2024-03-22T03:55:17.346 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-22T03:55:17.346 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --name pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/orig.config.yaml 2024-03-22T03:55:17.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T03:55:17.475 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927 branch: wip-batrick-testing-20240321.172515 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: pdonnell@redhat.com first_in_suite: false job_id: '7615927' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-batrick-testing-20240321.172515 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e kclient: syntax: v1 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240321.172515 sha1: c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 316 sha1: c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e sleep_before_teardown: 0 subset: 420/512 suite: fs suite_branch: wip-batrick-testing-20240321.172515 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPscXJYCGC+Y6ORG361G3Yy186lnUntU1Q7Mbs8PRDPUv/JsXU0OaJC4vPQHxfSoJ5xSvu8tDGn2eB3sAvggPtc= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDx+T9vmsFO1rwbEhqQ66c/mjZ446kth81nrjsA2cR/9py1eqhPY/yGlKG+PKmRKciCFFls8ssOVEgnSDY9R34w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_01:06:30 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-22T03:55:17.475 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/qa; will attempt to use it 2024-03-22T03:55:17.476 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/qa/tasks 2024-03-22T03:55:17.476 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-22T03:55:17.478 INFO:teuthology.task.internal:Checking packages... 2024-03-22T03:55:17.503 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e' 2024-03-22T03:55:17.503 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T03:55:17.503 INFO:teuthology.packaging:ref: None 2024-03-22T03:55:17.503 INFO:teuthology.packaging:tag: None 2024-03-22T03:55:17.504 INFO:teuthology.packaging:branch: wip-batrick-testing-20240321.172515 2024-03-22T03:55:17.504 INFO:teuthology.packaging:sha1: c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e 2024-03-22T03:55:17.504 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-batrick-testing-20240321.172515 2024-03-22T03:55:17.651 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T03:55:17.653 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-22T03:55:17.659 INFO:teuthology.task.internal:no buildpackages task found 2024-03-22T03:55:17.659 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-22T03:55:17.665 INFO:teuthology.task.internal:Saving configuration 2024-03-22T03:55:17.682 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-22T03:55:17.687 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T03:55:17.712 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 03:49:47.696768', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBtMvWCPNYWJo7H3xJDVit++vj16BdQ5v/+RRn3HobJA'} 2024-03-22T03:55:17.735 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 03:49:47.695009', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIID0hN2q8qM0sPdmVrnS+1I0+uh+CsKR9BQcGq3rhsqF'} 2024-03-22T03:55:17.735 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-22T03:55:17.741 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-22T03:55:17.742 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-22T03:55:17.742 INFO:teuthology.run_tasks:Running task console_log... 2024-03-22T03:55:17.816 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efe168564c0>, signals=[15]) 2024-03-22T03:55:17.816 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-22T03:55:17.821 INFO:teuthology.task.internal:Opening connections... 2024-03-22T03:55:17.822 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T03:55:17.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T03:55:17.913 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T03:55:17.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T03:55:17.993 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-22T03:55:17.999 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-03-22T03:55:18.004 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-03-22T03:55:18.005 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-03-22T03:55:18.050 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-22T03:55:18.050 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu" 2024-03-22T03:55:18.050 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="22.04" 2024-03-22T03:55:18.050 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-22T03:55:18.050 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=jammy 2024-03-22T03:55:18.050 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu 2024-03-22T03:55:18.050 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian 2024-03-22T03:55:18.051 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-22T03:55:18.051 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-22T03:55:18.051 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-22T03:55:18.051 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-22T03:55:18.051 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=jammy 2024-03-22T03:55:18.053 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-03-22T03:55:18.080 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-03-22T03:55:18.085 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-03-22T03:55:18.085 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-03-22T03:55:18.130 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Ubuntu" 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="22.04" 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_CODENAME=jammy 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:ID=ubuntu 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE=debian 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-22T03:55:18.131 INFO:teuthology.orchestra.run.smithi196.stdout:UBUNTU_CODENAME=jammy 2024-03-22T03:55:18.132 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-03-22T03:55:18.163 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-22T03:55:18.172 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-22T03:55:18.180 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-22T03:55:18.180 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-03-22T03:55:18.183 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-03-22T03:55:18.188 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-22T03:55:18.193 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-22T03:55:18.194 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-03-22T03:55:18.232 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-03-22T03:55:18.249 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-22T03:55:18.306 INFO:teuthology.run_tasks:Running task kernel... 2024-03-22T03:55:18.321 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-22T03:55:18.321 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-03-22T03:55:18.321 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-03-22T03:55:18.321 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T03:55:18.322 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T03:55:18.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T03:55:18.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-03-22T03:55:18.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T03:55:18.330 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-03-22T03:55:18.542 DEBUG:teuthology.task.kernel:sha1 for client.1 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T03:55:18.542 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-22T03:55:18.542 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-03-22T03:55:18.548 INFO:teuthology.orchestra.run.smithi196.stdout:5.15.0-56-generic 2024-03-22T03:55:18.548 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T03:55:18.548 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-22T03:55:18.549 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-03-22T03:55:18.549 DEBUG:teuthology.orchestra.run.smithi196:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-03-22T03:55:18.621 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-03-22T03:55:18.670 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T03:55:18.673 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-22T03:55:18.673 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-03-22T03:55:18.678 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-56-generic 2024-03-22T03:55:18.679 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T03:55:18.679 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-22T03:55:18.679 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-03-22T03:55:18.679 DEBUG:teuthology.orchestra.run.smithi086:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-03-22T03:55:18.742 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-03-22T03:55:18.777 INFO:teuthology.orchestra.run.smithi196.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-03-22T03:55:18.777 INFO:teuthology.orchestra.run.smithi196.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-03-22T03:55:18.778 INFO:teuthology.orchestra.run.smithi196.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-03-22T03:55:18.778 INFO:teuthology.orchestra.run.smithi196.stderr:hint: 2024-03-22T03:55:18.778 INFO:teuthology.orchestra.run.smithi196.stderr:hint: git config --global init.defaultBranch 2024-03-22T03:55:18.778 INFO:teuthology.orchestra.run.smithi196.stderr:hint: 2024-03-22T03:55:18.778 INFO:teuthology.orchestra.run.smithi196.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-03-22T03:55:18.778 INFO:teuthology.orchestra.run.smithi196.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-03-22T03:55:18.778 INFO:teuthology.orchestra.run.smithi196.stderr:hint: 2024-03-22T03:55:18.778 INFO:teuthology.orchestra.run.smithi196.stderr:hint: git branch -m 2024-03-22T03:55:18.779 INFO:teuthology.orchestra.run.smithi196.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-03-22T03:55:18.781 DEBUG:teuthology.orchestra.run.smithi196:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-03-22T03:55:18.844 DEBUG:teuthology.orchestra.run.smithi196:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-03-22T03:55:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: git config --global init.defaultBranch 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-03-22T03:55:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:hint: git branch -m 2024-03-22T03:55:18.879 INFO:teuthology.orchestra.run.smithi086.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-03-22T03:55:18.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-03-22T03:55:18.901 DEBUG:teuthology.orchestra.run.smithi196:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-03-22T03:55:18.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-03-22T03:55:19.003 DEBUG:teuthology.orchestra.run.smithi086:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-03-22T03:56:35.372 INFO:teuthology.orchestra.run.smithi196.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-03-22T03:56:35.372 INFO:teuthology.orchestra.run.smithi196.stderr: * [new branch] main -> origin/main 2024-03-22T03:56:35.411 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20190312 -> 20190312 2024-03-22T03:56:35.411 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20190416 -> 20190416 2024-03-22T03:56:35.411 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20190514 -> 20190514 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20190618 -> 20190618 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20190717 -> 20190717 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20190815 -> 20190815 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20190923 -> 20190923 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20191022 -> 20191022 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20191215 -> 20191215 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20200122 -> 20200122 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20200316 -> 20200316 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20200421 -> 20200421 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20200519 -> 20200519 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20200619 -> 20200619 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20200721 -> 20200721 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20200817 -> 20200817 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20200918 -> 20200918 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20201022 -> 20201022 2024-03-22T03:56:35.412 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20201118 -> 20201118 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20201218 -> 20201218 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20210208 -> 20210208 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20210315 -> 20210315 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20210511 -> 20210511 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20210716 -> 20210716 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20210818 -> 20210818 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20210919 -> 20210919 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20211027 -> 20211027 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20211216 -> 20211216 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20220209 -> 20220209 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20220310 -> 20220310 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20220411 -> 20220411 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20220509 -> 20220509 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20220610 -> 20220610 2024-03-22T03:56:35.413 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20220708 -> 20220708 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20220815 -> 20220815 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20220913 -> 20220913 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20221012 -> 20221012 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20221109 -> 20221109 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20221214 -> 20221214 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20230117 -> 20230117 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20230210 -> 20230210 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20230310 -> 20230310 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20230404 -> 20230404 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20230515 -> 20230515 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20230625 -> 20230625 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20230804 -> 20230804 2024-03-22T03:56:35.414 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20230919 -> 20230919 2024-03-22T03:56:35.415 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20231030 -> 20231030 2024-03-22T03:56:35.415 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20231111 -> 20231111 2024-03-22T03:56:35.415 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20231211 -> 20231211 2024-03-22T03:56:35.415 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20240115 -> 20240115 2024-03-22T03:56:35.415 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20240220 -> 20240220 2024-03-22T03:56:35.415 INFO:teuthology.orchestra.run.smithi196.stderr: * [new tag] 20240312 -> 20240312 2024-03-22T03:56:37.116 INFO:teuthology.orchestra.run.smithi086.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-03-22T03:56:37.116 INFO:teuthology.orchestra.run.smithi086.stderr: * [new branch] main -> origin/main 2024-03-22T03:56:37.149 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190312 -> 20190312 2024-03-22T03:56:37.149 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190416 -> 20190416 2024-03-22T03:56:37.149 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190514 -> 20190514 2024-03-22T03:56:37.149 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190618 -> 20190618 2024-03-22T03:56:37.149 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190717 -> 20190717 2024-03-22T03:56:37.149 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190815 -> 20190815 2024-03-22T03:56:37.150 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190923 -> 20190923 2024-03-22T03:56:37.150 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20191022 -> 20191022 2024-03-22T03:56:37.150 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20191215 -> 20191215 2024-03-22T03:56:37.150 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200122 -> 20200122 2024-03-22T03:56:37.150 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200316 -> 20200316 2024-03-22T03:56:37.150 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200421 -> 20200421 2024-03-22T03:56:37.151 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200519 -> 20200519 2024-03-22T03:56:37.151 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200619 -> 20200619 2024-03-22T03:56:37.151 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200721 -> 20200721 2024-03-22T03:56:37.151 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200817 -> 20200817 2024-03-22T03:56:37.151 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200918 -> 20200918 2024-03-22T03:56:37.152 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201022 -> 20201022 2024-03-22T03:56:37.152 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201118 -> 20201118 2024-03-22T03:56:37.152 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201218 -> 20201218 2024-03-22T03:56:37.152 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210208 -> 20210208 2024-03-22T03:56:37.152 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210315 -> 20210315 2024-03-22T03:56:37.155 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210511 -> 20210511 2024-03-22T03:56:37.155 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210716 -> 20210716 2024-03-22T03:56:37.155 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210818 -> 20210818 2024-03-22T03:56:37.155 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210919 -> 20210919 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20211027 -> 20211027 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20211216 -> 20211216 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220209 -> 20220209 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220310 -> 20220310 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220411 -> 20220411 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220509 -> 20220509 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220610 -> 20220610 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220708 -> 20220708 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220815 -> 20220815 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220913 -> 20220913 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221012 -> 20221012 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221109 -> 20221109 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221214 -> 20221214 2024-03-22T03:56:37.156 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230117 -> 20230117 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230210 -> 20230210 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230310 -> 20230310 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230404 -> 20230404 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230515 -> 20230515 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230625 -> 20230625 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230804 -> 20230804 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230919 -> 20230919 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231030 -> 20231030 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231111 -> 20231111 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231211 -> 20231211 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240115 -> 20240115 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240220 -> 20240220 2024-03-22T03:56:37.157 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240312 -> 20240312 2024-03-22T03:56:39.564 INFO:teuthology.orchestra.run.smithi196.stderr:Updating files: 52% (1873/3598) Updating files: 53% (1907/3598) Updating files: 54% (1943/3598) Updating files: 55% (1979/3598) Updating files: 56% (2015/3598) Updating files: 57% (2051/3598) Updating files: 58% (2087/3598) Updating files: 59% (2123/3598) Updating files: 60% (2159/3598) Updating files: 61% (2195/3598) Updating files: 62% (2231/3598) Updating files: 63% (2267/3598) Updating files: 64% (2303/3598) Updating files: 65% (2339/3598) Updating files: 65% (2352/3598) Updating files: 66% (2375/3598) Updating files: 67% (2411/3598) Updating files: 68% (2447/3598) Updating files: 69% (2483/3598) Updating files: 70% (2519/3598) Updating files: 71% (2555/3598) Updating files: 72% (2591/3598) Updating files: 73% (2627/3598) Updating files: 74% (2663/3598) Updating files: 75% (2699/3598) Updating files: 76% (2735/3598) Updating files: 77% (2771/3598) Updating files: 78% (2807/3598) Updating files: 79% (2843/3598) Updating files: 80% (2879/3598) Updating files: 80% (2887/3598) Updating files: 81% (2915/3598) Updating files: 82% (2951/3598) Updating files: 83% (2987/3598) Updating files: 84% (3023/3598) Updating files: 85% (3059/3598) Updating files: 86% (3095/3598) Updating files: 87% (3131/3598) Updating files: 88% (3167/3598) Updating files: 89% (3203/3598) Updating files: 90% (3239/3598) Updating files: 91% (3275/3598) Updating files: 92% (3311/3598) Updating files: 93% (3347/3598) Updating files: 94% (3383/3598) Updating files: 94% (3417/3598) Updating files: 95% (3419/3598) Updating files: 96% (3455/3598) Updating files: 97% (3491/3598) Updating files: 98% (3527/3598) Updating files: 99% (3563/3598) Updating files: 100% (3598/3598) Updating files: 100% (3598/3598), done. 2024-03-22T03:56:39.569 INFO:teuthology.orchestra.run.smithi196.stdout:HEAD is now at 22ee1d8c Merge branch 'robot/pr-0-1710861164' into 'main' 2024-03-22T03:56:39.590 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.1... 2024-03-22T03:56:39.590 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T03:56:39.811 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g5ec3e6e88b2e/ 2024-03-22T03:56:39.812 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /tmp/linux-image.deb && echo linux-image-6.8.0-g5ec3e6e88b2e_6.8.0-g5ec3e6e88b2e-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g5ec3e6e88b2e/ --input-file=- 2024-03-22T03:56:39.814 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi196.front.sepia.ceph.com to complete... 2024-03-22T03:56:41.305 INFO:teuthology.orchestra.run.smithi086.stderr:Updating files: 51% (1861/3598) Updating files: 52% (1871/3598) Updating files: 53% (1907/3598) Updating files: 54% (1943/3598) Updating files: 55% (1979/3598) Updating files: 56% (2015/3598) Updating files: 57% (2051/3598) Updating files: 58% (2087/3598) Updating files: 59% (2123/3598) Updating files: 60% (2159/3598) Updating files: 61% (2195/3598) Updating files: 62% (2231/3598) Updating files: 63% (2267/3598) Updating files: 64% (2303/3598) Updating files: 65% (2339/3598) Updating files: 65% (2351/3598) Updating files: 66% (2375/3598) Updating files: 67% (2411/3598) Updating files: 68% (2447/3598) Updating files: 69% (2483/3598) Updating files: 70% (2519/3598) Updating files: 71% (2555/3598) Updating files: 72% (2591/3598) Updating files: 73% (2627/3598) Updating files: 74% (2663/3598) Updating files: 75% (2699/3598) Updating files: 76% (2735/3598) Updating files: 77% (2771/3598) Updating files: 78% (2807/3598) Updating files: 79% (2843/3598) Updating files: 80% (2879/3598) Updating files: 80% (2887/3598) Updating files: 81% (2915/3598) Updating files: 82% (2951/3598) Updating files: 83% (2987/3598) Updating files: 84% (3023/3598) Updating files: 85% (3059/3598) Updating files: 86% (3095/3598) Updating files: 87% (3131/3598) Updating files: 88% (3167/3598) Updating files: 89% (3203/3598) Updating files: 90% (3239/3598) Updating files: 91% (3275/3598) Updating files: 92% (3311/3598) Updating files: 93% (3347/3598) Updating files: 94% (3383/3598) Updating files: 95% (3419/3598) Updating files: 95% (3431/3598) Updating files: 96% (3455/3598) Updating files: 97% (3491/3598) Updating files: 98% (3527/3598) Updating files: 99% (3563/3598) Updating files: 100% (3598/3598) Updating files: 100% (3598/3598), done. 2024-03-22T03:56:41.310 INFO:teuthology.orchestra.run.smithi086.stdout:HEAD is now at 22ee1d8c Merge branch 'robot/pr-0-1710861164' into 'main' 2024-03-22T03:56:41.334 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-22T03:56:41.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-22T03:56:41.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22 03:56:41 URL:https://3.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g5ec3e6e88b2e/linux-image-6.8.0-g5ec3e6e88b2e_6.8.0-g5ec3e6e88b2e-1_amd64.deb [39270258/39270258] -> "/tmp/linux-image.deb" [1] 2024-03-22T03:56:41.356 INFO:teuthology.orchestra.run.smithi196.stderr:FINISHED --2024-03-22 03:56:41-- 2024-03-22T03:56:41.356 INFO:teuthology.orchestra.run.smithi196.stderr:Total wall clock time: 1.5s 2024-03-22T03:56:41.356 INFO:teuthology.orchestra.run.smithi196.stderr:Downloaded: 1 files, 37M in 1.4s (26.9 MB/s) 2024-03-22T03:56:41.360 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.1... 2024-03-22T03:56:41.360 DEBUG:teuthology.orchestra.run.smithi196:> sudo dpkg -i /tmp/linux-image.deb 2024-03-22T03:56:41.482 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g5ec3e6e88b2e/ 2024-03-22T03:56:41.483 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/linux-image.deb && echo linux-image-6.8.0-g5ec3e6e88b2e_6.8.0-g5ec3e6e88b2e-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g5ec3e6e88b2e/ --input-file=- 2024-03-22T03:56:41.485 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-03-22T03:56:41.660 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-image-6.8.0-g5ec3e6e88b2e. 2024-03-22T03:56:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22 03:56:43 URL:https://3.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g5ec3e6e88b2e/linux-image-6.8.0-g5ec3e6e88b2e_6.8.0-g5ec3e6e88b2e-1_amd64.deb [39270258/39270258] -> "/tmp/linux-image.deb" [1] 2024-03-22T03:56:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-03-22 03:56:43-- 2024-03-22T03:56:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 1.5s 2024-03-22T03:56:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 37M in 1.4s (27.4 MB/s) 2024-03-22T03:56:42.993 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-22T03:56:42.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo dpkg -i /tmp/linux-image.deb 2024-03-22T03:56:43.247 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-image-6.8.0-g5ec3e6e88b2e. 2024-03-22T03:56:43.613 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-03-22T03:56:43.618 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-03-22T03:56:43.752 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-6.8.0-g5ec3e6e88b2e (6.8.0-g5ec3e6e88b2e-1) ... 2024-03-22T03:56:45.220 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-03-22T03:56:45.225 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-03-22T03:56:45.364 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-6.8.0-g5ec3e6e88b2e (6.8.0-g5ec3e6e88b2e-1) ... 2024-03-22T03:56:47.351 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-6.8.0-g5ec3e6e88b2e (6.8.0-g5ec3e6e88b2e-1) ... 2024-03-22T03:56:47.458 INFO:teuthology.orchestra.run.smithi196.stderr:update-initramfs: Generating /boot/initrd.img-6.8.0-g5ec3e6e88b2e 2024-03-22T03:56:48.933 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-6.8.0-g5ec3e6e88b2e (6.8.0-g5ec3e6e88b2e-1) ... 2024-03-22T03:56:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:update-initramfs: Generating /boot/initrd.img-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:09.132 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub' 2024-03-22T03:57:09.151 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-22T03:57:09.176 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-03-22T03:57:09.520 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:09.543 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:09.831 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-22T03:57:09.846 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-22T03:57:09.908 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-22T03:57:09.913 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-22T03:57:10.061 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-22T03:57:10.061 INFO:teuthology.orchestra.run.smithi196.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-22T03:57:10.061 INFO:teuthology.orchestra.run.smithi196.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-22T03:57:10.092 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-03-22T03:57:10.300 DEBUG:teuthology.orchestra.run.smithi196:> dpkg-deb -c /tmp/linux-image.deb 2024-03-22T03:57:10.308 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./ 2024-03-22T03:57:10.308 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./boot/ 2024-03-22T03:57:10.308 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 4474370 2024-03-19 20:13 ./boot/System.map-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:10.320 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 170904 2024-03-19 20:13 ./boot/config-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:10.320 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11555840 2024-03-19 20:13 ./boot/vmlinuz-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:10.338 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/postinst.d/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/postrm.d/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/preinst.d/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/prerm.d/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ 2024-03-22T03:57:10.339 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 490449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/aesni-intel.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 93081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 106697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 152145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 83857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-22T03:57:10.342 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/ 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/ 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/ 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-22T03:57:10.343 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpuid.ko 2024-03-22T03:57:10.344 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/msr.ko 2024-03-22T03:57:10.344 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/ 2024-03-22T03:57:10.344 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 354265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-22T03:57:10.344 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 886177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-22T03:57:10.344 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 2294937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-22T03:57:10.357 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ 2024-03-22T03:57:10.357 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/af_alg.ko 2024-03-22T03:57:10.357 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/algif_hash.ko 2024-03-22T03:57:10.357 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/algif_skcipher.ko 2024-03-22T03:57:10.358 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-22T03:57:10.358 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-22T03:57:10.358 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-22T03:57:10.358 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/asymmetric_keys/ 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/ 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-22T03:57:10.359 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cryptd.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_simd.ko 2024-03-22T03:57:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-22T03:57:10.361 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-22T03:57:10.361 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-22T03:57:10.361 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-22T03:57:10.361 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-22T03:57:10.361 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lrw.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-22T03:57:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 143033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xts.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_extlog.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-22T03:57:10.363 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/apei/ 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/apei/einj.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 104433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 89937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 88313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci.ko 2024-03-22T03:57:10.364 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-22T03:57:10.365 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-22T03:57:10.366 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-22T03:57:10.366 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-22T03:57:10.366 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-22T03:57:10.366 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-22T03:57:10.366 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 52177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_legacy.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-22T03:57:10.367 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_radisys.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rz1000.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-22T03:57:10.368 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 83257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-22T03:57:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/ 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 124457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/ 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/ 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 113833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/ 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 964817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-22T03:57:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 169705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-22T03:57:10.371 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/ 2024-03-22T03:57:10.371 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 94217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-22T03:57:10.371 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 116337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-22T03:57:10.371 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/ 2024-03-22T03:57:10.371 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 110329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-22T03:57:10.371 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 98193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-22T03:57:10.371 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 263641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/ 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 98121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/ 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/applicom.ko 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/ 2024-03-22T03:57:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-22T03:57:10.447 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-22T03:57:10.447 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-22T03:57:10.447 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-22T03:57:10.447 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-22T03:57:10.447 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 148521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 139361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/ 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 84025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/nvram.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-22T03:57:10.448 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/ 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/xillybus/ 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/xillybus/xillybus_class.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/xillybus/xillybus_core.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/ 2024-03-22T03:57:10.449 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/clk-palmas.ko 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/clk-si5351.ko 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/clk-twl6040.ko 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/clk-wm831x.ko 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/ 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-22T03:57:10.450 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 246841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/ 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/ 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/ 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 570497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/ 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/ 2024-03-22T03:57:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-22T03:57:10.452 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ 2024-03-22T03:57:10.452 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/ 2024-03-22T03:57:10.452 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-22T03:57:10.452 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ 2024-03-22T03:57:10.452 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 122817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-22T03:57:10.452 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ 2024-03-22T03:57:10.452 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 80305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-22T03:57:10.453 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-22T03:57:10.454 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-22T03:57:10.454 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 56737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-22T03:57:10.454 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-22T03:57:10.454 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/ 2024-03-22T03:57:10.454 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-gpio.ko 2024-03-22T03:57:10.455 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-max14577.ko 2024-03-22T03:57:10.455 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-max77693.ko 2024-03-22T03:57:10.455 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-max8997.ko 2024-03-22T03:57:10.455 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-palmas.ko 2024-03-22T03:57:10.455 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-rt8973a.ko 2024-03-22T03:57:10.455 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-sm5502.ko 2024-03-22T03:57:10.456 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/ 2024-03-22T03:57:10.456 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/dmi-sysfs.ko 2024-03-22T03:57:10.456 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/efi/ 2024-03-22T03:57:10.456 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/efi/efi-pstore.ko 2024-03-22T03:57:10.456 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-22T03:57:10.456 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/ 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-adp5520.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-amd8111.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-arizona.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-da9052.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-da9055.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-dln2.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 77753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-f7188x.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-generic.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-kempld.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-lp3943.ko 2024-03-22T03:57:10.457 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-max7300.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-max730x.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-max732x.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-pca953x.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-pcf857x.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-rdc321x.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-sch.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-sch311x.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-tps65912.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-twl4030.ko 2024-03-22T03:57:10.458 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-twl6040.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-vx855.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-wm831x.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-wm8350.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-wm8994.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/ 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-a4tech.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-apple.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-belkin.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-cherry.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-chicony.ko 2024-03-22T03:57:10.459 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 60665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-cp2112.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-cypress.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ezkey.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-generic.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-22T03:57:10.460 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kensington.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-22T03:57:10.461 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lg-g15.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 57977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 101545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 80921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-magicmouse.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-microsoft.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-monterey.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ntrig.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-22T03:57:10.462 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 72865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-22T03:57:10.463 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-22T03:57:10.464 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 143393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 269233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid.ko 2024-03-22T03:57:10.465 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/usbhid/ 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 125641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/usbhid/usbhid.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/usbhid/usbkbd.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/usbhid/usbmouse.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 218881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/ 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 64305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 90113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 304953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-22T03:57:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 97889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-22T03:57:10.467 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-22T03:57:10.468 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-22T03:57:10.468 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 74321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-22T03:57:10.468 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-22T03:57:10.468 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 69313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-22T03:57:10.468 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-22T03:57:10.468 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-22T03:57:10.468 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-22T03:57:10.469 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-22T03:57:10.469 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-22T03:57:10.469 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-22T03:57:10.469 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-22T03:57:10.469 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-22T03:57:10.469 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/da9052-hwmon.ko 2024-03-22T03:57:10.469 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/da9055-hwmon.ko 2024-03-22T03:57:10.470 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-22T03:57:10.470 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 99513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-22T03:57:10.470 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-22T03:57:10.470 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-22T03:57:10.470 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-22T03:57:10.471 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc2103.ko 2024-03-22T03:57:10.471 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-22T03:57:10.471 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-22T03:57:10.471 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 94601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-22T03:57:10.471 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-22T03:57:10.471 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-22T03:57:10.471 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-22T03:57:10.472 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-22T03:57:10.472 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-22T03:57:10.472 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-22T03:57:10.472 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-22T03:57:10.472 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hih6130.ko 2024-03-22T03:57:10.472 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-22T03:57:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-22T03:57:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-22T03:57:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-22T03:57:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-22T03:57:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-22T03:57:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-22T03:57:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 140137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-22T03:57:10.474 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/jc42.ko 2024-03-22T03:57:10.474 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-22T03:57:10.474 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-22T03:57:10.474 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-22T03:57:10.474 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-22T03:57:10.475 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-22T03:57:10.475 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-22T03:57:10.475 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-22T03:57:10.475 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-22T03:57:10.475 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-22T03:57:10.475 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-22T03:57:10.475 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 72745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-22T03:57:10.476 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-22T03:57:10.476 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-22T03:57:10.476 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-22T03:57:10.476 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 108753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-22T03:57:10.476 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-22T03:57:10.476 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-22T03:57:10.477 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-22T03:57:10.478 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-22T03:57:10.478 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-22T03:57:10.478 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-22T03:57:10.478 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-22T03:57:10.478 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-22T03:57:10.479 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-22T03:57:10.479 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mc13783-adc.ko 2024-03-22T03:57:10.479 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-22T03:57:10.479 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-03-22T03:57:10.479 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-22T03:57:10.479 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 119849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-22T03:57:10.480 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-22T03:57:10.480 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-22T03:57:10.480 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 82721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-22T03:57:10.480 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 54345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-22T03:57:10.480 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-22T03:57:10.480 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ 2024-03-22T03:57:10.481 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-22T03:57:10.481 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-22T03:57:10.481 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-22T03:57:10.481 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-22T03:57:10.481 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-22T03:57:10.481 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-22T03:57:10.482 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-22T03:57:10.482 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 82481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-22T03:57:10.482 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-22T03:57:10.482 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-22T03:57:10.482 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-22T03:57:10.482 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-22T03:57:10.483 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-22T03:57:10.483 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-22T03:57:10.483 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-22T03:57:10.483 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-22T03:57:10.483 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-22T03:57:10.483 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-22T03:57:10.484 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-22T03:57:10.484 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-22T03:57:10.484 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-22T03:57:10.484 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-22T03:57:10.484 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-22T03:57:10.484 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-22T03:57:10.485 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-22T03:57:10.485 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-22T03:57:10.485 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-22T03:57:10.485 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-22T03:57:10.485 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-22T03:57:10.485 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 43449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-22T03:57:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 52369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-22T03:57:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-22T03:57:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-22T03:57:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 79841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-22T03:57:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 71257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-22T03:57:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 63137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-22T03:57:10.487 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 71233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-22T03:57:10.487 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 101385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-22T03:57:10.487 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 119417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-22T03:57:10.487 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-22T03:57:10.487 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-22T03:57:10.488 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-03-22T03:57:10.488 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-03-22T03:57:10.488 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/ 2024-03-22T03:57:10.488 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/ 2024-03-22T03:57:10.488 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-22T03:57:10.488 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-22T03:57:10.489 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/ 2024-03-22T03:57:10.489 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-03-22T03:57:10.489 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-03-22T03:57:10.489 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-03-22T03:57:10.489 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-22T03:57:10.489 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-22T03:57:10.490 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-22T03:57:10.490 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-03-22T03:57:10.490 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-22T03:57:10.490 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-03-22T03:57:10.490 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-22T03:57:10.491 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-22T03:57:10.491 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-22T03:57:10.491 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-22T03:57:10.491 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-03-22T03:57:10.491 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-03-22T03:57:10.491 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-22T03:57:10.491 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-22T03:57:10.492 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-22T03:57:10.493 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-22T03:57:10.494 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-22T03:57:10.494 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-03-22T03:57:10.494 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-22T03:57:10.494 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-22T03:57:10.494 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-22T03:57:10.494 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/muxes/ 2024-03-22T03:57:10.494 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-03-22T03:57:10.495 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-03-22T03:57:10.495 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-03-22T03:57:10.495 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ 2024-03-22T03:57:10.495 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ 2024-03-22T03:57:10.495 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 273529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-22T03:57:10.495 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 883945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-22T03:57:10.495 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 69185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 324337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 97841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 282345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/ 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/ 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 493185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/ 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 420593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-22T03:57:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/ 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 741737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 296881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/ocrdma/ 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 153729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/usnic/ 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 169145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 266577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 100281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ 2024-03-22T03:57:10.497 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 111337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 128665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 129801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/ 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/evbug.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/ff-memless.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/ 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-22T03:57:10.498 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/ 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/lkkbd.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/lm8323.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/lm8333.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/newtonkbd.ko 2024-03-22T03:57:10.499 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/qt1070.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/qt2160.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/stowaway.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/sunkbd.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/xtkbd.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/matrix-keymap.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/ 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-22T03:57:10.500 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 123201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 89393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/gpio_mouse.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 335921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/psmouse.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/ 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 109601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/ 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/ct82c710.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-22T03:57:10.501 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/parkbd.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/pcips2.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/ps2mult.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serport.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/ 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-88pm860x.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-adp5520.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-bd2802.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-da903x.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-da9052.ko 2024-03-22T03:57:10.502 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-gpio.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3533.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm355x.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3642.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp8788.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-max8997.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-mc13783.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-menf21bmc.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-pca9532.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-pca955x.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-pca963x.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-tca6507.ko 2024-03-22T03:57:10.503 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-wm831x-status.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-wm8350.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-22T03:57:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-22T03:57:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/ 2024-03-22T03:57:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/ 2024-03-22T03:57:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 647121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-22T03:57:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-22T03:57:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 93321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bufio.ko 2024-03-22T03:57:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-22T03:57:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 155825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-22T03:57:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 107937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-22T03:57:10.506 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-22T03:57:10.506 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-era.ko 2024-03-22T03:57:10.506 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-22T03:57:10.506 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-22T03:57:10.506 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log.ko 2024-03-22T03:57:10.506 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-mirror.ko 2024-03-22T03:57:10.506 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 77881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-22T03:57:10.507 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-22T03:57:10.507 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-22T03:57:10.507 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-region-hash.ko 2024-03-22T03:57:10.507 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-22T03:57:10.507 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-22T03:57:10.507 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 109457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-snapshot.ko 2024-03-22T03:57:10.507 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-22T03:57:10.508 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 181777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-22T03:57:10.508 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-22T03:57:10.508 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-zero.ko 2024-03-22T03:57:10.508 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/ 2024-03-22T03:57:10.508 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 199881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-22T03:57:10.508 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-22T03:57:10.509 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 100841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-22T03:57:10.509 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 126369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-22T03:57:10.509 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 368497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-22T03:57:10.509 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/ 2024-03-22T03:57:10.509 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/ 2024-03-22T03:57:10.509 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 178049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-22T03:57:10.509 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 63241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-22T03:57:10.510 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-22T03:57:10.510 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-22T03:57:10.510 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 123233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-22T03:57:10.510 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 80921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-22T03:57:10.510 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 52865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-22T03:57:10.510 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/ 2024-03-22T03:57:10.510 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/88pm800.ko 2024-03-22T03:57:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/88pm805.ko 2024-03-22T03:57:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/88pm80x.ko 2024-03-22T03:57:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/arizona-i2c.ko 2024-03-22T03:57:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 123649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/arizona.ko 2024-03-22T03:57:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/bcm590xx.ko 2024-03-22T03:57:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/cros_ec_dev.ko 2024-03-22T03:57:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/dln2.ko 2024-03-22T03:57:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/janz-cmodio.ko 2024-03-22T03:57:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/kempld-core.ko 2024-03-22T03:57:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lm3533-core.ko 2024-03-22T03:57:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-03-22T03:57:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lp3943.ko 2024-03-22T03:57:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 56969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-22T03:57:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-22T03:57:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/max8907.ko 2024-03-22T03:57:10.512 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mc13xxx-core.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/menf21bmc.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/pcf50633-adc.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/pcf50633-gpio.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/pcf50633.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/retu-mfd.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 80345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/si476x-core.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/tps65010.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/tps6507x.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-22T03:57:10.513 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wm8994.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ad525x_dpot.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/ 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/c2port/ 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/c2port/core.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/ 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-22T03:57:10.514 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ds1682.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/dummy-irq.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/echo/ 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/echo/echo.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/ 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/genwqe/ 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 152073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hmc6352.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ibmasm/ 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 72561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-03-22T03:57:10.515 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ics932s401.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/ 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/ 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 126153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 56377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 313601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/phantom.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ti-st/ 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 74305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ti-st/st_drv.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-22T03:57:10.516 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/ 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 183569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/ 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 491609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/ 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/ 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 105937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/ 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 459001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-22T03:57:10.517 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/ 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 130665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 68785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 64617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 101617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/ 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 54625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-22T03:57:10.518 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/ 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 69169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/ 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/ 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/ 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 54617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 87457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/ 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/ 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 97201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-03-22T03:57:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 102497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 90209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/ 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 81321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 69545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/ 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 78385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 176697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1434505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 128369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/ 2024-03-22T03:57:10.520 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 131281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 314329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/ 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/ 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 323737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/ 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 193593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 347217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1045449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 166145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-22T03:57:10.521 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/ 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/ 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 204249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/ 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/ 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 69433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 141201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 62833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-22T03:57:10.522 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/ 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 60673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 62833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ec_bhf.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/ 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/ 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 281217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fujitsu/ 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 90305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/ 2024-03-22T03:57:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 295697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/ 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 584353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/ 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 246041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/ 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 944545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/ 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 288529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/ 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 513673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/ 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 93489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 735169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-03-22T03:57:10.524 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 151745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 86993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/ 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 102441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 122609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/ 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 727481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 278881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1122297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ 2024-03-22T03:57:10.525 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 52985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 95577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/ 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 108409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 73617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/ 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 151097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/ 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 136801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-22T03:57:10.526 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/ 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/ 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 227865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 87017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 672057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/ 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/ 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 83809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-22T03:57:10.527 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 194073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/samsung/ 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 104049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/ 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1067001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/ 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/ 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 54729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 63201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/ 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-22T03:57:10.528 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 73257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 68049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/ 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 495777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/ 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 103185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-22T03:57:10.529 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 166049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-22T03:57:10.633 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-22T03:57:10.633 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 78721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-22T03:57:10.633 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/ 2024-03-22T03:57:10.633 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-22T03:57:10.633 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/ 2024-03-22T03:57:10.633 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 92345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-22T03:57:10.633 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/ 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 79937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 83721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/ 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/ 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/fddi/ 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/fddi/defxx.ko 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/fddi/skfp/ 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 217113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/fddi/skfp/skfp.ko 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 74993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/ 2024-03-22T03:57:10.634 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 237265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/ 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 85857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 62897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/ 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-gpio.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-22T03:57:10.635 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/ 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-22T03:57:10.636 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 81961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 76569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 99009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/plip/ 2024-03-22T03:57:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 43769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/plip/plip.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sb1000.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/ 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-22T03:57:10.638 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 64561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/ 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 107233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 86825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/ 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 144361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-22T03:57:10.639 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/ 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 307193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/ 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 144017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/ 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/ 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 267745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 103361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/ 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 131329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 85217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-22T03:57:10.640 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/ 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/ 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pci-stub.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/ 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 146153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pcmcia.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 62081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pcmcia_core.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 112793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-22T03:57:10.641 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/ 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/broadcom/ 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/samsung/ 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/ 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/ 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 69033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/ 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/ 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec.ko 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-03-22T03:57:10.642 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/ 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 73601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 131945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-22T03:57:10.643 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/ibm_rtl.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/ 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/ 2024-03-22T03:57:10.644 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/88pm860x_battery.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/88pm860x_charger.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 54961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/bq2415x_charger.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/bq24190_charger.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/bq24735-charger.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/da9030_battery.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/da9052-battery.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ds2760_battery.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ds2780_battery.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ds2781_battery.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ds2782_battery.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/gpio-charger.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/isp1704_charger.ko 2024-03-22T03:57:10.645 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/lp8727_charger.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/max14577_charger.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/max17042_battery.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/max8903_charger.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/max8925_power.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/pcf50633-charger.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/sbs-battery.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/test_power.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/tps65090-charger.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/wm831x_backup.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/wm831x_power.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/wm8350_power.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/ 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-22T03:57:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/ 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/ 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/ 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-88pm80x.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-88pm860x.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-da9052.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-da9055.ko 2024-03-22T03:57:10.647 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-da9063.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 68481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-lp8788.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-22T03:57:10.648 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t86.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max8907.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max8925.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max8997.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max8998.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-palmas.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf50633.ko 2024-03-22T03:57:10.649 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rc5t583.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-s35390a.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-tps6586x.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-tps65910.ko 2024-03-22T03:57:10.650 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-wm831x.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-wm8350.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 77529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 57721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 135345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/ 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 271441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 100473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/ 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 356865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 295929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic94xx/ 2024-03-22T03:57:10.651 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 189145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/ 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 108209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/ 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 246937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/ 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1171953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/ 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 238401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/ 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 115329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-22T03:57:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/ 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 308961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/ 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 76041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 101425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 146753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 75977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/ 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 212385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 60505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/ 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 78225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-22T03:57:10.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 157729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fdomain.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/ 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 294953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 201729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/imm.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 43977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 265945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 101881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/ 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 317193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-22T03:57:10.654 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 52865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/ 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 315321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 145497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/ 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 216401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/ 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 2194089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/ 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 73473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 310689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 85521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/ 2024-03-22T03:57:10.655 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 659633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/ 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 119489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 78041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/ 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 81561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/ 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 366337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 124601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ppa.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/ 2024-03-22T03:57:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1876753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 85193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/ 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 558953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qlogicfas408.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 261081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 173929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 280113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 97625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 73865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 115649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-22T03:57:10.657 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/ 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 164889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/xen-scsifront.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 129393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/ 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/ 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 736761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/ 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-22T03:57:10.658 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 931521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 122489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/ 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 75521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/ 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/ 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/ 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-22T03:57:10.659 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-22T03:57:10.660 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 78353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 57777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/moxa.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/mxser.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 108489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/ 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/ 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/altera_uart.ko 2024-03-22T03:57:10.661 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 72553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/ 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 101993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/rp2.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/sc16is7xx.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/ 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 545401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/ 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-22T03:57:10.662 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_dmem_genirq.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_mf624.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_netx.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/ 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/c67x00/ 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 73225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/c67x00/c67x00.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 145793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-03-22T03:57:10.663 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/ 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 87009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 89769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/common/ 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/common/ulpi.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc2/ 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc3/ 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-03-22T03:57:10.664 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 452825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc3/dwc3.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/ 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/ 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 60329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/u_ether.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/u_serial.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 92409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 50945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 122257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-03-22T03:57:10.665 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 57633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 85521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/ 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-03-22T03:57:10.666 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 62369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 161233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/libcomposite.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/ 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/bdc/ 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 157617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 52929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-03-22T03:57:10.667 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 69617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 75969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/net2272.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 74985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/net2280.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 73017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 65881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 81409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 156337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/ 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/bcma-hcd.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 80017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/isp116x-hcd.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 87193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 78777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-22T03:57:10.668 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811_cs.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/ssb-hcd.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/ 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/cytherm.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ehset.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-22T03:57:10.669 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/lvstest.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/ 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 64745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbtest.ko 2024-03-22T03:57:10.670 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/mon/ 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 72825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/mon/usbmon.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/musb/ 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 301633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/musb/musb_hdrc.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/ 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-isp1301.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-tahvo.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-22T03:57:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/f81232.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 175105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 136985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 124745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-22T03:57:10.672 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 107433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/metro-usb.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 78129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 56249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mxuport.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-22T03:57:10.673 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 191825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 54265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-22T03:57:10.674 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 125529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usbserial.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/wishbone-serial.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 52761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-22T03:57:10.675 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 180329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/ 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 71025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 104065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/ 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/ 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 162865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-22T03:57:10.676 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 103673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 76441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/ 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 113665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 89057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/ 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/ 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/aat2870_bl.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/adp5520_bl.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/adp8860_bl.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/adp8870_bl.ko 2024-03-22T03:57:10.677 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/as3711_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/bd6107.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/da903x_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/da9052_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/gpio_backlight.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/kb3886_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lm3533_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lm3639_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lv5207lp.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/max8925_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/pandora_bl.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-03-22T03:57:10.678 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/wm831x_bl.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/ 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/matrox_w1.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/w1-gpio.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/ 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-22T03:57:10.679 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 56337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 103049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/acquirewdt.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/advantechwdt.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/cpu5wdt.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/da9052_wdt.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/da9055_wdt.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/da9063_wdt.ko 2024-03-22T03:57:10.680 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/dw_wdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/eurotechwdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/kempld_wdt.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-22T03:57:10.681 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/mena21_wdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pc87413_wdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/retu_wdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sc1200wdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-22T03:57:10.682 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/twl4030_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wafer5823wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wm831x_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wm8350_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/ 2024-03-22T03:57:10.683 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/ 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 176385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 71153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-scsiback.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/ 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/ 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 158281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/adfs/ 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/adfs/adfs.ko 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/ 2024-03-22T03:57:10.684 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 163417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/afs/ 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1019225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/afs/kafs.ko 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/autofs/ 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 103225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/autofs/autofs4.ko 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/ 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 64625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/bfs/ 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 41017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/bfs/bfs.ko 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/ 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 3487553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/ 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 305897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-22T03:57:10.685 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1335721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/ 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 97769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/configfs/ 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 108577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/configfs/configfs.ko 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/ 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/ 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 644249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/efs/ 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/efs/efs.ko 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/ 2024-03-22T03:57:10.686 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1527665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/ 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/freevxfs/ 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/freevxfs/freevxfs.ko 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/ 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/ 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1231361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/ 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 141713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/ 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 223281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hpfs/ 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 193545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hpfs/hpfs.ko 2024-03-22T03:57:10.687 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/ 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 74737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/ 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 420449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/ 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 281705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/ 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 88937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/ 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 490505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/ 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/ 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 81457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/ 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-22T03:57:10.688 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/ 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 117377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1058377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv2.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 120193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 2041825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/ 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/ 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1511665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/ 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 582529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/ 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-22T03:57:10.689 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ascii.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-22T03:57:10.690 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-22T03:57:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 94041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 142577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 165649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 118025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-22T03:57:10.692 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ntfs/ 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 220545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ntfs/ntfs.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 244673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 440681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ 2024-03-22T03:57:10.693 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 54177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 2590849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/omfs/ 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/omfs/omfs.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/ 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 360953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 56945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/qnx4/ 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/qnx4/qnx4.ko 2024-03-22T03:57:10.694 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/qnx6/ 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 46625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/qnx6/qnx6.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/quota/ 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/quota/quota_tree.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/quota/quota_v1.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/quota/quota_v2.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/ 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 532289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/ 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/ 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/ 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 2712601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/ 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-22T03:57:10.695 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/ 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 134809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/ 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 91329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/ 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 248409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 162785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/ 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 4036193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/ 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/locking/ 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 80489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/locking/locktorture.ko 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/time/ 2024-03-22T03:57:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/time/test_udelay.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 60265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc7.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/ 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/interval_tree_test.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/ 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-22T03:57:10.697 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/ 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/memory-notifier-error-inject.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/notifier-error-inject.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/percpu_test.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/pm-notifier-error-inject.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/ 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 143337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/rbtree_test.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/ 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test-kstrtox.ko 2024-03-22T03:57:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test-string_helpers.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 4863225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test_bpf.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test_firmware.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test_module.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test_user_copy.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/xz/ 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/xz/xz_dec_test.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/ 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ 2024-03-22T03:57:10.699 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/ 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 6905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/ 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-22T03:57:10.700 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/ 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 82897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/ 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 198833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/ 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 667545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/ 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 790673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-22T03:57:10.701 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-22T03:57:10.702 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/ 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1046825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/ 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 125705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/ 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 194505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-22T03:57:10.703 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 58745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/ 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 390089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/hsr/ 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 102073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/hsr/hsr.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 284721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 47881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-22T03:57:10.704 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 51217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 61513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 29081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-22T03:57:10.705 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ip_tables.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-22T03:57:10.706 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-22T03:57:10.707 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 45057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 79929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_gre.ko 2024-03-22T03:57:10.708 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 94169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 54273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/mip6.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-22T03:57:10.709 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-22T03:57:10.710 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 74881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/ 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 84401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/ 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 125209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 25817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-22T03:57:10.711 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 53233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/lapb/ 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/lapb/lapb.ko 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/ 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 268521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc2.ko 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/ 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 309417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/ 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 119705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-22T03:57:10.712 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 99993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 100281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 102553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 102345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 111673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 59193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 108345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 112145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 112409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 110337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 113289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ 2024-03-22T03:57:10.713 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 412513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-22T03:57:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 411769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 118161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 108313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-22T03:57:10.715 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 92233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 96521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 44137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 623473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 42513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-22T03:57:10.716 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 22937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 40809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-22T03:57:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_osf.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_synproxy.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tproxy.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 89233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/x_tables.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 18081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-22T03:57:10.718 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 38777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-22T03:57:10.719 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-22T03:57:10.720 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 43201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-22T03:57:10.721 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-22T03:57:10.722 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 16817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 8457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpudp.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/ 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 14441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/ 2024-03-22T03:57:10.723 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 12337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/ 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 384201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/ 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/phonet/ 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 112553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/phonet/phonet.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/phonet/pn_pep.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/ 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 253961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 165105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 68097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-22T03:57:10.724 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rxrpc/ 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 828297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rxrpc/rxrpc.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/ 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 37089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 43353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-22T03:57:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 19185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_cgroup.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 39521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 62657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 13321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 33425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-22T03:57:10.726 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 34593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq_codel.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 35457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 49849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 21977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 66449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 30225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 36153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 27409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-22T03:57:10.727 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 24777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 48537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 32817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 26793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 31321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 28273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/ 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 932945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/ 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/ 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 337009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 55041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 1513905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-22T03:57:10.728 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/ 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 704761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/ 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 10401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 867249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/ 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/ 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 87745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 67953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 98113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 11081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 17817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/x25/ 2024-03-22T03:57:10.729 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 131273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/x25/x25.ko 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/ 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_algo.ko 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 20553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 105217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_user.ko 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/ 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/ 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 15441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 642383 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 625850 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 7677 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 23498 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 9916 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 70732 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-22T03:57:10.730 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 147214 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 233213 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 210 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 72614 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 879 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 257955 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 312782 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./usr/ 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./usr/share/ 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./usr/share/doc/ 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./usr/share/doc/linux-image-6.8.0-g5ec3e6e88b2e/ 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 171 2024-03-19 20:13 ./usr/share/doc/linux-image-6.8.0-g5ec3e6e88b2e/changelog.Debian.gz 2024-03-22T03:57:10.731 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r-- root/root 691 2024-03-19 20:13 ./usr/share/doc/linux-image-6.8.0-g5ec3e6e88b2e/copyright 2024-03-22T03:57:10.736 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:10.736 INFO:teuthology.task.kernel:searching for kernel 6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:10.737 DEBUG:teuthology.orchestra.run.smithi196:> egrep '(submenu|menuentry.*6.8.0-g5ec3e6e88b2e).*{' /boot/grub/grub.cfg 2024-03-22T03:57:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2024-03-22T03:57:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-22T03:57:10.738 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-03-22T03:57:10.744 INFO:teuthology.orchestra.run.smithi196.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-22T03:57:10.744 INFO:teuthology.orchestra.run.smithi196.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g5ec3e6e88b2e' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g5ec3e6e88b2e-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-22T03:57:10.744 INFO:teuthology.orchestra.run.smithi196.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g5ec3e6e88b2e (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g5ec3e6e88b2e-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-22T03:57:10.744 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-03-22T03:57:10.745 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:10.745 DEBUG:teuthology.orchestra.run.smithi196:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-03-22T03:57:10.787 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi196.front.sepia.ceph.com to complete... 2024-03-22T03:57:11.031 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:11.047 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:11.160 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub' 2024-03-22T03:57:11.161 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-22T03:57:11.172 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-03-22T03:57:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-22T03:57:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-22T03:57:11.397 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:11.402 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:11.404 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-22T03:57:11.410 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-22T03:57:11.558 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-22T03:57:11.558 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-22T03:57:11.558 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-22T03:57:11.599 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-03-22T03:57:11.615 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-22T03:57:11.620 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-22T03:57:11.661 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-22T03:57:11.666 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-22T03:57:11.757 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-deb -c /tmp/linux-image.deb 2024-03-22T03:57:11.765 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./ 2024-03-22T03:57:11.765 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./boot/ 2024-03-22T03:57:11.765 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 4474370 2024-03-19 20:13 ./boot/System.map-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:11.776 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 170904 2024-03-19 20:13 ./boot/config-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:11.776 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11555840 2024-03-19 20:13 ./boot/vmlinuz-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:11.797 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/ 2024-03-22T03:57:11.797 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/ 2024-03-22T03:57:11.797 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/postinst.d/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/postrm.d/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/preinst.d/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./etc/kernel/prerm.d/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 490449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/aesni-intel.ko 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-22T03:57:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 106697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 152145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-22T03:57:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/ 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/ 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/ 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpuid.ko 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/msr.ko 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/ 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 354265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-22T03:57:11.800 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 886177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-22T03:57:11.801 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-22T03:57:11.801 INFO:teuthology.orchestra.run.smithi196.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-22T03:57:11.801 INFO:teuthology.orchestra.run.smithi196.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-22T03:57:11.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2294937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-22T03:57:11.814 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/af_alg.ko 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/algif_hash.ko 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/algif_skcipher.ko 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/asymmetric_keys/ 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-03-22T03:57:11.881 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/ 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-22T03:57:11.882 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cryptd.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_simd.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-22T03:57:11.883 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lrw.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 143033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-22T03:57:11.884 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xts.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_extlog.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/apei/ 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/apei/einj.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-22T03:57:11.885 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 104433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 88313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-22T03:57:11.886 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_legacy.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-22T03:57:11.887 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_radisys.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-22T03:57:11.888 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rz1000.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-22T03:57:11.889 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/ 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/ 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/ 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 113833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-22T03:57:11.890 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/ 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 964817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 169705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/ 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 94217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 116337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/ 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 110329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 98193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 263641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/ 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 98121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/ 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ 2024-03-22T03:57:11.891 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/applicom.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/ 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 148521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-22T03:57:11.892 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/ 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/nvram.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/ 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-22T03:57:11.893 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/xillybus/ 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/xillybus/xillybus_class.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/xillybus/xillybus_core.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/ 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/clk-palmas.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/clk-si5351.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/clk-twl6040.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/clk/clk-wm831x.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/ 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-22T03:57:11.894 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 246841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/ 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/ 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/ 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 570497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/ 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/ 2024-03-22T03:57:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/ 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 122817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-22T03:57:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/ 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-gpio.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-max14577.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-max77693.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-max8997.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-palmas.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-rt8973a.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/extcon/extcon-sm5502.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/ 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/dmi-sysfs.ko 2024-03-22T03:57:11.897 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/efi/ 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/efi/efi-pstore.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/ 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-adp5520.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-amd8111.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-arizona.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-da9052.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-da9055.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-dln2.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-f7188x.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-generic.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-22T03:57:11.898 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-kempld.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-lp3943.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-max7300.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-max730x.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-max732x.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-pca953x.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-pcf857x.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-rdc321x.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-sch.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-sch311x.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-tps65912.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-twl4030.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-twl6040.ko 2024-03-22T03:57:11.899 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-vx855.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-wm831x.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-wm8350.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-wm8994.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/ 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-a4tech.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-apple.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-belkin.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-cherry.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-chicony.ko 2024-03-22T03:57:11.900 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-cp2112.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-cypress.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ezkey.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-generic.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-22T03:57:11.901 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kensington.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lg-g15.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-magicmouse.ko 2024-03-22T03:57:11.902 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-microsoft.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-monterey.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ntrig.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-22T03:57:11.903 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-22T03:57:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 143393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 269233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid.ko 2024-03-22T03:57:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/usbhid/ 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 125641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/usbhid/usbhid.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/usbhid/usbkbd.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/usbhid/usbmouse.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 218881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/ 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 304953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-22T03:57:11.906 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-22T03:57:11.907 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/da9052-hwmon.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/da9055-hwmon.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 99513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-22T03:57:11.908 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc2103.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 94601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hih6130.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-22T03:57:11.909 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 140137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/jc42.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-22T03:57:11.910 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-22T03:57:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mc13783-adc.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-22T03:57:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-22T03:57:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-22T03:57:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-22T03:57:11.915 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/ 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/ 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/ 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-22T03:57:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-22T03:57:11.917 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-03-22T03:57:11.918 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/muxes/ 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 273529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-22T03:57:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 883945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 324337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 282345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/ 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/ 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 493185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/ 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 420593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/ 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 741737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 296881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-22T03:57:11.920 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/ocrdma/ 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 153729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/usnic/ 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 169145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 266577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 100281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 111337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 128665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ 2024-03-22T03:57:11.921 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 129801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/ 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/evbug.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/ff-memless.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/ 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/ 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-03-22T03:57:11.922 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/lkkbd.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/lm8323.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/lm8333.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/newtonkbd.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/qt1070.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/qt2160.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/stowaway.ko 2024-03-22T03:57:11.923 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/sunkbd.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/xtkbd.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/matrix-keymap.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/ 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 123201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/gpio_mouse.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 335921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/psmouse.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-22T03:57:11.924 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/ 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 109601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/ 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/ct82c710.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/parkbd.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/pcips2.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/ps2mult.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serport.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-22T03:57:11.925 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/ 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-88pm860x.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-adp5520.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-bd2802.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-da903x.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-da9052.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-gpio.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3533.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm355x.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3642.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp8788.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-max8997.ko 2024-03-22T03:57:11.926 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-mc13783.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-menf21bmc.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-pca9532.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-pca955x.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-pca963x.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-tca6507.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-wm831x-status.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-wm8350.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-22T03:57:11.927 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/ 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/ 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 647121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bufio.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 155825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-era.ko 2024-03-22T03:57:11.928 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-mirror.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-region-hash.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 109457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-snapshot.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 181777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-zero.ko 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/ 2024-03-22T03:57:11.929 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 199881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 100841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 368497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/ 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/ 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 178049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 123233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-22T03:57:11.930 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/ 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/88pm800.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/88pm805.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/88pm80x.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/arizona-i2c.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 123649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/arizona.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/bcm590xx.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/cros_ec_dev.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/dln2.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/janz-cmodio.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/kempld-core.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lm3533-core.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-03-22T03:57:11.931 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lp3943.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/max8907.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mc13xxx-core.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/menf21bmc.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/pcf50633-adc.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/pcf50633-gpio.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/pcf50633.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/retu-mfd.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/si476x-core.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-22T03:57:11.932 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/tps65010.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/tps6507x.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wm8994.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ad525x_dpot.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/ 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-22T03:57:11.933 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/c2port/ 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/c2port/core.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/ 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ds1682.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/dummy-irq.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/echo/ 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/echo/echo.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/ 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/genwqe/ 2024-03-22T03:57:11.934 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 152073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hmc6352.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ibmasm/ 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ics932s401.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/ 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/ 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-22T03:57:11.935 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 313601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/phantom.ko 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ti-st/ 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/ti-st/st_drv.ko 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/ 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 183569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/ 2024-03-22T03:57:11.936 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 491609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/ 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/ 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 105937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/ 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 459001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/ 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-22T03:57:11.937 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 130665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/ 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/ 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-22T03:57:11.938 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/ 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/ 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/ 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/ 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/ 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-03-22T03:57:11.939 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/ 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/ 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 176697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1434505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 128369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/ 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 314329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/ 2024-03-22T03:57:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/ 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 323737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/ 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 193593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 347217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1045449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 166145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-22T03:57:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/ 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/ 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 204249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/ 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/ 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 141201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/ 2024-03-22T03:57:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ec_bhf.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/ 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/ 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 281217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fujitsu/ 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/ 2024-03-22T03:57:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 295697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/ 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 584353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/ 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 246041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/ 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 944545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/ 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 288529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/ 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 513673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/ 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 735169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-22T03:57:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 151745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 86993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/ 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 122609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/ 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-03-22T03:57:11.945 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 727481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-22T03:57:12.028 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 278881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-22T03:57:12.028 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-03-22T03:57:12.028 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-03-22T03:57:12.028 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1122297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-22T03:57:12.028 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ 2024-03-22T03:57:12.028 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 95577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/ 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/ 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 151097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/ 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 136801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-22T03:57:12.029 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/ 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/ 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 227865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 672057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/ 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/ 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-22T03:57:12.030 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 194073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/samsung/ 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 104049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/ 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1067001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/ 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/ 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-22T03:57:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/ 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/ 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 495777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/ 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 166049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-22T03:57:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/ 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/ 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 92345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/ 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/ 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/ 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-22T03:57:12.033 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/fddi/ 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/fddi/defxx.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/fddi/skfp/ 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 217113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/fddi/skfp/skfp.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/ 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 237265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/ 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/ 2024-03-22T03:57:12.034 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-gpio.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/ 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-22T03:57:12.035 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 99009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-22T03:57:12.036 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/plip/ 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/plip/plip.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sb1000.ko 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/ 2024-03-22T03:57:12.037 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/ 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 86825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/ 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 144361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-22T03:57:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/ 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 307193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/ 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 144017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/ 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/ 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 267745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/ 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-22T03:57:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/ 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/ 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pci-stub.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/ 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 146153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pcmcia.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pcmcia_core.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 112793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-22T03:57:12.040 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/ 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/broadcom/ 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/samsung/ 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/ 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/ 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/ 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/ 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec.ko 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-03-22T03:57:12.041 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/ 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-22T03:57:12.042 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/ibm_rtl.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/ 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-22T03:57:12.043 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/ 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/88pm860x_battery.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/88pm860x_charger.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/bq2415x_charger.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/bq24190_charger.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/bq24735-charger.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/da9030_battery.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/da9052-battery.ko 2024-03-22T03:57:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ds2760_battery.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ds2780_battery.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ds2781_battery.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/ds2782_battery.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/gpio-charger.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/isp1704_charger.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/lp8727_charger.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/max14577_charger.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/max17042_battery.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/max8903_charger.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/max8925_power.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/pcf50633-charger.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/sbs-battery.ko 2024-03-22T03:57:12.045 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/test_power.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/tps65090-charger.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/wm831x_backup.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/wm831x_power.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/power/supply/wm8350_power.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/ 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/ 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/ 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-22T03:57:12.046 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/ 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-88pm80x.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-88pm860x.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-da9052.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-da9055.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-da9063.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-22T03:57:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-lp8788.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t86.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-22T03:57:12.048 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max8907.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max8925.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max8997.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max8998.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-palmas.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf50633.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rc5t583.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-22T03:57:12.049 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-s35390a.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-tps6586x.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-tps65910.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-wm831x.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-wm8350.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-22T03:57:12.050 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 135345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/ 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 271441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 100473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/ 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 356865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 295929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic94xx/ 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 189145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/ 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-22T03:57:12.051 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/ 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 246937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/ 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1171953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/ 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 238401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/ 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/ 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 308961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/ 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-03-22T03:57:12.052 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 146753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/ 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 212385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/ 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 157729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fdomain.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/ 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 294953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 201729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-22T03:57:12.053 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/imm.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 265945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/ 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 317193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/ 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 315321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 145497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/ 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 216401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-22T03:57:12.054 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/ 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2194089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/ 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 310689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/ 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 659633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/ 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/ 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-03-22T03:57:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/ 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 366337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ppa.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/ 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1876753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/ 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 558953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qlogicfas408.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 261081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-22T03:57:12.056 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 173929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 280113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/ 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 164889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/xen-scsifront.ko 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ 2024-03-22T03:57:12.057 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 129393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/ 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/ 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 736761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/ 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 931521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 122489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/ 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/ 2024-03-22T03:57:12.058 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/ 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/ 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-22T03:57:12.059 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/moxa.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/mxser.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/ 2024-03-22T03:57:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/ 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/altera_uart.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/ 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/rp2.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/sc16is7xx.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/ 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 545401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-22T03:57:12.061 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/ 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_dmem_genirq.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_mf624.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_netx.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/ 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/c67x00/ 2024-03-22T03:57:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/c67x00/c67x00.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 145793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/ 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/common/ 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/common/ulpi.ko 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc2/ 2024-03-22T03:57:12.063 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc3/ 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 452825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/dwc3/dwc3.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/ 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/ 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/u_ether.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/u_serial.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 92409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-03-22T03:57:12.064 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 122257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/ 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-03-22T03:57:12.065 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 161233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/libcomposite.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/ 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-03-22T03:57:12.066 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/bdc/ 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 157617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/net2272.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/net2280.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 156337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/ 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/bcma-hcd.ko 2024-03-22T03:57:12.067 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/isp116x-hcd.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811_cs.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/ssb-hcd.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/ 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/cytherm.ko 2024-03-22T03:57:12.068 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ehset.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/lvstest.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/ 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-22T03:57:12.069 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbtest.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/mon/ 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/mon/usbmon.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/musb/ 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 301633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/musb/musb_hdrc.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/ 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-isp1301.ko 2024-03-22T03:57:12.070 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-tahvo.ko 2024-03-22T03:57:12.073 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ 2024-03-22T03:57:12.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-22T03:57:12.073 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/f81232.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 175105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 136985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-22T03:57:12.074 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107433 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/metro-usb.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mxuport.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-22T03:57:12.075 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 191825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-22T03:57:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 125529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usbserial.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/wishbone-serial.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-22T03:57:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 180329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/ 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 104065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/ 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/ 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 162865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-22T03:57:12.078 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/ 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 113665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/ 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/ 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/aat2870_bl.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/adp5520_bl.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/adp8860_bl.ko 2024-03-22T03:57:12.079 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/adp8870_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/as3711_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/bd6107.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/da903x_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/da9052_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/gpio_backlight.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/kb3886_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lm3533_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lm3639_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lv5207lp.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/max8925_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/pandora_bl.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-03-22T03:57:12.080 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/wm831x_bl.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/ 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/matrox_w1.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/w1-gpio.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/ 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-22T03:57:12.081 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103049 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/acquirewdt.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/advantechwdt.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/cpu5wdt.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/da9052_wdt.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/da9055_wdt.ko 2024-03-22T03:57:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/da9063_wdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/dw_wdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/eurotechwdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-22T03:57:12.083 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21393 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/kempld_wdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/mena21_wdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pc87413_wdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/retu_wdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-22T03:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sc1200wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21481 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/twl4030_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wafer5823wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wm831x_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13689 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wm8350_wdt.ko 2024-03-22T03:57:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15609 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/ 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/ 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 176385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-scsiback.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/ 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/ 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 158281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-22T03:57:12.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/adfs/ 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/adfs/adfs.ko 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/ 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 163417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/afs/ 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1019225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/afs/kafs.ko 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/autofs/ 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/autofs/autofs4.ko 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/ 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/bfs/ 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/bfs/bfs.ko 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/ 2024-03-22T03:57:12.087 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 3487553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/ 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 305897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1335721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/ 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/configfs/ 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/configfs/configfs.ko 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/ 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/ 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 644249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/efs/ 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/efs/efs.ko 2024-03-22T03:57:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/ 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1527665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/ 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/freevxfs/ 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/freevxfs/freevxfs.ko 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/ 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/ 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1231361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/ 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 141713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/ 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 223281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hpfs/ 2024-03-22T03:57:12.089 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 193545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hpfs/hpfs.ko 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/ 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/ 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 420449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/ 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 281705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/ 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 88937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/ 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 490505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/ 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/ 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-22T03:57:12.090 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/ 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/ 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1058377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv2.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2041825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/ 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/ 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1511665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/ 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 582529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-22T03:57:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/ 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ascii.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-22T03:57:12.092 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-22T03:57:12.093 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 94041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 142577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 165649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 118025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-22T03:57:12.094 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ntfs/ 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 220545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ntfs/ntfs.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 244673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 440681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-22T03:57:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54177 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2590849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/omfs/ 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/omfs/omfs.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/ 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 360953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/qnx4/ 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/qnx4/qnx4.ko 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/qnx6/ 2024-03-22T03:57:12.096 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46625 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/qnx6/qnx6.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/quota/ 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/quota/quota_tree.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/quota/quota_v1.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/quota/quota_v2.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/ 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 532289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/ 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/ 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/ 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2712601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/ 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-22T03:57:12.097 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/ 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/ 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/ 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 248409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 162785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/ 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 4036193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/ 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/locking/ 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/locking/locktorture.ko 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/time/ 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/time/test_udelay.ko 2024-03-22T03:57:12.098 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc7.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/ 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/interval_tree_test.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/ 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-22T03:57:12.099 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/ 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/memory-notifier-error-inject.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/notifier-error-inject.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/percpu_test.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/pm-notifier-error-inject.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/ 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 143337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/rbtree_test.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/ 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test-kstrtox.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test-string_helpers.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 4863225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test_bpf.ko 2024-03-22T03:57:12.100 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test_firmware.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test_module.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/test_user_copy.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/xz/ 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/xz/xz_dec_test.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/ 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18129 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/ 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-22T03:57:12.101 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 6905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/ 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-22T03:57:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/ 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/ 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 198833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/ 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 667545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/ 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 790673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-22T03:57:12.103 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-22T03:57:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/ 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1046825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/ 2024-03-22T03:57:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 125705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-22T03:57:12.137 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/ 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 194505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/ 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 390089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12601 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/hsr/ 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102073 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/hsr/hsr.ko 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ 2024-03-22T03:57:12.138 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 284721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-22T03:57:12.139 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ip_tables.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9585 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14785 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-22T03:57:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15969 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-22T03:57:12.141 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10857 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ 2024-03-22T03:57:12.142 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_gre.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 94169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/mip6.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-22T03:57:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15025 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-22T03:57:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8913 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/ 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/ 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 125209 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36497 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-22T03:57:12.145 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/lapb/ 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/lapb/lapb.ko 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/ 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 268521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc2.ko 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/ 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 309417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/ 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119705 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-22T03:57:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28617 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 99993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 100281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 111673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108345 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 112145 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 112409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 110337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 113289 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-22T03:57:12.147 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 412513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11993 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10641 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9265 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13545 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7721 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-22T03:57:12.148 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 411769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15881 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 118161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 92233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-22T03:57:12.149 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12529 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44137 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 623473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42513 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30161 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-22T03:57:12.150 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49649 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9505 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20033 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14329 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11753 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_osf.ko 2024-03-22T03:57:12.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9985 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12841 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_synproxy.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tproxy.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/x_tables.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7633 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13281 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-22T03:57:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13801 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11561 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-22T03:57:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8233 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11257 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12305 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10889 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-22T03:57:12.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7769 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10569 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11825 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11697 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10121 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-22T03:57:12.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11729 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13737 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10017 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11897 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9385 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8169 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36361 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13241 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30001 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-22T03:57:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9417 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8713 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15681 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpudp.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9313 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/ 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/ 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12337 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/ 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 384201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-22T03:57:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10369 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11833 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/ 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13473 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/phonet/ 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 112553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/phonet/phonet.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55057 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/phonet/pn_pep.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/ 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 253961 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 165105 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68097 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rxrpc/ 2024-03-22T03:57:12.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 828297 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rxrpc/rxrpc.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/ 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23489 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23377 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43353 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20193 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35201 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36577 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33873 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19185 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_cgroup.ko 2024-03-22T03:57:12.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62657 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7465 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33425 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7921 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10929 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21673 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20865 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34593 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-22T03:57:12.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31521 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq_codel.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35457 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49849 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21977 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66449 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30225 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10809 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26089 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36153 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27409 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10665 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24777 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48537 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-22T03:57:12.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26793 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31321 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/ 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 932945 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/ 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/ 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 337009 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55041 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1513905 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-22T03:57:12.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/ 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 704761 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/ 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10401 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 867249 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/ 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15937 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/ 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87745 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67953 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 98113 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11081 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17817 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-22T03:57:12.163 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/x25/ 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131273 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/x25/x25.ko 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/ 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23065 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_algo.ko 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20553 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 105217 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_user.ko 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/ 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/ 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15441 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 642383 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 625850 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7677 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23498 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-22T03:57:12.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9916 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70732 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 147214 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 233213 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 210 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72614 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 879 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 257955 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 312782 2024-03-19 20:13 ./lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./usr/ 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./usr/share/ 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./usr/share/doc/ 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-03-19 20:13 ./usr/share/doc/linux-image-6.8.0-g5ec3e6e88b2e/ 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 171 2024-03-19 20:13 ./usr/share/doc/linux-image-6.8.0-g5ec3e6e88b2e/changelog.Debian.gz 2024-03-22T03:57:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 691 2024-03-19 20:13 ./usr/share/doc/linux-image-6.8.0-g5ec3e6e88b2e/copyright 2024-03-22T03:57:12.170 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:12.170 INFO:teuthology.task.kernel:searching for kernel 6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:12.171 DEBUG:teuthology.orchestra.run.smithi086:> egrep '(submenu|menuentry.*6.8.0-g5ec3e6e88b2e).*{' /boot/grub/grub.cfg 2024-03-22T03:57:12.186 INFO:teuthology.orchestra.run.smithi086.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-22T03:57:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g5ec3e6e88b2e' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g5ec3e6e88b2e-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-22T03:57:12.187 INFO:teuthology.orchestra.run.smithi086.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g5ec3e6e88b2e (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g5ec3e6e88b2e-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-22T03:57:12.187 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-03-22T03:57:12.187 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:12.187 DEBUG:teuthology.orchestra.run.smithi086:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-03-22T03:57:12.231 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-03-22T03:57:12.640 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2024-03-22T03:57:12.641 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-22T03:57:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-03-22T03:57:12.855 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:12.860 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-6.8.0-g5ec3e6e88b2e 2024-03-22T03:57:13.090 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-22T03:57:13.097 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-22T03:57:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-22T03:57:13.139 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-22T03:57:13.293 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-22T03:57:13.293 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-22T03:57:13.293 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-22T03:57:13.308 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-03-22T03:57:42.947 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-22T03:57:42.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-22T03:57:42.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T03:57:44.441 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-22T03:57:44.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T03:57:44.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T03:58:02.783 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T03:58:11.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T03:58:11.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T03:58:14.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-22T03:58:17.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-22T03:58:26.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-22T03:58:26.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T03:58:26.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T03:58:26.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T03:58:45.023 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-22T03:58:57.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-22T03:58:57.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T03:58:57.566 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-03-22T03:58:58.550 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-22T03:58:58.550 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-g5ec3e6e88b2e"... 2024-03-22T03:58:58.551 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-03-22T03:58:58.556 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-22T03:58:58.598 INFO:teuthology.orchestra.run.smithi196.stdout:6.8.0-g5ec3e6e88b2e 2024-03-22T03:58:58.598 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-22T03:58:58.599 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T03:58:58.599 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-22T03:58:59.209 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T03:58:59.209 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g5ec3e6e88b2e"... 2024-03-22T03:58:59.209 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-03-22T03:58:59.259 INFO:teuthology.orchestra.run.smithi086.stdout:6.8.0-g5ec3e6e88b2e 2024-03-22T03:58:59.259 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-22T03:58:59.259 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T03:58:59.259 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-22T03:58:59.600 DEBUG:teuthology.parallel:result is None 2024-03-22T03:59:00.260 DEBUG:teuthology.parallel:result is None 2024-03-22T03:59:00.261 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-22T03:59:00.268 INFO:teuthology.task.internal:Creating test directory... 2024-03-22T03:59:00.268 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T03:59:00.272 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T03:59:00.281 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-22T03:59:00.292 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-22T03:59:00.299 INFO:teuthology.task.internal:Creating archive directory... 2024-03-22T03:59:00.299 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T03:59:00.323 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T03:59:00.348 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-22T03:59:00.355 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-22T03:59:00.356 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T03:59:00.383 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T03:59:00.465 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T03:59:00.481 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T03:59:00.528 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T03:59:00.549 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T03:59:00.552 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-22T03:59:00.561 INFO:teuthology.task.internal:Configuring sudo... 2024-03-22T03:59:00.561 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T03:59:00.565 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T03:59:00.584 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-22T03:59:00.593 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-22T03:59:00.593 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T03:59:00.627 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T03:59:00.635 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T03:59:00.685 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T03:59:00.737 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T03:59:00.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T03:59:00.813 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T03:59:00.822 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T03:59:00.875 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T03:59:00.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T03:59:00.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-03-22T03:59:00.971 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-03-22T03:59:01.088 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-22T03:59:01.096 INFO:teuthology.task.internal:Starting timer... 2024-03-22T03:59:01.096 INFO:teuthology.run_tasks:Running task pcp... 2024-03-22T03:59:01.113 INFO:teuthology.run_tasks:Running task selinux... 2024-03-22T03:59:01.125 DEBUG:teuthology.task.selinux:Excluding smithi086: OS 'ubuntu' does not support SELinux 2024-03-22T03:59:01.125 DEBUG:teuthology.task.selinux:Excluding smithi196: OS 'ubuntu' does not support SELinux 2024-03-22T03:59:01.125 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-22T03:59:01.126 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-22T03:59:01.126 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-22T03:59:01.126 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-22T03:59:01.135 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-22T03:59:01.141 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-22T03:59:01.262 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-22T03:59:01.278 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-22T03:59:01.279 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-22T04:06:36.568 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-03-22T04:06:36.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T04:06:36.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T04:06:36.655 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-22T04:06:36.772 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-22T04:06:36.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-22T04:06:36.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T04:06:36.853 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-03-22T04:06:36.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-22T04:06:36.982 INFO:teuthology.run_tasks:Running task clock... 2024-03-22T04:06:36.992 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-22T04:06:36.992 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T04:06:36.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T04:06:36.995 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T04:06:36.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T04:06:37.030 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-22T04:06:37.030 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Command line: ntpd -gq 2024-03-22T04:06:37.031 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: ---------------------------------------------------- 2024-03-22T04:06:37.031 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: ntp-4 is maintained by Network Time Foundation, 2024-03-22T04:06:37.031 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-22T04:06:37.031 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: corporation. Support and training for ntp-4 are 2024-03-22T04:06:37.031 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: available at https://www.nwtime.org/support 2024-03-22T04:06:37.031 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: ---------------------------------------------------- 2024-03-22T04:06:37.032 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: proto: precision = 0.041 usec (-24) 2024-03-22T04:06:37.032 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: basedate set to 2022-02-04 2024-03-22T04:06:37.032 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: gps base set to 2022-02-06 (week 2196) 2024-03-22T04:06:37.032 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-22T04:06:37.032 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: restrict ::: KOD does nothing without LIMITED. 2024-03-22T04:06:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-22T04:06:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-22T04:06:37.034 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Listen and drop on 0 v6wildcard [::]:123 2024-03-22T04:06:37.034 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-22T04:06:37.034 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Listen normally on 2 lo 127.0.0.1:123 2024-03-22T04:06:37.034 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Listen normally on 3 enp3s0f1 172.21.15.86:123 2024-03-22T04:06:37.034 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Listen normally on 4 lo [::1]:123 2024-03-22T04:06:37.034 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d809%5]:123 2024-03-22T04:06:37.034 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:37 ntpd[17171]: Listening on routing socket on fd #22 for interface updates 2024-03-22T04:06:37.060 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-22T04:06:37.060 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Command line: ntpd -gq 2024-03-22T04:06:37.060 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: ---------------------------------------------------- 2024-03-22T04:06:37.060 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: ntp-4 is maintained by Network Time Foundation, 2024-03-22T04:06:37.060 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-22T04:06:37.060 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: corporation. Support and training for ntp-4 are 2024-03-22T04:06:37.060 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: available at https://www.nwtime.org/support 2024-03-22T04:06:37.060 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: ---------------------------------------------------- 2024-03-22T04:06:37.062 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: proto: precision = 0.042 usec (-24) 2024-03-22T04:06:37.062 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: basedate set to 2022-02-04 2024-03-22T04:06:37.062 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: gps base set to 2022-02-06 (week 2196) 2024-03-22T04:06:37.062 INFO:teuthology.orchestra.run.smithi196.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-22T04:06:37.062 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-22T04:06:37.063 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: restrict ::: KOD does nothing without LIMITED. 2024-03-22T04:06:37.063 INFO:teuthology.orchestra.run.smithi196.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-22T04:06:37.064 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Listen and drop on 0 v6wildcard [::]:123 2024-03-22T04:06:37.064 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-22T04:06:37.064 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Listen normally on 2 lo 127.0.0.1:123 2024-03-22T04:06:37.064 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Listen normally on 3 enp3s0f1 172.21.15.196:123 2024-03-22T04:06:37.064 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Listen normally on 4 lo [::1]:123 2024-03-22T04:06:37.064 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f47%5]:123 2024-03-22T04:06:37.064 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:37 ntpd[17159]: Listening on routing socket on fd #22 for interface updates 2024-03-22T04:06:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:38 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:38 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:38 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:38 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:38 ntpd[17171]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:38 ntpd[17171]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:38 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:39.031 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:39 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:39.031 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:39 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:40 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:40 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:40 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:40 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:40 ntpd[17171]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:40 ntpd[17171]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:40.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:40 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:42 ntpd[17171]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:42.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:42.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:42 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:44.033 INFO:teuthology.orchestra.run.smithi086.stdout:22 Mar 04:06:44 ntpd[17171]: ntpd: time slew -0.003449 s 2024-03-22T04:06:44.034 INFO:teuthology.orchestra.run.smithi086.stdout:ntpd: time slew -0.003449s 2024-03-22T04:06:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-22T04:06:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:22 Mar 04:06:44 ntpd[17171]: can't open /var/log/ntpstats/loopstats.20240322: Permission denied 2024-03-22T04:06:44.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.062 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.063 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.064 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:44.064 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:44.064 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-22T04:06:44.064 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: can't open /var/log/ntpstats/rawstats.20240322: Permission denied 2024-03-22T04:06:44.064 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-22T04:06:44.064 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: can't open /var/log/ntpstats/peerstats.20240322: Permission denied 2024-03-22T04:06:44.065 INFO:teuthology.orchestra.run.smithi196.stdout:22 Mar 04:06:44 ntpd[17159]: ntpd: time slew -0.000520 s 2024-03-22T04:06:44.065 INFO:teuthology.orchestra.run.smithi196.stdout:ntpd: time slew -0.000520s 2024-03-22T04:06:44.065 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-22T04:06:44.065 INFO:teuthology.orchestra.run.smithi196.stderr:22 Mar 04:06:44 ntpd[17159]: can't open /var/log/ntpstats/loopstats.20240322: Permission denied 2024-03-22T04:06:44.131 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2024-03-22T04:06:44.132 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2024-03-22T04:06:44.132 INFO:teuthology.orchestra.run.smithi086.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-22T04:06:44.132 INFO:teuthology.orchestra.run.smithi086.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-22T04:06:44.132 INFO:teuthology.orchestra.run.smithi086.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-22T04:06:44.132 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-22T04:06:44.154 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2024-03-22T04:06:44.155 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2024-03-22T04:06:44.155 INFO:teuthology.orchestra.run.smithi196.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-22T04:06:44.155 INFO:teuthology.orchestra.run.smithi196.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-22T04:06:44.155 INFO:teuthology.orchestra.run.smithi196.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-22T04:06:44.155 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-22T04:06:44.156 INFO:teuthology.run_tasks:Running task install... 2024-03-22T04:06:44.165 DEBUG:teuthology.task.install:project ceph 2024-03-22T04:06:44.165 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e'}} 2024-03-22T04:06:44.166 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e'} 2024-03-22T04:06:44.166 INFO:teuthology.task.install:Using flavor: default 2024-03-22T04:06:44.172 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-22T04:06:44.172 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-22T04:06:44.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-key list | grep Ceph 2024-03-22T04:06:44.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-key list | grep Ceph 2024-03-22T04:06:44.279 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-22T04:06:44.303 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-22T04:06:44.329 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-22T04:06:44.329 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph.com (release key) 2024-03-22T04:06:44.330 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-22T04:06:44.330 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-22T04:06:44.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e 2024-03-22T04:06:44.353 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-22T04:06:44.353 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph.com (release key) 2024-03-22T04:06:44.354 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-22T04:06:44.354 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-22T04:06:44.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e 2024-03-22T04:06:44.566 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default/ 2024-03-22T04:06:44.567 INFO:teuthology.task.install.deb:Package version is 19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T04:06:44.586 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default/ 2024-03-22T04:06:44.586 INFO:teuthology.task.install.deb:Package version is 19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T04:06:44.681 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:06:44.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-22T04:06:44.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2024-03-22T04:06:44.773 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:06:44.773 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-22T04:06:44.793 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-03-22T04:06:44.980 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-22T04:06:44.980 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-22T04:06:44.994 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy InRelease 2024-03-22T04:06:44.997 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-22T04:06:45.018 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-22T04:06:45.033 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-22T04:06:45.033 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-22T04:06:45.035 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-03-22T04:06:45.072 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-22T04:06:45.098 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy InRelease 2024-03-22T04:06:45.104 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-22T04:06:45.110 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-03-22T04:06:45.134 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-03-22T04:06:45.173 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-22T04:06:45.186 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-22T04:06:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-03-22T04:06:45.212 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-03-22T04:06:45.265 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-03-22T04:06:45.568 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 28.5 kB in 1s (51.3 kB/s) 2024-03-22T04:06:45.621 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 28.5 kB in 1s (40.6 kB/s) 2024-03-22T04:06:46.553 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-03-22T04:06:46.567 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2344-gc3a04c0e-1jammy cephadm=19.0.0-2344-gc3a04c0e-1jammy ceph-mds=19.0.0-2344-gc3a04c0e-1jammy ceph-mgr=19.0.0-2344-gc3a04c0e-1jammy ceph-common=19.0.0-2344-gc3a04c0e-1jammy ceph-fuse=19.0.0-2344-gc3a04c0e-1jammy ceph-test=19.0.0-2344-gc3a04c0e-1jammy ceph-volume=19.0.0-2344-gc3a04c0e-1jammy radosgw=19.0.0-2344-gc3a04c0e-1jammy python3-rados=19.0.0-2344-gc3a04c0e-1jammy python3-rgw=19.0.0-2344-gc3a04c0e-1jammy python3-cephfs=19.0.0-2344-gc3a04c0e-1jammy python3-rbd=19.0.0-2344-gc3a04c0e-1jammy libcephfs2=19.0.0-2344-gc3a04c0e-1jammy libcephfs-dev=19.0.0-2344-gc3a04c0e-1jammy librados2=19.0.0-2344-gc3a04c0e-1jammy librbd1=19.0.0-2344-gc3a04c0e-1jammy rbd-fuse=19.0.0-2344-gc3a04c0e-1jammy python3-cephfs=19.0.0-2344-gc3a04c0e-1jammy cephfs-shell=19.0.0-2344-gc3a04c0e-1jammy cephfs-top=19.0.0-2344-gc3a04c0e-1jammy cephfs-mirror=19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T04:06:46.578 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-03-22T04:06:46.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2344-gc3a04c0e-1jammy cephadm=19.0.0-2344-gc3a04c0e-1jammy ceph-mds=19.0.0-2344-gc3a04c0e-1jammy ceph-mgr=19.0.0-2344-gc3a04c0e-1jammy ceph-common=19.0.0-2344-gc3a04c0e-1jammy ceph-fuse=19.0.0-2344-gc3a04c0e-1jammy ceph-test=19.0.0-2344-gc3a04c0e-1jammy ceph-volume=19.0.0-2344-gc3a04c0e-1jammy radosgw=19.0.0-2344-gc3a04c0e-1jammy python3-rados=19.0.0-2344-gc3a04c0e-1jammy python3-rgw=19.0.0-2344-gc3a04c0e-1jammy python3-cephfs=19.0.0-2344-gc3a04c0e-1jammy python3-rbd=19.0.0-2344-gc3a04c0e-1jammy libcephfs2=19.0.0-2344-gc3a04c0e-1jammy libcephfs-dev=19.0.0-2344-gc3a04c0e-1jammy librados2=19.0.0-2344-gc3a04c0e-1jammy librbd1=19.0.0-2344-gc3a04c0e-1jammy rbd-fuse=19.0.0-2344-gc3a04c0e-1jammy python3-cephfs=19.0.0-2344-gc3a04c0e-1jammy cephfs-shell=19.0.0-2344-gc3a04c0e-1jammy cephfs-top=19.0.0-2344-gc3a04c0e-1jammy cephfs-mirror=19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T04:06:46.640 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-03-22T04:06:46.667 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-03-22T04:06:46.809 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-03-22T04:06:46.810 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-03-22T04:06:46.836 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-03-22T04:06:46.836 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-03-22T04:06:46.968 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-03-22T04:06:46.968 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-22T04:06:46.968 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-22T04:06:46.968 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-22T04:06:46.968 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-22T04:06:46.969 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-22T04:06:46.969 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-22T04:06:46.969 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-03-22T04:06:46.970 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-22T04:06:46.970 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-22T04:06:46.970 INFO:teuthology.orchestra.run.smithi196.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-22T04:06:46.970 INFO:teuthology.orchestra.run.smithi196.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-22T04:06:46.971 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-22T04:06:46.971 INFO:teuthology.orchestra.run.smithi196.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-22T04:06:46.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2024-03-22T04:06:46.973 INFO:teuthology.orchestra.run.smithi196.stdout: btrfs-tools 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-22T04:06:47.001 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-22T04:06:47.002 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-22T04:06:47.003 INFO:teuthology.orchestra.run.smithi086.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-22T04:06:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-22T04:06:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-22T04:06:47.004 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2024-03-22T04:06:47.004 INFO:teuthology.orchestra.run.smithi086.stdout: btrfs-tools 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-22T04:06:47.124 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-22T04:06:47.125 INFO:teuthology.orchestra.run.smithi196.stdout: socat unzip xmlstarlet zip 2024-03-22T04:06:47.151 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-03-22T04:06:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-22T04:06:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-22T04:06:47.151 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-22T04:06:47.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-22T04:06:47.153 INFO:teuthology.orchestra.run.smithi086.stdout: socat unzip xmlstarlet zip 2024-03-22T04:06:47.195 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 124 newly installed, 0 to remove and 288 not upgraded. 2024-03-22T04:06:47.195 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 179 MB of archives. 2024-03-22T04:06:47.195 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 820 MB of additional disk space will be used. 2024-03-22T04:06:47.195 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-22T04:06:47.216 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-22T04:06:47.223 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 124 newly installed, 0 to remove and 288 not upgraded. 2024-03-22T04:06:47.223 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 179 MB of archives. 2024-03-22T04:06:47.224 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 820 MB of additional disk space will be used. 2024-03-22T04:06:47.224 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-22T04:06:47.227 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-22T04:06:47.246 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-22T04:06:47.257 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-22T04:06:47.280 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-22T04:06:47.307 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-22T04:06:47.309 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-22T04:06:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-22T04:06:47.322 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-22T04:06:47.323 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2344-gc3a04c0e-1jammy [3,550 kB] 2024-03-22T04:06:47.339 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-22T04:06:47.343 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-22T04:06:47.358 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-22T04:06:47.364 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2344-gc3a04c0e-1jammy [3,550 kB] 2024-03-22T04:06:47.371 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-22T04:06:47.374 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-22T04:06:47.384 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-22T04:06:47.387 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-22T04:06:47.387 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-22T04:06:47.396 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-22T04:06:47.398 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-22T04:06:47.400 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-22T04:06:47.401 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-22T04:06:47.405 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-22T04:06:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-22T04:06:47.422 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-22T04:06:47.425 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-22T04:06:47.427 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-22T04:06:47.429 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-22T04:06:47.430 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-22T04:06:47.430 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-22T04:06:47.432 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-22T04:06:47.432 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-22T04:06:47.432 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-22T04:06:47.433 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-22T04:06:47.437 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-22T04:06:47.441 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-22T04:06:47.441 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-22T04:06:47.447 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-22T04:06:47.449 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-22T04:06:47.451 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-22T04:06:47.452 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-22T04:06:47.453 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-22T04:06:47.458 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-22T04:06:47.459 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-22T04:06:47.459 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-22T04:06:47.461 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-22T04:06:47.462 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-22T04:06:47.462 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-22T04:06:47.463 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-22T04:06:47.464 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-22T04:06:47.464 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-22T04:06:47.475 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-22T04:06:47.477 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-22T04:06:47.479 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-22T04:06:47.479 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-22T04:06:47.480 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-22T04:06:47.481 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-22T04:06:47.484 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-22T04:06:47.485 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-22T04:06:47.486 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-22T04:06:47.486 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-22T04:06:47.496 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-22T04:06:47.496 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-22T04:06:47.497 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-22T04:06:47.497 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-22T04:06:47.505 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-22T04:06:47.505 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-22T04:06:47.506 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-22T04:06:47.506 INFO:teuthology.orchestra.run.smithi196.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-22T04:06:47.507 INFO:teuthology.orchestra.run.smithi196.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-22T04:06:47.509 INFO:teuthology.orchestra.run.smithi196.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-22T04:06:47.518 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-22T04:06:47.519 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-22T04:06:47.521 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-22T04:06:47.522 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-22T04:06:47.523 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-22T04:06:47.523 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-22T04:06:47.529 INFO:teuthology.orchestra.run.smithi196.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-22T04:06:47.530 INFO:teuthology.orchestra.run.smithi196.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-22T04:06:47.531 INFO:teuthology.orchestra.run.smithi196.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-22T04:06:47.533 INFO:teuthology.orchestra.run.smithi196.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-22T04:06:47.542 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-22T04:06:47.543 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-22T04:06:47.543 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-22T04:06:47.544 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-22T04:06:47.553 INFO:teuthology.orchestra.run.smithi196.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-22T04:06:47.554 INFO:teuthology.orchestra.run.smithi196.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-22T04:06:47.556 INFO:teuthology.orchestra.run.smithi196.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-22T04:06:47.557 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2344-gc3a04c0e-1jammy [3,226 kB] 2024-03-22T04:06:47.576 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-22T04:06:47.577 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-22T04:06:47.587 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-22T04:06:47.587 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-22T04:06:47.588 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-22T04:06:47.588 INFO:teuthology.orchestra.run.smithi086.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-22T04:06:47.617 INFO:teuthology.orchestra.run.smithi196.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-22T04:06:47.618 INFO:teuthology.orchestra.run.smithi196.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-22T04:06:47.619 INFO:teuthology.orchestra.run.smithi086.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-22T04:06:47.620 INFO:teuthology.orchestra.run.smithi196.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-22T04:06:47.620 INFO:teuthology.orchestra.run.smithi196.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-22T04:06:47.621 INFO:teuthology.orchestra.run.smithi196.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-22T04:06:47.621 INFO:teuthology.orchestra.run.smithi086.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-22T04:06:47.622 INFO:teuthology.orchestra.run.smithi086.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-22T04:06:47.624 INFO:teuthology.orchestra.run.smithi086.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-22T04:06:47.645 INFO:teuthology.orchestra.run.smithi196.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-22T04:06:47.647 INFO:teuthology.orchestra.run.smithi196.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-22T04:06:47.648 INFO:teuthology.orchestra.run.smithi196.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-22T04:06:47.655 INFO:teuthology.orchestra.run.smithi086.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-22T04:06:47.657 INFO:teuthology.orchestra.run.smithi086.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-22T04:06:47.657 INFO:teuthology.orchestra.run.smithi086.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-22T04:06:47.658 INFO:teuthology.orchestra.run.smithi086.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-22T04:06:47.660 INFO:teuthology.orchestra.run.smithi086.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-22T04:06:47.661 INFO:teuthology.orchestra.run.smithi086.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2344-gc3a04c0e-1jammy [712 kB] 2024-03-22T04:06:47.679 INFO:teuthology.orchestra.run.smithi196.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-22T04:06:47.684 INFO:teuthology.orchestra.run.smithi086.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2344-gc3a04c0e-1jammy [356 kB] 2024-03-22T04:06:47.701 INFO:teuthology.orchestra.run.smithi196.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-22T04:06:47.702 INFO:teuthology.orchestra.run.smithi196.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-22T04:06:47.702 INFO:teuthology.orchestra.run.smithi196.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-22T04:06:47.703 INFO:teuthology.orchestra.run.smithi196.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-22T04:06:47.704 INFO:teuthology.orchestra.run.smithi196.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-22T04:06:47.718 INFO:teuthology.orchestra.run.smithi086.stdout:Get:57 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2344-gc3a04c0e-1jammy [32.8 kB] 2024-03-22T04:06:47.719 INFO:teuthology.orchestra.run.smithi086.stdout:Get:58 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2344-gc3a04c0e-1jammy [180 kB] 2024-03-22T04:06:47.729 INFO:teuthology.orchestra.run.smithi086.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-22T04:06:47.731 INFO:teuthology.orchestra.run.smithi086.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-22T04:06:47.732 INFO:teuthology.orchestra.run.smithi086.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-22T04:06:47.733 INFO:teuthology.orchestra.run.smithi086.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-22T04:06:47.734 INFO:teuthology.orchestra.run.smithi086.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-22T04:06:47.737 INFO:teuthology.orchestra.run.smithi086.stdout:Get:64 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2344-gc3a04c0e-1jammy [64.8 kB] 2024-03-22T04:06:47.738 INFO:teuthology.orchestra.run.smithi086.stdout:Get:65 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2344-gc3a04c0e-1jammy [330 kB] 2024-03-22T04:06:47.741 INFO:teuthology.orchestra.run.smithi196.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-22T04:06:47.742 INFO:teuthology.orchestra.run.smithi196.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-22T04:06:47.745 INFO:teuthology.orchestra.run.smithi196.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-22T04:06:47.747 INFO:teuthology.orchestra.run.smithi196.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-22T04:06:47.749 INFO:teuthology.orchestra.run.smithi196.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-22T04:06:47.757 INFO:teuthology.orchestra.run.smithi086.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-22T04:06:47.761 INFO:teuthology.orchestra.run.smithi086.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-22T04:06:47.761 INFO:teuthology.orchestra.run.smithi086.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-22T04:06:47.781 INFO:teuthology.orchestra.run.smithi196.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-22T04:06:47.782 INFO:teuthology.orchestra.run.smithi086.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-22T04:06:47.784 INFO:teuthology.orchestra.run.smithi196.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-22T04:06:47.787 INFO:teuthology.orchestra.run.smithi196.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-22T04:06:47.787 INFO:teuthology.orchestra.run.smithi196.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-22T04:06:47.788 INFO:teuthology.orchestra.run.smithi086.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-22T04:06:47.789 INFO:teuthology.orchestra.run.smithi196.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-22T04:06:47.789 INFO:teuthology.orchestra.run.smithi086.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-22T04:06:47.789 INFO:teuthology.orchestra.run.smithi086.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-22T04:06:47.790 INFO:teuthology.orchestra.run.smithi086.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-22T04:06:47.791 INFO:teuthology.orchestra.run.smithi086.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi196.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi196.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi196.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-22T04:06:47.811 INFO:teuthology.orchestra.run.smithi086.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-22T04:06:47.811 INFO:teuthology.orchestra.run.smithi086.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-22T04:06:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-22T04:06:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-22T04:06:47.815 INFO:teuthology.orchestra.run.smithi086.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-22T04:06:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-22T04:06:47.818 INFO:teuthology.orchestra.run.smithi086.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-22T04:06:47.825 INFO:teuthology.orchestra.run.smithi196.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-22T04:06:47.825 INFO:teuthology.orchestra.run.smithi196.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-22T04:06:47.827 INFO:teuthology.orchestra.run.smithi196.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-22T04:06:47.828 INFO:teuthology.orchestra.run.smithi196.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-22T04:06:47.836 INFO:teuthology.orchestra.run.smithi196.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-22T04:06:47.838 INFO:teuthology.orchestra.run.smithi086.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-22T04:06:47.843 INFO:teuthology.orchestra.run.smithi086.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-22T04:06:47.847 INFO:teuthology.orchestra.run.smithi086.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-22T04:06:47.848 INFO:teuthology.orchestra.run.smithi086.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-22T04:06:47.849 INFO:teuthology.orchestra.run.smithi086.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-22T04:06:47.852 INFO:teuthology.orchestra.run.smithi196.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-22T04:06:47.852 INFO:teuthology.orchestra.run.smithi196.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-22T04:06:47.854 INFO:teuthology.orchestra.run.smithi196.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-22T04:06:47.855 INFO:teuthology.orchestra.run.smithi196.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-22T04:06:47.873 INFO:teuthology.orchestra.run.smithi086.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-22T04:06:47.878 INFO:teuthology.orchestra.run.smithi086.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-22T04:06:47.879 INFO:teuthology.orchestra.run.smithi086.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-22T04:06:47.882 INFO:teuthology.orchestra.run.smithi086.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-22T04:06:47.882 INFO:teuthology.orchestra.run.smithi086.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-22T04:06:47.894 INFO:teuthology.orchestra.run.smithi086.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-22T04:06:47.895 INFO:teuthology.orchestra.run.smithi086.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-22T04:06:47.897 INFO:teuthology.orchestra.run.smithi086.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-22T04:06:47.898 INFO:teuthology.orchestra.run.smithi086.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-22T04:06:47.905 INFO:teuthology.orchestra.run.smithi086.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-22T04:06:47.917 INFO:teuthology.orchestra.run.smithi086.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-22T04:06:47.918 INFO:teuthology.orchestra.run.smithi086.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-22T04:06:47.920 INFO:teuthology.orchestra.run.smithi086.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-22T04:06:47.920 INFO:teuthology.orchestra.run.smithi086.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-22T04:06:48.048 INFO:teuthology.orchestra.run.smithi086.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2344-gc3a04c0e-1jammy [6,660 kB] 2024-03-22T04:06:48.287 INFO:teuthology.orchestra.run.smithi086.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2344-gc3a04c0e-1jammy [112 kB] 2024-03-22T04:06:48.289 INFO:teuthology.orchestra.run.smithi086.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2344-gc3a04c0e-1jammy [466 kB] 2024-03-22T04:06:48.313 INFO:teuthology.orchestra.run.smithi086.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2344-gc3a04c0e-1jammy [25.3 MB] 2024-03-22T04:06:48.986 INFO:teuthology.orchestra.run.smithi196.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2344-gc3a04c0e-1jammy [3,226 kB] 2024-03-22T04:06:49.303 INFO:teuthology.orchestra.run.smithi086.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2344-gc3a04c0e-1jammy [5,085 kB] 2024-03-22T04:06:49.491 INFO:teuthology.orchestra.run.smithi086.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2344-gc3a04c0e-1jammy [244 kB] 2024-03-22T04:06:49.500 INFO:teuthology.orchestra.run.smithi086.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2344-gc3a04c0e-1jammy [124 kB] 2024-03-22T04:06:49.607 INFO:teuthology.orchestra.run.smithi086.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2344-gc3a04c0e-1jammy [1,527 kB] 2024-03-22T04:06:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2344-gc3a04c0e-1jammy [6,256 kB] 2024-03-22T04:06:50.042 INFO:teuthology.orchestra.run.smithi196.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2344-gc3a04c0e-1jammy [712 kB] 2024-03-22T04:06:50.085 INFO:teuthology.orchestra.run.smithi086.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2344-gc3a04c0e-1jammy [22.8 MB] 2024-03-22T04:06:50.204 INFO:teuthology.orchestra.run.smithi196.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2344-gc3a04c0e-1jammy [356 kB] 2024-03-22T04:06:50.264 INFO:teuthology.orchestra.run.smithi196.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2344-gc3a04c0e-1jammy [32.8 kB] 2024-03-22T04:06:50.269 INFO:teuthology.orchestra.run.smithi196.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2344-gc3a04c0e-1jammy [180 kB] 2024-03-22T04:06:50.293 INFO:teuthology.orchestra.run.smithi196.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2344-gc3a04c0e-1jammy [64.8 kB] 2024-03-22T04:06:50.300 INFO:teuthology.orchestra.run.smithi196.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2344-gc3a04c0e-1jammy [330 kB] 2024-03-22T04:06:50.359 INFO:teuthology.orchestra.run.smithi196.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2344-gc3a04c0e-1jammy [6,660 kB] 2024-03-22T04:06:50.964 INFO:teuthology.orchestra.run.smithi086.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2344-gc3a04c0e-1jammy [14.0 kB] 2024-03-22T04:06:50.964 INFO:teuthology.orchestra.run.smithi086.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2344-gc3a04c0e-1jammy [870 kB] 2024-03-22T04:06:51.002 INFO:teuthology.orchestra.run.smithi086.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2344-gc3a04c0e-1jammy [2,435 kB] 2024-03-22T04:06:51.101 INFO:teuthology.orchestra.run.smithi086.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2344-gc3a04c0e-1jammy [779 kB] 2024-03-22T04:06:51.137 INFO:teuthology.orchestra.run.smithi086.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2344-gc3a04c0e-1jammy [146 kB] 2024-03-22T04:06:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2344-gc3a04c0e-1jammy [2,276 kB] 2024-03-22T04:06:51.239 INFO:teuthology.orchestra.run.smithi086.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2344-gc3a04c0e-1jammy [8,625 kB] 2024-03-22T04:06:51.517 INFO:teuthology.orchestra.run.smithi196.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2344-gc3a04c0e-1jammy [112 kB] 2024-03-22T04:06:51.527 INFO:teuthology.orchestra.run.smithi196.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2344-gc3a04c0e-1jammy [466 kB] 2024-03-22T04:06:51.588 INFO:teuthology.orchestra.run.smithi196.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2344-gc3a04c0e-1jammy [25.3 MB] 2024-03-22T04:06:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2344-gc3a04c0e-1jammy [14.2 kB] 2024-03-22T04:06:51.631 INFO:teuthology.orchestra.run.smithi086.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2344-gc3a04c0e-1jammy [50.3 MB] 2024-03-22T04:06:54.191 INFO:teuthology.orchestra.run.smithi196.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2344-gc3a04c0e-1jammy [5,085 kB] 2024-03-22T04:06:54.205 INFO:teuthology.orchestra.run.smithi086.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2344-gc3a04c0e-1jammy [121 kB] 2024-03-22T04:06:54.206 INFO:teuthology.orchestra.run.smithi086.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2344-gc3a04c0e-1jammy [229 kB] 2024-03-22T04:06:54.208 INFO:teuthology.orchestra.run.smithi086.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2344-gc3a04c0e-1jammy [34.0 kB] 2024-03-22T04:06:54.208 INFO:teuthology.orchestra.run.smithi086.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2344-gc3a04c0e-1jammy [23.4 kB] 2024-03-22T04:06:54.210 INFO:teuthology.orchestra.run.smithi086.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2344-gc3a04c0e-1jammy [38.9 kB] 2024-03-22T04:06:54.211 INFO:teuthology.orchestra.run.smithi086.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2344-gc3a04c0e-1jammy [13.2 MB] 2024-03-22T04:06:54.673 INFO:teuthology.orchestra.run.smithi196.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2344-gc3a04c0e-1jammy [244 kB] 2024-03-22T04:06:54.691 INFO:teuthology.orchestra.run.smithi196.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2344-gc3a04c0e-1jammy [124 kB] 2024-03-22T04:06:54.699 INFO:teuthology.orchestra.run.smithi196.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2344-gc3a04c0e-1jammy [1,527 kB] 2024-03-22T04:06:54.844 INFO:teuthology.orchestra.run.smithi196.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2344-gc3a04c0e-1jammy [6,256 kB] 2024-03-22T04:06:54.889 INFO:teuthology.orchestra.run.smithi086.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2344-gc3a04c0e-1jammy [91.6 kB] 2024-03-22T04:06:55.400 INFO:teuthology.orchestra.run.smithi196.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2344-gc3a04c0e-1jammy [22.8 MB] 2024-03-22T04:06:55.522 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 179 MB in 8s (23.2 MB/s) 2024-03-22T04:06:55.645 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-22T04:06:55.713 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132923 files and directories currently installed.) 2024-03-22T04:06:55.717 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-22T04:06:55.760 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-22T04:06:56.063 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-22T04:06:56.076 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-22T04:06:56.121 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-22T04:06:56.405 INFO:teuthology.orchestra.run.smithi196.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2344-gc3a04c0e-1jammy [14.0 kB] 2024-03-22T04:06:56.405 INFO:teuthology.orchestra.run.smithi196.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2344-gc3a04c0e-1jammy [870 kB] 2024-03-22T04:06:56.431 INFO:teuthology.orchestra.run.smithi196.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2344-gc3a04c0e-1jammy [2,435 kB] 2024-03-22T04:06:56.466 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cmd2. 2024-03-22T04:06:56.479 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-22T04:06:56.519 INFO:teuthology.orchestra.run.smithi196.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2344-gc3a04c0e-1jammy [779 kB] 2024-03-22T04:06:56.524 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-22T04:06:56.552 INFO:teuthology.orchestra.run.smithi196.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2344-gc3a04c0e-1jammy [146 kB] 2024-03-22T04:06:56.553 INFO:teuthology.orchestra.run.smithi196.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2344-gc3a04c0e-1jammy [2,276 kB] 2024-03-22T04:06:56.636 INFO:teuthology.orchestra.run.smithi196.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2344-gc3a04c0e-1jammy [8,625 kB] 2024-03-22T04:06:56.869 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-22T04:06:56.882 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-22T04:06:56.960 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-22T04:06:56.964 INFO:teuthology.orchestra.run.smithi196.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2344-gc3a04c0e-1jammy [14.2 kB] 2024-03-22T04:06:56.964 INFO:teuthology.orchestra.run.smithi196.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2344-gc3a04c0e-1jammy [50.3 MB] 2024-03-22T04:06:57.231 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-22T04:06:57.244 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-22T04:06:57.289 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-22T04:06:57.650 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-22T04:06:57.664 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-22T04:06:57.708 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-22T04:06:58.153 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-22T04:06:58.167 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-22T04:06:58.219 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:06:58.623 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-22T04:06:58.638 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-22T04:06:58.682 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:06:58.858 INFO:teuthology.orchestra.run.smithi196.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2344-gc3a04c0e-1jammy [121 kB] 2024-03-22T04:06:58.859 INFO:teuthology.orchestra.run.smithi196.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2344-gc3a04c0e-1jammy [229 kB] 2024-03-22T04:06:58.861 INFO:teuthology.orchestra.run.smithi196.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2344-gc3a04c0e-1jammy [34.0 kB] 2024-03-22T04:06:58.862 INFO:teuthology.orchestra.run.smithi196.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2344-gc3a04c0e-1jammy [23.4 kB] 2024-03-22T04:06:58.862 INFO:teuthology.orchestra.run.smithi196.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2344-gc3a04c0e-1jammy [38.9 kB] 2024-03-22T04:06:58.863 INFO:teuthology.orchestra.run.smithi196.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2344-gc3a04c0e-1jammy [13.2 MB] 2024-03-22T04:06:59.026 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-22T04:06:59.041 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-22T04:06:59.084 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:06:59.421 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-22T04:06:59.424 INFO:teuthology.orchestra.run.smithi196.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240321.172515/c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2344-gc3a04c0e-1jammy [91.6 kB] 2024-03-22T04:06:59.436 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-22T04:06:59.479 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-22T04:06:59.807 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librados2. 2024-03-22T04:06:59.822 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../010-librados2_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:06:59.857 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librados2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:00.150 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 179 MB in 12s (14.6 MB/s) 2024-03-22T04:07:00.293 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librbd1. 2024-03-22T04:07:00.306 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../011-librbd1_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:00.351 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librbd1 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:00.359 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-22T04:07:00.425 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132923 files and directories currently installed.) 2024-03-22T04:07:00.428 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-22T04:07:00.474 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-22T04:07:00.744 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-22T04:07:00.754 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs2. 2024-03-22T04:07:00.757 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-22T04:07:00.767 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:00.810 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-22T04:07:00.812 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:01.063 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cmd2. 2024-03-22T04:07:01.076 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-22T04:07:01.098 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rados. 2024-03-22T04:07:01.111 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:01.121 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-22T04:07:01.148 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rados (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:01.434 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-22T04:07:01.441 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-22T04:07:01.447 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:01.455 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-22T04:07:01.484 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-argparse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:01.541 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-22T04:07:01.711 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cephfs. 2024-03-22T04:07:01.725 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:01.761 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cephfs (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:01.894 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-22T04:07:01.908 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-22T04:07:01.952 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-22T04:07:02.056 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-22T04:07:02.069 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:02.139 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-common (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:02.289 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-22T04:07:02.303 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-22T04:07:02.339 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-22T04:07:02.467 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-prettytable. 2024-03-22T04:07:02.481 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-22T04:07:02.525 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-22T04:07:02.701 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-22T04:07:02.716 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-22T04:07:02.767 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:07:02.786 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rbd. 2024-03-22T04:07:02.800 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:02.844 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rbd (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:03.158 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-22T04:07:03.172 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-22T04:07:03.195 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-22T04:07:03.210 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-22T04:07:03.216 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-22T04:07:03.253 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:07:03.510 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-22T04:07:03.524 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-22T04:07:03.568 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-22T04:07:03.573 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-22T04:07:03.588 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-22T04:07:03.623 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:07:03.846 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-22T04:07:03.861 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-22T04:07:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-22T04:07:03.993 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-22T04:07:04.007 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-22T04:07:04.051 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-22T04:07:04.199 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua5.1. 2024-03-22T04:07:04.214 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-22T04:07:04.320 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librados2. 2024-03-22T04:07:04.335 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../010-librados2_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:04.379 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librados2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:04.558 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-22T04:07:04.823 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librbd1. 2024-03-22T04:07:04.838 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../011-librbd1_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:04.881 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librbd1 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-any. 2024-03-22T04:07:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-22T04:07:05.018 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-22T04:07:05.246 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package zip. 2024-03-22T04:07:05.261 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-22T04:07:05.304 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking zip (3.0-12build2) ... 2024-03-22T04:07:05.381 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs2. 2024-03-22T04:07:05.406 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:05.659 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:05.699 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package unzip. 2024-03-22T04:07:05.714 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-22T04:07:05.765 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-22T04:07:05.903 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rados. 2024-03-22T04:07:05.918 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:05.953 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rados (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:06.169 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package luarocks. 2024-03-22T04:07:06.184 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-22T04:07:06.222 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-22T04:07:06.227 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-22T04:07:06.237 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:06.280 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-argparse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:06.575 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cephfs. 2024-03-22T04:07:06.590 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:06.593 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librgw2. 2024-03-22T04:07:06.606 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../027-librgw2_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:06.633 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cephfs (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:06.651 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librgw2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:06.953 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-22T04:07:06.968 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:07.011 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-common (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rgw. 2024-03-22T04:07:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rgw (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:07.380 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-prettytable. 2024-03-22T04:07:07.395 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-22T04:07:07.438 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-22T04:07:07.675 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rbd. 2024-03-22T04:07:07.690 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:07.697 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-22T04:07:07.712 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-22T04:07:07.724 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rbd (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:07.764 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-22T04:07:08.094 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-22T04:07:08.109 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-22T04:07:08.152 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-22T04:07:08.159 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libradosstriper1. 2024-03-22T04:07:08.173 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:08.225 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libradosstriper1 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:08.430 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-22T04:07:08.445 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-22T04:07:08.488 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-22T04:07:08.662 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-common. 2024-03-22T04:07:08.676 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:08.728 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-common (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:08.799 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-22T04:07:08.815 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-22T04:07:08.874 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-22T04:07:09.152 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua5.1. 2024-03-22T04:07:09.167 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-22T04:07:09.211 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-22T04:07:09.505 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-any. 2024-03-22T04:07:09.520 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-22T04:07:09.563 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-22T04:07:09.850 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package zip. 2024-03-22T04:07:09.865 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-22T04:07:09.908 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking zip (3.0-12build2) ... 2024-03-22T04:07:09.958 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-base. 2024-03-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:10.034 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-base (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:10.202 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package unzip. 2024-03-22T04:07:10.218 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-22T04:07:10.254 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-22T04:07:10.621 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-22T04:07:10.634 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-22T04:07:10.656 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package luarocks. 2024-03-22T04:07:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-22T04:07:10.678 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-22T04:07:10.714 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-22T04:07:10.965 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cheroot. 2024-03-22T04:07:10.979 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-22T04:07:11.023 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-22T04:07:11.105 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librgw2. 2024-03-22T04:07:11.119 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../027-librgw2_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:11.163 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librgw2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:11.334 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-22T04:07:11.348 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-22T04:07:11.384 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-22T04:07:11.641 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rgw. 2024-03-22T04:07:11.655 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:11.687 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-22T04:07:11.691 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rgw (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:11.701 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-22T04:07:11.745 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-22T04:07:12.029 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-22T04:07:12.044 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-22T04:07:12.047 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-22T04:07:12.062 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-22T04:07:12.085 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-22T04:07:12.105 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-22T04:07:12.388 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libradosstriper1. 2024-03-22T04:07:12.400 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempora. 2024-03-22T04:07:12.403 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:12.415 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-22T04:07:12.446 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libradosstriper1 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:12.458 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-22T04:07:12.744 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-portend. 2024-03-22T04:07:12.759 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-22T04:07:12.802 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-22T04:07:12.899 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-common. 2024-03-22T04:07:12.914 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:12.949 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-common (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:13.113 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-22T04:07:13.128 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-22T04:07:13.172 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-22T04:07:13.483 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-22T04:07:13.498 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-22T04:07:13.541 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-22T04:07:13.853 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-natsort. 2024-03-22T04:07:13.867 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-22T04:07:13.903 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-22T04:07:14.055 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-base. 2024-03-22T04:07:14.070 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:14.122 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-base (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:14.231 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-logutils. 2024-03-22T04:07:14.245 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-22T04:07:14.289 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-22T04:07:14.592 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-22T04:07:14.606 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-22T04:07:14.650 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-22T04:07:14.700 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-22T04:07:14.714 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-22T04:07:14.758 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-22T04:07:15.036 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cheroot. 2024-03-22T04:07:15.051 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-22T04:07:15.086 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-22T04:07:15.144 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-mako. 2024-03-22T04:07:15.162 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-22T04:07:15.219 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-22T04:07:15.372 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-22T04:07:15.387 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-22T04:07:15.422 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-22T04:07:15.539 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-22T04:07:15.554 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-22T04:07:15.589 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-22T04:07:15.694 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-22T04:07:15.709 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-22T04:07:15.752 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-22T04:07:15.875 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-22T04:07:15.890 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-22T04:07:15.933 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-22T04:07:16.055 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-22T04:07:16.070 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-22T04:07:16.105 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-22T04:07:16.235 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webob. 2024-03-22T04:07:16.250 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-22T04:07:16.294 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-22T04:07:16.340 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempora. 2024-03-22T04:07:16.355 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-22T04:07:16.391 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-22T04:07:16.655 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-waitress. 2024-03-22T04:07:16.670 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-22T04:07:16.702 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-portend. 2024-03-22T04:07:16.716 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-22T04:07:16.717 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-22T04:07:16.760 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-22T04:07:17.041 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempita. 2024-03-22T04:07:17.056 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-22T04:07:17.096 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-22T04:07:17.099 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-22T04:07:17.111 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-22T04:07:17.154 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-22T04:07:17.386 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-paste. 2024-03-22T04:07:17.400 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-22T04:07:17.432 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-22T04:07:17.435 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-22T04:07:17.447 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-22T04:07:17.482 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-22T04:07:17.746 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-22T04:07:17.761 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-22T04:07:17.805 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-22T04:07:17.869 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-natsort. 2024-03-22T04:07:17.883 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-22T04:07:17.927 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-22T04:07:18.082 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-22T04:07:18.097 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-22T04:07:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-22T04:07:18.238 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-logutils. 2024-03-22T04:07:18.253 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-22T04:07:18.296 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-22T04:07:18.419 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webtest. 2024-03-22T04:07:18.434 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-22T04:07:18.485 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-22T04:07:18.616 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-22T04:07:18.631 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-22T04:07:18.674 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-22T04:07:18.821 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pecan. 2024-03-22T04:07:18.836 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-22T04:07:18.879 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-22T04:07:18.944 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-mako. 2024-03-22T04:07:18.959 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-22T04:07:19.002 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-22T04:07:19.217 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-certifi. 2024-03-22T04:07:19.231 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-22T04:07:19.275 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-22T04:07:19.330 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-22T04:07:19.345 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-22T04:07:19.388 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-22T04:07:19.595 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-idna. 2024-03-22T04:07:19.609 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-22T04:07:19.653 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-22T04:07:19.691 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-22T04:07:19.706 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-22T04:07:19.749 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-22T04:07:19.980 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-urllib3. 2024-03-22T04:07:19.995 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-22T04:07:20.039 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-22T04:07:20.677 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webob. 2024-03-22T04:07:20.693 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-22T04:07:20.727 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-22T04:07:21.092 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests. 2024-03-22T04:07:21.107 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-22T04:07:21.167 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-22T04:07:21.280 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-waitress. 2024-03-22T04:07:21.295 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-22T04:07:21.358 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-22T04:07:21.478 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-22T04:07:21.495 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-22T04:07:21.528 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-22T04:07:21.649 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempita. 2024-03-22T04:07:21.664 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-22T04:07:21.699 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-22T04:07:21.814 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-22T04:07:21.829 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:21.872 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:21.986 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-paste. 2024-03-22T04:07:22.000 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-22T04:07:22.044 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-22T04:07:22.252 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-22T04:07:22.266 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:22.335 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:22.380 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-22T04:07:22.394 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-22T04:07:22.438 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-22T04:07:22.646 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-22T04:07:22.660 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-22T04:07:22.704 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-22T04:07:22.741 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-22T04:07:22.756 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-22T04:07:22.799 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-22T04:07:22.998 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-openssl. 2024-03-22T04:07:23.013 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-22T04:07:23.048 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-22T04:07:23.127 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webtest. 2024-03-22T04:07:23.142 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-22T04:07:23.185 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-22T04:07:23.376 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr. 2024-03-22T04:07:23.391 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:23.434 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:23.472 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pecan. 2024-03-22T04:07:23.487 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-22T04:07:23.521 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-22T04:07:23.837 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mon. 2024-03-22T04:07:23.852 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:23.895 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mon (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:23.901 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-certifi. 2024-03-22T04:07:23.916 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-22T04:07:23.959 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-22T04:07:24.237 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-idna. 2024-03-22T04:07:24.249 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-22T04:07:24.278 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-22T04:07:24.465 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-osd. 2024-03-22T04:07:24.479 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:24.524 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-osd (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:24.581 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-urllib3. 2024-03-22T04:07:24.594 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-22T04:07:24.631 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-22T04:07:24.926 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests. 2024-03-22T04:07:24.938 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-22T04:07:24.975 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-22T04:07:25.270 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-22T04:07:25.286 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-22T04:07:25.353 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-22T04:07:25.419 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph. 2024-03-22T04:07:25.433 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../068-ceph_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:25.485 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:25.614 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-22T04:07:25.628 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:25.673 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:25.787 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-fuse. 2024-03-22T04:07:25.802 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:25.845 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-fuse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:26.061 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-22T04:07:26.074 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:26.144 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:26.232 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mds. 2024-03-22T04:07:26.246 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:26.289 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mds (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:26.397 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-22T04:07:26.410 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-22T04:07:26.447 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-22T04:07:26.659 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephadm. 2024-03-22T04:07:26.673 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../071-cephadm_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:26.717 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephadm (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:26.749 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-openssl. 2024-03-22T04:07:26.762 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-22T04:07:26.807 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-22T04:07:26.995 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-22T04:07:27.009 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-22T04:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-22T04:07:27.177 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr. 2024-03-22T04:07:27.190 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:27.235 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:27.381 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-22T04:07:27.395 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-22T04:07:27.438 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-22T04:07:27.580 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mon. 2024-03-22T04:07:27.593 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:27.638 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mon (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:28.200 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-osd. 2024-03-22T04:07:28.213 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:28.258 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-osd (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:28.393 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-babel. 2024-03-22T04:07:28.407 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-22T04:07:28.452 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-22T04:07:28.762 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jinja2. 2024-03-22T04:07:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-22T04:07:28.821 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-22T04:07:28.986 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph. 2024-03-22T04:07:28.999 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../068-ceph_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:29.044 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:29.157 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-22T04:07:29.170 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:29.215 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:29.380 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-fuse. 2024-03-22T04:07:29.394 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:29.438 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-fuse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:29.577 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-22T04:07:29.590 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-22T04:07:29.635 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-22T04:07:29.816 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mds. 2024-03-22T04:07:29.830 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:29.874 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mds (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:29.937 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-routes. 2024-03-22T04:07:29.951 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-22T04:07:29.996 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-22T04:07:30.252 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephadm. 2024-03-22T04:07:30.266 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../071-cephadm_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:30.302 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephadm (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-22T04:07:30.362 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:30.407 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:30.588 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-22T04:07:30.601 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-22T04:07:30.655 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-22T04:07:30.924 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-22T04:07:30.937 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-22T04:07:30.982 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-22T04:07:31.603 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-22T04:07:31.610 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-22T04:07:31.644 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-22T04:07:31.978 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-babel. 2024-03-22T04:07:31.991 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-22T04:07:32.028 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-22T04:07:32.140 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-joblib. 2024-03-22T04:07:32.147 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-22T04:07:32.189 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-22T04:07:32.339 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jinja2. 2024-03-22T04:07:32.351 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-22T04:07:32.389 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-22T04:07:32.551 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-22T04:07:32.560 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-22T04:07:32.678 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-22T04:07:32.759 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-22T04:07:32.771 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:32.808 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:32.956 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn. 2024-03-22T04:07:32.964 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-22T04:07:33.005 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-22T04:07:33.145 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-22T04:07:33.158 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-22T04:07:33.195 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-22T04:07:33.523 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-routes. 2024-03-22T04:07:33.536 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-22T04:07:33.581 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-22T04:07:33.705 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-22T04:07:33.714 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:33.747 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:33.917 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-22T04:07:33.933 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:33.975 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:34.650 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cachetools. 2024-03-22T04:07:34.659 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-22T04:07:34.700 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-22T04:07:35.020 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-22T04:07:35.028 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-22T04:07:35.069 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-22T04:07:35.180 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-22T04:07:35.187 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-22T04:07:35.230 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-22T04:07:35.372 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-22T04:07:35.384 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-22T04:07:35.422 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-22T04:07:35.725 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-joblib. 2024-03-22T04:07:35.734 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-22T04:07:35.766 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-22T04:07:35.984 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rsa. 2024-03-22T04:07:35.996 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-22T04:07:36.050 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-22T04:07:36.128 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-22T04:07:36.141 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-22T04:07:36.255 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-22T04:07:36.370 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-google-auth. 2024-03-22T04:07:36.384 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-22T04:07:36.429 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-22T04:07:36.725 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn. 2024-03-22T04:07:36.737 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-22T04:07:36.748 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-22T04:07:36.759 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-22T04:07:36.766 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-22T04:07:36.815 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-22T04:07:37.084 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-websocket. 2024-03-22T04:07:37.096 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-22T04:07:37.134 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-22T04:07:37.416 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-22T04:07:37.426 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:37.454 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-22T04:07:37.465 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-22T04:07:37.466 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:37.503 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-22T04:07:38.159 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-22T04:07:38.169 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:38.209 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:38.445 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cachetools. 2024-03-22T04:07:38.473 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-22T04:07:38.486 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-22T04:07:38.496 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-22T04:07:38.503 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-22T04:07:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-22T04:07:38.764 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-22T04:07:38.776 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-22T04:07:38.805 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-22T04:07:38.872 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-22T04:07:38.882 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-22T04:07:38.922 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-22T04:07:39.117 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-22T04:07:39.129 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-22T04:07:39.167 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-22T04:07:39.216 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package jq. 2024-03-22T04:07:39.229 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-22T04:07:39.266 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-22T04:07:39.470 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rsa. 2024-03-22T04:07:39.482 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-22T04:07:39.519 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package socat. 2024-03-22T04:07:39.519 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-22T04:07:39.530 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-22T04:07:39.568 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-22T04:07:39.838 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package xmlstarlet. 2024-03-22T04:07:39.849 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-22T04:07:39.865 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-google-auth. 2024-03-22T04:07:39.877 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-22T04:07:39.879 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-22T04:07:39.915 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-22T04:07:40.217 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-test. 2024-03-22T04:07:40.218 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-22T04:07:40.228 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:40.230 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-22T04:07:40.267 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-test (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:40.267 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-22T04:07:40.604 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-websocket. 2024-03-22T04:07:40.616 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-22T04:07:40.654 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-22T04:07:40.949 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-22T04:07:40.961 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-22T04:07:40.998 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-22T04:07:41.812 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-22T04:07:41.823 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:41.862 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:42.206 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-22T04:07:42.217 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-22T04:07:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-volume. 2024-03-22T04:07:42.248 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:42.256 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-22T04:07:42.287 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-volume (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:42.592 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-22T04:07:42.603 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-22T04:07:42.642 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-22T04:07:42.658 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-22T04:07:42.667 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:42.708 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-mirror (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package jq. 2024-03-22T04:07:42.889 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-22T04:07:42.945 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-22T04:07:43.046 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-colorama. 2024-03-22T04:07:43.056 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-22T04:07:43.087 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-22T04:07:43.181 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package socat. 2024-03-22T04:07:43.194 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-22T04:07:43.239 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-22T04:07:43.365 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-shell. 2024-03-22T04:07:43.374 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:43.414 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-shell (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:43.517 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package xmlstarlet. 2024-03-22T04:07:43.529 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-22T04:07:43.567 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-22T04:07:43.675 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-top. 2024-03-22T04:07:43.685 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:43.725 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-top (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:43.913 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-test. 2024-03-22T04:07:43.924 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:43.962 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-test (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:43.977 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-22T04:07:43.987 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:44.027 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs-dev (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:44.372 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-22T04:07:44.382 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-22T04:07:44.421 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-22T04:07:44.734 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-22T04:07:44.743 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-22T04:07:44.775 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-22T04:07:45.146 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package nvme-cli. 2024-03-22T04:07:45.156 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-22T04:07:45.195 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-22T04:07:45.633 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pkg-config. 2024-03-22T04:07:45.643 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-22T04:07:45.683 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-22T04:07:45.833 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-volume. 2024-03-22T04:07:45.845 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:45.883 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-volume (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:45.944 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-22T04:07:45.954 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-22T04:07:46.003 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-22T04:07:46.270 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-22T04:07:46.283 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:46.345 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-mirror (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:46.431 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-22T04:07:46.443 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-22T04:07:46.480 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-22T04:07:46.614 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-colorama. 2024-03-22T04:07:46.627 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-22T04:07:46.656 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-22T04:07:46.800 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastescript. 2024-03-22T04:07:46.811 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-22T04:07:46.849 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-22T04:07:46.884 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-shell. 2024-03-22T04:07:46.895 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:46.933 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-shell (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:47.145 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pluggy. 2024-03-22T04:07:47.155 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-22T04:07:47.194 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-22T04:07:47.228 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-top. 2024-03-22T04:07:47.240 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2344-gc3a04c0e-1jammy_all.deb ... 2024-03-22T04:07:47.278 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-top (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:47.506 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-psutil. 2024-03-22T04:07:47.516 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-22T04:07:47.530 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-22T04:07:47.542 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:47.555 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-22T04:07:47.580 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs-dev (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:47.866 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-22T04:07:47.881 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-22T04:07:47.908 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-py. 2024-03-22T04:07:47.919 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-22T04:07:47.957 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-22T04:07:47.958 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-22T04:07:48.236 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pygments. 2024-03-22T04:07:48.247 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-22T04:07:48.253 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-22T04:07:48.266 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-22T04:07:48.285 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-22T04:07:48.303 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-22T04:07:48.666 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package nvme-cli. 2024-03-22T04:07:48.679 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-22T04:07:48.732 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-22T04:07:48.741 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-22T04:07:48.748 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-22T04:07:48.781 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-22T04:07:49.109 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-toml. 2024-03-22T04:07:49.119 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-22T04:07:49.159 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-22T04:07:49.162 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package pkg-config. 2024-03-22T04:07:49.175 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-22T04:07:49.220 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-22T04:07:49.436 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pytest. 2024-03-22T04:07:49.447 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-22T04:07:49.486 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-22T04:07:49.489 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-22T04:07:49.502 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-22T04:07:49.572 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-22T04:07:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplejson. 2024-03-22T04:07:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-22T04:07:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-22T04:07:49.984 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-22T04:07:49.997 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-22T04:07:50.042 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-22T04:07:50.150 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-22T04:07:50.160 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-22T04:07:50.200 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-22T04:07:50.362 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastescript. 2024-03-22T04:07:50.375 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-22T04:07:50.411 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-22T04:07:50.656 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pluggy. 2024-03-22T04:07:50.669 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-22T04:07:50.714 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-22T04:07:50.987 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package radosgw. 2024-03-22T04:07:50.997 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../122-radosgw_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:51.025 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-psutil. 2024-03-22T04:07:51.028 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking radosgw (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:51.038 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-22T04:07:51.083 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-22T04:07:51.430 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-py. 2024-03-22T04:07:51.443 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-22T04:07:51.488 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-22T04:07:51.640 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package rbd-fuse. 2024-03-22T04:07:51.650 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:51.690 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking rbd-fuse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:51.842 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pygments. 2024-03-22T04:07:51.855 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-22T04:07:51.900 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-22T04:07:51.972 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-22T04:07:52.211 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-22T04:07:52.345 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-22T04:07:52.362 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-22T04:07:52.374 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-22T04:07:52.412 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-22T04:07:52.606 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-22T04:07:52.664 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-toml. 2024-03-22T04:07:52.676 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-22T04:07:52.737 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-22T04:07:53.007 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-22T04:07:53.042 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pytest. 2024-03-22T04:07:53.054 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-22T04:07:53.092 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-22T04:07:53.395 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplejson. 2024-03-22T04:07:53.407 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-22T04:07:53.445 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-22T04:07:53.614 INFO:teuthology.orchestra.run.smithi086.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-22T04:07:53.663 INFO:teuthology.orchestra.run.smithi086.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-03-22T04:07:53.698 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-22T04:07:53.705 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephadm (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:53.710 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-22T04:07:53.739 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-22T04:07:54.240 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user cephadm....done 2024-03-22T04:07:54.352 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-22T04:07:54.393 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package radosgw. 2024-03-22T04:07:54.404 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../122-radosgw_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:54.442 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking radosgw (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:54.637 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-22T04:07:54.905 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-22T04:07:55.054 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package rbd-fuse. 2024-03-22T04:07:55.066 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2344-gc3a04c0e-1jammy_amd64.deb ... 2024-03-22T04:07:55.072 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-22T04:07:55.104 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking rbd-fuse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:55.332 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-22T04:07:55.394 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-22T04:07:55.599 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-22T04:07:55.667 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-22T04:07:55.725 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-22T04:07:55.793 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-22T04:07:56.091 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-22T04:07:56.277 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-22T04:07:56.505 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-22T04:07:56.629 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-22T04:07:56.913 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-22T04:07:57.069 INFO:teuthology.orchestra.run.smithi196.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-22T04:07:57.109 INFO:teuthology.orchestra.run.smithi196.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-03-22T04:07:57.153 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephadm (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:57.181 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-22T04:07:57.465 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-22T04:07:57.671 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user cephadm....done 2024-03-22T04:07:57.767 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-22T04:07:57.800 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-22T04:07:58.001 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-22T04:07:58.060 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-22T04:07:58.252 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-22T04:07:58.342 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-22T04:07:58.470 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-22T04:07:58.520 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-argparse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:07:58.730 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-22T04:07:58.813 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-22T04:07:58.938 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-22T04:07:58.997 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-22T04:07:59.064 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-22T04:07:59.157 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-22T04:07:59.332 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-22T04:07:59.458 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-22T04:07:59.475 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-22T04:07:59.709 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-22T04:07:59.844 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-22T04:08:00.145 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-22T04:08:00.155 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-22T04:08:00.190 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-22T04:08:00.254 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-22T04:08:00.380 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-22T04:08:00.413 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-22T04:08:00.697 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-22T04:08:00.748 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-22T04:08:00.874 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-22T04:08:00.982 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-22T04:08:01.167 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-22T04:08:01.233 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-22T04:08:01.426 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-22T04:08:01.501 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-22T04:08:01.719 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up zip (3.0-12build2) ... 2024-03-22T04:08:01.785 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-argparse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:01.862 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-22T04:08:02.054 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-22T04:08:02.179 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-22T04:08:02.304 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-22T04:08:02.547 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-22T04:08:02.589 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-22T04:08:02.715 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-22T04:08:02.815 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-22T04:08:02.941 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-22T04:08:02.974 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-22T04:08:03.066 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-22T04:08:03.363 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-22T04:08:03.376 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-22T04:08:03.421 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-22T04:08:03.486 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-22T04:08:03.620 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-22T04:08:03.752 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-22T04:08:03.996 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-22T04:08:04.003 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-22T04:08:04.122 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-22T04:08:04.397 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-22T04:08:04.398 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-22T04:08:04.666 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-22T04:08:04.714 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-22T04:08:04.992 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up zip (3.0-12build2) ... 2024-03-22T04:08:05.040 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-22T04:08:05.101 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-22T04:08:05.392 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-22T04:08:05.635 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-22T04:08:05.785 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-common (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:05.786 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-22T04:08:06.063 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-22T04:08:06.095 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-idna (3.3-1) ... 2024-03-22T04:08:06.188 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-22T04:08:06.337 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-22T04:08:06.438 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-22T04:08:06.666 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-22T04:08:07.383 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-22T04:08:07.634 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-22T04:08:07.660 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-22T04:08:07.904 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:08:08.027 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-22T04:08:08.038 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-22T04:08:08.320 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-22T04:08:08.322 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-22T04:08:08.448 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-22T04:08:08.574 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-22T04:08:08.638 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-22T04:08:08.875 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-22T04:08:08.998 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-22T04:08:09.127 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-22T04:08:09.265 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-22T04:08:09.378 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:08:09.391 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-common (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:09.504 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-22T04:08:09.692 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-idna (3.3-1) ... 2024-03-22T04:08:09.814 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-22T04:08:10.060 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-22T04:08:10.089 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-22T04:08:10.366 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-22T04:08:10.633 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-22T04:08:10.876 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-22T04:08:11.135 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-22T04:08:11.266 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-22T04:08:11.270 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-22T04:08:11.526 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:08:11.554 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-22T04:08:11.651 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-22T04:08:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-22T04:08:11.911 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-22T04:08:12.015 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-22T04:08:12.037 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-22T04:08:12.188 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-22T04:08:12.324 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-22T04:08:12.490 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-22T04:08:12.584 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-22T04:08:12.685 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-22T04:08:12.757 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-22T04:08:12.844 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-22T04:08:12.932 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-22T04:08:13.016 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:08:13.142 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-22T04:08:13.171 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-22T04:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-22T04:08:13.444 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-22T04:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-22T04:08:13.711 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-22T04:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-22T04:08:14.021 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-22T04:08:14.100 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-22T04:08:14.288 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-22T04:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:08:14.460 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-22T04:08:14.573 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-22T04:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-22T04:08:14.840 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-22T04:08:14.949 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-22T04:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-22T04:08:15.217 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-22T04:08:15.318 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-22T04:08:15.414 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-22T04:08:15.617 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-22T04:08:15.628 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-22T04:08:15.868 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-22T04:08:15.945 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-22T04:08:16.144 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-22T04:08:16.213 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-22T04:08:16.339 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-22T04:08:16.412 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-22T04:08:16.465 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-22T04:08:16.553 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-22T04:08:16.783 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-22T04:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-22T04:08:16.909 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-22T04:08:17.090 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-22T04:08:17.327 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-22T04:08:17.443 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-22T04:08:17.577 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librados2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:17.603 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-22T04:08:17.678 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:17.729 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-22T04:08:17.804 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-22T04:08:18.022 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-22T04:08:18.148 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-22T04:08:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:18.290 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libradosstriper1 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:18.416 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-22T04:08:18.466 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-22T04:08:18.740 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librbd1 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:18.808 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-22T04:08:18.869 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-modules-core (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:18.994 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-fuse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:19.084 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-22T04:08:19.217 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-22T04:08:19.217 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-22T04:08:19.262 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-22T04:08:19.572 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-22T04:08:19.848 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-22T04:08:19.955 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs-dev (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:20.116 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-22T04:08:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rados (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:20.525 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librgw2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:20.526 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-22T04:08:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-top (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:20.751 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-22T04:08:20.803 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-22T04:08:21.155 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-22T04:08:21.281 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librados2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:21.406 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:21.532 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-22T04:08:21.886 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:21.995 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libradosstriper1 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.005 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rbd (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.121 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-22T04:08:22.130 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up rbd-fuse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.257 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rgw (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.414 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librbd1 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.515 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-modules-core (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.658 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-fuse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.708 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cephfs (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.825 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-shell (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:22.866 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-22T04:08:22.866 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-22T04:08:22.926 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-common (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:23.509 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs-dev (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:23.546 INFO:teuthology.orchestra.run.smithi086.stdout:Adding group ceph....done 2024-03-22T04:08:23.618 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rados (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:23.744 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librgw2 (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:23.870 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-top (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:23.930 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user ceph....done 2024-03-22T04:08:23.996 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-22T04:08:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:Setting system user ceph properties....done 2024-03-22T04:08:24.118 INFO:teuthology.orchestra.run.smithi086.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-22T04:08:24.264 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-22T04:08:24.662 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-22T04:08:25.274 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rbd (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:25.335 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-mirror (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:25.393 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up rbd-fuse (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:25.493 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rgw (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:25.535 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-22T04:08:25.535 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-22T04:08:25.602 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cephfs (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:25.720 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-shell (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:25.871 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-common (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:26.186 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-test (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:26.321 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up radosgw (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:26.423 INFO:teuthology.orchestra.run.smithi196.stdout:Adding group ceph....done 2024-03-22T04:08:26.783 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user ceph....done 2024-03-22T04:08:26.859 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-22T04:08:26.859 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-22T04:08:26.960 INFO:teuthology.orchestra.run.smithi196.stdout:Setting system user ceph properties....done 2024-03-22T04:08:26.981 INFO:teuthology.orchestra.run.smithi196.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-22T04:08:27.122 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-22T04:08:27.439 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-base (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:27.486 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-22T04:08:27.802 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-22T04:08:28.197 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-mirror (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:28.405 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-22T04:08:28.405 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-22T04:08:28.495 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mds (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:28.727 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-22T04:08:28.727 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-22T04:08:29.040 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-test (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:29.165 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up radosgw (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:29.363 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:29.602 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-22T04:08:29.602 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-22T04:08:29.710 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-22T04:08:29.710 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-22T04:08:30.248 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-osd (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:30.334 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-base (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:30.566 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-22T04:08:30.566 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-22T04:08:30.678 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-22T04:08:31.208 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:31.308 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:31.394 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mds (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:31.426 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mon (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:31.618 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-22T04:08:31.618 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-22T04:08:31.625 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-22T04:08:31.625 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-22T04:08:32.236 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-cephadm (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:32.263 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:32.379 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:32.480 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-22T04:08:32.480 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-22T04:08:32.505 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-dashboard (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:32.647 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-volume (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:33.008 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-22T04:08:33.109 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-22T04:08:33.260 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-22T04:08:33.616 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-osd (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:33.928 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-22T04:08:33.928 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-22T04:08:34.428 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-22T04:08:34.588 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:34.721 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:34.864 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mon (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:35.073 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-22T04:08:35.073 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-22T04:08:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-22T04:08:35.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-22T04:08:35.208 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-03-22T04:08:35.378 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-03-22T04:08:35.379 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-03-22T04:08:35.534 INFO:teuthology.orchestra.run.smithi086.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-22T04:08:35.534 INFO:teuthology.orchestra.run.smithi086.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-22T04:08:35.534 INFO:teuthology.orchestra.run.smithi086.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-22T04:08:35.534 INFO:teuthology.orchestra.run.smithi086.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-22T04:08:35.534 INFO:teuthology.orchestra.run.smithi086.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-22T04:08:35.534 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-03-22T04:08:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-22T04:08:35.534 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-22T04:08:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-22T04:08:35.535 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-22T04:08:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-22T04:08:35.536 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-22T04:08:35.536 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-03-22T04:08:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-22T04:08:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-22T04:08:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-22T04:08:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-22T04:08:35.536 INFO:teuthology.orchestra.run.smithi086.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-22T04:08:35.537 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-03-22T04:08:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-22T04:08:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-22T04:08:35.537 INFO:teuthology.orchestra.run.smithi086.stdout: libengine-pkcs11-openssl 2024-03-22T04:08:35.587 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-03-22T04:08:35.587 INFO:teuthology.orchestra.run.smithi086.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-22T04:08:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-22T04:08:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-22T04:08:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-22T04:08:35.588 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-22T04:08:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-22T04:08:35.589 INFO:teuthology.orchestra.run.smithi086.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-22T04:08:35.589 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2024-03-22T04:08:35.590 INFO:teuthology.orchestra.run.smithi086.stdout: libssl3 2024-03-22T04:08:35.732 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-cephadm (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:35.775 INFO:teuthology.orchestra.run.smithi086.stdout:1 upgraded, 39 newly installed, 0 to remove and 287 not upgraded. 2024-03-22T04:08:35.775 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 54.6 MB of archives. 2024-03-22T04:08:35.775 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-22T04:08:35.775 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-22T04:08:35.874 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:35.984 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-dashboard (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:36.118 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-volume (19.0.0-2344-gc3a04c0e-1jammy) ... 2024-03-22T04:08:36.277 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-22T04:08:36.496 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-22T04:08:36.540 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-22T04:08:36.576 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-22T04:08:36.577 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-22T04:08:36.583 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-22T04:08:36.586 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-22T04:08:36.587 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-22T04:08:36.590 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-22T04:08:36.593 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-22T04:08:36.604 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-22T04:08:36.756 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-22T04:08:36.757 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-22T04:08:36.758 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-22T04:08:37.014 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-22T04:08:37.857 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-22T04:08:38.030 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-22T04:08:38.032 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-22T04:08:38.032 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-22T04:08:38.035 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-22T04:08:38.036 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-22T04:08:38.064 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-22T04:08:38.065 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-22T04:08:38.073 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-22T04:08:38.236 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-22T04:08:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-22T04:08:38.578 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-22T04:08:38.651 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-03-22T04:08:38.823 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-03-22T04:08:38.823 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-03-22T04:08:38.980 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-22T04:08:38.980 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-22T04:08:38.982 INFO:teuthology.orchestra.run.smithi196.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-22T04:08:38.982 INFO:teuthology.orchestra.run.smithi196.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-22T04:08:38.982 INFO:teuthology.orchestra.run.smithi196.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-22T04:08:38.983 INFO:teuthology.orchestra.run.smithi196.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-22T04:08:38.983 INFO:teuthology.orchestra.run.smithi196.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-22T04:08:38.983 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-03-22T04:08:38.983 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-22T04:08:38.983 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-22T04:08:38.983 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-22T04:08:38.983 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-22T04:08:38.984 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-22T04:08:38.984 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-22T04:08:38.984 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-03-22T04:08:38.984 INFO:teuthology.orchestra.run.smithi196.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-22T04:08:38.984 INFO:teuthology.orchestra.run.smithi196.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-22T04:08:38.985 INFO:teuthology.orchestra.run.smithi196.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-22T04:08:38.985 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-22T04:08:38.985 INFO:teuthology.orchestra.run.smithi196.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-22T04:08:38.986 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-03-22T04:08:38.986 INFO:teuthology.orchestra.run.smithi196.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-22T04:08:38.986 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-22T04:08:38.986 INFO:teuthology.orchestra.run.smithi196.stdout: libengine-pkcs11-openssl 2024-03-22T04:08:39.006 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-22T04:08:39.035 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-03-22T04:08:39.036 INFO:teuthology.orchestra.run.smithi196.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-22T04:08:39.036 INFO:teuthology.orchestra.run.smithi196.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-22T04:08:39.036 INFO:teuthology.orchestra.run.smithi196.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-22T04:08:39.037 INFO:teuthology.orchestra.run.smithi196.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-22T04:08:39.037 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-22T04:08:39.037 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-22T04:08:39.037 INFO:teuthology.orchestra.run.smithi196.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-22T04:08:39.037 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2024-03-22T04:08:39.038 INFO:teuthology.orchestra.run.smithi196.stdout: libssl3 2024-03-22T04:08:39.084 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-22T04:08:39.092 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-22T04:08:39.093 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-22T04:08:39.094 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-22T04:08:39.095 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-22T04:08:39.123 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-22T04:08:39.126 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-22T04:08:39.222 INFO:teuthology.orchestra.run.smithi196.stdout:1 upgraded, 39 newly installed, 0 to remove and 287 not upgraded. 2024-03-22T04:08:39.222 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 54.6 MB of archives. 2024-03-22T04:08:39.222 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-22T04:08:39.222 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-22T04:08:39.716 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-22T04:08:39.752 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-22T04:08:39.834 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-22T04:08:39.835 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-22T04:08:39.835 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-22T04:08:39.930 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-22T04:08:39.946 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-22T04:08:39.974 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-22T04:08:40.009 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-22T04:08:40.010 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-22T04:08:40.016 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-22T04:08:40.019 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-22T04:08:40.019 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-22T04:08:40.022 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-22T04:08:40.025 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-22T04:08:40.190 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-22T04:08:40.190 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-22T04:08:40.392 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-22T04:08:40.394 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-22T04:08:40.449 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-22T04:08:40.783 INFO:teuthology.orchestra.run.smithi086.stdout:Preconfiguring packages ... 2024-03-22T04:08:40.927 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2024-03-22T04:08:41.069 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package flex. 2024-03-22T04:08:41.140 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140912 files and directories currently installed.) 2024-03-22T04:08:41.144 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-22T04:08:41.184 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-22T04:08:41.455 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-22T04:08:41.479 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-22T04:08:41.479 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-22T04:08:41.482 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-22T04:08:41.482 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-22T04:08:41.487 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-22T04:08:41.489 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-22T04:08:41.519 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-22T04:08:41.683 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-22T04:08:41.721 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-22T04:08:41.890 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-22T04:08:42.288 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-22T04:08:42.421 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-22T04:08:42.421 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-22T04:08:42.447 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-22T04:08:42.525 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-22T04:08:42.533 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-22T04:08:42.534 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-22T04:08:42.534 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-22T04:08:42.535 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-22T04:08:42.563 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-22T04:08:42.566 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-22T04:08:42.816 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package bison. 2024-03-22T04:08:42.887 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140985 files and directories currently installed.) 2024-03-22T04:08:42.891 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-22T04:08:42.943 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-22T04:08:43.192 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-22T04:08:43.275 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-22T04:08:43.276 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-22T04:08:43.276 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-22T04:08:43.335 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-22T04:08:43.349 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-22T04:08:43.365 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-22T04:08:43.371 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-22T04:08:43.394 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-22T04:08:43.739 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package dump. 2024-03-22T04:08:43.753 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-22T04:08:43.805 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking dump (0.4b47-3) ... 2024-03-22T04:08:43.836 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-22T04:08:43.839 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-22T04:08:44.075 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package indent. 2024-03-22T04:08:44.090 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:44.133 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-22T04:08:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:Preconfiguring packages ... 2024-03-22T04:08:44.369 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-22T04:08:44.384 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-22T04:08:44.388 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2024-03-22T04:08:44.427 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-22T04:08:44.530 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package flex. 2024-03-22T04:08:44.600 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140912 files and directories currently installed.) 2024-03-22T04:08:44.604 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-22T04:08:44.645 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-22T04:08:44.765 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-22T04:08:44.780 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-22T04:08:44.848 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-22T04:08:45.084 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-22T04:08:45.099 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-22T04:08:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-22T04:08:45.230 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-22T04:08:45.417 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-22T04:08:45.470 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-22T04:08:45.486 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-22T04:08:45.528 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-22T04:08:45.773 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-22T04:08:45.788 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-22T04:08:45.789 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-22T04:08:45.831 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-22T04:08:46.117 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjson-perl. 2024-03-22T04:08:46.133 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-22T04:08:46.175 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-22T04:08:46.293 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package bison. 2024-03-22T04:08:46.362 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140985 files and directories currently installed.) 2024-03-22T04:08:46.366 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-22T04:08:46.403 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-22T04:08:46.436 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-22T04:08:46.452 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-22T04:08:46.486 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-22T04:08:46.796 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-22T04:08:46.810 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-22T04:08:46.854 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-22T04:08:47.165 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package dump. 2024-03-22T04:08:47.179 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-22T04:08:47.215 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking dump (0.4b47-3) ... 2024-03-22T04:08:47.519 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package indent. 2024-03-22T04:08:47.533 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:47.577 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-22T04:08:47.837 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-22T04:08:47.851 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-22T04:08:47.856 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-22T04:08:47.872 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-22T04:08:47.896 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-22T04:08:47.914 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-22T04:08:48.166 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-22T04:08:48.182 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-22T04:08:48.249 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-22T04:08:48.263 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-22T04:08:48.299 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-22T04:08:48.341 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-22T04:08:48.562 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-22T04:08:48.576 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-22T04:08:48.620 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-22T04:08:48.702 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-22T04:08:48.718 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-22T04:08:48.760 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-22T04:08:48.949 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-22T04:08:48.963 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-22T04:08:49.024 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-22T04:08:49.080 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-22T04:08:49.095 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-22T04:08:49.138 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-22T04:08:49.259 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-22T04:08:49.273 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-22T04:08:49.309 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-22T04:08:49.466 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-22T04:08:49.482 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-22T04:08:49.537 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjson-perl. 2024-03-22T04:08:49.541 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-22T04:08:49.551 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-22T04:08:49.595 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-22T04:08:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-22T04:08:49.923 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-22T04:08:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:49.937 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-22T04:08:49.977 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:08:49.981 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-22T04:08:50.263 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-22T04:08:50.278 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:50.321 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-22T04:08:50.616 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-22T04:08:50.630 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:50.674 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-22T04:08:50.917 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-22T04:08:50.932 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-22T04:08:51.007 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-22T04:08:51.276 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-22T04:08:51.292 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-22T04:08:51.334 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-22T04:08:51.496 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-22T04:08:51.510 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-22T04:08:51.554 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-22T04:08:51.604 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-22T04:08:51.620 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-22T04:08:51.662 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-22T04:08:51.882 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package modemmanager. 2024-03-22T04:08:51.896 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-22T04:08:51.940 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-22T04:08:52.048 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-22T04:08:52.064 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-22T04:08:52.106 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-22T04:08:52.434 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-22T04:08:52.450 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-22T04:08:52.483 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-22T04:08:52.605 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager. 2024-03-22T04:08:52.619 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-22T04:08:52.670 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-22T04:08:52.828 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-22T04:08:52.844 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-22T04:08:52.886 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-22T04:08:53.256 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-22T04:08:53.272 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:53.303 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ppp. 2024-03-22T04:08:53.314 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:08:53.318 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-22T04:08:53.684 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-22T04:08:53.688 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-22T04:08:53.700 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:53.742 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-22T04:08:53.983 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pptp-linux. 2024-03-22T04:08:53.997 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-22T04:08:54.028 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-22T04:08:54.041 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-22T04:08:54.044 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:54.086 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-22T04:08:54.294 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-22T04:08:54.308 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-22T04:08:54.364 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-22T04:08:54.379 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-22T04:08:54.385 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-22T04:08:54.422 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-22T04:08:54.747 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-22T04:08:54.761 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-22T04:08:54.813 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-common (238) ... 2024-03-22T04:08:54.883 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-22T04:08:54.898 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-22T04:08:54.933 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-22T04:08:55.201 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-22T04:08:55.215 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:55.267 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:08:55.286 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package modemmanager. 2024-03-22T04:08:55.302 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-22T04:08:55.345 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-22T04:08:55.822 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-common. 2024-03-22T04:08:55.836 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-22T04:08:55.897 INFO:teuthology.orchestra.run.smithi086.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-22T04:08:55.929 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-common (238) ... 2024-03-22T04:08:55.976 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package network-manager. 2024-03-22T04:08:55.991 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-22T04:08:56.042 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-22T04:08:56.396 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-14. 2024-03-22T04:08:56.410 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:56.457 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:08:56.683 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ppp. 2024-03-22T04:08:56.697 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-22T04:08:57.057 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-22T04:08:57.380 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package pptp-linux. 2024-03-22T04:08:57.394 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-22T04:08:57.454 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-22T04:08:57.457 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql. 2024-03-22T04:08:57.472 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-22T04:08:57.515 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql (14+238) ... 2024-03-22T04:08:57.690 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-22T04:08:57.704 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-22T04:08:57.749 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-22T04:08:57.759 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client. 2024-03-22T04:08:57.774 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-22T04:08:57.809 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client (14+238) ... 2024-03-22T04:08:58.018 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-22T04:08:58.028 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-22T04:08:58.033 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-22T04:08:58.043 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-22T04:08:58.068 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client-common (238) ... 2024-03-22T04:08:58.111 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-22T04:08:58.405 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-22T04:08:58.420 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:58.455 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl8.6. 2024-03-22T04:08:58.471 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-22T04:08:58.514 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-22T04:08:58.555 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:08:58.800 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl. 2024-03-22T04:08:58.815 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-22T04:08:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-22T04:08:59.102 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-common. 2024-03-22T04:08:59.116 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-22T04:08:59.169 INFO:teuthology.orchestra.run.smithi196.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-22T04:08:59.200 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-common (238) ... 2024-03-22T04:08:59.625 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package wpasupplicant. 2024-03-22T04:08:59.642 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-22T04:08:59.659 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-14. 2024-03-22T04:08:59.673 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-22T04:08:59.677 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-22T04:08:59.719 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:09:00.016 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-22T04:09:00.032 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-22T04:09:00.074 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-22T04:09:00.452 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-22T04:09:00.468 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-22T04:09:00.510 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-22T04:09:00.778 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-common (238) ... 2024-03-22T04:09:00.886 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql. 2024-03-22T04:09:00.902 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-22T04:09:00.936 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql (14+238) ... 2024-03-22T04:09:00.991 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-22T04:09:01.117 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-22T04:09:01.163 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client. 2024-03-22T04:09:01.178 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-22T04:09:01.222 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client (14+238) ... 2024-03-22T04:09:01.243 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:09:01.361 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-22T04:09:01.470 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-22T04:09:01.474 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-22T04:09:01.488 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-22T04:09:01.532 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-22T04:09:01.596 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:09:01.835 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package tcl8.6. 2024-03-22T04:09:01.849 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-22T04:09:01.893 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-22T04:09:02.154 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package tcl. 2024-03-22T04:09:02.168 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-22T04:09:02.233 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-22T04:09:02.549 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-22T04:09:02.581 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package wpasupplicant. 2024-03-22T04:09:02.596 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-22T04:09:02.640 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-22T04:09:02.644 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-22T04:09:02.998 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-22T04:09:02.999 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-22T04:09:03.027 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-22T04:09:03.042 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-22T04:09:03.094 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-22T04:09:03.447 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-22T04:09:03.462 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-22T04:09:03.505 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-22T04:09:03.658 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-22T04:09:03.772 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client-common (238) ... 2024-03-22T04:09:03.780 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-22T04:09:03.907 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-22T04:09:04.033 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-22T04:09:04.159 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-22T04:09:04.284 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-22T04:09:04.378 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-22T04:09:04.427 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-22T04:09:04.504 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-22T04:09:04.672 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:09:04.798 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-22T04:09:04.924 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-22T04:09:05.033 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:09:05.133 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-22T04:09:05.259 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up dump (0.4b47-3) ... 2024-03-22T04:09:05.354 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-22T04:09:05.419 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-22T04:09:05.562 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-22T04:09:05.679 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-22T04:09:05.805 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-22T04:09:05.939 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-22T04:09:05.980 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-22T04:09:06.029 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-22T04:09:06.072 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-22T04:09:06.109 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-22T04:09:06.234 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-22T04:09:06.410 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-22T04:09:06.477 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-22T04:09:06.477 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-22T04:09:06.536 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-22T04:09:06.645 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-22T04:09:06.822 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client (14+238) ... 2024-03-22T04:09:06.948 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-22T04:09:07.183 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-22T04:09:07.283 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-22T04:09:07.384 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-22T04:09:07.510 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-22T04:09:07.542 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:07.542 INFO:teuthology.orchestra.run.smithi086.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-22T04:09:07.542 INFO:teuthology.orchestra.run.smithi086.stdout:which means they are currently configured by ifupdown: 2024-03-22T04:09:07.542 INFO:teuthology.orchestra.run.smithi086.stdout:- enp3s0f1 2024-03-22T04:09:07.542 INFO:teuthology.orchestra.run.smithi086.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-22T04:09:07.543 INFO:teuthology.orchestra.run.smithi086.stdout:remove their configuration from /etc/network/interfaces. 2024-03-22T04:09:07.543 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:07.645 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-22T04:09:07.675 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-22T04:09:07.771 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-22T04:09:07.930 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-22T04:09:08.088 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-22T04:09:08.483 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-22T04:09:08.527 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-22T04:09:08.703 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up dump (0.4b47-3) ... 2024-03-22T04:09:08.798 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-22T04:09:08.863 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-22T04:09:08.989 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-22T04:09:09.115 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-22T04:09:09.192 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-22T04:09:09.335 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-22T04:09:09.444 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-22T04:09:09.554 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-common (238) ... 2024-03-22T04:09:09.591 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-22T04:09:09.692 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-22T04:09:09.765 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-22T04:09:09.843 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-22T04:09:09.978 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-22T04:09:10.154 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-22T04:09:10.305 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-22T04:09:10.431 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-22T04:09:10.599 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client (14+238) ... 2024-03-22T04:09:10.717 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-22T04:09:10.861 INFO:teuthology.orchestra.run.smithi086.stdout:Adding user postgres to group ssl-cert 2024-03-22T04:09:11.086 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:11.086 INFO:teuthology.orchestra.run.smithi086.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-22T04:09:11.282 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:11.282 INFO:teuthology.orchestra.run.smithi196.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-22T04:09:11.282 INFO:teuthology.orchestra.run.smithi196.stdout:which means they are currently configured by ifupdown: 2024-03-22T04:09:11.282 INFO:teuthology.orchestra.run.smithi196.stdout:- enp3s0f1 2024-03-22T04:09:11.282 INFO:teuthology.orchestra.run.smithi196.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-22T04:09:11.282 INFO:teuthology.orchestra.run.smithi196.stdout:remove their configuration from /etc/network/interfaces. 2024-03-22T04:09:11.282 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:11.328 INFO:teuthology.orchestra.run.smithi086.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-22T04:09:11.328 INFO:teuthology.orchestra.run.smithi086.stdout:Removing obsolete dictionary files: 2024-03-22T04:09:11.414 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-22T04:09:11.810 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-22T04:09:12.031 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-22T04:09:12.204 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-22T04:09:12.572 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:09:12.919 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-22T04:09:13.005 INFO:teuthology.orchestra.run.smithi086.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-22T04:09:13.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout:This user must also own the server process. 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout:The database cluster will be initialized with locale "en_US". 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout:The default text search configuration will be set to "english". 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout:Data page checksums are disabled. 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:13.077 INFO:teuthology.orchestra.run.smithi086.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-22T04:09:13.078 INFO:teuthology.orchestra.run.smithi086.stdout:creating subdirectories ... ok 2024-03-22T04:09:13.078 INFO:teuthology.orchestra.run.smithi086.stdout:selecting dynamic shared memory implementation ... posix 2024-03-22T04:09:13.102 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default max_connections ... 100 2024-03-22T04:09:13.136 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default shared_buffers ... 128MB 2024-03-22T04:09:13.150 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default time zone ... Etc/UTC 2024-03-22T04:09:13.151 INFO:teuthology.orchestra.run.smithi086.stdout:creating configuration files ... ok 2024-03-22T04:09:13.204 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-22T04:09:13.315 INFO:teuthology.orchestra.run.smithi086.stdout:running bootstrap script ... ok 2024-03-22T04:09:13.330 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-22T04:09:13.481 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-common (238) ... 2024-03-22T04:09:13.923 INFO:teuthology.orchestra.run.smithi086.stdout:performing post-bootstrap initialization ... ok 2024-03-22T04:09:14.413 INFO:teuthology.orchestra.run.smithi196.stdout:Adding user postgres to group ssl-cert 2024-03-22T04:09:14.487 INFO:teuthology.orchestra.run.smithi086.stdout:syncing data to disk ... ok 2024-03-22T04:09:14.676 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:14.677 INFO:teuthology.orchestra.run.smithi196.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-22T04:09:14.936 INFO:teuthology.orchestra.run.smithi196.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-22T04:09:14.936 INFO:teuthology.orchestra.run.smithi196.stdout:Removing obsolete dictionary files: 2024-03-22T04:09:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-22T04:09:15.661 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-22T04:09:16.140 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-22T04:09:16.460 INFO:teuthology.orchestra.run.smithi196.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-22T04:09:16.479 INFO:teuthology.orchestra.run.smithi196.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout:This user must also own the server process. 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout:The database cluster will be initialized with locale "en_US". 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout:The default text search configuration will be set to "english". 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout:Data page checksums are disabled. 2024-03-22T04:09:16.533 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:16.534 INFO:teuthology.orchestra.run.smithi196.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-22T04:09:16.534 INFO:teuthology.orchestra.run.smithi196.stdout:creating subdirectories ... ok 2024-03-22T04:09:16.534 INFO:teuthology.orchestra.run.smithi196.stdout:selecting dynamic shared memory implementation ... posix 2024-03-22T04:09:16.558 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default max_connections ... 100 2024-03-22T04:09:16.591 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default shared_buffers ... 128MB 2024-03-22T04:09:16.605 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default time zone ... Etc/UTC 2024-03-22T04:09:16.605 INFO:teuthology.orchestra.run.smithi196.stdout:creating configuration files ... ok 2024-03-22T04:09:16.770 INFO:teuthology.orchestra.run.smithi196.stdout:running bootstrap script ... ok 2024-03-22T04:09:17.366 INFO:teuthology.orchestra.run.smithi196.stdout:performing post-bootstrap initialization ... ok 2024-03-22T04:09:17.921 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-22T04:09:17.930 INFO:teuthology.orchestra.run.smithi196.stdout:syncing data to disk ... ok 2024-03-22T04:09:18.047 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-22T04:09:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql (14+238) ... 2024-03-22T04:09:18.298 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-22T04:09:18.425 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-22T04:09:18.457 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-22T04:09:18.702 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-22T04:09:18.702 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-22T04:09:19.485 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-22T04:09:19.804 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-22T04:09:19.955 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-22T04:09:21.316 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-22T04:09:21.381 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-22T04:09:21.417 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-22T04:09:21.515 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-22T04:09:21.616 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql (14+238) ... 2024-03-22T04:09:21.717 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-22T04:09:21.868 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-22T04:09:21.937 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-22T04:09:21.944 DEBUG:teuthology.parallel:result is None 2024-03-22T04:09:22.118 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-22T04:09:22.119 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-22T04:09:22.854 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-22T04:09:23.181 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-22T04:09:23.307 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-22T04:09:24.751 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-22T04:09:24.851 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-22T04:09:25.404 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-22T04:09:25.411 DEBUG:teuthology.parallel:result is None 2024-03-22T04:09:25.411 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e 2024-03-22T04:09:25.635 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-query -W -f '${Version}' ceph 2024-03-22T04:09:25.659 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T04:09:25.660 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T04:09:25.660 INFO:teuthology.task.install:The correct ceph version 19.0.0-2344-gc3a04c0e-1jammy is installed. 2024-03-22T04:09:25.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e 2024-03-22T04:09:25.874 DEBUG:teuthology.orchestra.run.smithi196:> dpkg-query -W -f '${Version}' ceph 2024-03-22T04:09:25.896 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T04:09:25.897 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2344-gc3a04c0e-1jammy 2024-03-22T04:09:25.897 INFO:teuthology.task.install:The correct ceph version 19.0.0-2344-gc3a04c0e-1jammy is installed. 2024-03-22T04:09:25.899 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-22T04:09:25.899 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:25.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T04:09:25.920 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:09:25.921 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T04:09:25.956 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-22T04:09:25.957 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:25.957 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T04:09:25.984 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T04:09:26.048 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:09:26.048 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T04:09:26.067 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T04:09:26.127 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-22T04:09:26.127 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:26.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T04:09:26.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T04:09:26.212 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:09:26.212 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T04:09:26.231 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T04:09:26.296 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-22T04:09:26.296 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:26.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T04:09:26.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T04:09:26.380 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:09:26.380 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T04:09:26.399 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T04:09:26.460 INFO:teuthology.run_tasks:Running task ceph... 2024-03-22T04:09:26.552 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-22T04:09:26.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2024-03-22T04:09:26.556 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /var/log/ceph 2024-03-22T04:09:26.578 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-22T04:09:26.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-22T04:09:26.623 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-22T04:09:26.641 INFO:tasks.ceph:Creating extra log directories... 2024-03-22T04:09:26.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-22T04:09:26.687 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-22T04:09:26.713 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-22T04:09:26.713 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-22T04:09:26.713 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-22T04:09:26.713 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:26.713 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-22T04:09:26.756 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-22T04:09:26.848 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:09:26.849 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-22T04:09:26.857 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-22T04:09:26.946 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-22T04:09:26.947 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-22T04:09:26.947 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927', 'branch': 'wip-batrick-testing-20240321.172515', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7615927', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20240321.172515'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20240321.172515', 'sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 316, 'sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e', 'sleep_before_teardown': 0, 'subset': '420/512', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20240321.172515', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPscXJYCGC+Y6ORG361G3Yy186lnUntU1Q7Mbs8PRDPUv/JsXU0OaJC4vPQHxfSoJ5xSvu8tDGn2eB3sAvggPtc=', 'smithi196.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDx+T9vmsFO1rwbEhqQ66c/mjZ446kth81nrjsA2cR/9py1eqhPY/yGlKG+PKmRKciCFFls8ssOVEgnSDY9R34w='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-22_01:06:30', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-22T04:09:26.947 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-22T04:09:26.951 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-22T04:09:26.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2024-03-22T04:09:27.002 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/run/ceph 2024-03-22T04:09:27.051 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:27.052 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T04:09:27.072 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T04:09:27.072 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-03-22T04:09:27.124 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T04:09:27.124 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T04:09:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-03-22T04:09:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T04:09:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 04:03:06.475147966 +0000 2024-03-22T04:09:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 04:03:06.203150396 +0000 2024-03-22T04:09:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 04:03:06.203150396 +0000 2024-03-22T04:09:27.125 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 04:03:06.203150396 +0000 2024-03-22T04:09:27.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T04:09:27.186 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-22T04:09:27.186 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-22T04:09:27.186 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000396035 s, 1.3 MB/s 2024-03-22T04:09:27.188 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T04:09:27.246 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-03-22T04:09:27.300 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T04:09:27.300 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T04:09:27.300 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-03-22T04:09:27.301 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T04:09:27.301 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 04:03:07.031142999 +0000 2024-03-22T04:09:27.301 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 04:03:06.759145428 +0000 2024-03-22T04:09:27.301 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 04:03:06.759145428 +0000 2024-03-22T04:09:27.301 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 04:03:06.759145428 +0000 2024-03-22T04:09:27.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T04:09:27.362 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-22T04:09:27.362 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-22T04:09:27.362 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000387581 s, 1.3 MB/s 2024-03-22T04:09:27.364 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T04:09:27.418 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-03-22T04:09:27.468 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T04:09:27.468 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T04:09:27.468 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-22T04:09:27.468 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T04:09:27.468 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 04:03:07.547138389 +0000 2024-03-22T04:09:27.468 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 04:03:07.287140713 +0000 2024-03-22T04:09:27.469 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 04:03:07.287140713 +0000 2024-03-22T04:09:27.469 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 04:03:07.287140713 +0000 2024-03-22T04:09:27.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T04:09:27.531 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-22T04:09:27.531 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-22T04:09:27.531 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000672812 s, 761 kB/s 2024-03-22T04:09:27.533 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T04:09:27.586 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-03-22T04:09:27.636 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T04:09:27.636 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T04:09:27.636 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-03-22T04:09:27.636 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T04:09:27.637 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-22 04:03:08.043133959 +0000 2024-03-22T04:09:27.637 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-22 04:03:07.791136211 +0000 2024-03-22T04:09:27.637 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-22 04:03:07.791136211 +0000 2024-03-22T04:09:27.637 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-22 04:03:07.791136211 +0000 2024-03-22T04:09:27.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T04:09:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-22T04:09:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-22T04:09:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000395043 s, 1.3 MB/s 2024-03-22T04:09:27.700 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T04:09:27.758 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:27.759 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:09:27.759 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T04:09:27.767 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T04:09:27.768 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-03-22T04:09:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T04:09:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T04:09:27.818 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-03-22T04:09:27.818 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T04:09:27.818 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-22 04:03:08.054966126 +0000 2024-03-22T04:09:27.818 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-22 04:03:07.806968341 +0000 2024-03-22T04:09:27.818 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-22 04:03:07.806968341 +0000 2024-03-22T04:09:27.818 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-22 04:03:07.806968341 +0000 2024-03-22T04:09:27.818 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T04:09:27.879 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-22T04:09:27.879 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-22T04:09:27.879 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000387145 s, 1.3 MB/s 2024-03-22T04:09:27.881 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T04:09:27.935 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-03-22T04:09:27.985 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T04:09:27.985 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T04:09:27.985 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-03-22T04:09:27.985 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T04:09:27.985 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-22 04:03:08.570961518 +0000 2024-03-22T04:09:27.986 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-22 04:03:08.314963804 +0000 2024-03-22T04:09:27.986 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-22 04:03:08.314963804 +0000 2024-03-22T04:09:27.986 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-22 04:03:08.314963804 +0000 2024-03-22T04:09:27.986 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T04:09:28.042 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-22T04:09:28.043 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-22T04:09:28.043 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000360305 s, 1.4 MB/s 2024-03-22T04:09:28.044 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T04:09:28.095 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-03-22T04:09:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T04:09:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T04:09:28.146 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 925 Links: 1 2024-03-22T04:09:28.146 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T04:09:28.146 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-22 04:03:09.070957051 +0000 2024-03-22T04:09:28.146 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-22 04:03:08.822959267 +0000 2024-03-22T04:09:28.146 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-22 04:03:08.822959267 +0000 2024-03-22T04:09:28.146 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-22 04:03:08.822959267 +0000 2024-03-22T04:09:28.146 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T04:09:28.210 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-22T04:09:28.210 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-22T04:09:28.210 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000357738 s, 1.4 MB/s 2024-03-22T04:09:28.212 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T04:09:28.263 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-03-22T04:09:28.313 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T04:09:28.313 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T04:09:28.313 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 945 Links: 1 2024-03-22T04:09:28.313 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T04:09:28.314 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-22 04:03:09.566952620 +0000 2024-03-22T04:09:28.314 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-22 04:03:09.314954872 +0000 2024-03-22T04:09:28.314 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-22 04:03:09.314954872 +0000 2024-03-22T04:09:28.314 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-22 04:03:09.314954872 +0000 2024-03-22T04:09:28.314 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T04:09:28.374 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-22T04:09:28.375 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-22T04:09:28.375 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000364282 s, 1.4 MB/s 2024-03-22T04:09:28.376 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T04:09:28.427 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:28.427 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-22T04:09:28.427 INFO:tasks.ceph:Generating config... 2024-03-22T04:09:28.428 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-22T04:09:28.429 INFO:tasks.ceph:[client] debug client = 20 2024-03-22T04:09:28.429 INFO:tasks.ceph:[client] debug ms = 1 2024-03-22T04:09:28.429 INFO:tasks.ceph:[client] pid file = 2024-03-22T04:09:28.429 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-22T04:09:28.429 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-22T04:09:28.429 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-22T04:09:28.429 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-22T04:09:28.430 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-22T04:09:28.430 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-22T04:09:28.430 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-22T04:09:28.430 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-22T04:09:28.430 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-22T04:09:28.430 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-22T04:09:28.430 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-22T04:09:28.431 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-22T04:09:28.431 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-22T04:09:28.431 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-22T04:09:28.431 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-22T04:09:28.431 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-22T04:09:28.431 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-22T04:09:28.431 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-22T04:09:28.431 INFO:tasks.ceph:Setting up mon.a... 2024-03-22T04:09:28.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-22T04:09:28.477 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.keyring 2024-03-22T04:09:28.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-22T04:09:28.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-22T04:09:28.607 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.86'), ('mon.b', '172.21.15.196'), ('mon.c', '[v2:172.21.15.196:3301,v1:172.21.15.196:6790]')] 2024-03-22T04:09:28.608 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-22T04:09:28.609 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:28.609 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-22T04:09:28.655 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.86 --add b 172.21.15.196 --addv c '[v2:172.21.15.196:3301,v1:172.21.15.196:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:28.741 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:28.741 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: generated fsid 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:09:28.741 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-03-22T04:09:28.741 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 0 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:last_changed 2024-03-22T04:09:28.743788+0000 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-03-22T04:09:28.743788+0000 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:min_mon_release 17 (quincy) 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:election_strategy: 1 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.b 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:2: [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] mon.c 2024-03-22T04:09:28.742 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-22T04:09:28.744 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-22T04:09:28.752 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1365483d-2fb4-4579-a50b-2ce299f625b2... 2024-03-22T04:09:28.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:09:28.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:09:28.837 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-03-22T04:09:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-03-22T04:09:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:09:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-03-22T04:09:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.837 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:09:28.838 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-03-22T04:09:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.840 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.841 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-03-22T04:09:28.842 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-03-22T04:09:28.843 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-03-22T04:09:28.844 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.845 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:09:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-03-22T04:09:28.847 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.848 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:09:28.849 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-03-22T04:09:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on skipped message = False 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-03-22T04:09:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:09:28.852 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:09:28.853 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:09:28.854 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:09:28.855 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: mon down mkfs grace = 300 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:09:28.856 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-03-22T04:09:28.857 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:09:28.858 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:09:28.860 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-22T04:09:28.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-22T04:09:28.959 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-22T04:09:28.959 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:28.960 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-22T04:09:28.968 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:28.968 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-22T04:09:29.019 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:09:29.019 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:29.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-22T04:09:29.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-22T04:09:29.090 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:09:29.091 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:29.143 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:09:29.143 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:09:29.143 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-22T04:09:29.143 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-22T04:09:29.175 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:09:29.175 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:29.225 INFO:tasks.ceph:Setting up mon nodes... 2024-03-22T04:09:29.225 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-22T04:09:29.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-22T04:09:29.278 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-22T04:09:29.283 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-22T04:09:29.337 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-22T04:09:29.353 INFO:tasks.ceph:Setting up mds nodes... 2024-03-22T04:09:29.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-22T04:09:29.406 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-22T04:09:29.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-22T04:09:29.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-22T04:09:29.524 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-22T04:09:29.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-22T04:09:29.548 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-22T04:09:29.602 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-22T04:09:29.607 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-22T04:09:29.626 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-22T04:09:29.722 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-22T04:09:29.726 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-22T04:09:29.746 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-22T04:09:29.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-22T04:09:29.787 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-22T04:09:29.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-22T04:09:29.884 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-22T04:09:29.900 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-22T04:09:29.941 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-22T04:09:29.958 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-22T04:09:30.042 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-22T04:09:30.059 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-22T04:09:30.059 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-22T04:09:30.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-22T04:09:30.078 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:30.078 INFO:tasks.ceph:role: osd.0 2024-03-22T04:09:30.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:09:30.078 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-22T04:09:30.148 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-22T04:09:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-22T04:09:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-22T04:09:30.148 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-22T04:09:30.148 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-22T04:09:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-03-22T04:09:30.149 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-22T04:09:30.149 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-22T04:09:30.149 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-22T04:09:30.149 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-22T04:09:30.606 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-03-22T04:09:30.609 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-03-22T04:09:30.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-22T04:09:30.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-22T04:09:30.742 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-03-22T04:09:30.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:09:30.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-22T04:09:30.807 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:30.807 INFO:tasks.ceph:role: osd.1 2024-03-22T04:09:30.807 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:09:30.807 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-22T04:09:30.874 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-22T04:09:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-22T04:09:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-22T04:09:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-22T04:09:30.874 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-22T04:09:30.874 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-03-22T04:09:30.874 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-22T04:09:30.874 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-22T04:09:30.875 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-22T04:09:30.875 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-22T04:09:31.371 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-03-22T04:09:31.374 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-03-22T04:09:31.374 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-22T04:09:31.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-22T04:09:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-03-22T04:09:31.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:09:31.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-22T04:09:31.600 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:31.600 INFO:tasks.ceph:role: osd.2 2024-03-22T04:09:31.600 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:09:31.600 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-22T04:09:31.671 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-22T04:09:31.672 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-22T04:09:32.159 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-03-22T04:09:32.162 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-03-22T04:09:32.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-22T04:09:32.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-22T04:09:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-03-22T04:09:32.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:09:32.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-22T04:09:32.380 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:32.380 INFO:tasks.ceph:role: osd.3 2024-03-22T04:09:32.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:09:32.380 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-22T04:09:32.452 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-22T04:09:32.909 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-03-22T04:09:32.911 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-03-22T04:09:32.912 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-22T04:09:33.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-22T04:09:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-03-22T04:09:33.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:09:33.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:33.152+0000 7f834a61e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-22T04:09:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:33.156+0000 7f834a61e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-22T04:09:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:33.156+0000 7f834a61e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-22T04:09:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:33.156+0000 7f834a61e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-22T04:09:33.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:33.156+0000 7f834a61e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-22T04:09:33.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:33.160+0000 7f834a61e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-22T04:09:36.153 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T04:09:36.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:36.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:36.268+0000 7f229c3cf7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-22T04:09:36.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:36.268+0000 7f229c3cf7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-22T04:09:36.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:36.268+0000 7f229c3cf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-22T04:09:36.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:36.268+0000 7f229c3cf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-22T04:09:36.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:36.268+0000 7f229c3cf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-22T04:09:36.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:36.268+0000 7f229c3cf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-22T04:09:39.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T04:09:39.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:39.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:39.356+0000 7f98f35037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-22T04:09:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:39.356+0000 7f98f35037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-22T04:09:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:39.360+0000 7f98f35037c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-22T04:09:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:39.360+0000 7f98f35037c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-22T04:09:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:39.360+0000 7f98f35037c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-22T04:09:39.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:39.360+0000 7f98f35037c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-22T04:09:42.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T04:09:42.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:42.456+0000 7faf85b387c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-22T04:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:42.456+0000 7faf85b387c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-22T04:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:42.456+0000 7faf85b387c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-22T04:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:42.456+0000 7faf85b387c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-22T04:09:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:42.456+0000 7faf85b387c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-22T04:09:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:09:42.456+0000 7faf85b387c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-22T04:09:45.437 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T04:09:45.457 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-22T04:09:45.479 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:45.479 INFO:tasks.ceph:role: osd.4 2024-03-22T04:09:45.479 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:09:45.479 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-22T04:09:45.549 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-22T04:09:45.549 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-22T04:09:45.549 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-22T04:09:45.549 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-22T04:09:45.549 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-22T04:09:45.549 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-03-22T04:09:45.549 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-22T04:09:45.550 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-22T04:09:45.550 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-22T04:09:45.550 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-22T04:09:46.027 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-03-22T04:09:46.030 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-03-22T04:09:46.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-22T04:09:46.195 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-22T04:09:46.212 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-03-22T04:09:46.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:09:46.213 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-22T04:09:46.277 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:46.277 INFO:tasks.ceph:role: osd.5 2024-03-22T04:09:46.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:09:46.278 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-22T04:09:46.349 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-22T04:09:46.349 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-22T04:09:46.349 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-22T04:09:46.349 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-22T04:09:46.349 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-22T04:09:46.350 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-03-22T04:09:46.350 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-22T04:09:46.350 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-22T04:09:46.350 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-22T04:09:46.350 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-22T04:09:46.782 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-03-22T04:09:46.785 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-03-22T04:09:46.785 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-22T04:09:46.887 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-22T04:09:46.903 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-03-22T04:09:46.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:09:46.904 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-22T04:09:46.964 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:46.964 INFO:tasks.ceph:role: osd.6 2024-03-22T04:09:46.964 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:09:46.964 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-22T04:09:47.036 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-22T04:09:47.037 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-22T04:09:47.492 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-03-22T04:09:47.495 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-03-22T04:09:47.495 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-22T04:09:47.638 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-22T04:09:47.654 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-03-22T04:09:47.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:09:47.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-22T04:09:47.717 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-22T04:09:47.717 INFO:tasks.ceph:role: osd.7 2024-03-22T04:09:47.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:09:47.717 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-22T04:09:47.785 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-22T04:09:47.786 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-22T04:09:48.298 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-03-22T04:09:48.302 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-03-22T04:09:48.302 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-22T04:09:48.467 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-22T04:09:48.484 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-03-22T04:09:48.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:09:48.485 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:48.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:48.604+0000 7f39dcec97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-22T04:09:48.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:48.608+0000 7f39dcec97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-22T04:09:48.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:48.608+0000 7f39dcec97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-22T04:09:48.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:48.608+0000 7f39dcec97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-22T04:09:48.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:48.608+0000 7f39dcec97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-22T04:09:48.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:48.608+0000 7f39dcec97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-22T04:09:51.575 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T04:09:51.595 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:51.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:51.692+0000 7f1a114047c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-22T04:09:51.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:51.692+0000 7f1a114047c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-22T04:09:51.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:51.692+0000 7f1a114047c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-22T04:09:51.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:51.692+0000 7f1a114047c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-22T04:09:51.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:51.692+0000 7f1a114047c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-22T04:09:51.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:51.696+0000 7f1a114047c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-22T04:09:54.663 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T04:09:54.683 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:54.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:54.780+0000 7f15d2e8e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-22T04:09:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:54.780+0000 7f15d2e8e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-22T04:09:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:54.780+0000 7f15d2e8e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-22T04:09:54.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:54.780+0000 7f15d2e8e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-22T04:09:54.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:54.780+0000 7f15d2e8e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-22T04:09:54.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:54.780+0000 7f15d2e8e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-22T04:09:56.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:09:56.952 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:09:57.754 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-22T04:09:57.774 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:09:57.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:57.868+0000 7efe7cf197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-22T04:09:57.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:57.868+0000 7efe7cf197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-22T04:09:57.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:57.868+0000 7efe7cf197c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-22T04:09:57.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:57.868+0000 7efe7cf197c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-22T04:09:57.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:57.868+0000 7efe7cf197c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-22T04:09:57.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:09:57.872+0000 7efe7cf197c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-22T04:10:00.847 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-22T04:10:00.868 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-22T04:10:00.868 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:00.868 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-22T04:10:00.890 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:00.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-22T04:10:00.951 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:00.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-22T04:10:01.015 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:01.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-22T04:10:01.080 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:01.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-22T04:10:01.143 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:01.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-22T04:10:01.207 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:01.208 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-22T04:10:01.271 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.271 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-22T04:10:01.291 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.292 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-22T04:10:01.356 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.356 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-22T04:10:01.420 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.421 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-22T04:10:01.480 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.480 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-22T04:10:01.540 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.540 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-22T04:10:01.600 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-22T04:10:01.660 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:01.660 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-22T04:10:01.668 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:01.669 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-22T04:10:01.720 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.720 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-22T04:10:01.728 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:01.728 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-22T04:10:01.778 INFO:tasks.ceph:Adding keys to all mons... 2024-03-22T04:10:01.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-22T04:10:01.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-22T04:10:01.824 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.y] 2024-03-22T04:10:01.824 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ== 2024-03-22T04:10:01.824 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-03-22T04:10:01.824 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA== 2024-03-22T04:10:01.824 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q== 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA== 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA== 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg== 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.3] 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA== 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g== 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw== 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-03-22T04:10:01.825 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg== 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.4] 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ== 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.5] 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow== 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.6] 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg== 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.7] 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg== 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w== 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg== 2024-03-22T04:10:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2024-03-22T04:10:01.827 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w== 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.y] 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ== 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA== 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q== 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.0] 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA== 2024-03-22T04:10:01.838 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.1] 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA== 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.2] 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg== 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.3] 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA== 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.x] 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g== 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw== 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg== 2024-03-22T04:10:01.839 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.4] 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ== 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.5] 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow== 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.6] 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg== 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.7] 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg== 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout:[client.2] 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w== 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-03-22T04:10:01.840 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg== 2024-03-22T04:10:01.841 INFO:teuthology.orchestra.run.smithi196.stdout:[client.3] 2024-03-22T04:10:01.841 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w== 2024-03-22T04:10:01.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-22T04:10:01.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-22T04:10:01.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-22T04:10:01.943 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-22T04:10:02.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-22T04:10:02.003 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-22T04:10:02.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.062 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.119 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.178 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.231 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.234 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-22T04:10:02.291 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-22T04:10:02.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-22T04:10:02.347 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-22T04:10:02.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-22T04:10:02.406 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-22T04:10:02.462 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.465 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.522 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.578 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.631 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.634 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-22T04:10:02.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-22T04:10:02.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-22T04:10:02.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-22T04:10:02.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-22T04:10:02.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-22T04:10:02.809 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-22T04:10:02.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-22T04:10:02.868 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-22T04:10:02.924 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-22T04:10:02.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-22T04:10:02.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-22T04:10:03.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-22T04:10:03.071 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-22T04:10:03.090 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-22T04:10:03.205 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-22T04:10:03.224 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-22T04:10:03.283 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-22T04:10:03.392 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-22T04:10:03.411 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:10:03.414 DEBUG:teuthology.orchestra.run.smithi196:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-22T04:10:03.460 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-22T04:10:03.460 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-22T04:10:03.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-22T04:10:03.463 INFO:tasks.ceph.mon.a:Started 2024-03-22T04:10:03.463 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-22T04:10:03.463 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-22T04:10:03.504 INFO:tasks.ceph.mon.b:Started 2024-03-22T04:10:03.504 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-22T04:10:03.504 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-22T04:10:03.507 INFO:tasks.ceph.mon.c:Started 2024-03-22T04:10:03.507 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-22T04:10:03.507 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-22T04:10:03.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-22T04:10:03.510 INFO:tasks.ceph.mgr.y:Started 2024-03-22T04:10:03.510 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-22T04:10:03.510 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-22T04:10:03.513 INFO:tasks.ceph.mgr.x:Started 2024-03-22T04:10:03.513 INFO:tasks.ceph:Setting crush tunables to default 2024-03-22T04:10:03.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd crush tunables default 2024-03-22T04:10:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99e4a00640 1 Processor -- start 2024-03-22T04:10:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99e4a00640 1 -- start start 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99e4a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99e4a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99e4a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99e4a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f99e00b2740 con 0x7f99e00b3430 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99e4a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f99e015d710 con 0x7f99e00b2360 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99e4a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99e015fe40 con 0x7f99e015cee0 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99d5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99d5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49268/0 (socket says 172.21.15.86:49268) 2024-03-22T04:10:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 learned_addr learned my addr 172.21.15.86:0/1918146626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99dde00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99dde00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.651+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:03.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:03.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:03.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:03.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:03.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:03.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99dde00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99dde00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:03.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:03.855+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T04:10:04.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99dde00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:04.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99dde00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:04.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:04.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:04.259+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T04:10:05.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:05.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99dde00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99dde00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:05.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:05.063+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T04:10:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:06.655+0000 7f99d7e00640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 msgr2=0x7f99e006d6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:06.655+0000 7f99d7e00640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e015cee0 0x7f99e006d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:06.655+0000 7f99d7e00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 msgr2=0x7f99e00b2ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:06.655+0000 7f99d7e00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.196:3300/0 conn(0x7f99e00b2360 0x7f99e00b2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:06.655+0000 7f99d7e00640 1 -- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 msgr2=0x7f99e015c9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:06.655+0000 7f99d7e00640 1 --2- 172.21.15.86:0/1918146626 >> v2:172.21.15.86:3300/0 conn(0x7f99e00b3430 0x7f99e015c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:06.655+0000 7f99d7e00640 1 -- 172.21.15.86:0/1918146626 --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e0160120 con 0x7f99c4007330 2024-03-22T04:10:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:06.655+0000 7f99d7e00640 1 -- 172.21.15.86:0/1918146626 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c400c080 con 0x7f99c4003680 2024-03-22T04:10:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492459145 0 0) 0x7f99c400c080 con 0x7f99c4003680 2024-03-22T04:10:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99bc003530 con 0x7f99c4003680 2024-03-22T04:10:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1474523162 0 0) 0x7f99e0160120 con 0x7f99c4007330 2024-03-22T04:10:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99c400c080 con 0x7f99c4007330 2024-03-22T04:10:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3957551914 0 0) 0x7f99c400c080 con 0x7f99c4007330 2024-03-22T04:10:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 >> v1:172.21.15.196:6789/0 conn(0x7f99c4003680 legacy=0x7f99c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c400d320 con 0x7f99c4007330 2024-03-22T04:10:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f99c800f040 con 0x7f99c4007330 2024-03-22T04:10:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 >> v1:172.21.15.86:6789/0 conn(0x7f99c4007330 legacy=0x7f99c40097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 reap_dead start 2024-03-22T04:10:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99bc005010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 0x7f99bc003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0099c0 0x7f99bc00bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc00c320 con 0x7f99bc0099c0 2024-03-22T04:10:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc00c4d0 con 0x7f99bc004c30 2024-03-22T04:10:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99bc00c6d0 con 0x7f99bc0037b0 2024-03-22T04:10:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0099c0 0x7f99bc00bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 0x7f99bc003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99dde00640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99bc005010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 msgr2=0x7f99bc003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99d5400640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 msgr2=0x7f99bc003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 msgr2=0x7f99bc003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 0x7f99bc003c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99d5400640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 0x7f99bc003c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 0x7f99bc003c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 msgr2=0x7f99bc005010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99bc005010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.802+0000 7f99de800640 1 -- 172.21.15.86:0/1918146626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c400c080 con 0x7f99bc0099c0 2024-03-22T04:10:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.806+0000 7f99de800640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0099c0 0x7f99bc00bdb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99e006f3e0 tx=0x7f99d801d110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185e7efce64c86c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.806+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d80b3070 con 0x7f99bc0099c0 2024-03-22T04:10:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.806+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d80ae410 con 0x7f99bc0099c0 2024-03-22T04:10:08.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.806+0000 7f99dd400640 1 -- 172.21.15.86:0/1918146626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8003de0 con 0x7f99bc0099c0 2024-03-22T04:10:08.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- 172.21.15.86:0/1918146626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0099c0 msgr2=0x7f99bc00bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:08.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0099c0 0x7f99bc00bdb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99e006f3e0 tx=0x7f99d801d110 comp rx=0 tx=0).stop 2024-03-22T04:10:08.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- 172.21.15.86:0/1918146626 shutdown_connections 2024-03-22T04:10:08.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0099c0 0x7f99bc00bdb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0037b0 0x7f99bc003c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 --2- 172.21.15.86:0/1918146626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99bc005010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- 172.21.15.86:0/1918146626 >> 172.21.15.86:0/1918146626 conn(0x7f99e00cc940 msgr2=0x7f99e00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- 172.21.15.86:0/1918146626 shutdown_connections 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- 172.21.15.86:0/1918146626 wait complete. 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 Processor -- start 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- start start 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0037b0 0x7f99e01f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99e01f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 0x7f99e0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0160c00 con 0x7f99bc0037b0 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01606e0 con 0x7f99bc004c30 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99e4a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99e01f49d0 con 0x7f99bc0099c0 2024-03-22T04:10:08.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.810+0000 7f99dde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0037b0 0x7f99e01f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99dde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0037b0 0x7f99e01f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43096/0 (socket says 172.21.15.86:43096) 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99dde00640 1 -- 172.21.15.86:0/1760848303 learned_addr learned my addr 172.21.15.86:0/1760848303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d5400640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99e01f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99de800640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 0x7f99e0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99de800640 1 -- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 msgr2=0x7f99e0150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99de800640 1 -- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 msgr2=0x7f99e0150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99de800640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 0x7f99e0150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99de800640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 0x7f99e0150b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d5400640 1 -- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 msgr2=0x7f99e0150b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d5400640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 0x7f99e0150b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d5400640 1 -- 172.21.15.86:0/1760848303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0037b0 msgr2=0x7f99e01f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d5400640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0037b0 0x7f99e01f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d5400640 1 -- 172.21.15.86:0/1760848303 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e0152930 con 0x7f99bc004c30 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d5400640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99e01f3280 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c540 tx=0x7f99c8016ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d6a00640 1 -- 172.21.15.86:0/1760848303 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c800f040 con 0x7f99bc004c30 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d6a00640 1 -- 172.21.15.86:0/1760848303 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c8012c90 con 0x7f99bc004c30 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e0152c10 con 0x7f99bc004c30 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99d6a00640 1 -- 172.21.15.86:0/1760848303 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8004030 con 0x7f99bc004c30 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.814+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e00714d0 con 0x7f99bc004c30 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.818+0000 7f99d6a00640 1 -- 172.21.15.86:0/1760848303 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f99c8003620 con 0x7f99bc004c30 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.818+0000 7f99d6a00640 1 -- 172.21.15.86:0/1760848303 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f99c80233f0 con 0x7f99bc004c30 2024-03-22T04:10:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.818+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e0074470 con 0x7f99bc004c30 2024-03-22T04:10:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.822+0000 7f99d6a00640 1 -- 172.21.15.86:0/1760848303 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f99c801f020 con 0x7f99bc004c30 2024-03-22T04:10:08.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.886+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f99e00cfbe0 con 0x7f99bc004c30 2024-03-22T04:10:08.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.890+0000 7f99d6a00640 1 -- 172.21.15.86:0/1760848303 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f99c8019480 con 0x7f99bc004c30 2024-03-22T04:10:08.896 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-03-22T04:10:08.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 msgr2=0x7f99e01f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:08.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99e01f3280 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c540 tx=0x7f99c8016ed0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 shutdown_connections 2024-03-22T04:10:08.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99bc004c30 0x7f99e01f3280 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99bc0099c0 0x7f99e0150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 --2- 172.21.15.86:0/1760848303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99bc0037b0 0x7f99e01f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:08.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 >> 172.21.15.86:0/1760848303 conn(0x7f99e00cc940 msgr2=0x7f99e00b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:08.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 shutdown_connections 2024-03-22T04:10:08.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:08.894+0000 7f99e4a00640 1 -- 172.21.15.86:0/1760848303 wait complete. 2024-03-22T04:10:08.917 INFO:tasks.ceph:check_enable_crimson: False 2024-03-22T04:10:08.917 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-22T04:10:08.918 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:08.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-22T04:10:08.939 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:08.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-22T04:10:09.003 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:09.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-22T04:10:09.034 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:09.030+0000 7f47c1e48180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-22T04:10:09.053 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:09.053 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-22T04:10:09.069 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:09.067+0000 7f78bebd3180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-22T04:10:09.115 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:09.115 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-22T04:10:09.135 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:09.136 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-22T04:10:09.197 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:09.197 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-22T04:10:09.257 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:10:09.257 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-22T04:10:09.317 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 8e92443d-5bf1-45e4-81bd-0e22f9ecfae4 0 2024-03-22T04:10:09.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9c600640 1 Processor -- start 2024-03-22T04:10:09.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9c600640 1 -- start start 2024-03-22T04:10:09.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9c600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdd940b3610 0x7fdd9409b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9c600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 0x7fdd940b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9c600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd940cda00 con 0x7fdd9415c730 2024-03-22T04:10:09.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9c600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdd9409b910 con 0x7fdd940b3610 2024-03-22T04:10:09.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9c600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdd940b1710 con 0x7fdd940b11c0 2024-03-22T04:10:09.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd99200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdd940b3610 0x7fdd9409b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9a600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 0x7fdd940b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd99200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdd940b3610 0x7fdd9409b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49494/0 (socket says 172.21.15.196:49494) 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd99200640 1 -- 172.21.15.196:0/525635354 learned_addr learned my addr 172.21.15.196:0/525635354 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd99c00640 1 --1- 172.21.15.196:0/525635354 >> v1:172.21.15.196:6789/0 conn(0x7fdd9415c730 0x7fdd9409aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:51952/0 (socket says 172.21.15.196:51952) 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9a600640 1 -- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 msgr2=0x7fdd940b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9a600640 1 -- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 msgr2=0x7fdd940b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9a600640 1 --2- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 0x7fdd940b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd9a600640 1 --2- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 0x7fdd940b1ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd99200640 1 -- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 msgr2=0x7fdd940b1ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd99200640 1 --2- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 0x7fdd940b1ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd99200640 1 -- 172.21.15.196:0/525635354 >> v1:172.21.15.196:6789/0 conn(0x7fdd9415c730 legacy=0x7fdd9409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.491+0000 7fdd99200640 1 -- 172.21.15.196:0/525635354 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd940cdce0 con 0x7fdd940b3610 2024-03-22T04:10:09.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd99200640 1 --2- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd9409b180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009870 tx=0x7fdd8001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e110f3ea3bbaf8a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:09.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd93e00640 1 -- 172.21.15.196:0/525635354 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800b1070 con 0x7fdd940b3610 2024-03-22T04:10:09.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd93e00640 1 -- 172.21.15.196:0/525635354 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd800a8050 con 0x7fdd940b3610 2024-03-22T04:10:09.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd93e00640 1 -- 172.21.15.196:0/525635354 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800ac860 con 0x7fdd940b3610 2024-03-22T04:10:09.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd9c600640 1 -- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 msgr2=0x7fdd9409b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd9c600640 1 --2- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd9409b180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009870 tx=0x7fdd8001c4d0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd9c600640 1 -- 172.21.15.196:0/525635354 shutdown_connections 2024-03-22T04:10:09.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd9c600640 1 --2- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd940b11c0 0x7fdd940b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd9c600640 1 --2- 172.21.15.196:0/525635354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd9409b180 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd9c600640 1 -- 172.21.15.196:0/525635354 >> 172.21.15.196:0/525635354 conn(0x7fdd940b3bf0 msgr2=0x7fdd940b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:09.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd9c600640 1 -- 172.21.15.196:0/525635354 shutdown_connections 2024-03-22T04:10:09.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.495+0000 7fdd9c600640 1 -- 172.21.15.196:0/525635354 wait complete. 2024-03-22T04:10:09.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 Processor -- start 2024-03-22T04:10:09.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 -- start start 2024-03-22T04:10:09.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd940b11c0 0x7fdd94152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd940b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 0x7fdd940b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd940b2c80 con 0x7fdd940b11c0 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdd940b2b00 con 0x7fdd940b3610 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdd940b2e00 con 0x7fdd9415c730 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd940b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd940b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49500/0 (socket says 172.21.15.196:49500) 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 -- 172.21.15.196:0/3660625975 learned_addr learned my addr 172.21.15.196:0/3660625975 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9a600640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 0x7fdd940b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99c00640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd940b11c0 0x7fdd94152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9a600640 1 -- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 msgr2=0x7fdd940b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9a600640 1 -- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 msgr2=0x7fdd940b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9a600640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 0x7fdd940b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:09.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 -- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 msgr2=0x7fdd940b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9a600640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 0x7fdd940b0800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 0x7fdd940b0800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 -- 172.21.15.196:0/3660625975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd940b11c0 msgr2=0x7fdd94152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd940b11c0 0x7fdd94152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 -- 172.21.15.196:0/3660625975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9415da10 con 0x7fdd940b3610 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99c00640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd940b11c0 0x7fdd94152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd99200640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd940b02c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdd800097c0 tx=0x7fdd8001ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd91600640 1 -- 172.21.15.196:0/3660625975 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800b1040 con 0x7fdd940b3610 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd91600640 1 -- 172.21.15.196:0/3660625975 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd800bc070 con 0x7fdd940b3610 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd91600640 1 -- 172.21.15.196:0/3660625975 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800b7650 con 0x7fdd940b3610 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.499+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9415afa0 con 0x7fdd940b3610 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.503+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9415b560 con 0x7fdd940b3610 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.503+0000 7fdd91600640 1 -- 172.21.15.196:0/3660625975 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdd800b77f0 con 0x7fdd940b3610 2024-03-22T04:10:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.503+0000 7fdd91600640 1 -- 172.21.15.196:0/3660625975 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fdd800c19e0 con 0x7fdd940b3610 2024-03-22T04:10:09.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.503+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd94074470 con 0x7fdd940b3610 2024-03-22T04:10:09.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.503+0000 7fdd91600640 1 -- 172.21.15.196:0/3660625975 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdd9415da10 con 0x7fdd940b3610 2024-03-22T04:10:09.519 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:09.514+0000 7f47c1e48180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-22T04:10:09.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.571+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8e92443d-5bf1-45e4-81bd-0e22f9ecfae4", "id": 0} v 0) -- 0x7fdd94152680 con 0x7fdd940b3610 2024-03-22T04:10:09.573 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:09.571+0000 7f78bebd3180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-22T04:10:09.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd91600640 1 -- 172.21.15.196:0/3660625975 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8e92443d-5bf1-45e4-81bd-0e22f9ecfae4", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fdd80016150 con 0x7fdd940b3610 2024-03-22T04:10:09.579 INFO:teuthology.orchestra.run.smithi196.stdout:0 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 msgr2=0x7fdd940b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd940b02c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdd800097c0 tx=0x7fdd8001ad10 comp rx=0 tx=0).stop 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 shutdown_connections 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd940b3610 0x7fdd940b02c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd9415c730 0x7fdd940b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 --2- 172.21.15.196:0/3660625975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd940b11c0 0x7fdd94152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 >> 172.21.15.196:0/3660625975 conn(0x7fdd940b3bf0 msgr2=0x7fdd940aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 shutdown_connections 2024-03-22T04:10:09.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.579+0000 7fdd9c600640 1 -- 172.21.15.196:0/3660625975 wait complete. 2024-03-22T04:10:09.598 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 3c186e62-fd3b-40fa-93b4-72f33353c135 1 2024-03-22T04:10:09.717 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:09.714+0000 7f47c1e48180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-22T04:10:09.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0bc800640 1 Processor -- start 2024-03-22T04:10:09.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0bc800640 1 -- start start 2024-03-22T04:10:09.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0bc800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 0x7fd0b406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0bc800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0b4072650 0x7fd0b4072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0bc800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd0b40b1b80 0x7fd0b40b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0bc800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b4071350 con 0x7fd0b4072650 2024-03-22T04:10:09.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0bc800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b406f100 con 0x7fd0b40b1b80 2024-03-22T04:10:09.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0bc800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b4073150 con 0x7fd0b406e7e0 2024-03-22T04:10:09.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0b9e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 0x7fd0b406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0ba800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd0b40b1b80 0x7fd0b40b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0b9e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 0x7fd0b406ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:58970/0 (socket says 172.21.15.196:58970) 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0b9e00640 1 -- 172.21.15.196:0/2726628620 learned_addr learned my addr 172.21.15.196:0/2726628620 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0b9e00640 1 -- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 msgr2=0x7fd0b406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0b9e00640 1 -- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 msgr2=0x7fd0b406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0b9e00640 1 --2- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 0x7fd0b406ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0b9e00640 1 --2- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 0x7fd0b406ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.731+0000 7fd0b9400640 1 --2- 172.21.15.196:0/2726628620 >> v2:172.21.15.86:3300/0 conn(0x7fd0b4072650 0x7fd0b4072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0ba800640 1 -- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 msgr2=0x7fd0b406ebc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0ba800640 1 --2- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 0x7fd0b406ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0ba800640 1 -- 172.21.15.196:0/2726628620 >> v2:172.21.15.86:3300/0 conn(0x7fd0b4072650 msgr2=0x7fd0b4072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0ba800640 1 --2- 172.21.15.196:0/2726628620 >> v2:172.21.15.86:3300/0 conn(0x7fd0b4072650 0x7fd0b4072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0ba800640 1 -- 172.21.15.196:0/2726628620 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b40715a0 con 0x7fd0b40b1b80 2024-03-22T04:10:09.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0ba800640 1 --2- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b40b1b80 0x7fd0b40b2010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40cef70 tx=0x7fd0b001ea80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3eca35e9a3ed90 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:09.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0b8a00640 1 -- 172.21.15.196:0/2726628620 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00b4070 con 0x7fd0b40b1b80 2024-03-22T04:10:09.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0b8a00640 1 -- 172.21.15.196:0/2726628620 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b00a7a10 con 0x7fd0b40b1b80 2024-03-22T04:10:09.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0b8a00640 1 -- 172.21.15.196:0/2726628620 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00a7de0 con 0x7fd0b40b1b80 2024-03-22T04:10:09.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0bc800640 1 -- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b40b1b80 msgr2=0x7fd0b40b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0bc800640 1 --2- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b40b1b80 0x7fd0b40b2010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40cef70 tx=0x7fd0b001ea80 comp rx=0 tx=0).stop 2024-03-22T04:10:09.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0bc800640 1 -- 172.21.15.196:0/2726628620 shutdown_connections 2024-03-22T04:10:09.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0bc800640 1 --2- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b40b1b80 0x7fd0b40b2010 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0bc800640 1 --2- 172.21.15.196:0/2726628620 >> v2:172.21.15.86:3300/0 conn(0x7fd0b4072650 0x7fd0b4072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0bc800640 1 --2- 172.21.15.196:0/2726628620 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b406e7e0 0x7fd0b406ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0bc800640 1 -- 172.21.15.196:0/2726628620 >> 172.21.15.196:0/2726628620 conn(0x7fd0b40cc940 msgr2=0x7fd0b40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:09.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.735+0000 7fd0bc800640 1 -- 172.21.15.196:0/2726628620 shutdown_connections 2024-03-22T04:10:09.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 -- 172.21.15.196:0/2726628620 wait complete. 2024-03-22T04:10:09.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 Processor -- start 2024-03-22T04:10:09.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 -- start start 2024-03-22T04:10:09.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b406e7e0 0x7fd0b413cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 0x7fd0b41f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40b1b80 0x7fd0b41fa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4072080 con 0x7fd0b40b1b80 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b40ce930 con 0x7fd0b406e7e0 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b40cbd80 con 0x7fd0b4072650 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b406e7e0 0x7fd0b413cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 0x7fd0b41f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 0x7fd0b41f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:58980/0 (socket says 172.21.15.196:58980) 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9400640 1 -- 172.21.15.196:0/3408526824 learned_addr learned my addr 172.21.15.196:0/3408526824 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ba800640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40b1b80 0x7fd0b41fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9400640 1 -- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 msgr2=0x7fd0b41f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9400640 1 -- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 msgr2=0x7fd0b41f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9400640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 0x7fd0b41f9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:09.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9400640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 0x7fd0b41f9e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:09.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ba800640 1 -- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 msgr2=0x7fd0b41f9e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:09.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ba800640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 0x7fd0b41f9e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ba800640 1 -- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b406e7e0 msgr2=0x7fd0b413cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ba800640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b406e7e0 0x7fd0b413cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ba800640 1 -- 172.21.15.196:0/3408526824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b40cc060 con 0x7fd0b40b1b80 2024-03-22T04:10:09.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0b9e00640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b406e7e0 0x7fd0b413cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:09.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ba800640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40b1b80 0x7fd0b41fa340 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00ae000 tx=0x7fd0b001b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:09.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ae000640 1 -- 172.21.15.196:0/3408526824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00bf070 con 0x7fd0b40b1b80 2024-03-22T04:10:09.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ae000640 1 -- 172.21.15.196:0/3408526824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b00af440 con 0x7fd0b40b1b80 2024-03-22T04:10:09.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b413a3e0 con 0x7fd0b40b1b80 2024-03-22T04:10:09.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.739+0000 7fd0ae000640 1 -- 172.21.15.196:0/3408526824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00b4040 con 0x7fd0b40b1b80 2024-03-22T04:10:09.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.743+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b40cae20 con 0x7fd0b40b1b80 2024-03-22T04:10:09.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.743+0000 7fd0ae000640 1 -- 172.21.15.196:0/3408526824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd0b00c2070 con 0x7fd0b40b1b80 2024-03-22T04:10:09.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.743+0000 7fd0ae000640 1 -- 172.21.15.196:0/3408526824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd0b00c4b40 con 0x7fd0b40b1b80 2024-03-22T04:10:09.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.743+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b4074470 con 0x7fd0b40b1b80 2024-03-22T04:10:09.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.751+0000 7fd0ae000640 1 -- 172.21.15.196:0/3408526824 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd0b00c9070 con 0x7fd0b40b1b80 2024-03-22T04:10:09.798 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:09.795+0000 7f78bebd3180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-22T04:10:09.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.819+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3c186e62-fd3b-40fa-93b4-72f33353c135", "id": 1} v 0) -- 0x7fd0b4072200 con 0x7fd0b40b1b80 2024-03-22T04:10:09.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0ae000640 1 -- 172.21.15.196:0/3408526824 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3c186e62-fd3b-40fa-93b4-72f33353c135", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd0b0018730 con 0x7fd0b40b1b80 2024-03-22T04:10:09.832 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40b1b80 msgr2=0x7fd0b41fa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40b1b80 0x7fd0b41fa340 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00ae000 tx=0x7fd0b001b820 comp rx=0 tx=0).stop 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 shutdown_connections 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40b1b80 0x7fd0b41fa340 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0b4072650 0x7fd0b41f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 --2- 172.21.15.196:0/3408526824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0b406e7e0 0x7fd0b413cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 >> 172.21.15.196:0/3408526824 conn(0x7fd0b40cc940 msgr2=0x7fd0b414aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 shutdown_connections 2024-03-22T04:10:09.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.831+0000 7fd0bc800640 1 -- 172.21.15.196:0/3408526824 wait complete. 2024-03-22T04:10:09.851 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 5d711bf0-fab3-4e06-8190-04a685ccc537 2 2024-03-22T04:10:09.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.983+0000 7f2ae0200640 1 Processor -- start 2024-03-22T04:10:09.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.983+0000 7f2ae0200640 1 -- start start 2024-03-22T04:10:09.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2ae0200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2ad815c730 0x7f2ad809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2ae0200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ad80b3610 0x7f2ad809afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2ae0200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad80cda00 con 0x7f2ad80b11c0 2024-03-22T04:10:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2ae0200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad809b910 con 0x7f2ad815c730 2024-03-22T04:10:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2ae0200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad80b1710 con 0x7f2ad80b3610 2024-03-22T04:10:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2adce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ad80b3610 0x7f2ad809afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2adce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ad80b3610 0x7f2ad809afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49520/0 (socket says 172.21.15.196:49520) 2024-03-22T04:10:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2adce00640 1 -- 172.21.15.196:0/2643047941 learned_addr learned my addr 172.21.15.196:0/2643047941 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2adce00640 1 -- 172.21.15.196:0/2643047941 >> v2:172.21.15.86:3300/0 conn(0x7f2ad815c730 msgr2=0x7f2ad809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2adce00640 1 --2- 172.21.15.196:0/2643047941 >> v2:172.21.15.86:3300/0 conn(0x7f2ad815c730 0x7f2ad809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2adce00640 1 -- 172.21.15.196:0/2643047941 >> v1:172.21.15.86:6789/0 conn(0x7f2ad80b11c0 legacy=0x7f2ad80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2adce00640 1 -- 172.21.15.196:0/2643047941 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad80cdce0 con 0x7f2ad80b3610 2024-03-22T04:10:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2adce00640 1 --2- 172.21.15.196:0/2643047941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad809afb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009870 tx=0x7f2ac401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4c75978c46d5a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.987+0000 7f2ad7e00640 1 -- 172.21.15.196:0/2643047941 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40ab680 con 0x7f2ad80b3610 2024-03-22T04:10:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ad7e00640 1 -- 172.21.15.196:0/2643047941 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac40b0070 con 0x7f2ad80b3610 2024-03-22T04:10:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ad7e00640 1 -- 172.21.15.196:0/2643047941 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40aba40 con 0x7f2ad80b3610 2024-03-22T04:10:09.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- 172.21.15.196:0/2643047941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 msgr2=0x7f2ad809afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 --2- 172.21.15.196:0/2643047941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad809afb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009870 tx=0x7f2ac401c460 comp rx=0 tx=0).stop 2024-03-22T04:10:09.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- 172.21.15.196:0/2643047941 shutdown_connections 2024-03-22T04:10:09.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 --2- 172.21.15.196:0/2643047941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad809afb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 --2- 172.21.15.196:0/2643047941 >> v2:172.21.15.86:3300/0 conn(0x7f2ad815c730 0x7f2ad809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- 172.21.15.196:0/2643047941 >> 172.21.15.196:0/2643047941 conn(0x7f2ad80b3bf0 msgr2=0x7f2ad80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:09.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- 172.21.15.196:0/2643047941 shutdown_connections 2024-03-22T04:10:09.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- 172.21.15.196:0/2643047941 wait complete. 2024-03-22T04:10:09.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 Processor -- start 2024-03-22T04:10:09.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- start start 2024-03-22T04:10:09.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 0x7f2ad806e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad81666a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad815c730 0x7f2ad806f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:09.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8169510 con 0x7f2ad815c730 2024-03-22T04:10:09.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8169390 con 0x7f2ad80b3610 2024-03-22T04:10:09.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ae0200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad8169690 con 0x7f2ad80b11c0 2024-03-22T04:10:09.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2adce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad81666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2add800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 0x7f2ad806e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2adce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad81666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49522/0 (socket says 172.21.15.196:49522) 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2adce00640 1 -- 172.21.15.196:0/983009925 learned_addr learned my addr 172.21.15.196:0/983009925 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2ade200640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad815c730 0x7f2ad806f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2add800640 1 -- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 msgr2=0x7f2ad806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2add800640 1 -- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 msgr2=0x7f2ad806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2add800640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 0x7f2ad806e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2add800640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 0x7f2ad806e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2adce00640 1 -- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 msgr2=0x7f2ad806e140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2adce00640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 0x7f2ad806e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2adce00640 1 -- 172.21.15.196:0/983009925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad815c730 msgr2=0x7f2ad806f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2adce00640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad815c730 0x7f2ad806f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.991+0000 7f2adce00640 1 -- 172.21.15.196:0/983009925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad80d11e0 con 0x7f2ad80b3610 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2adce00640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad81666a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40ad040 tx=0x7f2ac401aca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ad5600640 1 -- 172.21.15.196:0/983009925 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40b4800 con 0x7f2ad80b3610 2024-03-22T04:10:09.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ad5600640 1 -- 172.21.15.196:0/983009925 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac40a7070 con 0x7f2ad80b3610 2024-03-22T04:10:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad806daf0 con 0x7f2ad80b3610 2024-03-22T04:10:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ad5600640 1 -- 172.21.15.196:0/983009925 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40b4c00 con 0x7f2ad80b3610 2024-03-22T04:10:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad806d570 con 0x7f2ad80b3610 2024-03-22T04:10:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ad5600640 1 -- 172.21.15.196:0/983009925 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ac40c77d0 con 0x7f2ad80b3610 2024-03-22T04:10:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ad5600640 1 -- 172.21.15.196:0/983009925 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f2ac40beae0 con 0x7f2ad80b3610 2024-03-22T04:10:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad8074470 con 0x7f2ad80b3610 2024-03-22T04:10:09.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:09.995+0000 7f2ad5600640 1 -- 172.21.15.196:0/983009925 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2ac4016110 con 0x7f2ad80b3610 2024-03-22T04:10:10.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.063+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5d711bf0-fab3-4e06-8190-04a685ccc537", "id": 2} v 0) -- 0x7f2ad806d8d0 con 0x7f2ad80b3610 2024-03-22T04:10:10.070 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.067+0000 7f2ad5600640 1 -- 172.21.15.196:0/983009925 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5d711bf0-fab3-4e06-8190-04a685ccc537", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f2ac40b4da0 con 0x7f2ad80b3610 2024-03-22T04:10:10.070 INFO:teuthology.orchestra.run.smithi196.stdout:2 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 msgr2=0x7f2ad81666a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad81666a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40ad040 tx=0x7f2ac401aca0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 shutdown_connections 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ad80b3610 0x7f2ad81666a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad815c730 0x7f2ad806f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 --2- 172.21.15.196:0/983009925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ad80b11c0 0x7f2ad806e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 >> 172.21.15.196:0/983009925 conn(0x7f2ad80b3bf0 msgr2=0x7f2ad80d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 shutdown_connections 2024-03-22T04:10:10.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.071+0000 7f2ae0200640 1 -- 172.21.15.196:0/983009925 wait complete. 2024-03-22T04:10:10.089 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new d846ee42-3859-4fac-9cd0-2476e0b5d4ef 3 2024-03-22T04:10:10.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff26e000640 1 Processor -- start 2024-03-22T04:10:10.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff26e000640 1 -- start start 2024-03-22T04:10:10.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff26e000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 0x7ff2680b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff26e000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff26809b180 0x7ff2680cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff26e000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26806dd70 con 0x7ff26806d8f0 2024-03-22T04:10:10.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff26e000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff2680b49a0 con 0x7ff26809b180 2024-03-22T04:10:10.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff26e000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff2680ccda0 con 0x7ff26815e970 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 0x7ff2680b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff26809b180 0x7ff2680cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff25fe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff26806d8f0 0x7ff26809aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:51962/0 (socket says 172.21.15.196:51962) 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff26809b180 0x7ff2680cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49532/0 (socket says 172.21.15.196:49532) 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 0x7ff2680b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:59008/0 (socket says 172.21.15.196:59008) 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff25fe00640 1 -- 172.21.15.196:0/1459891717 learned_addr learned my addr 172.21.15.196:0/1459891717 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267400640 1 -- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 msgr2=0x7ff2680b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267400640 1 -- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 msgr2=0x7ff2680b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267400640 1 --2- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 0x7ff2680b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267400640 1 --2- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 0x7ff2680b12e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267e00640 1 -- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 msgr2=0x7ff2680b12e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267e00640 1 --2- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 0x7ff2680b12e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267e00640 1 -- 172.21.15.196:0/1459891717 >> v1:172.21.15.196:6789/0 conn(0x7ff26806d8f0 legacy=0x7ff26809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff267e00640 1 -- 172.21.15.196:0/1459891717 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26806dff0 con 0x7ff26809b180 2024-03-22T04:10:10.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.223+0000 7ff266a00640 1 -- 172.21.15.196:0/1459891717 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136026266 0 0) 0x7ff250001120 con 0x7ff26806d8f0 2024-03-22T04:10:10.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.227+0000 7ff267e00640 1 --2- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26809b180 0x7ff2680cd230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff260009d90 tx=0x7ff26001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb2272233860794 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:10.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff266a00640 1 -- 172.21.15.196:0/1459891717 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600b2070 con 0x7ff26809b180 2024-03-22T04:10:10.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff266a00640 1 -- 172.21.15.196:0/1459891717 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2600a5a00 con 0x7ff26809b180 2024-03-22T04:10:10.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff266a00640 1 -- 172.21.15.196:0/1459891717 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600a5dd0 con 0x7ff26809b180 2024-03-22T04:10:10.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26809b180 msgr2=0x7ff2680cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 --2- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26809b180 0x7ff2680cd230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff260009d90 tx=0x7ff26001c940 comp rx=0 tx=0).stop 2024-03-22T04:10:10.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- 172.21.15.196:0/1459891717 shutdown_connections 2024-03-22T04:10:10.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 --2- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26809b180 0x7ff2680cd230 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 --2- 172.21.15.196:0/1459891717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26815e970 0x7ff2680b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- 172.21.15.196:0/1459891717 >> 172.21.15.196:0/1459891717 conn(0x7ff2680b3bf0 msgr2=0x7ff2680b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:10.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- 172.21.15.196:0/1459891717 shutdown_connections 2024-03-22T04:10:10.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- 172.21.15.196:0/1459891717 wait complete. 2024-03-22T04:10:10.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 Processor -- start 2024-03-22T04:10:10.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- start start 2024-03-22T04:10:10.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff26806d8f0 0x7ff2680b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 0x7ff268148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26815e970 0x7ff268146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2680cdf80 con 0x7ff26806d8f0 2024-03-22T04:10:10.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff26815ed50 con 0x7ff26815e970 2024-03-22T04:10:10.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff26e000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff2680ce100 con 0x7ff26809b180 2024-03-22T04:10:10.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff267e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26815e970 0x7ff268146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff25fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 0x7ff268148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff267e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26815e970 0x7ff268146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49542/0 (socket says 172.21.15.196:49542) 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff267e00640 1 -- 172.21.15.196:0/1465517194 learned_addr learned my addr 172.21.15.196:0/1465517194 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff267400640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff26806d8f0 0x7ff2680b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff25fe00640 1 -- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 msgr2=0x7ff268148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff25fe00640 1 -- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 msgr2=0x7ff268148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff25fe00640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 0x7ff268148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.231+0000 7ff25fe00640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 0x7ff268148280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff267e00640 1 -- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 msgr2=0x7ff268148280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff267e00640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 0x7ff268148280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff267e00640 1 -- 172.21.15.196:0/1465517194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff26806d8f0 msgr2=0x7ff2680b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff267e00640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff26806d8f0 0x7ff2680b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff267e00640 1 -- 172.21.15.196:0/1465517194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2681471e0 con 0x7ff26815e970 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff267e00640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26815e970 0x7ff268146690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff260009d90 tx=0x7ff2600196e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff25f400640 1 -- 172.21.15.196:0/1465517194 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600b2040 con 0x7ff26815e970 2024-03-22T04:10:10.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff25f400640 1 -- 172.21.15.196:0/1465517194 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2600ad540 con 0x7ff26815e970 2024-03-22T04:10:10.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff25f400640 1 -- 172.21.15.196:0/1465517194 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600ad910 con 0x7ff26815e970 2024-03-22T04:10:10.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2681474c0 con 0x7ff26815e970 2024-03-22T04:10:10.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff268147a80 con 0x7ff26815e970 2024-03-22T04:10:10.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff25f400640 1 -- 172.21.15.196:0/1465517194 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff2600c1020 con 0x7ff26815e970 2024-03-22T04:10:10.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff25f400640 1 -- 172.21.15.196:0/1465517194 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ff2600b8900 con 0x7ff26815e970 2024-03-22T04:10:10.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.235+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff268074470 con 0x7ff26815e970 2024-03-22T04:10:10.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.239+0000 7ff25f400640 1 -- 172.21.15.196:0/1465517194 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff2600c8020 con 0x7ff26815e970 2024-03-22T04:10:10.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.303+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d846ee42-3859-4fac-9cd0-2476e0b5d4ef", "id": 3} v 0) -- 0x7ff2680d09c0 con 0x7ff26815e970 2024-03-22T04:10:10.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff25f400640 1 -- 172.21.15.196:0/1465517194 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d846ee42-3859-4fac-9cd0-2476e0b5d4ef", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ff260016560 con 0x7ff26815e970 2024-03-22T04:10:10.312 INFO:teuthology.orchestra.run.smithi196.stdout:3 2024-03-22T04:10:10.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26815e970 msgr2=0x7ff268146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26815e970 0x7ff268146690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff260009d90 tx=0x7ff2600196e0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 shutdown_connections 2024-03-22T04:10:10.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff26815e970 0x7ff268146690 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff26809b180 0x7ff268148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 --2- 172.21.15.196:0/1465517194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff26806d8f0 0x7ff2680b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 >> 172.21.15.196:0/1465517194 conn(0x7ff2680b3bf0 msgr2=0x7ff26809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:10.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 shutdown_connections 2024-03-22T04:10:10.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.311+0000 7ff26e000640 1 -- 172.21.15.196:0/1465517194 wait complete. 2024-03-22T04:10:10.331 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 879fbacc-c9f4-4a38-a7db-fa4179109db9 4 2024-03-22T04:10:10.336 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.330+0000 7f47c1e48180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-22T04:10:10.397 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.394+0000 7f47c1e48180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-22T04:10:10.443 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:10.443+0000 7f78bebd3180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-22T04:10:10.457 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.454+0000 7f47c1e48180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-22T04:10:10.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c7600640 1 Processor -- start 2024-03-22T04:10:10.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c7600640 1 -- start start 2024-03-22T04:10:10.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c7600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71c015e970 0x7f71c00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c7600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 0x7f71c009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c7600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c006dd70 con 0x7f71c009afb0 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c7600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f71c00b49a0 con 0x7f71c015e970 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c7600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f71c00ccda0 con 0x7f71c006d8f0 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c4c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71c015e970 0x7f71c00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c4c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71c015e970 0x7f71c00b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49558/0 (socket says 172.21.15.196:49558) 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c4c00640 1 -- 172.21.15.196:0/460246726 learned_addr learned my addr 172.21.15.196:0/460246726 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71bfe00640 1 --2- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 0x7f71c009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71bfe00640 1 -- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 msgr2=0x7f71c009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71bfe00640 1 -- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 msgr2=0x7f71c009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71bfe00640 1 --2- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 0x7f71c009aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c4c00640 1 -- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 msgr2=0x7f71c009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71bfe00640 1 --2- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 0x7f71c009aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c4c00640 1 --2- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 0x7f71c009aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c4c00640 1 -- 172.21.15.196:0/460246726 >> v1:172.21.15.86:6789/0 conn(0x7f71c009afb0 legacy=0x7f71c00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.467+0000 7f71c4c00640 1 -- 172.21.15.196:0/460246726 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c006dff0 con 0x7f71c015e970 2024-03-22T04:10:10.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c4c00640 1 --2- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 0x7f71c00b12e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009870 tx=0x7f71b401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d45f28ce157fc17 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:10.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71bf400640 1 -- 172.21.15.196:0/460246726 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b40ab650 con 0x7f71c015e970 2024-03-22T04:10:10.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71bf400640 1 -- 172.21.15.196:0/460246726 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b40b0070 con 0x7f71c015e970 2024-03-22T04:10:10.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71bf400640 1 -- 172.21.15.196:0/460246726 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b40aba10 con 0x7f71c015e970 2024-03-22T04:10:10.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c7600640 1 -- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 msgr2=0x7f71c00b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c7600640 1 --2- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 0x7f71c00b12e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009870 tx=0x7f71b401c560 comp rx=0 tx=0).stop 2024-03-22T04:10:10.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c7600640 1 -- 172.21.15.196:0/460246726 shutdown_connections 2024-03-22T04:10:10.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c7600640 1 --2- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c006d8f0 0x7f71c009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c7600640 1 --2- 172.21.15.196:0/460246726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 0x7f71c00b12e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c7600640 1 -- 172.21.15.196:0/460246726 >> 172.21.15.196:0/460246726 conn(0x7f71c00b3bf0 msgr2=0x7f71c00b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:10.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c7600640 1 -- 172.21.15.196:0/460246726 shutdown_connections 2024-03-22T04:10:10.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.471+0000 7f71c7600640 1 -- 172.21.15.196:0/460246726 wait complete. 2024-03-22T04:10:10.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 Processor -- start 2024-03-22T04:10:10.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 -- start start 2024-03-22T04:10:10.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c006d8f0 0x7f71c0156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 0x7f71c0157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 0x7f71c0158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71c016b550 con 0x7f71c006d8f0 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71c015ed50 con 0x7f71c015e970 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f71c016b6d0 con 0x7f71c009afb0 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 0x7f71c0157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c5600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 0x7f71c0158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 0x7f71c0157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:59032/0 (socket says 172.21.15.196:59032) 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bfe00640 1 -- 172.21.15.196:0/3004763252 learned_addr learned my addr 172.21.15.196:0/3004763252 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c4c00640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c006d8f0 0x7f71c0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bfe00640 1 -- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 msgr2=0x7f71c0157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bfe00640 1 -- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 msgr2=0x7f71c0157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bfe00640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 0x7f71c0157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bfe00640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 0x7f71c0157fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c5600640 1 -- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 msgr2=0x7f71c0157fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:10.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c5600640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 0x7f71c0157fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c5600640 1 -- 172.21.15.196:0/3004763252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c006d8f0 msgr2=0x7f71c0156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c5600640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c006d8f0 0x7f71c0156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c5600640 1 -- 172.21.15.196:0/3004763252 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c0159030 con 0x7f71c015e970 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c5600640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 0x7f71c0158510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71b800c700 tx=0x7f71b800cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bcc00640 1 -- 172.21.15.196:0/3004763252 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8003b80 con 0x7f71c015e970 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bcc00640 1 -- 172.21.15.196:0/3004763252 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b8003d20 con 0x7f71c015e970 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71bcc00640 1 -- 172.21.15.196:0/3004763252 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800aa30 con 0x7f71c015e970 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c0156f30 con 0x7f71c015e970 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.475+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01574f0 con 0x7f71c015e970 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.479+0000 7f71bcc00640 1 -- 172.21.15.196:0/3004763252 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71b800abd0 con 0x7f71c015e970 2024-03-22T04:10:10.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.479+0000 7f71bcc00640 1 -- 172.21.15.196:0/3004763252 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f71b80055e0 con 0x7f71c015e970 2024-03-22T04:10:10.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.479+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c0074470 con 0x7f71c015e970 2024-03-22T04:10:10.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.479+0000 7f71bcc00640 1 -- 172.21.15.196:0/3004763252 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f71c0159030 con 0x7f71c015e970 2024-03-22T04:10:10.510 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:10.507+0000 7f78bebd3180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-22T04:10:10.545 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.542+0000 7f47c1e48180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-22T04:10:10.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.547+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "879fbacc-c9f4-4a38-a7db-fa4179109db9", "id": 4} v 0) -- 0x7f71c0075020 con 0x7f71c015e970 2024-03-22T04:10:10.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.551+0000 7f71bcc00640 1 -- 172.21.15.196:0/3004763252 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "879fbacc-c9f4-4a38-a7db-fa4179109db9", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f71c006dd70 con 0x7f71c015e970 2024-03-22T04:10:10.553 INFO:teuthology.orchestra.run.smithi196.stdout:4 2024-03-22T04:10:10.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.551+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 msgr2=0x7f71c0158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.551+0000 7f71c7600640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 0x7f71c0158510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71b800c700 tx=0x7f71b800cbc0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.551+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 shutdown_connections 2024-03-22T04:10:10.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.551+0000 7f71c7600640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71c015e970 0x7f71c0158510 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.551+0000 7f71c7600640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71c009afb0 0x7f71c0157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.551+0000 7f71c7600640 1 --2- 172.21.15.196:0/3004763252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71c006d8f0 0x7f71c0156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.551+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 >> 172.21.15.196:0/3004763252 conn(0x7f71c00b3bf0 msgr2=0x7f71c00b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:10.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.555+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 shutdown_connections 2024-03-22T04:10:10.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.555+0000 7f71c7600640 1 -- 172.21.15.196:0/3004763252 wait complete. 2024-03-22T04:10:10.578 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 7afce081-80c5-4cfd-9bb7-3becb644f0b5 5 2024-03-22T04:10:10.582 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:10.579+0000 7f78bebd3180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-22T04:10:10.606 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.606+0000 7f47c1e48180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-22T04:10:10.673 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:10.671+0000 7f78bebd3180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-22T04:10:10.693 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.694+0000 7f47c1e48180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-22T04:10:10.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.711+0000 7f59d8c00640 1 Processor -- start 2024-03-22T04:10:10.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.711+0000 7f59d8c00640 1 -- start start 2024-03-22T04:10:10.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59d40b6b40 0x7f59d406e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d8c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 0x7f59d40b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d40726b0 con 0x7f59d4144a80 2024-03-22T04:10:10.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d8c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f59d406ede0 con 0x7f59d40b6b40 2024-03-22T04:10:10.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d8c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f59d4072460 con 0x7f59d406e970 2024-03-22T04:10:10.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d2800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 0x7f59d40b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d2800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 0x7f59d40b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:59036/0 (socket says 172.21.15.196:59036) 2024-03-22T04:10:10.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d2800640 1 -- 172.21.15.196:0/2017864590 learned_addr learned my addr 172.21.15.196:0/2017864590 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:10.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d2800640 1 -- 172.21.15.196:0/2017864590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 msgr2=0x7f59d40b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-22T04:10:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d2800640 1 -- 172.21.15.196:0/2017864590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 msgr2=0x7f59d40b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d2800640 1 --2- 172.21.15.196:0/2017864590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 0x7f59d40b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59d2800640 1 --2- 172.21.15.196:0/2017864590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 0x7f59d40b41e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59cbe00640 1 --2- 172.21.15.196:0/2017864590 >> v2:172.21.15.86:3300/0 conn(0x7f59d40b6b40 0x7f59d406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59cbe00640 1 -- 172.21.15.196:0/2017864590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 msgr2=0x7f59d40b41e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59cbe00640 1 --2- 172.21.15.196:0/2017864590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 0x7f59d40b41e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59cbe00640 1 -- 172.21.15.196:0/2017864590 >> v1:172.21.15.86:6789/0 conn(0x7f59d4144a80 legacy=0x7f59d40b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:10.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.715+0000 7f59cbe00640 1 -- 172.21.15.196:0/2017864590 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d4072990 con 0x7f59d40b6b40 2024-03-22T04:10:10.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.719+0000 7f59cbe00640 1 --2- 172.21.15.196:0/2017864590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40b6b40 0x7f59d406e430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59d40739c0 tx=0x7f59bc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f226f46bee59b5be server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:10.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.719+0000 7f59d1400640 1 -- 172.21.15.196:0/2017864590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0b0070 con 0x7f59d40b6b40 2024-03-22T04:10:10.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.719+0000 7f59d1400640 1 -- 172.21.15.196:0/2017864590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc0ab600 con 0x7f59d40b6b40 2024-03-22T04:10:10.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.719+0000 7f59d1400640 1 -- 172.21.15.196:0/2017864590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0ab9d0 con 0x7f59d40b6b40 2024-03-22T04:10:10.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- 172.21.15.196:0/2017864590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40b6b40 msgr2=0x7f59d406e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 --2- 172.21.15.196:0/2017864590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40b6b40 0x7f59d406e430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59d40739c0 tx=0x7f59bc01c440 comp rx=0 tx=0).stop 2024-03-22T04:10:10.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- 172.21.15.196:0/2017864590 shutdown_connections 2024-03-22T04:10:10.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 --2- 172.21.15.196:0/2017864590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d406e970 0x7f59d40b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 --2- 172.21.15.196:0/2017864590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40b6b40 0x7f59d406e430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- 172.21.15.196:0/2017864590 >> 172.21.15.196:0/2017864590 conn(0x7f59d40d0fe0 msgr2=0x7f59d40d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:10.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- 172.21.15.196:0/2017864590 shutdown_connections 2024-03-22T04:10:10.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- 172.21.15.196:0/2017864590 wait complete. 2024-03-22T04:10:10.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 Processor -- start 2024-03-22T04:10:10.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- start start 2024-03-22T04:10:10.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d406e970 0x7f59d4157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d4144a80 0x7f59d413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 0x7f59d40b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59d40b50b0 con 0x7f59d4144a80 2024-03-22T04:10:10.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59d40b4f30 con 0x7f59d406e970 2024-03-22T04:10:10.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d8c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f59d40b5230 con 0x7f59d40b2220 2024-03-22T04:10:10.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d2800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 0x7f59d40b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d1e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d406e970 0x7f59d4157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d2800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 0x7f59d40b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:59052/0 (socket says 172.21.15.196:59052) 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d2800640 1 -- 172.21.15.196:0/872055874 learned_addr learned my addr 172.21.15.196:0/872055874 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59cbe00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d4144a80 0x7f59d413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d2800640 1 -- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 msgr2=0x7f59d40b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d2800640 1 -- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 msgr2=0x7f59d40b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d2800640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 0x7f59d40b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d2800640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 0x7f59d40b26b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59cbe00640 1 -- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 msgr2=0x7f59d40b26b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59cbe00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 0x7f59d40b26b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59cbe00640 1 -- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d406e970 msgr2=0x7f59d4157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59cbe00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d406e970 0x7f59d4157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59cbe00640 1 -- 172.21.15.196:0/872055874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d413d1e0 con 0x7f59d4144a80 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.723+0000 7f59d1e00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d406e970 0x7f59d4157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:10.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59cbe00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d4144a80 0x7f59d413dfa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0ab000 tx=0x7f59bc01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:10.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59ca000640 1 -- 172.21.15.196:0/872055874 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0b0050 con 0x7f59d4144a80 2024-03-22T04:10:10.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59ca000640 1 -- 172.21.15.196:0/872055874 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc0bc070 con 0x7f59d4144a80 2024-03-22T04:10:10.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59ca000640 1 -- 172.21.15.196:0/872055874 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0b5830 con 0x7f59d4144a80 2024-03-22T04:10:10.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d413d4c0 con 0x7f59d4144a80 2024-03-22T04:10:10.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d413da80 con 0x7f59d4144a80 2024-03-22T04:10:10.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59ca000640 1 -- 172.21.15.196:0/872055874 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f59bc0b59d0 con 0x7f59d4144a80 2024-03-22T04:10:10.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59ca000640 1 -- 172.21.15.196:0/872055874 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f59bc0b7450 con 0x7f59d4144a80 2024-03-22T04:10:10.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.727+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d4074470 con 0x7f59d4144a80 2024-03-22T04:10:10.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.731+0000 7f59ca000640 1 -- 172.21.15.196:0/872055874 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f59bc016030 con 0x7f59d4144a80 2024-03-22T04:10:10.739 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:10.739+0000 7f78bebd3180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-22T04:10:10.758 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.758+0000 7f47c1e48180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-22T04:10:10.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.795+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7afce081-80c5-4cfd-9bb7-3becb644f0b5", "id": 5} v 0) -- 0x7f59d406f030 con 0x7f59d4144a80 2024-03-22T04:10:10.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59ca000640 1 -- 172.21.15.196:0/872055874 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7afce081-80c5-4cfd-9bb7-3becb644f0b5", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f59bc0b5d60 con 0x7f59d4144a80 2024-03-22T04:10:10.800 INFO:teuthology.orchestra.run.smithi196.stdout:5 2024-03-22T04:10:10.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d4144a80 msgr2=0x7f59d413dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d4144a80 0x7f59d413dfa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0ab000 tx=0x7f59bc01ac80 comp rx=0 tx=0).stop 2024-03-22T04:10:10.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 shutdown_connections 2024-03-22T04:10:10.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d40b2220 0x7f59d40b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d4144a80 0x7f59d413dfa0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 --2- 172.21.15.196:0/872055874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d406e970 0x7f59d4157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 >> 172.21.15.196:0/872055874 conn(0x7f59d40d0fe0 msgr2=0x7f59d40d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:10.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 shutdown_connections 2024-03-22T04:10:10.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.799+0000 7f59d8c00640 1 -- 172.21.15.196:0/872055874 wait complete. 2024-03-22T04:10:10.820 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new d7750fe3-0854-4f19-b8a4-2ba996534d09 6 2024-03-22T04:10:10.834 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:10.835+0000 7f78bebd3180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-22T04:10:10.899 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:10.899+0000 7f78bebd3180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-22T04:10:10.922 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.922+0000 7f47c1e48180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-22T04:10:10.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95d5c00640 1 Processor -- start 2024-03-22T04:10:10.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95d5c00640 1 -- start start 2024-03-22T04:10:10.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95d5c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95d01608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95d5c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d0160df0 con 0x7f95d00ce3e0 2024-03-22T04:10:10.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95d5c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d015d3e0 con 0x7f95d00b6380 2024-03-22T04:10:10.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95d5c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f95d015d6c0 con 0x7f95d00ce000 2024-03-22T04:10:10.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95cea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95d01608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95cea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95d01608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:59056/0 (socket says 172.21.15.196:59056) 2024-03-22T04:10:10.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95cea00640 1 -- 172.21.15.196:0/141938149 learned_addr learned my addr 172.21.15.196:0/141938149 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:10.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95cea00640 1 -- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 msgr2=0x7f95d01608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:10.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95cea00640 1 -- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 msgr2=0x7f95d01608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:10.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95cea00640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95d01608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95cea00640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95d01608b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 389986972 0 0) 0x7f95d015d3e0 con 0x7f95d00b6380 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.955+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95b4003530 con 0x7f95d00b6380 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585193005 0 0) 0x7f95d0160df0 con 0x7f95d00ce3e0 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95d015d3e0 con 0x7f95d00ce3e0 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2569678511 0 0) 0x7f95b4003530 con 0x7f95d00b6380 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 msgr2=0x7f95d01608b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95d01608b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 >> v1:172.21.15.86:6789/0 conn(0x7f95d00ce3e0 legacy=0x7f95d015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d01600b0 con 0x7f95d00b6380 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f95c8001a00 con 0x7f95d00b6380 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 >> v1:172.21.15.196:6789/0 conn(0x7f95d00b6380 legacy=0x7f95d00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b4003b50 0x7f95b4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4004d50 0x7f95b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95d015d6c0 con 0x7f95b4003b50 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95b400bb60 con 0x7f95b4004d50 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f95b400bd60 con 0x7f95d00ce000 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95c7e00640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95cf400640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4004d50 0x7f95b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95cea00640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b4003b50 0x7f95b4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95cf400640 1 -- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 msgr2=0x7f95b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95cf400640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95cf400640 1 -- 172.21.15.196:0/141938149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b4003b50 msgr2=0x7f95b4003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:10.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95cf400640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b4003b50 0x7f95b4003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95cf400640 1 -- 172.21.15.196:0/141938149 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b4003530 con 0x7f95b4004d50 2024-03-22T04:10:10.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.959+0000 7f95c7e00640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95d00ce000 0x7f95b4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95cf400640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4004d50 0x7f95b40051d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c410 tx=0x7f95c801cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f05407677817e312 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80b2070 con 0x7f95b4004d50 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c80a5ba0 con 0x7f95b4004d50 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95ce000640 1 -- 172.21.15.196:0/141938149 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c800c0e0 con 0x7f95b4004d50 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4004d50 msgr2=0x7f95b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 --2- 172.21.15.196:0/141938149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4004d50 0x7f95b40051d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f95c800c410 tx=0x7f95c801cea0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95cf400640 1 -- 172.21.15.196:0/141938149 reap_dead start 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- 172.21.15.196:0/141938149 shutdown_connections 2024-03-22T04:10:10.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- 172.21.15.196:0/141938149 >> 172.21.15.196:0/141938149 conn(0x7f95d00d0fe0 msgr2=0x7f95d00d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:10.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- 172.21.15.196:0/141938149 shutdown_connections 2024-03-22T04:10:10.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- 172.21.15.196:0/141938149 wait complete. 2024-03-22T04:10:10.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 Processor -- start 2024-03-22T04:10:10.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- start start 2024-03-22T04:10:10.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 0x7f95d01488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d006b660 0x7f95d006bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95d00b4170 0x7f95d00b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95d00d0300 con 0x7f95d006b660 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95d00d0180 con 0x7f95d00b4170 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.963+0000 7f95d5c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f95d00d0480 con 0x7f95b4004d50 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 0x7f95d01488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95d00b4170 0x7f95d00b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 0x7f95d01488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:59076/0 (socket says 172.21.15.196:59076) 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cea00640 1 -- 172.21.15.196:0/635013039 learned_addr learned my addr 172.21.15.196:0/635013039 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95c7e00640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d006b660 0x7f95d006bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cea00640 1 -- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 msgr2=0x7f95d01488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-22T04:10:10.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cea00640 1 -- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 msgr2=0x7f95d01488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cea00640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 0x7f95d01488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cea00640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 0x7f95d01488b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cf400640 1 -- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 msgr2=0x7f95d01488b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cf400640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 0x7f95d01488b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cf400640 1 -- 172.21.15.196:0/635013039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d006b660 msgr2=0x7f95d006bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cf400640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d006b660 0x7f95d006bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cf400640 1 -- 172.21.15.196:0/635013039 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d00b3e30 con 0x7f95d00b4170 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95cf400640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95d00b4170 0x7f95d00b45f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95c80ac000 tx=0x7f95b4005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95c6a00640 1 -- 172.21.15.196:0/635013039 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80bb070 con 0x7f95d00b4170 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95c6a00640 1 -- 172.21.15.196:0/635013039 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c80a5980 con 0x7f95d00b4170 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95c6a00640 1 -- 172.21.15.196:0/635013039 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80b2040 con 0x7f95d00b4170 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d00b4e60 con 0x7f95d00b4170 2024-03-22T04:10:10.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d00b5420 con 0x7f95d00b4170 2024-03-22T04:10:10.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95c6a00640 1 -- 172.21.15.196:0/635013039 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f95c80b5070 con 0x7f95d00b4170 2024-03-22T04:10:10.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95c6a00640 1 -- 172.21.15.196:0/635013039 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f95c80a5b30 con 0x7f95d00b4170 2024-03-22T04:10:10.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.967+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d0074470 con 0x7f95d00b4170 2024-03-22T04:10:10.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:10.971+0000 7f95c6a00640 1 -- 172.21.15.196:0/635013039 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f95c80c5080 con 0x7f95d00b4170 2024-03-22T04:10:10.994 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:10.994+0000 7f47c1e48180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-22T04:10:11.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.035+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d7750fe3-0854-4f19-b8a4-2ba996534d09", "id": 6} v 0) -- 0x7f95d006e650 con 0x7f95d00b4170 2024-03-22T04:10:11.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95c6a00640 1 -- 172.21.15.196:0/635013039 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d7750fe3-0854-4f19-b8a4-2ba996534d09", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f95c8016bb0 con 0x7f95d00b4170 2024-03-22T04:10:11.044 INFO:teuthology.orchestra.run.smithi196.stdout:6 2024-03-22T04:10:11.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95d00b4170 msgr2=0x7f95d00b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95d00b4170 0x7f95d00b45f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95c80ac000 tx=0x7f95b4005840 comp rx=0 tx=0).stop 2024-03-22T04:10:11.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 shutdown_connections 2024-03-22T04:10:11.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95d00b4170 0x7f95d00b45f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d006b660 0x7f95d006bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 --2- 172.21.15.196:0/635013039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95b4004d50 0x7f95d01488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 >> 172.21.15.196:0/635013039 conn(0x7f95d00d0fe0 msgr2=0x7f95d015dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:11.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 shutdown_connections 2024-03-22T04:10:11.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.043+0000 7f95d5c00640 1 -- 172.21.15.196:0/635013039 wait complete. 2024-03-22T04:10:11.063 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 41569ccb-8394-4d80-8731-da57174ad08d 7 2024-03-22T04:10:11.072 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:11.071+0000 7f78bebd3180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-22T04:10:11.145 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:11.143+0000 7f78bebd3180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-22T04:10:11.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.203+0000 7feef3400640 1 Processor -- start 2024-03-22T04:10:11.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.203+0000 7feef3400640 1 -- start start 2024-03-22T04:10:11.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef3400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feeec155550 0x7feeec06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef3400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeec15f8a0 con 0x7feeec06efa0 2024-03-22T04:10:11.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef3400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeec06e360 con 0x7feeec0cced0 2024-03-22T04:10:11.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef3400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7feeec06ec00 con 0x7feeec155550 2024-03-22T04:10:11.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef0a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feeec155550 0x7feeec06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef0a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feeec155550 0x7feeec06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49588/0 (socket says 172.21.15.196:49588) 2024-03-22T04:10:11.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef0a00640 1 -- 172.21.15.196:0/1244311891 learned_addr learned my addr 172.21.15.196:0/1244311891 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:11.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef0a00640 1 -- 172.21.15.196:0/1244311891 >> v1:172.21.15.196:6789/0 conn(0x7feeec0cced0 legacy=0x7feeec0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef0a00640 1 -- 172.21.15.196:0/1244311891 >> v1:172.21.15.86:6789/0 conn(0x7feeec06efa0 legacy=0x7feeec15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.207+0000 7feef0a00640 1 -- 172.21.15.196:0/1244311891 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec06e640 con 0x7feeec155550 2024-03-22T04:10:11.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef0a00640 1 --2- 172.21.15.196:0/1244311891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 0x7feeec06dcf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feee0009870 tx=0x7feee001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5678177a72bbd834 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:11.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feeeb400640 1 -- 172.21.15.196:0/1244311891 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00aa600 con 0x7feeec155550 2024-03-22T04:10:11.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feeeb400640 1 -- 172.21.15.196:0/1244311891 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee00af070 con 0x7feeec155550 2024-03-22T04:10:11.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feeeb400640 1 -- 172.21.15.196:0/1244311891 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00aa9c0 con 0x7feeec155550 2024-03-22T04:10:11.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 -- 172.21.15.196:0/1244311891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 msgr2=0x7feeec06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 --2- 172.21.15.196:0/1244311891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 0x7feeec06dcf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feee0009870 tx=0x7feee001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 -- 172.21.15.196:0/1244311891 shutdown_connections 2024-03-22T04:10:11.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 --2- 172.21.15.196:0/1244311891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 0x7feeec06dcf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 -- 172.21.15.196:0/1244311891 >> 172.21.15.196:0/1244311891 conn(0x7feeec067370 msgr2=0x7feeec067780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:11.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 -- 172.21.15.196:0/1244311891 shutdown_connections 2024-03-22T04:10:11.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 -- 172.21.15.196:0/1244311891 wait complete. 2024-03-22T04:10:11.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 Processor -- start 2024-03-22T04:10:11.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.211+0000 7feef3400640 1 -- start start 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 0x7feeec155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeec0cced0 0x7feeec0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 0x7feeec0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feeec0ce710 con 0x7feeec0cced0 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feeec0b2e90 con 0x7feeec155550 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feeec0ce890 con 0x7feeec06efa0 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef0a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 0x7feeec155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef0a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 0x7feeec155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:59092/0 (socket says 172.21.15.196:59092) 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 0x7feeec0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef0a00640 1 -- 172.21.15.196:0/1571295663 learned_addr learned my addr 172.21.15.196:0/1571295663 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-22T04:10:11.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef0a00640 1 -- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 msgr2=0x7feeec155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef0a00640 1 -- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 msgr2=0x7feeec155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef0a00640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 0x7feeec155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef0a00640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 0x7feeec155f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feeebe00640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeec0cced0 0x7feeec0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef1400640 1 -- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 msgr2=0x7feeec155f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef1400640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 0x7feeec155f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef1400640 1 -- 172.21.15.196:0/1571295663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeec0cced0 msgr2=0x7feeec0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef1400640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeec0cced0 0x7feeec0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef1400640 1 -- 172.21.15.196:0/1571295663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec0b5010 con 0x7feeec155550 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feeebe00640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeec0cced0 0x7feeec0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef1400640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 0x7feeec0b3de0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feee400c700 tx=0x7feee400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feee8c00640 1 -- 172.21.15.196:0/1571295663 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4003b80 con 0x7feeec155550 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feee8c00640 1 -- 172.21.15.196:0/1571295663 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee4003d20 con 0x7feeec155550 2024-03-22T04:10:11.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec0b4d30 con 0x7feeec155550 2024-03-22T04:10:11.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feee8c00640 1 -- 172.21.15.196:0/1571295663 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4005310 con 0x7feeec155550 2024-03-22T04:10:11.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec14d760 con 0x7feeec155550 2024-03-22T04:10:11.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feee8c00640 1 -- 172.21.15.196:0/1571295663 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feee40054b0 con 0x7feeec155550 2024-03-22T04:10:11.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feee8c00640 1 -- 172.21.15.196:0/1571295663 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7feee4012a80 con 0x7feeec155550 2024-03-22T04:10:11.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.215+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeec14db40 con 0x7feeec155550 2024-03-22T04:10:11.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.219+0000 7feee8c00640 1 -- 172.21.15.196:0/1571295663 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7feeec14db40 con 0x7feeec155550 2024-03-22T04:10:11.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.283+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "41569ccb-8394-4d80-8731-da57174ad08d", "id": 7} v 0) -- 0x7feeec074470 con 0x7feeec155550 2024-03-22T04:10:11.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feee8c00640 1 -- 172.21.15.196:0/1571295663 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "41569ccb-8394-4d80-8731-da57174ad08d", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7feee4012d20 con 0x7feeec155550 2024-03-22T04:10:11.295 INFO:teuthology.orchestra.run.smithi196.stdout:7 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 msgr2=0x7feeec0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 0x7feeec0b3de0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feee400c700 tx=0x7feee400cbc0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 shutdown_connections 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeec155550 0x7feeec0b3de0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeec0cced0 0x7feeec0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 --2- 172.21.15.196:0/1571295663 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeec06efa0 0x7feeec155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 >> 172.21.15.196:0/1571295663 conn(0x7feeec067370 msgr2=0x7feeec161000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 shutdown_connections 2024-03-22T04:10:11.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:10:11.295+0000 7feef3400640 1 -- 172.21.15.196:0/1571295663 wait complete. 2024-03-22T04:10:11.313 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-22T04:10:11.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-22T04:10:11.318 INFO:tasks.ceph.osd.0:Started 2024-03-22T04:10:11.318 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-22T04:10:11.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-22T04:10:11.329 INFO:tasks.ceph.osd.1:Started 2024-03-22T04:10:11.329 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-22T04:10:11.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-22T04:10:11.333 INFO:tasks.ceph.osd.2:Started 2024-03-22T04:10:11.333 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-22T04:10:11.333 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-22T04:10:11.341 INFO:tasks.ceph.osd.3:Started 2024-03-22T04:10:11.341 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-22T04:10:11.342 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-22T04:10:11.345 INFO:tasks.ceph.osd.4:Started 2024-03-22T04:10:11.345 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-22T04:10:11.345 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-22T04:10:11.348 INFO:tasks.ceph.osd.5:Started 2024-03-22T04:10:11.348 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-22T04:10:11.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-22T04:10:11.351 INFO:tasks.ceph.osd.6:Started 2024-03-22T04:10:11.351 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-22T04:10:11.351 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-22T04:10:11.355 INFO:tasks.ceph.osd.7:Started 2024-03-22T04:10:11.355 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0feec00640 1 Processor -- start 2024-03-22T04:10:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0feec00640 1 -- start start 2024-03-22T04:10:11.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0feec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe815b500 0x7f0fe80aff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0feec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0fe80b0480 0x7f0fe80ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0feec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0fe80aec20 0x7f0fe80b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0feec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe80b37d0 con 0x7f0fe80b0480 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0feec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe80af4b0 con 0x7f0fe80aec20 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0feec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe80afd10 con 0x7f0fe815b500 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fdf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0fe80b0480 0x7f0fe80ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fdf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0fe80b0480 0x7f0fe80ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43136/0 (socket says 172.21.15.86:43136) 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fdf400640 1 -- 172.21.15.86:0/1263029155 learned_addr learned my addr 172.21.15.86:0/1263029155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fe7e00640 1 --2- 172.21.15.86:0/1263029155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe815b500 0x7f0fe80aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fecc00640 1 --2- 172.21.15.86:0/1263029155 >> v2:172.21.15.196:3300/0 conn(0x7f0fe80aec20 0x7f0fe80b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fdf400640 1 -- 172.21.15.86:0/1263029155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe815b500 msgr2=0x7f0fe80aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fdf400640 1 --2- 172.21.15.86:0/1263029155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe815b500 0x7f0fe80aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fdf400640 1 -- 172.21.15.86:0/1263029155 >> v2:172.21.15.196:3300/0 conn(0x7f0fe80aec20 msgr2=0x7f0fe80b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fdf400640 1 --2- 172.21.15.86:0/1263029155 >> v2:172.21.15.196:3300/0 conn(0x7f0fe80aec20 0x7f0fe80b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.546+0000 7f0fdf400640 1 -- 172.21.15.86:0/1263029155 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe815b8e0 con 0x7f0fe80b0480 2024-03-22T04:10:11.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0fdf400640 1 --2- 172.21.15.86:0/1263029155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe80ae6e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0fe806e0d0 tx=0x7f0fd001e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0fb613b0bab37f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:11.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0fe7400640 1 -- 172.21.15.86:0/1263029155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00b4070 con 0x7f0fe80b0480 2024-03-22T04:10:11.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0fe7400640 1 -- 172.21.15.86:0/1263029155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd00a7820 con 0x7f0fe80b0480 2024-03-22T04:10:11.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0fe7400640 1 -- 172.21.15.86:0/1263029155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00a7bf0 con 0x7f0fe80b0480 2024-03-22T04:10:11.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 -- 172.21.15.86:0/1263029155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 msgr2=0x7f0fe80ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 --2- 172.21.15.86:0/1263029155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe80ae6e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0fe806e0d0 tx=0x7f0fd001e8a0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 -- 172.21.15.86:0/1263029155 shutdown_connections 2024-03-22T04:10:11.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 --2- 172.21.15.86:0/1263029155 >> v2:172.21.15.196:3300/0 conn(0x7f0fe80aec20 0x7f0fe80b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 --2- 172.21.15.86:0/1263029155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe80ae6e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 --2- 172.21.15.86:0/1263029155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe815b500 0x7f0fe80aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 -- 172.21.15.86:0/1263029155 >> 172.21.15.86:0/1263029155 conn(0x7f0fe8072d20 msgr2=0x7f0fe80b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:11.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 -- 172.21.15.86:0/1263029155 shutdown_connections 2024-03-22T04:10:11.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.550+0000 7f0feec00640 1 -- 172.21.15.86:0/1263029155 wait complete. 2024-03-22T04:10:11.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.554+0000 7f0feec00640 1 Processor -- start 2024-03-22T04:10:11.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.554+0000 7f0feec00640 1 -- start start 2024-03-22T04:10:11.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe80aec20 0x7f0fe81f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe81454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fe815b500 0x7f0fe81643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe806b660 con 0x7f0fe80b0480 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80af240 con 0x7f0fe815b500 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe806b7e0 con 0x7f0fe80aec20 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe81454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe81454c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43150/0 (socket says 172.21.15.86:43150) 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 -- 172.21.15.86:0/202475325 learned_addr learned my addr 172.21.15.86:0/202475325 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fe7e00640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe80aec20 0x7f0fe81f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fecc00640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fe815b500 0x7f0fe81643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 -- 172.21.15.86:0/202475325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe80aec20 msgr2=0x7f0fe81f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe80aec20 0x7f0fe81f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 -- 172.21.15.86:0/202475325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fe815b500 msgr2=0x7f0fe81643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fe815b500 0x7f0fe81643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 -- 172.21.15.86:0/202475325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe8145a00 con 0x7f0fe80b0480 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fdf400640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe81454c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0fd001cf50 tx=0x7f0fd001b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fe4c00640 1 -- 172.21.15.86:0/202475325 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00b4070 con 0x7f0fe80b0480 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fe4c00640 1 -- 172.21.15.86:0/202475325 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd00b6070 con 0x7f0fe80b0480 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fe4c00640 1 -- 172.21.15.86:0/202475325 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00afb50 con 0x7f0fe80b0480 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe815d560 con 0x7f0fe80b0480 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe815db20 con 0x7f0fe80b0480 2024-03-22T04:10:11.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fe4c00640 1 -- 172.21.15.86:0/202475325 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0fd00bb800 con 0x7f0fe80b0480 2024-03-22T04:10:11.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0fe4c00640 1 -- 172.21.15.86:0/202475325 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0fd00c5c50 con 0x7f0fe80b0480 2024-03-22T04:10:11.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.558+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe8068d60 con 0x7f0fe80b0480 2024-03-22T04:10:11.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.562+0000 7f0fe4c00640 1 -- 172.21.15.86:0/202475325 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0fe8145a00 con 0x7f0fe80b0480 2024-03-22T04:10:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fe8146150 con 0x7f0fe80b0480 2024-03-22T04:10:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0fe4c00640 1 -- 172.21.15.86:0/202475325 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0fd00187e0 con 0x7f0fe80b0480 2024-03-22T04:10:11.618 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:11.618 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:11.288822+0000","last_up_change":"0.000000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 msgr2=0x7f0fe81454c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe81454c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0fd001cf50 tx=0x7f0fd001b640 comp rx=0 tx=0).stop 2024-03-22T04:10:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 shutdown_connections 2024-03-22T04:10:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fe815b500 0x7f0fe81643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe80b0480 0x7f0fe81454c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 --2- 172.21.15.86:0/202475325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fe80aec20 0x7f0fe81f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 >> 172.21.15.86:0/202475325 conn(0x7f0fe8072d20 msgr2=0x7f0fe8163300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 shutdown_connections 2024-03-22T04:10:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:11.618+0000 7f0feec00640 1 -- 172.21.15.86:0/202475325 wait complete. 2024-03-22T04:10:11.637 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-22T04:10:11.637 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-22T04:10:11.709 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-22T04:10:11.709 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-22T04:10:11.709 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-03-22T04:10:11.749 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:11.750+0000 7f47c1e48180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-22T04:10:11.817 INFO:tasks.ceph.mgr.x.smithi196.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-22T04:10:11.817 INFO:tasks.ceph.mgr.x.smithi196.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-22T04:10:11.817 INFO:tasks.ceph.mgr.x.smithi196.stderr: from numpy import show_config as show_numpy_config 2024-03-22T04:10:11.868 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:11.867+0000 7f78bebd3180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-22T04:10:11.897 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:11.898+0000 7f47c1e48180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-22T04:10:11.988 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-03-22T04:10:11.987+0000 7f15eec2b7c0 -1 Falling back to public interface 2024-03-22T04:10:11.991 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-03-22T04:10:11.991+0000 7f108a14e7c0 -1 Falling back to public interface 2024-03-22T04:10:12.002 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-03-22T04:10:12.002+0000 7fe7eab297c0 -1 Falling back to public interface 2024-03-22T04:10:12.007 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-03-22T04:10:12.007+0000 7fbd855197c0 -1 Falling back to public interface 2024-03-22T04:10:12.008 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-03-22T04:10:12.007+0000 7fe37429e7c0 -1 Falling back to public interface 2024-03-22T04:10:12.018 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-03-22T04:10:12.018+0000 7efeae2cd7c0 -1 Falling back to public interface 2024-03-22T04:10:12.023 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:12.023+0000 7f78bebd3180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-22T04:10:12.030 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-03-22T04:10:12.030+0000 7fde821b67c0 -1 Falling back to public interface 2024-03-22T04:10:12.034 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-03-22T04:10:12.034+0000 7f4dd3a547c0 -1 Falling back to public interface 2024-03-22T04:10:12.084 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:12.082+0000 7f47c1e48180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-22T04:10:12.157 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:12.158+0000 7f47c1e48180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-22T04:10:12.193 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:12.191+0000 7f78bebd3180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-22T04:10:12.267 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:12.267+0000 7f78bebd3180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-22T04:10:12.305 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:12.306+0000 7f47c1e48180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-22T04:10:12.418 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:12.415+0000 7f78bebd3180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-22T04:10:12.442 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:12.568 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.566+0000 7f0351800640 1 Processor -- start 2024-03-22T04:10:12.568 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.566+0000 7f0351800640 1 -- start start 2024-03-22T04:10:12.570 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0351800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 0x7f034c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0351800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f034c0b4930 0x7f034c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0351800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f034c0d1470 0x7f034c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0351800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f034c0d0aa0 con 0x7f034c0b4930 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0351800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f034c0d0490 con 0x7f034c0d1470 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0351800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f034c06cf50 con 0x7f034c15e510 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0343e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f034c0b4930 0x7f034c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0343e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f034c0b4930 0x7f034c0cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43162/0 (socket says 172.21.15.86:43162) 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0343e00640 1 -- 172.21.15.86:0/3890050349 learned_addr learned my addr 172.21.15.86:0/3890050349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f034b400640 1 --2- 172.21.15.86:0/3890050349 >> v2:172.21.15.196:3300/0 conn(0x7f034c0d1470 0x7f034c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f034aa00640 1 --2- 172.21.15.86:0/3890050349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 0x7f034c0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f034aa00640 1 -- 172.21.15.86:0/3890050349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 msgr2=0x7f034c0d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f034aa00640 1 -- 172.21.15.86:0/3890050349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 msgr2=0x7f034c0d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f034aa00640 1 --2- 172.21.15.86:0/3890050349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 0x7f034c0d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:12.571 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f034aa00640 1 --2- 172.21.15.86:0/3890050349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 0x7f034c0d0f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:12.572 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0343e00640 1 -- 172.21.15.86:0/3890050349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 msgr2=0x7f034c0d0f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:10:12.572 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0343e00640 1 --2- 172.21.15.86:0/3890050349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 0x7f034c0d0f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.572 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0343e00640 1 -- 172.21.15.86:0/3890050349 >> v2:172.21.15.196:3300/0 conn(0x7f034c0d1470 msgr2=0x7f034c0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:12.572 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0343e00640 1 --2- 172.21.15.86:0/3890050349 >> v2:172.21.15.196:3300/0 conn(0x7f034c0d1470 0x7f034c0ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.572 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.570+0000 7f0343e00640 1 -- 172.21.15.86:0/3890050349 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c06cc70 con 0x7f034c0b4930 2024-03-22T04:10:12.573 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0343e00640 1 --2- 172.21.15.86:0/3890050349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c0cc740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f034c0d0a10 tx=0x7f033401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b656e76b2dfcba server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:12.573 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f034a000640 1 -- 172.21.15.86:0/3890050349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340ab5f0 con 0x7f034c0b4930 2024-03-22T04:10:12.574 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f034a000640 1 -- 172.21.15.86:0/3890050349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03340b0070 con 0x7f034c0b4930 2024-03-22T04:10:12.574 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f034a000640 1 -- 172.21.15.86:0/3890050349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340ab9b0 con 0x7f034c0b4930 2024-03-22T04:10:12.574 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 -- 172.21.15.86:0/3890050349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 msgr2=0x7f034c0cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:12.574 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 --2- 172.21.15.86:0/3890050349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c0cc740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f034c0d0a10 tx=0x7f033401c520 comp rx=0 tx=0).stop 2024-03-22T04:10:12.575 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 -- 172.21.15.86:0/3890050349 shutdown_connections 2024-03-22T04:10:12.575 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 --2- 172.21.15.86:0/3890050349 >> v2:172.21.15.196:3300/0 conn(0x7f034c0d1470 0x7f034c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.575 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 --2- 172.21.15.86:0/3890050349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c0cc740 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.575 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 --2- 172.21.15.86:0/3890050349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15e510 0x7f034c0d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.575 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 -- 172.21.15.86:0/3890050349 >> 172.21.15.86:0/3890050349 conn(0x7f034c0b3be0 msgr2=0x7f034c0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:12.575 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 -- 172.21.15.86:0/3890050349 shutdown_connections 2024-03-22T04:10:12.575 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 -- 172.21.15.86:0/3890050349 wait complete. 2024-03-22T04:10:12.576 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 Processor -- start 2024-03-22T04:10:12.576 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.574+0000 7f0351800640 1 -- start start 2024-03-22T04:10:12.578 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f034c15e510 0x7f034c150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 0x7f034c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f034c09b600 con 0x7f034c0b4930 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f034c09b480 con 0x7f034c15e510 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f034c09b780 con 0x7f034c15bdb0 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43176/0 (socket says 172.21.15.86:43176) 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 -- 172.21.15.86:0/571668468 learned_addr learned my addr 172.21.15.86:0/571668468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0343e00640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f034c15e510 0x7f034c150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034b400640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 0x7f034c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034b400640 1 -- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 msgr2=0x7f034c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034b400640 1 -- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 msgr2=0x7f034c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034b400640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 0x7f034c15c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-22T04:10:12.579 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034b400640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 0x7f034c15c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 -- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 msgr2=0x7f034c15c260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 0x7f034c15c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 -- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f034c15e510 msgr2=0x7f034c150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f034c15e510 0x7f034c150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 -- 172.21.15.86:0/571668468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c0c5f90 con 0x7f034c0b4930 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f034aa00640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c150330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e7e0 tx=0x7f033c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0342a00640 1 -- 172.21.15.86:0/571668468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c017020 con 0x7f034c0b4930 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0342a00640 1 -- 172.21.15.86:0/571668468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033c00c6d0 con 0x7f034c0b4930 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0342a00640 1 -- 172.21.15.86:0/571668468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c00f040 con 0x7f034c0b4930 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c0c6270 con 0x7f034c0b4930 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c0b5330 con 0x7f034c0b4930 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0342a00640 1 -- 172.21.15.86:0/571668468 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f033c01a020 con 0x7f034c0b4930 2024-03-22T04:10:12.580 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0342a00640 1 -- 172.21.15.86:0/571668468 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f033c013b00 con 0x7f034c0b4930 2024-03-22T04:10:12.581 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.578+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c068f80 con 0x7f034c0b4930 2024-03-22T04:10:12.581 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.582+0000 7f0342a00640 1 -- 172.21.15.86:0/571668468 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f033c021020 con 0x7f034c0b4930 2024-03-22T04:10:12.638 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f034c0d1c10 con 0x7f034c0b4930 2024-03-22T04:10:12.639 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0342a00640 1 -- 172.21.15.86:0/571668468 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f033c013430 con 0x7f034c0b4930 2024-03-22T04:10:12.639 INFO:teuthology.misc.health.smithi086.stdout: 2024-03-22T04:10:12.639 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":10,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:11.288822+0000","last_up_change":"0.000000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:12.640 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 msgr2=0x7f034c150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:12.640 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c150330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e7e0 tx=0x7f033c00eca0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.640 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 shutdown_connections 2024-03-22T04:10:12.640 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f034c15e510 0x7f034c150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.640 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f034c15bdb0 0x7f034c15c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.640 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 --2- 172.21.15.86:0/571668468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f034c0b4930 0x7f034c150330 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:12.641 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 >> 172.21.15.86:0/571668468 conn(0x7f034c0b3be0 msgr2=0x7f034c0b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:12.641 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 shutdown_connections 2024-03-22T04:10:12.641 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:12.638+0000 7f0351800640 1 -- 172.21.15.86:0/571668468 wait complete. 2024-03-22T04:10:12.656 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-22T04:10:13.006 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:13.006+0000 7f47c1e48180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-22T04:10:13.092 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:13.090+0000 7f47c1e48180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-22T04:10:13.111 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:13.111+0000 7f78bebd3180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-22T04:10:13.198 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:13.195+0000 7f78bebd3180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-22T04:10:13.293 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:13.294+0000 7f47c1e48180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-22T04:10:13.377 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:13.378+0000 7f47c1e48180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-22T04:10:13.400 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:13.399+0000 7f78bebd3180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-22T04:10:13.485 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:13.483+0000 7f78bebd3180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-22T04:10:13.704 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:13.702+0000 7f47c1e48180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-22T04:10:13.775 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:13.774+0000 7f47c1e48180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-22T04:10:13.819 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:13.819+0000 7f78bebd3180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-22T04:10:13.891 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-03-22T04:10:13.891+0000 7f78bebd3180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-22T04:10:14.797 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:14.798+0000 7f4767400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-22T04:10:14.798 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-03-22T04:10:14.798+0000 7f4767400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-22T04:10:15.724 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-03-22T04:10:15.723+0000 7f15eec2b7c0 -1 osd.4 0 log_to_monitors true 2024-03-22T04:10:15.858 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-03-22T04:10:15.855+0000 7f108a14e7c0 -1 osd.5 0 log_to_monitors true 2024-03-22T04:10:15.890 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-03-22T04:10:15.887+0000 7fe37429e7c0 -1 osd.6 0 log_to_monitors true 2024-03-22T04:10:15.903 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-03-22T04:10:15.902+0000 7fe7eab297c0 -1 osd.1 0 log_to_monitors true 2024-03-22T04:10:15.926 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-03-22T04:10:15.923+0000 7fbd855197c0 -1 osd.7 0 log_to_monitors true 2024-03-22T04:10:15.940 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-03-22T04:10:15.938+0000 7efeae2cd7c0 -1 osd.0 0 log_to_monitors true 2024-03-22T04:10:16.000 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-03-22T04:10:15.998+0000 7fde821b67c0 -1 osd.2 0 log_to_monitors true 2024-03-22T04:10:16.010 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-03-22T04:10:16.010+0000 7f4dd3a547c0 -1 osd.3 0 log_to_monitors true 2024-03-22T04:10:17.746 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-03-22T04:10:17.742+0000 7f15e9000640 -1 osd.4 0 waiting for initial osdmap 2024-03-22T04:10:17.768 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-03-22T04:10:17.766+0000 7f15df000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T04:10:19.463 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:19.555 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-03-22T04:10:19.558+0000 7f1085000640 -1 osd.5 0 waiting for initial osdmap 2024-03-22T04:10:19.593 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-03-22T04:10:19.594+0000 7f107b000640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T04:10:19.604 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-03-22T04:10:19.606+0000 7fbd80400640 -1 osd.7 0 waiting for initial osdmap 2024-03-22T04:10:19.631 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-03-22T04:10:19.630+0000 7fe36f200640 -1 osd.6 0 waiting for initial osdmap 2024-03-22T04:10:19.640 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-03-22T04:10:19.642+0000 7fbd76400640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T04:10:19.648 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-03-22T04:10:19.650+0000 7fe365200640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T04:10:19.675 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.674+0000 7ff071000640 1 Processor -- start 2024-03-22T04:10:19.676 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.674+0000 7ff071000640 1 -- start start 2024-03-22T04:10:19.678 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff071000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:19.678 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff071000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff06c0cc790 0x7ff06c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:19.678 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff071000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06c0b3460 con 0x7ff06c06eeb0 2024-03-22T04:10:19.678 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff071000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff06c0cf940 con 0x7ff06c0cc790 2024-03-22T04:10:19.678 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff071000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff06c0714d0 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.679 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff069e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:19.679 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff069e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c15a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:49046/0 (socket says 172.21.15.86:49046) 2024-03-22T04:10:19.679 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff069e00640 1 -- 172.21.15.86:0/256311208 learned_addr learned my addr 172.21.15.86:0/256311208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:19.679 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff069400640 1 --2- 172.21.15.86:0/256311208 >> v2:172.21.15.86:3300/0 conn(0x7ff06c0cc790 0x7ff06c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:19.680 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff069e00640 1 -- 172.21.15.86:0/256311208 >> v2:172.21.15.86:3300/0 conn(0x7ff06c0cc790 msgr2=0x7ff06c06e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:19.680 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff069e00640 1 --2- 172.21.15.86:0/256311208 >> v2:172.21.15.86:3300/0 conn(0x7ff06c0cc790 0x7ff06c06e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:19.680 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff069e00640 1 -- 172.21.15.86:0/256311208 >> v1:172.21.15.196:6789/0 conn(0x7ff06c06eeb0 legacy=0x7ff06c15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:19.680 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff068a00640 1 -- 172.21.15.86:0/256311208 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2476817994 0 0) 0x7ff06c0b3460 con 0x7ff06c06eeb0 2024-03-22T04:10:19.681 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.678+0000 7ff069e00640 1 -- 172.21.15.86:0/256311208 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c0b3740 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.683 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff069e00640 1 --2- 172.21.15.86:0/256311208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c15a4d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff05800bb10 tx=0x7ff05801eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbc9959e6fdafb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:19.683 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff068a00640 1 -- 172.21.15.86:0/256311208 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580b4070 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.683 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff068a00640 1 -- 172.21.15.86:0/256311208 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0580a7990 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.684 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff068a00640 1 -- 172.21.15.86:0/256311208 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a7d60 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.684 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff071000640 1 -- 172.21.15.86:0/256311208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 msgr2=0x7ff06c15a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:19.685 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff071000640 1 --2- 172.21.15.86:0/256311208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c15a4d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff05800bb10 tx=0x7ff05801eba0 comp rx=0 tx=0).stop 2024-03-22T04:10:19.685 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff071000640 1 -- 172.21.15.86:0/256311208 shutdown_connections 2024-03-22T04:10:19.685 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff071000640 1 --2- 172.21.15.86:0/256311208 >> v2:172.21.15.86:3300/0 conn(0x7ff06c0cc790 0x7ff06c06e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:19.685 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff071000640 1 --2- 172.21.15.86:0/256311208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c15a4d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:19.685 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff071000640 1 -- 172.21.15.86:0/256311208 >> 172.21.15.86:0/256311208 conn(0x7ff06c0d00b0 msgr2=0x7ff06c0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:19.685 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff071000640 1 -- 172.21.15.86:0/256311208 shutdown_connections 2024-03-22T04:10:19.685 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.682+0000 7ff071000640 1 -- 172.21.15.86:0/256311208 wait complete. 2024-03-22T04:10:19.686 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff071000640 1 Processor -- start 2024-03-22T04:10:19.686 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff071000640 1 -- start start 2024-03-22T04:10:19.688 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff071000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff06c06eeb0 0x7ff06c1477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:19.688 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff071000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cc790 0x7ff06c15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:19.688 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff071000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:19.689 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff069e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff06c06eeb0 0x7ff06c1477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:19.689 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff069e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff06c06eeb0 0x7ff06c1477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52956/0 (socket says 172.21.15.86:52956) 2024-03-22T04:10:19.689 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff069e00640 1 -- 172.21.15.86:0/745144528 learned_addr learned my addr 172.21.15.86:0/745144528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:19.689 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c06fdf0 con 0x7ff06c06eeb0 2024-03-22T04:10:19.689 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c06fc70 con 0x7ff06c0cc790 2024-03-22T04:10:19.689 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff06c071f20 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.690 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.686+0000 7ff06a800640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:19.690 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff069400640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cc790 0x7ff06c15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff06a800640 1 -- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cc790 msgr2=0x7ff06c15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff06a800640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cc790 0x7ff06c15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff06a800640 1 -- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff06c06eeb0 msgr2=0x7ff06c1477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff06a800640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff06c06eeb0 0x7ff06c1477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff06a800640 1 -- 172.21.15.86:0/745144528 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c152be0 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff069e00640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff06c06eeb0 0x7ff06c1477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff06a800640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c153c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c700 tx=0x7ff06400cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff062000640 1 -- 172.21.15.86:0/745144528 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064017070 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff062000640 1 -- 172.21.15.86:0/745144528 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff064003dd0 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff062000640 1 -- 172.21.15.86:0/745144528 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640052b0 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.691 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c152ec0 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.692 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c153480 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.692 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.690+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06c0cf720 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.694 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.694+0000 7ff062000640 1 -- 172.21.15.86:0/745144528 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 4) ==== 87403+0+0 (secure 0 0 0) 0x7ff06401a070 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.694 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.694+0000 7ff062000640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff04c067ae0 0x7ff04c069fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:19.695 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.694+0000 7ff062000640 1 -- 172.21.15.86:0/745144528 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7ff064084620 con 0x7ff06c0d0bd0 2024-03-22T04:10:19.695 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.694+0000 7ff069e00640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff04c067ae0 0x7ff04c069fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:19.696 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.694+0000 7ff069e00640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff04c067ae0 0x7ff04c069fa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff05800bb10 tx=0x7ff0580a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:19.714 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:19.714+0000 7ff062000640 1 -- 172.21.15.86:0/745144528 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff064055820 con 0x7ff06c0d0bd0 2024-03-22T04:10:20.018 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.018+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff06c06f820 con 0x7ff06c0d0bd0 2024-03-22T04:10:20.019 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.018+0000 7ff062000640 1 -- 172.21.15.86:0/745144528 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7578 (secure 0 0 0) 0x7ff0640598c0 con 0x7ff06c0d0bd0 2024-03-22T04:10:20.020 INFO:teuthology.misc.health.smithi086.stdout: 2024-03-22T04:10:20.020 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":14,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:18.893634+0000","last_up_change":"2024-03-22T04:10:17.804144+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:20.023 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff04c067ae0 msgr2=0x7ff04c069fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:20.023 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff04c067ae0 0x7ff04c069fa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff05800bb10 tx=0x7ff0580a9070 comp rx=0 tx=0).stop 2024-03-22T04:10:20.023 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 msgr2=0x7ff06c153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:20.023 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c153c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c700 tx=0x7ff06400cbc0 comp rx=0 tx=0).stop 2024-03-22T04:10:20.023 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 shutdown_connections 2024-03-22T04:10:20.023 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff04c067ae0 0x7ff04c069fa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:20.023 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cc790 0x7ff06c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:20.023 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0d0bd0 0x7ff06c153c80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:20.024 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 --2- 172.21.15.86:0/745144528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff06c06eeb0 0x7ff06c1477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:20.024 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 >> 172.21.15.86:0/745144528 conn(0x7ff06c0d00b0 msgr2=0x7ff06c0cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:20.024 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 shutdown_connections 2024-03-22T04:10:20.024 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:20.022+0000 7ff071000640 1 -- 172.21.15.86:0/745144528 wait complete. 2024-03-22T04:10:20.055 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-03-22T04:10:20.230 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-03-22T04:10:20.230+0000 7efea9200640 -1 osd.0 0 waiting for initial osdmap 2024-03-22T04:10:20.241 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-03-22T04:10:20.238+0000 7fe7e6400640 -1 osd.1 0 waiting for initial osdmap 2024-03-22T04:10:20.263 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-03-22T04:10:20.262+0000 7efe9f200640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T04:10:20.282 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-03-22T04:10:20.282+0000 7fe7dba00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T04:10:20.294 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-03-22T04:10:20.294+0000 7fde7d000640 -1 osd.2 0 waiting for initial osdmap 2024-03-22T04:10:20.316 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-03-22T04:10:20.314+0000 7f4dcea00640 -1 osd.3 0 waiting for initial osdmap 2024-03-22T04:10:20.334 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-03-22T04:10:20.334+0000 7fde73000640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T04:10:20.354 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-03-22T04:10:20.354+0000 7f4dc4a00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T04:10:26.862 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:26.999 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe8600640 1 Processor -- start 2024-03-22T04:10:26.999 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe8600640 1 -- start start 2024-03-22T04:10:27.000 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe8600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafe0161e60 0x7fafe006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.000 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe8600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe006bba0 0x7fafe00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.000 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe8600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe00d1370 con 0x7fafe0161a80 2024-03-22T04:10:27.000 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe8600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fafe0073910 con 0x7fafe0161e60 2024-03-22T04:10:27.001 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe8600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fafe006bff0 con 0x7fafe006bba0 2024-03-22T04:10:27.001 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe5c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fafe0161a80 0x7fafe0073200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45296/0 (socket says 172.21.15.86:45296) 2024-03-22T04:10:27.001 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe5c00640 1 -- 172.21.15.86:0/2444871693 learned_addr learned my addr 172.21.15.86:0/2444871693 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe6600640 1 --2- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe006bba0 0x7fafe00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe5200640 1 --2- 172.21.15.86:0/2444871693 >> v2:172.21.15.196:3300/0 conn(0x7fafe0161e60 0x7fafe006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafdfe00640 1 -- 172.21.15.86:0/2444871693 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3702219066 0 0) 0x7fafe00d1370 con 0x7fafe0161a80 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafdfe00640 1 -- 172.21.15.86:0/2444871693 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc8003530 con 0x7fafe0161a80 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe5200640 1 -- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe006bba0 msgr2=0x7fafe00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe5200640 1 --2- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe006bba0 0x7fafe00cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe5200640 1 -- 172.21.15.86:0/2444871693 >> v1:172.21.15.86:6789/0 conn(0x7fafe0161a80 legacy=0x7fafe0073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe5200640 1 -- 172.21.15.86:0/2444871693 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe00d1650 con 0x7fafe0161e60 2024-03-22T04:10:27.002 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:26.997+0000 7fafe6600640 1 --2- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe006bba0 0x7fafe00cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:27.004 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe5200640 1 --2- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe0161e60 0x7fafe006b660 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0097f0 tx=0x7fafcc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3662b27439d9ee43 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.004 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafdfe00640 1 -- 172.21.15.86:0/2444871693 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0b0070 con 0x7fafe0161e60 2024-03-22T04:10:27.004 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafdfe00640 1 -- 172.21.15.86:0/2444871693 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafcc0ab600 con 0x7fafe0161e60 2024-03-22T04:10:27.005 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafdfe00640 1 -- 172.21.15.86:0/2444871693 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0ab9d0 con 0x7fafe0161e60 2024-03-22T04:10:27.005 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe8600640 1 -- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe0161e60 msgr2=0x7fafe006b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.005 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe8600640 1 --2- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe0161e60 0x7fafe006b660 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0097f0 tx=0x7fafcc01c410 comp rx=0 tx=0).stop 2024-03-22T04:10:27.005 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe8600640 1 -- 172.21.15.86:0/2444871693 shutdown_connections 2024-03-22T04:10:27.006 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe8600640 1 --2- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe006bba0 0x7fafe00cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.006 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe8600640 1 --2- 172.21.15.86:0/2444871693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe0161e60 0x7fafe006b660 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.006 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe8600640 1 -- 172.21.15.86:0/2444871693 >> 172.21.15.86:0/2444871693 conn(0x7fafe00678c0 msgr2=0x7fafe006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:27.006 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe8600640 1 -- 172.21.15.86:0/2444871693 shutdown_connections 2024-03-22T04:10:27.006 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.001+0000 7fafe8600640 1 -- 172.21.15.86:0/2444871693 wait complete. 2024-03-22T04:10:27.007 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 Processor -- start 2024-03-22T04:10:27.007 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 -- start start 2024-03-22T04:10:27.009 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe006bba0 0x7fafe0155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.009 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe0161a80 0x7fafe01469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.009 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafe0161e60 0x7fafe0160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.009 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafe00ceb30 con 0x7fafe0161e60 2024-03-22T04:10:27.009 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafe00ce9b0 con 0x7fafe006bba0 2024-03-22T04:10:27.009 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fafe00cecb0 con 0x7fafe0161a80 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafe0161e60 0x7fafe0160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafe0161e60 0x7fafe0160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40790/0 (socket says 172.21.15.86:40790) 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 -- 172.21.15.86:0/2727172787 learned_addr learned my addr 172.21.15.86:0/2727172787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe5200640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe0161a80 0x7fafe01469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe5c00640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe006bba0 0x7fafe0155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 -- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe0161a80 msgr2=0x7fafe01469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe0161a80 0x7fafe01469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 -- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe006bba0 msgr2=0x7fafe0155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe006bba0 0x7fafe0155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 -- 172.21.15.86:0/2727172787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe0153eb0 con 0x7fafe0161e60 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe5c00640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe006bba0 0x7fafe0155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe6600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafe0161e60 0x7fafe0160090 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009f00 tx=0x7fafd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafdd600640 1 -- 172.21.15.86:0/2727172787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8018030 con 0x7fafe0161e60 2024-03-22T04:10:27.010 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafdd600640 1 -- 172.21.15.86:0/2727172787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd800ce50 con 0x7fafe0161e60 2024-03-22T04:10:27.011 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafdd600640 1 -- 172.21.15.86:0/2727172787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8003bc0 con 0x7fafe0161e60 2024-03-22T04:10:27.011 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.005+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe01551a0 con 0x7fafe0161e60 2024-03-22T04:10:27.011 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.009+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe0146f30 con 0x7fafe0161e60 2024-03-22T04:10:27.011 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.009+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe00cf190 con 0x7fafe0161e60 2024-03-22T04:10:27.011 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.009+0000 7fafdd600640 1 -- 172.21.15.86:0/2727172787 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fafd801b070 con 0x7fafe0161e60 2024-03-22T04:10:27.012 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.009+0000 7fafdd600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafc0067b20 0x7fafc0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.012 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.009+0000 7fafdd600640 1 -- 172.21.15.86:0/2727172787 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fafd8086620 con 0x7fafe0161e60 2024-03-22T04:10:27.012 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.009+0000 7fafe5c00640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafc0067b20 0x7fafc0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.012 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.009+0000 7fafe5c00640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafc0067b20 0x7fafc0069fe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafd4005890 tx=0x7fafd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.017 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.013+0000 7fafdd600640 1 -- 172.21.15.86:0/2727172787 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fafd8022070 con 0x7fafe0161e60 2024-03-22T04:10:27.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:10:27.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:10:27.175 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.173+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafe0154190 con 0x7fafe0161e60 2024-03-22T04:10:27.176 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.173+0000 7fafdd600640 1 -- 172.21.15.86:0/2727172787 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7fafe0154190 con 0x7fafe0161e60 2024-03-22T04:10:27.176 INFO:teuthology.misc.health.smithi086.stdout: 2024-03-22T04:10:27.176 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":19,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:24.658944+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafc0067b20 msgr2=0x7fafc0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafc0067b20 0x7fafc0069fe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafd4005890 tx=0x7fafd4009040 comp rx=0 tx=0).stop 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafe0161e60 msgr2=0x7fafe0160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafe0161e60 0x7fafe0160090 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009f00 tx=0x7fafd800c490 comp rx=0 tx=0).stop 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 shutdown_connections 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafc0067b20 0x7fafc0069fe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafe0161e60 0x7fafe0160090 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafe0161a80 0x7fafe01469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.178 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 --2- 172.21.15.86:0/2727172787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafe006bba0 0x7fafe0155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.179 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 >> 172.21.15.86:0/2727172787 conn(0x7fafe00678c0 msgr2=0x7fafe00b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:27.179 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 shutdown_connections 2024-03-22T04:10:27.179 INFO:teuthology.misc.health.smithi086.stderr:2024-03-22T04:10:27.177+0000 7fafe8600640 1 -- 172.21.15.86:0/2727172787 wait complete. 2024-03-22T04:10:27.195 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-22T04:10:27.195 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-22T04:10:27.195 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-22T04:10:27.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-22T04:10:27.198 INFO:tasks.ceph.mds.a:Started 2024-03-22T04:10:27.199 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-22T04:10:27.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-22T04:10:27.202 INFO:tasks.ceph.mds.c:Started 2024-03-22T04:10:27.202 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-22T04:10:27.202 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-22T04:10:27.205 INFO:tasks.ceph.mds.b:Started 2024-03-22T04:10:27.205 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-22T04:10:27.205 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-22T04:10:27.208 INFO:tasks.ceph.mds.d:Started 2024-03-22T04:10:27.209 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-22T04:10:27.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:27.293 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-03-22T04:10:27.293 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:10:27.308 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:10:27.308 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:10:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1a00640 1 Processor -- start 2024-03-22T04:10:27.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1a00640 1 -- start start 2024-03-22T04:10:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec161cd0 0x7f67ec0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67ec09b180 0x7f67ec0d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67ec0b33b0 con 0x7f67ec0b4a90 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec0d1cc0 con 0x7f67ec09b180 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec0d0dd0 con 0x7f67ec161cd0 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67ec09b180 0x7f67ec0d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67ec09b180 0x7f67ec0d12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40844/0 (socket says 172.21.15.86:40844) 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1000640 1 -- 172.21.15.86:0/1586794835 learned_addr learned my addr 172.21.15.86:0/1586794835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67ebe00640 1 --2- 172.21.15.86:0/1586794835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec161cd0 0x7f67ec0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1000640 1 -- 172.21.15.86:0/1586794835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec161cd0 msgr2=0x7f67ec0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1000640 1 --2- 172.21.15.86:0/1586794835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec161cd0 0x7f67ec0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1000640 1 -- 172.21.15.86:0/1586794835 >> v1:172.21.15.196:6789/0 conn(0x7f67ec0b4a90 legacy=0x7f67ec09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.353+0000 7f67f1000640 1 -- 172.21.15.86:0/1586794835 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec0d1fa0 con 0x7f67ec09b180 2024-03-22T04:10:27.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1000640 1 --2- 172.21.15.86:0/1586794835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec0d12f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009870 tx=0x7f67e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cb1c3d942c035e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67eb400640 1 -- 172.21.15.86:0/1586794835 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40a7070 con 0x7f67ec09b180 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67eb400640 1 -- 172.21.15.86:0/1586794835 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e40b0070 con 0x7f67ec09b180 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67eb400640 1 -- 172.21.15.86:0/1586794835 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40ab890 con 0x7f67ec09b180 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1a00640 1 -- 172.21.15.86:0/1586794835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 msgr2=0x7f67ec0d12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1a00640 1 --2- 172.21.15.86:0/1586794835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec0d12f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009870 tx=0x7f67e401c560 comp rx=0 tx=0).stop 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1a00640 1 -- 172.21.15.86:0/1586794835 shutdown_connections 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1a00640 1 --2- 172.21.15.86:0/1586794835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec0d12f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1a00640 1 --2- 172.21.15.86:0/1586794835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec161cd0 0x7f67ec0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1a00640 1 -- 172.21.15.86:0/1586794835 >> 172.21.15.86:0/1586794835 conn(0x7f67ec0b3d60 msgr2=0x7f67ec0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:27.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1a00640 1 -- 172.21.15.86:0/1586794835 shutdown_connections 2024-03-22T04:10:27.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.357+0000 7f67f1a00640 1 -- 172.21.15.86:0/1586794835 wait complete. 2024-03-22T04:10:27.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 Processor -- start 2024-03-22T04:10:27.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 -- start start 2024-03-22T04:10:27.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec0b4a90 0x7f67ec070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ec161cd0 0x7f67ec158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec0b2770 con 0x7f67ec09b180 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec0b25f0 con 0x7f67ec161cd0 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec16b260 con 0x7f67ec0b4a90 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40850/0 (socket says 172.21.15.86:40850) 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 -- 172.21.15.86:0/2675616547 learned_addr learned my addr 172.21.15.86:0/2675616547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67e3400640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec0b4a90 0x7f67ec070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1000640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ec161cd0 0x7f67ec158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 -- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec0b4a90 msgr2=0x7f67ec070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec0b4a90 0x7f67ec070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 -- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ec161cd0 msgr2=0x7f67ec158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ec161cd0 0x7f67ec158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 -- 172.21.15.86:0/2675616547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec15acc0 con 0x7f67ec09b180 2024-03-22T04:10:27.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67e3400640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec0b4a90 0x7f67ec070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:27.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1000640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ec161cd0 0x7f67ec158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:27.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67ebe00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec070800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009d10 tx=0x7f67dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67e8c00640 1 -- 172.21.15.86:0/2675616547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc018020 con 0x7f67ec09b180 2024-03-22T04:10:27.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67e8c00640 1 -- 172.21.15.86:0/2675616547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67dc014480 con 0x7f67ec09b180 2024-03-22T04:10:27.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67e8c00640 1 -- 172.21.15.86:0/2675616547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc014830 con 0x7f67ec09b180 2024-03-22T04:10:27.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec15afa0 con 0x7f67ec09b180 2024-03-22T04:10:27.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec15b8e0 con 0x7f67ec09b180 2024-03-22T04:10:27.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.361+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ec074470 con 0x7f67ec09b180 2024-03-22T04:10:27.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.365+0000 7f67e8c00640 1 -- 172.21.15.86:0/2675616547 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f67dc01b020 con 0x7f67ec09b180 2024-03-22T04:10:27.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.365+0000 7f67e8c00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f67c8067b20 0x7f67c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.365+0000 7f67e8c00640 1 -- 172.21.15.86:0/2675616547 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f67dc086710 con 0x7f67ec09b180 2024-03-22T04:10:27.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.365+0000 7f67e3400640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f67c8067b20 0x7f67c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.365+0000 7f67e3400640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f67c8067b20 0x7f67c8069fe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f67ec09fc00 tx=0x7f67d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.369+0000 7f67e8c00640 1 -- 172.21.15.86:0/2675616547 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f67dc022070 con 0x7f67ec09b180 2024-03-22T04:10:27.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.525+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67ec0cf770 con 0x7f67ec09b180 2024-03-22T04:10:27.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.525+0000 7f67e8c00640 1 -- 172.21.15.86:0/2675616547 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7f67dc010040 con 0x7f67ec09b180 2024-03-22T04:10:27.528 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:27.528 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":19,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:24.658944+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:27.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f67c8067b20 msgr2=0x7f67c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f67c8067b20 0x7f67c8069fe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f67ec09fc00 tx=0x7f67d4009040 comp rx=0 tx=0).stop 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 msgr2=0x7f67ec070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec070800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009d10 tx=0x7f67dc00c490 comp rx=0 tx=0).stop 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 shutdown_connections 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f67c8067b20 0x7f67c8069fe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ec161cd0 0x7f67ec158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ec0b4a90 0x7f67ec070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 --2- 172.21.15.86:0/2675616547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67ec09b180 0x7f67ec070800 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 >> 172.21.15.86:0/2675616547 conn(0x7f67ec0b3d60 msgr2=0x7f67ec09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 shutdown_connections 2024-03-22T04:10:27.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.529+0000 7f67f1a00640 1 -- 172.21.15.86:0/2675616547 wait complete. 2024-03-22T04:10:27.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:10:27.549 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:10:27.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe663800640 1 Processor -- start 2024-03-22T04:10:27.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe663800640 1 -- start start 2024-03-22T04:10:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe663800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe663800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe65c0b1f70 con 0x7fe65c0b1670 2024-03-22T04:10:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe663800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe65c15fba0 con 0x7fe65c15b200 2024-03-22T04:10:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe663800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe65c0b2fc0 con 0x7fe65c0b1b00 2024-03-22T04:10:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe65be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe65c0b1670 0x7fe65c0ccc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45318/0 (socket says 172.21.15.86:45318) 2024-03-22T04:10:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe65be00640 1 -- 172.21.15.86:0/2262306563 learned_addr learned my addr 172.21.15.86:0/2262306563 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe661800640 1 --2- 172.21.15.86:0/2262306563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe65b400640 1 -- 172.21.15.86:0/2262306563 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880602873 0 0) 0x7fe65c0b1f70 con 0x7fe65c0b1670 2024-03-22T04:10:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe65b400640 1 -- 172.21.15.86:0/2262306563 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe644005740 con 0x7fe65c0b1670 2024-03-22T04:10:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe65b400640 1 -- 172.21.15.86:0/2262306563 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387890682 0 0) 0x7fe65c15fba0 con 0x7fe65c15b200 2024-03-22T04:10:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe65b400640 1 -- 172.21.15.86:0/2262306563 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe65c0b1f70 con 0x7fe65c15b200 2024-03-22T04:10:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe661800640 1 -- 172.21.15.86:0/2262306563 >> v1:172.21.15.196:6789/0 conn(0x7fe65c15b200 legacy=0x7fe65c0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe661800640 1 -- 172.21.15.86:0/2262306563 >> v1:172.21.15.86:6789/0 conn(0x7fe65c0b1670 legacy=0x7fe65c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.685+0000 7fe661800640 1 -- 172.21.15.86:0/2262306563 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c15ede0 con 0x7fe65c0b1b00 2024-03-22T04:10:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe661800640 1 --2- 172.21.15.86:0/2262306563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c0cd310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0b3140 tx=0x7fe65401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d30331b4215ee2f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe65b400640 1 -- 172.21.15.86:0/2262306563 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540a66d0 con 0x7fe65c0b1b00 2024-03-22T04:10:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe65b400640 1 -- 172.21.15.86:0/2262306563 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6540a6870 con 0x7fe65c0b1b00 2024-03-22T04:10:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe65b400640 1 -- 172.21.15.86:0/2262306563 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540a6c40 con 0x7fe65c0b1b00 2024-03-22T04:10:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe663800640 1 -- 172.21.15.86:0/2262306563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 msgr2=0x7fe65c0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe663800640 1 --2- 172.21.15.86:0/2262306563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c0cd310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0b3140 tx=0x7fe65401c900 comp rx=0 tx=0).stop 2024-03-22T04:10:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe663800640 1 -- 172.21.15.86:0/2262306563 shutdown_connections 2024-03-22T04:10:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe663800640 1 --2- 172.21.15.86:0/2262306563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c0cd310 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe663800640 1 -- 172.21.15.86:0/2262306563 >> 172.21.15.86:0/2262306563 conn(0x7fe65c067370 msgr2=0x7fe65c067780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe663800640 1 -- 172.21.15.86:0/2262306563 shutdown_connections 2024-03-22T04:10:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.689+0000 7fe663800640 1 -- 172.21.15.86:0/2262306563 wait complete. 2024-03-22T04:10:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 Processor -- start 2024-03-22T04:10:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 -- start start 2024-03-22T04:10:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65c0b1670 0x7fe65c143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c15b200 0x7fe65c150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c0b2bd0 con 0x7fe65c0b1670 2024-03-22T04:10:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c0cee80 con 0x7fe65c15b200 2024-03-22T04:10:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe65c0b2d50 con 0x7fe65c0b1b00 2024-03-22T04:10:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65c0b1670 0x7fe65c143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65c0b1670 0x7fe65c143680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40856/0 (socket says 172.21.15.86:40856) 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 -- 172.21.15.86:0/1849152495 learned_addr learned my addr 172.21.15.86:0/1849152495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe65be00640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe661800640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c15b200 0x7fe65c150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 -- 172.21.15.86:0/1849152495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 msgr2=0x7fe65c140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 -- 172.21.15.86:0/1849152495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c15b200 msgr2=0x7fe65c150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c15b200 0x7fe65c150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 -- 172.21.15.86:0/1849152495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c1413b0 con 0x7fe65c0b1670 2024-03-22T04:10:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe660e00640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65c0b1670 0x7fe65c143680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe65000c430 tx=0x7fe65000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe658c00640 1 -- 172.21.15.86:0/1849152495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650017070 con 0x7fe65c0b1670 2024-03-22T04:10:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe658c00640 1 -- 172.21.15.86:0/1849152495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6500052f0 con 0x7fe65c0b1670 2024-03-22T04:10:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c150430 con 0x7fe65c0b1670 2024-03-22T04:10:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe658c00640 1 -- 172.21.15.86:0/1849152495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650003840 con 0x7fe65c0b1670 2024-03-22T04:10:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.693+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c15e120 con 0x7fe65c0b1670 2024-03-22T04:10:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.697+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65c074470 con 0x7fe65c0b1670 2024-03-22T04:10:27.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.697+0000 7fe658c00640 1 -- 172.21.15.86:0/1849152495 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe65001a070 con 0x7fe65c0b1670 2024-03-22T04:10:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.697+0000 7fe658c00640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe63c067b20 0x7fe63c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.697+0000 7fe658c00640 1 -- 172.21.15.86:0/1849152495 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fe650084eb0 con 0x7fe65c0b1670 2024-03-22T04:10:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.697+0000 7fe65be00640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe63c067b20 0x7fe63c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.697+0000 7fe65be00640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe63c067b20 0x7fe63c069fe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe65c148e90 tx=0x7fe648007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.701+0000 7fe658c00640 1 -- 172.21.15.86:0/1849152495 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe65c1413b0 con 0x7fe65c0b1670 2024-03-22T04:10:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.857+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe65c069870 con 0x7fe65c0b1670 2024-03-22T04:10:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.857+0000 7fe658c00640 1 -- 172.21.15.86:0/1849152495 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7fe6500555b0 con 0x7fe65c0b1670 2024-03-22T04:10:27.861 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:10:27.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe63c067b20 msgr2=0x7fe63c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe63c067b20 0x7fe63c069fe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe65c148e90 tx=0x7fe648007d40 comp rx=0 tx=0).stop 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65c0b1670 msgr2=0x7fe65c143680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65c0b1670 0x7fe65c143680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe65000c430 tx=0x7fe65000c8f0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 shutdown_connections 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe63c067b20 0x7fe63c069fe0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65c15b200 0x7fe65c150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65c0b1b00 0x7fe65c140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 --2- 172.21.15.86:0/1849152495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65c0b1670 0x7fe65c143680 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 >> 172.21.15.86:0/1849152495 conn(0x7fe65c067370 msgr2=0x7fe65c06dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 shutdown_connections 2024-03-22T04:10:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:27.861+0000 7fe663800640 1 -- 172.21.15.86:0/1849152495 wait complete. 2024-03-22T04:10:27.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:10:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31ca200640 1 Processor -- start 2024-03-22T04:10:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31ca200640 1 -- start start 2024-03-22T04:10:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31ca200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f31c406df90 0x7f31c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31ca200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c415ac10 0x7f31c40b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31ca200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c415c550 con 0x7f31c415bc80 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31ca200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f31c415cb60 con 0x7f31c406df90 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31ca200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f31c406e8b0 con 0x7f31c415ac10 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31c3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f31c415bc80 0x7f31c415e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45320/0 (socket says 172.21.15.86:45320) 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31c3e00640 1 -- 172.21.15.86:0/2026079157 learned_addr learned my addr 172.21.15.86:0/2026079157 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31c2a00640 1 --2- 172.21.15.86:0/2026079157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c415ac10 0x7f31c40b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.013+0000 7f31c3400640 1 --2- 172.21.15.86:0/2026079157 >> v2:172.21.15.196:3300/0 conn(0x7f31c406df90 0x7f31c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2000640 1 -- 172.21.15.86:0/2026079157 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278670481 0 0) 0x7f31c415c550 con 0x7f31c415bc80 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2000640 1 -- 172.21.15.86:0/2026079157 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31a8003530 con 0x7f31c415bc80 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2a00640 1 -- 172.21.15.86:0/2026079157 >> v2:172.21.15.196:3300/0 conn(0x7f31c406df90 msgr2=0x7f31c406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2a00640 1 --2- 172.21.15.86:0/2026079157 >> v2:172.21.15.196:3300/0 conn(0x7f31c406df90 0x7f31c406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2a00640 1 -- 172.21.15.86:0/2026079157 >> v1:172.21.15.86:6789/0 conn(0x7f31c415bc80 legacy=0x7f31c415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2a00640 1 -- 172.21.15.86:0/2026079157 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c40b4750 con 0x7f31c415ac10 2024-03-22T04:10:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2a00640 1 --2- 172.21.15.86:0/2026079157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c415ac10 0x7f31c40b3c00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f31b00097c0 tx=0x7f31b001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd42e8e4729c996 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2000640 1 -- 172.21.15.86:0/2026079157 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00ab5f0 con 0x7f31c415ac10 2024-03-22T04:10:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2000640 1 -- 172.21.15.86:0/2026079157 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b00b0070 con 0x7f31c415ac10 2024-03-22T04:10:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.017+0000 7f31c2000640 1 -- 172.21.15.86:0/2026079157 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00ab9b0 con 0x7f31c415ac10 2024-03-22T04:10:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- 172.21.15.86:0/2026079157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c415ac10 msgr2=0x7f31c40b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 --2- 172.21.15.86:0/2026079157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c415ac10 0x7f31c40b3c00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f31b00097c0 tx=0x7f31b001c470 comp rx=0 tx=0).stop 2024-03-22T04:10:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- 172.21.15.86:0/2026079157 shutdown_connections 2024-03-22T04:10:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 --2- 172.21.15.86:0/2026079157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c415ac10 0x7f31c40b3c00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 --2- 172.21.15.86:0/2026079157 >> v2:172.21.15.196:3300/0 conn(0x7f31c406df90 0x7f31c406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- 172.21.15.86:0/2026079157 >> 172.21.15.86:0/2026079157 conn(0x7f31c40b1df0 msgr2=0x7f31c40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- 172.21.15.86:0/2026079157 shutdown_connections 2024-03-22T04:10:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- 172.21.15.86:0/2026079157 wait complete. 2024-03-22T04:10:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 Processor -- start 2024-03-22T04:10:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- start start 2024-03-22T04:10:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c406df90 0x7f31c4159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31c415ac10 0x7f31c414a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c415bc80 0x7f31c40ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31c415ee60 con 0x7f31c415bc80 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31c415ece0 con 0x7f31c415ac10 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31ca200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f31c415efe0 con 0x7f31c406df90 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c415bc80 0x7f31c40ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c415bc80 0x7f31c40ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40870/0 (socket says 172.21.15.86:40870) 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3e00640 1 -- 172.21.15.86:0/944033181 learned_addr learned my addr 172.21.15.86:0/944033181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c2a00640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31c415ac10 0x7f31c414a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3400640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c406df90 0x7f31c4159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3e00640 1 -- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c406df90 msgr2=0x7f31c4159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3e00640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c406df90 0x7f31c4159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3e00640 1 -- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31c415ac10 msgr2=0x7f31c414a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3e00640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31c415ac10 0x7f31c414a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3e00640 1 -- 172.21.15.86:0/944033181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c40b5270 con 0x7f31c415bc80 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.021+0000 7f31c3400640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c406df90 0x7f31c4159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31c3e00640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c415bc80 0x7f31c40ad4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00cad0 tx=0x7f31bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31bb400640 1 -- 172.21.15.86:0/944033181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc017070 con 0x7f31c415bc80 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31bb400640 1 -- 172.21.15.86:0/944033181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31bc00ac30 con 0x7f31c415bc80 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31bb400640 1 -- 172.21.15.86:0/944033181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc012400 con 0x7f31c415bc80 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c414e9e0 con 0x7f31c415bc80 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c40ad9f0 con 0x7f31c415bc80 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c4068650 con 0x7f31c415bc80 2024-03-22T04:10:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31bb400640 1 -- 172.21.15.86:0/944033181 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f31bc01a070 con 0x7f31c415bc80 2024-03-22T04:10:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31bb400640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31a4067b20 0x7f31a4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31bb400640 1 -- 172.21.15.86:0/944033181 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f31bc085910 con 0x7f31c415bc80 2024-03-22T04:10:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31c3400640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31a4067b20 0x7f31a4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.025+0000 7f31c3400640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31a4067b20 0x7f31a4069fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31b4007a70 tx=0x7f31b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.029+0000 7f31bb400640 1 -- 172.21.15.86:0/944033181 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f31bc055f80 con 0x7f31c415bc80 2024-03-22T04:10:28.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.217+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31c406ed60 con 0x7f31c415bc80 2024-03-22T04:10:28.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.217+0000 7f31bb400640 1 -- 172.21.15.86:0/944033181 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f31bc05c070 con 0x7f31c415bc80 2024-03-22T04:10:28.221 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:28.221 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-22T04:10:28.221 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-03-22T04:10:28.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31a4067b20 msgr2=0x7f31a4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31a4067b20 0x7f31a4069fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31b4007a70 tx=0x7f31b4008040 comp rx=0 tx=0).stop 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c415bc80 msgr2=0x7f31c40ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c415bc80 0x7f31c40ad4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00cad0 tx=0x7f31bc00cf90 comp rx=0 tx=0).stop 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 shutdown_connections 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31a4067b20 0x7f31a4069fe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31c415ac10 0x7f31c414a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c415bc80 0x7f31c40ad4b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 --2- 172.21.15.86:0/944033181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31c406df90 0x7f31c4159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 >> 172.21.15.86:0/944033181 conn(0x7f31c40b1df0 msgr2=0x7f31c4153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 shutdown_connections 2024-03-22T04:10:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:28.221+0000 7f31ca200640 1 -- 172.21.15.86:0/944033181 wait complete. 2024-03-22T04:10:30.242 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:10:30.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.385+0000 7f02e5400640 1 Processor -- start 2024-03-22T04:10:30.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.385+0000 7f02e5400640 1 -- start start 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.385+0000 7f02e5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e015cd00 0x7f02e01607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.385+0000 7f02e5400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e015d290 con 0x7f02e013f1b0 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e5400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e015d570 con 0x7f02e00700e0 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e5400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f02e0160320 con 0x7f02e015cd00 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02dfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f02e013f1b0 0x7f02e006f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45322/0 (socket says 172.21.15.86:45322) 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02dfe00640 1 -- 172.21.15.86:0/2133978520 learned_addr learned my addr 172.21.15.86:0/2133978520 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e4a00640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e015cd00 0x7f02e01607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352576475 0 0) 0x7f02e015d290 con 0x7f02e013f1b0 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02c4003530 con 0x7f02e013f1b0 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147922976 0 0) 0x7f02e015d570 con 0x7f02e00700e0 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02e015d290 con 0x7f02e00700e0 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1015002106 0 0) 0x7f02c4003530 con 0x7f02e013f1b0 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e015cd00 msgr2=0x7f02e01607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e015cd00 0x7f02e01607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 >> v1:172.21.15.196:6789/0 conn(0x7f02e00700e0 legacy=0x7f02e015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e015fcd0 con 0x7f02e013f1b0 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f02d0001680 con 0x7f02e013f1b0 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 >> v1:172.21.15.86:6789/0 conn(0x7f02e013f1b0 legacy=0x7f02e006f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c4003c60 0x7f02c4004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e015cd00 0x7f02c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4004d50 0x7f02c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02e0160320 con 0x7f02c4004d50 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02c400bbb0 con 0x7f02c4003c60 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f02c400bdb0 con 0x7f02e015cd00 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e4a00640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4004d50 0x7f02c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02dfe00640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c4003c60 0x7f02c4004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02d7400640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e015cd00 0x7f02c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e4a00640 1 -- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e015cd00 msgr2=0x7f02c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e4a00640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e015cd00 0x7f02c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e4a00640 1 -- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c4003c60 msgr2=0x7f02c4004060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e4a00640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c4003c60 0x7f02c4004060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02e4a00640 1 -- 172.21.15.86:0/2133978520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c4003530 con 0x7f02c4004d50 2024-03-22T04:10:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.389+0000 7f02dfe00640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02c4003c60 0x7f02c4004060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e4a00640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4004d50 0x7f02c40051d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009ae0 tx=0x7f02d801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8e826317ee1f6d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d80b2070 con 0x7f02c4004d50 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d80a5660 con 0x7f02c4004d50 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02df400640 1 -- 172.21.15.86:0/2133978520 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d80096b0 con 0x7f02c4004d50 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e5400640 1 -- 172.21.15.86:0/2133978520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4004d50 msgr2=0x7f02c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e5400640 1 --2- 172.21.15.86:0/2133978520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4004d50 0x7f02c40051d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009ae0 tx=0x7f02d801ca70 comp rx=0 tx=0).stop 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e4a00640 1 -- 172.21.15.86:0/2133978520 reap_dead start 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e5400640 1 -- 172.21.15.86:0/2133978520 shutdown_connections 2024-03-22T04:10:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e5400640 1 -- 172.21.15.86:0/2133978520 >> 172.21.15.86:0/2133978520 conn(0x7f02e006bc60 msgr2=0x7f02e00b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e5400640 1 -- 172.21.15.86:0/2133978520 shutdown_connections 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e5400640 1 -- 172.21.15.86:0/2133978520 wait complete. 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e5400640 1 Processor -- start 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.393+0000 7f02e5400640 1 -- start start 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02c4004d50 0x7f02e015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e013d0e0 0x7f02e013d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e013ac30 0x7f02e013b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01607b0 con 0x7f02e013d0e0 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02e00b49d0 con 0x7f02e013ac30 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f02e0160930 con 0x7f02c4004d50 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e013d0e0 0x7f02e013d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e013d0e0 0x7f02e013d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40892/0 (socket says 172.21.15.86:40892) 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 -- 172.21.15.86:0/1777687670 learned_addr learned my addr 172.21.15.86:0/1777687670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02dfe00640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02c4004d50 0x7f02e015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e4a00640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e013ac30 0x7f02e013b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 -- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02c4004d50 msgr2=0x7f02e015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02c4004d50 0x7f02e015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 -- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e013ac30 msgr2=0x7f02e013b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e013ac30 0x7f02e013b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 -- 172.21.15.86:0/1777687670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e015a670 con 0x7f02e013d0e0 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02dfe00640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02c4004d50 0x7f02e015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e4a00640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e013ac30 0x7f02e013b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02d7400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e013d0e0 0x7f02e013d560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02c8007bb0 tx=0x7f02c800f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02dcc00640 1 -- 172.21.15.86:0/1777687670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8010040 con 0x7f02e013d0e0 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02dcc00640 1 -- 172.21.15.86:0/1777687670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c801a070 con 0x7f02e013d0e0 2024-03-22T04:10:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e015a910 con 0x7f02e013d0e0 2024-03-22T04:10:30.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02dcc00640 1 -- 172.21.15.86:0/1777687670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8015850 con 0x7f02e013d0e0 2024-03-22T04:10:30.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e015aed0 con 0x7f02e013d0e0 2024-03-22T04:10:30.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e0074470 con 0x7f02e013d0e0 2024-03-22T04:10:30.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.397+0000 7f02dcc00640 1 -- 172.21.15.86:0/1777687670 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f02c80159f0 con 0x7f02e013d0e0 2024-03-22T04:10:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.401+0000 7f02dcc00640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02bc067b70 0x7f02bc06a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.401+0000 7f02dcc00640 1 -- 172.21.15.86:0/1777687670 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f02c8089380 con 0x7f02e013d0e0 2024-03-22T04:10:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.401+0000 7f02dfe00640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02bc067b70 0x7f02bc06a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.401+0000 7f02dfe00640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02bc067b70 0x7f02bc06a030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f02d0001020 tx=0x7f02d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:30.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.405+0000 7f02dcc00640 1 -- 172.21.15.86:0/1777687670 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f02c805a0c0 con 0x7f02e013d0e0 2024-03-22T04:10:30.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02e00b50f0 con 0x7f02e013d0e0 2024-03-22T04:10:30.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02dcc00640 1 -- 172.21.15.86:0/1777687670 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3862 (secure 0 0 0) 0x7f02c80599c0 con 0x7f02e013d0e0 2024-03-22T04:10:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:30.596 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":74171,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-22T04:10:30.596 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02bc067b70 msgr2=0x7f02bc06a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02bc067b70 0x7f02bc06a030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f02d0001020 tx=0x7f02d0009040 comp rx=0 tx=0).stop 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e013d0e0 msgr2=0x7f02e013d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e013d0e0 0x7f02e013d560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02c8007bb0 tx=0x7f02c800f540 comp rx=0 tx=0).stop 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 shutdown_connections 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02bc067b70 0x7f02bc06a030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e013ac30 0x7f02e013b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e013d0e0 0x7f02e013d560 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 --2- 172.21.15.86:0/1777687670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02c4004d50 0x7f02e015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.593+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 >> 172.21.15.86:0/1777687670 conn(0x7f02e006bc60 msgr2=0x7f02e00b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.597+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 shutdown_connections 2024-03-22T04:10:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.597+0000 7f02e5400640 1 -- 172.21.15.86:0/1777687670 wait complete. 2024-03-22T04:10:30.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:30.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8364200640 1 Processor -- start 2024-03-22T04:10:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8364200640 1 -- start start 2024-03-22T04:10:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8364200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f835c147880 0x7f835c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8364200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f835c0b4a60 0x7f835c15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8364200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c15ea90 con 0x7f835c06cec0 2024-03-22T04:10:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8364200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0b6320 con 0x7f835c0b4a60 2024-03-22T04:10:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8364200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f835c06d4f0 con 0x7f835c147880 2024-03-22T04:10:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8362200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f835c0b4a60 0x7f835c15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8362200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f835c0b4a60 0x7f835c15ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40898/0 (socket says 172.21.15.86:40898) 2024-03-22T04:10:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8362200640 1 -- 172.21.15.86:0/3764120845 learned_addr learned my addr 172.21.15.86:0/3764120845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.749+0000 7f8361800640 1 --2- 172.21.15.86:0/3764120845 >> v2:172.21.15.196:3300/0 conn(0x7f835c147880 0x7f835c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8362200640 1 -- 172.21.15.86:0/3764120845 >> v2:172.21.15.196:3300/0 conn(0x7f835c147880 msgr2=0x7f835c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8362200640 1 --2- 172.21.15.86:0/3764120845 >> v2:172.21.15.196:3300/0 conn(0x7f835c147880 0x7f835c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8362200640 1 -- 172.21.15.86:0/3764120845 >> v1:172.21.15.196:6789/0 conn(0x7f835c06cec0 legacy=0x7f835c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8362200640 1 -- 172.21.15.86:0/3764120845 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c06d7d0 con 0x7f835c0b4a60 2024-03-22T04:10:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8362200640 1 --2- 172.21.15.86:0/3764120845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c0b4a60 0x7f835c15ef40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8354009870 tx=0x7f835401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ec3c386f1dbcba server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f835be00640 1 -- 172.21.15.86:0/3764120845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540b0070 con 0x7f835c0b4a60 2024-03-22T04:10:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f835be00640 1 -- 172.21.15.86:0/3764120845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83540ab600 con 0x7f835c0b4a60 2024-03-22T04:10:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f835be00640 1 -- 172.21.15.86:0/3764120845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540ab9d0 con 0x7f835c0b4a60 2024-03-22T04:10:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8364200640 1 -- 172.21.15.86:0/3764120845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c0b4a60 msgr2=0x7f835c15ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8364200640 1 --2- 172.21.15.86:0/3764120845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c0b4a60 0x7f835c15ef40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8354009870 tx=0x7f835401c520 comp rx=0 tx=0).stop 2024-03-22T04:10:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8364200640 1 -- 172.21.15.86:0/3764120845 shutdown_connections 2024-03-22T04:10:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8364200640 1 --2- 172.21.15.86:0/3764120845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c0b4a60 0x7f835c15ef40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8364200640 1 --2- 172.21.15.86:0/3764120845 >> v2:172.21.15.196:3300/0 conn(0x7f835c147880 0x7f835c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8364200640 1 -- 172.21.15.86:0/3764120845 >> 172.21.15.86:0/3764120845 conn(0x7f835c06ba60 msgr2=0x7f835c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8364200640 1 -- 172.21.15.86:0/3764120845 shutdown_connections 2024-03-22T04:10:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.753+0000 7f8364200640 1 -- 172.21.15.86:0/3764120845 wait complete. 2024-03-22T04:10:30.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 Processor -- start 2024-03-22T04:10:30.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 -- start start 2024-03-22T04:10:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c06cec0 0x7f835c1679f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835c0b4a60 0x7f835c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c147880 0x7f835c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f835c15fe10 con 0x7f835c06cec0 2024-03-22T04:10:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f835c15fc90 con 0x7f835c147880 2024-03-22T04:10:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f835c15ff90 con 0x7f835c0b4a60 2024-03-22T04:10:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c06cec0 0x7f835c1679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c06cec0 0x7f835c1679f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40906/0 (socket says 172.21.15.86:40906) 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 -- 172.21.15.86:0/1289456042 learned_addr learned my addr 172.21.15.86:0/1289456042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8360e00640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835c0b4a60 0x7f835c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8362200640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c147880 0x7f835c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 -- 172.21.15.86:0/1289456042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835c0b4a60 msgr2=0x7f835c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835c0b4a60 0x7f835c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 -- 172.21.15.86:0/1289456042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c147880 msgr2=0x7f835c0cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c147880 0x7f835c0cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 -- 172.21.15.86:0/1289456042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c141750 con 0x7f835c06cec0 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8361800640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c06cec0 0x7f835c1679f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f835000efc0 tx=0x7f835000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8359600640 1 -- 172.21.15.86:0/1289456042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350018020 con 0x7f835c06cec0 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8359600640 1 -- 172.21.15.86:0/1289456042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8350014480 con 0x7f835c06cec0 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8359600640 1 -- 172.21.15.86:0/1289456042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350014850 con 0x7f835c06cec0 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c141a30 con 0x7f835c06cec0 2024-03-22T04:10:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.757+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c141ff0 con 0x7f835c06cec0 2024-03-22T04:10:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.761+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c068650 con 0x7f835c06cec0 2024-03-22T04:10:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.761+0000 7f8359600640 1 -- 172.21.15.86:0/1289456042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f835001b020 con 0x7f835c06cec0 2024-03-22T04:10:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.761+0000 7f8359600640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f833c067b20 0x7f833c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.761+0000 7f8359600640 1 -- 172.21.15.86:0/1289456042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f8350086a70 con 0x7f835c06cec0 2024-03-22T04:10:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.761+0000 7f8360e00640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f833c067b20 0x7f833c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.761+0000 7f8360e00640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f833c067b20 0x7f833c069fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f835c140580 tx=0x7f8348009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.765+0000 7f8359600640 1 -- 172.21.15.86:0/1289456042 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8350022030 con 0x7f835c06cec0 2024-03-22T04:10:30.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.925+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f835c072220 con 0x7f835c06cec0 2024-03-22T04:10:30.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.925+0000 7f8359600640 1 -- 172.21.15.86:0/1289456042 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7f8350010040 con 0x7f835c06cec0 2024-03-22T04:10:30.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:30.927 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":19,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:24.658944+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.925+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f833c067b20 msgr2=0x7f833c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.925+0000 7f8364200640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f833c067b20 0x7f833c069fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f835c140580 tx=0x7f8348009040 comp rx=0 tx=0).stop 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c06cec0 msgr2=0x7f835c1679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c06cec0 0x7f835c1679f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f835000efc0 tx=0x7f835000c490 comp rx=0 tx=0).stop 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 shutdown_connections 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f833c067b20 0x7f833c069fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c147880 0x7f835c0cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835c0b4a60 0x7f835c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 --2- 172.21.15.86:0/1289456042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c06cec0 0x7f835c1679f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 >> 172.21.15.86:0/1289456042 conn(0x7f835c06ba60 msgr2=0x7f835c0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 shutdown_connections 2024-03-22T04:10:30.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:30.929+0000 7f8364200640 1 -- 172.21.15.86:0/1289456042 wait complete. 2024-03-22T04:10:30.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:10:30.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:10:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.081+0000 7f73af400640 1 Processor -- start 2024-03-22T04:10:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.081+0000 7f73af400640 1 -- start start 2024-03-22T04:10:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73af400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73a80b3610 0x7f73a809b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73af400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73a80b11c0 0x7f73a80b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73af400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a80cda00 con 0x7f73a815c730 2024-03-22T04:10:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73af400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f73a809b910 con 0x7f73a80b3610 2024-03-22T04:10:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73af400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f73a80b1710 con 0x7f73a80b11c0 2024-03-22T04:10:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73a7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73a80b3610 0x7f73a809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73aca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f73a815c730 0x7f73a809aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45324/0 (socket says 172.21.15.86:45324) 2024-03-22T04:10:31.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73aca00640 1 -- 172.21.15.86:0/2485348466 learned_addr learned my addr 172.21.15.86:0/2485348466 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:31.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73ad400640 1 --2- 172.21.15.86:0/2485348466 >> v2:172.21.15.196:3300/0 conn(0x7f73a80b11c0 0x7f73a80b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73a7e00640 1 -- 172.21.15.86:0/2485348466 >> v2:172.21.15.196:3300/0 conn(0x7f73a80b11c0 msgr2=0x7f73a80b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:31.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73a7e00640 1 --2- 172.21.15.86:0/2485348466 >> v2:172.21.15.196:3300/0 conn(0x7f73a80b11c0 0x7f73a80b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73a7e00640 1 -- 172.21.15.86:0/2485348466 >> v1:172.21.15.86:6789/0 conn(0x7f73a815c730 legacy=0x7f73a809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73a7e00640 1 -- 172.21.15.86:0/2485348466 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a80cdce0 con 0x7f73a80b3610 2024-03-22T04:10:31.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.085+0000 7f73ad400640 1 --2- 172.21.15.86:0/2485348466 >> v2:172.21.15.196:3300/0 conn(0x7f73a80b11c0 0x7f73a80b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:10:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73a7e00640 1 --2- 172.21.15.86:0/2485348466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a809b180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f73940097c0 tx=0x7f739401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6587dbf0ded35568 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73a7400640 1 -- 172.21.15.86:0/2485348466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940b0070 con 0x7f73a80b3610 2024-03-22T04:10:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73a7400640 1 -- 172.21.15.86:0/2485348466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73940ab600 con 0x7f73a80b3610 2024-03-22T04:10:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73a7400640 1 -- 172.21.15.86:0/2485348466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940ab9d0 con 0x7f73a80b3610 2024-03-22T04:10:31.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 -- 172.21.15.86:0/2485348466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 msgr2=0x7f73a809b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 --2- 172.21.15.86:0/2485348466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a809b180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f73940097c0 tx=0x7f739401c3d0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 -- 172.21.15.86:0/2485348466 shutdown_connections 2024-03-22T04:10:31.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 --2- 172.21.15.86:0/2485348466 >> v2:172.21.15.196:3300/0 conn(0x7f73a80b11c0 0x7f73a80b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 --2- 172.21.15.86:0/2485348466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a809b180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 -- 172.21.15.86:0/2485348466 >> 172.21.15.86:0/2485348466 conn(0x7f73a80b3bf0 msgr2=0x7f73a80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 -- 172.21.15.86:0/2485348466 shutdown_connections 2024-03-22T04:10:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 -- 172.21.15.86:0/2485348466 wait complete. 2024-03-22T04:10:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 Processor -- start 2024-03-22T04:10:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.089+0000 7f73af400640 1 -- start start 2024-03-22T04:10:31.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a80b11c0 0x7f73a80b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815c730 0x7f73a8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73a80b2c80 con 0x7f73a80b3610 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73a80b2b00 con 0x7f73a80b11c0 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f73a80b2e00 con 0x7f73a815c730 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a806fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40934/0 (socket says 172.21.15.86:40934) 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 -- 172.21.15.86:0/2329551085 learned_addr learned my addr 172.21.15.86:0/2329551085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73aca00640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a80b11c0 0x7f73a80b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73ad400640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815c730 0x7f73a8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 -- 172.21.15.86:0/2329551085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815c730 msgr2=0x7f73a8155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815c730 0x7f73a8155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 -- 172.21.15.86:0/2329551085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a80b11c0 msgr2=0x7f73a80b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a80b11c0 0x7f73a80b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 -- 172.21.15.86:0/2329551085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a80b6cc0 con 0x7f73a80b3610 2024-03-22T04:10:31.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a7e00640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a806fe90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7394018fe0 tx=0x7f7394019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a4c00640 1 -- 172.21.15.86:0/2329551085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940b0040 con 0x7f73a80b3610 2024-03-22T04:10:31.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a4c00640 1 -- 172.21.15.86:0/2329551085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73940ba070 con 0x7f73a80b3610 2024-03-22T04:10:31.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a4c00640 1 -- 172.21.15.86:0/2329551085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940b4630 con 0x7f73a80b3610 2024-03-22T04:10:31.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a80b6fa0 con 0x7f73a80b3610 2024-03-22T04:10:31.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a80b7560 con 0x7f73a80b3610 2024-03-22T04:10:31.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a8074470 con 0x7f73a80b3610 2024-03-22T04:10:31.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.093+0000 7f73a4c00640 1 -- 172.21.15.86:0/2329551085 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f73940b47d0 con 0x7f73a80b3610 2024-03-22T04:10:31.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.097+0000 7f73a4c00640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7388067b50 0x7f738806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.097+0000 7f73a4c00640 1 -- 172.21.15.86:0/2329551085 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f739411f8d0 con 0x7f73a80b3610 2024-03-22T04:10:31.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.097+0000 7f73aca00640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7388067b50 0x7f738806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.097+0000 7f73aca00640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7388067b50 0x7f738806a010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f739c000f70 tx=0x7f739c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.101+0000 7f73a4c00640 1 -- 172.21.15.86:0/2329551085 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7394016050 con 0x7f73a80b3610 2024-03-22T04:10:31.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73a809b360 con 0x7f73a80b3610 2024-03-22T04:10:31.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73a4c00640 1 -- 172.21.15.86:0/2329551085 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f73940f0500 con 0x7f73a80b3610 2024-03-22T04:10:31.255 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7388067b50 msgr2=0x7f738806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7388067b50 0x7f738806a010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f739c000f70 tx=0x7f739c009040 comp rx=0 tx=0).stop 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 msgr2=0x7f73a806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a806fe90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7394018fe0 tx=0x7f7394019170 comp rx=0 tx=0).stop 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 shutdown_connections 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7388067b50 0x7f738806a010 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815c730 0x7f73a8155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a80b3610 0x7f73a806fe90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 --2- 172.21.15.86:0/2329551085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a80b11c0 0x7f73a80b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 >> 172.21.15.86:0/2329551085 conn(0x7f73a80b3bf0 msgr2=0x7f73a806c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 shutdown_connections 2024-03-22T04:10:31.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.253+0000 7f73af400640 1 -- 172.21.15.86:0/2329551085 wait complete. 2024-03-22T04:10:31.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T04:10:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1e600640 1 Processor -- start 2024-03-22T04:10:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1e600640 1 -- start start 2024-03-22T04:10:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1e600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd1815f860 0x7efd180ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1e600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd18073cf0 con 0x7efd1809b180 2024-03-22T04:10:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1e600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd180ce280 con 0x7efd180cdb00 2024-03-22T04:10:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1e600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efd180ce560 con 0x7efd1815f860 2024-03-22T04:10:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1dc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efd1809b180 0x7efd180b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45338/0 (socket says 172.21.15.86:45338) 2024-03-22T04:10:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1dc00640 1 -- 172.21.15.86:0/2973905701 learned_addr learned my addr 172.21.15.86:0/2973905701 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:31.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1d200640 1 --2- 172.21.15.86:0/2973905701 >> v2:172.21.15.196:3300/0 conn(0x7efd1815f860 0x7efd180ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 182885327 0 0) 0x7efd18073cf0 con 0x7efd1809b180 2024-03-22T04:10:31.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcfc003530 con 0x7efd1809b180 2024-03-22T04:10:31.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3410745993 0 0) 0x7efcfc003530 con 0x7efd1809b180 2024-03-22T04:10:31.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 >> v2:172.21.15.196:3300/0 conn(0x7efd1815f860 msgr2=0x7efd180ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 --2- 172.21.15.86:0/2973905701 >> v2:172.21.15.196:3300/0 conn(0x7efd1815f860 0x7efd180ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 >> v1:172.21.15.196:6789/0 conn(0x7efd180cdb00 legacy=0x7efd1809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd180ce840 con 0x7efd1809b180 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7efd10001a00 con 0x7efd1809b180 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 >> v1:172.21.15.86:6789/0 conn(0x7efd1809b180 legacy=0x7efd180b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc003aa0 0x7efcfc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc004810 0x7efcfc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcfc0051d0 0x7efcfc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd180ce560 con 0x7efcfc004810 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc00c0d0 con 0x7efcfc003aa0 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efcfc00c2d0 con 0x7efcfc0051d0 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd0fe00640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc004810 0x7efcfc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1d200640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc003aa0 0x7efcfc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1dc00640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcfc0051d0 0x7efcfc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd0fe00640 1 -- 172.21.15.86:0/2973905701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcfc0051d0 msgr2=0x7efcfc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd0fe00640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcfc0051d0 0x7efcfc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd0fe00640 1 -- 172.21.15.86:0/2973905701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc003aa0 msgr2=0x7efcfc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd0fe00640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc003aa0 0x7efcfc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd0fe00640 1 -- 172.21.15.86:0/2973905701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc003530 con 0x7efcfc004810 2024-03-22T04:10:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.409+0000 7efd1dc00640 1 -- 172.21.15.86:0/2973905701 reap_dead start 2024-03-22T04:10:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd0fe00640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc004810 0x7efcfc004c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efd00001860 tx=0x7efd0001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635c58dc7a2c3367 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00001460 con 0x7efcfc004810 2024-03-22T04:10:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd000b0070 con 0x7efcfc004810 2024-03-22T04:10:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd17e00640 1 -- 172.21.15.86:0/2973905701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd000ab870 con 0x7efcfc004810 2024-03-22T04:10:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd1e600640 1 -- 172.21.15.86:0/2973905701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc004810 msgr2=0x7efcfc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd1e600640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc004810 0x7efcfc004c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efd00001860 tx=0x7efd0001c470 comp rx=0 tx=0).stop 2024-03-22T04:10:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd1e600640 1 -- 172.21.15.86:0/2973905701 shutdown_connections 2024-03-22T04:10:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd1e600640 1 --2- 172.21.15.86:0/2973905701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc004810 0x7efcfc004c90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd1e600640 1 -- 172.21.15.86:0/2973905701 >> 172.21.15.86:0/2973905701 conn(0x7efd180b3d60 msgr2=0x7efd180b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd1e600640 1 -- 172.21.15.86:0/2973905701 shutdown_connections 2024-03-22T04:10:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.413+0000 7efd1e600640 1 -- 172.21.15.86:0/2973905701 wait complete. 2024-03-22T04:10:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 Processor -- start 2024-03-22T04:10:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 -- start start 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc004810 0x7efd1806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd180b50c0 0x7efd181569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18070570 0x7efd180709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd1809ae80 con 0x7efd180b50c0 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd180cdf30 con 0x7efcfc004810 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efd1809b000 con 0x7efd18070570 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd180b50c0 0x7efd181569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd180b50c0 0x7efd181569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40952/0 (socket says 172.21.15.86:40952) 2024-03-22T04:10:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 -- 172.21.15.86:0/3872507106 learned_addr learned my addr 172.21.15.86:0/3872507106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1d200640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc004810 0x7efd1806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 -- 172.21.15.86:0/3872507106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18070570 msgr2=0x7efd180709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18070570 0x7efd180709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 -- 172.21.15.86:0/3872507106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc004810 msgr2=0x7efd1806d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc004810 0x7efd1806d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 -- 172.21.15.86:0/3872507106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd180b5860 con 0x7efd180b50c0 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1d200640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc004810 0x7efd1806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd0fe00640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd180b50c0 0x7efd181569f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd0001c2a0 tx=0x7efcfc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd15600640 1 -- 172.21.15.86:0/3872507106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00006aa0 con 0x7efd180b50c0 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd15600640 1 -- 172.21.15.86:0/3872507106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd00006c40 con 0x7efd180b50c0 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd180b5b40 con 0x7efd180b50c0 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd15600640 1 -- 172.21.15.86:0/3872507106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00007040 con 0x7efd180b50c0 2024-03-22T04:10:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.417+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd180b6100 con 0x7efd180b50c0 2024-03-22T04:10:31.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.421+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd18070f30 con 0x7efd180b50c0 2024-03-22T04:10:31.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.421+0000 7efd15600640 1 -- 172.21.15.86:0/3872507106 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efd000c3060 con 0x7efd180b50c0 2024-03-22T04:10:31.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.421+0000 7efd15600640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efcf4067b20 0x7efcf4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.421+0000 7efd15600640 1 -- 172.21.15.86:0/3872507106 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7efd0011f4e0 con 0x7efd180b50c0 2024-03-22T04:10:31.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.421+0000 7efd1d200640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efcf4067b20 0x7efcf4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.421+0000 7efd1d200640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efcf4067b20 0x7efcf4069fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd08009b50 tx=0x7efd08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.425+0000 7efd15600640 1 -- 172.21.15.86:0/3872507106 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efd000161b0 con 0x7efd180b50c0 2024-03-22T04:10:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.581+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efd18074470 con 0x7efd180b50c0 2024-03-22T04:10:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.581+0000 7efd15600640 1 -- 172.21.15.86:0/3872507106 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7efd000f01a0 con 0x7efd180b50c0 2024-03-22T04:10:31.584 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:31.584 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:10:31.584 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:10:31.584 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T04:10:31.584 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T04:10:31.585 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:10:31.585 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:10:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efcf4067b20 msgr2=0x7efcf4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efcf4067b20 0x7efcf4069fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd08009b50 tx=0x7efd08008040 comp rx=0 tx=0).stop 2024-03-22T04:10:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd180b50c0 msgr2=0x7efd181569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd180b50c0 0x7efd181569f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd0001c2a0 tx=0x7efcfc005840 comp rx=0 tx=0).stop 2024-03-22T04:10:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 shutdown_connections 2024-03-22T04:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efcf4067b20 0x7efcf4069fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18070570 0x7efd180709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd180b50c0 0x7efd181569f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 --2- 172.21.15.86:0/3872507106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcfc004810 0x7efd1806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 >> 172.21.15.86:0/3872507106 conn(0x7efd180b3d60 msgr2=0x7efd1815c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 shutdown_connections 2024-03-22T04:10:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.585+0000 7efd1e600640 1 -- 172.21.15.86:0/3872507106 wait complete. 2024-03-22T04:10:31.604 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T04:10:31.605 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T04:10:31.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb799600640 1 Processor -- start 2024-03-22T04:10:31.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb799600640 1 -- start start 2024-03-22T04:10:31.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb799600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb79409acd0 0x7fb79409b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb799600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb7940d1160 0x7fb79406f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb799600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7940b4c60 con 0x7fb79409b5f0 2024-03-22T04:10:31.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb799600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb7940cdac0 con 0x7fb7940d1160 2024-03-22T04:10:31.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb799600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb7940cdda0 con 0x7fb79409acd0 2024-03-22T04:10:31.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb793400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb7940d1160 0x7fb79406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb792000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb79409b5f0 0x7fb79415f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45344/0 (socket says 172.21.15.86:45344) 2024-03-22T04:10:31.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb793400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb7940d1160 0x7fb79406f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40954/0 (socket says 172.21.15.86:40954) 2024-03-22T04:10:31.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb792000640 1 -- 172.21.15.86:0/2147037239 learned_addr learned my addr 172.21.15.86:0/2147037239 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:31.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb793400640 1 -- 172.21.15.86:0/2147037239 >> v2:172.21.15.196:3300/0 conn(0x7fb79409acd0 msgr2=0x7fb79409b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:31.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb792a00640 1 --2- 172.21.15.86:0/2147037239 >> v2:172.21.15.196:3300/0 conn(0x7fb79409acd0 0x7fb79409b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb791600640 1 -- 172.21.15.86:0/2147037239 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724462899 0 0) 0x7fb7940b4c60 con 0x7fb79409b5f0 2024-03-22T04:10:31.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb793400640 1 --2- 172.21.15.86:0/2147037239 >> v2:172.21.15.196:3300/0 conn(0x7fb79409acd0 0x7fb79409b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb793400640 1 -- 172.21.15.86:0/2147037239 >> v1:172.21.15.86:6789/0 conn(0x7fb79409b5f0 legacy=0x7fb79415f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb793400640 1 -- 172.21.15.86:0/2147037239 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7940ce080 con 0x7fb7940d1160 2024-03-22T04:10:31.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.741+0000 7fb792a00640 1 --2- 172.21.15.86:0/2147037239 >> v2:172.21.15.196:3300/0 conn(0x7fb79409acd0 0x7fb79409b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:10:31.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb793400640 1 --2- 172.21.15.86:0/2147037239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7940d1160 0x7fb79406f770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb794072ad0 tx=0x7fb78c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b0150349a84d71 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb791600640 1 -- 172.21.15.86:0/2147037239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0b2070 con 0x7fb7940d1160 2024-03-22T04:10:31.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb791600640 1 -- 172.21.15.86:0/2147037239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c0a5980 con 0x7fb7940d1160 2024-03-22T04:10:31.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb791600640 1 -- 172.21.15.86:0/2147037239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0a5d50 con 0x7fb7940d1160 2024-03-22T04:10:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb799600640 1 -- 172.21.15.86:0/2147037239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7940d1160 msgr2=0x7fb79406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb799600640 1 --2- 172.21.15.86:0/2147037239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7940d1160 0x7fb79406f770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb794072ad0 tx=0x7fb78c01c8a0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb799600640 1 -- 172.21.15.86:0/2147037239 shutdown_connections 2024-03-22T04:10:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb799600640 1 --2- 172.21.15.86:0/2147037239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7940d1160 0x7fb79406f770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb799600640 1 --2- 172.21.15.86:0/2147037239 >> v2:172.21.15.196:3300/0 conn(0x7fb79409acd0 0x7fb79409b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb799600640 1 -- 172.21.15.86:0/2147037239 >> 172.21.15.86:0/2147037239 conn(0x7fb7940b3d40 msgr2=0x7fb7940b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb799600640 1 -- 172.21.15.86:0/2147037239 shutdown_connections 2024-03-22T04:10:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.745+0000 7fb799600640 1 -- 172.21.15.86:0/2147037239 wait complete. 2024-03-22T04:10:31.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 Processor -- start 2024-03-22T04:10:31.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 -- start start 2024-03-22T04:10:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79409acd0 0x7fb7941645e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb79409b5f0 0x7fb79406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7940d1160 0x7fb794165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb794070890 con 0x7fb79409acd0 2024-03-22T04:10:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb794070710 con 0x7fb79409b5f0 2024-03-22T04:10:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb794070a10 con 0x7fb7940d1160 2024-03-22T04:10:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79409acd0 0x7fb7941645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79409acd0 0x7fb7941645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40962/0 (socket says 172.21.15.86:40962) 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 -- 172.21.15.86:0/2637365302 learned_addr learned my addr 172.21.15.86:0/2637365302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792000640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb79409b5f0 0x7fb79406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb793400640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7940d1160 0x7fb794165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 -- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7940d1160 msgr2=0x7fb794165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7940d1160 0x7fb794165160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 -- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb79409b5f0 msgr2=0x7fb79406da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb79409b5f0 0x7fb79406da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 -- 172.21.15.86:0/2637365302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79415ab00 con 0x7fb79409acd0 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb793400640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7940d1160 0x7fb794165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb792a00640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79409acd0 0x7fb7941645e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c7f0 tx=0x7fb77c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb78aa00640 1 -- 172.21.15.86:0/2637365302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c015070 con 0x7fb79409acd0 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb78aa00640 1 -- 172.21.15.86:0/2637365302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c007820 con 0x7fb79409acd0 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79415ade0 con 0x7fb79409acd0 2024-03-22T04:10:31.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb78aa00640 1 -- 172.21.15.86:0/2637365302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c007c10 con 0x7fb79409acd0 2024-03-22T04:10:31.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.749+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79406d590 con 0x7fb79409acd0 2024-03-22T04:10:31.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.753+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb794074580 con 0x7fb79409acd0 2024-03-22T04:10:31.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.753+0000 7fb78aa00640 1 -- 172.21.15.86:0/2637365302 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb77c018070 con 0x7fb79409acd0 2024-03-22T04:10:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.753+0000 7fb78aa00640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb770067b20 0x7fb770069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.753+0000 7fb78aa00640 1 -- 172.21.15.86:0/2637365302 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fb77c085cb0 con 0x7fb79409acd0 2024-03-22T04:10:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.753+0000 7fb792000640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb770067b20 0x7fb770069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.753+0000 7fb792000640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb770067b20 0x7fb770069fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb79406e3b0 tx=0x7fb784007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.757+0000 7fb78aa00640 1 -- 172.21.15.86:0/2637365302 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb77c056aa0 con 0x7fb79409acd0 2024-03-22T04:10:31.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:31.913+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb79409b350 con 0x7fb79409acd0 2024-03-22T04:10:32.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.893+0000 7fb78aa00640 1 -- 172.21.15.86:0/2637365302 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7fb77c0563a0 con 0x7fb79409acd0 2024-03-22T04:10:32.895 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T04:10:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb770067b20 msgr2=0x7fb770069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb770067b20 0x7fb770069fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb79406e3b0 tx=0x7fb784007b60 comp rx=0 tx=0).stop 2024-03-22T04:10:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79409acd0 msgr2=0x7fb7941645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79409acd0 0x7fb7941645e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c7f0 tx=0x7fb77c00ccb0 comp rx=0 tx=0).stop 2024-03-22T04:10:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 shutdown_connections 2024-03-22T04:10:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb770067b20 0x7fb770069fe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb79409b5f0 0x7fb79406da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7940d1160 0x7fb794165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 --2- 172.21.15.86:0/2637365302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79409acd0 0x7fb7941645e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 >> 172.21.15.86:0/2637365302 conn(0x7fb7940b3d40 msgr2=0x7fb7940af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 shutdown_connections 2024-03-22T04:10:32.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:32.897+0000 7fb799600640 1 -- 172.21.15.86:0/2637365302 wait complete. 2024-03-22T04:10:32.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-22T04:10:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.069+0000 7fee78c00640 1 Processor -- start 2024-03-22T04:10:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.069+0000 7fee78c00640 1 -- start start 2024-03-22T04:10:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee78c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 0x7fee740cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee78c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fee7415c920 0x7fee740d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee78c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7415ff50 con 0x7fee740cf030 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee78c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fee740cfb60 con 0x7fee7415c920 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee78c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fee74160230 con 0x7fee7413f330 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee72800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee740cf030 0x7fee740cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45348/0 (socket says 172.21.15.86:45348) 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee72800640 1 -- 172.21.15.86:0/136350897 learned_addr learned my addr 172.21.15.86:0/136350897 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee71400640 1 --2- 172.21.15.86:0/136350897 >> v2:172.21.15.196:3300/0 conn(0x7fee7415c920 0x7fee740d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee71e00640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 0x7fee740cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3864886540 0 0) 0x7fee7415ff50 con 0x7fee740cf030 2024-03-22T04:10:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee58003530 con 0x7fee740cf030 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1859689031 0 0) 0x7fee58003530 con 0x7fee740cf030 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 msgr2=0x7fee740cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 0x7fee740cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 >> v2:172.21.15.196:3300/0 conn(0x7fee7415c920 msgr2=0x7fee740d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 --2- 172.21.15.86:0/136350897 >> v2:172.21.15.196:3300/0 conn(0x7fee7415c920 0x7fee740d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee74160510 con 0x7fee740cf030 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fee6c001a00 con 0x7fee740cf030 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee71400640 1 --2- 172.21.15.86:0/136350897 >> v2:172.21.15.196:3300/0 conn(0x7fee7415c920 0x7fee740d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee71e00640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 0x7fee740cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 >> v1:172.21.15.86:6789/0 conn(0x7fee740cf030 legacy=0x7fee740cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 0x7fee58003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58003f70 0x7fee58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee58004d50 0x7fee580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee740cfb60 con 0x7fee58004d50 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee74160230 con 0x7fee58003f70 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fee5800b9d0 con 0x7fee7413f330 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee72800640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee58004d50 0x7fee580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee71e00640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 0x7fee58003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee71400640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58003f70 0x7fee58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee72800640 1 -- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 msgr2=0x7fee58003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee72800640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee7413f330 0x7fee58003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee72800640 1 -- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58003f70 msgr2=0x7fee58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee72800640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58003f70 0x7fee58004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.073+0000 7fee72800640 1 -- 172.21.15.86:0/136350897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee58003530 con 0x7fee58004d50 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee72800640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee58004d50 0x7fee580051d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00a880 tx=0x7fee6c01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8106cf25c7fa613b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00a4a0 con 0x7fee58004d50 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee6c0ad630 con 0x7fee58004d50 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee70a00640 1 -- 172.21.15.86:0/136350897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0adaa0 con 0x7fee58004d50 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee78c00640 1 -- 172.21.15.86:0/136350897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee58004d50 msgr2=0x7fee580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee78c00640 1 --2- 172.21.15.86:0/136350897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee58004d50 0x7fee580051d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00a880 tx=0x7fee6c01d3a0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee72800640 1 -- 172.21.15.86:0/136350897 reap_dead start 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee78c00640 1 -- 172.21.15.86:0/136350897 shutdown_connections 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee78c00640 1 -- 172.21.15.86:0/136350897 >> 172.21.15.86:0/136350897 conn(0x7fee7406ef50 msgr2=0x7fee7406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee78c00640 1 -- 172.21.15.86:0/136350897 shutdown_connections 2024-03-22T04:10:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee78c00640 1 -- 172.21.15.86:0/136350897 wait complete. 2024-03-22T04:10:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee78c00640 1 Processor -- start 2024-03-22T04:10:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.077+0000 7fee78c00640 1 -- start start 2024-03-22T04:10:33.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee741514a0 0x7fee74153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74151c40 0x7fee7413e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee74159790 0x7fee74159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee7413f710 con 0x7fee74151c40 2024-03-22T04:10:33.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee74160ad0 con 0x7fee74159790 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fee740cfda0 con 0x7fee741514a0 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74151c40 0x7fee7413e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74151c40 0x7fee7413e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40986/0 (socket says 172.21.15.86:40986) 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 -- 172.21.15.86:0/1891487026 learned_addr learned my addr 172.21.15.86:0/1891487026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71e00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee741514a0 0x7fee74153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee72800640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee74159790 0x7fee74159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 -- 172.21.15.86:0/1891487026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee741514a0 msgr2=0x7fee74153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee741514a0 0x7fee74153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 -- 172.21.15.86:0/1891487026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee74159790 msgr2=0x7fee74159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee74159790 0x7fee74159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 -- 172.21.15.86:0/1891487026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7413be70 con 0x7fee74151c40 2024-03-22T04:10:33.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee71400640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74151c40 0x7fee7413e1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee64011cd0 tx=0x7fee6400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:33.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee6a000640 1 -- 172.21.15.86:0/1891487026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6400fd20 con 0x7fee74151c40 2024-03-22T04:10:33.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee6a000640 1 -- 172.21.15.86:0/1891487026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee64004030 con 0x7fee74151c40 2024-03-22T04:10:33.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee6a000640 1 -- 172.21.15.86:0/1891487026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64004420 con 0x7fee74151c40 2024-03-22T04:10:33.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee741520d0 con 0x7fee74151c40 2024-03-22T04:10:33.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee74152690 con 0x7fee74151c40 2024-03-22T04:10:33.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.081+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee74074470 con 0x7fee74151c40 2024-03-22T04:10:33.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.085+0000 7fee6a000640 1 -- 172.21.15.86:0/1891487026 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fee64012040 con 0x7fee74151c40 2024-03-22T04:10:33.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.085+0000 7fee6a000640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee50067b20 0x7fee50069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:33.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.085+0000 7fee6a000640 1 -- 172.21.15.86:0/1891487026 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7fee64086290 con 0x7fee74151c40 2024-03-22T04:10:33.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.085+0000 7fee71e00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee50067b20 0x7fee50069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:33.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.085+0000 7fee71e00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee50067b20 0x7fee50069fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009bd0 tx=0x7fee5c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.089+0000 7fee6a000640 1 -- 172.21.15.86:0/1891487026 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fee64019070 con 0x7fee74151c40 2024-03-22T04:10:33.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.245+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fee7415f3d0 con 0x7fee74151c40 2024-03-22T04:10:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.901+0000 7fee6a000640 1 -- 172.21.15.86:0/1891487026 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) ==== 171+0+0 (secure 0 0 0) 0x7fee64056790 con 0x7fee74151c40 2024-03-22T04:10:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee50067b20 msgr2=0x7fee50069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee50067b20 0x7fee50069fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009bd0 tx=0x7fee5c009190 comp rx=0 tx=0).stop 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74151c40 msgr2=0x7fee7413e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74151c40 0x7fee7413e1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee64011cd0 tx=0x7fee6400f610 comp rx=0 tx=0).stop 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 shutdown_connections 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee50067b20 0x7fee50069fe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee74159790 0x7fee74159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74151c40 0x7fee7413e1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 --2- 172.21.15.86:0/1891487026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee741514a0 0x7fee74153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 >> 172.21.15.86:0/1891487026 conn(0x7fee7406ef50 msgr2=0x7fee740afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 shutdown_connections 2024-03-22T04:10:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:33.905+0000 7fee78c00640 1 -- 172.21.15.86:0/1891487026 wait complete. 2024-03-22T04:10:33.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T04:10:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.089+0000 7fe90be00640 1 Processor -- start 2024-03-22T04:10:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.089+0000 7fe90be00640 1 -- start start 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe90c0d2b30 0x7fe90c0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe90c0ce900 0x7fe90c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe90c0cf250 con 0x7fe90c15c230 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe90c0d05d0 con 0x7fe90c0d2b30 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90be00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe90c0cdd60 con 0x7fe90c0ce900 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe90c0d2b30 0x7fe90c0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe90c0d2b30 0x7fe90c0d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40992/0 (socket says 172.21.15.86:40992) 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90a000640 1 -- 172.21.15.86:0/3646226948 learned_addr learned my addr 172.21.15.86:0/3646226948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90b400640 1 --2- 172.21.15.86:0/3646226948 >> v2:172.21.15.196:3300/0 conn(0x7fe90c0ce900 0x7fe90c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90a000640 1 -- 172.21.15.86:0/3646226948 >> v2:172.21.15.196:3300/0 conn(0x7fe90c0ce900 msgr2=0x7fe90c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90a000640 1 --2- 172.21.15.86:0/3646226948 >> v2:172.21.15.196:3300/0 conn(0x7fe90c0ce900 0x7fe90c0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90a000640 1 -- 172.21.15.86:0/3646226948 >> v1:172.21.15.86:6789/0 conn(0x7fe90c15c230 legacy=0x7fe90c0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:34.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.093+0000 7fe90a000640 1 -- 172.21.15.86:0/3646226948 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c0cfca0 con 0x7fe90c0d2b30 2024-03-22T04:10:34.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90a000640 1 --2- 172.21.15.86:0/3646226948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c0d2b30 0x7fe90c0d0a60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0c58b0 tx=0x7fe8f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb02e0e7e7ef7e43 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe909600640 1 -- 172.21.15.86:0/3646226948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80ab430 con 0x7fe90c0d2b30 2024-03-22T04:10:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe909600640 1 -- 172.21.15.86:0/3646226948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f80b0070 con 0x7fe90c0d2b30 2024-03-22T04:10:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe909600640 1 -- 172.21.15.86:0/3646226948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80ab7f0 con 0x7fe90c0d2b30 2024-03-22T04:10:34.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- 172.21.15.86:0/3646226948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c0d2b30 msgr2=0x7fe90c0d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 --2- 172.21.15.86:0/3646226948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c0d2b30 0x7fe90c0d0a60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0c58b0 tx=0x7fe8f801c520 comp rx=0 tx=0).stop 2024-03-22T04:10:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- 172.21.15.86:0/3646226948 shutdown_connections 2024-03-22T04:10:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 --2- 172.21.15.86:0/3646226948 >> v2:172.21.15.196:3300/0 conn(0x7fe90c0ce900 0x7fe90c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 --2- 172.21.15.86:0/3646226948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c0d2b30 0x7fe90c0d0a60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- 172.21.15.86:0/3646226948 >> 172.21.15.86:0/3646226948 conn(0x7fe90c0d1f30 msgr2=0x7fe90c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- 172.21.15.86:0/3646226948 shutdown_connections 2024-03-22T04:10:34.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- 172.21.15.86:0/3646226948 wait complete. 2024-03-22T04:10:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 Processor -- start 2024-03-22T04:10:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- start start 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c0ce900 0x7fe90c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c0d2b30 0x7fe90c0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c15c230 0x7fe90c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c0b5920 con 0x7fe90c15c230 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c0b57a0 con 0x7fe90c0d2b30 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe90c168ff0 con 0x7fe90c0ce900 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c15c230 0x7fe90c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c15c230 0x7fe90c155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40996/0 (socket says 172.21.15.86:40996) 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90b400640 1 -- 172.21.15.86:0/1707056424 learned_addr learned my addr 172.21.15.86:0/1707056424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.097+0000 7fe90a000640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c0d2b30 0x7fe90c0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90aa00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c0ce900 0x7fe90c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90b400640 1 -- 172.21.15.86:0/1707056424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c0ce900 msgr2=0x7fe90c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90b400640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c0ce900 0x7fe90c06d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90b400640 1 -- 172.21.15.86:0/1707056424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c0d2b30 msgr2=0x7fe90c0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90b400640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c0d2b30 0x7fe90c0c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90b400640 1 -- 172.21.15.86:0/1707056424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c06cad0 con 0x7fe90c15c230 2024-03-22T04:10:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90b400640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c15c230 0x7fe90c155db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe904007e60 tx=0x7fe90400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe902a00640 1 -- 172.21.15.86:0/1707056424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904014420 con 0x7fe90c15c230 2024-03-22T04:10:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe902a00640 1 -- 172.21.15.86:0/1707056424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe904019070 con 0x7fe90c15c230 2024-03-22T04:10:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe902a00640 1 -- 172.21.15.86:0/1707056424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904014820 con 0x7fe90c15c230 2024-03-22T04:10:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c15b8d0 con 0x7fe90c15c230 2024-03-22T04:10:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c06d020 con 0x7fe90c15c230 2024-03-22T04:10:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c04f900 con 0x7fe90c15c230 2024-03-22T04:10:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe902a00640 1 -- 172.21.15.86:0/1707056424 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe9040149c0 con 0x7fe90c15c230 2024-03-22T04:10:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.101+0000 7fe902a00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe8ec067b20 0x7fe8ec069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.105+0000 7fe902a00640 1 -- 172.21.15.86:0/1707056424 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7fe9040867b0 con 0x7fe90c15c230 2024-03-22T04:10:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.105+0000 7fe90aa00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe8ec067b20 0x7fe8ec069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.105+0000 7fe90aa00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe8ec067b20 0x7fe8ec069fe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc001640 tx=0x7fe8fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.109+0000 7fe902a00640 1 -- 172.21.15.86:0/1707056424 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe9040570f0 con 0x7fe90c15c230 2024-03-22T04:10:34.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.293+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe90c0c8490 con 0x7fe90c15c230 2024-03-22T04:10:34.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.901+0000 7fe902a00640 1 -- 172.21.15.86:0/1707056424 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fe9040569f0 con 0x7fe90c15c230 2024-03-22T04:10:34.905 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe8ec067b20 msgr2=0x7fe8ec069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe8ec067b20 0x7fe8ec069fe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc001640 tx=0x7fe8fc009040 comp rx=0 tx=0).stop 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c15c230 msgr2=0x7fe90c155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c15c230 0x7fe90c155db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe904007e60 tx=0x7fe90400e4b0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 shutdown_connections 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe8ec067b20 0x7fe8ec069fe0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c0d2b30 0x7fe90c0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe90c15c230 0x7fe90c155db0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 --2- 172.21.15.86:0/1707056424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c0ce900 0x7fe90c06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 >> 172.21.15.86:0/1707056424 conn(0x7fe90c0d1f30 msgr2=0x7fe90c0b64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 shutdown_connections 2024-03-22T04:10:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:34.905+0000 7fe90be00640 1 -- 172.21.15.86:0/1707056424 wait complete. 2024-03-22T04:10:34.957 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-22T04:10:34.957 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-22T04:10:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41cc000640 1 Processor -- start 2024-03-22T04:10:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41cc000640 1 -- start start 2024-03-22T04:10:35.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41cc000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41c40d0420 0x7f41c4162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:35.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41cc000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f41c406ce80 0x7f41c4154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:35.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41cc000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c4070620 con 0x7f41c40cf090 2024-03-22T04:10:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41cc000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41c4162fa0 con 0x7f41c40d0420 2024-03-22T04:10:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41cc000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f41c4070900 con 0x7f41c406ce80 2024-03-22T04:10:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c9600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f41c40cf090 0x7f41c406c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45368/0 (socket says 172.21.15.86:45368) 2024-03-22T04:10:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c9600640 1 -- 172.21.15.86:0/1411655353 learned_addr learned my addr 172.21.15.86:0/1411655353 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c8c00640 1 --2- 172.21.15.86:0/1411655353 >> v2:172.21.15.86:3300/0 conn(0x7f41c40d0420 0x7f41c4162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c3e00640 1 -- 172.21.15.86:0/1411655353 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 135341802 0 0) 0x7f41c4070620 con 0x7f41c40cf090 2024-03-22T04:10:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c3e00640 1 -- 172.21.15.86:0/1411655353 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41ac003530 con 0x7f41c40cf090 2024-03-22T04:10:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41ca000640 1 --2- 172.21.15.86:0/1411655353 >> v2:172.21.15.196:3300/0 conn(0x7f41c406ce80 0x7f41c4154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c8c00640 1 -- 172.21.15.86:0/1411655353 >> v2:172.21.15.196:3300/0 conn(0x7f41c406ce80 msgr2=0x7f41c4154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c8c00640 1 --2- 172.21.15.86:0/1411655353 >> v2:172.21.15.196:3300/0 conn(0x7f41c406ce80 0x7f41c4154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c8c00640 1 -- 172.21.15.86:0/1411655353 >> v1:172.21.15.86:6789/0 conn(0x7f41c40cf090 legacy=0x7f41c406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41c8c00640 1 -- 172.21.15.86:0/1411655353 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c40cff00 con 0x7f41c40d0420 2024-03-22T04:10:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.097+0000 7f41ca000640 1 --2- 172.21.15.86:0/1411655353 >> v2:172.21.15.196:3300/0 conn(0x7f41c406ce80 0x7f41c4154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:10:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41c8c00640 1 --2- 172.21.15.86:0/1411655353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c4162a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41c406e930 tx=0x7f41b001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcdc14287609d99 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41c3e00640 1 -- 172.21.15.86:0/1411655353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00b1070 con 0x7f41c40d0420 2024-03-22T04:10:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41c3e00640 1 -- 172.21.15.86:0/1411655353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b00ac640 con 0x7f41c40d0420 2024-03-22T04:10:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41c3e00640 1 -- 172.21.15.86:0/1411655353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00aca10 con 0x7f41c40d0420 2024-03-22T04:10:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 -- 172.21.15.86:0/1411655353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 msgr2=0x7f41c4162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 --2- 172.21.15.86:0/1411655353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c4162a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41c406e930 tx=0x7f41b001c350 comp rx=0 tx=0).stop 2024-03-22T04:10:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 -- 172.21.15.86:0/1411655353 shutdown_connections 2024-03-22T04:10:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 --2- 172.21.15.86:0/1411655353 >> v2:172.21.15.196:3300/0 conn(0x7f41c406ce80 0x7f41c4154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 --2- 172.21.15.86:0/1411655353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c4162a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 -- 172.21.15.86:0/1411655353 >> 172.21.15.86:0/1411655353 conn(0x7f41c406b660 msgr2=0x7f41c406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 -- 172.21.15.86:0/1411655353 shutdown_connections 2024-03-22T04:10:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 -- 172.21.15.86:0/1411655353 wait complete. 2024-03-22T04:10:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.101+0000 7f41cc000640 1 Processor -- start 2024-03-22T04:10:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 -- start start 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41c406ce80 0x7f41c40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41c40cf090 0x7f41c4157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c40c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41c406df60 con 0x7f41c40d0420 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41c406dde0 con 0x7f41c406ce80 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f41c415ea90 con 0x7f41c40cf090 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c40c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c40c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41010/0 (socket says 172.21.15.86:41010) 2024-03-22T04:10:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 -- 172.21.15.86:0/3312860996 learned_addr learned my addr 172.21.15.86:0/3312860996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41c8c00640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41c40cf090 0x7f41c4157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41c9600640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41c406ce80 0x7f41c40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 -- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41c40cf090 msgr2=0x7f41c4157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41c40cf090 0x7f41c4157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 -- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41c406ce80 msgr2=0x7f41c40b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41c406ce80 0x7f41c40b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 -- 172.21.15.86:0/3312860996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c40ca4b0 con 0x7f41c40d0420 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41c9600640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41c406ce80 0x7f41c40b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41ca000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c40c9280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00cef0 tx=0x7f41bc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41c1600640 1 -- 172.21.15.86:0/3312860996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc01a070 con 0x7f41c40d0420 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41c1600640 1 -- 172.21.15.86:0/3312860996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41bc00d860 con 0x7f41c40d0420 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41c1600640 1 -- 172.21.15.86:0/3312860996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00dc50 con 0x7f41c40d0420 2024-03-22T04:10:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c40caa30 con 0x7f41c40d0420 2024-03-22T04:10:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c40caff0 con 0x7f41c40d0420 2024-03-22T04:10:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.105+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c4074580 con 0x7f41c40d0420 2024-03-22T04:10:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.109+0000 7f41c1600640 1 -- 172.21.15.86:0/3312860996 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f41bc01d070 con 0x7f41c40d0420 2024-03-22T04:10:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.109+0000 7f41c1600640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41a4067b20 0x7f41a4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.109+0000 7f41c1600640 1 -- 172.21.15.86:0/3312860996 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f41bc086760 con 0x7f41c40d0420 2024-03-22T04:10:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.109+0000 7f41c9600640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41a4067b20 0x7f41a4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.109+0000 7f41c9600640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41a4067b20 0x7f41a4069fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f41b8005890 tx=0x7f41b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:35.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.113+0000 7f41c1600640 1 -- 172.21.15.86:0/3312860996 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f41bc0572c0 con 0x7f41c40d0420 2024-03-22T04:10:35.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.273+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f41c40ca750 con 0x7f41c40d0420 2024-03-22T04:10:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.925+0000 7f41c1600640 1 -- 172.21.15.86:0/3312860996 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) ==== 157+0+0 (secure 0 0 0) 0x7f41c40ca750 con 0x7f41c40d0420 2024-03-22T04:10:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41a4067b20 msgr2=0x7f41a4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41a4067b20 0x7f41a4069fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f41b8005890 tx=0x7f41b8009040 comp rx=0 tx=0).stop 2024-03-22T04:10:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 msgr2=0x7f41c40c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c40c9280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00cef0 tx=0x7f41bc00a710 comp rx=0 tx=0).stop 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 shutdown_connections 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41a4067b20 0x7f41a4069fe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c40d0420 0x7f41c40c9280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41c40cf090 0x7f41c4157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 --2- 172.21.15.86:0/3312860996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41c406ce80 0x7f41c40b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 >> 172.21.15.86:0/3312860996 conn(0x7f41c406b660 msgr2=0x7f41c414ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 shutdown_connections 2024-03-22T04:10:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:35.929+0000 7f41cc000640 1 -- 172.21.15.86:0/3312860996 wait complete. 2024-03-22T04:10:35.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T04:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.093+0000 7fdbca800640 1 Processor -- start 2024-03-22T04:10:36.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.093+0000 7fdbca800640 1 -- start start 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbca800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 0x7fdbc415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbca800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbc406f710 0x7fdbc4162190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbca800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbc406fb70 con 0x7fdbc406d180 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbca800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc41626d0 con 0x7fdbc406f710 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbca800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc4162980 con 0x7fdbc40b1730 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbc2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbc406f710 0x7fdbc4162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbc2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbc406f710 0x7fdbc4162190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35028/0 (socket says 172.21.15.86:35028) 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbc2a00640 1 -- 172.21.15.86:0/658138727 learned_addr learned my addr 172.21.15.86:0/658138727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbc3400640 1 --2- 172.21.15.86:0/658138727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 0x7fdbc415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbc2a00640 1 -- 172.21.15.86:0/658138727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 msgr2=0x7fdbc415f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbc2a00640 1 --2- 172.21.15.86:0/658138727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 0x7fdbc415f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbc2a00640 1 -- 172.21.15.86:0/658138727 >> v1:172.21.15.196:6789/0 conn(0x7fdbc406d180 legacy=0x7fdbc40d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.097+0000 7fdbc2a00640 1 -- 172.21.15.86:0/658138727 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc4162c60 con 0x7fdbc406f710 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbc2a00640 1 --2- 172.21.15.86:0/658138727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406f710 0x7fdbc4162190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009870 tx=0x7fdbb001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f3a05bf77ecc2f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbc2000640 1 -- 172.21.15.86:0/658138727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00ab610 con 0x7fdbc406f710 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbc2000640 1 -- 172.21.15.86:0/658138727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb00b0070 con 0x7fdbc406f710 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbc2000640 1 -- 172.21.15.86:0/658138727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00ab9d0 con 0x7fdbc406f710 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- 172.21.15.86:0/658138727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406f710 msgr2=0x7fdbc4162190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 --2- 172.21.15.86:0/658138727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406f710 0x7fdbc4162190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009870 tx=0x7fdbb001c410 comp rx=0 tx=0).stop 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- 172.21.15.86:0/658138727 shutdown_connections 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 --2- 172.21.15.86:0/658138727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406f710 0x7fdbc4162190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 --2- 172.21.15.86:0/658138727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 0x7fdbc415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- 172.21.15.86:0/658138727 >> 172.21.15.86:0/658138727 conn(0x7fdbc40b3c40 msgr2=0x7fdbc40b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- 172.21.15.86:0/658138727 shutdown_connections 2024-03-22T04:10:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- 172.21.15.86:0/658138727 wait complete. 2024-03-22T04:10:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 Processor -- start 2024-03-22T04:10:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- start start 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406d180 0x7fdbc41543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406f710 0x7fdbc40b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 0x7fdbc4159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc406d9a0 con 0x7fdbc406d180 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc40d1720 con 0x7fdbc406f710 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbca800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc40b52b0 con 0x7fdbc40b1730 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbc3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406d180 0x7fdbc41543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbc3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406d180 0x7fdbc41543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35032/0 (socket says 172.21.15.86:35032) 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.101+0000 7fdbc3400640 1 -- 172.21.15.86:0/2283668725 learned_addr learned my addr 172.21.15.86:0/2283668725 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc2a00640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406f710 0x7fdbc40b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc3e00640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 0x7fdbc4159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc3400640 1 -- 172.21.15.86:0/2283668725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 msgr2=0x7fdbc4159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc3400640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 0x7fdbc4159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc3400640 1 -- 172.21.15.86:0/2283668725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406f710 msgr2=0x7fdbc40b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc3400640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406f710 0x7fdbc40b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc3400640 1 -- 172.21.15.86:0/2283668725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc40b25c0 con 0x7fdbc406d180 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc3400640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406d180 0x7fdbc41543e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400efc0 tx=0x7fdbb400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbbb400640 1 -- 172.21.15.86:0/2283668725 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4018020 con 0x7fdbc406d180 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbbb400640 1 -- 172.21.15.86:0/2283668725 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb4014480 con 0x7fdbc406d180 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc40b5550 con 0x7fdbc406d180 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbbb400640 1 -- 172.21.15.86:0/2283668725 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4014850 con 0x7fdbc406d180 2024-03-22T04:10:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc40b5b10 con 0x7fdbc406d180 2024-03-22T04:10:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc4074470 con 0x7fdbc406d180 2024-03-22T04:10:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbbb400640 1 -- 172.21.15.86:0/2283668725 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdbb401b020 con 0x7fdbc406d180 2024-03-22T04:10:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbbb400640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdba4067b20 0x7fdba4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbbb400640 1 -- 172.21.15.86:0/2283668725 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6007+0+0 (secure 0 0 0) 0x7fdbb4086ea0 con 0x7fdbc406d180 2024-03-22T04:10:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.105+0000 7fdbc2a00640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdba4067b20 0x7fdba4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.109+0000 7fdbc2a00640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdba4067b20 0x7fdba4069fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00af000 tx=0x7fdbb00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:36.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.109+0000 7fdbbb400640 1 -- 172.21.15.86:0/2283668725 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdbb4022030 con 0x7fdbc406d180 2024-03-22T04:10:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:36.265+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdbc406cf60 con 0x7fdbc406d180 2024-03-22T04:10:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.945+0000 7fdbbb400640 1 -- 172.21.15.86:0/2283668725 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) ==== 228+0+0 (secure 0 0 0) 0x7fdbb4010040 con 0x7fdbc406d180 2024-03-22T04:10:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' created 2024-03-22T04:10:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdba4067b20 msgr2=0x7fdba4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdba4067b20 0x7fdba4069fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00af000 tx=0x7fdbb00a7000 comp rx=0 tx=0).stop 2024-03-22T04:10:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406d180 msgr2=0x7fdbc41543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406d180 0x7fdbc41543e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400efc0 tx=0x7fdbb400c490 comp rx=0 tx=0).stop 2024-03-22T04:10:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 shutdown_connections 2024-03-22T04:10:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdba4067b20 0x7fdba4069fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406f710 0x7fdbc40b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40b1730 0x7fdbc4159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 --2- 172.21.15.86:0/2283668725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc406d180 0x7fdbc41543e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.949+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 >> 172.21.15.86:0/2283668725 conn(0x7fdbc40b3c40 msgr2=0x7fdbc40b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.953+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 shutdown_connections 2024-03-22T04:10:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:37.953+0000 7fdbca800640 1 -- 172.21.15.86:0/2283668725 wait complete. 2024-03-22T04:10:37.986 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-22T04:10:38.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.125+0000 7f9364a00640 1 Processor -- start 2024-03-22T04:10:38.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.125+0000 7f9364a00640 1 -- start start 2024-03-22T04:10:38.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f9364a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f936015dfe0 0x7f93600afc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:38.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f9364a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f936015e3c0 0x7f936006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:38.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f9364a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9360070320 con 0x7f93600b3eb0 2024-03-22T04:10:38.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f9364a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f93600b4340 con 0x7f936015dfe0 2024-03-22T04:10:38.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f9364a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f93600b5d20 con 0x7f936015e3c0 2024-03-22T04:10:38.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f93600b3eb0 0x7f9360072eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34572/0 (socket says 172.21.15.86:34572) 2024-03-22T04:10:38.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935ea00640 1 -- 172.21.15.86:0/945408963 learned_addr learned my addr 172.21.15.86:0/945408963 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:38.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935fe00640 1 --2- 172.21.15.86:0/945408963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f936015e3c0 0x7f936006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:38.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935f400640 1 --2- 172.21.15.86:0/945408963 >> v2:172.21.15.196:3300/0 conn(0x7f936015dfe0 0x7f93600afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:38.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935e000640 1 -- 172.21.15.86:0/945408963 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160772120 0 0) 0x7f9360070320 con 0x7f93600b3eb0 2024-03-22T04:10:38.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935e000640 1 -- 172.21.15.86:0/945408963 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9344003530 con 0x7f93600b3eb0 2024-03-22T04:10:38.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935fe00640 1 -- 172.21.15.86:0/945408963 >> v2:172.21.15.196:3300/0 conn(0x7f936015dfe0 msgr2=0x7f93600afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:38.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935fe00640 1 --2- 172.21.15.86:0/945408963 >> v2:172.21.15.196:3300/0 conn(0x7f936015dfe0 0x7f93600afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:38.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935fe00640 1 -- 172.21.15.86:0/945408963 >> v1:172.21.15.86:6789/0 conn(0x7f93600b3eb0 legacy=0x7f9360072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:38.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935fe00640 1 -- 172.21.15.86:0/945408963 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93600b6000 con 0x7f936015e3c0 2024-03-22T04:10:38.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.129+0000 7f935f400640 1 --2- 172.21.15.86:0/945408963 >> v2:172.21.15.196:3300/0 conn(0x7f936015dfe0 0x7f93600afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:38.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f935fe00640 1 --2- 172.21.15.86:0/945408963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f936015e3c0 0x7f936006ede0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93580097c0 tx=0x7f935801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c5ba823b9a1d23 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:38.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f935e000640 1 -- 172.21.15.86:0/945408963 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580b0070 con 0x7f936015e3c0 2024-03-22T04:10:38.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f935e000640 1 -- 172.21.15.86:0/945408963 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93580ab600 con 0x7f936015e3c0 2024-03-22T04:10:38.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f935e000640 1 -- 172.21.15.86:0/945408963 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580ab9d0 con 0x7f936015e3c0 2024-03-22T04:10:38.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 -- 172.21.15.86:0/945408963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f936015e3c0 msgr2=0x7f936006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:38.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 --2- 172.21.15.86:0/945408963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f936015e3c0 0x7f936006ede0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93580097c0 tx=0x7f935801c410 comp rx=0 tx=0).stop 2024-03-22T04:10:38.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 -- 172.21.15.86:0/945408963 shutdown_connections 2024-03-22T04:10:38.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 --2- 172.21.15.86:0/945408963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f936015e3c0 0x7f936006ede0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:38.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 --2- 172.21.15.86:0/945408963 >> v2:172.21.15.196:3300/0 conn(0x7f936015dfe0 0x7f93600afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:38.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 -- 172.21.15.86:0/945408963 >> 172.21.15.86:0/945408963 conn(0x7f93600b2fb0 msgr2=0x7f93600b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:38.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 -- 172.21.15.86:0/945408963 shutdown_connections 2024-03-22T04:10:38.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 -- 172.21.15.86:0/945408963 wait complete. 2024-03-22T04:10:38.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 Processor -- start 2024-03-22T04:10:38.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.133+0000 7f9364a00640 1 -- start start 2024-03-22T04:10:38.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f936015dfe0 0x7f936014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:38.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f936015e3c0 0x7f93601519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:38.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9360151f20 0x7f93600ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:38.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f936016af40 con 0x7f936015e3c0 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f936016adc0 con 0x7f936015dfe0 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f936016b0c0 con 0x7f9360151f20 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f936015e3c0 0x7f93601519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f936015dfe0 0x7f936014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f936015e3c0 0x7f93601519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35048/0 (socket says 172.21.15.86:35048) 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 -- 172.21.15.86:0/3253894576 learned_addr learned my addr 172.21.15.86:0/3253894576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935fe00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9360151f20 0x7f93600ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 -- 172.21.15.86:0/3253894576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9360151f20 msgr2=0x7f93600ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9360151f20 0x7f93600ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 -- 172.21.15.86:0/3253894576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f936015dfe0 msgr2=0x7f936014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f936015dfe0 0x7f936014fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 -- 172.21.15.86:0/3253894576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9360150110 con 0x7f936015e3c0 2024-03-22T04:10:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f935ea00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f936015e3c0 0x7f93601519e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f934c00c390 tx=0x7f934c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9357400640 1 -- 172.21.15.86:0/3253894576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c017070 con 0x7f936015e3c0 2024-03-22T04:10:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9357400640 1 -- 172.21.15.86:0/3253894576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c005d10 con 0x7f936015e3c0 2024-03-22T04:10:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9360161730 con 0x7f936015e3c0 2024-03-22T04:10:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9357400640 1 -- 172.21.15.86:0/3253894576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c007310 con 0x7f936015e3c0 2024-03-22T04:10:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93601528a0 con 0x7f936015e3c0 2024-03-22T04:10:38.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9360074470 con 0x7f936015e3c0 2024-03-22T04:10:38.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.137+0000 7f9357400640 1 -- 172.21.15.86:0/3253894576 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f934c01a070 con 0x7f936015e3c0 2024-03-22T04:10:38.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.141+0000 7f9357400640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9340067b50 0x7f934006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:38.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.141+0000 7f9357400640 1 -- 172.21.15.86:0/3253894576 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f934c085600 con 0x7f936015e3c0 2024-03-22T04:10:38.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.141+0000 7f935f400640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9340067b50 0x7f934006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:38.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.141+0000 7f935f400640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9340067b50 0x7f934006a010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9350009bd0 tx=0x7f935000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.145+0000 7f9357400640 1 -- 172.21.15.86:0/3253894576 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f934c0556c0 con 0x7f936015e3c0 2024-03-22T04:10:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.297+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f9360154c40 con 0x7f936015e3c0 2024-03-22T04:10:38.312 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:10:38.309+0000 7fc85b600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:10:38.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.949+0000 7f9357400640 1 -- 172.21.15.86:0/3253894576 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v26) ==== 169+0+0 (secure 0 0 0) 0x7f934c059760 con 0x7f936015e3c0 2024-03-22T04:10:38.951 INFO:teuthology.orchestra.run.smithi086.stderr:set pool 4 allow_ec_overwrites to true 2024-03-22T04:10:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9340067b50 msgr2=0x7f934006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9340067b50 0x7f934006a010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9350009bd0 tx=0x7f935000b040 comp rx=0 tx=0).stop 2024-03-22T04:10:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f936015e3c0 msgr2=0x7f93601519e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f936015e3c0 0x7f93601519e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f934c00c390 tx=0x7f934c00c850 comp rx=0 tx=0).stop 2024-03-22T04:10:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 shutdown_connections 2024-03-22T04:10:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9340067b50 0x7f934006a010 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9350009bd0 tx=0x7f935000b040 comp rx=0 tx=0).stop 2024-03-22T04:10:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9360151f20 0x7f93600ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f936015e3c0 0x7f93601519e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 --2- 172.21.15.86:0/3253894576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f936015dfe0 0x7f936014fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.953+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 >> 172.21.15.86:0/3253894576 conn(0x7f93600b2fb0 msgr2=0x7f9360155160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.957+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 shutdown_connections 2024-03-22T04:10:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:38.957+0000 7f9364a00640 1 -- 172.21.15.86:0/3253894576 wait complete. 2024-03-22T04:10:38.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-22T04:10:39.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.133+0000 7f137b200640 1 Processor -- start 2024-03-22T04:10:39.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.133+0000 7f137b200640 1 -- start start 2024-03-22T04:10:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.133+0000 7f137b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f137406e7e0 0x7f137406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.133+0000 7f137b200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1374072b00 0x7f13740b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.133+0000 7f137b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137415c770 con 0x7f1374072650 2024-03-22T04:10:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.133+0000 7f137b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1374071350 con 0x7f137406e7e0 2024-03-22T04:10:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.133+0000 7f137b200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f137406f100 con 0x7f1374072b00 2024-03-22T04:10:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f137406e7e0 0x7f137406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f137406e7e0 0x7f137406ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35064/0 (socket says 172.21.15.86:35064) 2024-03-22T04:10:39.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373e00640 1 -- 172.21.15.86:0/2353612517 learned_addr learned my addr 172.21.15.86:0/2353612517 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1379200640 1 --2- 172.21.15.86:0/2353612517 >> v2:172.21.15.196:3300/0 conn(0x7f1374072b00 0x7f13740b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373e00640 1 -- 172.21.15.86:0/2353612517 >> v2:172.21.15.196:3300/0 conn(0x7f1374072b00 msgr2=0x7f13740b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373e00640 1 --2- 172.21.15.86:0/2353612517 >> v2:172.21.15.196:3300/0 conn(0x7f1374072b00 0x7f13740b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373e00640 1 -- 172.21.15.86:0/2353612517 >> v1:172.21.15.86:6789/0 conn(0x7f1374072650 legacy=0x7f13740b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:39.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373e00640 1 -- 172.21.15.86:0/2353612517 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1374073150 con 0x7f137406e7e0 2024-03-22T04:10:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373e00640 1 --2- 172.21.15.86:0/2353612517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f137406ebc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f137406f7f0 tx=0x7f136401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9d88741660946d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373400640 1 -- 172.21.15.86:0/2353612517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640b2070 con 0x7f137406e7e0 2024-03-22T04:10:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373400640 1 -- 172.21.15.86:0/2353612517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13640ad680 con 0x7f137406e7e0 2024-03-22T04:10:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.137+0000 7f1373400640 1 -- 172.21.15.86:0/2353612517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640ada50 con 0x7f137406e7e0 2024-03-22T04:10:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- 172.21.15.86:0/2353612517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 msgr2=0x7f137406ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 --2- 172.21.15.86:0/2353612517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f137406ebc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f137406f7f0 tx=0x7f136401c490 comp rx=0 tx=0).stop 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- 172.21.15.86:0/2353612517 shutdown_connections 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 --2- 172.21.15.86:0/2353612517 >> v2:172.21.15.196:3300/0 conn(0x7f1374072b00 0x7f13740b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 --2- 172.21.15.86:0/2353612517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f137406ebc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- 172.21.15.86:0/2353612517 >> 172.21.15.86:0/2353612517 conn(0x7f13740cc940 msgr2=0x7f13740ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- 172.21.15.86:0/2353612517 shutdown_connections 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- 172.21.15.86:0/2353612517 wait complete. 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 Processor -- start 2024-03-22T04:10:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- start start 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f13740cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1374072650 0x7f1374163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1374072b00 0x7f13741f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1374072080 con 0x7f137406e7e0 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13740ce930 con 0x7f1374072650 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f137b200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f137415c9f0 con 0x7f1374072b00 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f13740cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f13740cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35074/0 (socket says 172.21.15.86:35074) 2024-03-22T04:10:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 -- 172.21.15.86:0/3691175384 learned_addr learned my addr 172.21.15.86:0/3691175384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f136b400640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1374072650 0x7f1374163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1379200640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1374072b00 0x7f13741f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 -- 172.21.15.86:0/3691175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1374072b00 msgr2=0x7f13741f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1374072b00 0x7f13741f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 -- 172.21.15.86:0/3691175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1374072650 msgr2=0x7f1374163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1374072650 0x7f1374163ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 -- 172.21.15.86:0/3691175384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137415ccd0 con 0x7f137406e7e0 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.141+0000 7f1373e00640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f13740cb830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f136401a0c0 tx=0x7f136401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f1370c00640 1 -- 172.21.15.86:0/3691175384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640bc020 con 0x7f137406e7e0 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f1370c00640 1 -- 172.21.15.86:0/3691175384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13640adc70 con 0x7f137406e7e0 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f1370c00640 1 -- 172.21.15.86:0/3691175384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640b2040 con 0x7f137406e7e0 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137415cfb0 con 0x7f137406e7e0 2024-03-22T04:10:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1374164330 con 0x7f137406e7e0 2024-03-22T04:10:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1374074470 con 0x7f137406e7e0 2024-03-22T04:10:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f1370c00640 1 -- 172.21.15.86:0/3691175384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f13640bf020 con 0x7f137406e7e0 2024-03-22T04:10:39.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f1370c00640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1350067b20 0x7f1350069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:39.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f1370c00640 1 -- 172.21.15.86:0/3691175384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6500+0+0 (secure 0 0 0) 0x7f13641202a0 con 0x7f137406e7e0 2024-03-22T04:10:39.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.145+0000 7f136b400640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1350067b20 0x7f1350069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:39.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.149+0000 7f136b400640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1350067b20 0x7f1350069fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f13741550e0 tx=0x7f135c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:39.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.149+0000 7f1370c00640 1 -- 172.21.15.86:0/3691175384 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f13640c6030 con 0x7f137406e7e0 2024-03-22T04:10:39.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.341+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f1374072200 con 0x7f137406e7e0 2024-03-22T04:10:39.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f1370c00640 1 -- 172.21.15.86:0/3691175384 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f1364016110 con 0x7f137406e7e0 2024-03-22T04:10:39.967 INFO:teuthology.orchestra.run.smithi086.stderr:added data pool 4 to fsmap 2024-03-22T04:10:39.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1350067b20 msgr2=0x7f1350069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1350067b20 0x7f1350069fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f13741550e0 tx=0x7f135c009040 comp rx=0 tx=0).stop 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 msgr2=0x7f13740cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f13740cb830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f136401a0c0 tx=0x7f136401acd0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 shutdown_connections 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1350067b20 0x7f1350069fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1374072650 0x7f1374163ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1374072b00 0x7f13741f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 --2- 172.21.15.86:0/3691175384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137406e7e0 0x7f13740cb830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 >> 172.21.15.86:0/3691175384 conn(0x7f13740cc940 msgr2=0x7f137415a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 shutdown_connections 2024-03-22T04:10:39.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:39.965+0000 7f137b200640 1 -- 172.21.15.86:0/3691175384 wait complete. 2024-03-22T04:10:39.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:10:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.121+0000 7f4b99800640 1 Processor -- start 2024-03-22T04:10:40.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.121+0000 7f4b99800640 1 -- start start 2024-03-22T04:10:40.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b99800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4b940ad710 0x7f4b94142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b99800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9415c7c0 con 0x7f4b940ad330 2024-03-22T04:10:40.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b99800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9406dea0 con 0x7f4b9406ef40 2024-03-22T04:10:40.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b99800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4b9406e180 con 0x7f4b940ad710 2024-03-22T04:10:40.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b92a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4b940ad330 0x7f4b9413fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34590/0 (socket says 172.21.15.86:34590) 2024-03-22T04:10:40.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b92a00640 1 -- 172.21.15.86:0/1605275236 learned_addr learned my addr 172.21.15.86:0/1605275236 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:40.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b93400640 1 --2- 172.21.15.86:0/1605275236 >> v2:172.21.15.196:3300/0 conn(0x7f4b940ad710 0x7f4b94142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b91600640 1 -- 172.21.15.86:0/1605275236 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726395667 0 0) 0x7f4b9415c7c0 con 0x7f4b940ad330 2024-03-22T04:10:40.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b91600640 1 -- 172.21.15.86:0/1605275236 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b78003530 con 0x7f4b940ad330 2024-03-22T04:10:40.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b91600640 1 -- 172.21.15.86:0/1605275236 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 936588730 0 0) 0x7f4b9406dea0 con 0x7f4b9406ef40 2024-03-22T04:10:40.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b91600640 1 -- 172.21.15.86:0/1605275236 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b9415c7c0 con 0x7f4b9406ef40 2024-03-22T04:10:40.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b93400640 1 -- 172.21.15.86:0/1605275236 >> v1:172.21.15.196:6789/0 conn(0x7f4b9406ef40 legacy=0x7f4b941402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b93400640 1 -- 172.21.15.86:0/1605275236 >> v1:172.21.15.86:6789/0 conn(0x7f4b940ad330 legacy=0x7f4b9413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.125+0000 7f4b93400640 1 -- 172.21.15.86:0/1605275236 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9406e460 con 0x7f4b940ad710 2024-03-22T04:10:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b93400640 1 --2- 172.21.15.86:0/1605275236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad710 0x7f4b94142690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0097c0 tx=0x7f4b8c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=119c1c7679b13178 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b91600640 1 -- 172.21.15.86:0/1605275236 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0aa5f0 con 0x7f4b940ad710 2024-03-22T04:10:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b91600640 1 -- 172.21.15.86:0/1605275236 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8c0af070 con 0x7f4b940ad710 2024-03-22T04:10:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b91600640 1 -- 172.21.15.86:0/1605275236 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0aa9b0 con 0x7f4b940ad710 2024-03-22T04:10:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 -- 172.21.15.86:0/1605275236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad710 msgr2=0x7f4b94142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 --2- 172.21.15.86:0/1605275236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad710 0x7f4b94142690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0097c0 tx=0x7f4b8c01c470 comp rx=0 tx=0).stop 2024-03-22T04:10:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 -- 172.21.15.86:0/1605275236 shutdown_connections 2024-03-22T04:10:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 --2- 172.21.15.86:0/1605275236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad710 0x7f4b94142690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 -- 172.21.15.86:0/1605275236 >> 172.21.15.86:0/1605275236 conn(0x7f4b940b1df0 msgr2=0x7f4b940b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 -- 172.21.15.86:0/1605275236 shutdown_connections 2024-03-22T04:10:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 -- 172.21.15.86:0/1605275236 wait complete. 2024-03-22T04:10:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 Processor -- start 2024-03-22T04:10:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.129+0000 7f4b99800640 1 -- start start 2024-03-22T04:10:40.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b9406ef40 0x7f4b9413f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad330 0x7f4b9415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b940ad710 0x7f4b94157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9406db10 con 0x7f4b940ad710 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941409d0 con 0x7f4b940ad330 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9416a780 con 0x7f4b9406ef40 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b940ad710 0x7f4b94157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b940ad710 0x7f4b94157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35086/0 (socket says 172.21.15.86:35086) 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b92a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b9406ef40 0x7f4b9413f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 -- 172.21.15.86:0/1331978344 learned_addr learned my addr 172.21.15.86:0/1331978344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b92000640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad330 0x7f4b9415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 -- 172.21.15.86:0/1331978344 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b9406ef40 msgr2=0x7f4b9413f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b9406ef40 0x7f4b9413f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 -- 172.21.15.86:0/1331978344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad330 msgr2=0x7f4b9415ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad330 0x7f4b9415ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 -- 172.21.15.86:0/1331978344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b94165590 con 0x7f4b940ad710 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b93400640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b940ad710 0x7f4b94157210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c01a0a0 tx=0x7f4b8c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b8aa00640 1 -- 172.21.15.86:0/1331978344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0be410 con 0x7f4b940ad710 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b8aa00640 1 -- 172.21.15.86:0/1331978344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8c0be5b0 con 0x7f4b940ad710 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b8aa00640 1 -- 172.21.15.86:0/1331978344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0be960 con 0x7f4b940ad710 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9413fbe0 con 0x7f4b940ad710 2024-03-22T04:10:40.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b94158760 con 0x7f4b940ad710 2024-03-22T04:10:40.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b94068650 con 0x7f4b940ad710 2024-03-22T04:10:40.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.133+0000 7f4b8aa00640 1 -- 172.21.15.86:0/1331978344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4b8c0bc020 con 0x7f4b940ad710 2024-03-22T04:10:40.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.137+0000 7f4b8aa00640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4b74067b20 0x7f4b74069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.137+0000 7f4b8aa00640 1 -- 172.21.15.86:0/1331978344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4b8c120660 con 0x7f4b940ad710 2024-03-22T04:10:40.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.137+0000 7f4b92a00640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4b74067b20 0x7f4b74069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.137+0000 7f4b92a00640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4b74067b20 0x7f4b74069fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4b840012f0 tx=0x7f4b84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.141+0000 7f4b8aa00640 1 -- 172.21.15.86:0/1331978344 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4b9413fbe0 con 0x7f4b940ad710 2024-03-22T04:10:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.325+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b9415c480 con 0x7f4b940ad710 2024-03-22T04:10:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.325+0000 7f4b8aa00640 1 -- 172.21.15.86:0/1331978344 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5179 (secure 0 0 0) 0x7f4b94165590 con 0x7f4b940ad710 2024-03-22T04:10:40.329 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:40.330 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:39.345095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:10:40.330 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-03-22T04:10:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4b74067b20 msgr2=0x7f4b74069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4b74067b20 0x7f4b74069fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4b840012f0 tx=0x7f4b84009040 comp rx=0 tx=0).stop 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b940ad710 msgr2=0x7f4b94157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b940ad710 0x7f4b94157210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c01a0a0 tx=0x7f4b8c01acb0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 shutdown_connections 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4b74067b20 0x7f4b74069fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b940ad330 0x7f4b9415ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b940ad710 0x7f4b94157210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 --2- 172.21.15.86:0/1331978344 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b9406ef40 0x7f4b9413f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 >> 172.21.15.86:0/1331978344 conn(0x7f4b940b1df0 msgr2=0x7f4b94141120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 shutdown_connections 2024-03-22T04:10:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.329+0000 7f4b99800640 1 -- 172.21.15.86:0/1331978344 wait complete. 2024-03-22T04:10:40.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.481+0000 7fed33600640 1 Processor -- start 2024-03-22T04:10:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.481+0000 7fed33600640 1 -- start start 2024-03-22T04:10:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed33600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed2c0730d0 0x7fed2c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed33600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fed2c0cc3e0 0x7fed2c0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed33600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c071920 con 0x7fed2c15b3f0 2024-03-22T04:10:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed33600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fed2c0716d0 con 0x7fed2c0730d0 2024-03-22T04:10:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed33600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fed2c06d990 con 0x7fed2c0cc3e0 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed2be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed2c0730d0 0x7fed2c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed30c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fed2c15b3f0 0x7fed2c0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:55926/0 (socket says 172.21.15.86:55926) 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed30c00640 1 -- 172.21.15.86:0/397851627 learned_addr learned my addr 172.21.15.86:0/397851627 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed31600640 1 --2- 172.21.15.86:0/397851627 >> v2:172.21.15.196:3300/0 conn(0x7fed2c0cc3e0 0x7fed2c0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed2b400640 1 -- 172.21.15.86:0/397851627 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3878757035 0 0) 0x7fed2c071920 con 0x7fed2c15b3f0 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed31600640 1 -- 172.21.15.86:0/397851627 >> v2:172.21.15.86:3300/0 conn(0x7fed2c0730d0 msgr2=0x7fed2c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed31600640 1 --2- 172.21.15.86:0/397851627 >> v2:172.21.15.86:3300/0 conn(0x7fed2c0730d0 0x7fed2c0ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed31600640 1 -- 172.21.15.86:0/397851627 >> v1:172.21.15.196:6789/0 conn(0x7fed2c15b3f0 legacy=0x7fed2c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed31600640 1 -- 172.21.15.86:0/397851627 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c071c00 con 0x7fed2c0cc3e0 2024-03-22T04:10:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.485+0000 7fed2be00640 1 --2- 172.21.15.86:0/397851627 >> v2:172.21.15.86:3300/0 conn(0x7fed2c0730d0 0x7fed2c0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:40.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed31600640 1 --2- 172.21.15.86:0/397851627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0cc3e0 0x7fed2c0ae370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed2c073b20 tx=0x7fed2401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ea7aa4ec89e5afe server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed2b400640 1 -- 172.21.15.86:0/397851627 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240b0070 con 0x7fed2c0cc3e0 2024-03-22T04:10:40.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed2b400640 1 -- 172.21.15.86:0/397851627 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed240ab600 con 0x7fed2c0cc3e0 2024-03-22T04:10:40.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed2b400640 1 -- 172.21.15.86:0/397851627 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240ab9d0 con 0x7fed2c0cc3e0 2024-03-22T04:10:40.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 -- 172.21.15.86:0/397851627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0cc3e0 msgr2=0x7fed2c0ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 --2- 172.21.15.86:0/397851627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0cc3e0 0x7fed2c0ae370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed2c073b20 tx=0x7fed2401c480 comp rx=0 tx=0).stop 2024-03-22T04:10:40.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 -- 172.21.15.86:0/397851627 shutdown_connections 2024-03-22T04:10:40.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 --2- 172.21.15.86:0/397851627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0cc3e0 0x7fed2c0ae370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 --2- 172.21.15.86:0/397851627 >> v2:172.21.15.86:3300/0 conn(0x7fed2c0730d0 0x7fed2c0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 -- 172.21.15.86:0/397851627 >> 172.21.15.86:0/397851627 conn(0x7fed2c0b1df0 msgr2=0x7fed2c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:40.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 -- 172.21.15.86:0/397851627 shutdown_connections 2024-03-22T04:10:40.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 -- 172.21.15.86:0/397851627 wait complete. 2024-03-22T04:10:40.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 Processor -- start 2024-03-22T04:10:40.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.489+0000 7fed33600640 1 -- start start 2024-03-22T04:10:40.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0730d0 0x7fed2c159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c0cc3e0 0x7fed2c0ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed2c15b3f0 0x7fed2c157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c06e110 con 0x7fed2c15b3f0 2024-03-22T04:10:40.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c06df90 con 0x7fed2c0730d0 2024-03-22T04:10:40.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c06e290 con 0x7fed2c0cc3e0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed2c15b3f0 0x7fed2c157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed2c15b3f0 0x7fed2c157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35098/0 (socket says 172.21.15.86:35098) 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 -- 172.21.15.86:0/3475541399 learned_addr learned my addr 172.21.15.86:0/3475541399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed30c00640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0730d0 0x7fed2c159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed2be00640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c0cc3e0 0x7fed2c0ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 -- 172.21.15.86:0/3475541399 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c0cc3e0 msgr2=0x7fed2c0ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c0cc3e0 0x7fed2c0ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 -- 172.21.15.86:0/3475541399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0730d0 msgr2=0x7fed2c159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0730d0 0x7fed2c159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 -- 172.21.15.86:0/3475541399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c157fa0 con 0x7fed2c15b3f0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed31600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed2c15b3f0 0x7fed2c157550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed2401a0b0 tx=0x7fed24019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed28c00640 1 -- 172.21.15.86:0/3475541399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240b9020 con 0x7fed2c15b3f0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed28c00640 1 -- 172.21.15.86:0/3475541399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed240b5480 con 0x7fed2c15b3f0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed28c00640 1 -- 172.21.15.86:0/3475541399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240b0040 con 0x7fed2c15b3f0 2024-03-22T04:10:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c158280 con 0x7fed2c15b3f0 2024-03-22T04:10:40.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c158b20 con 0x7fed2c15b3f0 2024-03-22T04:10:40.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c068650 con 0x7fed2c15b3f0 2024-03-22T04:10:40.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.493+0000 7fed28c00640 1 -- 172.21.15.86:0/3475541399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fed240bc020 con 0x7fed2c15b3f0 2024-03-22T04:10:40.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.497+0000 7fed28c00640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed0c067b20 0x7fed0c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.497+0000 7fed28c00640 1 -- 172.21.15.86:0/3475541399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fed2411fe10 con 0x7fed2c15b3f0 2024-03-22T04:10:40.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.497+0000 7fed30c00640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed0c067b20 0x7fed0c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.497+0000 7fed30c00640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed0c067b20 0x7fed0c069fe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0012f0 tx=0x7fed1c004d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.501+0000 7fed28c00640 1 -- 172.21.15.86:0/3475541399 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fed24123030 con 0x7fed2c15b3f0 2024-03-22T04:10:40.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.657+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed2c157230 con 0x7fed2c15b3f0 2024-03-22T04:10:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.657+0000 7fed28c00640 1 -- 172.21.15.86:0/3475541399 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7fed24016070 con 0x7fed2c15b3f0 2024-03-22T04:10:40.661 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:40.661 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:40.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed0c067b20 msgr2=0x7fed0c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed0c067b20 0x7fed0c069fe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0012f0 tx=0x7fed1c004d50 comp rx=0 tx=0).stop 2024-03-22T04:10:40.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed2c15b3f0 msgr2=0x7fed2c157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed2c15b3f0 0x7fed2c157550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed2401a0b0 tx=0x7fed24019220 comp rx=0 tx=0).stop 2024-03-22T04:10:40.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 shutdown_connections 2024-03-22T04:10:40.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed0c067b20 0x7fed0c069fe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed2c15b3f0 0x7fed2c157550 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c0cc3e0 0x7fed2c0ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 --2- 172.21.15.86:0/3475541399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0730d0 0x7fed2c159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 >> 172.21.15.86:0/3475541399 conn(0x7fed2c0b1df0 msgr2=0x7fed2c152e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:40.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 shutdown_connections 2024-03-22T04:10:40.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.661+0000 7fed33600640 1 -- 172.21.15.86:0/3475541399 wait complete. 2024-03-22T04:10:40.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.813+0000 7fba1e600640 1 Processor -- start 2024-03-22T04:10:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.813+0000 7fba1e600640 1 -- start start 2024-03-22T04:10:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba1e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba1806fc60 0x7fba180b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba1e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba180d1360 0x7fba181456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba1e600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba180d1810 con 0x7fba18070040 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba1e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fba180ce270 con 0x7fba180d1360 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba1e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fba180d1af0 con 0x7fba1806fc60 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba17e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba180d1360 0x7fba181456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba17e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba180d1360 0x7fba181456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35104/0 (socket says 172.21.15.86:35104) 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba16a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fba18070040 0x7fba180d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34594/0 (socket says 172.21.15.86:34594) 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba17e00640 1 -- 172.21.15.86:0/3656245327 learned_addr learned my addr 172.21.15.86:0/3656245327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba17400640 1 --2- 172.21.15.86:0/3656245327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba1806fc60 0x7fba180b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba17e00640 1 -- 172.21.15.86:0/3656245327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba1806fc60 msgr2=0x7fba180b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba17e00640 1 --2- 172.21.15.86:0/3656245327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba1806fc60 0x7fba180b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba17e00640 1 -- 172.21.15.86:0/3656245327 >> v1:172.21.15.86:6789/0 conn(0x7fba18070040 legacy=0x7fba180d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba16000640 1 -- 172.21.15.86:0/3656245327 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518724313 0 0) 0x7fba180d1810 con 0x7fba18070040 2024-03-22T04:10:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.817+0000 7fba17e00640 1 -- 172.21.15.86:0/3656245327 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba180d1dd0 con 0x7fba180d1360 2024-03-22T04:10:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba17e00640 1 --2- 172.21.15.86:0/3656245327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba181456f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba1001d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71aedf5d58db2662 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba16000640 1 -- 172.21.15.86:0/3656245327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100a5850 con 0x7fba180d1360 2024-03-22T04:10:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba16000640 1 -- 172.21.15.86:0/3656245327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba100a59f0 con 0x7fba180d1360 2024-03-22T04:10:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba16000640 1 -- 172.21.15.86:0/3656245327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100a5dc0 con 0x7fba180d1360 2024-03-22T04:10:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- 172.21.15.86:0/3656245327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 msgr2=0x7fba181456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 --2- 172.21.15.86:0/3656245327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba181456f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba1001d3d0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- 172.21.15.86:0/3656245327 shutdown_connections 2024-03-22T04:10:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 --2- 172.21.15.86:0/3656245327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba181456f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 --2- 172.21.15.86:0/3656245327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba1806fc60 0x7fba180b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- 172.21.15.86:0/3656245327 >> 172.21.15.86:0/3656245327 conn(0x7fba180b75b0 msgr2=0x7fba180b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- 172.21.15.86:0/3656245327 shutdown_connections 2024-03-22T04:10:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- 172.21.15.86:0/3656245327 wait complete. 2024-03-22T04:10:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 Processor -- start 2024-03-22T04:10:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- start start 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1806fc60 0x7fba1813bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba18070040 0x7fba181f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba18166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba180cf370 con 0x7fba180d1360 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba180cf1f0 con 0x7fba1806fc60 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba1e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fba180cf4f0 con 0x7fba18070040 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba18166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba18166250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35116/0 (socket says 172.21.15.86:35116) 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.821+0000 7fba17e00640 1 -- 172.21.15.86:0/616493267 learned_addr learned my addr 172.21.15.86:0/616493267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba16a00640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba18070040 0x7fba181f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba17400640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1806fc60 0x7fba1813bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba17e00640 1 -- 172.21.15.86:0/616493267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba18070040 msgr2=0x7fba181f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba17e00640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba18070040 0x7fba181f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba17e00640 1 -- 172.21.15.86:0/616493267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1806fc60 msgr2=0x7fba1813bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba17e00640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1806fc60 0x7fba1813bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba17e00640 1 -- 172.21.15.86:0/616493267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1813c000 con 0x7fba180d1360 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba17e00640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba18166250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba10019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba0f400640 1 -- 172.21.15.86:0/616493267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100bc030 con 0x7fba180d1360 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba0f400640 1 -- 172.21.15.86:0/616493267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba100b8470 con 0x7fba180d1360 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1812b2e0 con 0x7fba180d1360 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba0f400640 1 -- 172.21.15.86:0/616493267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100b8880 con 0x7fba180d1360 2024-03-22T04:10:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1812b8a0 con 0x7fba180d1360 2024-03-22T04:10:40.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba18068650 con 0x7fba180d1360 2024-03-22T04:10:40.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.825+0000 7fba0f400640 1 -- 172.21.15.86:0/616493267 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fba100bf070 con 0x7fba180d1360 2024-03-22T04:10:40.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.829+0000 7fba0f400640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9f8067b20 0x7fb9f8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:40.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.829+0000 7fba0f400640 1 -- 172.21.15.86:0/616493267 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fba10120f60 con 0x7fba180d1360 2024-03-22T04:10:40.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.829+0000 7fba17400640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9f8067b20 0x7fb9f8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:40.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.829+0000 7fba17400640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9f8067b20 0x7fb9f8069fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fba18166f30 tx=0x7fba08009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:40.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.833+0000 7fba0f400640 1 -- 172.21.15.86:0/616493267 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fba100c6070 con 0x7fba180d1360 2024-03-22T04:10:40.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.989+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba1815ffe0 con 0x7fba180d1360 2024-03-22T04:10:40.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba0f400640 1 -- 172.21.15.86:0/616493267 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7fba1813c000 con 0x7fba180d1360 2024-03-22T04:10:40.994 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9f8067b20 msgr2=0x7fb9f8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9f8067b20 0x7fb9f8069fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fba18166f30 tx=0x7fba08009190 comp rx=0 tx=0).stop 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 msgr2=0x7fba18166250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba18166250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba10019710 comp rx=0 tx=0).stop 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 shutdown_connections 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9f8067b20 0x7fb9f8069fe0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba180d1360 0x7fba18166250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba18070040 0x7fba181f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 --2- 172.21.15.86:0/616493267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba1806fc60 0x7fba1813bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 >> 172.21.15.86:0/616493267 conn(0x7fba180b75b0 msgr2=0x7fba1806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 shutdown_connections 2024-03-22T04:10:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:40.993+0000 7fba1e600640 1 -- 172.21.15.86:0/616493267 wait complete. 2024-03-22T04:10:41.031 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:10:41.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1f200640 1 Processor -- start 2024-03-22T04:10:41.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1f200640 1 -- start start 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 0x7efe20072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe2006b660 0x7efe200d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe2006bb60 con 0x7efe200cda50 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efe20146020 con 0x7efe2006b660 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe2006be40 con 0x7efe20145c40 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe2006b660 0x7efe200d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe2006b660 0x7efe200d0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35118/0 (socket says 172.21.15.86:35118) 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1d400640 1 -- 172.21.15.86:0/3021516428 learned_addr learned my addr 172.21.15.86:0/3021516428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1de00640 1 --2- 172.21.15.86:0/3021516428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 0x7efe20072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1d400640 1 -- 172.21.15.86:0/3021516428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 msgr2=0x7efe20072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1d400640 1 --2- 172.21.15.86:0/3021516428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 0x7efe20072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1d400640 1 -- 172.21.15.86:0/3021516428 >> v1:172.21.15.196:6789/0 conn(0x7efe200cda50 legacy=0x7efe20070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.165+0000 7efe1d400640 1 -- 172.21.15.86:0/3021516428 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2006fb00 con 0x7efe2006b660 2024-03-22T04:10:41.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1d400640 1 --2- 172.21.15.86:0/3021516428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe200d0750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009870 tx=0x7efe0c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a257f319eef720e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:41.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1ca00640 1 -- 172.21.15.86:0/3021516428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0aa430 con 0x7efe2006b660 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1ca00640 1 -- 172.21.15.86:0/3021516428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c0af070 con 0x7efe2006b660 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1ca00640 1 -- 172.21.15.86:0/3021516428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0aa7f0 con 0x7efe2006b660 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 -- 172.21.15.86:0/3021516428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 msgr2=0x7efe200d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 --2- 172.21.15.86:0/3021516428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe200d0750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009870 tx=0x7efe0c01c530 comp rx=0 tx=0).stop 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 -- 172.21.15.86:0/3021516428 shutdown_connections 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 --2- 172.21.15.86:0/3021516428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe200d0750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 --2- 172.21.15.86:0/3021516428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 0x7efe20072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 -- 172.21.15.86:0/3021516428 >> 172.21.15.86:0/3021516428 conn(0x7efe200d1000 msgr2=0x7efe200d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 -- 172.21.15.86:0/3021516428 shutdown_connections 2024-03-22T04:10:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 -- 172.21.15.86:0/3021516428 wait complete. 2024-03-22T04:10:41.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 Processor -- start 2024-03-22T04:10:41.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.169+0000 7efe1f200640 1 -- start start 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe2015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe200cda50 0x7efe201f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 0x7efe200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe200d0e10 con 0x7efe2006b660 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe200d0c90 con 0x7efe200cda50 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe20169180 con 0x7efe20145c40 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe2015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe2015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35128/0 (socket says 172.21.15.86:35128) 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 -- 172.21.15.86:0/1685489853 learned_addr learned my addr 172.21.15.86:0/1685489853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1d400640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe200cda50 0x7efe201f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1e800640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 0x7efe200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 -- 172.21.15.86:0/1685489853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 msgr2=0x7efe200b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 0x7efe200b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 -- 172.21.15.86:0/1685489853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe200cda50 msgr2=0x7efe201f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe200cda50 0x7efe201f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 -- 172.21.15.86:0/1685489853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe201f8570 con 0x7efe2006b660 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1de00640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe2015b2e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efe1000d5f0 tx=0x7efe1000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe16000640 1 -- 172.21.15.86:0/1685489853 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe100125f0 con 0x7efe2006b660 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe16000640 1 -- 172.21.15.86:0/1685489853 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe10019070 con 0x7efe2006b660 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe16000640 1 -- 172.21.15.86:0/1685489853 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe100129b0 con 0x7efe2006b660 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe201f8850 con 0x7efe2006b660 2024-03-22T04:10:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe201f8e10 con 0x7efe2006b660 2024-03-22T04:10:41.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe20074470 con 0x7efe2006b660 2024-03-22T04:10:41.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.173+0000 7efe16000640 1 -- 172.21.15.86:0/1685489853 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efe10012b50 con 0x7efe2006b660 2024-03-22T04:10:41.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.177+0000 7efe16000640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe00067b20 0x7efe00069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.177+0000 7efe16000640 1 -- 172.21.15.86:0/1685489853 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7efe10086600 con 0x7efe2006b660 2024-03-22T04:10:41.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.177+0000 7efe1d400640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe00067b20 0x7efe00069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.177+0000 7efe1d400640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe00067b20 0x7efe00069fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0ae000 tx=0x7efe0c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:41.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.181+0000 7efe16000640 1 -- 172.21.15.86:0/1685489853 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efe10056c30 con 0x7efe2006b660 2024-03-22T04:10:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.365+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe2006c3b0 con 0x7efe2006b660 2024-03-22T04:10:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe16000640 1 -- 172.21.15.86:0/1685489853 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5179 (secure 0 0 0) 0x7efe10056530 con 0x7efe2006b660 2024-03-22T04:10:41.370 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:41.370 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:39.345095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:10:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-03-22T04:10:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe00067b20 msgr2=0x7efe00069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe00067b20 0x7efe00069fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0ae000 tx=0x7efe0c0a7000 comp rx=0 tx=0).stop 2024-03-22T04:10:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 msgr2=0x7efe2015b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe2015b2e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efe1000d5f0 tx=0x7efe1000dab0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 shutdown_connections 2024-03-22T04:10:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe00067b20 0x7efe00069fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe200cda50 0x7efe201f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe20145c40 0x7efe200b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 --2- 172.21.15.86:0/1685489853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe2006b660 0x7efe2015b2e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 >> 172.21.15.86:0/1685489853 conn(0x7efe200d1000 msgr2=0x7efe200b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 shutdown_connections 2024-03-22T04:10:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.369+0000 7efe1f200640 1 -- 172.21.15.86:0/1685489853 wait complete. 2024-03-22T04:10:41.391 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.521+0000 7f5b1a400640 1 Processor -- start 2024-03-22T04:10:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b1a400640 1 -- start start 2024-03-22T04:10:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b1a400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5b140730d0 0x7f5b140ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b1a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b140cc3e0 0x7f5b140ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b1a400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b14071920 con 0x7f5b1415b3f0 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b1a400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5b140716d0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b1a400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5b1406d990 con 0x7f5b140730d0 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b13e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b140cc3e0 0x7f5b140ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b13400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5b1415b3f0 0x7f5b140729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34598/0 (socket says 172.21.15.86:34598) 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b13e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b140cc3e0 0x7f5b140ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35144/0 (socket says 172.21.15.86:35144) 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b13400640 1 -- 172.21.15.86:0/541589622 learned_addr learned my addr 172.21.15.86:0/541589622 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b0aa00640 1 --2- 172.21.15.86:0/541589622 >> v2:172.21.15.196:3300/0 conn(0x7f5b140730d0 0x7f5b140ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b12a00640 1 -- 172.21.15.86:0/541589622 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204294174 0 0) 0x7f5b14071920 con 0x7f5b1415b3f0 2024-03-22T04:10:41.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b12a00640 1 -- 172.21.15.86:0/541589622 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5af8003530 con 0x7f5b1415b3f0 2024-03-22T04:10:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b13e00640 1 -- 172.21.15.86:0/541589622 >> v2:172.21.15.196:3300/0 conn(0x7f5b140730d0 msgr2=0x7f5b140ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b13e00640 1 --2- 172.21.15.86:0/541589622 >> v2:172.21.15.196:3300/0 conn(0x7f5b140730d0 0x7f5b140ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b13e00640 1 -- 172.21.15.86:0/541589622 >> v1:172.21.15.86:6789/0 conn(0x7f5b1415b3f0 legacy=0x7f5b140729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b13e00640 1 -- 172.21.15.86:0/541589622 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b14071c00 con 0x7f5b140cc3e0 2024-03-22T04:10:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.525+0000 7f5b0aa00640 1 --2- 172.21.15.86:0/541589622 >> v2:172.21.15.196:3300/0 conn(0x7f5b140730d0 0x7f5b140ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b13e00640 1 --2- 172.21.15.86:0/541589622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b140ae370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5b1406f270 tx=0x7f5b0c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e414ce029620cc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b12a00640 1 -- 172.21.15.86:0/541589622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0b2070 con 0x7f5b140cc3e0 2024-03-22T04:10:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b12a00640 1 -- 172.21.15.86:0/541589622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0a58e0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b12a00640 1 -- 172.21.15.86:0/541589622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0a5cb0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b1a400640 1 -- 172.21.15.86:0/541589622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 msgr2=0x7f5b140ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b1a400640 1 --2- 172.21.15.86:0/541589622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b140ae370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5b1406f270 tx=0x7f5b0c01c930 comp rx=0 tx=0).stop 2024-03-22T04:10:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b1a400640 1 -- 172.21.15.86:0/541589622 shutdown_connections 2024-03-22T04:10:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b1a400640 1 --2- 172.21.15.86:0/541589622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b140ae370 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b1a400640 1 --2- 172.21.15.86:0/541589622 >> v2:172.21.15.196:3300/0 conn(0x7f5b140730d0 0x7f5b140ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b1a400640 1 -- 172.21.15.86:0/541589622 >> 172.21.15.86:0/541589622 conn(0x7f5b140b1df0 msgr2=0x7f5b140b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b1a400640 1 -- 172.21.15.86:0/541589622 shutdown_connections 2024-03-22T04:10:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.529+0000 7f5b1a400640 1 -- 172.21.15.86:0/541589622 wait complete. 2024-03-22T04:10:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 Processor -- start 2024-03-22T04:10:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 -- start start 2024-03-22T04:10:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b140730d0 0x7f5b1415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b1414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b1415b3f0 0x7f5b14150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b14168580 con 0x7f5b140cc3e0 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b14168400 con 0x7f5b1415b3f0 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5b14168700 con 0x7f5b140730d0 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b1414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b1414c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35152/0 (socket says 172.21.15.86:35152) 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 -- 172.21.15.86:0/60877636 learned_addr learned my addr 172.21.15.86:0/60877636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b13400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b140730d0 0x7f5b1415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b13e00640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b1415b3f0 0x7f5b14150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 -- 172.21.15.86:0/60877636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b140730d0 msgr2=0x7f5b1415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b140730d0 0x7f5b1415dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 -- 172.21.15.86:0/60877636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b1415b3f0 msgr2=0x7f5b14150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b1415b3f0 0x7f5b14150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 -- 172.21.15.86:0/60877636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1415e1f0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0aa00640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b1414c6e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00e9b0 tx=0x7f5afc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0be00640 1 -- 172.21.15.86:0/60877636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc017070 con 0x7f5b140cc3e0 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0be00640 1 -- 172.21.15.86:0/60877636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc00c9a0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1414f3c0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.533+0000 7f5b0be00640 1 -- 172.21.15.86:0/60877636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00cd70 con 0x7f5b140cc3e0 2024-03-22T04:10:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.537+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1415d1d0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.537+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b14068650 con 0x7f5b140cc3e0 2024-03-22T04:10:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.537+0000 7f5b0be00640 1 -- 172.21.15.86:0/60877636 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5afc01a070 con 0x7f5b140cc3e0 2024-03-22T04:10:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.537+0000 7f5b0be00640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5af0067b20 0x7f5af0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.537+0000 7f5b0be00640 1 -- 172.21.15.86:0/60877636 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5afc0860f0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.537+0000 7f5b13400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5af0067b20 0x7f5af0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.537+0000 7f5b13400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5af0067b20 0x7f5af0069fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b04007fd0 tx=0x7f5b04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.541+0000 7f5b0be00640 1 -- 172.21.15.86:0/60877636 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5afc0560d0 con 0x7f5b140cc3e0 2024-03-22T04:10:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.697+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b1415a980 con 0x7f5b140cc3e0 2024-03-22T04:10:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b0be00640 1 -- 172.21.15.86:0/60877636 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7f5afc05a170 con 0x7f5b140cc3e0 2024-03-22T04:10:41.702 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:41.703 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5af0067b20 msgr2=0x7f5af0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5af0067b20 0x7f5af0069fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b04007fd0 tx=0x7f5b04009040 comp rx=0 tx=0).stop 2024-03-22T04:10:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 msgr2=0x7f5b1414c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b1414c6e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00e9b0 tx=0x7f5afc00ee70 comp rx=0 tx=0).stop 2024-03-22T04:10:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 shutdown_connections 2024-03-22T04:10:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5af0067b20 0x7f5af0069fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b1415b3f0 0x7f5b14150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b140cc3e0 0x7f5b1414c6e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 --2- 172.21.15.86:0/60877636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b140730d0 0x7f5b1415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 >> 172.21.15.86:0/60877636 conn(0x7f5b140b1df0 msgr2=0x7f5b140b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 shutdown_connections 2024-03-22T04:10:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:41.701+0000 7f5b1a400640 1 -- 172.21.15.86:0/60877636 wait complete. 2024-03-22T04:10:42.125 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:10:34.298556+0000', 'modified': '2024-03-22T04:10:39.345095+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 74171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_74171': {'gid': 74171, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/180015071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6833', 'nonce': 180015071}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 180015071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 74171, 'qdb_cluster': [74171]} 2024-03-22T04:10:42.125 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:10:42.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T04:10:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14ad000640 1 Processor -- start 2024-03-22T04:10:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14ad000640 1 -- start start 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14ad000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14a80ad330 0x7f14a815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14ad000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f14a806ef40 0x7f14a806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14ad000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a80ad710 0x7f14a813fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14ad000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14a806dc40 con 0x7f14a80ad330 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14ad000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f14a815c7c0 con 0x7f14a806ef40 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14ad000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14a806dea0 con 0x7f14a80ad710 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14a80ad330 0x7f14a815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14a80ad330 0x7f14a815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35164/0 (socket says 172.21.15.86:35164) 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a5e00640 1 -- 172.21.15.86:0/576446739 learned_addr learned my addr 172.21.15.86:0/576446739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f149d400640 1 --2- 172.21.15.86:0/576446739 >> v2:172.21.15.196:3300/0 conn(0x7f14a806ef40 0x7f14a806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a6800640 1 --2- 172.21.15.86:0/576446739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a80ad710 0x7f14a813fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a5e00640 1 -- 172.21.15.86:0/576446739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a80ad710 msgr2=0x7f14a813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a5e00640 1 --2- 172.21.15.86:0/576446739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a80ad710 0x7f14a813fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a5e00640 1 -- 172.21.15.86:0/576446739 >> v2:172.21.15.196:3300/0 conn(0x7f14a806ef40 msgr2=0x7f14a806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a5e00640 1 --2- 172.21.15.86:0/576446739 >> v2:172.21.15.196:3300/0 conn(0x7f14a806ef40 0x7f14a806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.269+0000 7f14a5e00640 1 -- 172.21.15.86:0/576446739 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a806e180 con 0x7f14a80ad330 2024-03-22T04:10:42.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14a5e00640 1 --2- 172.21.15.86:0/576446739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad330 0x7f14a815d4a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f14a806fbe0 tx=0x7f149801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508ae5165b61d3a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14a5400640 1 -- 172.21.15.86:0/576446739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980a5840 con 0x7f14a80ad330 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14a5400640 1 -- 172.21.15.86:0/576446739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980a59e0 con 0x7f14a80ad330 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14a5400640 1 -- 172.21.15.86:0/576446739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980a5db0 con 0x7f14a80ad330 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 -- 172.21.15.86:0/576446739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad330 msgr2=0x7f14a815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 --2- 172.21.15.86:0/576446739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad330 0x7f14a815d4a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f14a806fbe0 tx=0x7f149801c9c0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 -- 172.21.15.86:0/576446739 shutdown_connections 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 --2- 172.21.15.86:0/576446739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a80ad710 0x7f14a813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 --2- 172.21.15.86:0/576446739 >> v2:172.21.15.196:3300/0 conn(0x7f14a806ef40 0x7f14a806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 --2- 172.21.15.86:0/576446739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad330 0x7f14a815d4a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 -- 172.21.15.86:0/576446739 >> 172.21.15.86:0/576446739 conn(0x7f14a80b1df0 msgr2=0x7f14a80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 -- 172.21.15.86:0/576446739 shutdown_connections 2024-03-22T04:10:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.273+0000 7f14ad000640 1 -- 172.21.15.86:0/576446739 wait complete. 2024-03-22T04:10:42.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 Processor -- start 2024-03-22T04:10:42.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 -- start start 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a806ef40 0x7f14a81f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a80ad330 0x7f14a81654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad710 0x7f14a809c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8141800 con 0x7f14a80ad710 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8141680 con 0x7f14a80ad330 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14a8141980 con 0x7f14a806ef40 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad710 0x7f14a809c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad710 0x7f14a809c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35180/0 (socket says 172.21.15.86:35180) 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a806ef40 0x7f14a81f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f149d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a80ad330 0x7f14a81654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 -- 172.21.15.86:0/978566041 learned_addr learned my addr 172.21.15.86:0/978566041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 -- 172.21.15.86:0/978566041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a806ef40 msgr2=0x7f14a81f9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a806ef40 0x7f14a81f9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 -- 172.21.15.86:0/978566041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a80ad330 msgr2=0x7f14a81654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a80ad330 0x7f14a81654e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 -- 172.21.15.86:0/978566041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a81fa4b0 con 0x7f14a80ad710 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14a6800640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad710 0x7f14a809c5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e7f0 tx=0x7f14a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f149ea00640 1 -- 172.21.15.86:0/978566041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0017070 con 0x7f14a80ad710 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f149ea00640 1 -- 172.21.15.86:0/978566041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a000c940 con 0x7f14a80ad710 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 -- 172.21.15.86:0/978566041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a809af60 con 0x7f14a80ad710 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f149ea00640 1 -- 172.21.15.86:0/978566041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a000eea0 con 0x7f14a80ad710 2024-03-22T04:10:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.277+0000 7f14ad000640 1 -- 172.21.15.86:0/978566041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a809b520 con 0x7f14a80ad710 2024-03-22T04:10:42.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f149ea00640 1 -- 172.21.15.86:0/978566041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f14a001a070 con 0x7f14a80ad710 2024-03-22T04:10:42.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f149ea00640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1484067b50 0x7f148406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f149ea00640 1 -- 172.21.15.86:0/978566041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f14a0086060 con 0x7f14a80ad710 2024-03-22T04:10:42.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14a5e00640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1484067b50 0x7f148406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14b05ac000 1 Processor -- start 2024-03-22T04:10:42.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14b05ac000 1 -- start start 2024-03-22T04:10:42.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14a5e00640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1484067b50 0x7f148406a010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f14980ae000 tx=0x7f14980a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e649250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 0x55cf5e5fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e5fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14b05ac000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e3fa140 con 0x55cf5e648e70 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14b05ac000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e44db10 con 0x55cf5e5fcc20 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14b05ac000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cf5e3d4c50 con 0x55cf5e649790 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14a5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e649250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14a5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e649250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35190/0 (socket says 172.21.15.86:35190) 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f14a5e00640 1 -- 172.21.15.86:0/997079212 learned_addr learned my addr 172.21.15.86:0/997079212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.281+0000 7f149d400640 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 0x55cf5e5fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a6800640 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e5fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 -- 172.21.15.86:0/997079212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 msgr2=0x55cf5e5fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 0x55cf5e5fc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 -- 172.21.15.86:0/997079212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 msgr2=0x55cf5e5fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e5fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 -- 172.21.15.86:0/997079212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5e492f20 con 0x55cf5e648e70 2024-03-22T04:10:42.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e649250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1498009e50 tx=0x7f1498004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850e7b45df46869d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f1496a00640 1 -- 172.21.15.86:0/997079212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980c6070 con 0x55cf5e648e70 2024-03-22T04:10:42.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f1496a00640 1 -- 172.21.15.86:0/997079212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980c2420 con 0x55cf5e648e70 2024-03-22T04:10:42.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f1496a00640 1 -- 172.21.15.86:0/997079212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980c27d0 con 0x55cf5e648e70 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- 172.21.15.86:0/997079212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 msgr2=0x55cf5e649250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e649250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1498009e50 tx=0x7f1498004ac0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- 172.21.15.86:0/997079212 shutdown_connections 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e5fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 0x55cf5e5fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 --2- 172.21.15.86:0/997079212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e649250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- 172.21.15.86:0/997079212 >> 172.21.15.86:0/997079212 conn(0x55cf5e629c10 msgr2=0x55cf5e62c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- 172.21.15.86:0/997079212 shutdown_connections 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- 172.21.15.86:0/997079212 wait complete. 2024-03-22T04:10:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- start start 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 Processor -- start 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e601810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e648e70 0x55cf5e601d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e649790 0x55cf5e602290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e407ab0 con 0x55cf5e649790 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e449ac0 con 0x55cf5e5fcc20 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cf5e48da60 con 0x55cf5e648e70 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14b05ac000 10 client.0 _ll_register_callbacks cb 0x55cf5e5ea380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e601810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e649790 0x55cf5e602290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e601810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:43150/0 (socket says 172.21.15.86:43150) 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f14a5e00640 1 -- 172.21.15.86:0/1462815660 learned_addr learned my addr 172.21.15.86:0/1462815660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.285+0000 7f149d400640 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e648e70 0x55cf5e601d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a6800640 1 -- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e648e70 msgr2=0x55cf5e601d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a6800640 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e648e70 0x55cf5e601d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a6800640 1 -- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 msgr2=0x55cf5e601810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a6800640 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e601810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a6800640 1 -- 172.21.15.86:0/1462815660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5e423570 con 0x55cf5e649790 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a5e00640 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e601810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a6800640 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e649790 0x55cf5e602290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f14a0055dd0 tx=0x7f14a005e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c7d4e46fb63588 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 1 -- 172.21.15.86:0/1462815660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a006e070 con 0x55cf5e649790 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 1 -- 172.21.15.86:0/1462815660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f14a007dda0 con 0x55cf5e649790 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 -- 172.21.15.86:0/1462815660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14980c2420 con 0x55cf5e649790 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 1 -- 172.21.15.86:0/1462815660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f14a0074050 con 0x55cf5e649790 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 10 client.4407 fetch_fsmap learned FSMap version 7 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 -- 172.21.15.86:0/1462815660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55cf5e419880 con 0x55cf5e649790 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 1 -- 172.21.15.86:0/1462815660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980c2420 con 0x55cf5e649790 2024-03-22T04:10:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 1 -- 172.21.15.86:0/1462815660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f14a0024060 con 0x55cf5e649790 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 10 client.4407 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 20 client.4407 populate_metadata read hostname 'smithi086' 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55cf5e602ca0 0x55cf5e65a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 4 client.4407 mds_command: new command op to 74171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 -- 172.21.15.86:0/1462815660 --> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf5e3e8bc0 con 0x55cf5e602ca0 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a5e00640 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55cf5e602ca0 0x55cf5e65a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14a5e00640 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55cf5e602ca0 0x55cf5e65a120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56a2e95268a6cef server_cookie=dd00a71ab7b691fd in_seq=0 out_seq=0 2024-03-22T04:10:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 10 client.4407 ms_handle_connect on v2:172.21.15.196:6833/180015071 2024-03-22T04:10:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 1 -- 172.21.15.86:0/1462815660 <== mds.0 v2:172.21.15.196:6833/180015071 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cf5e3e8bc0 con 0x55cf5e602ca0 2024-03-22T04:10:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 10 client.4407 handle_command_reply: tid=0 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 client.4407 shutdown 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 -- 172.21.15.86:0/1462815660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e649790 msgr2=0x55cf5e602290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e649790 0x55cf5e602290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f14a0055dd0 tx=0x7f14a005e9a0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 -- 172.21.15.86:0/1462815660 shutdown_connections 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55cf5e602ca0 0x55cf5e65a120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e649790 0x55cf5e602290 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e648e70 0x55cf5e601d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f148b400640 0 client.4407 ms_handle_reset on v2:172.21.15.196:6833/180015071 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 --2- 172.21.15.86:0/1462815660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e5fcc20 0x55cf5e601810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 -- 172.21.15.86:0/1462815660 >> 172.21.15.86:0/1462815660 conn(0x55cf5e629c10 msgr2=0x55cf5e62c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 -- 172.21.15.86:0/1462815660 shutdown_connections 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 1 -- 172.21.15.86:0/1462815660 wait complete. 2024-03-22T04:10:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.289+0000 7f14b05ac000 20 client.4407 trim_cache size 0 max 16384 2024-03-22T04:10:42.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 Processor -- start 2024-03-22T04:10:42.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- start start 2024-03-22T04:10:42.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e5fcc20 0x55cf5e65a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e65a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e649790 0x55cf5e65aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8204ef0 con 0x55cf5e648e70 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8203f40 con 0x55cf5e649790 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14a82032a0 con 0x55cf5e5fcc20 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e65a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e65a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35202/0 (socket says 172.21.15.86:35202) 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 -- 172.21.15.86:0/150043846 learned_addr learned my addr 172.21.15.86:0/150043846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14a6800640 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e649790 0x55cf5e65aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14a5e00640 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e5fcc20 0x55cf5e65a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 -- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e5fcc20 msgr2=0x55cf5e65a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e5fcc20 0x55cf5e65a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 -- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e649790 msgr2=0x55cf5e65aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e649790 0x55cf5e65aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 -- 172.21.15.86:0/150043846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5e3d5e80 con 0x55cf5e648e70 2024-03-22T04:10:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14a6800640 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e649790 0x55cf5e65aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f149d400640 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e65a660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f14a815f4c0 tx=0x7f1490009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ebf9b3eefd9e11 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f1496000640 1 -- 172.21.15.86:0/150043846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490023070 con 0x55cf5e648e70 2024-03-22T04:10:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f1496000640 1 -- 172.21.15.86:0/150043846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1490015b10 con 0x55cf5e648e70 2024-03-22T04:10:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f1496000640 1 -- 172.21.15.86:0/150043846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490015cc0 con 0x55cf5e648e70 2024-03-22T04:10:42.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- 172.21.15.86:0/150043846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 msgr2=0x55cf5e65a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e65a660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f14a815f4c0 tx=0x7f1490009d30 comp rx=0 tx=0).stop 2024-03-22T04:10:42.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- 172.21.15.86:0/150043846 shutdown_connections 2024-03-22T04:10:42.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e649790 0x55cf5e65aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e648e70 0x55cf5e65a660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 --2- 172.21.15.86:0/150043846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e5fcc20 0x55cf5e65a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- 172.21.15.86:0/150043846 >> 172.21.15.86:0/150043846 conn(0x55cf5e629c10 msgr2=0x55cf5e646f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- 172.21.15.86:0/150043846 shutdown_connections 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.309+0000 7f14b05ac000 1 -- 172.21.15.86:0/150043846 wait complete. 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 -- start start 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 Processor -- start 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e5fcc20 0x55cf5e6b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e648e70 0x55cf5e6b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 0x55cf5e673a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf5e509b30 con 0x55cf5e5fcc20 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14a82038f0 con 0x55cf5e648e70 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cf5e605110 con 0x55cf5e649790 2024-03-22T04:10:42.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 10 client.0 _ll_register_callbacks cb 0x7f14a0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e5fcc20 0x55cf5e6b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e5fcc20 0x55cf5e6b8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35212/0 (socket says 172.21.15.86:35212) 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 -- 172.21.15.86:0/2338612496 learned_addr learned my addr 172.21.15.86:0/2338612496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f149d400640 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e648e70 0x55cf5e6b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a6800640 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 0x55cf5e673a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 -- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 msgr2=0x55cf5e673a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 0x55cf5e673a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 -- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e648e70 msgr2=0x55cf5e6b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e648e70 0x55cf5e6b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 -- 172.21.15.86:0/2338612496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5e3e1750 con 0x55cf5e5fcc20 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14a5e00640 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e5fcc20 0x55cf5e6b8760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f149801d260 tx=0x7f1498004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af0e4fe0ed295e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f148b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f148b400640 1 -- 172.21.15.86:0/2338612496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980dc060 con 0x55cf5e5fcc20 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f148b400640 1 -- 172.21.15.86:0/2338612496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f14980d6b90 con 0x55cf5e5fcc20 2024-03-22T04:10:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 -- 172.21.15.86:0/2338612496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1490015b10 con 0x55cf5e5fcc20 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f148b400640 1 -- 172.21.15.86:0/2338612496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f14980ba040 con 0x55cf5e5fcc20 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 10 client.4413 fetch_fsmap learned FSMap version 7 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 -- 172.21.15.86:0/2338612496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55cf5e3f4450 con 0x55cf5e5fcc20 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f148b400640 1 -- 172.21.15.86:0/2338612496 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1490015b10 con 0x55cf5e5fcc20 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f148b400640 1 -- 172.21.15.86:0/2338612496 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f14980e5020 con 0x55cf5e5fcc20 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 10 client.4413 fetch_fsmap finished waiting for FSMap version 7 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 10 client.4413 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 20 client.4413 populate_metadata read hostname 'smithi086' 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55cf5e674430 0x55cf5e674810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 4 client.4413 mds_command: new command op to 74171 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f14b05ac000 1 -- 172.21.15.86:0/2338612496 --> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] -- command(tid 0: {"prefix": "status"}) -- 0x55cf5e45b050 con 0x55cf5e674430 2024-03-22T04:10:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.313+0000 7f149d400640 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55cf5e674430 0x55cf5e674810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f149d400640 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55cf5e674430 0x55cf5e674810 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f265e52835cd322 server_cookie=24b4c8d69f087a74 in_seq=0 out_seq=0 2024-03-22T04:10:42.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f148b400640 10 client.4413 ms_handle_connect on v2:172.21.15.196:6833/180015071 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f148b400640 1 -- 172.21.15.86:0/2338612496 <== mds.0 v2:172.21.15.196:6833/180015071 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55cf5e45b050 con 0x55cf5e674430 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f148b400640 10 client.4413 handle_command_reply: tid=0 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 client.4413 shutdown 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 -- 172.21.15.86:0/2338612496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e5fcc20 msgr2=0x55cf5e6b8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e5fcc20 0x55cf5e6b8760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f149801d260 tx=0x7f1498004a50 comp rx=0 tx=0).stop 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 -- 172.21.15.86:0/2338612496 shutdown_connections 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55cf5e674430 0x55cf5e674810 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cf5e648e70 0x55cf5e6b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cf5e649790 0x55cf5e673a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f148b400640 0 client.4413 ms_handle_reset on v2:172.21.15.196:6833/180015071 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 --2- 172.21.15.86:0/2338612496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf5e5fcc20 0x55cf5e6b8760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 -- 172.21.15.86:0/2338612496 >> 172.21.15.86:0/2338612496 conn(0x55cf5e629c10 msgr2=0x55cf5e6477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:42.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 -- 172.21.15.86:0/2338612496 shutdown_connections 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 1 -- 172.21.15.86:0/2338612496 wait complete. 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14b05ac000 20 client.4413 trim_cache size 0 max 16384 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 74171, 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:10:42.323 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 7.4109181690000003, 2024-03-22T04:10:42.324 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 7, 2024-03-22T04:10:42.324 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 27, 2024-03-22T04:10:42.324 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 27, 2024-03-22T04:10:42.324 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.004563395 2024-03-22T04:10:42.324 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:10:42.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 -- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1484067b50 msgr2=0x7f148406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1484067b50 0x7f148406a010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f14980ae000 tx=0x7f14980a7070 comp rx=0 tx=0).stop 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 -- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad710 msgr2=0x7f14a809c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad710 0x7f14a809c5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e7f0 tx=0x7f14a000ecb0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 -- 172.21.15.86:0/978566041 shutdown_connections 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1484067b50 0x7f148406a010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14a80ad330 0x7f14a81654e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a80ad710 0x7f14a809c5f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 --2- 172.21.15.86:0/978566041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14a806ef40 0x7f14a81f9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 -- 172.21.15.86:0/978566041 >> 172.21.15.86:0/978566041 conn(0x7f14a80b1df0 msgr2=0x7f14a8072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 -- 172.21.15.86:0/978566041 shutdown_connections 2024-03-22T04:10:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.317+0000 7f14ad000640 1 -- 172.21.15.86:0/978566041 wait complete. 2024-03-22T04:10:42.337 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T04:10:42.337 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:42.338 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:10:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.473+0000 7f3f89e00640 1 Processor -- start 2024-03-22T04:10:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.473+0000 7f3f89e00640 1 -- start start 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f89e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f84070050 0x7f3f840b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f89e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f84161e40 con 0x7f3f840b54c0 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f89e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f84162120 con 0x7f3f841413e0 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f89e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8415f860 con 0x7f3f84070050 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f84070050 0x7f3f840b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f89400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3f840b54c0 0x7f3f84143850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34614/0 (socket says 172.21.15.86:34614) 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f84070050 0x7f3f840b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35224/0 (socket says 172.21.15.86:35224) 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f88a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3f841413e0 0x7f3f8406f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:55954/0 (socket says 172.21.15.86:55954) 2024-03-22T04:10:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f89400640 1 -- 172.21.15.86:0/2694877117 learned_addr learned my addr 172.21.15.86:0/2694877117 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f83e00640 1 -- 172.21.15.86:0/2694877117 >> v1:172.21.15.196:6789/0 conn(0x7f3f841413e0 legacy=0x7f3f8406f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f83e00640 1 -- 172.21.15.86:0/2694877117 >> v1:172.21.15.86:6789/0 conn(0x7f3f840b54c0 legacy=0x7f3f84143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f83400640 1 -- 172.21.15.86:0/2694877117 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 11056175 0 0) 0x7f3f84162120 con 0x7f3f841413e0 2024-03-22T04:10:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.477+0000 7f3f83e00640 1 -- 172.21.15.86:0/2694877117 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f84162400 con 0x7f3f84070050 2024-03-22T04:10:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f83e00640 1 --2- 172.21.15.86:0/2694877117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f840b4f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3f700097c0 tx=0x7f3f7001c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9074eaab9621002 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f83400640 1 -- 172.21.15.86:0/2694877117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700b0070 con 0x7f3f84070050 2024-03-22T04:10:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f83400640 1 -- 172.21.15.86:0/2694877117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f700ab600 con 0x7f3f84070050 2024-03-22T04:10:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f83400640 1 -- 172.21.15.86:0/2694877117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700ab9d0 con 0x7f3f84070050 2024-03-22T04:10:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 -- 172.21.15.86:0/2694877117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 msgr2=0x7f3f840b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 --2- 172.21.15.86:0/2694877117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f840b4f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3f700097c0 tx=0x7f3f7001c2a0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 -- 172.21.15.86:0/2694877117 shutdown_connections 2024-03-22T04:10:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 --2- 172.21.15.86:0/2694877117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f840b4f80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 -- 172.21.15.86:0/2694877117 >> 172.21.15.86:0/2694877117 conn(0x7f3f840b3c80 msgr2=0x7f3f840b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 -- 172.21.15.86:0/2694877117 shutdown_connections 2024-03-22T04:10:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 -- 172.21.15.86:0/2694877117 wait complete. 2024-03-22T04:10:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 Processor -- start 2024-03-22T04:10:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.481+0000 7f3f89e00640 1 -- start start 2024-03-22T04:10:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f841570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f840b54c0 0x7f3f84154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f841413e0 0x7f3f8415b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84141960 con 0x7f3f84070050 2024-03-22T04:10:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841629c0 con 0x7f3f841413e0 2024-03-22T04:10:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3f84159140 con 0x7f3f840b54c0 2024-03-22T04:10:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f841570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f841570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35232/0 (socket says 172.21.15.86:35232) 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 -- 172.21.15.86:0/380366815 learned_addr learned my addr 172.21.15.86:0/380366815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f83e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f840b54c0 0x7f3f84154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89400640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f841413e0 0x7f3f8415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 -- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f840b54c0 msgr2=0x7f3f84154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f840b54c0 0x7f3f84154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 -- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f841413e0 msgr2=0x7f3f8415b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f841413e0 0x7f3f8415b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 -- 172.21.15.86:0/380366815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f841579e0 con 0x7f3f84070050 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89400640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f841413e0 0x7f3f8415b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f88a00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f841570e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c410 tx=0x7f3f7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f80c00640 1 -- 172.21.15.86:0/380366815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780074e0 con 0x7f3f84070050 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f80c00640 1 -- 172.21.15.86:0/380366815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f780047f0 con 0x7f3f84070050 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f84157cc0 con 0x7f3f84070050 2024-03-22T04:10:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f80c00640 1 -- 172.21.15.86:0/380366815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78007680 con 0x7f3f84070050 2024-03-22T04:10:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f84157620 con 0x7f3f84070050 2024-03-22T04:10:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f84158170 con 0x7f3f84070050 2024-03-22T04:10:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.485+0000 7f3f80c00640 1 -- 172.21.15.86:0/380366815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3f78011070 con 0x7f3f84070050 2024-03-22T04:10:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.489+0000 7f3f80c00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3f64067b20 0x7f3f64069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.489+0000 7f3f80c00640 1 -- 172.21.15.86:0/380366815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3f78085dd0 con 0x7f3f84070050 2024-03-22T04:10:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.489+0000 7f3f83e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3f64067b20 0x7f3f64069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.489+0000 7f3f83e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3f64067b20 0x7f3f64069fe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ad000 tx=0x7f3f700a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.493+0000 7f3f80c00640 1 -- 172.21.15.86:0/380366815 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3f780564a0 con 0x7f3f84070050 2024-03-22T04:10:42.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.689+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f84074470 con 0x7f3f84070050 2024-03-22T04:10:42.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.689+0000 7f3f80c00640 1 -- 172.21.15.86:0/380366815 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f3f78055da0 con 0x7f3f84070050 2024-03-22T04:10:42.693 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-22T04:10:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3f64067b20 msgr2=0x7f3f64069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3f64067b20 0x7f3f64069fe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ad000 tx=0x7f3f700a6000 comp rx=0 tx=0).stop 2024-03-22T04:10:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 msgr2=0x7f3f841570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f841570e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c410 tx=0x7f3f7800c8d0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 shutdown_connections 2024-03-22T04:10:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3f64067b20 0x7f3f64069fe0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f841413e0 0x7f3f8415b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f840b54c0 0x7f3f84154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 --2- 172.21.15.86:0/380366815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f84070050 0x7f3f841570e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 >> 172.21.15.86:0/380366815 conn(0x7f3f840b3c80 msgr2=0x7f3f8406f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 shutdown_connections 2024-03-22T04:10:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42.693+0000 7f3f89e00640 1 -- 172.21.15.86:0/380366815 wait complete. 2024-03-22T04:10:42.713 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:10:42.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:10:42.713 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:10:42.713 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-22T04:10:42.713 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:42.713 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:10:42.713 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T04:10:42.714 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:10:42.714 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-22T04:10:42.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:42.714 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:10:42.723 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42489sec preferred_lft 42489sec 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:10:42.724 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:10:42.725 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:10:42.726 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T04:10:42.726 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:10:42.726 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:10:42.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-03-22T04:10:42.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-03-22T04:10:42.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-03-22T04:10:42.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T04:10:42.726 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:10:42.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:10:42.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:10:42.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:42.887 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:10:42.952 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:10:42.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:42.955 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:10:43.235 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:10:43.235 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:10:43.235 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:10:43.235 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:10:43.235 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:10:43.235 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:10:43.236 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:10:43.236 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:10:43.236 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:10:43.236 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:10:43.236 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:10:43.236 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:10:43.236 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:10:43.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:10:43.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:10:43.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:10:43.237 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:10:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:10:43.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:10:43.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:43.332 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:10:43.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:43.341 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-03-22T04:10:43.394 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:10:43.394 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:10:43.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-22T04:10:43.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-22T04:10:43.394 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:10:43.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:10:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:10:43.523 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-22T04:10:43.523 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:10:43.523 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:10:43.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-22T04:10:43.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T04:10:43.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-22T04:10:43.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-22T04:10:43.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-22T04:10:43.524 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:10:43.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:10:43.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:10:43.678 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:10:43.678 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:10:43.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:10:43.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T04:10:43.678 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:10:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:10:43.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:10:43.808 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-22T04:10:43.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:10:43.808 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:43.817 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-22T04:10:43.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:10:43.817 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:43.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-03-22T04:10:43.929 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T04:10:43.979 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-03-22T04:10:43.980 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-03-22T04:10:43.981 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-03-22T04:10:43.982 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-03-22T04:10:43.982 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-03-22T04:10:43.982 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-03-22T04:10:43.982 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-03-22T04:10:43.982 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-22T04:10:43.982 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-22T04:10:43.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:43.983 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-03-22T04:10:44.033 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-22T04:10:44.034 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-22T04:10:44.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-03-22T04:10:44.101 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T04:10:44.150 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-03-22T04:10:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-22T04:10:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-22T04:10:44.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:44.153 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-03-22T04:10:44.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:2024-03-22T04:10:44.176+0000 7f917b3b4580 -1 init, newargv = 0x55d80e9637b0 newargc=15 2024-03-22T04:10:44.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[34054]: starting ceph client 2024-03-22T04:10:44.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[34054]: starting fuse 2024-03-22T04:10:45.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-03-22T04:10:45.315 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-03-22T04:10:45.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-03-22T04:10:45.372 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-03-22T04:10:45.373 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-22T04:10:45.374 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-22T04:10:45.374 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:45.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:45.375 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-03-22T04:10:45.426 INFO:teuthology.orchestra.run.smithi086.stdout:43 2024-03-22T04:10:45.426 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-03-22T04:10:45.427 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.admin")) 2024-03-22T04:10:45.428 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:10:45.557 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.admin.34054.asok 2024-03-22T04:10:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:10:45.570 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.34054.asok 2024-03-22T04:10:45.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:45.570 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.34054.asok status 2024-03-22T04:10:45.745 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e", 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-2344-gc3a04c0e (c3a04c0e75f6e20a134d0f5e6ebc95d5f534c43e) squid (dev)", 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "admin", 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "34054", 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-03-22T04:10:45.746 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4428, 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4428 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3480945757 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-03-22T04:10:45.747 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3480945757 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4428 192.168.0.1:0/3480945757", 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3480945757", 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 7, 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 27, 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-03-22T04:10:45.748 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:10:45.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:45.761 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-03-22T04:10:45.761 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:45.769 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-03-22T04:10:45.769 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:45.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:45.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:45.834 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-22T04:10:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:10:46.015 INFO:teuthology.orchestra.run.smithi086.stdout:# file: . 2024-03-22T04:10:46.015 INFO:teuthology.orchestra.run.smithi086.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-22T04:10:46.016 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:10:46.022 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:46.022 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:10:46.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.204+0000 7f5478600640 1 Processor -- start 2024-03-22T04:10:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.204+0000 7f5478600640 1 -- start start 2024-03-22T04:10:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.204+0000 7f5478600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547015b0d0 0x7f547015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.204+0000 7f5478600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f54700b2570 0x7f54700b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.204+0000 7f5478600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f54700b5150 0x7f547015eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.204+0000 7f5478600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f54700b22e0 con 0x7f54700b5150 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.204+0000 7f5478600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f54700b5570 con 0x7f54700b2570 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.204+0000 7f5478600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f547015bd30 con 0x7f547015b0d0 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f54700b5150 0x7f547015eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5475200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f54700b2570 0x7f54700b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f54700b5150 0x7f547015eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35210/0 (socket says 172.21.15.86:35210) 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 -- 172.21.15.86:0/3610220905 learned_addr learned my addr 172.21.15.86:0/3610220905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5475c00640 1 --2- 172.21.15.86:0/3610220905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547015b0d0 0x7f547015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 -- 172.21.15.86:0/3610220905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547015b0d0 msgr2=0x7f547015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 --2- 172.21.15.86:0/3610220905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547015b0d0 0x7f547015b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 -- 172.21.15.86:0/3610220905 >> v2:172.21.15.196:3300/0 conn(0x7f54700b2570 msgr2=0x7f54700b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 --2- 172.21.15.86:0/3610220905 >> v2:172.21.15.196:3300/0 conn(0x7f54700b2570 0x7f54700b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 -- 172.21.15.86:0/3610220905 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547015c010 con 0x7f54700b5150 2024-03-22T04:10:46.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f5476600640 1 --2- 172.21.15.86:0/3610220905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b5150 0x7f547015eed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f547015bf40 tx=0x7f546801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d4b48c69e2ca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f546fe00640 1 -- 172.21.15.86:0/3610220905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54680b1070 con 0x7f54700b5150 2024-03-22T04:10:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f546fe00640 1 -- 172.21.15.86:0/3610220905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54680ac410 con 0x7f54700b5150 2024-03-22T04:10:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.208+0000 7f546fe00640 1 -- 172.21.15.86:0/3610220905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54680ac7e0 con 0x7f54700b5150 2024-03-22T04:10:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- 172.21.15.86:0/3610220905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b5150 msgr2=0x7f547015eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 --2- 172.21.15.86:0/3610220905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b5150 0x7f547015eed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f547015bf40 tx=0x7f546801c500 comp rx=0 tx=0).stop 2024-03-22T04:10:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- 172.21.15.86:0/3610220905 shutdown_connections 2024-03-22T04:10:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 --2- 172.21.15.86:0/3610220905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b5150 0x7f547015eed0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 --2- 172.21.15.86:0/3610220905 >> v2:172.21.15.196:3300/0 conn(0x7f54700b2570 0x7f54700b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 --2- 172.21.15.86:0/3610220905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547015b0d0 0x7f547015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- 172.21.15.86:0/3610220905 >> 172.21.15.86:0/3610220905 conn(0x7f54700b6620 msgr2=0x7f54700b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:46.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- 172.21.15.86:0/3610220905 shutdown_connections 2024-03-22T04:10:46.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- 172.21.15.86:0/3610220905 wait complete. 2024-03-22T04:10:46.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 Processor -- start 2024-03-22T04:10:46.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- start start 2024-03-22T04:10:46.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b2570 0x7f5470144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54700b5150 0x7f5470148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547015b0d0 0x7f5470151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f547015cdd0 con 0x7f54700b2570 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f547015b9f0 con 0x7f547015b0d0 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5478600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f547015cf50 con 0x7f54700b5150 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b2570 0x7f5470144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b2570 0x7f5470144b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35214/0 (socket says 172.21.15.86:35214) 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475c00640 1 -- 172.21.15.86:0/1779259336 learned_addr learned my addr 172.21.15.86:0/1779259336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475200640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54700b5150 0x7f5470148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5476600640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547015b0d0 0x7f5470151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475c00640 1 -- 172.21.15.86:0/1779259336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54700b5150 msgr2=0x7f5470148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475c00640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54700b5150 0x7f5470148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475c00640 1 -- 172.21.15.86:0/1779259336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547015b0d0 msgr2=0x7f5470151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475c00640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547015b0d0 0x7f5470151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.212+0000 7f5475c00640 1 -- 172.21.15.86:0/1779259336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547014f8f0 con 0x7f54700b2570 2024-03-22T04:10:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f5475c00640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b2570 0x7f5470144b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f546400efc0 tx=0x7f546400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f546d600640 1 -- 172.21.15.86:0/1779259336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464018020 con 0x7f54700b2570 2024-03-22T04:10:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f546d600640 1 -- 172.21.15.86:0/1779259336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5464014480 con 0x7f54700b2570 2024-03-22T04:10:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f546d600640 1 -- 172.21.15.86:0/1779259336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464014850 con 0x7f54700b2570 2024-03-22T04:10:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547014fbd0 con 0x7f54700b2570 2024-03-22T04:10:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5470150190 con 0x7f54700b2570 2024-03-22T04:10:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5470068650 con 0x7f54700b2570 2024-03-22T04:10:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f546d600640 1 -- 172.21.15.86:0/1779259336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f546401b020 con 0x7f54700b2570 2024-03-22T04:10:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f546d600640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5450067b50 0x7f545006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f546d600640 1 -- 172.21.15.86:0/1779259336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5464086f50 con 0x7f54700b2570 2024-03-22T04:10:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.216+0000 7f5475200640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5450067b50 0x7f545006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.220+0000 7f5475200640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5450067b50 0x7f545006a010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5470155730 tx=0x7f545c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.220+0000 7f546d600640 1 -- 172.21.15.86:0/1779259336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5464022030 con 0x7f54700b2570 2024-03-22T04:10:46.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.376+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5470158490 con 0x7f54700b2570 2024-03-22T04:10:46.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f546d600640 1 -- 172.21.15.86:0/1779259336 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7f5464010040 con 0x7f54700b2570 2024-03-22T04:10:46.383 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:46.383 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5450067b50 msgr2=0x7f545006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5450067b50 0x7f545006a010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5470155730 tx=0x7f545c009190 comp rx=0 tx=0).stop 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b2570 msgr2=0x7f5470144b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b2570 0x7f5470144b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f546400efc0 tx=0x7f546400c490 comp rx=0 tx=0).stop 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 shutdown_connections 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5450067b50 0x7f545006a010 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547015b0d0 0x7f5470151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54700b5150 0x7f5470148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 --2- 172.21.15.86:0/1779259336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54700b2570 0x7f5470144b50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 >> 172.21.15.86:0/1779259336 conn(0x7f54700b6620 msgr2=0x7f54701559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 shutdown_connections 2024-03-22T04:10:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.380+0000 7f5478600640 1 -- 172.21.15.86:0/1779259336 wait complete. 2024-03-22T04:10:46.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:10:46.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:10:46.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0fd800640 1 Processor -- start 2024-03-22T04:10:46.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0fd800640 1 -- start start 2024-03-22T04:10:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0fd800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0f8070560 0x7fd0f80d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0fd800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f80d15f0 con 0x7fd0f80b4710 2024-03-22T04:10:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0fd800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f80d1310 con 0x7fd0f815e7e0 2024-03-22T04:10:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0fd800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f806f970 con 0x7fd0f8070560 2024-03-22T04:10:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0ee000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0f8070560 0x7fd0f80d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0ee000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0f8070560 0x7fd0f80d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35216/0 (socket says 172.21.15.86:35216) 2024-03-22T04:10:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0ee000640 1 -- 172.21.15.86:0/1835566725 learned_addr learned my addr 172.21.15.86:0/1835566725 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.540+0000 7fd0f7400640 1 --1- 172.21.15.86:0/1835566725 >> v1:172.21.15.86:6789/0 conn(0x7fd0f80b4710 0x7fd0f80cd360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59432/0 (socket says 172.21.15.86:59432) 2024-03-22T04:10:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0ee000640 1 -- 172.21.15.86:0/1835566725 >> v1:172.21.15.196:6789/0 conn(0x7fd0f815e7e0 legacy=0x7fd0f806fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0ee000640 1 -- 172.21.15.86:0/1835566725 >> v1:172.21.15.86:6789/0 conn(0x7fd0f80b4710 legacy=0x7fd0f80cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0ee000640 1 -- 172.21.15.86:0/1835566725 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f80cc880 con 0x7fd0f8070560 2024-03-22T04:10:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0ee000640 1 --2- 172.21.15.86:0/1835566725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f8070560 0x7fd0f80d0d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009870 tx=0x7fd0e001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29497ce90a2a0400 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0f6000640 1 -- 172.21.15.86:0/1835566725 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00b0070 con 0x7fd0f8070560 2024-03-22T04:10:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0f6000640 1 -- 172.21.15.86:0/1835566725 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e00ab600 con 0x7fd0f8070560 2024-03-22T04:10:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0f6000640 1 -- 172.21.15.86:0/1835566725 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00ab9d0 con 0x7fd0f8070560 2024-03-22T04:10:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0fd800640 1 -- 172.21.15.86:0/1835566725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f8070560 msgr2=0x7fd0f80d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0fd800640 1 --2- 172.21.15.86:0/1835566725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f8070560 0x7fd0f80d0d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009870 tx=0x7fd0e001c440 comp rx=0 tx=0).stop 2024-03-22T04:10:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0fd800640 1 -- 172.21.15.86:0/1835566725 shutdown_connections 2024-03-22T04:10:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0fd800640 1 --2- 172.21.15.86:0/1835566725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f8070560 0x7fd0f80d0d30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.544+0000 7fd0fd800640 1 -- 172.21.15.86:0/1835566725 >> 172.21.15.86:0/1835566725 conn(0x7fd0f80b3bf0 msgr2=0x7fd0f80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 -- 172.21.15.86:0/1835566725 shutdown_connections 2024-03-22T04:10:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 -- 172.21.15.86:0/1835566725 wait complete. 2024-03-22T04:10:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 Processor -- start 2024-03-22T04:10:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 -- start start 2024-03-22T04:10:46.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f8070560 0x7fd0f8157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f80b4710 0x7fd0f8154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f815e7e0 0x7fd0f815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f816c1b0 con 0x7fd0f80b4710 2024-03-22T04:10:46.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f80ce490 con 0x7fd0f8070560 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0fd800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f816c330 con 0x7fd0f815e7e0 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f80b4710 0x7fd0f8154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0f6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f8070560 0x7fd0f8157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f80b4710 0x7fd0f8154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35226/0 (socket says 172.21.15.86:35226) 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 -- 172.21.15.86:0/2309903093 learned_addr learned my addr 172.21.15.86:0/2309903093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0f7400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f815e7e0 0x7fd0f815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 -- 172.21.15.86:0/2309903093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f815e7e0 msgr2=0x7fd0f815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f815e7e0 0x7fd0f815cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 -- 172.21.15.86:0/2309903093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f8070560 msgr2=0x7fd0f8157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f8070560 0x7fd0f8157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 -- 172.21.15.86:0/2309903093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f80b5fc0 con 0x7fd0f80b4710 2024-03-22T04:10:46.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ee000640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f80b4710 0x7fd0f8154970 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0e001a070 tx=0x7fd0e00191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.548+0000 7fd0ef400640 1 -- 172.21.15.86:0/2309903093 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00b0040 con 0x7fd0f80b4710 2024-03-22T04:10:46.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0ef400640 1 -- 172.21.15.86:0/2309903093 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e00b4640 con 0x7fd0f80b4710 2024-03-22T04:10:46.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f80b62a0 con 0x7fd0f80b4710 2024-03-22T04:10:46.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0ef400640 1 -- 172.21.15.86:0/2309903093 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00b4a30 con 0x7fd0f80b4710 2024-03-22T04:10:46.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f80b6860 con 0x7fd0f80b4710 2024-03-22T04:10:46.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f8074470 con 0x7fd0f80b4710 2024-03-22T04:10:46.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0ef400640 1 -- 172.21.15.86:0/2309903093 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd0e00ab4c0 con 0x7fd0f80b4710 2024-03-22T04:10:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0ef400640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd0d4067b20 0x7fd0d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0ef400640 1 -- 172.21.15.86:0/2309903093 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd0e011fd70 con 0x7fd0f80b4710 2024-03-22T04:10:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.552+0000 7fd0f6a00640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd0d4067b20 0x7fd0d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.556+0000 7fd0f6a00640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd0d4067b20 0x7fd0d4069fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80012f0 tx=0x7fd0e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.556+0000 7fd0ef400640 1 -- 172.21.15.86:0/2309903093 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd0e0016150 con 0x7fd0f80b4710 2024-03-22T04:10:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.712+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0f80b18f0 con 0x7fd0f80b4710 2024-03-22T04:10:46.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.712+0000 7fd0ef400640 1 -- 172.21.15.86:0/2309903093 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fd0e00efdf0 con 0x7fd0f80b4710 2024-03-22T04:10:46.717 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd0d4067b20 msgr2=0x7fd0d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd0d4067b20 0x7fd0d4069fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80012f0 tx=0x7fd0e8009040 comp rx=0 tx=0).stop 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f80b4710 msgr2=0x7fd0f8154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f80b4710 0x7fd0f8154970 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0e001a070 tx=0x7fd0e00191e0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 shutdown_connections 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd0d4067b20 0x7fd0d4069fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f815e7e0 0x7fd0f815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f80b4710 0x7fd0f8154970 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 --2- 172.21.15.86:0/2309903093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f8070560 0x7fd0f8157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 >> 172.21.15.86:0/2309903093 conn(0x7fd0f80b3bf0 msgr2=0x7fd0f80b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 shutdown_connections 2024-03-22T04:10:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.716+0000 7fd0fd800640 1 -- 172.21.15.86:0/2309903093 wait complete. 2024-03-22T04:10:46.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:10:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7eac00640 1 Processor -- start 2024-03-22T04:10:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7eac00640 1 -- start start 2024-03-22T04:10:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7eac00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7e415e7e0 0x7fa7e40d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7eac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 0x7fa7e406fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7eac00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e40d15f0 con 0x7fa7e4070390 2024-03-22T04:10:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7eac00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e40b4b70 con 0x7fa7e415e7e0 2024-03-22T04:10:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7eac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e406f970 con 0x7fa7e40b4710 2024-03-22T04:10:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7e3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7e415e7e0 0x7fa7e40d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7e8c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa7e4070390 0x7fa7e409aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:46436/0 (socket says 172.21.15.86:46436) 2024-03-22T04:10:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7e3400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 0x7fa7e406fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7e8c00640 1 -- 172.21.15.86:0/2088503476 learned_addr learned my addr 172.21.15.86:0/2088503476 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7e3e00640 1 -- 172.21.15.86:0/2088503476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 msgr2=0x7fa7e406fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7e3e00640 1 --2- 172.21.15.86:0/2088503476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 0x7fa7e406fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7e3e00640 1 -- 172.21.15.86:0/2088503476 >> v1:172.21.15.196:6789/0 conn(0x7fa7e4070390 legacy=0x7fa7e409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.872+0000 7fa7e3e00640 1 -- 172.21.15.86:0/2088503476 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e4070820 con 0x7fa7e415e7e0 2024-03-22T04:10:46.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7e3e00640 1 --2- 172.21.15.86:0/2088503476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 0x7fa7e40d0d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009d90 tx=0x7fa7d401c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d4b1e71884c99a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7e2a00640 1 -- 172.21.15.86:0/2088503476 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40b2070 con 0x7fa7e415e7e0 2024-03-22T04:10:46.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7e2a00640 1 -- 172.21.15.86:0/2088503476 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d40a5800 con 0x7fa7e415e7e0 2024-03-22T04:10:46.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7e2a00640 1 -- 172.21.15.86:0/2088503476 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40a5bd0 con 0x7fa7e415e7e0 2024-03-22T04:10:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7eac00640 1 -- 172.21.15.86:0/2088503476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 msgr2=0x7fa7e40d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7eac00640 1 --2- 172.21.15.86:0/2088503476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 0x7fa7e40d0d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009d90 tx=0x7fa7d401c820 comp rx=0 tx=0).stop 2024-03-22T04:10:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7eac00640 1 -- 172.21.15.86:0/2088503476 shutdown_connections 2024-03-22T04:10:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7eac00640 1 --2- 172.21.15.86:0/2088503476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 0x7fa7e406fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7eac00640 1 --2- 172.21.15.86:0/2088503476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 0x7fa7e40d0d30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7eac00640 1 -- 172.21.15.86:0/2088503476 >> 172.21.15.86:0/2088503476 conn(0x7fa7e40b3bf0 msgr2=0x7fa7e40b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7eac00640 1 -- 172.21.15.86:0/2088503476 shutdown_connections 2024-03-22T04:10:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.876+0000 7fa7eac00640 1 -- 172.21.15.86:0/2088503476 wait complete. 2024-03-22T04:10:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7eac00640 1 Processor -- start 2024-03-22T04:10:46.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7eac00640 1 -- start start 2024-03-22T04:10:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7eac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e4070390 0x7fa7e414a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7eac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 0x7fa7e40b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7eac00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 0x7fa7e40b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7eac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40ce370 con 0x7fa7e4070390 2024-03-22T04:10:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7eac00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40cce40 con 0x7fa7e415e7e0 2024-03-22T04:10:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7eac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e40ce4f0 con 0x7fa7e40b4710 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e4070390 0x7fa7e414a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e4070390 0x7fa7e414a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35228/0 (socket says 172.21.15.86:35228) 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3e00640 1 -- 172.21.15.86:0/810065632 learned_addr learned my addr 172.21.15.86:0/810065632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3400640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 0x7fa7e40b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e8c00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 0x7fa7e40b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3e00640 1 -- 172.21.15.86:0/810065632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 msgr2=0x7fa7e40b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3e00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 0x7fa7e40b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3e00640 1 -- 172.21.15.86:0/810065632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 msgr2=0x7fa7e40b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3e00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 0x7fa7e40b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.880+0000 7fa7e3e00640 1 -- 172.21.15.86:0/810065632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e40b66c0 con 0x7fa7e4070390 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7e3e00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e4070390 0x7fa7e414a5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009d90 tx=0x7fa7d40195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7dbe00640 1 -- 172.21.15.86:0/810065632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40b2050 con 0x7fa7e4070390 2024-03-22T04:10:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7dbe00640 1 -- 172.21.15.86:0/810065632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d40be070 con 0x7fa7e4070390 2024-03-22T04:10:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7dbe00640 1 -- 172.21.15.86:0/810065632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40ad870 con 0x7fa7e4070390 2024-03-22T04:10:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e40b69a0 con 0x7fa7e4070390 2024-03-22T04:10:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e40b6f60 con 0x7fa7e4070390 2024-03-22T04:10:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e4074470 con 0x7fa7e4070390 2024-03-22T04:10:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7dbe00640 1 -- 172.21.15.86:0/810065632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa7d40ada10 con 0x7fa7e4070390 2024-03-22T04:10:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7dbe00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b50 0x7fa7c406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7dbe00640 1 -- 172.21.15.86:0/810065632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa7d4121260 con 0x7fa7e4070390 2024-03-22T04:10:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.884+0000 7fa7e3400640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b50 0x7fa7c406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:46.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.888+0000 7fa7e3400640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b50 0x7fa7c406a010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40b34d0 tx=0x7fa7d0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:46.888+0000 7fa7dbe00640 1 -- 172.21.15.86:0/810065632 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa7d4016560 con 0x7fa7e4070390 2024-03-22T04:10:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.048+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa7e406fc30 con 0x7fa7e4070390 2024-03-22T04:10:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.048+0000 7fa7dbe00640 1 -- 172.21.15.86:0/810065632 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fa7d40add00 con 0x7fa7e4070390 2024-03-22T04:10:47.053 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b50 msgr2=0x7fa7c406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b50 0x7fa7c406a010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40b34d0 tx=0x7fa7d0009190 comp rx=0 tx=0).stop 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e4070390 msgr2=0x7fa7e414a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e4070390 0x7fa7e414a5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009d90 tx=0x7fa7d40195c0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 shutdown_connections 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b50 0x7fa7c406a010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e415e7e0 0x7fa7e40b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e40b4710 0x7fa7e40b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 --2- 172.21.15.86:0/810065632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e4070390 0x7fa7e414a5a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 >> 172.21.15.86:0/810065632 conn(0x7fa7e40b3bf0 msgr2=0x7fa7e40b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 shutdown_connections 2024-03-22T04:10:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.052+0000 7fa7eac00640 1 -- 172.21.15.86:0/810065632 wait complete. 2024-03-22T04:10:47.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:10:47.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.208+0000 7ff7a5c00640 1 Processor -- start 2024-03-22T04:10:47.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.208+0000 7ff7a5c00640 1 -- start start 2024-03-22T04:10:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff7a00cf060 0x7ff7a015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff7a01558c0 0x7ff7a015f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a00cf560 con 0x7ff7a01554e0 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a00cf840 con 0x7ff7a01558c0 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a00cfb20 con 0x7ff7a00cf060 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff7a01558c0 0x7ff7a015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff79fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff7a01554e0 0x7ff7a0072d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59444/0 (socket says 172.21.15.86:59444) 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff79fe00640 1 -- 172.21.15.86:0/3475800154 learned_addr learned my addr 172.21.15.86:0/3475800154 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff79f400640 1 --2- 172.21.15.86:0/3475800154 >> v2:172.21.15.196:3300/0 conn(0x7ff7a00cf060 0x7ff7a015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5200640 1 -- 172.21.15.86:0/3475800154 >> v2:172.21.15.196:3300/0 conn(0x7ff7a00cf060 msgr2=0x7ff7a015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5200640 1 --2- 172.21.15.86:0/3475800154 >> v2:172.21.15.196:3300/0 conn(0x7ff7a00cf060 0x7ff7a015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5200640 1 -- 172.21.15.86:0/3475800154 >> v1:172.21.15.86:6789/0 conn(0x7ff7a01554e0 legacy=0x7ff7a0072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5200640 1 -- 172.21.15.86:0/3475800154 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a00cfe00 con 0x7ff7a01558c0 2024-03-22T04:10:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.212+0000 7ff7a5200640 1 --2- 172.21.15.86:0/3475800154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a01558c0 0x7ff7a015f210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff798009d90 tx=0x7ff79801eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe63be6a9380c57 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff79ea00640 1 -- 172.21.15.86:0/3475800154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980b4070 con 0x7ff7a01558c0 2024-03-22T04:10:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff79ea00640 1 -- 172.21.15.86:0/3475800154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7980a7b00 con 0x7ff7a01558c0 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff79ea00640 1 -- 172.21.15.86:0/3475800154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980af440 con 0x7ff7a01558c0 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- 172.21.15.86:0/3475800154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a01558c0 msgr2=0x7ff7a015f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/3475800154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a01558c0 0x7ff7a015f210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff798009d90 tx=0x7ff79801eb10 comp rx=0 tx=0).stop 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- 172.21.15.86:0/3475800154 shutdown_connections 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/3475800154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a01558c0 0x7ff7a015f210 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/3475800154 >> v2:172.21.15.196:3300/0 conn(0x7ff7a00cf060 0x7ff7a015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- 172.21.15.86:0/3475800154 >> 172.21.15.86:0/3475800154 conn(0x7ff7a00d0ec0 msgr2=0x7ff7a00d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- 172.21.15.86:0/3475800154 shutdown_connections 2024-03-22T04:10:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- 172.21.15.86:0/3475800154 wait complete. 2024-03-22T04:10:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 Processor -- start 2024-03-22T04:10:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- start start 2024-03-22T04:10:47.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a00cf060 0x7ff7a00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7a01554e0 0x7ff7a0141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01558c0 0x7ff7a00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a006c200 con 0x7ff7a00cf060 2024-03-22T04:10:47.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a006c080 con 0x7ff7a01558c0 2024-03-22T04:10:47.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a006c380 con 0x7ff7a01554e0 2024-03-22T04:10:47.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff79fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a00cf060 0x7ff7a00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff79f400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7a01554e0 0x7ff7a0141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff79fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a00cf060 0x7ff7a00b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35240/0 (socket says 172.21.15.86:35240) 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff79fe00640 1 -- 172.21.15.86:0/1535631584 learned_addr learned my addr 172.21.15.86:0/1535631584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.216+0000 7ff7a5200640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01558c0 0x7ff7a00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff79fe00640 1 -- 172.21.15.86:0/1535631584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7a01554e0 msgr2=0x7ff7a0141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff79fe00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7a01554e0 0x7ff7a0141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff79fe00640 1 -- 172.21.15.86:0/1535631584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01558c0 msgr2=0x7ff7a00b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff79fe00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01558c0 0x7ff7a00b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff79fe00640 1 -- 172.21.15.86:0/1535631584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a015c120 con 0x7ff7a00cf060 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff7a5200640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01558c0 0x7ff7a00b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:47.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff79fe00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a00cf060 0x7ff7a00b3f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c420 tx=0x7ff79000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff797e00640 1 -- 172.21.15.86:0/1535631584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790017070 con 0x7ff7a00cf060 2024-03-22T04:10:47.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff797e00640 1 -- 172.21.15.86:0/1535631584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff790005cf0 con 0x7ff7a00cf060 2024-03-22T04:10:47.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a00c73b0 con 0x7ff7a00cf060 2024-03-22T04:10:47.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff797e00640 1 -- 172.21.15.86:0/1535631584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900072f0 con 0x7ff7a00cf060 2024-03-22T04:10:47.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a0159530 con 0x7ff7a00cf060 2024-03-22T04:10:47.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a004f900 con 0x7ff7a00cf060 2024-03-22T04:10:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.220+0000 7ff797e00640 1 -- 172.21.15.86:0/1535631584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff79001a070 con 0x7ff7a00cf060 2024-03-22T04:10:47.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.224+0000 7ff797e00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff77c067b50 0x7ff77c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.224+0000 7ff797e00640 1 -- 172.21.15.86:0/1535631584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff790085560 con 0x7ff7a00cf060 2024-03-22T04:10:47.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.224+0000 7ff79f400640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff77c067b50 0x7ff77c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.224+0000 7ff79f400640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff77c067b50 0x7ff77c06a010 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00b5f90 tx=0x7ff788009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.228+0000 7ff797e00640 1 -- 172.21.15.86:0/1535631584 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff7900555d0 con 0x7ff7a00cf060 2024-03-22T04:10:47.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7a00b6e40 con 0x7ff7a00cf060 2024-03-22T04:10:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff797e00640 1 -- 172.21.15.86:0/1535631584 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ff790059670 con 0x7ff7a00cf060 2024-03-22T04:10:47.387 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff77c067b50 msgr2=0x7ff77c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff77c067b50 0x7ff77c06a010 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00b5f90 tx=0x7ff788009190 comp rx=0 tx=0).stop 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a00cf060 msgr2=0x7ff7a00b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a00cf060 0x7ff7a00b3f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c420 tx=0x7ff79000c8e0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 shutdown_connections 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff77c067b50 0x7ff77c06a010 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7a01558c0 0x7ff7a00b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7a01554e0 0x7ff7a0141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 --2- 172.21.15.86:0/1535631584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a00cf060 0x7ff7a00b3f20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 >> 172.21.15.86:0/1535631584 conn(0x7ff7a00d0ec0 msgr2=0x7ff7a0072d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 shutdown_connections 2024-03-22T04:10:47.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.384+0000 7ff7a5c00640 1 -- 172.21.15.86:0/1535631584 wait complete. 2024-03-22T04:10:47.407 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:10:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f4a800640 1 Processor -- start 2024-03-22T04:10:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f4a800640 1 -- start start 2024-03-22T04:10:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f4a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f440b3600 0x7f8f4415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f4a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f4415d710 con 0x7f8f440b2360 2024-03-22T04:10:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f4a800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f4415fe40 con 0x7f8f4415cee0 2024-03-22T04:10:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f4a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8f440b2740 con 0x7f8f440b3600 2024-03-22T04:10:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f43400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8f440b2360 0x7f8f440b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59446/0 (socket says 172.21.15.86:59446) 2024-03-22T04:10:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f42a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f440b3600 0x7f8f4415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f43400640 1 -- 172.21.15.86:0/159746643 learned_addr learned my addr 172.21.15.86:0/159746643 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f42a00640 1 -- 172.21.15.86:0/159746643 >> v1:172.21.15.196:6789/0 conn(0x7f8f4415cee0 legacy=0x7f8f440b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f42a00640 1 -- 172.21.15.86:0/159746643 >> v1:172.21.15.86:6789/0 conn(0x7f8f440b2360 legacy=0x7f8f440b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.544+0000 7f8f42a00640 1 -- 172.21.15.86:0/159746643 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f44160120 con 0x7f8f440b3600 2024-03-22T04:10:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f42a00640 1 --2- 172.21.15.86:0/159746643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b3600 0x7f8f4415c9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f300097c0 tx=0x7f8f3001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fc99b6e7b080e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f42000640 1 -- 172.21.15.86:0/159746643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f300a7070 con 0x7f8f440b3600 2024-03-22T04:10:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f42000640 1 -- 172.21.15.86:0/159746643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f300b0070 con 0x7f8f440b3600 2024-03-22T04:10:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f42000640 1 -- 172.21.15.86:0/159746643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f300ab890 con 0x7f8f440b3600 2024-03-22T04:10:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 -- 172.21.15.86:0/159746643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b3600 msgr2=0x7f8f4415c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 --2- 172.21.15.86:0/159746643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b3600 0x7f8f4415c9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f300097c0 tx=0x7f8f3001c470 comp rx=0 tx=0).stop 2024-03-22T04:10:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 -- 172.21.15.86:0/159746643 shutdown_connections 2024-03-22T04:10:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 --2- 172.21.15.86:0/159746643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b3600 0x7f8f4415c9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 -- 172.21.15.86:0/159746643 >> 172.21.15.86:0/159746643 conn(0x7f8f440cc940 msgr2=0x7f8f440ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 -- 172.21.15.86:0/159746643 shutdown_connections 2024-03-22T04:10:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 -- 172.21.15.86:0/159746643 wait complete. 2024-03-22T04:10:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 Processor -- start 2024-03-22T04:10:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.548+0000 7f8f4a800640 1 -- start start 2024-03-22T04:10:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b2360 0x7f8f4414ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b3600 0x7f8f44148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f4415cee0 0x7f8f4413a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4415d470 con 0x7f8f440b2360 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4415d2f0 con 0x7f8f440b3600 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4416a1f0 con 0x7f8f4415cee0 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b2360 0x7f8f4414ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b2360 0x7f8f4414ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35262/0 (socket says 172.21.15.86:35262) 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 -- 172.21.15.86:0/2978074456 learned_addr learned my addr 172.21.15.86:0/2978074456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f42a00640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b3600 0x7f8f44148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43e00640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f4415cee0 0x7f8f4413a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 -- 172.21.15.86:0/2978074456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f4415cee0 msgr2=0x7f8f4413a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f4415cee0 0x7f8f4413a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 -- 172.21.15.86:0/2978074456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b3600 msgr2=0x7f8f44148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b3600 0x7f8f44148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 -- 172.21.15.86:0/2978074456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f44150510 con 0x7f8f440b2360 2024-03-22T04:10:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f43400640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b2360 0x7f8f4414ff30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400c500 tx=0x7f8f3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f3b400640 1 -- 172.21.15.86:0/2978074456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34017070 con 0x7f8f440b2360 2024-03-22T04:10:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f3b400640 1 -- 172.21.15.86:0/2978074456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f34005cf0 con 0x7f8f440b2360 2024-03-22T04:10:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f3b400640 1 -- 172.21.15.86:0/2978074456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f340072f0 con 0x7f8f440b2360 2024-03-22T04:10:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f441507f0 con 0x7f8f440b2360 2024-03-22T04:10:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f44150db0 con 0x7f8f440b2360 2024-03-22T04:10:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f44074470 con 0x7f8f440b2360 2024-03-22T04:10:47.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.552+0000 7f8f3b400640 1 -- 172.21.15.86:0/2978074456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8f3401a070 con 0x7f8f440b2360 2024-03-22T04:10:47.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.556+0000 7f8f3b400640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f24067b20 0x7f8f24069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.556+0000 7f8f3b400640 1 -- 172.21.15.86:0/2978074456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8f340854d0 con 0x7f8f440b2360 2024-03-22T04:10:47.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.556+0000 7f8f42a00640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f24067b20 0x7f8f24069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.556+0000 7f8f42a00640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f24067b20 0x7f8f24069fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f300af000 tx=0x7f8f300a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.560+0000 7f8f3b400640 1 -- 172.21.15.86:0/2978074456 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8f340555e0 con 0x7f8f440b2360 2024-03-22T04:10:47.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8f4406e9f0 con 0x7f8f440b2360 2024-03-22T04:10:47.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f3b400640 1 -- 172.21.15.86:0/2978074456 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f8f34059680 con 0x7f8f440b2360 2024-03-22T04:10:47.719 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f24067b20 msgr2=0x7f8f24069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f24067b20 0x7f8f24069fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f300af000 tx=0x7f8f300a7000 comp rx=0 tx=0).stop 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b2360 msgr2=0x7f8f4414ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b2360 0x7f8f4414ff30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400c500 tx=0x7f8f3400c9c0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 shutdown_connections 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f24067b20 0x7f8f24069fe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b3600 0x7f8f44148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f4415cee0 0x7f8f4413a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 --2- 172.21.15.86:0/2978074456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f440b2360 0x7f8f4414ff30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 >> 172.21.15.86:0/2978074456 conn(0x7f8f440cc940 msgr2=0x7f8f440ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 shutdown_connections 2024-03-22T04:10:47.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.716+0000 7f8f4a800640 1 -- 172.21.15.86:0/2978074456 wait complete. 2024-03-22T04:10:47.740 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:10:47.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe76ba00640 1 Processor -- start 2024-03-22T04:10:47.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe76ba00640 1 -- start start 2024-03-22T04:10:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe76ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7640b4350 0x7fe7640b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe76ba00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe76415ef40 0x7fe76415f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe76ba00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76415ea90 con 0x7fe764147880 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe76ba00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe7640b6320 con 0x7fe76415ef40 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe76ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe7640b4cf0 con 0x7fe7640b4350 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe75be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7640b4350 0x7fe7640b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe75be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7640b4350 0x7fe7640b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:50968/0 (socket says 172.21.15.86:50968) 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.872+0000 7fe769000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe764147880 0x7fe76406c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:46452/0 (socket says 172.21.15.86:46452) 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe75be00640 1 -- 172.21.15.86:0/725783178 learned_addr learned my addr 172.21.15.86:0/725783178 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:47.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe769a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe76415ef40 0x7fe76415f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe763e00640 1 -- 172.21.15.86:0/725783178 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403043457 0 0) 0x7fe76415ea90 con 0x7fe764147880 2024-03-22T04:10:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe75be00640 1 -- 172.21.15.86:0/725783178 >> v2:172.21.15.196:3300/0 conn(0x7fe76415ef40 msgr2=0x7fe76415f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe75be00640 1 --2- 172.21.15.86:0/725783178 >> v2:172.21.15.196:3300/0 conn(0x7fe76415ef40 0x7fe76415f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe75be00640 1 -- 172.21.15.86:0/725783178 >> v1:172.21.15.196:6789/0 conn(0x7fe764147880 legacy=0x7fe76406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe75be00640 1 -- 172.21.15.86:0/725783178 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76406d4f0 con 0x7fe7640b4350 2024-03-22T04:10:47.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe769a00640 1 --2- 172.21.15.86:0/725783178 >> v2:172.21.15.196:3300/0 conn(0x7fe76415ef40 0x7fe76415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:47.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe75be00640 1 --2- 172.21.15.86:0/725783178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7640b4350 0x7fe7640b47b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009870 tx=0x7fe74c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7804c85f2421e46 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe763e00640 1 -- 172.21.15.86:0/725783178 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c0ab650 con 0x7fe7640b4350 2024-03-22T04:10:47.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe763e00640 1 -- 172.21.15.86:0/725783178 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c0b0070 con 0x7fe7640b4350 2024-03-22T04:10:47.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.876+0000 7fe763e00640 1 -- 172.21.15.86:0/725783178 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c0aba10 con 0x7fe7640b4350 2024-03-22T04:10:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- 172.21.15.86:0/725783178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7640b4350 msgr2=0x7fe7640b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 --2- 172.21.15.86:0/725783178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7640b4350 0x7fe7640b47b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009870 tx=0x7fe74c01c560 comp rx=0 tx=0).stop 2024-03-22T04:10:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- 172.21.15.86:0/725783178 shutdown_connections 2024-03-22T04:10:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 --2- 172.21.15.86:0/725783178 >> v2:172.21.15.196:3300/0 conn(0x7fe76415ef40 0x7fe76415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 --2- 172.21.15.86:0/725783178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7640b4350 0x7fe7640b47b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- 172.21.15.86:0/725783178 >> 172.21.15.86:0/725783178 conn(0x7fe76406ba60 msgr2=0x7fe76406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- 172.21.15.86:0/725783178 shutdown_connections 2024-03-22T04:10:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- 172.21.15.86:0/725783178 wait complete. 2024-03-22T04:10:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 Processor -- start 2024-03-22T04:10:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- start start 2024-03-22T04:10:47.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7640b4350 0x7fe764165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe764147880 0x7fe76415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76415ef40 0x7fe76413f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe76406d090 con 0x7fe7640b4350 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641480a0 con 0x7fe76415ef40 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe76ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe76406d210 con 0x7fe764147880 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe769000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7640b4350 0x7fe764165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe769000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7640b4350 0x7fe764165850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35264/0 (socket says 172.21.15.86:35264) 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe769000640 1 -- 172.21.15.86:0/366183410 learned_addr learned my addr 172.21.15.86:0/366183410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe75be00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe764147880 0x7fe76415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe769a00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76415ef40 0x7fe76413f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe769000640 1 -- 172.21.15.86:0/366183410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe764147880 msgr2=0x7fe76415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.880+0000 7fe769000640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe764147880 0x7fe76415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe769000640 1 -- 172.21.15.86:0/366183410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76415ef40 msgr2=0x7fe76413f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:47.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe769000640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76415ef40 0x7fe76413f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe769000640 1 -- 172.21.15.86:0/366183410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76415c330 con 0x7fe7640b4350 2024-03-22T04:10:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe769000640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7640b4350 0x7fe764165850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c520 tx=0x7fe75400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe761600640 1 -- 172.21.15.86:0/366183410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754018070 con 0x7fe7640b4350 2024-03-22T04:10:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe761600640 1 -- 172.21.15.86:0/366183410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7540074e0 con 0x7fe7640b4350 2024-03-22T04:10:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe761600640 1 -- 172.21.15.86:0/366183410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540049f0 con 0x7fe7640b4350 2024-03-22T04:10:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe764165f30 con 0x7fe7640b4350 2024-03-22T04:10:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641664f0 con 0x7fe7640b4350 2024-03-22T04:10:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.884+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764068650 con 0x7fe7640b4350 2024-03-22T04:10:47.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.888+0000 7fe761600640 1 -- 172.21.15.86:0/366183410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe754012400 con 0x7fe7640b4350 2024-03-22T04:10:47.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.888+0000 7fe761600640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe740067b20 0x7fe740069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:47.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.888+0000 7fe761600640 1 -- 172.21.15.86:0/366183410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe754085810 con 0x7fe7640b4350 2024-03-22T04:10:47.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.888+0000 7fe75be00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe740067b20 0x7fe740069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:47.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.888+0000 7fe75be00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe740067b20 0x7fe740069fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0af000 tx=0x7fe74c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:47.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:47.892+0000 7fe761600640 1 -- 172.21.15.86:0/366183410 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe754055970 con 0x7fe7640b4350 2024-03-22T04:10:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.056+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe764072220 con 0x7fe7640b4350 2024-03-22T04:10:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe761600640 1 -- 172.21.15.86:0/366183410 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fe754059a10 con 0x7fe7640b4350 2024-03-22T04:10:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:10:48.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe740067b20 msgr2=0x7fe740069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe740067b20 0x7fe740069fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0af000 tx=0x7fe74c0a7000 comp rx=0 tx=0).stop 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7640b4350 msgr2=0x7fe764165850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7640b4350 0x7fe764165850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c520 tx=0x7fe75400c920 comp rx=0 tx=0).stop 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 shutdown_connections 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe740067b20 0x7fe740069fe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76415ef40 0x7fe76413f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe764147880 0x7fe76415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 --2- 172.21.15.86:0/366183410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7640b4350 0x7fe764165850 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 >> 172.21.15.86:0/366183410 conn(0x7fe76406ba60 msgr2=0x7fe7640b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 shutdown_connections 2024-03-22T04:10:48.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.060+0000 7fe76ba00640 1 -- 172.21.15.86:0/366183410 wait complete. 2024-03-22T04:10:48.084 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:10:48.084 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:10:48.092 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:48.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa8b200640 1 Processor -- start 2024-03-22T04:10:48.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa8b200640 1 -- start start 2024-03-22T04:10:48.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa8b200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c0b0750 0x7faa8c15f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa8b200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa8c06f060 0x7faa8c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa8b200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa8c06be00 con 0x7faa8c15be40 2024-03-22T04:10:48.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa8b200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faa8c06bba0 con 0x7faa8c06f060 2024-03-22T04:10:48.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa8b200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faa8c06c0e0 con 0x7faa8c0b0750 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa81400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c0b0750 0x7faa8c15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa89e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7faa8c15be40 0x7faa8c0b0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:46460/0 (socket says 172.21.15.86:46460) 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa89e00640 1 -- 172.21.15.86:0/2438553677 learned_addr learned my addr 172.21.15.86:0/2438553677 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa8a800640 1 --2- 172.21.15.86:0/2438553677 >> v2:172.21.15.196:3300/0 conn(0x7faa8c06f060 0x7faa8c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa89400640 1 -- 172.21.15.86:0/2438553677 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121548164 0 0) 0x7faa8c06be00 con 0x7faa8c15be40 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa89400640 1 -- 172.21.15.86:0/2438553677 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa70003530 con 0x7faa8c15be40 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa81400640 1 -- 172.21.15.86:0/2438553677 >> v2:172.21.15.196:3300/0 conn(0x7faa8c06f060 msgr2=0x7faa8c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa81400640 1 --2- 172.21.15.86:0/2438553677 >> v2:172.21.15.196:3300/0 conn(0x7faa8c06f060 0x7faa8c06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.264+0000 7faa81400640 1 -- 172.21.15.86:0/2438553677 >> v1:172.21.15.196:6789/0 conn(0x7faa8c15be40 legacy=0x7faa8c0b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa81400640 1 -- 172.21.15.86:0/2438553677 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c15ec60 con 0x7faa8c0b0750 2024-03-22T04:10:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa81400640 1 --2- 172.21.15.86:0/2438553677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c0b0750 0x7faa8c15f460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faa740097c0 tx=0x7faa7401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15af3ed571cfc21c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa89400640 1 -- 172.21.15.86:0/2438553677 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa740b1070 con 0x7faa8c0b0750 2024-03-22T04:10:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa89400640 1 -- 172.21.15.86:0/2438553677 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa740ac600 con 0x7faa8c0b0750 2024-03-22T04:10:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa89400640 1 -- 172.21.15.86:0/2438553677 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa740ac9d0 con 0x7faa8c0b0750 2024-03-22T04:10:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa8b200640 1 -- 172.21.15.86:0/2438553677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c0b0750 msgr2=0x7faa8c15f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa8b200640 1 --2- 172.21.15.86:0/2438553677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c0b0750 0x7faa8c15f460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faa740097c0 tx=0x7faa7401c470 comp rx=0 tx=0).stop 2024-03-22T04:10:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa8b200640 1 -- 172.21.15.86:0/2438553677 shutdown_connections 2024-03-22T04:10:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa8b200640 1 --2- 172.21.15.86:0/2438553677 >> v2:172.21.15.196:3300/0 conn(0x7faa8c06f060 0x7faa8c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa8b200640 1 --2- 172.21.15.86:0/2438553677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c0b0750 0x7faa8c15f460 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa8b200640 1 -- 172.21.15.86:0/2438553677 >> 172.21.15.86:0/2438553677 conn(0x7faa8c06e540 msgr2=0x7faa8c06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa8b200640 1 -- 172.21.15.86:0/2438553677 shutdown_connections 2024-03-22T04:10:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.268+0000 7faa8b200640 1 -- 172.21.15.86:0/2438553677 wait complete. 2024-03-22T04:10:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 Processor -- start 2024-03-22T04:10:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 -- start start 2024-03-22T04:10:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8c06f060 0x7faa8c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa8c0b0750 0x7faa8c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c15be40 0x7faa8c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c0b49e0 con 0x7faa8c06f060 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c0b4860 con 0x7faa8c0b0750 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faa8c0b4b60 con 0x7faa8c15be40 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8c06f060 0x7faa8c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8c06f060 0x7faa8c0cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35274/0 (socket says 172.21.15.86:35274) 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 -- 172.21.15.86:0/999413462 learned_addr learned my addr 172.21.15.86:0/999413462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa81400640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa8c0b0750 0x7faa8c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8a800640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c15be40 0x7faa8c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 -- 172.21.15.86:0/999413462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c15be40 msgr2=0x7faa8c15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c15be40 0x7faa8c15a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 -- 172.21.15.86:0/999413462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa8c0b0750 msgr2=0x7faa8c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa8c0b0750 0x7faa8c15a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 -- 172.21.15.86:0/999413462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c1fa450 con 0x7faa8c06f060 2024-03-22T04:10:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa89e00640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8c06f060 0x7faa8c0cf180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00c3c0 tx=0x7faa7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa82a00640 1 -- 172.21.15.86:0/999413462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c017070 con 0x7faa8c06f060 2024-03-22T04:10:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa82a00640 1 -- 172.21.15.86:0/999413462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7c005cf0 con 0x7faa8c06f060 2024-03-22T04:10:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa82a00640 1 -- 172.21.15.86:0/999413462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0072f0 con 0x7faa8c06f060 2024-03-22T04:10:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c1fa730 con 0x7faa8c06f060 2024-03-22T04:10:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.272+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c1facf0 con 0x7faa8c06f060 2024-03-22T04:10:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.276+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8c074470 con 0x7faa8c06f060 2024-03-22T04:10:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.276+0000 7faa82a00640 1 -- 172.21.15.86:0/999413462 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7faa7c01a070 con 0x7faa8c06f060 2024-03-22T04:10:48.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.276+0000 7faa82a00640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa68067b20 0x7faa68069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.276+0000 7faa82a00640 1 -- 172.21.15.86:0/999413462 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7faa7c085530 con 0x7faa8c06f060 2024-03-22T04:10:48.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.276+0000 7faa81400640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa68067b20 0x7faa68069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.276+0000 7faa81400640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa68067b20 0x7faa68069fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa740ae000 tx=0x7faa740a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.280+0000 7faa82a00640 1 -- 172.21.15.86:0/999413462 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7faa7c055640 con 0x7faa8c06f060 2024-03-22T04:10:48.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.436+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa8c073750 con 0x7faa8c06f060 2024-03-22T04:10:48.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa82a00640 1 -- 172.21.15.86:0/999413462 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7faa7c0596e0 con 0x7faa8c06f060 2024-03-22T04:10:48.442 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:48.442 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa68067b20 msgr2=0x7faa68069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa68067b20 0x7faa68069fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa740ae000 tx=0x7faa740a7000 comp rx=0 tx=0).stop 2024-03-22T04:10:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8c06f060 msgr2=0x7faa8c0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8c06f060 0x7faa8c0cf180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00c3c0 tx=0x7faa7c00c880 comp rx=0 tx=0).stop 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 shutdown_connections 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa68067b20 0x7faa68069fe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa8c0b0750 0x7faa8c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa8c15be40 0x7faa8c15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 --2- 172.21.15.86:0/999413462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8c06f060 0x7faa8c0cf180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 >> 172.21.15.86:0/999413462 conn(0x7faa8c06e540 msgr2=0x7faa8c0b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 shutdown_connections 2024-03-22T04:10:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.440+0000 7faa8b200640 1 -- 172.21.15.86:0/999413462 wait complete. 2024-03-22T04:10:48.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:10:48.464 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:10:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.596+0000 7f2abb400640 1 Processor -- start 2024-03-22T04:10:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2abb400640 1 -- start start 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2abb400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ab40b2360 0x7f2ab40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2abb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2ab40b3430 0x7f2ab415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2abb400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab415d710 con 0x7f2ab415cee0 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2abb400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab40b2740 con 0x7f2ab40b3430 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2abb400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab415fe40 con 0x7f2ab40b2360 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2aabe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2ab40b3430 0x7f2ab415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2ab9400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2ab415cee0 0x7f2ab406d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59462/0 (socket says 172.21.15.86:59462) 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2ab9400640 1 -- 172.21.15.86:0/3984735156 learned_addr learned my addr 172.21.15.86:0/3984735156 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2ab8a00640 1 --2- 172.21.15.86:0/3984735156 >> v2:172.21.15.196:3300/0 conn(0x7f2ab40b2360 0x7f2ab40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2aabe00640 1 -- 172.21.15.86:0/3984735156 >> v2:172.21.15.196:3300/0 conn(0x7f2ab40b2360 msgr2=0x7f2ab40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:48.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2aabe00640 1 --2- 172.21.15.86:0/3984735156 >> v2:172.21.15.196:3300/0 conn(0x7f2ab40b2360 0x7f2ab40b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2aabe00640 1 -- 172.21.15.86:0/3984735156 >> v1:172.21.15.86:6789/0 conn(0x7f2ab415cee0 legacy=0x7f2ab406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2aabe00640 1 -- 172.21.15.86:0/3984735156 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab4160120 con 0x7f2ab40b3430 2024-03-22T04:10:48.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.600+0000 7f2ab3e00640 1 -- 172.21.15.86:0/3984735156 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744086153 0 0) 0x7f2ab415d710 con 0x7f2ab415cee0 2024-03-22T04:10:48.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2aabe00640 1 --2- 172.21.15.86:0/3984735156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab415c9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40683e0 tx=0x7f2a9c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ef348381051fa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2ab3e00640 1 -- 172.21.15.86:0/3984735156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0b0070 con 0x7f2ab40b3430 2024-03-22T04:10:48.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2ab3e00640 1 -- 172.21.15.86:0/3984735156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a9c0ab600 con 0x7f2ab40b3430 2024-03-22T04:10:48.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2ab3e00640 1 -- 172.21.15.86:0/3984735156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0ab9d0 con 0x7f2ab40b3430 2024-03-22T04:10:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 -- 172.21.15.86:0/3984735156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 msgr2=0x7f2ab415c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 --2- 172.21.15.86:0/3984735156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab415c9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40683e0 tx=0x7f2a9c01c480 comp rx=0 tx=0).stop 2024-03-22T04:10:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 -- 172.21.15.86:0/3984735156 shutdown_connections 2024-03-22T04:10:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 --2- 172.21.15.86:0/3984735156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab415c9a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 --2- 172.21.15.86:0/3984735156 >> v2:172.21.15.196:3300/0 conn(0x7f2ab40b2360 0x7f2ab40b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 -- 172.21.15.86:0/3984735156 >> 172.21.15.86:0/3984735156 conn(0x7f2ab40cc940 msgr2=0x7f2ab40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 -- 172.21.15.86:0/3984735156 shutdown_connections 2024-03-22T04:10:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 -- 172.21.15.86:0/3984735156 wait complete. 2024-03-22T04:10:48.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 Processor -- start 2024-03-22T04:10:48.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.604+0000 7f2abb400640 1 -- start start 2024-03-22T04:10:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab40b2360 0x7f2ab414a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab4149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab415cee0 0x7f2ab4155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4160c00 con 0x7f2ab40b3430 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab415d2c0 con 0x7f2ab415cee0 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab415bea0 con 0x7f2ab40b2360 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab4149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab4149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35290/0 (socket says 172.21.15.86:35290) 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 -- 172.21.15.86:0/738717459 learned_addr learned my addr 172.21.15.86:0/738717459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2ab8a00640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab40b2360 0x7f2ab414a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2ab9400640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab415cee0 0x7f2ab4155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 -- 172.21.15.86:0/738717459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab40b2360 msgr2=0x7f2ab414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab40b2360 0x7f2ab414a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 -- 172.21.15.86:0/738717459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab415cee0 msgr2=0x7f2ab4155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab415cee0 0x7f2ab4155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 -- 172.21.15.86:0/738717459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab415b320 con 0x7f2ab40b3430 2024-03-22T04:10:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2aabe00640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab4149be0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c01a0b0 tx=0x7f2a9c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2ab1600640 1 -- 172.21.15.86:0/738717459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0b0050 con 0x7f2ab40b3430 2024-03-22T04:10:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2ab1600640 1 -- 172.21.15.86:0/738717459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a9c0b1040 con 0x7f2ab40b3430 2024-03-22T04:10:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab415b600 con 0x7f2ab40b3430 2024-03-22T04:10:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2ab1600640 1 -- 172.21.15.86:0/738717459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c004220 con 0x7f2ab40b3430 2024-03-22T04:10:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab415bbc0 con 0x7f2ab40b3430 2024-03-22T04:10:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab4074470 con 0x7f2ab40b3430 2024-03-22T04:10:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.608+0000 7f2ab1600640 1 -- 172.21.15.86:0/738717459 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2a9c0be060 con 0x7f2ab40b3430 2024-03-22T04:10:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.612+0000 7f2ab1600640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a90067b20 0x7f2a90069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.612+0000 7f2ab1600640 1 -- 172.21.15.86:0/738717459 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2a9c11fb50 con 0x7f2ab40b3430 2024-03-22T04:10:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.612+0000 7f2ab8a00640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a90067b20 0x7f2a90069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.612+0000 7f2ab8a00640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a90067b20 0x7f2a90069fe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ab406dd00 tx=0x7f2aa4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.616+0000 7f2ab1600640 1 -- 172.21.15.86:0/738717459 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2a9c016070 con 0x7f2ab40b3430 2024-03-22T04:10:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.768+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ab406e9f0 con 0x7f2ab40b3430 2024-03-22T04:10:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.768+0000 7f2ab1600640 1 -- 172.21.15.86:0/738717459 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f2a9c0efbc0 con 0x7f2ab40b3430 2024-03-22T04:10:48.773 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:10:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a90067b20 msgr2=0x7f2a90069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a90067b20 0x7f2a90069fe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ab406dd00 tx=0x7f2aa4008040 comp rx=0 tx=0).stop 2024-03-22T04:10:48.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 msgr2=0x7f2ab4149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab4149be0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c01a0b0 tx=0x7f2a9c01acc0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 shutdown_connections 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a90067b20 0x7f2a90069fe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab415cee0 0x7f2ab4155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40b3430 0x7f2ab4149be0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 --2- 172.21.15.86:0/738717459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab40b2360 0x7f2ab414a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 >> 172.21.15.86:0/738717459 conn(0x7f2ab40cc940 msgr2=0x7f2ab415e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 shutdown_connections 2024-03-22T04:10:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.772+0000 7f2abb400640 1 -- 172.21.15.86:0/738717459 wait complete. 2024-03-22T04:10:48.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:10:48.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbffe00640 1 Processor -- start 2024-03-22T04:10:48.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbffe00640 1 -- start start 2024-03-22T04:10:48.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbffe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80b3600 0x7fdbf815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbffe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 0x7fdbf80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbffe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf815d710 con 0x7fdbf80b2360 2024-03-22T04:10:48.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbffe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf80b2740 con 0x7fdbf80b3600 2024-03-22T04:10:48.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbffe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf815fe40 con 0x7fdbf815cee0 2024-03-22T04:10:48.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80b3600 0x7fdbf815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfd400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdbf80b2360 0x7fdbf80b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59478/0 (socket says 172.21.15.86:59478) 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80b3600 0x7fdbf815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35302/0 (socket says 172.21.15.86:35302) 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfd400640 1 -- 172.21.15.86:0/3359371916 learned_addr learned my addr 172.21.15.86:0/3359371916 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfde00640 1 --2- 172.21.15.86:0/3359371916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 0x7fdbf80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfca00640 1 -- 172.21.15.86:0/3359371916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 msgr2=0x7fdbf80b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfca00640 1 --2- 172.21.15.86:0/3359371916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 0x7fdbf80b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfca00640 1 -- 172.21.15.86:0/3359371916 >> v1:172.21.15.86:6789/0 conn(0x7fdbf80b2360 legacy=0x7fdbf80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbfca00640 1 -- 172.21.15.86:0/3359371916 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf8160120 con 0x7fdbf80b3600 2024-03-22T04:10:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.932+0000 7fdbf7e00640 1 -- 172.21.15.86:0/3359371916 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3510538897 0 0) 0x7fdbf815d710 con 0x7fdbf80b2360 2024-03-22T04:10:48.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbfca00640 1 --2- 172.21.15.86:0/3359371916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b3600 0x7fdbf815c9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40097c0 tx=0x7fdbe401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a282f091fe5abf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbf7e00640 1 -- 172.21.15.86:0/3359371916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40a7070 con 0x7fdbf80b3600 2024-03-22T04:10:48.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbf7e00640 1 -- 172.21.15.86:0/3359371916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe40b0070 con 0x7fdbf80b3600 2024-03-22T04:10:48.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbf7e00640 1 -- 172.21.15.86:0/3359371916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40ab890 con 0x7fdbf80b3600 2024-03-22T04:10:48.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbffe00640 1 -- 172.21.15.86:0/3359371916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b3600 msgr2=0x7fdbf815c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbffe00640 1 --2- 172.21.15.86:0/3359371916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b3600 0x7fdbf815c9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40097c0 tx=0x7fdbe401c470 comp rx=0 tx=0).stop 2024-03-22T04:10:48.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbffe00640 1 -- 172.21.15.86:0/3359371916 shutdown_connections 2024-03-22T04:10:48.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbffe00640 1 --2- 172.21.15.86:0/3359371916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 0x7fdbf80b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbffe00640 1 --2- 172.21.15.86:0/3359371916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b3600 0x7fdbf815c9a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbffe00640 1 -- 172.21.15.86:0/3359371916 >> 172.21.15.86:0/3359371916 conn(0x7fdbf80cc940 msgr2=0x7fdbf80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:48.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbffe00640 1 -- 172.21.15.86:0/3359371916 shutdown_connections 2024-03-22T04:10:48.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.936+0000 7fdbffe00640 1 -- 172.21.15.86:0/3359371916 wait complete. 2024-03-22T04:10:48.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 Processor -- start 2024-03-22T04:10:48.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 -- start start 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b2360 0x7fdbf81538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80b3600 0x7fdbf814e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 0x7fdbf8145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf815d470 con 0x7fdbf80b2360 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf815d2f0 con 0x7fdbf80b3600 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf80b5ad0 con 0x7fdbf815cee0 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b2360 0x7fdbf81538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b2360 0x7fdbf81538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35308/0 (socket says 172.21.15.86:35308) 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 -- 172.21.15.86:0/920184217 learned_addr learned my addr 172.21.15.86:0/920184217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:48.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfde00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 0x7fdbf8145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfca00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80b3600 0x7fdbf814e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 -- 172.21.15.86:0/920184217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 msgr2=0x7fdbf8145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 0x7fdbf8145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 -- 172.21.15.86:0/920184217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80b3600 msgr2=0x7fdbf814e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80b3600 0x7fdbf814e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 -- 172.21.15.86:0/920184217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf814ed80 con 0x7fdbf80b2360 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbfd400640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b2360 0x7fdbf81538a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c580 tx=0x7fdbec00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbf5600640 1 -- 172.21.15.86:0/920184217 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec017070 con 0x7fdbf80b2360 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbf5600640 1 -- 172.21.15.86:0/920184217 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec005d10 con 0x7fdbf80b2360 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbf5600640 1 -- 172.21.15.86:0/920184217 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec007330 con 0x7fdbf80b2360 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf80afd20 con 0x7fdbf80b2360 2024-03-22T04:10:48.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.940+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf80b02e0 con 0x7fdbf80b2360 2024-03-22T04:10:48.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.944+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf8074470 con 0x7fdbf80b2360 2024-03-22T04:10:48.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.944+0000 7fdbf5600640 1 -- 172.21.15.86:0/920184217 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdbec01a070 con 0x7fdbf80b2360 2024-03-22T04:10:48.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.944+0000 7fdbf5600640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd8067b20 0x7fdbd8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:48.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.944+0000 7fdbf5600640 1 -- 172.21.15.86:0/920184217 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdbec085610 con 0x7fdbf80b2360 2024-03-22T04:10:48.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.944+0000 7fdbfca00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd8067b20 0x7fdbd8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:48.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.944+0000 7fdbfca00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd8067b20 0x7fdbd8069fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40af000 tx=0x7fdbe40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:48.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:48.948+0000 7fdbf5600640 1 -- 172.21.15.86:0/920184217 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdbec0556a0 con 0x7fdbf80b2360 2024-03-22T04:10:49.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdbf80af8f0 con 0x7fdbf80b2360 2024-03-22T04:10:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbf5600640 1 -- 172.21.15.86:0/920184217 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fdbec059740 con 0x7fdbf80b2360 2024-03-22T04:10:49.107 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:10:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd8067b20 msgr2=0x7fdbd8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd8067b20 0x7fdbd8069fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40af000 tx=0x7fdbe40a7000 comp rx=0 tx=0).stop 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b2360 msgr2=0x7fdbf81538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b2360 0x7fdbf81538a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c580 tx=0x7fdbec00ca40 comp rx=0 tx=0).stop 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 shutdown_connections 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd8067b20 0x7fdbd8069fe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80b3600 0x7fdbf814e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf815cee0 0x7fdbf8145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 --2- 172.21.15.86:0/920184217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80b2360 0x7fdbf81538a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 >> 172.21.15.86:0/920184217 conn(0x7fdbf80cc940 msgr2=0x7fdbf80d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 shutdown_connections 2024-03-22T04:10:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.104+0000 7fdbffe00640 1 -- 172.21.15.86:0/920184217 wait complete. 2024-03-22T04:10:49.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:10:49.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86f6800640 1 Processor -- start 2024-03-22T04:10:49.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86f6800640 1 -- start start 2024-03-22T04:10:49.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86f6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86f015e7e0 0x7f86f00d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86f6800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f00b4710 0x7f86f006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86f6800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86f0070390 0x7f86f009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86f6800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f86f00b4b70 con 0x7f86f015e7e0 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86f6800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f86f006f970 con 0x7f86f0070390 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86f6800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86f00d15f0 con 0x7f86f00b4710 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86ef400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86f015e7e0 0x7f86f00d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86ef400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86f015e7e0 0x7f86f00d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35318/0 (socket says 172.21.15.86:35318) 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86ef400640 1 -- 172.21.15.86:0/3855642372 learned_addr learned my addr 172.21.15.86:0/3855642372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86efe00640 1 --2- 172.21.15.86:0/3855642372 >> v2:172.21.15.196:3300/0 conn(0x7f86f0070390 0x7f86f009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86e6a00640 1 --2- 172.21.15.86:0/3855642372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f00b4710 0x7f86f006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86ef400640 1 -- 172.21.15.86:0/3855642372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f00b4710 msgr2=0x7f86f006fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86ef400640 1 --2- 172.21.15.86:0/3855642372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f00b4710 0x7f86f006fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86ef400640 1 -- 172.21.15.86:0/3855642372 >> v2:172.21.15.196:3300/0 conn(0x7f86f0070390 msgr2=0x7f86f009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86ef400640 1 --2- 172.21.15.86:0/3855642372 >> v2:172.21.15.196:3300/0 conn(0x7f86f0070390 0x7f86f009aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86ef400640 1 -- 172.21.15.86:0/3855642372 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f0070820 con 0x7f86f015e7e0 2024-03-22T04:10:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.264+0000 7f86e6a00640 1 --2- 172.21.15.86:0/3855642372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f00b4710 0x7f86f006fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:49.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86ef400640 1 --2- 172.21.15.86:0/3855642372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f00d0d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f86f00cc650 tx=0x7f86e001c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6e0dec86f0bfd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86eea00640 1 -- 172.21.15.86:0/3855642372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00b0070 con 0x7f86f015e7e0 2024-03-22T04:10:49.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86eea00640 1 -- 172.21.15.86:0/3855642372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e00ab600 con 0x7f86f015e7e0 2024-03-22T04:10:49.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86eea00640 1 -- 172.21.15.86:0/3855642372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00ab9d0 con 0x7f86f015e7e0 2024-03-22T04:10:49.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 -- 172.21.15.86:0/3855642372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 msgr2=0x7f86f00d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 --2- 172.21.15.86:0/3855642372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f00d0d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f86f00cc650 tx=0x7f86e001c110 comp rx=0 tx=0).stop 2024-03-22T04:10:49.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 -- 172.21.15.86:0/3855642372 shutdown_connections 2024-03-22T04:10:49.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 --2- 172.21.15.86:0/3855642372 >> v2:172.21.15.196:3300/0 conn(0x7f86f0070390 0x7f86f009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 --2- 172.21.15.86:0/3855642372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f00b4710 0x7f86f006fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 --2- 172.21.15.86:0/3855642372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f00d0d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 -- 172.21.15.86:0/3855642372 >> 172.21.15.86:0/3855642372 conn(0x7f86f00b3bf0 msgr2=0x7f86f00b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:49.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 -- 172.21.15.86:0/3855642372 shutdown_connections 2024-03-22T04:10:49.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.268+0000 7f86f6800640 1 -- 172.21.15.86:0/3855642372 wait complete. 2024-03-22T04:10:49.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 Processor -- start 2024-03-22T04:10:49.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 -- start start 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f0070390 0x7f86f0154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f00b4710 0x7f86f015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f015da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86f00cd360 con 0x7f86f015e7e0 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86f00cce40 con 0x7f86f00b4710 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86f00cd4e0 con 0x7f86f0070390 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f015da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35324/0 (socket says 172.21.15.86:35324) 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 -- 172.21.15.86:0/3214925178 learned_addr learned my addr 172.21.15.86:0/3214925178 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86e6a00640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f00b4710 0x7f86f015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86ef400640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f0070390 0x7f86f0154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 -- 172.21.15.86:0/3214925178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f0070390 msgr2=0x7f86f0154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f0070390 0x7f86f0154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 -- 172.21.15.86:0/3214925178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f00b4710 msgr2=0x7f86f015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f00b4710 0x7f86f015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 -- 172.21.15.86:0/3214925178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f0154fd0 con 0x7f86f015e7e0 2024-03-22T04:10:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86efe00640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f015da80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e960 tx=0x7f86e800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86e7e00640 1 -- 172.21.15.86:0/3214925178 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8017070 con 0x7f86f015e7e0 2024-03-22T04:10:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86e7e00640 1 -- 172.21.15.86:0/3214925178 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e800c9a0 con 0x7f86f015e7e0 2024-03-22T04:10:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86e7e00640 1 -- 172.21.15.86:0/3214925178 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e800cd70 con 0x7f86f015e7e0 2024-03-22T04:10:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f00b12d0 con 0x7f86f015e7e0 2024-03-22T04:10:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f00c65f0 con 0x7f86f015e7e0 2024-03-22T04:10:49.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.272+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f0074470 con 0x7f86f015e7e0 2024-03-22T04:10:49.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.276+0000 7f86e7e00640 1 -- 172.21.15.86:0/3214925178 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f86e801a070 con 0x7f86f015e7e0 2024-03-22T04:10:49.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.276+0000 7f86e7e00640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f86cc067b20 0x7f86cc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.276+0000 7f86e7e00640 1 -- 172.21.15.86:0/3214925178 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f86e8086130 con 0x7f86f015e7e0 2024-03-22T04:10:49.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.276+0000 7f86ef400640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f86cc067b20 0x7f86cc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.276+0000 7f86ef400640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f86cc067b20 0x7f86cc069fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86e00ad000 tx=0x7f86e00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.280+0000 7f86e7e00640 1 -- 172.21.15.86:0/3214925178 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f86f00b12d0 con 0x7f86f015e7e0 2024-03-22T04:10:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f86f006fc30 con 0x7f86f015e7e0 2024-03-22T04:10:49.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86e7e00640 1 -- 172.21.15.86:0/3214925178 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f86e8056240 con 0x7f86f015e7e0 2024-03-22T04:10:49.439 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:10:49.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f86cc067b20 msgr2=0x7f86cc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f86cc067b20 0x7f86cc069fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86e00ad000 tx=0x7f86e00a6000 comp rx=0 tx=0).stop 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 msgr2=0x7f86f015da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f015da80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e960 tx=0x7f86e800ee20 comp rx=0 tx=0).stop 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 shutdown_connections 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f86cc067b20 0x7f86cc069fe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f00b4710 0x7f86f015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f015e7e0 0x7f86f015da80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 --2- 172.21.15.86:0/3214925178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86f0070390 0x7f86f0154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 >> 172.21.15.86:0/3214925178 conn(0x7f86f00b3bf0 msgr2=0x7f86f00b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 shutdown_connections 2024-03-22T04:10:49.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.436+0000 7f86f6800640 1 -- 172.21.15.86:0/3214925178 wait complete. 2024-03-22T04:10:49.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:10:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.592+0000 7f254ac00640 1 Processor -- start 2024-03-22T04:10:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.592+0000 7f254ac00640 1 -- start start 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f254ac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f254415e7e0 0x7f25440d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f254ac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2544070560 0x7f25440cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f254ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25440d15f0 con 0x7f25440b4710 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f254ac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f25440b4b70 con 0x7f2544070560 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f254ac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f254406f970 con 0x7f254415e7e0 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f2548c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2544070560 0x7f25440cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f253b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f25440b4710 0x7f254406fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59484/0 (socket says 172.21.15.86:59484) 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f253b400640 1 -- 172.21.15.86:0/3797540060 learned_addr learned my addr 172.21.15.86:0/3797540060 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:49.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f2543e00640 1 --2- 172.21.15.86:0/3797540060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f254415e7e0 0x7f25440d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f2548c00640 1 -- 172.21.15.86:0/3797540060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f254415e7e0 msgr2=0x7f25440d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f2548c00640 1 --2- 172.21.15.86:0/3797540060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f254415e7e0 0x7f25440d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f2548c00640 1 -- 172.21.15.86:0/3797540060 >> v1:172.21.15.86:6789/0 conn(0x7f25440b4710 legacy=0x7f254406fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f2548c00640 1 -- 172.21.15.86:0/3797540060 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25440d1310 con 0x7f2544070560 2024-03-22T04:10:49.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.596+0000 7f2543400640 1 -- 172.21.15.86:0/3797540060 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824970064 0 0) 0x7f25440d15f0 con 0x7f25440b4710 2024-03-22T04:10:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f2548c00640 1 --2- 172.21.15.86:0/3797540060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2544070560 0x7f25440cd360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f253c009d90 tx=0x7f253c01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67af73a640e71295 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f2543400640 1 -- 172.21.15.86:0/3797540060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0b2070 con 0x7f2544070560 2024-03-22T04:10:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f2543400640 1 -- 172.21.15.86:0/3797540060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253c0a58e0 con 0x7f2544070560 2024-03-22T04:10:49.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f2543400640 1 -- 172.21.15.86:0/3797540060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0a5cb0 con 0x7f2544070560 2024-03-22T04:10:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- 172.21.15.86:0/3797540060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2544070560 msgr2=0x7f25440cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 --2- 172.21.15.86:0/3797540060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2544070560 0x7f25440cd360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f253c009d90 tx=0x7f253c01c820 comp rx=0 tx=0).stop 2024-03-22T04:10:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- 172.21.15.86:0/3797540060 shutdown_connections 2024-03-22T04:10:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 --2- 172.21.15.86:0/3797540060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2544070560 0x7f25440cd360 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 --2- 172.21.15.86:0/3797540060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f254415e7e0 0x7f25440d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- 172.21.15.86:0/3797540060 >> 172.21.15.86:0/3797540060 conn(0x7f25440b3bf0 msgr2=0x7f25440b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- 172.21.15.86:0/3797540060 shutdown_connections 2024-03-22T04:10:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- 172.21.15.86:0/3797540060 wait complete. 2024-03-22T04:10:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 Processor -- start 2024-03-22T04:10:49.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- start start 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2544070560 0x7f25440c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25440b4710 0x7f2544155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f254415e7e0 0x7f2544155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25440ce230 con 0x7f254415e7e0 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f25440ce0b0 con 0x7f25440b4710 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.600+0000 7f254ac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f25440ce3b0 con 0x7f2544070560 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f254415e7e0 0x7f2544155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f254415e7e0 0x7f2544155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35344/0 (socket says 172.21.15.86:35344) 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 -- 172.21.15.86:0/1063349422 learned_addr learned my addr 172.21.15.86:0/1063349422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:49.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2543e00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2544070560 0x7f25440c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f253b400640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25440b4710 0x7f2544155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 -- 172.21.15.86:0/1063349422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2544070560 msgr2=0x7f25440c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2544070560 0x7f25440c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 -- 172.21.15.86:0/1063349422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25440b4710 msgr2=0x7f2544155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25440b4710 0x7f2544155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 -- 172.21.15.86:0/1063349422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25440b31f0 con 0x7f254415e7e0 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2548c00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f254415e7e0 0x7f2544155bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f253c01aed0 tx=0x7f253c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2540c00640 1 -- 172.21.15.86:0/1063349422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0b2050 con 0x7f254415e7e0 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2540c00640 1 -- 172.21.15.86:0/1063349422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253c0be070 con 0x7f254415e7e0 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2540c00640 1 -- 172.21.15.86:0/1063349422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0ada50 con 0x7f254415e7e0 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254415c030 con 0x7f254415e7e0 2024-03-22T04:10:49.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25441580f0 con 0x7f254415e7e0 2024-03-22T04:10:49.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2544074470 con 0x7f254415e7e0 2024-03-22T04:10:49.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2540c00640 1 -- 172.21.15.86:0/1063349422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f253c0a5760 con 0x7f254415e7e0 2024-03-22T04:10:49.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2540c00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2520067b20 0x7f2520069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.604+0000 7f2540c00640 1 -- 172.21.15.86:0/1063349422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f253c121380 con 0x7f254415e7e0 2024-03-22T04:10:49.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.608+0000 7f2543e00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2520067b20 0x7f2520069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.608+0000 7f2543e00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2520067b20 0x7f2520069fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f254406e4d0 tx=0x7f2534009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.608+0000 7f2540c00640 1 -- 172.21.15.86:0/1063349422 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f253c016560 con 0x7f254415e7e0 2024-03-22T04:10:49.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.764+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f254406fc30 con 0x7f254415e7e0 2024-03-22T04:10:49.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.764+0000 7f2540c00640 1 -- 172.21.15.86:0/1063349422 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f253c0f1530 con 0x7f254415e7e0 2024-03-22T04:10:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:10:49.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2520067b20 msgr2=0x7f2520069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2520067b20 0x7f2520069fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f254406e4d0 tx=0x7f2534009190 comp rx=0 tx=0).stop 2024-03-22T04:10:49.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f254415e7e0 msgr2=0x7f2544155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f254415e7e0 0x7f2544155bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f253c01aed0 tx=0x7f253c0195c0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 shutdown_connections 2024-03-22T04:10:49.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2520067b20 0x7f2520069fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25440b4710 0x7f2544155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f254415e7e0 0x7f2544155bc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 --2- 172.21.15.86:0/1063349422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2544070560 0x7f25440c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 >> 172.21.15.86:0/1063349422 conn(0x7f25440b3bf0 msgr2=0x7f25440d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:49.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 shutdown_connections 2024-03-22T04:10:49.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.768+0000 7f254ac00640 1 -- 172.21.15.86:0/1063349422 wait complete. 2024-03-22T04:10:49.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:10:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.924+0000 7fbc18000640 1 Processor -- start 2024-03-22T04:10:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.924+0000 7fbc18000640 1 -- start start 2024-03-22T04:10:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc18000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc1015e630 0x7fbc100cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc18000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc18000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc101623d0 con 0x7fbc1006cc80 2024-03-22T04:10:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc18000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbc101620f0 con 0x7fbc1015e630 2024-03-22T04:10:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc18000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1015ec10 con 0x7fbc1006e2f0 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc15600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc1015e630 0x7fbc100cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc16000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbc1006cc80 0x7fbc1006d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59500/0 (socket says 172.21.15.86:59500) 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc15600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc1015e630 0x7fbc100cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35358/0 (socket says 172.21.15.86:35358) 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc16000640 1 -- 172.21.15.86:0/328235646 learned_addr learned my addr 172.21.15.86:0/328235646 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc14c00640 1 --2- 172.21.15.86:0/328235646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc15600640 1 -- 172.21.15.86:0/328235646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 msgr2=0x7fbc100d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc15600640 1 --2- 172.21.15.86:0/328235646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc15600640 1 -- 172.21.15.86:0/328235646 >> v1:172.21.15.86:6789/0 conn(0x7fbc1006cc80 legacy=0x7fbc1006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.928+0000 7fbc15600640 1 -- 172.21.15.86:0/328235646 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1015eef0 con 0x7fbc1015e630 2024-03-22T04:10:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc15600640 1 --2- 172.21.15.86:0/328235646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 0x7fbc100cfe70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbc040097c0 tx=0x7fbc0401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270c57639a49d89d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc0fe00640 1 -- 172.21.15.86:0/328235646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040b0070 con 0x7fbc1015e630 2024-03-22T04:10:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc0fe00640 1 -- 172.21.15.86:0/328235646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc040ab600 con 0x7fbc1015e630 2024-03-22T04:10:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc0fe00640 1 -- 172.21.15.86:0/328235646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040ab9d0 con 0x7fbc1015e630 2024-03-22T04:10:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 -- 172.21.15.86:0/328235646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 msgr2=0x7fbc100cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 --2- 172.21.15.86:0/328235646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 0x7fbc100cfe70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbc040097c0 tx=0x7fbc0401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 -- 172.21.15.86:0/328235646 shutdown_connections 2024-03-22T04:10:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 --2- 172.21.15.86:0/328235646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 --2- 172.21.15.86:0/328235646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 0x7fbc100cfe70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 -- 172.21.15.86:0/328235646 >> 172.21.15.86:0/328235646 conn(0x7fbc1006f910 msgr2=0x7fbc1006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 -- 172.21.15.86:0/328235646 shutdown_connections 2024-03-22T04:10:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 -- 172.21.15.86:0/328235646 wait complete. 2024-03-22T04:10:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 Processor -- start 2024-03-22T04:10:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.932+0000 7fbc18000640 1 -- start start 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc1006cc80 0x7fbc100c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 0x7fbc1015bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc100cfaf0 con 0x7fbc1015e630 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1015f9d0 con 0x7fbc1006cc80 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbc100cfc70 con 0x7fbc1006e2f0 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc16000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 0x7fbc1015bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc14c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc15600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc1006cc80 0x7fbc100c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc14c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:51074/0 (socket says 172.21.15.86:51074) 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc15600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc1006cc80 0x7fbc100c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:40944/0 (socket says 172.21.15.86:40944) 2024-03-22T04:10:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc14c00640 1 -- 172.21.15.86:0/491463834 learned_addr learned my addr 172.21.15.86:0/491463834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc16000640 1 -- 172.21.15.86:0/491463834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 msgr2=0x7fbc100b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc16000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc16000640 1 -- 172.21.15.86:0/491463834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc1006cc80 msgr2=0x7fbc100c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc16000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc1006cc80 0x7fbc100c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc16000640 1 -- 172.21.15.86:0/491463834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc100b56d0 con 0x7fbc1015e630 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc14c00640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc16000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 0x7fbc1015bf60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800caa0 tx=0x7fbc0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc0d600640 1 -- 172.21.15.86:0/491463834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08017070 con 0x7fbc1015e630 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc0d600640 1 -- 172.21.15.86:0/491463834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0800ab40 con 0x7fbc1015e630 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc100b59b0 con 0x7fbc1015e630 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc0d600640 1 -- 172.21.15.86:0/491463834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08012400 con 0x7fbc1015e630 2024-03-22T04:10:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc100b5f70 con 0x7fbc1015e630 2024-03-22T04:10:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc10068650 con 0x7fbc1015e630 2024-03-22T04:10:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.936+0000 7fbc0d600640 1 -- 172.21.15.86:0/491463834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbc0801a070 con 0x7fbc1015e630 2024-03-22T04:10:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.940+0000 7fbc0d600640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbbf0067b50 0x7fbbf006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.940+0000 7fbc0d600640 1 -- 172.21.15.86:0/491463834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbc08085ca0 con 0x7fbc1015e630 2024-03-22T04:10:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.940+0000 7fbc15600640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbbf0067b50 0x7fbbf006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.940+0000 7fbc15600640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbbf0067b50 0x7fbbf006a010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc040ad000 tx=0x7fbc040a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:49.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:49.944+0000 7fbc0d600640 1 -- 172.21.15.86:0/491463834 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbc08055db0 con 0x7fbc1015e630 2024-03-22T04:10:50.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.108+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbc1006dfa0 con 0x7fbc1015e630 2024-03-22T04:10:50.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc0d600640 1 -- 172.21.15.86:0/491463834 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fbc08059e50 con 0x7fbc1015e630 2024-03-22T04:10:50.114 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:10:50.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbbf0067b50 msgr2=0x7fbbf006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:50.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbbf0067b50 0x7fbbf006a010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc040ad000 tx=0x7fbc040a6000 comp rx=0 tx=0).stop 2024-03-22T04:10:50.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 msgr2=0x7fbc1015bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 0x7fbc1015bf60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800caa0 tx=0x7fbc0800cf60 comp rx=0 tx=0).stop 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 shutdown_connections 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbbf0067b50 0x7fbbf006a010 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc1015e630 0x7fbc1015bf60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc1006e2f0 0x7fbc100b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 --2- 172.21.15.86:0/491463834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc1006cc80 0x7fbc100c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 >> 172.21.15.86:0/491463834 conn(0x7fbc1006f910 msgr2=0x7fbc10168970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 shutdown_connections 2024-03-22T04:10:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:50.112+0000 7fbc18000640 1 -- 172.21.15.86:0/491463834 wait complete. 2024-03-22T04:10:50.135 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-03-22T04:10:50.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:50.136 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-22T04:10:50.355 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[34054]: fuse finished with error 0 and tester_r 0 2024-03-22T04:10:50.358 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:10:56.460 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:10:56.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:10:56.460 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-22T04:10:56.477 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:56.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd20d200640 1 Processor -- start 2024-03-22T04:10:56.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd20d200640 1 -- start start 2024-03-22T04:10:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd20d200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080ce6b0 0x7fd2080cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd20d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2080d11c0 con 0x7fd20815fed0 2024-03-22T04:10:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd20d200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2080cece0 con 0x7fd2080d0de0 2024-03-22T04:10:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd20d200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd2080cefc0 con 0x7fd2080ce6b0 2024-03-22T04:10:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd206800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd20815fed0 0x7fd208071620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57412/0 (socket says 172.21.15.86:57412) 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd206800640 1 -- 172.21.15.86:0/3424406902 learned_addr learned my addr 172.21.15.86:0/3424406902 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd1fd400640 1 --2- 172.21.15.86:0/3424406902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080ce6b0 0x7fd2080cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd205400640 1 -- 172.21.15.86:0/3424406902 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1229345063 0 0) 0x7fd2080d11c0 con 0x7fd20815fed0 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd205400640 1 -- 172.21.15.86:0/3424406902 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1e8003530 con 0x7fd20815fed0 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd205400640 1 -- 172.21.15.86:0/3424406902 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322937965 0 0) 0x7fd2080cece0 con 0x7fd2080d0de0 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd205400640 1 -- 172.21.15.86:0/3424406902 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2080d11c0 con 0x7fd2080d0de0 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd1fd400640 1 -- 172.21.15.86:0/3424406902 >> v1:172.21.15.196:6789/0 conn(0x7fd2080d0de0 legacy=0x7fd2080d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd1fd400640 1 -- 172.21.15.86:0/3424406902 >> v1:172.21.15.86:6789/0 conn(0x7fd20815fed0 legacy=0x7fd208071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.660+0000 7fd1fd400640 1 -- 172.21.15.86:0/3424406902 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2080cf2a0 con 0x7fd2080ce6b0 2024-03-22T04:10:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd1fd400640 1 --2- 172.21.15.86:0/3424406902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080ce6b0 0x7fd2080cfe70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00097f0 tx=0x7fd1f001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc2cf6968731aced server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd205400640 1 -- 172.21.15.86:0/3424406902 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00ab5f0 con 0x7fd2080ce6b0 2024-03-22T04:10:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd205400640 1 -- 172.21.15.86:0/3424406902 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f00b0070 con 0x7fd2080ce6b0 2024-03-22T04:10:56.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd205400640 1 -- 172.21.15.86:0/3424406902 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00ab9b0 con 0x7fd2080ce6b0 2024-03-22T04:10:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd20d200640 1 -- 172.21.15.86:0/3424406902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080ce6b0 msgr2=0x7fd2080cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd20d200640 1 --2- 172.21.15.86:0/3424406902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080ce6b0 0x7fd2080cfe70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00097f0 tx=0x7fd1f001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:10:56.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd20d200640 1 -- 172.21.15.86:0/3424406902 shutdown_connections 2024-03-22T04:10:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd20d200640 1 --2- 172.21.15.86:0/3424406902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080ce6b0 0x7fd2080cfe70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd20d200640 1 -- 172.21.15.86:0/3424406902 >> 172.21.15.86:0/3424406902 conn(0x7fd20806f910 msgr2=0x7fd20806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd20d200640 1 -- 172.21.15.86:0/3424406902 shutdown_connections 2024-03-22T04:10:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.664+0000 7fd20d200640 1 -- 172.21.15.86:0/3424406902 wait complete. 2024-03-22T04:10:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 Processor -- start 2024-03-22T04:10:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 -- start start 2024-03-22T04:10:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2080ce6b0 0x7fd2081528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080d0de0 0x7fd208153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd20815fed0 0x7fd208153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2080d0530 con 0x7fd2080ce6b0 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2080d03b0 con 0x7fd20815fed0 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd20816a860 con 0x7fd2080d0de0 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2080ce6b0 0x7fd2081528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2080ce6b0 0x7fd2081528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55794/0 (socket says 172.21.15.86:55794) 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 -- 172.21.15.86:0/2150479244 learned_addr learned my addr 172.21.15.86:0/2150479244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd1fd400640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080d0de0 0x7fd208153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd206800640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd20815fed0 0x7fd208153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 -- 172.21.15.86:0/2150479244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080d0de0 msgr2=0x7fd208153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080d0de0 0x7fd208153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 -- 172.21.15.86:0/2150479244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd20815fed0 msgr2=0x7fd208153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd20815fed0 0x7fd208153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 -- 172.21.15.86:0/2150479244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd208157b50 con 0x7fd2080ce6b0 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd205e00640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2080ce6b0 0x7fd2081528d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c4c0 tx=0x7fd1f800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd1fea00640 1 -- 172.21.15.86:0/2150479244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80052d0 con 0x7fd2080ce6b0 2024-03-22T04:10:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd1fea00640 1 -- 172.21.15.86:0/2150479244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f8005470 con 0x7fd2080ce6b0 2024-03-22T04:10:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd208157e10 con 0x7fd2080ce6b0 2024-03-22T04:10:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd1fea00640 1 -- 172.21.15.86:0/2150479244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8012870 con 0x7fd2080ce6b0 2024-03-22T04:10:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.668+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2081583d0 con 0x7fd2080ce6b0 2024-03-22T04:10:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.672+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd208068650 con 0x7fd2080ce6b0 2024-03-22T04:10:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.672+0000 7fd1fea00640 1 -- 172.21.15.86:0/2150479244 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd1f8012a10 con 0x7fd2080ce6b0 2024-03-22T04:10:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.672+0000 7fd1fea00640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd1e4067b20 0x7fd1e4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.672+0000 7fd1fea00640 1 -- 172.21.15.86:0/2150479244 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd1f8085f80 con 0x7fd2080ce6b0 2024-03-22T04:10:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.672+0000 7fd1fd400640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd1e4067b20 0x7fd1e4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.672+0000 7fd1fd400640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd1e4067b20 0x7fd1e4069fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00af000 tx=0x7fd1f00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:56.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.676+0000 7fd1fea00640 1 -- 172.21.15.86:0/2150479244 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd1f8056520 con 0x7fd2080ce6b0 2024-03-22T04:10:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.832+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd208069ae0 con 0x7fd2080ce6b0 2024-03-22T04:10:56.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.832+0000 7fd1fea00640 1 -- 172.21.15.86:0/2150479244 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7fd1f8055e20 con 0x7fd2080ce6b0 2024-03-22T04:10:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:56.838 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:56.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd1e4067b20 msgr2=0x7fd1e4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd1e4067b20 0x7fd1e4069fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00af000 tx=0x7fd1f00a7000 comp rx=0 tx=0).stop 2024-03-22T04:10:56.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2080ce6b0 msgr2=0x7fd2081528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2080ce6b0 0x7fd2081528d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c4c0 tx=0x7fd1f800c980 comp rx=0 tx=0).stop 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 shutdown_connections 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd1e4067b20 0x7fd1e4069fe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd20815fed0 0x7fd208153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2080d0de0 0x7fd208153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 --2- 172.21.15.86:0/2150479244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2080ce6b0 0x7fd2081528d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 >> 172.21.15.86:0/2150479244 conn(0x7fd20806f910 msgr2=0x7fd20815f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 shutdown_connections 2024-03-22T04:10:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.836+0000 7fd20d200640 1 -- 172.21.15.86:0/2150479244 wait complete. 2024-03-22T04:10:56.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcad6a00640 1 Processor -- start 2024-03-22T04:10:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcad6a00640 1 -- start start 2024-03-22T04:10:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcad6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad015b3f0 0x7fcad00729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcad6a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad0071920 con 0x7fcad00ae540 2024-03-22T04:10:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcad6a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad0071c00 con 0x7fcad00cc3e0 2024-03-22T04:10:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcad6a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcad00716d0 con 0x7fcad015b3f0 2024-03-22T04:10:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcad4a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcad00ae540 0x7fcad00b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57424/0 (socket says 172.21.15.86:57424) 2024-03-22T04:10:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcad4a00640 1 -- 172.21.15.86:0/3728519067 learned_addr learned my addr 172.21.15.86:0/3728519067 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcacfe00640 1 --2- 172.21.15.86:0/3728519067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad015b3f0 0x7fcad00729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcacea00640 1 -- 172.21.15.86:0/3728519067 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1375402292 0 0) 0x7fcad0071c00 con 0x7fcad00cc3e0 2024-03-22T04:10:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcacea00640 1 -- 172.21.15.86:0/3728519067 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcab4003530 con 0x7fcad00cc3e0 2024-03-22T04:10:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcacea00640 1 -- 172.21.15.86:0/3728519067 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 502381101 0 0) 0x7fcad0071920 con 0x7fcad00ae540 2024-03-22T04:10:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcacea00640 1 -- 172.21.15.86:0/3728519067 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcad0071c00 con 0x7fcad00ae540 2024-03-22T04:10:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcacfe00640 1 -- 172.21.15.86:0/3728519067 >> v1:172.21.15.196:6789/0 conn(0x7fcad00cc3e0 legacy=0x7fcad00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcacfe00640 1 -- 172.21.15.86:0/3728519067 >> v1:172.21.15.86:6789/0 conn(0x7fcad00ae540 legacy=0x7fcad00b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.992+0000 7fcacfe00640 1 -- 172.21.15.86:0/3728519067 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad0071ee0 con 0x7fcad015b3f0 2024-03-22T04:10:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcacfe00640 1 --2- 172.21.15.86:0/3728519067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad015b3f0 0x7fcad00729c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097f0 tx=0x7fcac001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b47d7cee644788 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcacea00640 1 -- 172.21.15.86:0/3728519067 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac00b0070 con 0x7fcad015b3f0 2024-03-22T04:10:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcacea00640 1 -- 172.21.15.86:0/3728519067 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac00ab600 con 0x7fcad015b3f0 2024-03-22T04:10:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcacea00640 1 -- 172.21.15.86:0/3728519067 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac00ab9d0 con 0x7fcad015b3f0 2024-03-22T04:10:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcad6a00640 1 -- 172.21.15.86:0/3728519067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad015b3f0 msgr2=0x7fcad00729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3728519067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad015b3f0 0x7fcad00729c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097f0 tx=0x7fcac001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcad6a00640 1 -- 172.21.15.86:0/3728519067 shutdown_connections 2024-03-22T04:10:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3728519067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad015b3f0 0x7fcad00729c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcad6a00640 1 -- 172.21.15.86:0/3728519067 >> 172.21.15.86:0/3728519067 conn(0x7fcad00b1df0 msgr2=0x7fcad00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcad6a00640 1 -- 172.21.15.86:0/3728519067 shutdown_connections 2024-03-22T04:10:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:56.996+0000 7fcad6a00640 1 -- 172.21.15.86:0/3728519067 wait complete. 2024-03-22T04:10:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 Processor -- start 2024-03-22T04:10:57.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 -- start start 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad00ae540 0x7fcad0157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad00cc3e0 0x7fcad0158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad015b3f0 0x7fcad0098e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcad00b30f0 con 0x7fcad00ae540 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcad00b4c60 con 0x7fcad015b3f0 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcad00b3270 con 0x7fcad00cc3e0 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad00ae540 0x7fcad0157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad00ae540 0x7fcad0157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55800/0 (socket says 172.21.15.86:55800) 2024-03-22T04:10:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 -- 172.21.15.86:0/853040105 learned_addr learned my addr 172.21.15.86:0/853040105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacf400640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad00cc3e0 0x7fcad0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad4a00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad015b3f0 0x7fcad0098e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 -- 172.21.15.86:0/853040105 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad00cc3e0 msgr2=0x7fcad0158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad00cc3e0 0x7fcad0158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 -- 172.21.15.86:0/853040105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad015b3f0 msgr2=0x7fcad0098e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad015b3f0 0x7fcad0098e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 -- 172.21.15.86:0/853040105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad0148670 con 0x7fcad00ae540 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcacfe00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad00ae540 0x7fcad0157cd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097f0 tx=0x7fcac0019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcac7e00640 1 -- 172.21.15.86:0/853040105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac00b0040 con 0x7fcad00ae540 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcac7e00640 1 -- 172.21.15.86:0/853040105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac00be070 con 0x7fcad00ae540 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcac7e00640 1 -- 172.21.15.86:0/853040105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0006e20 con 0x7fcad00ae540 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad0148950 con 0x7fcad00ae540 2024-03-22T04:10:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.000+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad0148f10 con 0x7fcad00ae540 2024-03-22T04:10:57.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.004+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad0068650 con 0x7fcad00ae540 2024-03-22T04:10:57.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.004+0000 7fcac7e00640 1 -- 172.21.15.86:0/853040105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcac00b3040 con 0x7fcad00ae540 2024-03-22T04:10:57.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.004+0000 7fcac7e00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcab0067b50 0x7fcab006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.004+0000 7fcac7e00640 1 -- 172.21.15.86:0/853040105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fcac011fe80 con 0x7fcad00ae540 2024-03-22T04:10:57.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.004+0000 7fcacf400640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcab0067b50 0x7fcab006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.004+0000 7fcacf400640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcab0067b50 0x7fcab006a010 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcabc007fd0 tx=0x7fcabc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:57.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.008+0000 7fcac7e00640 1 -- 172.21.15.86:0/853040105 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcac00c7020 con 0x7fcad00ae540 2024-03-22T04:10:57.075 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:10:57.078 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:10:57.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.168+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcad0153d50 con 0x7fcad00ae540 2024-03-22T04:10:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.168+0000 7fcac7e00640 1 -- 172.21.15.86:0/853040105 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7fcac00f6070 con 0x7fcad00ae540 2024-03-22T04:10:57.172 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:57.173 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcab0067b50 msgr2=0x7fcab006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcab0067b50 0x7fcab006a010 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcabc007fd0 tx=0x7fcabc009040 comp rx=0 tx=0).stop 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad00ae540 msgr2=0x7fcad0157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad00ae540 0x7fcad0157cd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097f0 tx=0x7fcac0019280 comp rx=0 tx=0).stop 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 shutdown_connections 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcab0067b50 0x7fcab006a010 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcad015b3f0 0x7fcad0098e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcad00cc3e0 0x7fcad0158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 --2- 172.21.15.86:0/853040105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad00ae540 0x7fcad0157cd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 >> 172.21.15.86:0/853040105 conn(0x7fcad00b1df0 msgr2=0x7fcad0157010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 shutdown_connections 2024-03-22T04:10:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.172+0000 7fcad6a00640 1 -- 172.21.15.86:0/853040105 wait complete. 2024-03-22T04:10:57.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T04:10:57.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf62800640 1 Processor -- start 2024-03-22T04:10:57.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf62800640 1 -- start start 2024-03-22T04:10:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf62800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf5c1554e0 0x7fdf5c072d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf62800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c0cf060 0x7fdf5c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf62800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdf5c1558c0 0x7fdf5c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf62800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdf5c0cf560 con 0x7fdf5c1554e0 2024-03-22T04:10:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf62800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdf5c0cf840 con 0x7fdf5c1558c0 2024-03-22T04:10:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf62800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c0cfb20 con 0x7fdf5c0cf060 2024-03-22T04:10:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf5c1554e0 0x7fdf5c072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf5c1554e0 0x7fdf5c072d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55812/0 (socket says 172.21.15.86:55812) 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61400640 1 -- 172.21.15.86:0/700237114 learned_addr learned my addr 172.21.15.86:0/700237114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf60a00640 1 --2- 172.21.15.86:0/700237114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c0cf060 0x7fdf5c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61e00640 1 --2- 172.21.15.86:0/700237114 >> v2:172.21.15.196:3300/0 conn(0x7fdf5c1558c0 0x7fdf5c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61400640 1 -- 172.21.15.86:0/700237114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c0cf060 msgr2=0x7fdf5c15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61400640 1 --2- 172.21.15.86:0/700237114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c0cf060 0x7fdf5c15ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61400640 1 -- 172.21.15.86:0/700237114 >> v2:172.21.15.196:3300/0 conn(0x7fdf5c1558c0 msgr2=0x7fdf5c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61400640 1 --2- 172.21.15.86:0/700237114 >> v2:172.21.15.196:3300/0 conn(0x7fdf5c1558c0 0x7fdf5c15f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.368+0000 7fdf61400640 1 -- 172.21.15.86:0/700237114 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c0cfe00 con 0x7fdf5c1554e0 2024-03-22T04:10:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf61400640 1 --2- 172.21.15.86:0/700237114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1554e0 0x7fdf5c072d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c073c30 tx=0x7fdf5001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3497a92ccf69a741 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf5be00640 1 -- 172.21.15.86:0/700237114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500a5860 con 0x7fdf5c1554e0 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf5be00640 1 -- 172.21.15.86:0/700237114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf500a5a00 con 0x7fdf5c1554e0 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf5be00640 1 -- 172.21.15.86:0/700237114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500a5dd0 con 0x7fdf5c1554e0 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 -- 172.21.15.86:0/700237114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1554e0 msgr2=0x7fdf5c072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 --2- 172.21.15.86:0/700237114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1554e0 0x7fdf5c072d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c073c30 tx=0x7fdf5001c9c0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 -- 172.21.15.86:0/700237114 shutdown_connections 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 --2- 172.21.15.86:0/700237114 >> v2:172.21.15.196:3300/0 conn(0x7fdf5c1558c0 0x7fdf5c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 --2- 172.21.15.86:0/700237114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c0cf060 0x7fdf5c15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 --2- 172.21.15.86:0/700237114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1554e0 0x7fdf5c072d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 -- 172.21.15.86:0/700237114 >> 172.21.15.86:0/700237114 conn(0x7fdf5c0d0ec0 msgr2=0x7fdf5c0d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 -- 172.21.15.86:0/700237114 shutdown_connections 2024-03-22T04:10:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.372+0000 7fdf62800640 1 -- 172.21.15.86:0/700237114 wait complete. 2024-03-22T04:10:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 Processor -- start 2024-03-22T04:10:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 -- start start 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5c0cf060 0x7fdf5c0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c1554e0 0x7fdf5c0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1558c0 0x7fdf5c0c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c15c2c0 con 0x7fdf5c1558c0 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c15c140 con 0x7fdf5c0cf060 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c15c440 con 0x7fdf5c1554e0 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1558c0 0x7fdf5c0c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5c0cf060 0x7fdf5c0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1558c0 0x7fdf5c0c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55814/0 (socket says 172.21.15.86:55814) 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 -- 172.21.15.86:0/2966286715 learned_addr learned my addr 172.21.15.86:0/2966286715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf60a00640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c1554e0 0x7fdf5c0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 -- 172.21.15.86:0/2966286715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c1554e0 msgr2=0x7fdf5c0c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c1554e0 0x7fdf5c0c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 -- 172.21.15.86:0/2966286715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5c0cf060 msgr2=0x7fdf5c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5c0cf060 0x7fdf5c0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 -- 172.21.15.86:0/2966286715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c15e060 con 0x7fdf5c1558c0 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf61e00640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1558c0 0x7fdf5c0c75c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400e7f0 tx=0x7fdf5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf59600640 1 -- 172.21.15.86:0/2966286715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54017070 con 0x7fdf5c1558c0 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf59600640 1 -- 172.21.15.86:0/2966286715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf5400c940 con 0x7fdf5c1558c0 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf59600640 1 -- 172.21.15.86:0/2966286715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400eea0 con 0x7fdf5c1558c0 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c15d2a0 con 0x7fdf5c1558c0 2024-03-22T04:10:57.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.376+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c15d860 con 0x7fdf5c1558c0 2024-03-22T04:10:57.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.380+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c04f900 con 0x7fdf5c1558c0 2024-03-22T04:10:57.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.380+0000 7fdf59600640 1 -- 172.21.15.86:0/2966286715 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdf5401a070 con 0x7fdf5c1558c0 2024-03-22T04:10:57.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.380+0000 7fdf59600640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf3c067b20 0x7fdf3c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:57.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.380+0000 7fdf59600640 1 -- 172.21.15.86:0/2966286715 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdf540863e0 con 0x7fdf5c1558c0 2024-03-22T04:10:57.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.380+0000 7fdf61400640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf3c067b20 0x7fdf3c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:57.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.380+0000 7fdf61400640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf3c067b20 0x7fdf3c069fe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf500ae000 tx=0x7fdf50004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:57.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.384+0000 7fdf59600640 1 -- 172.21.15.86:0/2966286715 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdf54056450 con 0x7fdf5c1558c0 2024-03-22T04:10:57.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.572+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdf5c0d04b0 con 0x7fdf5c1558c0 2024-03-22T04:10:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf59600640 1 -- 172.21.15.86:0/2966286715 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fdf5405a4f0 con 0x7fdf5c1558c0 2024-03-22T04:10:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf3c067b20 msgr2=0x7fdf3c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf3c067b20 0x7fdf3c069fe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf500ae000 tx=0x7fdf50004ac0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1558c0 msgr2=0x7fdf5c0c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1558c0 0x7fdf5c0c75c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400e7f0 tx=0x7fdf5400ecb0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 shutdown_connections 2024-03-22T04:10:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf3c067b20 0x7fdf3c069fe0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c1558c0 0x7fdf5c0c75c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5c1554e0 0x7fdf5c0c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 --2- 172.21.15.86:0/2966286715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5c0cf060 0x7fdf5c0b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 >> 172.21.15.86:0/2966286715 conn(0x7fdf5c0d0ec0 msgr2=0x7fdf5c06e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 shutdown_connections 2024-03-22T04:10:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:57.888+0000 7fdf62800640 1 -- 172.21.15.86:0/2966286715 wait complete. 2024-03-22T04:10:57.913 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-22T04:10:57.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-22T04:10:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.048+0000 7f2017e00640 1 Processor -- start 2024-03-22T04:10:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.048+0000 7f2017e00640 1 -- start start 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2017e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f201815c230 0x7f20180d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2017e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20180d2b30 0x7f20180cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2017e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20180cfca0 con 0x7f20180ce900 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2017e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f20180d05d0 con 0x7f201815c230 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2017e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f20180cff80 con 0x7f20180d2b30 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2016a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f201815c230 0x7f20180d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2017400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f20180ce900 0x7f20180b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57438/0 (socket says 172.21.15.86:57438) 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2017400640 1 -- 172.21.15.86:0/50651512 learned_addr learned my addr 172.21.15.86:0/50651512 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f200e000640 1 --2- 172.21.15.86:0/50651512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20180d2b30 0x7f20180cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2016a00640 1 -- 172.21.15.86:0/50651512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20180d2b30 msgr2=0x7f20180cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2016a00640 1 --2- 172.21.15.86:0/50651512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20180d2b30 0x7f20180cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2016a00640 1 -- 172.21.15.86:0/50651512 >> v1:172.21.15.86:6789/0 conn(0x7f20180ce900 legacy=0x7f20180b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2016a00640 1 -- 172.21.15.86:0/50651512 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20180d0260 con 0x7f201815c230 2024-03-22T04:10:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f2016000640 1 -- 172.21.15.86:0/50651512 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559950085 0 0) 0x7f20180cfca0 con 0x7f20180ce900 2024-03-22T04:10:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.052+0000 7f200e000640 1 --2- 172.21.15.86:0/50651512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20180d2b30 0x7f20180cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:10:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2016a00640 1 --2- 172.21.15.86:0/50651512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201815c230 0x7f20180d0a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20080097a0 tx=0x7f200801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0fd97938e0753f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2016000640 1 -- 172.21.15.86:0/50651512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20080b2070 con 0x7f201815c230 2024-03-22T04:10:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2016000640 1 -- 172.21.15.86:0/50651512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20080ad640 con 0x7f201815c230 2024-03-22T04:10:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2016000640 1 -- 172.21.15.86:0/50651512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20080ada10 con 0x7f201815c230 2024-03-22T04:10:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- 172.21.15.86:0/50651512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201815c230 msgr2=0x7f20180d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 --2- 172.21.15.86:0/50651512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201815c230 0x7f20180d0a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20080097a0 tx=0x7f200801c480 comp rx=0 tx=0).stop 2024-03-22T04:10:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- 172.21.15.86:0/50651512 shutdown_connections 2024-03-22T04:10:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 --2- 172.21.15.86:0/50651512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20180d2b30 0x7f20180cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 --2- 172.21.15.86:0/50651512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201815c230 0x7f20180d0a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- 172.21.15.86:0/50651512 >> 172.21.15.86:0/50651512 conn(0x7f20180d1f30 msgr2=0x7f20180d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- 172.21.15.86:0/50651512 shutdown_connections 2024-03-22T04:10:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- 172.21.15.86:0/50651512 wait complete. 2024-03-22T04:10:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 Processor -- start 2024-03-22T04:10:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- start start 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20180ce900 0x7f201814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20180d2b30 0x7f201814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f201815c230 0x7f201815d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2018169800 con 0x7f20180d2b30 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20180cdda0 con 0x7f20180ce900 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f2017e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2018169980 con 0x7f201815c230 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.056+0000 7f200e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20180d2b30 0x7f201814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20180d2b30 0x7f201814d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55834/0 (socket says 172.21.15.86:55834) 2024-03-22T04:10:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200e000640 1 -- 172.21.15.86:0/1417871816 learned_addr learned my addr 172.21.15.86:0/1417871816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f2016a00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20180ce900 0x7f201814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f2017400640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f201815c230 0x7f201815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200e000640 1 -- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f201815c230 msgr2=0x7f201815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200e000640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f201815c230 0x7f201815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200e000640 1 -- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20180ce900 msgr2=0x7f201814ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200e000640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20180ce900 0x7f201814ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200e000640 1 -- 172.21.15.86:0/1417871816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2018156980 con 0x7f20180d2b30 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f2016a00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20180ce900 0x7f201814ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200e000640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20180d2b30 0x7f201814d3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f200000d800 tx=0x7f200000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200f400640 1 -- 172.21.15.86:0/1417871816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000b6c0 con 0x7f20180d2b30 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200f400640 1 -- 172.21.15.86:0/1417871816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2000019070 con 0x7f20180d2b30 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200f400640 1 -- 172.21.15.86:0/1417871816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000baa0 con 0x7f20180d2b30 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20181552b0 con 0x7f20180d2b30 2024-03-22T04:10:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20181566a0 con 0x7f20180d2b30 2024-03-22T04:10:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201804f900 con 0x7f20180d2b30 2024-03-22T04:10:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200f400640 1 -- 172.21.15.86:0/1417871816 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f200000bc40 con 0x7f20180d2b30 2024-03-22T04:10:58.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200f400640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ff4067b50 0x7f1ff406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:58.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.060+0000 7f200f400640 1 -- 172.21.15.86:0/1417871816 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f20000864d0 con 0x7f20180d2b30 2024-03-22T04:10:58.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.064+0000 7f2016a00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ff4067b50 0x7f1ff406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:58.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.064+0000 7f2016a00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ff4067b50 0x7f1ff406a010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20080af000 tx=0x7f2008008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.064+0000 7f200f400640 1 -- 172.21.15.86:0/1417871816 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2000056540 con 0x7f20180d2b30 2024-03-22T04:10:58.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.252+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2018156cb0 con 0x7f20180d2b30 2024-03-22T04:10:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.892+0000 7f200f400640 1 -- 172.21.15.86:0/1417871816 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f200005a5e0 con 0x7f20180d2b30 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ff4067b50 msgr2=0x7f1ff406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ff4067b50 0x7f1ff406a010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20080af000 tx=0x7f2008008040 comp rx=0 tx=0).stop 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20180d2b30 msgr2=0x7f201814d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20180d2b30 0x7f201814d3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f200000d800 tx=0x7f200000dcc0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 shutdown_connections 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ff4067b50 0x7f1ff406a010 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f201815c230 0x7f201815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20180d2b30 0x7f201814d3b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 --2- 172.21.15.86:0/1417871816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20180ce900 0x7f201814ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 >> 172.21.15.86:0/1417871816 conn(0x7f20180d1f30 msgr2=0x7f201815a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 shutdown_connections 2024-03-22T04:10:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:58.896+0000 7f2017e00640 1 -- 172.21.15.86:0/1417871816 wait complete. 2024-03-22T04:10:58.922 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-22T04:10:58.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:10:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.060+0000 7f73fae00640 1 Processor -- start 2024-03-22T04:10:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.060+0000 7f73fae00640 1 -- start start 2024-03-22T04:10:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73fae00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73f415b3f0 0x7f73f40729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73fae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73f40cc3e0 0x7f73f4072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73fae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f4071920 con 0x7f73f40b30f0 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73fae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f73f40716d0 con 0x7f73f40cc3e0 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73fae00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f73f406d990 con 0x7f73f415b3f0 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73f40cc3e0 0x7f73f4072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f8e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f73f40b30f0 0x7f73f40ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57450/0 (socket says 172.21.15.86:57450) 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73f40cc3e0 0x7f73f4072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55838/0 (socket says 172.21.15.86:55838) 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f8e00640 1 -- 172.21.15.86:0/263328357 learned_addr learned my addr 172.21.15.86:0/263328357 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f3e00640 1 --2- 172.21.15.86:0/263328357 >> v2:172.21.15.196:3300/0 conn(0x7f73f415b3f0 0x7f73f40729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f3400640 1 -- 172.21.15.86:0/263328357 >> v2:172.21.15.196:3300/0 conn(0x7f73f415b3f0 msgr2=0x7f73f40729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f3400640 1 --2- 172.21.15.86:0/263328357 >> v2:172.21.15.196:3300/0 conn(0x7f73f415b3f0 0x7f73f40729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f3400640 1 -- 172.21.15.86:0/263328357 >> v1:172.21.15.86:6789/0 conn(0x7f73f40b30f0 legacy=0x7f73f40ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.064+0000 7f73f3400640 1 -- 172.21.15.86:0/263328357 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f4071c00 con 0x7f73f40cc3e0 2024-03-22T04:10:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73f3400640 1 --2- 172.21.15.86:0/263328357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4072f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73f406c510 tx=0x7f73e001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d29f78147d2ddc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73f2a00640 1 -- 172.21.15.86:0/263328357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00b0070 con 0x7f73f40cc3e0 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73f2a00640 1 -- 172.21.15.86:0/263328357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e00ab600 con 0x7f73f40cc3e0 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73f2a00640 1 -- 172.21.15.86:0/263328357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00ab9d0 con 0x7f73f40cc3e0 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 -- 172.21.15.86:0/263328357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 msgr2=0x7f73f4072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 --2- 172.21.15.86:0/263328357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4072f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73f406c510 tx=0x7f73e001c3f0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 -- 172.21.15.86:0/263328357 shutdown_connections 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 --2- 172.21.15.86:0/263328357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4072f00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 --2- 172.21.15.86:0/263328357 >> v2:172.21.15.196:3300/0 conn(0x7f73f415b3f0 0x7f73f40729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 -- 172.21.15.86:0/263328357 >> 172.21.15.86:0/263328357 conn(0x7f73f40b1df0 msgr2=0x7f73f40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 -- 172.21.15.86:0/263328357 shutdown_connections 2024-03-22T04:10:59.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 -- 172.21.15.86:0/263328357 wait complete. 2024-03-22T04:10:59.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 Processor -- start 2024-03-22T04:10:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 -- start start 2024-03-22T04:10:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73f40b30f0 0x7f73f414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f415b3f0 0x7f73f4159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4168580 con 0x7f73f40cc3e0 2024-03-22T04:10:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.068+0000 7f73fae00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4168400 con 0x7f73f415b3f0 2024-03-22T04:10:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73fae00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f73f4168700 con 0x7f73f40b30f0 2024-03-22T04:10:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73f40b30f0 0x7f73f414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55846/0 (socket says 172.21.15.86:55846) 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 -- 172.21.15.86:0/1551374807 learned_addr learned my addr 172.21.15.86:0/1551374807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f8e00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f415b3f0 0x7f73f4159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 -- 172.21.15.86:0/1551374807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73f40b30f0 msgr2=0x7f73f414bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73f40b30f0 0x7f73f414bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 -- 172.21.15.86:0/1551374807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f415b3f0 msgr2=0x7f73f4159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f415b3f0 0x7f73f4159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 -- 172.21.15.86:0/1551374807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f415d840 con 0x7f73f40cc3e0 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73f3400640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4159240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73e00ae040 tx=0x7f73e0019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73ebe00640 1 -- 172.21.15.86:0/1551374807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00b0040 con 0x7f73f40cc3e0 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73ebe00640 1 -- 172.21.15.86:0/1551374807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e00b9070 con 0x7f73f40cc3e0 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73ebe00640 1 -- 172.21.15.86:0/1551374807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00b4650 con 0x7f73f40cc3e0 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f41507a0 con 0x7f73f40cc3e0 2024-03-22T04:10:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f415cd60 con 0x7f73f40cc3e0 2024-03-22T04:10:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f4068650 con 0x7f73f40cc3e0 2024-03-22T04:10:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73ebe00640 1 -- 172.21.15.86:0/1551374807 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f73e00b47f0 con 0x7f73f40cc3e0 2024-03-22T04:10:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.072+0000 7f73ebe00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f73d4067b20 0x7f73d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.076+0000 7f73ebe00640 1 -- 172.21.15.86:0/1551374807 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f73e011fea0 con 0x7f73f40cc3e0 2024-03-22T04:10:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.076+0000 7f73f3e00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f73d4067b20 0x7f73d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.076+0000 7f73f3e00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f73d4067b20 0x7f73d4069fe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f73f414f2a0 tx=0x7f73e4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.080+0000 7f73ebe00640 1 -- 172.21.15.86:0/1551374807 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f73e0016070 con 0x7f73f40cc3e0 2024-03-22T04:10:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73f40ae260 con 0x7f73f40cc3e0 2024-03-22T04:10:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73ebe00640 1 -- 172.21.15.86:0/1551374807 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5179 (secure 0 0 0) 0x7f73e00effb0 con 0x7f73f40cc3e0 2024-03-22T04:10:59.271 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:59.271 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:58.258965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:10:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f73d4067b20 msgr2=0x7f73d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f73d4067b20 0x7f73d4069fe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f73f414f2a0 tx=0x7f73e4009190 comp rx=0 tx=0).stop 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 msgr2=0x7f73f4159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4159240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73e00ae040 tx=0x7f73e0019190 comp rx=0 tx=0).stop 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 shutdown_connections 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f73d4067b20 0x7f73d4069fe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f415b3f0 0x7f73f4159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73f40cc3e0 0x7f73f4159240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 --2- 172.21.15.86:0/1551374807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73f40b30f0 0x7f73f414bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 >> 172.21.15.86:0/1551374807 conn(0x7f73f40b1df0 msgr2=0x7f73f40b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 shutdown_connections 2024-03-22T04:10:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.268+0000 7f73fae00640 1 -- 172.21.15.86:0/1551374807 wait complete. 2024-03-22T04:10:59.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:10:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa064800640 1 Processor -- start 2024-03-22T04:10:59.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa064800640 1 -- start start 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa064800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa05c0ad330 0x7fa05c15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa064800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad710 0x7fa05c142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa064800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa05c15c7c0 con 0x7fa05c06ef40 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa064800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa05c06dea0 con 0x7fa05c0ad330 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa064800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa05c06e180 con 0x7fa05c0ad710 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa061400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa05c06ef40 0x7fa05c13fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57452/0 (socket says 172.21.15.86:57452) 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa061400640 1 -- 172.21.15.86:0/3816423863 learned_addr learned my addr 172.21.15.86:0/3816423863 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa061e00640 1 --2- 172.21.15.86:0/3816423863 >> v2:172.21.15.196:3300/0 conn(0x7fa05c0ad330 0x7fa05c15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa062800640 1 --2- 172.21.15.86:0/3816423863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad710 0x7fa05c142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa060a00640 1 -- 172.21.15.86:0/3816423863 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375675420 0 0) 0x7fa05c15c7c0 con 0x7fa05c06ef40 2024-03-22T04:10:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa060a00640 1 -- 172.21.15.86:0/3816423863 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa044003530 con 0x7fa05c06ef40 2024-03-22T04:10:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa062800640 1 -- 172.21.15.86:0/3816423863 >> v2:172.21.15.196:3300/0 conn(0x7fa05c0ad330 msgr2=0x7fa05c15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa062800640 1 --2- 172.21.15.86:0/3816423863 >> v2:172.21.15.196:3300/0 conn(0x7fa05c0ad330 0x7fa05c15d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa062800640 1 -- 172.21.15.86:0/3816423863 >> v1:172.21.15.86:6789/0 conn(0x7fa05c06ef40 legacy=0x7fa05c13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.424+0000 7fa062800640 1 -- 172.21.15.86:0/3816423863 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c06e460 con 0x7fa05c0ad710 2024-03-22T04:10:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa062800640 1 --2- 172.21.15.86:0/3816423863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad710 0x7fa05c142690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa058009d90 tx=0x7fa05801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d75e5ec78d5072 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa060a00640 1 -- 172.21.15.86:0/3816423863 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0580b2070 con 0x7fa05c0ad710 2024-03-22T04:10:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa060a00640 1 -- 172.21.15.86:0/3816423863 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0580a5a00 con 0x7fa05c0ad710 2024-03-22T04:10:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa060a00640 1 -- 172.21.15.86:0/3816423863 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0580a5dd0 con 0x7fa05c0ad710 2024-03-22T04:10:59.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa064800640 1 -- 172.21.15.86:0/3816423863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad710 msgr2=0x7fa05c142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa064800640 1 --2- 172.21.15.86:0/3816423863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad710 0x7fa05c142690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa058009d90 tx=0x7fa05801c940 comp rx=0 tx=0).stop 2024-03-22T04:10:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa064800640 1 -- 172.21.15.86:0/3816423863 shutdown_connections 2024-03-22T04:10:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa064800640 1 --2- 172.21.15.86:0/3816423863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad710 0x7fa05c142690 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa064800640 1 --2- 172.21.15.86:0/3816423863 >> v2:172.21.15.196:3300/0 conn(0x7fa05c0ad330 0x7fa05c15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa064800640 1 -- 172.21.15.86:0/3816423863 >> 172.21.15.86:0/3816423863 conn(0x7fa05c0b1df0 msgr2=0x7fa05c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa064800640 1 -- 172.21.15.86:0/3816423863 shutdown_connections 2024-03-22T04:10:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.428+0000 7fa064800640 1 -- 172.21.15.86:0/3816423863 wait complete. 2024-03-22T04:10:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 Processor -- start 2024-03-22T04:10:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 -- start start 2024-03-22T04:10:59.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa05c06ef40 0x7fa05c09dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad330 0x7fa05c1668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa05c0ad710 0x7fa05c15b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1690e0 con 0x7fa05c06ef40 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c168f60 con 0x7fa05c0ad710 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa05c169260 con 0x7fa05c0ad330 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa05c06ef40 0x7fa05c09dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa05c06ef40 0x7fa05c09dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55858/0 (socket says 172.21.15.86:55858) 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 -- 172.21.15.86:0/3414683206 learned_addr learned my addr 172.21.15.86:0/3414683206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061400640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad330 0x7fa05c1668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa062800640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa05c0ad710 0x7fa05c15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 -- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad330 msgr2=0x7fa05c1668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad330 0x7fa05c1668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 -- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa05c0ad710 msgr2=0x7fa05c15b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa05c0ad710 0x7fa05c15b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 -- 172.21.15.86:0/3414683206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c1593c0 con 0x7fa05c06ef40 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061400640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad330 0x7fa05c1668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa061e00640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa05c06ef40 0x7fa05c09dd10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e930 tx=0x7fa05000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa056000640 1 -- 172.21.15.86:0/3414683206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050017070 con 0x7fa05c06ef40 2024-03-22T04:10:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa056000640 1 -- 172.21.15.86:0/3414683206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05000ca00 con 0x7fa05c06ef40 2024-03-22T04:10:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa056000640 1 -- 172.21.15.86:0/3414683206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05000cdb0 con 0x7fa05c06ef40 2024-03-22T04:10:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c165e30 con 0x7fa05c06ef40 2024-03-22T04:10:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c1660d0 con 0x7fa05c06ef40 2024-03-22T04:10:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.432+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05c068650 con 0x7fa05c06ef40 2024-03-22T04:10:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.436+0000 7fa056000640 1 -- 172.21.15.86:0/3414683206 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa05001a070 con 0x7fa05c06ef40 2024-03-22T04:10:59.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.436+0000 7fa056000640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa040067b20 0x7fa040069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.436+0000 7fa056000640 1 -- 172.21.15.86:0/3414683206 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa0500862c0 con 0x7fa05c06ef40 2024-03-22T04:10:59.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.436+0000 7fa061400640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa040067b20 0x7fa040069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.436+0000 7fa061400640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa040067b20 0x7fa040069fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa05c1faa40 tx=0x7fa04c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.440+0000 7fa056000640 1 -- 172.21.15.86:0/3414683206 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa05c1593c0 con 0x7fa05c06ef40 2024-03-22T04:10:59.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.596+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa05c165c00 con 0x7fa05c06ef40 2024-03-22T04:10:59.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.596+0000 7fa056000640 1 -- 172.21.15.86:0/3414683206 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7fa050056360 con 0x7fa05c06ef40 2024-03-22T04:10:59.601 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:59.601 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:10:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa040067b20 msgr2=0x7fa040069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa040067b20 0x7fa040069fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa05c1faa40 tx=0x7fa04c009040 comp rx=0 tx=0).stop 2024-03-22T04:10:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa05c06ef40 msgr2=0x7fa05c09dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa05c06ef40 0x7fa05c09dd10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e930 tx=0x7fa05000edf0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 shutdown_connections 2024-03-22T04:10:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa040067b20 0x7fa040069fe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa05c0ad710 0x7fa05c15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa05c0ad330 0x7fa05c1668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 --2- 172.21.15.86:0/3414683206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa05c06ef40 0x7fa05c09dd10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 >> 172.21.15.86:0/3414683206 conn(0x7fa05c0b1df0 msgr2=0x7fa05c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:59.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 shutdown_connections 2024-03-22T04:10:59.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.600+0000 7fa064800640 1 -- 172.21.15.86:0/3414683206 wait complete. 2024-03-22T04:10:59.622 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T04:10:59.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:10:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.752+0000 7f8f9c200640 1 Processor -- start 2024-03-22T04:10:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.752+0000 7f8f9c200640 1 -- start start 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f9c200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f94147880 0x7f8f9406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f9c200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8f9406cec0 0x7f8f940b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f9c200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 0x7f8f94162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f9c200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8f940b6320 con 0x7f8f94147880 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f9c200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8f9415ea90 con 0x7f8f9406cec0 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f9c200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9406d4f0 con 0x7f8f940b4890 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f99800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f94147880 0x7f8f9406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f99800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f94147880 0x7f8f9406c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55872/0 (socket says 172.21.15.86:55872) 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f98e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8f9406cec0 0x7f8f940b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f9a200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 0x7f8f94162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f99800640 1 -- 172.21.15.86:0/1033088367 learned_addr learned my addr 172.21.15.86:0/1033088367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f99800640 1 -- 172.21.15.86:0/1033088367 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 msgr2=0x7f8f94162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f99800640 1 --2- 172.21.15.86:0/1033088367 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 0x7f8f94162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f99800640 1 -- 172.21.15.86:0/1033088367 >> v2:172.21.15.196:3300/0 conn(0x7f8f9406cec0 msgr2=0x7f8f940b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:10:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f99800640 1 --2- 172.21.15.86:0/1033088367 >> v2:172.21.15.196:3300/0 conn(0x7f8f9406cec0 0x7f8f940b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f99800640 1 -- 172.21.15.86:0/1033088367 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9406d7d0 con 0x7f8f94147880 2024-03-22T04:10:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.756+0000 7f8f98e00640 1 --2- 172.21.15.86:0/1033088367 >> v2:172.21.15.196:3300/0 conn(0x7f8f9406cec0 0x7f8f940b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:10:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f99800640 1 --2- 172.21.15.86:0/1033088367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9406c980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009870 tx=0x7f8f8801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c19fd800c3d7e21 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f93e00640 1 -- 172.21.15.86:0/1033088367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f880b0070 con 0x7f8f94147880 2024-03-22T04:10:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f93e00640 1 -- 172.21.15.86:0/1033088367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f880ab600 con 0x7f8f94147880 2024-03-22T04:10:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f93e00640 1 -- 172.21.15.86:0/1033088367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f880ab9d0 con 0x7f8f94147880 2024-03-22T04:10:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 -- 172.21.15.86:0/1033088367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 msgr2=0x7f8f9406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 --2- 172.21.15.86:0/1033088367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9406c980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009870 tx=0x7f8f8801c260 comp rx=0 tx=0).stop 2024-03-22T04:10:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 -- 172.21.15.86:0/1033088367 shutdown_connections 2024-03-22T04:10:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 --2- 172.21.15.86:0/1033088367 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 0x7f8f94162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 --2- 172.21.15.86:0/1033088367 >> v2:172.21.15.196:3300/0 conn(0x7f8f9406cec0 0x7f8f940b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 --2- 172.21.15.86:0/1033088367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9406c980 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 -- 172.21.15.86:0/1033088367 >> 172.21.15.86:0/1033088367 conn(0x7f8f9406ba60 msgr2=0x7f8f9406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 -- 172.21.15.86:0/1033088367 shutdown_connections 2024-03-22T04:10:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 -- 172.21.15.86:0/1033088367 wait complete. 2024-03-22T04:10:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 Processor -- start 2024-03-22T04:10:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.760+0000 7f8f9c200640 1 -- start start 2024-03-22T04:10:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f9406cec0 0x7f8f9413ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 0x7f8f94146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9413e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9415ef40 con 0x7f8f94147880 2024-03-22T04:10:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9406d2d0 con 0x7f8f9406cec0 2024-03-22T04:10:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9415f0c0 con 0x7f8f940b4890 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9413e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9413e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55878/0 (socket says 172.21.15.86:55878) 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 -- 172.21.15.86:0/292400879 learned_addr learned my addr 172.21.15.86:0/292400879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f99800640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f9406cec0 0x7f8f9413ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f98e00640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 0x7f8f94146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 -- 172.21.15.86:0/292400879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 msgr2=0x7f8f94146d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 0x7f8f94146d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 -- 172.21.15.86:0/292400879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f9406cec0 msgr2=0x7f8f9413ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f9406cec0 0x7f8f9413ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 -- 172.21.15.86:0/292400879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f941632a0 con 0x7f8f94147880 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9a200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9413e9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00efc0 tx=0x7f8f8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f91600640 1 -- 172.21.15.86:0/292400879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c018020 con 0x7f8f94147880 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f91600640 1 -- 172.21.15.86:0/292400879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c014480 con 0x7f8f94147880 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f91600640 1 -- 172.21.15.86:0/292400879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c014870 con 0x7f8f94147880 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f94163580 con 0x7f8f94147880 2024-03-22T04:10:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f94163b40 con 0x7f8f94147880 2024-03-22T04:10:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f94068650 con 0x7f8f94147880 2024-03-22T04:10:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.764+0000 7f8f91600640 1 -- 172.21.15.86:0/292400879 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8f8c01b020 con 0x7f8f94147880 2024-03-22T04:10:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.768+0000 7f8f91600640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f74067b20 0x7f8f74069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:10:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.768+0000 7f8f91600640 1 -- 172.21.15.86:0/292400879 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8f8c086b00 con 0x7f8f94147880 2024-03-22T04:10:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.768+0000 7f8f99800640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f74067b20 0x7f8f74069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:10:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.768+0000 7f8f99800640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f74067b20 0x7f8f74069fe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f880ad000 tx=0x7f8f880a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:10:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.772+0000 7f8f91600640 1 -- 172.21.15.86:0/292400879 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8f8c022070 con 0x7f8f94147880 2024-03-22T04:10:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.924+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f94072220 con 0x7f8f74067b20 2024-03-22T04:10:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.936+0000 7f8f91600640 1 -- 172.21.15.86:0/292400879 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f8f94072220 con 0x7f8f74067b20 2024-03-22T04:10:59.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:10:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:10:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f74067b20 msgr2=0x7f8f74069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f74067b20 0x7f8f74069fe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f880ad000 tx=0x7f8f880a6000 comp rx=0 tx=0).stop 2024-03-22T04:10:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 msgr2=0x7f8f9413e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:10:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9413e9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00efc0 tx=0x7f8f8c00c490 comp rx=0 tx=0).stop 2024-03-22T04:10:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 shutdown_connections 2024-03-22T04:10:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8f74067b20 0x7f8f74069fe0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94147880 0x7f8f9413e9c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f940b4890 0x7f8f94146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 --2- 172.21.15.86:0/292400879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f9406cec0 0x7f8f9413ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:10:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 >> 172.21.15.86:0/292400879 conn(0x7f8f9406ba60 msgr2=0x7f8f941600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:10:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 shutdown_connections 2024-03-22T04:10:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:10:59.940+0000 7f8f9c200640 1 -- 172.21.15.86:0/292400879 wait complete. 2024-03-22T04:10:59.963 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-03-22T04:10:59.796813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219592,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755128,"statfs":{"total":773094113280,"available":772869251072,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002971"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967926+0000","last_change":"2024-03-22T04:10:33.930897+0000","last_active":"2024-03-22T04:10:39.967926+0000","last_peered":"2024-03-22T04:10:39.967926+0000","last_clean":"2024-03-22T04:10:39.967926+0000","last_became_active":"2024-03-22T04:10:33.930543+0000","last_became_peered":"2024-03-22T04:10:33.930543+0000","last_unstale":"2024-03-22T04:10:39.967926+0000","last_undegraded":"2024-03-22T04:10:39.967926+0000","last_fullsized":"2024-03-22T04:10:39.967926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:57:52.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170514+0000","last_change":"2024-03-22T04:10:34.938515+0000","last_active":"2024-03-22T04:10:41.170514+0000","last_peered":"2024-03-22T04:10:41.170514+0000","last_clean":"2024-03-22T04:10:41.170514+0000","last_became_active":"2024-03-22T04:10:34.938169+0000","last_became_peered":"2024-03-22T04:10:34.938169+0000","last_unstale":"2024-03-22T04:10:41.170514+0000","last_undegraded":"2024-03-22T04:10:41.170514+0000","last_fullsized":"2024-03-22T04:10:41.170514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:34.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966658+0000","last_change":"2024-03-22T04:10:38.983720+0000","last_active":"2024-03-22T04:10:39.966658+0000","last_peered":"2024-03-22T04:10:39.966658+0000","last_clean":"2024-03-22T04:10:39.966658+0000","last_became_active":"2024-03-22T04:10:38.983348+0000","last_became_peered":"2024-03-22T04:10:38.983348+0000","last_unstale":"2024-03-22T04:10:39.966658+0000","last_undegraded":"2024-03-22T04:10:39.966658+0000","last_fullsized":"2024-03-22T04:10:39.966658+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:45.494650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170607+0000","last_change":"2024-03-22T04:10:33.934030+0000","last_active":"2024-03-22T04:10:41.170607+0000","last_peered":"2024-03-22T04:10:41.170607+0000","last_clean":"2024-03-22T04:10:41.170607+0000","last_became_active":"2024-03-22T04:10:33.933718+0000","last_became_peered":"2024-03-22T04:10:33.933718+0000","last_unstale":"2024-03-22T04:10:41.170607+0000","last_undegraded":"2024-03-22T04:10:41.170607+0000","last_fullsized":"2024-03-22T04:10:41.170607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:50:42.805996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170782+0000","last_change":"2024-03-22T04:10:34.942768+0000","last_active":"2024-03-22T04:10:41.170782+0000","last_peered":"2024-03-22T04:10:41.170782+0000","last_clean":"2024-03-22T04:10:41.170782+0000","last_became_active":"2024-03-22T04:10:34.941567+0000","last_became_peered":"2024-03-22T04:10:34.941567+0000","last_unstale":"2024-03-22T04:10:41.170782+0000","last_undegraded":"2024-03-22T04:10:41.170782+0000","last_fullsized":"2024-03-22T04:10:41.170782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:26.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964401+0000","last_change":"2024-03-22T04:10:38.988088+0000","last_active":"2024-03-22T04:10:39.964401+0000","last_peered":"2024-03-22T04:10:39.964401+0000","last_clean":"2024-03-22T04:10:39.964401+0000","last_became_active":"2024-03-22T04:10:38.987611+0000","last_became_peered":"2024-03-22T04:10:38.987611+0000","last_unstale":"2024-03-22T04:10:39.964401+0000","last_undegraded":"2024-03-22T04:10:39.964401+0000","last_fullsized":"2024-03-22T04:10:39.964401+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:15.151121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535943+0000","last_change":"2024-03-22T04:10:33.936193+0000","last_active":"2024-03-22T04:10:41.535943+0000","last_peered":"2024-03-22T04:10:41.535943+0000","last_clean":"2024-03-22T04:10:41.535943+0000","last_became_active":"2024-03-22T04:10:33.935883+0000","last_became_peered":"2024-03-22T04:10:33.935883+0000","last_unstale":"2024-03-22T04:10:41.535943+0000","last_undegraded":"2024-03-22T04:10:41.535943+0000","last_fullsized":"2024-03-22T04:10:41.535943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:00.244111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535389+0000","last_change":"2024-03-22T04:10:34.935504+0000","last_active":"2024-03-22T04:10:41.535389+0000","last_peered":"2024-03-22T04:10:41.535389+0000","last_clean":"2024-03-22T04:10:41.535389+0000","last_became_active":"2024-03-22T04:10:34.933502+0000","last_became_peered":"2024-03-22T04:10:34.933502+0000","last_unstale":"2024-03-22T04:10:41.535389+0000","last_undegraded":"2024-03-22T04:10:41.535389+0000","last_fullsized":"2024-03-22T04:10:41.535389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965123+0000","last_change":"2024-03-22T04:10:38.991103+0000","last_active":"2024-03-22T04:10:39.965123+0000","last_peered":"2024-03-22T04:10:39.965123+0000","last_clean":"2024-03-22T04:10:39.965123+0000","last_became_active":"2024-03-22T04:10:38.988847+0000","last_became_peered":"2024-03-22T04:10:38.988847+0000","last_unstale":"2024-03-22T04:10:39.965123+0000","last_undegraded":"2024-03-22T04:10:39.965123+0000","last_fullsized":"2024-03-22T04:10:39.965123+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:45.922825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535279+0000","last_change":"2024-03-22T04:10:33.936237+0000","last_active":"2024-03-22T04:10:41.535279+0000","last_peered":"2024-03-22T04:10:41.535279+0000","last_clean":"2024-03-22T04:10:41.535279+0000","last_became_active":"2024-03-22T04:10:33.935870+0000","last_became_peered":"2024-03-22T04:10:33.935870+0000","last_unstale":"2024-03-22T04:10:41.535279+0000","last_undegraded":"2024-03-22T04:10:41.535279+0000","last_fullsized":"2024-03-22T04:10:41.535279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:47.209113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922489+0000","last_change":"2024-03-22T04:10:34.930847+0000","last_active":"2024-03-22T04:10:40.922489+0000","last_peered":"2024-03-22T04:10:40.922489+0000","last_clean":"2024-03-22T04:10:40.922489+0000","last_became_active":"2024-03-22T04:10:34.930567+0000","last_became_peered":"2024-03-22T04:10:34.930567+0000","last_unstale":"2024-03-22T04:10:40.922489+0000","last_undegraded":"2024-03-22T04:10:40.922489+0000","last_fullsized":"2024-03-22T04:10:40.922489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:17.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964576+0000","last_change":"2024-03-22T04:10:38.992358+0000","last_active":"2024-03-22T04:10:39.964576+0000","last_peered":"2024-03-22T04:10:39.964576+0000","last_clean":"2024-03-22T04:10:39.964576+0000","last_became_active":"2024-03-22T04:10:38.991968+0000","last_became_peered":"2024-03-22T04:10:38.991968+0000","last_unstale":"2024-03-22T04:10:39.964576+0000","last_undegraded":"2024-03-22T04:10:39.964576+0000","last_fullsized":"2024-03-22T04:10:39.964576+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:12.109401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922040+0000","last_change":"2024-03-22T04:10:34.947353+0000","last_active":"2024-03-22T04:10:40.922040+0000","last_peered":"2024-03-22T04:10:40.922040+0000","last_clean":"2024-03-22T04:10:40.922040+0000","last_became_active":"2024-03-22T04:10:34.945247+0000","last_became_peered":"2024-03-22T04:10:34.945247+0000","last_unstale":"2024-03-22T04:10:40.922040+0000","last_undegraded":"2024-03-22T04:10:40.922040+0000","last_fullsized":"2024-03-22T04:10:40.922040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:31.574004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922244+0000","last_change":"2024-03-22T04:10:33.938771+0000","last_active":"2024-03-22T04:10:40.922244+0000","last_peered":"2024-03-22T04:10:40.922244+0000","last_clean":"2024-03-22T04:10:40.922244+0000","last_became_active":"2024-03-22T04:10:33.937907+0000","last_became_peered":"2024-03-22T04:10:33.937907+0000","last_unstale":"2024-03-22T04:10:40.922244+0000","last_undegraded":"2024-03-22T04:10:40.922244+0000","last_fullsized":"2024-03-22T04:10:40.922244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:43.867139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922386+0000","last_change":"2024-03-22T04:10:38.979866+0000","last_active":"2024-03-22T04:10:40.922386+0000","last_peered":"2024-03-22T04:10:40.922386+0000","last_clean":"2024-03-22T04:10:40.922386+0000","last_became_active":"2024-03-22T04:10:38.979321+0000","last_became_peered":"2024-03-22T04:10:38.979321+0000","last_unstale":"2024-03-22T04:10:40.922386+0000","last_undegraded":"2024-03-22T04:10:40.922386+0000","last_fullsized":"2024-03-22T04:10:40.922386+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:03:12.221975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967207+0000","last_change":"2024-03-22T04:10:34.951257+0000","last_active":"2024-03-22T04:10:39.967207+0000","last_peered":"2024-03-22T04:10:39.967207+0000","last_clean":"2024-03-22T04:10:39.967207+0000","last_became_active":"2024-03-22T04:10:34.950994+0000","last_became_peered":"2024-03-22T04:10:34.950994+0000","last_unstale":"2024-03-22T04:10:39.967207+0000","last_undegraded":"2024-03-22T04:10:39.967207+0000","last_fullsized":"2024-03-22T04:10:39.967207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536303+0000","last_change":"2024-03-22T04:10:33.933287+0000","last_active":"2024-03-22T04:10:41.536303+0000","last_peered":"2024-03-22T04:10:41.536303+0000","last_clean":"2024-03-22T04:10:41.536303+0000","last_became_active":"2024-03-22T04:10:33.932862+0000","last_became_peered":"2024-03-22T04:10:33.932862+0000","last_unstale":"2024-03-22T04:10:41.536303+0000","last_undegraded":"2024-03-22T04:10:41.536303+0000","last_fullsized":"2024-03-22T04:10:41.536303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534955+0000","last_change":"2024-03-22T04:10:38.986486+0000","last_active":"2024-03-22T04:10:41.534955+0000","last_peered":"2024-03-22T04:10:41.534955+0000","last_clean":"2024-03-22T04:10:41.534955+0000","last_became_active":"2024-03-22T04:10:38.985993+0000","last_became_peered":"2024-03-22T04:10:38.985993+0000","last_unstale":"2024-03-22T04:10:41.534955+0000","last_undegraded":"2024-03-22T04:10:41.534955+0000","last_fullsized":"2024-03-22T04:10:41.534955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:21.157155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966213+0000","last_change":"2024-03-22T04:10:34.946405+0000","last_active":"2024-03-22T04:10:39.966213+0000","last_peered":"2024-03-22T04:10:39.966213+0000","last_clean":"2024-03-22T04:10:39.966213+0000","last_became_active":"2024-03-22T04:10:34.946093+0000","last_became_peered":"2024-03-22T04:10:34.946093+0000","last_unstale":"2024-03-22T04:10:39.966213+0000","last_undegraded":"2024-03-22T04:10:39.966213+0000","last_fullsized":"2024-03-22T04:10:39.966213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:54:39.775612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966845+0000","last_change":"2024-03-22T04:10:33.931984+0000","last_active":"2024-03-22T04:10:39.966845+0000","last_peered":"2024-03-22T04:10:39.966845+0000","last_clean":"2024-03-22T04:10:39.966845+0000","last_became_active":"2024-03-22T04:10:33.931674+0000","last_became_peered":"2024-03-22T04:10:33.931674+0000","last_unstale":"2024-03-22T04:10:39.966845+0000","last_undegraded":"2024-03-22T04:10:39.966845+0000","last_fullsized":"2024-03-22T04:10:39.966845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:11.685927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967850+0000","last_change":"2024-03-22T04:10:38.992239+0000","last_active":"2024-03-22T04:10:39.967850+0000","last_peered":"2024-03-22T04:10:39.967850+0000","last_clean":"2024-03-22T04:10:39.967850+0000","last_became_active":"2024-03-22T04:10:38.991429+0000","last_became_peered":"2024-03-22T04:10:38.991429+0000","last_unstale":"2024-03-22T04:10:39.967850+0000","last_undegraded":"2024-03-22T04:10:39.967850+0000","last_fullsized":"2024-03-22T04:10:39.967850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:22.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921940+0000","last_change":"2024-03-22T04:10:34.939183+0000","last_active":"2024-03-22T04:10:40.921940+0000","last_peered":"2024-03-22T04:10:40.921940+0000","last_clean":"2024-03-22T04:10:40.921940+0000","last_became_active":"2024-03-22T04:10:34.938388+0000","last_became_peered":"2024-03-22T04:10:34.938388+0000","last_unstale":"2024-03-22T04:10:40.921940+0000","last_undegraded":"2024-03-22T04:10:40.921940+0000","last_fullsized":"2024-03-22T04:10:40.921940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:10:15.750612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534426+0000","last_change":"2024-03-22T04:10:33.933352+0000","last_active":"2024-03-22T04:10:41.534426+0000","last_peered":"2024-03-22T04:10:41.534426+0000","last_clean":"2024-03-22T04:10:41.534426+0000","last_became_active":"2024-03-22T04:10:33.933066+0000","last_became_peered":"2024-03-22T04:10:33.933066+0000","last_unstale":"2024-03-22T04:10:41.534426+0000","last_undegraded":"2024-03-22T04:10:41.534426+0000","last_fullsized":"2024-03-22T04:10:41.534426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:50:42.805996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964895+0000","last_change":"2024-03-22T04:10:38.987700+0000","last_active":"2024-03-22T04:10:39.964895+0000","last_peered":"2024-03-22T04:10:39.964895+0000","last_clean":"2024-03-22T04:10:39.964895+0000","last_became_active":"2024-03-22T04:10:38.987221+0000","last_became_peered":"2024-03-22T04:10:38.987221+0000","last_unstale":"2024-03-22T04:10:39.964895+0000","last_undegraded":"2024-03-22T04:10:39.964895+0000","last_fullsized":"2024-03-22T04:10:39.964895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:30.365392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967947+0000","last_change":"2024-03-22T04:10:34.953907+0000","last_active":"2024-03-22T04:10:39.967947+0000","last_peered":"2024-03-22T04:10:39.967947+0000","last_clean":"2024-03-22T04:10:39.967947+0000","last_became_active":"2024-03-22T04:10:34.953664+0000","last_became_peered":"2024-03-22T04:10:34.953664+0000","last_unstale":"2024-03-22T04:10:39.967947+0000","last_undegraded":"2024-03-22T04:10:39.967947+0000","last_fullsized":"2024-03-22T04:10:39.967947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:07.983679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922944+0000","last_change":"2024-03-22T04:10:33.931133+0000","last_active":"2024-03-22T04:10:40.922944+0000","last_peered":"2024-03-22T04:10:40.922944+0000","last_clean":"2024-03-22T04:10:40.922944+0000","last_became_active":"2024-03-22T04:10:33.930877+0000","last_became_peered":"2024-03-22T04:10:33.930877+0000","last_unstale":"2024-03-22T04:10:40.922944+0000","last_undegraded":"2024-03-22T04:10:40.922944+0000","last_fullsized":"2024-03-22T04:10:40.922944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:39.248649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964896+0000","last_change":"2024-03-22T04:10:38.983903+0000","last_active":"2024-03-22T04:10:39.964896+0000","last_peered":"2024-03-22T04:10:39.964896+0000","last_clean":"2024-03-22T04:10:39.964896+0000","last_became_active":"2024-03-22T04:10:38.983459+0000","last_became_peered":"2024-03-22T04:10:38.983459+0000","last_unstale":"2024-03-22T04:10:39.964896+0000","last_undegraded":"2024-03-22T04:10:39.964896+0000","last_fullsized":"2024-03-22T04:10:39.964896+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:38.915424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967821+0000","last_change":"2024-03-22T04:10:34.930898+0000","last_active":"2024-03-22T04:10:39.967821+0000","last_peered":"2024-03-22T04:10:39.967821+0000","last_clean":"2024-03-22T04:10:39.967821+0000","last_became_active":"2024-03-22T04:10:34.930562+0000","last_became_peered":"2024-03-22T04:10:34.930562+0000","last_unstale":"2024-03-22T04:10:39.967821+0000","last_undegraded":"2024-03-22T04:10:39.967821+0000","last_fullsized":"2024-03-22T04:10:39.967821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:46:40.225626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.968051+0000","last_change":"2024-03-22T04:10:33.934918+0000","last_active":"2024-03-22T04:10:39.968051+0000","last_peered":"2024-03-22T04:10:39.968051+0000","last_clean":"2024-03-22T04:10:39.968051+0000","last_became_active":"2024-03-22T04:10:33.933026+0000","last_became_peered":"2024-03-22T04:10:33.933026+0000","last_unstale":"2024-03-22T04:10:39.968051+0000","last_undegraded":"2024-03-22T04:10:39.968051+0000","last_fullsized":"2024-03-22T04:10:39.968051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:42:25.730112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966295+0000","last_change":"2024-03-22T04:10:38.984737+0000","last_active":"2024-03-22T04:10:39.966295+0000","last_peered":"2024-03-22T04:10:39.966295+0000","last_clean":"2024-03-22T04:10:39.966295+0000","last_became_active":"2024-03-22T04:10:38.984373+0000","last_became_peered":"2024-03-22T04:10:38.984373+0000","last_unstale":"2024-03-22T04:10:39.966295+0000","last_undegraded":"2024-03-22T04:10:39.966295+0000","last_fullsized":"2024-03-22T04:10:39.966295+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:23:32.419897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536493+0000","last_change":"2024-03-22T04:10:34.928433+0000","last_active":"2024-03-22T04:10:41.536493+0000","last_peered":"2024-03-22T04:10:41.536493+0000","last_clean":"2024-03-22T04:10:41.536493+0000","last_became_active":"2024-03-22T04:10:34.927912+0000","last_became_peered":"2024-03-22T04:10:34.927912+0000","last_unstale":"2024-03-22T04:10:41.536493+0000","last_undegraded":"2024-03-22T04:10:41.536493+0000","last_fullsized":"2024-03-22T04:10:41.536493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:12:01.577501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922703+0000","last_change":"2024-03-22T04:10:33.938458+0000","last_active":"2024-03-22T04:10:40.922703+0000","last_peered":"2024-03-22T04:10:40.922703+0000","last_clean":"2024-03-22T04:10:40.922703+0000","last_became_active":"2024-03-22T04:10:33.937789+0000","last_became_peered":"2024-03-22T04:10:33.937789+0000","last_unstale":"2024-03-22T04:10:40.922703+0000","last_undegraded":"2024-03-22T04:10:40.922703+0000","last_fullsized":"2024-03-22T04:10:40.922703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:40.441952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536228+0000","last_change":"2024-03-22T04:10:38.988981+0000","last_active":"2024-03-22T04:10:41.536228+0000","last_peered":"2024-03-22T04:10:41.536228+0000","last_clean":"2024-03-22T04:10:41.536228+0000","last_became_active":"2024-03-22T04:10:38.988442+0000","last_became_peered":"2024-03-22T04:10:38.988442+0000","last_unstale":"2024-03-22T04:10:41.536228+0000","last_undegraded":"2024-03-22T04:10:41.536228+0000","last_fullsized":"2024-03-22T04:10:41.536228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:38.915424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964614+0000","last_change":"2024-03-22T04:10:34.931657+0000","last_active":"2024-03-22T04:10:39.964614+0000","last_peered":"2024-03-22T04:10:39.964614+0000","last_clean":"2024-03-22T04:10:39.964614+0000","last_became_active":"2024-03-22T04:10:34.931362+0000","last_became_peered":"2024-03-22T04:10:34.931362+0000","last_unstale":"2024-03-22T04:10:39.964614+0000","last_undegraded":"2024-03-22T04:10:39.964614+0000","last_fullsized":"2024-03-22T04:10:39.964614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170215+0000","last_change":"2024-03-22T04:10:33.934747+0000","last_active":"2024-03-22T04:10:41.170215+0000","last_peered":"2024-03-22T04:10:41.170215+0000","last_clean":"2024-03-22T04:10:41.170215+0000","last_became_active":"2024-03-22T04:10:33.933151+0000","last_became_peered":"2024-03-22T04:10:33.933151+0000","last_unstale":"2024-03-22T04:10:41.170215+0000","last_undegraded":"2024-03-22T04:10:41.170215+0000","last_fullsized":"2024-03-22T04:10:41.170215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536019+0000","last_change":"2024-03-22T04:10:38.972402+0000","last_active":"2024-03-22T04:10:41.536019+0000","last_peered":"2024-03-22T04:10:41.536019+0000","last_clean":"2024-03-22T04:10:41.536019+0000","last_became_active":"2024-03-22T04:10:38.972049+0000","last_became_peered":"2024-03-22T04:10:38.972049+0000","last_unstale":"2024-03-22T04:10:41.536019+0000","last_undegraded":"2024-03-22T04:10:41.536019+0000","last_fullsized":"2024-03-22T04:10:41.536019+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:20.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535049+0000","last_change":"2024-03-22T04:10:34.931298+0000","last_active":"2024-03-22T04:10:41.535049+0000","last_peered":"2024-03-22T04:10:41.535049+0000","last_clean":"2024-03-22T04:10:41.535049+0000","last_became_active":"2024-03-22T04:10:34.930240+0000","last_became_peered":"2024-03-22T04:10:34.930240+0000","last_unstale":"2024-03-22T04:10:41.535049+0000","last_undegraded":"2024-03-22T04:10:41.535049+0000","last_fullsized":"2024-03-22T04:10:41.535049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:11.753753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922237+0000","last_change":"2024-03-22T04:10:33.938276+0000","last_active":"2024-03-22T04:10:40.922237+0000","last_peered":"2024-03-22T04:10:40.922237+0000","last_clean":"2024-03-22T04:10:40.922237+0000","last_became_active":"2024-03-22T04:10:33.933126+0000","last_became_peered":"2024-03-22T04:10:33.933126+0000","last_unstale":"2024-03-22T04:10:40.922237+0000","last_undegraded":"2024-03-22T04:10:40.922237+0000","last_fullsized":"2024-03-22T04:10:40.922237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:11.368879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536420+0000","last_change":"2024-03-22T04:10:38.974697+0000","last_active":"2024-03-22T04:10:41.536420+0000","last_peered":"2024-03-22T04:10:41.536420+0000","last_clean":"2024-03-22T04:10:41.536420+0000","last_became_active":"2024-03-22T04:10:38.974339+0000","last_became_peered":"2024-03-22T04:10:38.974339+0000","last_unstale":"2024-03-22T04:10:41.536420+0000","last_undegraded":"2024-03-22T04:10:41.536420+0000","last_fullsized":"2024-03-22T04:10:41.536420+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:59.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169762+0000","last_change":"2024-03-22T04:10:34.947104+0000","last_active":"2024-03-22T04:10:41.169762+0000","last_peered":"2024-03-22T04:10:41.169762+0000","last_clean":"2024-03-22T04:10:41.169762+0000","last_became_active":"2024-03-22T04:10:34.946787+0000","last_became_peered":"2024-03-22T04:10:34.946787+0000","last_unstale":"2024-03-22T04:10:41.169762+0000","last_undegraded":"2024-03-22T04:10:41.169762+0000","last_fullsized":"2024-03-22T04:10:41.169762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:11.753753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966701+0000","last_change":"2024-03-22T04:10:33.938271+0000","last_active":"2024-03-22T04:10:39.966701+0000","last_peered":"2024-03-22T04:10:39.966701+0000","last_clean":"2024-03-22T04:10:39.966701+0000","last_became_active":"2024-03-22T04:10:33.937651+0000","last_became_peered":"2024-03-22T04:10:33.937651+0000","last_unstale":"2024-03-22T04:10:39.966701+0000","last_undegraded":"2024-03-22T04:10:39.966701+0000","last_fullsized":"2024-03-22T04:10:39.966701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536236+0000","last_change":"2024-03-22T04:10:38.980629+0000","last_active":"2024-03-22T04:10:41.536236+0000","last_peered":"2024-03-22T04:10:41.536236+0000","last_clean":"2024-03-22T04:10:41.536236+0000","last_became_active":"2024-03-22T04:10:38.980157+0000","last_became_peered":"2024-03-22T04:10:38.980157+0000","last_unstale":"2024-03-22T04:10:41.536236+0000","last_undegraded":"2024-03-22T04:10:41.536236+0000","last_fullsized":"2024-03-22T04:10:41.536236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:30.365392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535927+0000","last_change":"2024-03-22T04:10:34.940387+0000","last_active":"2024-03-22T04:10:41.535927+0000","last_peered":"2024-03-22T04:10:41.535927+0000","last_clean":"2024-03-22T04:10:41.535927+0000","last_became_active":"2024-03-22T04:10:34.940009+0000","last_became_peered":"2024-03-22T04:10:34.940009+0000","last_unstale":"2024-03-22T04:10:41.535927+0000","last_undegraded":"2024-03-22T04:10:41.535927+0000","last_fullsized":"2024-03-22T04:10:41.535927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:12.376047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922076+0000","last_change":"2024-03-22T04:10:33.927977+0000","last_active":"2024-03-22T04:10:40.922076+0000","last_peered":"2024-03-22T04:10:40.922076+0000","last_clean":"2024-03-22T04:10:40.922076+0000","last_became_active":"2024-03-22T04:10:33.927635+0000","last_became_peered":"2024-03-22T04:10:33.927635+0000","last_unstale":"2024-03-22T04:10:40.922076+0000","last_undegraded":"2024-03-22T04:10:40.922076+0000","last_fullsized":"2024-03-22T04:10:40.922076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:00.244111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170363+0000","last_change":"2024-03-22T04:10:38.985749+0000","last_active":"2024-03-22T04:10:41.170363+0000","last_peered":"2024-03-22T04:10:41.170363+0000","last_clean":"2024-03-22T04:10:41.170363+0000","last_became_active":"2024-03-22T04:10:38.981278+0000","last_became_peered":"2024-03-22T04:10:38.981278+0000","last_unstale":"2024-03-22T04:10:41.170363+0000","last_undegraded":"2024-03-22T04:10:41.170363+0000","last_fullsized":"2024-03-22T04:10:41.170363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:40:38.204332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966651+0000","last_change":"2024-03-22T04:10:34.921916+0000","last_active":"2024-03-22T04:10:39.966651+0000","last_peered":"2024-03-22T04:10:39.966651+0000","last_clean":"2024-03-22T04:10:39.966651+0000","last_became_active":"2024-03-22T04:10:34.921675+0000","last_became_peered":"2024-03-22T04:10:34.921675+0000","last_unstale":"2024-03-22T04:10:39.966651+0000","last_undegraded":"2024-03-22T04:10:39.966651+0000","last_fullsized":"2024-03-22T04:10:39.966651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:23:07.242835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922298+0000","last_change":"2024-03-22T04:10:33.937845+0000","last_active":"2024-03-22T04:10:40.922298+0000","last_peered":"2024-03-22T04:10:40.922298+0000","last_clean":"2024-03-22T04:10:40.922298+0000","last_became_active":"2024-03-22T04:10:33.930400+0000","last_became_peered":"2024-03-22T04:10:33.930400+0000","last_unstale":"2024-03-22T04:10:40.922298+0000","last_undegraded":"2024-03-22T04:10:40.922298+0000","last_fullsized":"2024-03-22T04:10:40.922298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:03.705836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":177,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535319+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:10:41.535319+0000","last_peered":"2024-03-22T04:10:41.535319+0000","last_clean":"2024-03-22T04:10:41.535319+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:10:41.535319+0000","last_undegraded":"2024-03-22T04:10:41.535319+0000","last_fullsized":"2024-03-22T04:10:41.535319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922657+0000","last_change":"2024-03-22T04:10:38.982996+0000","last_active":"2024-03-22T04:10:40.922657+0000","last_peered":"2024-03-22T04:10:40.922657+0000","last_clean":"2024-03-22T04:10:40.922657+0000","last_became_active":"2024-03-22T04:10:38.981238+0000","last_became_peered":"2024-03-22T04:10:38.981238+0000","last_unstale":"2024-03-22T04:10:40.922657+0000","last_undegraded":"2024-03-22T04:10:40.922657+0000","last_fullsized":"2024-03-22T04:10:40.922657+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:09:02.614234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922533+0000","last_change":"2024-03-22T04:10:34.945879+0000","last_active":"2024-03-22T04:10:40.922533+0000","last_peered":"2024-03-22T04:10:40.922533+0000","last_clean":"2024-03-22T04:10:40.922533+0000","last_became_active":"2024-03-22T04:10:34.945056+0000","last_became_peered":"2024-03-22T04:10:34.945056+0000","last_unstale":"2024-03-22T04:10:40.922533+0000","last_undegraded":"2024-03-22T04:10:40.922533+0000","last_fullsized":"2024-03-22T04:10:40.922533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:17:47.020807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536542+0000","last_change":"2024-03-22T04:10:33.931838+0000","last_active":"2024-03-22T04:10:41.536542+0000","last_peered":"2024-03-22T04:10:41.536542+0000","last_clean":"2024-03-22T04:10:41.536542+0000","last_became_active":"2024-03-22T04:10:33.931611+0000","last_became_peered":"2024-03-22T04:10:33.931611+0000","last_unstale":"2024-03-22T04:10:41.536542+0000","last_undegraded":"2024-03-22T04:10:41.536542+0000","last_fullsized":"2024-03-22T04:10:41.536542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964623+0000","last_change":"2024-03-22T04:10:38.988757+0000","last_active":"2024-03-22T04:10:39.964623+0000","last_peered":"2024-03-22T04:10:39.964623+0000","last_clean":"2024-03-22T04:10:39.964623+0000","last_became_active":"2024-03-22T04:10:38.987185+0000","last_became_peered":"2024-03-22T04:10:38.987185+0000","last_unstale":"2024-03-22T04:10:39.964623+0000","last_undegraded":"2024-03-22T04:10:39.964623+0000","last_fullsized":"2024-03-22T04:10:39.964623+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:15.799282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535188+0000","last_change":"2024-03-22T04:10:34.948529+0000","last_active":"2024-03-22T04:10:41.535188+0000","last_peered":"2024-03-22T04:10:41.535188+0000","last_clean":"2024-03-22T04:10:41.535188+0000","last_became_active":"2024-03-22T04:10:34.948122+0000","last_became_peered":"2024-03-22T04:10:34.948122+0000","last_unstale":"2024-03-22T04:10:41.535188+0000","last_undegraded":"2024-03-22T04:10:41.535188+0000","last_fullsized":"2024-03-22T04:10:41.535188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:55.025923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170450+0000","last_change":"2024-03-22T04:10:33.938126+0000","last_active":"2024-03-22T04:10:41.170450+0000","last_peered":"2024-03-22T04:10:41.170450+0000","last_clean":"2024-03-22T04:10:41.170450+0000","last_became_active":"2024-03-22T04:10:33.937511+0000","last_became_peered":"2024-03-22T04:10:33.937511+0000","last_unstale":"2024-03-22T04:10:41.170450+0000","last_undegraded":"2024-03-22T04:10:41.170450+0000","last_fullsized":"2024-03-22T04:10:41.170450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169980+0000","last_change":"2024-03-22T04:10:38.988377+0000","last_active":"2024-03-22T04:10:41.169980+0000","last_peered":"2024-03-22T04:10:41.169980+0000","last_clean":"2024-03-22T04:10:41.169980+0000","last_became_active":"2024-03-22T04:10:38.984506+0000","last_became_peered":"2024-03-22T04:10:38.984506+0000","last_unstale":"2024-03-22T04:10:41.169980+0000","last_undegraded":"2024-03-22T04:10:41.169980+0000","last_fullsized":"2024-03-22T04:10:41.169980+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:17:51.066337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535990+0000","last_change":"2024-03-22T04:10:34.940162+0000","last_active":"2024-03-22T04:10:41.535990+0000","last_peered":"2024-03-22T04:10:41.535990+0000","last_clean":"2024-03-22T04:10:41.535990+0000","last_became_active":"2024-03-22T04:10:34.937040+0000","last_became_peered":"2024-03-22T04:10:34.937040+0000","last_unstale":"2024-03-22T04:10:41.535990+0000","last_undegraded":"2024-03-22T04:10:41.535990+0000","last_fullsized":"2024-03-22T04:10:41.535990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.104177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967999+0000","last_change":"2024-03-22T04:10:33.932000+0000","last_active":"2024-03-22T04:10:39.967999+0000","last_peered":"2024-03-22T04:10:39.967999+0000","last_clean":"2024-03-22T04:10:39.967999+0000","last_became_active":"2024-03-22T04:10:33.930159+0000","last_became_peered":"2024-03-22T04:10:33.930159+0000","last_unstale":"2024-03-22T04:10:39.967999+0000","last_undegraded":"2024-03-22T04:10:39.967999+0000","last_fullsized":"2024-03-22T04:10:39.967999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:11:21.741394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921794+0000","last_change":"2024-03-22T04:10:38.979495+0000","last_active":"2024-03-22T04:10:40.921794+0000","last_peered":"2024-03-22T04:10:40.921794+0000","last_clean":"2024-03-22T04:10:40.921794+0000","last_became_active":"2024-03-22T04:10:38.979058+0000","last_became_peered":"2024-03-22T04:10:38.979058+0000","last_unstale":"2024-03-22T04:10:40.921794+0000","last_undegraded":"2024-03-22T04:10:40.921794+0000","last_fullsized":"2024-03-22T04:10:40.921794+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:25.405176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965133+0000","last_change":"2024-03-22T04:10:34.936969+0000","last_active":"2024-03-22T04:10:39.965133+0000","last_peered":"2024-03-22T04:10:39.965133+0000","last_clean":"2024-03-22T04:10:39.965133+0000","last_became_active":"2024-03-22T04:10:34.936270+0000","last_became_peered":"2024-03-22T04:10:34.936270+0000","last_unstale":"2024-03-22T04:10:39.965133+0000","last_undegraded":"2024-03-22T04:10:39.965133+0000","last_fullsized":"2024-03-22T04:10:39.965133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:44.874307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536081+0000","last_change":"2024-03-22T04:10:33.929682+0000","last_active":"2024-03-22T04:10:41.536081+0000","last_peered":"2024-03-22T04:10:41.536081+0000","last_clean":"2024-03-22T04:10:41.536081+0000","last_became_active":"2024-03-22T04:10:33.929309+0000","last_became_peered":"2024-03-22T04:10:33.929309+0000","last_unstale":"2024-03-22T04:10:41.536081+0000","last_undegraded":"2024-03-22T04:10:41.536081+0000","last_fullsized":"2024-03-22T04:10:41.536081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:05.641869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535389+0000","last_change":"2024-03-22T04:10:38.979596+0000","last_active":"2024-03-22T04:10:41.535389+0000","last_peered":"2024-03-22T04:10:41.535389+0000","last_clean":"2024-03-22T04:10:41.535389+0000","last_became_active":"2024-03-22T04:10:38.979166+0000","last_became_peered":"2024-03-22T04:10:38.979166+0000","last_unstale":"2024-03-22T04:10:41.535389+0000","last_undegraded":"2024-03-22T04:10:41.535389+0000","last_fullsized":"2024-03-22T04:10:41.535389+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:08.758533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921851+0000","last_change":"2024-03-22T04:10:34.926802+0000","last_active":"2024-03-22T04:10:40.921851+0000","last_peered":"2024-03-22T04:10:40.921851+0000","last_clean":"2024-03-22T04:10:40.921851+0000","last_became_active":"2024-03-22T04:10:34.926514+0000","last_became_peered":"2024-03-22T04:10:34.926514+0000","last_unstale":"2024-03-22T04:10:40.921851+0000","last_undegraded":"2024-03-22T04:10:40.921851+0000","last_fullsized":"2024-03-22T04:10:40.921851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:02:47.197474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966395+0000","last_change":"2024-03-22T04:10:33.932545+0000","last_active":"2024-03-22T04:10:39.966395+0000","last_peered":"2024-03-22T04:10:39.966395+0000","last_clean":"2024-03-22T04:10:39.966395+0000","last_became_active":"2024-03-22T04:10:33.929875+0000","last_became_peered":"2024-03-22T04:10:33.929875+0000","last_unstale":"2024-03-22T04:10:39.966395+0000","last_undegraded":"2024-03-22T04:10:39.966395+0000","last_fullsized":"2024-03-22T04:10:39.966395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:59:03.501920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170524+0000","last_change":"2024-03-22T04:10:38.985437+0000","last_active":"2024-03-22T04:10:41.170524+0000","last_peered":"2024-03-22T04:10:41.170524+0000","last_clean":"2024-03-22T04:10:41.170524+0000","last_became_active":"2024-03-22T04:10:38.982076+0000","last_became_peered":"2024-03-22T04:10:38.982076+0000","last_unstale":"2024-03-22T04:10:41.170524+0000","last_undegraded":"2024-03-22T04:10:41.170524+0000","last_fullsized":"2024-03-22T04:10:41.170524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:21.687034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535147+0000","last_change":"2024-03-22T04:10:34.928089+0000","last_active":"2024-03-22T04:10:41.535147+0000","last_peered":"2024-03-22T04:10:41.535147+0000","last_clean":"2024-03-22T04:10:41.535147+0000","last_became_active":"2024-03-22T04:10:34.927828+0000","last_became_peered":"2024-03-22T04:10:34.927828+0000","last_unstale":"2024-03-22T04:10:41.535147+0000","last_undegraded":"2024-03-22T04:10:41.535147+0000","last_fullsized":"2024-03-22T04:10:41.535147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534615+0000","last_change":"2024-03-22T04:10:33.943460+0000","last_active":"2024-03-22T04:10:41.534615+0000","last_peered":"2024-03-22T04:10:41.534615+0000","last_clean":"2024-03-22T04:10:41.534615+0000","last_became_active":"2024-03-22T04:10:33.943039+0000","last_became_peered":"2024-03-22T04:10:33.943039+0000","last_unstale":"2024-03-22T04:10:41.534615+0000","last_undegraded":"2024-03-22T04:10:41.534615+0000","last_fullsized":"2024-03-22T04:10:41.534615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:40:51.297536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170801+0000","last_change":"2024-03-22T04:10:38.981354+0000","last_active":"2024-03-22T04:10:41.170801+0000","last_peered":"2024-03-22T04:10:41.170801+0000","last_clean":"2024-03-22T04:10:41.170801+0000","last_became_active":"2024-03-22T04:10:38.979192+0000","last_became_peered":"2024-03-22T04:10:38.979192+0000","last_unstale":"2024-03-22T04:10:41.170801+0000","last_undegraded":"2024-03-22T04:10:41.170801+0000","last_fullsized":"2024-03-22T04:10:41.170801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:05:23.818860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922177+0000","last_change":"2024-03-22T04:10:34.935166+0000","last_active":"2024-03-22T04:10:40.922177+0000","last_peered":"2024-03-22T04:10:40.922177+0000","last_clean":"2024-03-22T04:10:40.922177+0000","last_became_active":"2024-03-22T04:10:34.934859+0000","last_became_peered":"2024-03-22T04:10:34.934859+0000","last_unstale":"2024-03-22T04:10:40.922177+0000","last_undegraded":"2024-03-22T04:10:40.922177+0000","last_fullsized":"2024-03-22T04:10:40.922177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:34:47.128097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922197+0000","last_change":"2024-03-22T04:10:33.939004+0000","last_active":"2024-03-22T04:10:40.922197+0000","last_peered":"2024-03-22T04:10:40.922197+0000","last_clean":"2024-03-22T04:10:40.922197+0000","last_became_active":"2024-03-22T04:10:33.938036+0000","last_became_peered":"2024-03-22T04:10:33.938036+0000","last_unstale":"2024-03-22T04:10:40.922197+0000","last_undegraded":"2024-03-22T04:10:40.922197+0000","last_fullsized":"2024-03-22T04:10:40.922197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:47.209113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536345+0000","last_change":"2024-03-22T04:10:38.985663+0000","last_active":"2024-03-22T04:10:41.536345+0000","last_peered":"2024-03-22T04:10:41.536345+0000","last_clean":"2024-03-22T04:10:41.536345+0000","last_became_active":"2024-03-22T04:10:38.985235+0000","last_became_peered":"2024-03-22T04:10:38.985235+0000","last_unstale":"2024-03-22T04:10:41.536345+0000","last_undegraded":"2024-03-22T04:10:41.536345+0000","last_fullsized":"2024-03-22T04:10:41.536345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:45.922825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536323+0000","last_change":"2024-03-22T04:10:34.950266+0000","last_active":"2024-03-22T04:10:41.536323+0000","last_peered":"2024-03-22T04:10:41.536323+0000","last_clean":"2024-03-22T04:10:41.536323+0000","last_became_active":"2024-03-22T04:10:34.949946+0000","last_became_peered":"2024-03-22T04:10:34.949946+0000","last_unstale":"2024-03-22T04:10:41.536323+0000","last_undegraded":"2024-03-22T04:10:41.536323+0000","last_fullsized":"2024-03-22T04:10:41.536323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:04.713004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967305+0000","last_change":"2024-03-22T04:10:33.935184+0000","last_active":"2024-03-22T04:10:39.967305+0000","last_peered":"2024-03-22T04:10:39.967305+0000","last_clean":"2024-03-22T04:10:39.967305+0000","last_became_active":"2024-03-22T04:10:33.934904+0000","last_became_peered":"2024-03-22T04:10:33.934904+0000","last_unstale":"2024-03-22T04:10:39.967305+0000","last_undegraded":"2024-03-22T04:10:39.967305+0000","last_fullsized":"2024-03-22T04:10:39.967305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922445+0000","last_change":"2024-03-22T04:10:38.971479+0000","last_active":"2024-03-22T04:10:40.922445+0000","last_peered":"2024-03-22T04:10:40.922445+0000","last_clean":"2024-03-22T04:10:40.922445+0000","last_became_active":"2024-03-22T04:10:38.969629+0000","last_became_peered":"2024-03-22T04:10:38.969629+0000","last_unstale":"2024-03-22T04:10:40.922445+0000","last_undegraded":"2024-03-22T04:10:40.922445+0000","last_fullsized":"2024-03-22T04:10:40.922445+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:28:08.179423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535818+0000","last_change":"2024-03-22T04:10:34.932181+0000","last_active":"2024-03-22T04:10:41.535818+0000","last_peered":"2024-03-22T04:10:41.535818+0000","last_clean":"2024-03-22T04:10:41.535818+0000","last_became_active":"2024-03-22T04:10:34.931936+0000","last_became_peered":"2024-03-22T04:10:34.931936+0000","last_unstale":"2024-03-22T04:10:41.535818+0000","last_undegraded":"2024-03-22T04:10:41.535818+0000","last_fullsized":"2024-03-22T04:10:41.535818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:26.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:50.358530+0000","last_change":"2024-03-22T04:10:33.934598+0000","last_active":"2024-03-22T04:10:50.358530+0000","last_peered":"2024-03-22T04:10:50.358530+0000","last_clean":"2024-03-22T04:10:50.358530+0000","last_became_active":"2024-03-22T04:10:33.934129+0000","last_became_peered":"2024-03-22T04:10:33.934129+0000","last_unstale":"2024-03-22T04:10:50.358530+0000","last_undegraded":"2024-03-22T04:10:50.358530+0000","last_fullsized":"2024-03-22T04:10:50.358530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:25:11.544206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966493+0000","last_change":"2024-03-22T04:10:38.980975+0000","last_active":"2024-03-22T04:10:39.966493+0000","last_peered":"2024-03-22T04:10:39.966493+0000","last_clean":"2024-03-22T04:10:39.966493+0000","last_became_active":"2024-03-22T04:10:38.980594+0000","last_became_peered":"2024-03-22T04:10:38.980594+0000","last_unstale":"2024-03-22T04:10:39.966493+0000","last_undegraded":"2024-03-22T04:10:39.966493+0000","last_fullsized":"2024-03-22T04:10:39.966493+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:10:22.917144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922350+0000","last_change":"2024-03-22T04:10:34.939331+0000","last_active":"2024-03-22T04:10:40.922350+0000","last_peered":"2024-03-22T04:10:40.922350+0000","last_clean":"2024-03-22T04:10:40.922350+0000","last_became_active":"2024-03-22T04:10:34.938998+0000","last_became_peered":"2024-03-22T04:10:34.938998+0000","last_unstale":"2024-03-22T04:10:40.922350+0000","last_undegraded":"2024-03-22T04:10:40.922350+0000","last_fullsized":"2024-03-22T04:10:40.922350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.104177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967555+0000","last_change":"2024-03-22T04:10:33.938816+0000","last_active":"2024-03-22T04:10:39.967555+0000","last_peered":"2024-03-22T04:10:39.967555+0000","last_clean":"2024-03-22T04:10:39.967555+0000","last_became_active":"2024-03-22T04:10:33.937603+0000","last_became_peered":"2024-03-22T04:10:33.937603+0000","last_unstale":"2024-03-22T04:10:39.967555+0000","last_undegraded":"2024-03-22T04:10:39.967555+0000","last_fullsized":"2024-03-22T04:10:39.967555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964868+0000","last_change":"2024-03-22T04:10:38.987284+0000","last_active":"2024-03-22T04:10:39.964868+0000","last_peered":"2024-03-22T04:10:39.964868+0000","last_clean":"2024-03-22T04:10:39.964868+0000","last_became_active":"2024-03-22T04:10:38.986819+0000","last_became_peered":"2024-03-22T04:10:38.986819+0000","last_unstale":"2024-03-22T04:10:39.964868+0000","last_undegraded":"2024-03-22T04:10:39.964868+0000","last_fullsized":"2024-03-22T04:10:39.964868+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:20.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170831+0000","last_change":"2024-03-22T04:10:34.933050+0000","last_active":"2024-03-22T04:10:41.170831+0000","last_peered":"2024-03-22T04:10:41.170831+0000","last_clean":"2024-03-22T04:10:41.170831+0000","last_became_active":"2024-03-22T04:10:34.932756+0000","last_became_peered":"2024-03-22T04:10:34.932756+0000","last_unstale":"2024-03-22T04:10:41.170831+0000","last_undegraded":"2024-03-22T04:10:41.170831+0000","last_fullsized":"2024-03-22T04:10:41.170831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:41.877296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536138+0000","last_change":"2024-03-22T04:10:33.936205+0000","last_active":"2024-03-22T04:10:41.536138+0000","last_peered":"2024-03-22T04:10:41.536138+0000","last_clean":"2024-03-22T04:10:41.536138+0000","last_became_active":"2024-03-22T04:10:33.935904+0000","last_became_peered":"2024-03-22T04:10:33.935904+0000","last_unstale":"2024-03-22T04:10:41.536138+0000","last_undegraded":"2024-03-22T04:10:41.536138+0000","last_fullsized":"2024-03-22T04:10:41.536138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:30.304547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966245+0000","last_change":"2024-03-22T04:10:38.991949+0000","last_active":"2024-03-22T04:10:39.966245+0000","last_peered":"2024-03-22T04:10:39.966245+0000","last_clean":"2024-03-22T04:10:39.966245+0000","last_became_active":"2024-03-22T04:10:38.991459+0000","last_became_peered":"2024-03-22T04:10:38.991459+0000","last_unstale":"2024-03-22T04:10:39.966245+0000","last_undegraded":"2024-03-22T04:10:39.966245+0000","last_fullsized":"2024-03-22T04:10:39.966245+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:44.301347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965035+0000","last_change":"2024-03-22T04:10:34.947409+0000","last_active":"2024-03-22T04:10:39.965035+0000","last_peered":"2024-03-22T04:10:39.965035+0000","last_clean":"2024-03-22T04:10:39.965035+0000","last_became_active":"2024-03-22T04:10:34.947102+0000","last_became_peered":"2024-03-22T04:10:34.947102+0000","last_unstale":"2024-03-22T04:10:39.965035+0000","last_undegraded":"2024-03-22T04:10:39.965035+0000","last_fullsized":"2024-03-22T04:10:39.965035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:12.376047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535663+0000","last_change":"2024-03-22T04:10:33.922034+0000","last_active":"2024-03-22T04:10:41.535663+0000","last_peered":"2024-03-22T04:10:41.535663+0000","last_clean":"2024-03-22T04:10:41.535663+0000","last_became_active":"2024-03-22T04:10:33.921713+0000","last_became_peered":"2024-03-22T04:10:33.921713+0000","last_unstale":"2024-03-22T04:10:41.535663+0000","last_undegraded":"2024-03-22T04:10:41.535663+0000","last_fullsized":"2024-03-22T04:10:41.535663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966606+0000","last_change":"2024-03-22T04:10:38.983919+0000","last_active":"2024-03-22T04:10:39.966606+0000","last_peered":"2024-03-22T04:10:39.966606+0000","last_clean":"2024-03-22T04:10:39.966606+0000","last_became_active":"2024-03-22T04:10:38.983556+0000","last_became_peered":"2024-03-22T04:10:38.983556+0000","last_unstale":"2024-03-22T04:10:39.966606+0000","last_undegraded":"2024-03-22T04:10:39.966606+0000","last_fullsized":"2024-03-22T04:10:39.966606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:12:09.818595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967886+0000","last_change":"2024-03-22T04:10:38.985686+0000","last_active":"2024-03-22T04:10:39.967886+0000","last_peered":"2024-03-22T04:10:39.967886+0000","last_clean":"2024-03-22T04:10:39.967886+0000","last_became_active":"2024-03-22T04:10:38.985411+0000","last_became_peered":"2024-03-22T04:10:38.985411+0000","last_unstale":"2024-03-22T04:10:39.967886+0000","last_undegraded":"2024-03-22T04:10:39.967886+0000","last_fullsized":"2024-03-22T04:10:39.967886+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:36.068872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966752+0000","last_change":"2024-03-22T04:10:34.926900+0000","last_active":"2024-03-22T04:10:39.966752+0000","last_peered":"2024-03-22T04:10:39.966752+0000","last_clean":"2024-03-22T04:10:39.966752+0000","last_became_active":"2024-03-22T04:10:34.926599+0000","last_became_peered":"2024-03-22T04:10:34.926599+0000","last_unstale":"2024-03-22T04:10:39.966752+0000","last_undegraded":"2024-03-22T04:10:39.966752+0000","last_fullsized":"2024-03-22T04:10:39.966752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:21.635048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922002+0000","last_change":"2024-03-22T04:10:33.932457+0000","last_active":"2024-03-22T04:10:40.922002+0000","last_peered":"2024-03-22T04:10:40.922002+0000","last_clean":"2024-03-22T04:10:40.922002+0000","last_became_active":"2024-03-22T04:10:33.932048+0000","last_became_peered":"2024-03-22T04:10:33.932048+0000","last_unstale":"2024-03-22T04:10:40.922002+0000","last_undegraded":"2024-03-22T04:10:40.922002+0000","last_fullsized":"2024-03-22T04:10:40.922002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:30.304547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922792+0000","last_change":"2024-03-22T04:10:38.985629+0000","last_active":"2024-03-22T04:10:40.922792+0000","last_peered":"2024-03-22T04:10:40.922792+0000","last_clean":"2024-03-22T04:10:40.922792+0000","last_became_active":"2024-03-22T04:10:38.983094+0000","last_became_peered":"2024-03-22T04:10:38.983094+0000","last_unstale":"2024-03-22T04:10:40.922792+0000","last_undegraded":"2024-03-22T04:10:40.922792+0000","last_fullsized":"2024-03-22T04:10:40.922792+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:21:46.389313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169810+0000","last_change":"2024-03-22T04:10:34.925982+0000","last_active":"2024-03-22T04:10:41.169810+0000","last_peered":"2024-03-22T04:10:41.169810+0000","last_clean":"2024-03-22T04:10:41.169810+0000","last_became_active":"2024-03-22T04:10:34.925676+0000","last_became_peered":"2024-03-22T04:10:34.925676+0000","last_unstale":"2024-03-22T04:10:41.169810+0000","last_undegraded":"2024-03-22T04:10:41.169810+0000","last_fullsized":"2024-03-22T04:10:41.169810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:08.063872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536054+0000","last_change":"2024-03-22T04:10:33.933448+0000","last_active":"2024-03-22T04:10:41.536054+0000","last_peered":"2024-03-22T04:10:41.536054+0000","last_clean":"2024-03-22T04:10:41.536054+0000","last_became_active":"2024-03-22T04:10:33.933025+0000","last_became_peered":"2024-03-22T04:10:33.933025+0000","last_unstale":"2024-03-22T04:10:41.536054+0000","last_undegraded":"2024-03-22T04:10:41.536054+0000","last_fullsized":"2024-03-22T04:10:41.536054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922138+0000","last_change":"2024-03-22T04:10:33.935607+0000","last_active":"2024-03-22T04:10:40.922138+0000","last_peered":"2024-03-22T04:10:40.922138+0000","last_clean":"2024-03-22T04:10:40.922138+0000","last_became_active":"2024-03-22T04:10:33.932086+0000","last_became_peered":"2024-03-22T04:10:33.932086+0000","last_unstale":"2024-03-22T04:10:40.922138+0000","last_undegraded":"2024-03-22T04:10:40.922138+0000","last_fullsized":"2024-03-22T04:10:40.922138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:49.224368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967401+0000","last_change":"2024-03-22T04:10:34.934494+0000","last_active":"2024-03-22T04:10:39.967401+0000","last_peered":"2024-03-22T04:10:39.967401+0000","last_clean":"2024-03-22T04:10:39.967401+0000","last_became_active":"2024-03-22T04:10:34.934210+0000","last_became_peered":"2024-03-22T04:10:34.934210+0000","last_unstale":"2024-03-22T04:10:39.967401+0000","last_undegraded":"2024-03-22T04:10:39.967401+0000","last_fullsized":"2024-03-22T04:10:39.967401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:48:09.157711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967687+0000","last_change":"2024-03-22T04:10:38.988089+0000","last_active":"2024-03-22T04:10:39.967687+0000","last_peered":"2024-03-22T04:10:39.967687+0000","last_clean":"2024-03-22T04:10:39.967687+0000","last_became_active":"2024-03-22T04:10:38.987643+0000","last_became_peered":"2024-03-22T04:10:38.987643+0000","last_unstale":"2024-03-22T04:10:39.967687+0000","last_undegraded":"2024-03-22T04:10:39.967687+0000","last_fullsized":"2024-03-22T04:10:39.967687+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:35.902267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170109+0000","last_change":"2024-03-22T04:10:33.933057+0000","last_active":"2024-03-22T04:10:41.170109+0000","last_peered":"2024-03-22T04:10:41.170109+0000","last_clean":"2024-03-22T04:10:41.170109+0000","last_became_active":"2024-03-22T04:10:33.932776+0000","last_became_peered":"2024-03-22T04:10:33.932776+0000","last_unstale":"2024-03-22T04:10:41.170109+0000","last_undegraded":"2024-03-22T04:10:41.170109+0000","last_fullsized":"2024-03-22T04:10:41.170109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535263+0000","last_change":"2024-03-22T04:10:34.935154+0000","last_active":"2024-03-22T04:10:41.535263+0000","last_peered":"2024-03-22T04:10:41.535263+0000","last_clean":"2024-03-22T04:10:41.535263+0000","last_became_active":"2024-03-22T04:10:34.934888+0000","last_became_peered":"2024-03-22T04:10:34.934888+0000","last_unstale":"2024-03-22T04:10:41.535263+0000","last_undegraded":"2024-03-22T04:10:41.535263+0000","last_fullsized":"2024-03-22T04:10:41.535263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:01.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967502+0000","last_change":"2024-03-22T04:10:38.988529+0000","last_active":"2024-03-22T04:10:39.967502+0000","last_peered":"2024-03-22T04:10:39.967502+0000","last_clean":"2024-03-22T04:10:39.967502+0000","last_became_active":"2024-03-22T04:10:38.987847+0000","last_became_peered":"2024-03-22T04:10:38.987847+0000","last_unstale":"2024-03-22T04:10:39.967502+0000","last_undegraded":"2024-03-22T04:10:39.967502+0000","last_fullsized":"2024-03-22T04:10:39.967502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:12:05.623030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964372+0000","last_change":"2024-03-22T04:10:34.930193+0000","last_active":"2024-03-22T04:10:39.964372+0000","last_peered":"2024-03-22T04:10:39.964372+0000","last_clean":"2024-03-22T04:10:39.964372+0000","last_became_active":"2024-03-22T04:10:34.929910+0000","last_became_peered":"2024-03-22T04:10:34.929910+0000","last_unstale":"2024-03-22T04:10:39.964372+0000","last_undegraded":"2024-03-22T04:10:39.964372+0000","last_fullsized":"2024-03-22T04:10:39.964372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966601+0000","last_change":"2024-03-22T04:10:34.951954+0000","last_active":"2024-03-22T04:10:39.966601+0000","last_peered":"2024-03-22T04:10:39.966601+0000","last_clean":"2024-03-22T04:10:39.966601+0000","last_became_active":"2024-03-22T04:10:34.951651+0000","last_became_peered":"2024-03-22T04:10:34.951651+0000","last_unstale":"2024-03-22T04:10:39.966601+0000","last_undegraded":"2024-03-22T04:10:39.966601+0000","last_fullsized":"2024-03-22T04:10:39.966601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922008+0000","last_change":"2024-03-22T04:10:34.927015+0000","last_active":"2024-03-22T04:10:40.922008+0000","last_peered":"2024-03-22T04:10:40.922008+0000","last_clean":"2024-03-22T04:10:40.922008+0000","last_became_active":"2024-03-22T04:10:34.926098+0000","last_became_peered":"2024-03-22T04:10:34.926098+0000","last_unstale":"2024-03-22T04:10:40.922008+0000","last_undegraded":"2024-03-22T04:10:40.922008+0000","last_fullsized":"2024-03-22T04:10:40.922008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:05:19.773331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170027+0000","last_change":"2024-03-22T04:10:34.950558+0000","last_active":"2024-03-22T04:10:41.170027+0000","last_peered":"2024-03-22T04:10:41.170027+0000","last_clean":"2024-03-22T04:10:41.170027+0000","last_became_active":"2024-03-22T04:10:34.950242+0000","last_became_peered":"2024-03-22T04:10:34.950242+0000","last_unstale":"2024-03-22T04:10:41.170027+0000","last_undegraded":"2024-03-22T04:10:41.170027+0000","last_fullsized":"2024-03-22T04:10:41.170027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:11.105591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536489+0000","last_change":"2024-03-22T04:10:34.928518+0000","last_active":"2024-03-22T04:10:41.536489+0000","last_peered":"2024-03-22T04:10:41.536489+0000","last_clean":"2024-03-22T04:10:41.536489+0000","last_became_active":"2024-03-22T04:10:34.928228+0000","last_became_peered":"2024-03-22T04:10:34.928228+0000","last_unstale":"2024-03-22T04:10:41.536489+0000","last_undegraded":"2024-03-22T04:10:41.536489+0000","last_fullsized":"2024-03-22T04:10:41.536489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:41.449121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536201+0000","last_change":"2024-03-22T04:10:34.951914+0000","last_active":"2024-03-22T04:10:41.536201+0000","last_peered":"2024-03-22T04:10:41.536201+0000","last_clean":"2024-03-22T04:10:41.536201+0000","last_became_active":"2024-03-22T04:10:34.951608+0000","last_became_peered":"2024-03-22T04:10:34.951608+0000","last_unstale":"2024-03-22T04:10:41.536201+0000","last_undegraded":"2024-03-22T04:10:41.536201+0000","last_fullsized":"2024-03-22T04:10:41.536201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:18.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964949+0000","last_change":"2024-03-22T04:10:34.936728+0000","last_active":"2024-03-22T04:10:39.964949+0000","last_peered":"2024-03-22T04:10:39.964949+0000","last_clean":"2024-03-22T04:10:39.964949+0000","last_became_active":"2024-03-22T04:10:34.935969+0000","last_became_peered":"2024-03-22T04:10:34.935969+0000","last_unstale":"2024-03-22T04:10:39.964949+0000","last_undegraded":"2024-03-22T04:10:39.964949+0000","last_fullsized":"2024-03-22T04:10:39.964949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:48.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170702+0000","last_change":"2024-03-22T04:10:34.943850+0000","last_active":"2024-03-22T04:10:41.170702+0000","last_peered":"2024-03-22T04:10:41.170702+0000","last_clean":"2024-03-22T04:10:41.170702+0000","last_became_active":"2024-03-22T04:10:34.941930+0000","last_became_peered":"2024-03-22T04:10:34.941930+0000","last_unstale":"2024-03-22T04:10:41.170702+0000","last_undegraded":"2024-03-22T04:10:41.170702+0000","last_fullsized":"2024-03-22T04:10:41.170702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:55.025923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921894+0000","last_change":"2024-03-22T04:10:34.946337+0000","last_active":"2024-03-22T04:10:40.921894+0000","last_peered":"2024-03-22T04:10:40.921894+0000","last_clean":"2024-03-22T04:10:40.921894+0000","last_became_active":"2024-03-22T04:10:34.946083+0000","last_became_peered":"2024-03-22T04:10:34.946083+0000","last_unstale":"2024-03-22T04:10:40.921894+0000","last_undegraded":"2024-03-22T04:10:40.921894+0000","last_fullsized":"2024-03-22T04:10:40.921894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:15:33.976508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170005+0000","last_change":"2024-03-22T04:10:34.939061+0000","last_active":"2024-03-22T04:10:41.170005+0000","last_peered":"2024-03-22T04:10:41.170005+0000","last_clean":"2024-03-22T04:10:41.170005+0000","last_became_active":"2024-03-22T04:10:34.938566+0000","last_became_peered":"2024-03-22T04:10:34.938566+0000","last_unstale":"2024-03-22T04:10:41.170005+0000","last_undegraded":"2024-03-22T04:10:41.170005+0000","last_fullsized":"2024-03-22T04:10:41.170005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:44:57.681730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535168+0000","last_change":"2024-03-22T04:10:34.933002+0000","last_active":"2024-03-22T04:10:41.535168+0000","last_peered":"2024-03-22T04:10:41.535168+0000","last_clean":"2024-03-22T04:10:41.535168+0000","last_became_active":"2024-03-22T04:10:34.932710+0000","last_became_peered":"2024-03-22T04:10:34.932710+0000","last_unstale":"2024-03-22T04:10:41.535168+0000","last_undegraded":"2024-03-22T04:10:41.535168+0000","last_fullsized":"2024-03-22T04:10:41.535168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967328+0000","last_change":"2024-03-22T04:10:34.926419+0000","last_active":"2024-03-22T04:10:39.967328+0000","last_peered":"2024-03-22T04:10:39.967328+0000","last_clean":"2024-03-22T04:10:39.967328+0000","last_became_active":"2024-03-22T04:10:34.924952+0000","last_became_peered":"2024-03-22T04:10:34.924952+0000","last_unstale":"2024-03-22T04:10:39.967328+0000","last_undegraded":"2024-03-22T04:10:39.967328+0000","last_fullsized":"2024-03-22T04:10:39.967328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:30:41.171941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535087+0000","last_change":"2024-03-22T04:10:34.939886+0000","last_active":"2024-03-22T04:10:41.535087+0000","last_peered":"2024-03-22T04:10:41.535087+0000","last_clean":"2024-03-22T04:10:41.535087+0000","last_became_active":"2024-03-22T04:10:34.939633+0000","last_became_peered":"2024-03-22T04:10:34.939633+0000","last_unstale":"2024-03-22T04:10:41.535087+0000","last_undegraded":"2024-03-22T04:10:41.535087+0000","last_fullsized":"2024-03-22T04:10:41.535087+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:41.877296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922616+0000","last_change":"2024-03-22T04:10:34.930917+0000","last_active":"2024-03-22T04:10:40.922616+0000","last_peered":"2024-03-22T04:10:40.922616+0000","last_clean":"2024-03-22T04:10:40.922616+0000","last_became_active":"2024-03-22T04:10:34.930658+0000","last_became_peered":"2024-03-22T04:10:34.930658+0000","last_unstale":"2024-03-22T04:10:40.922616+0000","last_undegraded":"2024-03-22T04:10:40.922616+0000","last_fullsized":"2024-03-22T04:10:40.922616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:40:34.158803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967631+0000","last_change":"2024-03-22T04:10:34.953215+0000","last_active":"2024-03-22T04:10:39.967631+0000","last_peered":"2024-03-22T04:10:39.967631+0000","last_clean":"2024-03-22T04:10:39.967631+0000","last_became_active":"2024-03-22T04:10:34.952953+0000","last_became_peered":"2024-03-22T04:10:34.952953+0000","last_unstale":"2024-03-22T04:10:39.967631+0000","last_undegraded":"2024-03-22T04:10:39.967631+0000","last_fullsized":"2024-03-22T04:10:39.967631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922902+0000","last_change":"2024-03-22T04:10:34.937971+0000","last_active":"2024-03-22T04:10:40.922902+0000","last_peered":"2024-03-22T04:10:40.922902+0000","last_clean":"2024-03-22T04:10:40.922902+0000","last_became_active":"2024-03-22T04:10:34.937744+0000","last_became_peered":"2024-03-22T04:10:34.937744+0000","last_unstale":"2024-03-22T04:10:40.922902+0000","last_undegraded":"2024-03-22T04:10:40.922902+0000","last_fullsized":"2024-03-22T04:10:40.922902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:17.641505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921877+0000","last_change":"2024-03-22T04:10:34.929076+0000","last_active":"2024-03-22T04:10:40.921877+0000","last_peered":"2024-03-22T04:10:40.921877+0000","last_clean":"2024-03-22T04:10:40.921877+0000","last_became_active":"2024-03-22T04:10:34.928742+0000","last_became_peered":"2024-03-22T04:10:34.928742+0000","last_unstale":"2024-03-22T04:10:40.921877+0000","last_undegraded":"2024-03-22T04:10:40.921877+0000","last_fullsized":"2024-03-22T04:10:40.921877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:04.974724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169693+0000","last_change":"2024-03-22T04:10:34.935025+0000","last_active":"2024-03-22T04:10:41.169693+0000","last_peered":"2024-03-22T04:10:41.169693+0000","last_clean":"2024-03-22T04:10:41.169693+0000","last_became_active":"2024-03-22T04:10:34.934504+0000","last_became_peered":"2024-03-22T04:10:34.934504+0000","last_unstale":"2024-03-22T04:10:41.169693+0000","last_undegraded":"2024-03-22T04:10:41.169693+0000","last_fullsized":"2024-03-22T04:10:41.169693+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:18.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536006+0000","last_change":"2024-03-22T04:10:34.952611+0000","last_active":"2024-03-22T04:10:41.536006+0000","last_peered":"2024-03-22T04:10:41.536006+0000","last_clean":"2024-03-22T04:10:41.536006+0000","last_became_active":"2024-03-22T04:10:34.952289+0000","last_became_peered":"2024-03-22T04:10:34.952289+0000","last_unstale":"2024-03-22T04:10:41.536006+0000","last_undegraded":"2024-03-22T04:10:41.536006+0000","last_fullsized":"2024-03-22T04:10:41.536006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:44.874307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536345+0000","last_change":"2024-03-22T04:10:34.938439+0000","last_active":"2024-03-22T04:10:41.536345+0000","last_peered":"2024-03-22T04:10:41.536345+0000","last_clean":"2024-03-22T04:10:41.536345+0000","last_became_active":"2024-03-22T04:10:34.938160+0000","last_became_peered":"2024-03-22T04:10:34.938160+0000","last_unstale":"2024-03-22T04:10:41.536345+0000","last_undegraded":"2024-03-22T04:10:41.536345+0000","last_fullsized":"2024-03-22T04:10:41.536345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:40.255817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535047+0000","last_change":"2024-03-22T04:10:34.936432+0000","last_active":"2024-03-22T04:10:41.535047+0000","last_peered":"2024-03-22T04:10:41.535047+0000","last_clean":"2024-03-22T04:10:41.535047+0000","last_became_active":"2024-03-22T04:10:34.936175+0000","last_became_peered":"2024-03-22T04:10:34.936175+0000","last_unstale":"2024-03-22T04:10:41.535047+0000","last_undegraded":"2024-03-22T04:10:41.535047+0000","last_fullsized":"2024-03-22T04:10:41.535047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:34.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535725+0000","last_change":"2024-03-22T04:10:34.947651+0000","last_active":"2024-03-22T04:10:41.535725+0000","last_peered":"2024-03-22T04:10:41.535725+0000","last_clean":"2024-03-22T04:10:41.535725+0000","last_became_active":"2024-03-22T04:10:34.947262+0000","last_became_peered":"2024-03-22T04:10:34.947262+0000","last_unstale":"2024-03-22T04:10:41.535725+0000","last_undegraded":"2024-03-22T04:10:41.535725+0000","last_fullsized":"2024-03-22T04:10:41.535725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:16.846964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536037+0000","last_change":"2024-03-22T04:10:34.929968+0000","last_active":"2024-03-22T04:10:41.536037+0000","last_peered":"2024-03-22T04:10:41.536037+0000","last_clean":"2024-03-22T04:10:41.536037+0000","last_became_active":"2024-03-22T04:10:34.929650+0000","last_became_peered":"2024-03-22T04:10:34.929650+0000","last_unstale":"2024-03-22T04:10:41.536037+0000","last_undegraded":"2024-03-22T04:10:41.536037+0000","last_fullsized":"2024-03-22T04:10:41.536037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:04.974724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535701+0000","last_change":"2024-03-22T04:10:34.949078+0000","last_active":"2024-03-22T04:10:41.535701+0000","last_peered":"2024-03-22T04:10:41.535701+0000","last_clean":"2024-03-22T04:10:41.535701+0000","last_became_active":"2024-03-22T04:10:34.948775+0000","last_became_peered":"2024-03-22T04:10:34.948775+0000","last_unstale":"2024-03-22T04:10:41.535701+0000","last_undegraded":"2024-03-22T04:10:41.535701+0000","last_fullsized":"2024-03-22T04:10:41.535701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:48.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169713+0000","last_change":"2024-03-22T04:10:34.931467+0000","last_active":"2024-03-22T04:10:41.169713+0000","last_peered":"2024-03-22T04:10:41.169713+0000","last_clean":"2024-03-22T04:10:41.169713+0000","last_became_active":"2024-03-22T04:10:34.931119+0000","last_became_peered":"2024-03-22T04:10:34.931119+0000","last_unstale":"2024-03-22T04:10:41.169713+0000","last_undegraded":"2024-03-22T04:10:41.169713+0000","last_fullsized":"2024-03-22T04:10:41.169713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:18:49.597662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534582+0000","last_change":"2024-03-22T04:10:34.925425+0000","last_active":"2024-03-22T04:10:41.534582+0000","last_peered":"2024-03-22T04:10:41.534582+0000","last_clean":"2024-03-22T04:10:41.534582+0000","last_became_active":"2024-03-22T04:10:34.925151+0000","last_became_peered":"2024-03-22T04:10:34.925151+0000","last_unstale":"2024-03-22T04:10:41.534582+0000","last_undegraded":"2024-03-22T04:10:41.534582+0000","last_fullsized":"2024-03-22T04:10:41.534582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:11.105591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964686+0000","last_change":"2024-03-22T04:10:34.926711+0000","last_active":"2024-03-22T04:10:39.964686+0000","last_peered":"2024-03-22T04:10:39.964686+0000","last_clean":"2024-03-22T04:10:39.964686+0000","last_became_active":"2024-03-22T04:10:34.925765+0000","last_became_peered":"2024-03-22T04:10:34.925765+0000","last_unstale":"2024-03-22T04:10:39.964686+0000","last_undegraded":"2024-03-22T04:10:39.964686+0000","last_fullsized":"2024-03-22T04:10:39.964686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534972+0000","last_change":"2024-03-22T04:10:34.932009+0000","last_active":"2024-03-22T04:10:41.534972+0000","last_peered":"2024-03-22T04:10:41.534972+0000","last_clean":"2024-03-22T04:10:41.534972+0000","last_became_active":"2024-03-22T04:10:34.931692+0000","last_became_peered":"2024-03-22T04:10:34.931692+0000","last_unstale":"2024-03-22T04:10:41.534972+0000","last_undegraded":"2024-03-22T04:10:41.534972+0000","last_fullsized":"2024-03-22T04:10:41.534972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:08.063872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964556+0000","last_change":"2024-03-22T04:10:34.921659+0000","last_active":"2024-03-22T04:10:39.964556+0000","last_peered":"2024-03-22T04:10:39.964556+0000","last_clean":"2024-03-22T04:10:39.964556+0000","last_became_active":"2024-03-22T04:10:34.921420+0000","last_became_peered":"2024-03-22T04:10:34.921420+0000","last_unstale":"2024-03-22T04:10:39.964556+0000","last_undegraded":"2024-03-22T04:10:39.964556+0000","last_fullsized":"2024-03-22T04:10:39.964556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:01.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170351+0000","last_change":"2024-03-22T04:10:34.928037+0000","last_active":"2024-03-22T04:10:41.170351+0000","last_peered":"2024-03-22T04:10:41.170351+0000","last_clean":"2024-03-22T04:10:41.170351+0000","last_became_active":"2024-03-22T04:10:34.927746+0000","last_became_peered":"2024-03-22T04:10:34.927746+0000","last_unstale":"2024-03-22T04:10:41.170351+0000","last_undegraded":"2024-03-22T04:10:41.170351+0000","last_fullsized":"2024-03-22T04:10:41.170351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:29:34.250954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170416+0000","last_change":"2024-03-22T04:10:34.931962+0000","last_active":"2024-03-22T04:10:41.170416+0000","last_peered":"2024-03-22T04:10:41.170416+0000","last_clean":"2024-03-22T04:10:41.170416+0000","last_became_active":"2024-03-22T04:10:34.931736+0000","last_became_peered":"2024-03-22T04:10:34.931736+0000","last_unstale":"2024-03-22T04:10:41.170416+0000","last_undegraded":"2024-03-22T04:10:41.170416+0000","last_fullsized":"2024-03-22T04:10:41.170416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:16.846964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535903+0000","last_change":"2024-03-22T04:10:34.946756+0000","last_active":"2024-03-22T04:10:41.535903+0000","last_peered":"2024-03-22T04:10:41.535903+0000","last_clean":"2024-03-22T04:10:41.535903+0000","last_became_active":"2024-03-22T04:10:34.946501+0000","last_became_peered":"2024-03-22T04:10:34.946501+0000","last_unstale":"2024-03-22T04:10:41.535903+0000","last_undegraded":"2024-03-22T04:10:41.535903+0000","last_fullsized":"2024-03-22T04:10:41.535903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:02:47.197474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:10:59.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:11:00.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.136+0000 7f65bfe00640 1 Processor -- start 2024-03-22T04:11:00.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.136+0000 7f65bfe00640 1 -- start start 2024-03-22T04:11:00.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.136+0000 7f65bfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65b80d1c50 0x7f65b8071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.136+0000 7f65bfe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f65b80cfe70 0x7f65b80d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.136+0000 7f65bfe00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b80d0ca0 con 0x7f65b815c4a0 2024-03-22T04:11:00.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.136+0000 7f65bfe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f65b815c880 con 0x7f65b80d1c50 2024-03-22T04:11:00.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.136+0000 7f65bfe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f65b8072070 con 0x7f65b80cfe70 2024-03-22T04:11:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65b80d1c50 0x7f65b8071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65b80d1c50 0x7f65b8071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55884/0 (socket says 172.21.15.86:55884) 2024-03-22T04:11:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bca00640 1 -- 172.21.15.86:0/221799039 learned_addr learned my addr 172.21.15.86:0/221799039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bde00640 1 --2- 172.21.15.86:0/221799039 >> v2:172.21.15.196:3300/0 conn(0x7f65b80cfe70 0x7f65b80d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bca00640 1 -- 172.21.15.86:0/221799039 >> v2:172.21.15.196:3300/0 conn(0x7f65b80cfe70 msgr2=0x7f65b80d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bca00640 1 --2- 172.21.15.86:0/221799039 >> v2:172.21.15.196:3300/0 conn(0x7f65b80cfe70 0x7f65b80d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bca00640 1 -- 172.21.15.86:0/221799039 >> v1:172.21.15.196:6789/0 conn(0x7f65b815c4a0 legacy=0x7f65b80b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bca00640 1 -- 172.21.15.86:0/221799039 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b80ce5d0 con 0x7f65b80d1c50 2024-03-22T04:11:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65bca00640 1 --2- 172.21.15.86:0/221799039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b8071b30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f65b8073830 tx=0x7f65a401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e9fd120025acfb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65b7e00640 1 -- 172.21.15.86:0/221799039 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40ab430 con 0x7f65b80d1c50 2024-03-22T04:11:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65b7e00640 1 -- 172.21.15.86:0/221799039 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a40b0070 con 0x7f65b80d1c50 2024-03-22T04:11:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.140+0000 7f65b7e00640 1 -- 172.21.15.86:0/221799039 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40ab7f0 con 0x7f65b80d1c50 2024-03-22T04:11:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- 172.21.15.86:0/221799039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 msgr2=0x7f65b8071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 --2- 172.21.15.86:0/221799039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b8071b30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f65b8073830 tx=0x7f65a401c520 comp rx=0 tx=0).stop 2024-03-22T04:11:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- 172.21.15.86:0/221799039 shutdown_connections 2024-03-22T04:11:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 --2- 172.21.15.86:0/221799039 >> v2:172.21.15.196:3300/0 conn(0x7f65b80cfe70 0x7f65b80d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 --2- 172.21.15.86:0/221799039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b8071b30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- 172.21.15.86:0/221799039 >> 172.21.15.86:0/221799039 conn(0x7f65b806f930 msgr2=0x7f65b806fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- 172.21.15.86:0/221799039 shutdown_connections 2024-03-22T04:11:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- 172.21.15.86:0/221799039 wait complete. 2024-03-22T04:11:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 Processor -- start 2024-03-22T04:11:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- start start 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b80cfe70 0x7f65b80c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b80c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65b815c4a0 0x7f65b815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80b5880 con 0x7f65b80d1c50 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80ce900 con 0x7f65b80cfe70 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bfe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f65b80cf580 con 0x7f65b815c4a0 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b80c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b80c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55886/0 (socket says 172.21.15.86:55886) 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 -- 172.21.15.86:0/3372705577 learned_addr learned my addr 172.21.15.86:0/3372705577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bd400640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b80cfe70 0x7f65b80c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bde00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65b815c4a0 0x7f65b815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 -- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65b815c4a0 msgr2=0x7f65b815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65b815c4a0 0x7f65b815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 -- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b80cfe70 msgr2=0x7f65b80c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b80cfe70 0x7f65b80c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 -- 172.21.15.86:0/3372705577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b815bd50 con 0x7f65b80d1c50 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bd400640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b80cfe70 0x7f65b80c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.144+0000 7f65bca00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b80c7de0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f65a4019130 tx=0x7f65a401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65b5600640 1 -- 172.21.15.86:0/3372705577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40be5f0 con 0x7f65b80d1c50 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65b5600640 1 -- 172.21.15.86:0/3372705577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a40be790 con 0x7f65b80d1c50 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65b5600640 1 -- 172.21.15.86:0/3372705577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40beb60 con 0x7f65b80d1c50 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b80c9950 con 0x7f65b80d1c50 2024-03-22T04:11:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b80c9f10 con 0x7f65b80d1c50 2024-03-22T04:11:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b8068650 con 0x7f65b80d1c50 2024-03-22T04:11:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65b5600640 1 -- 172.21.15.86:0/3372705577 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f65a40b43f0 con 0x7f65b80d1c50 2024-03-22T04:11:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65b5600640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6598067b20 0x7f6598069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65b5600640 1 -- 172.21.15.86:0/3372705577 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f65a40a8030 con 0x7f65b80d1c50 2024-03-22T04:11:00.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65bd400640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6598067b20 0x7f6598069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.148+0000 7f65bd400640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6598067b20 0x7f6598069fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000f70 tx=0x7f65ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.152+0000 7f65b5600640 1 -- 172.21.15.86:0/3372705577 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f65a4016110 con 0x7f65b80d1c50 2024-03-22T04:11:00.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.304+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65b8071af0 con 0x7f6598067b20 2024-03-22T04:11:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.312+0000 7f65b5600640 1 -- 172.21.15.86:0/3372705577 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f65b8071af0 con 0x7f6598067b20 2024-03-22T04:11:00.319 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:00.321 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6598067b20 msgr2=0x7f6598069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6598067b20 0x7f6598069fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000f70 tx=0x7f65ac009040 comp rx=0 tx=0).stop 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 msgr2=0x7f65b80c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b80c7de0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f65a4019130 tx=0x7f65a401ad60 comp rx=0 tx=0).stop 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 shutdown_connections 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6598067b20 0x7f6598069fe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65b815c4a0 0x7f65b815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65b80d1c50 0x7f65b80c7de0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 --2- 172.21.15.86:0/3372705577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b80cfe70 0x7f65b80c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 >> 172.21.15.86:0/3372705577 conn(0x7f65b806f930 msgr2=0x7f65b815ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 shutdown_connections 2024-03-22T04:11:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.316+0000 7f65bfe00640 1 -- 172.21.15.86:0/3372705577 wait complete. 2024-03-22T04:11:00.338 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-03-22T04:10:59.796813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219592,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755128,"statfs":{"total":773094113280,"available":772869251072,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002971"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967926+0000","last_change":"2024-03-22T04:10:33.930897+0000","last_active":"2024-03-22T04:10:39.967926+0000","last_peered":"2024-03-22T04:10:39.967926+0000","last_clean":"2024-03-22T04:10:39.967926+0000","last_became_active":"2024-03-22T04:10:33.930543+0000","last_became_peered":"2024-03-22T04:10:33.930543+0000","last_unstale":"2024-03-22T04:10:39.967926+0000","last_undegraded":"2024-03-22T04:10:39.967926+0000","last_fullsized":"2024-03-22T04:10:39.967926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:57:52.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170514+0000","last_change":"2024-03-22T04:10:34.938515+0000","last_active":"2024-03-22T04:10:41.170514+0000","last_peered":"2024-03-22T04:10:41.170514+0000","last_clean":"2024-03-22T04:10:41.170514+0000","last_became_active":"2024-03-22T04:10:34.938169+0000","last_became_peered":"2024-03-22T04:10:34.938169+0000","last_unstale":"2024-03-22T04:10:41.170514+0000","last_undegraded":"2024-03-22T04:10:41.170514+0000","last_fullsized":"2024-03-22T04:10:41.170514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:34.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966658+0000","last_change":"2024-03-22T04:10:38.983720+0000","last_active":"2024-03-22T04:10:39.966658+0000","last_peered":"2024-03-22T04:10:39.966658+0000","last_clean":"2024-03-22T04:10:39.966658+0000","last_became_active":"2024-03-22T04:10:38.983348+0000","last_became_peered":"2024-03-22T04:10:38.983348+0000","last_unstale":"2024-03-22T04:10:39.966658+0000","last_undegraded":"2024-03-22T04:10:39.966658+0000","last_fullsized":"2024-03-22T04:10:39.966658+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:45.494650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170607+0000","last_change":"2024-03-22T04:10:33.934030+0000","last_active":"2024-03-22T04:10:41.170607+0000","last_peered":"2024-03-22T04:10:41.170607+0000","last_clean":"2024-03-22T04:10:41.170607+0000","last_became_active":"2024-03-22T04:10:33.933718+0000","last_became_peered":"2024-03-22T04:10:33.933718+0000","last_unstale":"2024-03-22T04:10:41.170607+0000","last_undegraded":"2024-03-22T04:10:41.170607+0000","last_fullsized":"2024-03-22T04:10:41.170607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:50:42.805996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170782+0000","last_change":"2024-03-22T04:10:34.942768+0000","last_active":"2024-03-22T04:10:41.170782+0000","last_peered":"2024-03-22T04:10:41.170782+0000","last_clean":"2024-03-22T04:10:41.170782+0000","last_became_active":"2024-03-22T04:10:34.941567+0000","last_became_peered":"2024-03-22T04:10:34.941567+0000","last_unstale":"2024-03-22T04:10:41.170782+0000","last_undegraded":"2024-03-22T04:10:41.170782+0000","last_fullsized":"2024-03-22T04:10:41.170782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:26.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964401+0000","last_change":"2024-03-22T04:10:38.988088+0000","last_active":"2024-03-22T04:10:39.964401+0000","last_peered":"2024-03-22T04:10:39.964401+0000","last_clean":"2024-03-22T04:10:39.964401+0000","last_became_active":"2024-03-22T04:10:38.987611+0000","last_became_peered":"2024-03-22T04:10:38.987611+0000","last_unstale":"2024-03-22T04:10:39.964401+0000","last_undegraded":"2024-03-22T04:10:39.964401+0000","last_fullsized":"2024-03-22T04:10:39.964401+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:15.151121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535943+0000","last_change":"2024-03-22T04:10:33.936193+0000","last_active":"2024-03-22T04:10:41.535943+0000","last_peered":"2024-03-22T04:10:41.535943+0000","last_clean":"2024-03-22T04:10:41.535943+0000","last_became_active":"2024-03-22T04:10:33.935883+0000","last_became_peered":"2024-03-22T04:10:33.935883+0000","last_unstale":"2024-03-22T04:10:41.535943+0000","last_undegraded":"2024-03-22T04:10:41.535943+0000","last_fullsized":"2024-03-22T04:10:41.535943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:00.244111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535389+0000","last_change":"2024-03-22T04:10:34.935504+0000","last_active":"2024-03-22T04:10:41.535389+0000","last_peered":"2024-03-22T04:10:41.535389+0000","last_clean":"2024-03-22T04:10:41.535389+0000","last_became_active":"2024-03-22T04:10:34.933502+0000","last_became_peered":"2024-03-22T04:10:34.933502+0000","last_unstale":"2024-03-22T04:10:41.535389+0000","last_undegraded":"2024-03-22T04:10:41.535389+0000","last_fullsized":"2024-03-22T04:10:41.535389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965123+0000","last_change":"2024-03-22T04:10:38.991103+0000","last_active":"2024-03-22T04:10:39.965123+0000","last_peered":"2024-03-22T04:10:39.965123+0000","last_clean":"2024-03-22T04:10:39.965123+0000","last_became_active":"2024-03-22T04:10:38.988847+0000","last_became_peered":"2024-03-22T04:10:38.988847+0000","last_unstale":"2024-03-22T04:10:39.965123+0000","last_undegraded":"2024-03-22T04:10:39.965123+0000","last_fullsized":"2024-03-22T04:10:39.965123+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:45.922825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535279+0000","last_change":"2024-03-22T04:10:33.936237+0000","last_active":"2024-03-22T04:10:41.535279+0000","last_peered":"2024-03-22T04:10:41.535279+0000","last_clean":"2024-03-22T04:10:41.535279+0000","last_became_active":"2024-03-22T04:10:33.935870+0000","last_became_peered":"2024-03-22T04:10:33.935870+0000","last_unstale":"2024-03-22T04:10:41.535279+0000","last_undegraded":"2024-03-22T04:10:41.535279+0000","last_fullsized":"2024-03-22T04:10:41.535279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:47.209113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922489+0000","last_change":"2024-03-22T04:10:34.930847+0000","last_active":"2024-03-22T04:10:40.922489+0000","last_peered":"2024-03-22T04:10:40.922489+0000","last_clean":"2024-03-22T04:10:40.922489+0000","last_became_active":"2024-03-22T04:10:34.930567+0000","last_became_peered":"2024-03-22T04:10:34.930567+0000","last_unstale":"2024-03-22T04:10:40.922489+0000","last_undegraded":"2024-03-22T04:10:40.922489+0000","last_fullsized":"2024-03-22T04:10:40.922489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:17.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964576+0000","last_change":"2024-03-22T04:10:38.992358+0000","last_active":"2024-03-22T04:10:39.964576+0000","last_peered":"2024-03-22T04:10:39.964576+0000","last_clean":"2024-03-22T04:10:39.964576+0000","last_became_active":"2024-03-22T04:10:38.991968+0000","last_became_peered":"2024-03-22T04:10:38.991968+0000","last_unstale":"2024-03-22T04:10:39.964576+0000","last_undegraded":"2024-03-22T04:10:39.964576+0000","last_fullsized":"2024-03-22T04:10:39.964576+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:12.109401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922040+0000","last_change":"2024-03-22T04:10:34.947353+0000","last_active":"2024-03-22T04:10:40.922040+0000","last_peered":"2024-03-22T04:10:40.922040+0000","last_clean":"2024-03-22T04:10:40.922040+0000","last_became_active":"2024-03-22T04:10:34.945247+0000","last_became_peered":"2024-03-22T04:10:34.945247+0000","last_unstale":"2024-03-22T04:10:40.922040+0000","last_undegraded":"2024-03-22T04:10:40.922040+0000","last_fullsized":"2024-03-22T04:10:40.922040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:31.574004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922244+0000","last_change":"2024-03-22T04:10:33.938771+0000","last_active":"2024-03-22T04:10:40.922244+0000","last_peered":"2024-03-22T04:10:40.922244+0000","last_clean":"2024-03-22T04:10:40.922244+0000","last_became_active":"2024-03-22T04:10:33.937907+0000","last_became_peered":"2024-03-22T04:10:33.937907+0000","last_unstale":"2024-03-22T04:10:40.922244+0000","last_undegraded":"2024-03-22T04:10:40.922244+0000","last_fullsized":"2024-03-22T04:10:40.922244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:43.867139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922386+0000","last_change":"2024-03-22T04:10:38.979866+0000","last_active":"2024-03-22T04:10:40.922386+0000","last_peered":"2024-03-22T04:10:40.922386+0000","last_clean":"2024-03-22T04:10:40.922386+0000","last_became_active":"2024-03-22T04:10:38.979321+0000","last_became_peered":"2024-03-22T04:10:38.979321+0000","last_unstale":"2024-03-22T04:10:40.922386+0000","last_undegraded":"2024-03-22T04:10:40.922386+0000","last_fullsized":"2024-03-22T04:10:40.922386+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:03:12.221975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967207+0000","last_change":"2024-03-22T04:10:34.951257+0000","last_active":"2024-03-22T04:10:39.967207+0000","last_peered":"2024-03-22T04:10:39.967207+0000","last_clean":"2024-03-22T04:10:39.967207+0000","last_became_active":"2024-03-22T04:10:34.950994+0000","last_became_peered":"2024-03-22T04:10:34.950994+0000","last_unstale":"2024-03-22T04:10:39.967207+0000","last_undegraded":"2024-03-22T04:10:39.967207+0000","last_fullsized":"2024-03-22T04:10:39.967207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536303+0000","last_change":"2024-03-22T04:10:33.933287+0000","last_active":"2024-03-22T04:10:41.536303+0000","last_peered":"2024-03-22T04:10:41.536303+0000","last_clean":"2024-03-22T04:10:41.536303+0000","last_became_active":"2024-03-22T04:10:33.932862+0000","last_became_peered":"2024-03-22T04:10:33.932862+0000","last_unstale":"2024-03-22T04:10:41.536303+0000","last_undegraded":"2024-03-22T04:10:41.536303+0000","last_fullsized":"2024-03-22T04:10:41.536303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534955+0000","last_change":"2024-03-22T04:10:38.986486+0000","last_active":"2024-03-22T04:10:41.534955+0000","last_peered":"2024-03-22T04:10:41.534955+0000","last_clean":"2024-03-22T04:10:41.534955+0000","last_became_active":"2024-03-22T04:10:38.985993+0000","last_became_peered":"2024-03-22T04:10:38.985993+0000","last_unstale":"2024-03-22T04:10:41.534955+0000","last_undegraded":"2024-03-22T04:10:41.534955+0000","last_fullsized":"2024-03-22T04:10:41.534955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:21.157155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966213+0000","last_change":"2024-03-22T04:10:34.946405+0000","last_active":"2024-03-22T04:10:39.966213+0000","last_peered":"2024-03-22T04:10:39.966213+0000","last_clean":"2024-03-22T04:10:39.966213+0000","last_became_active":"2024-03-22T04:10:34.946093+0000","last_became_peered":"2024-03-22T04:10:34.946093+0000","last_unstale":"2024-03-22T04:10:39.966213+0000","last_undegraded":"2024-03-22T04:10:39.966213+0000","last_fullsized":"2024-03-22T04:10:39.966213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:54:39.775612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966845+0000","last_change":"2024-03-22T04:10:33.931984+0000","last_active":"2024-03-22T04:10:39.966845+0000","last_peered":"2024-03-22T04:10:39.966845+0000","last_clean":"2024-03-22T04:10:39.966845+0000","last_became_active":"2024-03-22T04:10:33.931674+0000","last_became_peered":"2024-03-22T04:10:33.931674+0000","last_unstale":"2024-03-22T04:10:39.966845+0000","last_undegraded":"2024-03-22T04:10:39.966845+0000","last_fullsized":"2024-03-22T04:10:39.966845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:11.685927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967850+0000","last_change":"2024-03-22T04:10:38.992239+0000","last_active":"2024-03-22T04:10:39.967850+0000","last_peered":"2024-03-22T04:10:39.967850+0000","last_clean":"2024-03-22T04:10:39.967850+0000","last_became_active":"2024-03-22T04:10:38.991429+0000","last_became_peered":"2024-03-22T04:10:38.991429+0000","last_unstale":"2024-03-22T04:10:39.967850+0000","last_undegraded":"2024-03-22T04:10:39.967850+0000","last_fullsized":"2024-03-22T04:10:39.967850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:22.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921940+0000","last_change":"2024-03-22T04:10:34.939183+0000","last_active":"2024-03-22T04:10:40.921940+0000","last_peered":"2024-03-22T04:10:40.921940+0000","last_clean":"2024-03-22T04:10:40.921940+0000","last_became_active":"2024-03-22T04:10:34.938388+0000","last_became_peered":"2024-03-22T04:10:34.938388+0000","last_unstale":"2024-03-22T04:10:40.921940+0000","last_undegraded":"2024-03-22T04:10:40.921940+0000","last_fullsized":"2024-03-22T04:10:40.921940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:10:15.750612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534426+0000","last_change":"2024-03-22T04:10:33.933352+0000","last_active":"2024-03-22T04:10:41.534426+0000","last_peered":"2024-03-22T04:10:41.534426+0000","last_clean":"2024-03-22T04:10:41.534426+0000","last_became_active":"2024-03-22T04:10:33.933066+0000","last_became_peered":"2024-03-22T04:10:33.933066+0000","last_unstale":"2024-03-22T04:10:41.534426+0000","last_undegraded":"2024-03-22T04:10:41.534426+0000","last_fullsized":"2024-03-22T04:10:41.534426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:50:42.805996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964895+0000","last_change":"2024-03-22T04:10:38.987700+0000","last_active":"2024-03-22T04:10:39.964895+0000","last_peered":"2024-03-22T04:10:39.964895+0000","last_clean":"2024-03-22T04:10:39.964895+0000","last_became_active":"2024-03-22T04:10:38.987221+0000","last_became_peered":"2024-03-22T04:10:38.987221+0000","last_unstale":"2024-03-22T04:10:39.964895+0000","last_undegraded":"2024-03-22T04:10:39.964895+0000","last_fullsized":"2024-03-22T04:10:39.964895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:30.365392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967947+0000","last_change":"2024-03-22T04:10:34.953907+0000","last_active":"2024-03-22T04:10:39.967947+0000","last_peered":"2024-03-22T04:10:39.967947+0000","last_clean":"2024-03-22T04:10:39.967947+0000","last_became_active":"2024-03-22T04:10:34.953664+0000","last_became_peered":"2024-03-22T04:10:34.953664+0000","last_unstale":"2024-03-22T04:10:39.967947+0000","last_undegraded":"2024-03-22T04:10:39.967947+0000","last_fullsized":"2024-03-22T04:10:39.967947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:07.983679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922944+0000","last_change":"2024-03-22T04:10:33.931133+0000","last_active":"2024-03-22T04:10:40.922944+0000","last_peered":"2024-03-22T04:10:40.922944+0000","last_clean":"2024-03-22T04:10:40.922944+0000","last_became_active":"2024-03-22T04:10:33.930877+0000","last_became_peered":"2024-03-22T04:10:33.930877+0000","last_unstale":"2024-03-22T04:10:40.922944+0000","last_undegraded":"2024-03-22T04:10:40.922944+0000","last_fullsized":"2024-03-22T04:10:40.922944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:39.248649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964896+0000","last_change":"2024-03-22T04:10:38.983903+0000","last_active":"2024-03-22T04:10:39.964896+0000","last_peered":"2024-03-22T04:10:39.964896+0000","last_clean":"2024-03-22T04:10:39.964896+0000","last_became_active":"2024-03-22T04:10:38.983459+0000","last_became_peered":"2024-03-22T04:10:38.983459+0000","last_unstale":"2024-03-22T04:10:39.964896+0000","last_undegraded":"2024-03-22T04:10:39.964896+0000","last_fullsized":"2024-03-22T04:10:39.964896+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:38.915424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967821+0000","last_change":"2024-03-22T04:10:34.930898+0000","last_active":"2024-03-22T04:10:39.967821+0000","last_peered":"2024-03-22T04:10:39.967821+0000","last_clean":"2024-03-22T04:10:39.967821+0000","last_became_active":"2024-03-22T04:10:34.930562+0000","last_became_peered":"2024-03-22T04:10:34.930562+0000","last_unstale":"2024-03-22T04:10:39.967821+0000","last_undegraded":"2024-03-22T04:10:39.967821+0000","last_fullsized":"2024-03-22T04:10:39.967821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:46:40.225626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.968051+0000","last_change":"2024-03-22T04:10:33.934918+0000","last_active":"2024-03-22T04:10:39.968051+0000","last_peered":"2024-03-22T04:10:39.968051+0000","last_clean":"2024-03-22T04:10:39.968051+0000","last_became_active":"2024-03-22T04:10:33.933026+0000","last_became_peered":"2024-03-22T04:10:33.933026+0000","last_unstale":"2024-03-22T04:10:39.968051+0000","last_undegraded":"2024-03-22T04:10:39.968051+0000","last_fullsized":"2024-03-22T04:10:39.968051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:42:25.730112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966295+0000","last_change":"2024-03-22T04:10:38.984737+0000","last_active":"2024-03-22T04:10:39.966295+0000","last_peered":"2024-03-22T04:10:39.966295+0000","last_clean":"2024-03-22T04:10:39.966295+0000","last_became_active":"2024-03-22T04:10:38.984373+0000","last_became_peered":"2024-03-22T04:10:38.984373+0000","last_unstale":"2024-03-22T04:10:39.966295+0000","last_undegraded":"2024-03-22T04:10:39.966295+0000","last_fullsized":"2024-03-22T04:10:39.966295+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:23:32.419897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536493+0000","last_change":"2024-03-22T04:10:34.928433+0000","last_active":"2024-03-22T04:10:41.536493+0000","last_peered":"2024-03-22T04:10:41.536493+0000","last_clean":"2024-03-22T04:10:41.536493+0000","last_became_active":"2024-03-22T04:10:34.927912+0000","last_became_peered":"2024-03-22T04:10:34.927912+0000","last_unstale":"2024-03-22T04:10:41.536493+0000","last_undegraded":"2024-03-22T04:10:41.536493+0000","last_fullsized":"2024-03-22T04:10:41.536493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:12:01.577501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922703+0000","last_change":"2024-03-22T04:10:33.938458+0000","last_active":"2024-03-22T04:10:40.922703+0000","last_peered":"2024-03-22T04:10:40.922703+0000","last_clean":"2024-03-22T04:10:40.922703+0000","last_became_active":"2024-03-22T04:10:33.937789+0000","last_became_peered":"2024-03-22T04:10:33.937789+0000","last_unstale":"2024-03-22T04:10:40.922703+0000","last_undegraded":"2024-03-22T04:10:40.922703+0000","last_fullsized":"2024-03-22T04:10:40.922703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:40.441952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536228+0000","last_change":"2024-03-22T04:10:38.988981+0000","last_active":"2024-03-22T04:10:41.536228+0000","last_peered":"2024-03-22T04:10:41.536228+0000","last_clean":"2024-03-22T04:10:41.536228+0000","last_became_active":"2024-03-22T04:10:38.988442+0000","last_became_peered":"2024-03-22T04:10:38.988442+0000","last_unstale":"2024-03-22T04:10:41.536228+0000","last_undegraded":"2024-03-22T04:10:41.536228+0000","last_fullsized":"2024-03-22T04:10:41.536228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:38.915424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964614+0000","last_change":"2024-03-22T04:10:34.931657+0000","last_active":"2024-03-22T04:10:39.964614+0000","last_peered":"2024-03-22T04:10:39.964614+0000","last_clean":"2024-03-22T04:10:39.964614+0000","last_became_active":"2024-03-22T04:10:34.931362+0000","last_became_peered":"2024-03-22T04:10:34.931362+0000","last_unstale":"2024-03-22T04:10:39.964614+0000","last_undegraded":"2024-03-22T04:10:39.964614+0000","last_fullsized":"2024-03-22T04:10:39.964614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170215+0000","last_change":"2024-03-22T04:10:33.934747+0000","last_active":"2024-03-22T04:10:41.170215+0000","last_peered":"2024-03-22T04:10:41.170215+0000","last_clean":"2024-03-22T04:10:41.170215+0000","last_became_active":"2024-03-22T04:10:33.933151+0000","last_became_peered":"2024-03-22T04:10:33.933151+0000","last_unstale":"2024-03-22T04:10:41.170215+0000","last_undegraded":"2024-03-22T04:10:41.170215+0000","last_fullsized":"2024-03-22T04:10:41.170215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536019+0000","last_change":"2024-03-22T04:10:38.972402+0000","last_active":"2024-03-22T04:10:41.536019+0000","last_peered":"2024-03-22T04:10:41.536019+0000","last_clean":"2024-03-22T04:10:41.536019+0000","last_became_active":"2024-03-22T04:10:38.972049+0000","last_became_peered":"2024-03-22T04:10:38.972049+0000","last_unstale":"2024-03-22T04:10:41.536019+0000","last_undegraded":"2024-03-22T04:10:41.536019+0000","last_fullsized":"2024-03-22T04:10:41.536019+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:20.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535049+0000","last_change":"2024-03-22T04:10:34.931298+0000","last_active":"2024-03-22T04:10:41.535049+0000","last_peered":"2024-03-22T04:10:41.535049+0000","last_clean":"2024-03-22T04:10:41.535049+0000","last_became_active":"2024-03-22T04:10:34.930240+0000","last_became_peered":"2024-03-22T04:10:34.930240+0000","last_unstale":"2024-03-22T04:10:41.535049+0000","last_undegraded":"2024-03-22T04:10:41.535049+0000","last_fullsized":"2024-03-22T04:10:41.535049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:11.753753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922237+0000","last_change":"2024-03-22T04:10:33.938276+0000","last_active":"2024-03-22T04:10:40.922237+0000","last_peered":"2024-03-22T04:10:40.922237+0000","last_clean":"2024-03-22T04:10:40.922237+0000","last_became_active":"2024-03-22T04:10:33.933126+0000","last_became_peered":"2024-03-22T04:10:33.933126+0000","last_unstale":"2024-03-22T04:10:40.922237+0000","last_undegraded":"2024-03-22T04:10:40.922237+0000","last_fullsized":"2024-03-22T04:10:40.922237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:11.368879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536420+0000","last_change":"2024-03-22T04:10:38.974697+0000","last_active":"2024-03-22T04:10:41.536420+0000","last_peered":"2024-03-22T04:10:41.536420+0000","last_clean":"2024-03-22T04:10:41.536420+0000","last_became_active":"2024-03-22T04:10:38.974339+0000","last_became_peered":"2024-03-22T04:10:38.974339+0000","last_unstale":"2024-03-22T04:10:41.536420+0000","last_undegraded":"2024-03-22T04:10:41.536420+0000","last_fullsized":"2024-03-22T04:10:41.536420+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:59.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169762+0000","last_change":"2024-03-22T04:10:34.947104+0000","last_active":"2024-03-22T04:10:41.169762+0000","last_peered":"2024-03-22T04:10:41.169762+0000","last_clean":"2024-03-22T04:10:41.169762+0000","last_became_active":"2024-03-22T04:10:34.946787+0000","last_became_peered":"2024-03-22T04:10:34.946787+0000","last_unstale":"2024-03-22T04:10:41.169762+0000","last_undegraded":"2024-03-22T04:10:41.169762+0000","last_fullsized":"2024-03-22T04:10:41.169762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:11.753753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966701+0000","last_change":"2024-03-22T04:10:33.938271+0000","last_active":"2024-03-22T04:10:39.966701+0000","last_peered":"2024-03-22T04:10:39.966701+0000","last_clean":"2024-03-22T04:10:39.966701+0000","last_became_active":"2024-03-22T04:10:33.937651+0000","last_became_peered":"2024-03-22T04:10:33.937651+0000","last_unstale":"2024-03-22T04:10:39.966701+0000","last_undegraded":"2024-03-22T04:10:39.966701+0000","last_fullsized":"2024-03-22T04:10:39.966701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536236+0000","last_change":"2024-03-22T04:10:38.980629+0000","last_active":"2024-03-22T04:10:41.536236+0000","last_peered":"2024-03-22T04:10:41.536236+0000","last_clean":"2024-03-22T04:10:41.536236+0000","last_became_active":"2024-03-22T04:10:38.980157+0000","last_became_peered":"2024-03-22T04:10:38.980157+0000","last_unstale":"2024-03-22T04:10:41.536236+0000","last_undegraded":"2024-03-22T04:10:41.536236+0000","last_fullsized":"2024-03-22T04:10:41.536236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:30.365392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535927+0000","last_change":"2024-03-22T04:10:34.940387+0000","last_active":"2024-03-22T04:10:41.535927+0000","last_peered":"2024-03-22T04:10:41.535927+0000","last_clean":"2024-03-22T04:10:41.535927+0000","last_became_active":"2024-03-22T04:10:34.940009+0000","last_became_peered":"2024-03-22T04:10:34.940009+0000","last_unstale":"2024-03-22T04:10:41.535927+0000","last_undegraded":"2024-03-22T04:10:41.535927+0000","last_fullsized":"2024-03-22T04:10:41.535927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:12.376047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922076+0000","last_change":"2024-03-22T04:10:33.927977+0000","last_active":"2024-03-22T04:10:40.922076+0000","last_peered":"2024-03-22T04:10:40.922076+0000","last_clean":"2024-03-22T04:10:40.922076+0000","last_became_active":"2024-03-22T04:10:33.927635+0000","last_became_peered":"2024-03-22T04:10:33.927635+0000","last_unstale":"2024-03-22T04:10:40.922076+0000","last_undegraded":"2024-03-22T04:10:40.922076+0000","last_fullsized":"2024-03-22T04:10:40.922076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:00.244111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170363+0000","last_change":"2024-03-22T04:10:38.985749+0000","last_active":"2024-03-22T04:10:41.170363+0000","last_peered":"2024-03-22T04:10:41.170363+0000","last_clean":"2024-03-22T04:10:41.170363+0000","last_became_active":"2024-03-22T04:10:38.981278+0000","last_became_peered":"2024-03-22T04:10:38.981278+0000","last_unstale":"2024-03-22T04:10:41.170363+0000","last_undegraded":"2024-03-22T04:10:41.170363+0000","last_fullsized":"2024-03-22T04:10:41.170363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:40:38.204332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966651+0000","last_change":"2024-03-22T04:10:34.921916+0000","last_active":"2024-03-22T04:10:39.966651+0000","last_peered":"2024-03-22T04:10:39.966651+0000","last_clean":"2024-03-22T04:10:39.966651+0000","last_became_active":"2024-03-22T04:10:34.921675+0000","last_became_peered":"2024-03-22T04:10:34.921675+0000","last_unstale":"2024-03-22T04:10:39.966651+0000","last_undegraded":"2024-03-22T04:10:39.966651+0000","last_fullsized":"2024-03-22T04:10:39.966651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:23:07.242835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922298+0000","last_change":"2024-03-22T04:10:33.937845+0000","last_active":"2024-03-22T04:10:40.922298+0000","last_peered":"2024-03-22T04:10:40.922298+0000","last_clean":"2024-03-22T04:10:40.922298+0000","last_became_active":"2024-03-22T04:10:33.930400+0000","last_became_peered":"2024-03-22T04:10:33.930400+0000","last_unstale":"2024-03-22T04:10:40.922298+0000","last_undegraded":"2024-03-22T04:10:40.922298+0000","last_fullsized":"2024-03-22T04:10:40.922298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:03.705836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":177,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535319+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:10:41.535319+0000","last_peered":"2024-03-22T04:10:41.535319+0000","last_clean":"2024-03-22T04:10:41.535319+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:10:41.535319+0000","last_undegraded":"2024-03-22T04:10:41.535319+0000","last_fullsized":"2024-03-22T04:10:41.535319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922657+0000","last_change":"2024-03-22T04:10:38.982996+0000","last_active":"2024-03-22T04:10:40.922657+0000","last_peered":"2024-03-22T04:10:40.922657+0000","last_clean":"2024-03-22T04:10:40.922657+0000","last_became_active":"2024-03-22T04:10:38.981238+0000","last_became_peered":"2024-03-22T04:10:38.981238+0000","last_unstale":"2024-03-22T04:10:40.922657+0000","last_undegraded":"2024-03-22T04:10:40.922657+0000","last_fullsized":"2024-03-22T04:10:40.922657+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:09:02.614234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922533+0000","last_change":"2024-03-22T04:10:34.945879+0000","last_active":"2024-03-22T04:10:40.922533+0000","last_peered":"2024-03-22T04:10:40.922533+0000","last_clean":"2024-03-22T04:10:40.922533+0000","last_became_active":"2024-03-22T04:10:34.945056+0000","last_became_peered":"2024-03-22T04:10:34.945056+0000","last_unstale":"2024-03-22T04:10:40.922533+0000","last_undegraded":"2024-03-22T04:10:40.922533+0000","last_fullsized":"2024-03-22T04:10:40.922533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:17:47.020807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536542+0000","last_change":"2024-03-22T04:10:33.931838+0000","last_active":"2024-03-22T04:10:41.536542+0000","last_peered":"2024-03-22T04:10:41.536542+0000","last_clean":"2024-03-22T04:10:41.536542+0000","last_became_active":"2024-03-22T04:10:33.931611+0000","last_became_peered":"2024-03-22T04:10:33.931611+0000","last_unstale":"2024-03-22T04:10:41.536542+0000","last_undegraded":"2024-03-22T04:10:41.536542+0000","last_fullsized":"2024-03-22T04:10:41.536542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964623+0000","last_change":"2024-03-22T04:10:38.988757+0000","last_active":"2024-03-22T04:10:39.964623+0000","last_peered":"2024-03-22T04:10:39.964623+0000","last_clean":"2024-03-22T04:10:39.964623+0000","last_became_active":"2024-03-22T04:10:38.987185+0000","last_became_peered":"2024-03-22T04:10:38.987185+0000","last_unstale":"2024-03-22T04:10:39.964623+0000","last_undegraded":"2024-03-22T04:10:39.964623+0000","last_fullsized":"2024-03-22T04:10:39.964623+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:15.799282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535188+0000","last_change":"2024-03-22T04:10:34.948529+0000","last_active":"2024-03-22T04:10:41.535188+0000","last_peered":"2024-03-22T04:10:41.535188+0000","last_clean":"2024-03-22T04:10:41.535188+0000","last_became_active":"2024-03-22T04:10:34.948122+0000","last_became_peered":"2024-03-22T04:10:34.948122+0000","last_unstale":"2024-03-22T04:10:41.535188+0000","last_undegraded":"2024-03-22T04:10:41.535188+0000","last_fullsized":"2024-03-22T04:10:41.535188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:55.025923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170450+0000","last_change":"2024-03-22T04:10:33.938126+0000","last_active":"2024-03-22T04:10:41.170450+0000","last_peered":"2024-03-22T04:10:41.170450+0000","last_clean":"2024-03-22T04:10:41.170450+0000","last_became_active":"2024-03-22T04:10:33.937511+0000","last_became_peered":"2024-03-22T04:10:33.937511+0000","last_unstale":"2024-03-22T04:10:41.170450+0000","last_undegraded":"2024-03-22T04:10:41.170450+0000","last_fullsized":"2024-03-22T04:10:41.170450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169980+0000","last_change":"2024-03-22T04:10:38.988377+0000","last_active":"2024-03-22T04:10:41.169980+0000","last_peered":"2024-03-22T04:10:41.169980+0000","last_clean":"2024-03-22T04:10:41.169980+0000","last_became_active":"2024-03-22T04:10:38.984506+0000","last_became_peered":"2024-03-22T04:10:38.984506+0000","last_unstale":"2024-03-22T04:10:41.169980+0000","last_undegraded":"2024-03-22T04:10:41.169980+0000","last_fullsized":"2024-03-22T04:10:41.169980+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:17:51.066337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535990+0000","last_change":"2024-03-22T04:10:34.940162+0000","last_active":"2024-03-22T04:10:41.535990+0000","last_peered":"2024-03-22T04:10:41.535990+0000","last_clean":"2024-03-22T04:10:41.535990+0000","last_became_active":"2024-03-22T04:10:34.937040+0000","last_became_peered":"2024-03-22T04:10:34.937040+0000","last_unstale":"2024-03-22T04:10:41.535990+0000","last_undegraded":"2024-03-22T04:10:41.535990+0000","last_fullsized":"2024-03-22T04:10:41.535990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.104177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967999+0000","last_change":"2024-03-22T04:10:33.932000+0000","last_active":"2024-03-22T04:10:39.967999+0000","last_peered":"2024-03-22T04:10:39.967999+0000","last_clean":"2024-03-22T04:10:39.967999+0000","last_became_active":"2024-03-22T04:10:33.930159+0000","last_became_peered":"2024-03-22T04:10:33.930159+0000","last_unstale":"2024-03-22T04:10:39.967999+0000","last_undegraded":"2024-03-22T04:10:39.967999+0000","last_fullsized":"2024-03-22T04:10:39.967999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:11:21.741394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921794+0000","last_change":"2024-03-22T04:10:38.979495+0000","last_active":"2024-03-22T04:10:40.921794+0000","last_peered":"2024-03-22T04:10:40.921794+0000","last_clean":"2024-03-22T04:10:40.921794+0000","last_became_active":"2024-03-22T04:10:38.979058+0000","last_became_peered":"2024-03-22T04:10:38.979058+0000","last_unstale":"2024-03-22T04:10:40.921794+0000","last_undegraded":"2024-03-22T04:10:40.921794+0000","last_fullsized":"2024-03-22T04:10:40.921794+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:25.405176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965133+0000","last_change":"2024-03-22T04:10:34.936969+0000","last_active":"2024-03-22T04:10:39.965133+0000","last_peered":"2024-03-22T04:10:39.965133+0000","last_clean":"2024-03-22T04:10:39.965133+0000","last_became_active":"2024-03-22T04:10:34.936270+0000","last_became_peered":"2024-03-22T04:10:34.936270+0000","last_unstale":"2024-03-22T04:10:39.965133+0000","last_undegraded":"2024-03-22T04:10:39.965133+0000","last_fullsized":"2024-03-22T04:10:39.965133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:44.874307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536081+0000","last_change":"2024-03-22T04:10:33.929682+0000","last_active":"2024-03-22T04:10:41.536081+0000","last_peered":"2024-03-22T04:10:41.536081+0000","last_clean":"2024-03-22T04:10:41.536081+0000","last_became_active":"2024-03-22T04:10:33.929309+0000","last_became_peered":"2024-03-22T04:10:33.929309+0000","last_unstale":"2024-03-22T04:10:41.536081+0000","last_undegraded":"2024-03-22T04:10:41.536081+0000","last_fullsized":"2024-03-22T04:10:41.536081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:05.641869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535389+0000","last_change":"2024-03-22T04:10:38.979596+0000","last_active":"2024-03-22T04:10:41.535389+0000","last_peered":"2024-03-22T04:10:41.535389+0000","last_clean":"2024-03-22T04:10:41.535389+0000","last_became_active":"2024-03-22T04:10:38.979166+0000","last_became_peered":"2024-03-22T04:10:38.979166+0000","last_unstale":"2024-03-22T04:10:41.535389+0000","last_undegraded":"2024-03-22T04:10:41.535389+0000","last_fullsized":"2024-03-22T04:10:41.535389+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:08.758533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921851+0000","last_change":"2024-03-22T04:10:34.926802+0000","last_active":"2024-03-22T04:10:40.921851+0000","last_peered":"2024-03-22T04:10:40.921851+0000","last_clean":"2024-03-22T04:10:40.921851+0000","last_became_active":"2024-03-22T04:10:34.926514+0000","last_became_peered":"2024-03-22T04:10:34.926514+0000","last_unstale":"2024-03-22T04:10:40.921851+0000","last_undegraded":"2024-03-22T04:10:40.921851+0000","last_fullsized":"2024-03-22T04:10:40.921851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:02:47.197474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966395+0000","last_change":"2024-03-22T04:10:33.932545+0000","last_active":"2024-03-22T04:10:39.966395+0000","last_peered":"2024-03-22T04:10:39.966395+0000","last_clean":"2024-03-22T04:10:39.966395+0000","last_became_active":"2024-03-22T04:10:33.929875+0000","last_became_peered":"2024-03-22T04:10:33.929875+0000","last_unstale":"2024-03-22T04:10:39.966395+0000","last_undegraded":"2024-03-22T04:10:39.966395+0000","last_fullsized":"2024-03-22T04:10:39.966395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:59:03.501920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170524+0000","last_change":"2024-03-22T04:10:38.985437+0000","last_active":"2024-03-22T04:10:41.170524+0000","last_peered":"2024-03-22T04:10:41.170524+0000","last_clean":"2024-03-22T04:10:41.170524+0000","last_became_active":"2024-03-22T04:10:38.982076+0000","last_became_peered":"2024-03-22T04:10:38.982076+0000","last_unstale":"2024-03-22T04:10:41.170524+0000","last_undegraded":"2024-03-22T04:10:41.170524+0000","last_fullsized":"2024-03-22T04:10:41.170524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:21.687034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535147+0000","last_change":"2024-03-22T04:10:34.928089+0000","last_active":"2024-03-22T04:10:41.535147+0000","last_peered":"2024-03-22T04:10:41.535147+0000","last_clean":"2024-03-22T04:10:41.535147+0000","last_became_active":"2024-03-22T04:10:34.927828+0000","last_became_peered":"2024-03-22T04:10:34.927828+0000","last_unstale":"2024-03-22T04:10:41.535147+0000","last_undegraded":"2024-03-22T04:10:41.535147+0000","last_fullsized":"2024-03-22T04:10:41.535147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534615+0000","last_change":"2024-03-22T04:10:33.943460+0000","last_active":"2024-03-22T04:10:41.534615+0000","last_peered":"2024-03-22T04:10:41.534615+0000","last_clean":"2024-03-22T04:10:41.534615+0000","last_became_active":"2024-03-22T04:10:33.943039+0000","last_became_peered":"2024-03-22T04:10:33.943039+0000","last_unstale":"2024-03-22T04:10:41.534615+0000","last_undegraded":"2024-03-22T04:10:41.534615+0000","last_fullsized":"2024-03-22T04:10:41.534615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:40:51.297536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170801+0000","last_change":"2024-03-22T04:10:38.981354+0000","last_active":"2024-03-22T04:10:41.170801+0000","last_peered":"2024-03-22T04:10:41.170801+0000","last_clean":"2024-03-22T04:10:41.170801+0000","last_became_active":"2024-03-22T04:10:38.979192+0000","last_became_peered":"2024-03-22T04:10:38.979192+0000","last_unstale":"2024-03-22T04:10:41.170801+0000","last_undegraded":"2024-03-22T04:10:41.170801+0000","last_fullsized":"2024-03-22T04:10:41.170801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:05:23.818860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922177+0000","last_change":"2024-03-22T04:10:34.935166+0000","last_active":"2024-03-22T04:10:40.922177+0000","last_peered":"2024-03-22T04:10:40.922177+0000","last_clean":"2024-03-22T04:10:40.922177+0000","last_became_active":"2024-03-22T04:10:34.934859+0000","last_became_peered":"2024-03-22T04:10:34.934859+0000","last_unstale":"2024-03-22T04:10:40.922177+0000","last_undegraded":"2024-03-22T04:10:40.922177+0000","last_fullsized":"2024-03-22T04:10:40.922177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:34:47.128097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922197+0000","last_change":"2024-03-22T04:10:33.939004+0000","last_active":"2024-03-22T04:10:40.922197+0000","last_peered":"2024-03-22T04:10:40.922197+0000","last_clean":"2024-03-22T04:10:40.922197+0000","last_became_active":"2024-03-22T04:10:33.938036+0000","last_became_peered":"2024-03-22T04:10:33.938036+0000","last_unstale":"2024-03-22T04:10:40.922197+0000","last_undegraded":"2024-03-22T04:10:40.922197+0000","last_fullsized":"2024-03-22T04:10:40.922197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:47.209113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536345+0000","last_change":"2024-03-22T04:10:38.985663+0000","last_active":"2024-03-22T04:10:41.536345+0000","last_peered":"2024-03-22T04:10:41.536345+0000","last_clean":"2024-03-22T04:10:41.536345+0000","last_became_active":"2024-03-22T04:10:38.985235+0000","last_became_peered":"2024-03-22T04:10:38.985235+0000","last_unstale":"2024-03-22T04:10:41.536345+0000","last_undegraded":"2024-03-22T04:10:41.536345+0000","last_fullsized":"2024-03-22T04:10:41.536345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:45.922825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536323+0000","last_change":"2024-03-22T04:10:34.950266+0000","last_active":"2024-03-22T04:10:41.536323+0000","last_peered":"2024-03-22T04:10:41.536323+0000","last_clean":"2024-03-22T04:10:41.536323+0000","last_became_active":"2024-03-22T04:10:34.949946+0000","last_became_peered":"2024-03-22T04:10:34.949946+0000","last_unstale":"2024-03-22T04:10:41.536323+0000","last_undegraded":"2024-03-22T04:10:41.536323+0000","last_fullsized":"2024-03-22T04:10:41.536323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:04.713004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967305+0000","last_change":"2024-03-22T04:10:33.935184+0000","last_active":"2024-03-22T04:10:39.967305+0000","last_peered":"2024-03-22T04:10:39.967305+0000","last_clean":"2024-03-22T04:10:39.967305+0000","last_became_active":"2024-03-22T04:10:33.934904+0000","last_became_peered":"2024-03-22T04:10:33.934904+0000","last_unstale":"2024-03-22T04:10:39.967305+0000","last_undegraded":"2024-03-22T04:10:39.967305+0000","last_fullsized":"2024-03-22T04:10:39.967305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922445+0000","last_change":"2024-03-22T04:10:38.971479+0000","last_active":"2024-03-22T04:10:40.922445+0000","last_peered":"2024-03-22T04:10:40.922445+0000","last_clean":"2024-03-22T04:10:40.922445+0000","last_became_active":"2024-03-22T04:10:38.969629+0000","last_became_peered":"2024-03-22T04:10:38.969629+0000","last_unstale":"2024-03-22T04:10:40.922445+0000","last_undegraded":"2024-03-22T04:10:40.922445+0000","last_fullsized":"2024-03-22T04:10:40.922445+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:28:08.179423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535818+0000","last_change":"2024-03-22T04:10:34.932181+0000","last_active":"2024-03-22T04:10:41.535818+0000","last_peered":"2024-03-22T04:10:41.535818+0000","last_clean":"2024-03-22T04:10:41.535818+0000","last_became_active":"2024-03-22T04:10:34.931936+0000","last_became_peered":"2024-03-22T04:10:34.931936+0000","last_unstale":"2024-03-22T04:10:41.535818+0000","last_undegraded":"2024-03-22T04:10:41.535818+0000","last_fullsized":"2024-03-22T04:10:41.535818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:26.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:50.358530+0000","last_change":"2024-03-22T04:10:33.934598+0000","last_active":"2024-03-22T04:10:50.358530+0000","last_peered":"2024-03-22T04:10:50.358530+0000","last_clean":"2024-03-22T04:10:50.358530+0000","last_became_active":"2024-03-22T04:10:33.934129+0000","last_became_peered":"2024-03-22T04:10:33.934129+0000","last_unstale":"2024-03-22T04:10:50.358530+0000","last_undegraded":"2024-03-22T04:10:50.358530+0000","last_fullsized":"2024-03-22T04:10:50.358530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:25:11.544206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966493+0000","last_change":"2024-03-22T04:10:38.980975+0000","last_active":"2024-03-22T04:10:39.966493+0000","last_peered":"2024-03-22T04:10:39.966493+0000","last_clean":"2024-03-22T04:10:39.966493+0000","last_became_active":"2024-03-22T04:10:38.980594+0000","last_became_peered":"2024-03-22T04:10:38.980594+0000","last_unstale":"2024-03-22T04:10:39.966493+0000","last_undegraded":"2024-03-22T04:10:39.966493+0000","last_fullsized":"2024-03-22T04:10:39.966493+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:10:22.917144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922350+0000","last_change":"2024-03-22T04:10:34.939331+0000","last_active":"2024-03-22T04:10:40.922350+0000","last_peered":"2024-03-22T04:10:40.922350+0000","last_clean":"2024-03-22T04:10:40.922350+0000","last_became_active":"2024-03-22T04:10:34.938998+0000","last_became_peered":"2024-03-22T04:10:34.938998+0000","last_unstale":"2024-03-22T04:10:40.922350+0000","last_undegraded":"2024-03-22T04:10:40.922350+0000","last_fullsized":"2024-03-22T04:10:40.922350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.104177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967555+0000","last_change":"2024-03-22T04:10:33.938816+0000","last_active":"2024-03-22T04:10:39.967555+0000","last_peered":"2024-03-22T04:10:39.967555+0000","last_clean":"2024-03-22T04:10:39.967555+0000","last_became_active":"2024-03-22T04:10:33.937603+0000","last_became_peered":"2024-03-22T04:10:33.937603+0000","last_unstale":"2024-03-22T04:10:39.967555+0000","last_undegraded":"2024-03-22T04:10:39.967555+0000","last_fullsized":"2024-03-22T04:10:39.967555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964868+0000","last_change":"2024-03-22T04:10:38.987284+0000","last_active":"2024-03-22T04:10:39.964868+0000","last_peered":"2024-03-22T04:10:39.964868+0000","last_clean":"2024-03-22T04:10:39.964868+0000","last_became_active":"2024-03-22T04:10:38.986819+0000","last_became_peered":"2024-03-22T04:10:38.986819+0000","last_unstale":"2024-03-22T04:10:39.964868+0000","last_undegraded":"2024-03-22T04:10:39.964868+0000","last_fullsized":"2024-03-22T04:10:39.964868+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:20.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170831+0000","last_change":"2024-03-22T04:10:34.933050+0000","last_active":"2024-03-22T04:10:41.170831+0000","last_peered":"2024-03-22T04:10:41.170831+0000","last_clean":"2024-03-22T04:10:41.170831+0000","last_became_active":"2024-03-22T04:10:34.932756+0000","last_became_peered":"2024-03-22T04:10:34.932756+0000","last_unstale":"2024-03-22T04:10:41.170831+0000","last_undegraded":"2024-03-22T04:10:41.170831+0000","last_fullsized":"2024-03-22T04:10:41.170831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:41.877296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536138+0000","last_change":"2024-03-22T04:10:33.936205+0000","last_active":"2024-03-22T04:10:41.536138+0000","last_peered":"2024-03-22T04:10:41.536138+0000","last_clean":"2024-03-22T04:10:41.536138+0000","last_became_active":"2024-03-22T04:10:33.935904+0000","last_became_peered":"2024-03-22T04:10:33.935904+0000","last_unstale":"2024-03-22T04:10:41.536138+0000","last_undegraded":"2024-03-22T04:10:41.536138+0000","last_fullsized":"2024-03-22T04:10:41.536138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:30.304547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966245+0000","last_change":"2024-03-22T04:10:38.991949+0000","last_active":"2024-03-22T04:10:39.966245+0000","last_peered":"2024-03-22T04:10:39.966245+0000","last_clean":"2024-03-22T04:10:39.966245+0000","last_became_active":"2024-03-22T04:10:38.991459+0000","last_became_peered":"2024-03-22T04:10:38.991459+0000","last_unstale":"2024-03-22T04:10:39.966245+0000","last_undegraded":"2024-03-22T04:10:39.966245+0000","last_fullsized":"2024-03-22T04:10:39.966245+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:44.301347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965035+0000","last_change":"2024-03-22T04:10:34.947409+0000","last_active":"2024-03-22T04:10:39.965035+0000","last_peered":"2024-03-22T04:10:39.965035+0000","last_clean":"2024-03-22T04:10:39.965035+0000","last_became_active":"2024-03-22T04:10:34.947102+0000","last_became_peered":"2024-03-22T04:10:34.947102+0000","last_unstale":"2024-03-22T04:10:39.965035+0000","last_undegraded":"2024-03-22T04:10:39.965035+0000","last_fullsized":"2024-03-22T04:10:39.965035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:12.376047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535663+0000","last_change":"2024-03-22T04:10:33.922034+0000","last_active":"2024-03-22T04:10:41.535663+0000","last_peered":"2024-03-22T04:10:41.535663+0000","last_clean":"2024-03-22T04:10:41.535663+0000","last_became_active":"2024-03-22T04:10:33.921713+0000","last_became_peered":"2024-03-22T04:10:33.921713+0000","last_unstale":"2024-03-22T04:10:41.535663+0000","last_undegraded":"2024-03-22T04:10:41.535663+0000","last_fullsized":"2024-03-22T04:10:41.535663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966606+0000","last_change":"2024-03-22T04:10:38.983919+0000","last_active":"2024-03-22T04:10:39.966606+0000","last_peered":"2024-03-22T04:10:39.966606+0000","last_clean":"2024-03-22T04:10:39.966606+0000","last_became_active":"2024-03-22T04:10:38.983556+0000","last_became_peered":"2024-03-22T04:10:38.983556+0000","last_unstale":"2024-03-22T04:10:39.966606+0000","last_undegraded":"2024-03-22T04:10:39.966606+0000","last_fullsized":"2024-03-22T04:10:39.966606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:12:09.818595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967886+0000","last_change":"2024-03-22T04:10:38.985686+0000","last_active":"2024-03-22T04:10:39.967886+0000","last_peered":"2024-03-22T04:10:39.967886+0000","last_clean":"2024-03-22T04:10:39.967886+0000","last_became_active":"2024-03-22T04:10:38.985411+0000","last_became_peered":"2024-03-22T04:10:38.985411+0000","last_unstale":"2024-03-22T04:10:39.967886+0000","last_undegraded":"2024-03-22T04:10:39.967886+0000","last_fullsized":"2024-03-22T04:10:39.967886+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:36.068872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966752+0000","last_change":"2024-03-22T04:10:34.926900+0000","last_active":"2024-03-22T04:10:39.966752+0000","last_peered":"2024-03-22T04:10:39.966752+0000","last_clean":"2024-03-22T04:10:39.966752+0000","last_became_active":"2024-03-22T04:10:34.926599+0000","last_became_peered":"2024-03-22T04:10:34.926599+0000","last_unstale":"2024-03-22T04:10:39.966752+0000","last_undegraded":"2024-03-22T04:10:39.966752+0000","last_fullsized":"2024-03-22T04:10:39.966752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:21.635048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922002+0000","last_change":"2024-03-22T04:10:33.932457+0000","last_active":"2024-03-22T04:10:40.922002+0000","last_peered":"2024-03-22T04:10:40.922002+0000","last_clean":"2024-03-22T04:10:40.922002+0000","last_became_active":"2024-03-22T04:10:33.932048+0000","last_became_peered":"2024-03-22T04:10:33.932048+0000","last_unstale":"2024-03-22T04:10:40.922002+0000","last_undegraded":"2024-03-22T04:10:40.922002+0000","last_fullsized":"2024-03-22T04:10:40.922002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:30.304547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922792+0000","last_change":"2024-03-22T04:10:38.985629+0000","last_active":"2024-03-22T04:10:40.922792+0000","last_peered":"2024-03-22T04:10:40.922792+0000","last_clean":"2024-03-22T04:10:40.922792+0000","last_became_active":"2024-03-22T04:10:38.983094+0000","last_became_peered":"2024-03-22T04:10:38.983094+0000","last_unstale":"2024-03-22T04:10:40.922792+0000","last_undegraded":"2024-03-22T04:10:40.922792+0000","last_fullsized":"2024-03-22T04:10:40.922792+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:21:46.389313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169810+0000","last_change":"2024-03-22T04:10:34.925982+0000","last_active":"2024-03-22T04:10:41.169810+0000","last_peered":"2024-03-22T04:10:41.169810+0000","last_clean":"2024-03-22T04:10:41.169810+0000","last_became_active":"2024-03-22T04:10:34.925676+0000","last_became_peered":"2024-03-22T04:10:34.925676+0000","last_unstale":"2024-03-22T04:10:41.169810+0000","last_undegraded":"2024-03-22T04:10:41.169810+0000","last_fullsized":"2024-03-22T04:10:41.169810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:08.063872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536054+0000","last_change":"2024-03-22T04:10:33.933448+0000","last_active":"2024-03-22T04:10:41.536054+0000","last_peered":"2024-03-22T04:10:41.536054+0000","last_clean":"2024-03-22T04:10:41.536054+0000","last_became_active":"2024-03-22T04:10:33.933025+0000","last_became_peered":"2024-03-22T04:10:33.933025+0000","last_unstale":"2024-03-22T04:10:41.536054+0000","last_undegraded":"2024-03-22T04:10:41.536054+0000","last_fullsized":"2024-03-22T04:10:41.536054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922138+0000","last_change":"2024-03-22T04:10:33.935607+0000","last_active":"2024-03-22T04:10:40.922138+0000","last_peered":"2024-03-22T04:10:40.922138+0000","last_clean":"2024-03-22T04:10:40.922138+0000","last_became_active":"2024-03-22T04:10:33.932086+0000","last_became_peered":"2024-03-22T04:10:33.932086+0000","last_unstale":"2024-03-22T04:10:40.922138+0000","last_undegraded":"2024-03-22T04:10:40.922138+0000","last_fullsized":"2024-03-22T04:10:40.922138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:49.224368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967401+0000","last_change":"2024-03-22T04:10:34.934494+0000","last_active":"2024-03-22T04:10:39.967401+0000","last_peered":"2024-03-22T04:10:39.967401+0000","last_clean":"2024-03-22T04:10:39.967401+0000","last_became_active":"2024-03-22T04:10:34.934210+0000","last_became_peered":"2024-03-22T04:10:34.934210+0000","last_unstale":"2024-03-22T04:10:39.967401+0000","last_undegraded":"2024-03-22T04:10:39.967401+0000","last_fullsized":"2024-03-22T04:10:39.967401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:48:09.157711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967687+0000","last_change":"2024-03-22T04:10:38.988089+0000","last_active":"2024-03-22T04:10:39.967687+0000","last_peered":"2024-03-22T04:10:39.967687+0000","last_clean":"2024-03-22T04:10:39.967687+0000","last_became_active":"2024-03-22T04:10:38.987643+0000","last_became_peered":"2024-03-22T04:10:38.987643+0000","last_unstale":"2024-03-22T04:10:39.967687+0000","last_undegraded":"2024-03-22T04:10:39.967687+0000","last_fullsized":"2024-03-22T04:10:39.967687+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:35.902267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170109+0000","last_change":"2024-03-22T04:10:33.933057+0000","last_active":"2024-03-22T04:10:41.170109+0000","last_peered":"2024-03-22T04:10:41.170109+0000","last_clean":"2024-03-22T04:10:41.170109+0000","last_became_active":"2024-03-22T04:10:33.932776+0000","last_became_peered":"2024-03-22T04:10:33.932776+0000","last_unstale":"2024-03-22T04:10:41.170109+0000","last_undegraded":"2024-03-22T04:10:41.170109+0000","last_fullsized":"2024-03-22T04:10:41.170109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535263+0000","last_change":"2024-03-22T04:10:34.935154+0000","last_active":"2024-03-22T04:10:41.535263+0000","last_peered":"2024-03-22T04:10:41.535263+0000","last_clean":"2024-03-22T04:10:41.535263+0000","last_became_active":"2024-03-22T04:10:34.934888+0000","last_became_peered":"2024-03-22T04:10:34.934888+0000","last_unstale":"2024-03-22T04:10:41.535263+0000","last_undegraded":"2024-03-22T04:10:41.535263+0000","last_fullsized":"2024-03-22T04:10:41.535263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:01.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967502+0000","last_change":"2024-03-22T04:10:38.988529+0000","last_active":"2024-03-22T04:10:39.967502+0000","last_peered":"2024-03-22T04:10:39.967502+0000","last_clean":"2024-03-22T04:10:39.967502+0000","last_became_active":"2024-03-22T04:10:38.987847+0000","last_became_peered":"2024-03-22T04:10:38.987847+0000","last_unstale":"2024-03-22T04:10:39.967502+0000","last_undegraded":"2024-03-22T04:10:39.967502+0000","last_fullsized":"2024-03-22T04:10:39.967502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:12:05.623030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964372+0000","last_change":"2024-03-22T04:10:34.930193+0000","last_active":"2024-03-22T04:10:39.964372+0000","last_peered":"2024-03-22T04:10:39.964372+0000","last_clean":"2024-03-22T04:10:39.964372+0000","last_became_active":"2024-03-22T04:10:34.929910+0000","last_became_peered":"2024-03-22T04:10:34.929910+0000","last_unstale":"2024-03-22T04:10:39.964372+0000","last_undegraded":"2024-03-22T04:10:39.964372+0000","last_fullsized":"2024-03-22T04:10:39.964372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966601+0000","last_change":"2024-03-22T04:10:34.951954+0000","last_active":"2024-03-22T04:10:39.966601+0000","last_peered":"2024-03-22T04:10:39.966601+0000","last_clean":"2024-03-22T04:10:39.966601+0000","last_became_active":"2024-03-22T04:10:34.951651+0000","last_became_peered":"2024-03-22T04:10:34.951651+0000","last_unstale":"2024-03-22T04:10:39.966601+0000","last_undegraded":"2024-03-22T04:10:39.966601+0000","last_fullsized":"2024-03-22T04:10:39.966601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922008+0000","last_change":"2024-03-22T04:10:34.927015+0000","last_active":"2024-03-22T04:10:40.922008+0000","last_peered":"2024-03-22T04:10:40.922008+0000","last_clean":"2024-03-22T04:10:40.922008+0000","last_became_active":"2024-03-22T04:10:34.926098+0000","last_became_peered":"2024-03-22T04:10:34.926098+0000","last_unstale":"2024-03-22T04:10:40.922008+0000","last_undegraded":"2024-03-22T04:10:40.922008+0000","last_fullsized":"2024-03-22T04:10:40.922008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:05:19.773331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170027+0000","last_change":"2024-03-22T04:10:34.950558+0000","last_active":"2024-03-22T04:10:41.170027+0000","last_peered":"2024-03-22T04:10:41.170027+0000","last_clean":"2024-03-22T04:10:41.170027+0000","last_became_active":"2024-03-22T04:10:34.950242+0000","last_became_peered":"2024-03-22T04:10:34.950242+0000","last_unstale":"2024-03-22T04:10:41.170027+0000","last_undegraded":"2024-03-22T04:10:41.170027+0000","last_fullsized":"2024-03-22T04:10:41.170027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:11.105591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536489+0000","last_change":"2024-03-22T04:10:34.928518+0000","last_active":"2024-03-22T04:10:41.536489+0000","last_peered":"2024-03-22T04:10:41.536489+0000","last_clean":"2024-03-22T04:10:41.536489+0000","last_became_active":"2024-03-22T04:10:34.928228+0000","last_became_peered":"2024-03-22T04:10:34.928228+0000","last_unstale":"2024-03-22T04:10:41.536489+0000","last_undegraded":"2024-03-22T04:10:41.536489+0000","last_fullsized":"2024-03-22T04:10:41.536489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:41.449121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536201+0000","last_change":"2024-03-22T04:10:34.951914+0000","last_active":"2024-03-22T04:10:41.536201+0000","last_peered":"2024-03-22T04:10:41.536201+0000","last_clean":"2024-03-22T04:10:41.536201+0000","last_became_active":"2024-03-22T04:10:34.951608+0000","last_became_peered":"2024-03-22T04:10:34.951608+0000","last_unstale":"2024-03-22T04:10:41.536201+0000","last_undegraded":"2024-03-22T04:10:41.536201+0000","last_fullsized":"2024-03-22T04:10:41.536201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:18.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964949+0000","last_change":"2024-03-22T04:10:34.936728+0000","last_active":"2024-03-22T04:10:39.964949+0000","last_peered":"2024-03-22T04:10:39.964949+0000","last_clean":"2024-03-22T04:10:39.964949+0000","last_became_active":"2024-03-22T04:10:34.935969+0000","last_became_peered":"2024-03-22T04:10:34.935969+0000","last_unstale":"2024-03-22T04:10:39.964949+0000","last_undegraded":"2024-03-22T04:10:39.964949+0000","last_fullsized":"2024-03-22T04:10:39.964949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:48.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170702+0000","last_change":"2024-03-22T04:10:34.943850+0000","last_active":"2024-03-22T04:10:41.170702+0000","last_peered":"2024-03-22T04:10:41.170702+0000","last_clean":"2024-03-22T04:10:41.170702+0000","last_became_active":"2024-03-22T04:10:34.941930+0000","last_became_peered":"2024-03-22T04:10:34.941930+0000","last_unstale":"2024-03-22T04:10:41.170702+0000","last_undegraded":"2024-03-22T04:10:41.170702+0000","last_fullsized":"2024-03-22T04:10:41.170702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:55.025923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921894+0000","last_change":"2024-03-22T04:10:34.946337+0000","last_active":"2024-03-22T04:10:40.921894+0000","last_peered":"2024-03-22T04:10:40.921894+0000","last_clean":"2024-03-22T04:10:40.921894+0000","last_became_active":"2024-03-22T04:10:34.946083+0000","last_became_peered":"2024-03-22T04:10:34.946083+0000","last_unstale":"2024-03-22T04:10:40.921894+0000","last_undegraded":"2024-03-22T04:10:40.921894+0000","last_fullsized":"2024-03-22T04:10:40.921894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:15:33.976508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170005+0000","last_change":"2024-03-22T04:10:34.939061+0000","last_active":"2024-03-22T04:10:41.170005+0000","last_peered":"2024-03-22T04:10:41.170005+0000","last_clean":"2024-03-22T04:10:41.170005+0000","last_became_active":"2024-03-22T04:10:34.938566+0000","last_became_peered":"2024-03-22T04:10:34.938566+0000","last_unstale":"2024-03-22T04:10:41.170005+0000","last_undegraded":"2024-03-22T04:10:41.170005+0000","last_fullsized":"2024-03-22T04:10:41.170005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:44:57.681730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535168+0000","last_change":"2024-03-22T04:10:34.933002+0000","last_active":"2024-03-22T04:10:41.535168+0000","last_peered":"2024-03-22T04:10:41.535168+0000","last_clean":"2024-03-22T04:10:41.535168+0000","last_became_active":"2024-03-22T04:10:34.932710+0000","last_became_peered":"2024-03-22T04:10:34.932710+0000","last_unstale":"2024-03-22T04:10:41.535168+0000","last_undegraded":"2024-03-22T04:10:41.535168+0000","last_fullsized":"2024-03-22T04:10:41.535168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967328+0000","last_change":"2024-03-22T04:10:34.926419+0000","last_active":"2024-03-22T04:10:39.967328+0000","last_peered":"2024-03-22T04:10:39.967328+0000","last_clean":"2024-03-22T04:10:39.967328+0000","last_became_active":"2024-03-22T04:10:34.924952+0000","last_became_peered":"2024-03-22T04:10:34.924952+0000","last_unstale":"2024-03-22T04:10:39.967328+0000","last_undegraded":"2024-03-22T04:10:39.967328+0000","last_fullsized":"2024-03-22T04:10:39.967328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:30:41.171941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535087+0000","last_change":"2024-03-22T04:10:34.939886+0000","last_active":"2024-03-22T04:10:41.535087+0000","last_peered":"2024-03-22T04:10:41.535087+0000","last_clean":"2024-03-22T04:10:41.535087+0000","last_became_active":"2024-03-22T04:10:34.939633+0000","last_became_peered":"2024-03-22T04:10:34.939633+0000","last_unstale":"2024-03-22T04:10:41.535087+0000","last_undegraded":"2024-03-22T04:10:41.535087+0000","last_fullsized":"2024-03-22T04:10:41.535087+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:41.877296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922616+0000","last_change":"2024-03-22T04:10:34.930917+0000","last_active":"2024-03-22T04:10:40.922616+0000","last_peered":"2024-03-22T04:10:40.922616+0000","last_clean":"2024-03-22T04:10:40.922616+0000","last_became_active":"2024-03-22T04:10:34.930658+0000","last_became_peered":"2024-03-22T04:10:34.930658+0000","last_unstale":"2024-03-22T04:10:40.922616+0000","last_undegraded":"2024-03-22T04:10:40.922616+0000","last_fullsized":"2024-03-22T04:10:40.922616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:40:34.158803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967631+0000","last_change":"2024-03-22T04:10:34.953215+0000","last_active":"2024-03-22T04:10:39.967631+0000","last_peered":"2024-03-22T04:10:39.967631+0000","last_clean":"2024-03-22T04:10:39.967631+0000","last_became_active":"2024-03-22T04:10:34.952953+0000","last_became_peered":"2024-03-22T04:10:34.952953+0000","last_unstale":"2024-03-22T04:10:39.967631+0000","last_undegraded":"2024-03-22T04:10:39.967631+0000","last_fullsized":"2024-03-22T04:10:39.967631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922902+0000","last_change":"2024-03-22T04:10:34.937971+0000","last_active":"2024-03-22T04:10:40.922902+0000","last_peered":"2024-03-22T04:10:40.922902+0000","last_clean":"2024-03-22T04:10:40.922902+0000","last_became_active":"2024-03-22T04:10:34.937744+0000","last_became_peered":"2024-03-22T04:10:34.937744+0000","last_unstale":"2024-03-22T04:10:40.922902+0000","last_undegraded":"2024-03-22T04:10:40.922902+0000","last_fullsized":"2024-03-22T04:10:40.922902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:17.641505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921877+0000","last_change":"2024-03-22T04:10:34.929076+0000","last_active":"2024-03-22T04:10:40.921877+0000","last_peered":"2024-03-22T04:10:40.921877+0000","last_clean":"2024-03-22T04:10:40.921877+0000","last_became_active":"2024-03-22T04:10:34.928742+0000","last_became_peered":"2024-03-22T04:10:34.928742+0000","last_unstale":"2024-03-22T04:10:40.921877+0000","last_undegraded":"2024-03-22T04:10:40.921877+0000","last_fullsized":"2024-03-22T04:10:40.921877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:04.974724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169693+0000","last_change":"2024-03-22T04:10:34.935025+0000","last_active":"2024-03-22T04:10:41.169693+0000","last_peered":"2024-03-22T04:10:41.169693+0000","last_clean":"2024-03-22T04:10:41.169693+0000","last_became_active":"2024-03-22T04:10:34.934504+0000","last_became_peered":"2024-03-22T04:10:34.934504+0000","last_unstale":"2024-03-22T04:10:41.169693+0000","last_undegraded":"2024-03-22T04:10:41.169693+0000","last_fullsized":"2024-03-22T04:10:41.169693+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:18.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536006+0000","last_change":"2024-03-22T04:10:34.952611+0000","last_active":"2024-03-22T04:10:41.536006+0000","last_peered":"2024-03-22T04:10:41.536006+0000","last_clean":"2024-03-22T04:10:41.536006+0000","last_became_active":"2024-03-22T04:10:34.952289+0000","last_became_peered":"2024-03-22T04:10:34.952289+0000","last_unstale":"2024-03-22T04:10:41.536006+0000","last_undegraded":"2024-03-22T04:10:41.536006+0000","last_fullsized":"2024-03-22T04:10:41.536006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:44.874307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536345+0000","last_change":"2024-03-22T04:10:34.938439+0000","last_active":"2024-03-22T04:10:41.536345+0000","last_peered":"2024-03-22T04:10:41.536345+0000","last_clean":"2024-03-22T04:10:41.536345+0000","last_became_active":"2024-03-22T04:10:34.938160+0000","last_became_peered":"2024-03-22T04:10:34.938160+0000","last_unstale":"2024-03-22T04:10:41.536345+0000","last_undegraded":"2024-03-22T04:10:41.536345+0000","last_fullsized":"2024-03-22T04:10:41.536345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:40.255817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535047+0000","last_change":"2024-03-22T04:10:34.936432+0000","last_active":"2024-03-22T04:10:41.535047+0000","last_peered":"2024-03-22T04:10:41.535047+0000","last_clean":"2024-03-22T04:10:41.535047+0000","last_became_active":"2024-03-22T04:10:34.936175+0000","last_became_peered":"2024-03-22T04:10:34.936175+0000","last_unstale":"2024-03-22T04:10:41.535047+0000","last_undegraded":"2024-03-22T04:10:41.535047+0000","last_fullsized":"2024-03-22T04:10:41.535047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:34.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535725+0000","last_change":"2024-03-22T04:10:34.947651+0000","last_active":"2024-03-22T04:10:41.535725+0000","last_peered":"2024-03-22T04:10:41.535725+0000","last_clean":"2024-03-22T04:10:41.535725+0000","last_became_active":"2024-03-22T04:10:34.947262+0000","last_became_peered":"2024-03-22T04:10:34.947262+0000","last_unstale":"2024-03-22T04:10:41.535725+0000","last_undegraded":"2024-03-22T04:10:41.535725+0000","last_fullsized":"2024-03-22T04:10:41.535725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:16.846964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536037+0000","last_change":"2024-03-22T04:10:34.929968+0000","last_active":"2024-03-22T04:10:41.536037+0000","last_peered":"2024-03-22T04:10:41.536037+0000","last_clean":"2024-03-22T04:10:41.536037+0000","last_became_active":"2024-03-22T04:10:34.929650+0000","last_became_peered":"2024-03-22T04:10:34.929650+0000","last_unstale":"2024-03-22T04:10:41.536037+0000","last_undegraded":"2024-03-22T04:10:41.536037+0000","last_fullsized":"2024-03-22T04:10:41.536037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:04.974724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535701+0000","last_change":"2024-03-22T04:10:34.949078+0000","last_active":"2024-03-22T04:10:41.535701+0000","last_peered":"2024-03-22T04:10:41.535701+0000","last_clean":"2024-03-22T04:10:41.535701+0000","last_became_active":"2024-03-22T04:10:34.948775+0000","last_became_peered":"2024-03-22T04:10:34.948775+0000","last_unstale":"2024-03-22T04:10:41.535701+0000","last_undegraded":"2024-03-22T04:10:41.535701+0000","last_fullsized":"2024-03-22T04:10:41.535701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:48.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169713+0000","last_change":"2024-03-22T04:10:34.931467+0000","last_active":"2024-03-22T04:10:41.169713+0000","last_peered":"2024-03-22T04:10:41.169713+0000","last_clean":"2024-03-22T04:10:41.169713+0000","last_became_active":"2024-03-22T04:10:34.931119+0000","last_became_peered":"2024-03-22T04:10:34.931119+0000","last_unstale":"2024-03-22T04:10:41.169713+0000","last_undegraded":"2024-03-22T04:10:41.169713+0000","last_fullsized":"2024-03-22T04:10:41.169713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:18:49.597662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534582+0000","last_change":"2024-03-22T04:10:34.925425+0000","last_active":"2024-03-22T04:10:41.534582+0000","last_peered":"2024-03-22T04:10:41.534582+0000","last_clean":"2024-03-22T04:10:41.534582+0000","last_became_active":"2024-03-22T04:10:34.925151+0000","last_became_peered":"2024-03-22T04:10:34.925151+0000","last_unstale":"2024-03-22T04:10:41.534582+0000","last_undegraded":"2024-03-22T04:10:41.534582+0000","last_fullsized":"2024-03-22T04:10:41.534582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:11.105591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964686+0000","last_change":"2024-03-22T04:10:34.926711+0000","last_active":"2024-03-22T04:10:39.964686+0000","last_peered":"2024-03-22T04:10:39.964686+0000","last_clean":"2024-03-22T04:10:39.964686+0000","last_became_active":"2024-03-22T04:10:34.925765+0000","last_became_peered":"2024-03-22T04:10:34.925765+0000","last_unstale":"2024-03-22T04:10:39.964686+0000","last_undegraded":"2024-03-22T04:10:39.964686+0000","last_fullsized":"2024-03-22T04:10:39.964686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534972+0000","last_change":"2024-03-22T04:10:34.932009+0000","last_active":"2024-03-22T04:10:41.534972+0000","last_peered":"2024-03-22T04:10:41.534972+0000","last_clean":"2024-03-22T04:10:41.534972+0000","last_became_active":"2024-03-22T04:10:34.931692+0000","last_became_peered":"2024-03-22T04:10:34.931692+0000","last_unstale":"2024-03-22T04:10:41.534972+0000","last_undegraded":"2024-03-22T04:10:41.534972+0000","last_fullsized":"2024-03-22T04:10:41.534972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:08.063872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964556+0000","last_change":"2024-03-22T04:10:34.921659+0000","last_active":"2024-03-22T04:10:39.964556+0000","last_peered":"2024-03-22T04:10:39.964556+0000","last_clean":"2024-03-22T04:10:39.964556+0000","last_became_active":"2024-03-22T04:10:34.921420+0000","last_became_peered":"2024-03-22T04:10:34.921420+0000","last_unstale":"2024-03-22T04:10:39.964556+0000","last_undegraded":"2024-03-22T04:10:39.964556+0000","last_fullsized":"2024-03-22T04:10:39.964556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:01.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170351+0000","last_change":"2024-03-22T04:10:34.928037+0000","last_active":"2024-03-22T04:10:41.170351+0000","last_peered":"2024-03-22T04:10:41.170351+0000","last_clean":"2024-03-22T04:10:41.170351+0000","last_became_active":"2024-03-22T04:10:34.927746+0000","last_became_peered":"2024-03-22T04:10:34.927746+0000","last_unstale":"2024-03-22T04:10:41.170351+0000","last_undegraded":"2024-03-22T04:10:41.170351+0000","last_fullsized":"2024-03-22T04:10:41.170351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:29:34.250954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170416+0000","last_change":"2024-03-22T04:10:34.931962+0000","last_active":"2024-03-22T04:10:41.170416+0000","last_peered":"2024-03-22T04:10:41.170416+0000","last_clean":"2024-03-22T04:10:41.170416+0000","last_became_active":"2024-03-22T04:10:34.931736+0000","last_became_peered":"2024-03-22T04:10:34.931736+0000","last_unstale":"2024-03-22T04:10:41.170416+0000","last_undegraded":"2024-03-22T04:10:41.170416+0000","last_fullsized":"2024-03-22T04:10:41.170416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:16.846964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535903+0000","last_change":"2024-03-22T04:10:34.946756+0000","last_active":"2024-03-22T04:10:41.535903+0000","last_peered":"2024-03-22T04:10:41.535903+0000","last_clean":"2024-03-22T04:10:41.535903+0000","last_became_active":"2024-03-22T04:10:34.946501+0000","last_became_peered":"2024-03-22T04:10:34.946501+0000","last_unstale":"2024-03-22T04:10:41.535903+0000","last_undegraded":"2024-03-22T04:10:41.535903+0000","last_fullsized":"2024-03-22T04:10:41.535903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:02:47.197474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:11:00.345 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T04:11:00.345 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-22T04:11:00.345 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-22T04:11:00.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-22T04:11:00.346 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-22T04:11:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.512+0000 7f2092400640 1 Processor -- start 2024-03-22T04:11:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.512+0000 7f2092400640 1 -- start start 2024-03-22T04:11:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f2092400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f208c15b3f0 0x7f208c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f2092400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 0x7f208c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f2092400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f208c071920 con 0x7f208c0b30f0 2024-03-22T04:11:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f2092400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f208c0716d0 con 0x7f208c15b3f0 2024-03-22T04:11:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f2092400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f208c06d990 con 0x7f208c0cc3e0 2024-03-22T04:11:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f208c15b3f0 0x7f208c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f208c0b30f0 0x7f208c0ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57462/0 (socket says 172.21.15.86:57462) 2024-03-22T04:11:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f208c15b3f0 0x7f208c0729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55888/0 (socket says 172.21.15.86:55888) 2024-03-22T04:11:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208be00640 1 -- 172.21.15.86:0/2551565745 learned_addr learned my addr 172.21.15.86:0/2551565745 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208aa00640 1 --2- 172.21.15.86:0/2551565745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 0x7f208c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208b400640 1 -- 172.21.15.86:0/2551565745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 msgr2=0x7f208c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208b400640 1 --2- 172.21.15.86:0/2551565745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 0x7f208c072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208b400640 1 -- 172.21.15.86:0/2551565745 >> v1:172.21.15.86:6789/0 conn(0x7f208c0b30f0 legacy=0x7f208c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.516+0000 7f208b400640 1 -- 172.21.15.86:0/2551565745 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c071c00 con 0x7f208c15b3f0 2024-03-22T04:11:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f208b400640 1 --2- 172.21.15.86:0/2551565745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c0729c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f207c009d90 tx=0x7f207c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf0e2e437bfa396 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f208a000640 1 -- 172.21.15.86:0/2551565745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c0b2070 con 0x7f208c15b3f0 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f208a000640 1 -- 172.21.15.86:0/2551565745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c0a5890 con 0x7f208c15b3f0 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f208a000640 1 -- 172.21.15.86:0/2551565745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c0a5c60 con 0x7f208c15b3f0 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- 172.21.15.86:0/2551565745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 msgr2=0x7f208c0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 --2- 172.21.15.86:0/2551565745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c0729c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f207c009d90 tx=0x7f207c01c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- 172.21.15.86:0/2551565745 shutdown_connections 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 --2- 172.21.15.86:0/2551565745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 0x7f208c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 --2- 172.21.15.86:0/2551565745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c0729c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- 172.21.15.86:0/2551565745 >> 172.21.15.86:0/2551565745 conn(0x7f208c0b1df0 msgr2=0x7f208c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- 172.21.15.86:0/2551565745 shutdown_connections 2024-03-22T04:11:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- 172.21.15.86:0/2551565745 wait complete. 2024-03-22T04:11:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 Processor -- start 2024-03-22T04:11:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- start start 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b30f0 0x7f208c154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 0x7f208c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f208c168580 con 0x7f208c15b3f0 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f208c168400 con 0x7f208c0b30f0 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f2092400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f208c168700 con 0x7f208c0cc3e0 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f208be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f208be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c1567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55894/0 (socket says 172.21.15.86:55894) 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.520+0000 7f208be00640 1 -- 172.21.15.86:0/3357156330 learned_addr learned my addr 172.21.15.86:0/3357156330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208aa00640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 0x7f208c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208b400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b30f0 0x7f208c154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208be00640 1 -- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 msgr2=0x7f208c09dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208be00640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 0x7f208c09dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208be00640 1 -- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b30f0 msgr2=0x7f208c154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208be00640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b30f0 0x7f208c154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208be00640 1 -- 172.21.15.86:0/3357156330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c1551c0 con 0x7f208c15b3f0 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208b400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b30f0 0x7f208c154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f208be00640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c1567d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f208400c3c0 tx=0x7f208400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2083400640 1 -- 172.21.15.86:0/3357156330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084017070 con 0x7f208c15b3f0 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2083400640 1 -- 172.21.15.86:0/3357156330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2084005cf0 con 0x7f208c15b3f0 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208c1554a0 con 0x7f208c15b3f0 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2083400640 1 -- 172.21.15.86:0/3357156330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20840072f0 con 0x7f208c15b3f0 2024-03-22T04:11:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208c155a60 con 0x7f208c15b3f0 2024-03-22T04:11:00.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f208c068650 con 0x7f208c15b3f0 2024-03-22T04:11:00.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2083400640 1 -- 172.21.15.86:0/3357156330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f208401a070 con 0x7f208c15b3f0 2024-03-22T04:11:00.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2083400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f206c067b20 0x7f206c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.524+0000 7f2083400640 1 -- 172.21.15.86:0/3357156330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2084085480 con 0x7f208c15b3f0 2024-03-22T04:11:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.528+0000 7f208b400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f206c067b20 0x7f206c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.528+0000 7f208b400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f206c067b20 0x7f206c069fe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f207c0af000 tx=0x7f207c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.528+0000 7f2083400640 1 -- 172.21.15.86:0/3357156330 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2084055500 con 0x7f208c15b3f0 2024-03-22T04:11:00.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.720+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f208c0b07d0 con 0x7f208c15b3f0 2024-03-22T04:11:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.724+0000 7f2083400640 1 -- 172.21.15.86:0/3357156330 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+167120 (secure 0 0 0) 0x7f20840595a0 con 0x7f208c15b3f0 2024-03-22T04:11:00.730 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f206c067b20 msgr2=0x7f206c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f206c067b20 0x7f206c069fe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f207c0af000 tx=0x7f207c0a7070 comp rx=0 tx=0).stop 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 msgr2=0x7f208c1567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c1567d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f208400c3c0 tx=0x7f208400c880 comp rx=0 tx=0).stop 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 shutdown_connections 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f206c067b20 0x7f206c069fe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f208c15b3f0 0x7f208c1567d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc3e0 0x7f208c09dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 --2- 172.21.15.86:0/3357156330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b30f0 0x7f208c154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 >> 172.21.15.86:0/3357156330 conn(0x7f208c0b1df0 msgr2=0x7f208c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 shutdown_connections 2024-03-22T04:11:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.728+0000 7f2092400640 1 -- 172.21.15.86:0/3357156330 wait complete. 2024-03-22T04:11:00.750 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1135494009},{"type":"v1","addr":"172.21.15.86:6833","nonce":1135494009}]},"active_addr":"172.21.15.86:6833/1135494009","active_change":"2024-03-22T04:10:13.778210+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3232985974}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1628537781}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2198513763}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":577752599}]}]} 2024-03-22T04:11:00.754 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-22T04:11:00.755 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-22T04:11:00.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9189e00640 1 Processor -- start 2024-03-22T04:11:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9189e00640 1 -- start start 2024-03-22T04:11:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9189e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f91840d0de0 0x7f91840ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9189e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f91840d19a0 0x7f91840cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9189e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91840d11c0 con 0x7f918415fed0 2024-03-22T04:11:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9189e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f91840cece0 con 0x7f91840d19a0 2024-03-22T04:11:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9189e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f91840cefc0 con 0x7f91840d0de0 2024-03-22T04:11:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f917a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f91840d19a0 0x7f91840cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9183400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f918415fed0 0x7f91841602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57472/0 (socket says 172.21.15.86:57472) 2024-03-22T04:11:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9183400640 1 -- 172.21.15.86:0/2264709482 learned_addr learned my addr 172.21.15.86:0/2264709482 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9182a00640 1 --2- 172.21.15.86:0/2264709482 >> v2:172.21.15.196:3300/0 conn(0x7f91840d0de0 0x7f91840ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f917a000640 1 -- 172.21.15.86:0/2264709482 >> v2:172.21.15.196:3300/0 conn(0x7f91840d0de0 msgr2=0x7f91840ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f917a000640 1 --2- 172.21.15.86:0/2264709482 >> v2:172.21.15.196:3300/0 conn(0x7f91840d0de0 0x7f91840ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f917a000640 1 -- 172.21.15.86:0/2264709482 >> v1:172.21.15.86:6789/0 conn(0x7f918415fed0 legacy=0x7f91841602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f917a000640 1 -- 172.21.15.86:0/2264709482 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91840cf2a0 con 0x7f91840d19a0 2024-03-22T04:11:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.888+0000 7f9182000640 1 -- 172.21.15.86:0/2264709482 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4244178272 0 0) 0x7f91840d11c0 con 0x7f918415fed0 2024-03-22T04:11:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f917a000640 1 --2- 172.21.15.86:0/2264709482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91840d19a0 0x7f91840cfe70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f918406f4c0 tx=0x7f916c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e77aa0b6aadd11 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9182000640 1 -- 172.21.15.86:0/2264709482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c0b0070 con 0x7f91840d19a0 2024-03-22T04:11:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9182000640 1 -- 172.21.15.86:0/2264709482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916c0ab600 con 0x7f91840d19a0 2024-03-22T04:11:00.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9182000640 1 -- 172.21.15.86:0/2264709482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c0ab9d0 con 0x7f91840d19a0 2024-03-22T04:11:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9189e00640 1 -- 172.21.15.86:0/2264709482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91840d19a0 msgr2=0x7f91840cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9189e00640 1 --2- 172.21.15.86:0/2264709482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91840d19a0 0x7f91840cfe70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f918406f4c0 tx=0x7f916c01c480 comp rx=0 tx=0).stop 2024-03-22T04:11:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9189e00640 1 -- 172.21.15.86:0/2264709482 shutdown_connections 2024-03-22T04:11:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9189e00640 1 --2- 172.21.15.86:0/2264709482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91840d19a0 0x7f91840cfe70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9189e00640 1 --2- 172.21.15.86:0/2264709482 >> v2:172.21.15.196:3300/0 conn(0x7f91840d0de0 0x7f91840ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9189e00640 1 -- 172.21.15.86:0/2264709482 >> 172.21.15.86:0/2264709482 conn(0x7f918406f910 msgr2=0x7f918406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9189e00640 1 -- 172.21.15.86:0/2264709482 shutdown_connections 2024-03-22T04:11:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.892+0000 7f9189e00640 1 -- 172.21.15.86:0/2264709482 wait complete. 2024-03-22T04:11:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 Processor -- start 2024-03-22T04:11:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 -- start start 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91840d0de0 0x7f91840c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91840d19a0 0x7f9184159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918415fed0 0x7f918415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91840d1f70 con 0x7f918415fed0 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91840d1df0 con 0x7f91840d19a0 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9184071e30 con 0x7f91840d0de0 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918415fed0 0x7f918415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918415fed0 0x7f918415e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55914/0 (socket says 172.21.15.86:55914) 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 -- 172.21.15.86:0/673335261 learned_addr learned my addr 172.21.15.86:0/673335261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f917a000640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91840d19a0 0x7f9184159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9182a00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91840d0de0 0x7f91840c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 -- 172.21.15.86:0/673335261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91840d0de0 msgr2=0x7f91840c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91840d0de0 0x7f91840c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 -- 172.21.15.86:0/673335261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91840d19a0 msgr2=0x7f9184159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91840d19a0 0x7f9184159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 -- 172.21.15.86:0/673335261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9184154450 con 0x7f918415fed0 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9183400640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918415fed0 0x7f918415e9e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f917c00c7a0 tx=0x7f917c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f917be00640 1 -- 172.21.15.86:0/673335261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c018070 con 0x7f918415fed0 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f917be00640 1 -- 172.21.15.86:0/673335261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917c00a640 con 0x7f918415fed0 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9184154730 con 0x7f918415fed0 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f917be00640 1 -- 172.21.15.86:0/673335261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c00aab0 con 0x7f918415fed0 2024-03-22T04:11:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9184154cf0 con 0x7f918415fed0 2024-03-22T04:11:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.896+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9184068650 con 0x7f918415fed0 2024-03-22T04:11:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.900+0000 7f917be00640 1 -- 172.21.15.86:0/673335261 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f917c01b070 con 0x7f918415fed0 2024-03-22T04:11:00.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.900+0000 7f917be00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9160067b20 0x7f9160069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:00.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.900+0000 7f917be00640 1 -- 172.21.15.86:0/673335261 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f917c0858e0 con 0x7f918415fed0 2024-03-22T04:11:00.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.900+0000 7f9182a00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9160067b20 0x7f9160069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:00.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.900+0000 7f9182a00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9160067b20 0x7f9160069fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9174009ac0 tx=0x7f9174009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:00.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:00.904+0000 7f917be00640 1 -- 172.21.15.86:0/673335261 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f917c055830 con 0x7f918415fed0 2024-03-22T04:11:01.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9184166be0 con 0x7f918415fed0 2024-03-22T04:11:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f917be00640 1 -- 172.21.15.86:0/673335261 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7f917c0598d0 con 0x7f918415fed0 2024-03-22T04:11:01.067 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:01.068 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9160067b20 msgr2=0x7f9160069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9160067b20 0x7f9160069fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9174009ac0 tx=0x7f9174009190 comp rx=0 tx=0).stop 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918415fed0 msgr2=0x7f918415e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918415fed0 0x7f918415e9e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f917c00c7a0 tx=0x7f917c00cc60 comp rx=0 tx=0).stop 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 shutdown_connections 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9160067b20 0x7f9160069fe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91840d19a0 0x7f9184159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918415fed0 0x7f918415e9e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 --2- 172.21.15.86:0/673335261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91840d0de0 0x7f91840c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 >> 172.21.15.86:0/673335261 conn(0x7f918406f910 msgr2=0x7f918415a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 shutdown_connections 2024-03-22T04:11:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.064+0000 7f9189e00640 1 -- 172.21.15.86:0/673335261 wait complete. 2024-03-22T04:11:01.088 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-22T04:11:01.088 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:01.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f5346400640 1 Processor -- start 2024-03-22T04:11:01.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f5346400640 1 -- start start 2024-03-22T04:11:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f5346400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f534015e630 0x7f53400cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f5346400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006e2f0 0x7f53400d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f5346400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f534006cc80 0x7f534006d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f5346400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f53401623d0 con 0x7f534006cc80 2024-03-22T04:11:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f5346400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f53401620f0 con 0x7f534015e630 2024-03-22T04:11:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f5346400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f534015ec10 con 0x7f534006e2f0 2024-03-22T04:11:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f534006cc80 0x7f534006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f534006cc80 0x7f534006d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55918/0 (socket says 172.21.15.86:55918) 2024-03-22T04:11:01.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533fe00640 1 -- 172.21.15.86:0/466252521 learned_addr learned my addr 172.21.15.86:0/466252521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533ea00640 1 --2- 172.21.15.86:0/466252521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006e2f0 0x7f53400d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533f400640 1 --2- 172.21.15.86:0/466252521 >> v2:172.21.15.196:3300/0 conn(0x7f534015e630 0x7f53400cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533fe00640 1 -- 172.21.15.86:0/466252521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006e2f0 msgr2=0x7f53400d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533fe00640 1 --2- 172.21.15.86:0/466252521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006e2f0 0x7f53400d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533fe00640 1 -- 172.21.15.86:0/466252521 >> v2:172.21.15.196:3300/0 conn(0x7f534015e630 msgr2=0x7f53400cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533fe00640 1 --2- 172.21.15.86:0/466252521 >> v2:172.21.15.196:3300/0 conn(0x7f534015e630 0x7f53400cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.220+0000 7f533fe00640 1 -- 172.21.15.86:0/466252521 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534015eef0 con 0x7f534006cc80 2024-03-22T04:11:01.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f533fe00640 1 --2- 172.21.15.86:0/466252521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006cc80 0x7f534006d110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5340072430 tx=0x7f533801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226059333d7f884d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f533e000640 1 -- 172.21.15.86:0/466252521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53380b0070 con 0x7f534006cc80 2024-03-22T04:11:01.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f533e000640 1 -- 172.21.15.86:0/466252521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53380ab600 con 0x7f534006cc80 2024-03-22T04:11:01.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f533e000640 1 -- 172.21.15.86:0/466252521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53380ab9d0 con 0x7f534006cc80 2024-03-22T04:11:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 -- 172.21.15.86:0/466252521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006cc80 msgr2=0x7f534006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 --2- 172.21.15.86:0/466252521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006cc80 0x7f534006d110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5340072430 tx=0x7f533801c520 comp rx=0 tx=0).stop 2024-03-22T04:11:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 -- 172.21.15.86:0/466252521 shutdown_connections 2024-03-22T04:11:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 --2- 172.21.15.86:0/466252521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006cc80 0x7f534006d110 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 --2- 172.21.15.86:0/466252521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006e2f0 0x7f53400d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 --2- 172.21.15.86:0/466252521 >> v2:172.21.15.196:3300/0 conn(0x7f534015e630 0x7f53400cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 -- 172.21.15.86:0/466252521 >> 172.21.15.86:0/466252521 conn(0x7f534006f910 msgr2=0x7f534006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 -- 172.21.15.86:0/466252521 shutdown_connections 2024-03-22T04:11:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.224+0000 7f5346400640 1 -- 172.21.15.86:0/466252521 wait complete. 2024-03-22T04:11:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 Processor -- start 2024-03-22T04:11:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 -- start start 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006cc80 0x7f53400b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006e2f0 0x7f534015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534015e630 0x7f53400b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f534015f9d0 con 0x7f534006e2f0 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53400d1f60 con 0x7f534015e630 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53400cfaf0 con 0x7f534006cc80 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006e2f0 0x7f534015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006e2f0 0x7f534015b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55932/0 (socket says 172.21.15.86:55932) 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 -- 172.21.15.86:0/3486773275 learned_addr learned my addr 172.21.15.86:0/3486773275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533f400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006cc80 0x7f53400b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533fe00640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534015e630 0x7f53400b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 -- 172.21.15.86:0/3486773275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006cc80 msgr2=0x7f53400b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006cc80 0x7f53400b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 -- 172.21.15.86:0/3486773275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534015e630 msgr2=0x7f53400b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534015e630 0x7f53400b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 -- 172.21.15.86:0/3486773275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5340149260 con 0x7f534006e2f0 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f533ea00640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006e2f0 0x7f534015b130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f532c00e8a0 tx=0x7f532c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5337400640 1 -- 172.21.15.86:0/3486773275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c017070 con 0x7f534006e2f0 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53400b6e30 con 0x7f534006e2f0 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5337400640 1 -- 172.21.15.86:0/3486773275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532c00ee90 con 0x7f534006e2f0 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5337400640 1 -- 172.21.15.86:0/3486773275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c00cb80 con 0x7f534006e2f0 2024-03-22T04:11:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53400b73f0 con 0x7f534006e2f0 2024-03-22T04:11:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.228+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5340068650 con 0x7f534006e2f0 2024-03-22T04:11:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.232+0000 7f5337400640 1 -- 172.21.15.86:0/3486773275 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f532c01a070 con 0x7f534006e2f0 2024-03-22T04:11:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.232+0000 7f5337400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5320067b50 0x7f532006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.232+0000 7f5337400640 1 -- 172.21.15.86:0/3486773275 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f532c0860c0 con 0x7f534006e2f0 2024-03-22T04:11:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.232+0000 7f533f400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5320067b50 0x7f532006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.232+0000 7f533f400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5320067b50 0x7f532006a010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53400c5bf0 tx=0x7f533000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.236+0000 7f5337400640 1 -- 172.21.15.86:0/3486773275 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f532c0560a0 con 0x7f534006e2f0 2024-03-22T04:11:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f534006e980 con 0x7f534006e2f0 2024-03-22T04:11:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5337400640 1 -- 172.21.15.86:0/3486773275 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7f532c05a140 con 0x7f534006e2f0 2024-03-22T04:11:01.400 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:01.400 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5320067b50 msgr2=0x7f532006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5320067b50 0x7f532006a010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53400c5bf0 tx=0x7f533000b040 comp rx=0 tx=0).stop 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006e2f0 msgr2=0x7f534015b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006e2f0 0x7f534015b130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f532c00e8a0 tx=0x7f532c00ed60 comp rx=0 tx=0).stop 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 shutdown_connections 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5320067b50 0x7f532006a010 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534015e630 0x7f53400b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f534006e2f0 0x7f534015b130 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 --2- 172.21.15.86:0/3486773275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f534006cc80 0x7f53400b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 >> 172.21.15.86:0/3486773275 conn(0x7f534006f910 msgr2=0x7f53400ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 shutdown_connections 2024-03-22T04:11:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.396+0000 7f5346400640 1 -- 172.21.15.86:0/3486773275 wait complete. 2024-03-22T04:11:01.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-22T04:11:01.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-22T04:11:01.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-22T04:11:01.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-22T04:11:01.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-22T04:11:01.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-22T04:11:01.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-22T04:11:01.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-22T04:11:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.664+0000 7ff2f4600640 1 Processor -- start 2024-03-22T04:11:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.664+0000 7ff2f4600640 1 -- start start 2024-03-22T04:11:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.664+0000 7fac5ac00640 1 Processor -- start 2024-03-22T04:11:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.664+0000 7fac5ac00640 1 -- start start 2024-03-22T04:11:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f4600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff2e4047c20 0x7ff2e40eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2e4005ea0 0x7ff2e40062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f4600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e4034220 con 0x7ff2e4033e40 2024-03-22T04:11:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e4050210 con 0x7ff2e4005ea0 2024-03-22T04:11:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac5ac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540afc70 0x7fac5406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac5ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac5415b2e0 con 0x7fac5415ae80 2024-03-22T04:11:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac5ac00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac5406e660 con 0x7fac540b49b0 2024-03-22T04:11:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac5ac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fac5415b580 con 0x7fac540afc70 2024-03-22T04:11:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f4600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e4037bc0 con 0x7ff2e4047c20 2024-03-22T04:11:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac5a200640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fac540b49b0 0x7fac5415c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45856/0 (socket says 172.21.15.86:45856) 2024-03-22T04:11:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2e4005ea0 0x7ff2e40062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f1c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff2e4033e40 0x7ff2e40eb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57476/0 (socket says 172.21.15.86:57476) 2024-03-22T04:11:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f1c00640 1 -- 172.21.15.86:0/2999408081 learned_addr learned my addr 172.21.15.86:0/2999408081 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac5a200640 1 -- 172.21.15.86:0/1122918404 learned_addr learned my addr 172.21.15.86:0/1122918404 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f2600640 1 -- 172.21.15.86:0/2999408081 >> v2:172.21.15.196:3300/0 conn(0x7ff2e4047c20 msgr2=0x7ff2e40eb890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:11:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f2600640 1 --2- 172.21.15.86:0/2999408081 >> v2:172.21.15.196:3300/0 conn(0x7ff2e4047c20 0x7ff2e40eb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2987087821 0 0) 0x7fac5406e660 con 0x7fac540b49b0 2024-03-22T04:11:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac3c003450 con 0x7fac540b49b0 2024-03-22T04:11:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f2600640 1 -- 172.21.15.86:0/2999408081 >> v1:172.21.15.86:6789/0 conn(0x7ff2e4033e40 legacy=0x7ff2e40eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7ff2f2600640 1 -- 172.21.15.86:0/2999408081 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e404fa30 con 0x7ff2e4005ea0 2024-03-22T04:11:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2500550625 0 0) 0x7fac3c003450 con 0x7fac540b49b0 2024-03-22T04:11:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.668+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540afc70 msgr2=0x7fac5406f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac59800640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540afc70 0x7fac5406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540afc70 0x7fac5406f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 >> v1:172.21.15.86:6789/0 conn(0x7fac5415ae80 legacy=0x7fac540b42a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:11:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5415b860 con 0x7fac540b49b0 2024-03-22T04:11:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fac40001680 con 0x7fac540b49b0 2024-03-22T04:11:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 >> v1:172.21.15.196:6789/0 conn(0x7fac540b49b0 legacy=0x7fac5415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7ff2f2600640 1 --2- 172.21.15.86:0/2999408081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 0x7ff2e40062b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec069d60 tx=0x7ff2ec09a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8763083bcfd92a1d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac3c003b90 0x7fac3c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac3c00acb0 0x7fac3c00d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c00d8d0 0x7fac3c00fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac5415b580 con 0x7fac3c00d8d0 2024-03-22T04:11:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c010640 con 0x7fac3c003b90 2024-03-22T04:11:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fac3c010840 con 0x7fac3c00acb0 2024-03-22T04:11:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7ff2ebe00640 1 -- 172.21.15.86:0/2999408081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec12f070 con 0x7ff2e4005ea0 2024-03-22T04:11:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac59800640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540afc70 0x7fac5406f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7ff2ebe00640 1 -- 172.21.15.86:0/2999408081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ec12a600 con 0x7ff2e4005ea0 2024-03-22T04:11:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac5a200640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c00d8d0 0x7fac3c00fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7ff2ebe00640 1 -- 172.21.15.86:0/2999408081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec12aa50 con 0x7ff2e4005ea0 2024-03-22T04:11:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac5a200640 1 -- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac3c00acb0 msgr2=0x7fac3c00d120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-22T04:11:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac5a200640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac3c00acb0 0x7fac3c00d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac5a200640 1 -- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac3c003b90 msgr2=0x7fac3c003f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T04:11:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac5a200640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac3c003b90 0x7fac3c003f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7fac5a200640 1 -- 172.21.15.86:0/1122918404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c003450 con 0x7fac3c00d8d0 2024-03-22T04:11:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7ff2f4600640 1 -- 172.21.15.86:0/2999408081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 msgr2=0x7ff2e40062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.672+0000 7ff2f4600640 1 --2- 172.21.15.86:0/2999408081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 0x7ff2e40062b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec069d60 tx=0x7ff2ec09a500 comp rx=0 tx=0).stop 2024-03-22T04:11:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f2864e00640 1 Processor -- start 2024-03-22T04:11:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f2864e00640 1 -- start start 2024-03-22T04:11:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f46f4c00640 1 Processor -- start 2024-03-22T04:11:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f46f4c00640 1 -- start start 2024-03-22T04:11:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7ff2f4600640 1 -- 172.21.15.86:0/2999408081 shutdown_connections 2024-03-22T04:11:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7ff2f4600640 1 --2- 172.21.15.86:0/2999408081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 0x7ff2e40062b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7ff2f4600640 1 --2- 172.21.15.86:0/2999408081 >> v2:172.21.15.196:3300/0 conn(0x7ff2e4047c20 0x7ff2e40eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7ff2f4600640 1 -- 172.21.15.86:0/2999408081 >> 172.21.15.86:0/2999408081 conn(0x7ff2e4009fc0 msgr2=0x7ff2e400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7ff2f4600640 1 -- 172.21.15.86:0/2999408081 shutdown_connections 2024-03-22T04:11:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f2864e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f28600b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f2864e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28600b5640 con 0x7f2860071b30 2024-03-22T04:11:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f2864e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28600b4510 con 0x7f2860162b70 2024-03-22T04:11:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f2864e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f28600b5080 con 0x7f28600b40b0 2024-03-22T04:11:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f285de00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f2860162b70 0x7f286006e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45868/0 (socket says 172.21.15.86:45868) 2024-03-22T04:11:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7f285de00640 1 -- 172.21.15.86:0/2747193650 learned_addr learned my addr 172.21.15.86:0/2747193650 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285d400640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f28600b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46f4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f46f00b3800 0x7f46f006e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46f4c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f006e6f0 con 0x7f46f00999f0 2024-03-22T04:11:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46f4c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f006e9d0 con 0x7f46f0162a70 2024-03-22T04:11:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ee800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f46f00999f0 0x7f46f0071a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57496/0 (socket says 172.21.15.86:57496) 2024-03-22T04:11:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ee800640 1 -- 172.21.15.86:0/2694489159 learned_addr learned my addr 172.21.15.86:0/2694489159 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.676+0000 7ff2f4600640 1 -- 172.21.15.86:0/2999408081 wait complete. 2024-03-22T04:11:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7ff2f4600640 1 Processor -- start 2024-03-22T04:11:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46f4c00640 1 -- 172.21.15.86:0/2694489159 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f46f006ecb0 con 0x7f46f00b3800 2024-03-22T04:11:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ede00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f46f0162a70 0x7f46f00b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45874/0 (socket says 172.21.15.86:45874) 2024-03-22T04:11:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac5a200640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c00d8d0 0x7fac3c00fd60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fac4000c110 tx=0x7fac4001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447546724cf3ffc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40007c50 con 0x7fac3c00d8d0 2024-03-22T04:11:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2704451063 0 0) 0x7f46f006e6f0 con 0x7f46f00999f0 2024-03-22T04:11:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ed400640 1 --2- 172.21.15.86:0/2694489159 >> v2:172.21.15.86:3300/0 conn(0x7f46f00b3800 0x7f46f006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46d4003530 con 0x7f46f00999f0 2024-03-22T04:11:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac400a9020 con 0x7fac3c00d8d0 2024-03-22T04:11:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac53e00640 1 -- 172.21.15.86:0/1122918404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40005ae0 con 0x7fac3c00d8d0 2024-03-22T04:11:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7ff2f4600640 1 -- start start 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 159204022 0 0) 0x7f46f006e9d0 con 0x7f46f0162a70 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46f006e6f0 con 0x7f46f0162a70 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3097820448 0 0) 0x7f46d4003530 con 0x7f46f00999f0 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 >> v2:172.21.15.86:3300/0 conn(0x7f46f00b3800 msgr2=0x7f46f006e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 --2- 172.21.15.86:0/2694489159 >> v2:172.21.15.86:3300/0 conn(0x7f46f00b3800 0x7f46f006e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 >> v1:172.21.15.196:6789/0 conn(0x7f46f0162a70 legacy=0x7f46f00b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f006ef90 con 0x7f46f00999f0 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f46e8001a00 con 0x7f46f00999f0 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 >> v1:172.21.15.86:6789/0 conn(0x7f46f00999f0 legacy=0x7f46f0071a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46d4003b50 0x7f46d4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d4004810 0x7f46d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d40051d0 0x7f46d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46f006ecb0 con 0x7f46d4004810 2024-03-22T04:11:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46d400c050 con 0x7f46d40051d0 2024-03-22T04:11:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46d400c250 con 0x7f46d4003b50 2024-03-22T04:11:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ed400640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d4004810 0x7f46d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ede00640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46d4003b50 0x7f46d4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ee800640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d40051d0 0x7f46d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f19de600640 1 Processor -- start 2024-03-22T04:11:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac5ac00640 1 -- 172.21.15.86:0/1122918404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c00d8d0 msgr2=0x7fac3c00fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac5ac00640 1 --2- 172.21.15.86:0/1122918404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c00d8d0 0x7fac3c00fd60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fac4000c110 tx=0x7fac4001cba0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac5a200640 1 -- 172.21.15.86:0/1122918404 reap_dead start 2024-03-22T04:11:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac5ac00640 1 -- 172.21.15.86:0/1122918404 shutdown_connections 2024-03-22T04:11:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7fac5ac00640 1 -- 172.21.15.86:0/1122918404 >> 172.21.15.86:0/1122918404 conn(0x7fac5406b660 msgr2=0x7fac5406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3304865595 0 0) 0x7f28600b4510 con 0x7f2860162b70 2024-03-22T04:11:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2844003530 con 0x7f2860162b70 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 274960854 0 0) 0x7f28600b5640 con 0x7f2860071b30 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28600b4510 con 0x7f2860071b30 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3866473183 0 0) 0x7f2844003530 con 0x7f2860162b70 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 msgr2=0x7f28600b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f28600b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 >> v1:172.21.15.86:6789/0 conn(0x7f2860071b30 legacy=0x7f28600ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28600b5360 con 0x7f2860162b70 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f2850001680 con 0x7f2860162b70 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 >> v1:172.21.15.196:6789/0 conn(0x7f2860162b70 legacy=0x7f286006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 0x7ff2e40db0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2e4033e40 0x7ff2e40dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e4047c20 0x7ff2e40e45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 0x7ff2e40db0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7fac5ac00640 1 -- 172.21.15.86:0/1122918404 shutdown_connections 2024-03-22T04:11:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2e4033e40 0x7ff2e40dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 0x7ff2e40db0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55976/0 (socket says 172.21.15.86:55976) 2024-03-22T04:11:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2e4033e40 0x7ff2e40dffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:38912/0 (socket says 172.21.15.86:38912) 2024-03-22T04:11:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ee800640 1 -- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46d4003b50 msgr2=0x7f46d4003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ee800640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46d4003b50 0x7f46d4003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ee800640 1 -- 172.21.15.86:0/2694489159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d4004810 msgr2=0x7f46d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ee800640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d4004810 0x7f46d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f46ee800640 1 -- 172.21.15.86:0/2694489159 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f006ef90 con 0x7f46d40051d0 2024-03-22T04:11:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285ca00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2844004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285ca00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28440051a0 0x7f28440037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285ca00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2844003cf0 0x7f2844004150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28600b5080 con 0x7f28440051a0 2024-03-22T04:11:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f284400bc40 con 0x7f2844003cf0 2024-03-22T04:11:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f284400be40 con 0x7f28600b40b0 2024-03-22T04:11:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f46ed400640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d4004810 0x7f46d4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 -- 172.21.15.86:0/368455271 learned_addr learned my addr 172.21.15.86:0/368455271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e404fd80 con 0x7ff2e4005ea0 2024-03-22T04:11:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e404f090 con 0x7ff2e4047c20 2024-03-22T04:11:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e404f5c0 con 0x7ff2e4033e40 2024-03-22T04:11:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f2600640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e4047c20 0x7ff2e40e45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 -- 172.21.15.86:0/368455271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2e4033e40 msgr2=0x7ff2e40dffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2e4033e40 0x7ff2e40dffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 -- 172.21.15.86:0/368455271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e4047c20 msgr2=0x7ff2e40e45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e4047c20 0x7ff2e40e45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e40e0920 con 0x7ff2e4005ea0 2024-03-22T04:11:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285de00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2844004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7fac5ac00640 1 -- 172.21.15.86:0/1122918404 wait complete. 2024-03-22T04:11:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.680+0000 7f19de600640 1 -- start start 2024-03-22T04:11:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19de600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19d81567a0 0x7f19d80b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19de600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 0x7f19d80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19de600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d8156b80 con 0x7f19d80cefe0 2024-03-22T04:11:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19de600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f19d80cf430 con 0x7f19d81567a0 2024-03-22T04:11:01.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19de600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f19d80b5290 con 0x7f19d815d0d0 2024-03-22T04:11:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f1c00640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 0x7ff2e40db0e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000c450 tx=0x7ff2e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0017070 con 0x7ff2e4005ea0 2024-03-22T04:11:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e0005cf0 con 0x7ff2e4005ea0 2024-03-22T04:11:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0007290 con 0x7ff2e4005ea0 2024-03-22T04:11:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e40e0c00 con 0x7ff2e4005ea0 2024-03-22T04:11:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e40e1690 con 0x7ff2e4005ea0 2024-03-22T04:11:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285e800640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2844003cf0 0x7f2844004150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285d400640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28440051a0 0x7f28440037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19ddc00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f19d80cefe0 0x7f19d815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45880/0 (socket says 172.21.15.86:45880) 2024-03-22T04:11:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19ddc00640 1 -- 172.21.15.86:0/1659781059 learned_addr learned my addr 172.21.15.86:0/1659781059 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19dd200640 1 --2- 172.21.15.86:0/1659781059 >> v2:172.21.15.86:3300/0 conn(0x7f19d81567a0 0x7f19d80b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285de00640 1 -- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2844003cf0 msgr2=0x7f2844004150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285de00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2844003cf0 0x7f2844004150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff2e40053a0 con 0x7ff2e4005ea0 2024-03-22T04:11:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7fac5ac00640 1 Processor -- start 2024-03-22T04:11:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7fac5ac00640 1 -- start start 2024-03-22T04:11:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7fbef5000640 1 Processor -- start 2024-03-22T04:11:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7fbef5000640 1 -- start start 2024-03-22T04:11:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285de00640 1 -- 172.21.15.86:0/2747193650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28440051a0 msgr2=0x7f28440037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f285de00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28440051a0 0x7f28440037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19d7e00640 1 --2- 172.21.15.86:0/1659781059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 0x7f19d80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.684+0000 7f19d7400640 1 -- 172.21.15.86:0/1659781059 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549563566 0 0) 0x7f19d0001120 con 0x7f19d80cefe0 2024-03-22T04:11:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f19dd200640 1 -- 172.21.15.86:0/1659781059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 msgr2=0x7f19d80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f19dd200640 1 --2- 172.21.15.86:0/1659781059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 0x7f19d80ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f19dd200640 1 -- 172.21.15.86:0/1659781059 >> v1:172.21.15.196:6789/0 conn(0x7f19d80cefe0 legacy=0x7f19d815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c00d8d0 0x7fac54145e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5ac00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac540b1800 0x7fac540b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5ac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540a9a30 0x7fac540a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac5415c100 con 0x7fac3c00d8d0 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5ac00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac5415be20 con 0x7fac540b1800 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5ac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fac540b0050 con 0x7fac540a9a30 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac540b1800 0x7fac540b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac540b1800 0x7fac540b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52208/0 (socket says 172.21.15.86:52208) 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 -- 172.21.15.86:0/1029977890 learned_addr learned my addr 172.21.15.86:0/1029977890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5a200640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540a9a30 0x7fac540a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 -- 172.21.15.86:0/1029977890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540a9a30 msgr2=0x7fac540a9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac540a9a30 0x7fac540a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 -- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c00d8d0 msgr2=0x7fac54145e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3c00d8d0 0x7fac54145e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 -- 172.21.15.86:0/1029977890 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac54146da0 con 0x7fac540b1800 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac58e00640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac540b1800 0x7fac540b1c80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac4800b410 tx=0x7fac4800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48015020 con 0x7fac540b1800 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac48011830 con 0x7fac540b1800 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48011c60 con 0x7fac540b1800 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5ac00640 1 -- 172.21.15.86:0/1029977890 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac540aa3f0 con 0x7fac540b1800 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac5ac00640 1 -- 172.21.15.86:0/1029977890 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac54146ac0 con 0x7fac540b1800 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac30001040 con 0x7fac540b1800 2024-03-22T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbef5000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbef00d48d0 0x7fbef0162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbef5000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef00d3390 con 0x7fbef015e900 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbef5000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef006cd70 con 0x7fbef015e520 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbef5000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbef00cf760 con 0x7fbef00d48d0 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbeede00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fbef015e520 0x7fbef00cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45882/0 (socket says 172.21.15.86:45882) 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbeede00640 1 -- 172.21.15.86:0/496697464 learned_addr learned my addr 172.21.15.86:0/496697464 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbeee800640 1 --2- 172.21.15.86:0/496697464 >> v2:172.21.15.196:3300/0 conn(0x7fbef00d48d0 0x7fbef0162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbeee800640 1 -- 172.21.15.86:0/496697464 >> v1:172.21.15.196:6789/0 conn(0x7fbef015e520 legacy=0x7fbef00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbeee800640 1 -- 172.21.15.86:0/496697464 >> v1:172.21.15.86:6789/0 conn(0x7fbef015e900 legacy=0x7fbef00d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7fbeee800640 1 -- 172.21.15.86:0/496697464 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef00cf980 con 0x7fbef00d48d0 2024-03-22T04:11:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f285de00640 1 -- 172.21.15.86:0/2747193650 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28600b5640 con 0x7f28600b40b0 2024-03-22T04:11:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f285d400640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28440051a0 0x7f28440037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f46ee800640 1 -- 172.21.15.86:0/2694489159 reap_dead start 2024-03-22T04:11:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f46ee800640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d40051d0 0x7f46d400b9d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46e800a8b0 tx=0x7f46e801d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e722e90fad772dc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e800a480 con 0x7f46d40051d0 2024-03-22T04:11:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e0001d40 con 0x7f46d40051d0 2024-03-22T04:11:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f46eca00640 1 -- 172.21.15.86:0/2694489159 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e80ad850 con 0x7f46d40051d0 2024-03-22T04:11:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f19dd200640 1 -- 172.21.15.86:0/1659781059 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d80b5570 con 0x7f19d81567a0 2024-03-22T04:11:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.688+0000 7f19d7e00640 1 --2- 172.21.15.86:0/1659781059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 0x7f19d80ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f19dd200640 1 --2- 172.21.15.86:0/1659781059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19d81567a0 0x7f19d80b6330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097c0 tx=0x7f19cc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1af83c36d523af server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fac48018020 con 0x7fac540b1800 2024-03-22T04:11:01.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff2e001a070 con 0x7ff2e4005ea0 2024-03-22T04:11:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f46f4c00640 1 -- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d40051d0 msgr2=0x7f46d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f46f4c00640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d40051d0 0x7f46d400b9d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46e800a8b0 tx=0x7f46e801d340 comp rx=0 tx=0).stop 2024-03-22T04:11:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f46f4c00640 1 -- 172.21.15.86:0/2694489159 shutdown_connections 2024-03-22T04:11:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f46f4c00640 1 --2- 172.21.15.86:0/2694489159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d40051d0 0x7f46d400b9d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f46f4c00640 1 -- 172.21.15.86:0/2694489159 >> 172.21.15.86:0/2694489159 conn(0x7f46f00b1df0 msgr2=0x7f46f00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f19d7400640 1 -- 172.21.15.86:0/1659781059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0a7070 con 0x7f19d81567a0 2024-03-22T04:11:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7fac51600640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac34067b20 0x7fac34069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f19d7400640 1 -- 172.21.15.86:0/1659781059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc0b0070 con 0x7f19d81567a0 2024-03-22T04:11:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fac480861c0 con 0x7fac540b1800 2024-03-22T04:11:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7fac51600640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7fac34071220 0x7fac34073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac34073d30 con 0x7fac34071220 2024-03-22T04:11:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac4808d050 con 0x7fac540b1800 2024-03-22T04:11:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7ff2e9600640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2cc067b70 0x7ff2cc06a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7fac59800640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac34067b20 0x7fac34069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff2e00854c0 con 0x7ff2e4005ea0 2024-03-22T04:11:01.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19d7400640 1 -- 172.21.15.86:0/1659781059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0ab890 con 0x7f19d81567a0 2024-03-22T04:11:01.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f46f4c00640 1 -- 172.21.15.86:0/2694489159 shutdown_connections 2024-03-22T04:11:01.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.692+0000 7f46f4c00640 1 -- 172.21.15.86:0/2694489159 wait complete. 2024-03-22T04:11:01.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19de600640 1 -- 172.21.15.86:0/1659781059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19d81567a0 msgr2=0x7f19d80b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19de600640 1 --2- 172.21.15.86:0/1659781059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19d81567a0 0x7f19d80b6330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097c0 tx=0x7f19cc01c470 comp rx=0 tx=0).stop 2024-03-22T04:11:01.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7ff2e9600640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7ff2cc071270 0x7ff2cc0736b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2e0016ec0 con 0x7ff2cc071270 2024-03-22T04:11:01.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7ff2f1200640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2cc067b70 0x7ff2cc06a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7fac5a200640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7fac34071220 0x7fac34073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2e008c050 con 0x7ff2e4005ea0 2024-03-22T04:11:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7ff2f2600640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7ff2cc071270 0x7ff2cc0736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7fac5a200640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7fac34071220 0x7fac34073660 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7ff2f2600640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7ff2cc071270 0x7ff2cc0736b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19de600640 1 -- 172.21.15.86:0/1659781059 shutdown_connections 2024-03-22T04:11:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19de600640 1 --2- 172.21.15.86:0/1659781059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 0x7f19d80ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19de600640 1 --2- 172.21.15.86:0/1659781059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19d81567a0 0x7f19d80b6330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19de600640 1 -- 172.21.15.86:0/1659781059 >> 172.21.15.86:0/1659781059 conn(0x7f19d80b74a0 msgr2=0x7f19d80b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19de600640 1 -- 172.21.15.86:0/1659781059 shutdown_connections 2024-03-22T04:11:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7fac59800640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac34067b20 0x7fac34069fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fac5406c430 tx=0x7fac4c008280 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f19de600640 1 -- 172.21.15.86:0/1659781059 wait complete. 2024-03-22T04:11:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff2e0016ec0 con 0x7ff2cc071270 2024-03-22T04:11:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7fbeee800640 1 --2- 172.21.15.86:0/496697464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 0x7fbef0162d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0097c0 tx=0x7fbedc01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2637fdc9330de19a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f285de00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2844004bd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f285000c210 tx=0x7f285001cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2381e843bd963cd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850007cf0 con 0x7f28600b40b0 2024-03-22T04:11:01.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28500a5b00 con 0x7f28600b40b0 2024-03-22T04:11:01.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f46f4c00640 1 Processor -- start 2024-03-22T04:11:01.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7f46f4c00640 1 -- start start 2024-03-22T04:11:01.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7ff2f1200640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2cc067b70 0x7ff2cc06a030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009f70 tx=0x7ff2d8009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.696+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac34073d30 con 0x7fac34071220 2024-03-22T04:11:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f285ca00640 1 -- 172.21.15.86:0/2747193650 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850007cf0 con 0x7f28600b40b0 2024-03-22T04:11:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbeeca00640 1 -- 172.21.15.86:0/496697464 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0ab620 con 0x7fbef00d48d0 2024-03-22T04:11:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f2864e00640 1 -- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 msgr2=0x7f2844004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbeeca00640 1 -- 172.21.15.86:0/496697464 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc0b0070 con 0x7fbef00d48d0 2024-03-22T04:11:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f2864e00640 1 --2- 172.21.15.86:0/2747193650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2844004bd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f285000c210 tx=0x7f285001cc00 comp rx=0 tx=0).stop 2024-03-22T04:11:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46f4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d40051d0 0x7f46f0165920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46f4c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f00af370 0x7f46f01571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19de600640 1 Processor -- start 2024-03-22T04:11:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f285e800640 1 -- 172.21.15.86:0/2747193650 reap_dead start 2024-03-22T04:11:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f2864e00640 1 -- 172.21.15.86:0/2747193650 shutdown_connections 2024-03-22T04:11:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f2864e00640 1 -- 172.21.15.86:0/2747193650 >> 172.21.15.86:0/2747193650 conn(0x7f28600b5f30 msgr2=0x7f286006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d40051d0 0x7f46f0165920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d40051d0 0x7f46f0165920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56020/0 (socket says 172.21.15.86:56020) 2024-03-22T04:11:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f2864e00640 1 -- 172.21.15.86:0/2747193650 shutdown_connections 2024-03-22T04:11:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbef5000640 1 -- 172.21.15.86:0/496697464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 msgr2=0x7fbef0162d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbeeca00640 1 -- 172.21.15.86:0/496697464 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0ab620 con 0x7fbef00d48d0 2024-03-22T04:11:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46ede00640 1 -- 172.21.15.86:0/2345860751 learned_addr learned my addr 172.21.15.86:0/2345860751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbef5000640 1 --2- 172.21.15.86:0/496697464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 0x7fbef0162d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0097c0 tx=0x7fbedc01c420 comp rx=0 tx=0).stop 2024-03-22T04:11:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46f4c00640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f0164bf0 0x7f46f015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbef5000640 1 -- 172.21.15.86:0/496697464 shutdown_connections 2024-03-22T04:11:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbef5000640 1 --2- 172.21.15.86:0/496697464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 0x7fbef0162d50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbef5000640 1 -- 172.21.15.86:0/496697464 >> 172.21.15.86:0/496697464 conn(0x7fbef008c000 msgr2=0x7fbef009d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbef5000640 1 -- 172.21.15.86:0/496697464 shutdown_connections 2024-03-22T04:11:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7fbef5000640 1 -- 172.21.15.86:0/496697464 wait complete. 2024-03-22T04:11:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46ed400640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f00af370 0x7f46f01571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46ee800640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f0164bf0 0x7f46f015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46f006e110 con 0x7f46d40051d0 2024-03-22T04:11:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46f00b4000 con 0x7f46f0164bf0 2024-03-22T04:11:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19de600640 1 -- start start 2024-03-22T04:11:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19de600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19d80cefe0 0x7f19d81fb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19de600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d81567a0 0x7f19d815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19de600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 0x7f19d815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19de600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19d80b3da0 con 0x7f19d80cefe0 2024-03-22T04:11:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19de600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19d80b4f80 con 0x7f19d81567a0 2024-03-22T04:11:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19de600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f19d80b3f20 con 0x7f19d815d0d0 2024-03-22T04:11:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19ddc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 0x7f19d815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f19d7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d81567a0 0x7f19d815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19d7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d81567a0 0x7f19d815b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52242/0 (socket says 172.21.15.86:52242) 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19d7e00640 1 -- 172.21.15.86:0/3547718230 learned_addr learned my addr 172.21.15.86:0/3547718230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19dd200640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19d80cefe0 0x7f19d81fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f302b600640 1 Processor -- start 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f302b600640 1 -- start start 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f302b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3024162a70 0x7f302406e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f302b600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30240b3800 0x7f3024071a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f302b600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f302406e6f0 con 0x7f30240999f0 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f302b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f302406e9d0 con 0x7f3024162a70 2024-03-22T04:11:01.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f302b600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f302406ecb0 con 0x7f30240b3800 2024-03-22T04:11:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7fbef5000640 1 Processor -- start 2024-03-22T04:11:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7fbef5000640 1 -- start start 2024-03-22T04:11:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.700+0000 7f2864e00640 1 -- 172.21.15.86:0/2747193650 wait complete. 2024-03-22T04:11:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f2864e00640 1 Processor -- start 2024-03-22T04:11:01.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f2864e00640 1 -- start start 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46f006e290 con 0x7f46f00af370 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46ee800640 1 -- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f00af370 msgr2=0x7f46f01571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46ee800640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f00af370 0x7f46f01571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46ee800640 1 -- 172.21.15.86:0/2345860751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d40051d0 msgr2=0x7f46f0165920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46ee800640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d40051d0 0x7f46f0165920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46ee800640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f0163e90 con 0x7f46f0164bf0 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46ee800640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f0164bf0 0x7f46f015cbb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46f00b2e80 tx=0x7f46d4005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e800a480 con 0x7f46f0164bf0 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e80b4070 con 0x7f46f0164bf0 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e80b9890 con 0x7f46f0164bf0 2024-03-22T04:11:01.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f0165f00 con 0x7f46f0164bf0 2024-03-22T04:11:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19d7e00640 1 -- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 msgr2=0x7f19d815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19d7e00640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d815d0d0 0x7f19d815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19d7e00640 1 -- 172.21.15.86:0/3547718230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19d80cefe0 msgr2=0x7f19d81fb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19d7e00640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19d80cefe0 0x7f19d81fb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19d7e00640 1 -- 172.21.15.86:0/3547718230 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d80d1490 con 0x7f19d81567a0 2024-03-22T04:11:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f19d7e00640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d81567a0 0x7f19d815b830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f19c8007ef0 tx=0x7f19c800d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f01643f0 con 0x7f46f0164bf0 2024-03-22T04:11:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.704+0000 7f3023e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f30240999f0 0x7f30240b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45888/0 (socket says 172.21.15.86:45888) 2024-03-22T04:11:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3029600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30240b3800 0x7f3024071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023e00640 1 -- 172.21.15.86:0/2704968605 learned_addr learned my addr 172.21.15.86:0/2704968605 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3028c00640 1 --2- 172.21.15.86:0/2704968605 >> v2:172.21.15.86:3300/0 conn(0x7f3024162a70 0x7f302406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c801a070 con 0x7f19d81567a0 2024-03-22T04:11:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c8003ba0 con 0x7f19d81567a0 2024-03-22T04:11:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8004030 con 0x7f19d81567a0 2024-03-22T04:11:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701526004 0 0) 0x7f302406e6f0 con 0x7f30240999f0 2024-03-22T04:11:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46f01661e0 con 0x7f46f0164bf0 2024-03-22T04:11:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbef5000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 0x7fbef0151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3008003530 con 0x7f30240999f0 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d80d1770 con 0x7f19d81567a0 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 339214114 0 0) 0x7f3008003530 con 0x7f30240999f0 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 >> v2:172.21.15.196:3300/0 conn(0x7f30240b3800 msgr2=0x7f3024071a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 --2- 172.21.15.86:0/2704968605 >> v2:172.21.15.196:3300/0 conn(0x7f30240b3800 0x7f3024071a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 >> v2:172.21.15.86:3300/0 conn(0x7f3024162a70 msgr2=0x7f302406e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 --2- 172.21.15.86:0/2704968605 >> v2:172.21.15.86:3300/0 conn(0x7f3024162a70 0x7f302406e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302406ef90 con 0x7f30240999f0 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f301c001a00 con 0x7f30240999f0 2024-03-22T04:11:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3028c00640 1 --2- 172.21.15.86:0/2704968605 >> v2:172.21.15.86:3300/0 conn(0x7f3024162a70 0x7f302406e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 >> v1:172.21.15.196:6789/0 conn(0x7f30240999f0 legacy=0x7f30240b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080039c0 0x7f3008003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008004810 0x7f3008004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30080051d0 0x7f300800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f302406e9d0 con 0x7f3008004810 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f302406ecb0 con 0x7f30080039c0 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f300800c010 con 0x7f30080051d0 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeede00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 0x7fbef0151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeede00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 0x7fbef0151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52264/0 (socket says 172.21.15.86:52264) 2024-03-22T04:11:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbef5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef015e520 0x7fbef0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef015e520 0x7fbef0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeede00640 1 -- 172.21.15.86:0/112442147 learned_addr learned my addr 172.21.15.86:0/112442147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d80d1d30 con 0x7f19d81567a0 2024-03-22T04:11:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3029600640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30080051d0 0x7f300800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3028c00640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080039c0 0x7f3008003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbef5000640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef015e900 0x7fbef0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbef016bef0 con 0x7fbef015e520 2024-03-22T04:11:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbef016bd70 con 0x7fbef00d48d0 2024-03-22T04:11:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbef016c070 con 0x7fbef015e900 2024-03-22T04:11:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeee800640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef015e900 0x7fbef0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f2864e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2860202310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f2864e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2860200930 0x7f2860162f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19d8074470 con 0x7f19d81567a0 2024-03-22T04:11:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeed400640 1 -- 172.21.15.86:0/112442147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef015e900 msgr2=0x7fbef0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeed400640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef015e900 0x7fbef0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeed400640 1 -- 172.21.15.86:0/112442147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 msgr2=0x7fbef0151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeed400640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 0x7fbef0151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeed400640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef00b4eb0 con 0x7fbef015e520 2024-03-22T04:11:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7fbeede00640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef00d48d0 0x7fbef0151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f2864e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860200db0 0x7f28600cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f2864e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2860071fc0 con 0x7f2860200db0 2024-03-22T04:11:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f2864e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f286006dee0 con 0x7f2860200930 2024-03-22T04:11:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f2864e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2860072140 con 0x7f28600b40b0 2024-03-22T04:11:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2860202310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f46e80ad430 con 0x7f46f0164bf0 2024-03-22T04:11:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2860202310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:38966/0 (socket says 172.21.15.86:38966) 2024-03-22T04:11:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 -- 172.21.15.86:0/3808741302 learned_addr learned my addr 172.21.15.86:0/3808741302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285d400640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2860200930 0x7f2860162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285e800640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860200db0 0x7f28600cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7fbeed400640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef015e520 0x7fbef0156680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbee8002200 tx=0x7fbee800a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 -- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2860200930 msgr2=0x7f2860162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2860200930 0x7f2860162f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46e6000640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46d0067c20 0x7f46d006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f46e8120360 con 0x7f46f0164bf0 2024-03-22T04:11:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46e6000640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f46d00713a0 0x7f46d00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46e80c3e50 con 0x7f46d00713a0 2024-03-22T04:11:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46e8127050 con 0x7f46f0164bf0 2024-03-22T04:11:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 -- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860200db0 msgr2=0x7f28600cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860200db0 0x7f28600cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 -- 172.21.15.86:0/3808741302 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2860202850 con 0x7f28600b40b0 2024-03-22T04:11:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285e800640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860200db0 0x7f28600cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285de00640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2860202310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2850000c10 tx=0x7f2844005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f285d400640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2860200930 0x7f2860162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f19c801d070 con 0x7f19d81567a0 2024-03-22T04:11:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f19d4c00640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19bc067b70 0x7f19bc06a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46ed400640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f46d00713a0 0x7f46d00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46ede00640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46d0067c20 0x7f46d006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f19dd200640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19bc067b70 0x7f19bc06a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f46ed400640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f46d00713a0 0x7f46d00737e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28500073c0 con 0x7f28600b40b0 2024-03-22T04:11:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28500b5070 con 0x7f28600b40b0 2024-03-22T04:11:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850007760 con 0x7f28600b40b0 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f2864e00640 1 -- 172.21.15.86:0/3808741302 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2860163a10 con 0x7f28600b40b0 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f2864e00640 1 -- 172.21.15.86:0/3808741302 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2860159ac0 con 0x7f28600b40b0 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3029600640 1 -- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080039c0 msgr2=0x7f3008003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3029600640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080039c0 0x7f3008003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3029600640 1 -- 172.21.15.86:0/2704968605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008004810 msgr2=0x7f3008004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3029600640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3008004810 0x7f3008004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.708+0000 7f3029600640 1 -- 172.21.15.86:0/2704968605 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302406ef90 con 0x7f30080051d0 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.712+0000 7f3029600640 1 -- 172.21.15.86:0/2704968605 reap_dead start 2024-03-22T04:11:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f3028c00640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080039c0 0x7f3008003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46e80c3e50 con 0x7f46d00713a0 2024-03-22T04:11:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f3029600640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30080051d0 0x7f300800b9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f301400ba00 tx=0x7f301401c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427de803a6202650 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f46ede00640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46d0067c20 0x7f46d006a0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46e0001ec0 tx=0x7f46e0006940 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f19c80545e0 con 0x7f19d81567a0 2024-03-22T04:11:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f19d4c00640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7f19bc071f40 0x7f19bc0743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19bc074ab0 con 0x7f19bc071f40 2024-03-22T04:11:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8003b80 con 0x7fbef015e520 2024-03-22T04:11:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8003d40 con 0x7fbef015e520 2024-03-22T04:11:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee80052b0 con 0x7fbef015e520 2024-03-22T04:11:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef00b5190 con 0x7fbef015e520 2024-03-22T04:11:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef00b5750 con 0x7fbef015e520 2024-03-22T04:11:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f19dd200640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19bc067b70 0x7f19bc06a030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0af000 tx=0x7f19cc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301400b600 con 0x7f30080051d0 2024-03-22T04:11:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3014004220 con 0x7f30080051d0 2024-03-22T04:11:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f3023400640 1 -- 172.21.15.86:0/2704968605 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014003620 con 0x7f30080051d0 2024-03-22T04:11:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19c8052f70 con 0x7f19d81567a0 2024-03-22T04:11:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f2864e00640 1 -- 172.21.15.86:0/3808741302 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2838001040 con 0x7f28600b40b0 2024-03-22T04:11:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbef0068650 con 0x7fbef015e520 2024-03-22T04:11:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f302b600640 1 -- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30080051d0 msgr2=0x7f300800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f302b600640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30080051d0 0x7f300800b9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f301400ba00 tx=0x7f301401c6c0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f19ddc00640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7f19bc071f40 0x7f19bc0743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47cd200640 1 Processor -- start 2024-03-22T04:11:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.716+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f28500047f0 con 0x7f28600b40b0 2024-03-22T04:11:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f2856000640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2840067b70 0x7f284006a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2850120d50 con 0x7f28600b40b0 2024-03-22T04:11:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f2856000640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7f28400712f0 0x7f2840073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2850009eb0 con 0x7f28400712f0 2024-03-22T04:11:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2850123370 con 0x7f28600b40b0 2024-03-22T04:11:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47cd200640 1 -- start start 2024-03-22T04:11:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f19ddc00640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7f19bc071f40 0x7f19bc0743e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbee8005450 con 0x7fbef015e520 2024-03-22T04:11:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f285d400640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2840067b70 0x7f284006a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f302b600640 1 -- 172.21.15.86:0/2704968605 shutdown_connections 2024-03-22T04:11:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f302b600640 1 --2- 172.21.15.86:0/2704968605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30080051d0 0x7f300800b9d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f302b600640 1 -- 172.21.15.86:0/2704968605 >> 172.21.15.86:0/2704968605 conn(0x7f30240b1df0 msgr2=0x7f30240b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f285e800640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7f28400712f0 0x7f2840073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f302b600640 1 -- 172.21.15.86:0/2704968605 shutdown_connections 2024-03-22T04:11:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f302b600640 1 -- 172.21.15.86:0/2704968605 wait complete. 2024-03-22T04:11:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19bc074ab0 con 0x7f19bc071f40 2024-03-22T04:11:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47cd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47c8071e60 0x7f47c80cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47cd200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f47c80cc8f0 0x7f47c80d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47cd200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c8072240 con 0x7f47c8071100 2024-03-22T04:11:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47cd200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f47c80cce60 con 0x7f47c8071e60 2024-03-22T04:11:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47cd200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f47c80cd140 con 0x7f47c80cc8f0 2024-03-22T04:11:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f285e800640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7f28400712f0 0x7f2840073730 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7fbee6000640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbed0067bf0 0x7fbed006a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47c5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47c8071e60 0x7f47c80cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47c5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47c8071e60 0x7f47c80cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56092/0 (socket says 172.21.15.86:56092) 2024-03-22T04:11:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7f47c5e00640 1 -- 172.21.15.86:0/151866420 learned_addr learned my addr 172.21.15.86:0/151866420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f302b600640 1 Processor -- start 2024-03-22T04:11:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbee808e7e0 con 0x7fbef015e520 2024-03-22T04:11:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.720+0000 7fbee6000640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7fbed0071360 0x7fbed00737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f47c5400640 1 --2- 172.21.15.86:0/151866420 >> v2:172.21.15.196:3300/0 conn(0x7f47c80cc8f0 0x7f47c80d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f47c5e00640 1 -- 172.21.15.86:0/151866420 >> v2:172.21.15.196:3300/0 conn(0x7f47c80cc8f0 msgr2=0x7f47c80d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f47c5e00640 1 --2- 172.21.15.86:0/151866420 >> v2:172.21.15.196:3300/0 conn(0x7f47c80cc8f0 0x7f47c80d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f47c5e00640 1 -- 172.21.15.86:0/151866420 >> v1:172.21.15.86:6789/0 conn(0x7f47c8071100 legacy=0x7f47c806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f285d400640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2840067b70 0x7f284006a030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28600746c0 tx=0x7f284c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbed0073e70 con 0x7fbed0071360 2024-03-22T04:11:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbee808ebd0 con 0x7fbef015e520 2024-03-22T04:11:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f302b600640 1 -- start start 2024-03-22T04:11:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7fbeee800640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7fbed0071360 0x7fbed00737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7fbeede00640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbed0067bf0 0x7fbed006a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac30002de0 con 0x7fac34071220 2024-03-22T04:11:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f47c5e00640 1 -- 172.21.15.86:0/151866420 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c80cd420 con 0x7f47c8071e60 2024-03-22T04:11:01.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f302b600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080051d0 0x7f3024161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f302b600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3024150530 0x7f3024166420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f302b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cf1a0 0x7f3024166960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f302b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f302406e110 con 0x7f30240cf1a0 2024-03-22T04:11:01.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f302b600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30240b4000 con 0x7f30080051d0 2024-03-22T04:11:01.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f302b600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f302406e290 con 0x7f3024150530 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f3028c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080051d0 0x7f3024161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f3028c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080051d0 0x7f3024161940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52300/0 (socket says 172.21.15.86:52300) 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f3028c00640 1 -- 172.21.15.86:0/89461163 learned_addr learned my addr 172.21.15.86:0/89461163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f3023e00640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3024150530 0x7f3024166420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f3029600640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cf1a0 0x7f3024166960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3028c00640 1 -- 172.21.15.86:0/89461163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3024150530 msgr2=0x7f3024166420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3028c00640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3024150530 0x7f3024166420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3028c00640 1 -- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cf1a0 msgr2=0x7f3024166960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3028c00640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cf1a0 0x7f3024166960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3028c00640 1 -- 172.21.15.86:0/89461163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240cfa50 con 0x7f30080051d0 2024-03-22T04:11:01.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7fac51600640 1 -- 172.21.15.86:0/1029977890 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fac30002de0 con 0x7fac34071220 2024-03-22T04:11:01.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7fac34071220 msgr2=0x7fac34073660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7fac34071220 0x7fac34073660 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7fbeee800640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7fbed0071360 0x7fbed00737a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.724+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2850009eb0 con 0x7f28400712f0 2024-03-22T04:11:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3028c00640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080051d0 0x7f3024161940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3010006aa0 tx=0x7f3010011d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fbeede00640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbed0067bf0 0x7fbed006a0b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbee0007d60 tx=0x7fbee0007cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac34067b20 msgr2=0x7fac34069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac34067b20 0x7fac34069fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fac5406c430 tx=0x7fac4c008280 comp rx=0 tx=0).stop 2024-03-22T04:11:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac540b1800 msgr2=0x7fac540b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 --2- 172.21.15.86:0/1029977890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac540b1800 0x7fac540b1c80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac4800b410 tx=0x7fac4800b8d0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010009e60 con 0x7f30080051d0 2024-03-22T04:11:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3010004220 con 0x7f30080051d0 2024-03-22T04:11:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301000f9d0 con 0x7f30080051d0 2024-03-22T04:11:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30241542d0 con 0x7f30080051d0 2024-03-22T04:11:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3024154570 con 0x7f30080051d0 2024-03-22T04:11:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac5a200640 1 -- 172.21.15.86:0/1029977890 reap_dead start 2024-03-22T04:11:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 shutdown_connections 2024-03-22T04:11:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 >> 172.21.15.86:0/1029977890 conn(0x7fac5406b660 msgr2=0x7fac540b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff2e40e1ac0 con 0x7ff2cc071270 2024-03-22T04:11:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3024068650 con 0x7f30080051d0 2024-03-22T04:11:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbed0073e70 con 0x7fbed0071360 2024-03-22T04:11:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 shutdown_connections 2024-03-22T04:11:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f47c5e00640 1 --2- 172.21.15.86:0/151866420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071e60 0x7f47c80cfe70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47c806f470 tx=0x7f47c001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c4effee7bd4414 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7fac46a00640 1 -- 172.21.15.86:0/1029977890 wait complete. 2024-03-22T04:11:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f47c4a00640 1 -- 172.21.15.86:0/151866420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00a5880 con 0x7f47c8071e60 2024-03-22T04:11:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f47c4a00640 1 -- 172.21.15.86:0/151866420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c00a5a20 con 0x7f47c8071e60 2024-03-22T04:11:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.728+0000 7f47c4a00640 1 -- 172.21.15.86:0/151866420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00a5df0 con 0x7f47c8071e60 2024-03-22T04:11:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2e9600640 1 -- 172.21.15.86:0/368455271 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff2e40e1ac0 con 0x7ff2cc071270 2024-03-22T04:11:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f301000fb70 con 0x7f30080051d0 2024-03-22T04:11:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f3020c00640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067c20 0x7f300406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3010086e90 con 0x7f30080051d0 2024-03-22T04:11:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7ff2cc071270 msgr2=0x7ff2cc0736b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f4600640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7ff2cc071270 0x7ff2cc0736b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2cc067b70 msgr2=0x7ff2cc06a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f4600640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2cc067b70 0x7ff2cc06a030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009f70 tx=0x7ff2d8009060 comp rx=0 tx=0).stop 2024-03-22T04:11:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 msgr2=0x7ff2e40db0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f4600640 1 --2- 172.21.15.86:0/368455271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4005ea0 0x7ff2e40db0e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000c450 tx=0x7ff2e000c910 comp rx=0 tx=0).stop 2024-03-22T04:11:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f47cd200640 1 -- 172.21.15.86:0/151866420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071e60 msgr2=0x7f47c80cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f47cd200640 1 --2- 172.21.15.86:0/151866420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071e60 0x7f47c80cfe70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47c806f470 tx=0x7f47c001c9e0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f47cd200640 1 -- 172.21.15.86:0/151866420 shutdown_connections 2024-03-22T04:11:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f47cd200640 1 --2- 172.21.15.86:0/151866420 >> v2:172.21.15.196:3300/0 conn(0x7f47c80cc8f0 0x7f47c80d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f47cd200640 1 --2- 172.21.15.86:0/151866420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071e60 0x7f47c80cfe70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f47cd200640 1 -- 172.21.15.86:0/151866420 >> 172.21.15.86:0/151866420 conn(0x7f47c806f910 msgr2=0x7f47c806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f47cd200640 1 -- 172.21.15.86:0/151866420 shutdown_connections 2024-03-22T04:11:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f3020c00640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f3004071390 0x7f30040737d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3004073ea0 con 0x7f3004071390 2024-03-22T04:11:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3010087280 con 0x7f30080051d0 2024-03-22T04:11:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f47cd200640 1 -- 172.21.15.86:0/151866420 wait complete. 2024-03-22T04:11:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f2600640 1 -- 172.21.15.86:0/368455271 reap_dead start 2024-03-22T04:11:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 shutdown_connections 2024-03-22T04:11:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 >> 172.21.15.86:0/368455271 conn(0x7ff2e4009fc0 msgr2=0x7ff2e40e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.732+0000 7f3023e00640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067c20 0x7f300406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7f3029600640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f3004071390 0x7f30040737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7f3023e00640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067c20 0x7f300406a0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f301c001cd0 tx=0x7f301c00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7f3029600640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f3004071390 0x7f30040737d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7f47cd200640 1 Processor -- start 2024-03-22T04:11:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 shutdown_connections 2024-03-22T04:11:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7f47cd200640 1 -- start start 2024-03-22T04:11:01.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7ff2f4600640 1 -- 172.21.15.86:0/368455271 wait complete. 2024-03-22T04:11:01.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3004073ea0 con 0x7f3004071390 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.736+0000 7f47cd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071100 0x7f47c80cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47cd200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47c8071e60 0x7f47c8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47cd200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c80cc8f0 0x7f47c80c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47cd200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47c816f810 con 0x7f47c8071100 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47cd200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47c816f690 con 0x7f47c80cc8f0 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47cd200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f47c816f990 con 0x7f47c8071e60 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071100 0x7f47c80cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071100 0x7f47c80cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56114/0 (socket says 172.21.15.86:56114) 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5e00640 1 -- 172.21.15.86:0/1462595938 learned_addr learned my addr 172.21.15.86:0/1462595938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5400640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47c8071e60 0x7f47c8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c6800640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c80cc8f0 0x7f47c80c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5400640 1 -- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c80cc8f0 msgr2=0x7f47c80c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5400640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47c80cc8f0 0x7f47c80c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5400640 1 -- 172.21.15.86:0/1462595938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071100 msgr2=0x7f47c80cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5400640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8071100 0x7f47c80cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5400640 1 -- 172.21.15.86:0/1462595938 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c80c62d0 con 0x7f47c8071e60 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47c5400640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47c8071e60 0x7f47c8159070 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47b400d5f0 tx=0x7f47b400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b40125f0 con 0x7f47c8071e60 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b4019070 con 0x7f47c8071e60 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c80c65b0 con 0x7f47c8071e60 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b40129b0 con 0x7f47c8071e60 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c80c6b70 con 0x7f47c8071e60 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.740+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47c8068650 con 0x7f47c8071e60 2024-03-22T04:11:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.744+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f47b4012b50 con 0x7f47c8071e60 2024-03-22T04:11:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.744+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46f0068650 con 0x7f46d00713a0 2024-03-22T04:11:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.744+0000 7f47be000640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f47a8067bf0 0x7f47a806a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f46e6000640 1 -- 172.21.15.86:0/2345860751 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f46f0068650 con 0x7f46d00713a0 2024-03-22T04:11:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.744+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f47b4086540 con 0x7f47c8071e60 2024-03-22T04:11:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f47be000640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f47a8071370 0x7f47a80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47a8073e80 con 0x7f47a8071370 2024-03-22T04:11:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47b408d050 con 0x7f47c8071e60 2024-03-22T04:11:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f47c5e00640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f47a8067bf0 0x7f47a806a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f47c5e00640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f47a8067bf0 0x7f47a806a0b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f47c0000f90 tx=0x7f47c00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f46d00713a0 msgr2=0x7f46d00737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f46f4c00640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f46d00713a0 0x7f46d00737e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f47c6800640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f47a8071370 0x7f47a80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46d0067c20 msgr2=0x7f46d006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f46f4c00640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46d0067c20 0x7f46d006a0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46e0001ec0 tx=0x7f46e0006940 comp rx=0 tx=0).stop 2024-03-22T04:11:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f0164bf0 msgr2=0x7f46f015cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f46f4c00640 1 --2- 172.21.15.86:0/2345860751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f0164bf0 0x7f46f015cbb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46f00b2e80 tx=0x7f46d4005840 comp rx=0 tx=0).stop 2024-03-22T04:11:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.748+0000 7f46ee800640 1 -- 172.21.15.86:0/2345860751 reap_dead start 2024-03-22T04:11:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.752+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f19d8070650 con 0x7f19bc071f40 2024-03-22T04:11:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.752+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 shutdown_connections 2024-03-22T04:11:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.752+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 >> 172.21.15.86:0/2345860751 conn(0x7f46f00b1df0 msgr2=0x7f46f00b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.752+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 shutdown_connections 2024-03-22T04:11:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.752+0000 7f46f4c00640 1 -- 172.21.15.86:0/2345860751 wait complete. 2024-03-22T04:11:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.752+0000 7f47c6800640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f47a8071370 0x7f47a80737b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.752+0000 7f19d4c00640 1 -- 172.21.15.86:0/3547718230 <== osd.7 v2:172.21.15.196:6817/2847924285 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f19d8070650 con 0x7f19bc071f40 2024-03-22T04:11:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.752+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47a8073e80 con 0x7f47a8071370 2024-03-22T04:11:01.772 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:01.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-22T04:11:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7f19bc071f40 msgr2=0x7f19bc0743e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7f19bc071f40 0x7f19bc0743e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19bc067b70 msgr2=0x7f19bc06a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19bc067b70 0x7f19bc06a030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0af000 tx=0x7f19cc0a7000 comp rx=0 tx=0).stop 2024-03-22T04:11:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d81567a0 msgr2=0x7f19d815b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 --2- 172.21.15.86:0/3547718230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d81567a0 0x7f19d815b830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f19c8007ef0 tx=0x7f19c800d4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19ddc00640 1 -- 172.21.15.86:0/3547718230 reap_dead start 2024-03-22T04:11:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 shutdown_connections 2024-03-22T04:11:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 >> 172.21.15.86:0/3547718230 conn(0x7f19d80b74a0 msgr2=0x7f19d81601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 shutdown_connections 2024-03-22T04:11:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.756+0000 7f19de600640 1 -- 172.21.15.86:0/3547718230 wait complete. 2024-03-22T04:11:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.760+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbef00b65c0 con 0x7fbed0071360 2024-03-22T04:11:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.760+0000 7f2864e00640 1 -- 172.21.15.86:0/3808741302 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2838002de0 con 0x7f28400712f0 2024-03-22T04:11:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.760+0000 7fbee6000640 1 -- 172.21.15.86:0/112442147 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbef00b65c0 con 0x7fbed0071360 2024-03-22T04:11:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7fbed0071360 msgr2=0x7fbed00737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7fbed0071360 0x7fbed00737a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f2856000640 1 -- 172.21.15.86:0/3808741302 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2838002de0 con 0x7f28400712f0 2024-03-22T04:11:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbed0067bf0 msgr2=0x7fbed006a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbed0067bf0 0x7fbed006a0b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbee0007d60 tx=0x7fbee0007cb0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef015e520 msgr2=0x7fbef0156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 --2- 172.21.15.86:0/112442147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef015e520 0x7fbef0156680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbee8002200 tx=0x7fbee800a4f0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbeee800640 1 -- 172.21.15.86:0/112442147 reap_dead start 2024-03-22T04:11:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 shutdown_connections 2024-03-22T04:11:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 >> 172.21.15.86:0/112442147 conn(0x7fbef008c000 msgr2=0x7fbef00b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 -- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7f28400712f0 msgr2=0x7f2840073730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 shutdown_connections 2024-03-22T04:11:01.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7f28400712f0 0x7f2840073730 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7fbef5000640 1 -- 172.21.15.86:0/112442147 wait complete. 2024-03-22T04:11:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 -- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2840067b70 msgr2=0x7f284006a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2840067b70 0x7f284006a030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28600746c0 tx=0x7f284c008040 comp rx=0 tx=0).stop 2024-03-22T04:11:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 -- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 msgr2=0x7f2860202310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 --2- 172.21.15.86:0/3808741302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28600b40b0 0x7f2860202310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2850000c10 tx=0x7f2844005840 comp rx=0 tx=0).stop 2024-03-22T04:11:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f285e800640 1 -- 172.21.15.86:0/3808741302 reap_dead start 2024-03-22T04:11:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 -- 172.21.15.86:0/3808741302 shutdown_connections 2024-03-22T04:11:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 -- 172.21.15.86:0/3808741302 >> 172.21.15.86:0/3808741302 conn(0x7f28600b5f30 msgr2=0x7f286015cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 -- 172.21.15.86:0/3808741302 shutdown_connections 2024-03-22T04:11:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.764+0000 7f284b400640 1 -- 172.21.15.86:0/3808741302 wait complete. 2024-03-22T04:11:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.768+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30240cfdb0 con 0x7f3004071390 2024-03-22T04:11:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f3020c00640 1 -- 172.21.15.86:0/89461163 <== osd.0 v2:172.21.15.86:6808/2750248043 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f30240cfdb0 con 0x7f3004071390 2024-03-22T04:11:01.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f3004071390 msgr2=0x7f30040737d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f3004071390 0x7f30040737d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067c20 msgr2=0x7f300406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067c20 0x7f300406a0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f301c001cd0 tx=0x7f301c00c040 comp rx=0 tx=0).stop 2024-03-22T04:11:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080051d0 msgr2=0x7f3024161940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 --2- 172.21.15.86:0/89461163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30080051d0 0x7f3024161940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3010006aa0 tx=0x7f3010011d60 comp rx=0 tx=0).stop 2024-03-22T04:11:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f3029600640 1 -- 172.21.15.86:0/89461163 reap_dead start 2024-03-22T04:11:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 shutdown_connections 2024-03-22T04:11:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 >> 172.21.15.86:0/89461163 conn(0x7f30240b1df0 msgr2=0x7f3024072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47c806d040 con 0x7f47a8071370 2024-03-22T04:11:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f47be000640 1 -- 172.21.15.86:0/1462595938 <== osd.4 v2:172.21.15.196:6800/655314492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f47c806d040 con 0x7f47a8071370 2024-03-22T04:11:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 shutdown_connections 2024-03-22T04:11:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.772+0000 7f302b600640 1 -- 172.21.15.86:0/89461163 wait complete. 2024-03-22T04:11:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f47a8071370 msgr2=0x7f47a80737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f47a8071370 0x7f47a80737b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f47a8067bf0 msgr2=0x7f47a806a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f47a8067bf0 0x7f47a806a0b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f47c0000f90 tx=0x7f47c00a7070 comp rx=0 tx=0).stop 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47c8071e60 msgr2=0x7f47c8159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 --2- 172.21.15.86:0/1462595938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47c8071e60 0x7f47c8159070 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47b400d5f0 tx=0x7f47b400dab0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47c6800640 1 -- 172.21.15.86:0/1462595938 reap_dead start 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 shutdown_connections 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 >> 172.21.15.86:0/1462595938 conn(0x7f47c806f910 msgr2=0x7f47c8165990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 shutdown_connections 2024-03-22T04:11:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.776+0000 7f47cd200640 1 -- 172.21.15.86:0/1462595938 wait complete. 2024-03-22T04:11:01.785 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:01.785 INFO:teuthology.orchestra.run.smithi086.stdout:64424509450 2024-03-22T04:11:01.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-22T04:11:01.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-22T04:11:01.796 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:01.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-22T04:11:01.803 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:01.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-22T04:11:01.810 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:01.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-22T04:11:01.816 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:01.816 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-22T04:11:01.823 INFO:teuthology.orchestra.run.smithi086.stdout:55834574859 2024-03-22T04:11:01.823 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-22T04:11:01.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.984+0000 7fdd7d000640 1 Processor -- start 2024-03-22T04:11:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.984+0000 7fdd7d000640 1 -- start start 2024-03-22T04:11:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.984+0000 7fdd7d000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdd78154770 0x7fdd7815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.984+0000 7fdd7d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdd78154b50 0x7fdd7806ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.984+0000 7fdd7d000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd7806d980 con 0x7fdd7806c180 2024-03-22T04:11:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.984+0000 7fdd7d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7806dc60 con 0x7fdd78154b50 2024-03-22T04:11:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.984+0000 7fdd7d000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7806df40 con 0x7fdd78154770 2024-03-22T04:11:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.984+0000 7fdd75e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdd78154770 0x7fdd7815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.988+0000 7fdd75e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdd78154770 0x7fdd7815b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52316/0 (socket says 172.21.15.86:52316) 2024-03-22T04:11:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.988+0000 7fdd75e00640 1 -- 172.21.15.86:0/1610515933 learned_addr learned my addr 172.21.15.86:0/1610515933 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.988+0000 7fdd75400640 1 --2- 172.21.15.86:0/1610515933 >> v2:172.21.15.86:3300/0 conn(0x7fdd78154b50 0x7fdd7806ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.988+0000 7fdd75400640 1 -- 172.21.15.86:0/1610515933 >> v2:172.21.15.196:3300/0 conn(0x7fdd78154770 msgr2=0x7fdd7815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.988+0000 7fdd75400640 1 --2- 172.21.15.86:0/1610515933 >> v2:172.21.15.196:3300/0 conn(0x7fdd78154770 0x7fdd7815b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.988+0000 7fdd75400640 1 -- 172.21.15.86:0/1610515933 >> v1:172.21.15.196:6789/0 conn(0x7fdd7806c180 legacy=0x7fdd780b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.988+0000 7fdd75400640 1 -- 172.21.15.86:0/1610515933 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7806e220 con 0x7fdd78154b50 2024-03-22T04:11:02.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.992+0000 7fdd75400640 1 --2- 172.21.15.86:0/1610515933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154b50 0x7fdd7806ea00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd780d0980 tx=0x7fdd7001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6246601deb888e13 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:01.996+0000 7fdd74a00640 1 -- 172.21.15.86:0/1610515933 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd700b2070 con 0x7fdd78154b50 2024-03-22T04:11:02.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.000+0000 7fdd74a00640 1 -- 172.21.15.86:0/1610515933 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd700a5720 con 0x7fdd78154b50 2024-03-22T04:11:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.000+0000 7fdd74a00640 1 -- 172.21.15.86:0/1610515933 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd700a5af0 con 0x7fdd78154b50 2024-03-22T04:11:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.000+0000 7fdd7d000640 1 -- 172.21.15.86:0/1610515933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154b50 msgr2=0x7fdd7806ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.000+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1610515933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154b50 0x7fdd7806ea00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd780d0980 tx=0x7fdd7001c880 comp rx=0 tx=0).stop 2024-03-22T04:11:02.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.000+0000 7fdd7d000640 1 -- 172.21.15.86:0/1610515933 shutdown_connections 2024-03-22T04:11:02.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.000+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1610515933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154b50 0x7fdd7806ea00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.000+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1610515933 >> v2:172.21.15.196:3300/0 conn(0x7fdd78154770 0x7fdd7815b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.000+0000 7fdd7d000640 1 -- 172.21.15.86:0/1610515933 >> 172.21.15.86:0/1610515933 conn(0x7fdd7806b660 msgr2=0x7fdd7806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3eb400640 1 Processor -- start 2024-03-22T04:11:02.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3eb400640 1 -- start start 2024-03-22T04:11:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3eb400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd3e4071c10 0x7fd3e40b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40cff70 0x7fd3e40d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3eb400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e415aac0 con 0x7fd3e40b2500 2024-03-22T04:11:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3eb400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e40d0910 con 0x7fd3e4071c10 2024-03-22T04:11:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3eb400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e40b1bc0 con 0x7fd3e40cff70 2024-03-22T04:11:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fdd7d000640 1 -- 172.21.15.86:0/1610515933 shutdown_connections 2024-03-22T04:11:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fdd7d000640 1 -- 172.21.15.86:0/1610515933 wait complete. 2024-03-22T04:11:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fdd7d000640 1 Processor -- start 2024-03-22T04:11:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fdd7d000640 1 -- start start 2024-03-22T04:11:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3e8a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd3e4071c10 0x7fd3e40b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3e8a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd3e4071c10 0x7fd3e40b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52318/0 (socket says 172.21.15.86:52318) 2024-03-22T04:11:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.004+0000 7fd3e8a00640 1 -- 172.21.15.86:0/255027069 learned_addr learned my addr 172.21.15.86:0/255027069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fd3e9400640 1 --2- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40cff70 0x7fd3e40d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fd3e8a00640 1 -- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40cff70 msgr2=0x7fd3e40d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fd3e8a00640 1 --2- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40cff70 0x7fd3e40d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd7d000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd7806c180 0x7fdd782031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fd3e8a00640 1 -- 172.21.15.86:0/255027069 >> v1:172.21.15.86:6789/0 conn(0x7fd3e40b2500 legacy=0x7fd3e415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fd3e8a00640 1 -- 172.21.15.86:0/255027069 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e40b3f40 con 0x7fd3e4071c10 2024-03-22T04:11:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fd3e9400640 1 --2- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40cff70 0x7fd3e40d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd7d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154770 0x7fdd780cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd7d000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd78154b50 0x7fdd780b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd7d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd780cf650 con 0x7fdd78154770 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd7d000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdd780cf4d0 con 0x7fdd78154b50 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd7d000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdd780b3e60 con 0x7fdd7806c180 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd75e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd7806c180 0x7fdd782031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd75e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd7806c180 0x7fdd782031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39004/0 (socket says 172.21.15.86:39004) 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd75e00640 1 -- 172.21.15.86:0/1929828292 learned_addr learned my addr 172.21.15.86:0/1929828292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd76800640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd78154b50 0x7fdd780b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd75400640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154770 0x7fdd780cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd76800640 1 -- 172.21.15.86:0/1929828292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd7806c180 msgr2=0x7fdd782031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd76800640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd7806c180 0x7fdd782031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd76800640 1 -- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154770 msgr2=0x7fdd780cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd76800640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154770 0x7fdd780cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd76800640 1 -- 172.21.15.86:0/1929828292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd780ce6a0 con 0x7fdd78154b50 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd76800640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd78154b50 0x7fdd780b3190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000c410 tx=0x7fdd6000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd6e000640 1 -- 172.21.15.86:0/1929828292 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60017070 con 0x7fdd78154b50 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd6e000640 1 -- 172.21.15.86:0/1929828292 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd60003600 con 0x7fdd78154b50 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd6e000640 1 -- 172.21.15.86:0/1929828292 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd600039f0 con 0x7fdd78154b50 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.008+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd780ce980 con 0x7fdd78154b50 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.012+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd782036e0 con 0x7fdd78154b50 2024-03-22T04:11:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.012+0000 7f1c59a00640 1 Processor -- start 2024-03-22T04:11:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.012+0000 7f1c59a00640 1 -- start start 2024-03-22T04:11:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.012+0000 7fd3e8a00640 1 --2- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e40b1680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009870 tx=0x7fd3d001c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=733d616fb89c38d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.012+0000 7f1c59a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54161bf0 0x7f1c540af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.012+0000 7fd3e3400640 1 -- 172.21.15.86:0/255027069 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00b0070 con 0x7fd3e4071c10 2024-03-22T04:11:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c59a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1c540b3800 0x7f1c540d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fd3e3400640 1 -- 172.21.15.86:0/255027069 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d00ab600 con 0x7fd3e4071c10 2024-03-22T04:11:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fd3e3400640 1 -- 172.21.15.86:0/255027069 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00ab9d0 con 0x7fd3e4071c10 2024-03-22T04:11:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c52000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1c54071c10 0x7f1c540b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45912/0 (socket says 172.21.15.86:45912) 2024-03-22T04:11:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c52000640 1 -- 172.21.15.86:0/3339956067 learned_addr learned my addr 172.21.15.86:0/3339956067 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c53400640 1 --2- 172.21.15.86:0/3339956067 >> v2:172.21.15.196:3300/0 conn(0x7f1c540b3800 0x7f1c540d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c59a00640 1 -- 172.21.15.86:0/3339956067 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c540ad200 con 0x7f1c54071c10 2024-03-22T04:11:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c59a00640 1 -- 172.21.15.86:0/3339956067 --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1c540ad4e0 con 0x7f1c540b3800 2024-03-22T04:11:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.012+0000 7fdd6e000640 1 -- 172.21.15.86:0/1929828292 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdd6001a070 con 0x7fdd78154b50 2024-03-22T04:11:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fdd6e000640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdd54067bf0 0x7fdd5406a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fdd6e000640 1 -- 172.21.15.86:0/1929828292 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdd60085460 con 0x7fdd78154b50 2024-03-22T04:11:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd78074470 con 0x7fdd78154b50 2024-03-22T04:11:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c59a00640 1 -- 172.21.15.86:0/3339956067 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1c540ad7c0 con 0x7f1c54161bf0 2024-03-22T04:11:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c52a00640 1 --2- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54161bf0 0x7f1c540af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c53400640 1 -- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54161bf0 msgr2=0x7f1c540af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fdd75e00640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdd54067bf0 0x7fdd5406a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c53400640 1 --2- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54161bf0 0x7f1c540af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c53400640 1 -- 172.21.15.86:0/3339956067 >> v1:172.21.15.196:6789/0 conn(0x7f1c54071c10 legacy=0x7f1c540b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c53400640 1 -- 172.21.15.86:0/3339956067 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c540adaa0 con 0x7f1c540b3800 2024-03-22T04:11:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fd3eb400640 1 -- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 msgr2=0x7fd3e40b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fdd75e00640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdd54067bf0 0x7fdd5406a0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006aa0 tx=0x7fdd68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fd3eb400640 1 --2- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e40b1680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009870 tx=0x7fd3d001c310 comp rx=0 tx=0).stop 2024-03-22T04:11:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fd3eb400640 1 -- 172.21.15.86:0/255027069 shutdown_connections 2024-03-22T04:11:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7f1c52a00640 1 --2- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54161bf0 0x7f1c540af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fd3eb400640 1 --2- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40cff70 0x7fd3e40d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fd3eb400640 1 --2- 172.21.15.86:0/255027069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e40b1680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.016+0000 7fd3eb400640 1 -- 172.21.15.86:0/255027069 >> 172.21.15.86:0/255027069 conn(0x7fd3e40b5830 msgr2=0x7fd3e40b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.020+0000 7fd3eb400640 1 -- 172.21.15.86:0/255027069 shutdown_connections 2024-03-22T04:11:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.020+0000 7fd3eb400640 1 -- 172.21.15.86:0/255027069 wait complete. 2024-03-22T04:11:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.020+0000 7fdd6e000640 1 -- 172.21.15.86:0/1929828292 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdd60007e20 con 0x7fdd78154b50 2024-03-22T04:11:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.020+0000 7fd3eb400640 1 Processor -- start 2024-03-22T04:11:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.020+0000 7fd3eb400640 1 -- start start 2024-03-22T04:11:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7f1c53400640 1 --2- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 0x7f1c540d09e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009d90 tx=0x7f1c4c01c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfbe1a2534579136 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e4143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40b2500 0x7fd3e4143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3eb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e40cff70 0x7fd3e40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7f1c51600640 1 -- 172.21.15.86:0/3339956067 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0b2070 con 0x7f1c540b3800 2024-03-22T04:11:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7f1c51600640 1 -- 172.21.15.86:0/3339956067 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4c0a5710 con 0x7f1c540b3800 2024-03-22T04:11:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7f1c51600640 1 -- 172.21.15.86:0/3339956067 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0b2070 con 0x7f1c540b3800 2024-03-22T04:11:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3e8a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e4143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3e8a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e4143550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52356/0 (socket says 172.21.15.86:52356) 2024-03-22T04:11:02.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7f1c59a00640 1 -- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 msgr2=0x7f1c540d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7f1c59a00640 1 --2- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 0x7f1c540d09e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009d90 tx=0x7f1c4c01c870 comp rx=0 tx=0).stop 2024-03-22T04:11:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3eb400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e40ae060 con 0x7fd3e40cff70 2024-03-22T04:11:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3eb400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e40adee0 con 0x7fd3e4071c10 2024-03-22T04:11:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3eb400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e40ae1e0 con 0x7fd3e40b2500 2024-03-22T04:11:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3e3e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40b2500 0x7fd3e4143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3e3e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40b2500 0x7fd3e4143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39024/0 (socket says 172.21.15.86:39024) 2024-03-22T04:11:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3e3e00640 1 -- 172.21.15.86:0/132554425 learned_addr learned my addr 172.21.15.86:0/132554425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3e9400640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e40cff70 0x7fd3e40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7f1c59a00640 1 -- 172.21.15.86:0/3339956067 shutdown_connections 2024-03-22T04:11:02.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7f1c59a00640 1 --2- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 0x7f1c540d09e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7f1c59a00640 1 --2- 172.21.15.86:0/3339956067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54161bf0 0x7f1c540af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7f1c59a00640 1 -- 172.21.15.86:0/3339956067 >> 172.21.15.86:0/3339956067 conn(0x7f1c540b5830 msgr2=0x7f1c540b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.024+0000 7fd3e8a00640 1 -- 172.21.15.86:0/132554425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40b2500 msgr2=0x7fd3e4143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e8a00640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40b2500 0x7fd3e4143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e8a00640 1 -- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e40cff70 msgr2=0x7fd3e40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e8a00640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e40cff70 0x7fd3e40cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e8a00640 1 -- 172.21.15.86:0/132554425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e41443d0 con 0x7fd3e4071c10 2024-03-22T04:11:02.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e3e00640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40b2500 0x7fd3e4143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:02.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e8a00640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e4143550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00ab000 tx=0x7fd3d00190b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7f1c59a00640 1 -- 172.21.15.86:0/3339956067 shutdown_connections 2024-03-22T04:11:02.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e0c00640 1 -- 172.21.15.86:0/132554425 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00b0050 con 0x7fd3e4071c10 2024-03-22T04:11:02.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e0c00640 1 -- 172.21.15.86:0/132554425 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d00b1040 con 0x7fd3e4071c10 2024-03-22T04:11:02.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3e0c00640 1 -- 172.21.15.86:0/132554425 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00b47d0 con 0x7fd3e4071c10 2024-03-22T04:11:02.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e41446b0 con 0x7fd3e4071c10 2024-03-22T04:11:02.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e4144c70 con 0x7fd3e4071c10 2024-03-22T04:11:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.028+0000 7f1c59a00640 1 -- 172.21.15.86:0/3339956067 wait complete. 2024-03-22T04:11:02.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.032+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e4068650 con 0x7fd3e4071c10 2024-03-22T04:11:02.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.032+0000 7f1c59a00640 1 Processor -- start 2024-03-22T04:11:02.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.032+0000 7f1c59a00640 1 -- start start 2024-03-22T04:11:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.032+0000 7fd3e0c00640 1 -- 172.21.15.86:0/132554425 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd3d00bc030 con 0x7fd3e4071c10 2024-03-22T04:11:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.032+0000 7fd3e0c00640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3c4067b50 0x7fd3c406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.032+0000 7f1c59a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54071c10 0x7f1c541597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c59a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 0x7f1c541575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.032+0000 7fd3e0c00640 1 -- 172.21.15.86:0/132554425 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd3d011f7e0 con 0x7fd3e4071c10 2024-03-22T04:11:02.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54071c10 0x7f1c541597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54071c10 0x7f1c541597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39040/0 (socket says 172.21.15.86:39040) 2024-03-22T04:11:02.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 -- 172.21.15.86:0/215542293 learned_addr learned my addr 172.21.15.86:0/215542293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7fd3e3e00640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3c4067b50 0x7fd3c406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c59a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54161bf0 0x7f1c5415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52000640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 0x7f1c541575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c540aee40 con 0x7f1c54161bf0 2024-03-22T04:11:02.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c540aecc0 con 0x7f1c540b3800 2024-03-22T04:11:02.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1c540aefc0 con 0x7f1c54071c10 2024-03-22T04:11:02.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 -- 172.21.15.86:0/215542293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 msgr2=0x7f1c541575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 0x7f1c541575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 -- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54161bf0 msgr2=0x7f1c5415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54161bf0 0x7f1c5415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 -- 172.21.15.86:0/215542293 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c54157b20 con 0x7f1c54071c10 2024-03-22T04:11:02.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7fd3e3e00640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3c4067b50 0x7fd3c406a010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40b24d0 tx=0x7fd3d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c52a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54071c10 0x7f1c541597c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c7f0 tx=0x7f1c4000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c4aa00640 1 -- 172.21.15.86:0/215542293 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40015070 con 0x7f1c54071c10 2024-03-22T04:11:02.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.036+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c54157dd0 con 0x7f1c54071c10 2024-03-22T04:11:02.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.040+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c54158390 con 0x7f1c54071c10 2024-03-22T04:11:02.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.040+0000 7f1c4aa00640 1 -- 172.21.15.86:0/215542293 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c40007820 con 0x7f1c54071c10 2024-03-22T04:11:02.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.040+0000 7f1c4aa00640 1 -- 172.21.15.86:0/215542293 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40007c10 con 0x7f1c54071c10 2024-03-22T04:11:02.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.040+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c0052a0 con 0x7f1c54071c10 2024-03-22T04:11:02.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.040+0000 7f1c4aa00640 1 -- 172.21.15.86:0/215542293 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1c40018070 con 0x7f1c54071c10 2024-03-22T04:11:02.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.044+0000 7fd3e0c00640 1 -- 172.21.15.86:0/132554425 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd3d00c3060 con 0x7fd3e4071c10 2024-03-22T04:11:02.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.044+0000 7f1c4aa00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c34067b50 0x7f1c3406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.044+0000 7f1c4aa00640 1 -- 172.21.15.86:0/215542293 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1c40086090 con 0x7f1c54071c10 2024-03-22T04:11:02.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.044+0000 7f1c52000640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c34067b50 0x7f1c3406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.044+0000 7f1c52000640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c34067b50 0x7f1c3406a010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c54159670 tx=0x7f1c44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.044+0000 7fdba1a00640 1 Processor -- start 2024-03-22T04:11:02.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdba1a00640 1 -- start start 2024-03-22T04:11:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7f1c4aa00640 1 -- 172.21.15.86:0/215542293 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1c40056070 con 0x7f1c54071c10 2024-03-22T04:11:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdba1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 0x7fdb9c0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdba1a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdb9c15e510 0x7fdb9c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdba1a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9c15efa0 con 0x7fdb9c0d23c0 2024-03-22T04:11:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdba1a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c0cf950 con 0x7fdb9c15e510 2024-03-22T04:11:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdba1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c15f280 con 0x7fdb9c15e130 2024-03-22T04:11:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdb9b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdb9c15e510 0x7fdb9c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdb9b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdb9c15e510 0x7fdb9c15e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52380/0 (socket says 172.21.15.86:52380) 2024-03-22T04:11:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdb9b400640 1 -- 172.21.15.86:0/1957123068 learned_addr learned my addr 172.21.15.86:0/1957123068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.048+0000 7fdb9be00640 1 --2- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 0x7fdb9c0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.052+0000 7fdb9aa00640 1 -- 172.21.15.86:0/1957123068 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314023545 0 0) 0x7fdb9c15efa0 con 0x7fdb9c0d23c0 2024-03-22T04:11:02.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.052+0000 7fdb9aa00640 1 -- 172.21.15.86:0/1957123068 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb80003530 con 0x7fdb9c0d23c0 2024-03-22T04:11:02.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.052+0000 7fdb9b400640 1 -- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 msgr2=0x7fdb9c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:02.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.052+0000 7fdb9b400640 1 --2- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 0x7fdb9c0cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.052+0000 7fdb9b400640 1 -- 172.21.15.86:0/1957123068 >> v1:172.21.15.196:6789/0 conn(0x7fdb9c0d23c0 legacy=0x7fdb9c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.052+0000 7fdb9b400640 1 -- 172.21.15.86:0/1957123068 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c15f560 con 0x7fdb9c15e510 2024-03-22T04:11:02.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.052+0000 7fdb9be00640 1 --2- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 0x7fdb9c0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:02.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdb9b400640 1 --2- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15e970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009870 tx=0x7fdb9401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3d16cca053ef57 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdb9aa00640 1 -- 172.21.15.86:0/1957123068 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940b2070 con 0x7fdb9c15e510 2024-03-22T04:11:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fbae6a00640 1 Processor -- start 2024-03-22T04:11:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fbae6a00640 1 -- start start 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdb9aa00640 1 -- 172.21.15.86:0/1957123068 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb940a9070 con 0x7fdb9c15e510 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdb9aa00640 1 -- 172.21.15.86:0/1957123068 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940ad880 con 0x7fdb9c15e510 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdba1a00640 1 -- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 msgr2=0x7fdb9c15e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15e970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009870 tx=0x7fdb9401c560 comp rx=0 tx=0).stop 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdba1a00640 1 -- 172.21.15.86:0/1957123068 shutdown_connections 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15e970 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1957123068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 0x7fdb9c0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdba1a00640 1 -- 172.21.15.86:0/1957123068 >> 172.21.15.86:0/1957123068 conn(0x7fdb9c073390 msgr2=0x7fdb9c0d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdba1a00640 1 -- 172.21.15.86:0/1957123068 shutdown_connections 2024-03-22T04:11:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbae6a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbae015b370 0x7fbae00b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbae6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae00aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbae6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbae00b4090 0x7fbae00b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae00aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.056+0000 7fdba1a00640 1 -- 172.21.15.86:0/1957123068 wait complete. 2024-03-22T04:11:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fdba1a00640 1 Processor -- start 2024-03-22T04:11:02.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae00aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39058/0 (socket says 172.21.15.86:39058) 2024-03-22T04:11:02.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbae6a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbae00afb00 con 0x7fbae00b4090 2024-03-22T04:11:02.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbae6a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbae00b5410 con 0x7fbae015b370 2024-03-22T04:11:02.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbae6a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbae00af4b0 con 0x7fbae00ae770 2024-03-22T04:11:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fdba1a00640 1 -- start start 2024-03-22T04:11:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbad7e00640 1 -- 172.21.15.86:0/2576687059 learned_addr learned my addr 172.21.15.86:0/2576687059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbadfe00640 1 --2- 172.21.15.86:0/2576687059 >> v2:172.21.15.196:3300/0 conn(0x7fbae015b370 0x7fbae00b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbad7e00640 1 -- 172.21.15.86:0/2576687059 >> v2:172.21.15.196:3300/0 conn(0x7fbae015b370 msgr2=0x7fbae00b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbad7e00640 1 --2- 172.21.15.86:0/2576687059 >> v2:172.21.15.196:3300/0 conn(0x7fbae015b370 0x7fbae00b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbad7e00640 1 -- 172.21.15.86:0/2576687059 >> v2:172.21.15.86:3300/0 conn(0x7fbae00b4090 msgr2=0x7fbae00b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbad7e00640 1 --2- 172.21.15.86:0/2576687059 >> v2:172.21.15.86:3300/0 conn(0x7fbae00b4090 0x7fbae00b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.060+0000 7fbad7e00640 1 -- 172.21.15.86:0/2576687059 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae00af790 con 0x7fbae00ae770 2024-03-22T04:11:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c0d23c0 0x7fdb9c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 0x7fdb9c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c0b2c60 con 0x7fdb9c0d23c0 2024-03-22T04:11:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c162e90 con 0x7fdb9c15e510 2024-03-22T04:11:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c0b2de0 con 0x7fdb9c15e130 2024-03-22T04:11:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52390/0 (socket says 172.21.15.86:52390) 2024-03-22T04:11:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 -- 172.21.15.86:0/1702948980 learned_addr learned my addr 172.21.15.86:0/1702948980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 -- 172.21.15.86:0/1702948980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 msgr2=0x7fdb9c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 0x7fdb9c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 -- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c0d23c0 msgr2=0x7fdb9c15a7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T04:11:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c0d23c0 0x7fdb9c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 -- 172.21.15.86:0/1702948980 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c162790 con 0x7fdb9c15e510 2024-03-22T04:11:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1000640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15d930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400c430 tx=0x7fdb8400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdb93e00640 1 -- 172.21.15.86:0/1702948980 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84028070 con 0x7fdb9c15e510 2024-03-22T04:11:02.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdb93e00640 1 -- 172.21.15.86:0/1702948980 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb840050d0 con 0x7fdb9c15e510 2024-03-22T04:11:02.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdb93e00640 1 -- 172.21.15.86:0/1702948980 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb840054c0 con 0x7fdb9c15e510 2024-03-22T04:11:02.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.064+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c15ace0 con 0x7fdb9c15e510 2024-03-22T04:11:02.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c0cbf30 con 0x7fdb9c15e510 2024-03-22T04:11:02.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbad7e00640 1 --2- 172.21.15.86:0/2576687059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae00aebd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbae0073d10 tx=0x7fbac801ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f11367369d9571 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbadf400640 1 -- 172.21.15.86:0/2576687059 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80b4070 con 0x7fbae00ae770 2024-03-22T04:11:02.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbadf400640 1 -- 172.21.15.86:0/2576687059 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac80a7ac0 con 0x7fbae00ae770 2024-03-22T04:11:02.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbadf400640 1 -- 172.21.15.86:0/2576687059 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80af440 con 0x7fbae00ae770 2024-03-22T04:11:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c074470 con 0x7fdb9c15e510 2024-03-22T04:11:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 -- 172.21.15.86:0/2576687059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 msgr2=0x7fbae00aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 --2- 172.21.15.86:0/2576687059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae00aebd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbae0073d10 tx=0x7fbac801ec30 comp rx=0 tx=0).stop 2024-03-22T04:11:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 -- 172.21.15.86:0/2576687059 shutdown_connections 2024-03-22T04:11:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 --2- 172.21.15.86:0/2576687059 >> v2:172.21.15.86:3300/0 conn(0x7fbae00b4090 0x7fbae00b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 --2- 172.21.15.86:0/2576687059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae00aebd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 --2- 172.21.15.86:0/2576687059 >> v2:172.21.15.196:3300/0 conn(0x7fbae015b370 0x7fbae00b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 -- 172.21.15.86:0/2576687059 >> 172.21.15.86:0/2576687059 conn(0x7fbae0072d20 msgr2=0x7fbae00b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 -- 172.21.15.86:0/2576687059 shutdown_connections 2024-03-22T04:11:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fbae6a00640 1 -- 172.21.15.86:0/2576687059 wait complete. 2024-03-22T04:11:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.068+0000 7fdb93e00640 1 -- 172.21.15.86:0/1702948980 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdb8402b070 con 0x7fdb9c15e510 2024-03-22T04:11:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fdb93e00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb78067bf0 0x7fdb7806a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fdb93e00640 1 -- 172.21.15.86:0/1702948980 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdb84096410 con 0x7fdb9c15e510 2024-03-22T04:11:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fbae6a00640 1 Processor -- start 2024-03-22T04:11:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fbae6a00640 1 -- start start 2024-03-22T04:11:02.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fdb9be00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb78067bf0 0x7fdb7806a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fdb9be00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb78067bf0 0x7fdb7806a0b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e250 tx=0x7fdb8c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fbae6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae0154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fbae6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae00b4090 0x7fbae0151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fbae6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae015b370 0x7fbae0147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fbae6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbae015b8d0 con 0x7fbae015b370 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fbae6a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbae015b750 con 0x7fbae00b4090 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.072+0000 7fbae6a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbae006b660 con 0x7fbae00ae770 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae0154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae0154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39080/0 (socket says 172.21.15.86:39080) 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 -- 172.21.15.86:0/166188946 learned_addr learned my addr 172.21.15.86:0/166188946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbae4a00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae015b370 0x7fbae0147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbad7e00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae00b4090 0x7fbae0151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 -- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae00b4090 msgr2=0x7fbae0151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae00b4090 0x7fbae0151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 -- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae015b370 msgr2=0x7fbae0147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae015b370 0x7fbae0147310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 -- 172.21.15.86:0/166188946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae0147850 con 0x7fbae00ae770 2024-03-22T04:11:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbad7e00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae00b4090 0x7fbae0151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbae4a00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae015b370 0x7fbae0147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadfe00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae0154320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbad000e8a0 tx=0x7fbad000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadcc00640 1 -- 172.21.15.86:0/166188946 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0017070 con 0x7fbae00ae770 2024-03-22T04:11:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadcc00640 1 -- 172.21.15.86:0/166188946 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad000c940 con 0x7fbae00ae770 2024-03-22T04:11:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbadcc00640 1 -- 172.21.15.86:0/166188946 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad000ccf0 con 0x7fbae00ae770 2024-03-22T04:11:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae01522a0 con 0x7fbae00ae770 2024-03-22T04:11:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae0152860 con 0x7fbae00ae770 2024-03-22T04:11:02.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.076+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae0068d60 con 0x7fbae00ae770 2024-03-22T04:11:02.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.080+0000 7fbadcc00640 1 -- 172.21.15.86:0/166188946 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbad001a070 con 0x7fbae00ae770 2024-03-22T04:11:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.080+0000 7fbadcc00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbabc067b20 0x7fbabc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.080+0000 7fbadcc00640 1 -- 172.21.15.86:0/166188946 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbad0086210 con 0x7fbae00ae770 2024-03-22T04:11:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.080+0000 7fbad7e00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbabc067b20 0x7fbabc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.080+0000 7fbad7e00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbabc067b20 0x7fbabc069fe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbac80b1000 tx=0x7fbac8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.080+0000 7fdb93e00640 1 -- 172.21.15.86:0/1702948980 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdb9c15ace0 con 0x7fdb9c15e510 2024-03-22T04:11:02.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.084+0000 7fbadcc00640 1 -- 172.21.15.86:0/166188946 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbad0056160 con 0x7fbae00ae770 2024-03-22T04:11:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f8454200640 1 Processor -- start 2024-03-22T04:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f8454200640 1 -- start start 2024-03-22T04:11:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f02e7200640 1 Processor -- start 2024-03-22T04:11:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f02e7200640 1 -- start start 2024-03-22T04:11:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f8454200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f844404fbf0 0x7f84440078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f8454200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84440e5440 con 0x7f84440e5060 2024-03-22T04:11:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f8454200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8444006390 con 0x7f8444007430 2024-03-22T04:11:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f8454200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8444006670 con 0x7f844404fbf0 2024-03-22T04:11:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.108+0000 7f8452200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f844404fbf0 0x7f84440078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f8451800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f84440e5060 0x7f844403a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57540/0 (socket says 172.21.15.86:57540) 2024-03-22T04:11:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f8451800640 1 -- 172.21.15.86:0/273864562 learned_addr learned my addr 172.21.15.86:0/273864562 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f02e7200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f02e81445c0 0x7f02e80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f02e7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e80b41f0 con 0x7f02e806e410 2024-03-22T04:11:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f02e7200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e80b44d0 con 0x7f02e8143d20 2024-03-22T04:11:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f02e7200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f02e80b47b0 con 0x7f02e81445c0 2024-03-22T04:11:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f8452200640 1 -- 172.21.15.86:0/273864562 >> v1:172.21.15.196:6789/0 conn(0x7f8444007430 legacy=0x7f844404f4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T04:11:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f02e5e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f02e8143d20 0x7f02e815f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45950/0 (socket says 172.21.15.86:45950) 2024-03-22T04:11:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f02e5e00640 1 -- 172.21.15.86:0/4268175645 learned_addr learned my addr 172.21.15.86:0/4268175645 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f02e5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f02e81445c0 0x7f02e80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f8452200640 1 -- 172.21.15.86:0/273864562 >> v1:172.21.15.86:6789/0 conn(0x7f84440e5060 legacy=0x7f844403a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.112+0000 7f8452200640 1 -- 172.21.15.86:0/273864562 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8444006950 con 0x7f844404fbf0 2024-03-22T04:11:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.116+0000 7f8452200640 1 --2- 172.21.15.86:0/273864562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844404fbf0 0x7f84440078b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f844c069d60 tx=0x7f844c097aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b692ac5fd49dbc37 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.116+0000 7f02e4a00640 1 -- 172.21.15.86:0/4268175645 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584509689 0 0) 0x7f02e80b44d0 con 0x7f02e8143d20 2024-03-22T04:11:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.116+0000 7f02e4a00640 1 -- 172.21.15.86:0/4268175645 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02cc003530 con 0x7f02e8143d20 2024-03-22T04:11:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.116+0000 7f02e5400640 1 -- 172.21.15.86:0/4268175645 >> v1:172.21.15.196:6789/0 conn(0x7f02e8143d20 legacy=0x7f02e815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.116+0000 7f02e5400640 1 -- 172.21.15.86:0/4268175645 >> v1:172.21.15.86:6789/0 conn(0x7f02e806e410 legacy=0x7f02e80b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.116+0000 7f02e5400640 1 -- 172.21.15.86:0/4268175645 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e80b4a90 con 0x7f02e81445c0 2024-03-22T04:11:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f844be00640 1 -- 172.21.15.86:0/273864562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c120810 con 0x7f844404fbf0 2024-03-22T04:11:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f844be00640 1 -- 172.21.15.86:0/273864562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c1209b0 con 0x7f844404fbf0 2024-03-22T04:11:02.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f844be00640 1 -- 172.21.15.86:0/273864562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c120e40 con 0x7f844404fbf0 2024-03-22T04:11:02.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f02e5400640 1 --2- 172.21.15.86:0/4268175645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e81445c0 0x7f02e80ce720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f02d400ba60 tx=0x7f02d4020c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701fd0fdb125dd08 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f8454200640 1 -- 172.21.15.86:0/273864562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844404fbf0 msgr2=0x7f84440078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f8454200640 1 --2- 172.21.15.86:0/273864562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844404fbf0 0x7f84440078b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f844c069d60 tx=0x7f844c097aa0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f8454200640 1 -- 172.21.15.86:0/273864562 shutdown_connections 2024-03-22T04:11:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f02e4a00640 1 -- 172.21.15.86:0/4268175645 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d40b6070 con 0x7f02e81445c0 2024-03-22T04:11:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f8454200640 1 --2- 172.21.15.86:0/273864562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844404fbf0 0x7f84440078b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f02e4a00640 1 -- 172.21.15.86:0/4268175645 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d40a99f0 con 0x7f02e81445c0 2024-03-22T04:11:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f02e4a00640 1 -- 172.21.15.86:0/4268175645 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d40a9dc0 con 0x7f02e81445c0 2024-03-22T04:11:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f8454200640 1 -- 172.21.15.86:0/273864562 >> 172.21.15.86:0/273864562 conn(0x7f844400a410 msgr2=0x7f844400a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f8454200640 1 -- 172.21.15.86:0/273864562 shutdown_connections 2024-03-22T04:11:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f8454200640 1 -- 172.21.15.86:0/273864562 wait complete. 2024-03-22T04:11:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f02e7200640 1 -- 172.21.15.86:0/4268175645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e81445c0 msgr2=0x7f02e80ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.120+0000 7f02e7200640 1 --2- 172.21.15.86:0/4268175645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e81445c0 0x7f02e80ce720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f02d400ba60 tx=0x7f02d4020c40 comp rx=0 tx=0).stop 2024-03-22T04:11:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.124+0000 7f02e7200640 1 -- 172.21.15.86:0/4268175645 shutdown_connections 2024-03-22T04:11:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.124+0000 7f02e7200640 1 --2- 172.21.15.86:0/4268175645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e81445c0 0x7f02e80ce720 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.124+0000 7f02e7200640 1 -- 172.21.15.86:0/4268175645 >> 172.21.15.86:0/4268175645 conn(0x7f02e80d0f70 msgr2=0x7f02e80d1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.124+0000 7f02e7200640 1 -- 172.21.15.86:0/4268175645 shutdown_connections 2024-03-22T04:11:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.124+0000 7f02e7200640 1 -- 172.21.15.86:0/4268175645 wait complete. 2024-03-22T04:11:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.124+0000 7f8454200640 1 Processor -- start 2024-03-22T04:11:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.124+0000 7f8454200640 1 -- start start 2024-03-22T04:11:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.124+0000 7f02e7200640 1 Processor -- start 2024-03-22T04:11:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8454200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8444007430 0x7f84440e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8454200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f844404fbf0 0x7f844404e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f02e7200640 1 -- start start 2024-03-22T04:11:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8454200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440e5060 0x7f8444053be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8444007430 0x7f84440e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8454200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8444034530 con 0x7f84440e5060 2024-03-22T04:11:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8454200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84440343b0 con 0x7f8444007430 2024-03-22T04:11:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8444007430 0x7f84440e45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52408/0 (socket says 172.21.15.86:52408) 2024-03-22T04:11:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 -- 172.21.15.86:0/207449096 learned_addr learned my addr 172.21.15.86:0/207449096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8450e00640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f844404fbf0 0x7f844404e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f84440346b0 con 0x7f844404fbf0 2024-03-22T04:11:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8452200640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440e5060 0x7f8444053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 -- 172.21.15.86:0/207449096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f844404fbf0 msgr2=0x7f844404e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f844404fbf0 0x7f844404e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 -- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440e5060 msgr2=0x7f8444053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440e5060 0x7f8444053be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 -- 172.21.15.86:0/207449096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844404d280 con 0x7f8444007430 2024-03-22T04:11:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f02e7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e806e410 0x7f02e80cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8452200640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440e5060 0x7f8444053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8451800640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8444007430 0x7f84440e45c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f844000c810 tx=0x7f844000ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8449600640 1 -- 172.21.15.86:0/207449096 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440018070 con 0x7f8444007430 2024-03-22T04:11:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8449600640 1 -- 172.21.15.86:0/207449096 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84400074e0 con 0x7f8444007430 2024-03-22T04:11:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f02e7200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e8143d20 0x7f02e80c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8449600640 1 -- 172.21.15.86:0/207449096 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440004a30 con 0x7f8444007430 2024-03-22T04:11:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f02e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e806e410 0x7f02e80cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f02e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e806e410 0x7f02e80cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56198/0 (socket says 172.21.15.86:56198) 2024-03-22T04:11:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f02e5e00640 1 -- 172.21.15.86:0/2916185931 learned_addr learned my addr 172.21.15.86:0/2916185931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f02e7200640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e81445c0 0x7f02e80ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844404d560 con 0x7f8444007430 2024-03-22T04:11:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.128+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844404db20 con 0x7f8444007430 2024-03-22T04:11:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e6800640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e81445c0 0x7f02e80ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e5400640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e8143d20 0x7f02e80c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8144280 con 0x7f02e806e410 2024-03-22T04:11:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8144100 con 0x7f02e8143d20 2024-03-22T04:11:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f02e806be70 con 0x7f02e81445c0 2024-03-22T04:11:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e6800640 1 -- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e8143d20 msgr2=0x7f02e80c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e6800640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e8143d20 0x7f02e80c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e6800640 1 -- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e806e410 msgr2=0x7f02e80cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e6800640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e806e410 0x7f02e80cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e6800640 1 -- 172.21.15.86:0/2916185931 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e80ccf50 con 0x7f02e81445c0 2024-03-22T04:11:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e6800640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e81445c0 0x7f02e80ca0a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c760 tx=0x7f02e000cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02de000640 1 -- 172.21.15.86:0/2916185931 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0016020 con 0x7f02e81445c0 2024-03-22T04:11:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02de000640 1 -- 172.21.15.86:0/2916185931 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e0006d60 con 0x7f02e81445c0 2024-03-22T04:11:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02de000640 1 -- 172.21.15.86:0/2916185931 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0003bc0 con 0x7f02e81445c0 2024-03-22T04:11:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e80c84d0 con 0x7f02e81445c0 2024-03-22T04:11:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e80c8a90 con 0x7f02e81445c0 2024-03-22T04:11:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f02e5400640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e8143d20 0x7f02e80c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.132+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8444008e30 con 0x7f8444007430 2024-03-22T04:11:02.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.136+0000 7f02de000640 1 -- 172.21.15.86:0/2916185931 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f02e000a850 con 0x7f02e81445c0 2024-03-22T04:11:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.136+0000 7f8449600640 1 -- 172.21.15.86:0/207449096 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8440004bd0 con 0x7f8444007430 2024-03-22T04:11:02.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.136+0000 7f8449600640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f842c067b20 0x7f842c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.140+0000 7f8449600640 1 -- 172.21.15.86:0/207449096 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8440085810 con 0x7f8444007430 2024-03-22T04:11:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.136+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e80c6010 con 0x7f02e81445c0 2024-03-22T04:11:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.136+0000 7f02de000640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02c8067b20 0x7f02c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.136+0000 7f02de000640 1 -- 172.21.15.86:0/2916185931 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f02e0085d70 con 0x7f02e81445c0 2024-03-22T04:11:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.140+0000 7f02e5e00640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02c8067b20 0x7f02c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.140+0000 7f8450e00640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f842c067b20 0x7f842c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.140+0000 7f8450e00640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f842c067b20 0x7f842c069fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8444007d00 tx=0x7f8438009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.140+0000 7f02e5e00640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02c8067b20 0x7f02c8069fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02d8007fd0 tx=0x7f02d8007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.144+0000 7f8449600640 1 -- 172.21.15.86:0/207449096 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f84440e5440 con 0x7f8444007430 2024-03-22T04:11:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.144+0000 7f02de000640 1 -- 172.21.15.86:0/2916185931 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f02e0020020 con 0x7f02e81445c0 2024-03-22T04:11:02.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.160+0000 7fa98e200640 1 Processor -- start 2024-03-22T04:11:02.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.160+0000 7fa98e200640 1 -- start start 2024-03-22T04:11:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa98816eb70 0x7fa98816ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9880cc080 0x7fa98806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98e200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa9880cc4c0 0x7fa98806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa9880acda0 con 0x7fa98816eb70 2024-03-22T04:11:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98e200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa98816b4c0 con 0x7fa9880cc4c0 2024-03-22T04:11:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98e200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa9880cdad0 con 0x7fa9880cc080 2024-03-22T04:11:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98d800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa9880cc4c0 0x7fa98806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98d800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa9880cc4c0 0x7fa98806d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52420/0 (socket says 172.21.15.86:52420) 2024-03-22T04:11:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98d800640 1 -- 172.21.15.86:0/2857667018 learned_addr learned my addr 172.21.15.86:0/2857667018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98ce00640 1 --2- 172.21.15.86:0/2857667018 >> v2:172.21.15.86:3300/0 conn(0x7fa98816eb70 0x7fa98816ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98d800640 1 -- 172.21.15.86:0/2857667018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9880cc080 msgr2=0x7fa98806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98d800640 1 --2- 172.21.15.86:0/2857667018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9880cc080 0x7fa98806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98d800640 1 -- 172.21.15.86:0/2857667018 >> v2:172.21.15.86:3300/0 conn(0x7fa98816eb70 msgr2=0x7fa98816ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98d800640 1 --2- 172.21.15.86:0/2857667018 >> v2:172.21.15.86:3300/0 conn(0x7fa98816eb70 0x7fa98816ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.164+0000 7fa98d800640 1 -- 172.21.15.86:0/2857667018 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98816f490 con 0x7fa9880cc4c0 2024-03-22T04:11:02.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98d800640 1 --2- 172.21.15.86:0/2857667018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc4c0 0x7fa98806d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa98806ce20 tx=0x7fa97001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac1f0ba023eef9fe server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa987400640 1 -- 172.21.15.86:0/2857667018 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9700b5070 con 0x7fa9880cc4c0 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa987400640 1 -- 172.21.15.86:0/2857667018 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9700a89f0 con 0x7fa9880cc4c0 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa987400640 1 -- 172.21.15.86:0/2857667018 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9700a8e00 con 0x7fa9880cc4c0 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 -- 172.21.15.86:0/2857667018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc4c0 msgr2=0x7fa98806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 --2- 172.21.15.86:0/2857667018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc4c0 0x7fa98806d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa98806ce20 tx=0x7fa97001ed30 comp rx=0 tx=0).stop 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 -- 172.21.15.86:0/2857667018 shutdown_connections 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 --2- 172.21.15.86:0/2857667018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc4c0 0x7fa98806d9c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 --2- 172.21.15.86:0/2857667018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9880cc080 0x7fa98806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 --2- 172.21.15.86:0/2857667018 >> v2:172.21.15.86:3300/0 conn(0x7fa98816eb70 0x7fa98816ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 -- 172.21.15.86:0/2857667018 >> 172.21.15.86:0/2857667018 conn(0x7fa9880cc8e0 msgr2=0x7fa9880cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 -- 172.21.15.86:0/2857667018 shutdown_connections 2024-03-22T04:11:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.172+0000 7fa98e200640 1 -- 172.21.15.86:0/2857667018 wait complete. 2024-03-22T04:11:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.176+0000 7fa98e200640 1 Processor -- start 2024-03-22T04:11:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.176+0000 7fa98e200640 1 -- start start 2024-03-22T04:11:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.176+0000 7fa98e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc080 0x7fa988168930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.176+0000 7fa98e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa98816eb70 0x7fa98809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98816d480 0x7fa98816a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc080 0x7fa988168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc080 0x7fa988168930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:52428/0 (socket says 172.21.15.86:52428) 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa987e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa98816eb70 0x7fa98809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa987e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa98816eb70 0x7fa98809aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39118/0 (socket says 172.21.15.86:39118) 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98ce00640 1 -- 172.21.15.86:0/1706703946 learned_addr learned my addr 172.21.15.86:0/1706703946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98e200640 1 -- 172.21.15.86:0/1706703946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9880cdd70 con 0x7fa98816d480 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98e200640 1 -- 172.21.15.86:0/1706703946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9880ca790 con 0x7fa9880cc080 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98e200640 1 -- 172.21.15.86:0/1706703946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa98816f8c0 con 0x7fa98816eb70 2024-03-22T04:11:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98d800640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98816d480 0x7fa98816a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa987e00640 1 -- 172.21.15.86:0/1706703946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc080 msgr2=0x7fa988168930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa987e00640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc080 0x7fa988168930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa987e00640 1 -- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98816d480 msgr2=0x7fa98816a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa987e00640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98816d480 0x7fa98816a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa987e00640 1 -- 172.21.15.86:0/1706703946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9880b35a0 con 0x7fa98816eb70 2024-03-22T04:11:02.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa987e00640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa98816eb70 0x7fa98809aa70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa97800bd70 tx=0x7fa97800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa984c00640 1 -- 172.21.15.86:0/1706703946 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978019070 con 0x7fa98816eb70 2024-03-22T04:11:02.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa984c00640 1 -- 172.21.15.86:0/1706703946 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978014440 con 0x7fa98816eb70 2024-03-22T04:11:02.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98e200640 1 -- 172.21.15.86:0/1706703946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa988168e70 con 0x7fa98816eb70 2024-03-22T04:11:02.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa98e200640 1 -- 172.21.15.86:0/1706703946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98809b5a0 con 0x7fa98816eb70 2024-03-22T04:11:02.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.180+0000 7fa984c00640 1 -- 172.21.15.86:0/1706703946 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978014830 con 0x7fa98816eb70 2024-03-22T04:11:02.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.184+0000 7fa984c00640 1 -- 172.21.15.86:0/1706703946 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa97801c070 con 0x7fa98816eb70 2024-03-22T04:11:02.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.184+0000 7fa984c00640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa964067b20 0x7fa964069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.184+0000 7fa984c00640 1 -- 172.21.15.86:0/1706703946 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa9780861d0 con 0x7fa98816eb70 2024-03-22T04:11:02.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.188+0000 7fa98ce00640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa964067b20 0x7fa964069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.188+0000 7fa98ce00640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa964067b20 0x7fa964069fe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa980010cf0 tx=0x7fa980008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.192+0000 7fa97e000640 1 -- 172.21.15.86:0/1706703946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa988074470 con 0x7fa98816eb70 2024-03-22T04:11:02.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.196+0000 7fa984c00640 1 -- 172.21.15.86:0/1706703946 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa978056890 con 0x7fa98816eb70 2024-03-22T04:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.348+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdd78071810 con 0x7fdd78154b50 2024-03-22T04:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.348+0000 7fdd6e000640 1 -- 172.21.15.86:0/1929828292 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdd60055550 con 0x7fdd78154b50 2024-03-22T04:11:02.352 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:02.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdd54067bf0 msgr2=0x7fdd5406a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdd54067bf0 0x7fdd5406a0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006aa0 tx=0x7fdd68008040 comp rx=0 tx=0).stop 2024-03-22T04:11:02.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd78154b50 msgr2=0x7fdd780b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd78154b50 0x7fdd780b3190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000c410 tx=0x7fdd6000c8d0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 shutdown_connections 2024-03-22T04:11:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdd54067bf0 0x7fdd5406a0b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd78154b50 0x7fdd780b3190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd78154770 0x7fdd780cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 --2- 172.21.15.86:0/1929828292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd7806c180 0x7fdd782031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 >> 172.21.15.86:0/1929828292 conn(0x7fdd7806b660 msgr2=0x7fdd7815dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 shutdown_connections 2024-03-22T04:11:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.352+0000 7fdd7d000640 1 -- 172.21.15.86:0/1929828292 wait complete. 2024-03-22T04:11:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.380+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd3e406d9c0 con 0x7fd3e4071c10 2024-03-22T04:11:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.380+0000 7fd3e0c00640 1 -- 172.21.15.86:0/132554425 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd3e415aac0 con 0x7fd3e4071c10 2024-03-22T04:11:02.383 INFO:teuthology.orchestra.run.smithi086.stdout:64424509450 2024-03-22T04:11:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3c4067b50 msgr2=0x7fd3c406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3c4067b50 0x7fd3c406a010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40b24d0 tx=0x7fd3d8009040 comp rx=0 tx=0).stop 2024-03-22T04:11:02.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 msgr2=0x7fd3e4143550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e4143550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00ab000 tx=0x7fd3d00190b0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 shutdown_connections 2024-03-22T04:11:02.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3c4067b50 0x7fd3c406a010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3e40cff70 0x7fd3e40cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3e40b2500 0x7fd3e4143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 --2- 172.21.15.86:0/132554425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3e4071c10 0x7fd3e4143550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 >> 172.21.15.86:0/132554425 conn(0x7fd3e40b5830 msgr2=0x7fd3e4151740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 shutdown_connections 2024-03-22T04:11:02.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.2 2024-03-22T04:11:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.384+0000 7fd3eb400640 1 -- 172.21.15.86:0/132554425 wait complete. 2024-03-22T04:11:02.389 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:02.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.392+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1c2c005590 con 0x7f1c54071c10 2024-03-22T04:11:02.395 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:02.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.392+0000 7f1c4aa00640 1 -- 172.21.15.86:0/215542293 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1c4005a110 con 0x7f1c54071c10 2024-03-22T04:11:02.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c34067b50 msgr2=0x7f1c3406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c34067b50 0x7f1c3406a010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c54159670 tx=0x7f1c44009040 comp rx=0 tx=0).stop 2024-03-22T04:11:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54071c10 msgr2=0x7f1c541597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54071c10 0x7f1c541597c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c7f0 tx=0x7f1c4000ccb0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 shutdown_connections 2024-03-22T04:11:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c34067b50 0x7f1c3406a010 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c540b3800 0x7f1c541575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54161bf0 0x7f1c5415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 --2- 172.21.15.86:0/215542293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c54071c10 0x7f1c541597c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 >> 172.21.15.86:0/215542293 conn(0x7f1c540b5830 msgr2=0x7f1c5406ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 shutdown_connections 2024-03-22T04:11:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.396+0000 7f1c59a00640 1 -- 172.21.15.86:0/215542293 wait complete. 2024-03-22T04:11:02.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdb9c0b33b0 con 0x7fdb9c15e510 2024-03-22T04:11:02.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdb93e00640 1 -- 172.21.15.86:0/1702948980 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb84066470 con 0x7fdb9c15e510 2024-03-22T04:11:02.407 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb78067bf0 msgr2=0x7fdb7806a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb78067bf0 0x7fdb7806a0b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e250 tx=0x7fdb8c00b040 comp rx=0 tx=0).stop 2024-03-22T04:11:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 msgr2=0x7fdb9c15d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15d930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400c430 tx=0x7fdb8400c8f0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 shutdown_connections 2024-03-22T04:11:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb78067bf0 0x7fdb7806a0b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb9c15e510 0x7fdb9c15d930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb9c15e130 0x7fdb9c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 --2- 172.21.15.86:0/1702948980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c0d23c0 0x7fdb9c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 >> 172.21.15.86:0/1702948980 conn(0x7fdb9c073390 msgr2=0x7fdb9c0b92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.404+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 shutdown_connections 2024-03-22T04:11:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.408+0000 7fdba1a00640 1 -- 172.21.15.86:0/1702948980 wait complete. 2024-03-22T04:11:02.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.6 2024-03-22T04:11:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.416+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbae0165e20 con 0x7fbae00ae770 2024-03-22T04:11:02.422 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:02.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.416+0000 7fbadcc00640 1 -- 172.21.15.86:0/166188946 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbad005a200 con 0x7fbae00ae770 2024-03-22T04:11:02.422 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbabc067b20 msgr2=0x7fbabc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbabc067b20 0x7fbabc069fe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbac80b1000 tx=0x7fbac8008040 comp rx=0 tx=0).stop 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 msgr2=0x7fbae0154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae0154320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbad000e8a0 tx=0x7fbad000eca0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 shutdown_connections 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbabc067b20 0x7fbabc069fe0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae00b4090 0x7fbae0151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae015b370 0x7fbae0147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 --2- 172.21.15.86:0/166188946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae00ae770 0x7fbae0154320 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbad000e8a0 tx=0x7fbad000eca0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.420+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 >> 172.21.15.86:0/166188946 conn(0x7fbae0072d20 msgr2=0x7fbae00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.424+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 shutdown_connections 2024-03-22T04:11:02.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.424+0000 7fbae6a00640 1 -- 172.21.15.86:0/166188946 wait complete. 2024-03-22T04:11:02.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.1 2024-03-22T04:11:02.435 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:02.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.7 2024-03-22T04:11:02.437 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.452+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f84440057f0 con 0x7f8444007430 2024-03-22T04:11:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.452+0000 7f8449600640 1 -- 172.21.15.86:0/207449096 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8440055880 con 0x7f8444007430 2024-03-22T04:11:02.457 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:02.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.452+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f02e8074470 con 0x7f02e81445c0 2024-03-22T04:11:02.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.452+0000 7f02de000640 1 -- 172.21.15.86:0/2916185931 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02e0056430 con 0x7f02e81445c0 2024-03-22T04:11:02.458 INFO:teuthology.orchestra.run.smithi086.stdout:64424509451 2024-03-22T04:11:02.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.3 2024-03-22T04:11:02.460 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:02.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f842c067b20 msgr2=0x7f842c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f842c067b20 0x7f842c069fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8444007d00 tx=0x7f8438009060 comp rx=0 tx=0).stop 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8444007430 msgr2=0x7f84440e45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8444007430 0x7f84440e45c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f844000c810 tx=0x7f844000ccd0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 shutdown_connections 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f842c067b20 0x7f842c069fe0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440e5060 0x7f8444053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f844404fbf0 0x7f844404e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 --2- 172.21.15.86:0/207449096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8444007430 0x7f84440e45c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 >> 172.21.15.86:0/207449096 conn(0x7f844400a410 msgr2=0x7f844404f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 shutdown_connections 2024-03-22T04:11:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f8454200640 1 -- 172.21.15.86:0/207449096 wait complete. 2024-03-22T04:11:02.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02c8067b20 msgr2=0x7f02c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02c8067b20 0x7f02c8069fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02d8007fd0 tx=0x7f02d8007e20 comp rx=0 tx=0).stop 2024-03-22T04:11:02.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e81445c0 msgr2=0x7f02e80ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e81445c0 0x7f02e80ca0a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c760 tx=0x7f02e000cc20 comp rx=0 tx=0).stop 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 shutdown_connections 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f02c8067b20 0x7f02c8069fe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02e8143d20 0x7f02e80c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02e81445c0 0x7f02e80ca0a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 --2- 172.21.15.86:0/2916185931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02e806e410 0x7f02e80cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 >> 172.21.15.86:0/2916185931 conn(0x7f02e80d0f70 msgr2=0x7f02e80b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 shutdown_connections 2024-03-22T04:11:02.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.456+0000 7f02e7200640 1 -- 172.21.15.86:0/2916185931 wait complete. 2024-03-22T04:11:02.482 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.5 2024-03-22T04:11:02.482 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:02.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.0 2024-03-22T04:11:02.483 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 -- 172.21.15.86:0/1706703946 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa98816f6a0 con 0x7fa98816eb70 2024-03-22T04:11:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa984c00640 1 -- 172.21.15.86:0/1706703946 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa978056190 con 0x7fa98816eb70 2024-03-22T04:11:02.491 INFO:teuthology.orchestra.run.smithi086.stdout:55834574859 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 -- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa964067b20 msgr2=0x7fa964069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa964067b20 0x7fa964069fe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa980010cf0 tx=0x7fa980008040 comp rx=0 tx=0).stop 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 -- 172.21.15.86:0/1706703946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa98816eb70 msgr2=0x7fa98809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa98816eb70 0x7fa98809aa70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa97800bd70 tx=0x7fa97800be60 comp rx=0 tx=0).stop 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 -- 172.21.15.86:0/1706703946 shutdown_connections 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa964067b20 0x7fa964069fe0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98816d480 0x7fa98816a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa98816eb70 0x7fa98809aa70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 --2- 172.21.15.86:0/1706703946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9880cc080 0x7fa988168930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 -- 172.21.15.86:0/1706703946 >> 172.21.15.86:0/1706703946 conn(0x7fa9880cc8e0 msgr2=0x7fa9880acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.488+0000 7fa97e000640 1 -- 172.21.15.86:0/1706703946 shutdown_connections 2024-03-22T04:11:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.492+0000 7fa97e000640 1 -- 172.21.15.86:0/1706703946 wait complete. 2024-03-22T04:11:02.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-03-22T04:11:02.512 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:02.512 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-22T04:11:02.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:11:02.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.688+0000 7fe2ccc00640 1 Processor -- start 2024-03-22T04:11:02.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.688+0000 7fe2ccc00640 1 -- start start 2024-03-22T04:11:02.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.688+0000 7fe2ccc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe2c80730d0 0x7fe2c80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.688+0000 7fe2ccc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c8071920 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.688+0000 7fe2ccc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c8071c00 con 0x7fe2c815b3f0 2024-03-22T04:11:02.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2ccc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c80716d0 con 0x7fe2c80730d0 2024-03-22T04:11:02.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c6800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2c80cc3e0 0x7fe2c80b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57556/0 (socket says 172.21.15.86:57556) 2024-03-22T04:11:02.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c6800640 1 -- 172.21.15.86:0/905165753 learned_addr learned my addr 172.21.15.86:0/905165753 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:02.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c5400640 1 --2- 172.21.15.86:0/905165753 >> v2:172.21.15.196:3300/0 conn(0x7fe2c80730d0 0x7fe2c80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c4a00640 1 -- 172.21.15.86:0/905165753 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3910221695 0 0) 0x7fe2c8071920 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c4a00640 1 -- 172.21.15.86:0/905165753 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2ac003530 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c4a00640 1 -- 172.21.15.86:0/905165753 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182229915 0 0) 0x7fe2c8071c00 con 0x7fe2c815b3f0 2024-03-22T04:11:02.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c5400640 1 -- 172.21.15.86:0/905165753 >> v1:172.21.15.196:6789/0 conn(0x7fe2c815b3f0 legacy=0x7fe2c80729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c5400640 1 -- 172.21.15.86:0/905165753 >> v1:172.21.15.86:6789/0 conn(0x7fe2c80cc3e0 legacy=0x7fe2c80b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c5400640 1 -- 172.21.15.86:0/905165753 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c8071ee0 con 0x7fe2c80730d0 2024-03-22T04:11:02.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c5400640 1 --2- 172.21.15.86:0/905165753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c80730d0 0x7fe2c80ade30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40097c0 tx=0x7fe2b401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=990642476fbc27ad server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c4a00640 1 -- 172.21.15.86:0/905165753 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40b0070 con 0x7fe2c80730d0 2024-03-22T04:11:02.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c4a00640 1 -- 172.21.15.86:0/905165753 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b40ab600 con 0x7fe2c80730d0 2024-03-22T04:11:02.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.692+0000 7fe2c4a00640 1 -- 172.21.15.86:0/905165753 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40ab9b0 con 0x7fe2c80730d0 2024-03-22T04:11:02.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- 172.21.15.86:0/905165753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c80730d0 msgr2=0x7fe2c80ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 --2- 172.21.15.86:0/905165753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c80730d0 0x7fe2c80ade30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40097c0 tx=0x7fe2b401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- 172.21.15.86:0/905165753 shutdown_connections 2024-03-22T04:11:02.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 --2- 172.21.15.86:0/905165753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c80730d0 0x7fe2c80ade30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- 172.21.15.86:0/905165753 >> 172.21.15.86:0/905165753 conn(0x7fe2c80b1df0 msgr2=0x7fe2c80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- 172.21.15.86:0/905165753 shutdown_connections 2024-03-22T04:11:02.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- 172.21.15.86:0/905165753 wait complete. 2024-03-22T04:11:02.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 Processor -- start 2024-03-22T04:11:02.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- start start 2024-03-22T04:11:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2c80730d0 0x7fe2c81f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c80cc3e0 0x7fe2c80ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c815b3f0 0x7fe2c815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c80b30f0 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c80b3270 con 0x7fe2c815b3f0 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2ccc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c80b33f0 con 0x7fe2c80730d0 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c80cc3e0 0x7fe2c80ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c80cc3e0 0x7fe2c80ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56216/0 (socket says 172.21.15.86:56216) 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2c5400640 1 -- 172.21.15.86:0/4036027246 learned_addr learned my addr 172.21.15.86:0/4036027246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2c5e00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2c80730d0 0x7fe2c81f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2c6800640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c815b3f0 0x7fe2c815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.696+0000 7fe2c5400640 1 -- 172.21.15.86:0/4036027246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2c80730d0 msgr2=0x7fe2c81f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2c5400640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2c80730d0 0x7fe2c81f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2c5400640 1 -- 172.21.15.86:0/4036027246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c815b3f0 msgr2=0x7fe2c815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2c5400640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c815b3f0 0x7fe2c815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2c5400640 1 -- 172.21.15.86:0/4036027246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c80caf00 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2c5400640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c80cc3e0 0x7fe2c80ca4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4000c10 tx=0x7fe2b4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2be000640 1 -- 172.21.15.86:0/4036027246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40b0040 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2be000640 1 -- 172.21.15.86:0/4036027246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b40ab400 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c80cb1e0 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2be000640 1 -- 172.21.15.86:0/4036027246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4006e20 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c80cb7a0 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c8068650 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2be000640 1 -- 172.21.15.86:0/4036027246 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe2b40bc020 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2be000640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe2a8067b20 0x7fe2a8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2be000640 1 -- 172.21.15.86:0/4036027246 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe2b411f930 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.700+0000 7fe2c5e00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe2a8067b20 0x7fe2a8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.704+0000 7fe2c5e00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe2a8067b20 0x7fe2a8069fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80014f0 tx=0x7fe2b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.704+0000 7fe2be000640 1 -- 172.21.15.86:0/4036027246 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe2b4016090 con 0x7fe2c80cc3e0 2024-03-22T04:11:02.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.868+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2c80b4010 con 0x7fe2a8067b20 2024-03-22T04:11:02.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.876+0000 7fe2be000640 1 -- 172.21.15.86:0/4036027246 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fe2c80b4010 con 0x7fe2a8067b20 2024-03-22T04:11:02.881 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:11:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe2a8067b20 msgr2=0x7fe2a8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe2a8067b20 0x7fe2a8069fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80014f0 tx=0x7fe2b8009040 comp rx=0 tx=0).stop 2024-03-22T04:11:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c80cc3e0 msgr2=0x7fe2c80ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c80cc3e0 0x7fe2c80ca4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4000c10 tx=0x7fe2b4019240 comp rx=0 tx=0).stop 2024-03-22T04:11:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 shutdown_connections 2024-03-22T04:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe2a8067b20 0x7fe2a8069fe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c815b3f0 0x7fe2c815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2c80cc3e0 0x7fe2c80ca4b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 --2- 172.21.15.86:0/4036027246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2c80730d0 0x7fe2c81f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 >> 172.21.15.86:0/4036027246 conn(0x7fe2c80b1df0 msgr2=0x7fe2c80aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 shutdown_connections 2024-03-22T04:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:02.880+0000 7fe2ccc00640 1 -- 172.21.15.86:0/4036027246 wait complete. 2024-03-22T04:11:02.902 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-03-22T04:11:01.797157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219592,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755128,"statfs":{"total":773094113280,"available":772869251072,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002770"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967926+0000","last_change":"2024-03-22T04:10:33.930897+0000","last_active":"2024-03-22T04:10:39.967926+0000","last_peered":"2024-03-22T04:10:39.967926+0000","last_clean":"2024-03-22T04:10:39.967926+0000","last_became_active":"2024-03-22T04:10:33.930543+0000","last_became_peered":"2024-03-22T04:10:33.930543+0000","last_unstale":"2024-03-22T04:10:39.967926+0000","last_undegraded":"2024-03-22T04:10:39.967926+0000","last_fullsized":"2024-03-22T04:10:39.967926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:57:52.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170514+0000","last_change":"2024-03-22T04:10:34.938515+0000","last_active":"2024-03-22T04:10:41.170514+0000","last_peered":"2024-03-22T04:10:41.170514+0000","last_clean":"2024-03-22T04:10:41.170514+0000","last_became_active":"2024-03-22T04:10:34.938169+0000","last_became_peered":"2024-03-22T04:10:34.938169+0000","last_unstale":"2024-03-22T04:10:41.170514+0000","last_undegraded":"2024-03-22T04:10:41.170514+0000","last_fullsized":"2024-03-22T04:10:41.170514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:34.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966658+0000","last_change":"2024-03-22T04:10:38.983720+0000","last_active":"2024-03-22T04:10:39.966658+0000","last_peered":"2024-03-22T04:10:39.966658+0000","last_clean":"2024-03-22T04:10:39.966658+0000","last_became_active":"2024-03-22T04:10:38.983348+0000","last_became_peered":"2024-03-22T04:10:38.983348+0000","last_unstale":"2024-03-22T04:10:39.966658+0000","last_undegraded":"2024-03-22T04:10:39.966658+0000","last_fullsized":"2024-03-22T04:10:39.966658+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:45.494650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170607+0000","last_change":"2024-03-22T04:10:33.934030+0000","last_active":"2024-03-22T04:10:41.170607+0000","last_peered":"2024-03-22T04:10:41.170607+0000","last_clean":"2024-03-22T04:10:41.170607+0000","last_became_active":"2024-03-22T04:10:33.933718+0000","last_became_peered":"2024-03-22T04:10:33.933718+0000","last_unstale":"2024-03-22T04:10:41.170607+0000","last_undegraded":"2024-03-22T04:10:41.170607+0000","last_fullsized":"2024-03-22T04:10:41.170607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:50:42.805996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170782+0000","last_change":"2024-03-22T04:10:34.942768+0000","last_active":"2024-03-22T04:10:41.170782+0000","last_peered":"2024-03-22T04:10:41.170782+0000","last_clean":"2024-03-22T04:10:41.170782+0000","last_became_active":"2024-03-22T04:10:34.941567+0000","last_became_peered":"2024-03-22T04:10:34.941567+0000","last_unstale":"2024-03-22T04:10:41.170782+0000","last_undegraded":"2024-03-22T04:10:41.170782+0000","last_fullsized":"2024-03-22T04:10:41.170782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:26.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964401+0000","last_change":"2024-03-22T04:10:38.988088+0000","last_active":"2024-03-22T04:10:39.964401+0000","last_peered":"2024-03-22T04:10:39.964401+0000","last_clean":"2024-03-22T04:10:39.964401+0000","last_became_active":"2024-03-22T04:10:38.987611+0000","last_became_peered":"2024-03-22T04:10:38.987611+0000","last_unstale":"2024-03-22T04:10:39.964401+0000","last_undegraded":"2024-03-22T04:10:39.964401+0000","last_fullsized":"2024-03-22T04:10:39.964401+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:15.151121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535943+0000","last_change":"2024-03-22T04:10:33.936193+0000","last_active":"2024-03-22T04:10:41.535943+0000","last_peered":"2024-03-22T04:10:41.535943+0000","last_clean":"2024-03-22T04:10:41.535943+0000","last_became_active":"2024-03-22T04:10:33.935883+0000","last_became_peered":"2024-03-22T04:10:33.935883+0000","last_unstale":"2024-03-22T04:10:41.535943+0000","last_undegraded":"2024-03-22T04:10:41.535943+0000","last_fullsized":"2024-03-22T04:10:41.535943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:00.244111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535389+0000","last_change":"2024-03-22T04:10:34.935504+0000","last_active":"2024-03-22T04:10:41.535389+0000","last_peered":"2024-03-22T04:10:41.535389+0000","last_clean":"2024-03-22T04:10:41.535389+0000","last_became_active":"2024-03-22T04:10:34.933502+0000","last_became_peered":"2024-03-22T04:10:34.933502+0000","last_unstale":"2024-03-22T04:10:41.535389+0000","last_undegraded":"2024-03-22T04:10:41.535389+0000","last_fullsized":"2024-03-22T04:10:41.535389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965123+0000","last_change":"2024-03-22T04:10:38.991103+0000","last_active":"2024-03-22T04:10:39.965123+0000","last_peered":"2024-03-22T04:10:39.965123+0000","last_clean":"2024-03-22T04:10:39.965123+0000","last_became_active":"2024-03-22T04:10:38.988847+0000","last_became_peered":"2024-03-22T04:10:38.988847+0000","last_unstale":"2024-03-22T04:10:39.965123+0000","last_undegraded":"2024-03-22T04:10:39.965123+0000","last_fullsized":"2024-03-22T04:10:39.965123+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:45.922825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535279+0000","last_change":"2024-03-22T04:10:33.936237+0000","last_active":"2024-03-22T04:10:41.535279+0000","last_peered":"2024-03-22T04:10:41.535279+0000","last_clean":"2024-03-22T04:10:41.535279+0000","last_became_active":"2024-03-22T04:10:33.935870+0000","last_became_peered":"2024-03-22T04:10:33.935870+0000","last_unstale":"2024-03-22T04:10:41.535279+0000","last_undegraded":"2024-03-22T04:10:41.535279+0000","last_fullsized":"2024-03-22T04:10:41.535279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:47.209113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922489+0000","last_change":"2024-03-22T04:10:34.930847+0000","last_active":"2024-03-22T04:10:40.922489+0000","last_peered":"2024-03-22T04:10:40.922489+0000","last_clean":"2024-03-22T04:10:40.922489+0000","last_became_active":"2024-03-22T04:10:34.930567+0000","last_became_peered":"2024-03-22T04:10:34.930567+0000","last_unstale":"2024-03-22T04:10:40.922489+0000","last_undegraded":"2024-03-22T04:10:40.922489+0000","last_fullsized":"2024-03-22T04:10:40.922489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:17.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964576+0000","last_change":"2024-03-22T04:10:38.992358+0000","last_active":"2024-03-22T04:10:39.964576+0000","last_peered":"2024-03-22T04:10:39.964576+0000","last_clean":"2024-03-22T04:10:39.964576+0000","last_became_active":"2024-03-22T04:10:38.991968+0000","last_became_peered":"2024-03-22T04:10:38.991968+0000","last_unstale":"2024-03-22T04:10:39.964576+0000","last_undegraded":"2024-03-22T04:10:39.964576+0000","last_fullsized":"2024-03-22T04:10:39.964576+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:12.109401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922040+0000","last_change":"2024-03-22T04:10:34.947353+0000","last_active":"2024-03-22T04:10:40.922040+0000","last_peered":"2024-03-22T04:10:40.922040+0000","last_clean":"2024-03-22T04:10:40.922040+0000","last_became_active":"2024-03-22T04:10:34.945247+0000","last_became_peered":"2024-03-22T04:10:34.945247+0000","last_unstale":"2024-03-22T04:10:40.922040+0000","last_undegraded":"2024-03-22T04:10:40.922040+0000","last_fullsized":"2024-03-22T04:10:40.922040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:31.574004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922244+0000","last_change":"2024-03-22T04:10:33.938771+0000","last_active":"2024-03-22T04:10:40.922244+0000","last_peered":"2024-03-22T04:10:40.922244+0000","last_clean":"2024-03-22T04:10:40.922244+0000","last_became_active":"2024-03-22T04:10:33.937907+0000","last_became_peered":"2024-03-22T04:10:33.937907+0000","last_unstale":"2024-03-22T04:10:40.922244+0000","last_undegraded":"2024-03-22T04:10:40.922244+0000","last_fullsized":"2024-03-22T04:10:40.922244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:43.867139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922386+0000","last_change":"2024-03-22T04:10:38.979866+0000","last_active":"2024-03-22T04:10:40.922386+0000","last_peered":"2024-03-22T04:10:40.922386+0000","last_clean":"2024-03-22T04:10:40.922386+0000","last_became_active":"2024-03-22T04:10:38.979321+0000","last_became_peered":"2024-03-22T04:10:38.979321+0000","last_unstale":"2024-03-22T04:10:40.922386+0000","last_undegraded":"2024-03-22T04:10:40.922386+0000","last_fullsized":"2024-03-22T04:10:40.922386+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:03:12.221975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967207+0000","last_change":"2024-03-22T04:10:34.951257+0000","last_active":"2024-03-22T04:10:39.967207+0000","last_peered":"2024-03-22T04:10:39.967207+0000","last_clean":"2024-03-22T04:10:39.967207+0000","last_became_active":"2024-03-22T04:10:34.950994+0000","last_became_peered":"2024-03-22T04:10:34.950994+0000","last_unstale":"2024-03-22T04:10:39.967207+0000","last_undegraded":"2024-03-22T04:10:39.967207+0000","last_fullsized":"2024-03-22T04:10:39.967207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536303+0000","last_change":"2024-03-22T04:10:33.933287+0000","last_active":"2024-03-22T04:10:41.536303+0000","last_peered":"2024-03-22T04:10:41.536303+0000","last_clean":"2024-03-22T04:10:41.536303+0000","last_became_active":"2024-03-22T04:10:33.932862+0000","last_became_peered":"2024-03-22T04:10:33.932862+0000","last_unstale":"2024-03-22T04:10:41.536303+0000","last_undegraded":"2024-03-22T04:10:41.536303+0000","last_fullsized":"2024-03-22T04:10:41.536303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534955+0000","last_change":"2024-03-22T04:10:38.986486+0000","last_active":"2024-03-22T04:10:41.534955+0000","last_peered":"2024-03-22T04:10:41.534955+0000","last_clean":"2024-03-22T04:10:41.534955+0000","last_became_active":"2024-03-22T04:10:38.985993+0000","last_became_peered":"2024-03-22T04:10:38.985993+0000","last_unstale":"2024-03-22T04:10:41.534955+0000","last_undegraded":"2024-03-22T04:10:41.534955+0000","last_fullsized":"2024-03-22T04:10:41.534955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:21.157155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966213+0000","last_change":"2024-03-22T04:10:34.946405+0000","last_active":"2024-03-22T04:10:39.966213+0000","last_peered":"2024-03-22T04:10:39.966213+0000","last_clean":"2024-03-22T04:10:39.966213+0000","last_became_active":"2024-03-22T04:10:34.946093+0000","last_became_peered":"2024-03-22T04:10:34.946093+0000","last_unstale":"2024-03-22T04:10:39.966213+0000","last_undegraded":"2024-03-22T04:10:39.966213+0000","last_fullsized":"2024-03-22T04:10:39.966213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:54:39.775612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966845+0000","last_change":"2024-03-22T04:10:33.931984+0000","last_active":"2024-03-22T04:10:39.966845+0000","last_peered":"2024-03-22T04:10:39.966845+0000","last_clean":"2024-03-22T04:10:39.966845+0000","last_became_active":"2024-03-22T04:10:33.931674+0000","last_became_peered":"2024-03-22T04:10:33.931674+0000","last_unstale":"2024-03-22T04:10:39.966845+0000","last_undegraded":"2024-03-22T04:10:39.966845+0000","last_fullsized":"2024-03-22T04:10:39.966845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:11.685927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967850+0000","last_change":"2024-03-22T04:10:38.992239+0000","last_active":"2024-03-22T04:10:39.967850+0000","last_peered":"2024-03-22T04:10:39.967850+0000","last_clean":"2024-03-22T04:10:39.967850+0000","last_became_active":"2024-03-22T04:10:38.991429+0000","last_became_peered":"2024-03-22T04:10:38.991429+0000","last_unstale":"2024-03-22T04:10:39.967850+0000","last_undegraded":"2024-03-22T04:10:39.967850+0000","last_fullsized":"2024-03-22T04:10:39.967850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:22.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921940+0000","last_change":"2024-03-22T04:10:34.939183+0000","last_active":"2024-03-22T04:10:40.921940+0000","last_peered":"2024-03-22T04:10:40.921940+0000","last_clean":"2024-03-22T04:10:40.921940+0000","last_became_active":"2024-03-22T04:10:34.938388+0000","last_became_peered":"2024-03-22T04:10:34.938388+0000","last_unstale":"2024-03-22T04:10:40.921940+0000","last_undegraded":"2024-03-22T04:10:40.921940+0000","last_fullsized":"2024-03-22T04:10:40.921940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:10:15.750612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534426+0000","last_change":"2024-03-22T04:10:33.933352+0000","last_active":"2024-03-22T04:10:41.534426+0000","last_peered":"2024-03-22T04:10:41.534426+0000","last_clean":"2024-03-22T04:10:41.534426+0000","last_became_active":"2024-03-22T04:10:33.933066+0000","last_became_peered":"2024-03-22T04:10:33.933066+0000","last_unstale":"2024-03-22T04:10:41.534426+0000","last_undegraded":"2024-03-22T04:10:41.534426+0000","last_fullsized":"2024-03-22T04:10:41.534426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:50:42.805996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964895+0000","last_change":"2024-03-22T04:10:38.987700+0000","last_active":"2024-03-22T04:10:39.964895+0000","last_peered":"2024-03-22T04:10:39.964895+0000","last_clean":"2024-03-22T04:10:39.964895+0000","last_became_active":"2024-03-22T04:10:38.987221+0000","last_became_peered":"2024-03-22T04:10:38.987221+0000","last_unstale":"2024-03-22T04:10:39.964895+0000","last_undegraded":"2024-03-22T04:10:39.964895+0000","last_fullsized":"2024-03-22T04:10:39.964895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:30.365392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967947+0000","last_change":"2024-03-22T04:10:34.953907+0000","last_active":"2024-03-22T04:10:39.967947+0000","last_peered":"2024-03-22T04:10:39.967947+0000","last_clean":"2024-03-22T04:10:39.967947+0000","last_became_active":"2024-03-22T04:10:34.953664+0000","last_became_peered":"2024-03-22T04:10:34.953664+0000","last_unstale":"2024-03-22T04:10:39.967947+0000","last_undegraded":"2024-03-22T04:10:39.967947+0000","last_fullsized":"2024-03-22T04:10:39.967947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:07.983679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922944+0000","last_change":"2024-03-22T04:10:33.931133+0000","last_active":"2024-03-22T04:10:40.922944+0000","last_peered":"2024-03-22T04:10:40.922944+0000","last_clean":"2024-03-22T04:10:40.922944+0000","last_became_active":"2024-03-22T04:10:33.930877+0000","last_became_peered":"2024-03-22T04:10:33.930877+0000","last_unstale":"2024-03-22T04:10:40.922944+0000","last_undegraded":"2024-03-22T04:10:40.922944+0000","last_fullsized":"2024-03-22T04:10:40.922944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:39.248649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964896+0000","last_change":"2024-03-22T04:10:38.983903+0000","last_active":"2024-03-22T04:10:39.964896+0000","last_peered":"2024-03-22T04:10:39.964896+0000","last_clean":"2024-03-22T04:10:39.964896+0000","last_became_active":"2024-03-22T04:10:38.983459+0000","last_became_peered":"2024-03-22T04:10:38.983459+0000","last_unstale":"2024-03-22T04:10:39.964896+0000","last_undegraded":"2024-03-22T04:10:39.964896+0000","last_fullsized":"2024-03-22T04:10:39.964896+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:38.915424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967821+0000","last_change":"2024-03-22T04:10:34.930898+0000","last_active":"2024-03-22T04:10:39.967821+0000","last_peered":"2024-03-22T04:10:39.967821+0000","last_clean":"2024-03-22T04:10:39.967821+0000","last_became_active":"2024-03-22T04:10:34.930562+0000","last_became_peered":"2024-03-22T04:10:34.930562+0000","last_unstale":"2024-03-22T04:10:39.967821+0000","last_undegraded":"2024-03-22T04:10:39.967821+0000","last_fullsized":"2024-03-22T04:10:39.967821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:46:40.225626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.968051+0000","last_change":"2024-03-22T04:10:33.934918+0000","last_active":"2024-03-22T04:10:39.968051+0000","last_peered":"2024-03-22T04:10:39.968051+0000","last_clean":"2024-03-22T04:10:39.968051+0000","last_became_active":"2024-03-22T04:10:33.933026+0000","last_became_peered":"2024-03-22T04:10:33.933026+0000","last_unstale":"2024-03-22T04:10:39.968051+0000","last_undegraded":"2024-03-22T04:10:39.968051+0000","last_fullsized":"2024-03-22T04:10:39.968051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:42:25.730112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966295+0000","last_change":"2024-03-22T04:10:38.984737+0000","last_active":"2024-03-22T04:10:39.966295+0000","last_peered":"2024-03-22T04:10:39.966295+0000","last_clean":"2024-03-22T04:10:39.966295+0000","last_became_active":"2024-03-22T04:10:38.984373+0000","last_became_peered":"2024-03-22T04:10:38.984373+0000","last_unstale":"2024-03-22T04:10:39.966295+0000","last_undegraded":"2024-03-22T04:10:39.966295+0000","last_fullsized":"2024-03-22T04:10:39.966295+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:23:32.419897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536493+0000","last_change":"2024-03-22T04:10:34.928433+0000","last_active":"2024-03-22T04:10:41.536493+0000","last_peered":"2024-03-22T04:10:41.536493+0000","last_clean":"2024-03-22T04:10:41.536493+0000","last_became_active":"2024-03-22T04:10:34.927912+0000","last_became_peered":"2024-03-22T04:10:34.927912+0000","last_unstale":"2024-03-22T04:10:41.536493+0000","last_undegraded":"2024-03-22T04:10:41.536493+0000","last_fullsized":"2024-03-22T04:10:41.536493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:12:01.577501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922703+0000","last_change":"2024-03-22T04:10:33.938458+0000","last_active":"2024-03-22T04:10:40.922703+0000","last_peered":"2024-03-22T04:10:40.922703+0000","last_clean":"2024-03-22T04:10:40.922703+0000","last_became_active":"2024-03-22T04:10:33.937789+0000","last_became_peered":"2024-03-22T04:10:33.937789+0000","last_unstale":"2024-03-22T04:10:40.922703+0000","last_undegraded":"2024-03-22T04:10:40.922703+0000","last_fullsized":"2024-03-22T04:10:40.922703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:40.441952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536228+0000","last_change":"2024-03-22T04:10:38.988981+0000","last_active":"2024-03-22T04:10:41.536228+0000","last_peered":"2024-03-22T04:10:41.536228+0000","last_clean":"2024-03-22T04:10:41.536228+0000","last_became_active":"2024-03-22T04:10:38.988442+0000","last_became_peered":"2024-03-22T04:10:38.988442+0000","last_unstale":"2024-03-22T04:10:41.536228+0000","last_undegraded":"2024-03-22T04:10:41.536228+0000","last_fullsized":"2024-03-22T04:10:41.536228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:38.915424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964614+0000","last_change":"2024-03-22T04:10:34.931657+0000","last_active":"2024-03-22T04:10:39.964614+0000","last_peered":"2024-03-22T04:10:39.964614+0000","last_clean":"2024-03-22T04:10:39.964614+0000","last_became_active":"2024-03-22T04:10:34.931362+0000","last_became_peered":"2024-03-22T04:10:34.931362+0000","last_unstale":"2024-03-22T04:10:39.964614+0000","last_undegraded":"2024-03-22T04:10:39.964614+0000","last_fullsized":"2024-03-22T04:10:39.964614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170215+0000","last_change":"2024-03-22T04:10:33.934747+0000","last_active":"2024-03-22T04:10:41.170215+0000","last_peered":"2024-03-22T04:10:41.170215+0000","last_clean":"2024-03-22T04:10:41.170215+0000","last_became_active":"2024-03-22T04:10:33.933151+0000","last_became_peered":"2024-03-22T04:10:33.933151+0000","last_unstale":"2024-03-22T04:10:41.170215+0000","last_undegraded":"2024-03-22T04:10:41.170215+0000","last_fullsized":"2024-03-22T04:10:41.170215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536019+0000","last_change":"2024-03-22T04:10:38.972402+0000","last_active":"2024-03-22T04:10:41.536019+0000","last_peered":"2024-03-22T04:10:41.536019+0000","last_clean":"2024-03-22T04:10:41.536019+0000","last_became_active":"2024-03-22T04:10:38.972049+0000","last_became_peered":"2024-03-22T04:10:38.972049+0000","last_unstale":"2024-03-22T04:10:41.536019+0000","last_undegraded":"2024-03-22T04:10:41.536019+0000","last_fullsized":"2024-03-22T04:10:41.536019+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:20.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535049+0000","last_change":"2024-03-22T04:10:34.931298+0000","last_active":"2024-03-22T04:10:41.535049+0000","last_peered":"2024-03-22T04:10:41.535049+0000","last_clean":"2024-03-22T04:10:41.535049+0000","last_became_active":"2024-03-22T04:10:34.930240+0000","last_became_peered":"2024-03-22T04:10:34.930240+0000","last_unstale":"2024-03-22T04:10:41.535049+0000","last_undegraded":"2024-03-22T04:10:41.535049+0000","last_fullsized":"2024-03-22T04:10:41.535049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:11.753753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922237+0000","last_change":"2024-03-22T04:10:33.938276+0000","last_active":"2024-03-22T04:10:40.922237+0000","last_peered":"2024-03-22T04:10:40.922237+0000","last_clean":"2024-03-22T04:10:40.922237+0000","last_became_active":"2024-03-22T04:10:33.933126+0000","last_became_peered":"2024-03-22T04:10:33.933126+0000","last_unstale":"2024-03-22T04:10:40.922237+0000","last_undegraded":"2024-03-22T04:10:40.922237+0000","last_fullsized":"2024-03-22T04:10:40.922237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:11.368879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536420+0000","last_change":"2024-03-22T04:10:38.974697+0000","last_active":"2024-03-22T04:10:41.536420+0000","last_peered":"2024-03-22T04:10:41.536420+0000","last_clean":"2024-03-22T04:10:41.536420+0000","last_became_active":"2024-03-22T04:10:38.974339+0000","last_became_peered":"2024-03-22T04:10:38.974339+0000","last_unstale":"2024-03-22T04:10:41.536420+0000","last_undegraded":"2024-03-22T04:10:41.536420+0000","last_fullsized":"2024-03-22T04:10:41.536420+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:59.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169762+0000","last_change":"2024-03-22T04:10:34.947104+0000","last_active":"2024-03-22T04:10:41.169762+0000","last_peered":"2024-03-22T04:10:41.169762+0000","last_clean":"2024-03-22T04:10:41.169762+0000","last_became_active":"2024-03-22T04:10:34.946787+0000","last_became_peered":"2024-03-22T04:10:34.946787+0000","last_unstale":"2024-03-22T04:10:41.169762+0000","last_undegraded":"2024-03-22T04:10:41.169762+0000","last_fullsized":"2024-03-22T04:10:41.169762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:11.753753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966701+0000","last_change":"2024-03-22T04:10:33.938271+0000","last_active":"2024-03-22T04:10:39.966701+0000","last_peered":"2024-03-22T04:10:39.966701+0000","last_clean":"2024-03-22T04:10:39.966701+0000","last_became_active":"2024-03-22T04:10:33.937651+0000","last_became_peered":"2024-03-22T04:10:33.937651+0000","last_unstale":"2024-03-22T04:10:39.966701+0000","last_undegraded":"2024-03-22T04:10:39.966701+0000","last_fullsized":"2024-03-22T04:10:39.966701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536236+0000","last_change":"2024-03-22T04:10:38.980629+0000","last_active":"2024-03-22T04:10:41.536236+0000","last_peered":"2024-03-22T04:10:41.536236+0000","last_clean":"2024-03-22T04:10:41.536236+0000","last_became_active":"2024-03-22T04:10:38.980157+0000","last_became_peered":"2024-03-22T04:10:38.980157+0000","last_unstale":"2024-03-22T04:10:41.536236+0000","last_undegraded":"2024-03-22T04:10:41.536236+0000","last_fullsized":"2024-03-22T04:10:41.536236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:30.365392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535927+0000","last_change":"2024-03-22T04:10:34.940387+0000","last_active":"2024-03-22T04:10:41.535927+0000","last_peered":"2024-03-22T04:10:41.535927+0000","last_clean":"2024-03-22T04:10:41.535927+0000","last_became_active":"2024-03-22T04:10:34.940009+0000","last_became_peered":"2024-03-22T04:10:34.940009+0000","last_unstale":"2024-03-22T04:10:41.535927+0000","last_undegraded":"2024-03-22T04:10:41.535927+0000","last_fullsized":"2024-03-22T04:10:41.535927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:12.376047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922076+0000","last_change":"2024-03-22T04:10:33.927977+0000","last_active":"2024-03-22T04:10:40.922076+0000","last_peered":"2024-03-22T04:10:40.922076+0000","last_clean":"2024-03-22T04:10:40.922076+0000","last_became_active":"2024-03-22T04:10:33.927635+0000","last_became_peered":"2024-03-22T04:10:33.927635+0000","last_unstale":"2024-03-22T04:10:40.922076+0000","last_undegraded":"2024-03-22T04:10:40.922076+0000","last_fullsized":"2024-03-22T04:10:40.922076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:00.244111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170363+0000","last_change":"2024-03-22T04:10:38.985749+0000","last_active":"2024-03-22T04:10:41.170363+0000","last_peered":"2024-03-22T04:10:41.170363+0000","last_clean":"2024-03-22T04:10:41.170363+0000","last_became_active":"2024-03-22T04:10:38.981278+0000","last_became_peered":"2024-03-22T04:10:38.981278+0000","last_unstale":"2024-03-22T04:10:41.170363+0000","last_undegraded":"2024-03-22T04:10:41.170363+0000","last_fullsized":"2024-03-22T04:10:41.170363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:40:38.204332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966651+0000","last_change":"2024-03-22T04:10:34.921916+0000","last_active":"2024-03-22T04:10:39.966651+0000","last_peered":"2024-03-22T04:10:39.966651+0000","last_clean":"2024-03-22T04:10:39.966651+0000","last_became_active":"2024-03-22T04:10:34.921675+0000","last_became_peered":"2024-03-22T04:10:34.921675+0000","last_unstale":"2024-03-22T04:10:39.966651+0000","last_undegraded":"2024-03-22T04:10:39.966651+0000","last_fullsized":"2024-03-22T04:10:39.966651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:23:07.242835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922298+0000","last_change":"2024-03-22T04:10:33.937845+0000","last_active":"2024-03-22T04:10:40.922298+0000","last_peered":"2024-03-22T04:10:40.922298+0000","last_clean":"2024-03-22T04:10:40.922298+0000","last_became_active":"2024-03-22T04:10:33.930400+0000","last_became_peered":"2024-03-22T04:10:33.930400+0000","last_unstale":"2024-03-22T04:10:40.922298+0000","last_undegraded":"2024-03-22T04:10:40.922298+0000","last_fullsized":"2024-03-22T04:10:40.922298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:03.705836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":177,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535319+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:10:41.535319+0000","last_peered":"2024-03-22T04:10:41.535319+0000","last_clean":"2024-03-22T04:10:41.535319+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:10:41.535319+0000","last_undegraded":"2024-03-22T04:10:41.535319+0000","last_fullsized":"2024-03-22T04:10:41.535319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922657+0000","last_change":"2024-03-22T04:10:38.982996+0000","last_active":"2024-03-22T04:10:40.922657+0000","last_peered":"2024-03-22T04:10:40.922657+0000","last_clean":"2024-03-22T04:10:40.922657+0000","last_became_active":"2024-03-22T04:10:38.981238+0000","last_became_peered":"2024-03-22T04:10:38.981238+0000","last_unstale":"2024-03-22T04:10:40.922657+0000","last_undegraded":"2024-03-22T04:10:40.922657+0000","last_fullsized":"2024-03-22T04:10:40.922657+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:09:02.614234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922533+0000","last_change":"2024-03-22T04:10:34.945879+0000","last_active":"2024-03-22T04:10:40.922533+0000","last_peered":"2024-03-22T04:10:40.922533+0000","last_clean":"2024-03-22T04:10:40.922533+0000","last_became_active":"2024-03-22T04:10:34.945056+0000","last_became_peered":"2024-03-22T04:10:34.945056+0000","last_unstale":"2024-03-22T04:10:40.922533+0000","last_undegraded":"2024-03-22T04:10:40.922533+0000","last_fullsized":"2024-03-22T04:10:40.922533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:17:47.020807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536542+0000","last_change":"2024-03-22T04:10:33.931838+0000","last_active":"2024-03-22T04:10:41.536542+0000","last_peered":"2024-03-22T04:10:41.536542+0000","last_clean":"2024-03-22T04:10:41.536542+0000","last_became_active":"2024-03-22T04:10:33.931611+0000","last_became_peered":"2024-03-22T04:10:33.931611+0000","last_unstale":"2024-03-22T04:10:41.536542+0000","last_undegraded":"2024-03-22T04:10:41.536542+0000","last_fullsized":"2024-03-22T04:10:41.536542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964623+0000","last_change":"2024-03-22T04:10:38.988757+0000","last_active":"2024-03-22T04:10:39.964623+0000","last_peered":"2024-03-22T04:10:39.964623+0000","last_clean":"2024-03-22T04:10:39.964623+0000","last_became_active":"2024-03-22T04:10:38.987185+0000","last_became_peered":"2024-03-22T04:10:38.987185+0000","last_unstale":"2024-03-22T04:10:39.964623+0000","last_undegraded":"2024-03-22T04:10:39.964623+0000","last_fullsized":"2024-03-22T04:10:39.964623+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:15.799282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535188+0000","last_change":"2024-03-22T04:10:34.948529+0000","last_active":"2024-03-22T04:10:41.535188+0000","last_peered":"2024-03-22T04:10:41.535188+0000","last_clean":"2024-03-22T04:10:41.535188+0000","last_became_active":"2024-03-22T04:10:34.948122+0000","last_became_peered":"2024-03-22T04:10:34.948122+0000","last_unstale":"2024-03-22T04:10:41.535188+0000","last_undegraded":"2024-03-22T04:10:41.535188+0000","last_fullsized":"2024-03-22T04:10:41.535188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:55.025923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170450+0000","last_change":"2024-03-22T04:10:33.938126+0000","last_active":"2024-03-22T04:10:41.170450+0000","last_peered":"2024-03-22T04:10:41.170450+0000","last_clean":"2024-03-22T04:10:41.170450+0000","last_became_active":"2024-03-22T04:10:33.937511+0000","last_became_peered":"2024-03-22T04:10:33.937511+0000","last_unstale":"2024-03-22T04:10:41.170450+0000","last_undegraded":"2024-03-22T04:10:41.170450+0000","last_fullsized":"2024-03-22T04:10:41.170450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169980+0000","last_change":"2024-03-22T04:10:38.988377+0000","last_active":"2024-03-22T04:10:41.169980+0000","last_peered":"2024-03-22T04:10:41.169980+0000","last_clean":"2024-03-22T04:10:41.169980+0000","last_became_active":"2024-03-22T04:10:38.984506+0000","last_became_peered":"2024-03-22T04:10:38.984506+0000","last_unstale":"2024-03-22T04:10:41.169980+0000","last_undegraded":"2024-03-22T04:10:41.169980+0000","last_fullsized":"2024-03-22T04:10:41.169980+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:17:51.066337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535990+0000","last_change":"2024-03-22T04:10:34.940162+0000","last_active":"2024-03-22T04:10:41.535990+0000","last_peered":"2024-03-22T04:10:41.535990+0000","last_clean":"2024-03-22T04:10:41.535990+0000","last_became_active":"2024-03-22T04:10:34.937040+0000","last_became_peered":"2024-03-22T04:10:34.937040+0000","last_unstale":"2024-03-22T04:10:41.535990+0000","last_undegraded":"2024-03-22T04:10:41.535990+0000","last_fullsized":"2024-03-22T04:10:41.535990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.104177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967999+0000","last_change":"2024-03-22T04:10:33.932000+0000","last_active":"2024-03-22T04:10:39.967999+0000","last_peered":"2024-03-22T04:10:39.967999+0000","last_clean":"2024-03-22T04:10:39.967999+0000","last_became_active":"2024-03-22T04:10:33.930159+0000","last_became_peered":"2024-03-22T04:10:33.930159+0000","last_unstale":"2024-03-22T04:10:39.967999+0000","last_undegraded":"2024-03-22T04:10:39.967999+0000","last_fullsized":"2024-03-22T04:10:39.967999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:11:21.741394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921794+0000","last_change":"2024-03-22T04:10:38.979495+0000","last_active":"2024-03-22T04:10:40.921794+0000","last_peered":"2024-03-22T04:10:40.921794+0000","last_clean":"2024-03-22T04:10:40.921794+0000","last_became_active":"2024-03-22T04:10:38.979058+0000","last_became_peered":"2024-03-22T04:10:38.979058+0000","last_unstale":"2024-03-22T04:10:40.921794+0000","last_undegraded":"2024-03-22T04:10:40.921794+0000","last_fullsized":"2024-03-22T04:10:40.921794+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:25.405176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965133+0000","last_change":"2024-03-22T04:10:34.936969+0000","last_active":"2024-03-22T04:10:39.965133+0000","last_peered":"2024-03-22T04:10:39.965133+0000","last_clean":"2024-03-22T04:10:39.965133+0000","last_became_active":"2024-03-22T04:10:34.936270+0000","last_became_peered":"2024-03-22T04:10:34.936270+0000","last_unstale":"2024-03-22T04:10:39.965133+0000","last_undegraded":"2024-03-22T04:10:39.965133+0000","last_fullsized":"2024-03-22T04:10:39.965133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:44.874307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536081+0000","last_change":"2024-03-22T04:10:33.929682+0000","last_active":"2024-03-22T04:10:41.536081+0000","last_peered":"2024-03-22T04:10:41.536081+0000","last_clean":"2024-03-22T04:10:41.536081+0000","last_became_active":"2024-03-22T04:10:33.929309+0000","last_became_peered":"2024-03-22T04:10:33.929309+0000","last_unstale":"2024-03-22T04:10:41.536081+0000","last_undegraded":"2024-03-22T04:10:41.536081+0000","last_fullsized":"2024-03-22T04:10:41.536081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:05.641869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535389+0000","last_change":"2024-03-22T04:10:38.979596+0000","last_active":"2024-03-22T04:10:41.535389+0000","last_peered":"2024-03-22T04:10:41.535389+0000","last_clean":"2024-03-22T04:10:41.535389+0000","last_became_active":"2024-03-22T04:10:38.979166+0000","last_became_peered":"2024-03-22T04:10:38.979166+0000","last_unstale":"2024-03-22T04:10:41.535389+0000","last_undegraded":"2024-03-22T04:10:41.535389+0000","last_fullsized":"2024-03-22T04:10:41.535389+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:08.758533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921851+0000","last_change":"2024-03-22T04:10:34.926802+0000","last_active":"2024-03-22T04:10:40.921851+0000","last_peered":"2024-03-22T04:10:40.921851+0000","last_clean":"2024-03-22T04:10:40.921851+0000","last_became_active":"2024-03-22T04:10:34.926514+0000","last_became_peered":"2024-03-22T04:10:34.926514+0000","last_unstale":"2024-03-22T04:10:40.921851+0000","last_undegraded":"2024-03-22T04:10:40.921851+0000","last_fullsized":"2024-03-22T04:10:40.921851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:02:47.197474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966395+0000","last_change":"2024-03-22T04:10:33.932545+0000","last_active":"2024-03-22T04:10:39.966395+0000","last_peered":"2024-03-22T04:10:39.966395+0000","last_clean":"2024-03-22T04:10:39.966395+0000","last_became_active":"2024-03-22T04:10:33.929875+0000","last_became_peered":"2024-03-22T04:10:33.929875+0000","last_unstale":"2024-03-22T04:10:39.966395+0000","last_undegraded":"2024-03-22T04:10:39.966395+0000","last_fullsized":"2024-03-22T04:10:39.966395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:59:03.501920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170524+0000","last_change":"2024-03-22T04:10:38.985437+0000","last_active":"2024-03-22T04:10:41.170524+0000","last_peered":"2024-03-22T04:10:41.170524+0000","last_clean":"2024-03-22T04:10:41.170524+0000","last_became_active":"2024-03-22T04:10:38.982076+0000","last_became_peered":"2024-03-22T04:10:38.982076+0000","last_unstale":"2024-03-22T04:10:41.170524+0000","last_undegraded":"2024-03-22T04:10:41.170524+0000","last_fullsized":"2024-03-22T04:10:41.170524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:21.687034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535147+0000","last_change":"2024-03-22T04:10:34.928089+0000","last_active":"2024-03-22T04:10:41.535147+0000","last_peered":"2024-03-22T04:10:41.535147+0000","last_clean":"2024-03-22T04:10:41.535147+0000","last_became_active":"2024-03-22T04:10:34.927828+0000","last_became_peered":"2024-03-22T04:10:34.927828+0000","last_unstale":"2024-03-22T04:10:41.535147+0000","last_undegraded":"2024-03-22T04:10:41.535147+0000","last_fullsized":"2024-03-22T04:10:41.535147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534615+0000","last_change":"2024-03-22T04:10:33.943460+0000","last_active":"2024-03-22T04:10:41.534615+0000","last_peered":"2024-03-22T04:10:41.534615+0000","last_clean":"2024-03-22T04:10:41.534615+0000","last_became_active":"2024-03-22T04:10:33.943039+0000","last_became_peered":"2024-03-22T04:10:33.943039+0000","last_unstale":"2024-03-22T04:10:41.534615+0000","last_undegraded":"2024-03-22T04:10:41.534615+0000","last_fullsized":"2024-03-22T04:10:41.534615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:40:51.297536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170801+0000","last_change":"2024-03-22T04:10:38.981354+0000","last_active":"2024-03-22T04:10:41.170801+0000","last_peered":"2024-03-22T04:10:41.170801+0000","last_clean":"2024-03-22T04:10:41.170801+0000","last_became_active":"2024-03-22T04:10:38.979192+0000","last_became_peered":"2024-03-22T04:10:38.979192+0000","last_unstale":"2024-03-22T04:10:41.170801+0000","last_undegraded":"2024-03-22T04:10:41.170801+0000","last_fullsized":"2024-03-22T04:10:41.170801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:05:23.818860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922177+0000","last_change":"2024-03-22T04:10:34.935166+0000","last_active":"2024-03-22T04:10:40.922177+0000","last_peered":"2024-03-22T04:10:40.922177+0000","last_clean":"2024-03-22T04:10:40.922177+0000","last_became_active":"2024-03-22T04:10:34.934859+0000","last_became_peered":"2024-03-22T04:10:34.934859+0000","last_unstale":"2024-03-22T04:10:40.922177+0000","last_undegraded":"2024-03-22T04:10:40.922177+0000","last_fullsized":"2024-03-22T04:10:40.922177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:34:47.128097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922197+0000","last_change":"2024-03-22T04:10:33.939004+0000","last_active":"2024-03-22T04:10:40.922197+0000","last_peered":"2024-03-22T04:10:40.922197+0000","last_clean":"2024-03-22T04:10:40.922197+0000","last_became_active":"2024-03-22T04:10:33.938036+0000","last_became_peered":"2024-03-22T04:10:33.938036+0000","last_unstale":"2024-03-22T04:10:40.922197+0000","last_undegraded":"2024-03-22T04:10:40.922197+0000","last_fullsized":"2024-03-22T04:10:40.922197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:47.209113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536345+0000","last_change":"2024-03-22T04:10:38.985663+0000","last_active":"2024-03-22T04:10:41.536345+0000","last_peered":"2024-03-22T04:10:41.536345+0000","last_clean":"2024-03-22T04:10:41.536345+0000","last_became_active":"2024-03-22T04:10:38.985235+0000","last_became_peered":"2024-03-22T04:10:38.985235+0000","last_unstale":"2024-03-22T04:10:41.536345+0000","last_undegraded":"2024-03-22T04:10:41.536345+0000","last_fullsized":"2024-03-22T04:10:41.536345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:45.922825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536323+0000","last_change":"2024-03-22T04:10:34.950266+0000","last_active":"2024-03-22T04:10:41.536323+0000","last_peered":"2024-03-22T04:10:41.536323+0000","last_clean":"2024-03-22T04:10:41.536323+0000","last_became_active":"2024-03-22T04:10:34.949946+0000","last_became_peered":"2024-03-22T04:10:34.949946+0000","last_unstale":"2024-03-22T04:10:41.536323+0000","last_undegraded":"2024-03-22T04:10:41.536323+0000","last_fullsized":"2024-03-22T04:10:41.536323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:04.713004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967305+0000","last_change":"2024-03-22T04:10:33.935184+0000","last_active":"2024-03-22T04:10:39.967305+0000","last_peered":"2024-03-22T04:10:39.967305+0000","last_clean":"2024-03-22T04:10:39.967305+0000","last_became_active":"2024-03-22T04:10:33.934904+0000","last_became_peered":"2024-03-22T04:10:33.934904+0000","last_unstale":"2024-03-22T04:10:39.967305+0000","last_undegraded":"2024-03-22T04:10:39.967305+0000","last_fullsized":"2024-03-22T04:10:39.967305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922445+0000","last_change":"2024-03-22T04:10:38.971479+0000","last_active":"2024-03-22T04:10:40.922445+0000","last_peered":"2024-03-22T04:10:40.922445+0000","last_clean":"2024-03-22T04:10:40.922445+0000","last_became_active":"2024-03-22T04:10:38.969629+0000","last_became_peered":"2024-03-22T04:10:38.969629+0000","last_unstale":"2024-03-22T04:10:40.922445+0000","last_undegraded":"2024-03-22T04:10:40.922445+0000","last_fullsized":"2024-03-22T04:10:40.922445+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:28:08.179423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535818+0000","last_change":"2024-03-22T04:10:34.932181+0000","last_active":"2024-03-22T04:10:41.535818+0000","last_peered":"2024-03-22T04:10:41.535818+0000","last_clean":"2024-03-22T04:10:41.535818+0000","last_became_active":"2024-03-22T04:10:34.931936+0000","last_became_peered":"2024-03-22T04:10:34.931936+0000","last_unstale":"2024-03-22T04:10:41.535818+0000","last_undegraded":"2024-03-22T04:10:41.535818+0000","last_fullsized":"2024-03-22T04:10:41.535818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:26.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:50.358530+0000","last_change":"2024-03-22T04:10:33.934598+0000","last_active":"2024-03-22T04:10:50.358530+0000","last_peered":"2024-03-22T04:10:50.358530+0000","last_clean":"2024-03-22T04:10:50.358530+0000","last_became_active":"2024-03-22T04:10:33.934129+0000","last_became_peered":"2024-03-22T04:10:33.934129+0000","last_unstale":"2024-03-22T04:10:50.358530+0000","last_undegraded":"2024-03-22T04:10:50.358530+0000","last_fullsized":"2024-03-22T04:10:50.358530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:25:11.544206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966493+0000","last_change":"2024-03-22T04:10:38.980975+0000","last_active":"2024-03-22T04:10:39.966493+0000","last_peered":"2024-03-22T04:10:39.966493+0000","last_clean":"2024-03-22T04:10:39.966493+0000","last_became_active":"2024-03-22T04:10:38.980594+0000","last_became_peered":"2024-03-22T04:10:38.980594+0000","last_unstale":"2024-03-22T04:10:39.966493+0000","last_undegraded":"2024-03-22T04:10:39.966493+0000","last_fullsized":"2024-03-22T04:10:39.966493+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:10:22.917144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922350+0000","last_change":"2024-03-22T04:10:34.939331+0000","last_active":"2024-03-22T04:10:40.922350+0000","last_peered":"2024-03-22T04:10:40.922350+0000","last_clean":"2024-03-22T04:10:40.922350+0000","last_became_active":"2024-03-22T04:10:34.938998+0000","last_became_peered":"2024-03-22T04:10:34.938998+0000","last_unstale":"2024-03-22T04:10:40.922350+0000","last_undegraded":"2024-03-22T04:10:40.922350+0000","last_fullsized":"2024-03-22T04:10:40.922350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.104177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967555+0000","last_change":"2024-03-22T04:10:33.938816+0000","last_active":"2024-03-22T04:10:39.967555+0000","last_peered":"2024-03-22T04:10:39.967555+0000","last_clean":"2024-03-22T04:10:39.967555+0000","last_became_active":"2024-03-22T04:10:33.937603+0000","last_became_peered":"2024-03-22T04:10:33.937603+0000","last_unstale":"2024-03-22T04:10:39.967555+0000","last_undegraded":"2024-03-22T04:10:39.967555+0000","last_fullsized":"2024-03-22T04:10:39.967555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964868+0000","last_change":"2024-03-22T04:10:38.987284+0000","last_active":"2024-03-22T04:10:39.964868+0000","last_peered":"2024-03-22T04:10:39.964868+0000","last_clean":"2024-03-22T04:10:39.964868+0000","last_became_active":"2024-03-22T04:10:38.986819+0000","last_became_peered":"2024-03-22T04:10:38.986819+0000","last_unstale":"2024-03-22T04:10:39.964868+0000","last_undegraded":"2024-03-22T04:10:39.964868+0000","last_fullsized":"2024-03-22T04:10:39.964868+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:20.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170831+0000","last_change":"2024-03-22T04:10:34.933050+0000","last_active":"2024-03-22T04:10:41.170831+0000","last_peered":"2024-03-22T04:10:41.170831+0000","last_clean":"2024-03-22T04:10:41.170831+0000","last_became_active":"2024-03-22T04:10:34.932756+0000","last_became_peered":"2024-03-22T04:10:34.932756+0000","last_unstale":"2024-03-22T04:10:41.170831+0000","last_undegraded":"2024-03-22T04:10:41.170831+0000","last_fullsized":"2024-03-22T04:10:41.170831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:41.877296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536138+0000","last_change":"2024-03-22T04:10:33.936205+0000","last_active":"2024-03-22T04:10:41.536138+0000","last_peered":"2024-03-22T04:10:41.536138+0000","last_clean":"2024-03-22T04:10:41.536138+0000","last_became_active":"2024-03-22T04:10:33.935904+0000","last_became_peered":"2024-03-22T04:10:33.935904+0000","last_unstale":"2024-03-22T04:10:41.536138+0000","last_undegraded":"2024-03-22T04:10:41.536138+0000","last_fullsized":"2024-03-22T04:10:41.536138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:30.304547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966245+0000","last_change":"2024-03-22T04:10:38.991949+0000","last_active":"2024-03-22T04:10:39.966245+0000","last_peered":"2024-03-22T04:10:39.966245+0000","last_clean":"2024-03-22T04:10:39.966245+0000","last_became_active":"2024-03-22T04:10:38.991459+0000","last_became_peered":"2024-03-22T04:10:38.991459+0000","last_unstale":"2024-03-22T04:10:39.966245+0000","last_undegraded":"2024-03-22T04:10:39.966245+0000","last_fullsized":"2024-03-22T04:10:39.966245+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:44.301347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965035+0000","last_change":"2024-03-22T04:10:34.947409+0000","last_active":"2024-03-22T04:10:39.965035+0000","last_peered":"2024-03-22T04:10:39.965035+0000","last_clean":"2024-03-22T04:10:39.965035+0000","last_became_active":"2024-03-22T04:10:34.947102+0000","last_became_peered":"2024-03-22T04:10:34.947102+0000","last_unstale":"2024-03-22T04:10:39.965035+0000","last_undegraded":"2024-03-22T04:10:39.965035+0000","last_fullsized":"2024-03-22T04:10:39.965035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:12.376047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535663+0000","last_change":"2024-03-22T04:10:33.922034+0000","last_active":"2024-03-22T04:10:41.535663+0000","last_peered":"2024-03-22T04:10:41.535663+0000","last_clean":"2024-03-22T04:10:41.535663+0000","last_became_active":"2024-03-22T04:10:33.921713+0000","last_became_peered":"2024-03-22T04:10:33.921713+0000","last_unstale":"2024-03-22T04:10:41.535663+0000","last_undegraded":"2024-03-22T04:10:41.535663+0000","last_fullsized":"2024-03-22T04:10:41.535663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966606+0000","last_change":"2024-03-22T04:10:38.983919+0000","last_active":"2024-03-22T04:10:39.966606+0000","last_peered":"2024-03-22T04:10:39.966606+0000","last_clean":"2024-03-22T04:10:39.966606+0000","last_became_active":"2024-03-22T04:10:38.983556+0000","last_became_peered":"2024-03-22T04:10:38.983556+0000","last_unstale":"2024-03-22T04:10:39.966606+0000","last_undegraded":"2024-03-22T04:10:39.966606+0000","last_fullsized":"2024-03-22T04:10:39.966606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:12:09.818595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967886+0000","last_change":"2024-03-22T04:10:38.985686+0000","last_active":"2024-03-22T04:10:39.967886+0000","last_peered":"2024-03-22T04:10:39.967886+0000","last_clean":"2024-03-22T04:10:39.967886+0000","last_became_active":"2024-03-22T04:10:38.985411+0000","last_became_peered":"2024-03-22T04:10:38.985411+0000","last_unstale":"2024-03-22T04:10:39.967886+0000","last_undegraded":"2024-03-22T04:10:39.967886+0000","last_fullsized":"2024-03-22T04:10:39.967886+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:36.068872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966752+0000","last_change":"2024-03-22T04:10:34.926900+0000","last_active":"2024-03-22T04:10:39.966752+0000","last_peered":"2024-03-22T04:10:39.966752+0000","last_clean":"2024-03-22T04:10:39.966752+0000","last_became_active":"2024-03-22T04:10:34.926599+0000","last_became_peered":"2024-03-22T04:10:34.926599+0000","last_unstale":"2024-03-22T04:10:39.966752+0000","last_undegraded":"2024-03-22T04:10:39.966752+0000","last_fullsized":"2024-03-22T04:10:39.966752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:21.635048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922002+0000","last_change":"2024-03-22T04:10:33.932457+0000","last_active":"2024-03-22T04:10:40.922002+0000","last_peered":"2024-03-22T04:10:40.922002+0000","last_clean":"2024-03-22T04:10:40.922002+0000","last_became_active":"2024-03-22T04:10:33.932048+0000","last_became_peered":"2024-03-22T04:10:33.932048+0000","last_unstale":"2024-03-22T04:10:40.922002+0000","last_undegraded":"2024-03-22T04:10:40.922002+0000","last_fullsized":"2024-03-22T04:10:40.922002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:30.304547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922792+0000","last_change":"2024-03-22T04:10:38.985629+0000","last_active":"2024-03-22T04:10:40.922792+0000","last_peered":"2024-03-22T04:10:40.922792+0000","last_clean":"2024-03-22T04:10:40.922792+0000","last_became_active":"2024-03-22T04:10:38.983094+0000","last_became_peered":"2024-03-22T04:10:38.983094+0000","last_unstale":"2024-03-22T04:10:40.922792+0000","last_undegraded":"2024-03-22T04:10:40.922792+0000","last_fullsized":"2024-03-22T04:10:40.922792+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:21:46.389313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169810+0000","last_change":"2024-03-22T04:10:34.925982+0000","last_active":"2024-03-22T04:10:41.169810+0000","last_peered":"2024-03-22T04:10:41.169810+0000","last_clean":"2024-03-22T04:10:41.169810+0000","last_became_active":"2024-03-22T04:10:34.925676+0000","last_became_peered":"2024-03-22T04:10:34.925676+0000","last_unstale":"2024-03-22T04:10:41.169810+0000","last_undegraded":"2024-03-22T04:10:41.169810+0000","last_fullsized":"2024-03-22T04:10:41.169810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:08.063872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536054+0000","last_change":"2024-03-22T04:10:33.933448+0000","last_active":"2024-03-22T04:10:41.536054+0000","last_peered":"2024-03-22T04:10:41.536054+0000","last_clean":"2024-03-22T04:10:41.536054+0000","last_became_active":"2024-03-22T04:10:33.933025+0000","last_became_peered":"2024-03-22T04:10:33.933025+0000","last_unstale":"2024-03-22T04:10:41.536054+0000","last_undegraded":"2024-03-22T04:10:41.536054+0000","last_fullsized":"2024-03-22T04:10:41.536054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922138+0000","last_change":"2024-03-22T04:10:33.935607+0000","last_active":"2024-03-22T04:10:40.922138+0000","last_peered":"2024-03-22T04:10:40.922138+0000","last_clean":"2024-03-22T04:10:40.922138+0000","last_became_active":"2024-03-22T04:10:33.932086+0000","last_became_peered":"2024-03-22T04:10:33.932086+0000","last_unstale":"2024-03-22T04:10:40.922138+0000","last_undegraded":"2024-03-22T04:10:40.922138+0000","last_fullsized":"2024-03-22T04:10:40.922138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:49.224368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967401+0000","last_change":"2024-03-22T04:10:34.934494+0000","last_active":"2024-03-22T04:10:39.967401+0000","last_peered":"2024-03-22T04:10:39.967401+0000","last_clean":"2024-03-22T04:10:39.967401+0000","last_became_active":"2024-03-22T04:10:34.934210+0000","last_became_peered":"2024-03-22T04:10:34.934210+0000","last_unstale":"2024-03-22T04:10:39.967401+0000","last_undegraded":"2024-03-22T04:10:39.967401+0000","last_fullsized":"2024-03-22T04:10:39.967401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:48:09.157711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967687+0000","last_change":"2024-03-22T04:10:38.988089+0000","last_active":"2024-03-22T04:10:39.967687+0000","last_peered":"2024-03-22T04:10:39.967687+0000","last_clean":"2024-03-22T04:10:39.967687+0000","last_became_active":"2024-03-22T04:10:38.987643+0000","last_became_peered":"2024-03-22T04:10:38.987643+0000","last_unstale":"2024-03-22T04:10:39.967687+0000","last_undegraded":"2024-03-22T04:10:39.967687+0000","last_fullsized":"2024-03-22T04:10:39.967687+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:35.902267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170109+0000","last_change":"2024-03-22T04:10:33.933057+0000","last_active":"2024-03-22T04:10:41.170109+0000","last_peered":"2024-03-22T04:10:41.170109+0000","last_clean":"2024-03-22T04:10:41.170109+0000","last_became_active":"2024-03-22T04:10:33.932776+0000","last_became_peered":"2024-03-22T04:10:33.932776+0000","last_unstale":"2024-03-22T04:10:41.170109+0000","last_undegraded":"2024-03-22T04:10:41.170109+0000","last_fullsized":"2024-03-22T04:10:41.170109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535263+0000","last_change":"2024-03-22T04:10:34.935154+0000","last_active":"2024-03-22T04:10:41.535263+0000","last_peered":"2024-03-22T04:10:41.535263+0000","last_clean":"2024-03-22T04:10:41.535263+0000","last_became_active":"2024-03-22T04:10:34.934888+0000","last_became_peered":"2024-03-22T04:10:34.934888+0000","last_unstale":"2024-03-22T04:10:41.535263+0000","last_undegraded":"2024-03-22T04:10:41.535263+0000","last_fullsized":"2024-03-22T04:10:41.535263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:01.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967502+0000","last_change":"2024-03-22T04:10:38.988529+0000","last_active":"2024-03-22T04:10:39.967502+0000","last_peered":"2024-03-22T04:10:39.967502+0000","last_clean":"2024-03-22T04:10:39.967502+0000","last_became_active":"2024-03-22T04:10:38.987847+0000","last_became_peered":"2024-03-22T04:10:38.987847+0000","last_unstale":"2024-03-22T04:10:39.967502+0000","last_undegraded":"2024-03-22T04:10:39.967502+0000","last_fullsized":"2024-03-22T04:10:39.967502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:12:05.623030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964372+0000","last_change":"2024-03-22T04:10:34.930193+0000","last_active":"2024-03-22T04:10:39.964372+0000","last_peered":"2024-03-22T04:10:39.964372+0000","last_clean":"2024-03-22T04:10:39.964372+0000","last_became_active":"2024-03-22T04:10:34.929910+0000","last_became_peered":"2024-03-22T04:10:34.929910+0000","last_unstale":"2024-03-22T04:10:39.964372+0000","last_undegraded":"2024-03-22T04:10:39.964372+0000","last_fullsized":"2024-03-22T04:10:39.964372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966601+0000","last_change":"2024-03-22T04:10:34.951954+0000","last_active":"2024-03-22T04:10:39.966601+0000","last_peered":"2024-03-22T04:10:39.966601+0000","last_clean":"2024-03-22T04:10:39.966601+0000","last_became_active":"2024-03-22T04:10:34.951651+0000","last_became_peered":"2024-03-22T04:10:34.951651+0000","last_unstale":"2024-03-22T04:10:39.966601+0000","last_undegraded":"2024-03-22T04:10:39.966601+0000","last_fullsized":"2024-03-22T04:10:39.966601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922008+0000","last_change":"2024-03-22T04:10:34.927015+0000","last_active":"2024-03-22T04:10:40.922008+0000","last_peered":"2024-03-22T04:10:40.922008+0000","last_clean":"2024-03-22T04:10:40.922008+0000","last_became_active":"2024-03-22T04:10:34.926098+0000","last_became_peered":"2024-03-22T04:10:34.926098+0000","last_unstale":"2024-03-22T04:10:40.922008+0000","last_undegraded":"2024-03-22T04:10:40.922008+0000","last_fullsized":"2024-03-22T04:10:40.922008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:05:19.773331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170027+0000","last_change":"2024-03-22T04:10:34.950558+0000","last_active":"2024-03-22T04:10:41.170027+0000","last_peered":"2024-03-22T04:10:41.170027+0000","last_clean":"2024-03-22T04:10:41.170027+0000","last_became_active":"2024-03-22T04:10:34.950242+0000","last_became_peered":"2024-03-22T04:10:34.950242+0000","last_unstale":"2024-03-22T04:10:41.170027+0000","last_undegraded":"2024-03-22T04:10:41.170027+0000","last_fullsized":"2024-03-22T04:10:41.170027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:11.105591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536489+0000","last_change":"2024-03-22T04:10:34.928518+0000","last_active":"2024-03-22T04:10:41.536489+0000","last_peered":"2024-03-22T04:10:41.536489+0000","last_clean":"2024-03-22T04:10:41.536489+0000","last_became_active":"2024-03-22T04:10:34.928228+0000","last_became_peered":"2024-03-22T04:10:34.928228+0000","last_unstale":"2024-03-22T04:10:41.536489+0000","last_undegraded":"2024-03-22T04:10:41.536489+0000","last_fullsized":"2024-03-22T04:10:41.536489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:41.449121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536201+0000","last_change":"2024-03-22T04:10:34.951914+0000","last_active":"2024-03-22T04:10:41.536201+0000","last_peered":"2024-03-22T04:10:41.536201+0000","last_clean":"2024-03-22T04:10:41.536201+0000","last_became_active":"2024-03-22T04:10:34.951608+0000","last_became_peered":"2024-03-22T04:10:34.951608+0000","last_unstale":"2024-03-22T04:10:41.536201+0000","last_undegraded":"2024-03-22T04:10:41.536201+0000","last_fullsized":"2024-03-22T04:10:41.536201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:18.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964949+0000","last_change":"2024-03-22T04:10:34.936728+0000","last_active":"2024-03-22T04:10:39.964949+0000","last_peered":"2024-03-22T04:10:39.964949+0000","last_clean":"2024-03-22T04:10:39.964949+0000","last_became_active":"2024-03-22T04:10:34.935969+0000","last_became_peered":"2024-03-22T04:10:34.935969+0000","last_unstale":"2024-03-22T04:10:39.964949+0000","last_undegraded":"2024-03-22T04:10:39.964949+0000","last_fullsized":"2024-03-22T04:10:39.964949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:48.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170702+0000","last_change":"2024-03-22T04:10:34.943850+0000","last_active":"2024-03-22T04:10:41.170702+0000","last_peered":"2024-03-22T04:10:41.170702+0000","last_clean":"2024-03-22T04:10:41.170702+0000","last_became_active":"2024-03-22T04:10:34.941930+0000","last_became_peered":"2024-03-22T04:10:34.941930+0000","last_unstale":"2024-03-22T04:10:41.170702+0000","last_undegraded":"2024-03-22T04:10:41.170702+0000","last_fullsized":"2024-03-22T04:10:41.170702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:55.025923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921894+0000","last_change":"2024-03-22T04:10:34.946337+0000","last_active":"2024-03-22T04:10:40.921894+0000","last_peered":"2024-03-22T04:10:40.921894+0000","last_clean":"2024-03-22T04:10:40.921894+0000","last_became_active":"2024-03-22T04:10:34.946083+0000","last_became_peered":"2024-03-22T04:10:34.946083+0000","last_unstale":"2024-03-22T04:10:40.921894+0000","last_undegraded":"2024-03-22T04:10:40.921894+0000","last_fullsized":"2024-03-22T04:10:40.921894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:15:33.976508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170005+0000","last_change":"2024-03-22T04:10:34.939061+0000","last_active":"2024-03-22T04:10:41.170005+0000","last_peered":"2024-03-22T04:10:41.170005+0000","last_clean":"2024-03-22T04:10:41.170005+0000","last_became_active":"2024-03-22T04:10:34.938566+0000","last_became_peered":"2024-03-22T04:10:34.938566+0000","last_unstale":"2024-03-22T04:10:41.170005+0000","last_undegraded":"2024-03-22T04:10:41.170005+0000","last_fullsized":"2024-03-22T04:10:41.170005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:44:57.681730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535168+0000","last_change":"2024-03-22T04:10:34.933002+0000","last_active":"2024-03-22T04:10:41.535168+0000","last_peered":"2024-03-22T04:10:41.535168+0000","last_clean":"2024-03-22T04:10:41.535168+0000","last_became_active":"2024-03-22T04:10:34.932710+0000","last_became_peered":"2024-03-22T04:10:34.932710+0000","last_unstale":"2024-03-22T04:10:41.535168+0000","last_undegraded":"2024-03-22T04:10:41.535168+0000","last_fullsized":"2024-03-22T04:10:41.535168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967328+0000","last_change":"2024-03-22T04:10:34.926419+0000","last_active":"2024-03-22T04:10:39.967328+0000","last_peered":"2024-03-22T04:10:39.967328+0000","last_clean":"2024-03-22T04:10:39.967328+0000","last_became_active":"2024-03-22T04:10:34.924952+0000","last_became_peered":"2024-03-22T04:10:34.924952+0000","last_unstale":"2024-03-22T04:10:39.967328+0000","last_undegraded":"2024-03-22T04:10:39.967328+0000","last_fullsized":"2024-03-22T04:10:39.967328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:30:41.171941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535087+0000","last_change":"2024-03-22T04:10:34.939886+0000","last_active":"2024-03-22T04:10:41.535087+0000","last_peered":"2024-03-22T04:10:41.535087+0000","last_clean":"2024-03-22T04:10:41.535087+0000","last_became_active":"2024-03-22T04:10:34.939633+0000","last_became_peered":"2024-03-22T04:10:34.939633+0000","last_unstale":"2024-03-22T04:10:41.535087+0000","last_undegraded":"2024-03-22T04:10:41.535087+0000","last_fullsized":"2024-03-22T04:10:41.535087+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:41.877296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922616+0000","last_change":"2024-03-22T04:10:34.930917+0000","last_active":"2024-03-22T04:10:40.922616+0000","last_peered":"2024-03-22T04:10:40.922616+0000","last_clean":"2024-03-22T04:10:40.922616+0000","last_became_active":"2024-03-22T04:10:34.930658+0000","last_became_peered":"2024-03-22T04:10:34.930658+0000","last_unstale":"2024-03-22T04:10:40.922616+0000","last_undegraded":"2024-03-22T04:10:40.922616+0000","last_fullsized":"2024-03-22T04:10:40.922616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:40:34.158803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967631+0000","last_change":"2024-03-22T04:10:34.953215+0000","last_active":"2024-03-22T04:10:39.967631+0000","last_peered":"2024-03-22T04:10:39.967631+0000","last_clean":"2024-03-22T04:10:39.967631+0000","last_became_active":"2024-03-22T04:10:34.952953+0000","last_became_peered":"2024-03-22T04:10:34.952953+0000","last_unstale":"2024-03-22T04:10:39.967631+0000","last_undegraded":"2024-03-22T04:10:39.967631+0000","last_fullsized":"2024-03-22T04:10:39.967631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922902+0000","last_change":"2024-03-22T04:10:34.937971+0000","last_active":"2024-03-22T04:10:40.922902+0000","last_peered":"2024-03-22T04:10:40.922902+0000","last_clean":"2024-03-22T04:10:40.922902+0000","last_became_active":"2024-03-22T04:10:34.937744+0000","last_became_peered":"2024-03-22T04:10:34.937744+0000","last_unstale":"2024-03-22T04:10:40.922902+0000","last_undegraded":"2024-03-22T04:10:40.922902+0000","last_fullsized":"2024-03-22T04:10:40.922902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:17.641505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921877+0000","last_change":"2024-03-22T04:10:34.929076+0000","last_active":"2024-03-22T04:10:40.921877+0000","last_peered":"2024-03-22T04:10:40.921877+0000","last_clean":"2024-03-22T04:10:40.921877+0000","last_became_active":"2024-03-22T04:10:34.928742+0000","last_became_peered":"2024-03-22T04:10:34.928742+0000","last_unstale":"2024-03-22T04:10:40.921877+0000","last_undegraded":"2024-03-22T04:10:40.921877+0000","last_fullsized":"2024-03-22T04:10:40.921877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:04.974724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169693+0000","last_change":"2024-03-22T04:10:34.935025+0000","last_active":"2024-03-22T04:10:41.169693+0000","last_peered":"2024-03-22T04:10:41.169693+0000","last_clean":"2024-03-22T04:10:41.169693+0000","last_became_active":"2024-03-22T04:10:34.934504+0000","last_became_peered":"2024-03-22T04:10:34.934504+0000","last_unstale":"2024-03-22T04:10:41.169693+0000","last_undegraded":"2024-03-22T04:10:41.169693+0000","last_fullsized":"2024-03-22T04:10:41.169693+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:18.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536006+0000","last_change":"2024-03-22T04:10:34.952611+0000","last_active":"2024-03-22T04:10:41.536006+0000","last_peered":"2024-03-22T04:10:41.536006+0000","last_clean":"2024-03-22T04:10:41.536006+0000","last_became_active":"2024-03-22T04:10:34.952289+0000","last_became_peered":"2024-03-22T04:10:34.952289+0000","last_unstale":"2024-03-22T04:10:41.536006+0000","last_undegraded":"2024-03-22T04:10:41.536006+0000","last_fullsized":"2024-03-22T04:10:41.536006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:44.874307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536345+0000","last_change":"2024-03-22T04:10:34.938439+0000","last_active":"2024-03-22T04:10:41.536345+0000","last_peered":"2024-03-22T04:10:41.536345+0000","last_clean":"2024-03-22T04:10:41.536345+0000","last_became_active":"2024-03-22T04:10:34.938160+0000","last_became_peered":"2024-03-22T04:10:34.938160+0000","last_unstale":"2024-03-22T04:10:41.536345+0000","last_undegraded":"2024-03-22T04:10:41.536345+0000","last_fullsized":"2024-03-22T04:10:41.536345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:40.255817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535047+0000","last_change":"2024-03-22T04:10:34.936432+0000","last_active":"2024-03-22T04:10:41.535047+0000","last_peered":"2024-03-22T04:10:41.535047+0000","last_clean":"2024-03-22T04:10:41.535047+0000","last_became_active":"2024-03-22T04:10:34.936175+0000","last_became_peered":"2024-03-22T04:10:34.936175+0000","last_unstale":"2024-03-22T04:10:41.535047+0000","last_undegraded":"2024-03-22T04:10:41.535047+0000","last_fullsized":"2024-03-22T04:10:41.535047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:34.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535725+0000","last_change":"2024-03-22T04:10:34.947651+0000","last_active":"2024-03-22T04:10:41.535725+0000","last_peered":"2024-03-22T04:10:41.535725+0000","last_clean":"2024-03-22T04:10:41.535725+0000","last_became_active":"2024-03-22T04:10:34.947262+0000","last_became_peered":"2024-03-22T04:10:34.947262+0000","last_unstale":"2024-03-22T04:10:41.535725+0000","last_undegraded":"2024-03-22T04:10:41.535725+0000","last_fullsized":"2024-03-22T04:10:41.535725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:16.846964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536037+0000","last_change":"2024-03-22T04:10:34.929968+0000","last_active":"2024-03-22T04:10:41.536037+0000","last_peered":"2024-03-22T04:10:41.536037+0000","last_clean":"2024-03-22T04:10:41.536037+0000","last_became_active":"2024-03-22T04:10:34.929650+0000","last_became_peered":"2024-03-22T04:10:34.929650+0000","last_unstale":"2024-03-22T04:10:41.536037+0000","last_undegraded":"2024-03-22T04:10:41.536037+0000","last_fullsized":"2024-03-22T04:10:41.536037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:04.974724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535701+0000","last_change":"2024-03-22T04:10:34.949078+0000","last_active":"2024-03-22T04:10:41.535701+0000","last_peered":"2024-03-22T04:10:41.535701+0000","last_clean":"2024-03-22T04:10:41.535701+0000","last_became_active":"2024-03-22T04:10:34.948775+0000","last_became_peered":"2024-03-22T04:10:34.948775+0000","last_unstale":"2024-03-22T04:10:41.535701+0000","last_undegraded":"2024-03-22T04:10:41.535701+0000","last_fullsized":"2024-03-22T04:10:41.535701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:48.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169713+0000","last_change":"2024-03-22T04:10:34.931467+0000","last_active":"2024-03-22T04:10:41.169713+0000","last_peered":"2024-03-22T04:10:41.169713+0000","last_clean":"2024-03-22T04:10:41.169713+0000","last_became_active":"2024-03-22T04:10:34.931119+0000","last_became_peered":"2024-03-22T04:10:34.931119+0000","last_unstale":"2024-03-22T04:10:41.169713+0000","last_undegraded":"2024-03-22T04:10:41.169713+0000","last_fullsized":"2024-03-22T04:10:41.169713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:18:49.597662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534582+0000","last_change":"2024-03-22T04:10:34.925425+0000","last_active":"2024-03-22T04:10:41.534582+0000","last_peered":"2024-03-22T04:10:41.534582+0000","last_clean":"2024-03-22T04:10:41.534582+0000","last_became_active":"2024-03-22T04:10:34.925151+0000","last_became_peered":"2024-03-22T04:10:34.925151+0000","last_unstale":"2024-03-22T04:10:41.534582+0000","last_undegraded":"2024-03-22T04:10:41.534582+0000","last_fullsized":"2024-03-22T04:10:41.534582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:11.105591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964686+0000","last_change":"2024-03-22T04:10:34.926711+0000","last_active":"2024-03-22T04:10:39.964686+0000","last_peered":"2024-03-22T04:10:39.964686+0000","last_clean":"2024-03-22T04:10:39.964686+0000","last_became_active":"2024-03-22T04:10:34.925765+0000","last_became_peered":"2024-03-22T04:10:34.925765+0000","last_unstale":"2024-03-22T04:10:39.964686+0000","last_undegraded":"2024-03-22T04:10:39.964686+0000","last_fullsized":"2024-03-22T04:10:39.964686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534972+0000","last_change":"2024-03-22T04:10:34.932009+0000","last_active":"2024-03-22T04:10:41.534972+0000","last_peered":"2024-03-22T04:10:41.534972+0000","last_clean":"2024-03-22T04:10:41.534972+0000","last_became_active":"2024-03-22T04:10:34.931692+0000","last_became_peered":"2024-03-22T04:10:34.931692+0000","last_unstale":"2024-03-22T04:10:41.534972+0000","last_undegraded":"2024-03-22T04:10:41.534972+0000","last_fullsized":"2024-03-22T04:10:41.534972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:08.063872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964556+0000","last_change":"2024-03-22T04:10:34.921659+0000","last_active":"2024-03-22T04:10:39.964556+0000","last_peered":"2024-03-22T04:10:39.964556+0000","last_clean":"2024-03-22T04:10:39.964556+0000","last_became_active":"2024-03-22T04:10:34.921420+0000","last_became_peered":"2024-03-22T04:10:34.921420+0000","last_unstale":"2024-03-22T04:10:39.964556+0000","last_undegraded":"2024-03-22T04:10:39.964556+0000","last_fullsized":"2024-03-22T04:10:39.964556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:01.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170351+0000","last_change":"2024-03-22T04:10:34.928037+0000","last_active":"2024-03-22T04:10:41.170351+0000","last_peered":"2024-03-22T04:10:41.170351+0000","last_clean":"2024-03-22T04:10:41.170351+0000","last_became_active":"2024-03-22T04:10:34.927746+0000","last_became_peered":"2024-03-22T04:10:34.927746+0000","last_unstale":"2024-03-22T04:10:41.170351+0000","last_undegraded":"2024-03-22T04:10:41.170351+0000","last_fullsized":"2024-03-22T04:10:41.170351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:29:34.250954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170416+0000","last_change":"2024-03-22T04:10:34.931962+0000","last_active":"2024-03-22T04:10:41.170416+0000","last_peered":"2024-03-22T04:10:41.170416+0000","last_clean":"2024-03-22T04:10:41.170416+0000","last_became_active":"2024-03-22T04:10:34.931736+0000","last_became_peered":"2024-03-22T04:10:34.931736+0000","last_unstale":"2024-03-22T04:10:41.170416+0000","last_undegraded":"2024-03-22T04:10:41.170416+0000","last_fullsized":"2024-03-22T04:10:41.170416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:16.846964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535903+0000","last_change":"2024-03-22T04:10:34.946756+0000","last_active":"2024-03-22T04:10:41.535903+0000","last_peered":"2024-03-22T04:10:41.535903+0000","last_clean":"2024-03-22T04:10:41.535903+0000","last_became_active":"2024-03-22T04:10:34.946501+0000","last_became_peered":"2024-03-22T04:10:34.946501+0000","last_unstale":"2024-03-22T04:10:41.535903+0000","last_undegraded":"2024-03-22T04:10:41.535903+0000","last_fullsized":"2024-03-22T04:10:41.535903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:02:47.197474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:11:02.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:11:03.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb243400640 1 Processor -- start 2024-03-22T04:11:03.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb243400640 1 -- start start 2024-03-22T04:11:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb243400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb23c06df90 0x7fb23c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb243400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb23c15ac10 0x7fb23c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb243400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb23c15c550 con 0x7fb23c15bc80 2024-03-22T04:11:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb243400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb23c15cb60 con 0x7fb23c15ac10 2024-03-22T04:11:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb243400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb23c06e8b0 con 0x7fb23c06df90 2024-03-22T04:11:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb23be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb23c15ac10 0x7fb23c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb241400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb23c15bc80 0x7fb23c15e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57558/0 (socket says 172.21.15.86:57558) 2024-03-22T04:11:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb241400640 1 -- 172.21.15.86:0/282672105 learned_addr learned my addr 172.21.15.86:0/282672105 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.084+0000 7fb240a00640 1 --2- 172.21.15.86:0/282672105 >> v2:172.21.15.196:3300/0 conn(0x7fb23c06df90 0x7fb23c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23be00640 1 -- 172.21.15.86:0/282672105 >> v2:172.21.15.196:3300/0 conn(0x7fb23c06df90 msgr2=0x7fb23c06e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23be00640 1 --2- 172.21.15.86:0/282672105 >> v2:172.21.15.196:3300/0 conn(0x7fb23c06df90 0x7fb23c06e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23be00640 1 -- 172.21.15.86:0/282672105 >> v1:172.21.15.86:6789/0 conn(0x7fb23c15bc80 legacy=0x7fb23c15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23be00640 1 -- 172.21.15.86:0/282672105 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c0b4750 con 0x7fb23c15ac10 2024-03-22T04:11:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23b400640 1 -- 172.21.15.86:0/282672105 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3500995428 0 0) 0x7fb23c15c550 con 0x7fb23c15bc80 2024-03-22T04:11:03.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23be00640 1 --2- 172.21.15.86:0/282672105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0b3c00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0b3480 tx=0x7fb22c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad934ed48e0abe server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23b400640 1 -- 172.21.15.86:0/282672105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0ab650 con 0x7fb23c15ac10 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23b400640 1 -- 172.21.15.86:0/282672105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c0b0070 con 0x7fb23c15ac10 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb23b400640 1 -- 172.21.15.86:0/282672105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0aba10 con 0x7fb23c15ac10 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb243400640 1 -- 172.21.15.86:0/282672105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 msgr2=0x7fb23c0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb243400640 1 --2- 172.21.15.86:0/282672105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0b3c00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0b3480 tx=0x7fb22c01c450 comp rx=0 tx=0).stop 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb243400640 1 -- 172.21.15.86:0/282672105 shutdown_connections 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb243400640 1 --2- 172.21.15.86:0/282672105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0b3c00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb243400640 1 --2- 172.21.15.86:0/282672105 >> v2:172.21.15.196:3300/0 conn(0x7fb23c06df90 0x7fb23c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb243400640 1 -- 172.21.15.86:0/282672105 >> 172.21.15.86:0/282672105 conn(0x7fb23c0b1df0 msgr2=0x7fb23c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb243400640 1 -- 172.21.15.86:0/282672105 shutdown_connections 2024-03-22T04:11:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.088+0000 7fb243400640 1 -- 172.21.15.86:0/282672105 wait complete. 2024-03-22T04:11:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 Processor -- start 2024-03-22T04:11:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 -- start start 2024-03-22T04:11:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb23c06df90 0x7fb23c164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c15bc80 0x7fb23c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c06d880 con 0x7fb23c15ac10 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c06d700 con 0x7fb23c15bc80 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb23c06da00 con 0x7fb23c06df90 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56244/0 (socket says 172.21.15.86:56244) 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 -- 172.21.15.86:0/849541866 learned_addr learned my addr 172.21.15.86:0/849541866 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb240a00640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb23c06df90 0x7fb23c164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb241400640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c15bc80 0x7fb23c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 -- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb23c06df90 msgr2=0x7fb23c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb23c06df90 0x7fb23c164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 -- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c15bc80 msgr2=0x7fb23c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c15bc80 0x7fb23c0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 -- 172.21.15.86:0/849541866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c0ce8a0 con 0x7fb23c15ac10 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb241400640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c15bc80 0x7fb23c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb23be00640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0ae930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0ad040 tx=0x7fb22c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb238c00640 1 -- 172.21.15.86:0/849541866 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c006e20 con 0x7fb23c15ac10 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb238c00640 1 -- 172.21.15.86:0/849541866 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c004230 con 0x7fb23c15ac10 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb238c00640 1 -- 172.21.15.86:0/849541866 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0abe40 con 0x7fb23c15ac10 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c1fb460 con 0x7fb23c15ac10 2024-03-22T04:11:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.092+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c0cee00 con 0x7fb23c15ac10 2024-03-22T04:11:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.096+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c068650 con 0x7fb23c15ac10 2024-03-22T04:11:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.096+0000 7fb238c00640 1 -- 172.21.15.86:0/849541866 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb22c003620 con 0x7fb23c15ac10 2024-03-22T04:11:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.096+0000 7fb238c00640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb218067b20 0x7fb218069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.096+0000 7fb238c00640 1 -- 172.21.15.86:0/849541866 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb22c120da0 con 0x7fb23c15ac10 2024-03-22T04:11:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.096+0000 7fb240a00640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb218067b20 0x7fb218069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.096+0000 7fb240a00640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb218067b20 0x7fb218069fe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb224009b60 tx=0x7fb224009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.100+0000 7fb238c00640 1 -- 172.21.15.86:0/849541866 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb23c0ce8a0 con 0x7fb23c15ac10 2024-03-22T04:11:03.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.252+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb23c06ed60 con 0x7fb218067b20 2024-03-22T04:11:03.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.260+0000 7fb238c00640 1 -- 172.21.15.86:0/849541866 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fb23c06ed60 con 0x7fb218067b20 2024-03-22T04:11:03.267 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:03.269 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:11:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb218067b20 msgr2=0x7fb218069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb218067b20 0x7fb218069fe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb224009b60 tx=0x7fb224009190 comp rx=0 tx=0).stop 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 msgr2=0x7fb23c0ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0ae930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0ad040 tx=0x7fb22c01ac90 comp rx=0 tx=0).stop 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 shutdown_connections 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb218067b20 0x7fb218069fe0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb23c15bc80 0x7fb23c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23c15ac10 0x7fb23c0ae930 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 --2- 172.21.15.86:0/849541866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb23c06df90 0x7fb23c164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 >> 172.21.15.86:0/849541866 conn(0x7fb23c0b1df0 msgr2=0x7fb23c072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 shutdown_connections 2024-03-22T04:11:03.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.264+0000 7fb243400640 1 -- 172.21.15.86:0/849541866 wait complete. 2024-03-22T04:11:03.287 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-03-22T04:11:01.797157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219592,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755128,"statfs":{"total":773094113280,"available":772869251072,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002770"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967926+0000","last_change":"2024-03-22T04:10:33.930897+0000","last_active":"2024-03-22T04:10:39.967926+0000","last_peered":"2024-03-22T04:10:39.967926+0000","last_clean":"2024-03-22T04:10:39.967926+0000","last_became_active":"2024-03-22T04:10:33.930543+0000","last_became_peered":"2024-03-22T04:10:33.930543+0000","last_unstale":"2024-03-22T04:10:39.967926+0000","last_undegraded":"2024-03-22T04:10:39.967926+0000","last_fullsized":"2024-03-22T04:10:39.967926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:57:52.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170514+0000","last_change":"2024-03-22T04:10:34.938515+0000","last_active":"2024-03-22T04:10:41.170514+0000","last_peered":"2024-03-22T04:10:41.170514+0000","last_clean":"2024-03-22T04:10:41.170514+0000","last_became_active":"2024-03-22T04:10:34.938169+0000","last_became_peered":"2024-03-22T04:10:34.938169+0000","last_unstale":"2024-03-22T04:10:41.170514+0000","last_undegraded":"2024-03-22T04:10:41.170514+0000","last_fullsized":"2024-03-22T04:10:41.170514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:34.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966658+0000","last_change":"2024-03-22T04:10:38.983720+0000","last_active":"2024-03-22T04:10:39.966658+0000","last_peered":"2024-03-22T04:10:39.966658+0000","last_clean":"2024-03-22T04:10:39.966658+0000","last_became_active":"2024-03-22T04:10:38.983348+0000","last_became_peered":"2024-03-22T04:10:38.983348+0000","last_unstale":"2024-03-22T04:10:39.966658+0000","last_undegraded":"2024-03-22T04:10:39.966658+0000","last_fullsized":"2024-03-22T04:10:39.966658+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:45.494650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170607+0000","last_change":"2024-03-22T04:10:33.934030+0000","last_active":"2024-03-22T04:10:41.170607+0000","last_peered":"2024-03-22T04:10:41.170607+0000","last_clean":"2024-03-22T04:10:41.170607+0000","last_became_active":"2024-03-22T04:10:33.933718+0000","last_became_peered":"2024-03-22T04:10:33.933718+0000","last_unstale":"2024-03-22T04:10:41.170607+0000","last_undegraded":"2024-03-22T04:10:41.170607+0000","last_fullsized":"2024-03-22T04:10:41.170607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:50:42.805996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170782+0000","last_change":"2024-03-22T04:10:34.942768+0000","last_active":"2024-03-22T04:10:41.170782+0000","last_peered":"2024-03-22T04:10:41.170782+0000","last_clean":"2024-03-22T04:10:41.170782+0000","last_became_active":"2024-03-22T04:10:34.941567+0000","last_became_peered":"2024-03-22T04:10:34.941567+0000","last_unstale":"2024-03-22T04:10:41.170782+0000","last_undegraded":"2024-03-22T04:10:41.170782+0000","last_fullsized":"2024-03-22T04:10:41.170782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:26.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964401+0000","last_change":"2024-03-22T04:10:38.988088+0000","last_active":"2024-03-22T04:10:39.964401+0000","last_peered":"2024-03-22T04:10:39.964401+0000","last_clean":"2024-03-22T04:10:39.964401+0000","last_became_active":"2024-03-22T04:10:38.987611+0000","last_became_peered":"2024-03-22T04:10:38.987611+0000","last_unstale":"2024-03-22T04:10:39.964401+0000","last_undegraded":"2024-03-22T04:10:39.964401+0000","last_fullsized":"2024-03-22T04:10:39.964401+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:15.151121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535943+0000","last_change":"2024-03-22T04:10:33.936193+0000","last_active":"2024-03-22T04:10:41.535943+0000","last_peered":"2024-03-22T04:10:41.535943+0000","last_clean":"2024-03-22T04:10:41.535943+0000","last_became_active":"2024-03-22T04:10:33.935883+0000","last_became_peered":"2024-03-22T04:10:33.935883+0000","last_unstale":"2024-03-22T04:10:41.535943+0000","last_undegraded":"2024-03-22T04:10:41.535943+0000","last_fullsized":"2024-03-22T04:10:41.535943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:00.244111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535389+0000","last_change":"2024-03-22T04:10:34.935504+0000","last_active":"2024-03-22T04:10:41.535389+0000","last_peered":"2024-03-22T04:10:41.535389+0000","last_clean":"2024-03-22T04:10:41.535389+0000","last_became_active":"2024-03-22T04:10:34.933502+0000","last_became_peered":"2024-03-22T04:10:34.933502+0000","last_unstale":"2024-03-22T04:10:41.535389+0000","last_undegraded":"2024-03-22T04:10:41.535389+0000","last_fullsized":"2024-03-22T04:10:41.535389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965123+0000","last_change":"2024-03-22T04:10:38.991103+0000","last_active":"2024-03-22T04:10:39.965123+0000","last_peered":"2024-03-22T04:10:39.965123+0000","last_clean":"2024-03-22T04:10:39.965123+0000","last_became_active":"2024-03-22T04:10:38.988847+0000","last_became_peered":"2024-03-22T04:10:38.988847+0000","last_unstale":"2024-03-22T04:10:39.965123+0000","last_undegraded":"2024-03-22T04:10:39.965123+0000","last_fullsized":"2024-03-22T04:10:39.965123+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:45.922825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535279+0000","last_change":"2024-03-22T04:10:33.936237+0000","last_active":"2024-03-22T04:10:41.535279+0000","last_peered":"2024-03-22T04:10:41.535279+0000","last_clean":"2024-03-22T04:10:41.535279+0000","last_became_active":"2024-03-22T04:10:33.935870+0000","last_became_peered":"2024-03-22T04:10:33.935870+0000","last_unstale":"2024-03-22T04:10:41.535279+0000","last_undegraded":"2024-03-22T04:10:41.535279+0000","last_fullsized":"2024-03-22T04:10:41.535279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:47.209113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922489+0000","last_change":"2024-03-22T04:10:34.930847+0000","last_active":"2024-03-22T04:10:40.922489+0000","last_peered":"2024-03-22T04:10:40.922489+0000","last_clean":"2024-03-22T04:10:40.922489+0000","last_became_active":"2024-03-22T04:10:34.930567+0000","last_became_peered":"2024-03-22T04:10:34.930567+0000","last_unstale":"2024-03-22T04:10:40.922489+0000","last_undegraded":"2024-03-22T04:10:40.922489+0000","last_fullsized":"2024-03-22T04:10:40.922489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:17.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964576+0000","last_change":"2024-03-22T04:10:38.992358+0000","last_active":"2024-03-22T04:10:39.964576+0000","last_peered":"2024-03-22T04:10:39.964576+0000","last_clean":"2024-03-22T04:10:39.964576+0000","last_became_active":"2024-03-22T04:10:38.991968+0000","last_became_peered":"2024-03-22T04:10:38.991968+0000","last_unstale":"2024-03-22T04:10:39.964576+0000","last_undegraded":"2024-03-22T04:10:39.964576+0000","last_fullsized":"2024-03-22T04:10:39.964576+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:12.109401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922040+0000","last_change":"2024-03-22T04:10:34.947353+0000","last_active":"2024-03-22T04:10:40.922040+0000","last_peered":"2024-03-22T04:10:40.922040+0000","last_clean":"2024-03-22T04:10:40.922040+0000","last_became_active":"2024-03-22T04:10:34.945247+0000","last_became_peered":"2024-03-22T04:10:34.945247+0000","last_unstale":"2024-03-22T04:10:40.922040+0000","last_undegraded":"2024-03-22T04:10:40.922040+0000","last_fullsized":"2024-03-22T04:10:40.922040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:31.574004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922244+0000","last_change":"2024-03-22T04:10:33.938771+0000","last_active":"2024-03-22T04:10:40.922244+0000","last_peered":"2024-03-22T04:10:40.922244+0000","last_clean":"2024-03-22T04:10:40.922244+0000","last_became_active":"2024-03-22T04:10:33.937907+0000","last_became_peered":"2024-03-22T04:10:33.937907+0000","last_unstale":"2024-03-22T04:10:40.922244+0000","last_undegraded":"2024-03-22T04:10:40.922244+0000","last_fullsized":"2024-03-22T04:10:40.922244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:43.867139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922386+0000","last_change":"2024-03-22T04:10:38.979866+0000","last_active":"2024-03-22T04:10:40.922386+0000","last_peered":"2024-03-22T04:10:40.922386+0000","last_clean":"2024-03-22T04:10:40.922386+0000","last_became_active":"2024-03-22T04:10:38.979321+0000","last_became_peered":"2024-03-22T04:10:38.979321+0000","last_unstale":"2024-03-22T04:10:40.922386+0000","last_undegraded":"2024-03-22T04:10:40.922386+0000","last_fullsized":"2024-03-22T04:10:40.922386+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:03:12.221975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967207+0000","last_change":"2024-03-22T04:10:34.951257+0000","last_active":"2024-03-22T04:10:39.967207+0000","last_peered":"2024-03-22T04:10:39.967207+0000","last_clean":"2024-03-22T04:10:39.967207+0000","last_became_active":"2024-03-22T04:10:34.950994+0000","last_became_peered":"2024-03-22T04:10:34.950994+0000","last_unstale":"2024-03-22T04:10:39.967207+0000","last_undegraded":"2024-03-22T04:10:39.967207+0000","last_fullsized":"2024-03-22T04:10:39.967207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536303+0000","last_change":"2024-03-22T04:10:33.933287+0000","last_active":"2024-03-22T04:10:41.536303+0000","last_peered":"2024-03-22T04:10:41.536303+0000","last_clean":"2024-03-22T04:10:41.536303+0000","last_became_active":"2024-03-22T04:10:33.932862+0000","last_became_peered":"2024-03-22T04:10:33.932862+0000","last_unstale":"2024-03-22T04:10:41.536303+0000","last_undegraded":"2024-03-22T04:10:41.536303+0000","last_fullsized":"2024-03-22T04:10:41.536303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534955+0000","last_change":"2024-03-22T04:10:38.986486+0000","last_active":"2024-03-22T04:10:41.534955+0000","last_peered":"2024-03-22T04:10:41.534955+0000","last_clean":"2024-03-22T04:10:41.534955+0000","last_became_active":"2024-03-22T04:10:38.985993+0000","last_became_peered":"2024-03-22T04:10:38.985993+0000","last_unstale":"2024-03-22T04:10:41.534955+0000","last_undegraded":"2024-03-22T04:10:41.534955+0000","last_fullsized":"2024-03-22T04:10:41.534955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:21.157155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966213+0000","last_change":"2024-03-22T04:10:34.946405+0000","last_active":"2024-03-22T04:10:39.966213+0000","last_peered":"2024-03-22T04:10:39.966213+0000","last_clean":"2024-03-22T04:10:39.966213+0000","last_became_active":"2024-03-22T04:10:34.946093+0000","last_became_peered":"2024-03-22T04:10:34.946093+0000","last_unstale":"2024-03-22T04:10:39.966213+0000","last_undegraded":"2024-03-22T04:10:39.966213+0000","last_fullsized":"2024-03-22T04:10:39.966213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:54:39.775612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966845+0000","last_change":"2024-03-22T04:10:33.931984+0000","last_active":"2024-03-22T04:10:39.966845+0000","last_peered":"2024-03-22T04:10:39.966845+0000","last_clean":"2024-03-22T04:10:39.966845+0000","last_became_active":"2024-03-22T04:10:33.931674+0000","last_became_peered":"2024-03-22T04:10:33.931674+0000","last_unstale":"2024-03-22T04:10:39.966845+0000","last_undegraded":"2024-03-22T04:10:39.966845+0000","last_fullsized":"2024-03-22T04:10:39.966845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:11.685927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967850+0000","last_change":"2024-03-22T04:10:38.992239+0000","last_active":"2024-03-22T04:10:39.967850+0000","last_peered":"2024-03-22T04:10:39.967850+0000","last_clean":"2024-03-22T04:10:39.967850+0000","last_became_active":"2024-03-22T04:10:38.991429+0000","last_became_peered":"2024-03-22T04:10:38.991429+0000","last_unstale":"2024-03-22T04:10:39.967850+0000","last_undegraded":"2024-03-22T04:10:39.967850+0000","last_fullsized":"2024-03-22T04:10:39.967850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:22.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921940+0000","last_change":"2024-03-22T04:10:34.939183+0000","last_active":"2024-03-22T04:10:40.921940+0000","last_peered":"2024-03-22T04:10:40.921940+0000","last_clean":"2024-03-22T04:10:40.921940+0000","last_became_active":"2024-03-22T04:10:34.938388+0000","last_became_peered":"2024-03-22T04:10:34.938388+0000","last_unstale":"2024-03-22T04:10:40.921940+0000","last_undegraded":"2024-03-22T04:10:40.921940+0000","last_fullsized":"2024-03-22T04:10:40.921940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:10:15.750612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534426+0000","last_change":"2024-03-22T04:10:33.933352+0000","last_active":"2024-03-22T04:10:41.534426+0000","last_peered":"2024-03-22T04:10:41.534426+0000","last_clean":"2024-03-22T04:10:41.534426+0000","last_became_active":"2024-03-22T04:10:33.933066+0000","last_became_peered":"2024-03-22T04:10:33.933066+0000","last_unstale":"2024-03-22T04:10:41.534426+0000","last_undegraded":"2024-03-22T04:10:41.534426+0000","last_fullsized":"2024-03-22T04:10:41.534426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:50:42.805996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964895+0000","last_change":"2024-03-22T04:10:38.987700+0000","last_active":"2024-03-22T04:10:39.964895+0000","last_peered":"2024-03-22T04:10:39.964895+0000","last_clean":"2024-03-22T04:10:39.964895+0000","last_became_active":"2024-03-22T04:10:38.987221+0000","last_became_peered":"2024-03-22T04:10:38.987221+0000","last_unstale":"2024-03-22T04:10:39.964895+0000","last_undegraded":"2024-03-22T04:10:39.964895+0000","last_fullsized":"2024-03-22T04:10:39.964895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:30.365392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967947+0000","last_change":"2024-03-22T04:10:34.953907+0000","last_active":"2024-03-22T04:10:39.967947+0000","last_peered":"2024-03-22T04:10:39.967947+0000","last_clean":"2024-03-22T04:10:39.967947+0000","last_became_active":"2024-03-22T04:10:34.953664+0000","last_became_peered":"2024-03-22T04:10:34.953664+0000","last_unstale":"2024-03-22T04:10:39.967947+0000","last_undegraded":"2024-03-22T04:10:39.967947+0000","last_fullsized":"2024-03-22T04:10:39.967947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:07.983679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922944+0000","last_change":"2024-03-22T04:10:33.931133+0000","last_active":"2024-03-22T04:10:40.922944+0000","last_peered":"2024-03-22T04:10:40.922944+0000","last_clean":"2024-03-22T04:10:40.922944+0000","last_became_active":"2024-03-22T04:10:33.930877+0000","last_became_peered":"2024-03-22T04:10:33.930877+0000","last_unstale":"2024-03-22T04:10:40.922944+0000","last_undegraded":"2024-03-22T04:10:40.922944+0000","last_fullsized":"2024-03-22T04:10:40.922944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:39.248649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964896+0000","last_change":"2024-03-22T04:10:38.983903+0000","last_active":"2024-03-22T04:10:39.964896+0000","last_peered":"2024-03-22T04:10:39.964896+0000","last_clean":"2024-03-22T04:10:39.964896+0000","last_became_active":"2024-03-22T04:10:38.983459+0000","last_became_peered":"2024-03-22T04:10:38.983459+0000","last_unstale":"2024-03-22T04:10:39.964896+0000","last_undegraded":"2024-03-22T04:10:39.964896+0000","last_fullsized":"2024-03-22T04:10:39.964896+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:38.915424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967821+0000","last_change":"2024-03-22T04:10:34.930898+0000","last_active":"2024-03-22T04:10:39.967821+0000","last_peered":"2024-03-22T04:10:39.967821+0000","last_clean":"2024-03-22T04:10:39.967821+0000","last_became_active":"2024-03-22T04:10:34.930562+0000","last_became_peered":"2024-03-22T04:10:34.930562+0000","last_unstale":"2024-03-22T04:10:39.967821+0000","last_undegraded":"2024-03-22T04:10:39.967821+0000","last_fullsized":"2024-03-22T04:10:39.967821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:46:40.225626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.968051+0000","last_change":"2024-03-22T04:10:33.934918+0000","last_active":"2024-03-22T04:10:39.968051+0000","last_peered":"2024-03-22T04:10:39.968051+0000","last_clean":"2024-03-22T04:10:39.968051+0000","last_became_active":"2024-03-22T04:10:33.933026+0000","last_became_peered":"2024-03-22T04:10:33.933026+0000","last_unstale":"2024-03-22T04:10:39.968051+0000","last_undegraded":"2024-03-22T04:10:39.968051+0000","last_fullsized":"2024-03-22T04:10:39.968051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:42:25.730112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966295+0000","last_change":"2024-03-22T04:10:38.984737+0000","last_active":"2024-03-22T04:10:39.966295+0000","last_peered":"2024-03-22T04:10:39.966295+0000","last_clean":"2024-03-22T04:10:39.966295+0000","last_became_active":"2024-03-22T04:10:38.984373+0000","last_became_peered":"2024-03-22T04:10:38.984373+0000","last_unstale":"2024-03-22T04:10:39.966295+0000","last_undegraded":"2024-03-22T04:10:39.966295+0000","last_fullsized":"2024-03-22T04:10:39.966295+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:23:32.419897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536493+0000","last_change":"2024-03-22T04:10:34.928433+0000","last_active":"2024-03-22T04:10:41.536493+0000","last_peered":"2024-03-22T04:10:41.536493+0000","last_clean":"2024-03-22T04:10:41.536493+0000","last_became_active":"2024-03-22T04:10:34.927912+0000","last_became_peered":"2024-03-22T04:10:34.927912+0000","last_unstale":"2024-03-22T04:10:41.536493+0000","last_undegraded":"2024-03-22T04:10:41.536493+0000","last_fullsized":"2024-03-22T04:10:41.536493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:12:01.577501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922703+0000","last_change":"2024-03-22T04:10:33.938458+0000","last_active":"2024-03-22T04:10:40.922703+0000","last_peered":"2024-03-22T04:10:40.922703+0000","last_clean":"2024-03-22T04:10:40.922703+0000","last_became_active":"2024-03-22T04:10:33.937789+0000","last_became_peered":"2024-03-22T04:10:33.937789+0000","last_unstale":"2024-03-22T04:10:40.922703+0000","last_undegraded":"2024-03-22T04:10:40.922703+0000","last_fullsized":"2024-03-22T04:10:40.922703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:40.441952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536228+0000","last_change":"2024-03-22T04:10:38.988981+0000","last_active":"2024-03-22T04:10:41.536228+0000","last_peered":"2024-03-22T04:10:41.536228+0000","last_clean":"2024-03-22T04:10:41.536228+0000","last_became_active":"2024-03-22T04:10:38.988442+0000","last_became_peered":"2024-03-22T04:10:38.988442+0000","last_unstale":"2024-03-22T04:10:41.536228+0000","last_undegraded":"2024-03-22T04:10:41.536228+0000","last_fullsized":"2024-03-22T04:10:41.536228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:38.915424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964614+0000","last_change":"2024-03-22T04:10:34.931657+0000","last_active":"2024-03-22T04:10:39.964614+0000","last_peered":"2024-03-22T04:10:39.964614+0000","last_clean":"2024-03-22T04:10:39.964614+0000","last_became_active":"2024-03-22T04:10:34.931362+0000","last_became_peered":"2024-03-22T04:10:34.931362+0000","last_unstale":"2024-03-22T04:10:39.964614+0000","last_undegraded":"2024-03-22T04:10:39.964614+0000","last_fullsized":"2024-03-22T04:10:39.964614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170215+0000","last_change":"2024-03-22T04:10:33.934747+0000","last_active":"2024-03-22T04:10:41.170215+0000","last_peered":"2024-03-22T04:10:41.170215+0000","last_clean":"2024-03-22T04:10:41.170215+0000","last_became_active":"2024-03-22T04:10:33.933151+0000","last_became_peered":"2024-03-22T04:10:33.933151+0000","last_unstale":"2024-03-22T04:10:41.170215+0000","last_undegraded":"2024-03-22T04:10:41.170215+0000","last_fullsized":"2024-03-22T04:10:41.170215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536019+0000","last_change":"2024-03-22T04:10:38.972402+0000","last_active":"2024-03-22T04:10:41.536019+0000","last_peered":"2024-03-22T04:10:41.536019+0000","last_clean":"2024-03-22T04:10:41.536019+0000","last_became_active":"2024-03-22T04:10:38.972049+0000","last_became_peered":"2024-03-22T04:10:38.972049+0000","last_unstale":"2024-03-22T04:10:41.536019+0000","last_undegraded":"2024-03-22T04:10:41.536019+0000","last_fullsized":"2024-03-22T04:10:41.536019+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:20.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535049+0000","last_change":"2024-03-22T04:10:34.931298+0000","last_active":"2024-03-22T04:10:41.535049+0000","last_peered":"2024-03-22T04:10:41.535049+0000","last_clean":"2024-03-22T04:10:41.535049+0000","last_became_active":"2024-03-22T04:10:34.930240+0000","last_became_peered":"2024-03-22T04:10:34.930240+0000","last_unstale":"2024-03-22T04:10:41.535049+0000","last_undegraded":"2024-03-22T04:10:41.535049+0000","last_fullsized":"2024-03-22T04:10:41.535049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:11.753753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922237+0000","last_change":"2024-03-22T04:10:33.938276+0000","last_active":"2024-03-22T04:10:40.922237+0000","last_peered":"2024-03-22T04:10:40.922237+0000","last_clean":"2024-03-22T04:10:40.922237+0000","last_became_active":"2024-03-22T04:10:33.933126+0000","last_became_peered":"2024-03-22T04:10:33.933126+0000","last_unstale":"2024-03-22T04:10:40.922237+0000","last_undegraded":"2024-03-22T04:10:40.922237+0000","last_fullsized":"2024-03-22T04:10:40.922237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:11.368879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536420+0000","last_change":"2024-03-22T04:10:38.974697+0000","last_active":"2024-03-22T04:10:41.536420+0000","last_peered":"2024-03-22T04:10:41.536420+0000","last_clean":"2024-03-22T04:10:41.536420+0000","last_became_active":"2024-03-22T04:10:38.974339+0000","last_became_peered":"2024-03-22T04:10:38.974339+0000","last_unstale":"2024-03-22T04:10:41.536420+0000","last_undegraded":"2024-03-22T04:10:41.536420+0000","last_fullsized":"2024-03-22T04:10:41.536420+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:59.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169762+0000","last_change":"2024-03-22T04:10:34.947104+0000","last_active":"2024-03-22T04:10:41.169762+0000","last_peered":"2024-03-22T04:10:41.169762+0000","last_clean":"2024-03-22T04:10:41.169762+0000","last_became_active":"2024-03-22T04:10:34.946787+0000","last_became_peered":"2024-03-22T04:10:34.946787+0000","last_unstale":"2024-03-22T04:10:41.169762+0000","last_undegraded":"2024-03-22T04:10:41.169762+0000","last_fullsized":"2024-03-22T04:10:41.169762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:11.753753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966701+0000","last_change":"2024-03-22T04:10:33.938271+0000","last_active":"2024-03-22T04:10:39.966701+0000","last_peered":"2024-03-22T04:10:39.966701+0000","last_clean":"2024-03-22T04:10:39.966701+0000","last_became_active":"2024-03-22T04:10:33.937651+0000","last_became_peered":"2024-03-22T04:10:33.937651+0000","last_unstale":"2024-03-22T04:10:39.966701+0000","last_undegraded":"2024-03-22T04:10:39.966701+0000","last_fullsized":"2024-03-22T04:10:39.966701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536236+0000","last_change":"2024-03-22T04:10:38.980629+0000","last_active":"2024-03-22T04:10:41.536236+0000","last_peered":"2024-03-22T04:10:41.536236+0000","last_clean":"2024-03-22T04:10:41.536236+0000","last_became_active":"2024-03-22T04:10:38.980157+0000","last_became_peered":"2024-03-22T04:10:38.980157+0000","last_unstale":"2024-03-22T04:10:41.536236+0000","last_undegraded":"2024-03-22T04:10:41.536236+0000","last_fullsized":"2024-03-22T04:10:41.536236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:30.365392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535927+0000","last_change":"2024-03-22T04:10:34.940387+0000","last_active":"2024-03-22T04:10:41.535927+0000","last_peered":"2024-03-22T04:10:41.535927+0000","last_clean":"2024-03-22T04:10:41.535927+0000","last_became_active":"2024-03-22T04:10:34.940009+0000","last_became_peered":"2024-03-22T04:10:34.940009+0000","last_unstale":"2024-03-22T04:10:41.535927+0000","last_undegraded":"2024-03-22T04:10:41.535927+0000","last_fullsized":"2024-03-22T04:10:41.535927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:12.376047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922076+0000","last_change":"2024-03-22T04:10:33.927977+0000","last_active":"2024-03-22T04:10:40.922076+0000","last_peered":"2024-03-22T04:10:40.922076+0000","last_clean":"2024-03-22T04:10:40.922076+0000","last_became_active":"2024-03-22T04:10:33.927635+0000","last_became_peered":"2024-03-22T04:10:33.927635+0000","last_unstale":"2024-03-22T04:10:40.922076+0000","last_undegraded":"2024-03-22T04:10:40.922076+0000","last_fullsized":"2024-03-22T04:10:40.922076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:00.244111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170363+0000","last_change":"2024-03-22T04:10:38.985749+0000","last_active":"2024-03-22T04:10:41.170363+0000","last_peered":"2024-03-22T04:10:41.170363+0000","last_clean":"2024-03-22T04:10:41.170363+0000","last_became_active":"2024-03-22T04:10:38.981278+0000","last_became_peered":"2024-03-22T04:10:38.981278+0000","last_unstale":"2024-03-22T04:10:41.170363+0000","last_undegraded":"2024-03-22T04:10:41.170363+0000","last_fullsized":"2024-03-22T04:10:41.170363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:40:38.204332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966651+0000","last_change":"2024-03-22T04:10:34.921916+0000","last_active":"2024-03-22T04:10:39.966651+0000","last_peered":"2024-03-22T04:10:39.966651+0000","last_clean":"2024-03-22T04:10:39.966651+0000","last_became_active":"2024-03-22T04:10:34.921675+0000","last_became_peered":"2024-03-22T04:10:34.921675+0000","last_unstale":"2024-03-22T04:10:39.966651+0000","last_undegraded":"2024-03-22T04:10:39.966651+0000","last_fullsized":"2024-03-22T04:10:39.966651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:23:07.242835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922298+0000","last_change":"2024-03-22T04:10:33.937845+0000","last_active":"2024-03-22T04:10:40.922298+0000","last_peered":"2024-03-22T04:10:40.922298+0000","last_clean":"2024-03-22T04:10:40.922298+0000","last_became_active":"2024-03-22T04:10:33.930400+0000","last_became_peered":"2024-03-22T04:10:33.930400+0000","last_unstale":"2024-03-22T04:10:40.922298+0000","last_undegraded":"2024-03-22T04:10:40.922298+0000","last_fullsized":"2024-03-22T04:10:40.922298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:03.705836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":177,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535319+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:10:41.535319+0000","last_peered":"2024-03-22T04:10:41.535319+0000","last_clean":"2024-03-22T04:10:41.535319+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:10:41.535319+0000","last_undegraded":"2024-03-22T04:10:41.535319+0000","last_fullsized":"2024-03-22T04:10:41.535319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922657+0000","last_change":"2024-03-22T04:10:38.982996+0000","last_active":"2024-03-22T04:10:40.922657+0000","last_peered":"2024-03-22T04:10:40.922657+0000","last_clean":"2024-03-22T04:10:40.922657+0000","last_became_active":"2024-03-22T04:10:38.981238+0000","last_became_peered":"2024-03-22T04:10:38.981238+0000","last_unstale":"2024-03-22T04:10:40.922657+0000","last_undegraded":"2024-03-22T04:10:40.922657+0000","last_fullsized":"2024-03-22T04:10:40.922657+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:09:02.614234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922533+0000","last_change":"2024-03-22T04:10:34.945879+0000","last_active":"2024-03-22T04:10:40.922533+0000","last_peered":"2024-03-22T04:10:40.922533+0000","last_clean":"2024-03-22T04:10:40.922533+0000","last_became_active":"2024-03-22T04:10:34.945056+0000","last_became_peered":"2024-03-22T04:10:34.945056+0000","last_unstale":"2024-03-22T04:10:40.922533+0000","last_undegraded":"2024-03-22T04:10:40.922533+0000","last_fullsized":"2024-03-22T04:10:40.922533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:17:47.020807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536542+0000","last_change":"2024-03-22T04:10:33.931838+0000","last_active":"2024-03-22T04:10:41.536542+0000","last_peered":"2024-03-22T04:10:41.536542+0000","last_clean":"2024-03-22T04:10:41.536542+0000","last_became_active":"2024-03-22T04:10:33.931611+0000","last_became_peered":"2024-03-22T04:10:33.931611+0000","last_unstale":"2024-03-22T04:10:41.536542+0000","last_undegraded":"2024-03-22T04:10:41.536542+0000","last_fullsized":"2024-03-22T04:10:41.536542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964623+0000","last_change":"2024-03-22T04:10:38.988757+0000","last_active":"2024-03-22T04:10:39.964623+0000","last_peered":"2024-03-22T04:10:39.964623+0000","last_clean":"2024-03-22T04:10:39.964623+0000","last_became_active":"2024-03-22T04:10:38.987185+0000","last_became_peered":"2024-03-22T04:10:38.987185+0000","last_unstale":"2024-03-22T04:10:39.964623+0000","last_undegraded":"2024-03-22T04:10:39.964623+0000","last_fullsized":"2024-03-22T04:10:39.964623+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:30:15.799282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535188+0000","last_change":"2024-03-22T04:10:34.948529+0000","last_active":"2024-03-22T04:10:41.535188+0000","last_peered":"2024-03-22T04:10:41.535188+0000","last_clean":"2024-03-22T04:10:41.535188+0000","last_became_active":"2024-03-22T04:10:34.948122+0000","last_became_peered":"2024-03-22T04:10:34.948122+0000","last_unstale":"2024-03-22T04:10:41.535188+0000","last_undegraded":"2024-03-22T04:10:41.535188+0000","last_fullsized":"2024-03-22T04:10:41.535188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:55.025923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170450+0000","last_change":"2024-03-22T04:10:33.938126+0000","last_active":"2024-03-22T04:10:41.170450+0000","last_peered":"2024-03-22T04:10:41.170450+0000","last_clean":"2024-03-22T04:10:41.170450+0000","last_became_active":"2024-03-22T04:10:33.937511+0000","last_became_peered":"2024-03-22T04:10:33.937511+0000","last_unstale":"2024-03-22T04:10:41.170450+0000","last_undegraded":"2024-03-22T04:10:41.170450+0000","last_fullsized":"2024-03-22T04:10:41.170450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169980+0000","last_change":"2024-03-22T04:10:38.988377+0000","last_active":"2024-03-22T04:10:41.169980+0000","last_peered":"2024-03-22T04:10:41.169980+0000","last_clean":"2024-03-22T04:10:41.169980+0000","last_became_active":"2024-03-22T04:10:38.984506+0000","last_became_peered":"2024-03-22T04:10:38.984506+0000","last_unstale":"2024-03-22T04:10:41.169980+0000","last_undegraded":"2024-03-22T04:10:41.169980+0000","last_fullsized":"2024-03-22T04:10:41.169980+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:17:51.066337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535990+0000","last_change":"2024-03-22T04:10:34.940162+0000","last_active":"2024-03-22T04:10:41.535990+0000","last_peered":"2024-03-22T04:10:41.535990+0000","last_clean":"2024-03-22T04:10:41.535990+0000","last_became_active":"2024-03-22T04:10:34.937040+0000","last_became_peered":"2024-03-22T04:10:34.937040+0000","last_unstale":"2024-03-22T04:10:41.535990+0000","last_undegraded":"2024-03-22T04:10:41.535990+0000","last_fullsized":"2024-03-22T04:10:41.535990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.104177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967999+0000","last_change":"2024-03-22T04:10:33.932000+0000","last_active":"2024-03-22T04:10:39.967999+0000","last_peered":"2024-03-22T04:10:39.967999+0000","last_clean":"2024-03-22T04:10:39.967999+0000","last_became_active":"2024-03-22T04:10:33.930159+0000","last_became_peered":"2024-03-22T04:10:33.930159+0000","last_unstale":"2024-03-22T04:10:39.967999+0000","last_undegraded":"2024-03-22T04:10:39.967999+0000","last_fullsized":"2024-03-22T04:10:39.967999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:11:21.741394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921794+0000","last_change":"2024-03-22T04:10:38.979495+0000","last_active":"2024-03-22T04:10:40.921794+0000","last_peered":"2024-03-22T04:10:40.921794+0000","last_clean":"2024-03-22T04:10:40.921794+0000","last_became_active":"2024-03-22T04:10:38.979058+0000","last_became_peered":"2024-03-22T04:10:38.979058+0000","last_unstale":"2024-03-22T04:10:40.921794+0000","last_undegraded":"2024-03-22T04:10:40.921794+0000","last_fullsized":"2024-03-22T04:10:40.921794+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:25.405176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965133+0000","last_change":"2024-03-22T04:10:34.936969+0000","last_active":"2024-03-22T04:10:39.965133+0000","last_peered":"2024-03-22T04:10:39.965133+0000","last_clean":"2024-03-22T04:10:39.965133+0000","last_became_active":"2024-03-22T04:10:34.936270+0000","last_became_peered":"2024-03-22T04:10:34.936270+0000","last_unstale":"2024-03-22T04:10:39.965133+0000","last_undegraded":"2024-03-22T04:10:39.965133+0000","last_fullsized":"2024-03-22T04:10:39.965133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:44.874307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536081+0000","last_change":"2024-03-22T04:10:33.929682+0000","last_active":"2024-03-22T04:10:41.536081+0000","last_peered":"2024-03-22T04:10:41.536081+0000","last_clean":"2024-03-22T04:10:41.536081+0000","last_became_active":"2024-03-22T04:10:33.929309+0000","last_became_peered":"2024-03-22T04:10:33.929309+0000","last_unstale":"2024-03-22T04:10:41.536081+0000","last_undegraded":"2024-03-22T04:10:41.536081+0000","last_fullsized":"2024-03-22T04:10:41.536081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:05.641869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535389+0000","last_change":"2024-03-22T04:10:38.979596+0000","last_active":"2024-03-22T04:10:41.535389+0000","last_peered":"2024-03-22T04:10:41.535389+0000","last_clean":"2024-03-22T04:10:41.535389+0000","last_became_active":"2024-03-22T04:10:38.979166+0000","last_became_peered":"2024-03-22T04:10:38.979166+0000","last_unstale":"2024-03-22T04:10:41.535389+0000","last_undegraded":"2024-03-22T04:10:41.535389+0000","last_fullsized":"2024-03-22T04:10:41.535389+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:08.758533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921851+0000","last_change":"2024-03-22T04:10:34.926802+0000","last_active":"2024-03-22T04:10:40.921851+0000","last_peered":"2024-03-22T04:10:40.921851+0000","last_clean":"2024-03-22T04:10:40.921851+0000","last_became_active":"2024-03-22T04:10:34.926514+0000","last_became_peered":"2024-03-22T04:10:34.926514+0000","last_unstale":"2024-03-22T04:10:40.921851+0000","last_undegraded":"2024-03-22T04:10:40.921851+0000","last_fullsized":"2024-03-22T04:10:40.921851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:02:47.197474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966395+0000","last_change":"2024-03-22T04:10:33.932545+0000","last_active":"2024-03-22T04:10:39.966395+0000","last_peered":"2024-03-22T04:10:39.966395+0000","last_clean":"2024-03-22T04:10:39.966395+0000","last_became_active":"2024-03-22T04:10:33.929875+0000","last_became_peered":"2024-03-22T04:10:33.929875+0000","last_unstale":"2024-03-22T04:10:39.966395+0000","last_undegraded":"2024-03-22T04:10:39.966395+0000","last_fullsized":"2024-03-22T04:10:39.966395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:59:03.501920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170524+0000","last_change":"2024-03-22T04:10:38.985437+0000","last_active":"2024-03-22T04:10:41.170524+0000","last_peered":"2024-03-22T04:10:41.170524+0000","last_clean":"2024-03-22T04:10:41.170524+0000","last_became_active":"2024-03-22T04:10:38.982076+0000","last_became_peered":"2024-03-22T04:10:38.982076+0000","last_unstale":"2024-03-22T04:10:41.170524+0000","last_undegraded":"2024-03-22T04:10:41.170524+0000","last_fullsized":"2024-03-22T04:10:41.170524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:21.687034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535147+0000","last_change":"2024-03-22T04:10:34.928089+0000","last_active":"2024-03-22T04:10:41.535147+0000","last_peered":"2024-03-22T04:10:41.535147+0000","last_clean":"2024-03-22T04:10:41.535147+0000","last_became_active":"2024-03-22T04:10:34.927828+0000","last_became_peered":"2024-03-22T04:10:34.927828+0000","last_unstale":"2024-03-22T04:10:41.535147+0000","last_undegraded":"2024-03-22T04:10:41.535147+0000","last_fullsized":"2024-03-22T04:10:41.535147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:31.311715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534615+0000","last_change":"2024-03-22T04:10:33.943460+0000","last_active":"2024-03-22T04:10:41.534615+0000","last_peered":"2024-03-22T04:10:41.534615+0000","last_clean":"2024-03-22T04:10:41.534615+0000","last_became_active":"2024-03-22T04:10:33.943039+0000","last_became_peered":"2024-03-22T04:10:33.943039+0000","last_unstale":"2024-03-22T04:10:41.534615+0000","last_undegraded":"2024-03-22T04:10:41.534615+0000","last_fullsized":"2024-03-22T04:10:41.534615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:40:51.297536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170801+0000","last_change":"2024-03-22T04:10:38.981354+0000","last_active":"2024-03-22T04:10:41.170801+0000","last_peered":"2024-03-22T04:10:41.170801+0000","last_clean":"2024-03-22T04:10:41.170801+0000","last_became_active":"2024-03-22T04:10:38.979192+0000","last_became_peered":"2024-03-22T04:10:38.979192+0000","last_unstale":"2024-03-22T04:10:41.170801+0000","last_undegraded":"2024-03-22T04:10:41.170801+0000","last_fullsized":"2024-03-22T04:10:41.170801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:05:23.818860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922177+0000","last_change":"2024-03-22T04:10:34.935166+0000","last_active":"2024-03-22T04:10:40.922177+0000","last_peered":"2024-03-22T04:10:40.922177+0000","last_clean":"2024-03-22T04:10:40.922177+0000","last_became_active":"2024-03-22T04:10:34.934859+0000","last_became_peered":"2024-03-22T04:10:34.934859+0000","last_unstale":"2024-03-22T04:10:40.922177+0000","last_undegraded":"2024-03-22T04:10:40.922177+0000","last_fullsized":"2024-03-22T04:10:40.922177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:34:47.128097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922197+0000","last_change":"2024-03-22T04:10:33.939004+0000","last_active":"2024-03-22T04:10:40.922197+0000","last_peered":"2024-03-22T04:10:40.922197+0000","last_clean":"2024-03-22T04:10:40.922197+0000","last_became_active":"2024-03-22T04:10:33.938036+0000","last_became_peered":"2024-03-22T04:10:33.938036+0000","last_unstale":"2024-03-22T04:10:40.922197+0000","last_undegraded":"2024-03-22T04:10:40.922197+0000","last_fullsized":"2024-03-22T04:10:40.922197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:47.209113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536345+0000","last_change":"2024-03-22T04:10:38.985663+0000","last_active":"2024-03-22T04:10:41.536345+0000","last_peered":"2024-03-22T04:10:41.536345+0000","last_clean":"2024-03-22T04:10:41.536345+0000","last_became_active":"2024-03-22T04:10:38.985235+0000","last_became_peered":"2024-03-22T04:10:38.985235+0000","last_unstale":"2024-03-22T04:10:41.536345+0000","last_undegraded":"2024-03-22T04:10:41.536345+0000","last_fullsized":"2024-03-22T04:10:41.536345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:45.922825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536323+0000","last_change":"2024-03-22T04:10:34.950266+0000","last_active":"2024-03-22T04:10:41.536323+0000","last_peered":"2024-03-22T04:10:41.536323+0000","last_clean":"2024-03-22T04:10:41.536323+0000","last_became_active":"2024-03-22T04:10:34.949946+0000","last_became_peered":"2024-03-22T04:10:34.949946+0000","last_unstale":"2024-03-22T04:10:41.536323+0000","last_undegraded":"2024-03-22T04:10:41.536323+0000","last_fullsized":"2024-03-22T04:10:41.536323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:57:04.713004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967305+0000","last_change":"2024-03-22T04:10:33.935184+0000","last_active":"2024-03-22T04:10:39.967305+0000","last_peered":"2024-03-22T04:10:39.967305+0000","last_clean":"2024-03-22T04:10:39.967305+0000","last_became_active":"2024-03-22T04:10:33.934904+0000","last_became_peered":"2024-03-22T04:10:33.934904+0000","last_unstale":"2024-03-22T04:10:39.967305+0000","last_undegraded":"2024-03-22T04:10:39.967305+0000","last_fullsized":"2024-03-22T04:10:39.967305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922445+0000","last_change":"2024-03-22T04:10:38.971479+0000","last_active":"2024-03-22T04:10:40.922445+0000","last_peered":"2024-03-22T04:10:40.922445+0000","last_clean":"2024-03-22T04:10:40.922445+0000","last_became_active":"2024-03-22T04:10:38.969629+0000","last_became_peered":"2024-03-22T04:10:38.969629+0000","last_unstale":"2024-03-22T04:10:40.922445+0000","last_undegraded":"2024-03-22T04:10:40.922445+0000","last_fullsized":"2024-03-22T04:10:40.922445+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:28:08.179423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535818+0000","last_change":"2024-03-22T04:10:34.932181+0000","last_active":"2024-03-22T04:10:41.535818+0000","last_peered":"2024-03-22T04:10:41.535818+0000","last_clean":"2024-03-22T04:10:41.535818+0000","last_became_active":"2024-03-22T04:10:34.931936+0000","last_became_peered":"2024-03-22T04:10:34.931936+0000","last_unstale":"2024-03-22T04:10:41.535818+0000","last_undegraded":"2024-03-22T04:10:41.535818+0000","last_fullsized":"2024-03-22T04:10:41.535818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:10:26.319863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:50.358530+0000","last_change":"2024-03-22T04:10:33.934598+0000","last_active":"2024-03-22T04:10:50.358530+0000","last_peered":"2024-03-22T04:10:50.358530+0000","last_clean":"2024-03-22T04:10:50.358530+0000","last_became_active":"2024-03-22T04:10:33.934129+0000","last_became_peered":"2024-03-22T04:10:33.934129+0000","last_unstale":"2024-03-22T04:10:50.358530+0000","last_undegraded":"2024-03-22T04:10:50.358530+0000","last_fullsized":"2024-03-22T04:10:50.358530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:25:11.544206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966493+0000","last_change":"2024-03-22T04:10:38.980975+0000","last_active":"2024-03-22T04:10:39.966493+0000","last_peered":"2024-03-22T04:10:39.966493+0000","last_clean":"2024-03-22T04:10:39.966493+0000","last_became_active":"2024-03-22T04:10:38.980594+0000","last_became_peered":"2024-03-22T04:10:38.980594+0000","last_unstale":"2024-03-22T04:10:39.966493+0000","last_undegraded":"2024-03-22T04:10:39.966493+0000","last_fullsized":"2024-03-22T04:10:39.966493+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:10:22.917144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922350+0000","last_change":"2024-03-22T04:10:34.939331+0000","last_active":"2024-03-22T04:10:40.922350+0000","last_peered":"2024-03-22T04:10:40.922350+0000","last_clean":"2024-03-22T04:10:40.922350+0000","last_became_active":"2024-03-22T04:10:34.938998+0000","last_became_peered":"2024-03-22T04:10:34.938998+0000","last_unstale":"2024-03-22T04:10:40.922350+0000","last_undegraded":"2024-03-22T04:10:40.922350+0000","last_fullsized":"2024-03-22T04:10:40.922350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.104177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967555+0000","last_change":"2024-03-22T04:10:33.938816+0000","last_active":"2024-03-22T04:10:39.967555+0000","last_peered":"2024-03-22T04:10:39.967555+0000","last_clean":"2024-03-22T04:10:39.967555+0000","last_became_active":"2024-03-22T04:10:33.937603+0000","last_became_peered":"2024-03-22T04:10:33.937603+0000","last_unstale":"2024-03-22T04:10:39.967555+0000","last_undegraded":"2024-03-22T04:10:39.967555+0000","last_fullsized":"2024-03-22T04:10:39.967555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:02.803517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964868+0000","last_change":"2024-03-22T04:10:38.987284+0000","last_active":"2024-03-22T04:10:39.964868+0000","last_peered":"2024-03-22T04:10:39.964868+0000","last_clean":"2024-03-22T04:10:39.964868+0000","last_became_active":"2024-03-22T04:10:38.986819+0000","last_became_peered":"2024-03-22T04:10:38.986819+0000","last_unstale":"2024-03-22T04:10:39.964868+0000","last_undegraded":"2024-03-22T04:10:39.964868+0000","last_fullsized":"2024-03-22T04:10:39.964868+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:20.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170831+0000","last_change":"2024-03-22T04:10:34.933050+0000","last_active":"2024-03-22T04:10:41.170831+0000","last_peered":"2024-03-22T04:10:41.170831+0000","last_clean":"2024-03-22T04:10:41.170831+0000","last_became_active":"2024-03-22T04:10:34.932756+0000","last_became_peered":"2024-03-22T04:10:34.932756+0000","last_unstale":"2024-03-22T04:10:41.170831+0000","last_undegraded":"2024-03-22T04:10:41.170831+0000","last_fullsized":"2024-03-22T04:10:41.170831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:41.877296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536138+0000","last_change":"2024-03-22T04:10:33.936205+0000","last_active":"2024-03-22T04:10:41.536138+0000","last_peered":"2024-03-22T04:10:41.536138+0000","last_clean":"2024-03-22T04:10:41.536138+0000","last_became_active":"2024-03-22T04:10:33.935904+0000","last_became_peered":"2024-03-22T04:10:33.935904+0000","last_unstale":"2024-03-22T04:10:41.536138+0000","last_undegraded":"2024-03-22T04:10:41.536138+0000","last_fullsized":"2024-03-22T04:10:41.536138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:30.304547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966245+0000","last_change":"2024-03-22T04:10:38.991949+0000","last_active":"2024-03-22T04:10:39.966245+0000","last_peered":"2024-03-22T04:10:39.966245+0000","last_clean":"2024-03-22T04:10:39.966245+0000","last_became_active":"2024-03-22T04:10:38.991459+0000","last_became_peered":"2024-03-22T04:10:38.991459+0000","last_unstale":"2024-03-22T04:10:39.966245+0000","last_undegraded":"2024-03-22T04:10:39.966245+0000","last_fullsized":"2024-03-22T04:10:39.966245+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:44.301347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.965035+0000","last_change":"2024-03-22T04:10:34.947409+0000","last_active":"2024-03-22T04:10:39.965035+0000","last_peered":"2024-03-22T04:10:39.965035+0000","last_clean":"2024-03-22T04:10:39.965035+0000","last_became_active":"2024-03-22T04:10:34.947102+0000","last_became_peered":"2024-03-22T04:10:34.947102+0000","last_unstale":"2024-03-22T04:10:39.965035+0000","last_undegraded":"2024-03-22T04:10:39.965035+0000","last_fullsized":"2024-03-22T04:10:39.965035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:47:12.376047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535663+0000","last_change":"2024-03-22T04:10:33.922034+0000","last_active":"2024-03-22T04:10:41.535663+0000","last_peered":"2024-03-22T04:10:41.535663+0000","last_clean":"2024-03-22T04:10:41.535663+0000","last_became_active":"2024-03-22T04:10:33.921713+0000","last_became_peered":"2024-03-22T04:10:33.921713+0000","last_unstale":"2024-03-22T04:10:41.535663+0000","last_undegraded":"2024-03-22T04:10:41.535663+0000","last_fullsized":"2024-03-22T04:10:41.535663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966606+0000","last_change":"2024-03-22T04:10:38.983919+0000","last_active":"2024-03-22T04:10:39.966606+0000","last_peered":"2024-03-22T04:10:39.966606+0000","last_clean":"2024-03-22T04:10:39.966606+0000","last_became_active":"2024-03-22T04:10:38.983556+0000","last_became_peered":"2024-03-22T04:10:38.983556+0000","last_unstale":"2024-03-22T04:10:39.966606+0000","last_undegraded":"2024-03-22T04:10:39.966606+0000","last_fullsized":"2024-03-22T04:10:39.966606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:12:09.818595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967886+0000","last_change":"2024-03-22T04:10:38.985686+0000","last_active":"2024-03-22T04:10:39.967886+0000","last_peered":"2024-03-22T04:10:39.967886+0000","last_clean":"2024-03-22T04:10:39.967886+0000","last_became_active":"2024-03-22T04:10:38.985411+0000","last_became_peered":"2024-03-22T04:10:38.985411+0000","last_unstale":"2024-03-22T04:10:39.967886+0000","last_undegraded":"2024-03-22T04:10:39.967886+0000","last_fullsized":"2024-03-22T04:10:39.967886+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:36.068872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966752+0000","last_change":"2024-03-22T04:10:34.926900+0000","last_active":"2024-03-22T04:10:39.966752+0000","last_peered":"2024-03-22T04:10:39.966752+0000","last_clean":"2024-03-22T04:10:39.966752+0000","last_became_active":"2024-03-22T04:10:34.926599+0000","last_became_peered":"2024-03-22T04:10:34.926599+0000","last_unstale":"2024-03-22T04:10:39.966752+0000","last_undegraded":"2024-03-22T04:10:39.966752+0000","last_fullsized":"2024-03-22T04:10:39.966752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:21.635048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922002+0000","last_change":"2024-03-22T04:10:33.932457+0000","last_active":"2024-03-22T04:10:40.922002+0000","last_peered":"2024-03-22T04:10:40.922002+0000","last_clean":"2024-03-22T04:10:40.922002+0000","last_became_active":"2024-03-22T04:10:33.932048+0000","last_became_peered":"2024-03-22T04:10:33.932048+0000","last_unstale":"2024-03-22T04:10:40.922002+0000","last_undegraded":"2024-03-22T04:10:40.922002+0000","last_fullsized":"2024-03-22T04:10:40.922002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:22:30.304547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922792+0000","last_change":"2024-03-22T04:10:38.985629+0000","last_active":"2024-03-22T04:10:40.922792+0000","last_peered":"2024-03-22T04:10:40.922792+0000","last_clean":"2024-03-22T04:10:40.922792+0000","last_became_active":"2024-03-22T04:10:38.983094+0000","last_became_peered":"2024-03-22T04:10:38.983094+0000","last_unstale":"2024-03-22T04:10:40.922792+0000","last_undegraded":"2024-03-22T04:10:40.922792+0000","last_fullsized":"2024-03-22T04:10:40.922792+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:21:46.389313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169810+0000","last_change":"2024-03-22T04:10:34.925982+0000","last_active":"2024-03-22T04:10:41.169810+0000","last_peered":"2024-03-22T04:10:41.169810+0000","last_clean":"2024-03-22T04:10:41.169810+0000","last_became_active":"2024-03-22T04:10:34.925676+0000","last_became_peered":"2024-03-22T04:10:34.925676+0000","last_unstale":"2024-03-22T04:10:41.169810+0000","last_undegraded":"2024-03-22T04:10:41.169810+0000","last_fullsized":"2024-03-22T04:10:41.169810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:08.063872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536054+0000","last_change":"2024-03-22T04:10:33.933448+0000","last_active":"2024-03-22T04:10:41.536054+0000","last_peered":"2024-03-22T04:10:41.536054+0000","last_clean":"2024-03-22T04:10:41.536054+0000","last_became_active":"2024-03-22T04:10:33.933025+0000","last_became_peered":"2024-03-22T04:10:33.933025+0000","last_unstale":"2024-03-22T04:10:41.536054+0000","last_undegraded":"2024-03-22T04:10:41.536054+0000","last_fullsized":"2024-03-22T04:10:41.536054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:14.967579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922138+0000","last_change":"2024-03-22T04:10:33.935607+0000","last_active":"2024-03-22T04:10:40.922138+0000","last_peered":"2024-03-22T04:10:40.922138+0000","last_clean":"2024-03-22T04:10:40.922138+0000","last_became_active":"2024-03-22T04:10:33.932086+0000","last_became_peered":"2024-03-22T04:10:33.932086+0000","last_unstale":"2024-03-22T04:10:40.922138+0000","last_undegraded":"2024-03-22T04:10:40.922138+0000","last_fullsized":"2024-03-22T04:10:40.922138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:49.224368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967401+0000","last_change":"2024-03-22T04:10:34.934494+0000","last_active":"2024-03-22T04:10:39.967401+0000","last_peered":"2024-03-22T04:10:39.967401+0000","last_clean":"2024-03-22T04:10:39.967401+0000","last_became_active":"2024-03-22T04:10:34.934210+0000","last_became_peered":"2024-03-22T04:10:34.934210+0000","last_unstale":"2024-03-22T04:10:39.967401+0000","last_undegraded":"2024-03-22T04:10:39.967401+0000","last_fullsized":"2024-03-22T04:10:39.967401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:48:09.157711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967687+0000","last_change":"2024-03-22T04:10:38.988089+0000","last_active":"2024-03-22T04:10:39.967687+0000","last_peered":"2024-03-22T04:10:39.967687+0000","last_clean":"2024-03-22T04:10:39.967687+0000","last_became_active":"2024-03-22T04:10:38.987643+0000","last_became_peered":"2024-03-22T04:10:38.987643+0000","last_unstale":"2024-03-22T04:10:39.967687+0000","last_undegraded":"2024-03-22T04:10:39.967687+0000","last_fullsized":"2024-03-22T04:10:39.967687+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:35.902267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170109+0000","last_change":"2024-03-22T04:10:33.933057+0000","last_active":"2024-03-22T04:10:41.170109+0000","last_peered":"2024-03-22T04:10:41.170109+0000","last_clean":"2024-03-22T04:10:41.170109+0000","last_became_active":"2024-03-22T04:10:33.932776+0000","last_became_peered":"2024-03-22T04:10:33.932776+0000","last_unstale":"2024-03-22T04:10:41.170109+0000","last_undegraded":"2024-03-22T04:10:41.170109+0000","last_fullsized":"2024-03-22T04:10:41.170109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:32.889345+0000","last_clean_scrub_stamp":"2024-03-22T04:10:32.889345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:43.665213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535263+0000","last_change":"2024-03-22T04:10:34.935154+0000","last_active":"2024-03-22T04:10:41.535263+0000","last_peered":"2024-03-22T04:10:41.535263+0000","last_clean":"2024-03-22T04:10:41.535263+0000","last_became_active":"2024-03-22T04:10:34.934888+0000","last_became_peered":"2024-03-22T04:10:34.934888+0000","last_unstale":"2024-03-22T04:10:41.535263+0000","last_undegraded":"2024-03-22T04:10:41.535263+0000","last_fullsized":"2024-03-22T04:10:41.535263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:01.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967502+0000","last_change":"2024-03-22T04:10:38.988529+0000","last_active":"2024-03-22T04:10:39.967502+0000","last_peered":"2024-03-22T04:10:39.967502+0000","last_clean":"2024-03-22T04:10:39.967502+0000","last_became_active":"2024-03-22T04:10:38.987847+0000","last_became_peered":"2024-03-22T04:10:38.987847+0000","last_unstale":"2024-03-22T04:10:39.967502+0000","last_undegraded":"2024-03-22T04:10:39.967502+0000","last_fullsized":"2024-03-22T04:10:39.967502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:37.942043+0000","last_clean_scrub_stamp":"2024-03-22T04:10:37.942043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:12:05.623030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964372+0000","last_change":"2024-03-22T04:10:34.930193+0000","last_active":"2024-03-22T04:10:39.964372+0000","last_peered":"2024-03-22T04:10:39.964372+0000","last_clean":"2024-03-22T04:10:39.964372+0000","last_became_active":"2024-03-22T04:10:34.929910+0000","last_became_peered":"2024-03-22T04:10:34.929910+0000","last_unstale":"2024-03-22T04:10:39.964372+0000","last_undegraded":"2024-03-22T04:10:39.964372+0000","last_fullsized":"2024-03-22T04:10:39.964372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.966601+0000","last_change":"2024-03-22T04:10:34.951954+0000","last_active":"2024-03-22T04:10:39.966601+0000","last_peered":"2024-03-22T04:10:39.966601+0000","last_clean":"2024-03-22T04:10:39.966601+0000","last_became_active":"2024-03-22T04:10:34.951651+0000","last_became_peered":"2024-03-22T04:10:34.951651+0000","last_unstale":"2024-03-22T04:10:39.966601+0000","last_undegraded":"2024-03-22T04:10:39.966601+0000","last_fullsized":"2024-03-22T04:10:39.966601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:06.649038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922008+0000","last_change":"2024-03-22T04:10:34.927015+0000","last_active":"2024-03-22T04:10:40.922008+0000","last_peered":"2024-03-22T04:10:40.922008+0000","last_clean":"2024-03-22T04:10:40.922008+0000","last_became_active":"2024-03-22T04:10:34.926098+0000","last_became_peered":"2024-03-22T04:10:34.926098+0000","last_unstale":"2024-03-22T04:10:40.922008+0000","last_undegraded":"2024-03-22T04:10:40.922008+0000","last_fullsized":"2024-03-22T04:10:40.922008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:05:19.773331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170027+0000","last_change":"2024-03-22T04:10:34.950558+0000","last_active":"2024-03-22T04:10:41.170027+0000","last_peered":"2024-03-22T04:10:41.170027+0000","last_clean":"2024-03-22T04:10:41.170027+0000","last_became_active":"2024-03-22T04:10:34.950242+0000","last_became_peered":"2024-03-22T04:10:34.950242+0000","last_unstale":"2024-03-22T04:10:41.170027+0000","last_undegraded":"2024-03-22T04:10:41.170027+0000","last_fullsized":"2024-03-22T04:10:41.170027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:11.105591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536489+0000","last_change":"2024-03-22T04:10:34.928518+0000","last_active":"2024-03-22T04:10:41.536489+0000","last_peered":"2024-03-22T04:10:41.536489+0000","last_clean":"2024-03-22T04:10:41.536489+0000","last_became_active":"2024-03-22T04:10:34.928228+0000","last_became_peered":"2024-03-22T04:10:34.928228+0000","last_unstale":"2024-03-22T04:10:41.536489+0000","last_undegraded":"2024-03-22T04:10:41.536489+0000","last_fullsized":"2024-03-22T04:10:41.536489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:41.449121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536201+0000","last_change":"2024-03-22T04:10:34.951914+0000","last_active":"2024-03-22T04:10:41.536201+0000","last_peered":"2024-03-22T04:10:41.536201+0000","last_clean":"2024-03-22T04:10:41.536201+0000","last_became_active":"2024-03-22T04:10:34.951608+0000","last_became_peered":"2024-03-22T04:10:34.951608+0000","last_unstale":"2024-03-22T04:10:41.536201+0000","last_undegraded":"2024-03-22T04:10:41.536201+0000","last_fullsized":"2024-03-22T04:10:41.536201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:18.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964949+0000","last_change":"2024-03-22T04:10:34.936728+0000","last_active":"2024-03-22T04:10:39.964949+0000","last_peered":"2024-03-22T04:10:39.964949+0000","last_clean":"2024-03-22T04:10:39.964949+0000","last_became_active":"2024-03-22T04:10:34.935969+0000","last_became_peered":"2024-03-22T04:10:34.935969+0000","last_unstale":"2024-03-22T04:10:39.964949+0000","last_undegraded":"2024-03-22T04:10:39.964949+0000","last_fullsized":"2024-03-22T04:10:39.964949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:48.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170702+0000","last_change":"2024-03-22T04:10:34.943850+0000","last_active":"2024-03-22T04:10:41.170702+0000","last_peered":"2024-03-22T04:10:41.170702+0000","last_clean":"2024-03-22T04:10:41.170702+0000","last_became_active":"2024-03-22T04:10:34.941930+0000","last_became_peered":"2024-03-22T04:10:34.941930+0000","last_unstale":"2024-03-22T04:10:41.170702+0000","last_undegraded":"2024-03-22T04:10:41.170702+0000","last_fullsized":"2024-03-22T04:10:41.170702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:55.025923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921894+0000","last_change":"2024-03-22T04:10:34.946337+0000","last_active":"2024-03-22T04:10:40.921894+0000","last_peered":"2024-03-22T04:10:40.921894+0000","last_clean":"2024-03-22T04:10:40.921894+0000","last_became_active":"2024-03-22T04:10:34.946083+0000","last_became_peered":"2024-03-22T04:10:34.946083+0000","last_unstale":"2024-03-22T04:10:40.921894+0000","last_undegraded":"2024-03-22T04:10:40.921894+0000","last_fullsized":"2024-03-22T04:10:40.921894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:15:33.976508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170005+0000","last_change":"2024-03-22T04:10:34.939061+0000","last_active":"2024-03-22T04:10:41.170005+0000","last_peered":"2024-03-22T04:10:41.170005+0000","last_clean":"2024-03-22T04:10:41.170005+0000","last_became_active":"2024-03-22T04:10:34.938566+0000","last_became_peered":"2024-03-22T04:10:34.938566+0000","last_unstale":"2024-03-22T04:10:41.170005+0000","last_undegraded":"2024-03-22T04:10:41.170005+0000","last_fullsized":"2024-03-22T04:10:41.170005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:44:57.681730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535168+0000","last_change":"2024-03-22T04:10:34.933002+0000","last_active":"2024-03-22T04:10:41.535168+0000","last_peered":"2024-03-22T04:10:41.535168+0000","last_clean":"2024-03-22T04:10:41.535168+0000","last_became_active":"2024-03-22T04:10:34.932710+0000","last_became_peered":"2024-03-22T04:10:34.932710+0000","last_unstale":"2024-03-22T04:10:41.535168+0000","last_undegraded":"2024-03-22T04:10:41.535168+0000","last_fullsized":"2024-03-22T04:10:41.535168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967328+0000","last_change":"2024-03-22T04:10:34.926419+0000","last_active":"2024-03-22T04:10:39.967328+0000","last_peered":"2024-03-22T04:10:39.967328+0000","last_clean":"2024-03-22T04:10:39.967328+0000","last_became_active":"2024-03-22T04:10:34.924952+0000","last_became_peered":"2024-03-22T04:10:34.924952+0000","last_unstale":"2024-03-22T04:10:39.967328+0000","last_undegraded":"2024-03-22T04:10:39.967328+0000","last_fullsized":"2024-03-22T04:10:39.967328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:30:41.171941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535087+0000","last_change":"2024-03-22T04:10:34.939886+0000","last_active":"2024-03-22T04:10:41.535087+0000","last_peered":"2024-03-22T04:10:41.535087+0000","last_clean":"2024-03-22T04:10:41.535087+0000","last_became_active":"2024-03-22T04:10:34.939633+0000","last_became_peered":"2024-03-22T04:10:34.939633+0000","last_unstale":"2024-03-22T04:10:41.535087+0000","last_undegraded":"2024-03-22T04:10:41.535087+0000","last_fullsized":"2024-03-22T04:10:41.535087+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:45:41.877296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922616+0000","last_change":"2024-03-22T04:10:34.930917+0000","last_active":"2024-03-22T04:10:40.922616+0000","last_peered":"2024-03-22T04:10:40.922616+0000","last_clean":"2024-03-22T04:10:40.922616+0000","last_became_active":"2024-03-22T04:10:34.930658+0000","last_became_peered":"2024-03-22T04:10:34.930658+0000","last_unstale":"2024-03-22T04:10:40.922616+0000","last_undegraded":"2024-03-22T04:10:40.922616+0000","last_fullsized":"2024-03-22T04:10:40.922616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:40:34.158803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.967631+0000","last_change":"2024-03-22T04:10:34.953215+0000","last_active":"2024-03-22T04:10:39.967631+0000","last_peered":"2024-03-22T04:10:39.967631+0000","last_clean":"2024-03-22T04:10:39.967631+0000","last_became_active":"2024-03-22T04:10:34.952953+0000","last_became_peered":"2024-03-22T04:10:34.952953+0000","last_unstale":"2024-03-22T04:10:39.967631+0000","last_undegraded":"2024-03-22T04:10:39.967631+0000","last_fullsized":"2024-03-22T04:10:39.967631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.922902+0000","last_change":"2024-03-22T04:10:34.937971+0000","last_active":"2024-03-22T04:10:40.922902+0000","last_peered":"2024-03-22T04:10:40.922902+0000","last_clean":"2024-03-22T04:10:40.922902+0000","last_became_active":"2024-03-22T04:10:34.937744+0000","last_became_peered":"2024-03-22T04:10:34.937744+0000","last_unstale":"2024-03-22T04:10:40.922902+0000","last_undegraded":"2024-03-22T04:10:40.922902+0000","last_fullsized":"2024-03-22T04:10:40.922902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:17.641505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:40.921877+0000","last_change":"2024-03-22T04:10:34.929076+0000","last_active":"2024-03-22T04:10:40.921877+0000","last_peered":"2024-03-22T04:10:40.921877+0000","last_clean":"2024-03-22T04:10:40.921877+0000","last_became_active":"2024-03-22T04:10:34.928742+0000","last_became_peered":"2024-03-22T04:10:34.928742+0000","last_unstale":"2024-03-22T04:10:40.921877+0000","last_undegraded":"2024-03-22T04:10:40.921877+0000","last_fullsized":"2024-03-22T04:10:40.921877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:04.974724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169693+0000","last_change":"2024-03-22T04:10:34.935025+0000","last_active":"2024-03-22T04:10:41.169693+0000","last_peered":"2024-03-22T04:10:41.169693+0000","last_clean":"2024-03-22T04:10:41.169693+0000","last_became_active":"2024-03-22T04:10:34.934504+0000","last_became_peered":"2024-03-22T04:10:34.934504+0000","last_unstale":"2024-03-22T04:10:41.169693+0000","last_undegraded":"2024-03-22T04:10:41.169693+0000","last_fullsized":"2024-03-22T04:10:41.169693+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:31:18.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536006+0000","last_change":"2024-03-22T04:10:34.952611+0000","last_active":"2024-03-22T04:10:41.536006+0000","last_peered":"2024-03-22T04:10:41.536006+0000","last_clean":"2024-03-22T04:10:41.536006+0000","last_became_active":"2024-03-22T04:10:34.952289+0000","last_became_peered":"2024-03-22T04:10:34.952289+0000","last_unstale":"2024-03-22T04:10:41.536006+0000","last_undegraded":"2024-03-22T04:10:41.536006+0000","last_fullsized":"2024-03-22T04:10:41.536006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:45:44.874307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536345+0000","last_change":"2024-03-22T04:10:34.938439+0000","last_active":"2024-03-22T04:10:41.536345+0000","last_peered":"2024-03-22T04:10:41.536345+0000","last_clean":"2024-03-22T04:10:41.536345+0000","last_became_active":"2024-03-22T04:10:34.938160+0000","last_became_peered":"2024-03-22T04:10:34.938160+0000","last_unstale":"2024-03-22T04:10:41.536345+0000","last_undegraded":"2024-03-22T04:10:41.536345+0000","last_fullsized":"2024-03-22T04:10:41.536345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:40.255817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535047+0000","last_change":"2024-03-22T04:10:34.936432+0000","last_active":"2024-03-22T04:10:41.535047+0000","last_peered":"2024-03-22T04:10:41.535047+0000","last_clean":"2024-03-22T04:10:41.535047+0000","last_became_active":"2024-03-22T04:10:34.936175+0000","last_became_peered":"2024-03-22T04:10:34.936175+0000","last_unstale":"2024-03-22T04:10:41.535047+0000","last_undegraded":"2024-03-22T04:10:41.535047+0000","last_fullsized":"2024-03-22T04:10:41.535047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:34.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535725+0000","last_change":"2024-03-22T04:10:34.947651+0000","last_active":"2024-03-22T04:10:41.535725+0000","last_peered":"2024-03-22T04:10:41.535725+0000","last_clean":"2024-03-22T04:10:41.535725+0000","last_became_active":"2024-03-22T04:10:34.947262+0000","last_became_peered":"2024-03-22T04:10:34.947262+0000","last_unstale":"2024-03-22T04:10:41.535725+0000","last_undegraded":"2024-03-22T04:10:41.535725+0000","last_fullsized":"2024-03-22T04:10:41.535725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:16.846964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.536037+0000","last_change":"2024-03-22T04:10:34.929968+0000","last_active":"2024-03-22T04:10:41.536037+0000","last_peered":"2024-03-22T04:10:41.536037+0000","last_clean":"2024-03-22T04:10:41.536037+0000","last_became_active":"2024-03-22T04:10:34.929650+0000","last_became_peered":"2024-03-22T04:10:34.929650+0000","last_unstale":"2024-03-22T04:10:41.536037+0000","last_undegraded":"2024-03-22T04:10:41.536037+0000","last_fullsized":"2024-03-22T04:10:41.536037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:04.974724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535701+0000","last_change":"2024-03-22T04:10:34.949078+0000","last_active":"2024-03-22T04:10:41.535701+0000","last_peered":"2024-03-22T04:10:41.535701+0000","last_clean":"2024-03-22T04:10:41.535701+0000","last_became_active":"2024-03-22T04:10:34.948775+0000","last_became_peered":"2024-03-22T04:10:34.948775+0000","last_unstale":"2024-03-22T04:10:41.535701+0000","last_undegraded":"2024-03-22T04:10:41.535701+0000","last_fullsized":"2024-03-22T04:10:41.535701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:48.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.169713+0000","last_change":"2024-03-22T04:10:34.931467+0000","last_active":"2024-03-22T04:10:41.169713+0000","last_peered":"2024-03-22T04:10:41.169713+0000","last_clean":"2024-03-22T04:10:41.169713+0000","last_became_active":"2024-03-22T04:10:34.931119+0000","last_became_peered":"2024-03-22T04:10:34.931119+0000","last_unstale":"2024-03-22T04:10:41.169713+0000","last_undegraded":"2024-03-22T04:10:41.169713+0000","last_fullsized":"2024-03-22T04:10:41.169713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:18:49.597662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534582+0000","last_change":"2024-03-22T04:10:34.925425+0000","last_active":"2024-03-22T04:10:41.534582+0000","last_peered":"2024-03-22T04:10:41.534582+0000","last_clean":"2024-03-22T04:10:41.534582+0000","last_became_active":"2024-03-22T04:10:34.925151+0000","last_became_peered":"2024-03-22T04:10:34.925151+0000","last_unstale":"2024-03-22T04:10:41.534582+0000","last_undegraded":"2024-03-22T04:10:41.534582+0000","last_fullsized":"2024-03-22T04:10:41.534582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:11.105591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964686+0000","last_change":"2024-03-22T04:10:34.926711+0000","last_active":"2024-03-22T04:10:39.964686+0000","last_peered":"2024-03-22T04:10:39.964686+0000","last_clean":"2024-03-22T04:10:39.964686+0000","last_became_active":"2024-03-22T04:10:34.925765+0000","last_became_peered":"2024-03-22T04:10:34.925765+0000","last_unstale":"2024-03-22T04:10:39.964686+0000","last_undegraded":"2024-03-22T04:10:39.964686+0000","last_fullsized":"2024-03-22T04:10:39.964686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:50.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.534972+0000","last_change":"2024-03-22T04:10:34.932009+0000","last_active":"2024-03-22T04:10:41.534972+0000","last_peered":"2024-03-22T04:10:41.534972+0000","last_clean":"2024-03-22T04:10:41.534972+0000","last_became_active":"2024-03-22T04:10:34.931692+0000","last_became_peered":"2024-03-22T04:10:34.931692+0000","last_unstale":"2024-03-22T04:10:41.534972+0000","last_undegraded":"2024-03-22T04:10:41.534972+0000","last_fullsized":"2024-03-22T04:10:41.534972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:29:08.063872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:39.964556+0000","last_change":"2024-03-22T04:10:34.921659+0000","last_active":"2024-03-22T04:10:39.964556+0000","last_peered":"2024-03-22T04:10:39.964556+0000","last_clean":"2024-03-22T04:10:39.964556+0000","last_became_active":"2024-03-22T04:10:34.921420+0000","last_became_peered":"2024-03-22T04:10:34.921420+0000","last_unstale":"2024-03-22T04:10:39.964556+0000","last_undegraded":"2024-03-22T04:10:39.964556+0000","last_fullsized":"2024-03-22T04:10:39.964556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:01.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170351+0000","last_change":"2024-03-22T04:10:34.928037+0000","last_active":"2024-03-22T04:10:41.170351+0000","last_peered":"2024-03-22T04:10:41.170351+0000","last_clean":"2024-03-22T04:10:41.170351+0000","last_became_active":"2024-03-22T04:10:34.927746+0000","last_became_peered":"2024-03-22T04:10:34.927746+0000","last_unstale":"2024-03-22T04:10:41.170351+0000","last_undegraded":"2024-03-22T04:10:41.170351+0000","last_fullsized":"2024-03-22T04:10:41.170351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:29:34.250954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.170416+0000","last_change":"2024-03-22T04:10:34.931962+0000","last_active":"2024-03-22T04:10:41.170416+0000","last_peered":"2024-03-22T04:10:41.170416+0000","last_clean":"2024-03-22T04:10:41.170416+0000","last_became_active":"2024-03-22T04:10:34.931736+0000","last_became_peered":"2024-03-22T04:10:34.931736+0000","last_unstale":"2024-03-22T04:10:41.170416+0000","last_undegraded":"2024-03-22T04:10:41.170416+0000","last_fullsized":"2024-03-22T04:10:41.170416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:48:16.846964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-22T04:10:41.535903+0000","last_change":"2024-03-22T04:10:34.946756+0000","last_active":"2024-03-22T04:10:41.535903+0000","last_peered":"2024-03-22T04:10:41.535903+0000","last_clean":"2024-03-22T04:10:41.535903+0000","last_became_active":"2024-03-22T04:10:34.946501+0000","last_became_peered":"2024-03-22T04:10:34.946501+0000","last_unstale":"2024-03-22T04:10:41.535903+0000","last_undegraded":"2024-03-22T04:10:41.535903+0000","last_fullsized":"2024-03-22T04:10:41.535903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:33.896514+0000","last_clean_scrub_stamp":"2024-03-22T04:10:33.896514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:02:47.197474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:11:03.292 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-22T04:11:03.292 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-22T04:11:03.292 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-22T04:11:03.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-22T04:11:03.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef7c00640 1 Processor -- start 2024-03-22T04:11:03.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef7c00640 1 -- start start 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef7c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ef00cd480 0x7f4ef00cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef7c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4ef00cf240 0x7f4ef015fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef7c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef01602c0 0x7f4ef015c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef7c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4ef00cfa40 con 0x7f4ef00cd480 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef7c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4ef00cfd20 con 0x7f4ef00cf240 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef7c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef00d0000 con 0x7f4ef01602c0 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ef00cd480 0x7f4ef00cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ef00cd480 0x7f4ef00cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56250/0 (socket says 172.21.15.86:56250) 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5200640 1 -- 172.21.15.86:0/487613755 learned_addr learned my addr 172.21.15.86:0/487613755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4eefe00640 1 --2- 172.21.15.86:0/487613755 >> v2:172.21.15.196:3300/0 conn(0x7f4ef00cf240 0x7f4ef015fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5c00640 1 --2- 172.21.15.86:0/487613755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef01602c0 0x7f4ef015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5200640 1 -- 172.21.15.86:0/487613755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef01602c0 msgr2=0x7f4ef015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5200640 1 --2- 172.21.15.86:0/487613755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef01602c0 0x7f4ef015c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5200640 1 -- 172.21.15.86:0/487613755 >> v2:172.21.15.196:3300/0 conn(0x7f4ef00cf240 msgr2=0x7f4ef015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5200640 1 --2- 172.21.15.86:0/487613755 >> v2:172.21.15.196:3300/0 conn(0x7f4ef00cf240 0x7f4ef015fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.460+0000 7f4ef5200640 1 -- 172.21.15.86:0/487613755 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef00d02e0 con 0x7f4ef00cd480 2024-03-22T04:11:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef5200640 1 --2- 172.21.15.86:0/487613755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cd480 0x7f4ef00cd860 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00d13f0 tx=0x7f4ee001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2eef0e55e74b44 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4eef400640 1 -- 172.21.15.86:0/487613755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00b0070 con 0x7f4ef00cd480 2024-03-22T04:11:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4eef400640 1 -- 172.21.15.86:0/487613755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee00ab600 con 0x7f4ef00cd480 2024-03-22T04:11:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4eef400640 1 -- 172.21.15.86:0/487613755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00ab9d0 con 0x7f4ef00cd480 2024-03-22T04:11:03.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 -- 172.21.15.86:0/487613755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cd480 msgr2=0x7f4ef00cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/487613755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cd480 0x7f4ef00cd860 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00d13f0 tx=0x7f4ee001c520 comp rx=0 tx=0).stop 2024-03-22T04:11:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 -- 172.21.15.86:0/487613755 shutdown_connections 2024-03-22T04:11:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/487613755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef01602c0 0x7f4ef015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/487613755 >> v2:172.21.15.196:3300/0 conn(0x7f4ef00cf240 0x7f4ef015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/487613755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cd480 0x7f4ef00cd860 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 -- 172.21.15.86:0/487613755 >> 172.21.15.86:0/487613755 conn(0x7f4ef00d0860 msgr2=0x7f4ef00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 -- 172.21.15.86:0/487613755 shutdown_connections 2024-03-22T04:11:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.464+0000 7f4ef7c00640 1 -- 172.21.15.86:0/487613755 wait complete. 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 Processor -- start 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 -- start start 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef00cd480 0x7f4ef00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cf240 0x7f4ef00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef01602c0 0x7f4ef0155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef006bcd0 con 0x7f4ef00cf240 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef006bb50 con 0x7f4ef01602c0 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef006be50 con 0x7f4ef00cd480 2024-03-22T04:11:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cf240 0x7f4ef00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cf240 0x7f4ef00ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56252/0 (socket says 172.21.15.86:56252) 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 -- 172.21.15.86:0/4171045370 learned_addr learned my addr 172.21.15.86:0/4171045370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef5200640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef00cd480 0x7f4ef00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef5c00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef01602c0 0x7f4ef0155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 -- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef00cd480 msgr2=0x7f4ef00b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef00cd480 0x7f4ef00b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 -- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef01602c0 msgr2=0x7f4ef0155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef01602c0 0x7f4ef0155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 -- 172.21.15.86:0/4171045370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef00ae1a0 con 0x7f4ef00cf240 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef5c00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef01602c0 0x7f4ef0155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eefe00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cf240 0x7f4ef00ad8e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800e8b0 tx=0x7f4ed800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eecc00640 1 -- 172.21.15.86:0/4171045370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8017070 con 0x7f4ef00cf240 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eecc00640 1 -- 172.21.15.86:0/4171045370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed800c940 con 0x7f4ef00cf240 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4eecc00640 1 -- 172.21.15.86:0/4171045370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed800cd10 con 0x7f4ef00cf240 2024-03-22T04:11:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef00ae480 con 0x7f4ef00cf240 2024-03-22T04:11:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.468+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef00aea40 con 0x7f4ef00cf240 2024-03-22T04:11:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.472+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef0074550 con 0x7f4ef00cf240 2024-03-22T04:11:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.472+0000 7f4eecc00640 1 -- 172.21.15.86:0/4171045370 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4ed801a070 con 0x7f4ef00cf240 2024-03-22T04:11:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.472+0000 7f4eecc00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ecc067b20 0x7f4ecc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.472+0000 7f4eecc00640 1 -- 172.21.15.86:0/4171045370 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4ed8086520 con 0x7f4ef00cf240 2024-03-22T04:11:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.472+0000 7f4ef5200640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ecc067b20 0x7f4ecc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.472+0000 7f4ef5200640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ecc067b20 0x7f4ecc069fe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00ad000 tx=0x7f4ee0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.476+0000 7f4eecc00640 1 -- 172.21.15.86:0/4171045370 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4ed8056590 con 0x7f4ef00cf240 2024-03-22T04:11:03.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.664+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4ef006e0e0 con 0x7f4ef00cf240 2024-03-22T04:11:03.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4eecc00640 1 -- 172.21.15.86:0/4171045370 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4ed805a630 con 0x7f4ef00cf240 2024-03-22T04:11:03.671 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:03.671 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T04:11:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ecc067b20 msgr2=0x7f4ecc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ecc067b20 0x7f4ecc069fe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00ad000 tx=0x7f4ee0008040 comp rx=0 tx=0).stop 2024-03-22T04:11:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cf240 msgr2=0x7f4ef00ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cf240 0x7f4ef00ad8e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800e8b0 tx=0x7f4ed800ed70 comp rx=0 tx=0).stop 2024-03-22T04:11:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 shutdown_connections 2024-03-22T04:11:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ecc067b20 0x7f4ecc069fe0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef01602c0 0x7f4ef0155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef00cf240 0x7f4ef00ad8e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 --2- 172.21.15.86:0/4171045370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef00cd480 0x7f4ef00b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 >> 172.21.15.86:0/4171045370 conn(0x7f4ef00d0860 msgr2=0x7f4ef015e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 shutdown_connections 2024-03-22T04:11:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.668+0000 7f4ef7c00640 1 -- 172.21.15.86:0/4171045370 wait complete. 2024-03-22T04:11:03.691 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-22T04:11:03.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:03.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d36a00640 1 Processor -- start 2024-03-22T04:11:03.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d36a00640 1 -- start start 2024-03-22T04:11:03.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d36a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d300d0580 0x7f6d30071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d36a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d301623d0 con 0x7f6d3006e2f0 2024-03-22T04:11:03.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d36a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d301620f0 con 0x7f6d3015e630 2024-03-22T04:11:03.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d36a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3015ec10 con 0x7f6d300d0580 2024-03-22T04:11:03.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d2f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d300d0580 0x7f6d30071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d2f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d300d0580 0x7f6d30071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56264/0 (socket says 172.21.15.86:56264) 2024-03-22T04:11:03.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d2fe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f6d3015e630 0x7f6d300cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45968/0 (socket says 172.21.15.86:45968) 2024-03-22T04:11:03.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d2f400640 1 -- 172.21.15.86:0/3769125566 learned_addr learned my addr 172.21.15.86:0/3769125566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:03.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.824+0000 7f6d34a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6d3006e2f0 0x7f6d30071640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57564/0 (socket says 172.21.15.86:57564) 2024-03-22T04:11:03.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d2ea00640 1 -- 172.21.15.86:0/3769125566 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1633125590 0 0) 0x7f6d301620f0 con 0x7f6d3015e630 2024-03-22T04:11:03.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d2f400640 1 -- 172.21.15.86:0/3769125566 >> v1:172.21.15.196:6789/0 conn(0x7f6d3015e630 legacy=0x7f6d300cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d2f400640 1 -- 172.21.15.86:0/3769125566 >> v1:172.21.15.86:6789/0 conn(0x7f6d3006e2f0 legacy=0x7f6d30071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d2f400640 1 -- 172.21.15.86:0/3769125566 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3015eef0 con 0x7f6d300d0580 2024-03-22T04:11:03.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d2f400640 1 --2- 172.21.15.86:0/3769125566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d300d0580 0x7f6d30071100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009870 tx=0x7f6d1c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18442497626c74d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d2ea00640 1 -- 172.21.15.86:0/3769125566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0b1070 con 0x7f6d300d0580 2024-03-22T04:11:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d2ea00640 1 -- 172.21.15.86:0/3769125566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0ac5f0 con 0x7f6d300d0580 2024-03-22T04:11:03.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d2ea00640 1 -- 172.21.15.86:0/3769125566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0ac9c0 con 0x7f6d300d0580 2024-03-22T04:11:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d36a00640 1 -- 172.21.15.86:0/3769125566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d300d0580 msgr2=0x7f6d30071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d36a00640 1 --2- 172.21.15.86:0/3769125566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d300d0580 0x7f6d30071100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009870 tx=0x7f6d1c01c3f0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d36a00640 1 -- 172.21.15.86:0/3769125566 shutdown_connections 2024-03-22T04:11:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d36a00640 1 --2- 172.21.15.86:0/3769125566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d300d0580 0x7f6d30071100 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d36a00640 1 -- 172.21.15.86:0/3769125566 >> 172.21.15.86:0/3769125566 conn(0x7f6d3006f910 msgr2=0x7f6d3006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d36a00640 1 -- 172.21.15.86:0/3769125566 shutdown_connections 2024-03-22T04:11:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.828+0000 7f6d36a00640 1 -- 172.21.15.86:0/3769125566 wait complete. 2024-03-22T04:11:03.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 Processor -- start 2024-03-22T04:11:03.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 -- start start 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d3006e2f0 0x7f6d30155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d300d0580 0x7f6d300b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d3015e630 0x7f6d300cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d300d1f60 con 0x7f6d3015e630 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3015f4b0 con 0x7f6d300d0580 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3015f9d0 con 0x7f6d3006e2f0 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d3015e630 0x7f6d300cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d3015e630 0x7f6d300cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56280/0 (socket says 172.21.15.86:56280) 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 -- 172.21.15.86:0/862708012 learned_addr learned my addr 172.21.15.86:0/862708012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d2fe00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d3006e2f0 0x7f6d30155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d2f400640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d300d0580 0x7f6d300b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 -- 172.21.15.86:0/862708012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d3006e2f0 msgr2=0x7f6d30155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d3006e2f0 0x7f6d30155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 -- 172.21.15.86:0/862708012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d300d0580 msgr2=0x7f6d300b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d300d0580 0x7f6d300b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 -- 172.21.15.86:0/862708012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d300b5790 con 0x7f6d3015e630 2024-03-22T04:11:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d34a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d3015e630 0x7f6d300cd900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009e20 tx=0x7f6d2800c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d27e00640 1 -- 172.21.15.86:0/862708012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28019070 con 0x7f6d3015e630 2024-03-22T04:11:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d27e00640 1 -- 172.21.15.86:0/862708012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d28007830 con 0x7f6d3015e630 2024-03-22T04:11:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d27e00640 1 -- 172.21.15.86:0/862708012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28007c00 con 0x7f6d3015e630 2024-03-22T04:11:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d300b5a70 con 0x7f6d3015e630 2024-03-22T04:11:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.832+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d300b6030 con 0x7f6d3015e630 2024-03-22T04:11:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.836+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d30068650 con 0x7f6d3015e630 2024-03-22T04:11:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.836+0000 7f6d27e00640 1 -- 172.21.15.86:0/862708012 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6d2801c070 con 0x7f6d3015e630 2024-03-22T04:11:03.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.836+0000 7f6d27e00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d10067b20 0x7f6d10069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:03.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.836+0000 7f6d27e00640 1 -- 172.21.15.86:0/862708012 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6d28087db0 con 0x7f6d3015e630 2024-03-22T04:11:03.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.836+0000 7f6d2fe00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d10067b20 0x7f6d10069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:03.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.836+0000 7f6d2fe00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d10067b20 0x7f6d10069fe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6d20001950 tx=0x7f6d200018e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:03.840+0000 7f6d27e00640 1 -- 172.21.15.86:0/862708012 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6d2808c070 con 0x7f6d3015e630 2024-03-22T04:11:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.000+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d3015b410 con 0x7f6d3015e630 2024-03-22T04:11:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.000+0000 7f6d27e00640 1 -- 172.21.15.86:0/862708012 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7f6d28057e20 con 0x7f6d3015e630 2024-03-22T04:11:04.005 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:04.005 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d10067b20 msgr2=0x7f6d10069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d10067b20 0x7f6d10069fe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6d20001950 tx=0x7f6d200018e0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d3015e630 msgr2=0x7f6d300cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d3015e630 0x7f6d300cd900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009e20 tx=0x7f6d2800c430 comp rx=0 tx=0).stop 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 shutdown_connections 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d10067b20 0x7f6d10069fe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d300d0580 0x7f6d300b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d3015e630 0x7f6d300cd900 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 --2- 172.21.15.86:0/862708012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d3006e2f0 0x7f6d30155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 >> 172.21.15.86:0/862708012 conn(0x7f6d3006f910 msgr2=0x7f6d3006d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 shutdown_connections 2024-03-22T04:11:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.004+0000 7f6d36a00640 1 -- 172.21.15.86:0/862708012 wait complete. 2024-03-22T04:11:04.026 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:04.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:04.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.160+0000 7f7291600640 1 Processor -- start 2024-03-22T04:11:04.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.160+0000 7f7291600640 1 -- start start 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f7291600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f728c0c6f20 0x7f728c072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f7291600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 0x7f728c162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f7291600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c0cc7c0 con 0x7f728c0c7300 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f7291600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f728c0cb720 con 0x7f728c0c6f20 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f7291600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f728c0cba00 con 0x7f728c0cb1d0 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f728aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f728c0c6f20 0x7f728c072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f728b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f728c0c7300 0x7f728c0ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57574/0 (socket says 172.21.15.86:57574) 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f728b400640 1 -- 172.21.15.86:0/3921384662 learned_addr learned my addr 172.21.15.86:0/3921384662 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f728a000640 1 --2- 172.21.15.86:0/3921384662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 0x7f728c162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f728aa00640 1 -- 172.21.15.86:0/3921384662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 msgr2=0x7f728c162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:04.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f728aa00640 1 --2- 172.21.15.86:0/3921384662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 0x7f728c162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f728aa00640 1 -- 172.21.15.86:0/3921384662 >> v1:172.21.15.86:6789/0 conn(0x7f728c0c7300 legacy=0x7f728c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f728aa00640 1 -- 172.21.15.86:0/3921384662 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c0cbce0 con 0x7f728c0c6f20 2024-03-22T04:11:04.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.164+0000 7f7289600640 1 -- 172.21.15.86:0/3921384662 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2156431317 0 0) 0x7f728c0cc7c0 con 0x7f728c0c7300 2024-03-22T04:11:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f728aa00640 1 --2- 172.21.15.86:0/3921384662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c072840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f727c0097c0 tx=0x7f727c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95f9de883746a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7289600640 1 -- 172.21.15.86:0/3921384662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0b2070 con 0x7f728c0c6f20 2024-03-22T04:11:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7289600640 1 -- 172.21.15.86:0/3921384662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f727c0ad680 con 0x7f728c0c6f20 2024-03-22T04:11:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7289600640 1 -- 172.21.15.86:0/3921384662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0ada50 con 0x7f728c0c6f20 2024-03-22T04:11:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 -- 172.21.15.86:0/3921384662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 msgr2=0x7f728c072840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 --2- 172.21.15.86:0/3921384662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c072840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f727c0097c0 tx=0x7f727c01c470 comp rx=0 tx=0).stop 2024-03-22T04:11:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 -- 172.21.15.86:0/3921384662 shutdown_connections 2024-03-22T04:11:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 --2- 172.21.15.86:0/3921384662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 0x7f728c162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 --2- 172.21.15.86:0/3921384662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c072840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 -- 172.21.15.86:0/3921384662 >> 172.21.15.86:0/3921384662 conn(0x7f728c0d0730 msgr2=0x7f728c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:04.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 -- 172.21.15.86:0/3921384662 shutdown_connections 2024-03-22T04:11:04.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 -- 172.21.15.86:0/3921384662 wait complete. 2024-03-22T04:11:04.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 Processor -- start 2024-03-22T04:11:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 -- start start 2024-03-22T04:11:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.168+0000 7f7291600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f728c0c7300 0x7f728c06f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7291600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 0x7f728c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7291600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f728c072f00 con 0x7f728c0c6f20 2024-03-22T04:11:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7291600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f728c072d80 con 0x7f728c0c7300 2024-03-22T04:11:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7291600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f728c16b390 con 0x7f728c0cb1d0 2024-03-22T04:11:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56294/0 (socket says 172.21.15.86:56294) 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 -- 172.21.15.86:0/4206383989 learned_addr learned my addr 172.21.15.86:0/4206383989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728a000640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f728c0c7300 0x7f728c06f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728b400640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 0x7f728c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 -- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 msgr2=0x7f728c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 0x7f728c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 -- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f728c0c7300 msgr2=0x7f728c06f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f728c0c7300 0x7f728c06f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 -- 172.21.15.86:0/4206383989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c15a010 con 0x7f728c0c6f20 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728a000640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f728c0c7300 0x7f728c06f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f728aa00640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c152340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f727c01ab20 tx=0x7f727c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7282a00640 1 -- 172.21.15.86:0/4206383989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0b2040 con 0x7f728c0c6f20 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7282a00640 1 -- 172.21.15.86:0/4206383989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f727c0be070 con 0x7f728c0c6f20 2024-03-22T04:11:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7282a00640 1 -- 172.21.15.86:0/4206383989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0addd0 con 0x7f728c0c6f20 2024-03-22T04:11:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c15a2f0 con 0x7f728c0c6f20 2024-03-22T04:11:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c15a8b0 con 0x7f728c0c6f20 2024-03-22T04:11:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c074470 con 0x7f728c0c6f20 2024-03-22T04:11:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7282a00640 1 -- 172.21.15.86:0/4206383989 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f727c0c3460 con 0x7f728c0c6f20 2024-03-22T04:11:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.172+0000 7f7282a00640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f726c067b20 0x7f726c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.176+0000 7f7282a00640 1 -- 172.21.15.86:0/4206383989 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f727c1204e0 con 0x7f728c0c6f20 2024-03-22T04:11:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.176+0000 7f728a000640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f726c067b20 0x7f726c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.176+0000 7f728a000640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f726c067b20 0x7f726c069fe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f728c0ce740 tx=0x7f7278008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.176+0000 7f7282a00640 1 -- 172.21.15.86:0/4206383989 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f727c016090 con 0x7f728c0c6f20 2024-03-22T04:11:04.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f728c15aed0 con 0x7f728c0c6f20 2024-03-22T04:11:04.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7282a00640 1 -- 172.21.15.86:0/4206383989 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f727c0f0bb0 con 0x7f728c0c6f20 2024-03-22T04:11:04.335 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:04.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f726c067b20 msgr2=0x7f726c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f726c067b20 0x7f726c069fe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f728c0ce740 tx=0x7f7278008040 comp rx=0 tx=0).stop 2024-03-22T04:11:04.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 msgr2=0x7f728c152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c152340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f727c01ab20 tx=0x7f727c01acb0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 shutdown_connections 2024-03-22T04:11:04.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f726c067b20 0x7f726c069fe0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f728c0c7300 0x7f728c06f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f728c0cb1d0 0x7f728c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 --2- 172.21.15.86:0/4206383989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f728c0c6f20 0x7f728c152340 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 >> 172.21.15.86:0/4206383989 conn(0x7f728c0d0730 msgr2=0x7f728c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:04.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 shutdown_connections 2024-03-22T04:11:04.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.332+0000 7f7291600640 1 -- 172.21.15.86:0/4206383989 wait complete. 2024-03-22T04:11:04.355 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:04.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.488+0000 7f9bf6800640 1 Processor -- start 2024-03-22T04:11:04.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.488+0000 7f9bf6800640 1 -- start start 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9bf6800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf00b2030 0x7f9bf00ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9bf6800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf006d6d0 con 0x7f9bf00b1b80 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9bf6800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf00b2480 con 0x7f9bf0144a90 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9bf6800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf0144e70 con 0x7f9bf00b2030 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9be6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9bf00b1b80 0x7f9bf00cdf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57582/0 (socket says 172.21.15.86:57582) 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9be6a00640 1 -- 172.21.15.86:0/2602507806 learned_addr learned my addr 172.21.15.86:0/2602507806 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9bef400640 1 --1- 172.21.15.86:0/2602507806 >> v1:172.21.15.196:6789/0 conn(0x7f9bf0144a90 0x7f9bf015f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45980/0 (socket says 172.21.15.86:45980) 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9befe00640 1 --2- 172.21.15.86:0/2602507806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf00b2030 0x7f9bf00ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9beea00640 1 -- 172.21.15.86:0/2602507806 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861571737 0 0) 0x7f9bf00b2480 con 0x7f9bf0144a90 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9beea00640 1 -- 172.21.15.86:0/2602507806 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bd4003530 con 0x7f9bf0144a90 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9befe00640 1 -- 172.21.15.86:0/2602507806 >> v1:172.21.15.196:6789/0 conn(0x7f9bf0144a90 legacy=0x7f9bf015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9befe00640 1 -- 172.21.15.86:0/2602507806 >> v1:172.21.15.86:6789/0 conn(0x7f9bf00b1b80 legacy=0x7f9bf00cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9beea00640 1 -- 172.21.15.86:0/2602507806 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2951464950 0 0) 0x7f9bf006d6d0 con 0x7f9bf00b1b80 2024-03-22T04:11:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9befe00640 1 -- 172.21.15.86:0/2602507806 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf006d970 con 0x7f9bf00b2030 2024-03-22T04:11:04.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.492+0000 7f9befe00640 1 --2- 172.21.15.86:0/2602507806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf00b2030 0x7f9bf00ce620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9be80097f0 tx=0x7f9be801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582aa2784d78015e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9beea00640 1 -- 172.21.15.86:0/2602507806 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be80aa610 con 0x7f9bf00b2030 2024-03-22T04:11:04.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9beea00640 1 -- 172.21.15.86:0/2602507806 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be80af070 con 0x7f9bf00b2030 2024-03-22T04:11:04.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9beea00640 1 -- 172.21.15.86:0/2602507806 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be80aa9d0 con 0x7f9bf00b2030 2024-03-22T04:11:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- 172.21.15.86:0/2602507806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf00b2030 msgr2=0x7f9bf00ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 --2- 172.21.15.86:0/2602507806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf00b2030 0x7f9bf00ce620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9be80097f0 tx=0x7f9be801c4b0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- 172.21.15.86:0/2602507806 shutdown_connections 2024-03-22T04:11:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 --2- 172.21.15.86:0/2602507806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf00b2030 0x7f9bf00ce620 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- 172.21.15.86:0/2602507806 >> 172.21.15.86:0/2602507806 conn(0x7f9bf00cc940 msgr2=0x7f9bf00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- 172.21.15.86:0/2602507806 shutdown_connections 2024-03-22T04:11:04.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- 172.21.15.86:0/2602507806 wait complete. 2024-03-22T04:11:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 Processor -- start 2024-03-22T04:11:04.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- start start 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf00b1b80 0x7f9bf01587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf00b2030 0x7f9bf015a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf0144a90 0x7f9bf013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00adbf0 con 0x7f9bf00b2030 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf006e4f0 con 0x7f9bf00b1b80 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bf6800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf00add70 con 0x7f9bf0144a90 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9be6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf00b2030 0x7f9bf015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9be6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf00b2030 0x7f9bf015a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56302/0 (socket says 172.21.15.86:56302) 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9be6a00640 1 -- 172.21.15.86:0/1528290688 learned_addr learned my addr 172.21.15.86:0/1528290688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9befe00640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf0144a90 0x7f9bf013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.496+0000 7f9bef400640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf00b1b80 0x7f9bf01587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be6a00640 1 -- 172.21.15.86:0/1528290688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf0144a90 msgr2=0x7f9bf013f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be6a00640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf0144a90 0x7f9bf013f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be6a00640 1 -- 172.21.15.86:0/1528290688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf00b1b80 msgr2=0x7f9bf01587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be6a00640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf00b1b80 0x7f9bf01587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be6a00640 1 -- 172.21.15.86:0/1528290688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf015f380 con 0x7f9bf00b2030 2024-03-22T04:11:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be6a00640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf00b2030 0x7f9bf015a280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800c480 tx=0x7f9bd800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be7e00640 1 -- 172.21.15.86:0/1528290688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8018070 con 0x7f9bf00b2030 2024-03-22T04:11:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be7e00640 1 -- 172.21.15.86:0/1528290688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd8007500 con 0x7f9bf00b2030 2024-03-22T04:11:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf015a7c0 con 0x7f9bf00b2030 2024-03-22T04:11:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be7e00640 1 -- 172.21.15.86:0/1528290688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8004a30 con 0x7f9bf00b2030 2024-03-22T04:11:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf00b4620 con 0x7f9bf00b2030 2024-03-22T04:11:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf0074470 con 0x7f9bf00b2030 2024-03-22T04:11:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be7e00640 1 -- 172.21.15.86:0/1528290688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9bd801b070 con 0x7f9bf00b2030 2024-03-22T04:11:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be7e00640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9bcc067b20 0x7f9bcc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9be7e00640 1 -- 172.21.15.86:0/1528290688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9bd8085530 con 0x7f9bf00b2030 2024-03-22T04:11:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.500+0000 7f9bef400640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9bcc067b20 0x7f9bcc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.504+0000 7f9bef400640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9bcc067b20 0x7f9bcc069fe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9be0001570 tx=0x7f9be0007f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.504+0000 7f9be7e00640 1 -- 172.21.15.86:0/1528290688 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9bd80555a0 con 0x7f9bf00b2030 2024-03-22T04:11:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.660+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9bf006ef80 con 0x7f9bf00b2030 2024-03-22T04:11:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.660+0000 7f9be7e00640 1 -- 172.21.15.86:0/1528290688 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f9bd8059640 con 0x7f9bf00b2030 2024-03-22T04:11:04.665 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9bcc067b20 msgr2=0x7f9bcc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9bcc067b20 0x7f9bcc069fe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9be0001570 tx=0x7f9be0007f90 comp rx=0 tx=0).stop 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf00b2030 msgr2=0x7f9bf015a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf00b2030 0x7f9bf015a280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800c480 tx=0x7f9bd800c940 comp rx=0 tx=0).stop 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 shutdown_connections 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9bcc067b20 0x7f9bcc069fe0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf0144a90 0x7f9bf013f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf00b2030 0x7f9bf015a280 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 --2- 172.21.15.86:0/1528290688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf00b1b80 0x7f9bf01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 >> 172.21.15.86:0/1528290688 conn(0x7f9bf00cc940 msgr2=0x7f9bf0072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:04.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 shutdown_connections 2024-03-22T04:11:04.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.664+0000 7f9bf6800640 1 -- 172.21.15.86:0/1528290688 wait complete. 2024-03-22T04:11:04.686 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.820+0000 7fd848200640 1 Processor -- start 2024-03-22T04:11:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.820+0000 7fd848200640 1 -- start start 2024-03-22T04:11:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.820+0000 7fd848200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd8400b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.820+0000 7fd848200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8400b3430 0x7fd84015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.820+0000 7fd848200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd84015d710 con 0x7fd84015cee0 2024-03-22T04:11:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.820+0000 7fd848200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd8400b3890 con 0x7fd8400b3430 2024-03-22T04:11:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.820+0000 7fd848200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd8400b2740 con 0x7fd8400b2360 2024-03-22T04:11:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.820+0000 7fd845800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd8400b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd845800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd8400b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39202/0 (socket says 172.21.15.86:39202) 2024-03-22T04:11:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd845800640 1 -- 172.21.15.86:0/241704730 learned_addr learned my addr 172.21.15.86:0/241704730 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd846200640 1 --1- 172.21.15.86:0/241704730 >> v1:172.21.15.196:6789/0 conn(0x7fd84015cee0 0x7fd84006d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:45996/0 (socket says 172.21.15.86:45996) 2024-03-22T04:11:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd844e00640 1 --2- 172.21.15.86:0/241704730 >> v2:172.21.15.196:3300/0 conn(0x7fd8400b3430 0x7fd84015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd845800640 1 -- 172.21.15.86:0/241704730 >> v2:172.21.15.196:3300/0 conn(0x7fd8400b3430 msgr2=0x7fd84015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd845800640 1 --2- 172.21.15.86:0/241704730 >> v2:172.21.15.196:3300/0 conn(0x7fd8400b3430 0x7fd84015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd845800640 1 -- 172.21.15.86:0/241704730 >> v1:172.21.15.196:6789/0 conn(0x7fd84015cee0 legacy=0x7fd84006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd845800640 1 -- 172.21.15.86:0/241704730 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84015fe40 con 0x7fd8400b2360 2024-03-22T04:11:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd845800640 1 --2- 172.21.15.86:0/241704730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd8400b2ef0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd834009e50 tx=0x7fd83401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8dae75c73af07a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd83fe00640 1 -- 172.21.15.86:0/241704730 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340a57e0 con 0x7fd8400b2360 2024-03-22T04:11:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd83fe00640 1 -- 172.21.15.86:0/241704730 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8340a5980 con 0x7fd8400b2360 2024-03-22T04:11:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd83fe00640 1 -- 172.21.15.86:0/241704730 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340a5d50 con 0x7fd8400b2360 2024-03-22T04:11:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd848200640 1 -- 172.21.15.86:0/241704730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 msgr2=0x7fd8400b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.824+0000 7fd848200640 1 --2- 172.21.15.86:0/241704730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd8400b2ef0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd834009e50 tx=0x7fd83401ca00 comp rx=0 tx=0).stop 2024-03-22T04:11:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 -- 172.21.15.86:0/241704730 shutdown_connections 2024-03-22T04:11:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 --2- 172.21.15.86:0/241704730 >> v2:172.21.15.196:3300/0 conn(0x7fd8400b3430 0x7fd84015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 --2- 172.21.15.86:0/241704730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd8400b2ef0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 -- 172.21.15.86:0/241704730 >> 172.21.15.86:0/241704730 conn(0x7fd8400cc940 msgr2=0x7fd8400ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 -- 172.21.15.86:0/241704730 shutdown_connections 2024-03-22T04:11:04.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 -- 172.21.15.86:0/241704730 wait complete. 2024-03-22T04:11:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 Processor -- start 2024-03-22T04:11:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 -- start start 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd84014f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8400b3430 0x7fd840150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd84015cee0 0x7fd84014c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd840160c00 con 0x7fd8400b3430 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8400b5d90 con 0x7fd84015cee0 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd848200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd84014e2c0 con 0x7fd8400b2360 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd845800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd84014f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd844e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8400b3430 0x7fd840150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd845800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd84014f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39208/0 (socket says 172.21.15.86:39208) 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd845800640 1 -- 172.21.15.86:0/1309292655 learned_addr learned my addr 172.21.15.86:0/1309292655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd846200640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd84015cee0 0x7fd84014c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd844e00640 1 -- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 msgr2=0x7fd84014f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd844e00640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd84014f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd844e00640 1 -- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd84015cee0 msgr2=0x7fd84014c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd844e00640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd84015cee0 0x7fd84014c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd844e00640 1 -- 172.21.15.86:0/1309292655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd840150f70 con 0x7fd8400b3430 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd845800640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd84014f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd846200640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd84015cee0 0x7fd84014c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd844e00640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8400b3430 0x7fd840150790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00e8a0 tx=0x7fd82c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd83d600640 1 -- 172.21.15.86:0/1309292655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c017070 con 0x7fd8400b3430 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd83d600640 1 -- 172.21.15.86:0/1309292655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c00c940 con 0x7fd8400b3430 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.828+0000 7fd83d600640 1 -- 172.21.15.86:0/1309292655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c00ccf0 con 0x7fd8400b3430 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.832+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84014da60 con 0x7fd8400b3430 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.832+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84014dd00 con 0x7fd8400b3430 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.832+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84015a180 con 0x7fd8400b3430 2024-03-22T04:11:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.832+0000 7fd83d600640 1 -- 172.21.15.86:0/1309292655 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd82c01a070 con 0x7fd8400b3430 2024-03-22T04:11:04.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.832+0000 7fd83d600640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd820067b50 0x7fd82006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:04.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.832+0000 7fd83d600640 1 -- 172.21.15.86:0/1309292655 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd82c086120 con 0x7fd8400b3430 2024-03-22T04:11:04.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.832+0000 7fd845800640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd820067b50 0x7fd82006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:04.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.836+0000 7fd845800640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd820067b50 0x7fd82006a010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd834000f90 tx=0x7fd8340a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:04.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.836+0000 7fd83d600640 1 -- 172.21.15.86:0/1309292655 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd82c056190 con 0x7fd8400b3430 2024-03-22T04:11:04.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.992+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd840074470 con 0x7fd8400b3430 2024-03-22T04:11:04.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.992+0000 7fd83d600640 1 -- 172.21.15.86:0/1309292655 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fd82c05a230 con 0x7fd8400b3430 2024-03-22T04:11:04.997 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:04.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd820067b50 msgr2=0x7fd82006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd820067b50 0x7fd82006a010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd834000f90 tx=0x7fd8340a7070 comp rx=0 tx=0).stop 2024-03-22T04:11:04.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8400b3430 msgr2=0x7fd840150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:04.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8400b3430 0x7fd840150790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00e8a0 tx=0x7fd82c00ed60 comp rx=0 tx=0).stop 2024-03-22T04:11:04.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 shutdown_connections 2024-03-22T04:11:04.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd820067b50 0x7fd82006a010 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd84015cee0 0x7fd84014c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:04.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8400b3430 0x7fd840150790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 --2- 172.21.15.86:0/1309292655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8400b2360 0x7fd84014f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 >> 172.21.15.86:0/1309292655 conn(0x7fd8400cc940 msgr2=0x7fd8401561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:05.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 shutdown_connections 2024-03-22T04:11:05.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:04.996+0000 7fd848200640 1 -- 172.21.15.86:0/1309292655 wait complete. 2024-03-22T04:11:05.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:05.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.152+0000 7f83a2c00640 1 Processor -- start 2024-03-22T04:11:05.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.152+0000 7f83a2c00640 1 -- start start 2024-03-22T04:11:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a2c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f839c15e5b0 0x7f839c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a2c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c0b4eb0 0x7f839c162740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a2c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f839c06b660 0x7f839c06baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a2c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f839c1622b0 con 0x7f839c06b660 2024-03-22T04:11:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a2c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f839c0b5470 con 0x7f839c15e5b0 2024-03-22T04:11:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a2c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f839c0b5750 con 0x7f839c0b4eb0 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f839c06b660 0x7f839c06baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f839c06b660 0x7f839c06baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56316/0 (socket says 172.21.15.86:56316) 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a0c00640 1 -- 172.21.15.86:0/2428882364 learned_addr learned my addr 172.21.15.86:0/2428882364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f839be00640 1 --2- 172.21.15.86:0/2428882364 >> v2:172.21.15.196:3300/0 conn(0x7f839c15e5b0 0x7f839c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f839b400640 1 --2- 172.21.15.86:0/2428882364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c0b4eb0 0x7f839c162740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a0c00640 1 -- 172.21.15.86:0/2428882364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c0b4eb0 msgr2=0x7f839c162740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a0c00640 1 --2- 172.21.15.86:0/2428882364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c0b4eb0 0x7f839c162740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a0c00640 1 -- 172.21.15.86:0/2428882364 >> v2:172.21.15.196:3300/0 conn(0x7f839c15e5b0 msgr2=0x7f839c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a0c00640 1 --2- 172.21.15.86:0/2428882364 >> v2:172.21.15.196:3300/0 conn(0x7f839c15e5b0 0x7f839c0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.156+0000 7f83a0c00640 1 -- 172.21.15.86:0/2428882364 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0b5a30 con 0x7f839c06b660 2024-03-22T04:11:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a0c00640 1 --2- 172.21.15.86:0/2428882364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c06b660 0x7f839c06baf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f839c068f40 tx=0x7f839401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7005ee876af2133 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f839aa00640 1 -- 172.21.15.86:0/2428882364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940a5670 con 0x7f839c06b660 2024-03-22T04:11:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f839aa00640 1 -- 172.21.15.86:0/2428882364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83940a5810 con 0x7f839c06b660 2024-03-22T04:11:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f839aa00640 1 -- 172.21.15.86:0/2428882364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940a5be0 con 0x7f839c06b660 2024-03-22T04:11:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- 172.21.15.86:0/2428882364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c06b660 msgr2=0x7f839c06baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 --2- 172.21.15.86:0/2428882364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c06b660 0x7f839c06baf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f839c068f40 tx=0x7f839401c9f0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- 172.21.15.86:0/2428882364 shutdown_connections 2024-03-22T04:11:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 --2- 172.21.15.86:0/2428882364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c06b660 0x7f839c06baf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 --2- 172.21.15.86:0/2428882364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c0b4eb0 0x7f839c162740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 --2- 172.21.15.86:0/2428882364 >> v2:172.21.15.196:3300/0 conn(0x7f839c15e5b0 0x7f839c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- 172.21.15.86:0/2428882364 >> 172.21.15.86:0/2428882364 conn(0x7f839c0d0fa0 msgr2=0x7f839c0d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- 172.21.15.86:0/2428882364 shutdown_connections 2024-03-22T04:11:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- 172.21.15.86:0/2428882364 wait complete. 2024-03-22T04:11:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 Processor -- start 2024-03-22T04:11:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- start start 2024-03-22T04:11:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06b660 0x7f839c1fae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c0b4eb0 0x7f839c13bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15e5b0 0x7f839c154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f839c072df0 con 0x7f839c0b4eb0 2024-03-22T04:11:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f839c072c70 con 0x7f839c06b660 2024-03-22T04:11:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.160+0000 7f83a2c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f839c06e430 con 0x7f839c15e5b0 2024-03-22T04:11:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c0b4eb0 0x7f839c13bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c0b4eb0 0x7f839c13bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56330/0 (socket says 172.21.15.86:56330) 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 -- 172.21.15.86:0/1482670049 learned_addr learned my addr 172.21.15.86:0/1482670049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839be00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06b660 0x7f839c1fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f83a0c00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15e5b0 0x7f839c154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 -- 172.21.15.86:0/1482670049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15e5b0 msgr2=0x7f839c154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15e5b0 0x7f839c154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 -- 172.21.15.86:0/1482670049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06b660 msgr2=0x7f839c1fae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06b660 0x7f839c1fae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 -- 172.21.15.86:0/1482670049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c154ec0 con 0x7f839c0b4eb0 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f839b400640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c0b4eb0 0x7f839c13bb80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f838800e960 tx=0x7f838800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f8393e00640 1 -- 172.21.15.86:0/1482670049 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388017070 con 0x7f839c0b4eb0 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f8393e00640 1 -- 172.21.15.86:0/1482670049 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838800ca00 con 0x7f839c0b4eb0 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c160100 con 0x7f839c0b4eb0 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f8393e00640 1 -- 172.21.15.86:0/1482670049 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838800cdb0 con 0x7f839c0b4eb0 2024-03-22T04:11:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1fa660 con 0x7f839c0b4eb0 2024-03-22T04:11:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c074470 con 0x7f839c0b4eb0 2024-03-22T04:11:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.164+0000 7f8393e00640 1 -- 172.21.15.86:0/1482670049 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f838801a070 con 0x7f839c0b4eb0 2024-03-22T04:11:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.168+0000 7f8393e00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f837c067b20 0x7f837c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.168+0000 7f8393e00640 1 -- 172.21.15.86:0/1482670049 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8388086200 con 0x7f839c0b4eb0 2024-03-22T04:11:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.168+0000 7f839be00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f837c067b20 0x7f837c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.168+0000 7f839be00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f837c067b20 0x7f837c069fe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f838c009a40 tx=0x7f838c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.172+0000 7f8393e00640 1 -- 172.21.15.86:0/1482670049 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8388056280 con 0x7f839c0b4eb0 2024-03-22T04:11:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f839c162a60 con 0x7f839c0b4eb0 2024-03-22T04:11:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f8393e00640 1 -- 172.21.15.86:0/1482670049 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f838805a320 con 0x7f839c0b4eb0 2024-03-22T04:11:05.328 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f837c067b20 msgr2=0x7f837c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f837c067b20 0x7f837c069fe0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f838c009a40 tx=0x7f838c009190 comp rx=0 tx=0).stop 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c0b4eb0 msgr2=0x7f839c13bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c0b4eb0 0x7f839c13bb80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f838800e960 tx=0x7f838800ee20 comp rx=0 tx=0).stop 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 shutdown_connections 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f837c067b20 0x7f837c069fe0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15e5b0 0x7f839c154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f839c0b4eb0 0x7f839c13bb80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 --2- 172.21.15.86:0/1482670049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06b660 0x7f839c1fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 >> 172.21.15.86:0/1482670049 conn(0x7f839c0d0fa0 msgr2=0x7f839c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 shutdown_connections 2024-03-22T04:11:05.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.324+0000 7f83a2c00640 1 -- 172.21.15.86:0/1482670049 wait complete. 2024-03-22T04:11:05.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:11:05.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.480+0000 7fca18000640 1 Processor -- start 2024-03-22T04:11:05.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.480+0000 7fca18000640 1 -- start start 2024-03-22T04:11:05.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca18000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca1015b3f0 0x7fca100729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca18000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca100ae540 0x7fca10072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca18000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca10071920 con 0x7fca100cc3e0 2024-03-22T04:11:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca18000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fca100716d0 con 0x7fca100ae540 2024-03-22T04:11:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca18000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca1006d990 con 0x7fca1015b3f0 2024-03-22T04:11:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca14c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fca100cc3e0 0x7fca100ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:46008/0 (socket says 172.21.15.86:46008) 2024-03-22T04:11:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca15600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca1015b3f0 0x7fca100729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca14c00640 1 -- 172.21.15.86:0/3320274562 learned_addr learned my addr 172.21.15.86:0/3320274562 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca16000640 1 --2- 172.21.15.86:0/3320274562 >> v2:172.21.15.196:3300/0 conn(0x7fca100ae540 0x7fca10072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca0fe00640 1 -- 172.21.15.86:0/3320274562 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896898756 0 0) 0x7fca10071920 con 0x7fca100cc3e0 2024-03-22T04:11:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca0fe00640 1 -- 172.21.15.86:0/3320274562 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9f8005740 con 0x7fca100cc3e0 2024-03-22T04:11:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca15600640 1 -- 172.21.15.86:0/3320274562 >> v2:172.21.15.196:3300/0 conn(0x7fca100ae540 msgr2=0x7fca10072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca15600640 1 --2- 172.21.15.86:0/3320274562 >> v2:172.21.15.196:3300/0 conn(0x7fca100ae540 0x7fca10072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca15600640 1 -- 172.21.15.86:0/3320274562 >> v1:172.21.15.196:6789/0 conn(0x7fca100cc3e0 legacy=0x7fca100ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca15600640 1 -- 172.21.15.86:0/3320274562 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca10071c00 con 0x7fca1015b3f0 2024-03-22T04:11:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.484+0000 7fca16000640 1 --2- 172.21.15.86:0/3320274562 >> v2:172.21.15.196:3300/0 conn(0x7fca100ae540 0x7fca10072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca15600640 1 --2- 172.21.15.86:0/3320274562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca1015b3f0 0x7fca100729c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fca040097c0 tx=0x7fca0401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c714487f6cf9900d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca0fe00640 1 -- 172.21.15.86:0/3320274562 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040b0070 con 0x7fca1015b3f0 2024-03-22T04:11:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca0fe00640 1 -- 172.21.15.86:0/3320274562 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca040ab600 con 0x7fca1015b3f0 2024-03-22T04:11:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca0fe00640 1 -- 172.21.15.86:0/3320274562 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040ab9d0 con 0x7fca1015b3f0 2024-03-22T04:11:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 -- 172.21.15.86:0/3320274562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca1015b3f0 msgr2=0x7fca100729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 --2- 172.21.15.86:0/3320274562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca1015b3f0 0x7fca100729c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fca040097c0 tx=0x7fca0401c410 comp rx=0 tx=0).stop 2024-03-22T04:11:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 -- 172.21.15.86:0/3320274562 shutdown_connections 2024-03-22T04:11:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 --2- 172.21.15.86:0/3320274562 >> v2:172.21.15.196:3300/0 conn(0x7fca100ae540 0x7fca10072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 --2- 172.21.15.86:0/3320274562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca1015b3f0 0x7fca100729c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 -- 172.21.15.86:0/3320274562 >> 172.21.15.86:0/3320274562 conn(0x7fca100b1df0 msgr2=0x7fca100b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 -- 172.21.15.86:0/3320274562 shutdown_connections 2024-03-22T04:11:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 -- 172.21.15.86:0/3320274562 wait complete. 2024-03-22T04:11:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 Processor -- start 2024-03-22T04:11:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.488+0000 7fca18000640 1 -- start start 2024-03-22T04:11:05.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca100ae540 0x7fca10154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca100cc3e0 0x7fca1014fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca1015b3f0 0x7fca10151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca1006e320 con 0x7fca100ae540 2024-03-22T04:11:05.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca1006e1a0 con 0x7fca1015b3f0 2024-03-22T04:11:05.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca1006e4a0 con 0x7fca100cc3e0 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca100ae540 0x7fca10154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca100ae540 0x7fca10154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56336/0 (socket says 172.21.15.86:56336) 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 -- 172.21.15.86:0/3569871363 learned_addr learned my addr 172.21.15.86:0/3569871363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca16000640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca1015b3f0 0x7fca10151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca14c00640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca100cc3e0 0x7fca1014fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 -- 172.21.15.86:0/3569871363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca100cc3e0 msgr2=0x7fca1014fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca100cc3e0 0x7fca1014fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 -- 172.21.15.86:0/3569871363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca1015b3f0 msgr2=0x7fca10151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca1015b3f0 0x7fca10151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 -- 172.21.15.86:0/3569871363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca10150480 con 0x7fca100ae540 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca15600640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca100ae540 0x7fca10154800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fca040097c0 tx=0x7fca040191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca0d600640 1 -- 172.21.15.86:0/3569871363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040ba020 con 0x7fca100ae540 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca0d600640 1 -- 172.21.15.86:0/3569871363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca040b5460 con 0x7fca100ae540 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca10150760 con 0x7fca100ae540 2024-03-22T04:11:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca0d600640 1 -- 172.21.15.86:0/3569871363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040b0040 con 0x7fca100ae540 2024-03-22T04:11:05.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca10150d20 con 0x7fca100ae540 2024-03-22T04:11:05.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca10068650 con 0x7fca100ae540 2024-03-22T04:11:05.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.492+0000 7fca0d600640 1 -- 172.21.15.86:0/3569871363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fca040bd020 con 0x7fca100ae540 2024-03-22T04:11:05.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.496+0000 7fca0d600640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9f0067b20 0x7fc9f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.496+0000 7fca0d600640 1 -- 172.21.15.86:0/3569871363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fca041203e0 con 0x7fca100ae540 2024-03-22T04:11:05.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.496+0000 7fca14c00640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9f0067b20 0x7fc9f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.496+0000 7fca14c00640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9f0067b20 0x7fc9f0069fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc007fd0 tx=0x7fc9fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.500+0000 7fca0d600640 1 -- 172.21.15.86:0/3569871363 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fca040c4030 con 0x7fca100ae540 2024-03-22T04:11:05.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.684+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca100b1600 con 0x7fca100ae540 2024-03-22T04:11:05.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca0d600640 1 -- 172.21.15.86:0/3569871363 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5179 (secure 0 0 0) 0x7fca04016090 con 0x7fca100ae540 2024-03-22T04:11:05.690 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:05.690 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:58.258965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:11:05.690 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T04:11:05.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9f0067b20 msgr2=0x7fc9f0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9f0067b20 0x7fc9f0069fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc007fd0 tx=0x7fc9fc009040 comp rx=0 tx=0).stop 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca100ae540 msgr2=0x7fca10154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca100ae540 0x7fca10154800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fca040097c0 tx=0x7fca040191b0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 shutdown_connections 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9f0067b20 0x7fc9f0069fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca1015b3f0 0x7fca10151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca100cc3e0 0x7fca1014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 --2- 172.21.15.86:0/3569871363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca100ae540 0x7fca10154800 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 >> 172.21.15.86:0/3569871363 conn(0x7fca100b1df0 msgr2=0x7fca100b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 shutdown_connections 2024-03-22T04:11:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.688+0000 7fca18000640 1 -- 172.21.15.86:0/3569871363 wait complete. 2024-03-22T04:11:05.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:05.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2df800640 1 Processor -- start 2024-03-22T04:11:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2df800640 1 -- start start 2024-03-22T04:11:05.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2df800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2d80d0de0 0x7fd2d80ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2df800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d815fed0 0x7fd2d8160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2df800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d80d11c0 con 0x7fd2d80d19a0 2024-03-22T04:11:05.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2df800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d80d0580 con 0x7fd2d80d0de0 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2df800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d8160860 con 0x7fd2d815fed0 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2dce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2d80d0de0 0x7fd2d80ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2dce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2d80d0de0 0x7fd2d80ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56350/0 (socket says 172.21.15.86:56350) 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2dce00640 1 -- 172.21.15.86:0/246533625 learned_addr learned my addr 172.21.15.86:0/246533625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2dd800640 1 --2- 172.21.15.86:0/246533625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d815fed0 0x7fd2d8160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2dce00640 1 -- 172.21.15.86:0/246533625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d815fed0 msgr2=0x7fd2d8160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2dce00640 1 --2- 172.21.15.86:0/246533625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d815fed0 0x7fd2d8160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2dce00640 1 -- 172.21.15.86:0/246533625 >> v1:172.21.15.196:6789/0 conn(0x7fd2d80d19a0 legacy=0x7fd2d80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2dce00640 1 -- 172.21.15.86:0/246533625 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d80cece0 con 0x7fd2d80d0de0 2024-03-22T04:11:05.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.844+0000 7fd2d7400640 1 -- 172.21.15.86:0/246533625 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 864680955 0 0) 0x7fd2d80d11c0 con 0x7fd2d80d19a0 2024-03-22T04:11:05.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2dce00640 1 --2- 172.21.15.86:0/246533625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d80ce6b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009870 tx=0x7fd2c801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b234594a4cf22a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2d7400640 1 -- 172.21.15.86:0/246533625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80ab650 con 0x7fd2d80d0de0 2024-03-22T04:11:05.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2d7400640 1 -- 172.21.15.86:0/246533625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c80b0070 con 0x7fd2d80d0de0 2024-03-22T04:11:05.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2d7400640 1 -- 172.21.15.86:0/246533625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80aba10 con 0x7fd2d80d0de0 2024-03-22T04:11:05.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2df800640 1 -- 172.21.15.86:0/246533625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 msgr2=0x7fd2d80ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2df800640 1 --2- 172.21.15.86:0/246533625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d80ce6b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009870 tx=0x7fd2c801c560 comp rx=0 tx=0).stop 2024-03-22T04:11:05.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2df800640 1 -- 172.21.15.86:0/246533625 shutdown_connections 2024-03-22T04:11:05.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2df800640 1 --2- 172.21.15.86:0/246533625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d815fed0 0x7fd2d8160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2df800640 1 --2- 172.21.15.86:0/246533625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d80ce6b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2df800640 1 -- 172.21.15.86:0/246533625 >> 172.21.15.86:0/246533625 conn(0x7fd2d806f910 msgr2=0x7fd2d806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:05.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2df800640 1 -- 172.21.15.86:0/246533625 shutdown_connections 2024-03-22T04:11:05.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.848+0000 7fd2df800640 1 -- 172.21.15.86:0/246533625 wait complete. 2024-03-22T04:11:05.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 Processor -- start 2024-03-22T04:11:05.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 -- start start 2024-03-22T04:11:05.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d806d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d80d19a0 0x7fd2d81573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2d815fed0 0x7fd2d8165900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d80d1db0 con 0x7fd2d80d0de0 2024-03-22T04:11:05.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d8071100 con 0x7fd2d815fed0 2024-03-22T04:11:05.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d80d1f30 con 0x7fd2d80d19a0 2024-03-22T04:11:05.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d806d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d806d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56358/0 (socket says 172.21.15.86:56358) 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 -- 172.21.15.86:0/1128615535 learned_addr learned my addr 172.21.15.86:0/1128615535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dd800640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2d815fed0 0x7fd2d8165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2d7e00640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d80d19a0 0x7fd2d81573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 -- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d80d19a0 msgr2=0x7fd2d81573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d80d19a0 0x7fd2d81573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 -- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2d815fed0 msgr2=0x7fd2d8165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2d815fed0 0x7fd2d8165900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 -- 172.21.15.86:0/1128615535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d8166070 con 0x7fd2d80d0de0 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dd800640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2d815fed0 0x7fd2d8165900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2dce00640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d806d620 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80ad040 tx=0x7fd2c801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2d4c00640 1 -- 172.21.15.86:0/1128615535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80be5f0 con 0x7fd2d80d0de0 2024-03-22T04:11:05.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2d4c00640 1 -- 172.21.15.86:0/1128615535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c80be790 con 0x7fd2d80d0de0 2024-03-22T04:11:05.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d806c8a0 con 0x7fd2d80d0de0 2024-03-22T04:11:05.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2d4c00640 1 -- 172.21.15.86:0/1128615535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80beb80 con 0x7fd2d80d0de0 2024-03-22T04:11:05.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.852+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d806ce20 con 0x7fd2d80d0de0 2024-03-22T04:11:05.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.856+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d8068650 con 0x7fd2d80d0de0 2024-03-22T04:11:05.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.856+0000 7fd2d4c00640 1 -- 172.21.15.86:0/1128615535 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd2c80b43f0 con 0x7fd2d80d0de0 2024-03-22T04:11:05.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.856+0000 7fd2d4c00640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2b4067b20 0x7fd2b4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.856+0000 7fd2d4c00640 1 -- 172.21.15.86:0/1128615535 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd2c8121020 con 0x7fd2d80d0de0 2024-03-22T04:11:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.856+0000 7fd2d7e00640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2b4067b20 0x7fd2b4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.856+0000 7fd2d7e00640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2b4067b20 0x7fd2b4069fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2d815d4b0 tx=0x7fd2c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:05.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:05.860+0000 7fd2d4c00640 1 -- 172.21.15.86:0/1128615535 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd2c80c4060 con 0x7fd2d80d0de0 2024-03-22T04:11:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.020+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2d8070170 con 0x7fd2d80d0de0 2024-03-22T04:11:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.020+0000 7fd2d4c00640 1 -- 172.21.15.86:0/1128615535 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7fd2c8016150 con 0x7fd2d80d0de0 2024-03-22T04:11:06.025 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:06.026 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2b4067b20 msgr2=0x7fd2b4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2b4067b20 0x7fd2b4069fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2d815d4b0 tx=0x7fd2c0009040 comp rx=0 tx=0).stop 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 msgr2=0x7fd2d806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d806d620 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80ad040 tx=0x7fd2c801ada0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 shutdown_connections 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2b4067b20 0x7fd2b4069fe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2d815fed0 0x7fd2d8165900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2d80d19a0 0x7fd2d81573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 --2- 172.21.15.86:0/1128615535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d80d0de0 0x7fd2d806d620 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 >> 172.21.15.86:0/1128615535 conn(0x7fd2d806f910 msgr2=0x7fd2d815ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:06.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 shutdown_connections 2024-03-22T04:11:06.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.024+0000 7fd2df800640 1 -- 172.21.15.86:0/1128615535 wait complete. 2024-03-22T04:11:06.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:06.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:06.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6f000640 1 Processor -- start 2024-03-22T04:11:06.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6f000640 1 -- start start 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6f000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 0x7f9e680d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e680b4710 0x7f9e6806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6f000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e68070390 0x7f9e6809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9e680b4b70 con 0x7f9e680b4710 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6f000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9e6806f970 con 0x7f9e68070390 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6f000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e680d15f0 con 0x7f9e6815e7e0 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e5f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e680b4710 0x7f9e6806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e5f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e680b4710 0x7f9e6806fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60160/0 (socket says 172.21.15.86:60160) 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e5f400640 1 -- 172.21.15.86:0/2001992656 learned_addr learned my addr 172.21.15.86:0/2001992656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e6d000640 1 --2- 172.21.15.86:0/2001992656 >> v2:172.21.15.196:3300/0 conn(0x7f9e68070390 0x7f9e6809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e67e00640 1 --2- 172.21.15.86:0/2001992656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 0x7f9e680d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e5f400640 1 -- 172.21.15.86:0/2001992656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 msgr2=0x7f9e680d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e5f400640 1 --2- 172.21.15.86:0/2001992656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 0x7f9e680d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e5f400640 1 -- 172.21.15.86:0/2001992656 >> v2:172.21.15.196:3300/0 conn(0x7f9e68070390 msgr2=0x7f9e6809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e5f400640 1 --2- 172.21.15.86:0/2001992656 >> v2:172.21.15.196:3300/0 conn(0x7f9e68070390 0x7f9e6809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.184+0000 7f9e5f400640 1 -- 172.21.15.86:0/2001992656 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e68070820 con 0x7f9e680b4710 2024-03-22T04:11:06.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e5f400640 1 --2- 172.21.15.86:0/2001992656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e680b4710 0x7f9e6806fe50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e680cc650 tx=0x7f9e5001c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3f9e148c493439 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e67400640 1 -- 172.21.15.86:0/2001992656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500b0070 con 0x7f9e680b4710 2024-03-22T04:11:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e67400640 1 -- 172.21.15.86:0/2001992656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e500ab600 con 0x7f9e680b4710 2024-03-22T04:11:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e67400640 1 -- 172.21.15.86:0/2001992656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500ab9d0 con 0x7f9e680b4710 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 -- 172.21.15.86:0/2001992656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e680b4710 msgr2=0x7f9e6806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 --2- 172.21.15.86:0/2001992656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e680b4710 0x7f9e6806fe50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e680cc650 tx=0x7f9e5001c110 comp rx=0 tx=0).stop 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 -- 172.21.15.86:0/2001992656 shutdown_connections 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 --2- 172.21.15.86:0/2001992656 >> v2:172.21.15.196:3300/0 conn(0x7f9e68070390 0x7f9e6809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 --2- 172.21.15.86:0/2001992656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e680b4710 0x7f9e6806fe50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 --2- 172.21.15.86:0/2001992656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 0x7f9e680d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 -- 172.21.15.86:0/2001992656 >> 172.21.15.86:0/2001992656 conn(0x7f9e680b3bf0 msgr2=0x7f9e680b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 -- 172.21.15.86:0/2001992656 shutdown_connections 2024-03-22T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.188+0000 7f9e6f000640 1 -- 172.21.15.86:0/2001992656 wait complete. 2024-03-22T04:11:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 Processor -- start 2024-03-22T04:11:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 -- start start 2024-03-22T04:11:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e68070390 0x7f9e680cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e680b4710 0x7f9e680c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 0x7f9e680b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e680cd360 con 0x7f9e68070390 2024-03-22T04:11:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e680cce40 con 0x7f9e680b4710 2024-03-22T04:11:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e680cd4e0 con 0x7f9e6815e7e0 2024-03-22T04:11:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e68070390 0x7f9e680cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e68070390 0x7f9e680cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60166/0 (socket says 172.21.15.86:60166) 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 -- 172.21.15.86:0/623835007 learned_addr learned my addr 172.21.15.86:0/623835007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e5f400640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e680b4710 0x7f9e680c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6d000640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 0x7f9e680b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 -- 172.21.15.86:0/623835007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 msgr2=0x7f9e680b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 0x7f9e680b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 -- 172.21.15.86:0/623835007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e680b4710 msgr2=0x7f9e680c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e680b4710 0x7f9e680c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 -- 172.21.15.86:0/623835007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6814c3a0 con 0x7f9e68070390 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e67e00640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e68070390 0x7f9e680cbe70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e870 tx=0x7f9e5800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e64c00640 1 -- 172.21.15.86:0/623835007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58017070 con 0x7f9e68070390 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e64c00640 1 -- 172.21.15.86:0/623835007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5800c940 con 0x7f9e68070390 2024-03-22T04:11:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e64c00640 1 -- 172.21.15.86:0/623835007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800ccf0 con 0x7f9e68070390 2024-03-22T04:11:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6814c680 con 0x7f9e68070390 2024-03-22T04:11:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6814cc40 con 0x7f9e68070390 2024-03-22T04:11:06.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.192+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e68074470 con 0x7f9e68070390 2024-03-22T04:11:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.196+0000 7f9e64c00640 1 -- 172.21.15.86:0/623835007 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9e5801a070 con 0x7f9e68070390 2024-03-22T04:11:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.196+0000 7f9e64c00640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e44067b50 0x7f9e4406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.196+0000 7f9e64c00640 1 -- 172.21.15.86:0/623835007 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9e58086390 con 0x7f9e68070390 2024-03-22T04:11:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.196+0000 7f9e5f400640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e44067b50 0x7f9e4406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.196+0000 7f9e5f400640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e44067b50 0x7f9e4406a010 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e500ad000 tx=0x7f9e500a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.200+0000 7f9e64c00640 1 -- 172.21.15.86:0/623835007 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9e58056410 con 0x7f9e68070390 2024-03-22T04:11:06.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e6806fc30 con 0x7f9e68070390 2024-03-22T04:11:06.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e64c00640 1 -- 172.21.15.86:0/623835007 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9e5805a4b0 con 0x7f9e68070390 2024-03-22T04:11:06.359 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:06.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e44067b50 msgr2=0x7f9e4406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e44067b50 0x7f9e4406a010 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e500ad000 tx=0x7f9e500a6000 comp rx=0 tx=0).stop 2024-03-22T04:11:06.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e68070390 msgr2=0x7f9e680cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e68070390 0x7f9e680cbe70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e870 tx=0x7f9e5800ec70 comp rx=0 tx=0).stop 2024-03-22T04:11:06.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 shutdown_connections 2024-03-22T04:11:06.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e44067b50 0x7f9e4406a010 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e680b4710 0x7f9e680c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e6815e7e0 0x7f9e680b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 --2- 172.21.15.86:0/623835007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e68070390 0x7f9e680cbe70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 >> 172.21.15.86:0/623835007 conn(0x7f9e680b3bf0 msgr2=0x7f9e6806ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:06.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 shutdown_connections 2024-03-22T04:11:06.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.356+0000 7f9e6f000640 1 -- 172.21.15.86:0/623835007 wait complete. 2024-03-22T04:11:06.380 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:06.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.512+0000 7f4f6ea00640 1 Processor -- start 2024-03-22T04:11:06.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.512+0000 7f4f6ea00640 1 -- start start 2024-03-22T04:11:06.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f6ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4f6806e7e0 0x7f4f6806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f6ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68072650 0x7f4f68072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f6ea00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f6815c770 con 0x7f4f680b1b80 2024-03-22T04:11:06.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f6ea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4f68071350 con 0x7f4f6806e7e0 2024-03-22T04:11:06.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f6ea00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6806f100 con 0x7f4f68072650 2024-03-22T04:11:06.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f67e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4f6806e7e0 0x7f4f6806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f67e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4f6806e7e0 0x7f4f6806ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60170/0 (socket says 172.21.15.86:60170) 2024-03-22T04:11:06.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f67e00640 1 -- 172.21.15.86:0/2841450587 learned_addr learned my addr 172.21.15.86:0/2841450587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:06.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f5f400640 1 --2- 172.21.15.86:0/2841450587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68072650 0x7f4f68072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f67e00640 1 -- 172.21.15.86:0/2841450587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68072650 msgr2=0x7f4f68072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:06.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f67e00640 1 --2- 172.21.15.86:0/2841450587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68072650 0x7f4f68072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f67e00640 1 -- 172.21.15.86:0/2841450587 >> v1:172.21.15.196:6789/0 conn(0x7f4f680b1b80 legacy=0x7f4f680b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.516+0000 7f4f67e00640 1 -- 172.21.15.86:0/2841450587 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f68073150 con 0x7f4f6806e7e0 2024-03-22T04:11:06.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f67e00640 1 --2- 172.21.15.86:0/2841450587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6806ebc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009870 tx=0x7f4f5801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6259f882782e4a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f67400640 1 -- 172.21.15.86:0/2841450587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f580aa600 con 0x7f4f6806e7e0 2024-03-22T04:11:06.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f67400640 1 -- 172.21.15.86:0/2841450587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f580af070 con 0x7f4f6806e7e0 2024-03-22T04:11:06.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f67400640 1 -- 172.21.15.86:0/2841450587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f580aa9c0 con 0x7f4f6806e7e0 2024-03-22T04:11:06.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2841450587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 msgr2=0x7f4f6806ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2841450587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6806ebc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009870 tx=0x7f4f5801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2841450587 shutdown_connections 2024-03-22T04:11:06.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2841450587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68072650 0x7f4f68072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2841450587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6806ebc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2841450587 >> 172.21.15.86:0/2841450587 conn(0x7f4f680cc940 msgr2=0x7f4f680ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:06.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2841450587 shutdown_connections 2024-03-22T04:11:06.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2841450587 wait complete. 2024-03-22T04:11:06.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 Processor -- start 2024-03-22T04:11:06.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 -- start start 2024-03-22T04:11:06.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.520+0000 7f4f6ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68072650 0x7f4f681fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f680b1b80 0x7f4f681630d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68072080 con 0x7f4f6806e7e0 2024-03-22T04:11:06.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ea00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f680ce930 con 0x7f4f68072650 2024-03-22T04:11:06.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ea00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f681fb460 con 0x7f4f680b1b80 2024-03-22T04:11:06.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60182/0 (socket says 172.21.15.86:60182) 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 -- 172.21.15.86:0/2628815117 learned_addr learned my addr 172.21.15.86:0/2628815117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f5f400640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68072650 0x7f4f681fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ca00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f680b1b80 0x7f4f681630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 -- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f680b1b80 msgr2=0x7f4f681630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f680b1b80 0x7f4f681630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 -- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68072650 msgr2=0x7f4f681fa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68072650 0x7f4f681fa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 -- 172.21.15.86:0/2628815117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f68152360 con 0x7f4f6806e7e0 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ca00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f680b1b80 0x7f4f681630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f67e00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6815a790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4f580ac040 tx=0x7f4f5801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f64c00640 1 -- 172.21.15.86:0/2628815117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f580bd5f0 con 0x7f4f6806e7e0 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f64c00640 1 -- 172.21.15.86:0/2628815117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f580bd7b0 con 0x7f4f6806e7e0 2024-03-22T04:11:06.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f64c00640 1 -- 172.21.15.86:0/2628815117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f580bdba0 con 0x7f4f6806e7e0 2024-03-22T04:11:06.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f681fabc0 con 0x7f4f6806e7e0 2024-03-22T04:11:06.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681fb180 con 0x7f4f6806e7e0 2024-03-22T04:11:06.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f681f6eb0 con 0x7f4f6806e7e0 2024-03-22T04:11:06.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.524+0000 7f4f64c00640 1 -- 172.21.15.86:0/2628815117 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4f580a9070 con 0x7f4f6806e7e0 2024-03-22T04:11:06.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.528+0000 7f4f64c00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b20 0x7f4f44069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.528+0000 7f4f64c00640 1 -- 172.21.15.86:0/2628815117 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4f58120760 con 0x7f4f6806e7e0 2024-03-22T04:11:06.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.528+0000 7f4f5f400640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b20 0x7f4f44069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.528+0000 7f4f5f400640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b20 0x7f4f44069fe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f6806f050 tx=0x7f4f50009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.532+0000 7f4f64c00640 1 -- 172.21.15.86:0/2628815117 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4f681f6eb0 con 0x7f4f6806e7e0 2024-03-22T04:11:06.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.684+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f68074470 con 0x7f4f6806e7e0 2024-03-22T04:11:06.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.684+0000 7f4f64c00640 1 -- 172.21.15.86:0/2628815117 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f4f58016150 con 0x7f4f6806e7e0 2024-03-22T04:11:06.688 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b20 msgr2=0x7f4f44069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b20 0x7f4f44069fe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f6806f050 tx=0x7f4f50009190 comp rx=0 tx=0).stop 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 msgr2=0x7f4f6815a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6815a790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4f580ac040 tx=0x7f4f5801ace0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 shutdown_connections 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b20 0x7f4f44069fe0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68072650 0x7f4f681fa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f680b1b80 0x7f4f681630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 --2- 172.21.15.86:0/2628815117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f6806e7e0 0x7f4f6815a790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 >> 172.21.15.86:0/2628815117 conn(0x7f4f680cc940 msgr2=0x7f4f681653d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 shutdown_connections 2024-03-22T04:11:06.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.688+0000 7f4f6ea00640 1 -- 172.21.15.86:0/2628815117 wait complete. 2024-03-22T04:11:06.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe284a00640 1 Processor -- start 2024-03-22T04:11:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe284a00640 1 -- start start 2024-03-22T04:11:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe284a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe28015e7e0 0x7fe2800d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe284a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe280070560 0x7fe2800cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe284a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2800d15f0 con 0x7fe2800b4710 2024-03-22T04:11:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe284a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe2800b4b70 con 0x7fe280070560 2024-03-22T04:11:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe284a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe28006f970 con 0x7fe28015e7e0 2024-03-22T04:11:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe27e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe280070560 0x7fe2800cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe27e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe280070560 0x7fe2800cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60188/0 (socket says 172.21.15.86:60188) 2024-03-22T04:11:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe27e800640 1 -- 172.21.15.86:0/2469629910 learned_addr learned my addr 172.21.15.86:0/2469629910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe27d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2800b4710 0x7fe28006fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50048/0 (socket says 172.21.15.86:50048) 2024-03-22T04:11:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.844+0000 7fe27de00640 1 --2- 172.21.15.86:0/2469629910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe28015e7e0 0x7fe2800d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27e800640 1 -- 172.21.15.86:0/2469629910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe28015e7e0 msgr2=0x7fe2800d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27e800640 1 --2- 172.21.15.86:0/2469629910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe28015e7e0 0x7fe2800d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27e800640 1 -- 172.21.15.86:0/2469629910 >> v1:172.21.15.86:6789/0 conn(0x7fe2800b4710 legacy=0x7fe28006fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27e800640 1 -- 172.21.15.86:0/2469629910 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2800d1310 con 0x7fe280070560 2024-03-22T04:11:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27ca00640 1 -- 172.21.15.86:0/2469629910 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898275058 0 0) 0x7fe2800d15f0 con 0x7fe2800b4710 2024-03-22T04:11:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27e800640 1 --2- 172.21.15.86:0/2469629910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800cd360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe278009870 tx=0x7fe27801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56e2a28716f8c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27ca00640 1 -- 172.21.15.86:0/2469629910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2780ab610 con 0x7fe280070560 2024-03-22T04:11:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27ca00640 1 -- 172.21.15.86:0/2469629910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2780b0070 con 0x7fe280070560 2024-03-22T04:11:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe27ca00640 1 -- 172.21.15.86:0/2469629910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2780ab9d0 con 0x7fe280070560 2024-03-22T04:11:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe284a00640 1 -- 172.21.15.86:0/2469629910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 msgr2=0x7fe2800cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe284a00640 1 --2- 172.21.15.86:0/2469629910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800cd360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe278009870 tx=0x7fe27801c410 comp rx=0 tx=0).stop 2024-03-22T04:11:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe284a00640 1 -- 172.21.15.86:0/2469629910 shutdown_connections 2024-03-22T04:11:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe284a00640 1 --2- 172.21.15.86:0/2469629910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800cd360 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe284a00640 1 --2- 172.21.15.86:0/2469629910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe28015e7e0 0x7fe2800d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.848+0000 7fe284a00640 1 -- 172.21.15.86:0/2469629910 >> 172.21.15.86:0/2469629910 conn(0x7fe2800b3bf0 msgr2=0x7fe2800b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 -- 172.21.15.86:0/2469629910 shutdown_connections 2024-03-22T04:11:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 -- 172.21.15.86:0/2469629910 wait complete. 2024-03-22T04:11:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 Processor -- start 2024-03-22T04:11:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 -- start start 2024-03-22T04:11:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2800b4710 0x7fe280155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe28015e7e0 0x7fe280154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800ce230 con 0x7fe280070560 2024-03-22T04:11:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800ce0b0 con 0x7fe28015e7e0 2024-03-22T04:11:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe2800ce3b0 con 0x7fe2800b4710 2024-03-22T04:11:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60198/0 (socket says 172.21.15.86:60198) 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 -- 172.21.15.86:0/1151872372 learned_addr learned my addr 172.21.15.86:0/1151872372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27d400640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2800b4710 0x7fe280155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27e800640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe28015e7e0 0x7fe280154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 -- 172.21.15.86:0/1151872372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2800b4710 msgr2=0x7fe280155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2800b4710 0x7fe280155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 -- 172.21.15.86:0/1151872372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe28015e7e0 msgr2=0x7fe280154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe28015e7e0 0x7fe280154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 -- 172.21.15.86:0/1151872372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe280155480 con 0x7fe280070560 2024-03-22T04:11:06.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe27de00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800c7770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe27000efc0 tx=0x7fe27000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe276000640 1 -- 172.21.15.86:0/1151872372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270018020 con 0x7fe280070560 2024-03-22T04:11:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe276000640 1 -- 172.21.15.86:0/1151872372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe270014480 con 0x7fe280070560 2024-03-22T04:11:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe276000640 1 -- 172.21.15.86:0/1151872372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270014850 con 0x7fe280070560 2024-03-22T04:11:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe280155dd0 con 0x7fe280070560 2024-03-22T04:11:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.852+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe280156390 con 0x7fe280070560 2024-03-22T04:11:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.856+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe280074470 con 0x7fe280070560 2024-03-22T04:11:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.856+0000 7fe276000640 1 -- 172.21.15.86:0/1151872372 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe27001b020 con 0x7fe280070560 2024-03-22T04:11:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.856+0000 7fe276000640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe260067b20 0x7fe260069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.856+0000 7fe276000640 1 -- 172.21.15.86:0/1151872372 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe270086ad0 con 0x7fe280070560 2024-03-22T04:11:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.856+0000 7fe27d400640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe260067b20 0x7fe260069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:06.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.856+0000 7fe27d400640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe260067b20 0x7fe260069fe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe2800c72f0 tx=0x7fe26c007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:06.860+0000 7fe276000640 1 -- 172.21.15.86:0/1151872372 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe270022070 con 0x7fe280070560 2024-03-22T04:11:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.020+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe28006fc30 con 0x7fe280070560 2024-03-22T04:11:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe276000640 1 -- 172.21.15.86:0/1151872372 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fe280155480 con 0x7fe280070560 2024-03-22T04:11:07.026 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe260067b20 msgr2=0x7fe260069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe260067b20 0x7fe260069fe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe2800c72f0 tx=0x7fe26c007d60 comp rx=0 tx=0).stop 2024-03-22T04:11:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 msgr2=0x7fe2800c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800c7770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe27000efc0 tx=0x7fe27000c490 comp rx=0 tx=0).stop 2024-03-22T04:11:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 shutdown_connections 2024-03-22T04:11:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe260067b20 0x7fe260069fe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe28015e7e0 0x7fe280154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2800b4710 0x7fe280155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 --2- 172.21.15.86:0/1151872372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe280070560 0x7fe2800c7770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 >> 172.21.15.86:0/1151872372 conn(0x7fe2800b3bf0 msgr2=0x7fe2800d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 shutdown_connections 2024-03-22T04:11:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.024+0000 7fe284a00640 1 -- 172.21.15.86:0/1151872372 wait complete. 2024-03-22T04:11:07.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:07.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.180+0000 7ff986200640 1 Processor -- start 2024-03-22T04:11:07.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.180+0000 7ff986200640 1 -- start start 2024-03-22T04:11:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff986200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 0x7ff9800d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff986200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9800b4710 0x7ff98006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff986200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9800d15f0 con 0x7ff980070390 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff986200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff9800b4b70 con 0x7ff9800b4710 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff986200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff98006f970 con 0x7ff98015e7e0 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff976a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9800b4710 0x7ff98006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff976a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9800b4710 0x7ff98006fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60204/0 (socket says 172.21.15.86:60204) 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff976a00640 1 -- 172.21.15.86:0/4261547062 learned_addr learned my addr 172.21.15.86:0/4261547062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff97f400640 1 --2- 172.21.15.86:0/4261547062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 0x7ff9800d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff976a00640 1 -- 172.21.15.86:0/4261547062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 msgr2=0x7ff9800d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff976a00640 1 --2- 172.21.15.86:0/4261547062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 0x7ff9800d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff976a00640 1 -- 172.21.15.86:0/4261547062 >> v1:172.21.15.196:6789/0 conn(0x7ff980070390 legacy=0x7ff98009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.184+0000 7ff976a00640 1 -- 172.21.15.86:0/4261547062 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff980070820 con 0x7ff9800b4710 2024-03-22T04:11:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff976a00640 1 --2- 172.21.15.86:0/4261547062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9800b4710 0x7ff98006fe50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff968009870 tx=0x7ff96801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88a0eb83d741970 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff97ea00640 1 -- 172.21.15.86:0/4261547062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9680b0070 con 0x7ff9800b4710 2024-03-22T04:11:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff97ea00640 1 -- 172.21.15.86:0/4261547062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9680ab600 con 0x7ff9800b4710 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff97ea00640 1 -- 172.21.15.86:0/4261547062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9680ab9d0 con 0x7ff9800b4710 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 -- 172.21.15.86:0/4261547062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9800b4710 msgr2=0x7ff98006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 --2- 172.21.15.86:0/4261547062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9800b4710 0x7ff98006fe50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff968009870 tx=0x7ff96801c410 comp rx=0 tx=0).stop 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 -- 172.21.15.86:0/4261547062 shutdown_connections 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 --2- 172.21.15.86:0/4261547062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9800b4710 0x7ff98006fe50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 --2- 172.21.15.86:0/4261547062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 0x7ff9800d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 -- 172.21.15.86:0/4261547062 >> 172.21.15.86:0/4261547062 conn(0x7ff9800b3bf0 msgr2=0x7ff9800b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 -- 172.21.15.86:0/4261547062 shutdown_connections 2024-03-22T04:11:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 -- 172.21.15.86:0/4261547062 wait complete. 2024-03-22T04:11:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 Processor -- start 2024-03-22T04:11:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.188+0000 7ff986200640 1 -- start start 2024-03-22T04:11:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff980070390 0x7ff9801574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9800b4710 0x7ff9800c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 0x7ff9800b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800ce370 con 0x7ff980070390 2024-03-22T04:11:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800cce40 con 0x7ff9800b4710 2024-03-22T04:11:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff9800ce4f0 con 0x7ff98015e7e0 2024-03-22T04:11:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff980070390 0x7ff9801574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff980070390 0x7ff9801574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60212/0 (socket says 172.21.15.86:60212) 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 -- 172.21.15.86:0/3312643831 learned_addr learned my addr 172.21.15.86:0/3312643831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff976a00640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9800b4710 0x7ff9800c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97fe00640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 0x7ff9800b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 -- 172.21.15.86:0/3312643831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 msgr2=0x7ff9800b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 0x7ff9800b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 -- 172.21.15.86:0/3312643831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9800b4710 msgr2=0x7ff9800c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9800b4710 0x7ff9800c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 -- 172.21.15.86:0/3312643831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9800ca410 con 0x7ff980070390 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff97f400640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff980070390 0x7ff9801574b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff97000d830 tx=0x7ff97000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff977e00640 1 -- 172.21.15.86:0/3312643831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97000b6c0 con 0x7ff980070390 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff977e00640 1 -- 172.21.15.86:0/3312643831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff970019070 con 0x7ff980070390 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9800ca6f0 con 0x7ff980070390 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff977e00640 1 -- 172.21.15.86:0/3312643831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97000ba60 con 0x7ff980070390 2024-03-22T04:11:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff980158820 con 0x7ff980070390 2024-03-22T04:11:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff980074470 con 0x7ff980070390 2024-03-22T04:11:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff977e00640 1 -- 172.21.15.86:0/3312643831 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff97001e460 con 0x7ff980070390 2024-03-22T04:11:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.192+0000 7ff977e00640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff95c067b50 0x7ff95c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.196+0000 7ff977e00640 1 -- 172.21.15.86:0/3312643831 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff970086860 con 0x7ff980070390 2024-03-22T04:11:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.196+0000 7ff976a00640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff95c067b50 0x7ff95c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.196+0000 7ff976a00640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff95c067b50 0x7ff95c06a010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff9680ad000 tx=0x7ff968008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.200+0000 7ff977e00640 1 -- 172.21.15.86:0/3312643831 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff970056f20 con 0x7ff980070390 2024-03-22T04:11:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.352+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff98006fc30 con 0x7ff980070390 2024-03-22T04:11:07.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff977e00640 1 -- 172.21.15.86:0/3312643831 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff970056820 con 0x7ff980070390 2024-03-22T04:11:07.358 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff95c067b50 msgr2=0x7ff95c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff95c067b50 0x7ff95c06a010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff9680ad000 tx=0x7ff968008040 comp rx=0 tx=0).stop 2024-03-22T04:11:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff980070390 msgr2=0x7ff9801574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff980070390 0x7ff9801574b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff97000d830 tx=0x7ff97000dcf0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 shutdown_connections 2024-03-22T04:11:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff95c067b50 0x7ff95c06a010 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9800b4710 0x7ff9800c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98015e7e0 0x7ff9800b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 --2- 172.21.15.86:0/3312643831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff980070390 0x7ff9801574b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 >> 172.21.15.86:0/3312643831 conn(0x7ff9800b3bf0 msgr2=0x7ff9800b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 shutdown_connections 2024-03-22T04:11:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.356+0000 7ff986200640 1 -- 172.21.15.86:0/3312643831 wait complete. 2024-03-22T04:11:07.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:11:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.508+0000 7fc3b2200640 1 Processor -- start 2024-03-22T04:11:07.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.508+0000 7fc3b2200640 1 -- start start 2024-03-22T04:11:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3b2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3ac147880 0x7fc3ac06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3b2200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc3ac0b4a60 0x7fc3ac15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3b2200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3ac15ea90 con 0x7fc3ac06cec0 2024-03-22T04:11:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3b2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc3ac0b6320 con 0x7fc3ac147880 2024-03-22T04:11:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3b2200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc3ac06d4f0 con 0x7fc3ac0b4a60 2024-03-22T04:11:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3ab400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3ac147880 0x7fc3ac06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3a2a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc3ac06cec0 0x7fc3ac0b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50054/0 (socket says 172.21.15.86:50054) 2024-03-22T04:11:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3a2a00640 1 -- 172.21.15.86:0/152755578 learned_addr learned my addr 172.21.15.86:0/152755578 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3abe00640 1 --2- 172.21.15.86:0/152755578 >> v2:172.21.15.196:3300/0 conn(0x7fc3ac0b4a60 0x7fc3ac15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3ab400640 1 -- 172.21.15.86:0/152755578 >> v2:172.21.15.196:3300/0 conn(0x7fc3ac0b4a60 msgr2=0x7fc3ac15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3ab400640 1 --2- 172.21.15.86:0/152755578 >> v2:172.21.15.196:3300/0 conn(0x7fc3ac0b4a60 0x7fc3ac15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3ab400640 1 -- 172.21.15.86:0/152755578 >> v1:172.21.15.86:6789/0 conn(0x7fc3ac06cec0 legacy=0x7fc3ac0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.512+0000 7fc3ab400640 1 -- 172.21.15.86:0/152755578 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac06d7d0 con 0x7fc3ac147880 2024-03-22T04:11:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3ab400640 1 --2- 172.21.15.86:0/152755578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac147880 0x7fc3ac06c980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009d90 tx=0x7fc39c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8d0b710ba334fc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3aaa00640 1 -- 172.21.15.86:0/152755578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0b2070 con 0x7fc3ac147880 2024-03-22T04:11:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3aaa00640 1 -- 172.21.15.86:0/152755578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39c0a57f0 con 0x7fc3ac147880 2024-03-22T04:11:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3aaa00640 1 -- 172.21.15.86:0/152755578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0a5bc0 con 0x7fc3ac147880 2024-03-22T04:11:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- 172.21.15.86:0/152755578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac147880 msgr2=0x7fc3ac06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 --2- 172.21.15.86:0/152755578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac147880 0x7fc3ac06c980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009d90 tx=0x7fc39c01c930 comp rx=0 tx=0).stop 2024-03-22T04:11:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- 172.21.15.86:0/152755578 shutdown_connections 2024-03-22T04:11:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 --2- 172.21.15.86:0/152755578 >> v2:172.21.15.196:3300/0 conn(0x7fc3ac0b4a60 0x7fc3ac15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 --2- 172.21.15.86:0/152755578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac147880 0x7fc3ac06c980 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- 172.21.15.86:0/152755578 >> 172.21.15.86:0/152755578 conn(0x7fc3ac06ba60 msgr2=0x7fc3ac06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- 172.21.15.86:0/152755578 shutdown_connections 2024-03-22T04:11:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- 172.21.15.86:0/152755578 wait complete. 2024-03-22T04:11:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 Processor -- start 2024-03-22T04:11:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- start start 2024-03-22T04:11:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac06cec0 0x7fc3ac0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3ac0b4a60 0x7fc3ac15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3ac147880 0x7fc3ac1639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac15fe10 con 0x7fc3ac06cec0 2024-03-22T04:11:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac15fc90 con 0x7fc3ac0b4a60 2024-03-22T04:11:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3b2200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ac15ff90 con 0x7fc3ac147880 2024-03-22T04:11:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac06cec0 0x7fc3ac0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac06cec0 0x7fc3ac0cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60230/0 (socket says 172.21.15.86:60230) 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.516+0000 7fc3ab400640 1 -- 172.21.15.86:0/1345447785 learned_addr learned my addr 172.21.15.86:0/1345447785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3abe00640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3ac147880 0x7fc3ac1639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a2a00640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3ac0b4a60 0x7fc3ac15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3ab400640 1 -- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3ac147880 msgr2=0x7fc3ac1639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3ab400640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3ac147880 0x7fc3ac1639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3ab400640 1 -- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3ac0b4a60 msgr2=0x7fc3ac15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3ab400640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3ac0b4a60 0x7fc3ac15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3ab400640 1 -- 172.21.15.86:0/1345447785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac165d00 con 0x7fc3ac06cec0 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a2a00640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3ac0b4a60 0x7fc3ac15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3ab400640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac06cec0 0x7fc3ac0cb5d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0b0040 tx=0x7fc39c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a3e00640 1 -- 172.21.15.86:0/1345447785 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0bd020 con 0x7fc3ac06cec0 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a3e00640 1 -- 172.21.15.86:0/1345447785 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39c0b8470 con 0x7fc3ac06cec0 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a3e00640 1 -- 172.21.15.86:0/1345447785 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0b2040 con 0x7fc3ac06cec0 2024-03-22T04:11:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac164550 con 0x7fc3ac06cec0 2024-03-22T04:11:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac144fb0 con 0x7fc3ac06cec0 2024-03-22T04:11:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac068650 con 0x7fc3ac06cec0 2024-03-22T04:11:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a3e00640 1 -- 172.21.15.86:0/1345447785 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc39c0c0020 con 0x7fc3ac06cec0 2024-03-22T04:11:07.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a3e00640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc388067b20 0x7fc388069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a3e00640 1 -- 172.21.15.86:0/1345447785 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc39c120fe0 con 0x7fc3ac06cec0 2024-03-22T04:11:07.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.520+0000 7fc3a2a00640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc388067b20 0x7fc388069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.524+0000 7fc3a2a00640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc388067b20 0x7fc388069fe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac147330 tx=0x7fc394009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.524+0000 7fc3a3e00640 1 -- 172.21.15.86:0/1345447785 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc39c0c7030 con 0x7fc3ac06cec0 2024-03-22T04:11:07.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3ac072220 con 0x7fc3ac06cec0 2024-03-22T04:11:07.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3a3e00640 1 -- 172.21.15.86:0/1345447785 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5179 (secure 0 0 0) 0x7fc39c016520 con 0x7fc3ac06cec0 2024-03-22T04:11:07.716 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:07.716 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:58.258965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:11:07.716 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc388067b20 msgr2=0x7fc388069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc388067b20 0x7fc388069fe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac147330 tx=0x7fc394009040 comp rx=0 tx=0).stop 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac06cec0 msgr2=0x7fc3ac0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac06cec0 0x7fc3ac0cb5d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0b0040 tx=0x7fc39c0196d0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 shutdown_connections 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc388067b20 0x7fc388069fe0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3ac0b4a60 0x7fc3ac15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3ac147880 0x7fc3ac1639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 --2- 172.21.15.86:0/1345447785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3ac06cec0 0x7fc3ac0cb5d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 >> 172.21.15.86:0/1345447785 conn(0x7fc3ac06ba60 msgr2=0x7fc3ac0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 shutdown_connections 2024-03-22T04:11:07.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.712+0000 7fc3b2200640 1 -- 172.21.15.86:0/1345447785 wait complete. 2024-03-22T04:11:07.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.868+0000 7f1bcd600640 1 Processor -- start 2024-03-22T04:11:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.868+0000 7f1bcd600640 1 -- start start 2024-03-22T04:11:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bcd600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1bc80b4350 0x7f1bc80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bcd600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc815ea90 con 0x7f1bc815ef40 2024-03-22T04:11:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bcd600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc806d4f0 con 0x7f1bc8147880 2024-03-22T04:11:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bcd600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc80b6320 con 0x7f1bc80b4350 2024-03-22T04:11:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bc7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1bc815ef40 0x7f1bc815f390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50056/0 (socket says 172.21.15.86:50056) 2024-03-22T04:11:07.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bc7400640 1 -- 172.21.15.86:0/419865176 learned_addr learned my addr 172.21.15.86:0/419865176 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:07.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bbe000640 1 --2- 172.21.15.86:0/419865176 >> v2:172.21.15.196:3300/0 conn(0x7f1bc80b4350 0x7f1bc80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bc6000640 1 -- 172.21.15.86:0/419865176 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3238939723 0 0) 0x7f1bc815ea90 con 0x7f1bc815ef40 2024-03-22T04:11:07.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bc6000640 1 -- 172.21.15.86:0/419865176 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bac003530 con 0x7f1bc815ef40 2024-03-22T04:11:07.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bc6000640 1 -- 172.21.15.86:0/419865176 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1176451008 0 0) 0x7f1bc806d4f0 con 0x7f1bc8147880 2024-03-22T04:11:07.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bc6000640 1 -- 172.21.15.86:0/419865176 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bc815ea90 con 0x7f1bc8147880 2024-03-22T04:11:07.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bbe000640 1 -- 172.21.15.86:0/419865176 >> v1:172.21.15.196:6789/0 conn(0x7f1bc8147880 legacy=0x7f1bc806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bbe000640 1 -- 172.21.15.86:0/419865176 >> v1:172.21.15.86:6789/0 conn(0x7f1bc815ef40 legacy=0x7f1bc815f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.872+0000 7f1bbe000640 1 -- 172.21.15.86:0/419865176 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc806d7d0 con 0x7f1bc80b4350 2024-03-22T04:11:07.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bbe000640 1 --2- 172.21.15.86:0/419865176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc80b4350 0x7f1bc80b47e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009dc0 tx=0x7f1bb001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=961d32d25574cddf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bc6000640 1 -- 172.21.15.86:0/419865176 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00b2070 con 0x7f1bc80b4350 2024-03-22T04:11:07.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bc6000640 1 -- 172.21.15.86:0/419865176 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb00a5810 con 0x7f1bc80b4350 2024-03-22T04:11:07.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bc6000640 1 -- 172.21.15.86:0/419865176 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00a5be0 con 0x7f1bc80b4350 2024-03-22T04:11:07.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- 172.21.15.86:0/419865176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc80b4350 msgr2=0x7f1bc80b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 --2- 172.21.15.86:0/419865176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc80b4350 0x7f1bc80b47e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009dc0 tx=0x7f1bb001c950 comp rx=0 tx=0).stop 2024-03-22T04:11:07.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- 172.21.15.86:0/419865176 shutdown_connections 2024-03-22T04:11:07.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 --2- 172.21.15.86:0/419865176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc80b4350 0x7f1bc80b47e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- 172.21.15.86:0/419865176 >> 172.21.15.86:0/419865176 conn(0x7f1bc806ba60 msgr2=0x7f1bc806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:07.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- 172.21.15.86:0/419865176 shutdown_connections 2024-03-22T04:11:07.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- 172.21.15.86:0/419865176 wait complete. 2024-03-22T04:11:07.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 Processor -- start 2024-03-22T04:11:07.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- start start 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b4350 0x7f1bc8146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc8147880 0x7f1bc813be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc815ef40 0x7f1bc8140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc806d210 con 0x7f1bc8147880 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc806d090 con 0x7f1bc815ef40 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bcd600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc816c430 con 0x7f1bc80b4350 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bbe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc8147880 0x7f1bc813be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.876+0000 7f1bc6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b4350 0x7f1bc8146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc8147880 0x7f1bc813be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60238/0 (socket says 172.21.15.86:60238) 2024-03-22T04:11:07.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbe000640 1 -- 172.21.15.86:0/2220285734 learned_addr learned my addr 172.21.15.86:0/2220285734 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bc6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b4350 0x7f1bc8146f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42672/0 (socket says 172.21.15.86:42672) 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bc7400640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc815ef40 0x7f1bc8140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbe000640 1 -- 172.21.15.86:0/2220285734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b4350 msgr2=0x7f1bc8146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbe000640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b4350 0x7f1bc8146f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbe000640 1 -- 172.21.15.86:0/2220285734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc815ef40 msgr2=0x7f1bc8140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbe000640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc815ef40 0x7f1bc8140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbe000640 1 -- 172.21.15.86:0/2220285734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc8147440 con 0x7f1bc8147880 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbe000640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc8147880 0x7f1bc813be30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0019560 tx=0x7f1bb00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbf400640 1 -- 172.21.15.86:0/2220285734 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00b2050 con 0x7f1bc8147880 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbf400640 1 -- 172.21.15.86:0/2220285734 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb00b3040 con 0x7f1bc8147880 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbf400640 1 -- 172.21.15.86:0/2220285734 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00ad650 con 0x7f1bc8147880 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc8145ec0 con 0x7f1bc8147880 2024-03-22T04:11:07.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc8146700 con 0x7f1bc8147880 2024-03-22T04:11:07.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc8146250 con 0x7f1bc8147880 2024-03-22T04:11:07.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbf400640 1 -- 172.21.15.86:0/2220285734 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1bb00c0020 con 0x7f1bc8147880 2024-03-22T04:11:07.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbf400640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ba4067b20 0x7f1ba4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bbf400640 1 -- 172.21.15.86:0/2220285734 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1bb0120680 con 0x7f1bc8147880 2024-03-22T04:11:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.880+0000 7f1bc6a00640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ba4067b20 0x7f1ba4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.884+0000 7f1bc6a00640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ba4067b20 0x7f1ba4069fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80b5610 tx=0x7f1bb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:07.884+0000 7f1bbf400640 1 -- 172.21.15.86:0/2220285734 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1bb00c7020 con 0x7f1bc8147880 2024-03-22T04:11:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.044+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bc80b1aa0 con 0x7f1bc8147880 2024-03-22T04:11:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.044+0000 7f1bbf400640 1 -- 172.21.15.86:0/2220285734 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18518 (secure 0 0 0) 0x7f1bc8145ec0 con 0x7f1bc8147880 2024-03-22T04:11:08.049 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:08.050 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:10:39.344945+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ba4067b20 msgr2=0x7f1ba4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ba4067b20 0x7f1ba4069fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80b5610 tx=0x7f1bb8009040 comp rx=0 tx=0).stop 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc8147880 msgr2=0x7f1bc813be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc8147880 0x7f1bc813be30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0019560 tx=0x7f1bb00196f0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 shutdown_connections 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ba4067b20 0x7f1ba4069fe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bc815ef40 0x7f1bc8140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc8147880 0x7f1bc813be30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 --2- 172.21.15.86:0/2220285734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b4350 0x7f1bc8146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 >> 172.21.15.86:0/2220285734 conn(0x7f1bc806ba60 msgr2=0x7f1bc80b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 shutdown_connections 2024-03-22T04:11:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.048+0000 7f1bcd600640 1 -- 172.21.15.86:0/2220285734 wait complete. 2024-03-22T04:11:08.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:11:08.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.200+0000 7fa591a00640 1 Processor -- start 2024-03-22T04:11:08.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa591a00640 1 -- start start 2024-03-22T04:11:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa591a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa58c0704a0 0x7fa58c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa591a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa58c162740 con 0x7fa58c06be40 2024-03-22T04:11:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa591a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa58c162a20 con 0x7fa58c0700c0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa591a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa58c162d00 con 0x7fa58c0704a0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa58b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa58c06be40 0x7fa58c0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50070/0 (socket says 172.21.15.86:50070) 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa58b400640 1 -- 172.21.15.86:0/1294897251 learned_addr learned my addr 172.21.15.86:0/1294897251 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa582000640 1 --2- 172.21.15.86:0/1294897251 >> v2:172.21.15.196:3300/0 conn(0x7fa58c0704a0 0x7fa58c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa58a000640 1 -- 172.21.15.86:0/1294897251 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996029910 0 0) 0x7fa58c162740 con 0x7fa58c06be40 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa58a000640 1 -- 172.21.15.86:0/1294897251 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa570003530 con 0x7fa58c06be40 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa58a000640 1 -- 172.21.15.86:0/1294897251 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609279504 0 0) 0x7fa58c162a20 con 0x7fa58c0700c0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa58a000640 1 -- 172.21.15.86:0/1294897251 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa58c162740 con 0x7fa58c0700c0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa582000640 1 -- 172.21.15.86:0/1294897251 >> v1:172.21.15.196:6789/0 conn(0x7fa58c0700c0 legacy=0x7fa58c072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa582000640 1 -- 172.21.15.86:0/1294897251 >> v1:172.21.15.86:6789/0 conn(0x7fa58c06be40 legacy=0x7fa58c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.204+0000 7fa582000640 1 -- 172.21.15.86:0/1294897251 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c162fe0 con 0x7fa58c0704a0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa582000640 1 --2- 172.21.15.86:0/1294897251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c0704a0 0x7fa58c06b900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5740097c0 tx=0x7fa57401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2eebd9be42d3ae server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa58a000640 1 -- 172.21.15.86:0/1294897251 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740b0070 con 0x7fa58c0704a0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa58a000640 1 -- 172.21.15.86:0/1294897251 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5740ab600 con 0x7fa58c0704a0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa58a000640 1 -- 172.21.15.86:0/1294897251 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740ab9d0 con 0x7fa58c0704a0 2024-03-22T04:11:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 -- 172.21.15.86:0/1294897251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c0704a0 msgr2=0x7fa58c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 --2- 172.21.15.86:0/1294897251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c0704a0 0x7fa58c06b900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5740097c0 tx=0x7fa57401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 -- 172.21.15.86:0/1294897251 shutdown_connections 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 --2- 172.21.15.86:0/1294897251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c0704a0 0x7fa58c06b900 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 -- 172.21.15.86:0/1294897251 >> 172.21.15.86:0/1294897251 conn(0x7fa58c0b2d00 msgr2=0x7fa58c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 -- 172.21.15.86:0/1294897251 shutdown_connections 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 -- 172.21.15.86:0/1294897251 wait complete. 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 Processor -- start 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.208+0000 7fa591a00640 1 -- start start 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c06be40 0x7fa58c1451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c0700c0 0x7fa58c0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa58c0704a0 0x7fa58c0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c15f820 con 0x7fa58c0700c0 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c0d0b60 con 0x7fa58c06be40 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa58c15f9a0 con 0x7fa58c0704a0 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c0700c0 0x7fa58c0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c0700c0 0x7fa58c0c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60252/0 (socket says 172.21.15.86:60252) 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 -- 172.21.15.86:0/4140821822 learned_addr learned my addr 172.21.15.86:0/4140821822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa58aa00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c06be40 0x7fa58c1451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa58b400640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa58c0704a0 0x7fa58c0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 -- 172.21.15.86:0/4140821822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa58c0704a0 msgr2=0x7fa58c0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa58c0704a0 0x7fa58c0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 -- 172.21.15.86:0/4140821822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c06be40 msgr2=0x7fa58c1451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c06be40 0x7fa58c1451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 -- 172.21.15.86:0/4140821822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c0c95e0 con 0x7fa58c0700c0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa582000640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c0700c0 0x7fa58c0c90a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5740097c0 tx=0x7fa574019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa583400640 1 -- 172.21.15.86:0/4140821822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740b0040 con 0x7fa58c0700c0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa583400640 1 -- 172.21.15.86:0/4140821822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5740bc070 con 0x7fa58c0700c0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c0c9880 con 0x7fa58c0700c0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa583400640 1 -- 172.21.15.86:0/4140821822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740b5890 con 0x7fa58c0700c0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58c0b5380 con 0x7fa58c0700c0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa58c068650 con 0x7fa58c0700c0 2024-03-22T04:11:08.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.212+0000 7fa583400640 1 -- 172.21.15.86:0/4140821822 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa5740b3070 con 0x7fa58c0700c0 2024-03-22T04:11:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.216+0000 7fa583400640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa568067b20 0x7fa568069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.216+0000 7fa583400640 1 -- 172.21.15.86:0/4140821822 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa574120500 con 0x7fa58c0700c0 2024-03-22T04:11:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.216+0000 7fa58aa00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa568067b20 0x7fa568069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.216+0000 7fa58aa00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa568067b20 0x7fa568069fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa57c007fd0 tx=0x7fa57c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:08.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.220+0000 7fa583400640 1 -- 172.21.15.86:0/4140821822 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa574016090 con 0x7fa58c0700c0 2024-03-22T04:11:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.404+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa58c06d830 con 0x7fa58c0700c0 2024-03-22T04:11:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.404+0000 7fa583400640 1 -- 172.21.15.86:0/4140821822 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5179 (secure 0 0 0) 0x7fa5740f0b30 con 0x7fa58c0700c0 2024-03-22T04:11:08.408 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:08.409 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:58.258965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:11:08.409 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T04:11:08.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa568067b20 msgr2=0x7fa568069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa568067b20 0x7fa568069fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa57c007fd0 tx=0x7fa57c009040 comp rx=0 tx=0).stop 2024-03-22T04:11:08.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c0700c0 msgr2=0x7fa58c0c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c0700c0 0x7fa58c0c90a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5740097c0 tx=0x7fa574019240 comp rx=0 tx=0).stop 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 shutdown_connections 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa568067b20 0x7fa568069fe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa58c0704a0 0x7fa58c0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c0700c0 0x7fa58c0c90a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 --2- 172.21.15.86:0/4140821822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58c06be40 0x7fa58c1451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 >> 172.21.15.86:0/4140821822 conn(0x7fa58c0b2d00 msgr2=0x7fa58c15e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 shutdown_connections 2024-03-22T04:11:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.408+0000 7fa591a00640 1 -- 172.21.15.86:0/4140821822 wait complete. 2024-03-22T04:11:08.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:10:34.298556+0000', 'modified': '2024-03-22T04:10:58.258965+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 74171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_74171': {'gid': 74171, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/180015071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6833', 'nonce': 180015071}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 180015071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 74171, 'qdb_cluster': [74171]} 2024-03-22T04:11:08.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:11:08.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T04:11:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f4576a00640 1 Processor -- start 2024-03-22T04:11:08.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f4576a00640 1 -- start start 2024-03-22T04:11:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f4576a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f4576a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f457015fed0 0x7f4570160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f4576a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45700d11c0 con 0x7f45700d0de0 2024-03-22T04:11:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f4576a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4570160860 con 0x7f457015fed0 2024-03-22T04:11:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f4576a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f45700cece0 con 0x7f45700ce6b0 2024-03-22T04:11:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f456fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f45700d0de0 0x7f45700d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50072/0 (socket says 172.21.15.86:50072) 2024-03-22T04:11:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f456fe00640 1 -- 172.21.15.86:0/4206744272 learned_addr learned my addr 172.21.15.86:0/4206744272 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f4574a00640 1 --2- 172.21.15.86:0/4206744272 >> v2:172.21.15.196:3300/0 conn(0x7f457015fed0 0x7f4570160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f456f400640 1 --2- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4574a00640 1 -- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 msgr2=0x7f45700cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4574a00640 1 --2- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.968+0000 7f456ea00640 1 -- 172.21.15.86:0/4206744272 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 229302878 0 0) 0x7f45700d11c0 con 0x7f45700d0de0 2024-03-22T04:11:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4574a00640 1 -- 172.21.15.86:0/4206744272 >> v1:172.21.15.86:6789/0 conn(0x7f45700d0de0 legacy=0x7f45700d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4574a00640 1 -- 172.21.15.86:0/4206744272 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45700cefc0 con 0x7f457015fed0 2024-03-22T04:11:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f456f400640 1 --2- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4574a00640 1 --2- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 0x7f4570160320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f45680097c0 tx=0x7f456801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad793ee0303b8be1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f456ea00640 1 -- 172.21.15.86:0/4206744272 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680b0070 con 0x7f457015fed0 2024-03-22T04:11:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f456ea00640 1 -- 172.21.15.86:0/4206744272 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45680ab600 con 0x7f457015fed0 2024-03-22T04:11:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f456ea00640 1 -- 172.21.15.86:0/4206744272 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680ab9d0 con 0x7f457015fed0 2024-03-22T04:11:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4576a00640 1 -- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 msgr2=0x7f4570160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4576a00640 1 --2- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 0x7f4570160320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f45680097c0 tx=0x7f456801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4576a00640 1 -- 172.21.15.86:0/4206744272 shutdown_connections 2024-03-22T04:11:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4576a00640 1 --2- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 0x7f4570160320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4576a00640 1 --2- 172.21.15.86:0/4206744272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4576a00640 1 -- 172.21.15.86:0/4206744272 >> 172.21.15.86:0/4206744272 conn(0x7f457006f910 msgr2=0x7f457006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4576a00640 1 -- 172.21.15.86:0/4206744272 shutdown_connections 2024-03-22T04:11:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.972+0000 7f4576a00640 1 -- 172.21.15.86:0/4206744272 wait complete. 2024-03-22T04:11:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 Processor -- start 2024-03-22T04:11:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 -- start start 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45700d0de0 0x7f45700b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 0x7f45700b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45700d0530 con 0x7f45700d0de0 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45700d03b0 con 0x7f457015fed0 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4570071e30 con 0x7f45700ce6b0 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45700d0de0 0x7f45700b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45700d0de0 0x7f45700b61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60260/0 (socket says 172.21.15.86:60260) 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 -- 172.21.15.86:0/1491653438 learned_addr learned my addr 172.21.15.86:0/1491653438 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4574a00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 0x7f45700b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 -- 172.21.15.86:0/1491653438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 msgr2=0x7f45700c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 -- 172.21.15.86:0/1491653438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 msgr2=0x7f45700b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 0x7f45700b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 -- 172.21.15.86:0/1491653438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45700cc130 con 0x7f45700d0de0 2024-03-22T04:11:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4574a00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 0x7f45700b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:08.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f456f400640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45700d0de0 0x7f45700b61f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f455c00e980 tx=0x7f455c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:08.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4567e00640 1 -- 172.21.15.86:0/1491653438 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c017070 con 0x7f45700d0de0 2024-03-22T04:11:08.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4567e00640 1 -- 172.21.15.86:0/1491653438 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c00ca00 con 0x7f45700d0de0 2024-03-22T04:11:08.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 -- 172.21.15.86:0/1491653438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45700b6d60 con 0x7f45700d0de0 2024-03-22T04:11:08.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4567e00640 1 -- 172.21.15.86:0/1491653438 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c00cdd0 con 0x7f45700d0de0 2024-03-22T04:11:08.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.976+0000 7f4576a00640 1 -- 172.21.15.86:0/1491653438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45700b7320 con 0x7f45700d0de0 2024-03-22T04:11:08.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.980+0000 7f4567e00640 1 -- 172.21.15.86:0/1491653438 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f455c01a070 con 0x7f45700d0de0 2024-03-22T04:11:08.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.980+0000 7f4567e00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4550067b20 0x7f4550069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.980+0000 7f4567e00640 1 -- 172.21.15.86:0/1491653438 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f455c0860f0 con 0x7f45700d0de0 2024-03-22T04:11:08.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.980+0000 7f456fe00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4550067b20 0x7f4550069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.980+0000 7f4579f14000 1 Processor -- start 2024-03-22T04:11:08.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.980+0000 7f4579f14000 1 -- start start 2024-03-22T04:11:08.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.980+0000 7f456fe00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4550067b20 0x7f4550069fe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f45600012f0 tx=0x7f4560001770 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:08.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd59e70 0x55d3dfd5a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 0x55d3dfd0d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd0e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3dfb0b140 con 0x55d3dfd0dc20 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3dfb5eb10 con 0x55d3dfd5a790 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d3dfae5c50 con 0x55d3dfd59e70 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd0e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60264/0 (socket says 172.21.15.86:60264) 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 -- 172.21.15.86:0/3236723825 learned_addr learned my addr 172.21.15.86:0/3236723825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f456fe00640 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd59e70 0x55d3dfd5a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f456f400640 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 0x55d3dfd0d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 -- 172.21.15.86:0/3236723825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd59e70 msgr2=0x55d3dfd5a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd59e70 0x55d3dfd5a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 -- 172.21.15.86:0/3236723825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 msgr2=0x55d3dfd0d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 0x55d3dfd0d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 -- 172.21.15.86:0/3236723825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3dfba3f20 con 0x55d3dfd0dc20 2024-03-22T04:11:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4574a00640 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd0e000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f456801ab50 tx=0x7f45680a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b4f7fb2137e721 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:08.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4564c00640 1 -- 172.21.15.86:0/3236723825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680ba070 con 0x55d3dfd0dc20 2024-03-22T04:11:08.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4564c00640 1 -- 172.21.15.86:0/3236723825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45680abd60 con 0x55d3dfd0dc20 2024-03-22T04:11:08.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4564c00640 1 -- 172.21.15.86:0/3236723825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680b0040 con 0x55d3dfd0dc20 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- 172.21.15.86:0/3236723825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 msgr2=0x55d3dfd0e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd0e000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f456801ab50 tx=0x7f45680a6000 comp rx=0 tx=0).stop 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- 172.21.15.86:0/3236723825 shutdown_connections 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 0x55d3dfd0d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd0e000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 --2- 172.21.15.86:0/3236723825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd59e70 0x55d3dfd5a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- 172.21.15.86:0/3236723825 >> 172.21.15.86:0/3236723825 conn(0x55d3dfd3ac10 msgr2=0x55d3dfd3d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- 172.21.15.86:0/3236723825 shutdown_connections 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- 172.21.15.86:0/3236723825 wait complete. 2024-03-22T04:11:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 -- start start 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.984+0000 7f4579f14000 1 Processor -- start 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 0x55d3dfd127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd59e70 0x55d3dfd12d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 0x55d3dfd13260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3dfb18ab0 con 0x55d3dfd59e70 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3dfb5aac0 con 0x55d3dfd5a790 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d3dfc1ab30 con 0x55d3dfd0dc20 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 10 client.0 _ll_register_callbacks cb 0x55d3dfcfb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd59e70 0x55d3dfd12d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd59e70 0x55d3dfd12d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60268/0 (socket says 172.21.15.86:60268) 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 -- 172.21.15.86:0/3341012524 learned_addr learned my addr 172.21.15.86:0/3341012524 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456fe00640 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 0x55d3dfd127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 -- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 msgr2=0x55d3dfd127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 0x55d3dfd127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 -- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 msgr2=0x55d3dfd13260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 0x55d3dfd13260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 -- 172.21.15.86:0/3341012524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3dfb34570 con 0x55d3dfd59e70 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456f400640 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd59e70 0x55d3dfd12d20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f455c0525c0 tx=0x7f455c05a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5162da909c6c7959 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 1 -- 172.21.15.86:0/3341012524 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c021040 con 0x55d3dfd59e70 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 1 -- 172.21.15.86:0/3341012524 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f455c054460 con 0x55d3dfd59e70 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- 172.21.15.86:0/3341012524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45680abd60 con 0x55d3dfd59e70 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 1 -- 172.21.15.86:0/3341012524 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f455c0555b0 con 0x55d3dfd59e70 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 10 client.4791 fetch_fsmap learned FSMap version 9 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- 172.21.15.86:0/3341012524 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d3dfb2a880 con 0x55d3dfd59e70 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 1 -- 172.21.15.86:0/3341012524 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45680abd60 con 0x55d3dfd59e70 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 1 -- 172.21.15.86:0/3341012524 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f455c024060 con 0x55d3dfd59e70 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 10 client.4791 fetch_fsmap finished waiting for FSMap version 9 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 10 client.4791 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 20 client.4791 populate_metadata read hostname 'smithi086' 2024-03-22T04:11:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55d3dfd6d150 0x55d3dfd6f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 4 client.4791 mds_command: new command op to 74171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:11:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- 172.21.15.86:0/3341012524 --> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3dfaf9bc0 con 0x55d3dfd6d150 2024-03-22T04:11:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456fe00640 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55d3dfd6d150 0x55d3dfd6f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f456fe00640 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55d3dfd6d150 0x55d3dfd6f540 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ceb58ff4a94e4dc server_cookie=af5aee7cb855668c in_seq=0 out_seq=0 2024-03-22T04:11:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 10 client.4791 ms_handle_connect on v2:172.21.15.196:6833/180015071 2024-03-22T04:11:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 1 -- 172.21.15.86:0/3341012524 <== mds.0 v2:172.21.15.196:6833/180015071 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d3dfaf9bc0 con 0x55d3dfd6d150 2024-03-22T04:11:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 10 client.4791 handle_command_reply: tid=0 2024-03-22T04:11:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 client.4791 shutdown 2024-03-22T04:11:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- 172.21.15.86:0/3341012524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd59e70 msgr2=0x55d3dfd12d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd59e70 0x55d3dfd12d20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f455c0525c0 tx=0x7f455c05a200 comp rx=0 tx=0).stop 2024-03-22T04:11:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- 172.21.15.86:0/3341012524 shutdown_connections 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55d3dfd6d150 0x55d3dfd6f540 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd5a790 0x55d3dfd13260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd59e70 0x55d3dfd12d20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4559600640 0 client.4791 ms_handle_reset on v2:172.21.15.196:6833/180015071 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 --2- 172.21.15.86:0/3341012524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 0x55d3dfd127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.988+0000 7f4579f14000 1 -- 172.21.15.86:0/3341012524 >> 172.21.15.86:0/3341012524 conn(0x55d3dfd3ac10 msgr2=0x55d3dfd3d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.992+0000 7f4579f14000 1 -- 172.21.15.86:0/3341012524 shutdown_connections 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.992+0000 7f4579f14000 1 -- 172.21.15.86:0/3341012524 wait complete. 2024-03-22T04:11:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:08.992+0000 7f4579f14000 20 client.4791 trim_cache size 0 max 16384 2024-03-22T04:11:09.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 Processor -- start 2024-03-22T04:11:09.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 -- start start 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd3ac10 0x55d3dfd15010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd15550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 0x55d3dfd6f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45702007c0 con 0x55d3dfd0dc20 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45701ff7a0 con 0x55d3dfd59e70 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f457006e290 con 0x55d3dfd3ac10 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd15550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd15550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60282/0 (socket says 172.21.15.86:60282) 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 -- 172.21.15.86:0/1261426137 learned_addr learned my addr 172.21.15.86:0/1261426137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456fe00640 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd3ac10 0x55d3dfd15010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4574a00640 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 0x55d3dfd6f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 -- 172.21.15.86:0/1261426137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd3ac10 msgr2=0x55d3dfd15010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd3ac10 0x55d3dfd15010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 -- 172.21.15.86:0/1261426137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 msgr2=0x55d3dfd6f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 0x55d3dfd6f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 -- 172.21.15.86:0/1261426137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3dfae6e80 con 0x55d3dfd0dc20 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f456f400640 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd15550 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f455c009810 tx=0x7f455c05a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9c7efa863ab71c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f455be00640 1 -- 172.21.15.86:0/1261426137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c021070 con 0x55d3dfd0dc20 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f455be00640 1 -- 172.21.15.86:0/1261426137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c054460 con 0x55d3dfd0dc20 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f455be00640 1 -- 172.21.15.86:0/1261426137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c05c600 con 0x55d3dfd0dc20 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 -- 172.21.15.86:0/1261426137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 msgr2=0x55d3dfd15550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd15550 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f455c009810 tx=0x7f455c05a000 comp rx=0 tx=0).stop 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 -- 172.21.15.86:0/1261426137 shutdown_connections 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 0x55d3dfd6f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd0dc20 0x55d3dfd15550 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 --2- 172.21.15.86:0/1261426137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd3ac10 0x55d3dfd15010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.008+0000 7f4579f14000 1 -- 172.21.15.86:0/1261426137 >> 172.21.15.86:0/1261426137 conn(0x55d3dfd0cee0 msgr2=0x55d3dfd59210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- 172.21.15.86:0/1261426137 shutdown_connections 2024-03-22T04:11:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- 172.21.15.86:0/1261426137 wait complete. 2024-03-22T04:11:09.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- start start 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 Processor -- start 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 0x55d3dfd72650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd3ac10 0x55d3dfdcd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 0x55d3dfdcdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3dfb5de20 con 0x55d3dfd3ac10 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f457006e8e0 con 0x55d3dfd59e70 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d3dfaf6dc0 con 0x55d3dfd0dc20 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 10 client.0 _ll_register_callbacks cb 0x7f455c00cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd3ac10 0x55d3dfdcd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd3ac10 0x55d3dfdcd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60288/0 (socket says 172.21.15.86:60288) 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 -- 172.21.15.86:0/299550759 learned_addr learned my addr 172.21.15.86:0/299550759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456fe00640 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 0x55d3dfd72650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4574a00640 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 0x55d3dfdcdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 -- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 msgr2=0x55d3dfd72650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 0x55d3dfd72650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 -- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 msgr2=0x55d3dfdcdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 0x55d3dfdcdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 -- 172.21.15.86:0/299550759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3dfaf2750 con 0x55d3dfd3ac10 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456f400640 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd3ac10 0x55d3dfdcd9e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c520 tx=0x7f455c0841a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cde96df5f8cac6e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4559600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4559600640 1 -- 172.21.15.86:0/299550759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c021040 con 0x55d3dfd3ac10 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4559600640 1 -- 172.21.15.86:0/299550759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f455c074d00 con 0x55d3dfd3ac10 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- 172.21.15.86:0/299550759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455c054460 con 0x55d3dfd3ac10 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4559600640 1 -- 172.21.15.86:0/299550759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f455c051eb0 con 0x55d3dfd3ac10 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 10 client.4797 fetch_fsmap learned FSMap version 9 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- 172.21.15.86:0/299550759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d3dfb05450 con 0x55d3dfd3ac10 2024-03-22T04:11:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4559600640 1 -- 172.21.15.86:0/299550759 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c054460 con 0x55d3dfd3ac10 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4559600640 1 -- 172.21.15.86:0/299550759 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f455c052050 con 0x55d3dfd3ac10 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 10 client.4797 fetch_fsmap finished waiting for FSMap version 9 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 10 client.4797 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 20 client.4797 populate_metadata read hostname 'smithi086' 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55d3dfd5a790 0x55d3dfdca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 4 client.4797 mds_command: new command op to 74171 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f4579f14000 1 -- 172.21.15.86:0/299550759 --> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] -- command(tid 0: {"prefix": "status"}) -- 0x55d3dfb6c050 con 0x55d3dfd5a790 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.012+0000 7f456fe00640 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55d3dfd5a790 0x55d3dfdca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4559600640 10 client.4797 ms_handle_connect on v2:172.21.15.196:6833/180015071 2024-03-22T04:11:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f456fe00640 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55d3dfd5a790 0x55d3dfdca390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e066d5bf7d41e3b server_cookie=eeadd492f20a7274 in_seq=0 out_seq=0 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4559600640 1 -- 172.21.15.86:0/299550759 <== mds.0 v2:172.21.15.196:6833/180015071 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d3dfb6c050 con 0x55d3dfd5a790 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4559600640 10 client.4797 handle_command_reply: tid=0 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 client.4797 shutdown 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 -- 172.21.15.86:0/299550759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd3ac10 msgr2=0x55d3dfdcd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd3ac10 0x55d3dfdcd9e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c520 tx=0x7f455c0841a0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 -- 172.21.15.86:0/299550759 shutdown_connections 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:6833/180015071,v1:172.21.15.196:6835/180015071] conn(0x55d3dfd5a790 0x55d3dfdca390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3dfd59e70 0x55d3dfdcdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3dfd3ac10 0x55d3dfdcd9e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4559600640 0 client.4797 ms_handle_reset on v2:172.21.15.196:6833/180015071 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 --2- 172.21.15.86:0/299550759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d3dfd0dc20 0x55d3dfd72650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 -- 172.21.15.86:0/299550759 >> 172.21.15.86:0/299550759 conn(0x55d3dfd0cee0 msgr2=0x55d3dfd14000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:09.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 -- 172.21.15.86:0/299550759 shutdown_connections 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 1 -- 172.21.15.86:0/299550759 wait complete. 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4579f14000 20 client.4797 trim_cache size 0 max 16384 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 74171, 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 34.110423889000003, 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 9, 2024-03-22T04:11:09.024 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 27, 2024-03-22T04:11:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 27, 2024-03-22T04:11:09.025 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 41.704070246000001 2024-03-22T04:11:09.025 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:11:09.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 -- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4550067b20 msgr2=0x7f4550069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4550067b20 0x7f4550069fe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f45600012f0 tx=0x7f4560001770 comp rx=0 tx=0).stop 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 -- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45700d0de0 msgr2=0x7f45700b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45700d0de0 0x7f45700b61f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f455c00e980 tx=0x7f455c00ee40 comp rx=0 tx=0).stop 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 -- 172.21.15.86:0/1491653438 shutdown_connections 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4550067b20 0x7f4550069fe0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f457015fed0 0x7f45700b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45700d0de0 0x7f45700b61f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 --2- 172.21.15.86:0/1491653438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45700ce6b0 0x7f45700c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 -- 172.21.15.86:0/1491653438 >> 172.21.15.86:0/1491653438 conn(0x7f457006f910 msgr2=0x7f457015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 -- 172.21.15.86:0/1491653438 shutdown_connections 2024-03-22T04:11:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.016+0000 7f4576a00640 1 -- 172.21.15.86:0/1491653438 wait complete. 2024-03-22T04:11:09.038 INFO:teuthology.run_tasks:Running task sequential... 2024-03-22T04:11:09.046 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-22T04:11:09.047 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-22T04:11:09.047 INFO:teuthology.task.print:Enabling mgr modules 2024-03-22T04:11:09.047 INFO:teuthology.run_tasks:Running task kclient... 2024-03-22T04:11:09.105 INFO:tasks.kclient:Mounting kernel clients... 2024-03-22T04:11:09.105 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-22T04:11:09.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.105 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:11:09.115 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-22T04:11:09.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-22T04:11:09.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.178 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-22T04:11:09.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.360 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:11:09.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-22T04:11:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-03-22T04:11:09.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:11:09.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.433 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-03-22T04:11:09.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.443 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-03-22T04:11:09.506 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-03-22T04:11:09.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:11:09.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.509 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-03-22T04:11:09.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:09.561 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-03-22T04:11:09.630 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-03-22T04:11:09.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:11:09.632 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-22T04:11:09.632 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T04:11:09.632 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:11:09.632 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:11:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb83600640 1 Processor -- start 2024-03-22T04:11:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb83600640 1 -- start start 2024-03-22T04:11:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb83600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad330 0x7fdb7c15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb83600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdb7c0ad710 0x7fdb7c142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb83600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb7c15c7c0 con 0x7fdb7c06ef40 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb83600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7c06dea0 con 0x7fdb7c0ad710 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb83600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7c06e180 con 0x7fdb7c0ad330 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb73e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdb7c06ef40 0x7fdb7c13fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50078/0 (socket says 172.21.15.86:50078) 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb73e00640 1 -- 172.21.15.86:0/3601500945 learned_addr learned my addr 172.21.15.86:0/3601500945 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb81600640 1 --2- 172.21.15.86:0/3601500945 >> v2:172.21.15.86:3300/0 conn(0x7fdb7c0ad710 0x7fdb7c142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb80c00640 1 --2- 172.21.15.86:0/3601500945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad330 0x7fdb7c15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb7be00640 1 -- 172.21.15.86:0/3601500945 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1387353013 0 0) 0x7fdb7c15c7c0 con 0x7fdb7c06ef40 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb7be00640 1 -- 172.21.15.86:0/3601500945 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb60003530 con 0x7fdb7c06ef40 2024-03-22T04:11:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb81600640 1 -- 172.21.15.86:0/3601500945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad330 msgr2=0x7fdb7c15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb81600640 1 --2- 172.21.15.86:0/3601500945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad330 0x7fdb7c15d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb81600640 1 -- 172.21.15.86:0/3601500945 >> v1:172.21.15.86:6789/0 conn(0x7fdb7c06ef40 legacy=0x7fdb7c13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.764+0000 7fdb81600640 1 -- 172.21.15.86:0/3601500945 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c06e460 con 0x7fdb7c0ad710 2024-03-22T04:11:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb81600640 1 --2- 172.21.15.86:0/3601500945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad710 0x7fdb7c142690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009d90 tx=0x7fdb7401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5a7bc689812a4c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb7be00640 1 -- 172.21.15.86:0/3601500945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb740b2070 con 0x7fdb7c0ad710 2024-03-22T04:11:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb7be00640 1 -- 172.21.15.86:0/3601500945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb740a5a00 con 0x7fdb7c0ad710 2024-03-22T04:11:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb7be00640 1 -- 172.21.15.86:0/3601500945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb740a5db0 con 0x7fdb7c0ad710 2024-03-22T04:11:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb83600640 1 -- 172.21.15.86:0/3601500945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad710 msgr2=0x7fdb7c142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb83600640 1 --2- 172.21.15.86:0/3601500945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad710 0x7fdb7c142690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009d90 tx=0x7fdb7401c940 comp rx=0 tx=0).stop 2024-03-22T04:11:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb83600640 1 -- 172.21.15.86:0/3601500945 shutdown_connections 2024-03-22T04:11:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb83600640 1 --2- 172.21.15.86:0/3601500945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad710 0x7fdb7c142690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb83600640 1 --2- 172.21.15.86:0/3601500945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad330 0x7fdb7c15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb83600640 1 -- 172.21.15.86:0/3601500945 >> 172.21.15.86:0/3601500945 conn(0x7fdb7c0b1df0 msgr2=0x7fdb7c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb83600640 1 -- 172.21.15.86:0/3601500945 shutdown_connections 2024-03-22T04:11:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.768+0000 7fdb83600640 1 -- 172.21.15.86:0/3601500945 wait complete. 2024-03-22T04:11:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 Processor -- start 2024-03-22T04:11:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 -- start start 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c06ef40 0x7fdb7c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad330 0x7fdb7c1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad710 0x7fdb7c158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c1690e0 con 0x7fdb7c0ad330 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c168f60 con 0x7fdb7c06ef40 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7c169260 con 0x7fdb7c0ad710 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad330 0x7fdb7c1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad330 0x7fdb7c1586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60302/0 (socket says 172.21.15.86:60302) 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 -- 172.21.15.86:0/3432455862 learned_addr learned my addr 172.21.15.86:0/3432455862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb81600640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad710 0x7fdb7c158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb80c00640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c06ef40 0x7fdb7c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 -- 172.21.15.86:0/3432455862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad710 msgr2=0x7fdb7c158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad710 0x7fdb7c158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 -- 172.21.15.86:0/3432455862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c06ef40 msgr2=0x7fdb7c155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c06ef40 0x7fdb7c155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 -- 172.21.15.86:0/3432455862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c144320 con 0x7fdb7c0ad330 2024-03-22T04:11:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb73e00640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad330 0x7fdb7c1586d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400c3f0 tx=0x7fdb6400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb79600640 1 -- 172.21.15.86:0/3432455862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64017070 con 0x7fdb7c0ad330 2024-03-22T04:11:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb79600640 1 -- 172.21.15.86:0/3432455862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb64005cf0 con 0x7fdb7c0ad330 2024-03-22T04:11:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c09af40 con 0x7fdb7c0ad330 2024-03-22T04:11:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb79600640 1 -- 172.21.15.86:0/3432455862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb640072f0 con 0x7fdb7c0ad330 2024-03-22T04:11:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.772+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c1448a0 con 0x7fdb7c0ad330 2024-03-22T04:11:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.776+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7c068650 con 0x7fdb7c0ad330 2024-03-22T04:11:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.776+0000 7fdb79600640 1 -- 172.21.15.86:0/3432455862 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdb6401a070 con 0x7fdb7c0ad330 2024-03-22T04:11:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.776+0000 7fdb79600640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb58067b20 0x7fdb58069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.776+0000 7fdb79600640 1 -- 172.21.15.86:0/3432455862 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdb64085660 con 0x7fdb7c0ad330 2024-03-22T04:11:09.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.776+0000 7fdb80c00640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb58067b20 0x7fdb58069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:09.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.776+0000 7fdb80c00640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb58067b20 0x7fdb58069fe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009ac0 tx=0x7fdb6c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:09.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.780+0000 7fdb79600640 1 -- 172.21.15.86:0/3432455862 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdb640556d0 con 0x7fdb7c0ad330 2024-03-22T04:11:09.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdb7c0b5220 con 0x7fdb7c0ad330 2024-03-22T04:11:09.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb79600640 1 -- 172.21.15.86:0/3432455862 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fdb64059770 con 0x7fdb7c0ad330 2024-03-22T04:11:09.979 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-22T04:11:09.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb58067b20 msgr2=0x7fdb58069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb58067b20 0x7fdb58069fe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009ac0 tx=0x7fdb6c009190 comp rx=0 tx=0).stop 2024-03-22T04:11:09.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad330 msgr2=0x7fdb7c1586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:09.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad330 0x7fdb7c1586d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400c3f0 tx=0x7fdb6400c8b0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 shutdown_connections 2024-03-22T04:11:09.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdb58067b20 0x7fdb58069fe0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdb7c0ad710 0x7fdb7c158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb7c0ad330 0x7fdb7c1586d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 --2- 172.21.15.86:0/3432455862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c06ef40 0x7fdb7c155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:09.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 >> 172.21.15.86:0/3432455862 conn(0x7fdb7c0b1df0 msgr2=0x7fdb7c15ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:09.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 shutdown_connections 2024-03-22T04:11:09.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:09.976+0000 7fdb83600640 1 -- 172.21.15.86:0/3432455862 wait complete. 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:11:10.000 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:11:10.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:10.000 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:11:10.009 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42462sec preferred_lft 42462sec 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:11:10.010 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:10.011 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T04:11:10.011 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:10.012 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:10.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-03-22T04:11:10.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-03-22T04:11:10.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-03-22T04:11:10.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T04:11:10.012 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:10.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:10.167 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:11:10.227 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:11:10.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:10.229 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:11:10.281 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:11:10.281 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:11:10.281 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:11:10.281 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:11:10.281 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:11:10.281 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:11:10.281 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:11:10.282 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:10.282 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:10.282 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:10.282 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:11:10.282 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:10.282 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:10.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:11:10.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:11:10.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:11:10.282 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:10.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:10.414 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:11:10.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:10.423 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-03-22T04:11:10.478 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:10.478 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:10.478 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:11:10.478 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-22T04:11:10.478 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:10.604 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-22T04:11:10.604 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:10.604 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:10.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-22T04:11:10.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T04:11:10.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-22T04:11:10.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-22T04:11:10.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-22T04:11:10.604 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:10.739 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:10.739 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:10.740 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:11:10.740 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T04:11:10.740 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:10.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:10.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:11:10.868 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:11:10.877 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:11:10.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:11:10.877 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:11:10.925 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:11:10.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:10.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:11:11.020 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:11:11.020 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:11:11.020 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:11:11.505 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:11:11.505 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:11:11.505 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:11:11.506 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:11:11.506 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:11:11.518 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:11:11.518 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:11:11.518 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:11:11.518 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:11:11.518 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:11:11.518 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:11:11.518 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:11:11.518 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:11:11.522 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:11:11.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:11.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:11:11.542 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:11.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:11.643 INFO:teuthology.orchestra.run.smithi086.stdout:client4809 2024-03-22T04:11:11.643 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:11:11.643 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:11.652 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:11.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:11.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:11.749 INFO:teuthology.orchestra.run.smithi086.stdout:client4809 2024-03-22T04:11:11.749 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:11.846 INFO:teuthology.orchestra.run.smithi086.stdout:client4809 2024-03-22T04:11:11.846 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1365483d-2fb4-4579-a50b-2ce299f625b2.client4809/status') 2024-03-22T04:11:11.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:11.904 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.4809 (3)192.168.0.1:0/268242828 2024-03-22T04:11:11.905 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-03-22T04:11:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-03-22T04:11:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-03-22T04:11:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:68 bytes copied, 5.8406e-05 s, 1.2 MB/s 2024-03-22T04:11:11.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:11.913 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-03-22T04:11:11.913 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T04:11:11.913 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-22T04:11:11.914 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:11:12.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd441600640 1 Processor -- start 2024-03-22T04:11:12.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd441600640 1 -- start start 2024-03-22T04:11:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd441600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd43c0ad330 0x7fd43c15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd441600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c06ef40 0x7fd43c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd441600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd43c0ad710 0x7fd43c13fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd441600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd43c06dc40 con 0x7fd43c0ad330 2024-03-22T04:11:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd441600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd43c15c7c0 con 0x7fd43c0ad710 2024-03-22T04:11:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd441600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c06dea0 con 0x7fd43c06ef40 2024-03-22T04:11:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd43aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd43c0ad330 0x7fd43c15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd43aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd43c0ad330 0x7fd43c15d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60304/0 (socket says 172.21.15.86:60304) 2024-03-22T04:11:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd43aa00640 1 -- 172.21.15.86:0/2291724689 learned_addr learned my addr 172.21.15.86:0/2291724689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd43a000640 1 --2- 172.21.15.86:0/2291724689 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c06ef40 0x7fd43c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.084+0000 7fd43b400640 1 --2- 172.21.15.86:0/2291724689 >> v2:172.21.15.196:3300/0 conn(0x7fd43c0ad710 0x7fd43c13fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd43aa00640 1 -- 172.21.15.86:0/2291724689 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c06ef40 msgr2=0x7fd43c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd43aa00640 1 --2- 172.21.15.86:0/2291724689 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c06ef40 0x7fd43c06d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd43aa00640 1 -- 172.21.15.86:0/2291724689 >> v2:172.21.15.196:3300/0 conn(0x7fd43c0ad710 msgr2=0x7fd43c13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd43aa00640 1 --2- 172.21.15.86:0/2291724689 >> v2:172.21.15.196:3300/0 conn(0x7fd43c0ad710 0x7fd43c13fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd43aa00640 1 -- 172.21.15.86:0/2291724689 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c06e180 con 0x7fd43c0ad330 2024-03-22T04:11:12.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd43aa00640 1 --2- 172.21.15.86:0/2291724689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c0ad330 0x7fd43c15d4a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd43c06c5c0 tx=0x7fd42c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295a5555adc818ee server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:12.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd439600640 1 -- 172.21.15.86:0/2291724689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0aa610 con 0x7fd43c0ad330 2024-03-22T04:11:12.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd439600640 1 -- 172.21.15.86:0/2291724689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c0af070 con 0x7fd43c0ad330 2024-03-22T04:11:12.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd439600640 1 -- 172.21.15.86:0/2291724689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0aa9d0 con 0x7fd43c0ad330 2024-03-22T04:11:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 -- 172.21.15.86:0/2291724689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c0ad330 msgr2=0x7fd43c15d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 --2- 172.21.15.86:0/2291724689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c0ad330 0x7fd43c15d4a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd43c06c5c0 tx=0x7fd42c01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 -- 172.21.15.86:0/2291724689 shutdown_connections 2024-03-22T04:11:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 --2- 172.21.15.86:0/2291724689 >> v2:172.21.15.196:3300/0 conn(0x7fd43c0ad710 0x7fd43c13fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 --2- 172.21.15.86:0/2291724689 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c06ef40 0x7fd43c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 --2- 172.21.15.86:0/2291724689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c0ad330 0x7fd43c15d4a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 -- 172.21.15.86:0/2291724689 >> 172.21.15.86:0/2291724689 conn(0x7fd43c0b1df0 msgr2=0x7fd43c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 -- 172.21.15.86:0/2291724689 shutdown_connections 2024-03-22T04:11:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.088+0000 7fd441600640 1 -- 172.21.15.86:0/2291724689 wait complete. 2024-03-22T04:11:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 Processor -- start 2024-03-22T04:11:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 -- start start 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c06ef40 0x7fd43c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd43c0ad330 0x7fd43c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c0ad710 0x7fd43c15d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c141800 con 0x7fd43c06ef40 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c141680 con 0x7fd43c0ad330 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c141980 con 0x7fd43c0ad710 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c06ef40 0x7fd43c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c06ef40 0x7fd43c14ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60314/0 (socket says 172.21.15.86:60314) 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 -- 172.21.15.86:0/3704237633 learned_addr learned my addr 172.21.15.86:0/3704237633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:12.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43a000640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd43c0ad330 0x7fd43c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43b400640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c0ad710 0x7fd43c15d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 -- 172.21.15.86:0/3704237633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c0ad710 msgr2=0x7fd43c15d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c0ad710 0x7fd43c15d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 -- 172.21.15.86:0/3704237633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd43c0ad330 msgr2=0x7fd43c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd43c0ad330 0x7fd43c15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 -- 172.21.15.86:0/3704237633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c159e40 con 0x7fd43c06ef40 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd43aa00640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c06ef40 0x7fd43c14ffb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0ac040 tx=0x7fd42c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd432a00640 1 -- 172.21.15.86:0/3704237633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0bd3f0 con 0x7fd43c06ef40 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd432a00640 1 -- 172.21.15.86:0/3704237633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c0bd590 con 0x7fd43c06ef40 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd432a00640 1 -- 172.21.15.86:0/3704237633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0bd980 con 0x7fd43c06ef40 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.092+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c158760 con 0x7fd43c06ef40 2024-03-22T04:11:12.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.096+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c15bec0 con 0x7fd43c06ef40 2024-03-22T04:11:12.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.096+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c068650 con 0x7fd43c06ef40 2024-03-22T04:11:12.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.096+0000 7fd432a00640 1 -- 172.21.15.86:0/3704237633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd42c0b33f0 con 0x7fd43c06ef40 2024-03-22T04:11:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.096+0000 7fd432a00640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd41c067b20 0x7fd41c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.096+0000 7fd432a00640 1 -- 172.21.15.86:0/3704237633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd42c120410 con 0x7fd43c06ef40 2024-03-22T04:11:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.096+0000 7fd43a000640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd41c067b20 0x7fd41c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:12.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.096+0000 7fd43a000640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd41c067b20 0x7fd41c069fe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd43c15f300 tx=0x7fd428009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.100+0000 7fd432a00640 1 -- 172.21.15.86:0/3704237633 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd42c0a7070 con 0x7fd43c06ef40 2024-03-22T04:11:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd43c0b37b0 con 0x7fd43c06ef40 2024-03-22T04:11:12.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd432a00640 1 -- 172.21.15.86:0/3704237633 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd42c016110 con 0x7fd43c06ef40 2024-03-22T04:11:12.295 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-22T04:11:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd41c067b20 msgr2=0x7fd41c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd41c067b20 0x7fd41c069fe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd43c15f300 tx=0x7fd428009190 comp rx=0 tx=0).stop 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c06ef40 msgr2=0x7fd43c14ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c06ef40 0x7fd43c14ffb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0ac040 tx=0x7fd42c01ad30 comp rx=0 tx=0).stop 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 shutdown_connections 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd41c067b20 0x7fd41c069fe0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd43c0ad330 0x7fd43c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd43c0ad710 0x7fd43c15d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 --2- 172.21.15.86:0/3704237633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c06ef40 0x7fd43c14ffb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 >> 172.21.15.86:0/3704237633 conn(0x7fd43c0b1df0 msgr2=0x7fd43c158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 shutdown_connections 2024-03-22T04:11:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12.292+0000 7fd441600640 1 -- 172.21.15.86:0/3704237633 wait complete. 2024-03-22T04:11:12.315 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T04:11:12.315 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:11:12.316 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:11:12.316 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-22T04:11:12.316 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-22T04:11:12.316 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T04:11:12.316 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T04:11:12.316 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:11:12.316 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-22T04:11:12.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:12.316 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:11:12.325 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:11:12.325 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:11:12.325 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:11:12.325 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.325 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:11:12.326 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42459sec preferred_lft 42459sec 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:11:12.327 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:11:12.328 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-03-22T04:11:12.328 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:12.328 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:11:12.383 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:11:12.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:12.385 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:11:12.442 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:11:12.442 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:11:12.442 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:11:12.443 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:11:12.443 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:11:12.443 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:11:12.443 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:11:12.443 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:12.443 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:12.443 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:12.443 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:11:12.443 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:12.443 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:12.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:11:12.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:11:12.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:11:12.444 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:12.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:12.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:12.578 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:11:12.587 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:11:12.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:12.588 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-03-22T04:11:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-22T04:11:12.639 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:12.639 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:12.639 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:11:12.639 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-22T04:11:12.639 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:12.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:12.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:12.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 06:4f:34:46:de:75 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-22T04:11:12.783 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-22T04:11:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::44f:34ff:fe46:de75/64 scope link 2024-03-22T04:11:12.784 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:12.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 06:4f:34:46:de:75 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-22T04:11:12.846 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-22T04:11:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::44f:34ff:fe46:de75/64 scope link 2024-03-22T04:11:12.847 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:12.848 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-22T04:11:12.848 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:12.848 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:12.848 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-22T04:11:12.848 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-22T04:11:12.848 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-22T04:11:12.848 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-22T04:11:12.848 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-22T04:11:12.848 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:12.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:13.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:13.026 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:13.026 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:11:13.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.1 up 2024-03-22T04:11:13.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.1 master ceph-brx 2024-03-22T04:11:13.027 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:11:13.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:13.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:13.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:11:13.109 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-22T04:11:13.158 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-22T04:11:13.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:11:13.158 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-22T04:11:13.209 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:11:13.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:13.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-22T04:11:13.297 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-22T04:11:13.297 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-22T04:11:13.297 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:11:13.312 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-22T04:11:13.312 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-22T04:11:13.312 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:11:13.312 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:11:13.312 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:11:13.321 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-22T04:11:13.321 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-22T04:11:13.321 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:11:13.321 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:11:13.321 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:11:13.321 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:11:13.322 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:11:13.322 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:11:13.325 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:11:13.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:13.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-22T04:11:13.344 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:13.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:13.440 INFO:teuthology.orchestra.run.smithi086.stdout:client74567 2024-03-22T04:11:13.441 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:11:13.441 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:13.449 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:13.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:13.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:13.552 INFO:teuthology.orchestra.run.smithi086.stdout:client74567 2024-03-22T04:11:13.552 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:13.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:13.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:13.607 INFO:teuthology.orchestra.run.smithi086.stdout:client74567 2024-03-22T04:11:13.608 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1365483d-2fb4-4579-a50b-2ce299f625b2.client74567/status') 2024-03-22T04:11:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:13.665 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.74567 (3)172.21.15.86:0/1598841502 2024-03-22T04:11:13.665 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-03-22T04:11:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-03-22T04:11:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-03-22T04:11:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:71 bytes copied, 4.8785e-05 s, 1.5 MB/s 2024-03-22T04:11:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:13.673 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-22T04:11:13.673 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T04:11:13.674 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T04:11:13.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:11:13.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8cdc00640 1 Processor -- start 2024-03-22T04:11:13.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8cdc00640 1 -- start start 2024-03-22T04:11:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8cdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8c80afbe0 0x7fc8c815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8cdc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80cd140 con 0x7fc8c80724b0 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8cdc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80cd420 con 0x7fc8c80affc0 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8cdc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c806de00 con 0x7fc8c80afbe0 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8c7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8c80afbe0 0x7fc8c815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8c7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8c80afbe0 0x7fc8c815d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60318/0 (socket says 172.21.15.86:60318) 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8c7e00640 1 -- 172.21.15.86:0/3376330605 learned_addr learned my addr 172.21.15.86:0/3376330605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8cd200640 1 --1- 172.21.15.86:0/3376330605 >> v1:172.21.15.86:6789/0 conn(0x7fc8c80724b0 0x7fc8c806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50080/0 (socket says 172.21.15.86:50080) 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8c7e00640 1 -- 172.21.15.86:0/3376330605 >> v1:172.21.15.196:6789/0 conn(0x7fc8c80affc0 legacy=0x7fc8c8071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8c7e00640 1 -- 172.21.15.86:0/3376330605 >> v1:172.21.15.86:6789/0 conn(0x7fc8c80724b0 legacy=0x7fc8c806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8c7e00640 1 -- 172.21.15.86:0/3376330605 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80cd700 con 0x7fc8c80afbe0 2024-03-22T04:11:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.852+0000 7fc8c6a00640 1 -- 172.21.15.86:0/3376330605 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676265966 0 0) 0x7fc8c80cd140 con 0x7fc8c80724b0 2024-03-22T04:11:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8c7e00640 1 --2- 172.21.15.86:0/3376330605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c815d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc8c806df80 tx=0x7fc8b001cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e3f510864cf062 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8c6a00640 1 -- 172.21.15.86:0/3376330605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00ab650 con 0x7fc8c80afbe0 2024-03-22T04:11:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8c6a00640 1 -- 172.21.15.86:0/3376330605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b00b0070 con 0x7fc8c80afbe0 2024-03-22T04:11:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8c6a00640 1 -- 172.21.15.86:0/3376330605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00aba10 con 0x7fc8c80afbe0 2024-03-22T04:11:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8cdc00640 1 -- 172.21.15.86:0/3376330605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 msgr2=0x7fc8c815d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8cdc00640 1 --2- 172.21.15.86:0/3376330605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c815d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc8c806df80 tx=0x7fc8b001cfa0 comp rx=0 tx=0).stop 2024-03-22T04:11:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8cdc00640 1 -- 172.21.15.86:0/3376330605 shutdown_connections 2024-03-22T04:11:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8cdc00640 1 --2- 172.21.15.86:0/3376330605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c815d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8cdc00640 1 -- 172.21.15.86:0/3376330605 >> 172.21.15.86:0/3376330605 conn(0x7fc8c806b660 msgr2=0x7fc8c806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8cdc00640 1 -- 172.21.15.86:0/3376330605 shutdown_connections 2024-03-22T04:11:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.856+0000 7fc8cdc00640 1 -- 172.21.15.86:0/3376330605 wait complete. 2024-03-22T04:11:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 Processor -- start 2024-03-22T04:11:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 -- start start 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8c80724b0 0x7fc8c8147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c8148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80affc0 0x7fc8c81523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c806f540 con 0x7fc8c80afbe0 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c806f3c0 con 0x7fc8c80affc0 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c8169f10 con 0x7fc8c80724b0 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c8148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c8148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60332/0 (socket says 172.21.15.86:60332) 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 -- 172.21.15.86:0/1204121144 learned_addr learned my addr 172.21.15.86:0/1204121144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7e00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8c80724b0 0x7fc8c8147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cd200640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80affc0 0x7fc8c81523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 -- 172.21.15.86:0/1204121144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8c80724b0 msgr2=0x7fc8c8147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8c80724b0 0x7fc8c8147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 -- 172.21.15.86:0/1204121144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80affc0 msgr2=0x7fc8c81523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80affc0 0x7fc8c81523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 -- 172.21.15.86:0/1204121144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80688b0 con 0x7fc8c80afbe0 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8c7400640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c8148ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c500 tx=0x7fc8b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8bfe00640 1 -- 172.21.15.86:0/1204121144 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8016020 con 0x7fc8c80afbe0 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8bfe00640 1 -- 172.21.15.86:0/1204121144 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8005220 con 0x7fc8c80afbe0 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c8148680 con 0x7fc8c80afbe0 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8bfe00640 1 -- 172.21.15.86:0/1204121144 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8007030 con 0x7fc8c80afbe0 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c8154b60 con 0x7fc8c80afbe0 2024-03-22T04:11:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.860+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c806c6d0 con 0x7fc8c80afbe0 2024-03-22T04:11:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.864+0000 7fc8bfe00640 1 -- 172.21.15.86:0/1204121144 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc8b80071f0 con 0x7fc8c80afbe0 2024-03-22T04:11:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.864+0000 7fc8bfe00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8a4067b50 0x7fc8a406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.864+0000 7fc8bfe00640 1 -- 172.21.15.86:0/1204121144 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc8b8085a30 con 0x7fc8c80afbe0 2024-03-22T04:11:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.864+0000 7fc8c7e00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8a4067b50 0x7fc8a406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.864+0000 7fc8c7e00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8a4067b50 0x7fc8a406a010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00af000 tx=0x7fc8b0004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:13.868+0000 7fc8bfe00640 1 -- 172.21.15.86:0/1204121144 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc8b8020020 con 0x7fc8c80afbe0 2024-03-22T04:11:14.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.068+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8c806fc40 con 0x7fc8c80afbe0 2024-03-22T04:11:14.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.068+0000 7fc8bfe00640 1 -- 172.21.15.86:0/1204121144 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fc8c8148680 con 0x7fc8c80afbe0 2024-03-22T04:11:14.072 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-22T04:11:14.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8a4067b50 msgr2=0x7fc8a406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8a4067b50 0x7fc8a406a010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00af000 tx=0x7fc8b0004740 comp rx=0 tx=0).stop 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 msgr2=0x7fc8c8148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c8148ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c500 tx=0x7fc8b800c9c0 comp rx=0 tx=0).stop 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 shutdown_connections 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8a4067b50 0x7fc8a406a010 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80affc0 0x7fc8c81523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c80afbe0 0x7fc8c8148ef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 --2- 172.21.15.86:0/1204121144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8c80724b0 0x7fc8c8147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 >> 172.21.15.86:0/1204121144 conn(0x7fc8c806b660 msgr2=0x7fc8c815bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 shutdown_connections 2024-03-22T04:11:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:14.072+0000 7fc8cdc00640 1 -- 172.21.15.86:0/1204121144 wait complete. 2024-03-22T04:11:14.092 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T04:11:14.093 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:11:14.093 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-03-22T04:11:14.093 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-22T04:11:14.093 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T04:11:14.093 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T04:11:14.093 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T04:11:14.093 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:11:14.093 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-22T04:11:14.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:14.093 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-03-22T04:11:14.102 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:14.103 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42461sec preferred_lft 42461sec 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 02:42:12:2a:0f:7e brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:11:14.104 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:14.104 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T04:11:14.104 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:14.104 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:14.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add name ceph-brx type bridge 2024-03-22T04:11:14.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr flush dev ceph-brx 2024-03-22T04:11:14.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set ceph-brx up 2024-03-22T04:11:14.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T04:11:14.105 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:14.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:14.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:14.264 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:11:14.282 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-03-22T04:11:14.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:14.284 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-03-22T04:11:14.353 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:14.354 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:11:14.355 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:14.355 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:14.355 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:11:14.355 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:11:14.355 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:11:14.355 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:14.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:14.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:14.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:14.497 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-03-22T04:11:14.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:14.506 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-03-22T04:11:14.557 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:14.557 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:14.557 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T04:11:14.557 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-22T04:11:14.557 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:14.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:14.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:14.688 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-22T04:11:14.689 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:14.689 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:14.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-22T04:11:14.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T04:11:14.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-22T04:11:14.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-22T04:11:14.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-22T04:11:14.689 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:14.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:14.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:14.875 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:14.875 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:14.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-03-22T04:11:14.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T04:11:14.875 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:14.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:14.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:14.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:11:14.968 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-22T04:11:14.978 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-22T04:11:14.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:11:14.979 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-22T04:11:15.032 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:11:15.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:15.032 DEBUG:teuthology.orchestra.run.smithi196:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T04:11:15.130 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T04:11:15.130 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T04:11:15.130 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-03-22T04:11:15.660 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T04:11:15.660 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T04:11:15.660 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-03-22T04:11:15.660 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:11:15.660 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:11:15.672 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T04:11:15.672 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T04:11:15.672 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-03-22T04:11:15.673 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:11:15.673 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:11:15.673 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:11:15.673 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:11:15.673 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:11:15.676 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:11:15.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:15.676 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-22T04:11:15.695 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:15.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:15.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:15.806 INFO:teuthology.orchestra.run.smithi196.stdout:client74573 2024-03-22T04:11:15.806 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:11:15.806 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:15.815 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:15.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:15.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:15.913 INFO:teuthology.orchestra.run.smithi196.stdout:client74573 2024-03-22T04:11:15.913 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:15.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:15.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:15.970 INFO:teuthology.orchestra.run.smithi196.stdout:client74573 2024-03-22T04:11:15.970 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1365483d-2fb4-4579-a50b-2ce299f625b2.client74573/status') 2024-03-22T04:11:16.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:16.029 INFO:teuthology.orchestra.run.smithi196.stdout:instance: client.74573 (3)192.168.0.1:0/1147840346 2024-03-22T04:11:16.029 INFO:teuthology.orchestra.run.smithi196.stdout:blocklisted: false 2024-03-22T04:11:16.030 INFO:teuthology.orchestra.run.smithi196.stderr:0+1 records in 2024-03-22T04:11:16.030 INFO:teuthology.orchestra.run.smithi196.stderr:0+1 records out 2024-03-22T04:11:16.030 INFO:teuthology.orchestra.run.smithi196.stderr:70 bytes copied, 5.2375e-05 s, 1.3 MB/s 2024-03-22T04:11:16.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:16.037 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-03-22T04:11:16.038 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T04:11:16.038 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-22T04:11:16.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:11:16.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.180+0000 7f7da7a00640 1 Processor -- start 2024-03-22T04:11:16.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.180+0000 7f7da7a00640 1 -- start start 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da7a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7da006ef40 0x7f7da0142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da7a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da7a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da015c7c0 con 0x7f7da00ad330 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da7a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7da006dea0 con 0x7f7da006ef40 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da7a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7da006e180 con 0x7f7da00ad710 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da5000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7da00ad330 0x7f7da013fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35354/0 (socket says 172.21.15.86:35354) 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da5000640 1 -- 172.21.15.86:0/1274045135 learned_addr learned my addr 172.21.15.86:0/1274045135 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da5a00640 1 --2- 172.21.15.86:0/1274045135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7d9fe00640 1 --2- 172.21.15.86:0/1274045135 >> v2:172.21.15.196:3300/0 conn(0x7f7da006ef40 0x7f7da0142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7d9f400640 1 -- 172.21.15.86:0/1274045135 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787139158 0 0) 0x7f7da015c7c0 con 0x7f7da00ad330 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7d9f400640 1 -- 172.21.15.86:0/1274045135 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d88003530 con 0x7f7da00ad330 2024-03-22T04:11:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da5a00640 1 -- 172.21.15.86:0/1274045135 >> v2:172.21.15.196:3300/0 conn(0x7f7da006ef40 msgr2=0x7f7da0142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:16.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da5a00640 1 --2- 172.21.15.86:0/1274045135 >> v2:172.21.15.196:3300/0 conn(0x7f7da006ef40 0x7f7da0142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da5a00640 1 -- 172.21.15.86:0/1274045135 >> v1:172.21.15.86:6789/0 conn(0x7f7da00ad330 legacy=0x7f7da013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:16.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7da5a00640 1 -- 172.21.15.86:0/1274045135 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da006e460 con 0x7f7da00ad710 2024-03-22T04:11:16.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.184+0000 7f7d9fe00640 1 --2- 172.21.15.86:0/1274045135 >> v2:172.21.15.196:3300/0 conn(0x7f7da006ef40 0x7f7da0142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:16.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da5a00640 1 --2- 172.21.15.86:0/1274045135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015d4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009d90 tx=0x7f7d9801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9be48a6002e18b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:16.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7d9f400640 1 -- 172.21.15.86:0/1274045135 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d980b2070 con 0x7f7da00ad710 2024-03-22T04:11:16.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7d9f400640 1 -- 172.21.15.86:0/1274045135 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d980a5a00 con 0x7f7da00ad710 2024-03-22T04:11:16.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7d9f400640 1 -- 172.21.15.86:0/1274045135 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d980a5dd0 con 0x7f7da00ad710 2024-03-22T04:11:16.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 -- 172.21.15.86:0/1274045135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 msgr2=0x7f7da015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:16.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 --2- 172.21.15.86:0/1274045135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015d4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009d90 tx=0x7f7d9801c940 comp rx=0 tx=0).stop 2024-03-22T04:11:16.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 -- 172.21.15.86:0/1274045135 shutdown_connections 2024-03-22T04:11:16.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 --2- 172.21.15.86:0/1274045135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015d4a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 --2- 172.21.15.86:0/1274045135 >> v2:172.21.15.196:3300/0 conn(0x7f7da006ef40 0x7f7da0142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 -- 172.21.15.86:0/1274045135 >> 172.21.15.86:0/1274045135 conn(0x7f7da00b1df0 msgr2=0x7f7da00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 -- 172.21.15.86:0/1274045135 shutdown_connections 2024-03-22T04:11:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 -- 172.21.15.86:0/1274045135 wait complete. 2024-03-22T04:11:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 Processor -- start 2024-03-22T04:11:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.188+0000 7f7da7a00640 1 -- start start 2024-03-22T04:11:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da006ef40 0x7f7da015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7da00ad330 0x7f7da015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01690e0 con 0x7f7da006ef40 2024-03-22T04:11:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0168f60 con 0x7f7da00ad330 2024-03-22T04:11:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7da0169260 con 0x7f7da00ad710 2024-03-22T04:11:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da006ef40 0x7f7da015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da006ef40 0x7f7da015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38462/0 (socket says 172.21.15.86:38462) 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 -- 172.21.15.86:0/2553481911 learned_addr learned my addr 172.21.15.86:0/2553481911 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5a00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7d9fe00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7da00ad330 0x7f7da015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 -- 172.21.15.86:0/2553481911 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 msgr2=0x7f7da015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 -- 172.21.15.86:0/2553481911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7da00ad330 msgr2=0x7f7da015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7da00ad330 0x7f7da015dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 -- 172.21.15.86:0/2553481911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da01574e0 con 0x7f7da006ef40 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da5000640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da006ef40 0x7f7da015d860 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400c450 tx=0x7f7d9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7d9cc00640 1 -- 172.21.15.86:0/2553481911 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94017070 con 0x7f7da006ef40 2024-03-22T04:11:16.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7d9cc00640 1 -- 172.21.15.86:0/2553481911 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d94005cf0 con 0x7f7da006ef40 2024-03-22T04:11:16.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7d9cc00640 1 -- 172.21.15.86:0/2553481911 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94007290 con 0x7f7da006ef40 2024-03-22T04:11:16.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da01577c0 con 0x7f7da006ef40 2024-03-22T04:11:16.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da0157d80 con 0x7f7da006ef40 2024-03-22T04:11:16.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da0068650 con 0x7f7da006ef40 2024-03-22T04:11:16.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.191+0000 7f7d9cc00640 1 -- 172.21.15.86:0/2553481911 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7d9401a070 con 0x7f7da006ef40 2024-03-22T04:11:16.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.195+0000 7f7d9cc00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7d80067b20 0x7f7d80069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:16.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.195+0000 7f7d9cc00640 1 -- 172.21.15.86:0/2553481911 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f7d940855a0 con 0x7f7da006ef40 2024-03-22T04:11:16.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.195+0000 7f7d9fe00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7d80067b20 0x7f7d80069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:16.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.195+0000 7f7d9fe00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7d80067b20 0x7f7d80069fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7da01fa750 tx=0x7f7d8c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:16.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.199+0000 7f7d9cc00640 1 -- 172.21.15.86:0/2553481911 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7d940555b0 con 0x7f7da006ef40 2024-03-22T04:11:16.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7da00afae0 con 0x7f7da006ef40 2024-03-22T04:11:16.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7d9cc00640 1 -- 172.21.15.86:0/2553481911 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f7d94059650 con 0x7f7da006ef40 2024-03-22T04:11:16.392 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-22T04:11:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7d80067b20 msgr2=0x7f7d80069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7d80067b20 0x7f7d80069fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7da01fa750 tx=0x7f7d8c009190 comp rx=0 tx=0).stop 2024-03-22T04:11:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da006ef40 msgr2=0x7f7da015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da006ef40 0x7f7da015d860 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400c450 tx=0x7f7d9400c910 comp rx=0 tx=0).stop 2024-03-22T04:11:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 shutdown_connections 2024-03-22T04:11:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7d80067b20 0x7f7d80069fe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7da00ad330 0x7f7da015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7da00ad710 0x7f7da015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 --2- 172.21.15.86:0/2553481911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da006ef40 0x7f7da015d860 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.387+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 >> 172.21.15.86:0/2553481911 conn(0x7f7da00b1df0 msgr2=0x7f7da00b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.391+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 shutdown_connections 2024-03-22T04:11:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:16.391+0000 7f7da7a00640 1 -- 172.21.15.86:0/2553481911 wait complete. 2024-03-22T04:11:16.413 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T04:11:16.414 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:11:16.414 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-03-22T04:11:16.414 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-22T04:11:16.414 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-22T04:11:16.414 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T04:11:16.414 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T04:11:16.414 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:11:16.414 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-22T04:11:16.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:16.414 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:16.423 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42459sec preferred_lft 42459sec 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 02:42:12:2a:0f:7e brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:11:16.424 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-03-22T04:11:16.425 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:16.426 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:11:16.486 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-03-22T04:11:16.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:16.488 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-03-22T04:11:16.539 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:11:16.540 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:11:16.541 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:16.541 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:16.541 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:11:16.541 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:11:16.541 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:11:16.541 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:16.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:16.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:16.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:16.678 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-03-22T04:11:16.729 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-22T04:11:16.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:16.730 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-03-22T04:11:16.780 INFO:teuthology.orchestra.run.smithi196.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-22T04:11:16.781 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:16.781 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:16.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-22T04:11:16.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-22T04:11:16.782 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:16.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:16.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:16.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:16.908 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-22T04:11:16.928 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:11:16.928 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:11:16.928 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:11:16.928 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.929 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-03-22T04:11:16.929 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.929 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:11:16.929 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:6d:c7:96:8a:aa brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-22T04:11:16.929 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-22T04:11:16.929 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.929 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::286d:c7ff:fe96:8aaa/64 scope link 2024-03-22T04:11:16.929 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:16.930 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-22T04:11:16.994 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:6d:c7:96:8a:aa brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::286d:c7ff:fe96:8aaa/64 scope link 2024-03-22T04:11:16.995 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:11:16.997 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-22T04:11:16.997 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:16.997 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:16.997 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-22T04:11:16.997 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-22T04:11:16.997 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-22T04:11:16.997 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-22T04:11:16.997 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-22T04:11:16.997 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:17.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:17.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:17.175 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:11:17.175 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-22T04:11:17.175 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.1 up 2024-03-22T04:11:17.175 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.1 master ceph-brx 2024-03-22T04:11:17.175 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-22T04:11:17.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:17.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:17.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:11:17.260 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-22T04:11:17.268 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-22T04:11:17.268 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:11:17.268 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-22T04:11:17.320 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:11:17.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:17.320 DEBUG:teuthology.orchestra.run.smithi196:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-22T04:11:17.409 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-22T04:11:17.409 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-22T04:11:17.409 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-03-22T04:11:17.424 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-22T04:11:17.424 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-22T04:11:17.424 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-03-22T04:11:17.424 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:11:17.424 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:11:17.434 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-22T04:11:17.434 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-22T04:11:17.435 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-03-22T04:11:17.435 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:11:17.435 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:11:17.435 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:11:17.435 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:11:17.435 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:11:17.440 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:11:17.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:17.440 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-22T04:11:17.505 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:17.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:17.602 INFO:teuthology.orchestra.run.smithi196.stdout:client4804 2024-03-22T04:11:17.602 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:11:17.602 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:17.658 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:17.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:17.756 INFO:teuthology.orchestra.run.smithi196.stdout:client4804 2024-03-22T04:11:17.756 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:11:17.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:11:17.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:17.852 INFO:teuthology.orchestra.run.smithi196.stdout:client4804 2024-03-22T04:11:17.852 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1365483d-2fb4-4579-a50b-2ce299f625b2.client4804/status') 2024-03-22T04:11:17.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:11:17.911 INFO:teuthology.orchestra.run.smithi196.stdout:instance: client.4804 (3)192.168.0.2:0/1838996743 2024-03-22T04:11:17.911 INFO:teuthology.orchestra.run.smithi196.stdout:blocklisted: false 2024-03-22T04:11:17.911 INFO:teuthology.orchestra.run.smithi196.stderr:0+1 records in 2024-03-22T04:11:17.911 INFO:teuthology.orchestra.run.smithi196.stderr:0+1 records out 2024-03-22T04:11:17.911 INFO:teuthology.orchestra.run.smithi196.stderr:69 bytes copied, 6.7562e-05 s, 1.0 MB/s 2024-03-22T04:11:17.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-22T04:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:11:17.919 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-22T04:11:17.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7085a00640 1 Processor -- start 2024-03-22T04:11:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7085a00640 1 -- start start 2024-03-22T04:11:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7085a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708015e630 0x7f70800cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7085a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f708006e2f0 0x7f70800d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7085a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70801623d0 con 0x7f708006cc80 2024-03-22T04:11:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7085a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f70801620f0 con 0x7f708006e2f0 2024-03-22T04:11:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7085a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f708015ec10 con 0x7f708015e630 2024-03-22T04:11:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7076000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f708006e2f0 0x7f70800d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7076000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f708006e2f0 0x7f70800d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38466/0 (socket says 172.21.15.86:38466) 2024-03-22T04:11:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7076000640 1 -- 172.21.15.86:0/867526841 learned_addr learned my addr 172.21.15.86:0/867526841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:18.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f707ea00640 1 --2- 172.21.15.86:0/867526841 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708015e630 0x7f70800cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7076000640 1 -- 172.21.15.86:0/867526841 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708015e630 msgr2=0x7f70800cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:18.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7076000640 1 --2- 172.21.15.86:0/867526841 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708015e630 0x7f70800cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7076000640 1 -- 172.21.15.86:0/867526841 >> v1:172.21.15.196:6789/0 conn(0x7f708006cc80 legacy=0x7f708006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.079+0000 7f7076000640 1 -- 172.21.15.86:0/867526841 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708015eef0 con 0x7f708006e2f0 2024-03-22T04:11:18.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7076000640 1 --2- 172.21.15.86:0/867526841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f70800d03b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7068009870 tx=0x7f706801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d4df5ddfd80d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f707e000640 1 -- 172.21.15.86:0/867526841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680b2070 con 0x7f708006e2f0 2024-03-22T04:11:18.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f707e000640 1 -- 172.21.15.86:0/867526841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70680a9070 con 0x7f708006e2f0 2024-03-22T04:11:18.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f707e000640 1 -- 172.21.15.86:0/867526841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680ad880 con 0x7f708006e2f0 2024-03-22T04:11:18.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7085a00640 1 -- 172.21.15.86:0/867526841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 msgr2=0x7f70800d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7085a00640 1 --2- 172.21.15.86:0/867526841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f70800d03b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7068009870 tx=0x7f706801c560 comp rx=0 tx=0).stop 2024-03-22T04:11:18.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7085a00640 1 -- 172.21.15.86:0/867526841 shutdown_connections 2024-03-22T04:11:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7085a00640 1 --2- 172.21.15.86:0/867526841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f70800d03b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7085a00640 1 --2- 172.21.15.86:0/867526841 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708015e630 0x7f70800cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7085a00640 1 -- 172.21.15.86:0/867526841 >> 172.21.15.86:0/867526841 conn(0x7f708006f910 msgr2=0x7f708006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7085a00640 1 -- 172.21.15.86:0/867526841 shutdown_connections 2024-03-22T04:11:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.083+0000 7f7085a00640 1 -- 172.21.15.86:0/867526841 wait complete. 2024-03-22T04:11:18.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 Processor -- start 2024-03-22T04:11:18.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 -- start start 2024-03-22T04:11:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708006cc80 0x7f70801580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f7080153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708015e630 0x7f70800b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70800cfaf0 con 0x7f708006e2f0 2024-03-22T04:11:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f708015f9d0 con 0x7f708015e630 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f70800cfc70 con 0x7f708006cc80 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f7080153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f7080153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38472/0 (socket says 172.21.15.86:38472) 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 -- 172.21.15.86:0/3717520796 learned_addr learned my addr 172.21.15.86:0/3717520796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f707f400640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708015e630 0x7f70800b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f707ea00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708006cc80 0x7f70801580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 -- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708006cc80 msgr2=0x7f70801580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708006cc80 0x7f70801580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 -- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708015e630 msgr2=0x7f70800b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708015e630 0x7f70800b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 -- 172.21.15.86:0/3717520796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70800d3110 con 0x7f708006e2f0 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f707ea00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708006cc80 0x7f70801580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f707f400640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708015e630 0x7f70800b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7076000640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f7080153a60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f70680b0040 tx=0x7f7068019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7077400640 1 -- 172.21.15.86:0/3717520796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680bb070 con 0x7f708006e2f0 2024-03-22T04:11:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7077400640 1 -- 172.21.15.86:0/3717520796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70680add40 con 0x7f708006e2f0 2024-03-22T04:11:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70800b2f10 con 0x7f708006e2f0 2024-03-22T04:11:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7077400640 1 -- 172.21.15.86:0/3717520796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680b2040 con 0x7f708006e2f0 2024-03-22T04:11:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.087+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7080153fa0 con 0x7f708006e2f0 2024-03-22T04:11:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.091+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7080068650 con 0x7f708006e2f0 2024-03-22T04:11:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.091+0000 7f7077400640 1 -- 172.21.15.86:0/3717520796 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f70680be070 con 0x7f708006e2f0 2024-03-22T04:11:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.091+0000 7f7077400640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f705c067b20 0x7f705c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.091+0000 7f7077400640 1 -- 172.21.15.86:0/3717520796 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f706811f1b0 con 0x7f708006e2f0 2024-03-22T04:11:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.091+0000 7f707ea00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f705c067b20 0x7f705c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.091+0000 7f707ea00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f705c067b20 0x7f705c069fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f70700074c0 tx=0x7f707000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.095+0000 7f7077400640 1 -- 172.21.15.86:0/3717520796 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7068016150 con 0x7f708006e2f0 2024-03-22T04:11:18.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.251+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70800cffc0 con 0x7f708006e2f0 2024-03-22T04:11:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.251+0000 7f7077400640 1 -- 172.21.15.86:0/3717520796 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7f70680ef8a0 con 0x7f708006e2f0 2024-03-22T04:11:18.258 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f705c067b20 msgr2=0x7f705c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f705c067b20 0x7f705c069fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f70700074c0 tx=0x7f707000b040 comp rx=0 tx=0).stop 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 msgr2=0x7f7080153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f7080153a60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f70680b0040 tx=0x7f7068019300 comp rx=0 tx=0).stop 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 shutdown_connections 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f705c067b20 0x7f705c069fe0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708015e630 0x7f70800b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708006e2f0 0x7f7080153a60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 --2- 172.21.15.86:0/3717520796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708006cc80 0x7f70801580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 >> 172.21.15.86:0/3717520796 conn(0x7f708006f910 msgr2=0x7f7080168730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 shutdown_connections 2024-03-22T04:11:18.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.255+0000 7f7085a00640 1 -- 172.21.15.86:0/3717520796 wait complete. 2024-03-22T04:11:18.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:18.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:18.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.411+0000 7f985be00640 1 Processor -- start 2024-03-22T04:11:18.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.411+0000 7f985be00640 1 -- start start 2024-03-22T04:11:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f985c15e250 0x7f985c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c06d330 0x7f985c15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f985c15ff80 0x7f985c0ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f985c0d0440 con 0x7f985c15e250 2024-03-22T04:11:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985be00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f985c0d0720 con 0x7f985c15ff80 2024-03-22T04:11:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f985c0d0a00 con 0x7f985c06d330 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f985c15e250 0x7f985c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f985c15e250 0x7f985c0d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38478/0 (socket says 172.21.15.86:38478) 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 -- 172.21.15.86:0/2852303533 learned_addr learned my addr 172.21.15.86:0/2852303533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985b400640 1 --2- 172.21.15.86:0/2852303533 >> v2:172.21.15.196:3300/0 conn(0x7f985c15ff80 0x7f985c0ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985a000640 1 --2- 172.21.15.86:0/2852303533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c06d330 0x7f985c15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 -- 172.21.15.86:0/2852303533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c06d330 msgr2=0x7f985c15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 --2- 172.21.15.86:0/2852303533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c06d330 0x7f985c15fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 -- 172.21.15.86:0/2852303533 >> v2:172.21.15.196:3300/0 conn(0x7f985c15ff80 msgr2=0x7f985c0ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 --2- 172.21.15.86:0/2852303533 >> v2:172.21.15.196:3300/0 conn(0x7f985c15ff80 0x7f985c0ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 -- 172.21.15.86:0/2852303533 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985c0d0ce0 con 0x7f985c15e250 2024-03-22T04:11:18.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.415+0000 7f985aa00640 1 --2- 172.21.15.86:0/2852303533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c15e250 0x7f985c0d1200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f985c0d0070 tx=0x7f984c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a94f5a8e8c4dac5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f9859600640 1 -- 172.21.15.86:0/2852303533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c0a5640 con 0x7f985c15e250 2024-03-22T04:11:18.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f9859600640 1 -- 172.21.15.86:0/2852303533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c0a57e0 con 0x7f985c15e250 2024-03-22T04:11:18.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f9859600640 1 -- 172.21.15.86:0/2852303533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c0a5bb0 con 0x7f985c15e250 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- 172.21.15.86:0/2852303533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c15e250 msgr2=0x7f985c0d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 --2- 172.21.15.86:0/2852303533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c15e250 0x7f985c0d1200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f985c0d0070 tx=0x7f984c01c9c0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- 172.21.15.86:0/2852303533 shutdown_connections 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 --2- 172.21.15.86:0/2852303533 >> v2:172.21.15.196:3300/0 conn(0x7f985c15ff80 0x7f985c0ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 --2- 172.21.15.86:0/2852303533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c06d330 0x7f985c15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 --2- 172.21.15.86:0/2852303533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c15e250 0x7f985c0d1200 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- 172.21.15.86:0/2852303533 >> 172.21.15.86:0/2852303533 conn(0x7f985c0d1f30 msgr2=0x7f985c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- 172.21.15.86:0/2852303533 shutdown_connections 2024-03-22T04:11:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- 172.21.15.86:0/2852303533 wait complete. 2024-03-22T04:11:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 Processor -- start 2024-03-22T04:11:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- start start 2024-03-22T04:11:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c06d330 0x7f985c14f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c15e250 0x7f985c15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f985c15ff80 0x7f985c15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f985c162200 con 0x7f985c06d330 2024-03-22T04:11:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f985c162080 con 0x7f985c15ff80 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f985c162380 con 0x7f985c15e250 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c06d330 0x7f985c14f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c06d330 0x7f985c14f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38494/0 (socket says 172.21.15.86:38494) 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985aa00640 1 -- 172.21.15.86:0/1109958398 learned_addr learned my addr 172.21.15.86:0/1109958398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985a000640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c15e250 0x7f985c15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.419+0000 7f985b400640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f985c15ff80 0x7f985c15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985aa00640 1 -- 172.21.15.86:0/1109958398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c15e250 msgr2=0x7f985c15ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985aa00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c15e250 0x7f985c15ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985aa00640 1 -- 172.21.15.86:0/1109958398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f985c15ff80 msgr2=0x7f985c15f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985aa00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f985c15ff80 0x7f985c15f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985aa00640 1 -- 172.21.15.86:0/1109958398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985c1fbb60 con 0x7f985c06d330 2024-03-22T04:11:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985aa00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c06d330 0x7f985c14f0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f984c01d260 tx=0x7f984c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f9852a00640 1 -- 172.21.15.86:0/1109958398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c0bc020 con 0x7f985c06d330 2024-03-22T04:11:18.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f9852a00640 1 -- 172.21.15.86:0/1109958398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c0b8690 con 0x7f985c06d330 2024-03-22T04:11:18.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985c15bf20 con 0x7f985c06d330 2024-03-22T04:11:18.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f9852a00640 1 -- 172.21.15.86:0/1109958398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c009510 con 0x7f985c06d330 2024-03-22T04:11:18.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985c15c4e0 con 0x7f985c06d330 2024-03-22T04:11:18.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985c04f900 con 0x7f985c06d330 2024-03-22T04:11:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f9852a00640 1 -- 172.21.15.86:0/1109958398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f984c0bf020 con 0x7f985c06d330 2024-03-22T04:11:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f9852a00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f983c067b50 0x7f983c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f9852a00640 1 -- 172.21.15.86:0/1109958398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f984c1209b0 con 0x7f985c06d330 2024-03-22T04:11:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.423+0000 7f985a000640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f983c067b50 0x7f983c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.427+0000 7f985a000640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f983c067b50 0x7f983c06a010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f985c15fa30 tx=0x7f9848009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.427+0000 7f9852a00640 1 -- 172.21.15.86:0/1109958398 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f984c0c7020 con 0x7f985c06d330 2024-03-22T04:11:18.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.583+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f985c0b7eb0 con 0x7f985c06d330 2024-03-22T04:11:18.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.583+0000 7f9852a00640 1 -- 172.21.15.86:0/1109958398 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f984c016640 con 0x7f985c06d330 2024-03-22T04:11:18.589 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f983c067b50 msgr2=0x7f983c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f983c067b50 0x7f983c06a010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f985c15fa30 tx=0x7f9848009190 comp rx=0 tx=0).stop 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c06d330 msgr2=0x7f985c14f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c06d330 0x7f985c14f0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f984c01d260 tx=0x7f984c019760 comp rx=0 tx=0).stop 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 shutdown_connections 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f983c067b50 0x7f983c06a010 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f985c15ff80 0x7f985c15f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f985c15e250 0x7f985c15ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 --2- 172.21.15.86:0/1109958398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f985c06d330 0x7f985c14f0a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 >> 172.21.15.86:0/1109958398 conn(0x7f985c0d1f30 msgr2=0x7f985c15e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:18.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 shutdown_connections 2024-03-22T04:11:18.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.587+0000 7f985be00640 1 -- 172.21.15.86:0/1109958398 wait complete. 2024-03-22T04:11:18.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:18.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.743+0000 7fb25b800640 1 Processor -- start 2024-03-22T04:11:18.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.743+0000 7fb25b800640 1 -- start start 2024-03-22T04:11:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb25b800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb25415e970 0x7fb2540b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb25b800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb25409b180 0x7fb2540cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb25b800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25406dd70 con 0x7fb25406d8f0 2024-03-22T04:11:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb25b800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb2540b49a0 con 0x7fb25409b180 2024-03-22T04:11:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb25b800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb2540ccda0 con 0x7fb25415e970 2024-03-22T04:11:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb259800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb25409b180 0x7fb2540cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb258e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb25415e970 0x7fb2540b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb259800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb25409b180 0x7fb2540cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38500/0 (socket says 172.21.15.86:38500) 2024-03-22T04:11:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb259800640 1 -- 172.21.15.86:0/850780241 learned_addr learned my addr 172.21.15.86:0/850780241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb258e00640 1 -- 172.21.15.86:0/850780241 >> v2:172.21.15.86:3300/0 conn(0x7fb25409b180 msgr2=0x7fb2540cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb258e00640 1 --2- 172.21.15.86:0/850780241 >> v2:172.21.15.86:3300/0 conn(0x7fb25409b180 0x7fb2540cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb258e00640 1 -- 172.21.15.86:0/850780241 >> v1:172.21.15.196:6789/0 conn(0x7fb25406d8f0 legacy=0x7fb25409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb253e00640 1 -- 172.21.15.86:0/850780241 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947333904 0 0) 0x7fb23c001120 con 0x7fb25406d8f0 2024-03-22T04:11:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb258e00640 1 -- 172.21.15.86:0/850780241 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25406dff0 con 0x7fb25415e970 2024-03-22T04:11:18.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.747+0000 7fb259800640 1 --2- 172.21.15.86:0/850780241 >> v2:172.21.15.86:3300/0 conn(0x7fb25409b180 0x7fb2540cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:18.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb258e00640 1 --2- 172.21.15.86:0/850780241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25415e970 0x7fb2540b12e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb254069080 tx=0x7fb24401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450bf68e14a13c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb253e00640 1 -- 172.21.15.86:0/850780241 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2440ab650 con 0x7fb25415e970 2024-03-22T04:11:18.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb253e00640 1 -- 172.21.15.86:0/850780241 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2440b0070 con 0x7fb25415e970 2024-03-22T04:11:18.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb253e00640 1 -- 172.21.15.86:0/850780241 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2440aba10 con 0x7fb25415e970 2024-03-22T04:11:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 -- 172.21.15.86:0/850780241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25415e970 msgr2=0x7fb2540b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 --2- 172.21.15.86:0/850780241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25415e970 0x7fb2540b12e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb254069080 tx=0x7fb24401c450 comp rx=0 tx=0).stop 2024-03-22T04:11:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 -- 172.21.15.86:0/850780241 shutdown_connections 2024-03-22T04:11:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 --2- 172.21.15.86:0/850780241 >> v2:172.21.15.86:3300/0 conn(0x7fb25409b180 0x7fb2540cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 --2- 172.21.15.86:0/850780241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25415e970 0x7fb2540b12e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 -- 172.21.15.86:0/850780241 >> 172.21.15.86:0/850780241 conn(0x7fb2540b3bf0 msgr2=0x7fb2540b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 -- 172.21.15.86:0/850780241 shutdown_connections 2024-03-22T04:11:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 -- 172.21.15.86:0/850780241 wait complete. 2024-03-22T04:11:18.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 Processor -- start 2024-03-22T04:11:18.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.751+0000 7fb25b800640 1 -- start start 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb25406d8f0 0x7fb254160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25409b180 0x7fb2541588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb25415e970 0x7fb25415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2540cdf80 con 0x7fb25415e970 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb25415ed50 con 0x7fb25409b180 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb2540ce100 con 0x7fb25406d8f0 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb25415e970 0x7fb25415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb25415e970 0x7fb25415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38512/0 (socket says 172.21.15.86:38512) 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 -- 172.21.15.86:0/2877513008 learned_addr learned my addr 172.21.15.86:0/2877513008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb258e00640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb25406d8f0 0x7fb254160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb24be00640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25409b180 0x7fb2541588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 -- 172.21.15.86:0/2877513008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb25406d8f0 msgr2=0x7fb254160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb25406d8f0 0x7fb254160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 -- 172.21.15.86:0/2877513008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25409b180 msgr2=0x7fb2541588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25409b180 0x7fb2541588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 -- 172.21.15.86:0/2877513008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2541608d0 con 0x7fb25415e970 2024-03-22T04:11:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb259800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb25415e970 0x7fb25415a260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00ef90 tx=0x7fb24c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb251600640 1 -- 172.21.15.86:0/2877513008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00cbe0 con 0x7fb25415e970 2024-03-22T04:11:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb251600640 1 -- 172.21.15.86:0/2877513008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24c00cd80 con 0x7fb25415e970 2024-03-22T04:11:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb254158de0 con 0x7fb25415e970 2024-03-22T04:11:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb251600640 1 -- 172.21.15.86:0/2877513008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c003be0 con 0x7fb25415e970 2024-03-22T04:11:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2541594a0 con 0x7fb25415e970 2024-03-22T04:11:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb254074470 con 0x7fb25415e970 2024-03-22T04:11:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.755+0000 7fb251600640 1 -- 172.21.15.86:0/2877513008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb24c01b030 con 0x7fb25415e970 2024-03-22T04:11:18.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.759+0000 7fb251600640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb230067b20 0x7fb230069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:18.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.759+0000 7fb251600640 1 -- 172.21.15.86:0/2877513008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fb24c086650 con 0x7fb25415e970 2024-03-22T04:11:18.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.759+0000 7fb258e00640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb230067b20 0x7fb230069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:18.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.759+0000 7fb258e00640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb230067b20 0x7fb230069fe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2440af000 tx=0x7fb2440a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.763+0000 7fb251600640 1 -- 172.21.15.86:0/2877513008 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb24c022020 con 0x7fb25415e970 2024-03-22T04:11:18.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb254075020 con 0x7fb25415e970 2024-03-22T04:11:18.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb251600640 1 -- 172.21.15.86:0/2877513008 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fb24c010040 con 0x7fb25415e970 2024-03-22T04:11:18.924 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:18.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb230067b20 msgr2=0x7fb230069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb230067b20 0x7fb230069fe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2440af000 tx=0x7fb2440a7000 comp rx=0 tx=0).stop 2024-03-22T04:11:18.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb25415e970 msgr2=0x7fb25415a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:18.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb25415e970 0x7fb25415a260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00ef90 tx=0x7fb24c00c490 comp rx=0 tx=0).stop 2024-03-22T04:11:18.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 shutdown_connections 2024-03-22T04:11:18.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb230067b20 0x7fb230069fe0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb25409b180 0x7fb2541588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb25415e970 0x7fb25415a260 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 --2- 172.21.15.86:0/2877513008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb25406d8f0 0x7fb254160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:18.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 >> 172.21.15.86:0/2877513008 conn(0x7fb2540b3bf0 msgr2=0x7fb2540b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:18.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 shutdown_connections 2024-03-22T04:11:18.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:18.919+0000 7fb25b800640 1 -- 172.21.15.86:0/2877513008 wait complete. 2024-03-22T04:11:18.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:19.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f3005a00640 1 Processor -- start 2024-03-22T04:11:19.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f3005a00640 1 -- start start 2024-03-22T04:11:19.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f3005a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f3005a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30000b3430 0x7f300015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f3005a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300015d710 con 0x7f300015cee0 2024-03-22T04:11:19.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f3005a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f30000b3890 con 0x7f30000b3430 2024-03-22T04:11:19.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f3005a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30000b2740 con 0x7f30000b2360 2024-03-22T04:11:19.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2fff400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f300015cee0 0x7f300006d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35360/0 (socket says 172.21.15.86:35360) 2024-03-22T04:11:19.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2fff400640 1 -- 172.21.15.86:0/494954587 learned_addr learned my addr 172.21.15.86:0/494954587 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:19.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffea00640 1 --2- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffe000640 1 --2- 172.21.15.86:0/494954587 >> v2:172.21.15.196:3300/0 conn(0x7f30000b3430 0x7f300015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffd600640 1 -- 172.21.15.86:0/494954587 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1019574425 0 0) 0x7f300015d710 con 0x7f300015cee0 2024-03-22T04:11:19.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffd600640 1 -- 172.21.15.86:0/494954587 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fe4003530 con 0x7f300015cee0 2024-03-22T04:11:19.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffe000640 1 -- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 msgr2=0x7f30000b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:19.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffe000640 1 --2- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffe000640 1 -- 172.21.15.86:0/494954587 >> v1:172.21.15.86:6789/0 conn(0x7f300015cee0 legacy=0x7f300006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffe000640 1 -- 172.21.15.86:0/494954587 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300015fe40 con 0x7f30000b3430 2024-03-22T04:11:19.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.083+0000 7f2ffea00640 1 --2- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f2ffe000640 1 --2- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30000b3430 0x7f300015c9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0097c0 tx=0x7f2fec01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=234c109a207ed29 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f2ffd600640 1 -- 172.21.15.86:0/494954587 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0b2070 con 0x7f30000b3430 2024-03-22T04:11:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f2ffd600640 1 -- 172.21.15.86:0/494954587 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fec0ad640 con 0x7f30000b3430 2024-03-22T04:11:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f2ffd600640 1 -- 172.21.15.86:0/494954587 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0ada10 con 0x7f30000b3430 2024-03-22T04:11:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f3005a00640 1 -- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30000b3430 msgr2=0x7f300015c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f3005a00640 1 --2- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30000b3430 0x7f300015c9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0097c0 tx=0x7f2fec01c470 comp rx=0 tx=0).stop 2024-03-22T04:11:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f3005a00640 1 -- 172.21.15.86:0/494954587 shutdown_connections 2024-03-22T04:11:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f3005a00640 1 --2- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30000b3430 0x7f300015c9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f3005a00640 1 --2- 172.21.15.86:0/494954587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f3005a00640 1 -- 172.21.15.86:0/494954587 >> 172.21.15.86:0/494954587 conn(0x7f30000cc940 msgr2=0x7f30000ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:19.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f3005a00640 1 -- 172.21.15.86:0/494954587 shutdown_connections 2024-03-22T04:11:19.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.087+0000 7f3005a00640 1 -- 172.21.15.86:0/494954587 wait complete. 2024-03-22T04:11:19.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 Processor -- start 2024-03-22T04:11:19.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 -- start start 2024-03-22T04:11:19.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30000b3430 0x7f300014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300015cee0 0x7f3000154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3000160c00 con 0x7f30000b3430 2024-03-22T04:11:19.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30000b5d90 con 0x7f300015cee0 2024-03-22T04:11:19.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f300014c820 con 0x7f30000b2360 2024-03-22T04:11:19.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30000b3430 0x7f300014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30000b3430 0x7f300014c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38522/0 (socket says 172.21.15.86:38522) 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 -- 172.21.15.86:0/3491789907 learned_addr learned my addr 172.21.15.86:0/3491789907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffea00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 -- 172.21.15.86:0/3491789907 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 msgr2=0x7f30000730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 -- 172.21.15.86:0/3491789907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300015cee0 msgr2=0x7f3000154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300015cee0 0x7f3000154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 -- 172.21.15.86:0/3491789907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3000154f10 con 0x7f30000b3430 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ffe000640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30000b3430 0x7f300014c2e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0b0040 tx=0x7f2fec019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ff6a00640 1 -- 172.21.15.86:0/3491789907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0b2040 con 0x7f30000b3430 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ff6a00640 1 -- 172.21.15.86:0/3491789907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fec0bc070 con 0x7f30000b3430 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f2ff6a00640 1 -- 172.21.15.86:0/3491789907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec006e20 con 0x7f30000b3430 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30000725e0 con 0x7f30000b3430 2024-03-22T04:11:19.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.091+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3000072ba0 con 0x7f30000b3430 2024-03-22T04:11:19.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.095+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3000074470 con 0x7f30000b3430 2024-03-22T04:11:19.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.095+0000 7f2ff6a00640 1 -- 172.21.15.86:0/3491789907 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2fec0b6670 con 0x7f30000b3430 2024-03-22T04:11:19.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.095+0000 7f2ff6a00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2fe0067b20 0x7f2fe0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.095+0000 7f2ff6a00640 1 -- 172.21.15.86:0/3491789907 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f2fec120240 con 0x7f30000b3430 2024-03-22T04:11:19.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.095+0000 7f2ffea00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2fe0067b20 0x7f2fe0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.095+0000 7f2ffea00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2fe0067b20 0x7f2fe0069fe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f300015f420 tx=0x7f2ff0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.099+0000 7f2ff6a00640 1 -- 172.21.15.86:0/3491789907 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2fec016090 con 0x7f30000b3430 2024-03-22T04:11:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.251+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f300006e9f0 con 0x7f30000b3430 2024-03-22T04:11:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f2ff6a00640 1 -- 172.21.15.86:0/3491789907 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f2fec0f0920 con 0x7f30000b3430 2024-03-22T04:11:19.259 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2fe0067b20 msgr2=0x7f2fe0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2fe0067b20 0x7f2fe0069fe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f300015f420 tx=0x7f2ff0008040 comp rx=0 tx=0).stop 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30000b3430 msgr2=0x7f300014c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30000b3430 0x7f300014c2e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0b0040 tx=0x7f2fec019210 comp rx=0 tx=0).stop 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 shutdown_connections 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2fe0067b20 0x7f2fe0069fe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f300015cee0 0x7f3000154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30000b3430 0x7f300014c2e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 --2- 172.21.15.86:0/3491789907 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30000b2360 0x7f30000730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 >> 172.21.15.86:0/3491789907 conn(0x7f30000cc940 msgr2=0x7f300015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 shutdown_connections 2024-03-22T04:11:19.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.255+0000 7f3005a00640 1 -- 172.21.15.86:0/3491789907 wait complete. 2024-03-22T04:11:19.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:19.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e9600640 1 Processor -- start 2024-03-22T04:11:19.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e9600640 1 -- start start 2024-03-22T04:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e9600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40c6f20 0x7f58e4072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e9600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58e4162740 0x7f58e40c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e9600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e40cc7c0 con 0x7f58e40cb1d0 2024-03-22T04:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e9600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f58e40cb720 con 0x7f58e4162740 2024-03-22T04:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e9600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40cba00 con 0x7f58e40c6f20 2024-03-22T04:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58e4162740 0x7f58e40c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58dbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f58e40cb1d0 0x7f58e4162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35366/0 (socket says 172.21.15.86:35366) 2024-03-22T04:11:19.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58dbe00640 1 -- 172.21.15.86:0/2929228976 learned_addr learned my addr 172.21.15.86:0/2929228976 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:19.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e2a00640 1 --2- 172.21.15.86:0/2929228976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40c6f20 0x7f58e4072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e3400640 1 -- 172.21.15.86:0/2929228976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40c6f20 msgr2=0x7f58e4072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:19.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e3400640 1 --2- 172.21.15.86:0/2929228976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40c6f20 0x7f58e4072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e3400640 1 -- 172.21.15.86:0/2929228976 >> v1:172.21.15.86:6789/0 conn(0x7f58e40cb1d0 legacy=0x7f58e4162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e2000640 1 -- 172.21.15.86:0/2929228976 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3204102239 0 0) 0x7f58e40cc7c0 con 0x7f58e40cb1d0 2024-03-22T04:11:19.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.415+0000 7f58e3400640 1 -- 172.21.15.86:0/2929228976 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e40cbce0 con 0x7f58e4162740 2024-03-22T04:11:19.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e3400640 1 --2- 172.21.15.86:0/2929228976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4162740 0x7f58e40c55f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009d90 tx=0x7f58dc01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6453c621dec627b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e2000640 1 -- 172.21.15.86:0/2929228976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0b2070 con 0x7f58e4162740 2024-03-22T04:11:19.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e2000640 1 -- 172.21.15.86:0/2929228976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58dc0a5970 con 0x7f58e4162740 2024-03-22T04:11:19.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e2000640 1 -- 172.21.15.86:0/2929228976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0a5d40 con 0x7f58e4162740 2024-03-22T04:11:19.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 -- 172.21.15.86:0/2929228976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4162740 msgr2=0x7f58e40c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 --2- 172.21.15.86:0/2929228976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4162740 0x7f58e40c55f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009d90 tx=0x7f58dc01c8b0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 -- 172.21.15.86:0/2929228976 shutdown_connections 2024-03-22T04:11:19.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 --2- 172.21.15.86:0/2929228976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e4162740 0x7f58e40c55f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 --2- 172.21.15.86:0/2929228976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40c6f20 0x7f58e4072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 -- 172.21.15.86:0/2929228976 >> 172.21.15.86:0/2929228976 conn(0x7f58e40d0730 msgr2=0x7f58e40d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 -- 172.21.15.86:0/2929228976 shutdown_connections 2024-03-22T04:11:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 -- 172.21.15.86:0/2929228976 wait complete. 2024-03-22T04:11:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 Processor -- start 2024-03-22T04:11:19.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.419+0000 7f58e9600640 1 -- start start 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e40c6f20 0x7f58e4156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40cb1d0 0x7f58e40c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e4162740 0x7f58e415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40c64c0 con 0x7f58e40c6f20 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40c6340 con 0x7f58e4162740 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40c6640 con 0x7f58e40cb1d0 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e40c6f20 0x7f58e4156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e40c6f20 0x7f58e4156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38536/0 (socket says 172.21.15.86:38536) 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 -- 172.21.15.86:0/222956262 learned_addr learned my addr 172.21.15.86:0/222956262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58dbe00640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40cb1d0 0x7f58e40c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e3400640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e4162740 0x7f58e415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 -- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40cb1d0 msgr2=0x7f58e40c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40cb1d0 0x7f58e40c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 -- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e4162740 msgr2=0x7f58e415eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e4162740 0x7f58e415eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 -- 172.21.15.86:0/222956262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e40ca4a0 con 0x7f58e40c6f20 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e3400640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e4162740 0x7f58e415eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e2a00640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e40c6f20 0x7f58e4156c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58d400e870 tx=0x7f58d400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58daa00640 1 -- 172.21.15.86:0/222956262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4017070 con 0x7f58e40c6f20 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58daa00640 1 -- 172.21.15.86:0/222956262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d400c940 con 0x7f58e40c6f20 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58daa00640 1 -- 172.21.15.86:0/222956262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d400ccf0 con 0x7f58e40c6f20 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e415fd50 con 0x7f58e40c6f20 2024-03-22T04:11:19.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e406ed80 con 0x7f58e40c6f20 2024-03-22T04:11:19.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e4074470 con 0x7f58e40c6f20 2024-03-22T04:11:19.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.423+0000 7f58daa00640 1 -- 172.21.15.86:0/222956262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f58d401a070 con 0x7f58e40c6f20 2024-03-22T04:11:19.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.427+0000 7f58daa00640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f58c0067b20 0x7f58c0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.427+0000 7f58daa00640 1 -- 172.21.15.86:0/222956262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f58d4086520 con 0x7f58e40c6f20 2024-03-22T04:11:19.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.427+0000 7f58dbe00640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f58c0067b20 0x7f58c0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.427+0000 7f58dbe00640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f58c0067b20 0x7f58c0069fe0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f58e40b6700 tx=0x7f58cc007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.431+0000 7f58daa00640 1 -- 172.21.15.86:0/222956262 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f58d4056490 con 0x7f58e40c6f20 2024-03-22T04:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f58e406cca0 con 0x7f58e40c6f20 2024-03-22T04:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58daa00640 1 -- 172.21.15.86:0/222956262 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f58d405a530 con 0x7f58e40c6f20 2024-03-22T04:11:19.592 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f58c0067b20 msgr2=0x7f58c0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f58c0067b20 0x7f58c0069fe0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f58e40b6700 tx=0x7f58cc007d60 comp rx=0 tx=0).stop 2024-03-22T04:11:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e40c6f20 msgr2=0x7f58e4156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e40c6f20 0x7f58e4156c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58d400e870 tx=0x7f58d400ec70 comp rx=0 tx=0).stop 2024-03-22T04:11:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 shutdown_connections 2024-03-22T04:11:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f58c0067b20 0x7f58c0069fe0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e4162740 0x7f58e415eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e40cb1d0 0x7f58e40c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 --2- 172.21.15.86:0/222956262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58e40c6f20 0x7f58e4156c20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 >> 172.21.15.86:0/222956262 conn(0x7f58e40d0730 msgr2=0x7f58e40b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 shutdown_connections 2024-03-22T04:11:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.587+0000 7f58e9600640 1 -- 172.21.15.86:0/222956262 wait complete. 2024-03-22T04:11:19.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:19.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.743+0000 7fb13ee00640 1 Processor -- start 2024-03-22T04:11:19.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.743+0000 7fb13ee00640 1 -- start start 2024-03-22T04:11:19.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb13ee00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb138147880 0x7fb13806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb13ee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb1380b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb13ee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb13815ea90 con 0x7fb1380b4890 2024-03-22T04:11:19.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb13ee00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb1380b6320 con 0x7fb138147880 2024-03-22T04:11:19.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb13ee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb13806d4f0 con 0x7fb13806cec0 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb13ce00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb1380b4890 0x7fb138162080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35368/0 (socket says 172.21.15.86:35368) 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb13ce00640 1 -- 172.21.15.86:0/488079369 learned_addr learned my addr 172.21.15.86:0/488079369 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb137e00640 1 --2- 172.21.15.86:0/488079369 >> v2:172.21.15.196:3300/0 conn(0x7fb138147880 0x7fb13806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb12f400640 1 --2- 172.21.15.86:0/488079369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb1380b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb137400640 1 -- 172.21.15.86:0/488079369 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306738014 0 0) 0x7fb13815ea90 con 0x7fb1380b4890 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb137400640 1 -- 172.21.15.86:0/488079369 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb11c003530 con 0x7fb1380b4890 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb12f400640 1 -- 172.21.15.86:0/488079369 >> v2:172.21.15.196:3300/0 conn(0x7fb138147880 msgr2=0x7fb13806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb12f400640 1 --2- 172.21.15.86:0/488079369 >> v2:172.21.15.196:3300/0 conn(0x7fb138147880 0x7fb13806c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb12f400640 1 -- 172.21.15.86:0/488079369 >> v1:172.21.15.86:6789/0 conn(0x7fb1380b4890 legacy=0x7fb138162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb12f400640 1 -- 172.21.15.86:0/488079369 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13806d7d0 con 0x7fb13806cec0 2024-03-22T04:11:19.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.747+0000 7fb137e00640 1 --2- 172.21.15.86:0/488079369 >> v2:172.21.15.196:3300/0 conn(0x7fb138147880 0x7fb13806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:19.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb12f400640 1 --2- 172.21.15.86:0/488079369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb1380b4350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb1200097c0 tx=0x7fb12001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62abd15c71ce460c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb137400640 1 -- 172.21.15.86:0/488079369 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200b0070 con 0x7fb13806cec0 2024-03-22T04:11:19.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb137400640 1 -- 172.21.15.86:0/488079369 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1200ab600 con 0x7fb13806cec0 2024-03-22T04:11:19.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb137400640 1 -- 172.21.15.86:0/488079369 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200ab9d0 con 0x7fb13806cec0 2024-03-22T04:11:19.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- 172.21.15.86:0/488079369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 msgr2=0x7fb1380b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 --2- 172.21.15.86:0/488079369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb1380b4350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb1200097c0 tx=0x7fb12001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- 172.21.15.86:0/488079369 shutdown_connections 2024-03-22T04:11:19.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 --2- 172.21.15.86:0/488079369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb1380b4350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 --2- 172.21.15.86:0/488079369 >> v2:172.21.15.196:3300/0 conn(0x7fb138147880 0x7fb13806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- 172.21.15.86:0/488079369 >> 172.21.15.86:0/488079369 conn(0x7fb13806ba60 msgr2=0x7fb13806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:19.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- 172.21.15.86:0/488079369 shutdown_connections 2024-03-22T04:11:19.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- 172.21.15.86:0/488079369 wait complete. 2024-03-22T04:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 Processor -- start 2024-03-22T04:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- start start 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb138159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1380b4890 0x7fb138159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb138147880 0x7fb13815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb13816b420 con 0x7fb1380b4890 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1380b4ca0 con 0x7fb138147880 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.751+0000 7fb13ee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb13816b5a0 con 0x7fb13806cec0 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1380b4890 0x7fb138159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1380b4890 0x7fb138159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38552/0 (socket says 172.21.15.86:38552) 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 -- 172.21.15.86:0/2248682373 learned_addr learned my addr 172.21.15.86:0/2248682373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb137e00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb138159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb13ce00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb138147880 0x7fb13815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 -- 172.21.15.86:0/2248682373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 msgr2=0x7fb138159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb138159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 -- 172.21.15.86:0/2248682373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb138147880 msgr2=0x7fb13815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb138147880 0x7fb13815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 -- 172.21.15.86:0/2248682373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13813fc00 con 0x7fb1380b4890 2024-03-22T04:11:19.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb12f400640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1380b4890 0x7fb138159760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb12001a0d0 tx=0x7fb12001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb134c00640 1 -- 172.21.15.86:0/2248682373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200b0050 con 0x7fb1380b4890 2024-03-22T04:11:19.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb134c00640 1 -- 172.21.15.86:0/2248682373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1200bc070 con 0x7fb1380b4890 2024-03-22T04:11:19.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb134c00640 1 -- 172.21.15.86:0/2248682373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120006e20 con 0x7fb1380b4890 2024-03-22T04:11:19.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb138140750 con 0x7fb1380b4890 2024-03-22T04:11:19.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb138140d10 con 0x7fb1380b4890 2024-03-22T04:11:19.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1380b1e40 con 0x7fb1380b4890 2024-03-22T04:11:19.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.755+0000 7fb134c00640 1 -- 172.21.15.86:0/2248682373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb1200b5830 con 0x7fb1380b4890 2024-03-22T04:11:19.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.759+0000 7fb134c00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb114067b20 0x7fb114069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:19.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.759+0000 7fb134c00640 1 -- 172.21.15.86:0/2248682373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fb120120790 con 0x7fb1380b4890 2024-03-22T04:11:19.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.759+0000 7fb137e00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb114067b20 0x7fb114069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:19.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.759+0000 7fb137e00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb114067b20 0x7fb114069fe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb128009580 tx=0x7fb128009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:19.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.763+0000 7fb134c00640 1 -- 172.21.15.86:0/2248682373 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb120016090 con 0x7fb1380b4890 2024-03-22T04:11:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.919+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb138140390 con 0x7fb1380b4890 2024-03-22T04:11:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.919+0000 7fb134c00640 1 -- 172.21.15.86:0/2248682373 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7fb1200f07a0 con 0x7fb1380b4890 2024-03-22T04:11:19.925 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:19.926 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:19.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb114067b20 msgr2=0x7fb114069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb114067b20 0x7fb114069fe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb128009580 tx=0x7fb128009190 comp rx=0 tx=0).stop 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1380b4890 msgr2=0x7fb138159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1380b4890 0x7fb138159760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb12001a0d0 tx=0x7fb12001ace0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 shutdown_connections 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb114067b20 0x7fb114069fe0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb138147880 0x7fb13815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1380b4890 0x7fb138159760 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 --2- 172.21.15.86:0/2248682373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb13806cec0 0x7fb138159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 >> 172.21.15.86:0/2248682373 conn(0x7fb13806ba60 msgr2=0x7fb138069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 shutdown_connections 2024-03-22T04:11:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:19.923+0000 7fb13ee00640 1 -- 172.21.15.86:0/2248682373 wait complete. 2024-03-22T04:11:19.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:19.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:20.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.083+0000 7f43b0a00640 1 Processor -- start 2024-03-22T04:11:20.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.083+0000 7f43b0a00640 1 -- start start 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43b0a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f43ac15fb40 0x7f43ac070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43b0a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac09b180 0x7f43ac06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43b0a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43ac070cb0 con 0x7f43ac0aee30 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43b0a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f43ac0d0880 con 0x7f43ac15fb40 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43b0a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac0d0b60 con 0x7f43ac09b180 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43a9400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f43ac0aee30 0x7f43ac09aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35370/0 (socket says 172.21.15.86:35370) 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43a9400640 1 -- 172.21.15.86:0/3818553468 learned_addr learned my addr 172.21.15.86:0/3818553468 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43aa800640 1 --2- 172.21.15.86:0/3818553468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac09b180 0x7f43ac06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43a9e00640 1 --2- 172.21.15.86:0/3818553468 >> v2:172.21.15.196:3300/0 conn(0x7f43ac15fb40 0x7f43ac070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43a8a00640 1 -- 172.21.15.86:0/3818553468 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564854207 0 0) 0x7f43ac070cb0 con 0x7f43ac0aee30 2024-03-22T04:11:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43a8a00640 1 -- 172.21.15.86:0/3818553468 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4390003530 con 0x7f43ac0aee30 2024-03-22T04:11:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43aa800640 1 -- 172.21.15.86:0/3818553468 >> v2:172.21.15.196:3300/0 conn(0x7f43ac15fb40 msgr2=0x7f43ac070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43aa800640 1 --2- 172.21.15.86:0/3818553468 >> v2:172.21.15.196:3300/0 conn(0x7f43ac15fb40 0x7f43ac070770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43aa800640 1 -- 172.21.15.86:0/3818553468 >> v1:172.21.15.86:6789/0 conn(0x7f43ac0aee30 legacy=0x7f43ac09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.087+0000 7f43aa800640 1 -- 172.21.15.86:0/3818553468 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac0d0e40 con 0x7f43ac09b180 2024-03-22T04:11:20.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43aa800640 1 --2- 172.21.15.86:0/3818553468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac09b180 0x7f43ac06f440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43a40097c0 tx=0x7f43a401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a0670ff31bb802 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43a8a00640 1 -- 172.21.15.86:0/3818553468 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a40b0070 con 0x7f43ac09b180 2024-03-22T04:11:20.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43a8a00640 1 -- 172.21.15.86:0/3818553468 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a40ab600 con 0x7f43ac09b180 2024-03-22T04:11:20.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43a8a00640 1 -- 172.21.15.86:0/3818553468 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a40ab9d0 con 0x7f43ac09b180 2024-03-22T04:11:20.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 -- 172.21.15.86:0/3818553468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac09b180 msgr2=0x7f43ac06f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 --2- 172.21.15.86:0/3818553468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac09b180 0x7f43ac06f440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43a40097c0 tx=0x7f43a401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 -- 172.21.15.86:0/3818553468 shutdown_connections 2024-03-22T04:11:20.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 --2- 172.21.15.86:0/3818553468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac09b180 0x7f43ac06f440 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 --2- 172.21.15.86:0/3818553468 >> v2:172.21.15.196:3300/0 conn(0x7f43ac15fb40 0x7f43ac070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 -- 172.21.15.86:0/3818553468 >> 172.21.15.86:0/3818553468 conn(0x7f43ac0b3bf0 msgr2=0x7f43ac0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:20.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 -- 172.21.15.86:0/3818553468 shutdown_connections 2024-03-22T04:11:20.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 -- 172.21.15.86:0/3818553468 wait complete. 2024-03-22T04:11:20.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 Processor -- start 2024-03-22T04:11:20.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.091+0000 7f43b0a00640 1 -- start start 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac09b180 0x7f43ac0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43ac0aee30 0x7f43ac0c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac15fb40 0x7f43ac0c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac169460 con 0x7f43ac09b180 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1692e0 con 0x7f43ac0aee30 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac1695e0 con 0x7f43ac15fb40 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac09b180 0x7f43ac0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43ac0aee30 0x7f43ac0c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac09b180 0x7f43ac0ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38564/0 (socket says 172.21.15.86:38564) 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 -- 172.21.15.86:0/2615343564 learned_addr learned my addr 172.21.15.86:0/2615343564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43aa800640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac15fb40 0x7f43ac0c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 -- 172.21.15.86:0/2615343564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac15fb40 msgr2=0x7f43ac0c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac15fb40 0x7f43ac0c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 -- 172.21.15.86:0/2615343564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43ac0aee30 msgr2=0x7f43ac0c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43ac0aee30 0x7f43ac0c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 -- 172.21.15.86:0/2615343564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac0c81b0 con 0x7f43ac09b180 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac09b180 0x7f43ac0ca7f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f439c00efc0 tx=0x7f439c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a2000640 1 -- 172.21.15.86:0/2615343564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c018020 con 0x7f43ac09b180 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a2000640 1 -- 172.21.15.86:0/2615343564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c014480 con 0x7f43ac09b180 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a2000640 1 -- 172.21.15.86:0/2615343564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c014850 con 0x7f43ac09b180 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac0c78a0 con 0x7f43ac09b180 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac1faf60 con 0x7f43ac09b180 2024-03-22T04:11:20.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ac074470 con 0x7f43ac09b180 2024-03-22T04:11:20.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.095+0000 7f43a2000640 1 -- 172.21.15.86:0/2615343564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f439c01b020 con 0x7f43ac09b180 2024-03-22T04:11:20.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.099+0000 7f43a2000640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f438c067b20 0x7f438c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.099+0000 7f43a2000640 1 -- 172.21.15.86:0/2615343564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f439c086be0 con 0x7f43ac09b180 2024-03-22T04:11:20.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.099+0000 7f43a9400640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f438c067b20 0x7f438c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.099+0000 7f43a9400640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f438c067b20 0x7f438c069fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4398007fd0 tx=0x7f4398009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.103+0000 7f43a2000640 1 -- 172.21.15.86:0/2615343564 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f439c022070 con 0x7f43ac09b180 2024-03-22T04:11:20.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.267+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f43ac075170 con 0x7f43ac09b180 2024-03-22T04:11:20.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.267+0000 7f43a2000640 1 -- 172.21.15.86:0/2615343564 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f439c010040 con 0x7f43ac09b180 2024-03-22T04:11:20.272 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:20.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.267+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f438c067b20 msgr2=0x7f438c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.267+0000 7f43b0a00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f438c067b20 0x7f438c069fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4398007fd0 tx=0x7f4398009040 comp rx=0 tx=0).stop 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.267+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac09b180 msgr2=0x7f43ac0ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.267+0000 7f43b0a00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac09b180 0x7f43ac0ca7f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f439c00efc0 tx=0x7f439c00c490 comp rx=0 tx=0).stop 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.271+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 shutdown_connections 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.271+0000 7f43b0a00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f438c067b20 0x7f438c069fe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.271+0000 7f43b0a00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43ac0aee30 0x7f43ac0c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.271+0000 7f43b0a00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43ac15fb40 0x7f43ac0c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.271+0000 7f43b0a00640 1 --2- 172.21.15.86:0/2615343564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac09b180 0x7f43ac0ca7f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.271+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 >> 172.21.15.86:0/2615343564 conn(0x7f43ac0b3bf0 msgr2=0x7f43ac0c91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.271+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 shutdown_connections 2024-03-22T04:11:20.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.271+0000 7f43b0a00640 1 -- 172.21.15.86:0/2615343564 wait complete. 2024-03-22T04:11:20.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:20.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860f000640 1 Processor -- start 2024-03-22T04:11:20.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860f000640 1 -- start start 2024-03-22T04:11:20.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860f000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f860806e7e0 0x7f860806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860f000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8608072650 0x7f8608072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86080b1b80 0x7f86080b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8608071350 con 0x7f86080b1b80 2024-03-22T04:11:20.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860f000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f860806f100 con 0x7f860806e7e0 2024-03-22T04:11:20.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860f000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8608073150 con 0x7f8608072650 2024-03-22T04:11:20.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86080b1b80 0x7f86080b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86080b1b80 0x7f86080b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38568/0 (socket says 172.21.15.86:38568) 2024-03-22T04:11:20.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860d000640 1 -- 172.21.15.86:0/1134423969 learned_addr learned my addr 172.21.15.86:0/1134423969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:20.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860d000640 1 -- 172.21.15.86:0/1134423969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8608072650 msgr2=0x7f8608072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:20.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860d000640 1 --2- 172.21.15.86:0/1134423969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8608072650 0x7f8608072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860d000640 1 -- 172.21.15.86:0/1134423969 >> v2:172.21.15.196:3300/0 conn(0x7f860806e7e0 msgr2=0x7f860806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:20.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860d000640 1 --2- 172.21.15.86:0/1134423969 >> v2:172.21.15.196:3300/0 conn(0x7f860806e7e0 0x7f860806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.427+0000 7f860d000640 1 -- 172.21.15.86:0/1134423969 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86080715a0 con 0x7f86080b1b80 2024-03-22T04:11:20.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860d000640 1 --2- 172.21.15.86:0/1134423969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86080b1b80 0x7f86080b2010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f86080ce650 tx=0x7f860001c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1ade9513ab48a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f8607400640 1 -- 172.21.15.86:0/1134423969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86000a55d0 con 0x7f86080b1b80 2024-03-22T04:11:20.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f8607400640 1 -- 172.21.15.86:0/1134423969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86000a5770 con 0x7f86080b1b80 2024-03-22T04:11:20.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f8607400640 1 -- 172.21.15.86:0/1134423969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86000a5b40 con 0x7f86080b1b80 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 -- 172.21.15.86:0/1134423969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86080b1b80 msgr2=0x7f86080b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 --2- 172.21.15.86:0/1134423969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86080b1b80 0x7f86080b2010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f86080ce650 tx=0x7f860001c750 comp rx=0 tx=0).stop 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 -- 172.21.15.86:0/1134423969 shutdown_connections 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 --2- 172.21.15.86:0/1134423969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86080b1b80 0x7f86080b2010 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 --2- 172.21.15.86:0/1134423969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8608072650 0x7f8608072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 --2- 172.21.15.86:0/1134423969 >> v2:172.21.15.196:3300/0 conn(0x7f860806e7e0 0x7f860806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 -- 172.21.15.86:0/1134423969 >> 172.21.15.86:0/1134423969 conn(0x7f86080cc940 msgr2=0x7f86080ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 -- 172.21.15.86:0/1134423969 shutdown_connections 2024-03-22T04:11:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.431+0000 7f860f000640 1 -- 172.21.15.86:0/1134423969 wait complete. 2024-03-22T04:11:20.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 Processor -- start 2024-03-22T04:11:20.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 -- start start 2024-03-22T04:11:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860806e7e0 0x7f8608140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608072650 0x7f86080c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86080b1b80 0x7f8608157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8608072080 con 0x7f8608072650 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86080ce930 con 0x7f860806e7e0 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8608156ad0 con 0x7f86080b1b80 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608072650 0x7f86080c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608072650 0x7f86080c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38584/0 (socket says 172.21.15.86:38584) 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 -- 172.21.15.86:0/3781344867 learned_addr learned my addr 172.21.15.86:0/3781344867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f8607e00640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860806e7e0 0x7f8608140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860d000640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86080b1b80 0x7f8608157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 -- 172.21.15.86:0/3781344867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86080b1b80 msgr2=0x7f8608157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86080b1b80 0x7f8608157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 -- 172.21.15.86:0/3781344867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860806e7e0 msgr2=0x7f8608140850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860806e7e0 0x7f8608140850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 -- 172.21.15.86:0/3781344867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8608156db0 con 0x7f8608072650 2024-03-22T04:11:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f85ff400640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608072650 0x7f86080c4e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b410 tx=0x7f85f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f8604c00640 1 -- 172.21.15.86:0/3781344867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0015020 con 0x7f8608072650 2024-03-22T04:11:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f8604c00640 1 -- 172.21.15.86:0/3781344867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f0011830 con 0x7f8608072650 2024-03-22T04:11:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f8604c00640 1 -- 172.21.15.86:0/3781344867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0011c20 con 0x7f8608072650 2024-03-22T04:11:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8608157090 con 0x7f8608072650 2024-03-22T04:11:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8608157650 con 0x7f8608072650 2024-03-22T04:11:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.435+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8608074470 con 0x7f8608072650 2024-03-22T04:11:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.439+0000 7f8604c00640 1 -- 172.21.15.86:0/3781344867 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f85f0018020 con 0x7f8608072650 2024-03-22T04:11:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.439+0000 7f8604c00640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f85e4067b20 0x7f85e4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.439+0000 7f8604c00640 1 -- 172.21.15.86:0/3781344867 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f85f00861d0 con 0x7f8608072650 2024-03-22T04:11:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.439+0000 7f8607e00640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f85e4067b20 0x7f85e4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.439+0000 7f8607e00640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f85e4067b20 0x7f85e4069fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85f80098f0 tx=0x7f85f8006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.443+0000 7f8604c00640 1 -- 172.21.15.86:0/3781344867 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f85f001f020 con 0x7f8608072650 2024-03-22T04:11:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.595+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f86080b4c80 con 0x7f8608072650 2024-03-22T04:11:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.595+0000 7f8604c00640 1 -- 172.21.15.86:0/3781344867 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f85f0056140 con 0x7f8608072650 2024-03-22T04:11:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f85e4067b20 msgr2=0x7f85e4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f85e4067b20 0x7f85e4069fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85f80098f0 tx=0x7f85f8006f90 comp rx=0 tx=0).stop 2024-03-22T04:11:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608072650 msgr2=0x7f86080c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608072650 0x7f86080c4e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b410 tx=0x7f85f000b8d0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 shutdown_connections 2024-03-22T04:11:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f85e4067b20 0x7f85e4069fe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86080b1b80 0x7f8608157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608072650 0x7f86080c4e60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 --2- 172.21.15.86:0/3781344867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860806e7e0 0x7f8608140850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 >> 172.21.15.86:0/3781344867 conn(0x7f86080cc940 msgr2=0x7f8608154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 shutdown_connections 2024-03-22T04:11:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.599+0000 7f860f000640 1 -- 172.21.15.86:0/3781344867 wait complete. 2024-03-22T04:11:20.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f5464a00640 1 Processor -- start 2024-03-22T04:11:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f5464a00640 1 -- start start 2024-03-22T04:11:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f5464a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f54600d0a40 0x7f54600d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f5464a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f54600b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f5464a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54600d4d40 con 0x7f54600b5220 2024-03-22T04:11:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f5464a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f54600cfcf0 con 0x7f54600d0a40 2024-03-22T04:11:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f5464a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f54600d2a40 con 0x7f54600d2f40 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545f200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f54600b5220 0x7f54600b3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35384/0 (socket says 172.21.15.86:35384) 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545f200640 1 -- 172.21.15.86:0/2785423570 learned_addr learned my addr 172.21.15.86:0/2785423570 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545e800640 1 --2- 172.21.15.86:0/2785423570 >> v2:172.21.15.196:3300/0 conn(0x7f54600d0a40 0x7f54600d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545de00640 1 --2- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f54600b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545d400640 1 -- 172.21.15.86:0/2785423570 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149305996 0 0) 0x7f54600d4d40 con 0x7f54600b5220 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545d400640 1 -- 172.21.15.86:0/2785423570 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5444003530 con 0x7f54600b5220 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545e800640 1 -- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 msgr2=0x7f54600b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545e800640 1 --2- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f54600b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545e800640 1 -- 172.21.15.86:0/2785423570 >> v1:172.21.15.86:6789/0 conn(0x7f54600b5220 legacy=0x7f54600b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545e800640 1 -- 172.21.15.86:0/2785423570 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54600b4e00 con 0x7f54600d0a40 2024-03-22T04:11:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.755+0000 7f545de00640 1 --2- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f54600b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f545e800640 1 --2- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600d0a40 0x7f54600d0e20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f54500097c0 tx=0x7f545001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c88844abb290e03 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f545d400640 1 -- 172.21.15.86:0/2785423570 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500b1070 con 0x7f54600d0a40 2024-03-22T04:11:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f545d400640 1 -- 172.21.15.86:0/2785423570 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54500ac410 con 0x7f54600d0a40 2024-03-22T04:11:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f545d400640 1 -- 172.21.15.86:0/2785423570 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500ac7c0 con 0x7f54600d0a40 2024-03-22T04:11:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f5464a00640 1 -- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600d0a40 msgr2=0x7f54600d0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f5464a00640 1 --2- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600d0a40 0x7f54600d0e20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f54500097c0 tx=0x7f545001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f5464a00640 1 -- 172.21.15.86:0/2785423570 shutdown_connections 2024-03-22T04:11:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f5464a00640 1 --2- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f54600b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f5464a00640 1 --2- 172.21.15.86:0/2785423570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600d0a40 0x7f54600d0e20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f5464a00640 1 -- 172.21.15.86:0/2785423570 >> 172.21.15.86:0/2785423570 conn(0x7f54600d40d0 msgr2=0x7f54600d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f5464a00640 1 -- 172.21.15.86:0/2785423570 shutdown_connections 2024-03-22T04:11:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.759+0000 7f5464a00640 1 -- 172.21.15.86:0/2785423570 wait complete. 2024-03-22T04:11:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 Processor -- start 2024-03-22T04:11:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 -- start start 2024-03-22T04:11:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600b5220 0x7f54601578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54600d0a40 0x7f5460163220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f5460158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54600cf4f0 con 0x7f54600d0a40 2024-03-22T04:11:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54600b4890 con 0x7f54600b5220 2024-03-22T04:11:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f54600cf670 con 0x7f54600d2f40 2024-03-22T04:11:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54600d0a40 0x7f5460163220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54600d0a40 0x7f5460163220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38590/0 (socket says 172.21.15.86:38590) 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 -- 172.21.15.86:0/3705289459 learned_addr learned my addr 172.21.15.86:0/3705289459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545e800640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600b5220 0x7f54601578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545f200640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f5460158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 -- 172.21.15.86:0/3705289459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 msgr2=0x7f5460158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f5460158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 -- 172.21.15.86:0/3705289459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600b5220 msgr2=0x7f54601578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600b5220 0x7f54601578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 -- 172.21.15.86:0/3705289459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54600b6ad0 con 0x7f54600d0a40 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f545de00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54600d0a40 0x7f5460163220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f544c00e930 tx=0x7f544c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5456a00640 1 -- 172.21.15.86:0/3705289459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c017070 con 0x7f54600d0a40 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5456a00640 1 -- 172.21.15.86:0/3705289459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544c00ca00 con 0x7f54600d0a40 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54600c8bd0 con 0x7f54600d0a40 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.763+0000 7f5456a00640 1 -- 172.21.15.86:0/3705289459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c00cdd0 con 0x7f54600d0a40 2024-03-22T04:11:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.767+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5460162740 con 0x7f54600d0a40 2024-03-22T04:11:20.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.767+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546004f900 con 0x7f54600d0a40 2024-03-22T04:11:20.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.767+0000 7f5456a00640 1 -- 172.21.15.86:0/3705289459 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f544c01a070 con 0x7f54600d0a40 2024-03-22T04:11:20.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.767+0000 7f5456a00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5440067b20 0x7f5440069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:20.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.767+0000 7f5456a00640 1 -- 172.21.15.86:0/3705289459 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f544c086120 con 0x7f54600d0a40 2024-03-22T04:11:20.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.767+0000 7f545e800640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5440067b20 0x7f5440069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:20.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.767+0000 7f545e800640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5440067b20 0x7f5440069fe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54500ae000 tx=0x7f5450008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.771+0000 7f5456a00640 1 -- 172.21.15.86:0/3705289459 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f544c056160 con 0x7f54600d0a40 2024-03-22T04:11:20.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f546008c100 con 0x7f54600d0a40 2024-03-22T04:11:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5456a00640 1 -- 172.21.15.86:0/3705289459 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f544c05a200 con 0x7f54600d0a40 2024-03-22T04:11:20.932 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:20.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5440067b20 msgr2=0x7f5440069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5440067b20 0x7f5440069fe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54500ae000 tx=0x7f5450008040 comp rx=0 tx=0).stop 2024-03-22T04:11:20.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54600d0a40 msgr2=0x7f5460163220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:20.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54600d0a40 0x7f5460163220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f544c00e930 tx=0x7f544c00edf0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 shutdown_connections 2024-03-22T04:11:20.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5440067b20 0x7f5440069fe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54600d2f40 0x7f5460158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54600d0a40 0x7f5460163220 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 --2- 172.21.15.86:0/3705289459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54600b5220 0x7f54601578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:20.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 >> 172.21.15.86:0/3705289459 conn(0x7f54600d40d0 msgr2=0x7f546006ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:20.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 shutdown_connections 2024-03-22T04:11:20.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:20.927+0000 7f5464a00640 1 -- 172.21.15.86:0/3705289459 wait complete. 2024-03-22T04:11:20.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:21.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82be800640 1 Processor -- start 2024-03-22T04:11:21.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82be800640 1 -- start start 2024-03-22T04:11:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82be800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f82b80b2360 0x7f82b80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82be800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82be800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82b815d710 con 0x7f82b80b3430 2024-03-22T04:11:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82be800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f82b80b2740 con 0x7f82b80b2360 2024-03-22T04:11:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82be800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f82b815d4b0 con 0x7f82b815d0b0 2024-03-22T04:11:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f82b80b3430 0x7f82b815c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35396/0 (socket says 172.21.15.86:35396) 2024-03-22T04:11:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b6a00640 1 -- 172.21.15.86:0/3955165472 learned_addr learned my addr 172.21.15.86:0/3955165472 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b7400640 1 --2- 172.21.15.86:0/3955165472 >> v2:172.21.15.196:3300/0 conn(0x7f82b80b2360 0x7f82b80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b7e00640 1 --2- 172.21.15.86:0/3955165472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b6000640 1 -- 172.21.15.86:0/3955165472 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1060021286 0 0) 0x7f82b815d710 con 0x7f82b80b3430 2024-03-22T04:11:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b6000640 1 -- 172.21.15.86:0/3955165472 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f829c003530 con 0x7f82b80b3430 2024-03-22T04:11:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b7e00640 1 -- 172.21.15.86:0/3955165472 >> v2:172.21.15.196:3300/0 conn(0x7f82b80b2360 msgr2=0x7f82b80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:21.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b7e00640 1 --2- 172.21.15.86:0/3955165472 >> v2:172.21.15.196:3300/0 conn(0x7f82b80b2360 0x7f82b80b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b7e00640 1 -- 172.21.15.86:0/3955165472 >> v1:172.21.15.86:6789/0 conn(0x7f82b80b3430 legacy=0x7f82b815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b7e00640 1 -- 172.21.15.86:0/3955165472 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b815fe40 con 0x7f82b815d0b0 2024-03-22T04:11:21.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.087+0000 7f82b7400640 1 --2- 172.21.15.86:0/3955165472 >> v2:172.21.15.196:3300/0 conn(0x7f82b80b2360 0x7f82b80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:21.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82b7e00640 1 --2- 172.21.15.86:0/3955165472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b80b4d80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009d90 tx=0x7f82b001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3474c0c368647c1e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82b6000640 1 -- 172.21.15.86:0/3955165472 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b00b2070 con 0x7f82b815d0b0 2024-03-22T04:11:21.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82b6000640 1 -- 172.21.15.86:0/3955165472 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82b00a5920 con 0x7f82b815d0b0 2024-03-22T04:11:21.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82b6000640 1 -- 172.21.15.86:0/3955165472 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b00a5cf0 con 0x7f82b815d0b0 2024-03-22T04:11:21.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 -- 172.21.15.86:0/3955165472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 msgr2=0x7f82b80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 --2- 172.21.15.86:0/3955165472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b80b4d80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009d90 tx=0x7f82b001c970 comp rx=0 tx=0).stop 2024-03-22T04:11:21.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 -- 172.21.15.86:0/3955165472 shutdown_connections 2024-03-22T04:11:21.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 --2- 172.21.15.86:0/3955165472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b80b4d80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 --2- 172.21.15.86:0/3955165472 >> v2:172.21.15.196:3300/0 conn(0x7f82b80b2360 0x7f82b80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 -- 172.21.15.86:0/3955165472 >> 172.21.15.86:0/3955165472 conn(0x7f82b80cc940 msgr2=0x7f82b80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:21.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 -- 172.21.15.86:0/3955165472 shutdown_connections 2024-03-22T04:11:21.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 -- 172.21.15.86:0/3955165472 wait complete. 2024-03-22T04:11:21.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.091+0000 7f82be800640 1 Processor -- start 2024-03-22T04:11:21.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 -- start start 2024-03-22T04:11:21.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82b80b2360 0x7f82b814c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b80b3430 0x7f82b8151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82b8160c00 con 0x7f82b80b3430 2024-03-22T04:11:21.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82b80b5ad0 con 0x7f82b80b2360 2024-03-22T04:11:21.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f82b8151620 con 0x7f82b815d0b0 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b80b3430 0x7f82b8151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b80b3430 0x7f82b8151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38594/0 (socket says 172.21.15.86:38594) 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 -- 172.21.15.86:0/684338502 learned_addr learned my addr 172.21.15.86:0/684338502 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b7e00640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b7400640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82b80b2360 0x7f82b814c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 -- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 msgr2=0x7f82b815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 -- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82b80b2360 msgr2=0x7f82b814c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82b80b2360 0x7f82b814c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 -- 172.21.15.86:0/684338502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b8150aa0 con 0x7f82b80b3430 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b7400640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82b80b2360 0x7f82b814c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82b6a00640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b80b3430 0x7f82b8151e00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82a40010f0 tx=0x7f82a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82af400640 1 -- 172.21.15.86:0/684338502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4017070 con 0x7f82b80b3430 2024-03-22T04:11:21.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82af400640 1 -- 172.21.15.86:0/684338502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a4005d10 con 0x7f82b80b3430 2024-03-22T04:11:21.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82af400640 1 -- 172.21.15.86:0/684338502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4007310 con 0x7f82b80b3430 2024-03-22T04:11:21.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b8150d80 con 0x7f82b80b3430 2024-03-22T04:11:21.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b8151340 con 0x7f82b80b3430 2024-03-22T04:11:21.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b81f5230 con 0x7f82b80b3430 2024-03-22T04:11:21.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.095+0000 7f82af400640 1 -- 172.21.15.86:0/684338502 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f82a401a070 con 0x7f82b80b3430 2024-03-22T04:11:21.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.099+0000 7f82af400640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8298067b20 0x7f8298069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.099+0000 7f82af400640 1 -- 172.21.15.86:0/684338502 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f82a4085710 con 0x7f82b80b3430 2024-03-22T04:11:21.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.099+0000 7f82b7400640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8298067b20 0x7f8298069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.099+0000 7f82b7400640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8298067b20 0x7f8298069fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009bc0 tx=0x7f82a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.103+0000 7f82af400640 1 -- 172.21.15.86:0/684338502 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f82b81f5230 con 0x7f82b80b3430 2024-03-22T04:11:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.255+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f82b8074470 con 0x7f82b80b3430 2024-03-22T04:11:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.255+0000 7f82af400640 1 -- 172.21.15.86:0/684338502 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f82a4055680 con 0x7f82b80b3430 2024-03-22T04:11:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8298067b20 msgr2=0x7f8298069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8298067b20 0x7f8298069fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009bc0 tx=0x7f82a8008040 comp rx=0 tx=0).stop 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b80b3430 msgr2=0x7f82b8151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b80b3430 0x7f82b8151e00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82a40010f0 tx=0x7f82a400c8f0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 shutdown_connections 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8298067b20 0x7f8298069fe0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82b815d0b0 0x7f82b815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b80b3430 0x7f82b8151e00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 --2- 172.21.15.86:0/684338502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82b80b2360 0x7f82b814c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 >> 172.21.15.86:0/684338502 conn(0x7f82b80cc940 msgr2=0x7f82b80afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 shutdown_connections 2024-03-22T04:11:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.259+0000 7f82be800640 1 -- 172.21.15.86:0/684338502 wait complete. 2024-03-22T04:11:21.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.411+0000 7f07e5a00640 1 Processor -- start 2024-03-22T04:11:21.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.411+0000 7f07e5a00640 1 -- start start 2024-03-22T04:11:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.411+0000 7f07e5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07e0147880 0x7f07e006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.411+0000 7f07e5a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f07e006cec0 0x7f07e00b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.411+0000 7f07e5a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e015ea90 con 0x7f07e00b4890 2024-03-22T04:11:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.411+0000 7f07e5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f07e00b6320 con 0x7f07e0147880 2024-03-22T04:11:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.411+0000 7f07e5a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f07e00b4ce0 con 0x7f07e006cec0 2024-03-22T04:11:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.411+0000 7f07dea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07e0147880 0x7f07e006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07de000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f07e006cec0 0x7f07e00b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07e0147880 0x7f07e006c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38600/0 (socket says 172.21.15.86:38600) 2024-03-22T04:11:21.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dea00640 1 -- 172.21.15.86:0/4013999882 learned_addr learned my addr 172.21.15.86:0/4013999882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:21.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dea00640 1 -- 172.21.15.86:0/4013999882 >> v2:172.21.15.196:3300/0 conn(0x7f07e006cec0 msgr2=0x7f07e00b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:21.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dea00640 1 --2- 172.21.15.86:0/4013999882 >> v2:172.21.15.196:3300/0 conn(0x7f07e006cec0 0x7f07e00b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dea00640 1 -- 172.21.15.86:0/4013999882 >> v1:172.21.15.86:6789/0 conn(0x7f07e00b4890 legacy=0x7f07e0162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dea00640 1 -- 172.21.15.86:0/4013999882 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e006d4f0 con 0x7f07e0147880 2024-03-22T04:11:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dea00640 1 --2- 172.21.15.86:0/4013999882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e0147880 0x7f07e006c980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009870 tx=0x7f07d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28de0c04f056b5aa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dd600640 1 -- 172.21.15.86:0/4013999882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d00b0070 con 0x7f07e0147880 2024-03-22T04:11:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dd600640 1 -- 172.21.15.86:0/4013999882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d00ab600 con 0x7f07e0147880 2024-03-22T04:11:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.415+0000 7f07dd600640 1 -- 172.21.15.86:0/4013999882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d00ab9d0 con 0x7f07e0147880 2024-03-22T04:11:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- 172.21.15.86:0/4013999882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e0147880 msgr2=0x7f07e006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 --2- 172.21.15.86:0/4013999882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e0147880 0x7f07e006c980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009870 tx=0x7f07d001c520 comp rx=0 tx=0).stop 2024-03-22T04:11:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- 172.21.15.86:0/4013999882 shutdown_connections 2024-03-22T04:11:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 --2- 172.21.15.86:0/4013999882 >> v2:172.21.15.196:3300/0 conn(0x7f07e006cec0 0x7f07e00b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 --2- 172.21.15.86:0/4013999882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e0147880 0x7f07e006c980 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- 172.21.15.86:0/4013999882 >> 172.21.15.86:0/4013999882 conn(0x7f07e006ba60 msgr2=0x7f07e006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- 172.21.15.86:0/4013999882 shutdown_connections 2024-03-22T04:11:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- 172.21.15.86:0/4013999882 wait complete. 2024-03-22T04:11:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 Processor -- start 2024-03-22T04:11:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- start start 2024-03-22T04:11:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07e006cec0 0x7f07e0146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e00b4890 0x7f07e013ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07e0147880 0x7f07e0160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07e016b420 con 0x7f07e00b4890 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01480a0 con 0x7f07e0147880 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07e5a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f07e016b5a0 con 0x7f07e006cec0 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e00b4890 0x7f07e013ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e00b4890 0x7f07e013ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38604/0 (socket says 172.21.15.86:38604) 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 -- 172.21.15.86:0/2478195622 learned_addr learned my addr 172.21.15.86:0/2478195622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07dea00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07e006cec0 0x7f07e0146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07df400640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07e0147880 0x7f07e0160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 -- 172.21.15.86:0/2478195622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07e006cec0 msgr2=0x7f07e0146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07e006cec0 0x7f07e0146cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 -- 172.21.15.86:0/2478195622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07e0147880 msgr2=0x7f07e0160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07e0147880 0x7f07e0160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 -- 172.21.15.86:0/2478195622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e0145c20 con 0x7f07e00b4890 2024-03-22T04:11:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07de000640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e00b4890 0x7f07e013ff50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e930 tx=0x7f07cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.419+0000 7f07d6a00640 1 -- 172.21.15.86:0/2478195622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc017070 con 0x7f07e00b4890 2024-03-22T04:11:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07d6a00640 1 -- 172.21.15.86:0/2478195622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc00ca00 con 0x7f07e00b4890 2024-03-22T04:11:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e0145f30 con 0x7f07e00b4890 2024-03-22T04:11:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07d6a00640 1 -- 172.21.15.86:0/2478195622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00cdb0 con 0x7f07e00b4890 2024-03-22T04:11:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01464f0 con 0x7f07e00b4890 2024-03-22T04:11:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e0068650 con 0x7f07e00b4890 2024-03-22T04:11:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07d6a00640 1 -- 172.21.15.86:0/2478195622 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f07cc01a070 con 0x7f07e00b4890 2024-03-22T04:11:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07d6a00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f07c0067b20 0x7f07c0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07d6a00640 1 -- 172.21.15.86:0/2478195622 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f07cc086250 con 0x7f07e00b4890 2024-03-22T04:11:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07dea00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f07c0067b20 0x7f07c0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.423+0000 7f07dea00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f07c0067b20 0x7f07c0069fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07d00ad000 tx=0x7f07d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.427+0000 7f07d6a00640 1 -- 172.21.15.86:0/2478195622 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f07cc056310 con 0x7f07e00b4890 2024-03-22T04:11:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.583+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07e0072220 con 0x7f07e00b4890 2024-03-22T04:11:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.583+0000 7f07d6a00640 1 -- 172.21.15.86:0/2478195622 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7f07cc05a3b0 con 0x7f07e00b4890 2024-03-22T04:11:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:21.590 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:21.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f07c0067b20 msgr2=0x7f07c0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f07c0067b20 0x7f07c0069fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07d00ad000 tx=0x7f07d0008040 comp rx=0 tx=0).stop 2024-03-22T04:11:21.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e00b4890 msgr2=0x7f07e013ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e00b4890 0x7f07e013ff50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e930 tx=0x7f07cc00edf0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 shutdown_connections 2024-03-22T04:11:21.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f07c0067b20 0x7f07c0069fe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07e0147880 0x7f07e0160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e00b4890 0x7f07e013ff50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 --2- 172.21.15.86:0/2478195622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07e006cec0 0x7f07e0146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 >> 172.21.15.86:0/2478195622 conn(0x7f07e006ba60 msgr2=0x7f07e006f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:21.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 shutdown_connections 2024-03-22T04:11:21.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.587+0000 7f07e5a00640 1 -- 172.21.15.86:0/2478195622 wait complete. 2024-03-22T04:11:21.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:21.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:21.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.743+0000 7f6c2ae00640 1 Processor -- start 2024-03-22T04:11:21.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.743+0000 7f6c2ae00640 1 -- start start 2024-03-22T04:11:21.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2ae00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6c2415c230 0x7f6c240d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2ae00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240ce900 0x7f6c240ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2ae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c240cf250 con 0x7f6c240d2b30 2024-03-22T04:11:21.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2ae00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6c240d05d0 con 0x7f6c2415c230 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2ae00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6c240cdd60 con 0x7f6c240ce900 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c29000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6c240d2b30 0x7f6c240cd650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35412/0 (socket says 172.21.15.86:35412) 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c29000640 1 -- 172.21.15.86:0/2271424741 learned_addr learned my addr 172.21.15.86:0/2271424741 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c29a00640 1 --2- 172.21.15.86:0/2271424741 >> v2:172.21.15.196:3300/0 conn(0x7f6c2415c230 0x7f6c240d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2a400640 1 --2- 172.21.15.86:0/2271424741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240ce900 0x7f6c240ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c23e00640 1 -- 172.21.15.86:0/2271424741 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905175 0 0) 0x7f6c240cf250 con 0x7f6c240d2b30 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c23e00640 1 -- 172.21.15.86:0/2271424741 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c0c003530 con 0x7f6c240d2b30 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2a400640 1 -- 172.21.15.86:0/2271424741 >> v2:172.21.15.196:3300/0 conn(0x7f6c2415c230 msgr2=0x7f6c240d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2a400640 1 --2- 172.21.15.86:0/2271424741 >> v2:172.21.15.196:3300/0 conn(0x7f6c2415c230 0x7f6c240d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2a400640 1 -- 172.21.15.86:0/2271424741 >> v1:172.21.15.86:6789/0 conn(0x7f6c240d2b30 legacy=0x7f6c240cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.747+0000 7f6c2a400640 1 -- 172.21.15.86:0/2271424741 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c240cfca0 con 0x7f6c240ce900 2024-03-22T04:11:21.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2a400640 1 --2- 172.21.15.86:0/2271424741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240ce900 0x7f6c240ced10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009d90 tx=0x7f6c1c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74ded07fcd1b8520 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c23e00640 1 -- 172.21.15.86:0/2271424741 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0b2070 con 0x7f6c240ce900 2024-03-22T04:11:21.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c23e00640 1 -- 172.21.15.86:0/2271424741 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0a5940 con 0x7f6c240ce900 2024-03-22T04:11:21.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c23e00640 1 -- 172.21.15.86:0/2271424741 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0a5d10 con 0x7f6c240ce900 2024-03-22T04:11:21.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- 172.21.15.86:0/2271424741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240ce900 msgr2=0x7f6c240ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/2271424741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240ce900 0x7f6c240ced10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009d90 tx=0x7f6c1c01c970 comp rx=0 tx=0).stop 2024-03-22T04:11:21.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- 172.21.15.86:0/2271424741 shutdown_connections 2024-03-22T04:11:21.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/2271424741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240ce900 0x7f6c240ced10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/2271424741 >> v2:172.21.15.196:3300/0 conn(0x7f6c2415c230 0x7f6c240d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- 172.21.15.86:0/2271424741 >> 172.21.15.86:0/2271424741 conn(0x7f6c240d1f30 msgr2=0x7f6c240d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- 172.21.15.86:0/2271424741 shutdown_connections 2024-03-22T04:11:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- 172.21.15.86:0/2271424741 wait complete. 2024-03-22T04:11:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 Processor -- start 2024-03-22T04:11:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- start start 2024-03-22T04:11:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240ce900 0x7f6c24154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240d2b30 0x7f6c2406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2415c230 0x7f6c240b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c240b5920 con 0x7f6c240ce900 2024-03-22T04:11:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c240b57a0 con 0x7f6c2415c230 2024-03-22T04:11:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.751+0000 7f6c2ae00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6c24168ff0 con 0x7f6c240d2b30 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240ce900 0x7f6c24154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240ce900 0x7f6c24154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38610/0 (socket says 172.21.15.86:38610) 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 -- 172.21.15.86:0/4194929686 learned_addr learned my addr 172.21.15.86:0/4194929686 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29000640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240d2b30 0x7f6c2406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c2a400640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2415c230 0x7f6c240b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 -- 172.21.15.86:0/4194929686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240d2b30 msgr2=0x7f6c2406d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240d2b30 0x7f6c2406d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 -- 172.21.15.86:0/4194929686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2415c230 msgr2=0x7f6c240b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2415c230 0x7f6c240b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 -- 172.21.15.86:0/4194929686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2406d6c0 con 0x7f6c240ce900 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c29a00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240ce900 0x7f6c24154cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800e9b0 tx=0x7f6c1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c21600640 1 -- 172.21.15.86:0/4194929686 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18017070 con 0x7f6c240ce900 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c21600640 1 -- 172.21.15.86:0/4194929686 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1800c9a0 con 0x7f6c240ce900 2024-03-22T04:11:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c21600640 1 -- 172.21.15.86:0/4194929686 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800cd50 con 0x7f6c240ce900 2024-03-22T04:11:21.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c240c5900 con 0x7f6c240ce900 2024-03-22T04:11:21.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c240c5ec0 con 0x7f6c240ce900 2024-03-22T04:11:21.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2404f900 con 0x7f6c240ce900 2024-03-22T04:11:21.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c21600640 1 -- 172.21.15.86:0/4194929686 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6c1801a070 con 0x7f6c240ce900 2024-03-22T04:11:21.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c21600640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6c04067b20 0x7f6c04069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:21.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.755+0000 7f6c21600640 1 -- 172.21.15.86:0/4194929686 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f6c18086560 con 0x7f6c240ce900 2024-03-22T04:11:21.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.759+0000 7f6c29000640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6c04067b20 0x7f6c04069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:21.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.759+0000 7f6c29000640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6c04067b20 0x7f6c04069fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c10007fd0 tx=0x7f6c10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:21.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.759+0000 7f6c21600640 1 -- 172.21.15.86:0/4194929686 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6c18056570 con 0x7f6c240ce900 2024-03-22T04:11:21.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.915+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c240b7eb0 con 0x7f6c240ce900 2024-03-22T04:11:21.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c21600640 1 -- 172.21.15.86:0/4194929686 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f6c1805a610 con 0x7f6c240ce900 2024-03-22T04:11:21.923 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:21.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6c04067b20 msgr2=0x7f6c04069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6c04067b20 0x7f6c04069fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c10007fd0 tx=0x7f6c10009040 comp rx=0 tx=0).stop 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240ce900 msgr2=0x7f6c24154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240ce900 0x7f6c24154cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800e9b0 tx=0x7f6c1800ee70 comp rx=0 tx=0).stop 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 shutdown_connections 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6c04067b20 0x7f6c04069fe0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c2415c230 0x7f6c240b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c240d2b30 0x7f6c2406d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 --2- 172.21.15.86:0/4194929686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240ce900 0x7f6c24154cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 >> 172.21.15.86:0/4194929686 conn(0x7f6c240d1f30 msgr2=0x7f6c24157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 shutdown_connections 2024-03-22T04:11:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:21.919+0000 7f6c2ae00640 1 -- 172.21.15.86:0/4194929686 wait complete. 2024-03-22T04:11:21.944 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.079+0000 7f198f000640 1 Processor -- start 2024-03-22T04:11:22.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.079+0000 7f198f000640 1 -- start start 2024-03-22T04:11:22.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f198806f1a0 0x7f19880d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198f000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f19880d0640 0x7f198815c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198815fbe0 con 0x7f1988156620 2024-03-22T04:11:22.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f19880ad4b0 con 0x7f198806f1a0 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198f000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1988156a00 con 0x7f19880d0640 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f198806f1a0 0x7f19880d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198dc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1988156620 0x7f19880acda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35428/0 (socket says 172.21.15.86:35428) 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198dc00640 1 -- 172.21.15.86:0/1281009793 learned_addr learned my addr 172.21.15.86:0/1281009793 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198e600640 1 --2- 172.21.15.86:0/1281009793 >> v2:172.21.15.196:3300/0 conn(0x7f19880d0640 0x7f198815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f1987e00640 1 -- 172.21.15.86:0/1281009793 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494424207 0 0) 0x7f198815fbe0 con 0x7f1988156620 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f1987e00640 1 -- 172.21.15.86:0/1281009793 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1970003530 con 0x7f1988156620 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198d200640 1 -- 172.21.15.86:0/1281009793 >> v2:172.21.15.196:3300/0 conn(0x7f19880d0640 msgr2=0x7f198815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198d200640 1 --2- 172.21.15.86:0/1281009793 >> v2:172.21.15.196:3300/0 conn(0x7f19880d0640 0x7f198815c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198d200640 1 -- 172.21.15.86:0/1281009793 >> v1:172.21.15.86:6789/0 conn(0x7f1988156620 legacy=0x7f19880acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.083+0000 7f198d200640 1 -- 172.21.15.86:0/1281009793 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198815fe50 con 0x7f198806f1a0 2024-03-22T04:11:22.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198d200640 1 --2- 172.21.15.86:0/1281009793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198806f1a0 0x7f19880d0100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f19880af840 tx=0x7f197401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41911211b02f7769 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f1987e00640 1 -- 172.21.15.86:0/1281009793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740b0070 con 0x7f198806f1a0 2024-03-22T04:11:22.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f1987e00640 1 -- 172.21.15.86:0/1281009793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19740ab600 con 0x7f198806f1a0 2024-03-22T04:11:22.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f1987e00640 1 -- 172.21.15.86:0/1281009793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740ab9d0 con 0x7f198806f1a0 2024-03-22T04:11:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- 172.21.15.86:0/1281009793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198806f1a0 msgr2=0x7f19880d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 --2- 172.21.15.86:0/1281009793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198806f1a0 0x7f19880d0100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f19880af840 tx=0x7f197401c480 comp rx=0 tx=0).stop 2024-03-22T04:11:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- 172.21.15.86:0/1281009793 shutdown_connections 2024-03-22T04:11:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 --2- 172.21.15.86:0/1281009793 >> v2:172.21.15.196:3300/0 conn(0x7f19880d0640 0x7f198815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 --2- 172.21.15.86:0/1281009793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198806f1a0 0x7f19880d0100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- 172.21.15.86:0/1281009793 >> 172.21.15.86:0/1281009793 conn(0x7f198806ba60 msgr2=0x7f198806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- 172.21.15.86:0/1281009793 shutdown_connections 2024-03-22T04:11:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- 172.21.15.86:0/1281009793 wait complete. 2024-03-22T04:11:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 Processor -- start 2024-03-22T04:11:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- start start 2024-03-22T04:11:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f198806f1a0 0x7f1988147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19880d0640 0x7f198814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1988156620 0x7f198815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f198806d480 con 0x7f19880d0640 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f198806d300 con 0x7f198806f1a0 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198f000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f198815d2a0 con 0x7f1988156620 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.087+0000 7f198d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19880d0640 0x7f198814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19880d0640 0x7f198814d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38628/0 (socket says 172.21.15.86:38628) 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198d200640 1 -- 172.21.15.86:0/3905638390 learned_addr learned my addr 172.21.15.86:0/3905638390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198dc00640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f198806f1a0 0x7f1988147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198e600640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1988156620 0x7f198815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198d200640 1 -- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1988156620 msgr2=0x7f198815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198d200640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1988156620 0x7f198815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198d200640 1 -- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f198806f1a0 msgr2=0x7f1988147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198d200640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f198806f1a0 0x7f1988147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198d200640 1 -- 172.21.15.86:0/3905638390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1988149660 con 0x7f19880d0640 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198e600640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1988156620 0x7f198815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198dc00640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f198806f1a0 0x7f1988147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198d200640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19880d0640 0x7f198814d120 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19740ae040 tx=0x7f1974019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f1985600640 1 -- 172.21.15.86:0/3905638390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740b0040 con 0x7f19880d0640 2024-03-22T04:11:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f1985600640 1 -- 172.21.15.86:0/3905638390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19740ba070 con 0x7f19880d0640 2024-03-22T04:11:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1988149940 con 0x7f19880d0640 2024-03-22T04:11:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f1985600640 1 -- 172.21.15.86:0/3905638390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974006e20 con 0x7f19880d0640 2024-03-22T04:11:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1988149f00 con 0x7f19880d0640 2024-03-22T04:11:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198804f900 con 0x7f19880d0640 2024-03-22T04:11:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f1985600640 1 -- 172.21.15.86:0/3905638390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f19740b4650 con 0x7f19880d0640 2024-03-22T04:11:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f1985600640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1968067b20 0x7f1968069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.091+0000 7f1985600640 1 -- 172.21.15.86:0/3905638390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f197411fff0 con 0x7f19880d0640 2024-03-22T04:11:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.095+0000 7f198dc00640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1968067b20 0x7f1968069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.095+0000 7f198dc00640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1968067b20 0x7f1968069fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f197c005890 tx=0x7f197c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.099+0000 7f1985600640 1 -- 172.21.15.86:0/3905638390 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1974016070 con 0x7f19880d0640 2024-03-22T04:11:22.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.251+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f198815f870 con 0x7f19880d0640 2024-03-22T04:11:22.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.251+0000 7f1985600640 1 -- 172.21.15.86:0/3905638390 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f19740f0550 con 0x7f19880d0640 2024-03-22T04:11:22.257 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1968067b20 msgr2=0x7f1968069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1968067b20 0x7f1968069fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f197c005890 tx=0x7f197c009040 comp rx=0 tx=0).stop 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19880d0640 msgr2=0x7f198814d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19880d0640 0x7f198814d120 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19740ae040 tx=0x7f1974019220 comp rx=0 tx=0).stop 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 shutdown_connections 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1968067b20 0x7f1968069fe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1988156620 0x7f198815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19880d0640 0x7f198814d120 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 --2- 172.21.15.86:0/3905638390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f198806f1a0 0x7f1988147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 >> 172.21.15.86:0/3905638390 conn(0x7f198806ba60 msgr2=0x7f1988153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:22.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 shutdown_connections 2024-03-22T04:11:22.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.255+0000 7f198f000640 1 -- 172.21.15.86:0/3905638390 wait complete. 2024-03-22T04:11:22.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.411+0000 7f633fa00640 1 Processor -- start 2024-03-22T04:11:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.411+0000 7f633fa00640 1 -- start start 2024-03-22T04:11:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.411+0000 7f633fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f633806e7e0 0x7f633806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.411+0000 7f633fa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f633815c770 con 0x7f6338072650 2024-03-22T04:11:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.411+0000 7f633fa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63380b29a0 con 0x7f6338072b00 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.411+0000 7f633fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6338071350 con 0x7f633806e7e0 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.411+0000 7f633d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f633806e7e0 0x7f633806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f633d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f633806e7e0 0x7f633806ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38640/0 (socket says 172.21.15.86:38640) 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f633d000640 1 -- 172.21.15.86:0/1242419588 learned_addr learned my addr 172.21.15.86:0/1242419588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f6337e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6338072650 0x7f63380b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35434/0 (socket says 172.21.15.86:35434) 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f633d000640 1 -- 172.21.15.86:0/1242419588 >> v1:172.21.15.196:6789/0 conn(0x7f6338072b00 legacy=0x7f63380b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f633d000640 1 -- 172.21.15.86:0/1242419588 >> v1:172.21.15.86:6789/0 conn(0x7f6338072650 legacy=0x7f63380b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f633d000640 1 -- 172.21.15.86:0/1242419588 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633806f100 con 0x7f633806e7e0 2024-03-22T04:11:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f6337400640 1 -- 172.21.15.86:0/1242419588 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2587230486 0 0) 0x7f6324001120 con 0x7f6338072650 2024-03-22T04:11:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f633d000640 1 --2- 172.21.15.86:0/1242419588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f633806ebc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6328009e50 tx=0x7f632801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84096cbbfd5ff553 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f6337400640 1 -- 172.21.15.86:0/1242419588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280a5750 con 0x7f633806e7e0 2024-03-22T04:11:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f6337400640 1 -- 172.21.15.86:0/1242419588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63280a58f0 con 0x7f633806e7e0 2024-03-22T04:11:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f6337400640 1 -- 172.21.15.86:0/1242419588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280a5cc0 con 0x7f633806e7e0 2024-03-22T04:11:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f633fa00640 1 -- 172.21.15.86:0/1242419588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 msgr2=0x7f633806ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.415+0000 7f633fa00640 1 --2- 172.21.15.86:0/1242419588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f633806ebc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6328009e50 tx=0x7f632801c9a0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- 172.21.15.86:0/1242419588 shutdown_connections 2024-03-22T04:11:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 --2- 172.21.15.86:0/1242419588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f633806ebc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- 172.21.15.86:0/1242419588 >> 172.21.15.86:0/1242419588 conn(0x7f63380cc940 msgr2=0x7f63380ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- 172.21.15.86:0/1242419588 shutdown_connections 2024-03-22T04:11:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- 172.21.15.86:0/1242419588 wait complete. 2024-03-22T04:11:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 Processor -- start 2024-03-22T04:11:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- start start 2024-03-22T04:11:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f6338138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6338072650 0x7f6338158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6338072b00 0x7f633814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63380ce930 con 0x7f633806e7e0 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6338071b60 con 0x7f6338072b00 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6338072080 con 0x7f6338072650 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f6338138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f6338138fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38652/0 (socket says 172.21.15.86:38652) 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 -- 172.21.15.86:0/3298765136 learned_addr learned my addr 172.21.15.86:0/3298765136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f6337e00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6338072650 0x7f6338158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633da00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6338072b00 0x7f633814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 -- 172.21.15.86:0/3298765136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6338072650 msgr2=0x7f6338158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6338072650 0x7f6338158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 -- 172.21.15.86:0/3298765136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6338072b00 msgr2=0x7f633814c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6338072b00 0x7f633814c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 -- 172.21.15.86:0/3298765136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63381394f0 con 0x7f633806e7e0 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633d000640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f6338138fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63280195b0 tx=0x7f6328019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f6334c00640 1 -- 172.21.15.86:0/3298765136 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280b8470 con 0x7f633806e7e0 2024-03-22T04:11:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f6334c00640 1 -- 172.21.15.86:0/3298765136 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63280b8610 con 0x7f633806e7e0 2024-03-22T04:11:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.419+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6338153600 con 0x7f633806e7e0 2024-03-22T04:11:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.423+0000 7f6334c00640 1 -- 172.21.15.86:0/3298765136 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280b8a00 con 0x7f633806e7e0 2024-03-22T04:11:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.423+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6338152b20 con 0x7f633806e7e0 2024-03-22T04:11:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.423+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6338074470 con 0x7f633806e7e0 2024-03-22T04:11:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.423+0000 7f6334c00640 1 -- 172.21.15.86:0/3298765136 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f63280c0030 con 0x7f633806e7e0 2024-03-22T04:11:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.423+0000 7f6334c00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6318067b50 0x7f631806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.423+0000 7f6334c00640 1 -- 172.21.15.86:0/3298765136 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f6328120b30 con 0x7f633806e7e0 2024-03-22T04:11:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.423+0000 7f6337e00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6318067b50 0x7f631806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.423+0000 7f6337e00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6318067b50 0x7f631806a010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6338155e70 tx=0x7f6324007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.427+0000 7f6334c00640 1 -- 172.21.15.86:0/3298765136 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f63280c7060 con 0x7f633806e7e0 2024-03-22T04:11:22.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6338153bc0 con 0x7f633806e7e0 2024-03-22T04:11:22.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f6334c00640 1 -- 172.21.15.86:0/3298765136 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f6328016620 con 0x7f633806e7e0 2024-03-22T04:11:22.587 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6318067b50 msgr2=0x7f631806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6318067b50 0x7f631806a010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6338155e70 tx=0x7f6324007d10 comp rx=0 tx=0).stop 2024-03-22T04:11:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 msgr2=0x7f6338138fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f6338138fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63280195b0 tx=0x7f6328019740 comp rx=0 tx=0).stop 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 shutdown_connections 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6318067b50 0x7f631806a010 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6338072b00 0x7f633814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6338072650 0x7f6338158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 --2- 172.21.15.86:0/3298765136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633806e7e0 0x7f6338138fb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 >> 172.21.15.86:0/3298765136 conn(0x7f63380cc940 msgr2=0x7f63380b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 shutdown_connections 2024-03-22T04:11:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.583+0000 7f633fa00640 1 -- 172.21.15.86:0/3298765136 wait complete. 2024-03-22T04:11:22.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.739+0000 7f2379c00640 1 Processor -- start 2024-03-22T04:11:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.739+0000 7f2379c00640 1 -- start start 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2379c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 0x7f237406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2379c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2374072ad0 0x7f23740b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2379c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f237415c770 con 0x7f2374072650 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2379c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2374071350 con 0x7f2374072ad0 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2379c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f237406f100 con 0x7f237406e7e0 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2373400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2374072ad0 0x7f23740b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f236a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2374072650 0x7f23740b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35436/0 (socket says 172.21.15.86:35436) 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f236a000640 1 -- 172.21.15.86:0/2223595476 learned_addr learned my addr 172.21.15.86:0/2223595476 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2372a00640 1 --2- 172.21.15.86:0/2223595476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 0x7f237406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2373400640 1 -- 172.21.15.86:0/2223595476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 msgr2=0x7f237406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2373400640 1 --2- 172.21.15.86:0/2223595476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 0x7f237406ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2373400640 1 -- 172.21.15.86:0/2223595476 >> v1:172.21.15.86:6789/0 conn(0x7f2374072650 legacy=0x7f23740b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2373400640 1 -- 172.21.15.86:0/2223595476 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2374073150 con 0x7f2374072ad0 2024-03-22T04:11:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2372000640 1 -- 172.21.15.86:0/2223595476 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9713071 0 0) 0x7f235c001120 con 0x7f2374072650 2024-03-22T04:11:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.743+0000 7f2373400640 1 --2- 172.21.15.86:0/2223595476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072ad0 0x7f23740b2290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f236c009d90 tx=0x7f236c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9831d8dd8063ecf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2372000640 1 -- 172.21.15.86:0/2223595476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0b2070 con 0x7f2374072ad0 2024-03-22T04:11:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2372000640 1 -- 172.21.15.86:0/2223595476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236c0a5890 con 0x7f2374072ad0 2024-03-22T04:11:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2372000640 1 -- 172.21.15.86:0/2223595476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0a5c60 con 0x7f2374072ad0 2024-03-22T04:11:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- 172.21.15.86:0/2223595476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072ad0 msgr2=0x7f23740b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 --2- 172.21.15.86:0/2223595476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072ad0 0x7f23740b2290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f236c009d90 tx=0x7f236c01c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- 172.21.15.86:0/2223595476 shutdown_connections 2024-03-22T04:11:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 --2- 172.21.15.86:0/2223595476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072ad0 0x7f23740b2290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 --2- 172.21.15.86:0/2223595476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 0x7f237406ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- 172.21.15.86:0/2223595476 >> 172.21.15.86:0/2223595476 conn(0x7f23740cc940 msgr2=0x7f23740ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- 172.21.15.86:0/2223595476 shutdown_connections 2024-03-22T04:11:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- 172.21.15.86:0/2223595476 wait complete. 2024-03-22T04:11:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 Processor -- start 2024-03-22T04:11:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- start start 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 0x7f237413d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072650 0x7f23741573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2374072ad0 0x7f23740c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2374072080 con 0x7f2374072650 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23740ce930 con 0x7f2374072ad0 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2379c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f237414fb60 con 0x7f237406e7e0 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f236a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072650 0x7f23741573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f236a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072650 0x7f23741573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38676/0 (socket says 172.21.15.86:38676) 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f236a000640 1 -- 172.21.15.86:0/3260621172 learned_addr learned my addr 172.21.15.86:0/3260621172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2372a00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 0x7f237413d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.747+0000 7f2373400640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2374072ad0 0x7f23740c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236a000640 1 -- 172.21.15.86:0/3260621172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 msgr2=0x7f237413d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236a000640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 0x7f237413d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236a000640 1 -- 172.21.15.86:0/3260621172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2374072ad0 msgr2=0x7f23740c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236a000640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2374072ad0 0x7f23740c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236a000640 1 -- 172.21.15.86:0/3260621172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237414f880 con 0x7f2374072650 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236a000640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072650 0x7f23741573f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c4b0 tx=0x7f235c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236b400640 1 -- 172.21.15.86:0/3260621172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c018070 con 0x7f2374072650 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236b400640 1 -- 172.21.15.86:0/3260621172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c007500 con 0x7f2374072650 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23740c5000 con 0x7f2374072650 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236b400640 1 -- 172.21.15.86:0/3260621172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c004a30 con 0x7f2374072650 2024-03-22T04:11:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23740c55c0 con 0x7f2374072650 2024-03-22T04:11:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23740c57c0 con 0x7f2374072650 2024-03-22T04:11:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236b400640 1 -- 172.21.15.86:0/3260621172 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f235c01b070 con 0x7f2374072650 2024-03-22T04:11:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236b400640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2350067b20 0x7f2350069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f236b400640 1 -- 172.21.15.86:0/3260621172 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f235c085520 con 0x7f2374072650 2024-03-22T04:11:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.751+0000 7f2372a00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2350067b20 0x7f2350069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.755+0000 7f2372a00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2350067b20 0x7f2350069fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2364009ac0 tx=0x7f2364009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:22.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.755+0000 7f236b400640 1 -- 172.21.15.86:0/3260621172 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f23740c57c0 con 0x7f2374072650 2024-03-22T04:11:22.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.911+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2374074470 con 0x7f2374072650 2024-03-22T04:11:22.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f236b400640 1 -- 172.21.15.86:0/3260621172 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f237415c770 con 0x7f2374072650 2024-03-22T04:11:22.918 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:22.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2350067b20 msgr2=0x7f2350069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2350067b20 0x7f2350069fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2364009ac0 tx=0x7f2364009190 comp rx=0 tx=0).stop 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072650 msgr2=0x7f23741573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072650 0x7f23741573f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c4b0 tx=0x7f235c00c970 comp rx=0 tx=0).stop 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 shutdown_connections 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2350067b20 0x7f2350069fe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2374072ad0 0x7f23740c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2374072650 0x7f23741573f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 --2- 172.21.15.86:0/3260621172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f237406e7e0 0x7f237413d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 >> 172.21.15.86:0/3260621172 conn(0x7f23740cc940 msgr2=0x7f2374154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 shutdown_connections 2024-03-22T04:11:22.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:22.915+0000 7f2379c00640 1 -- 172.21.15.86:0/3260621172 wait complete. 2024-03-22T04:11:22.940 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ec00640 1 Processor -- start 2024-03-22T04:11:23.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ec00640 1 -- start start 2024-03-22T04:11:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7680d0e10 0x7fb76806e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb768162e80 con 0x7fb7680b4070 2024-03-22T04:11:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ec00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb768162300 con 0x7fb768147bd0 2024-03-22T04:11:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb7681625e0 con 0x7fb7680d0e10 2024-03-22T04:11:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7680d0e10 0x7fb76806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76d800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb7680b4070 0x7fb7680d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35444/0 (socket says 172.21.15.86:35444) 2024-03-22T04:11:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76d800640 1 -- 172.21.15.86:0/4207399174 learned_addr learned my addr 172.21.15.86:0/4207399174 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76e200640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb768147bd0 0x7fb76806e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:60330/0 (socket says 172.21.15.86:60330) 2024-03-22T04:11:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb767e00640 1 -- 172.21.15.86:0/4207399174 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509708508 0 0) 0x7fb768162e80 con 0x7fb7680b4070 2024-03-22T04:11:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ce00640 1 -- 172.21.15.86:0/4207399174 >> v1:172.21.15.196:6789/0 conn(0x7fb768147bd0 legacy=0x7fb76806e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ce00640 1 -- 172.21.15.86:0/4207399174 >> v1:172.21.15.86:6789/0 conn(0x7fb7680b4070 legacy=0x7fb7680d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.075+0000 7fb76ce00640 1 -- 172.21.15.86:0/4207399174 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7681628c0 con 0x7fb7680d0e10 2024-03-22T04:11:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb76ce00640 1 --2- 172.21.15.86:0/4207399174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7680d0e10 0x7fb76806e150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb754009770 tx=0x7fb76806f5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33cc91d527c2df00 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb767e00640 1 -- 172.21.15.86:0/4207399174 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7540b0070 con 0x7fb7680d0e10 2024-03-22T04:11:23.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb767e00640 1 -- 172.21.15.86:0/4207399174 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7540ab600 con 0x7fb7680d0e10 2024-03-22T04:11:23.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb767e00640 1 -- 172.21.15.86:0/4207399174 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7540ab9d0 con 0x7fb7680d0e10 2024-03-22T04:11:23.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb76ec00640 1 -- 172.21.15.86:0/4207399174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7680d0e10 msgr2=0x7fb76806e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb76ec00640 1 --2- 172.21.15.86:0/4207399174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7680d0e10 0x7fb76806e150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb754009770 tx=0x7fb76806f5b0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb76ec00640 1 -- 172.21.15.86:0/4207399174 shutdown_connections 2024-03-22T04:11:23.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb76ec00640 1 --2- 172.21.15.86:0/4207399174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7680d0e10 0x7fb76806e150 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb76ec00640 1 -- 172.21.15.86:0/4207399174 >> 172.21.15.86:0/4207399174 conn(0x7fb76806b660 msgr2=0x7fb76806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:23.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb76ec00640 1 -- 172.21.15.86:0/4207399174 shutdown_connections 2024-03-22T04:11:23.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.079+0000 7fb76ec00640 1 -- 172.21.15.86:0/4207399174 wait complete. 2024-03-22T04:11:23.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 Processor -- start 2024-03-22T04:11:23.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 -- start start 2024-03-22T04:11:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7680b4070 0x7fb76815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7680d0e10 0x7fb7680b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb768147bd0 0x7fb7680b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb76816c240 con 0x7fb7680d0e10 2024-03-22T04:11:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb76816c0c0 con 0x7fb7680b4070 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb76816c3c0 con 0x7fb768147bd0 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7680d0e10 0x7fb7680b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7680d0e10 0x7fb7680b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38678/0 (socket says 172.21.15.86:38678) 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 -- 172.21.15.86:0/920291433 learned_addr learned my addr 172.21.15.86:0/920291433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76d800640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7680b4070 0x7fb76815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76e200640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb768147bd0 0x7fb7680b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 -- 172.21.15.86:0/920291433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb768147bd0 msgr2=0x7fb7680b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb768147bd0 0x7fb7680b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 -- 172.21.15.86:0/920291433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7680b4070 msgr2=0x7fb76815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7680b4070 0x7fb76815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 -- 172.21.15.86:0/920291433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768140d50 con 0x7fb7680d0e10 2024-03-22T04:11:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ce00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7680d0e10 0x7fb7680b14e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb7540ae040 tx=0x7fb75401ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb765600640 1 -- 172.21.15.86:0/920291433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7540bc070 con 0x7fb7680d0e10 2024-03-22T04:11:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb765600640 1 -- 172.21.15.86:0/920291433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7540b1040 con 0x7fb7680d0e10 2024-03-22T04:11:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb765600640 1 -- 172.21.15.86:0/920291433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7540b0050 con 0x7fb7680d0e10 2024-03-22T04:11:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb768141030 con 0x7fb7680d0e10 2024-03-22T04:11:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.083+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76813e880 con 0x7fb7680d0e10 2024-03-22T04:11:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.087+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb768074470 con 0x7fb7680d0e10 2024-03-22T04:11:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.087+0000 7fb765600640 1 -- 172.21.15.86:0/920291433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb754004230 con 0x7fb7680d0e10 2024-03-22T04:11:23.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.087+0000 7fb765600640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb748067b20 0x7fb748069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.087+0000 7fb765600640 1 -- 172.21.15.86:0/920291433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fb75411fc20 con 0x7fb7680d0e10 2024-03-22T04:11:23.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.087+0000 7fb76d800640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb748067b20 0x7fb748069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.087+0000 7fb76d800640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb748067b20 0x7fb748069fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0011f0 tx=0x7fb75c001140 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.091+0000 7fb765600640 1 -- 172.21.15.86:0/920291433 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb754015f90 con 0x7fb7680d0e10 2024-03-22T04:11:23.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.251+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb76815fe20 con 0x7fb7680d0e10 2024-03-22T04:11:23.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.251+0000 7fb765600640 1 -- 172.21.15.86:0/920291433 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7fb7540efcc0 con 0x7fb7680d0e10 2024-03-22T04:11:23.257 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:23.257 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb748067b20 msgr2=0x7fb748069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb748067b20 0x7fb748069fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0011f0 tx=0x7fb75c001140 comp rx=0 tx=0).stop 2024-03-22T04:11:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7680d0e10 msgr2=0x7fb7680b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7680d0e10 0x7fb7680b14e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb7540ae040 tx=0x7fb75401ab20 comp rx=0 tx=0).stop 2024-03-22T04:11:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 shutdown_connections 2024-03-22T04:11:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb748067b20 0x7fb748069fe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb768147bd0 0x7fb7680b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7680d0e10 0x7fb7680b14e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 --2- 172.21.15.86:0/920291433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7680b4070 0x7fb76815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 >> 172.21.15.86:0/920291433 conn(0x7fb76806b660 msgr2=0x7fb7680d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 shutdown_connections 2024-03-22T04:11:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.255+0000 7fb76ec00640 1 -- 172.21.15.86:0/920291433 wait complete. 2024-03-22T04:11:23.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:23.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:23.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.411+0000 7f05a7e00640 1 Processor -- start 2024-03-22T04:11:23.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.411+0000 7f05a7e00640 1 -- start start 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a806bd70 0x7f05a80d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f05a80d05f0 0x7f05a806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a8162e80 con 0x7f05a815e4b0 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f05a815e890 con 0x7f05a80d05f0 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f05a8162950 con 0x7f05a806bd70 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f05a80d05f0 0x7f05a806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f059e000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a806bd70 0x7f05a80d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a6a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f05a815e4b0 0x7f05a806b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:60332/0 (socket says 172.21.15.86:60332) 2024-03-22T04:11:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a6a00640 1 -- 172.21.15.86:0/1547036468 learned_addr learned my addr 172.21.15.86:0/1547036468 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7400640 1 -- 172.21.15.86:0/1547036468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a806bd70 msgr2=0x7f05a80d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7400640 1 --2- 172.21.15.86:0/1547036468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a806bd70 0x7f05a80d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7400640 1 -- 172.21.15.86:0/1547036468 >> v1:172.21.15.196:6789/0 conn(0x7f05a815e4b0 legacy=0x7f05a806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f05a7400640 1 -- 172.21.15.86:0/1547036468 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a8162ba0 con 0x7f05a80d05f0 2024-03-22T04:11:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.415+0000 7f059e000640 1 --2- 172.21.15.86:0/1547036468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a806bd70 0x7f05a80d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7400640 1 --2- 172.21.15.86:0/1547036468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a80d05f0 0x7f05a806e4d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05a00097c0 tx=0x7f05a001c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10bdef1a087bf49 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a6000640 1 -- 172.21.15.86:0/1547036468 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00a7650 con 0x7f05a80d05f0 2024-03-22T04:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a6000640 1 -- 172.21.15.86:0/1547036468 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a00a77f0 con 0x7f05a80d05f0 2024-03-22T04:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a6000640 1 -- 172.21.15.86:0/1547036468 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00a7bc0 con 0x7f05a80d05f0 2024-03-22T04:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- 172.21.15.86:0/1547036468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a80d05f0 msgr2=0x7f05a806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 --2- 172.21.15.86:0/1547036468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a80d05f0 0x7f05a806e4d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05a00097c0 tx=0x7f05a001c9b0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- 172.21.15.86:0/1547036468 shutdown_connections 2024-03-22T04:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 --2- 172.21.15.86:0/1547036468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a80d05f0 0x7f05a806e4d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 --2- 172.21.15.86:0/1547036468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a806bd70 0x7f05a80d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- 172.21.15.86:0/1547036468 >> 172.21.15.86:0/1547036468 conn(0x7f05a80d0ec0 msgr2=0x7f05a80d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- 172.21.15.86:0/1547036468 shutdown_connections 2024-03-22T04:11:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- 172.21.15.86:0/1547036468 wait complete. 2024-03-22T04:11:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 Processor -- start 2024-03-22T04:11:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- start start 2024-03-22T04:11:23.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a806bd70 0x7f05a8154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a80d05f0 0x7f05a815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a815e4b0 0x7f05a815d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05a816be80 con 0x7f05a815e4b0 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05a816bd00 con 0x7f05a806bd70 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.419+0000 7f05a7e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f05a816c000 con 0x7f05a80d05f0 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a815e4b0 0x7f05a815d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a815e4b0 0x7f05a815d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38692/0 (socket says 172.21.15.86:38692) 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f059e000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a80d05f0 0x7f05a815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 -- 172.21.15.86:0/3508855180 learned_addr learned my addr 172.21.15.86:0/3508855180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a6a00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a806bd70 0x7f05a8154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 -- 172.21.15.86:0/3508855180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a80d05f0 msgr2=0x7f05a815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a80d05f0 0x7f05a815d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 -- 172.21.15.86:0/3508855180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a806bd70 msgr2=0x7f05a8154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a806bd70 0x7f05a8154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 -- 172.21.15.86:0/3508855180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a8159140 con 0x7f05a815e4b0 2024-03-22T04:11:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f059e000640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a80d05f0 0x7f05a815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7400640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a815e4b0 0x7f05a815d970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05a00097c0 tx=0x7f05a0019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f059f400640 1 -- 172.21.15.86:0/3508855180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00b8640 con 0x7f05a815e4b0 2024-03-22T04:11:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f059f400640 1 -- 172.21.15.86:0/3508855180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a00b87e0 con 0x7f05a815e4b0 2024-03-22T04:11:23.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a8159420 con 0x7f05a815e4b0 2024-03-22T04:11:23.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f059f400640 1 -- 172.21.15.86:0/3508855180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00b8bb0 con 0x7f05a815e4b0 2024-03-22T04:11:23.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a815cf10 con 0x7f05a815e4b0 2024-03-22T04:11:23.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a804f900 con 0x7f05a815e4b0 2024-03-22T04:11:23.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f059f400640 1 -- 172.21.15.86:0/3508855180 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f05a00a5040 con 0x7f05a815e4b0 2024-03-22T04:11:23.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.423+0000 7f059f400640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0584067b20 0x7f0584069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.427+0000 7f059f400640 1 -- 172.21.15.86:0/3508855180 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f05a011f680 con 0x7f05a815e4b0 2024-03-22T04:11:23.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.427+0000 7f05a6a00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0584067b20 0x7f0584069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.427+0000 7f05a6a00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0584067b20 0x7f0584069fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f05980012f0 tx=0x7f0598001850 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.427+0000 7f059f400640 1 -- 172.21.15.86:0/3508855180 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f05a00ab030 con 0x7f05a815e4b0 2024-03-22T04:11:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.583+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05a81595f0 con 0x7f05a815e4b0 2024-03-22T04:11:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f059f400640 1 -- 172.21.15.86:0/3508855180 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f05a0015fb0 con 0x7f05a815e4b0 2024-03-22T04:11:23.590 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0584067b20 msgr2=0x7f0584069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0584067b20 0x7f0584069fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f05980012f0 tx=0x7f0598001850 comp rx=0 tx=0).stop 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a815e4b0 msgr2=0x7f05a815d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a815e4b0 0x7f05a815d970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05a00097c0 tx=0x7f05a0019040 comp rx=0 tx=0).stop 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 shutdown_connections 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0584067b20 0x7f0584069fe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a815e4b0 0x7f05a815d970 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05a80d05f0 0x7f05a815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 --2- 172.21.15.86:0/3508855180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05a806bd70 0x7f05a8154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 >> 172.21.15.86:0/3508855180 conn(0x7f05a80d0ec0 msgr2=0x7f05a80cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 shutdown_connections 2024-03-22T04:11:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.587+0000 7f05a7e00640 1 -- 172.21.15.86:0/3508855180 wait complete. 2024-03-22T04:11:23.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:23.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.743+0000 7f9b52800640 1 Processor -- start 2024-03-22T04:11:23.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.743+0000 7f9b52800640 1 -- start start 2024-03-22T04:11:23.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b52800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9b4c144a90 0x7f9b4c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b52800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b4c0b1b80 0x7f9b4c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b52800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b52800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9b4c144e70 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b52800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9b4c15fab0 con 0x7f9b4c144a90 2024-03-22T04:11:23.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b52800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c06d6d0 con 0x7f9b4c1620e0 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b42a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b4c0b1b80 0x7f9b4c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b42a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b4c0b1b80 0x7f9b4c0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38700/0 (socket says 172.21.15.86:38700) 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b42a00640 1 -- 172.21.15.86:0/3572873667 learned_addr learned my addr 172.21.15.86:0/3572873667 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b4b400640 1 --2- 172.21.15.86:0/3572873667 >> v2:172.21.15.196:3300/0 conn(0x7f9b4c144a90 0x7f9b4c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b4be00640 1 --2- 172.21.15.86:0/3572873667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b42a00640 1 -- 172.21.15.86:0/3572873667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 msgr2=0x7f9b4c162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b42a00640 1 --2- 172.21.15.86:0/3572873667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b42a00640 1 -- 172.21.15.86:0/3572873667 >> v2:172.21.15.196:3300/0 conn(0x7f9b4c144a90 msgr2=0x7f9b4c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b42a00640 1 --2- 172.21.15.86:0/3572873667 >> v2:172.21.15.196:3300/0 conn(0x7f9b4c144a90 0x7f9b4c15f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.747+0000 7f9b42a00640 1 -- 172.21.15.86:0/3572873667 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c06d970 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b42a00640 1 --2- 172.21.15.86:0/3572873667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c0b2030 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c06f830 tx=0x7f9b3401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de15b00f4b7382f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b4aa00640 1 -- 172.21.15.86:0/3572873667 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b340b1070 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b4aa00640 1 -- 172.21.15.86:0/3572873667 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b340ac630 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b4aa00640 1 -- 172.21.15.86:0/3572873667 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b340a8050 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- 172.21.15.86:0/3572873667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 msgr2=0x7f9b4c0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 --2- 172.21.15.86:0/3572873667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c0b2030 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c06f830 tx=0x7f9b3401c520 comp rx=0 tx=0).stop 2024-03-22T04:11:23.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- 172.21.15.86:0/3572873667 shutdown_connections 2024-03-22T04:11:23.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 --2- 172.21.15.86:0/3572873667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 --2- 172.21.15.86:0/3572873667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c0b2030 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 --2- 172.21.15.86:0/3572873667 >> v2:172.21.15.196:3300/0 conn(0x7f9b4c144a90 0x7f9b4c15f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- 172.21.15.86:0/3572873667 >> 172.21.15.86:0/3572873667 conn(0x7f9b4c0cc940 msgr2=0x7f9b4c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:23.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- 172.21.15.86:0/3572873667 shutdown_connections 2024-03-22T04:11:23.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- 172.21.15.86:0/3572873667 wait complete. 2024-03-22T04:11:23.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 Processor -- start 2024-03-22T04:11:23.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- start start 2024-03-22T04:11:23.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b4c144a90 0x7f9b4c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c162e80 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c06e4f0 con 0x7f9b4c144a90 2024-03-22T04:11:23.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b52800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c0adbf0 con 0x7f9b4c1620e0 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c140f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38706/0 (socket says 172.21.15.86:38706) 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b4b400640 1 -- 172.21.15.86:0/56795791 learned_addr learned my addr 172.21.15.86:0/56795791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.751+0000 7f9b4be00640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b42a00640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b4c144a90 0x7f9b4c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b4b400640 1 -- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 msgr2=0x7f9b4c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b4b400640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b4b400640 1 -- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b4c144a90 msgr2=0x7f9b4c15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b4b400640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b4c144a90 0x7f9b4c15c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b4b400640 1 -- 172.21.15.86:0/56795791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c15bab0 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b4be00640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b4b400640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c140f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e930 tx=0x7f9b3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b43e00640 1 -- 172.21.15.86:0/56795791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c017070 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b43e00640 1 -- 172.21.15.86:0/56795791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3c00c9a0 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c155580 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b43e00640 1 -- 172.21.15.86:0/56795791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00cd50 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c141620 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4c074470 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b43e00640 1 -- 172.21.15.86:0/56795791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9b3c01a070 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b43e00640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b28067b20 0x7f9b28069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:23.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b43e00640 1 -- 172.21.15.86:0/56795791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f9b3c086530 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.755+0000 7f9b42a00640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b28067b20 0x7f9b28069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:23.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.759+0000 7f9b42a00640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b28067b20 0x7f9b28069fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b340ae000 tx=0x7f9b34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:23.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.759+0000 7f9b43e00640 1 -- 172.21.15.86:0/56795791 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9b3c056690 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.915+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9b4c15bc80 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.915+0000 7f9b43e00640 1 -- 172.21.15.86:0/56795791 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f9b3c05a730 con 0x7f9b4c0b1b80 2024-03-22T04:11:23.922 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b28067b20 msgr2=0x7f9b28069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b28067b20 0x7f9b28069fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b340ae000 tx=0x7f9b34009040 comp rx=0 tx=0).stop 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 msgr2=0x7f9b4c140f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c140f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e930 tx=0x7f9b3c00edf0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 shutdown_connections 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b28067b20 0x7f9b28069fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b4c144a90 0x7f9b4c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b4c1620e0 0x7f9b4c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 --2- 172.21.15.86:0/56795791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0b1b80 0x7f9b4c140f10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 >> 172.21.15.86:0/56795791 conn(0x7f9b4c0cc940 msgr2=0x7f9b4c071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 shutdown_connections 2024-03-22T04:11:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:23.919+0000 7f9b52800640 1 -- 172.21.15.86:0/56795791 wait complete. 2024-03-22T04:11:23.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:24.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3accc00640 1 Processor -- start 2024-03-22T04:11:24.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3accc00640 1 -- start start 2024-03-22T04:11:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3accc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac806e7e0 0x7f3ac806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3accc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ac815c770 con 0x7f3ac8072ad0 2024-03-22T04:11:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3accc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ac80b29a0 con 0x7f3ac8072650 2024-03-22T04:11:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3accc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac8071350 con 0x7f3ac806e7e0 2024-03-22T04:11:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac6800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3ac8072ad0 0x7f3ac80b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35448/0 (socket says 172.21.15.86:35448) 2024-03-22T04:11:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac6800640 1 -- 172.21.15.86:0/2550104913 learned_addr learned my addr 172.21.15.86:0/2550104913 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac5e00640 1 --2- 172.21.15.86:0/2550104913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac806e7e0 0x7f3ac806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac4a00640 1 -- 172.21.15.86:0/2550104913 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 760912774 0 0) 0x7f3ac80b29a0 con 0x7f3ac8072650 2024-03-22T04:11:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac4a00640 1 -- 172.21.15.86:0/2550104913 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aac005740 con 0x7f3ac8072650 2024-03-22T04:11:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac4a00640 1 -- 172.21.15.86:0/2550104913 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 269652408 0 0) 0x7f3ac00014a0 con 0x7f3ac8072ad0 2024-03-22T04:11:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac4a00640 1 -- 172.21.15.86:0/2550104913 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ac80b29a0 con 0x7f3ac8072ad0 2024-03-22T04:11:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac5e00640 1 -- 172.21.15.86:0/2550104913 >> v1:172.21.15.196:6789/0 conn(0x7f3ac8072650 legacy=0x7f3ac80b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac5e00640 1 -- 172.21.15.86:0/2550104913 >> v1:172.21.15.86:6789/0 conn(0x7f3ac8072ad0 legacy=0x7f3ac80b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.079+0000 7f3ac5e00640 1 -- 172.21.15.86:0/2550104913 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac806f100 con 0x7f3ac806e7e0 2024-03-22T04:11:24.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3ac5e00640 1 --2- 172.21.15.86:0/2550104913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac806e7e0 0x7f3ac806ebc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80097f0 tx=0x7f3ab801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a21259ee3d79631 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3ac4a00640 1 -- 172.21.15.86:0/2550104913 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80b0070 con 0x7f3ac806e7e0 2024-03-22T04:11:24.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3ac4a00640 1 -- 172.21.15.86:0/2550104913 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab80ab600 con 0x7f3ac806e7e0 2024-03-22T04:11:24.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3ac4a00640 1 -- 172.21.15.86:0/2550104913 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80ab9d0 con 0x7f3ac806e7e0 2024-03-22T04:11:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3accc00640 1 -- 172.21.15.86:0/2550104913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac806e7e0 msgr2=0x7f3ac806ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3accc00640 1 --2- 172.21.15.86:0/2550104913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac806e7e0 0x7f3ac806ebc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80097f0 tx=0x7f3ab801c450 comp rx=0 tx=0).stop 2024-03-22T04:11:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3accc00640 1 -- 172.21.15.86:0/2550104913 shutdown_connections 2024-03-22T04:11:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3accc00640 1 --2- 172.21.15.86:0/2550104913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac806e7e0 0x7f3ac806ebc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3accc00640 1 -- 172.21.15.86:0/2550104913 >> 172.21.15.86:0/2550104913 conn(0x7f3ac80cc940 msgr2=0x7f3ac80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3accc00640 1 -- 172.21.15.86:0/2550104913 shutdown_connections 2024-03-22T04:11:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.083+0000 7f3accc00640 1 -- 172.21.15.86:0/2550104913 wait complete. 2024-03-22T04:11:24.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 Processor -- start 2024-03-22T04:11:24.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 -- start start 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac806e7e0 0x7f3ac8152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8072650 0x7f3ac8159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac8072ad0 0x7f3ac8140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac80ce930 con 0x7f3ac806e7e0 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8071b60 con 0x7f3ac8072650 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac8072080 con 0x7f3ac8072ad0 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac806e7e0 0x7f3ac8152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac806e7e0 0x7f3ac8152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38708/0 (socket says 172.21.15.86:38708) 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 -- 172.21.15.86:0/277594310 learned_addr learned my addr 172.21.15.86:0/277594310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:24.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac6800640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac8072ad0 0x7f3ac8140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5400640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8072650 0x7f3ac8159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 -- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac8072ad0 msgr2=0x7f3ac8140430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac8072ad0 0x7f3ac8140430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 -- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8072650 msgr2=0x7f3ac8159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8072650 0x7f3ac8159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 -- 172.21.15.86:0/277594310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac80c9630 con 0x7f3ac806e7e0 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5400640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8072650 0x7f3ac8159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3ac5e00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac806e7e0 0x7f3ac8152530 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80097f0 tx=0x7f3ab801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3abe000640 1 -- 172.21.15.86:0/277594310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80b0050 con 0x7f3ac806e7e0 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3abe000640 1 -- 172.21.15.86:0/277594310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab80bc070 con 0x7f3ac806e7e0 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3abe000640 1 -- 172.21.15.86:0/277594310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8006e20 con 0x7f3ac806e7e0 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac80c9910 con 0x7f3ac806e7e0 2024-03-22T04:11:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac80c9ed0 con 0x7f3ac806e7e0 2024-03-22T04:11:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.087+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac8074470 con 0x7f3ac806e7e0 2024-03-22T04:11:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.091+0000 7f3abe000640 1 -- 172.21.15.86:0/277594310 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3ab80b5850 con 0x7f3ac806e7e0 2024-03-22T04:11:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.091+0000 7f3abe000640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3aa8067b20 0x7f3aa8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.091+0000 7f3abe000640 1 -- 172.21.15.86:0/277594310 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f3ab8120920 con 0x7f3ac806e7e0 2024-03-22T04:11:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.091+0000 7f3ac5400640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3aa8067b20 0x7f3aa8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.091+0000 7f3ac5400640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3aa8067b20 0x7f3aa8069fe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8166d10 tx=0x7f3ab4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.095+0000 7f3abe000640 1 -- 172.21.15.86:0/277594310 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3ab80160d0 con 0x7f3ac806e7e0 2024-03-22T04:11:24.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ac8156580 con 0x7f3ac806e7e0 2024-03-22T04:11:24.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3abe000640 1 -- 172.21.15.86:0/277594310 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f3ab80f0f70 con 0x7f3ac806e7e0 2024-03-22T04:11:24.255 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3aa8067b20 msgr2=0x7f3aa8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3aa8067b20 0x7f3aa8069fe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8166d10 tx=0x7f3ab4009040 comp rx=0 tx=0).stop 2024-03-22T04:11:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac806e7e0 msgr2=0x7f3ac8152530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac806e7e0 0x7f3ac8152530 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80097f0 tx=0x7f3ab801ac90 comp rx=0 tx=0).stop 2024-03-22T04:11:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 shutdown_connections 2024-03-22T04:11:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3aa8067b20 0x7f3aa8069fe0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8072650 0x7f3ac8159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ac8072ad0 0x7f3ac8140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 --2- 172.21.15.86:0/277594310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ac806e7e0 0x7f3ac8152530 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 >> 172.21.15.86:0/277594310 conn(0x7f3ac80cc940 msgr2=0x7f3ac815ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:24.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 shutdown_connections 2024-03-22T04:11:24.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.251+0000 7f3accc00640 1 -- 172.21.15.86:0/277594310 wait complete. 2024-03-22T04:11:24.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.407+0000 7fc8ecc00640 1 Processor -- start 2024-03-22T04:11:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.407+0000 7fc8ecc00640 1 -- start start 2024-03-22T04:11:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8ecc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8e815d0b0 0x7fc8e81671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8ecc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e815d710 con 0x7fc8e80b2360 2024-03-22T04:11:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8ecc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e815fe40 con 0x7fc8e80b3600 2024-03-22T04:11:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8ecc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e80b2740 con 0x7fc8e815d0b0 2024-03-22T04:11:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8e6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8e815d0b0 0x7fc8e81671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8e6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8e815d0b0 0x7fc8e81671d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38724/0 (socket says 172.21.15.86:38724) 2024-03-22T04:11:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8e6800640 1 -- 172.21.15.86:0/571706568 learned_addr learned my addr 172.21.15.86:0/571706568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8e6800640 1 -- 172.21.15.86:0/571706568 >> v1:172.21.15.196:6789/0 conn(0x7fc8e80b3600 legacy=0x7fc8e815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8e6800640 1 -- 172.21.15.86:0/571706568 >> v1:172.21.15.86:6789/0 conn(0x7fc8e80b2360 legacy=0x7fc8e80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8e6800640 1 -- 172.21.15.86:0/571706568 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e8160120 con 0x7fc8e815d0b0 2024-03-22T04:11:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.411+0000 7fc8e4a00640 1 -- 172.21.15.86:0/571706568 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1306230888 0 0) 0x7fc8e815fe40 con 0x7fc8e80b3600 2024-03-22T04:11:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8e6800640 1 --2- 172.21.15.86:0/571706568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e815d0b0 0x7fc8e81671d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009870 tx=0x7fc8e001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65252d454317722 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8e4a00640 1 -- 172.21.15.86:0/571706568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00ab650 con 0x7fc8e815d0b0 2024-03-22T04:11:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8e4a00640 1 -- 172.21.15.86:0/571706568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e00b0070 con 0x7fc8e815d0b0 2024-03-22T04:11:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8e4a00640 1 -- 172.21.15.86:0/571706568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00aba10 con 0x7fc8e815d0b0 2024-03-22T04:11:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 -- 172.21.15.86:0/571706568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e815d0b0 msgr2=0x7fc8e81671d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 --2- 172.21.15.86:0/571706568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e815d0b0 0x7fc8e81671d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009870 tx=0x7fc8e001c560 comp rx=0 tx=0).stop 2024-03-22T04:11:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 -- 172.21.15.86:0/571706568 shutdown_connections 2024-03-22T04:11:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 --2- 172.21.15.86:0/571706568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e815d0b0 0x7fc8e81671d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 -- 172.21.15.86:0/571706568 >> 172.21.15.86:0/571706568 conn(0x7fc8e80cc940 msgr2=0x7fc8e80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 -- 172.21.15.86:0/571706568 shutdown_connections 2024-03-22T04:11:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 -- 172.21.15.86:0/571706568 wait complete. 2024-03-22T04:11:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 Processor -- start 2024-03-22T04:11:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.415+0000 7fc8ecc00640 1 -- start start 2024-03-22T04:11:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e80b2360 0x7fc8e81556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e80b3600 0x7fc8e8153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e815d0b0 0x7fc8e80727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8160c00 con 0x7fc8e80b3600 2024-03-22T04:11:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8169f30 con 0x7fc8e80b2360 2024-03-22T04:11:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e8071710 con 0x7fc8e815d0b0 2024-03-22T04:11:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e80b3600 0x7fc8e8153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e80b3600 0x7fc8e8153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38740/0 (socket says 172.21.15.86:38740) 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 -- 172.21.15.86:0/1970914077 learned_addr learned my addr 172.21.15.86:0/1970914077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5e00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e80b2360 0x7fc8e81556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e6800640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e815d0b0 0x7fc8e80727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 -- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e815d0b0 msgr2=0x7fc8e80727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e815d0b0 0x7fc8e80727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 -- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e80b2360 msgr2=0x7fc8e81556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e80b2360 0x7fc8e81556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 -- 172.21.15.86:0/1970914077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e80719f0 con 0x7fc8e80b3600 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5e00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e80b2360 0x7fc8e81556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8e5400640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e80b3600 0x7fc8e8153cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c480 tx=0x7fc8d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8de000640 1 -- 172.21.15.86:0/1970914077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4005480 con 0x7fc8e80b3600 2024-03-22T04:11:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8de000640 1 -- 172.21.15.86:0/1970914077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d4003600 con 0x7fc8e80b3600 2024-03-22T04:11:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8de000640 1 -- 172.21.15.86:0/1970914077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40039f0 con 0x7fc8e80b3600 2024-03-22T04:11:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e8071cd0 con 0x7fc8e80b3600 2024-03-22T04:11:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e8072290 con 0x7fc8e80b3600 2024-03-22T04:11:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e8074470 con 0x7fc8e80b3600 2024-03-22T04:11:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.419+0000 7fc8de000640 1 -- 172.21.15.86:0/1970914077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc8d4003b90 con 0x7fc8e80b3600 2024-03-22T04:11:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.423+0000 7fc8de000640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c8067b20 0x7fc8c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.423+0000 7fc8de000640 1 -- 172.21.15.86:0/1970914077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fc8d4085ec0 con 0x7fc8e80b3600 2024-03-22T04:11:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.423+0000 7fc8e5e00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c8067b20 0x7fc8c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.423+0000 7fc8e5e00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c8067b20 0x7fc8c8069fe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80012f0 tx=0x7fc8d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.427+0000 7fc8de000640 1 -- 172.21.15.86:0/1970914077 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc8d40565b0 con 0x7fc8e80b3600 2024-03-22T04:11:24.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.583+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc8e80cc6c0 con 0x7fc8e80b3600 2024-03-22T04:11:24.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.583+0000 7fc8de000640 1 -- 172.21.15.86:0/1970914077 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fc8d4055eb0 con 0x7fc8e80b3600 2024-03-22T04:11:24.589 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:24.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c8067b20 msgr2=0x7fc8c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c8067b20 0x7fc8c8069fe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80012f0 tx=0x7fc8d8009040 comp rx=0 tx=0).stop 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e80b3600 msgr2=0x7fc8e8153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e80b3600 0x7fc8e8153cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c480 tx=0x7fc8d400c940 comp rx=0 tx=0).stop 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 shutdown_connections 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c8067b20 0x7fc8c8069fe0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e815d0b0 0x7fc8e80727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e80b3600 0x7fc8e8153cf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 --2- 172.21.15.86:0/1970914077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e80b2360 0x7fc8e81556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 >> 172.21.15.86:0/1970914077 conn(0x7fc8e80cc940 msgr2=0x7fc8e80cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 shutdown_connections 2024-03-22T04:11:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.587+0000 7fc8ecc00640 1 -- 172.21.15.86:0/1970914077 wait complete. 2024-03-22T04:11:24.632 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-22T04:11:24.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:24.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.763+0000 7fdaf8800640 1 Processor -- start 2024-03-22T04:11:24.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.763+0000 7fdaf8800640 1 -- start start 2024-03-22T04:11:24.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf8800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdaf01444f0 0x7fdaf01692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf8800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf00ae5a0 con 0x7fdaf006cd80 2024-03-22T04:11:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf8800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf0144b80 con 0x7fdaf00aff50 2024-03-22T04:11:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf8800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf0162aa0 con 0x7fdaf01444f0 2024-03-22T04:11:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdaf006cd80 0x7fdaf0143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35458/0 (socket says 172.21.15.86:35458) 2024-03-22T04:11:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf5400640 1 -- 172.21.15.86:0/717265848 learned_addr learned my addr 172.21.15.86:0/717265848 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf6800640 1 --2- 172.21.15.86:0/717265848 >> v2:172.21.15.196:3300/0 conn(0x7fdaf01444f0 0x7fdaf01692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf4a00640 1 -- 172.21.15.86:0/717265848 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116848590 0 0) 0x7fdae0001120 con 0x7fdaf006cd80 2024-03-22T04:11:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf4a00640 1 -- 172.21.15.86:0/717265848 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdad8003530 con 0x7fdaf006cd80 2024-03-22T04:11:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf6800640 1 -- 172.21.15.86:0/717265848 >> v1:172.21.15.196:6789/0 conn(0x7fdaf00aff50 legacy=0x7fdaf006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf6800640 1 -- 172.21.15.86:0/717265848 >> v1:172.21.15.86:6789/0 conn(0x7fdaf006cd80 legacy=0x7fdaf0143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.767+0000 7fdaf6800640 1 -- 172.21.15.86:0/717265848 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf0162170 con 0x7fdaf01444f0 2024-03-22T04:11:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf6800640 1 --2- 172.21.15.86:0/717265848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf01444f0 0x7fdaf01692f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009d90 tx=0x7fdaec01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56219c3d18abbf94 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf4a00640 1 -- 172.21.15.86:0/717265848 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0b2070 con 0x7fdaf01444f0 2024-03-22T04:11:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf4a00640 1 -- 172.21.15.86:0/717265848 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec0a57a0 con 0x7fdaf01444f0 2024-03-22T04:11:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf4a00640 1 -- 172.21.15.86:0/717265848 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0a5b70 con 0x7fdaf01444f0 2024-03-22T04:11:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- 172.21.15.86:0/717265848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf01444f0 msgr2=0x7fdaf01692f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 --2- 172.21.15.86:0/717265848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf01444f0 0x7fdaf01692f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009d90 tx=0x7fdaec01c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- 172.21.15.86:0/717265848 shutdown_connections 2024-03-22T04:11:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 --2- 172.21.15.86:0/717265848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf01444f0 0x7fdaf01692f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- 172.21.15.86:0/717265848 >> 172.21.15.86:0/717265848 conn(0x7fdaf0071fe0 msgr2=0x7fdaf00723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- 172.21.15.86:0/717265848 shutdown_connections 2024-03-22T04:11:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- 172.21.15.86:0/717265848 wait complete. 2024-03-22T04:11:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 Processor -- start 2024-03-22T04:11:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- start start 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf006cd80 0x7fdaf0141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf00aff50 0x7fdaf013df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaf01444f0 0x7fdaf0143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00b04b0 con 0x7fdaf00aff50 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00b0330 con 0x7fdaf006cd80 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf8800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf016c460 con 0x7fdaf01444f0 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf00aff50 0x7fdaf013df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf00aff50 0x7fdaf013df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38742/0 (socket says 172.21.15.86:38742) 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.771+0000 7fdaf5400640 1 -- 172.21.15.86:0/3433953525 learned_addr learned my addr 172.21.15.86:0/3433953525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf6800640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaf01444f0 0x7fdaf0143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf5e00640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf006cd80 0x7fdaf0141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf5400640 1 -- 172.21.15.86:0/3433953525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaf01444f0 msgr2=0x7fdaf0143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf5400640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaf01444f0 0x7fdaf0143680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf5400640 1 -- 172.21.15.86:0/3433953525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf006cd80 msgr2=0x7fdaf0141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf5400640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf006cd80 0x7fdaf0141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf5400640 1 -- 172.21.15.86:0/3433953525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf013a780 con 0x7fdaf00aff50 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf5400640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf00aff50 0x7fdaf013df10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdae000c730 tx=0x7fdae000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaea000640 1 -- 172.21.15.86:0/3433953525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0017070 con 0x7fdaf00aff50 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaea000640 1 -- 172.21.15.86:0/3433953525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0005ac0 con 0x7fdaf00aff50 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf013aa60 con 0x7fdaf00aff50 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaea000640 1 -- 172.21.15.86:0/3433953525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0007070 con 0x7fdaf00aff50 2024-03-22T04:11:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf013e450 con 0x7fdaf00aff50 2024-03-22T04:11:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf0140b90 con 0x7fdaf00aff50 2024-03-22T04:11:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaea000640 1 -- 172.21.15.86:0/3433953525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdae001a070 con 0x7fdaf00aff50 2024-03-22T04:11:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaea000640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdad4067b20 0x7fdad4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaea000640 1 -- 172.21.15.86:0/3433953525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fdae0085650 con 0x7fdaf00aff50 2024-03-22T04:11:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.775+0000 7fdaf5e00640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdad4067b20 0x7fdad4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.779+0000 7fdaf5e00640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdad4067b20 0x7fdad4069fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdae4001810 tx=0x7fdae4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.779+0000 7fdaea000640 1 -- 172.21.15.86:0/3433953525 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdaf0140b90 con 0x7fdaf00aff50 2024-03-22T04:11:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdaf0068650 con 0x7fdaf00aff50 2024-03-22T04:11:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaea000640 1 -- 172.21.15.86:0/3433953525 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7fdaf00ae5a0 con 0x7fdaf00aff50 2024-03-22T04:11:24.948 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:24.948 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdad4067b20 msgr2=0x7fdad4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdad4067b20 0x7fdad4069fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdae4001810 tx=0x7fdae4009040 comp rx=0 tx=0).stop 2024-03-22T04:11:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf00aff50 msgr2=0x7fdaf013df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf00aff50 0x7fdaf013df10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdae000c730 tx=0x7fdae000cbf0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 shutdown_connections 2024-03-22T04:11:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdad4067b20 0x7fdad4069fe0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaf01444f0 0x7fdaf0143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf00aff50 0x7fdaf013df10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 --2- 172.21.15.86:0/3433953525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaf006cd80 0x7fdaf0141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 >> 172.21.15.86:0/3433953525 conn(0x7fdaf0071fe0 msgr2=0x7fdaf006c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.943+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 shutdown_connections 2024-03-22T04:11:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:24.947+0000 7fdaf8800640 1 -- 172.21.15.86:0/3433953525 wait complete. 2024-03-22T04:11:24.969 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:24.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:25.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de8000640 1 Processor -- start 2024-03-22T04:11:25.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de8000640 1 -- start start 2024-03-22T04:11:25.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de8000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8de0072650 0x7f8de0072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de8000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de015b760 0x7f8de015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de8000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de0073150 con 0x7f8de006e7e0 2024-03-22T04:11:25.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de8000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8de0071350 con 0x7f8de0072650 2024-03-22T04:11:25.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de8000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8de00715a0 con 0x7f8de015b760 2024-03-22T04:11:25.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8de0072650 0x7f8de0072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de5600640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f8de006e7e0 0x7f8de006ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:60362/0 (socket says 172.21.15.86:60362) 2024-03-22T04:11:25.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de5600640 1 -- 172.21.15.86:0/1949095921 learned_addr learned my addr 172.21.15.86:0/1949095921 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:25.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de6000640 1 --2- 172.21.15.86:0/1949095921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de015b760 0x7f8de015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de4c00640 1 -- 172.21.15.86:0/1949095921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de015b760 msgr2=0x7f8de015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:25.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de4c00640 1 --2- 172.21.15.86:0/1949095921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de015b760 0x7f8de015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de4c00640 1 -- 172.21.15.86:0/1949095921 >> v1:172.21.15.196:6789/0 conn(0x7f8de006e7e0 legacy=0x7f8de006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8ddfe00640 1 -- 172.21.15.86:0/1949095921 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237172103 0 0) 0x7f8de0073150 con 0x7f8de006e7e0 2024-03-22T04:11:25.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.103+0000 7f8de4c00640 1 -- 172.21.15.86:0/1949095921 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0071880 con 0x7f8de0072650 2024-03-22T04:11:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de4c00640 1 --2- 172.21.15.86:0/1949095921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0072650 0x7f8de0072b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0097c0 tx=0x7f8dcc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b719ebfdb143fa6e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8ddfe00640 1 -- 172.21.15.86:0/1949095921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0b1070 con 0x7f8de0072650 2024-03-22T04:11:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8ddfe00640 1 -- 172.21.15.86:0/1949095921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dcc0ac600 con 0x7f8de0072650 2024-03-22T04:11:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8ddfe00640 1 -- 172.21.15.86:0/1949095921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0ac9d0 con 0x7f8de0072650 2024-03-22T04:11:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de8000640 1 -- 172.21.15.86:0/1949095921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0072650 msgr2=0x7f8de0072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de8000640 1 --2- 172.21.15.86:0/1949095921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0072650 0x7f8de0072b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0097c0 tx=0x7f8dcc01c3e0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de8000640 1 -- 172.21.15.86:0/1949095921 shutdown_connections 2024-03-22T04:11:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de8000640 1 --2- 172.21.15.86:0/1949095921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de015b760 0x7f8de015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de8000640 1 --2- 172.21.15.86:0/1949095921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0072650 0x7f8de0072b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de8000640 1 -- 172.21.15.86:0/1949095921 >> 172.21.15.86:0/1949095921 conn(0x7f8de00cc940 msgr2=0x7f8de00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de8000640 1 -- 172.21.15.86:0/1949095921 shutdown_connections 2024-03-22T04:11:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.107+0000 7f8de8000640 1 -- 172.21.15.86:0/1949095921 wait complete. 2024-03-22T04:11:25.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 Processor -- start 2024-03-22T04:11:25.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 -- start start 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8de006e7e0 0x7f8de0155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de0072650 0x7f8de006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de015b760 0x7f8de015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0155c20 con 0x7f8de015b760 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0155aa0 con 0x7f8de006e7e0 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8de0167df0 con 0x7f8de0072650 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de015b760 0x7f8de015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de015b760 0x7f8de015d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38766/0 (socket says 172.21.15.86:38766) 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 -- 172.21.15.86:0/138666210 learned_addr learned my addr 172.21.15.86:0/138666210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de5600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8de006e7e0 0x7f8de0155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de4c00640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de0072650 0x7f8de006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 -- 172.21.15.86:0/138666210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de0072650 msgr2=0x7f8de006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de0072650 0x7f8de006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 -- 172.21.15.86:0/138666210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8de006e7e0 msgr2=0x7f8de0155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8de006e7e0 0x7f8de0155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 -- 172.21.15.86:0/138666210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0151a40 con 0x7f8de015b760 2024-03-22T04:11:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de6000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de015b760 0x7f8de015d830 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800e7f0 tx=0x7f8dd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8ddd600640 1 -- 172.21.15.86:0/138666210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8017070 con 0x7f8de015b760 2024-03-22T04:11:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8ddd600640 1 -- 172.21.15.86:0/138666210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd800c940 con 0x7f8de015b760 2024-03-22T04:11:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8ddd600640 1 -- 172.21.15.86:0/138666210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800eea0 con 0x7f8de015b760 2024-03-22T04:11:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de0151d20 con 0x7f8de015b760 2024-03-22T04:11:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.111+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de01522e0 con 0x7f8de015b760 2024-03-22T04:11:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.115+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de0074470 con 0x7f8de015b760 2024-03-22T04:11:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.115+0000 7f8ddd600640 1 -- 172.21.15.86:0/138666210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8dd801a070 con 0x7f8de015b760 2024-03-22T04:11:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.115+0000 7f8ddd600640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dc0067b20 0x7f8dc0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.115+0000 7f8ddd600640 1 -- 172.21.15.86:0/138666210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f8dd8086440 con 0x7f8de015b760 2024-03-22T04:11:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.115+0000 7f8de5600640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dc0067b20 0x7f8dc0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.115+0000 7f8de5600640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dc0067b20 0x7f8dc0069fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40012f0 tx=0x7f8dd4001690 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.119+0000 7f8ddd600640 1 -- 172.21.15.86:0/138666210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8dd8056450 con 0x7f8de015b760 2024-03-22T04:11:25.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.271+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8de0072dc0 con 0x7f8de015b760 2024-03-22T04:11:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.271+0000 7f8ddd600640 1 -- 172.21.15.86:0/138666210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f8dd805a4f0 con 0x7f8de015b760 2024-03-22T04:11:25.278 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dc0067b20 msgr2=0x7f8dc0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dc0067b20 0x7f8dc0069fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40012f0 tx=0x7f8dd4001690 comp rx=0 tx=0).stop 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de015b760 msgr2=0x7f8de015d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de015b760 0x7f8de015d830 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800e7f0 tx=0x7f8dd800ecb0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 shutdown_connections 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dc0067b20 0x7f8dc0069fe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de015b760 0x7f8de015d830 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8de0072650 0x7f8de006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 --2- 172.21.15.86:0/138666210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8de006e7e0 0x7f8de0155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 >> 172.21.15.86:0/138666210 conn(0x7f8de00cc940 msgr2=0x7f8de00694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 shutdown_connections 2024-03-22T04:11:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.275+0000 7f8de8000640 1 -- 172.21.15.86:0/138666210 wait complete. 2024-03-22T04:11:25.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:25.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.431+0000 7f9aad600640 1 Processor -- start 2024-03-22T04:11:25.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.431+0000 7f9aad600640 1 -- start start 2024-03-22T04:11:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.431+0000 7f9aad600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9aa8147d20 0x7f9aa806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.431+0000 7f9aad600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9aa806bba0 0x7f9aa806fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.431+0000 7f9aad600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa8070770 con 0x7f9aa806ff80 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.431+0000 7f9aad600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa806c010 con 0x7f9aa806bba0 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.431+0000 7f9aad600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa8148100 con 0x7f9aa8147d20 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9aa806bba0 0x7f9aa806fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9aa806bba0 0x7f9aa806fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38772/0 (socket says 172.21.15.86:38772) 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6000640 1 -- 172.21.15.86:0/308617759 learned_addr learned my addr 172.21.15.86:0/308617759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa7400640 1 --1- 172.21.15.86:0/308617759 >> v1:172.21.15.86:6789/0 conn(0x7f9aa806ff80 0x7f9aa8161f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35464/0 (socket says 172.21.15.86:35464) 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6a00640 1 --2- 172.21.15.86:0/308617759 >> v2:172.21.15.196:3300/0 conn(0x7f9aa8147d20 0x7f9aa806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6000640 1 -- 172.21.15.86:0/308617759 >> v2:172.21.15.196:3300/0 conn(0x7f9aa8147d20 msgr2=0x7f9aa806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6000640 1 --2- 172.21.15.86:0/308617759 >> v2:172.21.15.196:3300/0 conn(0x7f9aa8147d20 0x7f9aa806b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6000640 1 -- 172.21.15.86:0/308617759 >> v1:172.21.15.86:6789/0 conn(0x7f9aa806ff80 legacy=0x7f9aa8161f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6000640 1 -- 172.21.15.86:0/308617759 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa8073470 con 0x7f9aa806bba0 2024-03-22T04:11:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa6000640 1 --2- 172.21.15.86:0/308617759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806bba0 0x7f9aa806fa40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009870 tx=0x7f9a9401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5f174f506bc278 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa5600640 1 -- 172.21.15.86:0/308617759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940ab620 con 0x7f9aa806bba0 2024-03-22T04:11:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa5600640 1 -- 172.21.15.86:0/308617759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a940b0070 con 0x7f9aa806bba0 2024-03-22T04:11:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.435+0000 7f9aa5600640 1 -- 172.21.15.86:0/308617759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940ab9e0 con 0x7f9aa806bba0 2024-03-22T04:11:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- 172.21.15.86:0/308617759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806bba0 msgr2=0x7f9aa806fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 --2- 172.21.15.86:0/308617759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806bba0 0x7f9aa806fa40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009870 tx=0x7f9a9401c500 comp rx=0 tx=0).stop 2024-03-22T04:11:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- 172.21.15.86:0/308617759 shutdown_connections 2024-03-22T04:11:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 --2- 172.21.15.86:0/308617759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806bba0 0x7f9aa806fa40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 --2- 172.21.15.86:0/308617759 >> v2:172.21.15.196:3300/0 conn(0x7f9aa8147d20 0x7f9aa806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- 172.21.15.86:0/308617759 >> 172.21.15.86:0/308617759 conn(0x7f9aa80d0fa0 msgr2=0x7f9aa80d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- 172.21.15.86:0/308617759 shutdown_connections 2024-03-22T04:11:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- 172.21.15.86:0/308617759 wait complete. 2024-03-22T04:11:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 Processor -- start 2024-03-22T04:11:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- start start 2024-03-22T04:11:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aa806bba0 0x7f9aa813a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806ff80 0x7f9aa8160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9aa8147d20 0x7f9aa8161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa80ce260 con 0x7f9aa806ff80 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa80cdf80 con 0x7f9aa806bba0 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aad600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa8070390 con 0x7f9aa8147d20 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806ff80 0x7f9aa8160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806ff80 0x7f9aa8160cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38786/0 (socket says 172.21.15.86:38786) 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 -- 172.21.15.86:0/3330945258 learned_addr learned my addr 172.21.15.86:0/3330945258 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6a00640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aa806bba0 0x7f9aa813a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa7400640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9aa8147d20 0x7f9aa8161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 -- 172.21.15.86:0/3330945258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9aa8147d20 msgr2=0x7f9aa8161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9aa8147d20 0x7f9aa8161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 -- 172.21.15.86:0/3330945258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aa806bba0 msgr2=0x7f9aa813a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aa806bba0 0x7f9aa813a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 -- 172.21.15.86:0/3330945258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa80cc450 con 0x7f9aa806ff80 2024-03-22T04:11:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.439+0000 7f9aa6000640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806ff80 0x7f9aa8160cd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a94019110 tx=0x7f9a9401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9a9ea00640 1 -- 172.21.15.86:0/3330945258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940abcd0 con 0x7f9aa806ff80 2024-03-22T04:11:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9a9ea00640 1 -- 172.21.15.86:0/3330945258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a940be3f0 con 0x7f9aa806ff80 2024-03-22T04:11:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9a9ea00640 1 -- 172.21.15.86:0/3330945258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940be7e0 con 0x7f9aa806ff80 2024-03-22T04:11:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa815d060 con 0x7f9aa806ff80 2024-03-22T04:11:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa815d620 con 0x7f9aa806ff80 2024-03-22T04:11:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa8074470 con 0x7f9aa806ff80 2024-03-22T04:11:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9a9ea00640 1 -- 172.21.15.86:0/3330945258 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9a940be980 con 0x7f9aa806ff80 2024-03-22T04:11:25.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9a9ea00640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a88067b50 0x7f9a8806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9a9ea00640 1 -- 172.21.15.86:0/3330945258 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f9a941202f0 con 0x7f9aa806ff80 2024-03-22T04:11:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.443+0000 7f9aa6a00640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a88067b50 0x7f9a8806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.447+0000 7f9aa6a00640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a88067b50 0x7f9a8806a010 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009b60 tx=0x7f9a98009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.447+0000 7f9a9ea00640 1 -- 172.21.15.86:0/3330945258 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9a940c4060 con 0x7f9aa806ff80 2024-03-22T04:11:25.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9aa80ca590 con 0x7f9aa806ff80 2024-03-22T04:11:25.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9a9ea00640 1 -- 172.21.15.86:0/3330945258 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f9a940160f0 con 0x7f9aa806ff80 2024-03-22T04:11:25.607 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a88067b50 msgr2=0x7f9a8806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a88067b50 0x7f9a8806a010 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009b60 tx=0x7f9a98009190 comp rx=0 tx=0).stop 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806ff80 msgr2=0x7f9aa8160cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806ff80 0x7f9aa8160cd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a94019110 tx=0x7f9a9401ad40 comp rx=0 tx=0).stop 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 shutdown_connections 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a88067b50 0x7f9a8806a010 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9aa8147d20 0x7f9aa8161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa806ff80 0x7f9aa8160cd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 --2- 172.21.15.86:0/3330945258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aa806bba0 0x7f9aa813a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 >> 172.21.15.86:0/3330945258 conn(0x7f9aa80d0fa0 msgr2=0x7f9aa8157710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:25.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 shutdown_connections 2024-03-22T04:11:25.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.603+0000 7f9aad600640 1 -- 172.21.15.86:0/3330945258 wait complete. 2024-03-22T04:11:25.627 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.759+0000 7f6500e00640 1 Processor -- start 2024-03-22T04:11:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.759+0000 7f6500e00640 1 -- start start 2024-03-22T04:11:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.759+0000 7f6500e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64fc072650 0x7f64fc072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.759+0000 7f6500e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc15b760 0x7f64fc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.759+0000 7f6500e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc073150 con 0x7f64fc06e7e0 2024-03-22T04:11:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.759+0000 7f6500e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc071350 con 0x7f64fc072650 2024-03-22T04:11:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.759+0000 7f6500e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0715a0 con 0x7f64fc15b760 2024-03-22T04:11:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f64fc06e7e0 0x7f64fc06ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35468/0 (socket says 172.21.15.86:35468) 2024-03-22T04:11:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64fc072650 0x7f64fc072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f9e00640 1 -- 172.21.15.86:0/1896298661 learned_addr learned my addr 172.21.15.86:0/1896298661 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64fa800640 1 --2- 172.21.15.86:0/1896298661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc15b760 0x7f64fc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f1400640 1 -- 172.21.15.86:0/1896298661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc15b760 msgr2=0x7f64fc15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f1400640 1 --2- 172.21.15.86:0/1896298661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc15b760 0x7f64fc15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f1400640 1 -- 172.21.15.86:0/1896298661 >> v1:172.21.15.86:6789/0 conn(0x7f64fc06e7e0 legacy=0x7f64fc06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f1400640 1 -- 172.21.15.86:0/1896298661 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc071880 con 0x7f64fc072650 2024-03-22T04:11:25.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f1400640 1 --2- 172.21.15.86:0/1896298661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc072650 0x7f64fc072b00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64e40097c0 tx=0x7f64e401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a389eb1edbd0e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f9400640 1 -- 172.21.15.86:0/1896298661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e40b0070 con 0x7f64fc072650 2024-03-22T04:11:25.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f9400640 1 -- 172.21.15.86:0/1896298661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e40ab400 con 0x7f64fc072650 2024-03-22T04:11:25.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.763+0000 7f64f9400640 1 -- 172.21.15.86:0/1896298661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e40ab7d0 con 0x7f64fc072650 2024-03-22T04:11:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- 172.21.15.86:0/1896298661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc072650 msgr2=0x7f64fc072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 --2- 172.21.15.86:0/1896298661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc072650 0x7f64fc072b00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64e40097c0 tx=0x7f64e401c270 comp rx=0 tx=0).stop 2024-03-22T04:11:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- 172.21.15.86:0/1896298661 shutdown_connections 2024-03-22T04:11:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 --2- 172.21.15.86:0/1896298661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc15b760 0x7f64fc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 --2- 172.21.15.86:0/1896298661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc072650 0x7f64fc072b00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- 172.21.15.86:0/1896298661 >> 172.21.15.86:0/1896298661 conn(0x7f64fc0cc940 msgr2=0x7f64fc0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- 172.21.15.86:0/1896298661 shutdown_connections 2024-03-22T04:11:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- 172.21.15.86:0/1896298661 wait complete. 2024-03-22T04:11:25.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 Processor -- start 2024-03-22T04:11:25.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- start start 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc06e7e0 0x7f64fc168a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64fc072650 0x7f64fc14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc15b760 0x7f64fc1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc155c20 con 0x7f64fc15b760 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc155aa0 con 0x7f64fc072650 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f6500e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc165de0 con 0x7f64fc06e7e0 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f64fa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc15b760 0x7f64fc1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f64f9e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc06e7e0 0x7f64fc168a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f64f1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64fc072650 0x7f64fc14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f64fa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc15b760 0x7f64fc1581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38810/0 (socket says 172.21.15.86:38810) 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f64f1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64fc072650 0x7f64fc14de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:56448/0 (socket says 172.21.15.86:56448) 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f64fa800640 1 -- 172.21.15.86:0/883166715 learned_addr learned my addr 172.21.15.86:0/883166715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f64fa800640 1 -- 172.21.15.86:0/883166715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc06e7e0 msgr2=0x7f64fc168a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.767+0000 7f64fa800640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc06e7e0 0x7f64fc168a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64fa800640 1 -- 172.21.15.86:0/883166715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64fc072650 msgr2=0x7f64fc14de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64fa800640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64fc072650 0x7f64fc14de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64fa800640 1 -- 172.21.15.86:0/883166715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc15bf60 con 0x7f64fc15b760 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64f1400640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64fc072650 0x7f64fc14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64fa800640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc15b760 0x7f64fc1581f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f64f400efc0 tx=0x7f64f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64f2a00640 1 -- 172.21.15.86:0/883166715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4018020 con 0x7f64fc15b760 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64f2a00640 1 -- 172.21.15.86:0/883166715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f4014480 con 0x7f64fc15b760 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64f2a00640 1 -- 172.21.15.86:0/883166715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4014870 con 0x7f64fc15b760 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc148d00 con 0x7f64fc15b760 2024-03-22T04:11:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc148a20 con 0x7f64fc15b760 2024-03-22T04:11:25.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc074470 con 0x7f64fc15b760 2024-03-22T04:11:25.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64f2a00640 1 -- 172.21.15.86:0/883166715 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f64f401b020 con 0x7f64fc15b760 2024-03-22T04:11:25.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64f2a00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d8067b20 0x7f64d8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:25.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64f2a00640 1 -- 172.21.15.86:0/883166715 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f64f4086b60 con 0x7f64fc15b760 2024-03-22T04:11:25.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.771+0000 7f64f9e00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d8067b20 0x7f64d8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:25.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.775+0000 7f64f9e00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d8067b20 0x7f64d8069fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f64ec000f70 tx=0x7f64ec007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:25.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.775+0000 7f64f2a00640 1 -- 172.21.15.86:0/883166715 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f64f4022070 con 0x7f64fc15b760 2024-03-22T04:11:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f64fc0cf480 con 0x7f64fc15b760 2024-03-22T04:11:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f64f2a00640 1 -- 172.21.15.86:0/883166715 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f64f4010040 con 0x7f64fc15b760 2024-03-22T04:11:25.939 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d8067b20 msgr2=0x7f64d8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d8067b20 0x7f64d8069fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f64ec000f70 tx=0x7f64ec007e30 comp rx=0 tx=0).stop 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc15b760 msgr2=0x7f64fc1581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc15b760 0x7f64fc1581f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f64f400efc0 tx=0x7f64f400c490 comp rx=0 tx=0).stop 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 shutdown_connections 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d8067b20 0x7f64d8069fe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64fc072650 0x7f64fc14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc15b760 0x7f64fc1581f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 --2- 172.21.15.86:0/883166715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64fc06e7e0 0x7f64fc168a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 >> 172.21.15.86:0/883166715 conn(0x7f64fc0cc940 msgr2=0x7f64fc0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 shutdown_connections 2024-03-22T04:11:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:25.935+0000 7f6500e00640 1 -- 172.21.15.86:0/883166715 wait complete. 2024-03-22T04:11:25.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:26.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f6071c00640 1 Processor -- start 2024-03-22T04:11:26.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f6071c00640 1 -- start start 2024-03-22T04:11:26.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f6071c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c6f20 0x7f606c072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f6071c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f606c0cb1d0 0x7f606c162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f6071c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606c0cc7c0 con 0x7f606c0c7300 2024-03-22T04:11:26.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f6071c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f606c0cb720 con 0x7f606c0cb1d0 2024-03-22T04:11:26.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f6071c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f606c0cba00 con 0x7f606c0c6f20 2024-03-22T04:11:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f606a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f606c0cb1d0 0x7f606c162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f606a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f606c0cb1d0 0x7f606c162030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57216/0 (socket says 172.21.15.86:57216) 2024-03-22T04:11:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f606a000640 1 -- 172.21.15.86:0/3003403968 learned_addr learned my addr 172.21.15.86:0/3003403968 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f606aa00640 1 --2- 172.21.15.86:0/3003403968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c6f20 0x7f606c072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f606a000640 1 -- 172.21.15.86:0/3003403968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c6f20 msgr2=0x7f606c072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f606a000640 1 --2- 172.21.15.86:0/3003403968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c6f20 0x7f606c072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f606a000640 1 -- 172.21.15.86:0/3003403968 >> v1:172.21.15.196:6789/0 conn(0x7f606c0c7300 legacy=0x7f606c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.095+0000 7f606a000640 1 -- 172.21.15.86:0/3003403968 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c0cbce0 con 0x7f606c0cb1d0 2024-03-22T04:11:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f606a000640 1 --2- 172.21.15.86:0/3003403968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0cb1d0 0x7f606c162030 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6058009870 tx=0x7f605801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1720b1ba6a90ba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6069600640 1 -- 172.21.15.86:0/3003403968 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60580b1070 con 0x7f606c0cb1d0 2024-03-22T04:11:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6069600640 1 -- 172.21.15.86:0/3003403968 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60580ac410 con 0x7f606c0cb1d0 2024-03-22T04:11:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6069600640 1 -- 172.21.15.86:0/3003403968 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60580ac7e0 con 0x7f606c0cb1d0 2024-03-22T04:11:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6071c00640 1 -- 172.21.15.86:0/3003403968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0cb1d0 msgr2=0x7f606c162030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6071c00640 1 --2- 172.21.15.86:0/3003403968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0cb1d0 0x7f606c162030 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6058009870 tx=0x7f605801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6071c00640 1 -- 172.21.15.86:0/3003403968 shutdown_connections 2024-03-22T04:11:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6071c00640 1 --2- 172.21.15.86:0/3003403968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0cb1d0 0x7f606c162030 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6071c00640 1 --2- 172.21.15.86:0/3003403968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c6f20 0x7f606c072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6071c00640 1 -- 172.21.15.86:0/3003403968 >> 172.21.15.86:0/3003403968 conn(0x7f606c0d0730 msgr2=0x7f606c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6071c00640 1 -- 172.21.15.86:0/3003403968 shutdown_connections 2024-03-22T04:11:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.099+0000 7f6071c00640 1 -- 172.21.15.86:0/3003403968 wait complete. 2024-03-22T04:11:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 Processor -- start 2024-03-22T04:11:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 -- start start 2024-03-22T04:11:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0c6f20 0x7f606c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c7300 0x7f606c0c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f606c0cb1d0 0x7f606c156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f606c072f00 con 0x7f606c0c6f20 2024-03-22T04:11:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f606c072d80 con 0x7f606c0cb1d0 2024-03-22T04:11:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f606c16b390 con 0x7f606c0c7300 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0c6f20 0x7f606c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0c6f20 0x7f606c154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57220/0 (socket says 172.21.15.86:57220) 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 -- 172.21.15.86:0/3459215461 learned_addr learned my addr 172.21.15.86:0/3459215461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606a000640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c7300 0x7f606c0c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606b400640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f606c0cb1d0 0x7f606c156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 -- 172.21.15.86:0/3459215461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c7300 msgr2=0x7f606c0c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c7300 0x7f606c0c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 -- 172.21.15.86:0/3459215461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f606c0cb1d0 msgr2=0x7f606c156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f606c0cb1d0 0x7f606c156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 -- 172.21.15.86:0/3459215461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c0c8bd0 con 0x7f606c0c6f20 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f606aa00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0c6f20 0x7f606c154e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f605c00e870 tx=0x7f605c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6062a00640 1 -- 172.21.15.86:0/3459215461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c017070 con 0x7f606c0c6f20 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6062a00640 1 -- 172.21.15.86:0/3459215461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605c00c940 con 0x7f606c0c6f20 2024-03-22T04:11:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6062a00640 1 -- 172.21.15.86:0/3459215461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c00ccf0 con 0x7f606c0c6f20 2024-03-22T04:11:26.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c0ae140 con 0x7f606c0c6f20 2024-03-22T04:11:26.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.103+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c0ae700 con 0x7f606c0c6f20 2024-03-22T04:11:26.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.107+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c074470 con 0x7f606c0c6f20 2024-03-22T04:11:26.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.107+0000 7f6062a00640 1 -- 172.21.15.86:0/3459215461 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f605c01a070 con 0x7f606c0c6f20 2024-03-22T04:11:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.107+0000 7f6062a00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f604c067b20 0x7f604c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.107+0000 7f6062a00640 1 -- 172.21.15.86:0/3459215461 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f605c086520 con 0x7f606c0c6f20 2024-03-22T04:11:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.107+0000 7f606a000640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f604c067b20 0x7f604c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.107+0000 7f606a000640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f604c067b20 0x7f604c069fe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f60580ae000 tx=0x7f60580a6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.111+0000 7f6062a00640 1 -- 172.21.15.86:0/3459215461 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f605c056490 con 0x7f606c0c6f20 2024-03-22T04:11:26.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.263+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f606c15ae50 con 0x7f606c0c6f20 2024-03-22T04:11:26.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.263+0000 7f6062a00640 1 -- 172.21.15.86:0/3459215461 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f605c05a530 con 0x7f606c0c6f20 2024-03-22T04:11:26.269 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f604c067b20 msgr2=0x7f604c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f604c067b20 0x7f604c069fe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f60580ae000 tx=0x7f60580a6040 comp rx=0 tx=0).stop 2024-03-22T04:11:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0c6f20 msgr2=0x7f606c154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0c6f20 0x7f606c154e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f605c00e870 tx=0x7f605c00ec70 comp rx=0 tx=0).stop 2024-03-22T04:11:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 shutdown_connections 2024-03-22T04:11:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f604c067b20 0x7f604c069fe0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f606c0cb1d0 0x7f606c156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f606c0c7300 0x7f606c0c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 --2- 172.21.15.86:0/3459215461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f606c0c6f20 0x7f606c154e40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 >> 172.21.15.86:0/3459215461 conn(0x7f606c0d0730 msgr2=0x7f606c157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 shutdown_connections 2024-03-22T04:11:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.267+0000 7f6071c00640 1 -- 172.21.15.86:0/3459215461 wait complete. 2024-03-22T04:11:26.290 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-22T04:11:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.419+0000 7f6abb800640 1 Processor -- start 2024-03-22T04:11:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.419+0000 7f6abb800640 1 -- start start 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6abb800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6ab40b4350 0x7f6ab40b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6abb800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ab415ef40 0x7f6ab415f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6abb800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab415ea90 con 0x7f6ab4147880 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6abb800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab40b6320 con 0x7f6ab415ef40 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6abb800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab406d4f0 con 0x7f6ab40b4350 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6ab9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ab415ef40 0x7f6ab415f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6aabe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6ab40b4350 0x7f6ab40b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6aabe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6ab40b4350 0x7f6ab40b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:42000/0 (socket says 172.21.15.86:42000) 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6aabe00640 1 -- 172.21.15.86:0/1512155788 learned_addr learned my addr 172.21.15.86:0/1512155788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6ab9800640 1 --2- 172.21.15.86:0/1512155788 >> v2:172.21.15.86:3300/0 conn(0x7f6ab415ef40 0x7f6ab415f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57224/0 (socket says 172.21.15.86:57224) 2024-03-22T04:11:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6aabe00640 1 -- 172.21.15.86:0/1512155788 >> v2:172.21.15.86:3300/0 conn(0x7f6ab415ef40 msgr2=0x7f6ab415f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:26.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6aabe00640 1 --2- 172.21.15.86:0/1512155788 >> v2:172.21.15.86:3300/0 conn(0x7f6ab415ef40 0x7f6ab415f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6aabe00640 1 -- 172.21.15.86:0/1512155788 >> v1:172.21.15.196:6789/0 conn(0x7f6ab4147880 legacy=0x7f6ab406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6aabe00640 1 -- 172.21.15.86:0/1512155788 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab406d7d0 con 0x7f6ab40b4350 2024-03-22T04:11:26.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6ab9800640 1 --2- 172.21.15.86:0/1512155788 >> v2:172.21.15.86:3300/0 conn(0x7f6ab415ef40 0x7f6ab415f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:26.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6aabe00640 1 --2- 172.21.15.86:0/1512155788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab40b4350 0x7f6ab40b47e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009870 tx=0x7f6a9c01c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4697f0a4a8110c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:26.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6ab3e00640 1 -- 172.21.15.86:0/1512155788 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0b0070 con 0x7f6ab40b4350 2024-03-22T04:11:26.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.423+0000 7f6ab3e00640 1 -- 172.21.15.86:0/1512155788 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c0ab600 con 0x7f6ab40b4350 2024-03-22T04:11:26.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab3e00640 1 -- 172.21.15.86:0/1512155788 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0ab9d0 con 0x7f6ab40b4350 2024-03-22T04:11:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- 172.21.15.86:0/1512155788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab40b4350 msgr2=0x7f6ab40b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 --2- 172.21.15.86:0/1512155788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab40b4350 0x7f6ab40b47e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009870 tx=0x7f6a9c01c520 comp rx=0 tx=0).stop 2024-03-22T04:11:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- 172.21.15.86:0/1512155788 shutdown_connections 2024-03-22T04:11:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 --2- 172.21.15.86:0/1512155788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab40b4350 0x7f6ab40b47e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 --2- 172.21.15.86:0/1512155788 >> v2:172.21.15.86:3300/0 conn(0x7f6ab415ef40 0x7f6ab415f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- 172.21.15.86:0/1512155788 >> 172.21.15.86:0/1512155788 conn(0x7f6ab406ba60 msgr2=0x7f6ab406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- 172.21.15.86:0/1512155788 shutdown_connections 2024-03-22T04:11:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- 172.21.15.86:0/1512155788 wait complete. 2024-03-22T04:11:26.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 Processor -- start 2024-03-22T04:11:26.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- start start 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ab40b4350 0x7f6ab413a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab4147880 0x7f6ab41f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab415ef40 0x7f6ab4142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab406d210 con 0x7f6ab415ef40 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab406d090 con 0x7f6ab4147880 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6abb800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab416b420 con 0x7f6ab40b4350 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab415ef40 0x7f6ab4142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab415ef40 0x7f6ab4142910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57232/0 (socket says 172.21.15.86:57232) 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab9800640 1 -- 172.21.15.86:0/2387115131 learned_addr learned my addr 172.21.15.86:0/2387115131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6aabe00640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab4147880 0x7f6ab41f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab8e00640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ab40b4350 0x7f6ab413a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab9800640 1 -- 172.21.15.86:0/2387115131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ab40b4350 msgr2=0x7f6ab413a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab9800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ab40b4350 0x7f6ab413a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab9800640 1 -- 172.21.15.86:0/2387115131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab4147880 msgr2=0x7f6ab41f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab9800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab4147880 0x7f6ab41f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.427+0000 7f6ab9800640 1 -- 172.21.15.86:0/2387115131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab41459f0 con 0x7f6ab415ef40 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab9800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab415ef40 0x7f6ab4142910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00efc0 tx=0x7f6aac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab1600640 1 -- 172.21.15.86:0/2387115131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac018030 con 0x7f6ab415ef40 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab1600640 1 -- 172.21.15.86:0/2387115131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aac00cdf0 con 0x7f6ab415ef40 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab1600640 1 -- 172.21.15.86:0/2387115131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac003bc0 con 0x7f6ab415ef40 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab4145cd0 con 0x7f6ab415ef40 2024-03-22T04:11:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab4146290 con 0x7f6ab415ef40 2024-03-22T04:11:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab4068650 con 0x7f6ab415ef40 2024-03-22T04:11:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab1600640 1 -- 172.21.15.86:0/2387115131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6aac01b070 con 0x7f6ab415ef40 2024-03-22T04:11:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab1600640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6a90067b20 0x7f6a90069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab1600640 1 -- 172.21.15.86:0/2387115131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f6aac086b00 con 0x7f6ab415ef40 2024-03-22T04:11:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab8e00640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6a90067b20 0x7f6a90069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:26.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.431+0000 7f6ab8e00640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6a90067b20 0x7f6a90069fe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4139060 tx=0x7f6aa4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.435+0000 7f6ab1600640 1 -- 172.21.15.86:0/2387115131 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6aac010040 con 0x7f6ab415ef40 2024-03-22T04:11:26.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.643+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f6ab4072220 con 0x7f6ab415ef40 2024-03-22T04:11:27.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6ab1600640 1 -- 172.21.15.86:0/2387115131 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v75) ==== 145+0+0 (secure 0 0 0) 0x7f6aac056b00 con 0x7f6ab415ef40 2024-03-22T04:11:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6a90067b20 msgr2=0x7f6a90069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6a90067b20 0x7f6a90069fe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4139060 tx=0x7f6aa4009040 comp rx=0 tx=0).stop 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab415ef40 msgr2=0x7f6ab4142910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab415ef40 0x7f6ab4142910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00efc0 tx=0x7f6aac00c490 comp rx=0 tx=0).stop 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 shutdown_connections 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6a90067b20 0x7f6a90069fe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab4147880 0x7f6ab41f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab415ef40 0x7f6ab4142910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 --2- 172.21.15.86:0/2387115131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ab40b4350 0x7f6ab413a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 >> 172.21.15.86:0/2387115131 conn(0x7f6ab406ba60 msgr2=0x7f6ab40b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 shutdown_connections 2024-03-22T04:11:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:26.999+0000 7f6abb800640 1 -- 172.21.15.86:0/2387115131 wait complete. 2024-03-22T04:11:27.025 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-22T04:11:27.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:11:27.148 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:11:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f849b000640 1 Processor -- start 2024-03-22T04:11:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f849b000640 1 -- start start 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f849b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84940b2360 0x7f84940b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f849b000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b3430 0x7f849415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f849b000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f849415cee0 0x7f849406d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f849b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f84940b2740 con 0x7f84940b2360 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f849b000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f849415d710 con 0x7f849415cee0 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f849b000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f849415fe40 con 0x7f84940b3430 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8493e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84940b2360 0x7f84940b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8493e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84940b2360 0x7f84940b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57246/0 (socket says 172.21.15.86:57246) 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8493e00640 1 -- 172.21.15.86:0/1482889561 learned_addr learned my addr 172.21.15.86:0/1482889561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f848be00640 1 --2- 172.21.15.86:0/1482889561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b3430 0x7f849415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8499000640 1 --2- 172.21.15.86:0/1482889561 >> v2:172.21.15.196:3300/0 conn(0x7f849415cee0 0x7f849406d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8493e00640 1 -- 172.21.15.86:0/1482889561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b3430 msgr2=0x7f849415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8493e00640 1 --2- 172.21.15.86:0/1482889561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b3430 0x7f849415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8493e00640 1 -- 172.21.15.86:0/1482889561 >> v2:172.21.15.196:3300/0 conn(0x7f849415cee0 msgr2=0x7f849406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8493e00640 1 --2- 172.21.15.86:0/1482889561 >> v2:172.21.15.196:3300/0 conn(0x7f849415cee0 0x7f849406d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.159+0000 7f8493e00640 1 -- 172.21.15.86:0/1482889561 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8494160120 con 0x7f84940b2360 2024-03-22T04:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f8493e00640 1 --2- 172.21.15.86:0/1482889561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b2360 0x7f84940b2ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f849406f590 tx=0x7f848401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63649f0bb87a8a05 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f8493400640 1 -- 172.21.15.86:0/1482889561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84840b0070 con 0x7f84940b2360 2024-03-22T04:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f8493400640 1 -- 172.21.15.86:0/1482889561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84840ab600 con 0x7f84940b2360 2024-03-22T04:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f8493400640 1 -- 172.21.15.86:0/1482889561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84840ab9d0 con 0x7f84940b2360 2024-03-22T04:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 -- 172.21.15.86:0/1482889561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b2360 msgr2=0x7f84940b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 --2- 172.21.15.86:0/1482889561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b2360 0x7f84940b2ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f849406f590 tx=0x7f848401c260 comp rx=0 tx=0).stop 2024-03-22T04:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 -- 172.21.15.86:0/1482889561 shutdown_connections 2024-03-22T04:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 --2- 172.21.15.86:0/1482889561 >> v2:172.21.15.196:3300/0 conn(0x7f849415cee0 0x7f849406d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 --2- 172.21.15.86:0/1482889561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b3430 0x7f849415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 --2- 172.21.15.86:0/1482889561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b2360 0x7f84940b2ef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 -- 172.21.15.86:0/1482889561 >> 172.21.15.86:0/1482889561 conn(0x7f84940cc940 msgr2=0x7f84940ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 -- 172.21.15.86:0/1482889561 shutdown_connections 2024-03-22T04:11:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.163+0000 7f849b000640 1 -- 172.21.15.86:0/1482889561 wait complete. 2024-03-22T04:11:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 Processor -- start 2024-03-22T04:11:27.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 -- start start 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b2360 0x7f849414a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b3430 0x7f849414aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849415cee0 0x7f849414cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f849415d470 con 0x7f84940b3430 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f849415d2f0 con 0x7f849415cee0 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f84940b4d80 con 0x7f84940b2360 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f848be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b3430 0x7f849414aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f8493e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b2360 0x7f849414a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f8493e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b2360 0x7f849414a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42146/0 (socket says 172.21.15.86:42146) 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f8493e00640 1 -- 172.21.15.86:0/2116182889 learned_addr learned my addr 172.21.15.86:0/2116182889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f8499000640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849415cee0 0x7f849414cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f848be00640 1 -- 172.21.15.86:0/2116182889 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b2360 msgr2=0x7f849414a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f848be00640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b2360 0x7f849414a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f848be00640 1 -- 172.21.15.86:0/2116182889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849415cee0 msgr2=0x7f849414cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f848be00640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849415cee0 0x7f849414cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f848be00640 1 -- 172.21.15.86:0/2116182889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8494148950 con 0x7f84940b3430 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f848be00640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b3430 0x7f849414aaa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f847c00e870 tx=0x7f847c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f8490c00640 1 -- 172.21.15.86:0/2116182889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c017070 con 0x7f84940b3430 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f8490c00640 1 -- 172.21.15.86:0/2116182889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847c00c940 con 0x7f84940b3430 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f8490c00640 1 -- 172.21.15.86:0/2116182889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c00ccf0 con 0x7f84940b3430 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.167+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8494148c30 con 0x7f84940b3430 2024-03-22T04:11:27.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.171+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84941491f0 con 0x7f84940b3430 2024-03-22T04:11:27.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.171+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8494074470 con 0x7f84940b3430 2024-03-22T04:11:27.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.171+0000 7f8490c00640 1 -- 172.21.15.86:0/2116182889 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f847c01a070 con 0x7f84940b3430 2024-03-22T04:11:27.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.171+0000 7f8490c00640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8470067b20 0x7f8470069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.171+0000 7f8490c00640 1 -- 172.21.15.86:0/2116182889 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f847c086180 con 0x7f84940b3430 2024-03-22T04:11:27.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.171+0000 7f8493e00640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8470067b20 0x7f8470069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.171+0000 7f8493e00640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8470067b20 0x7f8470069fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f848401c2a0 tx=0x7f8484008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.175+0000 7f8490c00640 1 -- 172.21.15.86:0/2116182889 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f847c056190 con 0x7f84940b3430 2024-03-22T04:11:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.331+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f849406e9f0 con 0x7f84940b3430 2024-03-22T04:11:27.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.335+0000 7f8490c00640 1 -- 172.21.15.86:0/2116182889 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f847c05a230 con 0x7f84940b3430 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8470067b20 msgr2=0x7f8470069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8470067b20 0x7f8470069fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f848401c2a0 tx=0x7f8484008040 comp rx=0 tx=0).stop 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b3430 msgr2=0x7f849414aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b3430 0x7f849414aaa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f847c00e870 tx=0x7f847c00ed30 comp rx=0 tx=0).stop 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 shutdown_connections 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8470067b20 0x7f8470069fe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f849415cee0 0x7f849414cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84940b3430 0x7f849414aaa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 --2- 172.21.15.86:0/2116182889 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84940b2360 0x7f849414a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 >> 172.21.15.86:0/2116182889 conn(0x7f84940cc940 msgr2=0x7f84940afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 shutdown_connections 2024-03-22T04:11:27.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.339+0000 7f849b000640 1 -- 172.21.15.86:0/2116182889 wait complete. 2024-03-22T04:11:27.362 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:11:27.362 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:11:27.461 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:11:27.556 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:11:27.556 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:11:27.556 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:27.565 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:11:27.565 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:27.613 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:11:27.613 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:27.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd053800640 1 Processor -- start 2024-03-22T04:11:27.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd053800640 1 -- start start 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd053800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd04c0ad330 0x7fd04c15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd053800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd04c0ad710 0x7fd04c142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd053800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd04c15c7c0 con 0x7fd04c06ef40 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd053800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd04c06dea0 con 0x7fd04c0ad710 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd053800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd04c06e180 con 0x7fd04c0ad330 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd051800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd04c0ad710 0x7fd04c142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd050e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd04c0ad330 0x7fd04c15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd051800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd04c0ad710 0x7fd04c142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57262/0 (socket says 172.21.15.86:57262) 2024-03-22T04:11:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.835+0000 7fd051800640 1 -- 172.21.15.86:0/86022793 learned_addr learned my addr 172.21.15.86:0/86022793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd051800640 1 -- 172.21.15.86:0/86022793 >> v2:172.21.15.196:3300/0 conn(0x7fd04c0ad330 msgr2=0x7fd04c15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd051800640 1 --2- 172.21.15.86:0/86022793 >> v2:172.21.15.196:3300/0 conn(0x7fd04c0ad330 0x7fd04c15d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd051800640 1 -- 172.21.15.86:0/86022793 >> v1:172.21.15.196:6789/0 conn(0x7fd04c06ef40 legacy=0x7fd04c13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd051800640 1 -- 172.21.15.86:0/86022793 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c06e460 con 0x7fd04c0ad710 2024-03-22T04:11:27.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd051800640 1 --2- 172.21.15.86:0/86022793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad710 0x7fd04c142690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd04c06d080 tx=0x7fd04401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed37de82292cd2b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:27.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd04be00640 1 -- 172.21.15.86:0/86022793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440a5880 con 0x7fd04c0ad710 2024-03-22T04:11:27.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd04be00640 1 -- 172.21.15.86:0/86022793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0440a5a20 con 0x7fd04c0ad710 2024-03-22T04:11:27.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd04be00640 1 -- 172.21.15.86:0/86022793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440a5df0 con 0x7fd04c0ad710 2024-03-22T04:11:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd053800640 1 -- 172.21.15.86:0/86022793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad710 msgr2=0x7fd04c142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd053800640 1 --2- 172.21.15.86:0/86022793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad710 0x7fd04c142690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd04c06d080 tx=0x7fd04401c9c0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd053800640 1 -- 172.21.15.86:0/86022793 shutdown_connections 2024-03-22T04:11:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd053800640 1 --2- 172.21.15.86:0/86022793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad710 0x7fd04c142690 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd053800640 1 --2- 172.21.15.86:0/86022793 >> v2:172.21.15.196:3300/0 conn(0x7fd04c0ad330 0x7fd04c15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd053800640 1 -- 172.21.15.86:0/86022793 >> 172.21.15.86:0/86022793 conn(0x7fd04c0b1df0 msgr2=0x7fd04c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:27.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.839+0000 7fd053800640 1 -- 172.21.15.86:0/86022793 shutdown_connections 2024-03-22T04:11:27.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 -- 172.21.15.86:0/86022793 wait complete. 2024-03-22T04:11:27.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 Processor -- start 2024-03-22T04:11:27.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 -- start start 2024-03-22T04:11:27.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c06ef40 0x7fd04c1419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad330 0x7fd04c13c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd04c0ad710 0x7fd04c154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1690e0 con 0x7fd04c0ad330 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c168f60 con 0x7fd04c06ef40 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd053800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c169260 con 0x7fd04c0ad710 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd043e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad330 0x7fd04c13c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c06ef40 0x7fd04c1419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c06ef40 0x7fd04c1419d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:42048/0 (socket says 172.21.15.86:42048) 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 -- 172.21.15.86:0/2399281916 learned_addr learned my addr 172.21.15.86:0/2399281916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd051800640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd04c0ad710 0x7fd04c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 -- 172.21.15.86:0/2399281916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd04c0ad710 msgr2=0x7fd04c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd04c0ad710 0x7fd04c154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 -- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad330 msgr2=0x7fd04c13c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:27.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad330 0x7fd04c13c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:27.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 -- 172.21.15.86:0/2399281916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c1545a0 con 0x7fd04c06ef40 2024-03-22T04:11:27.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd050e00640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c06ef40 0x7fd04c1419d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e930 tx=0x7fd03c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:27.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.843+0000 7fd049600640 1 -- 172.21.15.86:0/2399281916 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c017070 con 0x7fd04c06ef40 2024-03-22T04:11:27.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd049600640 1 -- 172.21.15.86:0/2399281916 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03c00ca00 con 0x7fd04c06ef40 2024-03-22T04:11:27.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd049600640 1 -- 172.21.15.86:0/2399281916 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00cdb0 con 0x7fd04c06ef40 2024-03-22T04:11:27.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c071a00 con 0x7fd04c06ef40 2024-03-22T04:11:27.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c071fc0 con 0x7fd04c06ef40 2024-03-22T04:11:27.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c068650 con 0x7fd04c06ef40 2024-03-22T04:11:27.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd049600640 1 -- 172.21.15.86:0/2399281916 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd03c01a070 con 0x7fd04c06ef40 2024-03-22T04:11:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd049600640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd028067b20 0x7fd028069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd049600640 1 -- 172.21.15.86:0/2399281916 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fd03c086ab0 con 0x7fd04c06ef40 2024-03-22T04:11:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd043e00640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd028067b20 0x7fd028069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.847+0000 7fd043e00640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd028067b20 0x7fd028069fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd04c13fb80 tx=0x7fd034009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:27.859+0000 7fd049600640 1 -- 172.21.15.86:0/2399281916 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd03c056b60 con 0x7fd04c06ef40 2024-03-22T04:11:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.015+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd04c156020 con 0x7fd04c06ef40 2024-03-22T04:11:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd049600640 1 -- 172.21.15.86:0/2399281916 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7fd03c05ac00 con 0x7fd04c06ef40 2024-03-22T04:11:28.022 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:28.023 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd028067b20 msgr2=0x7fd028069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd028067b20 0x7fd028069fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd04c13fb80 tx=0x7fd034009040 comp rx=0 tx=0).stop 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c06ef40 msgr2=0x7fd04c1419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c06ef40 0x7fd04c1419d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e930 tx=0x7fd03c00edf0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 shutdown_connections 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd028067b20 0x7fd028069fe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd04c0ad710 0x7fd04c154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c0ad330 0x7fd04c13c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 --2- 172.21.15.86:0/2399281916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c06ef40 0x7fd04c1419d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 >> 172.21.15.86:0/2399281916 conn(0x7fd04c0b1df0 msgr2=0x7fd04c154ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 shutdown_connections 2024-03-22T04:11:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.019+0000 7fd053800640 1 -- 172.21.15.86:0/2399281916 wait complete. 2024-03-22T04:11:28.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:28.043 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52bc600640 1 Processor -- start 2024-03-22T04:11:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52bc600640 1 -- start start 2024-03-22T04:11:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52bc600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52b40b2360 0x7f52b40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52bc600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f52b40b3430 0x7f52b415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52bc600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b415d710 con 0x7f52b415cee0 2024-03-22T04:11:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52bc600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52b40b2740 con 0x7f52b40b2360 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52bc600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f52b415fe40 con 0x7f52b40b3430 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52b9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52b40b2360 0x7f52b40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.179+0000 7f52ba600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f52b415cee0 0x7f52b406d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52630/0 (socket says 172.21.15.86:52630) 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52b40b2360 0x7f52b40b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57280/0 (socket says 172.21.15.86:57280) 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52ba600640 1 -- 172.21.15.86:0/2828800384 learned_addr learned my addr 172.21.15.86:0/2828800384 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b9200640 1 --2- 172.21.15.86:0/2828800384 >> v2:172.21.15.196:3300/0 conn(0x7f52b40b3430 0x7f52b415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b9c00640 1 -- 172.21.15.86:0/2828800384 >> v2:172.21.15.196:3300/0 conn(0x7f52b40b3430 msgr2=0x7f52b415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b9c00640 1 --2- 172.21.15.86:0/2828800384 >> v2:172.21.15.196:3300/0 conn(0x7f52b40b3430 0x7f52b415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b9c00640 1 -- 172.21.15.86:0/2828800384 >> v1:172.21.15.86:6789/0 conn(0x7f52b415cee0 legacy=0x7f52b406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b9c00640 1 -- 172.21.15.86:0/2828800384 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b4160120 con 0x7f52b40b2360 2024-03-22T04:11:28.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b9c00640 1 --2- 172.21.15.86:0/2828800384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b40b2360 0x7f52b40b2ef0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f52b40afb10 tx=0x7f52a801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8afc749d5ce8c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b3e00640 1 -- 172.21.15.86:0/2828800384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80b2070 con 0x7f52b40b2360 2024-03-22T04:11:28.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b3e00640 1 -- 172.21.15.86:0/2828800384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a80a5800 con 0x7f52b40b2360 2024-03-22T04:11:28.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.183+0000 7f52b3e00640 1 -- 172.21.15.86:0/2828800384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80a5bd0 con 0x7f52b40b2360 2024-03-22T04:11:28.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- 172.21.15.86:0/2828800384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b40b2360 msgr2=0x7f52b40b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 --2- 172.21.15.86:0/2828800384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b40b2360 0x7f52b40b2ef0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f52b40afb10 tx=0x7f52a801c930 comp rx=0 tx=0).stop 2024-03-22T04:11:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- 172.21.15.86:0/2828800384 shutdown_connections 2024-03-22T04:11:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 --2- 172.21.15.86:0/2828800384 >> v2:172.21.15.196:3300/0 conn(0x7f52b40b3430 0x7f52b415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 --2- 172.21.15.86:0/2828800384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b40b2360 0x7f52b40b2ef0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- 172.21.15.86:0/2828800384 >> 172.21.15.86:0/2828800384 conn(0x7f52b40cc940 msgr2=0x7f52b40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- 172.21.15.86:0/2828800384 shutdown_connections 2024-03-22T04:11:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- 172.21.15.86:0/2828800384 wait complete. 2024-03-22T04:11:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 Processor -- start 2024-03-22T04:11:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- start start 2024-03-22T04:11:28.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52b40b2360 0x7f52b414da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52b40b3430 0x7f52b414e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b415cee0 0x7f52b414ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4160c00 con 0x7f52b415cee0 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52b415d2c0 con 0x7f52b40b2360 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52bc600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f52b4154810 con 0x7f52b40b3430 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52ba600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b415cee0 0x7f52b414ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52b9c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52b40b2360 0x7f52b414da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52b9200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52b40b3430 0x7f52b414e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52ba600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b415cee0 0x7f52b414ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57290/0 (socket says 172.21.15.86:57290) 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52b9c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52b40b2360 0x7f52b414da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:42058/0 (socket says 172.21.15.86:42058) 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52ba600640 1 -- 172.21.15.86:0/2290383811 learned_addr learned my addr 172.21.15.86:0/2290383811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52ba600640 1 -- 172.21.15.86:0/2290383811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52b40b3430 msgr2=0x7f52b414e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52ba600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52b40b3430 0x7f52b414e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52ba600640 1 -- 172.21.15.86:0/2290383811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52b40b2360 msgr2=0x7f52b414da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52ba600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52b40b2360 0x7f52b414da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52ba600640 1 -- 172.21.15.86:0/2290383811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b415ba60 con 0x7f52b415cee0 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.187+0000 7f52b9c00640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52b40b2360 0x7f52b414da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52ba600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b415cee0 0x7f52b414ed10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00c400 tx=0x7f52ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52b1600640 1 -- 172.21.15.86:0/2290383811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac017070 con 0x7f52b415cee0 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52b1600640 1 -- 172.21.15.86:0/2290383811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ac005cf0 con 0x7f52b415cee0 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b415bd40 con 0x7f52b415cee0 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52b1600640 1 -- 172.21.15.86:0/2290383811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0072d0 con 0x7f52b415cee0 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b414df40 con 0x7f52b415cee0 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b415c1f0 con 0x7f52b415cee0 2024-03-22T04:11:28.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52b1600640 1 -- 172.21.15.86:0/2290383811 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f52ac01a070 con 0x7f52b415cee0 2024-03-22T04:11:28.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52b1600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5294067b50 0x7f529406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52b1600640 1 -- 172.21.15.86:0/2290383811 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f52ac0855d0 con 0x7f52b415cee0 2024-03-22T04:11:28.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52b9c00640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5294067b50 0x7f529406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.191+0000 7f52b9c00640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5294067b50 0x7f529406a010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f52a80af000 tx=0x7f52a80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.195+0000 7f52b1600640 1 -- 172.21.15.86:0/2290383811 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f52ac055540 con 0x7f52b415cee0 2024-03-22T04:11:28.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.351+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f52b4074470 con 0x7f52b415cee0 2024-03-22T04:11:28.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.351+0000 7f52b1600640 1 -- 172.21.15.86:0/2290383811 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f52ac0595e0 con 0x7f52b415cee0 2024-03-22T04:11:28.358 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5294067b50 msgr2=0x7f529406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5294067b50 0x7f529406a010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f52a80af000 tx=0x7f52a80a7070 comp rx=0 tx=0).stop 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b415cee0 msgr2=0x7f52b414ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b415cee0 0x7f52b414ed10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00c400 tx=0x7f52ac00c8c0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 shutdown_connections 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5294067b50 0x7f529406a010 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52b415cee0 0x7f52b414ed10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52b40b3430 0x7f52b414e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 --2- 172.21.15.86:0/2290383811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52b40b2360 0x7f52b414da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 >> 172.21.15.86:0/2290383811 conn(0x7f52b40cc940 msgr2=0x7f52b40b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 shutdown_connections 2024-03-22T04:11:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.355+0000 7f52bc600640 1 -- 172.21.15.86:0/2290383811 wait complete. 2024-03-22T04:11:28.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.511+0000 7f07c7a00640 1 Processor -- start 2024-03-22T04:11:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.511+0000 7f07c7a00640 1 -- start start 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.511+0000 7f07c7a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f07c00c6f20 0x7f07c0072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.511+0000 7f07c7a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00cb1d0 0x7f07c0162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.511+0000 7f07c7a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07c00c7300 0x7f07c0162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.511+0000 7f07c7a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f07c00cc7c0 con 0x7f07c00c7300 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.511+0000 7f07c7a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f07c00cb720 con 0x7f07c00c6f20 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.511+0000 7f07c7a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f07c00cba00 con 0x7f07c00cb1d0 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07c00c7300 0x7f07c0162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07c00c7300 0x7f07c0162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57302/0 (socket says 172.21.15.86:57302) 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 -- 172.21.15.86:0/421607270 learned_addr learned my addr 172.21.15.86:0/421607270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07b7e00640 1 --2- 172.21.15.86:0/421607270 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00cb1d0 0x7f07c0162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5000640 1 --2- 172.21.15.86:0/421607270 >> v2:172.21.15.196:3300/0 conn(0x7f07c00c6f20 0x7f07c0072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 -- 172.21.15.86:0/421607270 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00cb1d0 msgr2=0x7f07c0162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 --2- 172.21.15.86:0/421607270 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00cb1d0 0x7f07c0162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 -- 172.21.15.86:0/421607270 >> v2:172.21.15.196:3300/0 conn(0x7f07c00c6f20 msgr2=0x7f07c0072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 --2- 172.21.15.86:0/421607270 >> v2:172.21.15.196:3300/0 conn(0x7f07c00c6f20 0x7f07c0072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 -- 172.21.15.86:0/421607270 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c00cbce0 con 0x7f07c00c7300 2024-03-22T04:11:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5000640 1 --2- 172.21.15.86:0/421607270 >> v2:172.21.15.196:3300/0 conn(0x7f07c00c6f20 0x7f07c0072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07c5a00640 1 --2- 172.21.15.86:0/421607270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c7300 0x7f07c0162570 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f07c006d5b0 tx=0x7f07b801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd260219c0130aca server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07bfe00640 1 -- 172.21.15.86:0/421607270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80a5840 con 0x7f07c00c7300 2024-03-22T04:11:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07bfe00640 1 -- 172.21.15.86:0/421607270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b80a59e0 con 0x7f07c00c7300 2024-03-22T04:11:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.515+0000 7f07bfe00640 1 -- 172.21.15.86:0/421607270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80a5db0 con 0x7f07c00c7300 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- 172.21.15.86:0/421607270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c7300 msgr2=0x7f07c0162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 --2- 172.21.15.86:0/421607270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c7300 0x7f07c0162570 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f07c006d5b0 tx=0x7f07b801c9c0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- 172.21.15.86:0/421607270 shutdown_connections 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 --2- 172.21.15.86:0/421607270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c7300 0x7f07c0162570 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 --2- 172.21.15.86:0/421607270 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00cb1d0 0x7f07c0162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 --2- 172.21.15.86:0/421607270 >> v2:172.21.15.196:3300/0 conn(0x7f07c00c6f20 0x7f07c0072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- 172.21.15.86:0/421607270 >> 172.21.15.86:0/421607270 conn(0x7f07c00d0730 msgr2=0x7f07c00d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- 172.21.15.86:0/421607270 shutdown_connections 2024-03-22T04:11:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- 172.21.15.86:0/421607270 wait complete. 2024-03-22T04:11:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 Processor -- start 2024-03-22T04:11:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- start start 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c6f20 0x7f07c014b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00c7300 0x7f07c01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c00cb1d0 0x7f07c0155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07c00ceea0 con 0x7f07c00c6f20 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07c00ced20 con 0x7f07c00cb1d0 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c7a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f07c00cf020 con 0x7f07c00c7300 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c6f20 0x7f07c014b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c6f20 0x7f07c014b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57318/0 (socket says 172.21.15.86:57318) 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5000640 1 -- 172.21.15.86:0/2765251276 learned_addr learned my addr 172.21.15.86:0/2765251276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07b7e00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00c7300 0x7f07c01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5a00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c00cb1d0 0x7f07c0155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5000640 1 -- 172.21.15.86:0/2765251276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00c7300 msgr2=0x7f07c01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5000640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00c7300 0x7f07c01552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5000640 1 -- 172.21.15.86:0/2765251276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c00cb1d0 msgr2=0x7f07c0155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5000640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c00cb1d0 0x7f07c0155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.519+0000 7f07c5000640 1 -- 172.21.15.86:0/2765251276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c00c9de0 con 0x7f07c00c6f20 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07c5000640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c6f20 0x7f07c014b860 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e810 tx=0x7f07b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07bd600640 1 -- 172.21.15.86:0/2765251276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0017070 con 0x7f07c00c6f20 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07bd600640 1 -- 172.21.15.86:0/2765251276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b000c8d0 con 0x7f07c00c6f20 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07bd600640 1 -- 172.21.15.86:0/2765251276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b000f040 con 0x7f07c00c6f20 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c00c9540 con 0x7f07c00c6f20 2024-03-22T04:11:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c00c9b00 con 0x7f07c00c6f20 2024-03-22T04:11:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c0074470 con 0x7f07c00c6f20 2024-03-22T04:11:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07bd600640 1 -- 172.21.15.86:0/2765251276 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f07b001a070 con 0x7f07c00c6f20 2024-03-22T04:11:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07bd600640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f079c067b20 0x7f079c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07bd600640 1 -- 172.21.15.86:0/2765251276 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f07b0085d40 con 0x7f07c00c6f20 2024-03-22T04:11:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.523+0000 7f07b7e00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f079c067b20 0x7f079c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.527+0000 7f07b7e00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f079c067b20 0x7f079c069fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07c01fa650 tx=0x7f07a8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.527+0000 7f07bd600640 1 -- 172.21.15.86:0/2765251276 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f07b0055dc0 con 0x7f07c00c6f20 2024-03-22T04:11:28.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.679+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f07c00c5ce0 con 0x7f07c00c6f20 2024-03-22T04:11:28.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07bd600640 1 -- 172.21.15.86:0/2765251276 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f07b0059e60 con 0x7f07c00c6f20 2024-03-22T04:11:28.687 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f079c067b20 msgr2=0x7f079c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f079c067b20 0x7f079c069fe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07c01fa650 tx=0x7f07a8009190 comp rx=0 tx=0).stop 2024-03-22T04:11:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c6f20 msgr2=0x7f07c014b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c6f20 0x7f07c014b860 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e810 tx=0x7f07b000ecd0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 shutdown_connections 2024-03-22T04:11:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f079c067b20 0x7f079c069fe0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c00cb1d0 0x7f07c0155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c00c7300 0x7f07c01552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 --2- 172.21.15.86:0/2765251276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00c6f20 0x7f07c014b860 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 >> 172.21.15.86:0/2765251276 conn(0x7f07c00d0730 msgr2=0x7f07c00b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 shutdown_connections 2024-03-22T04:11:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.683+0000 7f07c7a00640 1 -- 172.21.15.86:0/2765251276 wait complete. 2024-03-22T04:11:28.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.839+0000 7fce56c00640 1 Processor -- start 2024-03-22T04:11:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.839+0000 7fce56c00640 1 -- start start 2024-03-22T04:11:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.839+0000 7fce56c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fce500b2360 0x7fce500b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.839+0000 7fce56c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce500b3430 0x7fce5015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.839+0000 7fce56c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce5015d710 con 0x7fce5015cee0 2024-03-22T04:11:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.839+0000 7fce56c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fce500b2740 con 0x7fce500b3430 2024-03-22T04:11:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.839+0000 7fce56c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fce5015fe40 con 0x7fce500b2360 2024-03-22T04:11:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce500b3430 0x7fce5015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce54c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fce5015cee0 0x7fce5006d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52646/0 (socket says 172.21.15.86:52646) 2024-03-22T04:11:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce54c00640 1 -- 172.21.15.86:0/4105356016 learned_addr learned my addr 172.21.15.86:0/4105356016 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4fe00640 1 --2- 172.21.15.86:0/4105356016 >> v2:172.21.15.196:3300/0 conn(0x7fce500b2360 0x7fce500b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4ea00640 1 -- 172.21.15.86:0/4105356016 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491731570 0 0) 0x7fce5015d710 con 0x7fce5015cee0 2024-03-22T04:11:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4ea00640 1 -- 172.21.15.86:0/4105356016 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce34003530 con 0x7fce5015cee0 2024-03-22T04:11:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4f400640 1 -- 172.21.15.86:0/4105356016 >> v2:172.21.15.196:3300/0 conn(0x7fce500b2360 msgr2=0x7fce500b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4f400640 1 --2- 172.21.15.86:0/4105356016 >> v2:172.21.15.196:3300/0 conn(0x7fce500b2360 0x7fce500b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4f400640 1 -- 172.21.15.86:0/4105356016 >> v1:172.21.15.86:6789/0 conn(0x7fce5015cee0 legacy=0x7fce5006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4f400640 1 -- 172.21.15.86:0/4105356016 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce50160120 con 0x7fce500b3430 2024-03-22T04:11:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4fe00640 1 --2- 172.21.15.86:0/4105356016 >> v2:172.21.15.196:3300/0 conn(0x7fce500b2360 0x7fce500b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:28.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4f400640 1 --2- 172.21.15.86:0/4105356016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b3430 0x7fce5015c9a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fce500683e0 tx=0x7fce3c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660f34b0c85d3d73 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4ea00640 1 -- 172.21.15.86:0/4105356016 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0b0070 con 0x7fce500b3430 2024-03-22T04:11:28.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4ea00640 1 -- 172.21.15.86:0/4105356016 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3c0ab600 con 0x7fce500b3430 2024-03-22T04:11:28.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.843+0000 7fce4ea00640 1 -- 172.21.15.86:0/4105356016 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0ab9d0 con 0x7fce500b3430 2024-03-22T04:11:28.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- 172.21.15.86:0/4105356016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b3430 msgr2=0x7fce5015c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 --2- 172.21.15.86:0/4105356016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b3430 0x7fce5015c9a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fce500683e0 tx=0x7fce3c01c480 comp rx=0 tx=0).stop 2024-03-22T04:11:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- 172.21.15.86:0/4105356016 shutdown_connections 2024-03-22T04:11:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 --2- 172.21.15.86:0/4105356016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b3430 0x7fce5015c9a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 --2- 172.21.15.86:0/4105356016 >> v2:172.21.15.196:3300/0 conn(0x7fce500b2360 0x7fce500b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- 172.21.15.86:0/4105356016 >> 172.21.15.86:0/4105356016 conn(0x7fce500cc940 msgr2=0x7fce500ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- 172.21.15.86:0/4105356016 shutdown_connections 2024-03-22T04:11:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- 172.21.15.86:0/4105356016 wait complete. 2024-03-22T04:11:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 Processor -- start 2024-03-22T04:11:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- start start 2024-03-22T04:11:28.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b2360 0x7fce5014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce500b3430 0x7fce5013e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce5015cee0 0x7fce50072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce50160c00 con 0x7fce500b2360 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce5015d2c0 con 0x7fce500b3430 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce56c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce5014faf0 con 0x7fce5015cee0 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b2360 0x7fce5014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b2360 0x7fce5014f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57336/0 (socket says 172.21.15.86:57336) 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4fe00640 1 -- 172.21.15.86:0/1945106360 learned_addr learned my addr 172.21.15.86:0/1945106360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4f400640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce500b3430 0x7fce5013e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce54c00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce5015cee0 0x7fce50072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4fe00640 1 -- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce5015cee0 msgr2=0x7fce50072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4fe00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce5015cee0 0x7fce50072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4fe00640 1 -- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce500b3430 msgr2=0x7fce5013e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4fe00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce500b3430 0x7fce5013e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce4fe00640 1 -- 172.21.15.86:0/1945106360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce500b3cb0 con 0x7fce500b2360 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.847+0000 7fce54c00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce5015cee0 0x7fce50072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce4fe00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b2360 0x7fce5014f5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fce4000efc0 tx=0x7fce4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce47e00640 1 -- 172.21.15.86:0/1945106360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40018020 con 0x7fce500b2360 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce47e00640 1 -- 172.21.15.86:0/1945106360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce40014680 con 0x7fce500b2360 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce47e00640 1 -- 172.21.15.86:0/1945106360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40014a50 con 0x7fce500b2360 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce500cbf40 con 0x7fce500b2360 2024-03-22T04:11:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce501500a0 con 0x7fce500b2360 2024-03-22T04:11:28.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce50074470 con 0x7fce500b2360 2024-03-22T04:11:28.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce47e00640 1 -- 172.21.15.86:0/1945106360 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fce4001b020 con 0x7fce500b2360 2024-03-22T04:11:28.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce47e00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce30067b20 0x7fce30069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:28.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.851+0000 7fce47e00640 1 -- 172.21.15.86:0/1945106360 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fce40087030 con 0x7fce500b2360 2024-03-22T04:11:28.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.855+0000 7fce4f400640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce30067b20 0x7fce30069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:28.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.855+0000 7fce4f400640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce30067b20 0x7fce30069fe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0ad000 tx=0x7fce3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:28.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:28.859+0000 7fce47e00640 1 -- 172.21.15.86:0/1945106360 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fce40022030 con 0x7fce500b2360 2024-03-22T04:11:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.011+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce5006e9f0 con 0x7fce500b2360 2024-03-22T04:11:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.011+0000 7fce47e00640 1 -- 172.21.15.86:0/1945106360 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fce40010040 con 0x7fce500b2360 2024-03-22T04:11:29.018 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce30067b20 msgr2=0x7fce30069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce30067b20 0x7fce30069fe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0ad000 tx=0x7fce3c008040 comp rx=0 tx=0).stop 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b2360 msgr2=0x7fce5014f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b2360 0x7fce5014f5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fce4000efc0 tx=0x7fce4000c490 comp rx=0 tx=0).stop 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 shutdown_connections 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce30067b20 0x7fce30069fe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce500b3430 0x7fce5013e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce5015cee0 0x7fce50072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 --2- 172.21.15.86:0/1945106360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce500b2360 0x7fce5014f5b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 >> 172.21.15.86:0/1945106360 conn(0x7fce500cc940 msgr2=0x7fce500716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 shutdown_connections 2024-03-22T04:11:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.015+0000 7fce56c00640 1 -- 172.21.15.86:0/1945106360 wait complete. 2024-03-22T04:11:29.039 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:29.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.171+0000 7f9993800640 1 Processor -- start 2024-03-22T04:11:29.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.171+0000 7f9993800640 1 -- start start 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.171+0000 7f9993800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f998c0b3600 0x7f998c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.171+0000 7f9993800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c15cee0 0x7f998c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.171+0000 7f9993800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f998c15d710 con 0x7f998c0b2360 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.171+0000 7f9993800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f998c0b2740 con 0x7f998c0b3600 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.171+0000 7f9993800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f998c15fe40 con 0x7f998c15cee0 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f9990e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f998c0b2360 0x7f998c0b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52662/0 (socket says 172.21.15.86:52662) 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f9990e00640 1 -- 172.21.15.86:0/4006065267 learned_addr learned my addr 172.21.15.86:0/4006065267 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f9991800640 1 --2- 172.21.15.86:0/4006065267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c15cee0 0x7f998c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f998be00640 1 --2- 172.21.15.86:0/4006065267 >> v2:172.21.15.196:3300/0 conn(0x7f998c0b3600 0x7f998c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f998b400640 1 -- 172.21.15.86:0/4006065267 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1757557496 0 0) 0x7f998c15d710 con 0x7f998c0b2360 2024-03-22T04:11:29.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f998b400640 1 -- 172.21.15.86:0/4006065267 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9974003530 con 0x7f998c0b2360 2024-03-22T04:11:29.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f9991800640 1 -- 172.21.15.86:0/4006065267 >> v2:172.21.15.196:3300/0 conn(0x7f998c0b3600 msgr2=0x7f998c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:29.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f9991800640 1 --2- 172.21.15.86:0/4006065267 >> v2:172.21.15.196:3300/0 conn(0x7f998c0b3600 0x7f998c15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f9991800640 1 -- 172.21.15.86:0/4006065267 >> v1:172.21.15.86:6789/0 conn(0x7f998c0b2360 legacy=0x7f998c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f9991800640 1 -- 172.21.15.86:0/4006065267 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c160120 con 0x7f998c15cee0 2024-03-22T04:11:29.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f998be00640 1 --2- 172.21.15.86:0/4006065267 >> v2:172.21.15.196:3300/0 conn(0x7f998c0b3600 0x7f998c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:29.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f9991800640 1 --2- 172.21.15.86:0/4006065267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c15cee0 0x7f998c0b4d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9984009d90 tx=0x7f998401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79bffbf66fabbfa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:29.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.175+0000 7f998b400640 1 -- 172.21.15.86:0/4006065267 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840b2070 con 0x7f998c15cee0 2024-03-22T04:11:29.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f998b400640 1 -- 172.21.15.86:0/4006065267 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99840a5a00 con 0x7f998c15cee0 2024-03-22T04:11:29.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f998b400640 1 -- 172.21.15.86:0/4006065267 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840a5dd0 con 0x7f998c15cee0 2024-03-22T04:11:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- 172.21.15.86:0/4006065267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c15cee0 msgr2=0x7f998c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 --2- 172.21.15.86:0/4006065267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c15cee0 0x7f998c0b4d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9984009d90 tx=0x7f998401c940 comp rx=0 tx=0).stop 2024-03-22T04:11:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- 172.21.15.86:0/4006065267 shutdown_connections 2024-03-22T04:11:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 --2- 172.21.15.86:0/4006065267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c15cee0 0x7f998c0b4d80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 --2- 172.21.15.86:0/4006065267 >> v2:172.21.15.196:3300/0 conn(0x7f998c0b3600 0x7f998c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- 172.21.15.86:0/4006065267 >> 172.21.15.86:0/4006065267 conn(0x7f998c0cc940 msgr2=0x7f998c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- 172.21.15.86:0/4006065267 shutdown_connections 2024-03-22T04:11:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- 172.21.15.86:0/4006065267 wait complete. 2024-03-22T04:11:29.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 Processor -- start 2024-03-22T04:11:29.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- start start 2024-03-22T04:11:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f998c0b2360 0x7f998c15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c0b3600 0x7f998c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f998c15cee0 0x7f998c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f998c15d4d0 con 0x7f998c0b2360 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f998c15d350 con 0x7f998c15cee0 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9993800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f998c0b5ad0 con 0x7f998c0b3600 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9990e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f998c0b2360 0x7f998c15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9990e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f998c0b2360 0x7f998c15e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57344/0 (socket says 172.21.15.86:57344) 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9990e00640 1 -- 172.21.15.86:0/1061304186 learned_addr learned my addr 172.21.15.86:0/1061304186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9991800640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f998c15cee0 0x7f998c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f998be00640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c0b3600 0x7f998c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9990e00640 1 -- 172.21.15.86:0/1061304186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c0b3600 msgr2=0x7f998c15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9990e00640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c0b3600 0x7f998c15b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9990e00640 1 -- 172.21.15.86:0/1061304186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f998c15cee0 msgr2=0x7f998c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9990e00640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f998c15cee0 0x7f998c15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.179+0000 7f9990e00640 1 -- 172.21.15.86:0/1061304186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c153f70 con 0x7f998c0b2360 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9990e00640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f998c0b2360 0x7f998c15e0d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f99800010f0 tx=0x7f998000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9988c00640 1 -- 172.21.15.86:0/1061304186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980017070 con 0x7f998c0b2360 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9988c00640 1 -- 172.21.15.86:0/1061304186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980005d10 con 0x7f998c0b2360 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9988c00640 1 -- 172.21.15.86:0/1061304186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980007310 con 0x7f998c0b2360 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c14e230 con 0x7f998c0b2360 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c14e7f0 con 0x7f998c0b2360 2024-03-22T04:11:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998c074470 con 0x7f998c0b2360 2024-03-22T04:11:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9988c00640 1 -- 172.21.15.86:0/1061304186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f998001a070 con 0x7f998c0b2360 2024-03-22T04:11:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9988c00640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f996c067b20 0x7f996c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f9988c00640 1 -- 172.21.15.86:0/1061304186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f9980085600 con 0x7f998c0b2360 2024-03-22T04:11:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.183+0000 7f998be00640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f996c067b20 0x7f996c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.187+0000 7f998be00640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f996c067b20 0x7f996c069fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9978009580 tx=0x7f9978009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:29.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.187+0000 7f9988c00640 1 -- 172.21.15.86:0/1061304186 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f99800556a0 con 0x7f998c0b2360 2024-03-22T04:11:29.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.339+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f998c14af70 con 0x7f998c0b2360 2024-03-22T04:11:29.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9988c00640 1 -- 172.21.15.86:0/1061304186 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f9980059740 con 0x7f998c0b2360 2024-03-22T04:11:29.347 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f996c067b20 msgr2=0x7f996c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f996c067b20 0x7f996c069fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9978009580 tx=0x7f9978009190 comp rx=0 tx=0).stop 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f998c0b2360 msgr2=0x7f998c15e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f998c0b2360 0x7f998c15e0d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f99800010f0 tx=0x7f998000c8f0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 shutdown_connections 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f996c067b20 0x7f996c069fe0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f998c15cee0 0x7f998c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f998c0b3600 0x7f998c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 --2- 172.21.15.86:0/1061304186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f998c0b2360 0x7f998c15e0d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 >> 172.21.15.86:0/1061304186 conn(0x7f998c0cc940 msgr2=0x7f998c0cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 shutdown_connections 2024-03-22T04:11:29.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.343+0000 7f9993800640 1 -- 172.21.15.86:0/1061304186 wait complete. 2024-03-22T04:11:29.368 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:11:29.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f4100c00640 1 Processor -- start 2024-03-22T04:11:29.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f4100c00640 1 -- start start 2024-03-22T04:11:29.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f4100c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f40fc147880 0x7f40fc06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f4100c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f4100c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f40fc0b4890 0x7f40fc162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f4100c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f40fc0b6320 con 0x7f40fc147880 2024-03-22T04:11:29.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f4100c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f40fc15ea90 con 0x7f40fc0b4890 2024-03-22T04:11:29.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f4100c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f40fc06d4f0 con 0x7f40fc06cec0 2024-03-22T04:11:29.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f40fc147880 0x7f40fc06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f40fc147880 0x7f40fc06c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57358/0 (socket says 172.21.15.86:57358) 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f9e00640 1 -- 172.21.15.86:0/1511675337 learned_addr learned my addr 172.21.15.86:0/1511675337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f1400640 1 --2- 172.21.15.86:0/1511675337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40fa800640 1 --2- 172.21.15.86:0/1511675337 >> v2:172.21.15.196:3300/0 conn(0x7f40fc0b4890 0x7f40fc162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f9e00640 1 -- 172.21.15.86:0/1511675337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 msgr2=0x7f40fc0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f9e00640 1 --2- 172.21.15.86:0/1511675337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc0b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f9e00640 1 -- 172.21.15.86:0/1511675337 >> v2:172.21.15.196:3300/0 conn(0x7f40fc0b4890 msgr2=0x7f40fc162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f9e00640 1 --2- 172.21.15.86:0/1511675337 >> v2:172.21.15.196:3300/0 conn(0x7f40fc0b4890 0x7f40fc162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f9e00640 1 -- 172.21.15.86:0/1511675337 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc06d7d0 con 0x7f40fc147880 2024-03-22T04:11:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.499+0000 7f40f1400640 1 --2- 172.21.15.86:0/1511675337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:29.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f40f9e00640 1 --2- 172.21.15.86:0/1511675337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc147880 0x7f40fc06c980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40fc163120 tx=0x7f40ec01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74a3d1778bb787e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:29.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f40f9400640 1 -- 172.21.15.86:0/1511675337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0b0070 con 0x7f40fc147880 2024-03-22T04:11:29.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f40f9400640 1 -- 172.21.15.86:0/1511675337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ec0ab600 con 0x7f40fc147880 2024-03-22T04:11:29.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f40f9400640 1 -- 172.21.15.86:0/1511675337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0ab9d0 con 0x7f40fc147880 2024-03-22T04:11:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 -- 172.21.15.86:0/1511675337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc147880 msgr2=0x7f40fc06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 --2- 172.21.15.86:0/1511675337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc147880 0x7f40fc06c980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40fc163120 tx=0x7f40ec01c260 comp rx=0 tx=0).stop 2024-03-22T04:11:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 -- 172.21.15.86:0/1511675337 shutdown_connections 2024-03-22T04:11:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 --2- 172.21.15.86:0/1511675337 >> v2:172.21.15.196:3300/0 conn(0x7f40fc0b4890 0x7f40fc162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 --2- 172.21.15.86:0/1511675337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 --2- 172.21.15.86:0/1511675337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc147880 0x7f40fc06c980 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 -- 172.21.15.86:0/1511675337 >> 172.21.15.86:0/1511675337 conn(0x7f40fc06ba60 msgr2=0x7f40fc06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 -- 172.21.15.86:0/1511675337 shutdown_connections 2024-03-22T04:11:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.503+0000 7f4100c00640 1 -- 172.21.15.86:0/1511675337 wait complete. 2024-03-22T04:11:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 Processor -- start 2024-03-22T04:11:29.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 -- start start 2024-03-22T04:11:29.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc13ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc0b4890 0x7f40fc13ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc147880 0x7f40fc13e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc15ef40 con 0x7f40fc0b4890 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc0b4ca0 con 0x7f40fc147880 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f40fc15f0c0 con 0x7f40fc06cec0 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc0b4890 0x7f40fc13ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc0b4890 0x7f40fc13ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57360/0 (socket says 172.21.15.86:57360) 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 -- 172.21.15.86:0/1060349532 learned_addr learned my addr 172.21.15.86:0/1060349532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f9e00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc13ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40fa800640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc147880 0x7f40fc13e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 -- 172.21.15.86:0/1060349532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 msgr2=0x7f40fc13ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc13ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 -- 172.21.15.86:0/1060349532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc147880 msgr2=0x7f40fc13e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc147880 0x7f40fc13e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 -- 172.21.15.86:0/1060349532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc13e850 con 0x7f40fc0b4890 2024-03-22T04:11:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f1400640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc0b4890 0x7f40fc13ea40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f40e400ec10 tx=0x7f40e400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f2a00640 1 -- 172.21.15.86:0/1060349532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4017070 con 0x7f40fc0b4890 2024-03-22T04:11:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f2a00640 1 -- 172.21.15.86:0/1060349532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e400cc30 con 0x7f40fc0b4890 2024-03-22T04:11:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc13f190 con 0x7f40fc0b4890 2024-03-22T04:11:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f40f2a00640 1 -- 172.21.15.86:0/1060349532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4013470 con 0x7f40fc0b4890 2024-03-22T04:11:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc13f750 con 0x7f40fc0b4890 2024-03-22T04:11:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.507+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40fc068650 con 0x7f40fc0b4890 2024-03-22T04:11:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.511+0000 7f40f2a00640 1 -- 172.21.15.86:0/1060349532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f40e401a070 con 0x7f40fc0b4890 2024-03-22T04:11:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.511+0000 7f40f2a00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f40d8067b20 0x7f40d8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.511+0000 7f40f2a00640 1 -- 172.21.15.86:0/1060349532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f40e4086240 con 0x7f40fc0b4890 2024-03-22T04:11:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.511+0000 7f40f9e00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f40d8067b20 0x7f40d8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:29.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.511+0000 7f40f9e00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f40d8067b20 0x7f40d8069fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0ad000 tx=0x7f40ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:29.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.515+0000 7f40f2a00640 1 -- 172.21.15.86:0/1060349532 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f40e40561b0 con 0x7f40fc0b4890 2024-03-22T04:11:29.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.683+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40fc072220 con 0x7f40fc0b4890 2024-03-22T04:11:29.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.683+0000 7f40f2a00640 1 -- 172.21.15.86:0/1060349532 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f40e405a250 con 0x7f40fc0b4890 2024-03-22T04:11:29.689 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:11:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f40d8067b20 msgr2=0x7f40d8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f40d8067b20 0x7f40d8069fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0ad000 tx=0x7f40ec008040 comp rx=0 tx=0).stop 2024-03-22T04:11:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc0b4890 msgr2=0x7f40fc13ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc0b4890 0x7f40fc13ea40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f40e400ec10 tx=0x7f40e400c520 comp rx=0 tx=0).stop 2024-03-22T04:11:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 shutdown_connections 2024-03-22T04:11:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f40d8067b20 0x7f40d8069fe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc147880 0x7f40fc13e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40fc0b4890 0x7f40fc13ea40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 --2- 172.21.15.86:0/1060349532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc06cec0 0x7f40fc13ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 >> 172.21.15.86:0/1060349532 conn(0x7f40fc06ba60 msgr2=0x7f40fc0b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 shutdown_connections 2024-03-22T04:11:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:29.687+0000 7f4100c00640 1 -- 172.21.15.86:0/1060349532 wait complete. 2024-03-22T04:11:29.710 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:11:29.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:29.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-22T04:11:29.748 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:11:29.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:29.749 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:11:29.803 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:11:29.803 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:29.812 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:11:29.812 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:29.865 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:11:29.865 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:29.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:30.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f52a1a00640 1 Processor -- start 2024-03-22T04:11:30.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f52a1a00640 1 -- start start 2024-03-22T04:11:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f52a1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 0x7f529c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f52a1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f529c162540 0x7f529c15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f52a1a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c0b2bd0 con 0x7f529c147e10 2024-03-22T04:11:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f52a1a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f529c0b2610 con 0x7f529c162540 2024-03-22T04:11:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f52a1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f529c0b28f0 con 0x7f529c06c270 2024-03-22T04:11:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f529b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f529c162540 0x7f529c15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f529b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f529c162540 0x7f529c15fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57364/0 (socket says 172.21.15.86:57364) 2024-03-22T04:11:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f529b400640 1 -- 172.21.15.86:0/2488371716 learned_addr learned my addr 172.21.15.86:0/2488371716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:30.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f529aa00640 1 --1- 172.21.15.86:0/2488371716 >> v1:172.21.15.86:6789/0 conn(0x7f529c147e10 0x7f529c0b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52676/0 (socket says 172.21.15.86:52676) 2024-03-22T04:11:30.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f5292000640 1 --2- 172.21.15.86:0/2488371716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 0x7f529c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f529b400640 1 -- 172.21.15.86:0/2488371716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 msgr2=0x7f529c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:30.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f529b400640 1 --2- 172.21.15.86:0/2488371716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 0x7f529c162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f529b400640 1 -- 172.21.15.86:0/2488371716 >> v1:172.21.15.86:6789/0 conn(0x7f529c147e10 legacy=0x7f529c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.047+0000 7f529b400640 1 -- 172.21.15.86:0/2488371716 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c0b5ce0 con 0x7f529c162540 2024-03-22T04:11:30.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f529b400640 1 --2- 172.21.15.86:0/2488371716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c162540 0x7f529c15fb20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5294009e50 tx=0x7f529401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfbbde77cf3c26b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f529a000640 1 -- 172.21.15.86:0/2488371716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52940a5810 con 0x7f529c162540 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f529a000640 1 -- 172.21.15.86:0/2488371716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52940a59b0 con 0x7f529c162540 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f529a000640 1 -- 172.21.15.86:0/2488371716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52940a5d80 con 0x7f529c162540 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f52a1a00640 1 -- 172.21.15.86:0/2488371716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c162540 msgr2=0x7f529c15fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f52a1a00640 1 --2- 172.21.15.86:0/2488371716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c162540 0x7f529c15fb20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5294009e50 tx=0x7f529401ca30 comp rx=0 tx=0).stop 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f52a1a00640 1 -- 172.21.15.86:0/2488371716 shutdown_connections 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f52a1a00640 1 --2- 172.21.15.86:0/2488371716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c162540 0x7f529c15fb20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f52a1a00640 1 --2- 172.21.15.86:0/2488371716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 0x7f529c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f52a1a00640 1 -- 172.21.15.86:0/2488371716 >> 172.21.15.86:0/2488371716 conn(0x7f529c0b6620 msgr2=0x7f529c0b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f52a1a00640 1 -- 172.21.15.86:0/2488371716 shutdown_connections 2024-03-22T04:11:30.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.051+0000 7f52a1a00640 1 -- 172.21.15.86:0/2488371716 wait complete. 2024-03-22T04:11:30.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 Processor -- start 2024-03-22T04:11:30.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 -- start start 2024-03-22T04:11:30.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 0x7f529c15c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c147e10 0x7f529c13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f529c162540 0x7f529c13c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f529c160c00 con 0x7f529c147e10 2024-03-22T04:11:30.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f529c160a80 con 0x7f529c162540 2024-03-22T04:11:30.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f529c160d80 con 0x7f529c06c270 2024-03-22T04:11:30.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c147e10 0x7f529c13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f529aa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 0x7f529c15c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c147e10 0x7f529c13bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57378/0 (socket says 172.21.15.86:57378) 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 -- 172.21.15.86:0/1710235292 learned_addr learned my addr 172.21.15.86:0/1710235292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f529b400640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f529c162540 0x7f529c13c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 -- 172.21.15.86:0/1710235292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 msgr2=0x7f529c15c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 0x7f529c15c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 -- 172.21.15.86:0/1710235292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f529c162540 msgr2=0x7f529c13c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f529c162540 0x7f529c13c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 -- 172.21.15.86:0/1710235292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c15ad50 con 0x7f529c147e10 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5292000640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c147e10 0x7f529c13bf80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f528400d5f0 tx=0x7f528400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5293400640 1 -- 172.21.15.86:0/1710235292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52840125f0 con 0x7f529c147e10 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5293400640 1 -- 172.21.15.86:0/1710235292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5284019070 con 0x7f529c147e10 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529c15b030 con 0x7f529c147e10 2024-03-22T04:11:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f5293400640 1 -- 172.21.15.86:0/1710235292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52840129b0 con 0x7f529c147e10 2024-03-22T04:11:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529c15ce60 con 0x7f529c147e10 2024-03-22T04:11:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.055+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529c068650 con 0x7f529c147e10 2024-03-22T04:11:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.059+0000 7f5293400640 1 -- 172.21.15.86:0/1710235292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5284012b50 con 0x7f529c147e10 2024-03-22T04:11:30.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.059+0000 7f5293400640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5278067b20 0x7f5278069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.059+0000 7f5293400640 1 -- 172.21.15.86:0/1710235292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f52840865d0 con 0x7f529c147e10 2024-03-22T04:11:30.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.059+0000 7f529aa00640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5278067b20 0x7f5278069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.059+0000 7f529aa00640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5278067b20 0x7f5278069fe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f528c000f70 tx=0x7f528c007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.063+0000 7f5293400640 1 -- 172.21.15.86:0/1710235292 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f52840565d0 con 0x7f529c147e10 2024-03-22T04:11:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.215+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f529c15b550 con 0x7f529c147e10 2024-03-22T04:11:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f5293400640 1 -- 172.21.15.86:0/1710235292 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7f528405a670 con 0x7f529c147e10 2024-03-22T04:11:30.224 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:30.224 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5278067b20 msgr2=0x7f5278069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5278067b20 0x7f5278069fe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f528c000f70 tx=0x7f528c007e90 comp rx=0 tx=0).stop 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c147e10 msgr2=0x7f529c13bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c147e10 0x7f529c13bf80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f528400d5f0 tx=0x7f528400dab0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 shutdown_connections 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5278067b20 0x7f5278069fe0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f529c162540 0x7f529c13c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c147e10 0x7f529c13bf80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 --2- 172.21.15.86:0/1710235292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c06c270 0x7f529c15c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 >> 172.21.15.86:0/1710235292 conn(0x7f529c0b6620 msgr2=0x7f529c1688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 shutdown_connections 2024-03-22T04:11:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.219+0000 7f52a1a00640 1 -- 172.21.15.86:0/1710235292 wait complete. 2024-03-22T04:11:30.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:30.245 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1207600640 1 Processor -- start 2024-03-22T04:11:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1207600640 1 -- start start 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1207600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f12000b4eb0 0x7f1200162740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1207600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f120006b660 0x7f120006baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1207600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12000b5470 con 0x7f120015e5b0 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1207600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f12001622b0 con 0x7f120006b660 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1207600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f12000b5750 con 0x7f12000b4eb0 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1205600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f120006b660 0x7f120006baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1205600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f120006b660 0x7f120006baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57382/0 (socket says 172.21.15.86:57382) 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1205600640 1 -- 172.21.15.86:0/3264467508 learned_addr learned my addr 172.21.15.86:0/3264467508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f11ffe00640 1 --2- 172.21.15.86:0/3264467508 >> v2:172.21.15.196:3300/0 conn(0x7f12000b4eb0 0x7f1200162740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1205600640 1 -- 172.21.15.86:0/3264467508 >> v2:172.21.15.196:3300/0 conn(0x7f12000b4eb0 msgr2=0x7f1200162740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:30.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1205600640 1 --2- 172.21.15.86:0/3264467508 >> v2:172.21.15.196:3300/0 conn(0x7f12000b4eb0 0x7f1200162740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1205600640 1 -- 172.21.15.86:0/3264467508 >> v1:172.21.15.196:6789/0 conn(0x7f120015e5b0 legacy=0x7f12000b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.379+0000 7f1205600640 1 -- 172.21.15.86:0/3264467508 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12000b5a30 con 0x7f120006b660 2024-03-22T04:11:30.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1205600640 1 --2- 172.21.15.86:0/3264467508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120006b660 0x7f120006baf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f12000b7310 tx=0x7f11f801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d276f21fb44d4f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f11ff400640 1 -- 172.21.15.86:0/3264467508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80b1070 con 0x7f120006b660 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f11ff400640 1 -- 172.21.15.86:0/3264467508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f80ac600 con 0x7f120006b660 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f11ff400640 1 -- 172.21.15.86:0/3264467508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80ac9d0 con 0x7f120006b660 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1207600640 1 -- 172.21.15.86:0/3264467508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120006b660 msgr2=0x7f120006baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1207600640 1 --2- 172.21.15.86:0/3264467508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120006b660 0x7f120006baf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f12000b7310 tx=0x7f11f801c490 comp rx=0 tx=0).stop 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1207600640 1 -- 172.21.15.86:0/3264467508 shutdown_connections 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1207600640 1 --2- 172.21.15.86:0/3264467508 >> v2:172.21.15.196:3300/0 conn(0x7f12000b4eb0 0x7f1200162740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1207600640 1 --2- 172.21.15.86:0/3264467508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120006b660 0x7f120006baf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1207600640 1 -- 172.21.15.86:0/3264467508 >> 172.21.15.86:0/3264467508 conn(0x7f12000d0fa0 msgr2=0x7f12000d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1207600640 1 -- 172.21.15.86:0/3264467508 shutdown_connections 2024-03-22T04:11:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.383+0000 7f1207600640 1 -- 172.21.15.86:0/3264467508 wait complete. 2024-03-22T04:11:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 Processor -- start 2024-03-22T04:11:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 -- start start 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f120006b660 0x7f1200138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12000b4eb0 0x7f1200157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120015e5b0 0x7f12000ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f120016b2e0 con 0x7f120015e5b0 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1200072e80 con 0x7f12000b4eb0 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f120016b460 con 0x7f120006b660 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120015e5b0 0x7f12000ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120015e5b0 0x7f12000ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57388/0 (socket says 172.21.15.86:57388) 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 -- 172.21.15.86:0/4103285580 learned_addr learned my addr 172.21.15.86:0/4103285580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1204c00640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f120006b660 0x7f1200138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f11ffe00640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12000b4eb0 0x7f1200157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 -- 172.21.15.86:0/4103285580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f120006b660 msgr2=0x7f1200138930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f120006b660 0x7f1200138930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 -- 172.21.15.86:0/4103285580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12000b4eb0 msgr2=0x7f1200157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12000b4eb0 0x7f1200157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 -- 172.21.15.86:0/4103285580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120014a7c0 con 0x7f120015e5b0 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1205600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120015e5b0 0x7f12000ae7e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f11f80190a0 tx=0x7f11f801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f11fcc00640 1 -- 172.21.15.86:0/4103285580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80b1040 con 0x7f120015e5b0 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f11fcc00640 1 -- 172.21.15.86:0/4103285580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f80bc070 con 0x7f120015e5b0 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1200138e70 con 0x7f120015e5b0 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f11fcc00640 1 -- 172.21.15.86:0/4103285580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80b7690 con 0x7f120015e5b0 2024-03-22T04:11:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.387+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120014aa30 con 0x7f120015e5b0 2024-03-22T04:11:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.391+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1200074470 con 0x7f120015e5b0 2024-03-22T04:11:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.391+0000 7f11fcc00640 1 -- 172.21.15.86:0/4103285580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f11f80b7830 con 0x7f120015e5b0 2024-03-22T04:11:30.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.391+0000 7f11fcc00640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f11e0067b20 0x7f11e0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.391+0000 7f11fcc00640 1 -- 172.21.15.86:0/4103285580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f11f81203c0 con 0x7f120015e5b0 2024-03-22T04:11:30.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.391+0000 7f1204c00640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f11e0067b20 0x7f11e0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.391+0000 7f1204c00640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f11e0067b20 0x7f11e0069fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f120014a6f0 tx=0x7f11f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.395+0000 7f11fcc00640 1 -- 172.21.15.86:0/4103285580 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f11f8016110 con 0x7f120015e5b0 2024-03-22T04:11:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.551+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1200068ae0 con 0x7f120015e5b0 2024-03-22T04:11:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.551+0000 7f11fcc00640 1 -- 172.21.15.86:0/4103285580 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f11f80f0330 con 0x7f120015e5b0 2024-03-22T04:11:30.557 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f11e0067b20 msgr2=0x7f11e0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f11e0067b20 0x7f11e0069fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f120014a6f0 tx=0x7f11f0009040 comp rx=0 tx=0).stop 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120015e5b0 msgr2=0x7f12000ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120015e5b0 0x7f12000ae7e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f11f80190a0 tx=0x7f11f801acd0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 shutdown_connections 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f11e0067b20 0x7f11e0069fe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12000b4eb0 0x7f1200157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120015e5b0 0x7f12000ae7e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 --2- 172.21.15.86:0/4103285580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f120006b660 0x7f1200138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 >> 172.21.15.86:0/4103285580 conn(0x7f12000d0fa0 msgr2=0x7f120006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 shutdown_connections 2024-03-22T04:11:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.555+0000 7f1207600640 1 -- 172.21.15.86:0/4103285580 wait complete. 2024-03-22T04:11:30.577 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:30.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcdfc00640 1 Processor -- start 2024-03-22T04:11:30.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcdfc00640 1 -- start start 2024-03-22T04:11:30.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcdfc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd8070560 0x7fdcd80d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcdfc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd80d15f0 con 0x7fdcd815e7e0 2024-03-22T04:11:30.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcdfc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd80d1310 con 0x7fdcd80b4710 2024-03-22T04:11:30.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcdfc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd80b4ba0 con 0x7fdcd8070560 2024-03-22T04:11:30.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcdd200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdcd815e7e0 0x7fdcd806fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52692/0 (socket says 172.21.15.86:52692) 2024-03-22T04:11:30.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcdd200640 1 -- 172.21.15.86:0/1326030918 learned_addr learned my addr 172.21.15.86:0/1326030918 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:30.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcd7e00640 1 --2- 172.21.15.86:0/1326030918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd8070560 0x7fdcd80d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcd7400640 1 -- 172.21.15.86:0/1326030918 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592127744 0 0) 0x7fdcd80d15f0 con 0x7fdcd815e7e0 2024-03-22T04:11:30.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcd7400640 1 -- 172.21.15.86:0/1326030918 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcbc003530 con 0x7fdcd815e7e0 2024-03-22T04:11:30.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcd7400640 1 -- 172.21.15.86:0/1326030918 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4088296250 0 0) 0x7fdcd80d1310 con 0x7fdcd80b4710 2024-03-22T04:11:30.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcd7400640 1 -- 172.21.15.86:0/1326030918 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcd80d15f0 con 0x7fdcd80b4710 2024-03-22T04:11:30.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcd7e00640 1 -- 172.21.15.86:0/1326030918 >> v1:172.21.15.196:6789/0 conn(0x7fdcd80b4710 legacy=0x7fdcd80cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcd7e00640 1 -- 172.21.15.86:0/1326030918 >> v1:172.21.15.86:6789/0 conn(0x7fdcd815e7e0 legacy=0x7fdcd806fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.711+0000 7fdcd7e00640 1 -- 172.21.15.86:0/1326030918 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd80cc880 con 0x7fdcd8070560 2024-03-22T04:11:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcd7e00640 1 --2- 172.21.15.86:0/1326030918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd8070560 0x7fdcd80d0d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40097f0 tx=0x7fdcc401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df311bd3a572fcb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcd7400640 1 -- 172.21.15.86:0/1326030918 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40b0070 con 0x7fdcd8070560 2024-03-22T04:11:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcd7400640 1 -- 172.21.15.86:0/1326030918 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc40ab600 con 0x7fdcd8070560 2024-03-22T04:11:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcd7400640 1 -- 172.21.15.86:0/1326030918 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40ab9d0 con 0x7fdcd8070560 2024-03-22T04:11:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcdfc00640 1 -- 172.21.15.86:0/1326030918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd8070560 msgr2=0x7fdcd80d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcdfc00640 1 --2- 172.21.15.86:0/1326030918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd8070560 0x7fdcd80d0d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40097f0 tx=0x7fdcc401c3c0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcdfc00640 1 -- 172.21.15.86:0/1326030918 shutdown_connections 2024-03-22T04:11:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcdfc00640 1 --2- 172.21.15.86:0/1326030918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd8070560 0x7fdcd80d0d30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcdfc00640 1 -- 172.21.15.86:0/1326030918 >> 172.21.15.86:0/1326030918 conn(0x7fdcd80b3bf0 msgr2=0x7fdcd80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcdfc00640 1 -- 172.21.15.86:0/1326030918 shutdown_connections 2024-03-22T04:11:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.715+0000 7fdcdfc00640 1 -- 172.21.15.86:0/1326030918 wait complete. 2024-03-22T04:11:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 Processor -- start 2024-03-22T04:11:30.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 -- start start 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd8070560 0x7fdcd81551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd80b4710 0x7fdcd80ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd815e7e0 0x7fdcd80ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd816c1b0 con 0x7fdcd80b4710 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd80ce4d0 con 0x7fdcd8070560 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd816c330 con 0x7fdcd815e7e0 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd80b4710 0x7fdcd80ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd80b4710 0x7fdcd80ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57402/0 (socket says 172.21.15.86:57402) 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 -- 172.21.15.86:0/3129938465 learned_addr learned my addr 172.21.15.86:0/3129938465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdd200640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd8070560 0x7fdcd81551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcddc00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd815e7e0 0x7fdcd80ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 -- 172.21.15.86:0/3129938465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd815e7e0 msgr2=0x7fdcd80ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd815e7e0 0x7fdcd80ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 -- 172.21.15.86:0/3129938465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd8070560 msgr2=0x7fdcd81551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd8070560 0x7fdcd81551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 -- 172.21.15.86:0/3129938465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd815b730 con 0x7fdcd80b4710 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd7e00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd80b4710 0x7fdcd80ca220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4018fd0 tx=0x7fdcc4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd4c00640 1 -- 172.21.15.86:0/3129938465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40b0040 con 0x7fdcd80b4710 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd4c00640 1 -- 172.21.15.86:0/3129938465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc40b9070 con 0x7fdcd80b4710 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd815ba10 con 0x7fdcd80b4710 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcd4c00640 1 -- 172.21.15.86:0/3129938465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40b4690 con 0x7fdcd80b4710 2024-03-22T04:11:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd8155730 con 0x7fdcd80b4710 2024-03-22T04:11:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.719+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd8074470 con 0x7fdcd80b4710 2024-03-22T04:11:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.723+0000 7fdcd4c00640 1 -- 172.21.15.86:0/3129938465 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdcc40b4830 con 0x7fdcd80b4710 2024-03-22T04:11:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.723+0000 7fdcd4c00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdcb8067b20 0x7fdcb8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:30.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.723+0000 7fdcd4c00640 1 -- 172.21.15.86:0/3129938465 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fdcc411fe70 con 0x7fdcd80b4710 2024-03-22T04:11:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.723+0000 7fdcdd200640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdcb8067b20 0x7fdcb8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:30.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.723+0000 7fdcdd200640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdcb8067b20 0x7fdcb8069fe0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80012f0 tx=0x7fdcc8001a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:30.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.727+0000 7fdcd4c00640 1 -- 172.21.15.86:0/3129938465 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdcc40160d0 con 0x7fdcd80b4710 2024-03-22T04:11:30.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdcd806fc30 con 0x7fdcd80b4710 2024-03-22T04:11:30.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcd4c00640 1 -- 172.21.15.86:0/3129938465 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fdcc40efdf0 con 0x7fdcd80b4710 2024-03-22T04:11:30.888 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdcb8067b20 msgr2=0x7fdcb8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdcb8067b20 0x7fdcb8069fe0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80012f0 tx=0x7fdcc8001a40 comp rx=0 tx=0).stop 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd80b4710 msgr2=0x7fdcd80ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd80b4710 0x7fdcd80ca220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4018fd0 tx=0x7fdcc4019160 comp rx=0 tx=0).stop 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 shutdown_connections 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdcb8067b20 0x7fdcb8069fe0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcd815e7e0 0x7fdcd80ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd80b4710 0x7fdcd80ca220 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 --2- 172.21.15.86:0/3129938465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcd8070560 0x7fdcd81551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 >> 172.21.15.86:0/3129938465 conn(0x7fdcd80b3bf0 msgr2=0x7fdcd80b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:30.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 shutdown_connections 2024-03-22T04:11:30.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:30.883+0000 7fdcdfc00640 1 -- 172.21.15.86:0/3129938465 wait complete. 2024-03-22T04:11:30.908 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:31.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f3131000640 1 Processor -- start 2024-03-22T04:11:31.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f3131000640 1 -- start start 2024-03-22T04:11:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f3131000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c15fb40 0x7f312c070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f3131000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312c070cb0 con 0x7f312c09b180 2024-03-22T04:11:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f3131000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312c0d0880 con 0x7f312c0aee30 2024-03-22T04:11:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f3131000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f312c0d0b60 con 0x7f312c15fb40 2024-03-22T04:11:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f312a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f312c09b180 0x7f312c06f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52702/0 (socket says 172.21.15.86:52702) 2024-03-22T04:11:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f312a800640 1 -- 172.21.15.86:0/990172562 learned_addr learned my addr 172.21.15.86:0/990172562 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.047+0000 7f3129e00640 1 --2- 172.21.15.86:0/990172562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c15fb40 0x7f312c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3129400640 1 -- 172.21.15.86:0/990172562 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3169171794 0 0) 0x7f312c070cb0 con 0x7f312c09b180 2024-03-22T04:11:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3129e00640 1 -- 172.21.15.86:0/990172562 >> v1:172.21.15.196:6789/0 conn(0x7f312c0aee30 legacy=0x7f312c09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3129e00640 1 -- 172.21.15.86:0/990172562 >> v1:172.21.15.86:6789/0 conn(0x7f312c09b180 legacy=0x7f312c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3129e00640 1 -- 172.21.15.86:0/990172562 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c0d0e40 con 0x7f312c15fb40 2024-03-22T04:11:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3129e00640 1 --2- 172.21.15.86:0/990172562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c15fb40 0x7f312c070770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f311c009770 tx=0x7f311c01c310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d8a784b85b3d99 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3129400640 1 -- 172.21.15.86:0/990172562 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b0070 con 0x7f312c15fb40 2024-03-22T04:11:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3129400640 1 -- 172.21.15.86:0/990172562 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c0ab600 con 0x7f312c15fb40 2024-03-22T04:11:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3129400640 1 -- 172.21.15.86:0/990172562 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0ab9d0 con 0x7f312c15fb40 2024-03-22T04:11:31.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3131000640 1 -- 172.21.15.86:0/990172562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c15fb40 msgr2=0x7f312c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3131000640 1 --2- 172.21.15.86:0/990172562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c15fb40 0x7f312c070770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f311c009770 tx=0x7f311c01c310 comp rx=0 tx=0).stop 2024-03-22T04:11:31.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3131000640 1 -- 172.21.15.86:0/990172562 shutdown_connections 2024-03-22T04:11:31.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3131000640 1 --2- 172.21.15.86:0/990172562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c15fb40 0x7f312c070770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3131000640 1 -- 172.21.15.86:0/990172562 >> 172.21.15.86:0/990172562 conn(0x7f312c0b3bf0 msgr2=0x7f312c0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.051+0000 7f3131000640 1 -- 172.21.15.86:0/990172562 shutdown_connections 2024-03-22T04:11:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 -- 172.21.15.86:0/990172562 wait complete. 2024-03-22T04:11:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 Processor -- start 2024-03-22T04:11:31.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 -- start start 2024-03-22T04:11:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c09b180 0x7f312c14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f312c0aee30 0x7f312c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f312c15fb40 0x7f312c157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f312c06fb50 con 0x7f312c15fb40 2024-03-22T04:11:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f312c0b49e0 con 0x7f312c0aee30 2024-03-22T04:11:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f312c06fcd0 con 0x7f312c09b180 2024-03-22T04:11:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f312c15fb40 0x7f312c157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3121400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f312c0aee30 0x7f312c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3129e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c09b180 0x7f312c14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f312c15fb40 0x7f312c157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57408/0 (socket says 172.21.15.86:57408) 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 -- 172.21.15.86:0/488511457 learned_addr learned my addr 172.21.15.86:0/488511457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 -- 172.21.15.86:0/488511457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c09b180 msgr2=0x7f312c14c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c09b180 0x7f312c14c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 -- 172.21.15.86:0/488511457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f312c0aee30 msgr2=0x7f312c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f312c0aee30 0x7f312c159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 -- 172.21.15.86:0/488511457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c14cb00 con 0x7f312c15fb40 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f312a800640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f312c15fb40 0x7f312c157390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f312400c480 tx=0x7f312400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3122a00640 1 -- 172.21.15.86:0/488511457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124018070 con 0x7f312c15fb40 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3122a00640 1 -- 172.21.15.86:0/488511457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31240074e0 con 0x7f312c15fb40 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3122a00640 1 -- 172.21.15.86:0/488511457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124004a30 con 0x7f312c15fb40 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1562f0 con 0x7f312c15fb40 2024-03-22T04:11:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.055+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1568b0 con 0x7f312c15fb40 2024-03-22T04:11:31.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.059+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312c074470 con 0x7f312c15fb40 2024-03-22T04:11:31.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.059+0000 7f3122a00640 1 -- 172.21.15.86:0/488511457 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3124004bd0 con 0x7f312c15fb40 2024-03-22T04:11:31.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.059+0000 7f3122a00640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3108067b20 0x7f3108069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.059+0000 7f3122a00640 1 -- 172.21.15.86:0/488511457 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f31240855e0 con 0x7f312c15fb40 2024-03-22T04:11:31.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.059+0000 7f3129e00640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3108067b20 0x7f3108069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.059+0000 7f3129e00640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3108067b20 0x7f3108069fe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f311c0ad000 tx=0x7f311c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.063+0000 7f3122a00640 1 -- 172.21.15.86:0/488511457 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f31240555e0 con 0x7f312c15fb40 2024-03-22T04:11:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.219+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f312c0b4350 con 0x7f312c15fb40 2024-03-22T04:11:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.219+0000 7f3122a00640 1 -- 172.21.15.86:0/488511457 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f3124059680 con 0x7f312c15fb40 2024-03-22T04:11:31.225 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3108067b20 msgr2=0x7f3108069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3108067b20 0x7f3108069fe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f311c0ad000 tx=0x7f311c0a6000 comp rx=0 tx=0).stop 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f312c15fb40 msgr2=0x7f312c157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f312c15fb40 0x7f312c157390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f312400c480 tx=0x7f312400c940 comp rx=0 tx=0).stop 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 shutdown_connections 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3108067b20 0x7f3108069fe0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f312c0aee30 0x7f312c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f312c15fb40 0x7f312c157390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 --2- 172.21.15.86:0/488511457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f312c09b180 0x7f312c14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 >> 172.21.15.86:0/488511457 conn(0x7f312c0b3bf0 msgr2=0x7f312c0b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 shutdown_connections 2024-03-22T04:11:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.223+0000 7f3131000640 1 -- 172.21.15.86:0/488511457 wait complete. 2024-03-22T04:11:31.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:31.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abee00640 1 Processor -- start 2024-03-22T04:11:31.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abee00640 1 -- start start 2024-03-22T04:11:31.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab815c1a0 0x7f2ab815c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2ab8074eb0 0x7f2ab80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abee00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ab8071b70 0x7f2ab8072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab80d02c0 con 0x7f2ab8074eb0 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abee00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab806dca0 con 0x7f2ab8071b70 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab806df80 con 0x7f2ab815c1a0 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abda00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab815c1a0 0x7f2ab815c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abe400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ab8071b70 0x7f2ab8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abd000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2ab8074eb0 0x7f2ab80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abda00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab815c1a0 0x7f2ab815c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42304/0 (socket says 172.21.15.86:42304) 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abda00640 1 -- 172.21.15.86:0/727869522 learned_addr learned my addr 172.21.15.86:0/727869522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abe400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ab8071b70 0x7f2ab8072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:42162/0 (socket says 172.21.15.86:42162) 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abda00640 1 -- 172.21.15.86:0/727869522 >> v2:172.21.15.196:3300/0 conn(0x7f2ab8071b70 msgr2=0x7f2ab8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:31.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abda00640 1 --2- 172.21.15.86:0/727869522 >> v2:172.21.15.196:3300/0 conn(0x7f2ab8071b70 0x7f2ab8072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abda00640 1 -- 172.21.15.86:0/727869522 >> v2:172.21.15.86:3300/0 conn(0x7f2ab8074eb0 msgr2=0x7f2ab80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:31.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abda00640 1 --2- 172.21.15.86:0/727869522 >> v2:172.21.15.86:3300/0 conn(0x7f2ab8074eb0 0x7f2ab80cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.379+0000 7f2abda00640 1 -- 172.21.15.86:0/727869522 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab806e260 con 0x7f2ab815c1a0 2024-03-22T04:11:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abda00640 1 --2- 172.21.15.86:0/727869522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab815c1a0 0x7f2ab815c580 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8074c60 tx=0x7f2aac01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6838599df62774d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2ab7e00640 1 -- 172.21.15.86:0/727869522 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0b0070 con 0x7f2ab815c1a0 2024-03-22T04:11:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2ab7e00640 1 -- 172.21.15.86:0/727869522 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aac0ab600 con 0x7f2ab815c1a0 2024-03-22T04:11:31.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2ab7e00640 1 -- 172.21.15.86:0/727869522 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0ab9d0 con 0x7f2ab815c1a0 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 -- 172.21.15.86:0/727869522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab815c1a0 msgr2=0x7f2ab815c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 --2- 172.21.15.86:0/727869522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab815c1a0 0x7f2ab815c580 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8074c60 tx=0x7f2aac01c490 comp rx=0 tx=0).stop 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 -- 172.21.15.86:0/727869522 shutdown_connections 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 --2- 172.21.15.86:0/727869522 >> v2:172.21.15.196:3300/0 conn(0x7f2ab8071b70 0x7f2ab8072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 --2- 172.21.15.86:0/727869522 >> v2:172.21.15.86:3300/0 conn(0x7f2ab8074eb0 0x7f2ab80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 --2- 172.21.15.86:0/727869522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab815c1a0 0x7f2ab815c580 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 -- 172.21.15.86:0/727869522 >> 172.21.15.86:0/727869522 conn(0x7f2ab80d0580 msgr2=0x7f2ab80d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 -- 172.21.15.86:0/727869522 shutdown_connections 2024-03-22T04:11:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.383+0000 7f2abee00640 1 -- 172.21.15.86:0/727869522 wait complete. 2024-03-22T04:11:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 Processor -- start 2024-03-22T04:11:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 -- start start 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab8071b70 0x7f2ab815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab8074eb0 0x7f2ab813d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab815c1a0 0x7f2ab8159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab806f600 con 0x7f2ab815c1a0 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab806f480 con 0x7f2ab8071b70 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab806f780 con 0x7f2ab8074eb0 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abe400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab815c1a0 0x7f2ab8159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abe400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab815c1a0 0x7f2ab8159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57436/0 (socket says 172.21.15.86:57436) 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abe400640 1 -- 172.21.15.86:0/2524461917 learned_addr learned my addr 172.21.15.86:0/2524461917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abda00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab8071b70 0x7f2ab815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abd000640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab8074eb0 0x7f2ab813d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abda00640 1 -- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab8074eb0 msgr2=0x7f2ab813d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abda00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab8074eb0 0x7f2ab813d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abda00640 1 -- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab815c1a0 msgr2=0x7f2ab8159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abda00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab815c1a0 0x7f2ab8159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abda00640 1 -- 172.21.15.86:0/2524461917 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab81f7c00 con 0x7f2ab8071b70 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abe400640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab815c1a0 0x7f2ab8159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abda00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab8071b70 0x7f2ab815b2a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0ab000 tx=0x7f2aac01acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2ab5600640 1 -- 172.21.15.86:0/2524461917 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0ba020 con 0x7f2ab8071b70 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2ab5600640 1 -- 172.21.15.86:0/2524461917 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aac0b6420 con 0x7f2ab8071b70 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab815b8d0 con 0x7f2ab8071b70 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2ab5600640 1 -- 172.21.15.86:0/2524461917 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0b0040 con 0x7f2ab8071b70 2024-03-22T04:11:31.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.387+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab81f8180 con 0x7f2ab8071b70 2024-03-22T04:11:31.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.391+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab804f900 con 0x7f2ab8071b70 2024-03-22T04:11:31.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.391+0000 7f2ab5600640 1 -- 172.21.15.86:0/2524461917 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2aac0bd020 con 0x7f2ab8071b70 2024-03-22T04:11:31.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.391+0000 7f2ab5600640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a98067b20 0x7f2a98069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.391+0000 7f2ab5600640 1 -- 172.21.15.86:0/2524461917 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f2aac1201d0 con 0x7f2ab8071b70 2024-03-22T04:11:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.391+0000 7f2abd000640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a98067b20 0x7f2a98069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.391+0000 7f2abd000640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a98067b20 0x7f2a98069fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ab81591e0 tx=0x7f2aa4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.395+0000 7f2ab5600640 1 -- 172.21.15.86:0/2524461917 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2aac0c4030 con 0x7f2ab8071b70 2024-03-22T04:11:31.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.547+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2ab815c880 con 0x7f2ab8071b70 2024-03-22T04:11:31.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.547+0000 7f2ab5600640 1 -- 172.21.15.86:0/2524461917 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f2aac016110 con 0x7f2ab8071b70 2024-03-22T04:11:31.553 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a98067b20 msgr2=0x7f2a98069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a98067b20 0x7f2a98069fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ab81591e0 tx=0x7f2aa4009190 comp rx=0 tx=0).stop 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab8071b70 msgr2=0x7f2ab815b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab8071b70 0x7f2ab815b2a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0ab000 tx=0x7f2aac01acd0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 shutdown_connections 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a98067b20 0x7f2a98069fe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab815c1a0 0x7f2ab8159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ab8074eb0 0x7f2ab813d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 --2- 172.21.15.86:0/2524461917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ab8071b70 0x7f2ab815b2a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 >> 172.21.15.86:0/2524461917 conn(0x7f2ab80d0580 msgr2=0x7f2ab80b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 shutdown_connections 2024-03-22T04:11:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.551+0000 7f2abee00640 1 -- 172.21.15.86:0/2524461917 wait complete. 2024-03-22T04:11:31.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:11:31.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca52200640 1 Processor -- start 2024-03-22T04:11:31.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca52200640 1 -- start start 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca52200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca4c147e10 0x7fca4c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca52200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c162710 0x7fca4c15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca52200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4c0b2bd0 con 0x7fca4c06c270 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca52200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fca4c0b2610 con 0x7fca4c147e10 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca52200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c0b28f0 con 0x7fca4c162710 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca4c147e10 0x7fca4c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fca4c06c270 0x7fca4c162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52718/0 (socket says 172.21.15.86:52718) 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4aa00640 1 -- 172.21.15.86:0/2386427269 learned_addr learned my addr 172.21.15.86:0/2386427269 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:31.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4be00640 1 --2- 172.21.15.86:0/2386427269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c162710 0x7fca4c15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4b400640 1 -- 172.21.15.86:0/2386427269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c162710 msgr2=0x7fca4c15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4b400640 1 --2- 172.21.15.86:0/2386427269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c162710 0x7fca4c15fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4b400640 1 -- 172.21.15.86:0/2386427269 >> v1:172.21.15.86:6789/0 conn(0x7fca4c06c270 legacy=0x7fca4c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4b400640 1 -- 172.21.15.86:0/2386427269 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c0b5ce0 con 0x7fca4c147e10 2024-03-22T04:11:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.707+0000 7fca4a000640 1 -- 172.21.15.86:0/2386427269 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638421464 0 0) 0x7fca4c0b2bd0 con 0x7fca4c06c270 2024-03-22T04:11:31.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca4b400640 1 --2- 172.21.15.86:0/2386427269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c147e10 0x7fca4c0b4100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0097c0 tx=0x7fca3c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1eb4c955661f91 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca4a000640 1 -- 172.21.15.86:0/2386427269 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0b1070 con 0x7fca4c147e10 2024-03-22T04:11:31.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca4a000640 1 -- 172.21.15.86:0/2386427269 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c0ac410 con 0x7fca4c147e10 2024-03-22T04:11:31.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca4a000640 1 -- 172.21.15.86:0/2386427269 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0ac7e0 con 0x7fca4c147e10 2024-03-22T04:11:31.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca52200640 1 -- 172.21.15.86:0/2386427269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c147e10 msgr2=0x7fca4c0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca52200640 1 --2- 172.21.15.86:0/2386427269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c147e10 0x7fca4c0b4100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0097c0 tx=0x7fca3c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca52200640 1 -- 172.21.15.86:0/2386427269 shutdown_connections 2024-03-22T04:11:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca52200640 1 --2- 172.21.15.86:0/2386427269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c162710 0x7fca4c15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca52200640 1 --2- 172.21.15.86:0/2386427269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c147e10 0x7fca4c0b4100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca52200640 1 -- 172.21.15.86:0/2386427269 >> 172.21.15.86:0/2386427269 conn(0x7fca4c0b6620 msgr2=0x7fca4c0b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca52200640 1 -- 172.21.15.86:0/2386427269 shutdown_connections 2024-03-22T04:11:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.711+0000 7fca52200640 1 -- 172.21.15.86:0/2386427269 wait complete. 2024-03-22T04:11:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 Processor -- start 2024-03-22T04:11:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 -- start start 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c06c270 0x7fca4c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c147e10 0x7fca4c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca4c162710 0x7fca4c1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c160c40 con 0x7fca4c06c270 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c160ac0 con 0x7fca4c162710 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c16b150 con 0x7fca4c147e10 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c06c270 0x7fca4c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c06c270 0x7fca4c0cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57458/0 (socket says 172.21.15.86:57458) 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 -- 172.21.15.86:0/2048970630 learned_addr learned my addr 172.21.15.86:0/2048970630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4aa00640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c147e10 0x7fca4c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4be00640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca4c162710 0x7fca4c1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 -- 172.21.15.86:0/2048970630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c147e10 msgr2=0x7fca4c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c147e10 0x7fca4c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 -- 172.21.15.86:0/2048970630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca4c162710 msgr2=0x7fca4c1586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca4c162710 0x7fca4c1586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 -- 172.21.15.86:0/2048970630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c0cb400 con 0x7fca4c06c270 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca4b400640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c06c270 0x7fca4c0cc4a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fca3c01a0d0 tx=0x7fca3c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca43400640 1 -- 172.21.15.86:0/2048970630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0b9070 con 0x7fca4c06c270 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca43400640 1 -- 172.21.15.86:0/2048970630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c0acd90 con 0x7fca4c06c270 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca43400640 1 -- 172.21.15.86:0/2048970630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0b1040 con 0x7fca4c06c270 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c0cb6e0 con 0x7fca4c06c270 2024-03-22T04:11:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c0cbca0 con 0x7fca4c06c270 2024-03-22T04:11:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.715+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c068650 con 0x7fca4c06c270 2024-03-22T04:11:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.719+0000 7fca43400640 1 -- 172.21.15.86:0/2048970630 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fca3c0bc070 con 0x7fca4c06c270 2024-03-22T04:11:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.719+0000 7fca43400640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca2c067b20 0x7fca2c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.719+0000 7fca43400640 1 -- 172.21.15.86:0/2048970630 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fca3c11fc30 con 0x7fca4c06c270 2024-03-22T04:11:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.719+0000 7fca4aa00640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca2c067b20 0x7fca2c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.719+0000 7fca4aa00640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca2c067b20 0x7fca2c069fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca4c159070 tx=0x7fca38007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:31.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.723+0000 7fca43400640 1 -- 172.21.15.86:0/2048970630 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fca3c016090 con 0x7fca4c06c270 2024-03-22T04:11:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fca4c0b4190 con 0x7fca4c06c270 2024-03-22T04:11:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca43400640 1 -- 172.21.15.86:0/2048970630 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fca3c0efcd0 con 0x7fca4c06c270 2024-03-22T04:11:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:11:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca2c067b20 msgr2=0x7fca2c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca2c067b20 0x7fca2c069fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca4c159070 tx=0x7fca38007d60 comp rx=0 tx=0).stop 2024-03-22T04:11:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c06c270 msgr2=0x7fca4c0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c06c270 0x7fca4c0cc4a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fca3c01a0d0 tx=0x7fca3c019240 comp rx=0 tx=0).stop 2024-03-22T04:11:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 shutdown_connections 2024-03-22T04:11:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca2c067b20 0x7fca2c069fe0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca4c162710 0x7fca4c1586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca4c147e10 0x7fca4c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 --2- 172.21.15.86:0/2048970630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4c06c270 0x7fca4c0cc4a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:31.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 >> 172.21.15.86:0/2048970630 conn(0x7fca4c0b6620 msgr2=0x7fca4c06f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:31.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 shutdown_connections 2024-03-22T04:11:31.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:31.891+0000 7fca52200640 1 -- 172.21.15.86:0/2048970630 wait complete. 2024-03-22T04:11:31.916 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-22T04:11:31.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:31.916 DEBUG:teuthology.orchestra.run.smithi196:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-22T04:11:31.952 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:11:31.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:31.953 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-22T04:11:32.004 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:11:32.004 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:32.013 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:11:32.013 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:32.062 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:11:32.062 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:32.114 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.287+0000 7f1b73c00640 1 Processor -- start 2024-03-22T04:11:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.287+0000 7f1b73c00640 1 -- start start 2024-03-22T04:11:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b73c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b6c0ad710 0x7f1b6c142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b73c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6c15c7c0 con 0x7f1b6c0ad330 2024-03-22T04:11:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b73c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6c06dea0 con 0x7f1b6c06ef40 2024-03-22T04:11:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b73c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1b6c06e180 con 0x7f1b6c0ad710 2024-03-22T04:11:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b71200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1b6c0ad330 0x7f1b6c13fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52732/0 (socket says 172.21.15.86:52732) 2024-03-22T04:11:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b71200640 1 -- 172.21.15.86:0/3014644347 learned_addr learned my addr 172.21.15.86:0/3014644347 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b71c00640 1 --2- 172.21.15.86:0/3014644347 >> v2:172.21.15.196:3300/0 conn(0x7f1b6c0ad710 0x7f1b6c142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b6b400640 1 -- 172.21.15.86:0/3014644347 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575993902 0 0) 0x7f1b6c15c7c0 con 0x7f1b6c0ad330 2024-03-22T04:11:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b6b400640 1 -- 172.21.15.86:0/3014644347 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b50003530 con 0x7f1b6c0ad330 2024-03-22T04:11:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b71c00640 1 -- 172.21.15.86:0/3014644347 >> v1:172.21.15.196:6789/0 conn(0x7f1b6c06ef40 legacy=0x7f1b6c1402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b71c00640 1 -- 172.21.15.86:0/3014644347 >> v1:172.21.15.86:6789/0 conn(0x7f1b6c0ad330 legacy=0x7f1b6c13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b6b400640 1 -- 172.21.15.86:0/3014644347 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156268832 0 0) 0x7f1b6c06dea0 con 0x7f1b6c06ef40 2024-03-22T04:11:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.291+0000 7f1b71c00640 1 -- 172.21.15.86:0/3014644347 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c06e460 con 0x7f1b6c0ad710 2024-03-22T04:11:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b71c00640 1 --2- 172.21.15.86:0/3014644347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 0x7f1b6c142690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1b640097c0 tx=0x7f1b6401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b5430fd3b9aa37 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b6b400640 1 -- 172.21.15.86:0/3014644347 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640aa5f0 con 0x7f1b6c0ad710 2024-03-22T04:11:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b6b400640 1 -- 172.21.15.86:0/3014644347 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b640af070 con 0x7f1b6c0ad710 2024-03-22T04:11:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b6b400640 1 -- 172.21.15.86:0/3014644347 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640aa9b0 con 0x7f1b6c0ad710 2024-03-22T04:11:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- 172.21.15.86:0/3014644347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 msgr2=0x7f1b6c142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 --2- 172.21.15.86:0/3014644347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 0x7f1b6c142690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1b640097c0 tx=0x7f1b6401c470 comp rx=0 tx=0).stop 2024-03-22T04:11:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- 172.21.15.86:0/3014644347 shutdown_connections 2024-03-22T04:11:32.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 --2- 172.21.15.86:0/3014644347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 0x7f1b6c142690 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- 172.21.15.86:0/3014644347 >> 172.21.15.86:0/3014644347 conn(0x7f1b6c0b1df0 msgr2=0x7f1b6c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- 172.21.15.86:0/3014644347 shutdown_connections 2024-03-22T04:11:32.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- 172.21.15.86:0/3014644347 wait complete. 2024-03-22T04:11:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 Processor -- start 2024-03-22T04:11:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- start start 2024-03-22T04:11:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c06ef40 0x7f1b6c1f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b6c0ad330 0x7f1b6c09a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 0x7f1b6c141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c06db10 con 0x7f1b6c06ef40 2024-03-22T04:11:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1409d0 con 0x7f1b6c0ad710 2024-03-22T04:11:32.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b73c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6c16a780 con 0x7f1b6c0ad330 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c06ef40 0x7f1b6c1f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c06ef40 0x7f1b6c1f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57464/0 (socket says 172.21.15.86:57464) 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.295+0000 7f1b71200640 1 -- 172.21.15.86:0/61601418 learned_addr learned my addr 172.21.15.86:0/61601418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b71c00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 0x7f1b6c141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b6be00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b6c0ad330 0x7f1b6c09a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b71200640 1 -- 172.21.15.86:0/61601418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b6c0ad330 msgr2=0x7f1b6c09a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b71200640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b6c0ad330 0x7f1b6c09a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b71200640 1 -- 172.21.15.86:0/61601418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 msgr2=0x7f1b6c141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b71200640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 0x7f1b6c141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b71200640 1 -- 172.21.15.86:0/61601418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c09ded0 con 0x7f1b6c06ef40 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b71200640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c06ef40 0x7f1b6c1f5ad0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00cbb0 tx=0x7f1b5c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b68c00640 1 -- 172.21.15.86:0/61601418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c017070 con 0x7f1b6c06ef40 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b68c00640 1 -- 172.21.15.86:0/61601418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c00ad00 con 0x7f1b6c06ef40 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c09a720 con 0x7f1b6c06ef40 2024-03-22T04:11:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b68c00640 1 -- 172.21.15.86:0/61601418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c012640 con 0x7f1b6c06ef40 2024-03-22T04:11:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c158c50 con 0x7f1b6c06ef40 2024-03-22T04:11:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c068650 con 0x7f1b6c06ef40 2024-03-22T04:11:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b68c00640 1 -- 172.21.15.86:0/61601418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1b5c01a070 con 0x7f1b6c06ef40 2024-03-22T04:11:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b68c00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b4c067b50 0x7f1b4c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b68c00640 1 -- 172.21.15.86:0/61601418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f1b5c085ff0 con 0x7f1b6c06ef40 2024-03-22T04:11:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.299+0000 7f1b6be00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b4c067b50 0x7f1b4c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.303+0000 7f1b6be00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b4c067b50 0x7f1b4c06a010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c1671f0 tx=0x7f1b58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.303+0000 7f1b68c00640 1 -- 172.21.15.86:0/61601418 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1b5c055ff0 con 0x7f1b6c06ef40 2024-03-22T04:11:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.463+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b6c0b0c80 con 0x7f1b6c06ef40 2024-03-22T04:11:32.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.463+0000 7f1b68c00640 1 -- 172.21.15.86:0/61601418 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7f1b5c05a090 con 0x7f1b6c06ef40 2024-03-22T04:11:32.469 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:32.469 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:32.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b4c067b50 msgr2=0x7f1b4c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b4c067b50 0x7f1b4c06a010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c1671f0 tx=0x7f1b58009040 comp rx=0 tx=0).stop 2024-03-22T04:11:32.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c06ef40 msgr2=0x7f1b6c1f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c06ef40 0x7f1b6c1f5ad0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00cbb0 tx=0x7f1b5c00a430 comp rx=0 tx=0).stop 2024-03-22T04:11:32.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 shutdown_connections 2024-03-22T04:11:32.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b4c067b50 0x7f1b4c06a010 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b6c0ad710 0x7f1b6c141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b6c0ad330 0x7f1b6c09a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 --2- 172.21.15.86:0/61601418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6c06ef40 0x7f1b6c1f5ad0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 >> 172.21.15.86:0/61601418 conn(0x7f1b6c0b1df0 msgr2=0x7f1b6c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:32.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 shutdown_connections 2024-03-22T04:11:32.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.467+0000 7f1b73c00640 1 -- 172.21.15.86:0/61601418 wait complete. 2024-03-22T04:11:32.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:32.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e60600640 1 Processor -- start 2024-03-22T04:11:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e60600640 1 -- start start 2024-03-22T04:11:32.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e60600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e580cb1d0 0x7f0e58162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e60600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 0x7f0e58162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e60600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e580cc7c0 con 0x7f0e580c6f20 2024-03-22T04:11:32.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e60600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0e580cb720 con 0x7f0e580cb1d0 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e60600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0e580cba00 con 0x7f0e580c7300 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e5d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e580cb1d0 0x7f0e58162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e5dc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0e580c6f20 0x7f0e58072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52746/0 (socket says 172.21.15.86:52746) 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e5dc00640 1 -- 172.21.15.86:0/2021819522 learned_addr learned my addr 172.21.15.86:0/2021819522 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.623+0000 7f0e5e600640 1 --2- 172.21.15.86:0/2021819522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 0x7f0e58162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e5d200640 1 -- 172.21.15.86:0/2021819522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 msgr2=0x7f0e58162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e5d200640 1 --2- 172.21.15.86:0/2021819522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 0x7f0e58162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e5d200640 1 -- 172.21.15.86:0/2021819522 >> v1:172.21.15.86:6789/0 conn(0x7f0e580c6f20 legacy=0x7f0e58072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e5d200640 1 -- 172.21.15.86:0/2021819522 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e580cbce0 con 0x7f0e580cb1d0 2024-03-22T04:11:32.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e57e00640 1 -- 172.21.15.86:0/2021819522 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2261442483 0 0) 0x7f0e580cc7c0 con 0x7f0e580c6f20 2024-03-22T04:11:32.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e5d200640 1 --2- 172.21.15.86:0/2021819522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cb1d0 0x7f0e58162030 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0e440097c0 tx=0x7f0e4401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f450d2adfb2fbd0a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e57e00640 1 -- 172.21.15.86:0/2021819522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e440b2070 con 0x7f0e580cb1d0 2024-03-22T04:11:32.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e57e00640 1 -- 172.21.15.86:0/2021819522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e440ad640 con 0x7f0e580cb1d0 2024-03-22T04:11:32.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e57e00640 1 -- 172.21.15.86:0/2021819522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e440ada10 con 0x7f0e580cb1d0 2024-03-22T04:11:32.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e60600640 1 -- 172.21.15.86:0/2021819522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cb1d0 msgr2=0x7f0e58162030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e60600640 1 --2- 172.21.15.86:0/2021819522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cb1d0 0x7f0e58162030 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0e440097c0 tx=0x7f0e4401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e60600640 1 -- 172.21.15.86:0/2021819522 shutdown_connections 2024-03-22T04:11:32.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e60600640 1 --2- 172.21.15.86:0/2021819522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 0x7f0e58162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e60600640 1 --2- 172.21.15.86:0/2021819522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cb1d0 0x7f0e58162030 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e60600640 1 -- 172.21.15.86:0/2021819522 >> 172.21.15.86:0/2021819522 conn(0x7f0e580d0730 msgr2=0x7f0e580d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:32.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e60600640 1 -- 172.21.15.86:0/2021819522 shutdown_connections 2024-03-22T04:11:32.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.627+0000 7f0e60600640 1 -- 172.21.15.86:0/2021819522 wait complete. 2024-03-22T04:11:32.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 Processor -- start 2024-03-22T04:11:32.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 -- start start 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580c6f20 0x7f0e580b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 0x7f0e580c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e580cb1d0 0x7f0e58141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e580c6190 con 0x7f0e580c6f20 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e580c6010 con 0x7f0e580cb1d0 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0e580c6310 con 0x7f0e580c7300 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580c6f20 0x7f0e580b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580c6f20 0x7f0e580b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57492/0 (socket says 172.21.15.86:57492) 2024-03-22T04:11:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 -- 172.21.15.86:0/2210532759 learned_addr learned my addr 172.21.15.86:0/2210532759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5d200640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 0x7f0e580c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5e600640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e580cb1d0 0x7f0e58141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 -- 172.21.15.86:0/2210532759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 msgr2=0x7f0e580c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 0x7f0e580c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 -- 172.21.15.86:0/2210532759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e580cb1d0 msgr2=0x7f0e58141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e580cb1d0 0x7f0e58141880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 -- 172.21.15.86:0/2210532759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e58141dc0 con 0x7f0e580c6f20 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e5dc00640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580c6f20 0x7f0e580b5270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c007fd0 tx=0x7f0e4c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e55600640 1 -- 172.21.15.86:0/2210532759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c017070 con 0x7f0e580c6f20 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e55600640 1 -- 172.21.15.86:0/2210532759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e4c005d10 con 0x7f0e580c6f20 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e55600640 1 -- 172.21.15.86:0/2210532759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c007310 con 0x7f0e580c6f20 2024-03-22T04:11:32.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e580b59e0 con 0x7f0e580c6f20 2024-03-22T04:11:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.631+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e580b5fa0 con 0x7f0e580c6f20 2024-03-22T04:11:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.635+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e58074470 con 0x7f0e580c6f20 2024-03-22T04:11:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.635+0000 7f0e55600640 1 -- 172.21.15.86:0/2210532759 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0e4c01a070 con 0x7f0e580c6f20 2024-03-22T04:11:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.635+0000 7f0e55600640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e38067b20 0x7f0e38069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.635+0000 7f0e55600640 1 -- 172.21.15.86:0/2210532759 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f0e4c085640 con 0x7f0e580c6f20 2024-03-22T04:11:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.635+0000 7f0e5d200640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e38067b20 0x7f0e38069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.635+0000 7f0e5d200640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e38067b20 0x7f0e38069fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e440af000 tx=0x7f0e44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.639+0000 7f0e55600640 1 -- 172.21.15.86:0/2210532759 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0e4c055640 con 0x7f0e580c6f20 2024-03-22T04:11:32.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.795+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e58071be0 con 0x7f0e580c6f20 2024-03-22T04:11:32.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.795+0000 7f0e55600640 1 -- 172.21.15.86:0/2210532759 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f0e4c0596e0 con 0x7f0e580c6f20 2024-03-22T04:11:32.801 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e38067b20 msgr2=0x7f0e38069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e38067b20 0x7f0e38069fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e440af000 tx=0x7f0e44008040 comp rx=0 tx=0).stop 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580c6f20 msgr2=0x7f0e580b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580c6f20 0x7f0e580b5270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c007fd0 tx=0x7f0e4c00c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 shutdown_connections 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e38067b20 0x7f0e38069fe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e580cb1d0 0x7f0e58141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580c7300 0x7f0e580c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 --2- 172.21.15.86:0/2210532759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580c6f20 0x7f0e580b5270 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 >> 172.21.15.86:0/2210532759 conn(0x7f0e580d0730 msgr2=0x7f0e580cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 shutdown_connections 2024-03-22T04:11:32.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.799+0000 7f0e60600640 1 -- 172.21.15.86:0/2210532759 wait complete. 2024-03-22T04:11:32.822 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7129400640 1 Processor -- start 2024-03-22T04:11:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7129400640 1 -- start start 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7129400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7124072650 0x7f7124072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7129400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7124073150 con 0x7f712415b760 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7129400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71240715a0 con 0x7f712406e7e0 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7129400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7124071350 con 0x7f7124072650 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7123200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f712415b760 0x7f712415bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52758/0 (socket says 172.21.15.86:52758) 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7123200640 1 -- 172.21.15.86:0/646613659 learned_addr learned my addr 172.21.15.86:0/646613659 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7122800640 1 --1- 172.21.15.86:0/646613659 >> v1:172.21.15.196:6789/0 conn(0x7f712406e7e0 0x7f712406ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:58980/0 (socket says 172.21.15.86:58980) 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7121e00640 1 --2- 172.21.15.86:0/646613659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7124072650 0x7f7124072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7121400640 1 -- 172.21.15.86:0/646613659 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1384405372 0 0) 0x7f7124073150 con 0x7f712415b760 2024-03-22T04:11:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7121400640 1 -- 172.21.15.86:0/646613659 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7108003530 con 0x7f712415b760 2024-03-22T04:11:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7121400640 1 -- 172.21.15.86:0/646613659 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2575039114 0 0) 0x7f71240715a0 con 0x7f712406e7e0 2024-03-22T04:11:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7121400640 1 -- 172.21.15.86:0/646613659 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7124073150 con 0x7f712406e7e0 2024-03-22T04:11:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7121e00640 1 -- 172.21.15.86:0/646613659 >> v1:172.21.15.196:6789/0 conn(0x7f712406e7e0 legacy=0x7f712406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7121e00640 1 -- 172.21.15.86:0/646613659 >> v1:172.21.15.86:6789/0 conn(0x7f712415b760 legacy=0x7f712415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.955+0000 7f7121e00640 1 -- 172.21.15.86:0/646613659 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7124071880 con 0x7f7124072650 2024-03-22T04:11:32.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7121e00640 1 --2- 172.21.15.86:0/646613659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7124072650 0x7f7124072ad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71140097f0 tx=0x7f711401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7646b6e060c6ce server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7121400640 1 -- 172.21.15.86:0/646613659 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140ab620 con 0x7f7124072650 2024-03-22T04:11:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7121400640 1 -- 172.21.15.86:0/646613659 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71140b0070 con 0x7f7124072650 2024-03-22T04:11:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7121400640 1 -- 172.21.15.86:0/646613659 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140ab9e0 con 0x7f7124072650 2024-03-22T04:11:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7129400640 1 -- 172.21.15.86:0/646613659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7124072650 msgr2=0x7f7124072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7129400640 1 --2- 172.21.15.86:0/646613659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7124072650 0x7f7124072ad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71140097f0 tx=0x7f711401c420 comp rx=0 tx=0).stop 2024-03-22T04:11:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7129400640 1 -- 172.21.15.86:0/646613659 shutdown_connections 2024-03-22T04:11:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7129400640 1 --2- 172.21.15.86:0/646613659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7124072650 0x7f7124072ad0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7129400640 1 -- 172.21.15.86:0/646613659 >> 172.21.15.86:0/646613659 conn(0x7f71240cc940 msgr2=0x7f71240ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7129400640 1 -- 172.21.15.86:0/646613659 shutdown_connections 2024-03-22T04:11:32.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.959+0000 7f7129400640 1 -- 172.21.15.86:0/646613659 wait complete. 2024-03-22T04:11:32.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 Processor -- start 2024-03-22T04:11:32.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 -- start start 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712406e7e0 0x7f7124136650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124072650 0x7f71241473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f712415b760 0x7f712414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7124072080 con 0x7f712406e7e0 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7124155aa0 con 0x7f7124072650 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7124134930 con 0x7f712415b760 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712406e7e0 0x7f7124136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712406e7e0 0x7f7124136650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57500/0 (socket says 172.21.15.86:57500) 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 -- 172.21.15.86:0/3459107421 learned_addr learned my addr 172.21.15.86:0/3459107421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7121e00640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124072650 0x7f71241473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7123200640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f712415b760 0x7f712414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 -- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f712415b760 msgr2=0x7f712414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f712415b760 0x7f712414e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 -- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124072650 msgr2=0x7f71241473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124072650 0x7f71241473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 -- 172.21.15.86:0/3459107421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7124134600 con 0x7f712406e7e0 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7123200640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f712415b760 0x7f712414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7121e00640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124072650 0x7f71241473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7122800640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712406e7e0 0x7f7124136650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c3c0 tx=0x7f710c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f711aa00640 1 -- 172.21.15.86:0/3459107421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c0074e0 con 0x7f712406e7e0 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f711aa00640 1 -- 172.21.15.86:0/3459107421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f710c0047f0 con 0x7f712406e7e0 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f711aa00640 1 -- 172.21.15.86:0/3459107421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c0076a0 con 0x7f712406e7e0 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7124136ee0 con 0x7f712406e7e0 2024-03-22T04:11:32.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.963+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241374a0 con 0x7f712406e7e0 2024-03-22T04:11:32.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.967+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7124074470 con 0x7f712406e7e0 2024-03-22T04:11:32.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.967+0000 7f711aa00640 1 -- 172.21.15.86:0/3459107421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f710c01a020 con 0x7f712406e7e0 2024-03-22T04:11:32.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.967+0000 7f711aa00640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7100067b50 0x7f710006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:32.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.967+0000 7f711aa00640 1 -- 172.21.15.86:0/3459107421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f710c085ab0 con 0x7f712406e7e0 2024-03-22T04:11:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.967+0000 7f7121e00640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7100067b50 0x7f710006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:32.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.967+0000 7f7121e00640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7100067b50 0x7f710006a010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71140af000 tx=0x7f71140a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:32.971+0000 7f711aa00640 1 -- 172.21.15.86:0/3459107421 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f710c056070 con 0x7f712406e7e0 2024-03-22T04:11:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7124072200 con 0x7f712406e7e0 2024-03-22T04:11:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f711aa00640 1 -- 172.21.15.86:0/3459107421 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f710c055970 con 0x7f712406e7e0 2024-03-22T04:11:33.132 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:33.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7100067b50 msgr2=0x7f710006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f7129400640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7100067b50 0x7f710006a010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71140af000 tx=0x7f71140a7000 comp rx=0 tx=0).stop 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712406e7e0 msgr2=0x7f7124136650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f7129400640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712406e7e0 0x7f7124136650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c3c0 tx=0x7f710c00c880 comp rx=0 tx=0).stop 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 shutdown_connections 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f7129400640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7100067b50 0x7f710006a010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.127+0000 7f7129400640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124072650 0x7f71241473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.131+0000 7f7129400640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f712415b760 0x7f712414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.131+0000 7f7129400640 1 --2- 172.21.15.86:0/3459107421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712406e7e0 0x7f7124136650 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.131+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 >> 172.21.15.86:0/3459107421 conn(0x7f71240cc940 msgr2=0x7f71240ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.131+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 shutdown_connections 2024-03-22T04:11:33.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.131+0000 7f7129400640 1 -- 172.21.15.86:0/3459107421 wait complete. 2024-03-22T04:11:33.152 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff696600640 1 Processor -- start 2024-03-22T04:11:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff696600640 1 -- start start 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff696600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff69006e7e0 0x7ff69006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff696600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 0x7ff690072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff696600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6900b1b80 0x7ff6900b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff696600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff690071350 con 0x7ff6900b1b80 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff696600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff69006f100 con 0x7ff69006e7e0 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff696600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff690073150 con 0x7ff690072650 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6900b1b80 0x7ff6900b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff69006e7e0 0x7ff69006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6900b1b80 0x7ff6900b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57506/0 (socket says 172.21.15.86:57506) 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68fe00640 1 -- 172.21.15.86:0/3253248100 learned_addr learned my addr 172.21.15.86:0/3253248100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff687e00640 1 --2- 172.21.15.86:0/3253248100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 0x7ff690072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68fe00640 1 -- 172.21.15.86:0/3253248100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 msgr2=0x7ff690072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68fe00640 1 --2- 172.21.15.86:0/3253248100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 0x7ff690072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68fe00640 1 -- 172.21.15.86:0/3253248100 >> v2:172.21.15.196:3300/0 conn(0x7ff69006e7e0 msgr2=0x7ff69006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68fe00640 1 --2- 172.21.15.86:0/3253248100 >> v2:172.21.15.196:3300/0 conn(0x7ff69006e7e0 0x7ff69006ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.287+0000 7ff68fe00640 1 -- 172.21.15.86:0/3253248100 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6900715a0 con 0x7ff6900b1b80 2024-03-22T04:11:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff68fe00640 1 --2- 172.21.15.86:0/3253248100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6900b1b80 0x7ff6900b2010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6900ce650 tx=0x7ff68801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6383b532a9cce66b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff68ea00640 1 -- 172.21.15.86:0/3253248100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6880a55d0 con 0x7ff6900b1b80 2024-03-22T04:11:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff68ea00640 1 -- 172.21.15.86:0/3253248100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6880a5770 con 0x7ff6900b1b80 2024-03-22T04:11:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff68ea00640 1 -- 172.21.15.86:0/3253248100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6880a5b40 con 0x7ff6900b1b80 2024-03-22T04:11:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 -- 172.21.15.86:0/3253248100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6900b1b80 msgr2=0x7ff6900b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 --2- 172.21.15.86:0/3253248100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6900b1b80 0x7ff6900b2010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6900ce650 tx=0x7ff68801c750 comp rx=0 tx=0).stop 2024-03-22T04:11:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 -- 172.21.15.86:0/3253248100 shutdown_connections 2024-03-22T04:11:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 --2- 172.21.15.86:0/3253248100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6900b1b80 0x7ff6900b2010 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 --2- 172.21.15.86:0/3253248100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 0x7ff690072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 --2- 172.21.15.86:0/3253248100 >> v2:172.21.15.196:3300/0 conn(0x7ff69006e7e0 0x7ff69006ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 -- 172.21.15.86:0/3253248100 >> 172.21.15.86:0/3253248100 conn(0x7ff6900cc940 msgr2=0x7ff6900ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 -- 172.21.15.86:0/3253248100 shutdown_connections 2024-03-22T04:11:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.291+0000 7ff696600640 1 -- 172.21.15.86:0/3253248100 wait complete. 2024-03-22T04:11:33.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 Processor -- start 2024-03-22T04:11:33.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 -- start start 2024-03-22T04:11:33.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff69006e7e0 0x7ff690148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 0x7ff69014aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6900b1b80 0x7ff69014f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff690072080 con 0x7ff69006e7e0 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6900ce930 con 0x7ff6900b1b80 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff690149020 con 0x7ff690072650 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff69006e7e0 0x7ff690148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff69006e7e0 0x7ff690148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57508/0 (socket says 172.21.15.86:57508) 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 -- 172.21.15.86:0/1055794072 learned_addr learned my addr 172.21.15.86:0/1055794072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff687e00640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 0x7ff69014aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68fe00640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6900b1b80 0x7ff69014f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 -- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 msgr2=0x7ff69014aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 0x7ff69014aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 -- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6900b1b80 msgr2=0x7ff69014f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6900b1b80 0x7ff69014f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 -- 172.21.15.86:0/1055794072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69014afc0 con 0x7ff69006e7e0 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68fe00640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6900b1b80 0x7ff69014f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff68f400640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff69006e7e0 0x7ff690148ae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff68000e870 tx=0x7ff68000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff687400640 1 -- 172.21.15.86:0/1055794072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680017070 con 0x7ff69006e7e0 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff687400640 1 -- 172.21.15.86:0/1055794072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68000ee60 con 0x7ff69006e7e0 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6901f84b0 con 0x7ff69006e7e0 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff687400640 1 -- 172.21.15.86:0/1055794072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68000cb80 con 0x7ff69006e7e0 2024-03-22T04:11:33.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff690148000 con 0x7ff69006e7e0 2024-03-22T04:11:33.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.295+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff690074470 con 0x7ff69006e7e0 2024-03-22T04:11:33.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.299+0000 7ff687400640 1 -- 172.21.15.86:0/1055794072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff68001a070 con 0x7ff69006e7e0 2024-03-22T04:11:33.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.299+0000 7ff687400640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff66c067b20 0x7ff66c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.299+0000 7ff687400640 1 -- 172.21.15.86:0/1055794072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff680086490 con 0x7ff69006e7e0 2024-03-22T04:11:33.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.299+0000 7ff687e00640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff66c067b20 0x7ff66c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.299+0000 7ff687e00640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff66c067b20 0x7ff66c069fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff690150370 tx=0x7ff678009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.303+0000 7ff687400640 1 -- 172.21.15.86:0/1055794072 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff680056400 con 0x7ff69006e7e0 2024-03-22T04:11:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.459+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff69006e110 con 0x7ff69006e7e0 2024-03-22T04:11:33.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.459+0000 7ff687400640 1 -- 172.21.15.86:0/1055794072 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7ff68005a4a0 con 0x7ff69006e7e0 2024-03-22T04:11:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff66c067b20 msgr2=0x7ff66c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff66c067b20 0x7ff66c069fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff690150370 tx=0x7ff678009190 comp rx=0 tx=0).stop 2024-03-22T04:11:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff69006e7e0 msgr2=0x7ff690148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff69006e7e0 0x7ff690148ae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff68000e870 tx=0x7ff68000ed30 comp rx=0 tx=0).stop 2024-03-22T04:11:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 shutdown_connections 2024-03-22T04:11:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff66c067b20 0x7ff66c069fe0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6900b1b80 0x7ff69014f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff690072650 0x7ff69014aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 --2- 172.21.15.86:0/1055794072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff69006e7e0 0x7ff690148ae0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 >> 172.21.15.86:0/1055794072 conn(0x7ff6900cc940 msgr2=0x7ff6900cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 shutdown_connections 2024-03-22T04:11:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.463+0000 7ff696600640 1 -- 172.21.15.86:0/1055794072 wait complete. 2024-03-22T04:11:33.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a7e00640 1 Processor -- start 2024-03-22T04:11:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a7e00640 1 -- start start 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe2a015e7e0 0x7fe2a00d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a00b4710 0x7fe2a006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a7e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a00d15f0 con 0x7fe2a0070390 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a7e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a00b4b70 con 0x7fe2a015e7e0 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a7e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a006f970 con 0x7fe2a00b4710 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2a0070390 0x7fe2a009aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52768/0 (socket says 172.21.15.86:52768) 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a5e00640 1 -- 172.21.15.86:0/4133061046 learned_addr learned my addr 172.21.15.86:0/4133061046 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a5400640 1 --2- 172.21.15.86:0/4133061046 >> v2:172.21.15.196:3300/0 conn(0x7fe2a015e7e0 0x7fe2a00d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a00b4710 0x7fe2a006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe29fe00640 1 -- 172.21.15.86:0/4133061046 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971593488 0 0) 0x7fe2a00d15f0 con 0x7fe2a0070390 2024-03-22T04:11:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe29fe00640 1 -- 172.21.15.86:0/4133061046 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe288003530 con 0x7fe2a0070390 2024-03-22T04:11:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a5400640 1 -- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a00b4710 msgr2=0x7fe2a006fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a5400640 1 --2- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a00b4710 0x7fe2a006fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a5400640 1 -- 172.21.15.86:0/4133061046 >> v1:172.21.15.86:6789/0 conn(0x7fe2a0070390 legacy=0x7fe2a009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a5400640 1 -- 172.21.15.86:0/4133061046 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a0070820 con 0x7fe2a015e7e0 2024-03-22T04:11:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.619+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a00b4710 0x7fe2a006fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:33.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a5400640 1 --2- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e7e0 0x7fe2a00d0d30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2940097c0 tx=0x7fe29401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed597245619252c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe29fe00640 1 -- 172.21.15.86:0/4133061046 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2940b0070 con 0x7fe2a015e7e0 2024-03-22T04:11:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe29fe00640 1 -- 172.21.15.86:0/4133061046 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2940ab600 con 0x7fe2a015e7e0 2024-03-22T04:11:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe29fe00640 1 -- 172.21.15.86:0/4133061046 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2940ab9b0 con 0x7fe2a015e7e0 2024-03-22T04:11:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a7e00640 1 -- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e7e0 msgr2=0x7fe2a00d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e7e0 0x7fe2a00d0d30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2940097c0 tx=0x7fe29401c350 comp rx=0 tx=0).stop 2024-03-22T04:11:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a7e00640 1 -- 172.21.15.86:0/4133061046 shutdown_connections 2024-03-22T04:11:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a00b4710 0x7fe2a006fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/4133061046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e7e0 0x7fe2a00d0d30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a7e00640 1 -- 172.21.15.86:0/4133061046 >> 172.21.15.86:0/4133061046 conn(0x7fe2a00b3bf0 msgr2=0x7fe2a00b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a7e00640 1 -- 172.21.15.86:0/4133061046 shutdown_connections 2024-03-22T04:11:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.623+0000 7fe2a7e00640 1 -- 172.21.15.86:0/4133061046 wait complete. 2024-03-22T04:11:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 Processor -- start 2024-03-22T04:11:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 -- start start 2024-03-22T04:11:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a0070390 0x7fe2a00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a00b4710 0x7fe2a00b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a015e7e0 0x7fe2a0154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a00ce370 con 0x7fe2a00b4710 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a00cce40 con 0x7fe2a0070390 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a00ce4f0 con 0x7fe2a015e7e0 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a00b4710 0x7fe2a00b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a00b4710 0x7fe2a00b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57518/0 (socket says 172.21.15.86:57518) 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 -- 172.21.15.86:0/6254563 learned_addr learned my addr 172.21.15.86:0/6254563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a5400640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a0070390 0x7fe2a00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a5e00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a015e7e0 0x7fe2a0154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 -- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a015e7e0 msgr2=0x7fe2a0154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a015e7e0 0x7fe2a0154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 -- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a0070390 msgr2=0x7fe2a00b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a0070390 0x7fe2a00b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 -- 172.21.15.86:0/6254563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a0155510 con 0x7fe2a00b4710 2024-03-22T04:11:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a5400640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a0070390 0x7fe2a00b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a00b4710 0x7fe2a00b0680 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00e980 tx=0x7fe28c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe29d600640 1 -- 172.21.15.86:0/6254563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c017070 con 0x7fe2a00b4710 2024-03-22T04:11:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe29d600640 1 -- 172.21.15.86:0/6254563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28c00ca00 con 0x7fe2a00b4710 2024-03-22T04:11:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a01557f0 con 0x7fe2a00b4710 2024-03-22T04:11:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe29d600640 1 -- 172.21.15.86:0/6254563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00cdd0 con 0x7fe2a00b4710 2024-03-22T04:11:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.627+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a0155db0 con 0x7fe2a00b4710 2024-03-22T04:11:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.631+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a0074470 con 0x7fe2a00b4710 2024-03-22T04:11:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.631+0000 7fe29d600640 1 -- 172.21.15.86:0/6254563 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe28c01a070 con 0x7fe2a00b4710 2024-03-22T04:11:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.631+0000 7fe29d600640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.631+0000 7fe29d600640 1 -- 172.21.15.86:0/6254563 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fe28c086480 con 0x7fe2a00b4710 2024-03-22T04:11:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.631+0000 7fe2a5400640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.631+0000 7fe2a5400640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2940ad000 tx=0x7fe2940a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.635+0000 7fe29d600640 1 -- 172.21.15.86:0/6254563 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe28c056480 con 0x7fe2a00b4710 2024-03-22T04:11:33.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.787+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe2a006fc30 con 0x7fe2a00b4710 2024-03-22T04:11:33.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe29d600640 1 -- 172.21.15.86:0/6254563 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fe28c05a520 con 0x7fe2a00b4710 2024-03-22T04:11:33.795 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 msgr2=0x7fe280069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2940ad000 tx=0x7fe2940a6000 comp rx=0 tx=0).stop 2024-03-22T04:11:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a00b4710 msgr2=0x7fe2a00b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a00b4710 0x7fe2a00b0680 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00e980 tx=0x7fe28c00ee40 comp rx=0 tx=0).stop 2024-03-22T04:11:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 shutdown_connections 2024-03-22T04:11:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a015e7e0 0x7fe2a0154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a00b4710 0x7fe2a00b0680 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/6254563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a0070390 0x7fe2a00b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 >> 172.21.15.86:0/6254563 conn(0x7fe2a00b3bf0 msgr2=0x7fe2a006e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 shutdown_connections 2024-03-22T04:11:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.791+0000 7fe2a7e00640 1 -- 172.21.15.86:0/6254563 wait complete. 2024-03-22T04:11:33.816 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:11:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.947+0000 7f1d52600640 1 Processor -- start 2024-03-22T04:11:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.947+0000 7f1d52600640 1 -- start start 2024-03-22T04:11:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d52600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d4c0730d0 0x7f1d4c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d52600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d4c071920 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d52600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d4c071c00 con 0x7f1d4c15b3f0 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d52600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1d4c0716d0 con 0x7f1d4c0730d0 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d42a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d4c0730d0 0x7f1d4c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d4be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1d4c0cc3e0 0x7f1d4c0b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52772/0 (socket says 172.21.15.86:52772) 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d4be00640 1 -- 172.21.15.86:0/1884188422 learned_addr learned my addr 172.21.15.86:0/1884188422 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d42a00640 1 -- 172.21.15.86:0/1884188422 >> v1:172.21.15.196:6789/0 conn(0x7f1d4c15b3f0 legacy=0x7f1d4c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d42a00640 1 -- 172.21.15.86:0/1884188422 >> v1:172.21.15.86:6789/0 conn(0x7f1d4c0cc3e0 legacy=0x7f1d4c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.951+0000 7f1d42a00640 1 -- 172.21.15.86:0/1884188422 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c071ee0 con 0x7f1d4c0730d0 2024-03-22T04:11:33.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d42a00640 1 --2- 172.21.15.86:0/1884188422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0730d0 0x7f1d4c0ade30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1d340097c0 tx=0x7f1d3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b4a78e5ff8c3e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d4aa00640 1 -- 172.21.15.86:0/1884188422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340b0070 con 0x7f1d4c0730d0 2024-03-22T04:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d4aa00640 1 -- 172.21.15.86:0/1884188422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d340ab600 con 0x7f1d4c0730d0 2024-03-22T04:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d4aa00640 1 -- 172.21.15.86:0/1884188422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340ab9d0 con 0x7f1d4c0730d0 2024-03-22T04:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- 172.21.15.86:0/1884188422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0730d0 msgr2=0x7f1d4c0ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 --2- 172.21.15.86:0/1884188422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0730d0 0x7f1d4c0ade30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1d340097c0 tx=0x7f1d3401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- 172.21.15.86:0/1884188422 shutdown_connections 2024-03-22T04:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 --2- 172.21.15.86:0/1884188422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0730d0 0x7f1d4c0ade30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- 172.21.15.86:0/1884188422 >> 172.21.15.86:0/1884188422 conn(0x7f1d4c0b1df0 msgr2=0x7f1d4c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- 172.21.15.86:0/1884188422 shutdown_connections 2024-03-22T04:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- 172.21.15.86:0/1884188422 wait complete. 2024-03-22T04:11:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 Processor -- start 2024-03-22T04:11:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- start start 2024-03-22T04:11:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d4c0730d0 0x7f1d4c147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cc3e0 0x7f1d4c158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d4c15b3f0 0x7f1d4c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c0b30f0 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c0b3270 con 0x7f1d4c15b3f0 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.955+0000 7f1d52600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c0b33f0 con 0x7f1d4c0730d0 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cc3e0 0x7f1d4c158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cc3e0 0x7f1d4c158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57524/0 (socket says 172.21.15.86:57524) 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 -- 172.21.15.86:0/2464944661 learned_addr learned my addr 172.21.15.86:0/2464944661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d4be00640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d4c15b3f0 0x7f1d4c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d4b400640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d4c0730d0 0x7f1d4c147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 -- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d4c0730d0 msgr2=0x7f1d4c147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d4c0730d0 0x7f1d4c147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 -- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d4c15b3f0 msgr2=0x7f1d4c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d4c15b3f0 0x7f1d4c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 -- 172.21.15.86:0/2464944661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c14d960 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d4b400640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d4c0730d0 0x7f1d4c147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d42a00640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cc3e0 0x7f1d4c158390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d340ae040 tx=0x7f1d34019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d43e00640 1 -- 172.21.15.86:0/2464944661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340b0040 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d43e00640 1 -- 172.21.15.86:0/2464944661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d340b9070 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d43e00640 1 -- 172.21.15.86:0/2464944661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340aabb0 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c14cba0 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c14d160 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c068650 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d43e00640 1 -- 172.21.15.86:0/2464944661 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1d340ab4c0 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.959+0000 7f1d43e00640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1d28067b20 0x7f1d28069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.963+0000 7f1d43e00640 1 -- 172.21.15.86:0/2464944661 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f1d341203c0 con 0x7f1d4c0cc3e0 2024-03-22T04:11:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.963+0000 7f1d4b400640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1d28067b20 0x7f1d28069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.963+0000 7f1d4b400640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1d28067b20 0x7f1d28069fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c09e0f0 tx=0x7f1d3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:33.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:33.967+0000 7f1d43e00640 1 -- 172.21.15.86:0/2464944661 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1d34016090 con 0x7f1d4c0cc3e0 2024-03-22T04:11:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.131+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d4c157260 con 0x7f1d4c0cc3e0 2024-03-22T04:11:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.131+0000 7f1d43e00640 1 -- 172.21.15.86:0/2464944661 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f1d340f0a10 con 0x7f1d4c0cc3e0 2024-03-22T04:11:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1d28067b20 msgr2=0x7f1d28069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1d28067b20 0x7f1d28069fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c09e0f0 tx=0x7f1d3c009040 comp rx=0 tx=0).stop 2024-03-22T04:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cc3e0 msgr2=0x7f1d4c158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cc3e0 0x7f1d4c158390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d340ae040 tx=0x7f1d34019240 comp rx=0 tx=0).stop 2024-03-22T04:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 shutdown_connections 2024-03-22T04:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1d28067b20 0x7f1d28069fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d4c15b3f0 0x7f1d4c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cc3e0 0x7f1d4c158390 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 --2- 172.21.15.86:0/2464944661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d4c0730d0 0x7f1d4c147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 >> 172.21.15.86:0/2464944661 conn(0x7f1d4c0b1df0 msgr2=0x7f1d4c0aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 shutdown_connections 2024-03-22T04:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.135+0000 7f1d52600640 1 -- 172.21.15.86:0/2464944661 wait complete. 2024-03-22T04:11:34.157 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-22T04:11:34.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:34.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-22T04:11:34.180 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:11:34.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:34.180 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-22T04:11:34.230 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:11:34.231 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:34.239 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:11:34.240 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:34.289 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:11:34.289 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:11:34.337 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.467+0000 7fdbfd200640 1 Processor -- start 2024-03-22T04:11:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.467+0000 7fdbfd200640 1 -- start start 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.467+0000 7fdbfd200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.467+0000 7fdbfd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80ad710 0x7fdbf815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.467+0000 7fdbfd200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf815c7c0 con 0x7fdbf80ad330 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbfd200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf806dea0 con 0x7fdbf80ad710 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbfd200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf806e180 con 0x7fdbf806ef40 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80ad710 0x7fdbf815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80ad710 0x7fdbf815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57536/0 (socket says 172.21.15.86:57536) 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf6800640 1 -- 172.21.15.86:0/3768538516 learned_addr learned my addr 172.21.15.86:0/3768538516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbed400640 1 --2- 172.21.15.86:0/3768538516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf6800640 1 -- 172.21.15.86:0/3768538516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 msgr2=0x7fdbf8142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf6800640 1 --2- 172.21.15.86:0/3768538516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf6800640 1 -- 172.21.15.86:0/3768538516 >> v1:172.21.15.196:6789/0 conn(0x7fdbf80ad330 legacy=0x7fdbf813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf6800640 1 -- 172.21.15.86:0/3768538516 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf806e460 con 0x7fdbf80ad710 2024-03-22T04:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf6800640 1 --2- 172.21.15.86:0/3768538516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815d4a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009e50 tx=0x7fdbf001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d8aab5dbff4047 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf5400640 1 -- 172.21.15.86:0/3768538516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00a58d0 con 0x7fdbf80ad710 2024-03-22T04:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf5400640 1 -- 172.21.15.86:0/3768538516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf00a5a70 con 0x7fdbf80ad710 2024-03-22T04:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.471+0000 7fdbf5400640 1 -- 172.21.15.86:0/3768538516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00a5e40 con 0x7fdbf80ad710 2024-03-22T04:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- 172.21.15.86:0/3768538516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 msgr2=0x7fdbf815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 --2- 172.21.15.86:0/3768538516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815d4a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009e50 tx=0x7fdbf001ca00 comp rx=0 tx=0).stop 2024-03-22T04:11:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- 172.21.15.86:0/3768538516 shutdown_connections 2024-03-22T04:11:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 --2- 172.21.15.86:0/3768538516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 --2- 172.21.15.86:0/3768538516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815d4a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- 172.21.15.86:0/3768538516 >> 172.21.15.86:0/3768538516 conn(0x7fdbf80b1df0 msgr2=0x7fdbf80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- 172.21.15.86:0/3768538516 shutdown_connections 2024-03-22T04:11:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- 172.21.15.86:0/3768538516 wait complete. 2024-03-22T04:11:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 Processor -- start 2024-03-22T04:11:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- start start 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80ad330 0x7fdbf815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81690e0 con 0x7fdbf80ad710 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf8168f60 con 0x7fdbf80ad330 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbfd200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf8169260 con 0x7fdbf806ef40 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8099ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42416/0 (socket says 172.21.15.86:42416) 2024-03-22T04:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57548/0 (socket says 172.21.15.86:57548) 2024-03-22T04:11:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf5e00640 1 -- 172.21.15.86:0/542223623 learned_addr learned my addr 172.21.15.86:0/542223623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbed400640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80ad330 0x7fdbf815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf6800640 1 -- 172.21.15.86:0/542223623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 msgr2=0x7fdbf8099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf6800640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8099ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf6800640 1 -- 172.21.15.86:0/542223623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80ad330 msgr2=0x7fdbf815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf6800640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80ad330 0x7fdbf815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.475+0000 7fdbf6800640 1 -- 172.21.15.86:0/542223623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf80cfab0 con 0x7fdbf80ad710 2024-03-22T04:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbf6800640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815e8f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00ab000 tx=0x7fdbf00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbeea00640 1 -- 172.21.15.86:0/542223623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00a58d0 con 0x7fdbf80ad710 2024-03-22T04:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbeea00640 1 -- 172.21.15.86:0/542223623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf00090d0 con 0x7fdbf80ad710 2024-03-22T04:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbeea00640 1 -- 172.21.15.86:0/542223623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0009510 con 0x7fdbf80ad710 2024-03-22T04:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf8072170 con 0x7fdbf80ad710 2024-03-22T04:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf815f510 con 0x7fdbf80ad710 2024-03-22T04:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf8068650 con 0x7fdbf80ad710 2024-03-22T04:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbeea00640 1 -- 172.21.15.86:0/542223623 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdbf00c0020 con 0x7fdbf80ad710 2024-03-22T04:11:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbeea00640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd4067b20 0x7fdbd4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbeea00640 1 -- 172.21.15.86:0/542223623 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fdbf0120ff0 con 0x7fdbf80ad710 2024-03-22T04:11:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.479+0000 7fdbf5e00640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd4067b20 0x7fdbd4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.483+0000 7fdbf5e00640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd4067b20 0x7fdbd4069fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8007fd0 tx=0x7fdbe8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:34.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.483+0000 7fdbeea00640 1 -- 172.21.15.86:0/542223623 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdbf00c7020 con 0x7fdbf80ad710 2024-03-22T04:11:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.643+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbf806c670 con 0x7fdbf80ad710 2024-03-22T04:11:34.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.643+0000 7fdbeea00640 1 -- 172.21.15.86:0/542223623 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7fdbf0016620 con 0x7fdbf80ad710 2024-03-22T04:11:34.649 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:34.650 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd4067b20 msgr2=0x7fdbd4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd4067b20 0x7fdbd4069fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8007fd0 tx=0x7fdbe8009040 comp rx=0 tx=0).stop 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 msgr2=0x7fdbf815e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815e8f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00ab000 tx=0x7fdbf00197a0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 shutdown_connections 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdbd4067b20 0x7fdbd4069fe0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbf80ad330 0x7fdbf815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80ad710 0x7fdbf815e8f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 --2- 172.21.15.86:0/542223623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbf806ef40 0x7fdbf8099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 >> 172.21.15.86:0/542223623 conn(0x7fdbf80b1df0 msgr2=0x7fdbf8141d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 shutdown_connections 2024-03-22T04:11:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.647+0000 7fdbfd200640 1 -- 172.21.15.86:0/542223623 wait complete. 2024-03-22T04:11:34.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:34.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50ee400640 1 Processor -- start 2024-03-22T04:11:34.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50ee400640 1 -- start start 2024-03-22T04:11:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50ee400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f50e8072b00 0x7f50e815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50ee400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50e8073150 con 0x7f50e8072650 2024-03-22T04:11:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50ee400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50e80715a0 con 0x7f50e806e7e0 2024-03-22T04:11:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50ee400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f50e8071350 con 0x7f50e8072b00 2024-03-22T04:11:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50e6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f50e8072650 0x7f50e815b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52786/0 (socket says 172.21.15.86:52786) 2024-03-22T04:11:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50e6a00640 1 -- 172.21.15.86:0/74275163 learned_addr learned my addr 172.21.15.86:0/74275163 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50e7e00640 1 --2- 172.21.15.86:0/74275163 >> v2:172.21.15.86:3300/0 conn(0x7f50e8072b00 0x7f50e815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50e7e00640 1 -- 172.21.15.86:0/74275163 >> v1:172.21.15.196:6789/0 conn(0x7f50e806e7e0 legacy=0x7f50e806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50e7e00640 1 -- 172.21.15.86:0/74275163 >> v1:172.21.15.86:6789/0 conn(0x7f50e8072650 legacy=0x7f50e815b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50e7e00640 1 -- 172.21.15.86:0/74275163 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e8071880 con 0x7f50e8072b00 2024-03-22T04:11:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.807+0000 7f50e6000640 1 -- 172.21.15.86:0/74275163 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790533005 0 0) 0x7f50e80715a0 con 0x7f50e806e7e0 2024-03-22T04:11:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50e7e00640 1 --2- 172.21.15.86:0/74275163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e8072b00 0x7f50e815be70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009d90 tx=0x7f50e001c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ff8db799658a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50e6000640 1 -- 172.21.15.86:0/74275163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e00a66d0 con 0x7f50e8072b00 2024-03-22T04:11:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50e6000640 1 -- 172.21.15.86:0/74275163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e00a6870 con 0x7f50e8072b00 2024-03-22T04:11:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50e6000640 1 -- 172.21.15.86:0/74275163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e00a6c40 con 0x7f50e8072b00 2024-03-22T04:11:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 -- 172.21.15.86:0/74275163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e8072b00 msgr2=0x7f50e815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 --2- 172.21.15.86:0/74275163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e8072b00 0x7f50e815be70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009d90 tx=0x7f50e001c770 comp rx=0 tx=0).stop 2024-03-22T04:11:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 -- 172.21.15.86:0/74275163 shutdown_connections 2024-03-22T04:11:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 --2- 172.21.15.86:0/74275163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e8072b00 0x7f50e815be70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 -- 172.21.15.86:0/74275163 >> 172.21.15.86:0/74275163 conn(0x7f50e80cc940 msgr2=0x7f50e80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 -- 172.21.15.86:0/74275163 shutdown_connections 2024-03-22T04:11:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 -- 172.21.15.86:0/74275163 wait complete. 2024-03-22T04:11:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 Processor -- start 2024-03-22T04:11:34.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.811+0000 7f50ee400640 1 -- start start 2024-03-22T04:11:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e806e7e0 0x7f50e8169fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50e8072650 0x7f50e8154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8072b00 0x7f50e814e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50e8155c20 con 0x7f50e806e7e0 2024-03-22T04:11:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50e8155aa0 con 0x7f50e8072b00 2024-03-22T04:11:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f50e806d6d0 con 0x7f50e8072650 2024-03-22T04:11:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e806e7e0 0x7f50e8169fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e806e7e0 0x7f50e8169fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57562/0 (socket says 172.21.15.86:57562) 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 -- 172.21.15.86:0/3446589781 learned_addr learned my addr 172.21.15.86:0/3446589781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7e00640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8072b00 0x7f50e814e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e6a00640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50e8072650 0x7f50e8154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 -- 172.21.15.86:0/3446589781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50e8072650 msgr2=0x7f50e8154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50e8072650 0x7f50e8154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 -- 172.21.15.86:0/3446589781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8072b00 msgr2=0x7f50e814e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8072b00 0x7f50e814e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 -- 172.21.15.86:0/3446589781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e816a750 con 0x7f50e806e7e0 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50e7400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e806e7e0 0x7f50e8169fd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c4d0 tx=0x7f50d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50df400640 1 -- 172.21.15.86:0/3446589781 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8017070 con 0x7f50e806e7e0 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50df400640 1 -- 172.21.15.86:0/3446589781 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d8005480 con 0x7f50e806e7e0 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e816aa30 con 0x7f50e806e7e0 2024-03-22T04:11:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50df400640 1 -- 172.21.15.86:0/3446589781 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8003840 con 0x7f50e806e7e0 2024-03-22T04:11:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e81545c0 con 0x7f50e806e7e0 2024-03-22T04:11:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e813a720 con 0x7f50e806e7e0 2024-03-22T04:11:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.815+0000 7f50df400640 1 -- 172.21.15.86:0/3446589781 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f50d801a070 con 0x7f50e806e7e0 2024-03-22T04:11:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.819+0000 7f50df400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50c8067b20 0x7f50c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.819+0000 7f50df400640 1 -- 172.21.15.86:0/3446589781 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f50d8085590 con 0x7f50e806e7e0 2024-03-22T04:11:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.819+0000 7f50e6a00640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50c8067b20 0x7f50c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.819+0000 7f50e6a00640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50c8067b20 0x7f50c8069fe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50d4000f70 tx=0x7f50d4007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:34.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.823+0000 7f50df400640 1 -- 172.21.15.86:0/3446589781 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f50e813a720 con 0x7f50e806e7e0 2024-03-22T04:11:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50e816aee0 con 0x7f50e806e7e0 2024-03-22T04:11:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50df400640 1 -- 172.21.15.86:0/3446589781 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f50d80556f0 con 0x7f50e806e7e0 2024-03-22T04:11:34.996 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50c8067b20 msgr2=0x7f50c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50c8067b20 0x7f50c8069fe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50d4000f70 tx=0x7f50d4007e30 comp rx=0 tx=0).stop 2024-03-22T04:11:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e806e7e0 msgr2=0x7f50e8169fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e806e7e0 0x7f50e8169fd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c4d0 tx=0x7f50d800c990 comp rx=0 tx=0).stop 2024-03-22T04:11:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 shutdown_connections 2024-03-22T04:11:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50c8067b20 0x7f50c8069fe0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50e8072b00 0x7f50e814e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50e8072650 0x7f50e8154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 --2- 172.21.15.86:0/3446589781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50e806e7e0 0x7f50e8169fd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 >> 172.21.15.86:0/3446589781 conn(0x7f50e80cc940 msgr2=0x7f50e815a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 shutdown_connections 2024-03-22T04:11:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:34.991+0000 7f50ee400640 1 -- 172.21.15.86:0/3446589781 wait complete. 2024-03-22T04:11:35.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:35.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0f600640 1 Processor -- start 2024-03-22T04:11:35.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0f600640 1 -- start start 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0f600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda0806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0f600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fda08072650 0x7fda08072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0f600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0815c770 con 0x7fda080b1b80 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0f600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fda08071350 con 0x7fda08072650 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0f600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fda0806f100 con 0x7fda0806e7e0 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0cc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda0806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0cc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda0806ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42440/0 (socket says 172.21.15.86:42440) 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda0cc00640 1 -- 172.21.15.86:0/1396332391 learned_addr learned my addr 172.21.15.86:0/1396332391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:35.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.151+0000 7fda07e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fda08072650 0x7fda08072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda0cc00640 1 -- 172.21.15.86:0/1396332391 >> v2:172.21.15.196:3300/0 conn(0x7fda08072650 msgr2=0x7fda08072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:35.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda0cc00640 1 --2- 172.21.15.86:0/1396332391 >> v2:172.21.15.196:3300/0 conn(0x7fda08072650 0x7fda08072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda0cc00640 1 -- 172.21.15.86:0/1396332391 >> v1:172.21.15.196:6789/0 conn(0x7fda080b1b80 legacy=0x7fda080b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda0cc00640 1 -- 172.21.15.86:0/1396332391 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda08073150 con 0x7fda0806e7e0 2024-03-22T04:11:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda0cc00640 1 --2- 172.21.15.86:0/1396332391 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda0806ebc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800bb10 tx=0x7fd9f801eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be21b2f6346e123 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda07400640 1 -- 172.21.15.86:0/1396332391 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80b4070 con 0x7fda0806e7e0 2024-03-22T04:11:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda07400640 1 -- 172.21.15.86:0/1396332391 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f80a7b40 con 0x7fda0806e7e0 2024-03-22T04:11:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda07400640 1 -- 172.21.15.86:0/1396332391 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80af440 con 0x7fda0806e7e0 2024-03-22T04:11:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda0f600640 1 -- 172.21.15.86:0/1396332391 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 msgr2=0x7fda0806ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.155+0000 7fda0f600640 1 --2- 172.21.15.86:0/1396332391 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda0806ebc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800bb10 tx=0x7fd9f801eb70 comp rx=0 tx=0).stop 2024-03-22T04:11:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 -- 172.21.15.86:0/1396332391 shutdown_connections 2024-03-22T04:11:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 --2- 172.21.15.86:0/1396332391 >> v2:172.21.15.196:3300/0 conn(0x7fda08072650 0x7fda08072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 --2- 172.21.15.86:0/1396332391 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda0806ebc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 -- 172.21.15.86:0/1396332391 >> 172.21.15.86:0/1396332391 conn(0x7fda080cc940 msgr2=0x7fda080ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 -- 172.21.15.86:0/1396332391 shutdown_connections 2024-03-22T04:11:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 -- 172.21.15.86:0/1396332391 wait complete. 2024-03-22T04:11:35.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 Processor -- start 2024-03-22T04:11:35.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 -- start start 2024-03-22T04:11:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda080cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda08072650 0x7fda08158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda080b1b80 0x7fda08152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda08072080 con 0x7fda08072650 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda080ce930 con 0x7fda080b1b80 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0f600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fda08153020 con 0x7fda0806e7e0 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda08072650 0x7fda08158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda08072650 0x7fda08158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57568/0 (socket says 172.21.15.86:57568) 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 -- 172.21.15.86:0/570361946 learned_addr learned my addr 172.21.15.86:0/570361946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0cc00640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda080cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda0d600640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda080b1b80 0x7fda08152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 -- 172.21.15.86:0/570361946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 msgr2=0x7fda080cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda080cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 -- 172.21.15.86:0/570361946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda080b1b80 msgr2=0x7fda08152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda080b1b80 0x7fda08152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 -- 172.21.15.86:0/570361946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda080c5c60 con 0x7fda08072650 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.159+0000 7fda07e00640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda08072650 0x7fda08158280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000e870 tx=0x7fd9f000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda04c00640 1 -- 172.21.15.86:0/570361946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0017070 con 0x7fda08072650 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda04c00640 1 -- 172.21.15.86:0/570361946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f000c940 con 0x7fda08072650 2024-03-22T04:11:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda04c00640 1 -- 172.21.15.86:0/570361946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000ccf0 con 0x7fda08072650 2024-03-22T04:11:35.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda081574c0 con 0x7fda08072650 2024-03-22T04:11:35.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda08157a80 con 0x7fda08072650 2024-03-22T04:11:35.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda08074470 con 0x7fda08072650 2024-03-22T04:11:35.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda04c00640 1 -- 172.21.15.86:0/570361946 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd9f001a070 con 0x7fda08072650 2024-03-22T04:11:35.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda04c00640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd9e4067b20 0x7fd9e4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda04c00640 1 -- 172.21.15.86:0/570361946 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fd9f0086120 con 0x7fda08072650 2024-03-22T04:11:35.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda0cc00640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd9e4067b20 0x7fd9e4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.163+0000 7fda0cc00640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd9e4067b20 0x7fd9e4069fe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80b1000 tx=0x7fd9f80a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.167+0000 7fda04c00640 1 -- 172.21.15.86:0/570361946 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd9f0056120 con 0x7fda08072650 2024-03-22T04:11:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.319+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fda0815e960 con 0x7fda08072650 2024-03-22T04:11:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda04c00640 1 -- 172.21.15.86:0/570361946 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fd9f005a1c0 con 0x7fda08072650 2024-03-22T04:11:35.327 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd9e4067b20 msgr2=0x7fd9e4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd9e4067b20 0x7fd9e4069fe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80b1000 tx=0x7fd9f80a9070 comp rx=0 tx=0).stop 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda08072650 msgr2=0x7fda08158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda08072650 0x7fda08158280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000e870 tx=0x7fd9f000ec70 comp rx=0 tx=0).stop 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 shutdown_connections 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd9e4067b20 0x7fd9e4069fe0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda080b1b80 0x7fda08152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda08072650 0x7fda08158280 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 --2- 172.21.15.86:0/570361946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda0806e7e0 0x7fda080cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 >> 172.21.15.86:0/570361946 conn(0x7fda080cc940 msgr2=0x7fda08154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 shutdown_connections 2024-03-22T04:11:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.323+0000 7fda0f600640 1 -- 172.21.15.86:0/570361946 wait complete. 2024-03-22T04:11:35.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.479+0000 7f64d3c00640 1 Processor -- start 2024-03-22T04:11:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.479+0000 7f64d3c00640 1 -- start start 2024-03-22T04:11:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64d3c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f64cc070560 0x7f64cc0d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64d3c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0b4710 0x7f64cc0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64d3c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64cc0d15f0 con 0x7f64cc15e7e0 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64d3c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f64cc0b4ba0 con 0x7f64cc070560 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64d3c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64cc06f970 con 0x7f64cc0b4710 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64d1200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f64cc15e7e0 0x7f64cc06fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52792/0 (socket says 172.21.15.86:52792) 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64d1200640 1 -- 172.21.15.86:0/3317430736 learned_addr learned my addr 172.21.15.86:0/3317430736 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64c3e00640 1 --2- 172.21.15.86:0/3317430736 >> v2:172.21.15.196:3300/0 conn(0x7f64cc070560 0x7f64cc0d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64d1c00640 1 --2- 172.21.15.86:0/3317430736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0b4710 0x7f64cc0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64cbe00640 1 -- 172.21.15.86:0/3317430736 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668082481 0 0) 0x7f64cc0d15f0 con 0x7f64cc15e7e0 2024-03-22T04:11:35.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64c3e00640 1 -- 172.21.15.86:0/3317430736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0b4710 msgr2=0x7f64cc0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:35.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64c3e00640 1 --2- 172.21.15.86:0/3317430736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0b4710 0x7f64cc0cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64c3e00640 1 -- 172.21.15.86:0/3317430736 >> v1:172.21.15.86:6789/0 conn(0x7f64cc15e7e0 legacy=0x7f64cc06fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64c3e00640 1 -- 172.21.15.86:0/3317430736 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64cc0d1310 con 0x7f64cc070560 2024-03-22T04:11:35.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64c3e00640 1 --2- 172.21.15.86:0/3317430736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc070560 0x7f64cc0d0d30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f64b40097c0 tx=0x7f64b401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564e5db30c9c97d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64cbe00640 1 -- 172.21.15.86:0/3317430736 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b40b0070 con 0x7f64cc070560 2024-03-22T04:11:35.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.483+0000 7f64cbe00640 1 -- 172.21.15.86:0/3317430736 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4004230 con 0x7f64cc070560 2024-03-22T04:11:35.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64cbe00640 1 -- 172.21.15.86:0/3317430736 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4003620 con 0x7f64cc070560 2024-03-22T04:11:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- 172.21.15.86:0/3317430736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc070560 msgr2=0x7f64cc0d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 --2- 172.21.15.86:0/3317430736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc070560 0x7f64cc0d0d30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f64b40097c0 tx=0x7f64b401c350 comp rx=0 tx=0).stop 2024-03-22T04:11:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- 172.21.15.86:0/3317430736 shutdown_connections 2024-03-22T04:11:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 --2- 172.21.15.86:0/3317430736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0b4710 0x7f64cc0cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 --2- 172.21.15.86:0/3317430736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc070560 0x7f64cc0d0d30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- 172.21.15.86:0/3317430736 >> 172.21.15.86:0/3317430736 conn(0x7f64cc0b3bf0 msgr2=0x7f64cc0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- 172.21.15.86:0/3317430736 shutdown_connections 2024-03-22T04:11:35.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- 172.21.15.86:0/3317430736 wait complete. 2024-03-22T04:11:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 Processor -- start 2024-03-22T04:11:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- start start 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc070560 0x7f64cc0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0b4710 0x7f64cc0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc15e7e0 0x7f64cc154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc0ce230 con 0x7f64cc0b4710 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc0ce0b0 con 0x7f64cc15e7e0 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d3c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64cc0ce3b0 con 0x7f64cc070560 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0b4710 0x7f64cc0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0b4710 0x7f64cc0cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57584/0 (socket says 172.21.15.86:57584) 2024-03-22T04:11:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64c3e00640 1 -- 172.21.15.86:0/705095868 learned_addr learned my addr 172.21.15.86:0/705095868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d1200640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc070560 0x7f64cc0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64d1c00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc15e7e0 0x7f64cc154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64c3e00640 1 -- 172.21.15.86:0/705095868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc070560 msgr2=0x7f64cc0c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64c3e00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc070560 0x7f64cc0c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64c3e00640 1 -- 172.21.15.86:0/705095868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc15e7e0 msgr2=0x7f64cc154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64c3e00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc15e7e0 0x7f64cc154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.487+0000 7f64c3e00640 1 -- 172.21.15.86:0/705095868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64cc155d30 con 0x7f64cc0b4710 2024-03-22T04:11:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64c3e00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0b4710 0x7f64cc0cb9e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64b40ae040 tx=0x7f64b40190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64c9600640 1 -- 172.21.15.86:0/705095868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b40b9020 con 0x7f64cc0b4710 2024-03-22T04:11:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64c9600640 1 -- 172.21.15.86:0/705095868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4003df0 con 0x7f64cc0b4710 2024-03-22T04:11:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64cc156010 con 0x7f64cc0b4710 2024-03-22T04:11:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64c9600640 1 -- 172.21.15.86:0/705095868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b40b0040 con 0x7f64cc0b4710 2024-03-22T04:11:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64cc156870 con 0x7f64cc0b4710 2024-03-22T04:11:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64cc074470 con 0x7f64cc0b4710 2024-03-22T04:11:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64c9600640 1 -- 172.21.15.86:0/705095868 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f64b40bc020 con 0x7f64cc0b4710 2024-03-22T04:11:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64c9600640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64a8067b20 0x7f64a8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64c9600640 1 -- 172.21.15.86:0/705095868 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f64b41200c0 con 0x7f64cc0b4710 2024-03-22T04:11:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.491+0000 7f64d1200640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64a8067b20 0x7f64a8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.495+0000 7f64d1200640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64a8067b20 0x7f64a8069fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f64bc007fd0 tx=0x7f64bc007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.495+0000 7f64c9600640 1 -- 172.21.15.86:0/705095868 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f64b40c3030 con 0x7f64cc0b4710 2024-03-22T04:11:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f64cc168cf0 con 0x7f64cc0b4710 2024-03-22T04:11:35.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64c9600640 1 -- 172.21.15.86:0/705095868 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f64b4016090 con 0x7f64cc0b4710 2024-03-22T04:11:35.656 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64a8067b20 msgr2=0x7f64a8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64a8067b20 0x7f64a8069fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f64bc007fd0 tx=0x7f64bc007d60 comp rx=0 tx=0).stop 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0b4710 msgr2=0x7f64cc0cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0b4710 0x7f64cc0cb9e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64b40ae040 tx=0x7f64b40190f0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 shutdown_connections 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64a8067b20 0x7f64a8069fe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc15e7e0 0x7f64cc154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0b4710 0x7f64cc0cb9e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 --2- 172.21.15.86:0/705095868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc070560 0x7f64cc0c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 >> 172.21.15.86:0/705095868 conn(0x7f64cc0b3bf0 msgr2=0x7f64cc15cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 shutdown_connections 2024-03-22T04:11:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.651+0000 7f64d3c00640 1 -- 172.21.15.86:0/705095868 wait complete. 2024-03-22T04:11:35.676 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:35.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07bc000640 1 Processor -- start 2024-03-22T04:11:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07bc000640 1 -- start start 2024-03-22T04:11:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07bc000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b40b2360 0x7f07b40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07bc000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f07b415d0b0 0x7f07b40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07bc000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b415d710 con 0x7f07b40b3430 2024-03-22T04:11:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07bc000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f07b40b3890 con 0x7f07b415d0b0 2024-03-22T04:11:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07bc000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f07b40b2740 con 0x7f07b40b2360 2024-03-22T04:11:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07b9600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b40b2360 0x7f07b40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07ba000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f07b415d0b0 0x7f07b40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07b9600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b40b2360 0x7f07b40b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42476/0 (socket says 172.21.15.86:42476) 2024-03-22T04:11:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07b9600640 1 -- 172.21.15.86:0/3547040733 learned_addr learned my addr 172.21.15.86:0/3547040733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07b8c00640 1 --1- 172.21.15.86:0/3547040733 >> v1:172.21.15.196:6789/0 conn(0x7f07b40b3430 0x7f07b415c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:59020/0 (socket says 172.21.15.86:59020) 2024-03-22T04:11:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07b9600640 1 -- 172.21.15.86:0/3547040733 >> v2:172.21.15.196:3300/0 conn(0x7f07b415d0b0 msgr2=0x7f07b40b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07b9600640 1 --2- 172.21.15.86:0/3547040733 >> v2:172.21.15.196:3300/0 conn(0x7f07b415d0b0 0x7f07b40b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07b9600640 1 -- 172.21.15.86:0/3547040733 >> v1:172.21.15.196:6789/0 conn(0x7f07b40b3430 legacy=0x7f07b415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.811+0000 7f07b9600640 1 -- 172.21.15.86:0/3547040733 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b415fe40 con 0x7f07b40b2360 2024-03-22T04:11:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07b9600640 1 --2- 172.21.15.86:0/3547040733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b40b2360 0x7f07b40b2ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009870 tx=0x7f07a801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2297bafda48d319 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07b3e00640 1 -- 172.21.15.86:0/3547040733 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a80ab650 con 0x7f07b40b2360 2024-03-22T04:11:35.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07b3e00640 1 -- 172.21.15.86:0/3547040733 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a80b0070 con 0x7f07b40b2360 2024-03-22T04:11:35.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07b3e00640 1 -- 172.21.15.86:0/3547040733 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a80aba10 con 0x7f07b40b2360 2024-03-22T04:11:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07bc000640 1 -- 172.21.15.86:0/3547040733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b40b2360 msgr2=0x7f07b40b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07bc000640 1 --2- 172.21.15.86:0/3547040733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b40b2360 0x7f07b40b2ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009870 tx=0x7f07a801c560 comp rx=0 tx=0).stop 2024-03-22T04:11:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07bc000640 1 -- 172.21.15.86:0/3547040733 shutdown_connections 2024-03-22T04:11:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07bc000640 1 --2- 172.21.15.86:0/3547040733 >> v2:172.21.15.196:3300/0 conn(0x7f07b415d0b0 0x7f07b40b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07bc000640 1 --2- 172.21.15.86:0/3547040733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b40b2360 0x7f07b40b2ef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07bc000640 1 -- 172.21.15.86:0/3547040733 >> 172.21.15.86:0/3547040733 conn(0x7f07b40cc940 msgr2=0x7f07b40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07bc000640 1 -- 172.21.15.86:0/3547040733 shutdown_connections 2024-03-22T04:11:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.815+0000 7f07bc000640 1 -- 172.21.15.86:0/3547040733 wait complete. 2024-03-22T04:11:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 Processor -- start 2024-03-22T04:11:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 -- start start 2024-03-22T04:11:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07b40b2360 0x7f07b40ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b40b3430 0x7f07b4147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b415d0b0 0x7f07b40ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4160c00 con 0x7f07b40b3430 2024-03-22T04:11:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07b40b5ad0 con 0x7f07b40b2360 2024-03-22T04:11:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f07b40ac220 con 0x7f07b415d0b0 2024-03-22T04:11:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b40b3430 0x7f07b4147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b9600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07b40b2360 0x7f07b40ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b40b3430 0x7f07b4147800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57588/0 (socket says 172.21.15.86:57588) 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 -- 172.21.15.86:0/4113803189 learned_addr learned my addr 172.21.15.86:0/4113803189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07ba000640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b415d0b0 0x7f07b40ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 -- 172.21.15.86:0/4113803189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b415d0b0 msgr2=0x7f07b40ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b415d0b0 0x7f07b40ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 -- 172.21.15.86:0/4113803189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07b40b2360 msgr2=0x7f07b40ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07b40b2360 0x7f07b40ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 -- 172.21.15.86:0/4113803189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b40ac500 con 0x7f07b40b3430 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b9600640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07b40b2360 0x7f07b40ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b8c00640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b40b3430 0x7f07b4147800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c420 tx=0x7f07a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b1600640 1 -- 172.21.15.86:0/4113803189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a00074e0 con 0x7f07b40b3430 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b1600640 1 -- 172.21.15.86:0/4113803189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a0007680 con 0x7f07b40b3430 2024-03-22T04:11:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07b1600640 1 -- 172.21.15.86:0/4113803189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0004a30 con 0x7f07b40b3430 2024-03-22T04:11:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b40ac7e0 con 0x7f07b40b3430 2024-03-22T04:11:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b4147d40 con 0x7f07b40b3430 2024-03-22T04:11:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.819+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4074470 con 0x7f07b40b3430 2024-03-22T04:11:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.823+0000 7f07b1600640 1 -- 172.21.15.86:0/4113803189 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f07a001a020 con 0x7f07b40b3430 2024-03-22T04:11:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.823+0000 7f07b1600640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0794067b20 0x7f0794069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.823+0000 7f07b1600640 1 -- 172.21.15.86:0/4113803189 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f07a0085980 con 0x7f07b40b3430 2024-03-22T04:11:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.823+0000 7f07b9600640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0794067b20 0x7f0794069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.823+0000 7f07b9600640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0794067b20 0x7f0794069fe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07a80af000 tx=0x7f07a80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:35.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.827+0000 7f07b1600640 1 -- 172.21.15.86:0/4113803189 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f07a0055990 con 0x7f07b40b3430 2024-03-22T04:11:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.979+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f07b406e9f0 con 0x7f07b40b3430 2024-03-22T04:11:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.979+0000 7f07b1600640 1 -- 172.21.15.86:0/4113803189 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f07a0059a30 con 0x7f07b40b3430 2024-03-22T04:11:35.986 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0794067b20 msgr2=0x7f0794069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0794067b20 0x7f0794069fe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07a80af000 tx=0x7f07a80a7000 comp rx=0 tx=0).stop 2024-03-22T04:11:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b40b3430 msgr2=0x7f07b4147800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b40b3430 0x7f07b4147800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c420 tx=0x7f07a000c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 shutdown_connections 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0794067b20 0x7f0794069fe0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07b415d0b0 0x7f07b40ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b40b3430 0x7f07b4147800 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 --2- 172.21.15.86:0/4113803189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07b40b2360 0x7f07b40ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 >> 172.21.15.86:0/4113803189 conn(0x7f07b40cc940 msgr2=0x7f07b4155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 shutdown_connections 2024-03-22T04:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:35.983+0000 7f07bc000640 1 -- 172.21.15.86:0/4113803189 wait complete. 2024-03-22T04:11:36.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:11:36.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b19e00640 1 Processor -- start 2024-03-22T04:11:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b19e00640 1 -- start start 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b19e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b1415b3f0 0x7f8b140729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b19e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8b140cc3e0 0x7f8b14072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b19e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b14071920 con 0x7f8b140b30f0 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b19e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8b140716d0 con 0x7f8b1415b3f0 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b19e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1406d990 con 0x7f8b140cc3e0 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b12a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b1415b3f0 0x7f8b140729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b12a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b1415b3f0 0x7f8b140729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59388/0 (socket says 172.21.15.86:59388) 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b12a00640 1 -- 172.21.15.86:0/434756388 learned_addr learned my addr 172.21.15.86:0/434756388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.139+0000 7f8b0a000640 1 --2- 172.21.15.86:0/434756388 >> v2:172.21.15.196:3300/0 conn(0x7f8b140cc3e0 0x7f8b14072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b12a00640 1 -- 172.21.15.86:0/434756388 >> v2:172.21.15.196:3300/0 conn(0x7f8b140cc3e0 msgr2=0x7f8b14072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:36.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b12a00640 1 --2- 172.21.15.86:0/434756388 >> v2:172.21.15.196:3300/0 conn(0x7f8b140cc3e0 0x7f8b14072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b12a00640 1 -- 172.21.15.86:0/434756388 >> v1:172.21.15.196:6789/0 conn(0x7f8b140b30f0 legacy=0x7f8b140ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b12a00640 1 -- 172.21.15.86:0/434756388 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b14071c00 con 0x7f8b1415b3f0 2024-03-22T04:11:36.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b12a00640 1 --2- 172.21.15.86:0/434756388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b140729c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b14070ed0 tx=0x7f8b0401e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e48172cc4bffd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b12000640 1 -- 172.21.15.86:0/434756388 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040ab660 con 0x7f8b1415b3f0 2024-03-22T04:11:36.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b12000640 1 -- 172.21.15.86:0/434756388 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b040ab800 con 0x7f8b1415b3f0 2024-03-22T04:11:36.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b12000640 1 -- 172.21.15.86:0/434756388 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040abbd0 con 0x7f8b1415b3f0 2024-03-22T04:11:36.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b19e00640 1 -- 172.21.15.86:0/434756388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 msgr2=0x7f8b140729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b19e00640 1 --2- 172.21.15.86:0/434756388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b140729c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b14070ed0 tx=0x7f8b0401e6a0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b19e00640 1 -- 172.21.15.86:0/434756388 shutdown_connections 2024-03-22T04:11:36.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b19e00640 1 --2- 172.21.15.86:0/434756388 >> v2:172.21.15.196:3300/0 conn(0x7f8b140cc3e0 0x7f8b14072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b19e00640 1 --2- 172.21.15.86:0/434756388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b140729c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b19e00640 1 -- 172.21.15.86:0/434756388 >> 172.21.15.86:0/434756388 conn(0x7f8b140b1df0 msgr2=0x7f8b140b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:36.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b19e00640 1 -- 172.21.15.86:0/434756388 shutdown_connections 2024-03-22T04:11:36.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.143+0000 7f8b19e00640 1 -- 172.21.15.86:0/434756388 wait complete. 2024-03-22T04:11:36.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 Processor -- start 2024-03-22T04:11:36.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 -- start start 2024-03-22T04:11:36.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140b30f0 0x7f8b14149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b140cc3e0 0x7f8b1414cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b141553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1406e2d0 con 0x7f8b1415b3f0 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1406e150 con 0x7f8b140b30f0 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1406e450 con 0x7f8b140cc3e0 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b141553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b141553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59400/0 (socket says 172.21.15.86:59400) 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 -- 172.21.15.86:0/2404920195 learned_addr learned my addr 172.21.15.86:0/2404920195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b12a00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140b30f0 0x7f8b14149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b0a000640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b140cc3e0 0x7f8b1414cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 -- 172.21.15.86:0/2404920195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b140cc3e0 msgr2=0x7f8b1414cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b140cc3e0 0x7f8b1414cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 -- 172.21.15.86:0/2404920195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140b30f0 msgr2=0x7f8b14149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140b30f0 0x7f8b14149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 -- 172.21.15.86:0/2404920195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b14158470 con 0x7f8b1415b3f0 2024-03-22T04:11:36.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b13400640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b141553e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00c770 tx=0x7f8b0c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b0be00640 1 -- 172.21.15.86:0/2404920195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c003b80 con 0x7f8b1415b3f0 2024-03-22T04:11:36.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b0be00640 1 -- 172.21.15.86:0/2404920195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0c003d20 con 0x7f8b1415b3f0 2024-03-22T04:11:36.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b0be00640 1 -- 172.21.15.86:0/2404920195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c00aa90 con 0x7f8b1415b3f0 2024-03-22T04:11:36.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.147+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1414a240 con 0x7f8b1415b3f0 2024-03-22T04:11:36.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.151+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1414c8b0 con 0x7f8b1415b3f0 2024-03-22T04:11:36.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.151+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1414cab0 con 0x7f8b1415b3f0 2024-03-22T04:11:36.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.151+0000 7f8b0be00640 1 -- 172.21.15.86:0/2404920195 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8b0c00ac30 con 0x7f8b1415b3f0 2024-03-22T04:11:36.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.151+0000 7f8b0be00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.151+0000 7f8b0be00640 1 -- 172.21.15.86:0/2404920195 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f8b0c086250 con 0x7f8b1415b3f0 2024-03-22T04:11:36.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.151+0000 7f8b12a00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.151+0000 7f8b12a00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b040b0000 tx=0x7f8b04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.155+0000 7f8b0be00640 1 -- 172.21.15.86:0/2404920195 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8b1414cab0 con 0x7f8b1415b3f0 2024-03-22T04:11:36.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.323+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b14068650 con 0x7f8b1415b3f0 2024-03-22T04:11:36.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.323+0000 7f8b0be00640 1 -- 172.21.15.86:0/2404920195 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f8b0c056810 con 0x7f8b1415b3f0 2024-03-22T04:11:36.329 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:11:36.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 msgr2=0x7f8af0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b040b0000 tx=0x7f8b04009040 comp rx=0 tx=0).stop 2024-03-22T04:11:36.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 msgr2=0x7f8b141553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b141553e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00c770 tx=0x7f8b0c00cc30 comp rx=0 tx=0).stop 2024-03-22T04:11:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 shutdown_connections 2024-03-22T04:11:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b1415b3f0 0x7f8b141553e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b140cc3e0 0x7f8b1414cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 --2- 172.21.15.86:0/2404920195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140b30f0 0x7f8b14149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 >> 172.21.15.86:0/2404920195 conn(0x7f8b140b1df0 msgr2=0x7f8b1415c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 shutdown_connections 2024-03-22T04:11:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.327+0000 7f8b19e00640 1 -- 172.21.15.86:0/2404920195 wait complete. 2024-03-22T04:11:36.351 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-22T04:11:36.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:36.351 DEBUG:teuthology.orchestra.run.smithi196:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-22T04:11:36.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:11:36.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:11:36.375 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-22T04:11:36.427 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-22T04:11:36.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:11:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca32000640 1 Processor -- start 2024-03-22T04:11:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca32000640 1 -- start start 2024-03-22T04:11:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca32000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c0aff50 0x7fca2c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca32000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca2c06cbb0 0x7fca2c143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca32000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca2c0ae5a0 con 0x7fca2c144320 2024-03-22T04:11:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca32000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fca2c06d010 con 0x7fca2c06cbb0 2024-03-22T04:11:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca32000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca2c162aa0 con 0x7fca2c0aff50 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2b400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c0aff50 0x7fca2c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca2c06cbb0 0x7fca2c143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2be00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fca2c144320 0x7fca2c06e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:51636/0 (socket says 172.21.15.86:51636) 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2be00640 1 -- 172.21.15.86:0/3333170534 learned_addr learned my addr 172.21.15.86:0/3333170534 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2b400640 1 -- 172.21.15.86:0/3333170534 >> v2:172.21.15.196:3300/0 conn(0x7fca2c06cbb0 msgr2=0x7fca2c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2b400640 1 --2- 172.21.15.86:0/3333170534 >> v2:172.21.15.196:3300/0 conn(0x7fca2c06cbb0 0x7fca2c143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2b400640 1 -- 172.21.15.86:0/3333170534 >> v1:172.21.15.196:6789/0 conn(0x7fca2c144320 legacy=0x7fca2c06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2b400640 1 -- 172.21.15.86:0/3333170534 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c144b80 con 0x7fca2c0aff50 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2aa00640 1 --2- 172.21.15.86:0/3333170534 >> v2:172.21.15.196:3300/0 conn(0x7fca2c06cbb0 0x7fca2c143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:36.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.559+0000 7fca2a000640 1 -- 172.21.15.86:0/3333170534 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3996468507 0 0) 0x7fca240014a0 con 0x7fca2c144320 2024-03-22T04:11:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca2b400640 1 --2- 172.21.15.86:0/3333170534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c0aff50 0x7fca2c06c670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17544488dda3dcc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca2a000640 1 -- 172.21.15.86:0/3333170534 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b0070 con 0x7fca2c0aff50 2024-03-22T04:11:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca2a000640 1 -- 172.21.15.86:0/3333170534 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c0ab600 con 0x7fca2c0aff50 2024-03-22T04:11:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca2a000640 1 -- 172.21.15.86:0/3333170534 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0ab9d0 con 0x7fca2c0aff50 2024-03-22T04:11:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca32000640 1 -- 172.21.15.86:0/3333170534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c0aff50 msgr2=0x7fca2c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca32000640 1 --2- 172.21.15.86:0/3333170534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c0aff50 0x7fca2c06c670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c410 comp rx=0 tx=0).stop 2024-03-22T04:11:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca32000640 1 -- 172.21.15.86:0/3333170534 shutdown_connections 2024-03-22T04:11:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca32000640 1 --2- 172.21.15.86:0/3333170534 >> v2:172.21.15.196:3300/0 conn(0x7fca2c06cbb0 0x7fca2c143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca32000640 1 --2- 172.21.15.86:0/3333170534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c0aff50 0x7fca2c06c670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca32000640 1 -- 172.21.15.86:0/3333170534 >> 172.21.15.86:0/3333170534 conn(0x7fca2c071fe0 msgr2=0x7fca2c0723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca32000640 1 -- 172.21.15.86:0/3333170534 shutdown_connections 2024-03-22T04:11:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.563+0000 7fca32000640 1 -- 172.21.15.86:0/3333170534 wait complete. 2024-03-22T04:11:36.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 Processor -- start 2024-03-22T04:11:36.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 -- start start 2024-03-22T04:11:36.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c06cbb0 0x7fca2c15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c0aff50 0x7fca2c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c144320 0x7fca2c143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c162f30 con 0x7fca2c0aff50 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c0b0330 con 0x7fca2c06cbb0 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca2c158e30 con 0x7fca2c144320 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c0aff50 0x7fca2c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c0aff50 0x7fca2c156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59408/0 (socket says 172.21.15.86:59408) 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 -- 172.21.15.86:0/2639726377 learned_addr learned my addr 172.21.15.86:0/2639726377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2be00640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c144320 0x7fca2c143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2b400640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c06cbb0 0x7fca2c15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 -- 172.21.15.86:0/2639726377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c144320 msgr2=0x7fca2c143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c144320 0x7fca2c143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 -- 172.21.15.86:0/2639726377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c06cbb0 msgr2=0x7fca2c15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c06cbb0 0x7fca2c15c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 -- 172.21.15.86:0/2639726377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c159110 con 0x7fca2c0aff50 2024-03-22T04:11:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca2aa00640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c0aff50 0x7fca2c156440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fca1800ebe0 tx=0x7fca1800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca23400640 1 -- 172.21.15.86:0/2639726377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18017070 con 0x7fca2c0aff50 2024-03-22T04:11:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca23400640 1 -- 172.21.15.86:0/2639726377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1800cc30 con 0x7fca2c0aff50 2024-03-22T04:11:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c1593f0 con 0x7fca2c0aff50 2024-03-22T04:11:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca23400640 1 -- 172.21.15.86:0/2639726377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18013470 con 0x7fca2c0aff50 2024-03-22T04:11:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.567+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c1411c0 con 0x7fca2c0aff50 2024-03-22T04:11:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.571+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2c068650 con 0x7fca2c0aff50 2024-03-22T04:11:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.571+0000 7fca23400640 1 -- 172.21.15.86:0/2639726377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fca1801a070 con 0x7fca2c0aff50 2024-03-22T04:11:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.571+0000 7fca23400640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca0c067b20 0x7fca0c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.571+0000 7fca23400640 1 -- 172.21.15.86:0/2639726377 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fca180865b0 con 0x7fca2c0aff50 2024-03-22T04:11:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.571+0000 7fca2b400640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca0c067b20 0x7fca0c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.571+0000 7fca2b400640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca0c067b20 0x7fca0c069fe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ad000 tx=0x7fca1c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.575+0000 7fca23400640 1 -- 172.21.15.86:0/2639726377 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fca18056520 con 0x7fca2c0aff50 2024-03-22T04:11:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.763+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca2c069480 con 0x7fca2c0aff50 2024-03-22T04:11:36.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.763+0000 7fca23400640 1 -- 172.21.15.86:0/2639726377 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5179 (secure 0 0 0) 0x7fca1805a5c0 con 0x7fca2c0aff50 2024-03-22T04:11:36.769 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:36.769 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:58.258965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:11:36.769 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca0c067b20 msgr2=0x7fca0c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca0c067b20 0x7fca0c069fe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ad000 tx=0x7fca1c0a6000 comp rx=0 tx=0).stop 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c0aff50 msgr2=0x7fca2c156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c0aff50 0x7fca2c156440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fca1800ebe0 tx=0x7fca1800c520 comp rx=0 tx=0).stop 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 shutdown_connections 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca0c067b20 0x7fca0c069fe0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2c144320 0x7fca2c143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c0aff50 0x7fca2c156440 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 --2- 172.21.15.86:0/2639726377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2c06cbb0 0x7fca2c15c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 >> 172.21.15.86:0/2639726377 conn(0x7fca2c071fe0 msgr2=0x7fca2c0b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 shutdown_connections 2024-03-22T04:11:36.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.767+0000 7fca32000640 1 -- 172.21.15.86:0/2639726377 wait complete. 2024-03-22T04:11:36.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:36.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.919+0000 7f612fe00640 1 Processor -- start 2024-03-22T04:11:36.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.919+0000 7f612fe00640 1 -- start start 2024-03-22T04:11:36.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f612815b3f0 0x7f61280729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6128071920 con 0x7f61280ae540 2024-03-22T04:11:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612fe00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6128071c00 con 0x7f61280cc3e0 2024-03-22T04:11:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612fe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f61280716d0 con 0x7f612815b3f0 2024-03-22T04:11:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f61280ae540 0x7f61280b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54670/0 (socket says 172.21.15.86:54670) 2024-03-22T04:11:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612de00640 1 -- 172.21.15.86:0/3690985004 learned_addr learned my addr 172.21.15.86:0/3690985004 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612d400640 1 --2- 172.21.15.86:0/3690985004 >> v2:172.21.15.196:3300/0 conn(0x7f612815b3f0 0x7f61280729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f6127e00640 1 -- 172.21.15.86:0/3690985004 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2286477715 0 0) 0x7f6128071920 con 0x7f61280ae540 2024-03-22T04:11:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f6127e00640 1 -- 172.21.15.86:0/3690985004 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6110005740 con 0x7f61280ae540 2024-03-22T04:11:36.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f6127e00640 1 -- 172.21.15.86:0/3690985004 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408849362 0 0) 0x7f6128071c00 con 0x7f61280cc3e0 2024-03-22T04:11:36.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612d400640 1 -- 172.21.15.86:0/3690985004 >> v1:172.21.15.196:6789/0 conn(0x7f61280cc3e0 legacy=0x7f61280ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612d400640 1 -- 172.21.15.86:0/3690985004 >> v1:172.21.15.86:6789/0 conn(0x7f61280ae540 legacy=0x7f61280b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612d400640 1 -- 172.21.15.86:0/3690985004 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6128071ee0 con 0x7f612815b3f0 2024-03-22T04:11:36.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.923+0000 7f612d400640 1 --2- 172.21.15.86:0/3690985004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612815b3f0 0x7f61280729c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f611c0097c0 tx=0x7f611c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b8f2c1d7b1d526 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f6127e00640 1 -- 172.21.15.86:0/3690985004 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b0070 con 0x7f612815b3f0 2024-03-22T04:11:36.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f6127e00640 1 -- 172.21.15.86:0/3690985004 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c0ab600 con 0x7f612815b3f0 2024-03-22T04:11:36.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f6127e00640 1 -- 172.21.15.86:0/3690985004 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0ab9d0 con 0x7f612815b3f0 2024-03-22T04:11:36.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- 172.21.15.86:0/3690985004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612815b3f0 msgr2=0x7f61280729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 --2- 172.21.15.86:0/3690985004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612815b3f0 0x7f61280729c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f611c0097c0 tx=0x7f611c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- 172.21.15.86:0/3690985004 shutdown_connections 2024-03-22T04:11:36.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 --2- 172.21.15.86:0/3690985004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612815b3f0 0x7f61280729c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- 172.21.15.86:0/3690985004 >> 172.21.15.86:0/3690985004 conn(0x7f61280b1df0 msgr2=0x7f61280b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:36.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- 172.21.15.86:0/3690985004 shutdown_connections 2024-03-22T04:11:36.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- 172.21.15.86:0/3690985004 wait complete. 2024-03-22T04:11:36.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 Processor -- start 2024-03-22T04:11:36.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- start start 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61280ae540 0x7f6128154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61280cc3e0 0x7f612814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612815b3f0 0x7f612814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61280b30f0 con 0x7f612815b3f0 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61280b4c60 con 0x7f61280cc3e0 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f61280b3270 con 0x7f61280ae540 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612815b3f0 0x7f612814a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612d400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61280ae540 0x7f6128154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612815b3f0 0x7f612814a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59412/0 (socket says 172.21.15.86:59412) 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612de00640 1 -- 172.21.15.86:0/256867881 learned_addr learned my addr 172.21.15.86:0/256867881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612ca00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61280cc3e0 0x7f612814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612de00640 1 -- 172.21.15.86:0/256867881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61280ae540 msgr2=0x7f6128154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612de00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61280ae540 0x7f6128154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.927+0000 7f612de00640 1 -- 172.21.15.86:0/256867881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61280cc3e0 msgr2=0x7f612814b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f612de00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61280cc3e0 0x7f612814b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f612de00640 1 -- 172.21.15.86:0/256867881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6128155100 con 0x7f612815b3f0 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f612de00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612815b3f0 0x7f612814a6a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f612000cbe0 tx=0x7f612000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f6125600640 1 -- 172.21.15.86:0/256867881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120017070 con 0x7f612815b3f0 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f6125600640 1 -- 172.21.15.86:0/256867881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612000ad40 con 0x7f612815b3f0 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6128149bc0 con 0x7f612815b3f0 2024-03-22T04:11:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f6125600640 1 -- 172.21.15.86:0/256867881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120012600 con 0x7f612815b3f0 2024-03-22T04:11:36.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612814b7d0 con 0x7f612815b3f0 2024-03-22T04:11:36.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6128068650 con 0x7f612815b3f0 2024-03-22T04:11:36.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f6125600640 1 -- 172.21.15.86:0/256867881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f612001a070 con 0x7f612815b3f0 2024-03-22T04:11:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f6125600640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.931+0000 7f6125600640 1 -- 172.21.15.86:0/256867881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f6120085fe0 con 0x7f612815b3f0 2024-03-22T04:11:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.935+0000 7f612d400640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:36.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.935+0000 7f612d400640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f611c0ad000 tx=0x7f611c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:36.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:36.935+0000 7f6125600640 1 -- 172.21.15.86:0/256867881 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6120055fb0 con 0x7f612815b3f0 2024-03-22T04:11:37.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.091+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6128151950 con 0x7f612815b3f0 2024-03-22T04:11:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f6125600640 1 -- 172.21.15.86:0/256867881 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7f612005a050 con 0x7f612815b3f0 2024-03-22T04:11:37.099 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:37.099 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 msgr2=0x7f6108069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f611c0ad000 tx=0x7f611c008040 comp rx=0 tx=0).stop 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612815b3f0 msgr2=0x7f612814a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612815b3f0 0x7f612814a6a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f612000cbe0 tx=0x7f612000a430 comp rx=0 tx=0).stop 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 shutdown_connections 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61280cc3e0 0x7f612814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612815b3f0 0x7f612814a6a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 --2- 172.21.15.86:0/256867881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61280ae540 0x7f6128154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 >> 172.21.15.86:0/256867881 conn(0x7f61280b1df0 msgr2=0x7f61280b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 shutdown_connections 2024-03-22T04:11:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.095+0000 7f612fe00640 1 -- 172.21.15.86:0/256867881 wait complete. 2024-03-22T04:11:37.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:10:31.926650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:10:33.257136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-22T04:10:36.954718+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T04:11:37.122 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:37.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.255+0000 7f1ed7200640 1 Processor -- start 2024-03-22T04:11:37.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.255+0000 7f1ed7200640 1 -- start start 2024-03-22T04:11:37.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed815e640 0x7f1ed80d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed7200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ed80b0de0 0x7f1ed80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed80ceee0 con 0x7f1ed815ea20 2024-03-22T04:11:37.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed7200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed80cecb0 con 0x7f1ed80b0de0 2024-03-22T04:11:37.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed7200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed80cf1c0 con 0x7f1ed815e640 2024-03-22T04:11:37.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1ed815ea20 0x7f1ed80cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54678/0 (socket says 172.21.15.86:54678) 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed5400640 1 -- 172.21.15.86:0/3744155664 learned_addr learned my addr 172.21.15.86:0/3744155664 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/3744155664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed815e640 0x7f1ed80d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed6800640 1 --2- 172.21.15.86:0/3744155664 >> v2:172.21.15.196:3300/0 conn(0x7f1ed80b0de0 0x7f1ed80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed4a00640 1 -- 172.21.15.86:0/3744155664 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553153563 0 0) 0x7f1ed80ceee0 con 0x7f1ed815ea20 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed4a00640 1 -- 172.21.15.86:0/3744155664 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ebc003530 con 0x7f1ed815ea20 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed5e00640 1 -- 172.21.15.86:0/3744155664 >> v2:172.21.15.196:3300/0 conn(0x7f1ed80b0de0 msgr2=0x7f1ed80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/3744155664 >> v2:172.21.15.196:3300/0 conn(0x7f1ed80b0de0 0x7f1ed80b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed5e00640 1 -- 172.21.15.86:0/3744155664 >> v1:172.21.15.86:6789/0 conn(0x7f1ed815ea20 legacy=0x7f1ed80cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed5e00640 1 -- 172.21.15.86:0/3744155664 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80cf4a0 con 0x7f1ed815e640 2024-03-22T04:11:37.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.259+0000 7f1ed6800640 1 --2- 172.21.15.86:0/3744155664 >> v2:172.21.15.196:3300/0 conn(0x7f1ed80b0de0 0x7f1ed80b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/3744155664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed815e640 0x7f1ed80d2a20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00097c0 tx=0x7f1ec001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cb4804b29d51e04 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed4a00640 1 -- 172.21.15.86:0/3744155664 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec00b1070 con 0x7f1ed815e640 2024-03-22T04:11:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed4a00640 1 -- 172.21.15.86:0/3744155664 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec00ac410 con 0x7f1ed815e640 2024-03-22T04:11:37.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed4a00640 1 -- 172.21.15.86:0/3744155664 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec00ac7e0 con 0x7f1ed815e640 2024-03-22T04:11:37.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- 172.21.15.86:0/3744155664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed815e640 msgr2=0x7f1ed80d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 --2- 172.21.15.86:0/3744155664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed815e640 0x7f1ed80d2a20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00097c0 tx=0x7f1ec001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- 172.21.15.86:0/3744155664 shutdown_connections 2024-03-22T04:11:37.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 --2- 172.21.15.86:0/3744155664 >> v2:172.21.15.196:3300/0 conn(0x7f1ed80b0de0 0x7f1ed80b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 --2- 172.21.15.86:0/3744155664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed815e640 0x7f1ed80d2a20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- 172.21.15.86:0/3744155664 >> 172.21.15.86:0/3744155664 conn(0x7f1ed80d4110 msgr2=0x7f1ed80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:37.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- 172.21.15.86:0/3744155664 shutdown_connections 2024-03-22T04:11:37.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- 172.21.15.86:0/3744155664 wait complete. 2024-03-22T04:11:37.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 Processor -- start 2024-03-22T04:11:37.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- start start 2024-03-22T04:11:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed80b0de0 0x7f1ed80b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed815e640 0x7f1ed80b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815ea20 0x7f1ed80b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed8162ce0 con 0x7f1ed815ea20 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed8162b60 con 0x7f1ed815e640 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed7200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed8162e60 con 0x7f1ed80b0de0 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815ea20 0x7f1ed80b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815ea20 0x7f1ed80b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59414/0 (socket says 172.21.15.86:59414) 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed815e640 0x7f1ed80b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed6800640 1 -- 172.21.15.86:0/1193660149 learned_addr learned my addr 172.21.15.86:0/1193660149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.263+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed80b0de0 0x7f1ed80b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed6800640 1 -- 172.21.15.86:0/1193660149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed80b0de0 msgr2=0x7f1ed80b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed6800640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed80b0de0 0x7f1ed80b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed6800640 1 -- 172.21.15.86:0/1193660149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed815e640 msgr2=0x7f1ed80b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed6800640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed815e640 0x7f1ed80b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed6800640 1 -- 172.21.15.86:0/1193660149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80b9150 con 0x7f1ed815ea20 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed5400640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed815e640 0x7f1ed80b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:37.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed6800640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815ea20 0x7f1ed80b7240 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009f00 tx=0x7f1ed000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ece000640 1 -- 172.21.15.86:0/1193660149 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0018030 con 0x7f1ed815ea20 2024-03-22T04:11:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ece000640 1 -- 172.21.15.86:0/1193660149 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed000ce50 con 0x7f1ed815ea20 2024-03-22T04:11:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed80b9430 con 0x7f1ed815ea20 2024-03-22T04:11:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ece000640 1 -- 172.21.15.86:0/1193660149 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003bc0 con 0x7f1ed815ea20 2024-03-22T04:11:37.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed80b99f0 con 0x7f1ed815ea20 2024-03-22T04:11:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed804f900 con 0x7f1ed815ea20 2024-03-22T04:11:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ece000640 1 -- 172.21.15.86:0/1193660149 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1ed001b070 con 0x7f1ed815ea20 2024-03-22T04:11:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ece000640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1eb4067b20 0x7f1eb4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ece000640 1 -- 172.21.15.86:0/1193660149 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f1ed0086cf0 con 0x7f1ed815ea20 2024-03-22T04:11:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.267+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1eb4067b20 0x7f1eb4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.271+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1eb4067b20 0x7f1eb4069fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00ae000 tx=0x7f1ec0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.271+0000 7f1ece000640 1 -- 172.21.15.86:0/1193660149 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1ed0010040 con 0x7f1ed815ea20 2024-03-22T04:11:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.427+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ed80d01d0 con 0x7f1ed815ea20 2024-03-22T04:11:37.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.427+0000 7f1ece000640 1 -- 172.21.15.86:0/1193660149 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f1ed00572b0 con 0x7f1ed815ea20 2024-03-22T04:11:37.433 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1eb4067b20 msgr2=0x7f1eb4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1eb4067b20 0x7f1eb4069fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00ae000 tx=0x7f1ec0008040 comp rx=0 tx=0).stop 2024-03-22T04:11:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815ea20 msgr2=0x7f1ed80b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815ea20 0x7f1ed80b7240 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009f00 tx=0x7f1ed000c490 comp rx=0 tx=0).stop 2024-03-22T04:11:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 shutdown_connections 2024-03-22T04:11:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1eb4067b20 0x7f1eb4069fe0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed815e640 0x7f1ed80b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815ea20 0x7f1ed80b7240 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 --2- 172.21.15.86:0/1193660149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed80b0de0 0x7f1ed80b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 >> 172.21.15.86:0/1193660149 conn(0x7f1ed80d4110 msgr2=0x7f1ed80d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:37.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 shutdown_connections 2024-03-22T04:11:37.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.431+0000 7f1ed7200640 1 -- 172.21.15.86:0/1193660149 wait complete. 2024-03-22T04:11:37.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:11:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8afd600640 1 Processor -- start 2024-03-22T04:11:37.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8afd600640 1 -- start start 2024-03-22T04:11:37.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8afd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8af8072430 0x7f8af80cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8afd600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af80af900 con 0x7f8af815a830 2024-03-22T04:11:37.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8afd600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af80ce5d0 con 0x7f8af8071860 2024-03-22T04:11:37.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8afd600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8af80cfbb0 con 0x7f8af8072430 2024-03-22T04:11:37.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8af6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8af815a830 0x7f8af815ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54694/0 (socket says 172.21.15.86:54694) 2024-03-22T04:11:37.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8af6a00640 1 -- 172.21.15.86:0/4240112272 learned_addr learned my addr 172.21.15.86:0/4240112272 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:37.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.587+0000 7f8af7400640 1 --2- 172.21.15.86:0/4240112272 >> v2:172.21.15.86:3300/0 conn(0x7f8af8072430 0x7f8af80cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8af7400640 1 -- 172.21.15.86:0/4240112272 >> v1:172.21.15.196:6789/0 conn(0x7f8af8071860 legacy=0x7f8af8071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8af7400640 1 -- 172.21.15.86:0/4240112272 >> v1:172.21.15.86:6789/0 conn(0x7f8af815a830 legacy=0x7f8af815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8af5600640 1 -- 172.21.15.86:0/4240112272 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773239831 0 0) 0x7f8af80ce5d0 con 0x7f8af8071860 2024-03-22T04:11:37.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8af7400640 1 -- 172.21.15.86:0/4240112272 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af80ce8b0 con 0x7f8af8072430 2024-03-22T04:11:37.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8af7400640 1 --2- 172.21.15.86:0/4240112272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8072430 0x7f8af80cf670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8af00097c0 tx=0x7f8af001d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0d6c016629f07 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8af5600640 1 -- 172.21.15.86:0/4240112272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af00b1070 con 0x7f8af8072430 2024-03-22T04:11:37.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8af5600640 1 -- 172.21.15.86:0/4240112272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af00ac600 con 0x7f8af8072430 2024-03-22T04:11:37.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8af5600640 1 -- 172.21.15.86:0/4240112272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af00ac9d0 con 0x7f8af8072430 2024-03-22T04:11:37.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8afd600640 1 -- 172.21.15.86:0/4240112272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8072430 msgr2=0x7f8af80cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8afd600640 1 --2- 172.21.15.86:0/4240112272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8072430 0x7f8af80cf670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8af00097c0 tx=0x7f8af001d440 comp rx=0 tx=0).stop 2024-03-22T04:11:37.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8afd600640 1 -- 172.21.15.86:0/4240112272 shutdown_connections 2024-03-22T04:11:37.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8afd600640 1 --2- 172.21.15.86:0/4240112272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8072430 0x7f8af80cf670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.591+0000 7f8afd600640 1 -- 172.21.15.86:0/4240112272 >> 172.21.15.86:0/4240112272 conn(0x7f8af80cc940 msgr2=0x7f8af80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:37.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 -- 172.21.15.86:0/4240112272 shutdown_connections 2024-03-22T04:11:37.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 -- 172.21.15.86:0/4240112272 wait complete. 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 Processor -- start 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 -- start start 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8071860 0x7f8af80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8072430 0x7f8af8154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af815a830 0x7f8af8154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8af806dae0 con 0x7f8af8071860 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8af806d960 con 0x7f8af8072430 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8af806dc60 con 0x7f8af815a830 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8071860 0x7f8af80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8071860 0x7f8af80b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59432/0 (socket says 172.21.15.86:59432) 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 -- 172.21.15.86:0/2921459312 learned_addr learned my addr 172.21.15.86:0/2921459312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af7400640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af815a830 0x7f8af8154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6000640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8072430 0x7f8af8154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 -- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af815a830 msgr2=0x7f8af8154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af815a830 0x7f8af8154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 -- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8072430 msgr2=0x7f8af8154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8072430 0x7f8af8154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 -- 172.21.15.86:0/2921459312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af80b4b10 con 0x7f8af8071860 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6000640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8072430 0x7f8af8154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8af6a00640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8071860 0x7f8af80b3750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800caf0 tx=0x7f8ae80020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8aeea00640 1 -- 172.21.15.86:0/2921459312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8018070 con 0x7f8af8071860 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8aeea00640 1 -- 172.21.15.86:0/2921459312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae800a700 con 0x7f8af8071860 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af80b4df0 con 0x7f8af8071860 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8aeea00640 1 -- 172.21.15.86:0/2921459312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800aad0 con 0x7f8af8071860 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.595+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af80b53b0 con 0x7f8af8071860 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.599+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af8074470 con 0x7f8af8071860 2024-03-22T04:11:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.599+0000 7f8aeea00640 1 -- 172.21.15.86:0/2921459312 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8ae801b070 con 0x7f8af8071860 2024-03-22T04:11:37.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.599+0000 7f8aeea00640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad8067b20 0x7f8ad8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.599+0000 7f8aeea00640 1 -- 172.21.15.86:0/2921459312 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f8ae8085a10 con 0x7f8af8071860 2024-03-22T04:11:37.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.599+0000 7f8af6000640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad8067b20 0x7f8ad8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.599+0000 7f8af6000640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad8067b20 0x7f8ad8069fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8af81fa8c0 tx=0x7f8ae4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.603+0000 7f8aeea00640 1 -- 172.21.15.86:0/2921459312 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8ae800ac70 con 0x7f8af8071860 2024-03-22T04:11:37.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8af80ce0e0 con 0x7f8af8071860 2024-03-22T04:11:37.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8aeea00640 1 -- 172.21.15.86:0/2921459312 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f8ae8055a20 con 0x7f8af8071860 2024-03-22T04:11:37.763 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:37.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad8067b20 msgr2=0x7f8ad8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad8067b20 0x7f8ad8069fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8af81fa8c0 tx=0x7f8ae4009040 comp rx=0 tx=0).stop 2024-03-22T04:11:37.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8071860 msgr2=0x7f8af80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8071860 0x7f8af80b3750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800caf0 tx=0x7f8ae80020b0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 shutdown_connections 2024-03-22T04:11:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad8067b20 0x7f8ad8069fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8072430 0x7f8af8154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af815a830 0x7f8af8154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 --2- 172.21.15.86:0/2921459312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8071860 0x7f8af80b3750 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 >> 172.21.15.86:0/2921459312 conn(0x7f8af80cc940 msgr2=0x7f8af815d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 shutdown_connections 2024-03-22T04:11:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.759+0000 7f8afd600640 1 -- 172.21.15.86:0/2921459312 wait complete. 2024-03-22T04:11:37.784 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:11:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb8b200640 1 Processor -- start 2024-03-22T04:11:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb8b200640 1 -- start start 2024-03-22T04:11:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb8b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb8415e970 0x7fcb840b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb8b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8406dd70 con 0x7fcb8409b180 2024-03-22T04:11:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb8b200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8406dff0 con 0x7fcb8406d8f0 2024-03-22T04:11:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb8b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcb840b49a0 con 0x7fcb8415e970 2024-03-22T04:11:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb8415e970 0x7fcb840b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb89200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcb8409b180 0x7fcb840cd230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54702/0 (socket says 172.21.15.86:54702) 2024-03-22T04:11:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb89200640 1 -- 172.21.15.86:0/457618540 learned_addr learned my addr 172.21.15.86:0/457618540 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb83e00640 1 -- 172.21.15.86:0/457618540 >> v1:172.21.15.196:6789/0 conn(0x7fcb8406d8f0 legacy=0x7fcb8409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb83e00640 1 -- 172.21.15.86:0/457618540 >> v1:172.21.15.86:6789/0 conn(0x7fcb8409b180 legacy=0x7fcb840cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.919+0000 7fcb83e00640 1 -- 172.21.15.86:0/457618540 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8406e2d0 con 0x7fcb8415e970 2024-03-22T04:11:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb83e00640 1 --2- 172.21.15.86:0/457618540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8415e970 0x7fcb840b12e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcb740097c0 tx=0x7fcb7401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cbbd2d36b45a5c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb82a00640 1 -- 172.21.15.86:0/457618540 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb740b1070 con 0x7fcb8415e970 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb82a00640 1 -- 172.21.15.86:0/457618540 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb740ac5f0 con 0x7fcb8415e970 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb82a00640 1 -- 172.21.15.86:0/457618540 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb740ac9c0 con 0x7fcb8415e970 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb8b200640 1 -- 172.21.15.86:0/457618540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8415e970 msgr2=0x7fcb840b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb8b200640 1 --2- 172.21.15.86:0/457618540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8415e970 0x7fcb840b12e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcb740097c0 tx=0x7fcb7401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb8b200640 1 -- 172.21.15.86:0/457618540 shutdown_connections 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb8b200640 1 --2- 172.21.15.86:0/457618540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8415e970 0x7fcb840b12e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb8b200640 1 -- 172.21.15.86:0/457618540 >> 172.21.15.86:0/457618540 conn(0x7fcb840b3bf0 msgr2=0x7fcb840b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb8b200640 1 -- 172.21.15.86:0/457618540 shutdown_connections 2024-03-22T04:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.923+0000 7fcb8b200640 1 -- 172.21.15.86:0/457618540 wait complete. 2024-03-22T04:11:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb8b200640 1 Processor -- start 2024-03-22T04:11:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb8b200640 1 -- start start 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb8b200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8406d8f0 0x7fcb84153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb8b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8409b180 0x7fcb840c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb8b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb8415e970 0x7fcb8414a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb8b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb840ce2d0 con 0x7fcb8409b180 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb8b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb840ce150 con 0x7fcb8415e970 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb8b200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcb840ce450 con 0x7fcb8406d8f0 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8409b180 0x7fcb840c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8409b180 0x7fcb840c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59452/0 (socket says 172.21.15.86:59452) 2024-03-22T04:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 -- 172.21.15.86:0/3395373709 learned_addr learned my addr 172.21.15.86:0/3395373709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83e00640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8406d8f0 0x7fcb84153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb89200640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb8415e970 0x7fcb8414a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 -- 172.21.15.86:0/3395373709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8406d8f0 msgr2=0x7fcb84153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8406d8f0 0x7fcb84153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 -- 172.21.15.86:0/3395373709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb8415e970 msgr2=0x7fcb8414a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb8415e970 0x7fcb8414a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 -- 172.21.15.86:0/3395373709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb840c5ff0 con 0x7fcb8409b180 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb83400640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8409b180 0x7fcb840c7090 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000c370 tx=0x7fcb7000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.927+0000 7fcb7be00640 1 -- 172.21.15.86:0/3395373709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb700052d0 con 0x7fcb8409b180 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb7be00640 1 -- 172.21.15.86:0/3395373709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb70005470 con 0x7fcb8409b180 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb7be00640 1 -- 172.21.15.86:0/3395373709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70007260 con 0x7fcb8409b180 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb840c62d0 con 0x7fcb8409b180 2024-03-22T04:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb840c6890 con 0x7fcb8409b180 2024-03-22T04:11:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb84074470 con 0x7fcb8409b180 2024-03-22T04:11:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb7be00640 1 -- 172.21.15.86:0/3395373709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcb70007400 con 0x7fcb8409b180 2024-03-22T04:11:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb7be00640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb64067b20 0x7fcb64069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb7be00640 1 -- 172.21.15.86:0/3395373709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fcb70085730 con 0x7fcb8409b180 2024-03-22T04:11:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.931+0000 7fcb83e00640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb64067b20 0x7fcb64069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.935+0000 7fcb83e00640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb64067b20 0x7fcb64069fe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb740ae000 tx=0x7fcb740a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:37.935+0000 7fcb7be00640 1 -- 172.21.15.86:0/3395373709 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcb700556a0 con 0x7fcb8409b180 2024-03-22T04:11:38.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb8409ac10 con 0x7fcb8409b180 2024-03-22T04:11:38.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb7be00640 1 -- 172.21.15.86:0/3395373709 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fcb70059740 con 0x7fcb8409b180 2024-03-22T04:11:38.100 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:11:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb64067b20 msgr2=0x7fcb64069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb64067b20 0x7fcb64069fe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb740ae000 tx=0x7fcb740a7000 comp rx=0 tx=0).stop 2024-03-22T04:11:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8409b180 msgr2=0x7fcb840c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8409b180 0x7fcb840c7090 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000c370 tx=0x7fcb7000c830 comp rx=0 tx=0).stop 2024-03-22T04:11:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 shutdown_connections 2024-03-22T04:11:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb64067b20 0x7fcb64069fe0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb8415e970 0x7fcb8414a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8409b180 0x7fcb840c7090 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 --2- 172.21.15.86:0/3395373709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8406d8f0 0x7fcb84153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 >> 172.21.15.86:0/3395373709 conn(0x7fcb840b3bf0 msgr2=0x7fcb840cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 shutdown_connections 2024-03-22T04:11:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.095+0000 7fcb8b200640 1 -- 172.21.15.86:0/3395373709 wait complete. 2024-03-22T04:11:38.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-22T04:11:38.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf70000640 1 Processor -- start 2024-03-22T04:11:38.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf70000640 1 -- start start 2024-03-22T04:11:38.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf70000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 0x7fcf68162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf70000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf680cc7c0 con 0x7fcf680c6f20 2024-03-22T04:11:38.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf70000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf680cb720 con 0x7fcf680c7300 2024-03-22T04:11:38.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf70000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf680cba00 con 0x7fcf680cb1d0 2024-03-22T04:11:38.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6d600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcf680c6f20 0x7fcf68072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54712/0 (socket says 172.21.15.86:54712) 2024-03-22T04:11:38.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6d600640 1 -- 172.21.15.86:0/3790240671 learned_addr learned my addr 172.21.15.86:0/3790240671 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:38.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6cc00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 0x7fcf68162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 89127107 0 0) 0x7fcf680cc7c0 con 0x7fcf680c6f20 2024-03-22T04:11:38.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf50003530 con 0x7fcf680c6f20 2024-03-22T04:11:38.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358957840 0 0) 0x7fcf680cb720 con 0x7fcf680c7300 2024-03-22T04:11:38.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf680cc7c0 con 0x7fcf680c7300 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1937006549 0 0) 0x7fcf50003530 con 0x7fcf680c6f20 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 msgr2=0x7fcf68162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 0x7fcf68162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 >> v1:172.21.15.196:6789/0 conn(0x7fcf680c7300 legacy=0x7fcf680c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf680cbce0 con 0x7fcf680c6f20 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fcf5c001a00 con 0x7fcf680c6f20 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 >> v1:172.21.15.86:6789/0 conn(0x7fcf680c6f20 legacy=0x7fcf68072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 0x7fcf50003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf50004130 0x7fcf50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004d50 0x7fcf500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680cba00 con 0x7fcf50004130 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5000bb10 con 0x7fcf50004d50 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5000bd10 con 0x7fcf680cb1d0 2024-03-22T04:11:38.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6cc00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf50004130 0x7fcf50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6d600640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 0x7fcf50003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6e000640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004d50 0x7fcf500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6cc00640 1 -- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 msgr2=0x7fcf50003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6cc00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 0x7fcf50003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6cc00640 1 -- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004d50 msgr2=0x7fcf500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6cc00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004d50 0x7fcf500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.255+0000 7fcf6cc00640 1 -- 172.21.15.86:0/3790240671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf50003530 con 0x7fcf50004130 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf6d600640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680cb1d0 0x7fcf50003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:38.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf6e000640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004d50 0x7fcf500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf6cc00640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf50004130 0x7fcf50004810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009ae0 tx=0x7fcf5401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5be38605fd97b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf540b2070 con 0x7fcf50004130 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf540a55a0 con 0x7fcf50004130 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf67e00640 1 -- 172.21.15.86:0/3790240671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf540096b0 con 0x7fcf50004130 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf70000640 1 -- 172.21.15.86:0/3790240671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf50004130 msgr2=0x7fcf50004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf70000640 1 --2- 172.21.15.86:0/3790240671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf50004130 0x7fcf50004810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009ae0 tx=0x7fcf5401c9b0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf6e000640 1 -- 172.21.15.86:0/3790240671 reap_dead start 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf70000640 1 -- 172.21.15.86:0/3790240671 shutdown_connections 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf70000640 1 -- 172.21.15.86:0/3790240671 >> 172.21.15.86:0/3790240671 conn(0x7fcf680d0730 msgr2=0x7fcf680d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf70000640 1 -- 172.21.15.86:0/3790240671 shutdown_connections 2024-03-22T04:11:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.259+0000 7fcf70000640 1 -- 172.21.15.86:0/3790240671 wait complete. 2024-03-22T04:11:38.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 Processor -- start 2024-03-22T04:11:38.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 -- start start 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004130 0x7fcf680ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680b55b0 0x7fcf680b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf680b5f70 0x7fcf68147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf68162030 con 0x7fcf680b5f70 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680cc2a0 con 0x7fcf50004130 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf681621b0 con 0x7fcf680b55b0 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf680b5f70 0x7fcf68147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf680b5f70 0x7fcf68147810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59466/0 (socket says 172.21.15.86:59466) 2024-03-22T04:11:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 -- 172.21.15.86:0/779174409 learned_addr learned my addr 172.21.15.86:0/779174409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6d600640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004130 0x7fcf680ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6cc00640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680b55b0 0x7fcf680b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 -- 172.21.15.86:0/779174409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680b55b0 msgr2=0x7fcf680b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680b55b0 0x7fcf680b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 -- 172.21.15.86:0/779174409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004130 msgr2=0x7fcf680ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004130 0x7fcf680ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 -- 172.21.15.86:0/779174409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6806e730 con 0x7fcf680b5f70 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf6e000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf680b5f70 0x7fcf68147810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcf680cad30 tx=0x7fcf6000f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf65600640 1 -- 172.21.15.86:0/779174409 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60010040 con 0x7fcf680b5f70 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf65600640 1 -- 172.21.15.86:0/779174409 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6001a070 con 0x7fcf680b5f70 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf65600640 1 -- 172.21.15.86:0/779174409 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60015410 con 0x7fcf680b5f70 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6806e9f0 con 0x7fcf680b5f70 2024-03-22T04:11:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.263+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6806efb0 con 0x7fcf680b5f70 2024-03-22T04:11:38.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.267+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf68074470 con 0x7fcf680b5f70 2024-03-22T04:11:38.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.267+0000 7fcf65600640 1 -- 172.21.15.86:0/779174409 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcf600155b0 con 0x7fcf680b5f70 2024-03-22T04:11:38.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.267+0000 7fcf65600640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf48067b50 0x7fcf4806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.267+0000 7fcf65600640 1 -- 172.21.15.86:0/779174409 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fcf60089590 con 0x7fcf680b5f70 2024-03-22T04:11:38.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.267+0000 7fcf6d600640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf48067b50 0x7fcf4806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.267+0000 7fcf6d600640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf48067b50 0x7fcf4806a010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c005e20 tx=0x7fcf5c002100 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.271+0000 7fcf65600640 1 -- 172.21.15.86:0/779174409 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcf600595a0 con 0x7fcf680b5f70 2024-03-22T04:11:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.423+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcf681e85e0 con 0x7fcf680b5f70 2024-03-22T04:11:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.423+0000 7fcf65600640 1 -- 172.21.15.86:0/779174409 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fcf6005d640 con 0x7fcf680b5f70 2024-03-22T04:11:38.429 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf48067b50 msgr2=0x7fcf4806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf48067b50 0x7fcf4806a010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c005e20 tx=0x7fcf5c002100 comp rx=0 tx=0).stop 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf680b5f70 msgr2=0x7fcf68147810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf680b5f70 0x7fcf68147810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcf680cad30 tx=0x7fcf6000f540 comp rx=0 tx=0).stop 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 shutdown_connections 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf48067b50 0x7fcf4806a010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf680b5f70 0x7fcf68147810 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf680b55b0 0x7fcf680b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 --2- 172.21.15.86:0/779174409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf50004130 0x7fcf680ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 >> 172.21.15.86:0/779174409 conn(0x7fcf680d0730 msgr2=0x7fcf681e8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:38.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 shutdown_connections 2024-03-22T04:11:38.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.427+0000 7fcf70000640 1 -- 172.21.15.86:0/779174409 wait complete. 2024-03-22T04:11:38.449 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:11:38.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4fc00640 1 Processor -- start 2024-03-22T04:11:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4fc00640 1 -- start start 2024-03-22T04:11:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4fc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f48071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f4806e2f0 0x7f9f48071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4fc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f481623d0 con 0x7f9f4815e630 2024-03-22T04:11:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9f481620f0 con 0x7f9f4806e2f0 2024-03-22T04:11:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4fc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4815ec10 con 0x7f9f480d0580 2024-03-22T04:11:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f4806e2f0 0x7f9f48071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4d200640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f9f4815e630 0x7f9f480cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:51680/0 (socket says 172.21.15.86:51680) 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4d200640 1 -- 172.21.15.86:0/4258713857 learned_addr learned my addr 172.21.15.86:0/4258713857 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f3fe00640 1 --2- 172.21.15.86:0/4258713857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f48071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4dc00640 1 -- 172.21.15.86:0/4258713857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 msgr2=0x7f9f48071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4dc00640 1 --2- 172.21.15.86:0/4258713857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f48071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4dc00640 1 -- 172.21.15.86:0/4258713857 >> v1:172.21.15.196:6789/0 conn(0x7f9f4815e630 legacy=0x7f9f480cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f47e00640 1 -- 172.21.15.86:0/4258713857 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2610073918 0 0) 0x7f9f481623d0 con 0x7f9f4815e630 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f4dc00640 1 -- 172.21.15.86:0/4258713857 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4815eef0 con 0x7f9f4806e2f0 2024-03-22T04:11:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.579+0000 7f9f3fe00640 1 --2- 172.21.15.86:0/4258713857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f48071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:38.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4dc00640 1 --2- 172.21.15.86:0/4258713857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48071640 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009d90 tx=0x7f9f4001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8cc81c571a2347 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f47e00640 1 -- 172.21.15.86:0/4258713857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400b2070 con 0x7f9f4806e2f0 2024-03-22T04:11:38.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f47e00640 1 -- 172.21.15.86:0/4258713857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f400a5830 con 0x7f9f4806e2f0 2024-03-22T04:11:38.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f47e00640 1 -- 172.21.15.86:0/4258713857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400a5c00 con 0x7f9f4806e2f0 2024-03-22T04:11:38.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4fc00640 1 -- 172.21.15.86:0/4258713857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 msgr2=0x7f9f48071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/4258713857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48071640 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009d90 tx=0x7f9f4001c970 comp rx=0 tx=0).stop 2024-03-22T04:11:38.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4fc00640 1 -- 172.21.15.86:0/4258713857 shutdown_connections 2024-03-22T04:11:38.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/4258713857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f48071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/4258713857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48071640 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4fc00640 1 -- 172.21.15.86:0/4258713857 >> 172.21.15.86:0/4258713857 conn(0x7f9f4806f910 msgr2=0x7f9f4806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:38.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4fc00640 1 -- 172.21.15.86:0/4258713857 shutdown_connections 2024-03-22T04:11:38.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.583+0000 7f9f4fc00640 1 -- 172.21.15.86:0/4258713857 wait complete. 2024-03-22T04:11:38.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 Processor -- start 2024-03-22T04:11:38.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 -- start start 2024-03-22T04:11:38.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f480b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f4815e630 0x7f9f480c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48071f90 con 0x7f9f4806e2f0 2024-03-22T04:11:38.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4815f9d0 con 0x7f9f4815e630 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9f480cfaf0 con 0x7f9f480d0580 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59478/0 (socket says 172.21.15.86:59478) 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 -- 172.21.15.86:0/2169162681 learned_addr learned my addr 172.21.15.86:0/2169162681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4dc00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f4815e630 0x7f9f480c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f3fe00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f480b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 -- 172.21.15.86:0/2169162681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 msgr2=0x7f9f480b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f480b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 -- 172.21.15.86:0/2169162681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f4815e630 msgr2=0x7f9f480c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f4815e630 0x7f9f480c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 -- 172.21.15.86:0/2169162681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f48155e70 con 0x7f9f4806e2f0 2024-03-22T04:11:38.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4d200640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48155930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9f3800c480 tx=0x7f9f3800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f45600640 1 -- 172.21.15.86:0/2169162681 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38018070 con 0x7f9f4806e2f0 2024-03-22T04:11:38.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f45600640 1 -- 172.21.15.86:0/2169162681 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f38007500 con 0x7f9f4806e2f0 2024-03-22T04:11:38.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f480b55c0 con 0x7f9f4806e2f0 2024-03-22T04:11:38.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f45600640 1 -- 172.21.15.86:0/2169162681 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38004a30 con 0x7f9f4806e2f0 2024-03-22T04:11:38.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f48158480 con 0x7f9f4806e2f0 2024-03-22T04:11:38.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.587+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f48068650 con 0x7f9f4806e2f0 2024-03-22T04:11:38.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.591+0000 7f9f45600640 1 -- 172.21.15.86:0/2169162681 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9f3801b070 con 0x7f9f4806e2f0 2024-03-22T04:11:38.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.591+0000 7f9f45600640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f24067b20 0x7f9f24069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.591+0000 7f9f45600640 1 -- 172.21.15.86:0/2169162681 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f9f38085490 con 0x7f9f4806e2f0 2024-03-22T04:11:38.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.591+0000 7f9f3fe00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f24067b20 0x7f9f24069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.591+0000 7f9f3fe00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f24067b20 0x7f9f24069fe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009580 tx=0x7f9f30009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.595+0000 7f9f45600640 1 -- 172.21.15.86:0/2169162681 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9f48155e70 con 0x7f9f4806e2f0 2024-03-22T04:11:38.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.779+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f480b57f0 con 0x7f9f4806e2f0 2024-03-22T04:11:38.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f45600640 1 -- 172.21.15.86:0/2169162681 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5179 (secure 0 0 0) 0x7f9f38055530 con 0x7f9f4806e2f0 2024-03-22T04:11:38.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:38.787 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:58.258965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:11:38.787 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T04:11:38.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f24067b20 msgr2=0x7f9f24069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f24067b20 0x7f9f24069fe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009580 tx=0x7f9f30009190 comp rx=0 tx=0).stop 2024-03-22T04:11:38.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 msgr2=0x7f9f48155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48155930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9f3800c480 tx=0x7f9f3800c940 comp rx=0 tx=0).stop 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 shutdown_connections 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f24067b20 0x7f9f24069fe0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f4815e630 0x7f9f480c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f480d0580 0x7f9f480b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 --2- 172.21.15.86:0/2169162681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4806e2f0 0x7f9f48155930 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 >> 172.21.15.86:0/2169162681 conn(0x7f9f4806f910 msgr2=0x7f9f480cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 shutdown_connections 2024-03-22T04:11:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.783+0000 7f9f4fc00640 1 -- 172.21.15.86:0/2169162681 wait complete. 2024-03-22T04:11:38.808 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.939+0000 7f7933400640 1 Processor -- start 2024-03-22T04:11:38.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.939+0000 7f7933400640 1 -- start start 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7933400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7933400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792c0d11c0 con 0x7f792c0d1dc0 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7933400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792c0cece0 con 0x7f792c0d19a0 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7933400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f792c0d0580 con 0x7f792c0d0de0 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7931400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f792c0d1dc0 0x7f792c15fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54720/0 (socket says 172.21.15.86:54720) 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7931400640 1 -- 172.21.15.86:0/258713131 learned_addr learned my addr 172.21.15.86:0/258713131 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7930a00640 1 --2- 172.21.15.86:0/258713131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f792b400640 1 -- 172.21.15.86:0/258713131 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3834059158 0 0) 0x7f792c0d11c0 con 0x7f792c0d1dc0 2024-03-22T04:11:38.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f792b400640 1 -- 172.21.15.86:0/258713131 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7914003530 con 0x7f792c0d1dc0 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7930a00640 1 -- 172.21.15.86:0/258713131 >> v1:172.21.15.196:6789/0 conn(0x7f792c0d19a0 legacy=0x7f792c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7930a00640 1 -- 172.21.15.86:0/258713131 >> v1:172.21.15.86:6789/0 conn(0x7f792c0d1dc0 legacy=0x7f792c15fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.943+0000 7f7930a00640 1 -- 172.21.15.86:0/258713131 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c0cefc0 con 0x7f792c0d0de0 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7930a00640 1 --2- 172.21.15.86:0/258713131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0ce6b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79200097f0 tx=0x7f792001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232ff19ab47e8d8d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f792b400640 1 -- 172.21.15.86:0/258713131 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200b1070 con 0x7f792c0d0de0 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f792b400640 1 -- 172.21.15.86:0/258713131 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79200ac630 con 0x7f792c0d0de0 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f792b400640 1 -- 172.21.15.86:0/258713131 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200a8050 con 0x7f792c0d0de0 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7933400640 1 -- 172.21.15.86:0/258713131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 msgr2=0x7f792c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7933400640 1 --2- 172.21.15.86:0/258713131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0ce6b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79200097f0 tx=0x7f792001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7933400640 1 -- 172.21.15.86:0/258713131 shutdown_connections 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7933400640 1 --2- 172.21.15.86:0/258713131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0ce6b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7933400640 1 -- 172.21.15.86:0/258713131 >> 172.21.15.86:0/258713131 conn(0x7f792c06f910 msgr2=0x7f792c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7933400640 1 -- 172.21.15.86:0/258713131 shutdown_connections 2024-03-22T04:11:38.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7933400640 1 -- 172.21.15.86:0/258713131 wait complete. 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.947+0000 7f7933400640 1 Processor -- start 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 -- start start 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f792c0d19a0 0x7f792c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f792c0d1dc0 0x7f792c0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f792c160760 con 0x7f792c0d19a0 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1605e0 con 0x7f792c0d1dc0 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f792c071e30 con 0x7f792c0d0de0 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f792c0d19a0 0x7f792c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f792c0d19a0 0x7f792c0b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59482/0 (socket says 172.21.15.86:59482) 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 -- 172.21.15.86:0/2489252559 learned_addr learned my addr 172.21.15.86:0/2489252559 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7930a00640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7931400640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f792c0d1dc0 0x7f792c0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 -- 172.21.15.86:0/2489252559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 msgr2=0x7f792c0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 -- 172.21.15.86:0/2489252559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f792c0d1dc0 msgr2=0x7f792c0b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f792c0d1dc0 0x7f792c0b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 -- 172.21.15.86:0/2489252559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c0cc2c0 con 0x7f792c0d19a0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f792be00640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f792c0d19a0 0x7f792c0b7180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f791800c420 tx=0x7f791800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7928c00640 1 -- 172.21.15.86:0/2489252559 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79180052d0 con 0x7f792c0d19a0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7928c00640 1 -- 172.21.15.86:0/2489252559 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7918005470 con 0x7f792c0d19a0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c0cbfe0 con 0x7f792c0d19a0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7928c00640 1 -- 172.21.15.86:0/2489252559 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918007260 con 0x7f792c0d19a0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c0b6980 con 0x7f792c0d19a0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.951+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c0b64c0 con 0x7f792c0d19a0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.955+0000 7f7928c00640 1 -- 172.21.15.86:0/2489252559 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7918007400 con 0x7f792c0d19a0 2024-03-22T04:11:38.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.955+0000 7f7928c00640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f790c067b20 0x7f790c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:38.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.955+0000 7f7928c00640 1 -- 172.21.15.86:0/2489252559 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f7918085680 con 0x7f792c0d19a0 2024-03-22T04:11:38.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.955+0000 7f7930a00640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f790c067b20 0x7f790c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:38.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.955+0000 7f7930a00640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f790c067b20 0x7f790c069fe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79200ae000 tx=0x7f7920008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:38.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:38.959+0000 7f7928c00640 1 -- 172.21.15.86:0/2489252559 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f792c0b64c0 con 0x7f792c0d19a0 2024-03-22T04:11:39.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f792c068650 con 0x7f792c0d19a0 2024-03-22T04:11:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7928c00640 1 -- 172.21.15.86:0/2489252559 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7f7918055720 con 0x7f792c0d19a0 2024-03-22T04:11:39.124 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:39.124 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f790c067b20 msgr2=0x7f790c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f790c067b20 0x7f790c069fe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79200ae000 tx=0x7f7920008040 comp rx=0 tx=0).stop 2024-03-22T04:11:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f792c0d19a0 msgr2=0x7f792c0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f792c0d19a0 0x7f792c0b7180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f791800c420 tx=0x7f791800c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 shutdown_connections 2024-03-22T04:11:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f790c067b20 0x7f790c069fe0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f792c0d1dc0 0x7f792c0b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f792c0d19a0 0x7f792c0b7180 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 --2- 172.21.15.86:0/2489252559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f792c0d0de0 0x7f792c0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 >> 172.21.15.86:0/2489252559 conn(0x7f792c06f910 msgr2=0x7f792c0cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 shutdown_connections 2024-03-22T04:11:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.119+0000 7f7933400640 1 -- 172.21.15.86:0/2489252559 wait complete. 2024-03-22T04:11:39.146 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:11:39.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:11:39.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe49ba00640 1 Processor -- start 2024-03-22T04:11:39.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe49ba00640 1 -- start start 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe49ba00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe49406ef40 0x7fe494142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe49ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4940ad710 0x7fe49415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe49ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49415c7c0 con 0x7fe4940ad330 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe49ba00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe49406dea0 con 0x7fe49406ef40 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe49ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe49406e180 con 0x7fe4940ad710 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe499000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe4940ad330 0x7fe49413fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54734/0 (socket says 172.21.15.86:54734) 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe499000640 1 -- 172.21.15.86:0/161861649 learned_addr learned my addr 172.21.15.86:0/161861649 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe499a00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4940ad710 0x7fe49415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe48be00640 1 --2- 172.21.15.86:0/161861649 >> v2:172.21.15.196:3300/0 conn(0x7fe49406ef40 0x7fe494142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.275+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3268532131 0 0) 0x7fe49415c7c0 con 0x7fe4940ad330 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe478003530 con 0x7fe4940ad330 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4213512383 0 0) 0x7fe478003530 con 0x7fe4940ad330 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4940ad710 msgr2=0x7fe49415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4940ad710 0x7fe49415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 >> v2:172.21.15.196:3300/0 conn(0x7fe49406ef40 msgr2=0x7fe494142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 --2- 172.21.15.86:0/161861649 >> v2:172.21.15.196:3300/0 conn(0x7fe49406ef40 0x7fe494142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49406e460 con 0x7fe4940ad330 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fe484001680 con 0x7fe4940ad330 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe48be00640 1 --2- 172.21.15.86:0/161861649 >> v2:172.21.15.196:3300/0 conn(0x7fe49406ef40 0x7fe494142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:39.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 >> v1:172.21.15.86:6789/0 conn(0x7fe4940ad330 legacy=0x7fe49413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4780039c0 0x7fe478003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4940ad710 0x7fe478004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe478004d50 0x7fe4780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe49406dea0 con 0x7fe478004d50 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe49406e180 con 0x7fe4780039c0 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe47800b9d0 con 0x7fe4940ad710 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe499a00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe478004d50 0x7fe4780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe499000640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4780039c0 0x7fe478003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe48be00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4940ad710 0x7fe478004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe499a00640 1 -- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4940ad710 msgr2=0x7fe478004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe499a00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4940ad710 0x7fe478004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe499a00640 1 -- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4780039c0 msgr2=0x7fe478003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe499a00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4780039c0 0x7fe478003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.279+0000 7fe499a00640 1 -- 172.21.15.86:0/161861649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe478003530 con 0x7fe478004d50 2024-03-22T04:11:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe499a00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe478004d50 0x7fe4780051d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009fc0 tx=0x7fe48c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595d519c95bed40 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0b2070 con 0x7fe478004d50 2024-03-22T04:11:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48c0a5c90 con 0x7fe478004d50 2024-03-22T04:11:39.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe493e00640 1 -- 172.21.15.86:0/161861649 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c009c10 con 0x7fe478004d50 2024-03-22T04:11:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- 172.21.15.86:0/161861649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe478004d50 msgr2=0x7fe4780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 --2- 172.21.15.86:0/161861649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe478004d50 0x7fe4780051d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009fc0 tx=0x7fe48c01cc80 comp rx=0 tx=0).stop 2024-03-22T04:11:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe499a00640 1 -- 172.21.15.86:0/161861649 reap_dead start 2024-03-22T04:11:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- 172.21.15.86:0/161861649 shutdown_connections 2024-03-22T04:11:39.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- 172.21.15.86:0/161861649 >> 172.21.15.86:0/161861649 conn(0x7fe4940b1df0 msgr2=0x7fe4940b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- 172.21.15.86:0/161861649 shutdown_connections 2024-03-22T04:11:39.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- 172.21.15.86:0/161861649 wait complete. 2024-03-22T04:11:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 Processor -- start 2024-03-22T04:11:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- start start 2024-03-22T04:11:39.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe478004d50 0x7fe494157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4941f3e20 0x7fe49415a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49409a1e0 0x7fe494159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe49415d4a0 con 0x7fe49409a1e0 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe49406ea20 con 0x7fe478004d50 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe49ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe49415d620 con 0x7fe4941f3e20 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe499a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49409a1e0 0x7fe494159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe499000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe478004d50 0x7fe494157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe48be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4941f3e20 0x7fe49415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe499000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe478004d50 0x7fe494157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:37590/0 (socket says 172.21.15.86:37590) 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.283+0000 7fe499000640 1 -- 172.21.15.86:0/135204780 learned_addr learned my addr 172.21.15.86:0/135204780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe499a00640 1 -- 172.21.15.86:0/135204780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4941f3e20 msgr2=0x7fe49415a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe499a00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4941f3e20 0x7fe49415a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe499a00640 1 -- 172.21.15.86:0/135204780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe478004d50 msgr2=0x7fe494157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe499a00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe478004d50 0x7fe494157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe499a00640 1 -- 172.21.15.86:0/135204780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4941f43e0 con 0x7fe49409a1e0 2024-03-22T04:11:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe499a00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49409a1e0 0x7fe494159520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0ac000 tx=0x7fe48c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe491600640 1 -- 172.21.15.86:0/135204780 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0b2040 con 0x7fe49409a1e0 2024-03-22T04:11:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe491600640 1 -- 172.21.15.86:0/135204780 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48c0a5a90 con 0x7fe49409a1e0 2024-03-22T04:11:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe491600640 1 -- 172.21.15.86:0/135204780 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c009c10 con 0x7fe49409a1e0 2024-03-22T04:11:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4941f46c0 con 0x7fe49409a1e0 2024-03-22T04:11:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4941f4c80 con 0x7fe49409a1e0 2024-03-22T04:11:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe494068650 con 0x7fe49409a1e0 2024-03-22T04:11:39.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe491600640 1 -- 172.21.15.86:0/135204780 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe48c0be020 con 0x7fe49409a1e0 2024-03-22T04:11:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe491600640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe470067b10 0x7fe470069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe491600640 1 -- 172.21.15.86:0/135204780 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fe48c120260 con 0x7fe49409a1e0 2024-03-22T04:11:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.287+0000 7fe499000640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe470067b10 0x7fe470069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.291+0000 7fe499000640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe470067b10 0x7fe470069fd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe494140fd0 tx=0x7fe484007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:39.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.291+0000 7fe491600640 1 -- 172.21.15.86:0/135204780 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe48c0c5020 con 0x7fe49409a1e0 2024-03-22T04:11:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.479+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4940b17d0 con 0x7fe49409a1e0 2024-03-22T04:11:39.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe491600640 1 -- 172.21.15.86:0/135204780 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5179 (secure 0 0 0) 0x7fe48c016990 con 0x7fe49409a1e0 2024-03-22T04:11:39.486 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:39.486 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:58.258965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:11:39.487 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T04:11:39.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe470067b10 msgr2=0x7fe470069fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe470067b10 0x7fe470069fd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe494140fd0 tx=0x7fe484007900 comp rx=0 tx=0).stop 2024-03-22T04:11:39.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49409a1e0 msgr2=0x7fe494159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49409a1e0 0x7fe494159520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0ac000 tx=0x7fe48c009750 comp rx=0 tx=0).stop 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 shutdown_connections 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe470067b10 0x7fe470069fd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49409a1e0 0x7fe494159520 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4941f3e20 0x7fe49415a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 --2- 172.21.15.86:0/135204780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe478004d50 0x7fe494157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 >> 172.21.15.86:0/135204780 conn(0x7fe4940b1df0 msgr2=0x7fe4940b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 shutdown_connections 2024-03-22T04:11:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.483+0000 7fe49ba00640 1 -- 172.21.15.86:0/135204780 wait complete. 2024-03-22T04:11:39.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:11:39.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5bb200640 1 Processor -- start 2024-03-22T04:11:39.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5bb200640 1 -- start start 2024-03-22T04:11:39.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5bb200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd5b4161880 0x7fd5b40b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5bb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd5b4161c60 0x7fd5b40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5bb200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b406d700 0x7fd5b406db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5bb200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b415ee80 con 0x7fd5b4161c60 2024-03-22T04:11:39.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5bb200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b406e0b0 con 0x7fd5b4161880 2024-03-22T04:11:39.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5bb200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b406e350 con 0x7fd5b406d700 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd5b4161c60 0x7fd5b40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd5b4161880 0x7fd5b40b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd5b4161c60 0x7fd5b40b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59512/0 (socket says 172.21.15.86:59512) 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b9200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b406d700 0x7fd5b406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3400640 1 -- 172.21.15.86:0/1854364209 learned_addr learned my addr 172.21.15.86:0/1854364209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3e00640 1 -- 172.21.15.86:0/1854364209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b406d700 msgr2=0x7fd5b406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3e00640 1 --2- 172.21.15.86:0/1854364209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b406d700 0x7fd5b406db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3e00640 1 -- 172.21.15.86:0/1854364209 >> v2:172.21.15.86:3300/0 conn(0x7fd5b4161c60 msgr2=0x7fd5b40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3e00640 1 --2- 172.21.15.86:0/1854364209 >> v2:172.21.15.86:3300/0 conn(0x7fd5b4161c60 0x7fd5b40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3e00640 1 -- 172.21.15.86:0/1854364209 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b406e630 con 0x7fd5b4161880 2024-03-22T04:11:39.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.639+0000 7fd5b3400640 1 --2- 172.21.15.86:0/1854364209 >> v2:172.21.15.86:3300/0 conn(0x7fd5b4161c60 0x7fd5b40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:39.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5b3e00640 1 --2- 172.21.15.86:0/1854364209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 0x7fd5b40b30f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5b406c510 tx=0x7fd5a401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a66418bf519b9c13 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5b2a00640 1 -- 172.21.15.86:0/1854364209 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a40b2070 con 0x7fd5b4161880 2024-03-22T04:11:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5b2a00640 1 -- 172.21.15.86:0/1854364209 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a40ad680 con 0x7fd5b4161880 2024-03-22T04:11:39.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5b2a00640 1 -- 172.21.15.86:0/1854364209 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a40ada50 con 0x7fd5b4161880 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 -- 172.21.15.86:0/1854364209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 msgr2=0x7fd5b40b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1854364209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 0x7fd5b40b30f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5b406c510 tx=0x7fd5a401c3c0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 -- 172.21.15.86:0/1854364209 shutdown_connections 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1854364209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b406d700 0x7fd5b406db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1854364209 >> v2:172.21.15.86:3300/0 conn(0x7fd5b4161c60 0x7fd5b40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1854364209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 0x7fd5b40b30f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 -- 172.21.15.86:0/1854364209 >> 172.21.15.86:0/1854364209 conn(0x7fd5b40b1df0 msgr2=0x7fd5b40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 -- 172.21.15.86:0/1854364209 shutdown_connections 2024-03-22T04:11:39.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 -- 172.21.15.86:0/1854364209 wait complete. 2024-03-22T04:11:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.643+0000 7fd5bb200640 1 Processor -- start 2024-03-22T04:11:39.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 -- start start 2024-03-22T04:11:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b406d700 0x7fd5b4099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 0x7fd5b4159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b4161c60 0x7fd5b415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4162a60 con 0x7fd5b406d700 2024-03-22T04:11:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b41628e0 con 0x7fd5b4161880 2024-03-22T04:11:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b4162be0 con 0x7fd5b4161c60 2024-03-22T04:11:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b406d700 0x7fd5b4099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b406d700 0x7fd5b4099880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59520/0 (socket says 172.21.15.86:59520) 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 -- 172.21.15.86:0/1263426821 learned_addr learned my addr 172.21.15.86:0/1263426821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3400640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 0x7fd5b4159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b9200640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b4161c60 0x7fd5b415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 -- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b4161c60 msgr2=0x7fd5b415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b4161c60 0x7fd5b415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 -- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 msgr2=0x7fd5b4159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 0x7fd5b4159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 -- 172.21.15.86:0/1263426821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b4158b60 con 0x7fd5b406d700 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b9200640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b4161c60 0x7fd5b415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5b3e00640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b406d700 0x7fd5b4099880 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd5a401cc60 tx=0x7fd5a401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5abe00640 1 -- 172.21.15.86:0/1263426821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a40b2040 con 0x7fd5b406d700 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5abe00640 1 -- 172.21.15.86:0/1263426821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a40be070 con 0x7fd5b406d700 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b4158e40 con 0x7fd5b406d700 2024-03-22T04:11:39.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5abe00640 1 -- 172.21.15.86:0/1263426821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a40ade10 con 0x7fd5b406d700 2024-03-22T04:11:39.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b4159400 con 0x7fd5b406d700 2024-03-22T04:11:39.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.647+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b4068650 con 0x7fd5b406d700 2024-03-22T04:11:39.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.651+0000 7fd5abe00640 1 -- 172.21.15.86:0/1263426821 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd5a40c3460 con 0x7fd5b406d700 2024-03-22T04:11:39.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.651+0000 7fd5abe00640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd594067b20 0x7fd594069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:39.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.651+0000 7fd5abe00640 1 -- 172.21.15.86:0/1263426821 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fd5a4120500 con 0x7fd5b406d700 2024-03-22T04:11:39.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.651+0000 7fd5b3400640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd594067b20 0x7fd594069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:39.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.651+0000 7fd5b3400640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd594067b20 0x7fd594069fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009630 tx=0x7fd5a0009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:39.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.655+0000 7fd5abe00640 1 -- 172.21.15.86:0/1263426821 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd5a4016070 con 0x7fd5b406d700 2024-03-22T04:11:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.843+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5b40b1640 con 0x7fd5b406d700 2024-03-22T04:11:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.843+0000 7fd5abe00640 1 -- 172.21.15.86:0/1263426821 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5179 (secure 0 0 0) 0x7fd5a40f0510 con 0x7fd5b406d700 2024-03-22T04:11:39.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:39.850 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/417854358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":417854358},{"type":"v1","addr":"172.21.15.86:6836","nonce":417854358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/692673174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":692673174},{"type":"v1","addr":"172.21.15.86:6837","nonce":692673174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1972723344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1972723344},{"type":"v1","addr":"172.21.15.196:6834","nonce":1972723344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:10:34.298556+0000","modified":"2024-03-22T04:10:58.258965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74171":{"gid":74171,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/180015071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":180015071},{"type":"v1","addr":"172.21.15.196:6835","nonce":180015071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74171,"qdb_cluster":[74171]},"id":1}]} 2024-03-22T04:11:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-22T04:11:39.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd594067b20 msgr2=0x7fd594069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd594067b20 0x7fd594069fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009630 tx=0x7fd5a0009240 comp rx=0 tx=0).stop 2024-03-22T04:11:39.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b406d700 msgr2=0x7fd5b4099880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b406d700 0x7fd5b4099880 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd5a401cc60 tx=0x7fd5a401ac00 comp rx=0 tx=0).stop 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 shutdown_connections 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd594067b20 0x7fd594069fe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5b4161880 0x7fd5b4159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5b4161c60 0x7fd5b415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 --2- 172.21.15.86:0/1263426821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b406d700 0x7fd5b4099880 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 >> 172.21.15.86:0/1263426821 conn(0x7fd5b40b1df0 msgr2=0x7fd5b4160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 shutdown_connections 2024-03-22T04:11:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:39.847+0000 7fd5bb200640 1 -- 172.21.15.86:0/1263426821 wait complete. 2024-03-22T04:11:39.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5994200640 1 Processor -- start 2024-03-22T04:11:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5994200640 1 -- start start 2024-03-22T04:11:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5994200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f598c0d0de0 0x7f598c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5994200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f598c0d11c0 con 0x7f598c0d19a0 2024-03-22T04:11:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5994200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f598c0cece0 con 0x7f598c15fed0 2024-03-22T04:11:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5994200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f598c0d0580 con 0x7f598c0d0de0 2024-03-22T04:11:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5990e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f598c0d19a0 0x7f598c0cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54744/0 (socket says 172.21.15.86:54744) 2024-03-22T04:11:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5990e00640 1 -- 172.21.15.86:0/3365039535 learned_addr learned my addr 172.21.15.86:0/3365039535 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5991800640 1 --2- 172.21.15.86:0/3365039535 >> v2:172.21.15.196:3300/0 conn(0x7f598c0d0de0 0x7f598c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f598be00640 1 -- 172.21.15.86:0/3365039535 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765074280 0 0) 0x7f598c0d11c0 con 0x7f598c0d19a0 2024-03-22T04:11:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f598be00640 1 -- 172.21.15.86:0/3365039535 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5974003530 con 0x7f598c0d19a0 2024-03-22T04:11:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f598be00640 1 -- 172.21.15.86:0/3365039535 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009907434 0 0) 0x7f598c0cece0 con 0x7f598c15fed0 2024-03-22T04:11:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f598be00640 1 -- 172.21.15.86:0/3365039535 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f598c0d11c0 con 0x7f598c15fed0 2024-03-22T04:11:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5991800640 1 -- 172.21.15.86:0/3365039535 >> v1:172.21.15.196:6789/0 conn(0x7f598c15fed0 legacy=0x7f598c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5991800640 1 -- 172.21.15.86:0/3365039535 >> v1:172.21.15.86:6789/0 conn(0x7f598c0d19a0 legacy=0x7f598c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.003+0000 7f5991800640 1 -- 172.21.15.86:0/3365039535 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598c0cefc0 con 0x7f598c0d0de0 2024-03-22T04:11:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5991800640 1 --2- 172.21.15.86:0/3365039535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 0x7f598c0ce6b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f59800097c0 tx=0x7f598001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c64449424154635 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f598be00640 1 -- 172.21.15.86:0/3365039535 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59800b1070 con 0x7f598c0d0de0 2024-03-22T04:11:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f598be00640 1 -- 172.21.15.86:0/3365039535 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59800ac5f0 con 0x7f598c0d0de0 2024-03-22T04:11:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f598be00640 1 -- 172.21.15.86:0/3365039535 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59800ac9c0 con 0x7f598c0d0de0 2024-03-22T04:11:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 -- 172.21.15.86:0/3365039535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 msgr2=0x7f598c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 --2- 172.21.15.86:0/3365039535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 0x7f598c0ce6b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f59800097c0 tx=0x7f598001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 -- 172.21.15.86:0/3365039535 shutdown_connections 2024-03-22T04:11:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 --2- 172.21.15.86:0/3365039535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 0x7f598c0ce6b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 -- 172.21.15.86:0/3365039535 >> 172.21.15.86:0/3365039535 conn(0x7f598c06f910 msgr2=0x7f598c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 -- 172.21.15.86:0/3365039535 shutdown_connections 2024-03-22T04:11:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 -- 172.21.15.86:0/3365039535 wait complete. 2024-03-22T04:11:40.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 Processor -- start 2024-03-22T04:11:40.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.007+0000 7f5994200640 1 -- start start 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 0x7f598c153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d19a0 0x7f598c1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f598c15fed0 0x7f598c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f598c0d1f30 con 0x7f598c0d19a0 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f598c0d1db0 con 0x7f598c0d0de0 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f598c071e30 con 0x7f598c15fed0 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d19a0 0x7f598c1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d19a0 0x7f598c1587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59524/0 (socket says 172.21.15.86:59524) 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 -- 172.21.15.86:0/4194066995 learned_addr learned my addr 172.21.15.86:0/4194066995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5991800640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 0x7f598c153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5992200640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f598c15fed0 0x7f598c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 -- 172.21.15.86:0/4194066995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f598c15fed0 msgr2=0x7f598c1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f598c15fed0 0x7f598c1543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 -- 172.21.15.86:0/4194066995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 msgr2=0x7f598c153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 0x7f598c153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 -- 172.21.15.86:0/4194066995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598c155f00 con 0x7f598c0d19a0 2024-03-22T04:11:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5990e00640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d19a0 0x7f598c1587e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f597800c500 tx=0x7f597800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5989600640 1 -- 172.21.15.86:0/4194066995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978017070 con 0x7f598c0d19a0 2024-03-22T04:11:40.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5989600640 1 -- 172.21.15.86:0/4194066995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5978005cf0 con 0x7f598c0d19a0 2024-03-22T04:11:40.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598c1561e0 con 0x7f598c0d19a0 2024-03-22T04:11:40.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5989600640 1 -- 172.21.15.86:0/4194066995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978007330 con 0x7f598c0d19a0 2024-03-22T04:11:40.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f598c159c60 con 0x7f598c0d19a0 2024-03-22T04:11:40.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.011+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f598c068650 con 0x7f598c0d19a0 2024-03-22T04:11:40.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.015+0000 7f5989600640 1 -- 172.21.15.86:0/4194066995 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f597801a070 con 0x7f598c0d19a0 2024-03-22T04:11:40.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.015+0000 7f5989600640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f596c067b20 0x7f596c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.015+0000 7f5989600640 1 -- 172.21.15.86:0/4194066995 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f59780856a0 con 0x7f598c0d19a0 2024-03-22T04:11:40.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.015+0000 7f5991800640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f596c067b20 0x7f596c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.015+0000 7f5991800640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f596c067b20 0x7f596c069fe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59800ae000 tx=0x7f5980008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.019+0000 7f5989600640 1 -- 172.21.15.86:0/4194066995 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f59780556a0 con 0x7f598c0d19a0 2024-03-22T04:11:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.175+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f598c15e8b0 con 0x7f598c0d19a0 2024-03-22T04:11:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.175+0000 7f5989600640 1 -- 172.21.15.86:0/4194066995 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18707 (secure 0 0 0) 0x7f5978059740 con 0x7f598c0d19a0 2024-03-22T04:11:40.181 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:40.181 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:14.959620+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:10:31.926650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-22T04:10:33.257136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-22T04:10:36.954718+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.8","mappings":[{"from":3,"to":0}]},{"pgid":"4.10","mappings":[{"from":1,"to":4}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:40.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f596c067b20 msgr2=0x7f596c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f596c067b20 0x7f596c069fe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59800ae000 tx=0x7f5980008040 comp rx=0 tx=0).stop 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d19a0 msgr2=0x7f598c1587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d19a0 0x7f598c1587e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f597800c500 tx=0x7f597800c9c0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 shutdown_connections 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f596c067b20 0x7f596c069fe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f598c15fed0 0x7f598c1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d19a0 0x7f598c1587e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 --2- 172.21.15.86:0/4194066995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f598c0d0de0 0x7f598c153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 >> 172.21.15.86:0/4194066995 conn(0x7f598c06f910 msgr2=0x7f598c0cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 shutdown_connections 2024-03-22T04:11:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.179+0000 7f5994200640 1 -- 172.21.15.86:0/4194066995 wait complete. 2024-03-22T04:11:40.203 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:11:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde71800640 1 Processor -- start 2024-03-22T04:11:40.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde71800640 1 -- start start 2024-03-22T04:11:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde71800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fde6c0ad330 0x7fde6c15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde71800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad710 0x7fde6c142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde71800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde6c15c7c0 con 0x7fde6c06ef40 2024-03-22T04:11:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde71800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fde6c06dea0 con 0x7fde6c0ad330 2024-03-22T04:11:40.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde71800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fde6c06e180 con 0x7fde6c0ad710 2024-03-22T04:11:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde6b400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad710 0x7fde6c142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde6aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fde6c0ad330 0x7fde6c15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde62000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fde6c06ef40 0x7fde6c13fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:51694/0 (socket says 172.21.15.86:51694) 2024-03-22T04:11:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde62000640 1 -- 172.21.15.86:0/618873801 learned_addr learned my addr 172.21.15.86:0/618873801 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:40.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde6a000640 1 -- 172.21.15.86:0/618873801 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4195260613 0 0) 0x7fde6c15c7c0 con 0x7fde6c06ef40 2024-03-22T04:11:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde6aa00640 1 -- 172.21.15.86:0/618873801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad710 msgr2=0x7fde6c142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde6aa00640 1 --2- 172.21.15.86:0/618873801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad710 0x7fde6c142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde6aa00640 1 -- 172.21.15.86:0/618873801 >> v1:172.21.15.196:6789/0 conn(0x7fde6c06ef40 legacy=0x7fde6c13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde6aa00640 1 -- 172.21.15.86:0/618873801 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c06e460 con 0x7fde6c0ad330 2024-03-22T04:11:40.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.335+0000 7fde6b400640 1 --2- 172.21.15.86:0/618873801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad710 0x7fde6c142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde6aa00640 1 --2- 172.21.15.86:0/618873801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c0ad330 0x7fde6c15d4a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009d90 tx=0x7fde5c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d8837be5dfd586 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde6a000640 1 -- 172.21.15.86:0/618873801 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0b2070 con 0x7fde6c0ad330 2024-03-22T04:11:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde6a000640 1 -- 172.21.15.86:0/618873801 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde5c0a5a00 con 0x7fde6c0ad330 2024-03-22T04:11:40.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde6a000640 1 -- 172.21.15.86:0/618873801 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0a5dd0 con 0x7fde6c0ad330 2024-03-22T04:11:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde71800640 1 -- 172.21.15.86:0/618873801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c0ad330 msgr2=0x7fde6c15d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde71800640 1 --2- 172.21.15.86:0/618873801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c0ad330 0x7fde6c15d4a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009d90 tx=0x7fde5c01c940 comp rx=0 tx=0).stop 2024-03-22T04:11:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde71800640 1 -- 172.21.15.86:0/618873801 shutdown_connections 2024-03-22T04:11:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde71800640 1 --2- 172.21.15.86:0/618873801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad710 0x7fde6c142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde71800640 1 --2- 172.21.15.86:0/618873801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c0ad330 0x7fde6c15d4a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde71800640 1 -- 172.21.15.86:0/618873801 >> 172.21.15.86:0/618873801 conn(0x7fde6c0b1df0 msgr2=0x7fde6c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde71800640 1 -- 172.21.15.86:0/618873801 shutdown_connections 2024-03-22T04:11:40.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.339+0000 7fde71800640 1 -- 172.21.15.86:0/618873801 wait complete. 2024-03-22T04:11:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 Processor -- start 2024-03-22T04:11:40.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 -- start start 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c06ef40 0x7fde6c153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad330 0x7fde6c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c0ad710 0x7fde6c0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1690e0 con 0x7fde6c0ad710 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c168f60 con 0x7fde6c06ef40 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fde6c169260 con 0x7fde6c0ad330 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c0ad710 0x7fde6c0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c0ad710 0x7fde6c0b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59540/0 (socket says 172.21.15.86:59540) 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 -- 172.21.15.86:0/529605238 learned_addr learned my addr 172.21.15.86:0/529605238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde62000640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad330 0x7fde6c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6aa00640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c06ef40 0x7fde6c153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 -- 172.21.15.86:0/529605238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad330 msgr2=0x7fde6c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad330 0x7fde6c15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 -- 172.21.15.86:0/529605238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c06ef40 msgr2=0x7fde6c153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c06ef40 0x7fde6c153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 -- 172.21.15.86:0/529605238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c15a000 con 0x7fde6c0ad710 2024-03-22T04:11:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde6b400640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c0ad710 0x7fde6c0b38a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fde6400e930 tx=0x7fde6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde63400640 1 -- 172.21.15.86:0/529605238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64017070 con 0x7fde6c0ad710 2024-03-22T04:11:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde63400640 1 -- 172.21.15.86:0/529605238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde64004020 con 0x7fde6c0ad710 2024-03-22T04:11:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c15a2e0 con 0x7fde6c0ad710 2024-03-22T04:11:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde63400640 1 -- 172.21.15.86:0/529605238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64004430 con 0x7fde6c0ad710 2024-03-22T04:11:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6c153d70 con 0x7fde6c0ad710 2024-03-22T04:11:40.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.343+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6c068650 con 0x7fde6c0ad710 2024-03-22T04:11:40.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.347+0000 7fde63400640 1 -- 172.21.15.86:0/529605238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fde6400c940 con 0x7fde6c0ad710 2024-03-22T04:11:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.347+0000 7fde63400640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde48067b20 0x7fde48069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.347+0000 7fde63400640 1 -- 172.21.15.86:0/529605238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fde640863e0 con 0x7fde6c0ad710 2024-03-22T04:11:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.347+0000 7fde6aa00640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde48067b20 0x7fde48069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.347+0000 7fde6aa00640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde48067b20 0x7fde48069fe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0af000 tx=0x7fde5c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.351+0000 7fde63400640 1 -- 172.21.15.86:0/529605238 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fde640563e0 con 0x7fde6c0ad710 2024-03-22T04:11:40.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.539+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fde6c15d060 con 0x7fde6c0ad710 2024-03-22T04:11:40.543 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:11:40.539+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:11:40.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.539+0000 7fde63400640 1 -- 172.21.15.86:0/529605238 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fde6405a480 con 0x7fde6c0ad710 2024-03-22T04:11:40.545 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde48067b20 msgr2=0x7fde48069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde48067b20 0x7fde48069fe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0af000 tx=0x7fde5c0a7070 comp rx=0 tx=0).stop 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c0ad710 msgr2=0x7fde6c0b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c0ad710 0x7fde6c0b38a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fde6400e930 tx=0x7fde6400edf0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 shutdown_connections 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde48067b20 0x7fde48069fe0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde6c0ad710 0x7fde6c0b38a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde6c0ad330 0x7fde6c15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 --2- 172.21.15.86:0/529605238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde6c06ef40 0x7fde6c153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 >> 172.21.15.86:0/529605238 conn(0x7fde6c0b1df0 msgr2=0x7fde6c155640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 shutdown_connections 2024-03-22T04:11:40.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.543+0000 7fde71800640 1 -- 172.21.15.86:0/529605238 wait complete. 2024-03-22T04:11:40.561 INFO:tasks.ceph.mds.b.smithi196.stderr: -5641> 2024-03-22T04:10:38.309+0000 7fc85b600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:11:40.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T04:11:40.609 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:11:40.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.699+0000 7f4e56e00640 1 Processor -- start 2024-03-22T04:11:40.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.699+0000 7f4e56e00640 1 -- start start 2024-03-22T04:11:40.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e56e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4e5006e7e0 0x7f4e5006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e56e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e5015c770 con 0x7f4e50072650 2024-03-22T04:11:40.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e56e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e500b29a0 con 0x7f4e50072b00 2024-03-22T04:11:40.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e56e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4e50071350 con 0x7f4e5006e7e0 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4e50072650 0x7f4e500b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54748/0 (socket says 172.21.15.86:54748) 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4f400640 1 -- 172.21.15.86:0/526224412 learned_addr learned my addr 172.21.15.86:0/526224412 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4fe00640 1 --2- 172.21.15.86:0/526224412 >> v2:172.21.15.196:3300/0 conn(0x7f4e5006e7e0 0x7f4e5006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 90741194 0 0) 0x7f4e3c001120 con 0x7f4e50072650 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e34003530 con 0x7f4e50072650 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 333372695 0 0) 0x7f4e34003530 con 0x7f4e50072650 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 >> v2:172.21.15.196:3300/0 conn(0x7f4e5006e7e0 msgr2=0x7f4e5006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 --2- 172.21.15.86:0/526224412 >> v2:172.21.15.196:3300/0 conn(0x7f4e5006e7e0 0x7f4e5006ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 >> v1:172.21.15.196:6789/0 conn(0x7f4e50072b00 legacy=0x7f4e500b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5006f100 con 0x7f4e50072650 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f4e3c001850 con 0x7f4e50072650 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 >> v1:172.21.15.86:6789/0 conn(0x7f4e50072650 legacy=0x7f4e500b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e34004810 0x7f4e34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e340051d0 0x7f4e3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50071350 con 0x7f4e340039c0 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3400c170 con 0x7f4e340051d0 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3400c370 con 0x7f4e34004810 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4fe00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4f400640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e34004810 0x7f4e34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e54e00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e340051d0 0x7f4e3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4fe00640 1 -- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e34004810 msgr2=0x7f4e34004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4fe00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e34004810 0x7f4e34004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4fe00640 1 -- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e340051d0 msgr2=0x7f4e3400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4fe00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e340051d0 0x7f4e3400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4fe00640 1 -- 172.21.15.86:0/526224412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e34003530 con 0x7f4e340039c0 2024-03-22T04:11:40.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e4f400640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e34004810 0x7f4e34004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:40.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.703+0000 7f4e54e00640 1 -- 172.21.15.86:0/526224412 reap_dead start 2024-03-22T04:11:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e4fe00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e34003de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009ae0 tx=0x7f4e4001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614aa9c354758768 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e400096b0 con 0x7f4e340039c0 2024-03-22T04:11:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e400a6450 con 0x7f4e340039c0 2024-03-22T04:11:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e4ea00640 1 -- 172.21.15.86:0/526224412 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e400a6840 con 0x7f4e340039c0 2024-03-22T04:11:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 -- 172.21.15.86:0/526224412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 msgr2=0x7f4e34003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e34003de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009ae0 tx=0x7f4e4001ca70 comp rx=0 tx=0).stop 2024-03-22T04:11:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 -- 172.21.15.86:0/526224412 shutdown_connections 2024-03-22T04:11:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 --2- 172.21.15.86:0/526224412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e34003de0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 -- 172.21.15.86:0/526224412 >> 172.21.15.86:0/526224412 conn(0x7f4e500cc940 msgr2=0x7f4e500ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:40.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 -- 172.21.15.86:0/526224412 shutdown_connections 2024-03-22T04:11:40.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 -- 172.21.15.86:0/526224412 wait complete. 2024-03-22T04:11:40.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 Processor -- start 2024-03-22T04:11:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.707+0000 7f4e56e00640 1 -- start start 2024-03-22T04:11:40.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e50155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50158090 0x7f4e5014dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e5014c8b0 0x7f4e5014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e500ce930 con 0x7f4e340039c0 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50071b60 con 0x7f4e50158090 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4e50072080 con 0x7f4e5014c8b0 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e50155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e50155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59560/0 (socket says 172.21.15.86:59560) 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 -- 172.21.15.86:0/3335649253 learned_addr learned my addr 172.21.15.86:0/3335649253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e54e00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e5014c8b0 0x7f4e5014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4f400640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50158090 0x7f4e5014dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 -- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e5014c8b0 msgr2=0x7f4e5014cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e5014c8b0 0x7f4e5014cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 -- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50158090 msgr2=0x7f4e5014dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50158090 0x7f4e5014dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 -- 172.21.15.86:0/3335649253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e500b5b70 con 0x7f4e340039c0 2024-03-22T04:11:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4f400640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50158090 0x7f4e5014dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e4fe00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e50155aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009ae0 tx=0x7f4e40019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e47e00640 1 -- 172.21.15.86:0/3335649253 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e400be070 con 0x7f4e340039c0 2024-03-22T04:11:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e47e00640 1 -- 172.21.15.86:0/3335649253 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4000f000 con 0x7f4e340039c0 2024-03-22T04:11:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5015d6f0 con 0x7f4e340039c0 2024-03-22T04:11:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e47e00640 1 -- 172.21.15.86:0/3335649253 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e400a7430 con 0x7f4e340039c0 2024-03-22T04:11:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501564b0 con 0x7f4e340039c0 2024-03-22T04:11:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e50074470 con 0x7f4e340039c0 2024-03-22T04:11:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.711+0000 7f4e47e00640 1 -- 172.21.15.86:0/3335649253 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4e400c1070 con 0x7f4e340039c0 2024-03-22T04:11:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.715+0000 7f4e47e00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e30067b50 0x7f4e3006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.715+0000 7f4e47e00640 1 -- 172.21.15.86:0/3335649253 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6734+0+0 (secure 0 0 0) 0x7f4e4011fb10 con 0x7f4e340039c0 2024-03-22T04:11:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.715+0000 7f4e4f400640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e30067b50 0x7f4e3006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.715+0000 7f4e4f400640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e30067b50 0x7f4e3006a010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0012c0 tx=0x7f4e3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:40.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.719+0000 7f4e47e00640 1 -- 172.21.15.86:0/3335649253 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4e400ac020 con 0x7f4e340039c0 2024-03-22T04:11:40.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:40.907+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4e50153600 con 0x7f4e340039c0 2024-03-22T04:11:41.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.543+0000 7f4e47e00640 1 -- 172.21.15.86:0/3335649253 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f4e40016780 con 0x7f4e340039c0 2024-03-22T04:11:41.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e30067b50 msgr2=0x7f4e3006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:41.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e30067b50 0x7f4e3006a010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0012c0 tx=0x7f4e3c009040 comp rx=0 tx=0).stop 2024-03-22T04:11:41.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 msgr2=0x7f4e50155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e50155aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009ae0 tx=0x7f4e40019810 comp rx=0 tx=0).stop 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 shutdown_connections 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e30067b50 0x7f4e3006a010 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50158090 0x7f4e5014dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e5014c8b0 0x7f4e5014cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 --2- 172.21.15.86:0/3335649253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e340039c0 0x7f4e50155aa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 >> 172.21.15.86:0/3335649253 conn(0x7f4e500cc940 msgr2=0x7f4e500b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 shutdown_connections 2024-03-22T04:11:41.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.547+0000 7f4e56e00640 1 -- 172.21.15.86:0/3335649253 wait complete. 2024-03-22T04:11:41.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T04:11:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc56e200640 1 Processor -- start 2024-03-22T04:11:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc56e200640 1 -- start start 2024-03-22T04:11:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc56e200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5680ce2e0 0x7fc568160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc56e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5680cc820 con 0x7fc56815c210 2024-03-22T04:11:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc56e200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5680ccb00 con 0x7fc5680d0410 2024-03-22T04:11:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc56e200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc5680ce8e0 con 0x7fc5680ce2e0 2024-03-22T04:11:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc567400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc56815c210 0x7fc5680cfd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54764/0 (socket says 172.21.15.86:54764) 2024-03-22T04:11:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc567400640 1 -- 172.21.15.86:0/720476920 learned_addr learned my addr 172.21.15.86:0/720476920 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc567e00640 1 --2- 172.21.15.86:0/720476920 >> v2:172.21.15.196:3300/0 conn(0x7fc5680ce2e0 0x7fc568160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394504000 0 0) 0x7fc5680cc820 con 0x7fc56815c210 2024-03-22T04:11:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc54c003530 con 0x7fc56815c210 2024-03-22T04:11:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1366163590 0 0) 0x7fc54c003530 con 0x7fc56815c210 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 >> v2:172.21.15.196:3300/0 conn(0x7fc5680ce2e0 msgr2=0x7fc568160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 --2- 172.21.15.86:0/720476920 >> v2:172.21.15.196:3300/0 conn(0x7fc5680ce2e0 0x7fc568160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 >> v1:172.21.15.196:6789/0 conn(0x7fc5680d0410 legacy=0x7fc5680d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5680b4f80 con 0x7fc56815c210 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fc558001680 con 0x7fc56815c210 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 >> v1:172.21.15.86:6789/0 conn(0x7fc56815c210 legacy=0x7fc5680cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc54c003a10 0x7fc54c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc54c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54c005220 0x7fc54c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5680ce8e0 con 0x7fc54c004810 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c00c170 con 0x7fc54c005220 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c00c370 con 0x7fc54c003a10 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.707+0000 7fc55fe00640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc54c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc567e00640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54c005220 0x7fc54c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc567400640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc54c003a10 0x7fc54c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc55fe00640 1 -- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc54c003a10 msgr2=0x7fc54c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc55fe00640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc54c003a10 0x7fc54c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc55fe00640 1 -- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54c005220 msgr2=0x7fc54c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc55fe00640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54c005220 0x7fc54c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc55fe00640 1 -- 172.21.15.86:0/720476920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c003530 con 0x7fc54c004810 2024-03-22T04:11:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc567e00640 1 -- 172.21.15.86:0/720476920 reap_dead start 2024-03-22T04:11:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc567400640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc54c003a10 0x7fc54c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:41.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc55fe00640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc54c004c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc550001750 tx=0x7fc55001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6a157753a824f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:41.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5500013a0 con 0x7fc54c004810 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5500052f0 con 0x7fc54c004810 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc566a00640 1 -- 172.21.15.86:0/720476920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc550003840 con 0x7fc54c004810 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc56e200640 1 -- 172.21.15.86:0/720476920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 msgr2=0x7fc54c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.711+0000 7fc56e200640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc54c004c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc550001750 tx=0x7fc55001c3c0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 -- 172.21.15.86:0/720476920 shutdown_connections 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 --2- 172.21.15.86:0/720476920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc54c004c90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 -- 172.21.15.86:0/720476920 >> 172.21.15.86:0/720476920 conn(0x7fc56806b660 msgr2=0x7fc56806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 -- 172.21.15.86:0/720476920 shutdown_connections 2024-03-22T04:11:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 -- 172.21.15.86:0/720476920 wait complete. 2024-03-22T04:11:41.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 Processor -- start 2024-03-22T04:11:41.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 -- start start 2024-03-22T04:11:41.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc5680ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:41.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc56806c770 0x7fc56815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:41.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5681e97b0 0x7fc56815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:41.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5680d1530 con 0x7fc54c004810 2024-03-22T04:11:41.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5680ccde0 con 0x7fc56806c770 2024-03-22T04:11:41.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc56e200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc5680cf350 con 0x7fc5681e97b0 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc5680ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc5680ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59568/0 (socket says 172.21.15.86:59568) 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 -- 172.21.15.86:0/3157310525 learned_addr learned my addr 172.21.15.86:0/3157310525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc55fe00640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc56806c770 0x7fc56815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567e00640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5681e97b0 0x7fc56815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 -- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5681e97b0 msgr2=0x7fc56815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5681e97b0 0x7fc56815abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 -- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc56806c770 msgr2=0x7fc56815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc56806c770 0x7fc56815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 -- 172.21.15.86:0/3157310525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56806cc00 con 0x7fc54c004810 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc55fe00640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc56806c770 0x7fc56815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc567400640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc5680ad8f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc55800e5b0 tx=0x7fc55800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.715+0000 7fc55f400640 1 -- 172.21.15.86:0/3157310525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558012650 con 0x7fc54c004810 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc55f400640 1 -- 172.21.15.86:0/3157310525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc558018070 con 0x7fc54c004810 2024-03-22T04:11:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc55f400640 1 -- 172.21.15.86:0/3157310525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558012a10 con 0x7fc54c004810 2024-03-22T04:11:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56806cee0 con 0x7fc54c004810 2024-03-22T04:11:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc56806d4a0 con 0x7fc54c004810 2024-03-22T04:11:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc568074580 con 0x7fc54c004810 2024-03-22T04:11:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc55f400640 1 -- 172.21.15.86:0/3157310525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc558012bb0 con 0x7fc54c004810 2024-03-22T04:11:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc55f400640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc544067b20 0x7fc544069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc55f400640 1 -- 172.21.15.86:0/3157310525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6734+0+0 (secure 0 0 0) 0x7fc558085ff0 con 0x7fc54c004810 2024-03-22T04:11:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc55fe00640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc544067b20 0x7fc544069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.719+0000 7fc55fe00640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc544067b20 0x7fc544069fe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc54c004790 tx=0x7fc5500011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.723+0000 7fc55f400640 1 -- 172.21.15.86:0/3157310525 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc558056690 con 0x7fc54c004810 2024-03-22T04:11:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:41.879+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc5680b36b0 con 0x7fc54c004810 2024-03-22T04:11:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.547+0000 7fc55f400640 1 -- 172.21.15.86:0/3157310525 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7fc558055f90 con 0x7fc54c004810 2024-03-22T04:11:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T04:11:42.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc544067b20 msgr2=0x7fc544069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:42.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc544067b20 0x7fc544069fe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc54c004790 tx=0x7fc5500011a0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 msgr2=0x7fc5680ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:42.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc5680ad8f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc55800e5b0 tx=0x7fc55800ea70 comp rx=0 tx=0).stop 2024-03-22T04:11:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 shutdown_connections 2024-03-22T04:11:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc544067b20 0x7fc544069fe0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc56806c770 0x7fc56815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5681e97b0 0x7fc56815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 --2- 172.21.15.86:0/3157310525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c004810 0x7fc5680ad8f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 >> 172.21.15.86:0/3157310525 conn(0x7fc56806b660 msgr2=0x7fc5680b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 shutdown_connections 2024-03-22T04:11:42.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.551+0000 7fc56e200640 1 -- 172.21.15.86:0/3157310525 wait complete. 2024-03-22T04:11:42.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T04:11:42.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.723+0000 7f35cc600640 1 Processor -- start 2024-03-22T04:11:42.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.723+0000 7f35cc600640 1 -- start start 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.723+0000 7f35cc600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f35c415c4b0 0x7f35c40cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.723+0000 7f35cc600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35c40cdd50 0x7f35c40aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.723+0000 7f35cc600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c40ce8f0 con 0x7f35c40b0f90 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35cc600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35c415c890 con 0x7f35c40cdd50 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35cc600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f35c409b620 con 0x7f35c415c4b0 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35c40cdd50 0x7f35c40aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35c40cdd50 0x7f35c40aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59572/0 (socket says 172.21.15.86:59572) 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35ca600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f35c40b0f90 0x7f35c40d0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54772/0 (socket says 172.21.15.86:54772) 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9200640 1 -- 172.21.15.86:0/2145851115 learned_addr learned my addr 172.21.15.86:0/2145851115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9c00640 1 --2- 172.21.15.86:0/2145851115 >> v2:172.21.15.196:3300/0 conn(0x7f35c415c4b0 0x7f35c40cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:42.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9200640 1 -- 172.21.15.86:0/2145851115 >> v2:172.21.15.196:3300/0 conn(0x7f35c415c4b0 msgr2=0x7f35c40cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:42.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9200640 1 --2- 172.21.15.86:0/2145851115 >> v2:172.21.15.196:3300/0 conn(0x7f35c415c4b0 0x7f35c40cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9200640 1 -- 172.21.15.86:0/2145851115 >> v1:172.21.15.86:6789/0 conn(0x7f35c40b0f90 legacy=0x7f35c40d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:42.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9200640 1 -- 172.21.15.86:0/2145851115 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c409b900 con 0x7f35c40cdd50 2024-03-22T04:11:42.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c3e00640 1 -- 172.21.15.86:0/2145851115 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826657991 0 0) 0x7f35c40ce8f0 con 0x7f35c40b0f90 2024-03-22T04:11:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c9200640 1 --2- 172.21.15.86:0/2145851115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c40cdd50 0x7f35c40aecc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f35c40728d0 tx=0x7f35b001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35a5c08553a5d3e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c3e00640 1 -- 172.21.15.86:0/2145851115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b00aa650 con 0x7f35c40cdd50 2024-03-22T04:11:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c3e00640 1 -- 172.21.15.86:0/2145851115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b00af070 con 0x7f35c40cdd50 2024-03-22T04:11:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.727+0000 7f35c3e00640 1 -- 172.21.15.86:0/2145851115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b00aaa10 con 0x7f35c40cdd50 2024-03-22T04:11:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- 172.21.15.86:0/2145851115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c40cdd50 msgr2=0x7f35c40aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 --2- 172.21.15.86:0/2145851115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c40cdd50 0x7f35c40aecc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f35c40728d0 tx=0x7f35b001c4f0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- 172.21.15.86:0/2145851115 shutdown_connections 2024-03-22T04:11:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 --2- 172.21.15.86:0/2145851115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c40cdd50 0x7f35c40aecc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 --2- 172.21.15.86:0/2145851115 >> v2:172.21.15.196:3300/0 conn(0x7f35c415c4b0 0x7f35c40cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- 172.21.15.86:0/2145851115 >> 172.21.15.86:0/2145851115 conn(0x7f35c40b3ba0 msgr2=0x7f35c40b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- 172.21.15.86:0/2145851115 shutdown_connections 2024-03-22T04:11:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- 172.21.15.86:0/2145851115 wait complete. 2024-03-22T04:11:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 Processor -- start 2024-03-22T04:11:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- start start 2024-03-22T04:11:42.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40b0f90 0x7f35c40cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35c40cdd50 0x7f35c414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c415c4b0 0x7f35c415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35c40d0f20 con 0x7f35c415c4b0 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35c40d0da0 con 0x7f35c40b0f90 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35cc600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f35c40cfe90 con 0x7f35c40cdd50 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35ca600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c415c4b0 0x7f35c415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35ca600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c415c4b0 0x7f35c415ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59582/0 (socket says 172.21.15.86:59582) 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35ca600640 1 -- 172.21.15.86:0/1189001111 learned_addr learned my addr 172.21.15.86:0/1189001111 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35c9c00640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40b0f90 0x7f35c40cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35c9200640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35c40cdd50 0x7f35c414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35ca600640 1 -- 172.21.15.86:0/1189001111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35c40cdd50 msgr2=0x7f35c414c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35ca600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35c40cdd50 0x7f35c414c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35ca600640 1 -- 172.21.15.86:0/1189001111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40b0f90 msgr2=0x7f35c40cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35ca600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40b0f90 0x7f35c40cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.731+0000 7f35ca600640 1 -- 172.21.15.86:0/1189001111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c414ca20 con 0x7f35c415c4b0 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35ca600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c415c4b0 0x7f35c415ca70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00c7a0 tx=0x7f35bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35c1600640 1 -- 172.21.15.86:0/1189001111 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc018070 con 0x7f35c415c4b0 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35c1600640 1 -- 172.21.15.86:0/1189001111 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35bc00a640 con 0x7f35c415c4b0 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35c1600640 1 -- 172.21.15.86:0/1189001111 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00aab0 con 0x7f35c415c4b0 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c415f0a0 con 0x7f35c415c4b0 2024-03-22T04:11:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c415f660 con 0x7f35c415c4b0 2024-03-22T04:11:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c40b46c0 con 0x7f35c415c4b0 2024-03-22T04:11:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35c1600640 1 -- 172.21.15.86:0/1189001111 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f35bc01b070 con 0x7f35c415c4b0 2024-03-22T04:11:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35c1600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35a4067b20 0x7f35a4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35c1600640 1 -- 172.21.15.86:0/1189001111 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6296+0+0 (secure 0 0 0) 0x7f35bc0857f0 con 0x7f35c415c4b0 2024-03-22T04:11:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.735+0000 7f35c9c00640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35a4067b20 0x7f35a4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.739+0000 7f35c9c00640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35a4067b20 0x7f35a4069fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35b800b930 tx=0x7f35b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:42.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.739+0000 7f35c1600640 1 -- 172.21.15.86:0/1189001111 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f35bc055f10 con 0x7f35c415c4b0 2024-03-22T04:11:42.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:42.895+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35c406dce0 con 0x7f35c415c4b0 2024-03-22T04:11:43.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35c1600640 1 -- 172.21.15.86:0/1189001111 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f35bc055810 con 0x7f35c415c4b0 2024-03-22T04:11:43.555 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35a4067b20 msgr2=0x7f35a4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35a4067b20 0x7f35a4069fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35b800b930 tx=0x7f35b8008040 comp rx=0 tx=0).stop 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c415c4b0 msgr2=0x7f35c415ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c415c4b0 0x7f35c415ca70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00c7a0 tx=0x7f35bc00cc60 comp rx=0 tx=0).stop 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 shutdown_connections 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35a4067b20 0x7f35a4069fe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35c415c4b0 0x7f35c415ca70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35c40cdd50 0x7f35c414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 --2- 172.21.15.86:0/1189001111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40b0f90 0x7f35c40cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 >> 172.21.15.86:0/1189001111 conn(0x7f35c40b3ba0 msgr2=0x7f35c406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 shutdown_connections 2024-03-22T04:11:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.551+0000 7f35cc600640 1 -- 172.21.15.86:0/1189001111 wait complete. 2024-03-22T04:11:43.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-22T04:11:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.723+0000 7f56a1600640 1 Processor -- start 2024-03-22T04:11:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.723+0000 7f56a1600640 1 -- start start 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.723+0000 7f56a1600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f569c0b1cd0 0x7f569c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.723+0000 7f56a1600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f569c06de70 0x7f569c13f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.723+0000 7f56a1600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569c0b25f0 con 0x7f569c0cfa70 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.723+0000 7f56a1600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f569c0cea80 con 0x7f569c06de70 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.723+0000 7f56a1600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f569c06e4d0 con 0x7f569c0b1cd0 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f569b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f569c06de70 0x7f569c13f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f569b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f569c06de70 0x7f569c13f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59594/0 (socket says 172.21.15.86:59594) 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f569b400640 1 -- 172.21.15.86:0/1055877656 learned_addr learned my addr 172.21.15.86:0/1055877656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f569b400640 1 -- 172.21.15.86:0/1055877656 >> v2:172.21.15.196:3300/0 conn(0x7f569c0b1cd0 msgr2=0x7f569c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f569b400640 1 --2- 172.21.15.86:0/1055877656 >> v2:172.21.15.196:3300/0 conn(0x7f569c0b1cd0 0x7f569c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f569b400640 1 -- 172.21.15.86:0/1055877656 >> v1:172.21.15.196:6789/0 conn(0x7f569c0cfa70 legacy=0x7f569c06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f569b400640 1 -- 172.21.15.86:0/1055877656 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c0cecb0 con 0x7f569c06de70 2024-03-22T04:11:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f569b400640 1 --2- 172.21.15.86:0/1055877656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c13f8d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f569c075b20 tx=0x7f569401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849704031ab7056a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f5699600640 1 -- 172.21.15.86:0/1055877656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940b1070 con 0x7f569c06de70 2024-03-22T04:11:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f5699600640 1 -- 172.21.15.86:0/1055877656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56940ac630 con 0x7f569c06de70 2024-03-22T04:11:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.727+0000 7f5699600640 1 -- 172.21.15.86:0/1055877656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940a8050 con 0x7f569c06de70 2024-03-22T04:11:43.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- 172.21.15.86:0/1055877656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 msgr2=0x7f569c13f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:43.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 --2- 172.21.15.86:0/1055877656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c13f8d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f569c075b20 tx=0x7f569401c4e0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- 172.21.15.86:0/1055877656 shutdown_connections 2024-03-22T04:11:43.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 --2- 172.21.15.86:0/1055877656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c13f8d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 --2- 172.21.15.86:0/1055877656 >> v2:172.21.15.196:3300/0 conn(0x7f569c0b1cd0 0x7f569c0b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- 172.21.15.86:0/1055877656 >> 172.21.15.86:0/1055877656 conn(0x7f569c06b660 msgr2=0x7f569c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:43.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- 172.21.15.86:0/1055877656 shutdown_connections 2024-03-22T04:11:43.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- 172.21.15.86:0/1055877656 wait complete. 2024-03-22T04:11:43.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 Processor -- start 2024-03-22T04:11:43.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- start start 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0b1cd0 0x7f569c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c0cfa70 0x7f569c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f569c13f5e0 con 0x7f569c06de70 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f569c13f460 con 0x7f569c0cfa70 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f56a1600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f569c168d80 con 0x7f569c0b1cd0 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c136480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c136480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59606/0 (socket says 172.21.15.86:59606) 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569aa00640 1 -- 172.21.15.86:0/3231285091 learned_addr learned my addr 172.21.15.86:0/3231285091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569a000640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0b1cd0 0x7f569c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569b400640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c0cfa70 0x7f569c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569aa00640 1 -- 172.21.15.86:0/3231285091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0b1cd0 msgr2=0x7f569c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569aa00640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0b1cd0 0x7f569c0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569aa00640 1 -- 172.21.15.86:0/3231285091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c0cfa70 msgr2=0x7f569c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569aa00640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c0cfa70 0x7f569c15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.731+0000 7f569aa00640 1 -- 172.21.15.86:0/3231285091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c0b0c60 con 0x7f569c06de70 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f569aa00640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c136480 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f568c00ba50 tx=0x7f568c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f5692a00640 1 -- 172.21.15.86:0/3231285091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c019070 con 0x7f569c06de70 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f5692a00640 1 -- 172.21.15.86:0/3231285091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f568c00cc00 con 0x7f569c06de70 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f5692a00640 1 -- 172.21.15.86:0/3231285091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c014440 con 0x7f569c06de70 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569c0b0f40 con 0x7f569c06de70 2024-03-22T04:11:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569c0b1500 con 0x7f569c06de70 2024-03-22T04:11:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569c074580 con 0x7f569c06de70 2024-03-22T04:11:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f5692a00640 1 -- 172.21.15.86:0/3231285091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f568c01c070 con 0x7f569c06de70 2024-03-22T04:11:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f5692a00640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f567c067b20 0x7f567c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f5692a00640 1 -- 172.21.15.86:0/3231285091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5853+0+0 (secure 0 0 0) 0x7f568c085d70 con 0x7f569c06de70 2024-03-22T04:11:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f569a000640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f567c067b20 0x7f567c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:43.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.735+0000 7f569a000640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f567c067b20 0x7f567c069fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f569c145d80 tx=0x7f5688008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:43.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.739+0000 7f5692a00640 1 -- 172.21.15.86:0/3231285091 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f568c0566f0 con 0x7f569c06de70 2024-03-22T04:11:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:43.899+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f569c1369c0 con 0x7f569c06de70 2024-03-22T04:11:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.567+0000 7f5692a00640 1 -- 172.21.15.86:0/3231285091 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) ==== 178+0+0 (secure 0 0 0) 0x7f568c055ff0 con 0x7f569c06de70 2024-03-22T04:11:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' removed 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f567c067b20 msgr2=0x7f567c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f567c067b20 0x7f567c069fe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f569c145d80 tx=0x7f5688008040 comp rx=0 tx=0).stop 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 msgr2=0x7f569c136480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c136480 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f568c00ba50 tx=0x7f568c00bf10 comp rx=0 tx=0).stop 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 shutdown_connections 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f567c067b20 0x7f567c069fe0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c0cfa70 0x7f569c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0b1cd0 0x7f569c0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 --2- 172.21.15.86:0/3231285091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569c06de70 0x7f569c136480 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 >> 172.21.15.86:0/3231285091 conn(0x7f569c06b660 msgr2=0x7f569c06d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 shutdown_connections 2024-03-22T04:11:44.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.571+0000 7f56a1600640 1 -- 172.21.15.86:0/3231285091 wait complete. 2024-03-22T04:11:44.599 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-22T04:11:44.600 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-22T04:11:44.600 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-22T04:11:44.600 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:11:44.600 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-22T04:11:44.600 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-22T04:11:44.600 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-22T04:11:44.601 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:11:44.601 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-22T04:11:44.601 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-22T04:11:44.601 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-22T04:11:44.601 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:11:44.601 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-22T04:11:44.601 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-22T04:11:44.602 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-22T04:11:44.602 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:11:44.612 INFO:tasks.ceph.mds.a:Stopped 2024-03-22T04:11:44.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-22T04:11:44.614 INFO:tasks.ceph.mds.c:Stopped 2024-03-22T04:11:44.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-22T04:11:44.615 INFO:tasks.ceph.mds.b:Stopped 2024-03-22T04:11:44.615 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-22T04:11:44.617 INFO:tasks.ceph.mds.d:Stopped 2024-03-22T04:11:44.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-22T04:11:44.660 INFO:tasks.ceph.mds.a:Started 2024-03-22T04:11:44.660 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:44.662 INFO:tasks.ceph.mds.c:Started 2024-03-22T04:11:44.662 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:44.663 INFO:tasks.ceph.mds.b:Started 2024-03-22T04:11:44.664 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:44.666 INFO:tasks.ceph.mds.d:Started 2024-03-22T04:11:44.666 DEBUG:teuthology.parallel:result is None 2024-03-22T04:11:44.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-22T04:11:44.747 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:11:44.747 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-03-22T04:11:44.756 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:11:44.756 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:11:44.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd521200640 1 Processor -- start 2024-03-22T04:11:44.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd521200640 1 -- start start 2024-03-22T04:11:44.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd521200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:44.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd521200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd51c0cdb00 0x7fd51c09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:44.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd521200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51c073cf0 con 0x7fd51c09afb0 2024-03-22T04:11:44.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd521200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd51c09b3e0 con 0x7fd51c0cdb00 2024-03-22T04:11:44.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd521200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c0ce280 con 0x7fd51c15f860 2024-03-22T04:11:44.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd512a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd51c0cdb00 0x7fd51c09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd51be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd51c09afb0 0x7fd51c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54780/0 (socket says 172.21.15.86:54780) 2024-03-22T04:11:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd51be00640 1 -- 172.21.15.86:0/3419731661 learned_addr learned my addr 172.21.15.86:0/3419731661 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd51b400640 1 --2- 172.21.15.86:0/3419731661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd512a00640 1 -- 172.21.15.86:0/3419731661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 msgr2=0x7fd51c0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd512a00640 1 --2- 172.21.15.86:0/3419731661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd512a00640 1 -- 172.21.15.86:0/3419731661 >> v1:172.21.15.86:6789/0 conn(0x7fd51c09afb0 legacy=0x7fd51c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd512a00640 1 -- 172.21.15.86:0/3419731661 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c0ce560 con 0x7fd51c0cdb00 2024-03-22T04:11:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.807+0000 7fd51aa00640 1 -- 172.21.15.86:0/3419731661 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1018831030 0 0) 0x7fd51c073cf0 con 0x7fd51c09afb0 2024-03-22T04:11:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd512a00640 1 --2- 172.21.15.86:0/3419731661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c09aa70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd5040097c0 tx=0x7fd50401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2be3a90e32add2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd51aa00640 1 -- 172.21.15.86:0/3419731661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5040a7070 con 0x7fd51c0cdb00 2024-03-22T04:11:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd51aa00640 1 -- 172.21.15.86:0/3419731661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5040b0070 con 0x7fd51c0cdb00 2024-03-22T04:11:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd51aa00640 1 -- 172.21.15.86:0/3419731661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5040ab890 con 0x7fd51c0cdb00 2024-03-22T04:11:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd521200640 1 -- 172.21.15.86:0/3419731661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 msgr2=0x7fd51c09aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd521200640 1 --2- 172.21.15.86:0/3419731661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c09aa70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd5040097c0 tx=0x7fd50401c470 comp rx=0 tx=0).stop 2024-03-22T04:11:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd521200640 1 -- 172.21.15.86:0/3419731661 shutdown_connections 2024-03-22T04:11:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd521200640 1 --2- 172.21.15.86:0/3419731661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c09aa70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd521200640 1 --2- 172.21.15.86:0/3419731661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd521200640 1 -- 172.21.15.86:0/3419731661 >> 172.21.15.86:0/3419731661 conn(0x7fd51c0b3d60 msgr2=0x7fd51c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd521200640 1 -- 172.21.15.86:0/3419731661 shutdown_connections 2024-03-22T04:11:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.811+0000 7fd521200640 1 -- 172.21.15.86:0/3419731661 wait complete. 2024-03-22T04:11:44.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 Processor -- start 2024-03-22T04:11:44.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 -- start start 2024-03-22T04:11:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c09afb0 0x7fd51c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c1fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c15fc40 con 0x7fd51c0cdb00 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c0cdf40 con 0x7fd51c09afb0 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c15fdc0 con 0x7fd51c15f860 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c1fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c1fa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59648/0 (socket says 172.21.15.86:59648) 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 -- 172.21.15.86:0/987329008 learned_addr learned my addr 172.21.15.86:0/987329008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd51b400640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c09afb0 0x7fd51c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 -- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 msgr2=0x7fd51c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd51be00640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 -- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c09afb0 msgr2=0x7fd51c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c09afb0 0x7fd51c15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 -- 172.21.15.86:0/987329008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c0b5360 con 0x7fd51c0cdb00 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd51b400640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c09afb0 0x7fd51c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd51be00640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd512a00640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c1fa7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd5040ad040 tx=0x7fd50401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd513e00640 1 -- 172.21.15.86:0/987329008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5040bf610 con 0x7fd51c0cdb00 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd513e00640 1 -- 172.21.15.86:0/987329008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5040bf7f0 con 0x7fd51c0cdb00 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd513e00640 1 -- 172.21.15.86:0/987329008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5040bfbe0 con 0x7fd51c0cdb00 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c0b5640 con 0x7fd51c0cdb00 2024-03-22T04:11:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.815+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c0b5c00 con 0x7fd51c0cdb00 2024-03-22T04:11:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.819+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c074470 con 0x7fd51c0cdb00 2024-03-22T04:11:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.819+0000 7fd513e00640 1 -- 172.21.15.86:0/987329008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd5040bd020 con 0x7fd51c0cdb00 2024-03-22T04:11:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.819+0000 7fd513e00640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4f8067b20 0x7fd4f8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.819+0000 7fd513e00640 1 -- 172.21.15.86:0/987329008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5335+0+0 (secure 0 0 0) 0x7fd504120120 con 0x7fd51c0cdb00 2024-03-22T04:11:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.819+0000 7fd51b400640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4f8067b20 0x7fd4f8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:44.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.819+0000 7fd51b400640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4f8067b20 0x7fd4f8069fe0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009ae0 tx=0x7fd50c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:44.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.823+0000 7fd513e00640 1 -- 172.21.15.86:0/987329008 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd5040c6030 con 0x7fd51c0cdb00 2024-03-22T04:11:44.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:44.991+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd51c0cc150 con 0x7fd51c0cdb00 2024-03-22T04:11:45.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.587+0000 7fd513e00640 1 -- 172.21.15.86:0/987329008 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) ==== 97+0+0 (secure 0 0 0) 0x7fd504016090 con 0x7fd51c0cdb00 2024-03-22T04:11:45.595 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-03-22T04:11:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4f8067b20 msgr2=0x7fd4f8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4f8067b20 0x7fd4f8069fe0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009ae0 tx=0x7fd50c008040 comp rx=0 tx=0).stop 2024-03-22T04:11:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 msgr2=0x7fd51c1fa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c1fa7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd5040ad040 tx=0x7fd50401acb0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 shutdown_connections 2024-03-22T04:11:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4f8067b20 0x7fd4f8069fe0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c15f860 0x7fd51c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51c0cdb00 0x7fd51c1fa7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 --2- 172.21.15.86:0/987329008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c09afb0 0x7fd51c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 >> 172.21.15.86:0/987329008 conn(0x7fd51c0b3d60 msgr2=0x7fd51c0af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 shutdown_connections 2024-03-22T04:11:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.591+0000 7fd521200640 1 -- 172.21.15.86:0/987329008 wait complete. 2024-03-22T04:11:45.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-22T04:11:45.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.755+0000 7f4064200640 1 Processor -- start 2024-03-22T04:11:45.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.755+0000 7f4064200640 1 -- start start 2024-03-22T04:11:45.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.755+0000 7f4064200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f405c15b3f0 0x7f405c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:45.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.755+0000 7f4064200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f405c0ae540 0x7f405c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:45.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.755+0000 7f4064200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f405c071920 con 0x7f405c0cc3e0 2024-03-22T04:11:45.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.755+0000 7f4064200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f405c0716d0 con 0x7f405c0ae540 2024-03-22T04:11:45.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.755+0000 7f4064200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f405c06d990 con 0x7f405c15b3f0 2024-03-22T04:11:45.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f4062200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f405c0ae540 0x7f405c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:45.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f4062200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f405c0ae540 0x7f405c072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59650/0 (socket says 172.21.15.86:59650) 2024-03-22T04:11:45.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f4062200640 1 -- 172.21.15.86:0/1392719382 learned_addr learned my addr 172.21.15.86:0/1392719382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:45.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f4062200640 1 -- 172.21.15.86:0/1392719382 >> v2:172.21.15.196:3300/0 conn(0x7f405c15b3f0 msgr2=0x7f405c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:45.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f4062200640 1 --2- 172.21.15.86:0/1392719382 >> v2:172.21.15.196:3300/0 conn(0x7f405c15b3f0 0x7f405c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f4062200640 1 -- 172.21.15.86:0/1392719382 >> v1:172.21.15.196:6789/0 conn(0x7f405c0cc3e0 legacy=0x7f405c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:45.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f4062200640 1 -- 172.21.15.86:0/1392719382 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c071c00 con 0x7f405c0ae540 2024-03-22T04:11:45.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f4062200640 1 --2- 172.21.15.86:0/1392719382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c0ae540 0x7f405c072f00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f405c06c510 tx=0x7f405401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403ef36dc8c4aea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:45.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f405be00640 1 -- 172.21.15.86:0/1392719382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540a56d0 con 0x7f405c0ae540 2024-03-22T04:11:45.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f405be00640 1 -- 172.21.15.86:0/1392719382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40540a5870 con 0x7f405c0ae540 2024-03-22T04:11:45.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.759+0000 7f405be00640 1 -- 172.21.15.86:0/1392719382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540a5c40 con 0x7f405c0ae540 2024-03-22T04:11:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- 172.21.15.86:0/1392719382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c0ae540 msgr2=0x7f405c072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 --2- 172.21.15.86:0/1392719382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c0ae540 0x7f405c072f00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f405c06c510 tx=0x7f405401c920 comp rx=0 tx=0).stop 2024-03-22T04:11:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- 172.21.15.86:0/1392719382 shutdown_connections 2024-03-22T04:11:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 --2- 172.21.15.86:0/1392719382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c0ae540 0x7f405c072f00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 --2- 172.21.15.86:0/1392719382 >> v2:172.21.15.196:3300/0 conn(0x7f405c15b3f0 0x7f405c0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- 172.21.15.86:0/1392719382 >> 172.21.15.86:0/1392719382 conn(0x7f405c0b1df0 msgr2=0x7f405c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- 172.21.15.86:0/1392719382 shutdown_connections 2024-03-22T04:11:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- 172.21.15.86:0/1392719382 wait complete. 2024-03-22T04:11:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 Processor -- start 2024-03-22T04:11:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- start start 2024-03-22T04:11:45.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f405c0ae540 0x7f405c14eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f405c0cc3e0 0x7f405c150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c15b3f0 0x7f405c14c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f405c168580 con 0x7f405c15b3f0 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f405c168400 con 0x7f405c0cc3e0 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4064200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f405c168700 con 0x7f405c0ae540 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4062200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c15b3f0 0x7f405c14c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4062200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c15b3f0 0x7f405c14c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59656/0 (socket says 172.21.15.86:59656) 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4062200640 1 -- 172.21.15.86:0/1781865179 learned_addr learned my addr 172.21.15.86:0/1781865179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4061800640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f405c0ae540 0x7f405c14eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4060e00640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f405c0cc3e0 0x7f405c150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4062200640 1 -- 172.21.15.86:0/1781865179 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f405c0ae540 msgr2=0x7f405c14eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4062200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f405c0ae540 0x7f405c14eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4062200640 1 -- 172.21.15.86:0/1781865179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f405c0cc3e0 msgr2=0x7f405c150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4062200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f405c0cc3e0 0x7f405c150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.763+0000 7f4062200640 1 -- 172.21.15.86:0/1781865179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c14f3a0 con 0x7f405c15b3f0 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4062200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c15b3f0 0x7f405c14c5a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f40540af040 tx=0x7f40540196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4059600640 1 -- 172.21.15.86:0/1781865179 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540bc020 con 0x7f405c15b3f0 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4059600640 1 -- 172.21.15.86:0/1781865179 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40540b8670 con 0x7f405c15b3f0 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4059600640 1 -- 172.21.15.86:0/1781865179 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540b8a60 con 0x7f405c15b3f0 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405c14f680 con 0x7f405c15b3f0 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405c14fc40 con 0x7f405c15b3f0 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405c068650 con 0x7f405c15b3f0 2024-03-22T04:11:45.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4059600640 1 -- 172.21.15.86:0/1781865179 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f40540bf020 con 0x7f405c15b3f0 2024-03-22T04:11:45.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4059600640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4040067b20 0x7f4040069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:45.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4059600640 1 -- 172.21.15.86:0/1781865179 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f4054120d40 con 0x7f405c15b3f0 2024-03-22T04:11:45.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.767+0000 7f4061800640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4040067b20 0x7f4040069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:45.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.771+0000 7f4061800640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4040067b20 0x7f4040069fe0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f405c14ea60 tx=0x7f404c009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:45.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.771+0000 7f4059600640 1 -- 172.21.15.86:0/1781865179 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f40540c6030 con 0x7f405c15b3f0 2024-03-22T04:11:45.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.979+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f405c15aac0 con 0x7f405c15b3f0 2024-03-22T04:11:45.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.979+0000 7f4059600640 1 -- 172.21.15.86:0/1781865179 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+6645 (secure 0 0 0) 0x7f40540165a0 con 0x7f405c15b3f0 2024-03-22T04:11:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:45.986 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA==", 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw==", 2024-03-22T04:11:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q==", 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:11:45.988 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg==", 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-03-22T04:11:45.989 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA==", 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA==", 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:11:45.990 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg==", 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-03-22T04:11:45.991 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA==", 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ==", 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.992 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow==", 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg==", 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:11:45.993 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg==", 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.994 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA==", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg==", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:11:45.995 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w==", 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-03-22T04:11:45.996 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w==", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB4BP1lvLoQORAADwVEr78V9aUc1YblO47Gaw==", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-mds", 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCgBP1llALsLhAA43bNq65ox6ZTUUJs32gBrw==", 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-mds" 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-mgr", 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCgBP1l+g3sLhAASIl+GfzJKSHjAJ+/PXyGmg==", 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-22T04:11:45.998 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-osd", 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCgBP1lXRbsLhAAYtPotGjJDfCWfalMvBJiiw==", 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-osd" 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rbd", 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCgBP1lrCLsLhAAOnfulKgDZ1K9vrgpluZtuQ==", 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:45.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCgBP1lAivsLhAAmyHsXffgNYsQu2lfKY0ZqA==", 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rgw", 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCgBP1lgTPsLhAALvXzad89N32HAcfC3BQwkw==", 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:46.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g==", 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ==", 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:11:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:11:46.002 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:46.002 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:46.002 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T04:11:46.002 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4040067b20 msgr2=0x7f4040069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4040067b20 0x7f4040069fe0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f405c14ea60 tx=0x7f404c009110 comp rx=0 tx=0).stop 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c15b3f0 msgr2=0x7f405c14c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c15b3f0 0x7f405c14c5a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f40540af040 tx=0x7f40540196c0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 shutdown_connections 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4040067b20 0x7f4040069fe0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f405c0cc3e0 0x7f405c150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f405c15b3f0 0x7f405c14c5a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 --2- 172.21.15.86:0/1781865179 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f405c0ae540 0x7f405c14eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 >> 172.21.15.86:0/1781865179 conn(0x7f405c0b1df0 msgr2=0x7f405c15b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 shutdown_connections 2024-03-22T04:11:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:45.983+0000 7f4064200640 1 -- 172.21.15.86:0/1781865179 wait complete. 2024-03-22T04:11:46.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-22T04:11:46.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f471a600640 1 Processor -- start 2024-03-22T04:11:46.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f471a600640 1 -- start start 2024-03-22T04:11:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f471a600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47140d0630 0x7f47140cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f471a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f471415f090 con 0x7f471415ead0 2024-03-22T04:11:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f471a600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f471415f370 con 0x7f471406b660 2024-03-22T04:11:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f471a600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f471415f650 con 0x7f47140d0630 2024-03-22T04:11:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4712a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f471415ead0 0x7f471406e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48664/0 (socket says 172.21.15.86:48664) 2024-03-22T04:11:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4712a00640 1 -- 172.21.15.86:0/179495461 learned_addr learned my addr 172.21.15.86:0/179495461 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4713400640 1 --2- 172.21.15.86:0/179495461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47140d0630 0x7f47140cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4712000640 1 -- 172.21.15.86:0/179495461 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436365633 0 0) 0x7f471415f090 con 0x7f471415ead0 2024-03-22T04:11:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4712000640 1 -- 172.21.15.86:0/179495461 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46f8003530 con 0x7f471415ead0 2024-03-22T04:11:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4712000640 1 -- 172.21.15.86:0/179495461 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2550535774 0 0) 0x7f471415f370 con 0x7f471406b660 2024-03-22T04:11:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4713400640 1 -- 172.21.15.86:0/179495461 >> v1:172.21.15.196:6789/0 conn(0x7f471406b660 legacy=0x7f47140d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4713400640 1 -- 172.21.15.86:0/179495461 >> v1:172.21.15.86:6789/0 conn(0x7f471415ead0 legacy=0x7f471406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.139+0000 7f4713400640 1 -- 172.21.15.86:0/179495461 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471415f930 con 0x7f47140d0630 2024-03-22T04:11:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f4713400640 1 --2- 172.21.15.86:0/179495461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47140d0630 0x7f47140cf6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f47040097f0 tx=0x7f470401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbe60175036ed20 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f4712000640 1 -- 172.21.15.86:0/179495461 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040aa660 con 0x7f47140d0630 2024-03-22T04:11:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f4712000640 1 -- 172.21.15.86:0/179495461 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47040af070 con 0x7f47140d0630 2024-03-22T04:11:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f4712000640 1 -- 172.21.15.86:0/179495461 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040aaa20 con 0x7f47140d0630 2024-03-22T04:11:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f471a600640 1 -- 172.21.15.86:0/179495461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47140d0630 msgr2=0x7f47140cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f471a600640 1 --2- 172.21.15.86:0/179495461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47140d0630 0x7f47140cf6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f47040097f0 tx=0x7f470401c4e0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f471a600640 1 -- 172.21.15.86:0/179495461 shutdown_connections 2024-03-22T04:11:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f471a600640 1 --2- 172.21.15.86:0/179495461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47140d0630 0x7f47140cf6f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f471a600640 1 -- 172.21.15.86:0/179495461 >> 172.21.15.86:0/179495461 conn(0x7f471406c770 msgr2=0x7f471406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f471a600640 1 -- 172.21.15.86:0/179495461 shutdown_connections 2024-03-22T04:11:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.143+0000 7f471a600640 1 -- 172.21.15.86:0/179495461 wait complete. 2024-03-22T04:11:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 Processor -- start 2024-03-22T04:11:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 -- start start 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f471406b660 0x7f47140cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47140d0630 0x7f4714154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471415ead0 0x7f4714141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f471416ccd0 con 0x7f471406b660 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f471406bea0 con 0x7f47140d0630 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f471416ce50 con 0x7f471415ead0 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f471406b660 0x7f47140cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f471406b660 0x7f47140cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53450/0 (socket says 172.21.15.86:53450) 2024-03-22T04:11:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 -- 172.21.15.86:0/1902339155 learned_addr learned my addr 172.21.15.86:0/1902339155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 -- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471415ead0 msgr2=0x7f4714141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4712a00640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47140d0630 0x7f4714154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713e00640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471415ead0 0x7f4714141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471415ead0 0x7f4714141ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 -- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47140d0630 msgr2=0x7f4714154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47140d0630 0x7f4714154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 -- 172.21.15.86:0/1902339155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4714140a20 con 0x7f471406b660 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713e00640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471415ead0 0x7f4714141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4712a00640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47140d0630 0x7f4714154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f4713400640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f471406b660 0x7f47140cc9e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47040ac040 tx=0x7f470401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f470b400640 1 -- 172.21.15.86:0/1902339155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040b4600 con 0x7f471406b660 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f470b400640 1 -- 172.21.15.86:0/1902339155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47040aae50 con 0x7f471406b660 2024-03-22T04:11:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f470b400640 1 -- 172.21.15.86:0/1902339155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040b4a00 con 0x7f471406b660 2024-03-22T04:11:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4714140d00 con 0x7f471406b660 2024-03-22T04:11:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.147+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47141412c0 con 0x7f471406b660 2024-03-22T04:11:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.151+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4714068650 con 0x7f471406b660 2024-03-22T04:11:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.151+0000 7f470b400640 1 -- 172.21.15.86:0/1902339155 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f47040bc070 con 0x7f471406b660 2024-03-22T04:11:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.151+0000 7f470b400640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f4067b20 0x7f46f4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.151+0000 7f470b400640 1 -- 172.21.15.86:0/1902339155 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f470411fc60 con 0x7f471406b660 2024-03-22T04:11:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.151+0000 7f4712a00640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f4067b20 0x7f46f4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.151+0000 7f4712a00640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f4067b20 0x7f46f4069fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f47140b1890 tx=0x7f4700009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.155+0000 7f470b400640 1 -- 172.21.15.86:0/1902339155 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f47040160d0 con 0x7f471406b660 2024-03-22T04:11:46.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.363+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f47140ce250 con 0x7f471406b660 2024-03-22T04:11:46.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.363+0000 7f470b400640 1 -- 172.21.15.86:0/1902339155 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f47040f0260 con 0x7f471406b660 2024-03-22T04:11:46.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f4067b20 msgr2=0x7f46f4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f4067b20 0x7f46f4069fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f47140b1890 tx=0x7f4700009040 comp rx=0 tx=0).stop 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f471406b660 msgr2=0x7f47140cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f471406b660 0x7f47140cc9e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47040ac040 tx=0x7f470401ad20 comp rx=0 tx=0).stop 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 shutdown_connections 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f4067b20 0x7f46f4069fe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47140d0630 0x7f4714154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471415ead0 0x7f4714141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 --2- 172.21.15.86:0/1902339155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f471406b660 0x7f47140cc9e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 >> 172.21.15.86:0/1902339155 conn(0x7f471406c770 msgr2=0x7f471415c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 shutdown_connections 2024-03-22T04:11:46.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.367+0000 7f471a600640 1 -- 172.21.15.86:0/1902339155 wait complete. 2024-03-22T04:11:46.392 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-22T04:11:46.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.523+0000 7ff48a600640 1 Processor -- start 2024-03-22T04:11:46.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.523+0000 7ff48a600640 1 -- start start 2024-03-22T04:11:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.523+0000 7ff48a600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4840700e0 0x7ff4841607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.523+0000 7ff48a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff48415d290 con 0x7ff48413f1b0 2024-03-22T04:11:46.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.523+0000 7ff48a600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff48415d570 con 0x7ff48415c5f0 2024-03-22T04:11:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.523+0000 7ff48a600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff484160320 con 0x7ff4840700e0 2024-03-22T04:11:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff48413f1b0 0x7ff48406f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48678/0 (socket says 172.21.15.86:48678) 2024-03-22T04:11:46.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489200640 1 -- 172.21.15.86:0/2574035287 learned_addr learned my addr 172.21.15.86:0/2574035287 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483e00640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4840700e0 0x7ff4841607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685896085 0 0) 0x7ff48415d290 con 0x7ff48413f1b0 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff468003530 con 0x7ff48413f1b0 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445520917 0 0) 0x7ff48415d570 con 0x7ff48415c5f0 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff48415d290 con 0x7ff48415c5f0 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2922710499 0 0) 0x7ff468003530 con 0x7ff48413f1b0 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4840700e0 msgr2=0x7ff4841607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4840700e0 0x7ff4841607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 >> v1:172.21.15.196:6789/0 conn(0x7ff48415c5f0 legacy=0x7ff48415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48415fcd0 con 0x7ff48413f1b0 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7ff474001680 con 0x7ff48413f1b0 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 >> v1:172.21.15.86:6789/0 conn(0x7ff48413f1b0 legacy=0x7ff48406f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4840700e0 0x7ff468003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff468004210 0x7ff468004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff468004df0 0x7ff468005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff484160320 con 0x7ff468004df0 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff46800bd00 con 0x7ff468004210 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff46800bf00 con 0x7ff4840700e0 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489c00640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff468004df0 0x7ff468005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489200640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4840700e0 0x7ff468003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff483e00640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff468004210 0x7ff468004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489c00640 1 -- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4840700e0 msgr2=0x7ff468003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:46.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489c00640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4840700e0 0x7ff468003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489c00640 1 -- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff468004210 msgr2=0x7ff468004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:46.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489c00640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff468004210 0x7ff468004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.527+0000 7ff489c00640 1 -- 172.21.15.86:0/2574035287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff468003530 con 0x7ff468004df0 2024-03-22T04:11:46.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff489c00640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff468004df0 0x7ff468005270 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff47c001fd0 tx=0x7ff47c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bf7a6718fcd341 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c001c20 con 0x7ff468004df0 2024-03-22T04:11:46.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c003ba0 con 0x7ff468004df0 2024-03-22T04:11:46.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff483400640 1 -- 172.21.15.86:0/2574035287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0050d0 con 0x7ff468004df0 2024-03-22T04:11:46.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff48a600640 1 -- 172.21.15.86:0/2574035287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff468004df0 msgr2=0x7ff468005270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff48a600640 1 --2- 172.21.15.86:0/2574035287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff468004df0 0x7ff468005270 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff47c001fd0 tx=0x7ff47c01c900 comp rx=0 tx=0).stop 2024-03-22T04:11:46.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff489c00640 1 -- 172.21.15.86:0/2574035287 reap_dead start 2024-03-22T04:11:46.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff48a600640 1 -- 172.21.15.86:0/2574035287 shutdown_connections 2024-03-22T04:11:46.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff48a600640 1 -- 172.21.15.86:0/2574035287 >> 172.21.15.86:0/2574035287 conn(0x7ff48406bc60 msgr2=0x7ff4840b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:46.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff48a600640 1 -- 172.21.15.86:0/2574035287 shutdown_connections 2024-03-22T04:11:46.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff48a600640 1 -- 172.21.15.86:0/2574035287 wait complete. 2024-03-22T04:11:46.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff48a600640 1 Processor -- start 2024-03-22T04:11:46.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.531+0000 7ff48a600640 1 -- start start 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4841fa070 0x7ff48406e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4841fa450 0x7ff4840b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4840b3ec0 0x7ff484143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4840d1590 con 0x7ff4840b3ec0 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4840d1410 con 0x7ff4841fa070 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff4840d1710 con 0x7ff4841fa450 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4840b3ec0 0x7ff484143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4840b3ec0 0x7ff484143ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53470/0 (socket says 172.21.15.86:53470) 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 -- 172.21.15.86:0/2283550971 learned_addr learned my addr 172.21.15.86:0/2283550971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff483e00640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4841fa450 0x7ff4840b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489200640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4841fa070 0x7ff48406e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 -- 172.21.15.86:0/2283550971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4841fa450 msgr2=0x7ff4840b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4841fa450 0x7ff4840b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 -- 172.21.15.86:0/2283550971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4841fa070 msgr2=0x7ff48406e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4841fa070 0x7ff48406e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 -- 172.21.15.86:0/2283550971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4840a97e0 con 0x7ff4840b3ec0 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff489c00640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4840b3ec0 0x7ff484143ac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff47c005040 tx=0x7ff468005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff480c00640 1 -- 172.21.15.86:0/2283550971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c001c00 con 0x7ff4840b3ec0 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff480c00640 1 -- 172.21.15.86:0/2283550971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c003d50 con 0x7ff4840b3ec0 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff480c00640 1 -- 172.21.15.86:0/2283550971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c006d60 con 0x7ff4840b3ec0 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48406cac0 con 0x7ff4840b3ec0 2024-03-22T04:11:46.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48406d080 con 0x7ff4840b3ec0 2024-03-22T04:11:46.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff484074470 con 0x7ff4840b3ec0 2024-03-22T04:11:46.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.535+0000 7ff480c00640 1 -- 172.21.15.86:0/2283550971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff47c0052e0 con 0x7ff4840b3ec0 2024-03-22T04:11:46.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.539+0000 7ff480c00640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff460067b20 0x7ff460069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.539+0000 7ff480c00640 1 -- 172.21.15.86:0/2283550971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7ff47c11ef90 con 0x7ff4840b3ec0 2024-03-22T04:11:46.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.539+0000 7ff489200640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff460067b20 0x7ff460069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.539+0000 7ff489200640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff460067b20 0x7ff460069fe0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff474000c10 tx=0x7ff4740013a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.543+0000 7ff480c00640 1 -- 172.21.15.86:0/2283550971 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff47c0bd030 con 0x7ff4840b3ec0 2024-03-22T04:11:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.755+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7ff4840b50f0 con 0x7ff4840b3ec0 2024-03-22T04:11:46.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff480c00640 1 -- 172.21.15.86:0/2283550971 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7ff47c016610 con 0x7ff4840b3ec0 2024-03-22T04:11:46.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff460067b20 msgr2=0x7ff460069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff460067b20 0x7ff460069fe0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff474000c10 tx=0x7ff4740013a0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4840b3ec0 msgr2=0x7ff484143ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4840b3ec0 0x7ff484143ac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff47c005040 tx=0x7ff468005840 comp rx=0 tx=0).stop 2024-03-22T04:11:46.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 shutdown_connections 2024-03-22T04:11:46.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff460067b20 0x7ff460069fe0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4840b3ec0 0x7ff484143ac0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4841fa450 0x7ff4840b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 --2- 172.21.15.86:0/2283550971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4841fa070 0x7ff48406e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 >> 172.21.15.86:0/2283550971 conn(0x7ff48406bc60 msgr2=0x7ff484160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:46.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 shutdown_connections 2024-03-22T04:11:46.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.759+0000 7ff48a600640 1 -- 172.21.15.86:0/2283550971 wait complete. 2024-03-22T04:11:46.784 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-22T04:11:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.915+0000 7f1972c00640 1 Processor -- start 2024-03-22T04:11:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.915+0000 7f1972c00640 1 -- start start 2024-03-22T04:11:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.915+0000 7f1972c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f196c0d0580 0x7f196c071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.915+0000 7f1972c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f196c06e2f0 0x7f196c071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.915+0000 7f1972c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196c1623d0 con 0x7f196c15e630 2024-03-22T04:11:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.915+0000 7f1972c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f196c1620f0 con 0x7f196c06e2f0 2024-03-22T04:11:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.915+0000 7f1972c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f196c15ec10 con 0x7f196c0d0580 2024-03-22T04:11:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f1970c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f196c06e2f0 0x7f196c071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f196be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f196c15e630 0x7f196c0cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48694/0 (socket says 172.21.15.86:48694) 2024-03-22T04:11:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f196be00640 1 -- 172.21.15.86:0/723432533 learned_addr learned my addr 172.21.15.86:0/723432533 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f1963e00640 1 --2- 172.21.15.86:0/723432533 >> v2:172.21.15.196:3300/0 conn(0x7f196c0d0580 0x7f196c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f1970c00640 1 -- 172.21.15.86:0/723432533 >> v2:172.21.15.196:3300/0 conn(0x7f196c0d0580 msgr2=0x7f196c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f1970c00640 1 --2- 172.21.15.86:0/723432533 >> v2:172.21.15.196:3300/0 conn(0x7f196c0d0580 0x7f196c071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f1970c00640 1 -- 172.21.15.86:0/723432533 >> v1:172.21.15.86:6789/0 conn(0x7f196c15e630 legacy=0x7f196c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f1970c00640 1 -- 172.21.15.86:0/723432533 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c15eef0 con 0x7f196c06e2f0 2024-03-22T04:11:46.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f196b400640 1 -- 172.21.15.86:0/723432533 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2441875069 0 0) 0x7f196c1623d0 con 0x7f196c15e630 2024-03-22T04:11:46.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f1970c00640 1 --2- 172.21.15.86:0/723432533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c06e2f0 0x7f196c071640 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f196c06c5e0 tx=0x7f196401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aec4298ce5ff8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f196b400640 1 -- 172.21.15.86:0/723432533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640b2070 con 0x7f196c06e2f0 2024-03-22T04:11:46.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f196b400640 1 -- 172.21.15.86:0/723432533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19640a57f0 con 0x7f196c06e2f0 2024-03-22T04:11:46.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.919+0000 7f196b400640 1 -- 172.21.15.86:0/723432533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640a5bc0 con 0x7f196c06e2f0 2024-03-22T04:11:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- 172.21.15.86:0/723432533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c06e2f0 msgr2=0x7f196c071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 --2- 172.21.15.86:0/723432533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c06e2f0 0x7f196c071640 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f196c06c5e0 tx=0x7f196401c930 comp rx=0 tx=0).stop 2024-03-22T04:11:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- 172.21.15.86:0/723432533 shutdown_connections 2024-03-22T04:11:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 --2- 172.21.15.86:0/723432533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c06e2f0 0x7f196c071640 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 --2- 172.21.15.86:0/723432533 >> v2:172.21.15.196:3300/0 conn(0x7f196c0d0580 0x7f196c071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- 172.21.15.86:0/723432533 >> 172.21.15.86:0/723432533 conn(0x7f196c06f910 msgr2=0x7f196c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- 172.21.15.86:0/723432533 shutdown_connections 2024-03-22T04:11:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- 172.21.15.86:0/723432533 wait complete. 2024-03-22T04:11:46.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 Processor -- start 2024-03-22T04:11:46.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- start start 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c06e2f0 0x7f196c0b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c0d0580 0x7f196c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c15e630 0x7f196c157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f196c071f90 con 0x7f196c0d0580 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f196c15f9d0 con 0x7f196c15e630 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1972c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f196c0cfaf0 con 0x7f196c06e2f0 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1963e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c0d0580 0x7f196c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1963e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c0d0580 0x7f196c156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53500/0 (socket says 172.21.15.86:53500) 2024-03-22T04:11:46.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1963e00640 1 -- 172.21.15.86:0/3447483278 learned_addr learned my addr 172.21.15.86:0/3447483278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1970c00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c15e630 0x7f196c157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f196be00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c06e2f0 0x7f196c0b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1963e00640 1 -- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c06e2f0 msgr2=0x7f196c0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1963e00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c06e2f0 0x7f196c0b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1963e00640 1 -- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c15e630 msgr2=0x7f196c157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1963e00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c15e630 0x7f196c157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f1963e00640 1 -- 172.21.15.86:0/3447483278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c0c69f0 con 0x7f196c0d0580 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.923+0000 7f196be00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c06e2f0 0x7f196c0b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:46.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1963e00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c0d0580 0x7f196c156a70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f195400e930 tx=0x7f195400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1968c00640 1 -- 172.21.15.86:0/3447483278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954017070 con 0x7f196c0d0580 2024-03-22T04:11:46.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1968c00640 1 -- 172.21.15.86:0/3447483278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195400ca00 con 0x7f196c0d0580 2024-03-22T04:11:46.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c0c6cd0 con 0x7f196c0d0580 2024-03-22T04:11:46.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1968c00640 1 -- 172.21.15.86:0/3447483278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195400cdb0 con 0x7f196c0d0580 2024-03-22T04:11:46.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c15c2c0 con 0x7f196c0d0580 2024-03-22T04:11:46.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c068650 con 0x7f196c0d0580 2024-03-22T04:11:46.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1968c00640 1 -- 172.21.15.86:0/3447483278 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f195401a070 con 0x7f196c0d0580 2024-03-22T04:11:46.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1968c00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:46.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f1968c00640 1 -- 172.21.15.86:0/3447483278 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f19540861c0 con 0x7f196c0d0580 2024-03-22T04:11:46.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.927+0000 7f196be00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:46.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.931+0000 7f196be00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f195c007fd0 tx=0x7f195c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:46.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:46.931+0000 7f1968c00640 1 -- 172.21.15.86:0/3447483278 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1954056db0 con 0x7f196c0d0580 2024-03-22T04:11:47.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.143+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f196c06dd30 con 0x7f196c0d0580 2024-03-22T04:11:47.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.143+0000 7f1968c00640 1 -- 172.21.15.86:0/3447483278 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f19540566b0 con 0x7f196c0d0580 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 msgr2=0x7f1948069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f195c007fd0 tx=0x7f195c009040 comp rx=0 tx=0).stop 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c0d0580 msgr2=0x7f196c156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c0d0580 0x7f196c156a70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f195400e930 tx=0x7f195400edf0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 shutdown_connections 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c15e630 0x7f196c157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c0d0580 0x7f196c156a70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 --2- 172.21.15.86:0/3447483278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c06e2f0 0x7f196c0b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 >> 172.21.15.86:0/3447483278 conn(0x7f196c06f910 msgr2=0x7f196c1611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 shutdown_connections 2024-03-22T04:11:47.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.147+0000 7f1972c00640 1 -- 172.21.15.86:0/3447483278 wait complete. 2024-03-22T04:11:47.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-22T04:11:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6240600640 1 Processor -- start 2024-03-22T04:11:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6240600640 1 -- start start 2024-03-22T04:11:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6240600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62380ae540 0x7f62380b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6240600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6238071920 con 0x7f623815b3f0 2024-03-22T04:11:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6240600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6238071c00 con 0x7f62380730d0 2024-03-22T04:11:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6240600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62380716d0 con 0x7f62380ae540 2024-03-22T04:11:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623dc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f623815b3f0 0x7f62380729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48702/0 (socket says 172.21.15.86:48702) 2024-03-22T04:11:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623dc00640 1 -- 172.21.15.86:0/3987902079 learned_addr learned my addr 172.21.15.86:0/3987902079 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405529323 0 0) 0x7f6238071920 con 0x7f623815b3f0 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623e600640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62380ae540 0x7f62380b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6220005740 con 0x7f623815b3f0 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 890392052 0 0) 0x7f6220005740 con 0x7f623815b3f0 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62380ae540 msgr2=0x7f62380b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62380ae540 0x7f62380b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 >> v1:172.21.15.196:6789/0 conn(0x7f62380730d0 legacy=0x7f62380ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6238071ee0 con 0x7f623815b3f0 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f622c001680 con 0x7f623815b3f0 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 >> v1:172.21.15.86:6789/0 conn(0x7f623815b3f0 legacy=0x7f62380729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62380ae540 0x7f6220005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6220006180 0x7f6220006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006f60 0x7f62200073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62380716d0 con 0x7f6220006180 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f622000dcc0 con 0x7f6220006f60 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f622000dec0 con 0x7f62380ae540 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623d200640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6220006180 0x7f6220006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623dc00640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62380ae540 0x7f6220005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623e600640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006f60 0x7f62200073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623d200640 1 -- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62380ae540 msgr2=0x7f6220005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623d200640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62380ae540 0x7f6220005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623d200640 1 -- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006f60 msgr2=0x7f62200073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623d200640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006f60 0x7f62200073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623d200640 1 -- 172.21.15.86:0/3987902079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6220005740 con 0x7f6220006180 2024-03-22T04:11:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.303+0000 7f623e600640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006f60 0x7f62200073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:47.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f623d200640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6220006180 0x7f6220006a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6224001780 tx=0x7f622401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815979772d07519f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240013a0 con 0x7f6220006180 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62240052f0 con 0x7f6220006180 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6237e00640 1 -- 172.21.15.86:0/3987902079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224003840 con 0x7f6220006180 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6240600640 1 -- 172.21.15.86:0/3987902079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6220006180 msgr2=0x7f6220006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6240600640 1 --2- 172.21.15.86:0/3987902079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6220006180 0x7f6220006a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6224001780 tx=0x7f622401c420 comp rx=0 tx=0).stop 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f623e600640 1 -- 172.21.15.86:0/3987902079 reap_dead start 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6240600640 1 -- 172.21.15.86:0/3987902079 shutdown_connections 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6240600640 1 -- 172.21.15.86:0/3987902079 >> 172.21.15.86:0/3987902079 conn(0x7f62380b1df0 msgr2=0x7f62380b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6240600640 1 -- 172.21.15.86:0/3987902079 shutdown_connections 2024-03-22T04:11:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.307+0000 7f6240600640 1 -- 172.21.15.86:0/3987902079 wait complete. 2024-03-22T04:11:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 Processor -- start 2024-03-22T04:11:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 -- start start 2024-03-22T04:11:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006180 0x7f6238158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6238150820 0x7f62380cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62380d0290 0x7f623814fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62380cc3e0 con 0x7f62380d0290 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f623806d990 con 0x7f6220006180 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62380cc560 con 0x7f6238150820 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62380d0290 0x7f623814fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62380d0290 0x7f623814fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53516/0 (socket says 172.21.15.86:53516) 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 -- 172.21.15.86:0/1219786324 learned_addr learned my addr 172.21.15.86:0/1219786324 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623dc00640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006180 0x7f6238158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623d200640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6238150820 0x7f62380cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 -- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6238150820 msgr2=0x7f62380cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6238150820 0x7f62380cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 -- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006180 msgr2=0x7f6238158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006180 0x7f6238158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 -- 172.21.15.86:0/1219786324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62380cca20 con 0x7f62380d0290 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623d200640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6238150820 0x7f62380cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f623e600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62380d0290 0x7f623814fb90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6238070f70 tx=0x7f6230010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6235600640 1 -- 172.21.15.86:0/1219786324 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300039a0 con 0x7f62380d0290 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6235600640 1 -- 172.21.15.86:0/1219786324 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6230003b60 con 0x7f62380d0290 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6235600640 1 -- 172.21.15.86:0/1219786324 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230004020 con 0x7f62380d0290 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62380ccd00 con 0x7f62380d0290 2024-03-22T04:11:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.311+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62381f4440 con 0x7f62380d0290 2024-03-22T04:11:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.315+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6238068650 con 0x7f62380d0290 2024-03-22T04:11:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.315+0000 7f6235600640 1 -- 172.21.15.86:0/1219786324 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f623001e030 con 0x7f62380d0290 2024-03-22T04:11:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.315+0000 7f6235600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6218067b20 0x7f6218069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.315+0000 7f6235600640 1 -- 172.21.15.86:0/1219786324 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f62300864c0 con 0x7f62380d0290 2024-03-22T04:11:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.315+0000 7f623dc00640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6218067b20 0x7f6218069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.315+0000 7f623dc00640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6218067b20 0x7f6218069fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f622c001610 tx=0x7f622c0015a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:47.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.319+0000 7f6235600640 1 -- 172.21.15.86:0/1219786324 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6230025020 con 0x7f62380d0290 2024-03-22T04:11:47.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.527+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f62380b0e40 con 0x7f62380d0290 2024-03-22T04:11:47.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.527+0000 7f6235600640 1 -- 172.21.15.86:0/1219786324 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v13) ==== 90+0+0 (secure 0 0 0) 0x7f6230057120 con 0x7f62380d0290 2024-03-22T04:11:47.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6218067b20 msgr2=0x7f6218069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6218067b20 0x7f6218069fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f622c001610 tx=0x7f622c0015a0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62380d0290 msgr2=0x7f623814fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62380d0290 0x7f623814fb90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6238070f70 tx=0x7f6230010590 comp rx=0 tx=0).stop 2024-03-22T04:11:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 shutdown_connections 2024-03-22T04:11:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6218067b20 0x7f6218069fe0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62380d0290 0x7f623814fb90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6238150820 0x7f62380cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 --2- 172.21.15.86:0/1219786324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6220006180 0x7f6238158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 >> 172.21.15.86:0/1219786324 conn(0x7f62380b1df0 msgr2=0x7f6238159990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 shutdown_connections 2024-03-22T04:11:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.531+0000 7f6240600640 1 -- 172.21.15.86:0/1219786324 wait complete. 2024-03-22T04:11:47.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-22T04:11:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.687+0000 7f93d9600640 1 Processor -- start 2024-03-22T04:11:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.687+0000 7f93d9600640 1 -- start start 2024-03-22T04:11:47.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.687+0000 7f93d9600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f93d40ae540 0x7f93d40b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.687+0000 7f93d9600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d4071920 con 0x7f93d40730d0 2024-03-22T04:11:47.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.687+0000 7f93d9600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d4071c00 con 0x7f93d415b3f0 2024-03-22T04:11:47.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.687+0000 7f93d9600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f93d40716d0 con 0x7f93d40ae540 2024-03-22T04:11:47.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f93d40ae540 0x7f93d40b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93ca000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f93d40730d0 0x7f93d40ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48710/0 (socket says 172.21.15.86:48710) 2024-03-22T04:11:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93ca000640 1 -- 172.21.15.86:0/2181157176 learned_addr learned my addr 172.21.15.86:0/2181157176 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d3400640 1 -- 172.21.15.86:0/2181157176 >> v1:172.21.15.196:6789/0 conn(0x7f93d415b3f0 legacy=0x7f93d40729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d3400640 1 -- 172.21.15.86:0/2181157176 >> v1:172.21.15.86:6789/0 conn(0x7f93d40730d0 legacy=0x7f93d40ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d3400640 1 -- 172.21.15.86:0/2181157176 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d4071ee0 con 0x7f93d40ae540 2024-03-22T04:11:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d2000640 1 -- 172.21.15.86:0/2181157176 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2713929184 0 0) 0x7f93d4071920 con 0x7f93d40730d0 2024-03-22T04:11:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d3400640 1 --2- 172.21.15.86:0/2181157176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40ae540 0x7f93d40b4300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009d90 tx=0x7f93cc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5919479fb3a21e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d2000640 1 -- 172.21.15.86:0/2181157176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0b2070 con 0x7f93d40ae540 2024-03-22T04:11:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d2000640 1 -- 172.21.15.86:0/2181157176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93cc0a5730 con 0x7f93d40ae540 2024-03-22T04:11:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.691+0000 7f93d2000640 1 -- 172.21.15.86:0/2181157176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0a5b00 con 0x7f93d40ae540 2024-03-22T04:11:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- 172.21.15.86:0/2181157176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40ae540 msgr2=0x7f93d40b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 --2- 172.21.15.86:0/2181157176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40ae540 0x7f93d40b4300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009d90 tx=0x7f93cc01c970 comp rx=0 tx=0).stop 2024-03-22T04:11:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- 172.21.15.86:0/2181157176 shutdown_connections 2024-03-22T04:11:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 --2- 172.21.15.86:0/2181157176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40ae540 0x7f93d40b4300 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- 172.21.15.86:0/2181157176 >> 172.21.15.86:0/2181157176 conn(0x7f93d40b1df0 msgr2=0x7f93d40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- 172.21.15.86:0/2181157176 shutdown_connections 2024-03-22T04:11:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- 172.21.15.86:0/2181157176 wait complete. 2024-03-22T04:11:47.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 Processor -- start 2024-03-22T04:11:47.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- start start 2024-03-22T04:11:47.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40730d0 0x7f93d415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d40ae540 0x7f93d41470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93d415b3f0 0x7f93d4155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40b4c50 con 0x7f93d40730d0 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40b4ad0 con 0x7f93d40ae540 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d9600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f93d4169a10 con 0x7f93d415b3f0 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40730d0 0x7f93d415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40730d0 0x7f93d415a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53526/0 (socket says 172.21.15.86:53526) 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 -- 172.21.15.86:0/2568083058 learned_addr learned my addr 172.21.15.86:0/2568083058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93ca000640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d40ae540 0x7f93d41470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d3400640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93d415b3f0 0x7f93d4155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 -- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93d415b3f0 msgr2=0x7f93d4155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93d415b3f0 0x7f93d4155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 -- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d40ae540 msgr2=0x7f93d41470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d40ae540 0x7f93d41470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 -- 172.21.15.86:0/2568083058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d415bb10 con 0x7f93d40730d0 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d3400640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93d415b3f0 0x7f93d4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.695+0000 7f93d2a00640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40730d0 0x7f93d415a560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f93c400c530 tx=0x7f93c400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93cb400640 1 -- 172.21.15.86:0/2568083058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4016020 con 0x7f93d40730d0 2024-03-22T04:11:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93cb400640 1 -- 172.21.15.86:0/2568083058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c4005220 con 0x7f93d40730d0 2024-03-22T04:11:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d415bdf0 con 0x7f93d40730d0 2024-03-22T04:11:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93cb400640 1 -- 172.21.15.86:0/2568083058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4007030 con 0x7f93d40730d0 2024-03-22T04:11:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d415c3b0 con 0x7f93d40730d0 2024-03-22T04:11:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d4068650 con 0x7f93d40730d0 2024-03-22T04:11:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93cb400640 1 -- 172.21.15.86:0/2568083058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f93c40071d0 con 0x7f93d40730d0 2024-03-22T04:11:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93cb400640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93b0067b20 0x7f93b0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93cb400640 1 -- 172.21.15.86:0/2568083058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f93c4085770 con 0x7f93d40730d0 2024-03-22T04:11:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93ca000640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93b0067b20 0x7f93b0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:47.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.699+0000 7f93ca000640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93b0067b20 0x7f93b0069fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93d41fab80 tx=0x7f93bc007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:47.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.703+0000 7f93cb400640 1 -- 172.21.15.86:0/2568083058 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f93c4020020 con 0x7f93d40730d0 2024-03-22T04:11:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.915+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f93d40ca590 con 0x7f93d40730d0 2024-03-22T04:11:47.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93cb400640 1 -- 172.21.15.86:0/2568083058 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v14) ==== 97+0+0 (secure 0 0 0) 0x7f93c40074c0 con 0x7f93d40730d0 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93b0067b20 msgr2=0x7f93b0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93b0067b20 0x7f93b0069fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93d41fab80 tx=0x7f93bc007d10 comp rx=0 tx=0).stop 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40730d0 msgr2=0x7f93d415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40730d0 0x7f93d415a560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f93c400c530 tx=0x7f93c400c9f0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 shutdown_connections 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93b0067b20 0x7f93b0069fe0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d40ae540 0x7f93d41470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93d415b3f0 0x7f93d4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 --2- 172.21.15.86:0/2568083058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93d40730d0 0x7f93d415a560 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 >> 172.21.15.86:0/2568083058 conn(0x7f93d40b1df0 msgr2=0x7f93d4154400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 shutdown_connections 2024-03-22T04:11:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:47.919+0000 7f93d9600640 1 -- 172.21.15.86:0/2568083058 wait complete. 2024-03-22T04:11:47.944 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-22T04:11:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.075+0000 7f15fe800640 1 Processor -- start 2024-03-22T04:11:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.075+0000 7f15fe800640 1 -- start start 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.075+0000 7f15fe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15f80ad330 0x7f15f815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.075+0000 7f15fe800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f15f806ef40 0x7f15f806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.075+0000 7f15fe800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 0x7f15f813fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.075+0000 7f15fe800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f15f806dc40 con 0x7f15f80ad330 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.075+0000 7f15fe800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f15f815c7c0 con 0x7f15f806ef40 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.075+0000 7f15fe800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15f806dea0 con 0x7f15f80ad710 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15f80ad330 0x7f15f815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15f80ad330 0x7f15f815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53536/0 (socket says 172.21.15.86:53536) 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 -- 172.21.15.86:0/3838208487 learned_addr learned my addr 172.21.15.86:0/3838208487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15eea00640 1 --2- 172.21.15.86:0/3838208487 >> v2:172.21.15.196:3300/0 conn(0x7f15f806ef40 0x7f15f806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7e00640 1 --2- 172.21.15.86:0/3838208487 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 0x7f15f813fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 -- 172.21.15.86:0/3838208487 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 msgr2=0x7f15f813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 --2- 172.21.15.86:0/3838208487 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 0x7f15f813fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 -- 172.21.15.86:0/3838208487 >> v2:172.21.15.196:3300/0 conn(0x7f15f806ef40 msgr2=0x7f15f806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 --2- 172.21.15.86:0/3838208487 >> v2:172.21.15.196:3300/0 conn(0x7f15f806ef40 0x7f15f806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 -- 172.21.15.86:0/3838208487 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f806e180 con 0x7f15f80ad330 2024-03-22T04:11:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15eea00640 1 --2- 172.21.15.86:0/3838208487 >> v2:172.21.15.196:3300/0 conn(0x7f15f806ef40 0x7f15f806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f7400640 1 --2- 172.21.15.86:0/3838208487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f80ad330 0x7f15f815d4a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f15f806fbe0 tx=0x7f15e801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f0d3e6200c9aa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f6a00640 1 -- 172.21.15.86:0/3838208487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80a5840 con 0x7f15f80ad330 2024-03-22T04:11:48.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f6a00640 1 -- 172.21.15.86:0/3838208487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e80a59e0 con 0x7f15f80ad330 2024-03-22T04:11:48.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.079+0000 7f15f6a00640 1 -- 172.21.15.86:0/3838208487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80a5db0 con 0x7f15f80ad330 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- 172.21.15.86:0/3838208487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f80ad330 msgr2=0x7f15f815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 --2- 172.21.15.86:0/3838208487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f80ad330 0x7f15f815d4a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f15f806fbe0 tx=0x7f15e801c9c0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- 172.21.15.86:0/3838208487 shutdown_connections 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 --2- 172.21.15.86:0/3838208487 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 0x7f15f813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 --2- 172.21.15.86:0/3838208487 >> v2:172.21.15.196:3300/0 conn(0x7f15f806ef40 0x7f15f806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 --2- 172.21.15.86:0/3838208487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f80ad330 0x7f15f815d4a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- 172.21.15.86:0/3838208487 >> 172.21.15.86:0/3838208487 conn(0x7f15f80b1df0 msgr2=0x7f15f80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- 172.21.15.86:0/3838208487 shutdown_connections 2024-03-22T04:11:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- 172.21.15.86:0/3838208487 wait complete. 2024-03-22T04:11:48.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 Processor -- start 2024-03-22T04:11:48.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- start start 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f806ef40 0x7f15f80cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15f80ad330 0x7f15f81515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 0x7f15f815f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8141800 con 0x7f15f806ef40 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8141680 con 0x7f15f80ad330 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15fe800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15f8141980 con 0x7f15f80ad710 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f806ef40 0x7f15f80cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f806ef40 0x7f15f80cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53542/0 (socket says 172.21.15.86:53542) 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15f7400640 1 -- 172.21.15.86:0/3111542133 learned_addr learned my addr 172.21.15.86:0/3111542133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15f7e00640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 0x7f15f815f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15eea00640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15f80ad330 0x7f15f81515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15f7400640 1 -- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 msgr2=0x7f15f815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15f7400640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 0x7f15f815f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.083+0000 7f15f7400640 1 -- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15f80ad330 msgr2=0x7f15f81515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15f7400640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15f80ad330 0x7f15f81515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15f7400640 1 -- 172.21.15.86:0/3111542133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f80b0bf0 con 0x7f15f806ef40 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15eea00640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15f80ad330 0x7f15f81515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15f7400640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f806ef40 0x7f15f80cf3a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f15e801b070 tx=0x7f15e8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15efe00640 1 -- 172.21.15.86:0/3111542133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80bc020 con 0x7f15f806ef40 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15efe00640 1 -- 172.21.15.86:0/3111542133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e80b8670 con 0x7f15f806ef40 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f80b0ed0 con 0x7f15f806ef40 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15efe00640 1 -- 172.21.15.86:0/3111542133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80b8a60 con 0x7f15f806ef40 2024-03-22T04:11:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f815ede0 con 0x7f15f806ef40 2024-03-22T04:11:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f815f840 con 0x7f15f806ef40 2024-03-22T04:11:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15efe00640 1 -- 172.21.15.86:0/3111542133 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f15e80bf020 con 0x7f15f806ef40 2024-03-22T04:11:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15efe00640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f15d4067b20 0x7f15d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15efe00640 1 -- 172.21.15.86:0/3111542133 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f15e8120dc0 con 0x7f15f806ef40 2024-03-22T04:11:48.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.087+0000 7f15eea00640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f15d4067b20 0x7f15d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.091+0000 7f15eea00640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f15d4067b20 0x7f15d4069fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15f8153000 tx=0x7f15e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.091+0000 7f15efe00640 1 -- 172.21.15.86:0/3111542133 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f15f815f840 con 0x7f15f806ef40 2024-03-22T04:11:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.299+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f15f8068650 con 0x7f15f806ef40 2024-03-22T04:11:48.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.299+0000 7f15efe00640 1 -- 172.21.15.86:0/3111542133 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v15) ==== 90+0+0 (secure 0 0 0) 0x7f15e80c6030 con 0x7f15f806ef40 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f15d4067b20 msgr2=0x7f15d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f15d4067b20 0x7f15d4069fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15f8153000 tx=0x7f15e0008040 comp rx=0 tx=0).stop 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f806ef40 msgr2=0x7f15f80cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f806ef40 0x7f15f80cf3a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f15e801b070 tx=0x7f15e8019760 comp rx=0 tx=0).stop 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 shutdown_connections 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f15d4067b20 0x7f15d4069fe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15f80ad330 0x7f15f81515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15f80ad710 0x7f15f815f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 --2- 172.21.15.86:0/3111542133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f806ef40 0x7f15f80cf3a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 >> 172.21.15.86:0/3111542133 conn(0x7f15f80b1df0 msgr2=0x7f15f80b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:48.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 shutdown_connections 2024-03-22T04:11:48.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.303+0000 7f15fe800640 1 -- 172.21.15.86:0/3111542133 wait complete. 2024-03-22T04:11:48.327 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.455+0000 7f5295a00640 1 Processor -- start 2024-03-22T04:11:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.455+0000 7f5295a00640 1 -- start start 2024-03-22T04:11:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f5295a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f529015b3f0 0x7f52900729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f5295a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f52900cc3e0 0x7f5290072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f5295a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5290071920 con 0x7f52900b30f0 2024-03-22T04:11:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f5295a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52900716d0 con 0x7f529015b3f0 2024-03-22T04:11:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f5295a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f529006d990 con 0x7f52900cc3e0 2024-03-22T04:11:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f528ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f529015b3f0 0x7f52900729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f528ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f529015b3f0 0x7f52900729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53558/0 (socket says 172.21.15.86:53558) 2024-03-22T04:11:48.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f528ea00640 1 -- 172.21.15.86:0/2853246007 learned_addr learned my addr 172.21.15.86:0/2853246007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:48.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f5286000640 1 --2- 172.21.15.86:0/2853246007 >> v2:172.21.15.196:3300/0 conn(0x7f52900cc3e0 0x7f5290072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f528ea00640 1 -- 172.21.15.86:0/2853246007 >> v2:172.21.15.196:3300/0 conn(0x7f52900cc3e0 msgr2=0x7f5290072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:48.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f528ea00640 1 --2- 172.21.15.86:0/2853246007 >> v2:172.21.15.196:3300/0 conn(0x7f52900cc3e0 0x7f5290072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f528ea00640 1 -- 172.21.15.86:0/2853246007 >> v1:172.21.15.196:6789/0 conn(0x7f52900b30f0 legacy=0x7f52900ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:48.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.459+0000 7f528ea00640 1 -- 172.21.15.86:0/2853246007 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5290071c00 con 0x7f529015b3f0 2024-03-22T04:11:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f528ea00640 1 --2- 172.21.15.86:0/2853246007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f52900729c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5290070ed0 tx=0x7f528001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94799c25263d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f528e000640 1 -- 172.21.15.86:0/2853246007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52800a56d0 con 0x7f529015b3f0 2024-03-22T04:11:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f528e000640 1 -- 172.21.15.86:0/2853246007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52800a5870 con 0x7f529015b3f0 2024-03-22T04:11:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f528e000640 1 -- 172.21.15.86:0/2853246007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52800a5c40 con 0x7f529015b3f0 2024-03-22T04:11:48.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 -- 172.21.15.86:0/2853246007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 msgr2=0x7f52900729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 --2- 172.21.15.86:0/2853246007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f52900729c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5290070ed0 tx=0x7f528001c920 comp rx=0 tx=0).stop 2024-03-22T04:11:48.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 -- 172.21.15.86:0/2853246007 shutdown_connections 2024-03-22T04:11:48.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 --2- 172.21.15.86:0/2853246007 >> v2:172.21.15.196:3300/0 conn(0x7f52900cc3e0 0x7f5290072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 --2- 172.21.15.86:0/2853246007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f52900729c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 -- 172.21.15.86:0/2853246007 >> 172.21.15.86:0/2853246007 conn(0x7f52900b1df0 msgr2=0x7f52900b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:48.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 -- 172.21.15.86:0/2853246007 shutdown_connections 2024-03-22T04:11:48.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 -- 172.21.15.86:0/2853246007 wait complete. 2024-03-22T04:11:48.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 Processor -- start 2024-03-22T04:11:48.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.463+0000 7f5295a00640 1 -- start start 2024-03-22T04:11:48.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52900b30f0 0x7f52900ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52900cc3e0 0x7f5290152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f529014aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5290168580 con 0x7f529015b3f0 2024-03-22T04:11:48.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5290168400 con 0x7f52900cc3e0 2024-03-22T04:11:48.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5290168700 con 0x7f52900b30f0 2024-03-22T04:11:48.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f529014aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f529014aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53570/0 (socket says 172.21.15.86:53570) 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 -- 172.21.15.86:0/2430285292 learned_addr learned my addr 172.21.15.86:0/2430285292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5286000640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52900cc3e0 0x7f5290152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528ea00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52900b30f0 0x7f52900ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 -- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52900b30f0 msgr2=0x7f52900ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52900b30f0 0x7f52900ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 -- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52900cc3e0 msgr2=0x7f5290152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52900cc3e0 0x7f5290152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 -- 172.21.15.86:0/2430285292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52901538d0 con 0x7f529015b3f0 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5286000640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52900cc3e0 0x7f5290152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f528f400640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f529014aa40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f528800b410 tx=0x7f528800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5287400640 1 -- 172.21.15.86:0/2430285292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288015020 con 0x7f529015b3f0 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5287400640 1 -- 172.21.15.86:0/2430285292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5288011870 con 0x7f529015b3f0 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5287400640 1 -- 172.21.15.86:0/2430285292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288011c60 con 0x7f529015b3f0 2024-03-22T04:11:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5290153bb0 con 0x7f529015b3f0 2024-03-22T04:11:48.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5290154170 con 0x7f529015b3f0 2024-03-22T04:11:48.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5290068650 con 0x7f529015b3f0 2024-03-22T04:11:48.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.467+0000 7f5287400640 1 -- 172.21.15.86:0/2430285292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5288018020 con 0x7f529015b3f0 2024-03-22T04:11:48.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.471+0000 7f5287400640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f526c067b50 0x7f526c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.471+0000 7f5287400640 1 -- 172.21.15.86:0/2430285292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f52880858d0 con 0x7f529015b3f0 2024-03-22T04:11:48.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.471+0000 7f528ea00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f526c067b50 0x7f526c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.471+0000 7f528ea00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f526c067b50 0x7f526c06a010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52800ae000 tx=0x7f52800a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.475+0000 7f5287400640 1 -- 172.21.15.86:0/2430285292 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f528801f030 con 0x7f529015b3f0 2024-03-22T04:11:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.631+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f529006ed30 con 0x7f529015b3f0 2024-03-22T04:11:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.631+0000 7f5287400640 1 -- 172.21.15.86:0/2430285292 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13164 (secure 0 0 0) 0x7f5288018310 con 0x7f529015b3f0 2024-03-22T04:11:48.638 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:48.638 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:45.608427+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f526c067b50 msgr2=0x7f526c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f526c067b50 0x7f526c06a010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52800ae000 tx=0x7f52800a7070 comp rx=0 tx=0).stop 2024-03-22T04:11:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 msgr2=0x7f529014aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f529014aa40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f528800b410 tx=0x7f528800b8d0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 shutdown_connections 2024-03-22T04:11:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f526c067b50 0x7f526c06a010 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52900cc3e0 0x7f5290152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529015b3f0 0x7f529014aa40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 --2- 172.21.15.86:0/2430285292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52900b30f0 0x7f52900ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 >> 172.21.15.86:0/2430285292 conn(0x7f52900b1df0 msgr2=0x7f52900b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 shutdown_connections 2024-03-22T04:11:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.635+0000 7f5295a00640 1 -- 172.21.15.86:0/2430285292 wait complete. 2024-03-22T04:11:48.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:11:48.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:11:48.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.791+0000 7f27abc00640 1 Processor -- start 2024-03-22T04:11:48.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.791+0000 7f27abc00640 1 -- start start 2024-03-22T04:11:48.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27abc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27a406e7e0 0x7f27a406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27abc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a415c770 con 0x7f27a4072b00 2024-03-22T04:11:48.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27abc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a40b29a0 con 0x7f27a4072650 2024-03-22T04:11:48.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27abc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f27a4071350 con 0x7f27a406e7e0 2024-03-22T04:11:48.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27a9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27a406e7e0 0x7f27a406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27a9c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f27a4072b00 0x7f27a40b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48714/0 (socket says 172.21.15.86:48714) 2024-03-22T04:11:48.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27a9c00640 1 -- 172.21.15.86:0/3652075241 learned_addr learned my addr 172.21.15.86:0/3652075241 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:48.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27a9200640 1 -- 172.21.15.86:0/3652075241 >> v1:172.21.15.196:6789/0 conn(0x7f27a4072650 legacy=0x7f27a40b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27a9200640 1 -- 172.21.15.86:0/3652075241 >> v1:172.21.15.86:6789/0 conn(0x7f27a4072b00 legacy=0x7f27a40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27a3400640 1 -- 172.21.15.86:0/3652075241 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2126855088 0 0) 0x7f279c0014a0 con 0x7f27a4072b00 2024-03-22T04:11:48.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.795+0000 7f27a9200640 1 -- 172.21.15.86:0/3652075241 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a406f100 con 0x7f27a406e7e0 2024-03-22T04:11:48.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27a9200640 1 --2- 172.21.15.86:0/3652075241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a406ebc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f27980097c0 tx=0x7f279801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0aa2a12c4c1603 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27a3400640 1 -- 172.21.15.86:0/3652075241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980b0070 con 0x7f27a406e7e0 2024-03-22T04:11:48.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27a3400640 1 -- 172.21.15.86:0/3652075241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27980ab600 con 0x7f27a406e7e0 2024-03-22T04:11:48.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27a3400640 1 -- 172.21.15.86:0/3652075241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980ab9d0 con 0x7f27a406e7e0 2024-03-22T04:11:48.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- 172.21.15.86:0/3652075241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 msgr2=0x7f27a406ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 --2- 172.21.15.86:0/3652075241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a406ebc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f27980097c0 tx=0x7f279801c410 comp rx=0 tx=0).stop 2024-03-22T04:11:48.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- 172.21.15.86:0/3652075241 shutdown_connections 2024-03-22T04:11:48.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 --2- 172.21.15.86:0/3652075241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a406ebc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- 172.21.15.86:0/3652075241 >> 172.21.15.86:0/3652075241 conn(0x7f27a40cc940 msgr2=0x7f27a40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:48.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- 172.21.15.86:0/3652075241 shutdown_connections 2024-03-22T04:11:48.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- 172.21.15.86:0/3652075241 wait complete. 2024-03-22T04:11:48.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 Processor -- start 2024-03-22T04:11:48.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- start start 2024-03-22T04:11:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a40c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27a4072650 0x7f27a4149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27a4072b00 0x7f27a4150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27a40ce930 con 0x7f27a406e7e0 2024-03-22T04:11:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27a4071b60 con 0x7f27a4072b00 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.799+0000 7f27abc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f27a4072080 con 0x7f27a4072650 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a40c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a40c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53584/0 (socket says 172.21.15.86:53584) 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 -- 172.21.15.86:0/1862021596 learned_addr learned my addr 172.21.15.86:0/1862021596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9c00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27a4072b00 0x7f27a4150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a3e00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27a4072650 0x7f27a4149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 -- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27a4072650 msgr2=0x7f27a4149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27a4072650 0x7f27a4149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 -- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27a4072b00 msgr2=0x7f27a4150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27a4072b00 0x7f27a4150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 -- 172.21.15.86:0/1862021596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a40ca6d0 con 0x7f27a406e7e0 2024-03-22T04:11:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a3e00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27a4072650 0x7f27a4149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a9200640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a40c9270 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f27980ae040 tx=0x7f279801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a0c00640 1 -- 172.21.15.86:0/1862021596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980b0050 con 0x7f27a406e7e0 2024-03-22T04:11:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a0c00640 1 -- 172.21.15.86:0/1862021596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27980bc070 con 0x7f27a406e7e0 2024-03-22T04:11:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a40ca9b0 con 0x7f27a406e7e0 2024-03-22T04:11:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a0c00640 1 -- 172.21.15.86:0/1862021596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980b5a70 con 0x7f27a406e7e0 2024-03-22T04:11:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a40ca3a0 con 0x7f27a406e7e0 2024-03-22T04:11:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a4074470 con 0x7f27a406e7e0 2024-03-22T04:11:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a0c00640 1 -- 172.21.15.86:0/1862021596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f27980ab4c0 con 0x7f27a406e7e0 2024-03-22T04:11:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.803+0000 7f27a0c00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2784067b50 0x7f278406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.807+0000 7f27a0c00640 1 -- 172.21.15.86:0/1862021596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f27981205e0 con 0x7f27a406e7e0 2024-03-22T04:11:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.807+0000 7f27a3e00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2784067b50 0x7f278406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.807+0000 7f27a3e00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2784067b50 0x7f278406a010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f27a41533f0 tx=0x7f2790009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.811+0000 7f27a0c00640 1 -- 172.21.15.86:0/1862021596 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2798016090 con 0x7f27a406e7e0 2024-03-22T04:11:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27a4153e30 con 0x7f27a406e7e0 2024-03-22T04:11:48.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27a0c00640 1 -- 172.21.15.86:0/1862021596 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7f27980f1110 con 0x7f27a406e7e0 2024-03-22T04:11:48.972 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:11:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2784067b50 msgr2=0x7f278406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2784067b50 0x7f278406a010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f27a41533f0 tx=0x7f2790009040 comp rx=0 tx=0).stop 2024-03-22T04:11:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 msgr2=0x7f27a40c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a40c9270 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f27980ae040 tx=0x7f279801ac50 comp rx=0 tx=0).stop 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 shutdown_connections 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2784067b50 0x7f278406a010 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27a4072b00 0x7f27a4150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27a4072650 0x7f27a4149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 --2- 172.21.15.86:0/1862021596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27a406e7e0 0x7f27a40c9270 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 >> 172.21.15.86:0/1862021596 conn(0x7f27a40cc940 msgr2=0x7f27a4159740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 shutdown_connections 2024-03-22T04:11:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:48.967+0000 7f27abc00640 1 -- 172.21.15.86:0/1862021596 wait complete. 2024-03-22T04:11:48.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T04:11:49.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.123+0000 7fe603e00640 1 Processor -- start 2024-03-22T04:11:49.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.123+0000 7fe603e00640 1 -- start start 2024-03-22T04:11:49.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe603e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe5fc15b3f0 0x7fe5fc0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe603e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe5fc0ae540 0x7fe5fc072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe603e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5fc071920 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe603e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe5fc0716d0 con 0x7fe5fc0ae540 2024-03-22T04:11:49.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe603e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe5fc06d990 con 0x7fe5fc15b3f0 2024-03-22T04:11:49.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe601e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe5fc0ae540 0x7fe5fc072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe601400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe5fc15b3f0 0x7fe5fc0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe601e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe5fc0ae540 0x7fe5fc072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53588/0 (socket says 172.21.15.86:53588) 2024-03-22T04:11:49.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe601e00640 1 -- 172.21.15.86:0/3787686511 learned_addr learned my addr 172.21.15.86:0/3787686511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:49.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe600a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe5fc0cc3e0 0x7fe5fc0ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:59586/0 (socket says 172.21.15.86:59586) 2024-03-22T04:11:49.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe601e00640 1 -- 172.21.15.86:0/3787686511 >> v2:172.21.15.196:3300/0 conn(0x7fe5fc15b3f0 msgr2=0x7fe5fc0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:49.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe601e00640 1 --2- 172.21.15.86:0/3787686511 >> v2:172.21.15.196:3300/0 conn(0x7fe5fc15b3f0 0x7fe5fc0729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe601e00640 1 -- 172.21.15.86:0/3787686511 >> v1:172.21.15.196:6789/0 conn(0x7fe5fc0cc3e0 legacy=0x7fe5fc0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.127+0000 7fe601e00640 1 -- 172.21.15.86:0/3787686511 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc071c00 con 0x7fe5fc0ae540 2024-03-22T04:11:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe601e00640 1 --2- 172.21.15.86:0/3787686511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0ae540 0x7fe5fc072f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc06c510 tx=0x7fe5f401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827f0721396043f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe5fbe00640 1 -- 172.21.15.86:0/3787686511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40b2070 con 0x7fe5fc0ae540 2024-03-22T04:11:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe5fbe00640 1 -- 172.21.15.86:0/3787686511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f40ad680 con 0x7fe5fc0ae540 2024-03-22T04:11:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe5fbe00640 1 -- 172.21.15.86:0/3787686511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40ada50 con 0x7fe5fc0ae540 2024-03-22T04:11:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- 172.21.15.86:0/3787686511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0ae540 msgr2=0x7fe5fc072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 --2- 172.21.15.86:0/3787686511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0ae540 0x7fe5fc072f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc06c510 tx=0x7fe5f401c490 comp rx=0 tx=0).stop 2024-03-22T04:11:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- 172.21.15.86:0/3787686511 shutdown_connections 2024-03-22T04:11:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 --2- 172.21.15.86:0/3787686511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0ae540 0x7fe5fc072f00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 --2- 172.21.15.86:0/3787686511 >> v2:172.21.15.196:3300/0 conn(0x7fe5fc15b3f0 0x7fe5fc0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- 172.21.15.86:0/3787686511 >> 172.21.15.86:0/3787686511 conn(0x7fe5fc0b1df0 msgr2=0x7fe5fc0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- 172.21.15.86:0/3787686511 shutdown_connections 2024-03-22T04:11:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- 172.21.15.86:0/3787686511 wait complete. 2024-03-22T04:11:49.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 Processor -- start 2024-03-22T04:11:49.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- start start 2024-03-22T04:11:49.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5fc0ae540 0x7fe5fc1fad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0cc3e0 0x7fe5fc15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5fc15b3f0 0x7fe5fc155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc168580 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc168400 con 0x7fe5fc0ae540 2024-03-22T04:11:49.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.131+0000 7fe603e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5fc168700 con 0x7fe5fc15b3f0 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0cc3e0 0x7fe5fc15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0cc3e0 0x7fe5fc15cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53590/0 (socket says 172.21.15.86:53590) 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 -- 172.21.15.86:0/3115295262 learned_addr learned my addr 172.21.15.86:0/3115295262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe601400640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5fc0ae540 0x7fe5fc1fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe601e00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5fc15b3f0 0x7fe5fc155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 -- 172.21.15.86:0/3115295262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5fc15b3f0 msgr2=0x7fe5fc155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5fc15b3f0 0x7fe5fc155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 -- 172.21.15.86:0/3115295262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5fc0ae540 msgr2=0x7fe5fc1fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5fc0ae540 0x7fe5fc1fad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 -- 172.21.15.86:0/3115295262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc157bd0 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe600a00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0cc3e0 0x7fe5fc15cdf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c450 tx=0x7fe5e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe5f9600640 1 -- 172.21.15.86:0/3115295262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8018070 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe5f9600640 1 -- 172.21.15.86:0/3115295262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e8007480 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe5f9600640 1 -- 172.21.15.86:0/3115295262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e80047f0 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc157eb0 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc0cb7e0 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5fc0cb9e0 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe5f9600640 1 -- 172.21.15.86:0/3115295262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe5e801b070 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.135+0000 7fe5f9600640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5dc067ba0 0x7fe5dc06a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.139+0000 7fe5f9600640 1 -- 172.21.15.86:0/3115295262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fe5e8084f00 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.139+0000 7fe601400640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5dc067ba0 0x7fe5dc06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.139+0000 7fe601400640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5dc067ba0 0x7fe5dc06a060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00113e0 tx=0x7fe5f000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:49.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.139+0000 7fe5f9600640 1 -- 172.21.15.86:0/3115295262 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe5fc0cb9e0 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe5fc068650 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe5f9600640 1 -- 172.21.15.86:0/3115295262 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) ==== 84+0+65 (secure 0 0 0) 0x7fe5e8055560 con 0x7fe5fc0cc3e0 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:11:49.304 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:11:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5dc067ba0 msgr2=0x7fe5dc06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5dc067ba0 0x7fe5dc06a060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00113e0 tx=0x7fe5f000b040 comp rx=0 tx=0).stop 2024-03-22T04:11:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0cc3e0 msgr2=0x7fe5fc15cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0cc3e0 0x7fe5fc15cdf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c450 tx=0x7fe5e800c910 comp rx=0 tx=0).stop 2024-03-22T04:11:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 shutdown_connections 2024-03-22T04:11:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5dc067ba0 0x7fe5dc06a060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5fc15b3f0 0x7fe5fc155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5fc0cc3e0 0x7fe5fc15cdf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 --2- 172.21.15.86:0/3115295262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5fc0ae540 0x7fe5fc1fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 >> 172.21.15.86:0/3115295262 conn(0x7fe5fc0b1df0 msgr2=0x7fe5fc15da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:49.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 shutdown_connections 2024-03-22T04:11:49.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.299+0000 7fe603e00640 1 -- 172.21.15.86:0/3115295262 wait complete. 2024-03-22T04:11:49.325 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T04:11:49.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T04:11:49.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa07aa00640 1 Processor -- start 2024-03-22T04:11:49.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa07aa00640 1 -- start start 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa07aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa074144a90 0x7fa0740727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa07aa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0740b37c0 0x7fa0740ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa07aa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa074144e70 con 0x7fa074072cf0 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa07aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa0740b3e00 con 0x7fa074144a90 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa07aa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa0740b40e0 con 0x7fa0740b37c0 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa073e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa074144a90 0x7fa0740727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa073e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa074144a90 0x7fa0740727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53604/0 (socket says 172.21.15.86:53604) 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa073e00640 1 -- 172.21.15.86:0/3496563102 learned_addr learned my addr 172.21.15.86:0/3496563102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa078a00640 1 --2- 172.21.15.86:0/3496563102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0740b37c0 0x7fa0740ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa073e00640 1 -- 172.21.15.86:0/3496563102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0740b37c0 msgr2=0x7fa0740ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:49.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa073e00640 1 --2- 172.21.15.86:0/3496563102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0740b37c0 0x7fa0740ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa073e00640 1 -- 172.21.15.86:0/3496563102 >> v1:172.21.15.196:6789/0 conn(0x7fa074072cf0 legacy=0x7fa0740b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.459+0000 7fa073e00640 1 -- 172.21.15.86:0/3496563102 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0740cdbd0 con 0x7fa074144a90 2024-03-22T04:11:49.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa073e00640 1 --2- 172.21.15.86:0/3496563102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa074144a90 0x7fa0740727b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa06400bb10 tx=0x7fa06401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb05ad741769e045 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa073400640 1 -- 172.21.15.86:0/3496563102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0640b4070 con 0x7fa074144a90 2024-03-22T04:11:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa073400640 1 -- 172.21.15.86:0/3496563102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0640a7b00 con 0x7fa074144a90 2024-03-22T04:11:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa073400640 1 -- 172.21.15.86:0/3496563102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0640af440 con 0x7fa074144a90 2024-03-22T04:11:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa07aa00640 1 -- 172.21.15.86:0/3496563102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa074144a90 msgr2=0x7fa0740727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa07aa00640 1 --2- 172.21.15.86:0/3496563102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa074144a90 0x7fa0740727b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa06400bb10 tx=0x7fa06401ec30 comp rx=0 tx=0).stop 2024-03-22T04:11:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa07aa00640 1 -- 172.21.15.86:0/3496563102 shutdown_connections 2024-03-22T04:11:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa07aa00640 1 --2- 172.21.15.86:0/3496563102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0740b37c0 0x7fa0740ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa07aa00640 1 --2- 172.21.15.86:0/3496563102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa074144a90 0x7fa0740727b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa07aa00640 1 -- 172.21.15.86:0/3496563102 >> 172.21.15.86:0/3496563102 conn(0x7fa07406b660 msgr2=0x7fa07406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa07aa00640 1 -- 172.21.15.86:0/3496563102 shutdown_connections 2024-03-22T04:11:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.463+0000 7fa07aa00640 1 -- 172.21.15.86:0/3496563102 wait complete. 2024-03-22T04:11:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 Processor -- start 2024-03-22T04:11:49.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 -- start start 2024-03-22T04:11:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa074072cf0 0x7fa07415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0740b37c0 0x7fa0740aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa074144a90 0x7fa07413b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0740cf4f0 con 0x7fa0740b37c0 2024-03-22T04:11:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0740cf370 con 0x7fa074144a90 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa0740cf670 con 0x7fa074072cf0 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0740b37c0 0x7fa0740aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0740b37c0 0x7fa0740aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53618/0 (socket says 172.21.15.86:53618) 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 -- 172.21.15.86:0/1992639335 learned_addr learned my addr 172.21.15.86:0/1992639335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa073e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa074072cf0 0x7fa07415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa078a00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa074144a90 0x7fa07413b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 -- 172.21.15.86:0/1992639335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa074072cf0 msgr2=0x7fa07415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa074072cf0 0x7fa07415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 -- 172.21.15.86:0/1992639335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa074144a90 msgr2=0x7fa07413b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa074144a90 0x7fa07413b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 -- 172.21.15.86:0/1992639335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa074136b90 con 0x7fa0740b37c0 2024-03-22T04:11:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa06b400640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0740b37c0 0x7fa0740aea10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00c370 tx=0x7fa05c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa070c00640 1 -- 172.21.15.86:0/1992639335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0052d0 con 0x7fa0740b37c0 2024-03-22T04:11:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa070c00640 1 -- 172.21.15.86:0/1992639335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c005470 con 0x7fa0740b37c0 2024-03-22T04:11:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa074136e70 con 0x7fa0740b37c0 2024-03-22T04:11:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa070c00640 1 -- 172.21.15.86:0/1992639335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c007260 con 0x7fa0740b37c0 2024-03-22T04:11:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa074137430 con 0x7fa0740b37c0 2024-03-22T04:11:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.467+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa074074580 con 0x7fa0740b37c0 2024-03-22T04:11:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.471+0000 7fa070c00640 1 -- 172.21.15.86:0/1992639335 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa05c007400 con 0x7fa0740b37c0 2024-03-22T04:11:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.471+0000 7fa070c00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa050067b20 0x7fa050069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.471+0000 7fa070c00640 1 -- 172.21.15.86:0/1992639335 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fa05c0851a0 con 0x7fa0740b37c0 2024-03-22T04:11:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.471+0000 7fa073e00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa050067b20 0x7fa050069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.471+0000 7fa073e00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa050067b20 0x7fa050069fe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0640b1000 tx=0x7fa064008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:49.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.475+0000 7fa070c00640 1 -- 172.21.15.86:0/1992639335 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa05c055680 con 0x7fa0740b37c0 2024-03-22T04:11:49.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:49.631+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa074140520 con 0x7fa0740b37c0 2024-03-22T04:11:50.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.603+0000 7fa070c00640 1 -- 172.21.15.86:0/1992639335 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) ==== 138+0+0 (secure 0 0 0) 0x7fa05c059720 con 0x7fa0740b37c0 2024-03-22T04:11:50.610 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa050067b20 msgr2=0x7fa050069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa050067b20 0x7fa050069fe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0640b1000 tx=0x7fa064008040 comp rx=0 tx=0).stop 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0740b37c0 msgr2=0x7fa0740aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0740b37c0 0x7fa0740aea10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00c370 tx=0x7fa05c00c830 comp rx=0 tx=0).stop 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 shutdown_connections 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa050067b20 0x7fa050069fe0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa074144a90 0x7fa07413b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0740b37c0 0x7fa0740aea10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 --2- 172.21.15.86:0/1992639335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa074072cf0 0x7fa07415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 >> 172.21.15.86:0/1992639335 conn(0x7fa07406b660 msgr2=0x7fa0740b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:50.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 shutdown_connections 2024-03-22T04:11:50.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.607+0000 7fa07aa00640 1 -- 172.21.15.86:0/1992639335 wait complete. 2024-03-22T04:11:50.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T04:11:50.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f588e800640 1 Processor -- start 2024-03-22T04:11:50.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f588e800640 1 -- start start 2024-03-22T04:11:50.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f588e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58880726e0 0x7f588806fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f588e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 0x7f5888072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f588e800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5888073000 con 0x7f588806b660 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f588e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5888070d00 con 0x7f58880726e0 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f588e800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58880d0f10 con 0x7f58880705b0 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f5887400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58880726e0 0x7f588806fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f5887400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58880726e0 0x7f588806fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53626/0 (socket says 172.21.15.86:53626) 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f5887400640 1 -- 172.21.15.86:0/1113328793 learned_addr learned my addr 172.21.15.86:0/1113328793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f5886a00640 1 --2- 172.21.15.86:0/1113328793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 0x7f5888072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f5887400640 1 -- 172.21.15.86:0/1113328793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 msgr2=0x7f5888072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f5887400640 1 --2- 172.21.15.86:0/1113328793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 0x7f5888072ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f5887400640 1 -- 172.21.15.86:0/1113328793 >> v1:172.21.15.196:6789/0 conn(0x7f588806b660 legacy=0x7f588806bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.775+0000 7f5887400640 1 -- 172.21.15.86:0/1113328793 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58880d0150 con 0x7f58880726e0 2024-03-22T04:11:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f5887400640 1 --2- 172.21.15.86:0/1113328793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58880726e0 0x7f588806fba0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5878009e50 tx=0x7f587801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4dffe8c48300a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f5886000640 1 -- 172.21.15.86:0/1113328793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780a5700 con 0x7f58880726e0 2024-03-22T04:11:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f5886000640 1 -- 172.21.15.86:0/1113328793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58780a58a0 con 0x7f58880726e0 2024-03-22T04:11:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f5886000640 1 -- 172.21.15.86:0/1113328793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780a5c70 con 0x7f58880726e0 2024-03-22T04:11:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 -- 172.21.15.86:0/1113328793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58880726e0 msgr2=0x7f588806fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 --2- 172.21.15.86:0/1113328793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58880726e0 0x7f588806fba0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5878009e50 tx=0x7f587801ca30 comp rx=0 tx=0).stop 2024-03-22T04:11:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 -- 172.21.15.86:0/1113328793 shutdown_connections 2024-03-22T04:11:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 --2- 172.21.15.86:0/1113328793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 0x7f5888072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 --2- 172.21.15.86:0/1113328793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58880726e0 0x7f588806fba0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 -- 172.21.15.86:0/1113328793 >> 172.21.15.86:0/1113328793 conn(0x7f58880b2e50 msgr2=0x7f58880b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 -- 172.21.15.86:0/1113328793 shutdown_connections 2024-03-22T04:11:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 -- 172.21.15.86:0/1113328793 wait complete. 2024-03-22T04:11:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 Processor -- start 2024-03-22T04:11:50.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.779+0000 7f588e800640 1 -- start start 2024-03-22T04:11:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f588806b660 0x7f5888161460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 0x7f58880aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58880726e0 0x7f58880b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5888070990 con 0x7f588806b660 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58880cfe00 con 0x7f58880726e0 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f588806f670 con 0x7f58880705b0 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f588806b660 0x7f5888161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f588806b660 0x7f5888161460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53638/0 (socket says 172.21.15.86:53638) 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 -- 172.21.15.86:0/3737002631 learned_addr learned my addr 172.21.15.86:0/3737002631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5886a00640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 0x7f58880aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887e00640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58880726e0 0x7f58880b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 -- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 msgr2=0x7f58880aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 0x7f58880aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 -- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58880726e0 msgr2=0x7f58880b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58880726e0 0x7f58880b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:50.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 -- 172.21.15.86:0/3737002631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58880ada80 con 0x7f588806b660 2024-03-22T04:11:50.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887e00640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58880726e0 0x7f58880b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:50.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f5887400640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f588806b660 0x7f5888161460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f58780af040 tx=0x7f58780197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:50.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f587f400640 1 -- 172.21.15.86:0/3737002631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780b8670 con 0x7f588806b660 2024-03-22T04:11:50.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f587f400640 1 -- 172.21.15.86:0/3737002631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58780be070 con 0x7f588806b660 2024-03-22T04:11:50.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f587f400640 1 -- 172.21.15.86:0/3737002631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780b8a50 con 0x7f588806b660 2024-03-22T04:11:50.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58880add60 con 0x7f588806b660 2024-03-22T04:11:50.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58880ae320 con 0x7f588806b660 2024-03-22T04:11:50.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5888074470 con 0x7f588806b660 2024-03-22T04:11:50.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.783+0000 7f587f400640 1 -- 172.21.15.86:0/3737002631 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f58780b8bf0 con 0x7f588806b660 2024-03-22T04:11:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.787+0000 7f587f400640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5868067b20 0x7f5868069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.787+0000 7f587f400640 1 -- 172.21.15.86:0/3737002631 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5619+0+0 (secure 0 0 0) 0x7f58781211c0 con 0x7f588806b660 2024-03-22T04:11:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.787+0000 7f5886a00640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5868067b20 0x7f5868069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.787+0000 7f5886a00640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5868067b20 0x7f5868069fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f58880b0880 tx=0x7f5874008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.791+0000 7f587f400640 1 -- 172.21.15.86:0/3737002631 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5878016620 con 0x7f588806b660 2024-03-22T04:11:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:50.946+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5888148230 con 0x7f588806b660 2024-03-22T04:11:51.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.606+0000 7f587f400640 1 -- 172.21.15.86:0/3737002631 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) ==== 170+0+0 (secure 0 0 0) 0x7f58780f1ca0 con 0x7f588806b660 2024-03-22T04:11:51.614 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5868067b20 msgr2=0x7f5868069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5868067b20 0x7f5868069fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f58880b0880 tx=0x7f5874008040 comp rx=0 tx=0).stop 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f588806b660 msgr2=0x7f5888161460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f588806b660 0x7f5888161460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f58780af040 tx=0x7f58780197d0 comp rx=0 tx=0).stop 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 shutdown_connections 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5868067b20 0x7f5868069fe0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58880726e0 0x7f58880b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58880705b0 0x7f58880aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 --2- 172.21.15.86:0/3737002631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f588806b660 0x7f5888161460 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 >> 172.21.15.86:0/3737002631 conn(0x7f58880b2e50 msgr2=0x7f58880b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 shutdown_connections 2024-03-22T04:11:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.610+0000 7f588e800640 1 -- 172.21.15.86:0/3737002631 wait complete. 2024-03-22T04:11:51.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T04:11:51.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.782+0000 7f1c51800640 1 Processor -- start 2024-03-22T04:11:51.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.782+0000 7f1c51800640 1 -- start start 2024-03-22T04:11:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c51800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1c4c162740 0x7f1c4c169380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c51800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c4c0cc7c0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c51800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c4c0cb720 con 0x7f1c4c0cb1d0 2024-03-22T04:11:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c51800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1c4c0cba00 con 0x7f1c4c162740 2024-03-22T04:11:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c4aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1c4c0c6f20 0x7f1c4c072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48722/0 (socket says 172.21.15.86:48722) 2024-03-22T04:11:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c4aa00640 1 -- 172.21.15.86:0/539881039 learned_addr learned my addr 172.21.15.86:0/539881039 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c4b400640 1 --2- 172.21.15.86:0/539881039 >> v2:172.21.15.196:3300/0 conn(0x7f1c4c162740 0x7f1c4c169380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c49600640 1 -- 172.21.15.86:0/539881039 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 799751704 0 0) 0x7f1c4c0cc7c0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c49600640 1 -- 172.21.15.86:0/539881039 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c30003530 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c49600640 1 -- 172.21.15.86:0/539881039 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 565930328 0 0) 0x7f1c4c0cb720 con 0x7f1c4c0cb1d0 2024-03-22T04:11:51.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c49600640 1 -- 172.21.15.86:0/539881039 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c4c0cc7c0 con 0x7f1c4c0cb1d0 2024-03-22T04:11:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c4b400640 1 -- 172.21.15.86:0/539881039 >> v1:172.21.15.196:6789/0 conn(0x7f1c4c0cb1d0 legacy=0x7f1c4c162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c4b400640 1 -- 172.21.15.86:0/539881039 >> v1:172.21.15.86:6789/0 conn(0x7f1c4c0c6f20 legacy=0x7f1c4c072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c49600640 1 -- 172.21.15.86:0/539881039 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 956190739 0 0) 0x7f1c30003530 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.786+0000 7f1c4b400640 1 -- 172.21.15.86:0/539881039 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c0cbce0 con 0x7f1c4c162740 2024-03-22T04:11:51.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c4b400640 1 --2- 172.21.15.86:0/539881039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c162740 0x7f1c4c169380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009d90 tx=0x7f1c4401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce9cc6e7f3202c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c49600640 1 -- 172.21.15.86:0/539881039 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440b2070 con 0x7f1c4c162740 2024-03-22T04:11:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c49600640 1 -- 172.21.15.86:0/539881039 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c440a5920 con 0x7f1c4c162740 2024-03-22T04:11:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c49600640 1 -- 172.21.15.86:0/539881039 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440a5cf0 con 0x7f1c4c162740 2024-03-22T04:11:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 -- 172.21.15.86:0/539881039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c162740 msgr2=0x7f1c4c169380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 --2- 172.21.15.86:0/539881039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c162740 0x7f1c4c169380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009d90 tx=0x7f1c4401c970 comp rx=0 tx=0).stop 2024-03-22T04:11:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 -- 172.21.15.86:0/539881039 shutdown_connections 2024-03-22T04:11:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 --2- 172.21.15.86:0/539881039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c162740 0x7f1c4c169380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 -- 172.21.15.86:0/539881039 >> 172.21.15.86:0/539881039 conn(0x7f1c4c0d0730 msgr2=0x7f1c4c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 -- 172.21.15.86:0/539881039 shutdown_connections 2024-03-22T04:11:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 -- 172.21.15.86:0/539881039 wait complete. 2024-03-22T04:11:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 Processor -- start 2024-03-22T04:11:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.790+0000 7f1c51800640 1 -- start start 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c4c0c6f20 0x7f1c4c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c0cb1d0 0x7f1c4c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c4c162740 0x7f1c4c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c16c670 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c16c4f0 con 0x7f1c4c0cb1d0 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1c4c16c7f0 con 0x7f1c4c162740 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c4c0c6f20 0x7f1c4c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c4c0c6f20 0x7f1c4c06fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53650/0 (socket says 172.21.15.86:53650) 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 -- 172.21.15.86:0/3431067088 learned_addr learned my addr 172.21.15.86:0/3431067088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4a000640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c0cb1d0 0x7f1c4c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4b400640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c4c162740 0x7f1c4c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 -- 172.21.15.86:0/3431067088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c4c162740 msgr2=0x7f1c4c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c4c162740 0x7f1c4c15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 -- 172.21.15.86:0/3431067088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c0cb1d0 msgr2=0x7f1c4c15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c0cb1d0 0x7f1c4c15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 -- 172.21.15.86:0/3431067088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c15dcd0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c4aa00640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c4c0c6f20 0x7f1c4c06fce0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1c340011f0 tx=0x7f1c3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c42a00640 1 -- 172.21.15.86:0/3431067088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34017070 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c42a00640 1 -- 172.21.15.86:0/3431067088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34005cf0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c42a00640 1 -- 172.21.15.86:0/3431067088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c340072f0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c15dfb0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c143ed0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4c1440d0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.794+0000 7f1c42a00640 1 -- 172.21.15.86:0/3431067088 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1c3401a070 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.798+0000 7f1c42a00640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c28067b20 0x7f1c28069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:51.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.798+0000 7f1c42a00640 1 -- 172.21.15.86:0/3431067088 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6001+0+0 (secure 0 0 0) 0x7f1c34085950 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.798+0000 7f1c4a000640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c28067b20 0x7f1c28069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:51.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.798+0000 7f1c4a000640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c28067b20 0x7f1c28069fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0b66d0 tx=0x7f1c3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.802+0000 7f1c42a00640 1 -- 172.21.15.86:0/3431067088 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1c4c1440d0 con 0x7f1c4c0c6f20 2024-03-22T04:11:51.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:51.990+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1c4c074470 con 0x7f1c4c0c6f20 2024-03-22T04:11:52.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.642+0000 7f1c42a00640 1 -- 172.21.15.86:0/3431067088 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f1c34021020 con 0x7f1c4c0c6f20 2024-03-22T04:11:52.650 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 5 and data pool 6 2024-03-22T04:11:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.646+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c28067b20 msgr2=0x7f1c28069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.646+0000 7f1c51800640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c28067b20 0x7f1c28069fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0b66d0 tx=0x7f1c3c009040 comp rx=0 tx=0).stop 2024-03-22T04:11:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.646+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c4c0c6f20 msgr2=0x7f1c4c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.646+0000 7f1c51800640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c4c0c6f20 0x7f1c4c06fce0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1c340011f0 tx=0x7f1c3400c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.650+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 shutdown_connections 2024-03-22T04:11:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.650+0000 7f1c51800640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c28067b20 0x7f1c28069fe0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.650+0000 7f1c51800640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c4c0cb1d0 0x7f1c4c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.650+0000 7f1c51800640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1c4c162740 0x7f1c4c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.650+0000 7f1c51800640 1 --2- 172.21.15.86:0/3431067088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c4c0c6f20 0x7f1c4c06fce0 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1c340011f0 tx=0x7f1c3400c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.650+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 >> 172.21.15.86:0/3431067088 conn(0x7f1c4c0d0730 msgr2=0x7f1c4c0b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.650+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 shutdown_connections 2024-03-22T04:11:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.650+0000 7f1c51800640 1 -- 172.21.15.86:0/3431067088 wait complete. 2024-03-22T04:11:52.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.818+0000 7f146f800640 1 Processor -- start 2024-03-22T04:11:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.818+0000 7f146f800640 1 -- start start 2024-03-22T04:11:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146f800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f14680d0580 0x7f1468071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146806e2f0 0x7f1468071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146f800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14681623d0 con 0x7f146815e630 2024-03-22T04:11:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146f800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f14681620f0 con 0x7f14680d0580 2024-03-22T04:11:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f146815ec10 con 0x7f146806e2f0 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146d800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146806e2f0 0x7f1468071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f1467e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f14680d0580 0x7f1468071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146d800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146806e2f0 0x7f1468071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42460/0 (socket says 172.21.15.86:42460) 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146d800640 1 -- 172.21.15.86:0/3111598008 learned_addr learned my addr 172.21.15.86:0/3111598008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f1467e00640 1 -- 172.21.15.86:0/3111598008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146806e2f0 msgr2=0x7f1468071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f1467e00640 1 --2- 172.21.15.86:0/3111598008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146806e2f0 0x7f1468071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f1467e00640 1 -- 172.21.15.86:0/3111598008 >> v1:172.21.15.196:6789/0 conn(0x7f146815e630 legacy=0x7f14680cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f1467e00640 1 -- 172.21.15.86:0/3111598008 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146815eef0 con 0x7f14680d0580 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f1467400640 1 -- 172.21.15.86:0/3111598008 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 211826420 0 0) 0x7f14681623d0 con 0x7f146815e630 2024-03-22T04:11:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.822+0000 7f146d800640 1 --2- 172.21.15.86:0/3111598008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146806e2f0 0x7f1468071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f1467e00640 1 --2- 172.21.15.86:0/3111598008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14680d0580 0x7f1468071100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14540097c0 tx=0x7f145401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4284dce53772d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f1467400640 1 -- 172.21.15.86:0/3111598008 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14540b0070 con 0x7f14680d0580 2024-03-22T04:11:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f1467400640 1 -- 172.21.15.86:0/3111598008 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14540ab600 con 0x7f14680d0580 2024-03-22T04:11:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f1467400640 1 -- 172.21.15.86:0/3111598008 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14540ab9d0 con 0x7f14680d0580 2024-03-22T04:11:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 -- 172.21.15.86:0/3111598008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14680d0580 msgr2=0x7f1468071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 --2- 172.21.15.86:0/3111598008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14680d0580 0x7f1468071100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14540097c0 tx=0x7f145401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 -- 172.21.15.86:0/3111598008 shutdown_connections 2024-03-22T04:11:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 --2- 172.21.15.86:0/3111598008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146806e2f0 0x7f1468071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 --2- 172.21.15.86:0/3111598008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14680d0580 0x7f1468071100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 -- 172.21.15.86:0/3111598008 >> 172.21.15.86:0/3111598008 conn(0x7f146806f910 msgr2=0x7f146806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 -- 172.21.15.86:0/3111598008 shutdown_connections 2024-03-22T04:11:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 -- 172.21.15.86:0/3111598008 wait complete. 2024-03-22T04:11:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 Processor -- start 2024-03-22T04:11:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.826+0000 7f146f800640 1 -- start start 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146806e2f0 0x7f14680c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14680d0580 0x7f14680b7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146815e630 0x7f14680c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1468071f90 con 0x7f146806e2f0 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f146815f9d0 con 0x7f14680d0580 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14680cfaf0 con 0x7f146815e630 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146806e2f0 0x7f14680c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146806e2f0 0x7f14680c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53654/0 (socket says 172.21.15.86:53654) 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 -- 172.21.15.86:0/2382520616 learned_addr learned my addr 172.21.15.86:0/2382520616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 -- 172.21.15.86:0/2382520616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146815e630 msgr2=0x7f14680c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146815e630 0x7f14680c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 -- 172.21.15.86:0/2382520616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14680d0580 msgr2=0x7f14680b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14680d0580 0x7f14680b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 -- 172.21.15.86:0/2382520616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14680b5af0 con 0x7f146806e2f0 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146ce00640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146806e2f0 0x7f14680c87c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f145c00c500 tx=0x7f145c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f1464c00640 1 -- 172.21.15.86:0/2382520616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c017070 con 0x7f146806e2f0 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f1464c00640 1 -- 172.21.15.86:0/2382520616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f145c003600 con 0x7f146806e2f0 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14680b5dd0 con 0x7f146806e2f0 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f1464c00640 1 -- 172.21.15.86:0/2382520616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c0054a0 con 0x7f146806e2f0 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14680b6390 con 0x7f146806e2f0 2024-03-22T04:11:52.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1468068650 con 0x7f146806e2f0 2024-03-22T04:11:52.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.830+0000 7f1464c00640 1 -- 172.21.15.86:0/2382520616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f145c01a070 con 0x7f146806e2f0 2024-03-22T04:11:52.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.834+0000 7f1464c00640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1448067b50 0x7f144806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:52.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.834+0000 7f1464c00640 1 -- 172.21.15.86:0/2382520616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f145c085450 con 0x7f146806e2f0 2024-03-22T04:11:52.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.834+0000 7f1467e00640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1448067b50 0x7f144806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:52.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.834+0000 7f1467e00640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1448067b50 0x7f144806a010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f14540ad000 tx=0x7f1454004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:52.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.838+0000 7f1464c00640 1 -- 172.21.15.86:0/2382520616 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f145c055d80 con 0x7f146806e2f0 2024-03-22T04:11:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14680cdf20 con 0x7f146806e2f0 2024-03-22T04:11:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f1464c00640 1 -- 172.21.15.86:0/2382520616 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16955 (secure 0 0 0) 0x7f145c055680 con 0x7f146806e2f0 2024-03-22T04:11:53.000 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:53.000 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":40,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:51.996084+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:11:49.644732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-22T04:11:50.961986+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:53.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1448067b50 msgr2=0x7f144806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1448067b50 0x7f144806a010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f14540ad000 tx=0x7f1454004740 comp rx=0 tx=0).stop 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146806e2f0 msgr2=0x7f14680c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146806e2f0 0x7f14680c87c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f145c00c500 tx=0x7f145c00c9c0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 shutdown_connections 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1448067b50 0x7f144806a010 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14680d0580 0x7f14680b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146815e630 0x7f14680c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 --2- 172.21.15.86:0/2382520616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f146806e2f0 0x7f14680c87c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 >> 172.21.15.86:0/2382520616 conn(0x7f146806f910 msgr2=0x7f14680b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 shutdown_connections 2024-03-22T04:11:53.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:52.994+0000 7f146f800640 1 -- 172.21.15.86:0/2382520616 wait complete. 2024-03-22T04:11:53.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.150+0000 7f37b1600640 1 Processor -- start 2024-03-22T04:11:53.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.150+0000 7f37b1600640 1 -- start start 2024-03-22T04:11:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.150+0000 7f37b1600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac0ad330 0x7f37ac15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.150+0000 7f37b1600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f37ac06ef40 0x7f37ac06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.150+0000 7f37b1600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ac15c7c0 con 0x7f37ac0ad710 2024-03-22T04:11:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.150+0000 7f37b1600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f37ac06dc40 con 0x7f37ac06ef40 2024-03-22T04:11:53.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.150+0000 7f37b1600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac06dea0 con 0x7f37ac0ad330 2024-03-22T04:11:53.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aaa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac0ad330 0x7f37ac15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aaa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac0ad330 0x7f37ac15d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42468/0 (socket says 172.21.15.86:42468) 2024-03-22T04:11:53.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aaa00640 1 -- 172.21.15.86:0/3799483033 learned_addr learned my addr 172.21.15.86:0/3799483033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:53.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37a2000640 1 --2- 172.21.15.86:0/3799483033 >> v2:172.21.15.196:3300/0 conn(0x7f37ac06ef40 0x7f37ac06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aaa00640 1 -- 172.21.15.86:0/3799483033 >> v2:172.21.15.196:3300/0 conn(0x7f37ac06ef40 msgr2=0x7f37ac06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:53.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aaa00640 1 --2- 172.21.15.86:0/3799483033 >> v2:172.21.15.196:3300/0 conn(0x7f37ac06ef40 0x7f37ac06d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aaa00640 1 -- 172.21.15.86:0/3799483033 >> v1:172.21.15.196:6789/0 conn(0x7f37ac0ad710 legacy=0x7f37ac13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aa000640 1 -- 172.21.15.86:0/3799483033 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 269686696 0 0) 0x7f37ac15c7c0 con 0x7f37ac0ad710 2024-03-22T04:11:53.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aaa00640 1 -- 172.21.15.86:0/3799483033 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac06e180 con 0x7f37ac0ad330 2024-03-22T04:11:53.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.154+0000 7f37aaa00640 1 --2- 172.21.15.86:0/3799483033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac0ad330 0x7f37ac15d4a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f379c009870 tx=0x7f379c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3616dab954b2ada server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:53.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37aa000640 1 -- 172.21.15.86:0/3799483033 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c0ab640 con 0x7f37ac0ad330 2024-03-22T04:11:53.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37aa000640 1 -- 172.21.15.86:0/3799483033 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f379c0b0070 con 0x7f37ac0ad330 2024-03-22T04:11:53.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37aa000640 1 -- 172.21.15.86:0/3799483033 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c0aba00 con 0x7f37ac0ad330 2024-03-22T04:11:53.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- 172.21.15.86:0/3799483033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac0ad330 msgr2=0x7f37ac15d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 --2- 172.21.15.86:0/3799483033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac0ad330 0x7f37ac15d4a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f379c009870 tx=0x7f379c01c530 comp rx=0 tx=0).stop 2024-03-22T04:11:53.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- 172.21.15.86:0/3799483033 shutdown_connections 2024-03-22T04:11:53.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 --2- 172.21.15.86:0/3799483033 >> v2:172.21.15.196:3300/0 conn(0x7f37ac06ef40 0x7f37ac06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 --2- 172.21.15.86:0/3799483033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac0ad330 0x7f37ac15d4a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- 172.21.15.86:0/3799483033 >> 172.21.15.86:0/3799483033 conn(0x7f37ac0b1df0 msgr2=0x7f37ac0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:53.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- 172.21.15.86:0/3799483033 shutdown_connections 2024-03-22T04:11:53.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- 172.21.15.86:0/3799483033 wait complete. 2024-03-22T04:11:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 Processor -- start 2024-03-22T04:11:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- start start 2024-03-22T04:11:53.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac06ef40 0x7f37ac0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ac0ad330 0x7f37ac0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ac0ad710 0x7f37ac14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1690e0 con 0x7f37ac0ad330 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac168f60 con 0x7f37ac0ad710 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37b1600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac169260 con 0x7f37ac06ef40 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37a2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ac0ad330 0x7f37ac0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37a2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ac0ad330 0x7f37ac0d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53668/0 (socket says 172.21.15.86:53668) 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.158+0000 7f37a2000640 1 -- 172.21.15.86:0/2651802812 learned_addr learned my addr 172.21.15.86:0/2651802812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37ab400640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ac0ad710 0x7f37ac14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37aaa00640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac06ef40 0x7f37ac0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a2000640 1 -- 172.21.15.86:0/2651802812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac06ef40 msgr2=0x7f37ac0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a2000640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac06ef40 0x7f37ac0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a2000640 1 -- 172.21.15.86:0/2651802812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ac0ad710 msgr2=0x7f37ac14aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a2000640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ac0ad710 0x7f37ac14aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a2000640 1 -- 172.21.15.86:0/2651802812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac137a30 con 0x7f37ac0ad330 2024-03-22T04:11:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a2000640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ac0ad330 0x7f37ac0d06a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f379400e870 tx=0x7f379400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a3400640 1 -- 172.21.15.86:0/2651802812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794017070 con 0x7f37ac0ad330 2024-03-22T04:11:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a3400640 1 -- 172.21.15.86:0/2651802812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f379400c940 con 0x7f37ac0ad330 2024-03-22T04:11:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac138290 con 0x7f37ac0ad330 2024-03-22T04:11:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a3400640 1 -- 172.21.15.86:0/2651802812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379400ccf0 con 0x7f37ac0ad330 2024-03-22T04:11:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac137fb0 con 0x7f37ac0ad330 2024-03-22T04:11:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac068650 con 0x7f37ac0ad330 2024-03-22T04:11:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a3400640 1 -- 172.21.15.86:0/2651802812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f379401a070 con 0x7f37ac0ad330 2024-03-22T04:11:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.162+0000 7f37a3400640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3788067b20 0x7f3788069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.166+0000 7f37a3400640 1 -- 172.21.15.86:0/2651802812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f3794085f90 con 0x7f37ac0ad330 2024-03-22T04:11:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.166+0000 7f37aaa00640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3788067b20 0x7f3788069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.166+0000 7f37aaa00640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3788067b20 0x7f3788069fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f379c0af000 tx=0x7f379c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.170+0000 7f37a3400640 1 -- 172.21.15.86:0/2651802812 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3794056120 con 0x7f37ac0ad330 2024-03-22T04:11:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37ac072b40 con 0x7f37ac0ad330 2024-03-22T04:11:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37a3400640 1 -- 172.21.15.86:0/2651802812 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16955 (secure 0 0 0) 0x7f379405a1c0 con 0x7f37ac0ad330 2024-03-22T04:11:53.332 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:53.332 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":40,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:51.996084+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:11:49.644732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-22T04:11:50.961986+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3788067b20 msgr2=0x7f3788069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3788067b20 0x7f3788069fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f379c0af000 tx=0x7f379c0a7000 comp rx=0 tx=0).stop 2024-03-22T04:11:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ac0ad330 msgr2=0x7f37ac0d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ac0ad330 0x7f37ac0d06a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f379400e870 tx=0x7f379400ec70 comp rx=0 tx=0).stop 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 shutdown_connections 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3788067b20 0x7f3788069fe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37ac0ad710 0x7f37ac14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ac0ad330 0x7f37ac0d06a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 --2- 172.21.15.86:0/2651802812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37ac06ef40 0x7f37ac0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 >> 172.21.15.86:0/2651802812 conn(0x7f37ac0b1df0 msgr2=0x7f37ac0716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.326+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 shutdown_connections 2024-03-22T04:11:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.330+0000 7f37b1600640 1 -- 172.21.15.86:0/2651802812 wait complete. 2024-03-22T04:11:53.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T04:11:53.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.486+0000 7fa64de00640 1 Processor -- start 2024-03-22T04:11:53.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.486+0000 7fa64de00640 1 -- start start 2024-03-22T04:11:53.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.486+0000 7fa64de00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa6480b2360 0x7fa6480b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.486+0000 7fa64de00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.486+0000 7fa64de00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64815d710 con 0x7fa64815cee0 2024-03-22T04:11:53.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.486+0000 7fa64de00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa6480b2740 con 0x7fa6480b2360 2024-03-22T04:11:53.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.486+0000 7fa64de00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa64815fe40 con 0x7fa6480b3430 2024-03-22T04:11:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa6480b2360 0x7fa6480b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42478/0 (socket says 172.21.15.86:42478) 2024-03-22T04:11:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646000640 1 -- 172.21.15.86:0/4264536039 learned_addr learned my addr 172.21.15.86:0/4264536039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646000640 1 -- 172.21.15.86:0/4264536039 >> v2:172.21.15.196:3300/0 conn(0x7fa6480b2360 msgr2=0x7fa6480b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646000640 1 --2- 172.21.15.86:0/4264536039 >> v2:172.21.15.196:3300/0 conn(0x7fa6480b2360 0x7fa6480b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646000640 1 -- 172.21.15.86:0/4264536039 >> v1:172.21.15.196:6789/0 conn(0x7fa64815cee0 legacy=0x7fa64806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646000640 1 -- 172.21.15.86:0/4264536039 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa648160120 con 0x7fa6480b3430 2024-03-22T04:11:53.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.490+0000 7fa646000640 1 --2- 172.21.15.86:0/4264536039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815c9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa634009870 tx=0x7fa63401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e736ef07f235ef9e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:53.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa645600640 1 -- 172.21.15.86:0/4264536039 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340b0070 con 0x7fa6480b3430 2024-03-22T04:11:53.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa645600640 1 -- 172.21.15.86:0/4264536039 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6340ab600 con 0x7fa6480b3430 2024-03-22T04:11:53.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa645600640 1 -- 172.21.15.86:0/4264536039 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340ab9d0 con 0x7fa6480b3430 2024-03-22T04:11:53.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- 172.21.15.86:0/4264536039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 msgr2=0x7fa64815c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 --2- 172.21.15.86:0/4264536039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815c9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa634009870 tx=0x7fa63401c560 comp rx=0 tx=0).stop 2024-03-22T04:11:53.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- 172.21.15.86:0/4264536039 shutdown_connections 2024-03-22T04:11:53.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 --2- 172.21.15.86:0/4264536039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815c9a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 --2- 172.21.15.86:0/4264536039 >> v2:172.21.15.196:3300/0 conn(0x7fa6480b2360 0x7fa6480b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- 172.21.15.86:0/4264536039 >> 172.21.15.86:0/4264536039 conn(0x7fa6480cc940 msgr2=0x7fa6480ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:53.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- 172.21.15.86:0/4264536039 shutdown_connections 2024-03-22T04:11:53.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- 172.21.15.86:0/4264536039 wait complete. 2024-03-22T04:11:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 Processor -- start 2024-03-22T04:11:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- start start 2024-03-22T04:11:53.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6480b2360 0x7fa648147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa64815cee0 0x7fa64814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa648160c00 con 0x7fa6480b2360 2024-03-22T04:11:53.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa64815d2f0 con 0x7fa64815cee0 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa64de00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa64815da90 con 0x7fa6480b3430 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa646a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6480b2360 0x7fa648147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa646a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6480b2360 0x7fa648147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53670/0 (socket says 172.21.15.86:53670) 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.494+0000 7fa646a00640 1 -- 172.21.15.86:0/3904960518 learned_addr learned my addr 172.21.15.86:0/3904960518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa647400640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa64815cee0 0x7fa64814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa646000640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa646a00640 1 -- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 msgr2=0x7fa64815ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa646a00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa646a00640 1 -- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa64815cee0 msgr2=0x7fa64814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa646a00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa64815cee0 0x7fa64814fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa646a00640 1 -- 172.21.15.86:0/3904960518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64815dd70 con 0x7fa6480b2360 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa647400640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa64815cee0 0x7fa64814fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa646000640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:11:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa646a00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6480b2360 0x7fa648147cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa63800efc0 tx=0x7fa63800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:53.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa63f400640 1 -- 172.21.15.86:0/3904960518 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638018020 con 0x7fa6480b2360 2024-03-22T04:11:53.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa63f400640 1 -- 172.21.15.86:0/3904960518 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa638014480 con 0x7fa6480b2360 2024-03-22T04:11:53.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64815e050 con 0x7fa6480b2360 2024-03-22T04:11:53.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa63f400640 1 -- 172.21.15.86:0/3904960518 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638014850 con 0x7fa6480b2360 2024-03-22T04:11:53.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64815e610 con 0x7fa6480b2360 2024-03-22T04:11:53.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa648074470 con 0x7fa6480b2360 2024-03-22T04:11:53.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa63f400640 1 -- 172.21.15.86:0/3904960518 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa63801b020 con 0x7fa6480b2360 2024-03-22T04:11:53.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa63f400640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa628067b20 0x7fa628069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:53.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.498+0000 7fa63f400640 1 -- 172.21.15.86:0/3904960518 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa638086bc0 con 0x7fa6480b2360 2024-03-22T04:11:53.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.502+0000 7fa646000640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa628067b20 0x7fa628069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.502+0000 7fa646000640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa628067b20 0x7fa628069fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa6340ad000 tx=0x7fa634008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:53.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.502+0000 7fa63f400640 1 -- 172.21.15.86:0/3904960518 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa638022030 con 0x7fa6480b2360 2024-03-22T04:11:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:53.698+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa64806e9f0 con 0x7fa6480b2360 2024-03-22T04:11:54.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.650+0000 7fa63f400640 1 -- 172.21.15.86:0/3904960518 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fa638010040 con 0x7fa6480b2360 2024-03-22T04:11:54.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa628067b20 msgr2=0x7fa628069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:54.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa628067b20 0x7fa628069fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa6340ad000 tx=0x7fa634008040 comp rx=0 tx=0).stop 2024-03-22T04:11:54.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6480b2360 msgr2=0x7fa648147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:54.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6480b2360 0x7fa648147cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa63800efc0 tx=0x7fa63800c490 comp rx=0 tx=0).stop 2024-03-22T04:11:54.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 shutdown_connections 2024-03-22T04:11:54.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa628067b20 0x7fa628069fe0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa64815cee0 0x7fa64814fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6480b3430 0x7fa64815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 --2- 172.21.15.86:0/3904960518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6480b2360 0x7fa648147cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 >> 172.21.15.86:0/3904960518 conn(0x7fa6480cc940 msgr2=0x7fa6480b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:54.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 shutdown_connections 2024-03-22T04:11:54.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.654+0000 7fa64de00640 1 -- 172.21.15.86:0/3904960518 wait complete. 2024-03-22T04:11:54.681 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-22T04:11:54.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-22T04:11:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.818+0000 7fbb26400640 1 Processor -- start 2024-03-22T04:11:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.818+0000 7fbb26400640 1 -- start start 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.818+0000 7fbb26400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbb2006e7e0 0x7fbb2006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.818+0000 7fbb26400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb20072650 0x7fbb20072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.818+0000 7fbb26400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2015c770 con 0x7fbb200b1b80 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.818+0000 7fbb26400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbb20071350 con 0x7fbb2006e7e0 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.818+0000 7fbb26400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2006f100 con 0x7fbb20072650 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.818+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb20072650 0x7fbb20072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbb2006e7e0 0x7fbb2006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb20072650 0x7fbb20072ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42492/0 (socket says 172.21.15.86:42492) 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1ea00640 1 -- 172.21.15.86:0/4261319890 learned_addr learned my addr 172.21.15.86:0/4261319890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1fe00640 1 --1- 172.21.15.86:0/4261319890 >> v1:172.21.15.196:6789/0 conn(0x7fbb200b1b80 0x7fbb200b2010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:59648/0 (socket says 172.21.15.86:59648) 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1ea00640 1 -- 172.21.15.86:0/4261319890 >> v2:172.21.15.196:3300/0 conn(0x7fbb2006e7e0 msgr2=0x7fbb2006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/4261319890 >> v2:172.21.15.196:3300/0 conn(0x7fbb2006e7e0 0x7fbb2006ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1ea00640 1 -- 172.21.15.86:0/4261319890 >> v1:172.21.15.196:6789/0 conn(0x7fbb200b1b80 legacy=0x7fbb200b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:54.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1ea00640 1 -- 172.21.15.86:0/4261319890 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb20073150 con 0x7fbb20072650 2024-03-22T04:11:54.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1e000640 1 -- 172.21.15.86:0/4261319890 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924568186 0 0) 0x7fbb180014a0 con 0x7fbb200b1b80 2024-03-22T04:11:54.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.822+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/4261319890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb20072650 0x7fbb20072ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009870 tx=0x7fbb0c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b698c9f2f7904d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1e000640 1 -- 172.21.15.86:0/4261319890 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0aa600 con 0x7fbb20072650 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1e000640 1 -- 172.21.15.86:0/4261319890 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c0af070 con 0x7fbb20072650 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1e000640 1 -- 172.21.15.86:0/4261319890 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0aa9c0 con 0x7fbb20072650 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- 172.21.15.86:0/4261319890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb20072650 msgr2=0x7fbb20072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 --2- 172.21.15.86:0/4261319890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb20072650 0x7fbb20072ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009870 tx=0x7fbb0c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- 172.21.15.86:0/4261319890 shutdown_connections 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 --2- 172.21.15.86:0/4261319890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb20072650 0x7fbb20072ad0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 --2- 172.21.15.86:0/4261319890 >> v2:172.21.15.196:3300/0 conn(0x7fbb2006e7e0 0x7fbb2006ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- 172.21.15.86:0/4261319890 >> 172.21.15.86:0/4261319890 conn(0x7fbb200cc940 msgr2=0x7fbb200ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- 172.21.15.86:0/4261319890 shutdown_connections 2024-03-22T04:11:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- 172.21.15.86:0/4261319890 wait complete. 2024-03-22T04:11:54.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 Processor -- start 2024-03-22T04:11:54.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- start start 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2006e7e0 0x7fbb20152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb20072650 0x7fbb20163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb200b1b80 0x7fbb20164300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20072080 con 0x7fbb20072650 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb200ce930 con 0x7fbb2006e7e0 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb26400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbb20163270 con 0x7fbb200b1b80 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb20072650 0x7fbb20163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb20072650 0x7fbb20163dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53682/0 (socket says 172.21.15.86:53682) 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1ea00640 1 -- 172.21.15.86:0/2416936418 learned_addr learned my addr 172.21.15.86:0/2416936418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2006e7e0 0x7fbb20152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:54.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1fe00640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb200b1b80 0x7fbb20164300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1ea00640 1 -- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb200b1b80 msgr2=0x7fbb20164300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb200b1b80 0x7fbb20164300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1ea00640 1 -- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2006e7e0 msgr2=0x7fbb20152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.826+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2006e7e0 0x7fbb20152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb1ea00640 1 -- 172.21.15.86:0/2416936418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb20164840 con 0x7fbb20072650 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2006e7e0 0x7fbb20152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb20072650 0x7fbb20163dc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c01a0d0 tx=0x7fbb0c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb17400640 1 -- 172.21.15.86:0/2416936418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0b9070 con 0x7fbb20072650 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb17400640 1 -- 172.21.15.86:0/2416936418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c004220 con 0x7fbb20072650 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb20153390 con 0x7fbb20072650 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb17400640 1 -- 172.21.15.86:0/2416936418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0aa850 con 0x7fbb20072650 2024-03-22T04:11:54.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb20164d90 con 0x7fbb20072650 2024-03-22T04:11:54.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb20153560 con 0x7fbb20072650 2024-03-22T04:11:54.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb17400640 1 -- 172.21.15.86:0/2416936418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbb0c0a9070 con 0x7fbb20072650 2024-03-22T04:11:54.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb17400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbb00067b50 0x7fbb0006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:54.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb17400640 1 -- 172.21.15.86:0/2416936418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fbb0c11fb70 con 0x7fbb20072650 2024-03-22T04:11:54.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.830+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbb00067b50 0x7fbb0006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:54.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.834+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbb00067b50 0x7fbb0006a010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009eb0 tx=0x7fbb10009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:54.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:54.838+0000 7fbb17400640 1 -- 172.21.15.86:0/2416936418 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbb0c016150 con 0x7fbb20072650 2024-03-22T04:11:55.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.022+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbb20072200 con 0x7fbb20072650 2024-03-22T04:11:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.650+0000 7fbb17400640 1 -- 172.21.15.86:0/2416936418 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fbb0c0efd00 con 0x7fbb20072650 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbb00067b50 msgr2=0x7fbb0006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbb00067b50 0x7fbb0006a010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009eb0 tx=0x7fbb10009190 comp rx=0 tx=0).stop 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb20072650 msgr2=0x7fbb20163dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb20072650 0x7fbb20163dc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c01a0d0 tx=0x7fbb0c01ace0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 shutdown_connections 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbb00067b50 0x7fbb0006a010 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb200b1b80 0x7fbb20164300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb20072650 0x7fbb20163dc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 --2- 172.21.15.86:0/2416936418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2006e7e0 0x7fbb20152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 >> 172.21.15.86:0/2416936418 conn(0x7fbb200cc940 msgr2=0x7fbb200ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 shutdown_connections 2024-03-22T04:11:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.654+0000 7fbb26400640 1 -- 172.21.15.86:0/2416936418 wait complete. 2024-03-22T04:11:55.684 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-22T04:11:55.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:11:55.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c7e00640 1 Processor -- start 2024-03-22T04:11:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c7e00640 1 -- start start 2024-03-22T04:11:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23c00cc3e0 0x7f23c00ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c7e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c0071920 con 0x7f23c015b3f0 2024-03-22T04:11:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f23c00716d0 con 0x7f23c00cc3e0 2024-03-22T04:11:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c7e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f23c006d990 con 0x7f23c00730d0 2024-03-22T04:11:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23c00cc3e0 0x7f23c00ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23c00cc3e0 0x7f23c00ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53684/0 (socket says 172.21.15.86:53684) 2024-03-22T04:11:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c5e00640 1 -- 172.21.15.86:0/1167249832 learned_addr learned my addr 172.21.15.86:0/1167249832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c4a00640 1 --2- 172.21.15.86:0/1167249832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c5e00640 1 -- 172.21.15.86:0/1167249832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 msgr2=0x7f23c00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c5e00640 1 --2- 172.21.15.86:0/1167249832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c00ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c5e00640 1 -- 172.21.15.86:0/1167249832 >> v1:172.21.15.196:6789/0 conn(0x7f23c015b3f0 legacy=0x7f23c00729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23bfe00640 1 -- 172.21.15.86:0/1167249832 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176255868 0 0) 0x7f23c0071920 con 0x7f23c015b3f0 2024-03-22T04:11:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c5e00640 1 -- 172.21.15.86:0/1167249832 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c0071c00 con 0x7f23c00cc3e0 2024-03-22T04:11:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.818+0000 7f23c4a00640 1 --2- 172.21.15.86:0/1167249832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:11:55.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c5e00640 1 --2- 172.21.15.86:0/1167249832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c00cc3e0 0x7f23c00ae370 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009e50 tx=0x7f23b801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cb10db6453953d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:55.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23bfe00640 1 -- 172.21.15.86:0/1167249832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80a57f0 con 0x7f23c00cc3e0 2024-03-22T04:11:55.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23bfe00640 1 -- 172.21.15.86:0/1167249832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b80a5990 con 0x7f23c00cc3e0 2024-03-22T04:11:55.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23bfe00640 1 -- 172.21.15.86:0/1167249832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80a5d60 con 0x7f23c00cc3e0 2024-03-22T04:11:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c7e00640 1 -- 172.21.15.86:0/1167249832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c00cc3e0 msgr2=0x7f23c00ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1167249832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c00cc3e0 0x7f23c00ae370 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009e50 tx=0x7f23b801ca30 comp rx=0 tx=0).stop 2024-03-22T04:11:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c7e00640 1 -- 172.21.15.86:0/1167249832 shutdown_connections 2024-03-22T04:11:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1167249832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1167249832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c00cc3e0 0x7f23c00ae370 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c7e00640 1 -- 172.21.15.86:0/1167249832 >> 172.21.15.86:0/1167249832 conn(0x7f23c00b1df0 msgr2=0x7f23c00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c7e00640 1 -- 172.21.15.86:0/1167249832 shutdown_connections 2024-03-22T04:11:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.822+0000 7f23c7e00640 1 -- 172.21.15.86:0/1167249832 wait complete. 2024-03-22T04:11:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 Processor -- start 2024-03-22T04:11:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 -- start start 2024-03-22T04:11:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c0156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c00cc3e0 0x7f23c009a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c015b3f0 0x7f23c009cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0168580 con 0x7f23c015b3f0 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0168400 con 0x7f23c00cc3e0 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f23c0168700 con 0x7f23c00730d0 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c015b3f0 0x7f23c009cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c015b3f0 0x7f23c009cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53694/0 (socket says 172.21.15.86:53694) 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 -- 172.21.15.86:0/1888170200 learned_addr learned my addr 172.21.15.86:0/1888170200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c4a00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c00cc3e0 0x7f23c009a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5400640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c0156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 -- 172.21.15.86:0/1888170200 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 msgr2=0x7f23c0156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c0156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 -- 172.21.15.86:0/1888170200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c00cc3e0 msgr2=0x7f23c009a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:55.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c00cc3e0 0x7f23c009a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:55.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 -- 172.21.15.86:0/1888170200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c00cf5c0 con 0x7f23c015b3f0 2024-03-22T04:11:55.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c5e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c015b3f0 0x7f23c009cd60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23b80af040 tx=0x7f23b80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:55.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23bd600640 1 -- 172.21.15.86:0/1888170200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80bc020 con 0x7f23c015b3f0 2024-03-22T04:11:55.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23bd600640 1 -- 172.21.15.86:0/1888170200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b80b8670 con 0x7f23c015b3f0 2024-03-22T04:11:55.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23bd600640 1 -- 172.21.15.86:0/1888170200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80b8a60 con 0x7f23c015b3f0 2024-03-22T04:11:55.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c00cf8a0 con 0x7f23c015b3f0 2024-03-22T04:11:55.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.826+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c00cfe60 con 0x7f23c015b3f0 2024-03-22T04:11:55.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.830+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c0068650 con 0x7f23c015b3f0 2024-03-22T04:11:55.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.830+0000 7f23bd600640 1 -- 172.21.15.86:0/1888170200 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f23b80bf020 con 0x7f23c015b3f0 2024-03-22T04:11:55.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.830+0000 7f23bd600640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f23a0067b20 0x7f23a0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:55.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.830+0000 7f23bd600640 1 -- 172.21.15.86:0/1888170200 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f23b8121060 con 0x7f23c015b3f0 2024-03-22T04:11:55.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.830+0000 7f23c5400640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f23a0067b20 0x7f23a0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:55.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.830+0000 7f23c5400640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f23a0067b20 0x7f23a0069fe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000f70 tx=0x7f23b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:55.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:55.834+0000 7f23bd600640 1 -- 172.21.15.86:0/1888170200 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f23b80c6030 con 0x7f23c015b3f0 2024-03-22T04:11:56.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.022+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23c00cc160 con 0x7f23c015b3f0 2024-03-22T04:11:56.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.022+0000 7f23bd600640 1 -- 172.21.15.86:0/1888170200 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f23b8016620 con 0x7f23c015b3f0 2024-03-22T04:11:56.031 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:56.031 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:11:51.996418+0000","modified":"2024-03-22T04:11:55.030984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74930":{"gid":74930,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1426902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1426902111},{"type":"v1","addr":"172.21.15.196:6835","nonce":1426902111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74930,"qdb_cluster":[74930]},"id":2}]} 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f23a0067b20 msgr2=0x7f23a0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f23a0067b20 0x7f23a0069fe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000f70 tx=0x7f23b4009040 comp rx=0 tx=0).stop 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c015b3f0 msgr2=0x7f23c009cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c015b3f0 0x7f23c009cd60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23b80af040 tx=0x7f23b80197d0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 shutdown_connections 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f23a0067b20 0x7f23a0069fe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23c00cc3e0 0x7f23c009a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23c015b3f0 0x7f23c009cd60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 --2- 172.21.15.86:0/1888170200 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23c00730d0 0x7f23c0156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 >> 172.21.15.86:0/1888170200 conn(0x7f23c00b1df0 msgr2=0x7f23c00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 shutdown_connections 2024-03-22T04:11:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.026+0000 7f23c7e00640 1 -- 172.21.15.86:0/1888170200 wait complete. 2024-03-22T04:11:56.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:11:56.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.178+0000 7fe597a00640 1 Processor -- start 2024-03-22T04:11:56.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.178+0000 7fe597a00640 1 -- start start 2024-03-22T04:11:56.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe597a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe5900d1360 0x7fe5900b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe597a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5901457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe597a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5900d1810 con 0x7fe59006fc60 2024-03-22T04:11:56.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe597a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe5900ce270 con 0x7fe5900d1360 2024-03-22T04:11:56.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe597a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5900d1af0 con 0x7fe590070040 2024-03-22T04:11:56.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe595000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe59006fc60 0x7fe5900d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40344/0 (socket says 172.21.15.86:40344) 2024-03-22T04:11:56.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe595000640 1 -- 172.21.15.86:0/1102656259 learned_addr learned my addr 172.21.15.86:0/1102656259 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:56.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe595a00640 1 --2- 172.21.15.86:0/1102656259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5901457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe58fe00640 1 --2- 172.21.15.86:0/1102656259 >> v2:172.21.15.196:3300/0 conn(0x7fe5900d1360 0x7fe5900b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe58f400640 1 -- 172.21.15.86:0/1102656259 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437206055 0 0) 0x7fe5900d1810 con 0x7fe59006fc60 2024-03-22T04:11:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe58f400640 1 -- 172.21.15.86:0/1102656259 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe578003530 con 0x7fe59006fc60 2024-03-22T04:11:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe595a00640 1 -- 172.21.15.86:0/1102656259 >> v2:172.21.15.196:3300/0 conn(0x7fe5900d1360 msgr2=0x7fe5900b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe595a00640 1 --2- 172.21.15.86:0/1102656259 >> v2:172.21.15.196:3300/0 conn(0x7fe5900d1360 0x7fe5900b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe595a00640 1 -- 172.21.15.86:0/1102656259 >> v1:172.21.15.86:6789/0 conn(0x7fe59006fc60 legacy=0x7fe5900d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe595a00640 1 -- 172.21.15.86:0/1102656259 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900d1dd0 con 0x7fe590070040 2024-03-22T04:11:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.182+0000 7fe58fe00640 1 --2- 172.21.15.86:0/1102656259 >> v2:172.21.15.196:3300/0 conn(0x7fe5900d1360 0x7fe5900b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe595a00640 1 --2- 172.21.15.86:0/1102656259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5901457f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe588009d90 tx=0x7fe58801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14453ca74a0d955d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe58f400640 1 -- 172.21.15.86:0/1102656259 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5880b2070 con 0x7fe590070040 2024-03-22T04:11:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe58f400640 1 -- 172.21.15.86:0/1102656259 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5880a5a20 con 0x7fe590070040 2024-03-22T04:11:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe58f400640 1 -- 172.21.15.86:0/1102656259 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5880a5df0 con 0x7fe590070040 2024-03-22T04:11:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 -- 172.21.15.86:0/1102656259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 msgr2=0x7fe5901457f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 --2- 172.21.15.86:0/1102656259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5901457f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe588009d90 tx=0x7fe58801c940 comp rx=0 tx=0).stop 2024-03-22T04:11:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 -- 172.21.15.86:0/1102656259 shutdown_connections 2024-03-22T04:11:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 --2- 172.21.15.86:0/1102656259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5901457f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 --2- 172.21.15.86:0/1102656259 >> v2:172.21.15.196:3300/0 conn(0x7fe5900d1360 0x7fe5900b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 -- 172.21.15.86:0/1102656259 >> 172.21.15.86:0/1102656259 conn(0x7fe5900b75b0 msgr2=0x7fe5900b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 -- 172.21.15.86:0/1102656259 shutdown_connections 2024-03-22T04:11:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 -- 172.21.15.86:0/1102656259 wait complete. 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 Processor -- start 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.186+0000 7fe597a00640 1 -- start start 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe59006fc60 0x7fe590158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5900b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5900d1360 0x7fe59013b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900cf370 con 0x7fe59006fc60 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900cf1f0 con 0x7fe5900d1360 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5900cf4f0 con 0x7fe590070040 2024-03-22T04:11:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe59006fc60 0x7fe590158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe59006fc60 0x7fe590158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36664/0 (socket says 172.21.15.86:36664) 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 -- 172.21.15.86:0/1112364704 learned_addr learned my addr 172.21.15.86:0/1112364704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595a00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5900d1360 0x7fe59013b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe58fe00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5900b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 -- 172.21.15.86:0/1112364704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 msgr2=0x7fe5900b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5900b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 -- 172.21.15.86:0/1112364704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5900d1360 msgr2=0x7fe59013b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5900d1360 0x7fe59013b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 -- 172.21.15.86:0/1112364704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe590165f40 con 0x7fe59006fc60 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe595000640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe59006fc60 0x7fe590158e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe58400c450 tx=0x7fe58400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe58cc00640 1 -- 172.21.15.86:0/1112364704 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584017070 con 0x7fe59006fc60 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe58cc00640 1 -- 172.21.15.86:0/1112364704 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe584005cf0 con 0x7fe59006fc60 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe58cc00640 1 -- 172.21.15.86:0/1112364704 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584007290 con 0x7fe59006fc60 2024-03-22T04:11:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901594a0 con 0x7fe59006fc60 2024-03-22T04:11:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe590159a60 con 0x7fe59006fc60 2024-03-22T04:11:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe590068650 con 0x7fe59006fc60 2024-03-22T04:11:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.190+0000 7fe58cc00640 1 -- 172.21.15.86:0/1112364704 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe58401a070 con 0x7fe59006fc60 2024-03-22T04:11:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.194+0000 7fe58cc00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe570067b50 0x7fe57006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.194+0000 7fe58cc00640 1 -- 172.21.15.86:0/1112364704 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fe584085370 con 0x7fe59006fc60 2024-03-22T04:11:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.194+0000 7fe58fe00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe570067b50 0x7fe57006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.194+0000 7fe58fe00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe570067b50 0x7fe57006a010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe5900b20e0 tx=0x7fe57c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.198+0000 7fe58cc00640 1 -- 172.21.15.86:0/1112364704 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe584055590 con 0x7fe59006fc60 2024-03-22T04:11:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.354+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5900b6a50 con 0x7fe59006fc60 2024-03-22T04:11:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe58cc00640 1 -- 172.21.15.86:0/1112364704 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16942 (secure 0 0 0) 0x7fe584059630 con 0x7fe59006fc60 2024-03-22T04:11:56.363 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:56.363 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":41,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:53.644770+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:11:49.644732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-22T04:11:50.961986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:11:56.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe570067b50 msgr2=0x7fe57006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe570067b50 0x7fe57006a010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe5900b20e0 tx=0x7fe57c008040 comp rx=0 tx=0).stop 2024-03-22T04:11:56.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe59006fc60 msgr2=0x7fe590158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe59006fc60 0x7fe590158e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe58400c450 tx=0x7fe58400c910 comp rx=0 tx=0).stop 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 shutdown_connections 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe570067b50 0x7fe57006a010 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5900d1360 0x7fe59013b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe590070040 0x7fe5900b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 --2- 172.21.15.86:0/1112364704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe59006fc60 0x7fe590158e10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 >> 172.21.15.86:0/1112364704 conn(0x7fe5900b75b0 msgr2=0x7fe590069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 shutdown_connections 2024-03-22T04:11:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.358+0000 7fe597a00640 1 -- 172.21.15.86:0/1112364704 wait complete. 2024-03-22T04:11:56.384 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T04:11:56.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:11:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f9a800640 1 Processor -- start 2024-03-22T04:11:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f9a800640 1 -- start start 2024-03-22T04:11:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f9a800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f9415fed0 0x7f6f941602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f9a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f940d11c0 con 0x7f6f940ce6b0 2024-03-22T04:11:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f9a800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f94160820 con 0x7f6f940d0de0 2024-03-22T04:11:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f9a800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6f940d0580 con 0x7f6f9415fed0 2024-03-22T04:11:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f92a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6f940ce6b0 0x7f6f940cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40356/0 (socket says 172.21.15.86:40356) 2024-03-22T04:11:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f92a00640 1 -- 172.21.15.86:0/3547079576 learned_addr learned my addr 172.21.15.86:0/3547079576 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f93e00640 1 --2- 172.21.15.86:0/3547079576 >> v2:172.21.15.196:3300/0 conn(0x7f6f9415fed0 0x7f6f941602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f92000640 1 -- 172.21.15.86:0/3547079576 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2556740867 0 0) 0x7f6f940d11c0 con 0x7f6f940ce6b0 2024-03-22T04:11:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f92000640 1 -- 172.21.15.86:0/3547079576 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f78003530 con 0x7f6f940ce6b0 2024-03-22T04:11:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f92000640 1 -- 172.21.15.86:0/3547079576 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316823293 0 0) 0x7f6f7c001120 con 0x7f6f940d0de0 2024-03-22T04:11:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f92000640 1 -- 172.21.15.86:0/3547079576 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f940d11c0 con 0x7f6f940d0de0 2024-03-22T04:11:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f93e00640 1 -- 172.21.15.86:0/3547079576 >> v1:172.21.15.196:6789/0 conn(0x7f6f940d0de0 legacy=0x7f6f940d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f93e00640 1 -- 172.21.15.86:0/3547079576 >> v1:172.21.15.86:6789/0 conn(0x7f6f940ce6b0 legacy=0x7f6f940cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.514+0000 7f6f93e00640 1 -- 172.21.15.86:0/3547079576 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f940cece0 con 0x7f6f9415fed0 2024-03-22T04:11:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f93e00640 1 --2- 172.21.15.86:0/3547079576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f9415fed0 0x7f6f941602e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009d90 tx=0x7f6f8c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c989bfa33ee52d55 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f92000640 1 -- 172.21.15.86:0/3547079576 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0b2070 con 0x7f6f9415fed0 2024-03-22T04:11:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f92000640 1 -- 172.21.15.86:0/3547079576 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f8c0a5830 con 0x7f6f9415fed0 2024-03-22T04:11:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f92000640 1 -- 172.21.15.86:0/3547079576 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0a5be0 con 0x7f6f9415fed0 2024-03-22T04:11:56.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f9a800640 1 -- 172.21.15.86:0/3547079576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f9415fed0 msgr2=0x7f6f941602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f9a800640 1 --2- 172.21.15.86:0/3547079576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f9415fed0 0x7f6f941602e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009d90 tx=0x7f6f8c01c970 comp rx=0 tx=0).stop 2024-03-22T04:11:56.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f9a800640 1 -- 172.21.15.86:0/3547079576 shutdown_connections 2024-03-22T04:11:56.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f9a800640 1 --2- 172.21.15.86:0/3547079576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f9415fed0 0x7f6f941602e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f9a800640 1 -- 172.21.15.86:0/3547079576 >> 172.21.15.86:0/3547079576 conn(0x7f6f9406f910 msgr2=0x7f6f9406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:56.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f9a800640 1 -- 172.21.15.86:0/3547079576 shutdown_connections 2024-03-22T04:11:56.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.518+0000 7f6f9a800640 1 -- 172.21.15.86:0/3547079576 wait complete. 2024-03-22T04:11:56.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 Processor -- start 2024-03-22T04:11:56.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 -- start start 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f940ce6b0 0x7f6f94151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f940d0de0 0x7f6f940b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9415fed0 0x7f6f9415ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94071e30 con 0x7f6f940d0de0 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94071100 con 0x7f6f940ce6b0 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f94071fb0 con 0x7f6f9415fed0 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f940d0de0 0x7f6f940b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f940d0de0 0x7f6f940b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36680/0 (socket says 172.21.15.86:36680) 2024-03-22T04:11:56.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 -- 172.21.15.86:0/3453338146 learned_addr learned my addr 172.21.15.86:0/3453338146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f93400640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f940ce6b0 0x7f6f94151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f93e00640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9415fed0 0x7f6f9415ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 -- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9415fed0 msgr2=0x7f6f9415ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9415fed0 0x7f6f9415ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 -- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f940ce6b0 msgr2=0x7f6f94151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f940ce6b0 0x7f6f94151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 -- 172.21.15.86:0/3453338146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f941517d0 con 0x7f6f940d0de0 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f93400640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f940ce6b0 0x7f6f94151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f92a00640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f940d0de0 0x7f6f940b3510 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c420 tx=0x7f6f8400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f8b400640 1 -- 172.21.15.86:0/3453338146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84017070 con 0x7f6f940d0de0 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f8b400640 1 -- 172.21.15.86:0/3453338146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f84005cf0 con 0x7f6f940d0de0 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f8b400640 1 -- 172.21.15.86:0/3453338146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84007310 con 0x7f6f940d0de0 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f94152430 con 0x7f6f940d0de0 2024-03-22T04:11:56.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f941529f0 con 0x7f6f940d0de0 2024-03-22T04:11:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.522+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f94068650 con 0x7f6f940d0de0 2024-03-22T04:11:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.526+0000 7f6f8b400640 1 -- 172.21.15.86:0/3453338146 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6f8401a070 con 0x7f6f940d0de0 2024-03-22T04:11:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.526+0000 7f6f8b400640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f70067b20 0x7f6f70069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.526+0000 7f6f8b400640 1 -- 172.21.15.86:0/3453338146 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f6f840853c0 con 0x7f6f940d0de0 2024-03-22T04:11:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.526+0000 7f6f93400640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f70067b20 0x7f6f70069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.526+0000 7f6f93400640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f70067b20 0x7f6f70069fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0012c0 tx=0x7f6f7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.530+0000 7f6f8b400640 1 -- 172.21.15.86:0/3453338146 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6f84055660 con 0x7f6f940d0de0 2024-03-22T04:11:56.596 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:11:56.594+0000 7f9eac000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:11:56.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.682+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f9406cd90 con 0x7f6f70067b20 2024-03-22T04:11:56.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f8b400640 1 -- 172.21.15.86:0/3453338146 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242394 (secure 0 0 0) 0x7f6f9406cd90 con 0x7f6f70067b20 2024-03-22T04:11:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:56.699 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:11:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f70067b20 msgr2=0x7f6f70069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f9a800640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f70067b20 0x7f6f70069fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0012c0 tx=0x7f6f7c009040 comp rx=0 tx=0).stop 2024-03-22T04:11:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f940d0de0 msgr2=0x7f6f940b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f9a800640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f940d0de0 0x7f6f940b3510 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c420 tx=0x7f6f8400c8e0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 shutdown_connections 2024-03-22T04:11:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f9a800640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f70067b20 0x7f6f70069fe0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f9a800640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9415fed0 0x7f6f9415ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.690+0000 7f6f9a800640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f940d0de0 0x7f6f940b3510 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.694+0000 7f6f9a800640 1 --2- 172.21.15.86:0/3453338146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f940ce6b0 0x7f6f94151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.694+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 >> 172.21.15.86:0/3453338146 conn(0x7f6f9406f910 msgr2=0x7f6f9415b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:56.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.694+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 shutdown_connections 2024-03-22T04:11:56.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.694+0000 7f6f9a800640 1 -- 172.21.15.86:0/3453338146 wait complete. 2024-03-22T04:11:56.717 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-03-22T04:11:55.808113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":92,"acting":92,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220696,"kb_used_data":4504,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754024,"statfs":{"total":773094113280,"available":772868120576,"internally_reserved":0,"allocated":4612096,"data_stored":2599861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.197155"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.669236+0000","last_change":"2024-03-22T04:11:52.662881+0000","last_active":"2024-03-22T04:11:52.669236+0000","last_peered":"2024-03-22T04:11:52.669236+0000","last_clean":"2024-03-22T04:11:52.669236+0000","last_became_active":"2024-03-22T04:11:52.662454+0000","last_became_peered":"2024-03-22T04:11:52.662454+0000","last_unstale":"2024-03-22T04:11:52.669236+0000","last_undegraded":"2024-03-22T04:11:52.669236+0000","last_fullsized":"2024-03-22T04:11:52.669236+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:23.935409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.669282+0000","last_change":"2024-03-22T04:11:52.662047+0000","last_active":"2024-03-22T04:11:52.669282+0000","last_peered":"2024-03-22T04:11:52.669282+0000","last_clean":"2024-03-22T04:11:52.669282+0000","last_became_active":"2024-03-22T04:11:52.661357+0000","last_became_peered":"2024-03-22T04:11:52.661357+0000","last_unstale":"2024-03-22T04:11:52.669282+0000","last_undegraded":"2024-03-22T04:11:52.669282+0000","last_fullsized":"2024-03-22T04:11:52.669282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:15:57.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.663438+0000","last_change":"2024-03-22T04:11:52.655700+0000","last_active":"2024-03-22T04:11:52.663438+0000","last_peered":"2024-03-22T04:11:52.663438+0000","last_clean":"2024-03-22T04:11:52.663438+0000","last_became_active":"2024-03-22T04:11:52.655442+0000","last_became_peered":"2024-03-22T04:11:52.655442+0000","last_unstale":"2024-03-22T04:11:52.663438+0000","last_undegraded":"2024-03-22T04:11:52.663438+0000","last_fullsized":"2024-03-22T04:11:52.663438+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:19:04.849981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667934+0000","last_change":"2024-03-22T04:11:52.658845+0000","last_active":"2024-03-22T04:11:52.667934+0000","last_peered":"2024-03-22T04:11:52.667934+0000","last_clean":"2024-03-22T04:11:52.667934+0000","last_became_active":"2024-03-22T04:11:52.657778+0000","last_became_peered":"2024-03-22T04:11:52.657778+0000","last_unstale":"2024-03-22T04:11:52.667934+0000","last_undegraded":"2024-03-22T04:11:52.667934+0000","last_fullsized":"2024-03-22T04:11:52.667934+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:46:39.954425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667894+0000","last_change":"2024-03-22T04:11:52.658420+0000","last_active":"2024-03-22T04:11:52.667894+0000","last_peered":"2024-03-22T04:11:52.667894+0000","last_clean":"2024-03-22T04:11:52.667894+0000","last_became_active":"2024-03-22T04:11:52.657062+0000","last_became_peered":"2024-03-22T04:11:52.657062+0000","last_unstale":"2024-03-22T04:11:52.667894+0000","last_undegraded":"2024-03-22T04:11:52.667894+0000","last_fullsized":"2024-03-22T04:11:52.667894+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:47.563475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.656938+0000","last_change":"2024-03-22T04:11:52.652956+0000","last_active":"2024-03-22T04:11:52.656938+0000","last_peered":"2024-03-22T04:11:52.656938+0000","last_clean":"2024-03-22T04:11:52.656938+0000","last_became_active":"2024-03-22T04:11:52.652568+0000","last_became_peered":"2024-03-22T04:11:52.652568+0000","last_unstale":"2024-03-22T04:11:52.656938+0000","last_undegraded":"2024-03-22T04:11:52.656938+0000","last_fullsized":"2024-03-22T04:11:52.656938+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:35:18.455069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667915+0000","last_change":"2024-03-22T04:11:52.658632+0000","last_active":"2024-03-22T04:11:52.667915+0000","last_peered":"2024-03-22T04:11:52.667915+0000","last_clean":"2024-03-22T04:11:52.667915+0000","last_became_active":"2024-03-22T04:11:52.657426+0000","last_became_peered":"2024-03-22T04:11:52.657426+0000","last_unstale":"2024-03-22T04:11:52.667915+0000","last_undegraded":"2024-03-22T04:11:52.667915+0000","last_fullsized":"2024-03-22T04:11:52.667915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:59:50.201070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613235+0000","last_change":"2024-03-22T04:11:50.613235+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613235+0000","last_undegraded":"2024-03-22T04:11:50.613235+0000","last_fullsized":"2024-03-22T04:11:50.613235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615708+0000","last_change":"2024-03-22T04:11:50.615708+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615708+0000","last_undegraded":"2024-03-22T04:11:50.615708+0000","last_fullsized":"2024-03-22T04:11:50.615708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.679524+0000","last_change":"2024-03-22T04:11:51.636087+0000","last_active":"2024-03-22T04:11:52.679524+0000","last_peered":"2024-03-22T04:11:52.679524+0000","last_clean":"2024-03-22T04:11:52.679524+0000","last_became_active":"2024-03-22T04:11:51.635794+0000","last_became_peered":"2024-03-22T04:11:51.635794+0000","last_unstale":"2024-03-22T04:11:52.679524+0000","last_undegraded":"2024-03-22T04:11:52.679524+0000","last_fullsized":"2024-03-22T04:11:52.679524+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:29.018276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613272+0000","last_change":"2024-03-22T04:11:50.613272+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613272+0000","last_undegraded":"2024-03-22T04:11:50.613272+0000","last_fullsized":"2024-03-22T04:11:50.613272+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667952+0000","last_change":"2024-03-22T04:11:52.659045+0000","last_active":"2024-03-22T04:11:52.667952+0000","last_peered":"2024-03-22T04:11:52.667952+0000","last_clean":"2024-03-22T04:11:52.667952+0000","last_became_active":"2024-03-22T04:11:52.658165+0000","last_became_peered":"2024-03-22T04:11:52.658165+0000","last_unstale":"2024-03-22T04:11:52.667952+0000","last_undegraded":"2024-03-22T04:11:52.667952+0000","last_fullsized":"2024-03-22T04:11:52.667952+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:12.332645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613232+0000","last_change":"2024-03-22T04:11:50.613232+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613232+0000","last_undegraded":"2024-03-22T04:11:50.613232+0000","last_fullsized":"2024-03-22T04:11:50.613232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615507+0000","last_change":"2024-03-22T04:11:50.615507+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615507+0000","last_undegraded":"2024-03-22T04:11:50.615507+0000","last_fullsized":"2024-03-22T04:11:50.615507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.638844+0000","last_change":"2024-03-22T04:11:51.645291+0000","last_active":"2024-03-22T04:11:52.638844+0000","last_peered":"2024-03-22T04:11:52.638844+0000","last_clean":"2024-03-22T04:11:52.638844+0000","last_became_active":"2024-03-22T04:11:51.644994+0000","last_became_peered":"2024-03-22T04:11:51.644994+0000","last_unstale":"2024-03-22T04:11:52.638844+0000","last_undegraded":"2024-03-22T04:11:52.638844+0000","last_fullsized":"2024-03-22T04:11:52.638844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:36:32.547421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615494+0000","last_change":"2024-03-22T04:11:50.615494+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615494+0000","last_undegraded":"2024-03-22T04:11:50.615494+0000","last_fullsized":"2024-03-22T04:11:50.615494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.657919+0000","last_change":"2024-03-22T04:11:52.652881+0000","last_active":"2024-03-22T04:11:52.657919+0000","last_peered":"2024-03-22T04:11:52.657919+0000","last_clean":"2024-03-22T04:11:52.657919+0000","last_became_active":"2024-03-22T04:11:52.652190+0000","last_became_peered":"2024-03-22T04:11:52.652190+0000","last_unstale":"2024-03-22T04:11:52.657919+0000","last_undegraded":"2024-03-22T04:11:52.657919+0000","last_fullsized":"2024-03-22T04:11:52.657919+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:08:26.601087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639424+0000","last_change":"2024-03-22T04:11:51.635044+0000","last_active":"2024-03-22T04:11:52.639424+0000","last_peered":"2024-03-22T04:11:52.639424+0000","last_clean":"2024-03-22T04:11:52.639424+0000","last_became_active":"2024-03-22T04:11:51.633837+0000","last_became_peered":"2024-03-22T04:11:51.633837+0000","last_unstale":"2024-03-22T04:11:52.639424+0000","last_undegraded":"2024-03-22T04:11:52.639424+0000","last_fullsized":"2024-03-22T04:11:52.639424+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:31:32.200080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619235+0000","last_change":"2024-03-22T04:11:50.619235+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619235+0000","last_undegraded":"2024-03-22T04:11:50.619235+0000","last_fullsized":"2024-03-22T04:11:50.619235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.665977+0000","last_change":"2024-03-22T04:11:52.656994+0000","last_active":"2024-03-22T04:11:52.665977+0000","last_peered":"2024-03-22T04:11:52.665977+0000","last_clean":"2024-03-22T04:11:52.665977+0000","last_became_active":"2024-03-22T04:11:52.656688+0000","last_became_peered":"2024-03-22T04:11:52.656688+0000","last_unstale":"2024-03-22T04:11:52.665977+0000","last_undegraded":"2024-03-22T04:11:52.665977+0000","last_fullsized":"2024-03-22T04:11:52.665977+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:18:30.069733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.620324+0000","last_change":"2024-03-22T04:11:50.620324+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.620324+0000","last_undegraded":"2024-03-22T04:11:50.620324+0000","last_fullsized":"2024-03-22T04:11:50.620324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.681884+0000","last_change":"2024-03-22T04:11:51.644715+0000","last_active":"2024-03-22T04:11:52.681884+0000","last_peered":"2024-03-22T04:11:52.681884+0000","last_clean":"2024-03-22T04:11:52.681884+0000","last_became_active":"2024-03-22T04:11:51.644365+0000","last_became_peered":"2024-03-22T04:11:51.644365+0000","last_unstale":"2024-03-22T04:11:52.681884+0000","last_undegraded":"2024-03-22T04:11:52.681884+0000","last_fullsized":"2024-03-22T04:11:52.681884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:49:10.202707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.656068+0000","last_change":"2024-03-22T04:11:52.646801+0000","last_active":"2024-03-22T04:11:52.656068+0000","last_peered":"2024-03-22T04:11:52.656068+0000","last_clean":"2024-03-22T04:11:52.656068+0000","last_became_active":"2024-03-22T04:11:52.646491+0000","last_became_peered":"2024-03-22T04:11:52.646491+0000","last_unstale":"2024-03-22T04:11:52.656068+0000","last_undegraded":"2024-03-22T04:11:52.656068+0000","last_fullsized":"2024-03-22T04:11:52.656068+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:44:29.455556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.679634+0000","last_change":"2024-03-22T04:11:51.635750+0000","last_active":"2024-03-22T04:11:52.679634+0000","last_peered":"2024-03-22T04:11:52.679634+0000","last_clean":"2024-03-22T04:11:52.679634+0000","last_became_active":"2024-03-22T04:11:51.634686+0000","last_became_peered":"2024-03-22T04:11:51.634686+0000","last_unstale":"2024-03-22T04:11:52.679634+0000","last_undegraded":"2024-03-22T04:11:52.679634+0000","last_fullsized":"2024-03-22T04:11:52.679634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:56:45.221222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616261+0000","last_change":"2024-03-22T04:11:50.616261+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616261+0000","last_undegraded":"2024-03-22T04:11:50.616261+0000","last_fullsized":"2024-03-22T04:11:50.616261+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619979+0000","last_change":"2024-03-22T04:11:50.619979+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619979+0000","last_undegraded":"2024-03-22T04:11:50.619979+0000","last_fullsized":"2024-03-22T04:11:50.619979+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613507+0000","last_change":"2024-03-22T04:11:50.613507+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613507+0000","last_undegraded":"2024-03-22T04:11:50.613507+0000","last_fullsized":"2024-03-22T04:11:50.613507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615586+0000","last_change":"2024-03-22T04:11:50.615586+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615586+0000","last_undegraded":"2024-03-22T04:11:50.615586+0000","last_fullsized":"2024-03-22T04:11:50.615586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618788+0000","last_change":"2024-03-22T04:11:50.618788+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618788+0000","last_undegraded":"2024-03-22T04:11:50.618788+0000","last_fullsized":"2024-03-22T04:11:50.618788+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616139+0000","last_change":"2024-03-22T04:11:50.616139+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616139+0000","last_undegraded":"2024-03-22T04:11:50.616139+0000","last_fullsized":"2024-03-22T04:11:50.616139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617590+0000","last_change":"2024-03-22T04:11:50.617590+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617590+0000","last_undegraded":"2024-03-22T04:11:50.617590+0000","last_fullsized":"2024-03-22T04:11:50.617590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618135+0000","last_change":"2024-03-22T04:11:50.618135+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618135+0000","last_undegraded":"2024-03-22T04:11:50.618135+0000","last_fullsized":"2024-03-22T04:11:50.618135+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.666825+0000","last_change":"2024-03-22T04:11:52.657565+0000","last_active":"2024-03-22T04:11:52.666825+0000","last_peered":"2024-03-22T04:11:52.666825+0000","last_clean":"2024-03-22T04:11:52.666825+0000","last_became_active":"2024-03-22T04:11:52.657329+0000","last_became_peered":"2024-03-22T04:11:52.657329+0000","last_unstale":"2024-03-22T04:11:52.666825+0000","last_undegraded":"2024-03-22T04:11:52.666825+0000","last_fullsized":"2024-03-22T04:11:52.666825+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:51.727928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616872+0000","last_change":"2024-03-22T04:11:50.616872+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616872+0000","last_undegraded":"2024-03-22T04:11:50.616872+0000","last_fullsized":"2024-03-22T04:11:50.616872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615564+0000","last_change":"2024-03-22T04:11:50.615564+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615564+0000","last_undegraded":"2024-03-22T04:11:50.615564+0000","last_fullsized":"2024-03-22T04:11:50.615564+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667509+0000","last_change":"2024-03-22T04:11:52.657915+0000","last_active":"2024-03-22T04:11:52.667509+0000","last_peered":"2024-03-22T04:11:52.667509+0000","last_clean":"2024-03-22T04:11:52.667509+0000","last_became_active":"2024-03-22T04:11:52.657728+0000","last_became_peered":"2024-03-22T04:11:52.657728+0000","last_unstale":"2024-03-22T04:11:52.667509+0000","last_undegraded":"2024-03-22T04:11:52.667509+0000","last_fullsized":"2024-03-22T04:11:52.667509+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:53:28.532936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615718+0000","last_change":"2024-03-22T04:11:50.615718+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615718+0000","last_undegraded":"2024-03-22T04:11:50.615718+0000","last_fullsized":"2024-03-22T04:11:50.615718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618931+0000","last_change":"2024-03-22T04:11:50.618931+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618931+0000","last_undegraded":"2024-03-22T04:11:50.618931+0000","last_fullsized":"2024-03-22T04:11:50.618931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.614675+0000","last_change":"2024-03-22T04:11:50.614675+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.614675+0000","last_undegraded":"2024-03-22T04:11:50.614675+0000","last_fullsized":"2024-03-22T04:11:50.614675+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615612+0000","last_change":"2024-03-22T04:11:50.615612+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615612+0000","last_undegraded":"2024-03-22T04:11:50.615612+0000","last_fullsized":"2024-03-22T04:11:50.615612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639215+0000","last_change":"2024-03-22T04:11:51.635806+0000","last_active":"2024-03-22T04:11:52.639215+0000","last_peered":"2024-03-22T04:11:52.639215+0000","last_clean":"2024-03-22T04:11:52.639215+0000","last_became_active":"2024-03-22T04:11:51.633614+0000","last_became_peered":"2024-03-22T04:11:51.633614+0000","last_unstale":"2024-03-22T04:11:52.639215+0000","last_undegraded":"2024-03-22T04:11:52.639215+0000","last_fullsized":"2024-03-22T04:11:52.639215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:14.289312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617268+0000","last_change":"2024-03-22T04:11:50.617268+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617268+0000","last_undegraded":"2024-03-22T04:11:50.617268+0000","last_fullsized":"2024-03-22T04:11:50.617268+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619612+0000","last_change":"2024-03-22T04:11:50.619612+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619612+0000","last_undegraded":"2024-03-22T04:11:50.619612+0000","last_fullsized":"2024-03-22T04:11:50.619612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639907+0000","last_change":"2024-03-22T04:11:51.641613+0000","last_active":"2024-03-22T04:11:52.639907+0000","last_peered":"2024-03-22T04:11:52.639907+0000","last_clean":"2024-03-22T04:11:52.639907+0000","last_became_active":"2024-03-22T04:11:51.641256+0000","last_became_peered":"2024-03-22T04:11:51.641256+0000","last_unstale":"2024-03-22T04:11:52.639907+0000","last_undegraded":"2024-03-22T04:11:52.639907+0000","last_fullsized":"2024-03-22T04:11:52.639907+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:27.877295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617293+0000","last_change":"2024-03-22T04:11:50.617293+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617293+0000","last_undegraded":"2024-03-22T04:11:50.617293+0000","last_fullsized":"2024-03-22T04:11:50.617293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-22T04:11:50.615841+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:11:50.615841+0000","last_peered":"2024-03-22T04:11:50.615841+0000","last_clean":"2024-03-22T04:11:50.615841+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:11:50.615841+0000","last_undegraded":"2024-03-22T04:11:50.615841+0000","last_fullsized":"2024-03-22T04:11:50.615841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1011712,"data_stored":763484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":763484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":194895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:11:56.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:11:56.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.890+0000 7f3598000640 1 Processor -- start 2024-03-22T04:11:56.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.890+0000 7f3598000640 1 -- start start 2024-03-22T04:11:56.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3598000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3590158200 0x7f3590071d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3598000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3590072250 0x7f359006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3598000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f359006e6f0 con 0x7f359006e260 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3598000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35900726b0 con 0x7f3590072250 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3598000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f359015bc10 con 0x7f3590158200 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3594c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3590072250 0x7f359006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3595600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3590158200 0x7f3590071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3594c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3590072250 0x7f359006dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36690/0 (socket says 172.21.15.86:36690) 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3594c00640 1 -- 172.21.15.86:0/2581279477 learned_addr learned my addr 172.21.15.86:0/2581279477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3596000640 1 --1- 172.21.15.86:0/2581279477 >> v1:172.21.15.196:6789/0 conn(0x7f359006e260 0x7f359015e040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:36704/0 (socket says 172.21.15.86:36704) 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3594c00640 1 -- 172.21.15.86:0/2581279477 >> v2:172.21.15.196:3300/0 conn(0x7f3590158200 msgr2=0x7f3590071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3594c00640 1 --2- 172.21.15.86:0/2581279477 >> v2:172.21.15.196:3300/0 conn(0x7f3590158200 0x7f3590071d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3594c00640 1 -- 172.21.15.86:0/2581279477 >> v1:172.21.15.196:6789/0 conn(0x7f359006e260 legacy=0x7f359015e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f3594c00640 1 -- 172.21.15.86:0/2581279477 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359006e9a0 con 0x7f3590072250 2024-03-22T04:11:56.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.894+0000 7f358fe00640 1 -- 172.21.15.86:0/2581279477 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594671658 0 0) 0x7f359006e6f0 con 0x7f359006e260 2024-03-22T04:11:56.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3594c00640 1 --2- 172.21.15.86:0/2581279477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590072250 0x7f359006dd20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3590072990 tx=0x7f357c01cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489f652c0eda13cf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f358fe00640 1 -- 172.21.15.86:0/2581279477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c0b2070 con 0x7f3590072250 2024-03-22T04:11:56.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f358fe00640 1 -- 172.21.15.86:0/2581279477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f357c0a5be0 con 0x7f3590072250 2024-03-22T04:11:56.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f358fe00640 1 -- 172.21.15.86:0/2581279477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c0ad440 con 0x7f3590072250 2024-03-22T04:11:56.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 -- 172.21.15.86:0/2581279477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590072250 msgr2=0x7f359006dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 --2- 172.21.15.86:0/2581279477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590072250 0x7f359006dd20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3590072990 tx=0x7f357c01cd30 comp rx=0 tx=0).stop 2024-03-22T04:11:56.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 -- 172.21.15.86:0/2581279477 shutdown_connections 2024-03-22T04:11:56.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 --2- 172.21.15.86:0/2581279477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3590072250 0x7f359006dd20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 --2- 172.21.15.86:0/2581279477 >> v2:172.21.15.196:3300/0 conn(0x7f3590158200 0x7f3590071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 -- 172.21.15.86:0/2581279477 >> 172.21.15.86:0/2581279477 conn(0x7f359006b660 msgr2=0x7f359006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:56.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 -- 172.21.15.86:0/2581279477 shutdown_connections 2024-03-22T04:11:56.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 -- 172.21.15.86:0/2581279477 wait complete. 2024-03-22T04:11:56.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 Processor -- start 2024-03-22T04:11:56.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 -- start start 2024-03-22T04:11:56.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359006e260 0x7f3590164a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.898+0000 7f3598000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3590072250 0x7f35900b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3598000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3590158200 0x7f3590149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3598000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35900d00f0 con 0x7f359006e260 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3598000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35900cebc0 con 0x7f3590158200 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3598000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f35901671b0 con 0x7f3590072250 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359006e260 0x7f3590164a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359006e260 0x7f3590164a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36702/0 (socket says 172.21.15.86:36702) 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 -- 172.21.15.86:0/1617947628 learned_addr learned my addr 172.21.15.86:0/1617947628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3596000640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3590158200 0x7f3590149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3594c00640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3590072250 0x7f35900b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 -- 172.21.15.86:0/1617947628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3590072250 msgr2=0x7f35900b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3590072250 0x7f35900b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 -- 172.21.15.86:0/1617947628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3590158200 msgr2=0x7f3590149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3590158200 0x7f3590149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:56.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 -- 172.21.15.86:0/1617947628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35901566d0 con 0x7f359006e260 2024-03-22T04:11:56.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3595600640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359006e260 0x7f3590164a90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f358400e930 tx=0x7f358400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f358d600640 1 -- 172.21.15.86:0/1617947628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584017070 con 0x7f359006e260 2024-03-22T04:11:56.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f358d600640 1 -- 172.21.15.86:0/1617947628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358400ca00 con 0x7f359006e260 2024-03-22T04:11:56.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f358d600640 1 -- 172.21.15.86:0/1617947628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358400cdb0 con 0x7f359006e260 2024-03-22T04:11:56.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35901569b0 con 0x7f359006e260 2024-03-22T04:11:56.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35900b4a30 con 0x7f359006e260 2024-03-22T04:11:56.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3590068650 con 0x7f359006e260 2024-03-22T04:11:56.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.902+0000 7f358d600640 1 -- 172.21.15.86:0/1617947628 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f358401a070 con 0x7f359006e260 2024-03-22T04:11:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.906+0000 7f358d600640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3570067b20 0x7f3570069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.906+0000 7f358d600640 1 -- 172.21.15.86:0/1617947628 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f3584085fe0 con 0x7f359006e260 2024-03-22T04:11:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.906+0000 7f3594c00640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3570067b20 0x7f3570069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.906+0000 7f3594c00640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3570067b20 0x7f3570069fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f357c0af000 tx=0x7f357c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:56.910+0000 7f358d600640 1 -- 172.21.15.86:0/1617947628 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3584056360 con 0x7f359006e260 2024-03-22T04:11:57.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.058+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35900cce20 con 0x7f3570067b20 2024-03-22T04:11:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.066+0000 7f358d600640 1 -- 172.21.15.86:0/1617947628 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242394 (secure 0 0 0) 0x7f35900cce20 con 0x7f3570067b20 2024-03-22T04:11:57.075 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3570067b20 msgr2=0x7f3570069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3570067b20 0x7f3570069fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f357c0af000 tx=0x7f357c009040 comp rx=0 tx=0).stop 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359006e260 msgr2=0x7f3590164a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359006e260 0x7f3590164a90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f358400e930 tx=0x7f358400edf0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 shutdown_connections 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3570067b20 0x7f3570069fe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3590158200 0x7f3590149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3590072250 0x7f35900b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 --2- 172.21.15.86:0/1617947628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359006e260 0x7f3590164a90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 >> 172.21.15.86:0/1617947628 conn(0x7f359006b660 msgr2=0x7f359015afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:57.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 shutdown_connections 2024-03-22T04:11:57.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.070+0000 7f3598000640 1 -- 172.21.15.86:0/1617947628 wait complete. 2024-03-22T04:11:57.095 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-03-22T04:11:55.808113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":92,"acting":92,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220696,"kb_used_data":4504,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754024,"statfs":{"total":773094113280,"available":772868120576,"internally_reserved":0,"allocated":4612096,"data_stored":2599861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.197155"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.669236+0000","last_change":"2024-03-22T04:11:52.662881+0000","last_active":"2024-03-22T04:11:52.669236+0000","last_peered":"2024-03-22T04:11:52.669236+0000","last_clean":"2024-03-22T04:11:52.669236+0000","last_became_active":"2024-03-22T04:11:52.662454+0000","last_became_peered":"2024-03-22T04:11:52.662454+0000","last_unstale":"2024-03-22T04:11:52.669236+0000","last_undegraded":"2024-03-22T04:11:52.669236+0000","last_fullsized":"2024-03-22T04:11:52.669236+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:23.935409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.669282+0000","last_change":"2024-03-22T04:11:52.662047+0000","last_active":"2024-03-22T04:11:52.669282+0000","last_peered":"2024-03-22T04:11:52.669282+0000","last_clean":"2024-03-22T04:11:52.669282+0000","last_became_active":"2024-03-22T04:11:52.661357+0000","last_became_peered":"2024-03-22T04:11:52.661357+0000","last_unstale":"2024-03-22T04:11:52.669282+0000","last_undegraded":"2024-03-22T04:11:52.669282+0000","last_fullsized":"2024-03-22T04:11:52.669282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:15:57.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.663438+0000","last_change":"2024-03-22T04:11:52.655700+0000","last_active":"2024-03-22T04:11:52.663438+0000","last_peered":"2024-03-22T04:11:52.663438+0000","last_clean":"2024-03-22T04:11:52.663438+0000","last_became_active":"2024-03-22T04:11:52.655442+0000","last_became_peered":"2024-03-22T04:11:52.655442+0000","last_unstale":"2024-03-22T04:11:52.663438+0000","last_undegraded":"2024-03-22T04:11:52.663438+0000","last_fullsized":"2024-03-22T04:11:52.663438+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:19:04.849981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667934+0000","last_change":"2024-03-22T04:11:52.658845+0000","last_active":"2024-03-22T04:11:52.667934+0000","last_peered":"2024-03-22T04:11:52.667934+0000","last_clean":"2024-03-22T04:11:52.667934+0000","last_became_active":"2024-03-22T04:11:52.657778+0000","last_became_peered":"2024-03-22T04:11:52.657778+0000","last_unstale":"2024-03-22T04:11:52.667934+0000","last_undegraded":"2024-03-22T04:11:52.667934+0000","last_fullsized":"2024-03-22T04:11:52.667934+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:46:39.954425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667894+0000","last_change":"2024-03-22T04:11:52.658420+0000","last_active":"2024-03-22T04:11:52.667894+0000","last_peered":"2024-03-22T04:11:52.667894+0000","last_clean":"2024-03-22T04:11:52.667894+0000","last_became_active":"2024-03-22T04:11:52.657062+0000","last_became_peered":"2024-03-22T04:11:52.657062+0000","last_unstale":"2024-03-22T04:11:52.667894+0000","last_undegraded":"2024-03-22T04:11:52.667894+0000","last_fullsized":"2024-03-22T04:11:52.667894+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:47.563475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.656938+0000","last_change":"2024-03-22T04:11:52.652956+0000","last_active":"2024-03-22T04:11:52.656938+0000","last_peered":"2024-03-22T04:11:52.656938+0000","last_clean":"2024-03-22T04:11:52.656938+0000","last_became_active":"2024-03-22T04:11:52.652568+0000","last_became_peered":"2024-03-22T04:11:52.652568+0000","last_unstale":"2024-03-22T04:11:52.656938+0000","last_undegraded":"2024-03-22T04:11:52.656938+0000","last_fullsized":"2024-03-22T04:11:52.656938+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:35:18.455069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667915+0000","last_change":"2024-03-22T04:11:52.658632+0000","last_active":"2024-03-22T04:11:52.667915+0000","last_peered":"2024-03-22T04:11:52.667915+0000","last_clean":"2024-03-22T04:11:52.667915+0000","last_became_active":"2024-03-22T04:11:52.657426+0000","last_became_peered":"2024-03-22T04:11:52.657426+0000","last_unstale":"2024-03-22T04:11:52.667915+0000","last_undegraded":"2024-03-22T04:11:52.667915+0000","last_fullsized":"2024-03-22T04:11:52.667915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:59:50.201070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613235+0000","last_change":"2024-03-22T04:11:50.613235+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613235+0000","last_undegraded":"2024-03-22T04:11:50.613235+0000","last_fullsized":"2024-03-22T04:11:50.613235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615708+0000","last_change":"2024-03-22T04:11:50.615708+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615708+0000","last_undegraded":"2024-03-22T04:11:50.615708+0000","last_fullsized":"2024-03-22T04:11:50.615708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.679524+0000","last_change":"2024-03-22T04:11:51.636087+0000","last_active":"2024-03-22T04:11:52.679524+0000","last_peered":"2024-03-22T04:11:52.679524+0000","last_clean":"2024-03-22T04:11:52.679524+0000","last_became_active":"2024-03-22T04:11:51.635794+0000","last_became_peered":"2024-03-22T04:11:51.635794+0000","last_unstale":"2024-03-22T04:11:52.679524+0000","last_undegraded":"2024-03-22T04:11:52.679524+0000","last_fullsized":"2024-03-22T04:11:52.679524+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:29.018276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613272+0000","last_change":"2024-03-22T04:11:50.613272+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613272+0000","last_undegraded":"2024-03-22T04:11:50.613272+0000","last_fullsized":"2024-03-22T04:11:50.613272+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667952+0000","last_change":"2024-03-22T04:11:52.659045+0000","last_active":"2024-03-22T04:11:52.667952+0000","last_peered":"2024-03-22T04:11:52.667952+0000","last_clean":"2024-03-22T04:11:52.667952+0000","last_became_active":"2024-03-22T04:11:52.658165+0000","last_became_peered":"2024-03-22T04:11:52.658165+0000","last_unstale":"2024-03-22T04:11:52.667952+0000","last_undegraded":"2024-03-22T04:11:52.667952+0000","last_fullsized":"2024-03-22T04:11:52.667952+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:12.332645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613232+0000","last_change":"2024-03-22T04:11:50.613232+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613232+0000","last_undegraded":"2024-03-22T04:11:50.613232+0000","last_fullsized":"2024-03-22T04:11:50.613232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615507+0000","last_change":"2024-03-22T04:11:50.615507+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615507+0000","last_undegraded":"2024-03-22T04:11:50.615507+0000","last_fullsized":"2024-03-22T04:11:50.615507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.638844+0000","last_change":"2024-03-22T04:11:51.645291+0000","last_active":"2024-03-22T04:11:52.638844+0000","last_peered":"2024-03-22T04:11:52.638844+0000","last_clean":"2024-03-22T04:11:52.638844+0000","last_became_active":"2024-03-22T04:11:51.644994+0000","last_became_peered":"2024-03-22T04:11:51.644994+0000","last_unstale":"2024-03-22T04:11:52.638844+0000","last_undegraded":"2024-03-22T04:11:52.638844+0000","last_fullsized":"2024-03-22T04:11:52.638844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:36:32.547421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615494+0000","last_change":"2024-03-22T04:11:50.615494+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615494+0000","last_undegraded":"2024-03-22T04:11:50.615494+0000","last_fullsized":"2024-03-22T04:11:50.615494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.657919+0000","last_change":"2024-03-22T04:11:52.652881+0000","last_active":"2024-03-22T04:11:52.657919+0000","last_peered":"2024-03-22T04:11:52.657919+0000","last_clean":"2024-03-22T04:11:52.657919+0000","last_became_active":"2024-03-22T04:11:52.652190+0000","last_became_peered":"2024-03-22T04:11:52.652190+0000","last_unstale":"2024-03-22T04:11:52.657919+0000","last_undegraded":"2024-03-22T04:11:52.657919+0000","last_fullsized":"2024-03-22T04:11:52.657919+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:08:26.601087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639424+0000","last_change":"2024-03-22T04:11:51.635044+0000","last_active":"2024-03-22T04:11:52.639424+0000","last_peered":"2024-03-22T04:11:52.639424+0000","last_clean":"2024-03-22T04:11:52.639424+0000","last_became_active":"2024-03-22T04:11:51.633837+0000","last_became_peered":"2024-03-22T04:11:51.633837+0000","last_unstale":"2024-03-22T04:11:52.639424+0000","last_undegraded":"2024-03-22T04:11:52.639424+0000","last_fullsized":"2024-03-22T04:11:52.639424+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:31:32.200080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619235+0000","last_change":"2024-03-22T04:11:50.619235+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619235+0000","last_undegraded":"2024-03-22T04:11:50.619235+0000","last_fullsized":"2024-03-22T04:11:50.619235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.665977+0000","last_change":"2024-03-22T04:11:52.656994+0000","last_active":"2024-03-22T04:11:52.665977+0000","last_peered":"2024-03-22T04:11:52.665977+0000","last_clean":"2024-03-22T04:11:52.665977+0000","last_became_active":"2024-03-22T04:11:52.656688+0000","last_became_peered":"2024-03-22T04:11:52.656688+0000","last_unstale":"2024-03-22T04:11:52.665977+0000","last_undegraded":"2024-03-22T04:11:52.665977+0000","last_fullsized":"2024-03-22T04:11:52.665977+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:18:30.069733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.620324+0000","last_change":"2024-03-22T04:11:50.620324+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.620324+0000","last_undegraded":"2024-03-22T04:11:50.620324+0000","last_fullsized":"2024-03-22T04:11:50.620324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.681884+0000","last_change":"2024-03-22T04:11:51.644715+0000","last_active":"2024-03-22T04:11:52.681884+0000","last_peered":"2024-03-22T04:11:52.681884+0000","last_clean":"2024-03-22T04:11:52.681884+0000","last_became_active":"2024-03-22T04:11:51.644365+0000","last_became_peered":"2024-03-22T04:11:51.644365+0000","last_unstale":"2024-03-22T04:11:52.681884+0000","last_undegraded":"2024-03-22T04:11:52.681884+0000","last_fullsized":"2024-03-22T04:11:52.681884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:49:10.202707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.656068+0000","last_change":"2024-03-22T04:11:52.646801+0000","last_active":"2024-03-22T04:11:52.656068+0000","last_peered":"2024-03-22T04:11:52.656068+0000","last_clean":"2024-03-22T04:11:52.656068+0000","last_became_active":"2024-03-22T04:11:52.646491+0000","last_became_peered":"2024-03-22T04:11:52.646491+0000","last_unstale":"2024-03-22T04:11:52.656068+0000","last_undegraded":"2024-03-22T04:11:52.656068+0000","last_fullsized":"2024-03-22T04:11:52.656068+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:44:29.455556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.679634+0000","last_change":"2024-03-22T04:11:51.635750+0000","last_active":"2024-03-22T04:11:52.679634+0000","last_peered":"2024-03-22T04:11:52.679634+0000","last_clean":"2024-03-22T04:11:52.679634+0000","last_became_active":"2024-03-22T04:11:51.634686+0000","last_became_peered":"2024-03-22T04:11:51.634686+0000","last_unstale":"2024-03-22T04:11:52.679634+0000","last_undegraded":"2024-03-22T04:11:52.679634+0000","last_fullsized":"2024-03-22T04:11:52.679634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:56:45.221222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616261+0000","last_change":"2024-03-22T04:11:50.616261+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616261+0000","last_undegraded":"2024-03-22T04:11:50.616261+0000","last_fullsized":"2024-03-22T04:11:50.616261+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619979+0000","last_change":"2024-03-22T04:11:50.619979+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619979+0000","last_undegraded":"2024-03-22T04:11:50.619979+0000","last_fullsized":"2024-03-22T04:11:50.619979+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613507+0000","last_change":"2024-03-22T04:11:50.613507+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613507+0000","last_undegraded":"2024-03-22T04:11:50.613507+0000","last_fullsized":"2024-03-22T04:11:50.613507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615586+0000","last_change":"2024-03-22T04:11:50.615586+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615586+0000","last_undegraded":"2024-03-22T04:11:50.615586+0000","last_fullsized":"2024-03-22T04:11:50.615586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618788+0000","last_change":"2024-03-22T04:11:50.618788+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618788+0000","last_undegraded":"2024-03-22T04:11:50.618788+0000","last_fullsized":"2024-03-22T04:11:50.618788+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616139+0000","last_change":"2024-03-22T04:11:50.616139+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616139+0000","last_undegraded":"2024-03-22T04:11:50.616139+0000","last_fullsized":"2024-03-22T04:11:50.616139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617590+0000","last_change":"2024-03-22T04:11:50.617590+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617590+0000","last_undegraded":"2024-03-22T04:11:50.617590+0000","last_fullsized":"2024-03-22T04:11:50.617590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618135+0000","last_change":"2024-03-22T04:11:50.618135+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618135+0000","last_undegraded":"2024-03-22T04:11:50.618135+0000","last_fullsized":"2024-03-22T04:11:50.618135+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.666825+0000","last_change":"2024-03-22T04:11:52.657565+0000","last_active":"2024-03-22T04:11:52.666825+0000","last_peered":"2024-03-22T04:11:52.666825+0000","last_clean":"2024-03-22T04:11:52.666825+0000","last_became_active":"2024-03-22T04:11:52.657329+0000","last_became_peered":"2024-03-22T04:11:52.657329+0000","last_unstale":"2024-03-22T04:11:52.666825+0000","last_undegraded":"2024-03-22T04:11:52.666825+0000","last_fullsized":"2024-03-22T04:11:52.666825+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:51.727928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616872+0000","last_change":"2024-03-22T04:11:50.616872+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616872+0000","last_undegraded":"2024-03-22T04:11:50.616872+0000","last_fullsized":"2024-03-22T04:11:50.616872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615564+0000","last_change":"2024-03-22T04:11:50.615564+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615564+0000","last_undegraded":"2024-03-22T04:11:50.615564+0000","last_fullsized":"2024-03-22T04:11:50.615564+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667509+0000","last_change":"2024-03-22T04:11:52.657915+0000","last_active":"2024-03-22T04:11:52.667509+0000","last_peered":"2024-03-22T04:11:52.667509+0000","last_clean":"2024-03-22T04:11:52.667509+0000","last_became_active":"2024-03-22T04:11:52.657728+0000","last_became_peered":"2024-03-22T04:11:52.657728+0000","last_unstale":"2024-03-22T04:11:52.667509+0000","last_undegraded":"2024-03-22T04:11:52.667509+0000","last_fullsized":"2024-03-22T04:11:52.667509+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:53:28.532936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615718+0000","last_change":"2024-03-22T04:11:50.615718+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615718+0000","last_undegraded":"2024-03-22T04:11:50.615718+0000","last_fullsized":"2024-03-22T04:11:50.615718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618931+0000","last_change":"2024-03-22T04:11:50.618931+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618931+0000","last_undegraded":"2024-03-22T04:11:50.618931+0000","last_fullsized":"2024-03-22T04:11:50.618931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.614675+0000","last_change":"2024-03-22T04:11:50.614675+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.614675+0000","last_undegraded":"2024-03-22T04:11:50.614675+0000","last_fullsized":"2024-03-22T04:11:50.614675+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615612+0000","last_change":"2024-03-22T04:11:50.615612+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615612+0000","last_undegraded":"2024-03-22T04:11:50.615612+0000","last_fullsized":"2024-03-22T04:11:50.615612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639215+0000","last_change":"2024-03-22T04:11:51.635806+0000","last_active":"2024-03-22T04:11:52.639215+0000","last_peered":"2024-03-22T04:11:52.639215+0000","last_clean":"2024-03-22T04:11:52.639215+0000","last_became_active":"2024-03-22T04:11:51.633614+0000","last_became_peered":"2024-03-22T04:11:51.633614+0000","last_unstale":"2024-03-22T04:11:52.639215+0000","last_undegraded":"2024-03-22T04:11:52.639215+0000","last_fullsized":"2024-03-22T04:11:52.639215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:14.289312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617268+0000","last_change":"2024-03-22T04:11:50.617268+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617268+0000","last_undegraded":"2024-03-22T04:11:50.617268+0000","last_fullsized":"2024-03-22T04:11:50.617268+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619612+0000","last_change":"2024-03-22T04:11:50.619612+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619612+0000","last_undegraded":"2024-03-22T04:11:50.619612+0000","last_fullsized":"2024-03-22T04:11:50.619612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639907+0000","last_change":"2024-03-22T04:11:51.641613+0000","last_active":"2024-03-22T04:11:52.639907+0000","last_peered":"2024-03-22T04:11:52.639907+0000","last_clean":"2024-03-22T04:11:52.639907+0000","last_became_active":"2024-03-22T04:11:51.641256+0000","last_became_peered":"2024-03-22T04:11:51.641256+0000","last_unstale":"2024-03-22T04:11:52.639907+0000","last_undegraded":"2024-03-22T04:11:52.639907+0000","last_fullsized":"2024-03-22T04:11:52.639907+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:27.877295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617293+0000","last_change":"2024-03-22T04:11:50.617293+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617293+0000","last_undegraded":"2024-03-22T04:11:50.617293+0000","last_fullsized":"2024-03-22T04:11:50.617293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-22T04:11:50.615841+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:11:50.615841+0000","last_peered":"2024-03-22T04:11:50.615841+0000","last_clean":"2024-03-22T04:11:50.615841+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:11:50.615841+0000","last_undegraded":"2024-03-22T04:11:50.615841+0000","last_fullsized":"2024-03-22T04:11:50.615841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1011712,"data_stored":763484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":763484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":194895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:11:57.098 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:11:57.203 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:11:57.206 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:11:57.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.266+0000 7efcc5800640 1 Processor -- start 2024-03-22T04:11:57.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.266+0000 7efcc5800640 1 -- start start 2024-03-22T04:11:57.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcc5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efcc0158200 0x7efcc0071d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcc5800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efcc006e430 0x7efcc00cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcc5800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc015bc10 con 0x7efcc0072250 2024-03-22T04:11:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcc5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efcc00726b0 con 0x7efcc0158200 2024-03-22T04:11:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcc5800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efcc006e9a0 con 0x7efcc006e430 2024-03-22T04:11:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcb6000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efcc0072250 0x7efcc006dd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40368/0 (socket says 172.21.15.86:40368) 2024-03-22T04:11:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcb6000640 1 -- 172.21.15.86:0/3971394029 learned_addr learned my addr 172.21.15.86:0/3971394029 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcbea00640 1 --2- 172.21.15.86:0/3971394029 >> v2:172.21.15.86:3300/0 conn(0x7efcc0158200 0x7efcc0071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcbf400640 1 --2- 172.21.15.86:0/3971394029 >> v2:172.21.15.196:3300/0 conn(0x7efcc006e430 0x7efcc00cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcbe000640 1 -- 172.21.15.86:0/3971394029 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375456954 0 0) 0x7efcc015bc10 con 0x7efcc0072250 2024-03-22T04:11:57.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcbe000640 1 -- 172.21.15.86:0/3971394029 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efca4003530 con 0x7efcc0072250 2024-03-22T04:11:57.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcbea00640 1 -- 172.21.15.86:0/3971394029 >> v2:172.21.15.196:3300/0 conn(0x7efcc006e430 msgr2=0x7efcc00cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:11:57.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcbea00640 1 --2- 172.21.15.86:0/3971394029 >> v2:172.21.15.196:3300/0 conn(0x7efcc006e430 0x7efcc00cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcbea00640 1 -- 172.21.15.86:0/3971394029 >> v1:172.21.15.86:6789/0 conn(0x7efcc0072250 legacy=0x7efcc006dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.270+0000 7efcbea00640 1 -- 172.21.15.86:0/3971394029 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc006ec80 con 0x7efcc0158200 2024-03-22T04:11:57.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcbea00640 1 --2- 172.21.15.86:0/3971394029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0158200 0x7efcc0071d10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efcc015bb90 tx=0x7efcb001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5b4d8cd51650c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:57.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcbe000640 1 -- 172.21.15.86:0/3971394029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00b2070 con 0x7efcc0158200 2024-03-22T04:11:57.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcbe000640 1 -- 172.21.15.86:0/3971394029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb00a5740 con 0x7efcc0158200 2024-03-22T04:11:57.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcbe000640 1 -- 172.21.15.86:0/3971394029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00a5b10 con 0x7efcc0158200 2024-03-22T04:11:57.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 -- 172.21.15.86:0/3971394029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0158200 msgr2=0x7efcc0071d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 --2- 172.21.15.86:0/3971394029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0158200 0x7efcc0071d10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efcc015bb90 tx=0x7efcb001c8a0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 -- 172.21.15.86:0/3971394029 shutdown_connections 2024-03-22T04:11:57.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 --2- 172.21.15.86:0/3971394029 >> v2:172.21.15.196:3300/0 conn(0x7efcc006e430 0x7efcc00cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 --2- 172.21.15.86:0/3971394029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0158200 0x7efcc0071d10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 -- 172.21.15.86:0/3971394029 >> 172.21.15.86:0/3971394029 conn(0x7efcc006b660 msgr2=0x7efcc006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:57.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 -- 172.21.15.86:0/3971394029 shutdown_connections 2024-03-22T04:11:57.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 -- 172.21.15.86:0/3971394029 wait complete. 2024-03-22T04:11:57.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 Processor -- start 2024-03-22T04:11:57.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.274+0000 7efcc5800640 1 -- start start 2024-03-22T04:11:57.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcc006e430 0x7efcc014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0072250 0x7efcc0151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0158200 0x7efcc01558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcc00cffb0 con 0x7efcc0072250 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efcc00cfe30 con 0x7efcc006e430 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efcc00d0130 con 0x7efcc0158200 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0072250 0x7efcc0151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0072250 0x7efcc0151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36714/0 (socket says 172.21.15.86:36714) 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcbea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcc006e430 0x7efcc014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 -- 172.21.15.86:0/3343071283 learned_addr learned my addr 172.21.15.86:0/3343071283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcbf400640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0158200 0x7efcc01558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 -- 172.21.15.86:0/3343071283 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0158200 msgr2=0x7efcc01558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0158200 0x7efcc01558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 -- 172.21.15.86:0/3343071283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcc006e430 msgr2=0x7efcc014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcc006e430 0x7efcc014f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 -- 172.21.15.86:0/3343071283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc01553c0 con 0x7efcc0072250 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb6000640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0072250 0x7efcc0151470 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efca800c3f0 tx=0x7efca800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb7400640 1 -- 172.21.15.86:0/3343071283 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8017070 con 0x7efcc0072250 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb7400640 1 -- 172.21.15.86:0/3343071283 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca8005cf0 con 0x7efcc0072250 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc014f970 con 0x7efcc0072250 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb7400640 1 -- 172.21.15.86:0/3343071283 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80072f0 con 0x7efcc0072250 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc014ff30 con 0x7efcc0072250 2024-03-22T04:11:57.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc0155590 con 0x7efcc0072250 2024-03-22T04:11:57.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.278+0000 7efcb7400640 1 -- 172.21.15.86:0/3343071283 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efca801a070 con 0x7efcc0072250 2024-03-22T04:11:57.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.282+0000 7efcb7400640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc9c067b20 0x7efc9c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.282+0000 7efcb7400640 1 -- 172.21.15.86:0/3343071283 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7efca8085540 con 0x7efcc0072250 2024-03-22T04:11:57.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.282+0000 7efcbea00640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc9c067b20 0x7efc9c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.282+0000 7efcbea00640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc9c067b20 0x7efc9c069fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efcb00af000 tx=0x7efcb00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:57.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.286+0000 7efcb7400640 1 -- 172.21.15.86:0/3343071283 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efca80556d0 con 0x7efcc0072250 2024-03-22T04:11:57.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.442+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efcc00cce20 con 0x7efc9c067b20 2024-03-22T04:11:57.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.450+0000 7efcb7400640 1 -- 172.21.15.86:0/3343071283 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242394 (secure 0 0 0) 0x7efcc00cce20 con 0x7efc9c067b20 2024-03-22T04:11:57.457 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.450+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc9c067b20 msgr2=0x7efc9c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.450+0000 7efcc5800640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc9c067b20 0x7efc9c069fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efcb00af000 tx=0x7efcb00a7070 comp rx=0 tx=0).stop 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0072250 msgr2=0x7efcc0151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0072250 0x7efcc0151470 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efca800c3f0 tx=0x7efca800c8b0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 shutdown_connections 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc9c067b20 0x7efc9c069fe0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0158200 0x7efcc01558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcc0072250 0x7efcc0151470 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 --2- 172.21.15.86:0/3343071283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcc006e430 0x7efcc014f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 >> 172.21.15.86:0/3343071283 conn(0x7efcc006b660 msgr2=0x7efcc015afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 shutdown_connections 2024-03-22T04:11:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.454+0000 7efcc5800640 1 -- 172.21.15.86:0/3343071283 wait complete. 2024-03-22T04:11:57.477 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-03-22T04:11:55.808113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":92,"acting":92,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220696,"kb_used_data":4504,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754024,"statfs":{"total":773094113280,"available":772868120576,"internally_reserved":0,"allocated":4612096,"data_stored":2599861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.197155"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.669236+0000","last_change":"2024-03-22T04:11:52.662881+0000","last_active":"2024-03-22T04:11:52.669236+0000","last_peered":"2024-03-22T04:11:52.669236+0000","last_clean":"2024-03-22T04:11:52.669236+0000","last_became_active":"2024-03-22T04:11:52.662454+0000","last_became_peered":"2024-03-22T04:11:52.662454+0000","last_unstale":"2024-03-22T04:11:52.669236+0000","last_undegraded":"2024-03-22T04:11:52.669236+0000","last_fullsized":"2024-03-22T04:11:52.669236+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:23.935409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.669282+0000","last_change":"2024-03-22T04:11:52.662047+0000","last_active":"2024-03-22T04:11:52.669282+0000","last_peered":"2024-03-22T04:11:52.669282+0000","last_clean":"2024-03-22T04:11:52.669282+0000","last_became_active":"2024-03-22T04:11:52.661357+0000","last_became_peered":"2024-03-22T04:11:52.661357+0000","last_unstale":"2024-03-22T04:11:52.669282+0000","last_undegraded":"2024-03-22T04:11:52.669282+0000","last_fullsized":"2024-03-22T04:11:52.669282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:15:57.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.663438+0000","last_change":"2024-03-22T04:11:52.655700+0000","last_active":"2024-03-22T04:11:52.663438+0000","last_peered":"2024-03-22T04:11:52.663438+0000","last_clean":"2024-03-22T04:11:52.663438+0000","last_became_active":"2024-03-22T04:11:52.655442+0000","last_became_peered":"2024-03-22T04:11:52.655442+0000","last_unstale":"2024-03-22T04:11:52.663438+0000","last_undegraded":"2024-03-22T04:11:52.663438+0000","last_fullsized":"2024-03-22T04:11:52.663438+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:19:04.849981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667934+0000","last_change":"2024-03-22T04:11:52.658845+0000","last_active":"2024-03-22T04:11:52.667934+0000","last_peered":"2024-03-22T04:11:52.667934+0000","last_clean":"2024-03-22T04:11:52.667934+0000","last_became_active":"2024-03-22T04:11:52.657778+0000","last_became_peered":"2024-03-22T04:11:52.657778+0000","last_unstale":"2024-03-22T04:11:52.667934+0000","last_undegraded":"2024-03-22T04:11:52.667934+0000","last_fullsized":"2024-03-22T04:11:52.667934+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:46:39.954425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667894+0000","last_change":"2024-03-22T04:11:52.658420+0000","last_active":"2024-03-22T04:11:52.667894+0000","last_peered":"2024-03-22T04:11:52.667894+0000","last_clean":"2024-03-22T04:11:52.667894+0000","last_became_active":"2024-03-22T04:11:52.657062+0000","last_became_peered":"2024-03-22T04:11:52.657062+0000","last_unstale":"2024-03-22T04:11:52.667894+0000","last_undegraded":"2024-03-22T04:11:52.667894+0000","last_fullsized":"2024-03-22T04:11:52.667894+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:47.563475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.656938+0000","last_change":"2024-03-22T04:11:52.652956+0000","last_active":"2024-03-22T04:11:52.656938+0000","last_peered":"2024-03-22T04:11:52.656938+0000","last_clean":"2024-03-22T04:11:52.656938+0000","last_became_active":"2024-03-22T04:11:52.652568+0000","last_became_peered":"2024-03-22T04:11:52.652568+0000","last_unstale":"2024-03-22T04:11:52.656938+0000","last_undegraded":"2024-03-22T04:11:52.656938+0000","last_fullsized":"2024-03-22T04:11:52.656938+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:35:18.455069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667915+0000","last_change":"2024-03-22T04:11:52.658632+0000","last_active":"2024-03-22T04:11:52.667915+0000","last_peered":"2024-03-22T04:11:52.667915+0000","last_clean":"2024-03-22T04:11:52.667915+0000","last_became_active":"2024-03-22T04:11:52.657426+0000","last_became_peered":"2024-03-22T04:11:52.657426+0000","last_unstale":"2024-03-22T04:11:52.667915+0000","last_undegraded":"2024-03-22T04:11:52.667915+0000","last_fullsized":"2024-03-22T04:11:52.667915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:59:50.201070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613235+0000","last_change":"2024-03-22T04:11:50.613235+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613235+0000","last_undegraded":"2024-03-22T04:11:50.613235+0000","last_fullsized":"2024-03-22T04:11:50.613235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615708+0000","last_change":"2024-03-22T04:11:50.615708+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615708+0000","last_undegraded":"2024-03-22T04:11:50.615708+0000","last_fullsized":"2024-03-22T04:11:50.615708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.679524+0000","last_change":"2024-03-22T04:11:51.636087+0000","last_active":"2024-03-22T04:11:52.679524+0000","last_peered":"2024-03-22T04:11:52.679524+0000","last_clean":"2024-03-22T04:11:52.679524+0000","last_became_active":"2024-03-22T04:11:51.635794+0000","last_became_peered":"2024-03-22T04:11:51.635794+0000","last_unstale":"2024-03-22T04:11:52.679524+0000","last_undegraded":"2024-03-22T04:11:52.679524+0000","last_fullsized":"2024-03-22T04:11:52.679524+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:29.018276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613272+0000","last_change":"2024-03-22T04:11:50.613272+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613272+0000","last_undegraded":"2024-03-22T04:11:50.613272+0000","last_fullsized":"2024-03-22T04:11:50.613272+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667952+0000","last_change":"2024-03-22T04:11:52.659045+0000","last_active":"2024-03-22T04:11:52.667952+0000","last_peered":"2024-03-22T04:11:52.667952+0000","last_clean":"2024-03-22T04:11:52.667952+0000","last_became_active":"2024-03-22T04:11:52.658165+0000","last_became_peered":"2024-03-22T04:11:52.658165+0000","last_unstale":"2024-03-22T04:11:52.667952+0000","last_undegraded":"2024-03-22T04:11:52.667952+0000","last_fullsized":"2024-03-22T04:11:52.667952+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:12.332645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613232+0000","last_change":"2024-03-22T04:11:50.613232+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613232+0000","last_undegraded":"2024-03-22T04:11:50.613232+0000","last_fullsized":"2024-03-22T04:11:50.613232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615507+0000","last_change":"2024-03-22T04:11:50.615507+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615507+0000","last_undegraded":"2024-03-22T04:11:50.615507+0000","last_fullsized":"2024-03-22T04:11:50.615507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.638844+0000","last_change":"2024-03-22T04:11:51.645291+0000","last_active":"2024-03-22T04:11:52.638844+0000","last_peered":"2024-03-22T04:11:52.638844+0000","last_clean":"2024-03-22T04:11:52.638844+0000","last_became_active":"2024-03-22T04:11:51.644994+0000","last_became_peered":"2024-03-22T04:11:51.644994+0000","last_unstale":"2024-03-22T04:11:52.638844+0000","last_undegraded":"2024-03-22T04:11:52.638844+0000","last_fullsized":"2024-03-22T04:11:52.638844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:36:32.547421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615494+0000","last_change":"2024-03-22T04:11:50.615494+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615494+0000","last_undegraded":"2024-03-22T04:11:50.615494+0000","last_fullsized":"2024-03-22T04:11:50.615494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.657919+0000","last_change":"2024-03-22T04:11:52.652881+0000","last_active":"2024-03-22T04:11:52.657919+0000","last_peered":"2024-03-22T04:11:52.657919+0000","last_clean":"2024-03-22T04:11:52.657919+0000","last_became_active":"2024-03-22T04:11:52.652190+0000","last_became_peered":"2024-03-22T04:11:52.652190+0000","last_unstale":"2024-03-22T04:11:52.657919+0000","last_undegraded":"2024-03-22T04:11:52.657919+0000","last_fullsized":"2024-03-22T04:11:52.657919+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:08:26.601087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639424+0000","last_change":"2024-03-22T04:11:51.635044+0000","last_active":"2024-03-22T04:11:52.639424+0000","last_peered":"2024-03-22T04:11:52.639424+0000","last_clean":"2024-03-22T04:11:52.639424+0000","last_became_active":"2024-03-22T04:11:51.633837+0000","last_became_peered":"2024-03-22T04:11:51.633837+0000","last_unstale":"2024-03-22T04:11:52.639424+0000","last_undegraded":"2024-03-22T04:11:52.639424+0000","last_fullsized":"2024-03-22T04:11:52.639424+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:31:32.200080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619235+0000","last_change":"2024-03-22T04:11:50.619235+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619235+0000","last_undegraded":"2024-03-22T04:11:50.619235+0000","last_fullsized":"2024-03-22T04:11:50.619235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.665977+0000","last_change":"2024-03-22T04:11:52.656994+0000","last_active":"2024-03-22T04:11:52.665977+0000","last_peered":"2024-03-22T04:11:52.665977+0000","last_clean":"2024-03-22T04:11:52.665977+0000","last_became_active":"2024-03-22T04:11:52.656688+0000","last_became_peered":"2024-03-22T04:11:52.656688+0000","last_unstale":"2024-03-22T04:11:52.665977+0000","last_undegraded":"2024-03-22T04:11:52.665977+0000","last_fullsized":"2024-03-22T04:11:52.665977+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:18:30.069733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.620324+0000","last_change":"2024-03-22T04:11:50.620324+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.620324+0000","last_undegraded":"2024-03-22T04:11:50.620324+0000","last_fullsized":"2024-03-22T04:11:50.620324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.681884+0000","last_change":"2024-03-22T04:11:51.644715+0000","last_active":"2024-03-22T04:11:52.681884+0000","last_peered":"2024-03-22T04:11:52.681884+0000","last_clean":"2024-03-22T04:11:52.681884+0000","last_became_active":"2024-03-22T04:11:51.644365+0000","last_became_peered":"2024-03-22T04:11:51.644365+0000","last_unstale":"2024-03-22T04:11:52.681884+0000","last_undegraded":"2024-03-22T04:11:52.681884+0000","last_fullsized":"2024-03-22T04:11:52.681884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:49:10.202707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.656068+0000","last_change":"2024-03-22T04:11:52.646801+0000","last_active":"2024-03-22T04:11:52.656068+0000","last_peered":"2024-03-22T04:11:52.656068+0000","last_clean":"2024-03-22T04:11:52.656068+0000","last_became_active":"2024-03-22T04:11:52.646491+0000","last_became_peered":"2024-03-22T04:11:52.646491+0000","last_unstale":"2024-03-22T04:11:52.656068+0000","last_undegraded":"2024-03-22T04:11:52.656068+0000","last_fullsized":"2024-03-22T04:11:52.656068+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:44:29.455556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.679634+0000","last_change":"2024-03-22T04:11:51.635750+0000","last_active":"2024-03-22T04:11:52.679634+0000","last_peered":"2024-03-22T04:11:52.679634+0000","last_clean":"2024-03-22T04:11:52.679634+0000","last_became_active":"2024-03-22T04:11:51.634686+0000","last_became_peered":"2024-03-22T04:11:51.634686+0000","last_unstale":"2024-03-22T04:11:52.679634+0000","last_undegraded":"2024-03-22T04:11:52.679634+0000","last_fullsized":"2024-03-22T04:11:52.679634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:56:45.221222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616261+0000","last_change":"2024-03-22T04:11:50.616261+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616261+0000","last_undegraded":"2024-03-22T04:11:50.616261+0000","last_fullsized":"2024-03-22T04:11:50.616261+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619979+0000","last_change":"2024-03-22T04:11:50.619979+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619979+0000","last_undegraded":"2024-03-22T04:11:50.619979+0000","last_fullsized":"2024-03-22T04:11:50.619979+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.613507+0000","last_change":"2024-03-22T04:11:50.613507+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.613507+0000","last_undegraded":"2024-03-22T04:11:50.613507+0000","last_fullsized":"2024-03-22T04:11:50.613507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615586+0000","last_change":"2024-03-22T04:11:50.615586+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615586+0000","last_undegraded":"2024-03-22T04:11:50.615586+0000","last_fullsized":"2024-03-22T04:11:50.615586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618788+0000","last_change":"2024-03-22T04:11:50.618788+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618788+0000","last_undegraded":"2024-03-22T04:11:50.618788+0000","last_fullsized":"2024-03-22T04:11:50.618788+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616139+0000","last_change":"2024-03-22T04:11:50.616139+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616139+0000","last_undegraded":"2024-03-22T04:11:50.616139+0000","last_fullsized":"2024-03-22T04:11:50.616139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617590+0000","last_change":"2024-03-22T04:11:50.617590+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617590+0000","last_undegraded":"2024-03-22T04:11:50.617590+0000","last_fullsized":"2024-03-22T04:11:50.617590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618135+0000","last_change":"2024-03-22T04:11:50.618135+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618135+0000","last_undegraded":"2024-03-22T04:11:50.618135+0000","last_fullsized":"2024-03-22T04:11:50.618135+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.666825+0000","last_change":"2024-03-22T04:11:52.657565+0000","last_active":"2024-03-22T04:11:52.666825+0000","last_peered":"2024-03-22T04:11:52.666825+0000","last_clean":"2024-03-22T04:11:52.666825+0000","last_became_active":"2024-03-22T04:11:52.657329+0000","last_became_peered":"2024-03-22T04:11:52.657329+0000","last_unstale":"2024-03-22T04:11:52.666825+0000","last_undegraded":"2024-03-22T04:11:52.666825+0000","last_fullsized":"2024-03-22T04:11:52.666825+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:51.727928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.616872+0000","last_change":"2024-03-22T04:11:50.616872+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.616872+0000","last_undegraded":"2024-03-22T04:11:50.616872+0000","last_fullsized":"2024-03-22T04:11:50.616872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615564+0000","last_change":"2024-03-22T04:11:50.615564+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615564+0000","last_undegraded":"2024-03-22T04:11:50.615564+0000","last_fullsized":"2024-03-22T04:11:50.615564+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667509+0000","last_change":"2024-03-22T04:11:52.657915+0000","last_active":"2024-03-22T04:11:52.667509+0000","last_peered":"2024-03-22T04:11:52.667509+0000","last_clean":"2024-03-22T04:11:52.667509+0000","last_became_active":"2024-03-22T04:11:52.657728+0000","last_became_peered":"2024-03-22T04:11:52.657728+0000","last_unstale":"2024-03-22T04:11:52.667509+0000","last_undegraded":"2024-03-22T04:11:52.667509+0000","last_fullsized":"2024-03-22T04:11:52.667509+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:53:28.532936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615718+0000","last_change":"2024-03-22T04:11:50.615718+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615718+0000","last_undegraded":"2024-03-22T04:11:50.615718+0000","last_fullsized":"2024-03-22T04:11:50.615718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.618931+0000","last_change":"2024-03-22T04:11:50.618931+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.618931+0000","last_undegraded":"2024-03-22T04:11:50.618931+0000","last_fullsized":"2024-03-22T04:11:50.618931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.614675+0000","last_change":"2024-03-22T04:11:50.614675+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.614675+0000","last_undegraded":"2024-03-22T04:11:50.614675+0000","last_fullsized":"2024-03-22T04:11:50.614675+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.615612+0000","last_change":"2024-03-22T04:11:50.615612+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.615612+0000","last_undegraded":"2024-03-22T04:11:50.615612+0000","last_fullsized":"2024-03-22T04:11:50.615612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639215+0000","last_change":"2024-03-22T04:11:51.635806+0000","last_active":"2024-03-22T04:11:52.639215+0000","last_peered":"2024-03-22T04:11:52.639215+0000","last_clean":"2024-03-22T04:11:52.639215+0000","last_became_active":"2024-03-22T04:11:51.633614+0000","last_became_peered":"2024-03-22T04:11:51.633614+0000","last_unstale":"2024-03-22T04:11:52.639215+0000","last_undegraded":"2024-03-22T04:11:52.639215+0000","last_fullsized":"2024-03-22T04:11:52.639215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:14.289312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617268+0000","last_change":"2024-03-22T04:11:50.617268+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617268+0000","last_undegraded":"2024-03-22T04:11:50.617268+0000","last_fullsized":"2024-03-22T04:11:50.617268+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.619612+0000","last_change":"2024-03-22T04:11:50.619612+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.619612+0000","last_undegraded":"2024-03-22T04:11:50.619612+0000","last_fullsized":"2024-03-22T04:11:50.619612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639907+0000","last_change":"2024-03-22T04:11:51.641613+0000","last_active":"2024-03-22T04:11:52.639907+0000","last_peered":"2024-03-22T04:11:52.639907+0000","last_clean":"2024-03-22T04:11:52.639907+0000","last_became_active":"2024-03-22T04:11:51.641256+0000","last_became_peered":"2024-03-22T04:11:51.641256+0000","last_unstale":"2024-03-22T04:11:52.639907+0000","last_undegraded":"2024-03-22T04:11:52.639907+0000","last_fullsized":"2024-03-22T04:11:52.639907+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:27.877295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-22T04:11:51.610479+0000","last_change":"2024-03-22T04:11:51.610479+0000","last_active":"2024-03-22T04:11:51.610479+0000","last_peered":"2024-03-22T04:11:51.610479+0000","last_clean":"2024-03-22T04:11:51.610479+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:51.610479+0000","last_undegraded":"2024-03-22T04:11:51.610479+0000","last_fullsized":"2024-03-22T04:11:51.610479+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-22T04:11:50.617293+0000","last_change":"2024-03-22T04:11:50.617293+0000","last_active":"2024-03-22T04:11:50.603350+0000","last_peered":"2024-03-22T04:11:50.603350+0000","last_clean":"2024-03-22T04:11:50.603350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:11:50.617293+0000","last_undegraded":"2024-03-22T04:11:50.617293+0000","last_fullsized":"2024-03-22T04:11:50.617293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-22T04:11:50.615841+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:11:50.615841+0000","last_peered":"2024-03-22T04:11:50.615841+0000","last_clean":"2024-03-22T04:11:50.615841+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:11:50.615841+0000","last_undegraded":"2024-03-22T04:11:50.615841+0000","last_fullsized":"2024-03-22T04:11:50.615841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1011712,"data_stored":763484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":173116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":763484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":194895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:11:57.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-03-22T04:11:57.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-03-22T04:11:57.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-03-22T04:11:57.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-03-22T04:11:57.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-03-22T04:11:57.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-03-22T04:11:57.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-03-22T04:11:57.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-03-22T04:11:57.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-03-22T04:11:57.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-03-22T04:11:57.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.613235+0000', 'last_change': '2024-03-22T04:11:50.613235+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.613235+0000', 'last_undegraded': '2024-03-22T04:11:50.613235+0000', 'last_fullsized': '2024-03-22T04:11:50.613235+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-03-22T04:11:57.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.615708+0000', 'last_change': '2024-03-22T04:11:50.615708+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.615708+0000', 'last_undegraded': '2024-03-22T04:11:50.615708+0000', 'last_fullsized': '2024-03-22T04:11:50.615708+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.613272+0000', 'last_change': '2024-03-22T04:11:50.613272+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.613272+0000', 'last_undegraded': '2024-03-22T04:11:50.613272+0000', 'last_fullsized': '2024-03-22T04:11:50.613272+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.613232+0000', 'last_change': '2024-03-22T04:11:50.613232+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.613232+0000', 'last_undegraded': '2024-03-22T04:11:50.613232+0000', 'last_fullsized': '2024-03-22T04:11:50.613232+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.615507+0000', 'last_change': '2024-03-22T04:11:50.615507+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.615507+0000', 'last_undegraded': '2024-03-22T04:11:50.615507+0000', 'last_fullsized': '2024-03-22T04:11:50.615507+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.615494+0000', 'last_change': '2024-03-22T04:11:50.615494+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.615494+0000', 'last_undegraded': '2024-03-22T04:11:50.615494+0000', 'last_fullsized': '2024-03-22T04:11:50.615494+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-03-22T04:11:57.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.619235+0000', 'last_change': '2024-03-22T04:11:50.619235+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.619235+0000', 'last_undegraded': '2024-03-22T04:11:50.619235+0000', 'last_fullsized': '2024-03-22T04:11:50.619235+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.620324+0000', 'last_change': '2024-03-22T04:11:50.620324+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.620324+0000', 'last_undegraded': '2024-03-22T04:11:50.620324+0000', 'last_fullsized': '2024-03-22T04:11:50.620324+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.616261+0000', 'last_change': '2024-03-22T04:11:50.616261+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.616261+0000', 'last_undegraded': '2024-03-22T04:11:50.616261+0000', 'last_fullsized': '2024-03-22T04:11:50.616261+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.619979+0000', 'last_change': '2024-03-22T04:11:50.619979+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.619979+0000', 'last_undegraded': '2024-03-22T04:11:50.619979+0000', 'last_fullsized': '2024-03-22T04:11:50.619979+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.613507+0000', 'last_change': '2024-03-22T04:11:50.613507+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.613507+0000', 'last_undegraded': '2024-03-22T04:11:50.613507+0000', 'last_fullsized': '2024-03-22T04:11:50.613507+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.615586+0000', 'last_change': '2024-03-22T04:11:50.615586+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.615586+0000', 'last_undegraded': '2024-03-22T04:11:50.615586+0000', 'last_fullsized': '2024-03-22T04:11:50.615586+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-03-22T04:11:57.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.618788+0000', 'last_change': '2024-03-22T04:11:50.618788+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.618788+0000', 'last_undegraded': '2024-03-22T04:11:50.618788+0000', 'last_fullsized': '2024-03-22T04:11:50.618788+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.616139+0000', 'last_change': '2024-03-22T04:11:50.616139+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.616139+0000', 'last_undegraded': '2024-03-22T04:11:50.616139+0000', 'last_fullsized': '2024-03-22T04:11:50.616139+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.617590+0000', 'last_change': '2024-03-22T04:11:50.617590+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.617590+0000', 'last_undegraded': '2024-03-22T04:11:50.617590+0000', 'last_fullsized': '2024-03-22T04:11:50.617590+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-03-22T04:11:57.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.618135+0000', 'last_change': '2024-03-22T04:11:50.618135+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.618135+0000', 'last_undegraded': '2024-03-22T04:11:50.618135+0000', 'last_fullsized': '2024-03-22T04:11:50.618135+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.616872+0000', 'last_change': '2024-03-22T04:11:50.616872+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.616872+0000', 'last_undegraded': '2024-03-22T04:11:50.616872+0000', 'last_fullsized': '2024-03-22T04:11:50.616872+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.615564+0000', 'last_change': '2024-03-22T04:11:50.615564+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.615564+0000', 'last_undegraded': '2024-03-22T04:11:50.615564+0000', 'last_fullsized': '2024-03-22T04:11:50.615564+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.615718+0000', 'last_change': '2024-03-22T04:11:50.615718+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.615718+0000', 'last_undegraded': '2024-03-22T04:11:50.615718+0000', 'last_fullsized': '2024-03-22T04:11:50.615718+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-03-22T04:11:57.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.618931+0000', 'last_change': '2024-03-22T04:11:50.618931+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.618931+0000', 'last_undegraded': '2024-03-22T04:11:50.618931+0000', 'last_fullsized': '2024-03-22T04:11:50.618931+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.614675+0000', 'last_change': '2024-03-22T04:11:50.614675+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.614675+0000', 'last_undegraded': '2024-03-22T04:11:50.614675+0000', 'last_fullsized': '2024-03-22T04:11:50.614675+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.615612+0000', 'last_change': '2024-03-22T04:11:50.615612+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.615612+0000', 'last_undegraded': '2024-03-22T04:11:50.615612+0000', 'last_fullsized': '2024-03-22T04:11:50.615612+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-03-22T04:11:57.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.617268+0000', 'last_change': '2024-03-22T04:11:50.617268+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.617268+0000', 'last_undegraded': '2024-03-22T04:11:50.617268+0000', 'last_fullsized': '2024-03-22T04:11:50.617268+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.619612+0000', 'last_change': '2024-03-22T04:11:50.619612+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.619612+0000', 'last_undegraded': '2024-03-22T04:11:50.619612+0000', 'last_fullsized': '2024-03-22T04:11:50.619612+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-03-22T04:11:57.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-03-22T04:11:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-22T04:11:51.610479+0000', 'last_change': '2024-03-22T04:11:51.610479+0000', 'last_active': '2024-03-22T04:11:51.610479+0000', 'last_peered': '2024-03-22T04:11:51.610479+0000', 'last_clean': '2024-03-22T04:11:51.610479+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:51.610479+0000', 'last_undegraded': '2024-03-22T04:11:51.610479+0000', 'last_fullsized': '2024-03-22T04:11:51.610479+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:51.610479+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-22T04:11:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-03-22T04:11:57.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:11:50.617293+0000', 'last_change': '2024-03-22T04:11:50.617293+0000', 'last_active': '2024-03-22T04:11:50.603350+0000', 'last_peered': '2024-03-22T04:11:50.603350+0000', 'last_clean': '2024-03-22T04:11:50.603350+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:11:50.617293+0000', 'last_undegraded': '2024-03-22T04:11:50.617293+0000', 'last_fullsized': '2024-03-22T04:11:50.617293+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'last_clean_scrub_stamp': '2024-03-22T04:11:50.603350+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:11:57.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-22T04:11:57.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d5ba00640 1 Processor -- start 2024-03-22T04:11:57.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d5ba00640 1 -- start start 2024-03-22T04:11:57.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d5ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415ecb0 0x7f9d5406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d5ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d54072f90 con 0x7f9d540add00 2024-03-22T04:11:57.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d5ba00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d540721d0 con 0x7f9d5415b380 2024-03-22T04:11:57.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d5ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d540724b0 con 0x7f9d5415ecb0 2024-03-22T04:11:57.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d53e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f9d5415b380 0x7f9d5415f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:36718/0 (socket says 172.21.15.86:36718) 2024-03-22T04:11:57.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d53e00640 1 -- 172.21.15.86:0/1956882613 learned_addr learned my addr 172.21.15.86:0/1956882613 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:57.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d59a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9d540add00 0x7f9d540ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40374/0 (socket says 172.21.15.86:40374) 2024-03-22T04:11:57.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d59000640 1 --2- 172.21.15.86:0/1956882613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415ecb0 0x7f9d5406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d59000640 1 -- 172.21.15.86:0/1956882613 >> v1:172.21.15.196:6789/0 conn(0x7f9d5415b380 legacy=0x7f9d5415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d59000640 1 -- 172.21.15.86:0/1956882613 >> v1:172.21.15.86:6789/0 conn(0x7f9d540add00 legacy=0x7f9d540ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d59000640 1 -- 172.21.15.86:0/1956882613 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d54072790 con 0x7f9d5415ecb0 2024-03-22T04:11:57.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.646+0000 7f9d53400640 1 -- 172.21.15.86:0/1956882613 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445324609 0 0) 0x7f9d540721d0 con 0x7f9d5415b380 2024-03-22T04:11:57.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d59000640 1 --2- 172.21.15.86:0/1956882613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415ecb0 0x7f9d5406d940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009dc0 tx=0x7f9d4401c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a333a197189faf3e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:57.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d53400640 1 -- 172.21.15.86:0/1956882613 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d440b2070 con 0x7f9d5415ecb0 2024-03-22T04:11:57.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d53400640 1 -- 172.21.15.86:0/1956882613 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d440a5880 con 0x7f9d5415ecb0 2024-03-22T04:11:57.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d53400640 1 -- 172.21.15.86:0/1956882613 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d440a5c50 con 0x7f9d5415ecb0 2024-03-22T04:11:57.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d5ba00640 1 -- 172.21.15.86:0/1956882613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415ecb0 msgr2=0x7f9d5406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d5ba00640 1 --2- 172.21.15.86:0/1956882613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415ecb0 0x7f9d5406d940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009dc0 tx=0x7f9d4401c820 comp rx=0 tx=0).stop 2024-03-22T04:11:57.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d5ba00640 1 -- 172.21.15.86:0/1956882613 shutdown_connections 2024-03-22T04:11:57.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d5ba00640 1 --2- 172.21.15.86:0/1956882613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415ecb0 0x7f9d5406d940 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d5ba00640 1 -- 172.21.15.86:0/1956882613 >> 172.21.15.86:0/1956882613 conn(0x7f9d540b1c80 msgr2=0x7f9d540b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:57.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d5ba00640 1 -- 172.21.15.86:0/1956882613 shutdown_connections 2024-03-22T04:11:57.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.650+0000 7f9d5ba00640 1 -- 172.21.15.86:0/1956882613 wait complete. 2024-03-22T04:11:57.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 Processor -- start 2024-03-22T04:11:57.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 -- start start 2024-03-22T04:11:57.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d540add00 0x7f9d54146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415b380 0x7f9d541e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5415ecb0 0x7f9d541e8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d540ae8a0 con 0x7f9d540add00 2024-03-22T04:11:57.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d540b3f90 con 0x7f9d5415ecb0 2024-03-22T04:11:57.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d540aea20 con 0x7f9d5415b380 2024-03-22T04:11:57.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d540add00 0x7f9d54146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d540add00 0x7f9d54146b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36726/0 (socket says 172.21.15.86:36726) 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 -- 172.21.15.86:0/730669913 learned_addr learned my addr 172.21.15.86:0/730669913 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d53e00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415b380 0x7f9d541e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59a00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5415ecb0 0x7f9d541e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 -- 172.21.15.86:0/730669913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415b380 msgr2=0x7f9d541e65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415b380 0x7f9d541e65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 -- 172.21.15.86:0/730669913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5415ecb0 msgr2=0x7f9d541e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5415ecb0 0x7f9d541e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 -- 172.21.15.86:0/730669913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d541e70a0 con 0x7f9d540add00 2024-03-22T04:11:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d59000640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d540add00 0x7f9d54146b40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d4401cd00 tx=0x7f9d440195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:57.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d50c00640 1 -- 172.21.15.86:0/730669913 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d440b2040 con 0x7f9d540add00 2024-03-22T04:11:57.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d50c00640 1 -- 172.21.15.86:0/730669913 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d440a5680 con 0x7f9d540add00 2024-03-22T04:11:57.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d50c00640 1 -- 172.21.15.86:0/730669913 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d440add70 con 0x7f9d540add00 2024-03-22T04:11:57.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d541e7380 con 0x7f9d540add00 2024-03-22T04:11:57.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.654+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541e7940 con 0x7f9d540add00 2024-03-22T04:11:57.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.658+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d54074550 con 0x7f9d540add00 2024-03-22T04:11:57.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.658+0000 7f9d50c00640 1 -- 172.21.15.86:0/730669913 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9d440c1020 con 0x7f9d540add00 2024-03-22T04:11:57.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.658+0000 7f9d50c00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d34067b20 0x7f9d34069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:57.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.658+0000 7f9d50c00640 1 -- 172.21.15.86:0/730669913 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f9d44120e80 con 0x7f9d540add00 2024-03-22T04:11:57.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.658+0000 7f9d53e00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d34067b20 0x7f9d34069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:57.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.658+0000 7f9d53e00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d34067b20 0x7f9d34069fe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b0310 tx=0x7f9d40007200 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:57.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.662+0000 7f9d50c00640 1 -- 172.21.15.86:0/730669913 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9d440164a0 con 0x7f9d540add00 2024-03-22T04:11:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9d540d09b0 con 0x7f9d540add00 2024-03-22T04:11:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d50c00640 1 -- 172.21.15.86:0/730669913 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1175 (secure 0 0 0) 0x7f9d440f10d0 con 0x7f9d540add00 2024-03-22T04:11:57.879 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:57.879 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":108,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":8,"num_up_osds":8,"osd_up_since":1711080620,"num_in_osds":8,"osd_in_since":1711080611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":51},{"state_name":"creating+peering","count":25},{"state_name":"active+clean","count":21}],"num_pgs":97,"num_pools":3,"num_objects":9,"data_bytes":592014,"bytes_used":225992704,"bytes_avail":772868120576,"bytes_total":773094113280,"unknown_pgs_ratio":0.52577316761016846,"inactive_pgs_ratio":0.25773194432258606,"write_bytes_sec":1970,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":74930}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-03-22T04:11:39.805364+0000","services":{}},"progress_events":{}} 2024-03-22T04:11:57.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d34067b20 msgr2=0x7f9d34069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d34067b20 0x7f9d34069fe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b0310 tx=0x7f9d40007200 comp rx=0 tx=0).stop 2024-03-22T04:11:57.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d540add00 msgr2=0x7f9d54146b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:57.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d540add00 0x7f9d54146b40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d4401cd00 tx=0x7f9d440195c0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 shutdown_connections 2024-03-22T04:11:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d34067b20 0x7f9d34069fe0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5415ecb0 0x7f9d541e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5415b380 0x7f9d541e65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 --2- 172.21.15.86:0/730669913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d540add00 0x7f9d54146b40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 >> 172.21.15.86:0/730669913 conn(0x7f9d540b1c80 msgr2=0x7f9d540b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 shutdown_connections 2024-03-22T04:11:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:57.874+0000 7f9d5ba00640 1 -- 172.21.15.86:0/730669913 wait complete. 2024-03-22T04:11:57.900 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-22T04:11:57.901 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:11:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f4744c00640 1 Processor -- start 2024-03-22T04:11:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f4744c00640 1 -- start start 2024-03-22T04:11:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f4744c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47400ad330 0x7f474015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f4744c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f474015c7c0 con 0x7f474006ef40 2024-03-22T04:11:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f4744c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f474006dea0 con 0x7f47400ad710 2024-03-22T04:11:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f4744c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f474006e180 con 0x7f47400ad330 2024-03-22T04:11:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f474006ef40 0x7f474013fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40386/0 (socket says 172.21.15.86:40386) 2024-03-22T04:11:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473d400640 1 -- 172.21.15.86:0/612600450 learned_addr learned my addr 172.21.15.86:0/612600450 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:11:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473de00640 1 --2- 172.21.15.86:0/612600450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47400ad330 0x7f474015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473ca00640 1 -- 172.21.15.86:0/612600450 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024030848 0 0) 0x7f474015c7c0 con 0x7f474006ef40 2024-03-22T04:11:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473ca00640 1 -- 172.21.15.86:0/612600450 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4724003530 con 0x7f474006ef40 2024-03-22T04:11:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473ca00640 1 -- 172.21.15.86:0/612600450 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325668299 0 0) 0x7f474006dea0 con 0x7f47400ad710 2024-03-22T04:11:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473de00640 1 -- 172.21.15.86:0/612600450 >> v1:172.21.15.196:6789/0 conn(0x7f47400ad710 legacy=0x7f47401402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473de00640 1 -- 172.21.15.86:0/612600450 >> v1:172.21.15.86:6789/0 conn(0x7f474006ef40 legacy=0x7f474013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.030+0000 7f473de00640 1 -- 172.21.15.86:0/612600450 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474006e460 con 0x7f47400ad330 2024-03-22T04:11:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f473de00640 1 --2- 172.21.15.86:0/612600450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47400ad330 0x7f474015d4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4730009dc0 tx=0x7f473001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a022d4181c9e55a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f473ca00640 1 -- 172.21.15.86:0/612600450 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47300b2070 con 0x7f47400ad330 2024-03-22T04:11:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f473ca00640 1 -- 172.21.15.86:0/612600450 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47300a5930 con 0x7f47400ad330 2024-03-22T04:11:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f473ca00640 1 -- 172.21.15.86:0/612600450 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47300a5d00 con 0x7f47400ad330 2024-03-22T04:11:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f4744c00640 1 -- 172.21.15.86:0/612600450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47400ad330 msgr2=0x7f474015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f4744c00640 1 --2- 172.21.15.86:0/612600450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47400ad330 0x7f474015d4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4730009dc0 tx=0x7f473001c960 comp rx=0 tx=0).stop 2024-03-22T04:11:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f4744c00640 1 -- 172.21.15.86:0/612600450 shutdown_connections 2024-03-22T04:11:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f4744c00640 1 --2- 172.21.15.86:0/612600450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47400ad330 0x7f474015d4a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f4744c00640 1 -- 172.21.15.86:0/612600450 >> 172.21.15.86:0/612600450 conn(0x7f47400b1df0 msgr2=0x7f47400b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f4744c00640 1 -- 172.21.15.86:0/612600450 shutdown_connections 2024-03-22T04:11:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.034+0000 7f4744c00640 1 -- 172.21.15.86:0/612600450 wait complete. 2024-03-22T04:11:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 Processor -- start 2024-03-22T04:11:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 -- start start 2024-03-22T04:11:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f474006ef40 0x7f474009dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47400ad330 0x7f47400b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47400ad710 0x7f4740153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f474016a780 con 0x7f47400ad710 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f474006db10 con 0x7f47400ad330 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f474016a900 con 0x7f474006ef40 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47400ad710 0x7f4740153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47400ad710 0x7f4740153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36732/0 (socket says 172.21.15.86:36732) 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 -- 172.21.15.86:0/3481169208 learned_addr learned my addr 172.21.15.86:0/3481169208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473d400640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47400ad330 0x7f47400b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473de00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f474006ef40 0x7f474009dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 -- 172.21.15.86:0/3481169208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f474006ef40 msgr2=0x7f474009dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f474006ef40 0x7f474009dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 -- 172.21.15.86:0/3481169208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47400ad330 msgr2=0x7f47400b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47400ad330 0x7f47400b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 -- 172.21.15.86:0/3481169208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47400cf5d0 con 0x7f47400ad710 2024-03-22T04:11:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f473e800640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47400ad710 0x7f4740153aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f473800c440 tx=0x7f473800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4736000640 1 -- 172.21.15.86:0/3481169208 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738017070 con 0x7f47400ad710 2024-03-22T04:11:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4736000640 1 -- 172.21.15.86:0/3481169208 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4738003600 con 0x7f47400ad710 2024-03-22T04:11:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4736000640 1 -- 172.21.15.86:0/3481169208 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47380039f0 con 0x7f47400ad710 2024-03-22T04:11:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474015c370 con 0x7f47400ad710 2024-03-22T04:11:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4740159540 con 0x7f47400ad710 2024-03-22T04:11:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.038+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4740068650 con 0x7f47400ad710 2024-03-22T04:11:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.042+0000 7f4736000640 1 -- 172.21.15.86:0/3481169208 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f473801a070 con 0x7f47400ad710 2024-03-22T04:11:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.042+0000 7f4736000640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4720067b50 0x7f472006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:11:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.042+0000 7f4736000640 1 -- 172.21.15.86:0/3481169208 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f4738085350 con 0x7f47400ad710 2024-03-22T04:11:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.042+0000 7f473de00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4720067b50 0x7f472006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:11:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.042+0000 7f473de00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4720067b50 0x7f472006a010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f47300af000 tx=0x7f47300a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:11:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.046+0000 7f4736000640 1 -- 172.21.15.86:0/3481169208 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4738055570 con 0x7f47400ad710 2024-03-22T04:11:58.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.198+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47400b4f70 con 0x7f4720067b50 2024-03-22T04:11:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.206+0000 7f4736000640 1 -- 172.21.15.86:0/3481169208 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251943 (secure 0 0 0) 0x7f4730003c80 con 0x7f4720067b50 2024-03-22T04:11:58.213 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:11:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:11:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4720067b50 msgr2=0x7f472006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4720067b50 0x7f472006a010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f47300af000 tx=0x7f47300a7070 comp rx=0 tx=0).stop 2024-03-22T04:11:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47400ad710 msgr2=0x7f4740153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:11:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47400ad710 0x7f4740153aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f473800c440 tx=0x7f473800c840 comp rx=0 tx=0).stop 2024-03-22T04:11:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 shutdown_connections 2024-03-22T04:11:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4720067b50 0x7f472006a010 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47400ad330 0x7f47400b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47400ad710 0x7f4740153aa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 --2- 172.21.15.86:0/3481169208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f474006ef40 0x7f474009dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:11:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 >> 172.21.15.86:0/3481169208 conn(0x7f47400b1df0 msgr2=0x7f47400b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:11:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 shutdown_connections 2024-03-22T04:11:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:11:58.210+0000 7f4744c00640 1 -- 172.21.15.86:0/3481169208 wait complete. 2024-03-22T04:11:58.234 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-03-22T04:11:57.808538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221052,"kb_used_data":4860,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753668,"statfs":{"total":773094113280,"available":772867756032,"internally_reserved":0,"allocated":4976640,"data_stored":2777672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.193682"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.669236+0000","last_change":"2024-03-22T04:11:52.662881+0000","last_active":"2024-03-22T04:11:52.669236+0000","last_peered":"2024-03-22T04:11:52.669236+0000","last_clean":"2024-03-22T04:11:52.669236+0000","last_became_active":"2024-03-22T04:11:52.662454+0000","last_became_peered":"2024-03-22T04:11:52.662454+0000","last_unstale":"2024-03-22T04:11:52.669236+0000","last_undegraded":"2024-03-22T04:11:52.669236+0000","last_fullsized":"2024-03-22T04:11:52.669236+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:23.935409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656654+0000","last_change":"2024-03-22T04:11:52.659739+0000","last_active":"2024-03-22T04:11:53.656654+0000","last_peered":"2024-03-22T04:11:53.656654+0000","last_clean":"2024-03-22T04:11:53.656654+0000","last_became_active":"2024-03-22T04:11:52.659450+0000","last_became_peered":"2024-03-22T04:11:52.659450+0000","last_unstale":"2024-03-22T04:11:53.656654+0000","last_undegraded":"2024-03-22T04:11:53.656654+0000","last_fullsized":"2024-03-22T04:11:53.656654+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.600899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.669282+0000","last_change":"2024-03-22T04:11:52.662047+0000","last_active":"2024-03-22T04:11:52.669282+0000","last_peered":"2024-03-22T04:11:52.669282+0000","last_clean":"2024-03-22T04:11:52.669282+0000","last_became_active":"2024-03-22T04:11:52.661357+0000","last_became_peered":"2024-03-22T04:11:52.661357+0000","last_unstale":"2024-03-22T04:11:52.669282+0000","last_undegraded":"2024-03-22T04:11:52.669282+0000","last_fullsized":"2024-03-22T04:11:52.669282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:15:57.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.663438+0000","last_change":"2024-03-22T04:11:52.655700+0000","last_active":"2024-03-22T04:11:52.663438+0000","last_peered":"2024-03-22T04:11:52.663438+0000","last_clean":"2024-03-22T04:11:52.663438+0000","last_became_active":"2024-03-22T04:11:52.655442+0000","last_became_peered":"2024-03-22T04:11:52.655442+0000","last_unstale":"2024-03-22T04:11:52.663438+0000","last_undegraded":"2024-03-22T04:11:52.663438+0000","last_fullsized":"2024-03-22T04:11:52.663438+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:19:04.849981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655387+0000","last_change":"2024-03-22T04:11:52.665671+0000","last_active":"2024-03-22T04:11:53.655387+0000","last_peered":"2024-03-22T04:11:53.655387+0000","last_clean":"2024-03-22T04:11:53.655387+0000","last_became_active":"2024-03-22T04:11:52.665384+0000","last_became_peered":"2024-03-22T04:11:52.665384+0000","last_unstale":"2024-03-22T04:11:53.655387+0000","last_undegraded":"2024-03-22T04:11:53.655387+0000","last_fullsized":"2024-03-22T04:11:53.655387+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:23:00.057749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656097+0000","last_change":"2024-03-22T04:11:52.653999+0000","last_active":"2024-03-22T04:11:53.656097+0000","last_peered":"2024-03-22T04:11:53.656097+0000","last_clean":"2024-03-22T04:11:53.656097+0000","last_became_active":"2024-03-22T04:11:52.653699+0000","last_became_peered":"2024-03-22T04:11:52.653699+0000","last_unstale":"2024-03-22T04:11:53.656097+0000","last_undegraded":"2024-03-22T04:11:53.656097+0000","last_fullsized":"2024-03-22T04:11:53.656097+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:47.474967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131452+0000","last_change":"2024-03-22T04:11:52.666658+0000","last_active":"2024-03-22T04:11:54.131452+0000","last_peered":"2024-03-22T04:11:54.131452+0000","last_clean":"2024-03-22T04:11:54.131452+0000","last_became_active":"2024-03-22T04:11:52.666408+0000","last_became_peered":"2024-03-22T04:11:52.666408+0000","last_unstale":"2024-03-22T04:11:54.131452+0000","last_undegraded":"2024-03-22T04:11:54.131452+0000","last_fullsized":"2024-03-22T04:11:54.131452+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:23:00.057749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667934+0000","last_change":"2024-03-22T04:11:52.658845+0000","last_active":"2024-03-22T04:11:52.667934+0000","last_peered":"2024-03-22T04:11:52.667934+0000","last_clean":"2024-03-22T04:11:52.667934+0000","last_became_active":"2024-03-22T04:11:52.657778+0000","last_became_peered":"2024-03-22T04:11:52.657778+0000","last_unstale":"2024-03-22T04:11:52.667934+0000","last_undegraded":"2024-03-22T04:11:52.667934+0000","last_fullsized":"2024-03-22T04:11:52.667934+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:46:39.954425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656254+0000","last_change":"2024-03-22T04:11:52.665877+0000","last_active":"2024-03-22T04:11:53.656254+0000","last_peered":"2024-03-22T04:11:53.656254+0000","last_clean":"2024-03-22T04:11:53.656254+0000","last_became_active":"2024-03-22T04:11:52.665559+0000","last_became_peered":"2024-03-22T04:11:52.665559+0000","last_unstale":"2024-03-22T04:11:53.656254+0000","last_undegraded":"2024-03-22T04:11:53.656254+0000","last_fullsized":"2024-03-22T04:11:53.656254+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:48:52.128428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131943+0000","last_change":"2024-03-22T04:11:52.652350+0000","last_active":"2024-03-22T04:11:54.131943+0000","last_peered":"2024-03-22T04:11:54.131943+0000","last_clean":"2024-03-22T04:11:54.131943+0000","last_became_active":"2024-03-22T04:11:52.651964+0000","last_became_peered":"2024-03-22T04:11:52.651964+0000","last_unstale":"2024-03-22T04:11:54.131943+0000","last_undegraded":"2024-03-22T04:11:54.131943+0000","last_fullsized":"2024-03-22T04:11:54.131943+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:23.206659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661723+0000","last_change":"2024-03-22T04:11:52.656438+0000","last_active":"2024-03-22T04:11:53.661723+0000","last_peered":"2024-03-22T04:11:53.661723+0000","last_clean":"2024-03-22T04:11:53.661723+0000","last_became_active":"2024-03-22T04:11:52.655889+0000","last_became_peered":"2024-03-22T04:11:52.655889+0000","last_unstale":"2024-03-22T04:11:53.661723+0000","last_undegraded":"2024-03-22T04:11:53.661723+0000","last_fullsized":"2024-03-22T04:11:53.661723+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:45:11.033321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655653+0000","last_change":"2024-03-22T04:11:52.661339+0000","last_active":"2024-03-22T04:11:53.655653+0000","last_peered":"2024-03-22T04:11:53.655653+0000","last_clean":"2024-03-22T04:11:53.655653+0000","last_became_active":"2024-03-22T04:11:52.660933+0000","last_became_peered":"2024-03-22T04:11:52.660933+0000","last_unstale":"2024-03-22T04:11:53.655653+0000","last_undegraded":"2024-03-22T04:11:53.655653+0000","last_fullsized":"2024-03-22T04:11:53.655653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:47:59.233121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131386+0000","last_change":"2024-03-22T04:11:52.648731+0000","last_active":"2024-03-22T04:11:54.131386+0000","last_peered":"2024-03-22T04:11:54.131386+0000","last_clean":"2024-03-22T04:11:54.131386+0000","last_became_active":"2024-03-22T04:11:52.648451+0000","last_became_peered":"2024-03-22T04:11:52.648451+0000","last_unstale":"2024-03-22T04:11:54.131386+0000","last_undegraded":"2024-03-22T04:11:54.131386+0000","last_fullsized":"2024-03-22T04:11:54.131386+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:10:16.282670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131091+0000","last_change":"2024-03-22T04:11:52.645044+0000","last_active":"2024-03-22T04:11:54.131091+0000","last_peered":"2024-03-22T04:11:54.131091+0000","last_clean":"2024-03-22T04:11:54.131091+0000","last_became_active":"2024-03-22T04:11:52.644788+0000","last_became_peered":"2024-03-22T04:11:52.644788+0000","last_unstale":"2024-03-22T04:11:54.131091+0000","last_undegraded":"2024-03-22T04:11:54.131091+0000","last_fullsized":"2024-03-22T04:11:54.131091+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:15:20.514843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653534+0000","last_change":"2024-03-22T04:11:52.646944+0000","last_active":"2024-03-22T04:11:53.653534+0000","last_peered":"2024-03-22T04:11:53.653534+0000","last_clean":"2024-03-22T04:11:53.653534+0000","last_became_active":"2024-03-22T04:11:52.646677+0000","last_became_peered":"2024-03-22T04:11:52.646677+0000","last_unstale":"2024-03-22T04:11:53.653534+0000","last_undegraded":"2024-03-22T04:11:53.653534+0000","last_fullsized":"2024-03-22T04:11:53.653534+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:55:53.302385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131789+0000","last_change":"2024-03-22T04:11:52.665504+0000","last_active":"2024-03-22T04:11:54.131789+0000","last_peered":"2024-03-22T04:11:54.131789+0000","last_clean":"2024-03-22T04:11:54.131789+0000","last_became_active":"2024-03-22T04:11:52.665186+0000","last_became_peered":"2024-03-22T04:11:52.665186+0000","last_unstale":"2024-03-22T04:11:54.131789+0000","last_undegraded":"2024-03-22T04:11:54.131789+0000","last_fullsized":"2024-03-22T04:11:54.131789+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:48:52.128428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.654017+0000","last_change":"2024-03-22T04:11:52.658480+0000","last_active":"2024-03-22T04:11:53.654017+0000","last_peered":"2024-03-22T04:11:53.654017+0000","last_clean":"2024-03-22T04:11:53.654017+0000","last_became_active":"2024-03-22T04:11:52.658213+0000","last_became_peered":"2024-03-22T04:11:52.658213+0000","last_unstale":"2024-03-22T04:11:53.654017+0000","last_undegraded":"2024-03-22T04:11:53.654017+0000","last_fullsized":"2024-03-22T04:11:53.654017+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:07.174862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653915+0000","last_change":"2024-03-22T04:11:52.658663+0000","last_active":"2024-03-22T04:11:53.653915+0000","last_peered":"2024-03-22T04:11:53.653915+0000","last_clean":"2024-03-22T04:11:53.653915+0000","last_became_active":"2024-03-22T04:11:52.658389+0000","last_became_peered":"2024-03-22T04:11:52.658389+0000","last_unstale":"2024-03-22T04:11:53.653915+0000","last_undegraded":"2024-03-22T04:11:53.653915+0000","last_fullsized":"2024-03-22T04:11:53.653915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:01:24.998993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132674+0000","last_change":"2024-03-22T04:11:52.657515+0000","last_active":"2024-03-22T04:11:54.132674+0000","last_peered":"2024-03-22T04:11:54.132674+0000","last_clean":"2024-03-22T04:11:54.132674+0000","last_became_active":"2024-03-22T04:11:52.657199+0000","last_became_peered":"2024-03-22T04:11:52.657199+0000","last_unstale":"2024-03-22T04:11:54.132674+0000","last_undegraded":"2024-03-22T04:11:54.132674+0000","last_fullsized":"2024-03-22T04:11:54.132674+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:15:03.000496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661434+0000","last_change":"2024-03-22T04:11:52.659905+0000","last_active":"2024-03-22T04:11:53.661434+0000","last_peered":"2024-03-22T04:11:53.661434+0000","last_clean":"2024-03-22T04:11:53.661434+0000","last_became_active":"2024-03-22T04:11:52.659333+0000","last_became_peered":"2024-03-22T04:11:52.659333+0000","last_unstale":"2024-03-22T04:11:53.661434+0000","last_undegraded":"2024-03-22T04:11:53.661434+0000","last_fullsized":"2024-03-22T04:11:53.661434+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:37:01.810257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.662140+0000","last_change":"2024-03-22T04:11:52.663400+0000","last_active":"2024-03-22T04:11:53.662140+0000","last_peered":"2024-03-22T04:11:53.662140+0000","last_clean":"2024-03-22T04:11:53.662140+0000","last_became_active":"2024-03-22T04:11:52.662511+0000","last_became_peered":"2024-03-22T04:11:52.662511+0000","last_unstale":"2024-03-22T04:11:53.662140+0000","last_undegraded":"2024-03-22T04:11:53.662140+0000","last_fullsized":"2024-03-22T04:11:53.662140+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:43:40.916261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661358+0000","last_change":"2024-03-22T04:11:52.664517+0000","last_active":"2024-03-22T04:11:53.661358+0000","last_peered":"2024-03-22T04:11:53.661358+0000","last_clean":"2024-03-22T04:11:53.661358+0000","last_became_active":"2024-03-22T04:11:52.664245+0000","last_became_peered":"2024-03-22T04:11:52.664245+0000","last_unstale":"2024-03-22T04:11:53.661358+0000","last_undegraded":"2024-03-22T04:11:53.661358+0000","last_fullsized":"2024-03-22T04:11:53.661358+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:07:26.114489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653590+0000","last_change":"2024-03-22T04:11:52.655515+0000","last_active":"2024-03-22T04:11:53.653590+0000","last_peered":"2024-03-22T04:11:53.653590+0000","last_clean":"2024-03-22T04:11:53.653590+0000","last_became_active":"2024-03-22T04:11:52.655189+0000","last_became_peered":"2024-03-22T04:11:52.655189+0000","last_unstale":"2024-03-22T04:11:53.653590+0000","last_undegraded":"2024-03-22T04:11:53.653590+0000","last_fullsized":"2024-03-22T04:11:53.653590+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:16.270410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667894+0000","last_change":"2024-03-22T04:11:52.658420+0000","last_active":"2024-03-22T04:11:52.667894+0000","last_peered":"2024-03-22T04:11:52.667894+0000","last_clean":"2024-03-22T04:11:52.667894+0000","last_became_active":"2024-03-22T04:11:52.657062+0000","last_became_peered":"2024-03-22T04:11:52.657062+0000","last_unstale":"2024-03-22T04:11:52.667894+0000","last_undegraded":"2024-03-22T04:11:52.667894+0000","last_fullsized":"2024-03-22T04:11:52.667894+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:47.563475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131795+0000","last_change":"2024-03-22T04:11:52.650350+0000","last_active":"2024-03-22T04:11:54.131795+0000","last_peered":"2024-03-22T04:11:54.131795+0000","last_clean":"2024-03-22T04:11:54.131795+0000","last_became_active":"2024-03-22T04:11:52.650085+0000","last_became_peered":"2024-03-22T04:11:52.650085+0000","last_unstale":"2024-03-22T04:11:54.131795+0000","last_undegraded":"2024-03-22T04:11:54.131795+0000","last_fullsized":"2024-03-22T04:11:54.131795+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:03:05.060737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.656938+0000","last_change":"2024-03-22T04:11:52.652956+0000","last_active":"2024-03-22T04:11:52.656938+0000","last_peered":"2024-03-22T04:11:52.656938+0000","last_clean":"2024-03-22T04:11:52.656938+0000","last_became_active":"2024-03-22T04:11:52.652568+0000","last_became_peered":"2024-03-22T04:11:52.652568+0000","last_unstale":"2024-03-22T04:11:52.656938+0000","last_undegraded":"2024-03-22T04:11:52.656938+0000","last_fullsized":"2024-03-22T04:11:52.656938+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:35:18.455069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661860+0000","last_change":"2024-03-22T04:11:52.665778+0000","last_active":"2024-03-22T04:11:53.661860+0000","last_peered":"2024-03-22T04:11:53.661860+0000","last_clean":"2024-03-22T04:11:53.661860+0000","last_became_active":"2024-03-22T04:11:52.665516+0000","last_became_peered":"2024-03-22T04:11:52.665516+0000","last_unstale":"2024-03-22T04:11:53.661860+0000","last_undegraded":"2024-03-22T04:11:53.661860+0000","last_fullsized":"2024-03-22T04:11:53.661860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:13:33.572221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131655+0000","last_change":"2024-03-22T04:11:52.652480+0000","last_active":"2024-03-22T04:11:54.131655+0000","last_peered":"2024-03-22T04:11:54.131655+0000","last_clean":"2024-03-22T04:11:54.131655+0000","last_became_active":"2024-03-22T04:11:52.652170+0000","last_became_peered":"2024-03-22T04:11:52.652170+0000","last_unstale":"2024-03-22T04:11:54.131655+0000","last_undegraded":"2024-03-22T04:11:54.131655+0000","last_fullsized":"2024-03-22T04:11:54.131655+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:47:59.233121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653702+0000","last_change":"2024-03-22T04:11:52.656760+0000","last_active":"2024-03-22T04:11:53.653702+0000","last_peered":"2024-03-22T04:11:53.653702+0000","last_clean":"2024-03-22T04:11:53.653702+0000","last_became_active":"2024-03-22T04:11:52.654908+0000","last_became_peered":"2024-03-22T04:11:52.654908+0000","last_unstale":"2024-03-22T04:11:53.653702+0000","last_undegraded":"2024-03-22T04:11:53.653702+0000","last_fullsized":"2024-03-22T04:11:53.653702+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:45:59.503409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653629+0000","last_change":"2024-03-22T04:11:52.655520+0000","last_active":"2024-03-22T04:11:53.653629+0000","last_peered":"2024-03-22T04:11:53.653629+0000","last_clean":"2024-03-22T04:11:53.653629+0000","last_became_active":"2024-03-22T04:11:52.655233+0000","last_became_peered":"2024-03-22T04:11:52.655233+0000","last_unstale":"2024-03-22T04:11:53.653629+0000","last_undegraded":"2024-03-22T04:11:53.653629+0000","last_fullsized":"2024-03-22T04:11:53.653629+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:35:20.592890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131533+0000","last_change":"2024-03-22T04:11:52.650033+0000","last_active":"2024-03-22T04:11:54.131533+0000","last_peered":"2024-03-22T04:11:54.131533+0000","last_clean":"2024-03-22T04:11:54.131533+0000","last_became_active":"2024-03-22T04:11:52.648288+0000","last_became_peered":"2024-03-22T04:11:52.648288+0000","last_unstale":"2024-03-22T04:11:54.131533+0000","last_undegraded":"2024-03-22T04:11:54.131533+0000","last_fullsized":"2024-03-22T04:11:54.131533+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:48:59.705079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667915+0000","last_change":"2024-03-22T04:11:52.658632+0000","last_active":"2024-03-22T04:11:52.667915+0000","last_peered":"2024-03-22T04:11:52.667915+0000","last_clean":"2024-03-22T04:11:52.667915+0000","last_became_active":"2024-03-22T04:11:52.657426+0000","last_became_peered":"2024-03-22T04:11:52.657426+0000","last_unstale":"2024-03-22T04:11:52.667915+0000","last_undegraded":"2024-03-22T04:11:52.667915+0000","last_fullsized":"2024-03-22T04:11:52.667915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:59:50.201070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653910+0000","last_change":"2024-03-22T04:11:52.646488+0000","last_active":"2024-03-22T04:11:53.653910+0000","last_peered":"2024-03-22T04:11:53.653910+0000","last_clean":"2024-03-22T04:11:53.653910+0000","last_became_active":"2024-03-22T04:11:52.646297+0000","last_became_peered":"2024-03-22T04:11:52.646297+0000","last_unstale":"2024-03-22T04:11:53.653910+0000","last_undegraded":"2024-03-22T04:11:53.653910+0000","last_fullsized":"2024-03-22T04:11:53.653910+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:54:16.199170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131668+0000","last_change":"2024-03-22T04:11:51.636676+0000","last_active":"2024-03-22T04:11:54.131668+0000","last_peered":"2024-03-22T04:11:54.131668+0000","last_clean":"2024-03-22T04:11:54.131668+0000","last_became_active":"2024-03-22T04:11:51.636389+0000","last_became_peered":"2024-03-22T04:11:51.636389+0000","last_unstale":"2024-03-22T04:11:54.131668+0000","last_undegraded":"2024-03-22T04:11:54.131668+0000","last_fullsized":"2024-03-22T04:11:54.131668+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:48:58.697949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656600+0000","last_change":"2024-03-22T04:11:52.663434+0000","last_active":"2024-03-22T04:11:53.656600+0000","last_peered":"2024-03-22T04:11:53.656600+0000","last_clean":"2024-03-22T04:11:53.656600+0000","last_became_active":"2024-03-22T04:11:52.663176+0000","last_became_peered":"2024-03-22T04:11:52.663176+0000","last_unstale":"2024-03-22T04:11:53.656600+0000","last_undegraded":"2024-03-22T04:11:53.656600+0000","last_fullsized":"2024-03-22T04:11:53.656600+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:15:03.000496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656772+0000","last_change":"2024-03-22T04:11:51.630512+0000","last_active":"2024-03-22T04:11:53.656772+0000","last_peered":"2024-03-22T04:11:53.656772+0000","last_clean":"2024-03-22T04:11:53.656772+0000","last_became_active":"2024-03-22T04:11:51.628986+0000","last_became_peered":"2024-03-22T04:11:51.628986+0000","last_unstale":"2024-03-22T04:11:53.656772+0000","last_undegraded":"2024-03-22T04:11:53.656772+0000","last_fullsized":"2024-03-22T04:11:53.656772+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:47:58.225991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.654082+0000","last_change":"2024-03-22T04:11:52.645360+0000","last_active":"2024-03-22T04:11:53.654082+0000","last_peered":"2024-03-22T04:11:53.654082+0000","last_clean":"2024-03-22T04:11:53.654082+0000","last_became_active":"2024-03-22T04:11:52.645030+0000","last_became_peered":"2024-03-22T04:11:52.645030+0000","last_unstale":"2024-03-22T04:11:53.654082+0000","last_undegraded":"2024-03-22T04:11:53.654082+0000","last_fullsized":"2024-03-22T04:11:53.654082+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:50:44.542135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.679524+0000","last_change":"2024-03-22T04:11:51.636087+0000","last_active":"2024-03-22T04:11:52.679524+0000","last_peered":"2024-03-22T04:11:52.679524+0000","last_clean":"2024-03-22T04:11:52.679524+0000","last_became_active":"2024-03-22T04:11:51.635794+0000","last_became_peered":"2024-03-22T04:11:51.635794+0000","last_unstale":"2024-03-22T04:11:52.679524+0000","last_undegraded":"2024-03-22T04:11:52.679524+0000","last_fullsized":"2024-03-22T04:11:52.679524+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:29.018276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656657+0000","last_change":"2024-03-22T04:11:52.662695+0000","last_active":"2024-03-22T04:11:53.656657+0000","last_peered":"2024-03-22T04:11:53.656657+0000","last_clean":"2024-03-22T04:11:53.656657+0000","last_became_active":"2024-03-22T04:11:52.662338+0000","last_became_peered":"2024-03-22T04:11:52.662338+0000","last_unstale":"2024-03-22T04:11:53.656657+0000","last_undegraded":"2024-03-22T04:11:53.656657+0000","last_fullsized":"2024-03-22T04:11:53.656657+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:45:14.997775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131226+0000","last_change":"2024-03-22T04:11:51.641684+0000","last_active":"2024-03-22T04:11:54.131226+0000","last_peered":"2024-03-22T04:11:54.131226+0000","last_clean":"2024-03-22T04:11:54.131226+0000","last_became_active":"2024-03-22T04:11:51.641520+0000","last_became_peered":"2024-03-22T04:11:51.641520+0000","last_unstale":"2024-03-22T04:11:54.131226+0000","last_undegraded":"2024-03-22T04:11:54.131226+0000","last_fullsized":"2024-03-22T04:11:54.131226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:47:58.225991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667952+0000","last_change":"2024-03-22T04:11:52.659045+0000","last_active":"2024-03-22T04:11:52.667952+0000","last_peered":"2024-03-22T04:11:52.667952+0000","last_clean":"2024-03-22T04:11:52.667952+0000","last_became_active":"2024-03-22T04:11:52.658165+0000","last_became_peered":"2024-03-22T04:11:52.658165+0000","last_unstale":"2024-03-22T04:11:52.667952+0000","last_undegraded":"2024-03-22T04:11:52.667952+0000","last_fullsized":"2024-03-22T04:11:52.667952+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:12.332645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131108+0000","last_change":"2024-03-22T04:11:51.640008+0000","last_active":"2024-03-22T04:11:54.131108+0000","last_peered":"2024-03-22T04:11:54.131108+0000","last_clean":"2024-03-22T04:11:54.131108+0000","last_became_active":"2024-03-22T04:11:51.637341+0000","last_became_peered":"2024-03-22T04:11:51.637341+0000","last_unstale":"2024-03-22T04:11:54.131108+0000","last_undegraded":"2024-03-22T04:11:54.131108+0000","last_fullsized":"2024-03-22T04:11:54.131108+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:10:15.275541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661326+0000","last_change":"2024-03-22T04:11:52.660384+0000","last_active":"2024-03-22T04:11:53.661326+0000","last_peered":"2024-03-22T04:11:53.661326+0000","last_clean":"2024-03-22T04:11:53.661326+0000","last_became_active":"2024-03-22T04:11:52.660042+0000","last_became_peered":"2024-03-22T04:11:52.660042+0000","last_unstale":"2024-03-22T04:11:53.661326+0000","last_undegraded":"2024-03-22T04:11:53.661326+0000","last_fullsized":"2024-03-22T04:11:53.661326+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.600899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131442+0000","last_change":"2024-03-22T04:11:51.638506+0000","last_active":"2024-03-22T04:11:54.131442+0000","last_peered":"2024-03-22T04:11:54.131442+0000","last_clean":"2024-03-22T04:11:54.131442+0000","last_became_active":"2024-03-22T04:11:51.638256+0000","last_became_peered":"2024-03-22T04:11:51.638256+0000","last_unstale":"2024-03-22T04:11:54.131442+0000","last_undegraded":"2024-03-22T04:11:54.131442+0000","last_fullsized":"2024-03-22T04:11:54.131442+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:16:50.683344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653699+0000","last_change":"2024-03-22T04:11:52.660266+0000","last_active":"2024-03-22T04:11:53.653699+0000","last_peered":"2024-03-22T04:11:53.653699+0000","last_clean":"2024-03-22T04:11:53.653699+0000","last_became_active":"2024-03-22T04:11:52.659895+0000","last_became_peered":"2024-03-22T04:11:52.659895+0000","last_unstale":"2024-03-22T04:11:53.653699+0000","last_undegraded":"2024-03-22T04:11:53.653699+0000","last_fullsized":"2024-03-22T04:11:53.653699+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:35.693232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.638844+0000","last_change":"2024-03-22T04:11:51.645291+0000","last_active":"2024-03-22T04:11:52.638844+0000","last_peered":"2024-03-22T04:11:52.638844+0000","last_clean":"2024-03-22T04:11:52.638844+0000","last_became_active":"2024-03-22T04:11:51.644994+0000","last_became_peered":"2024-03-22T04:11:51.644994+0000","last_unstale":"2024-03-22T04:11:52.638844+0000","last_undegraded":"2024-03-22T04:11:52.638844+0000","last_fullsized":"2024-03-22T04:11:52.638844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:36:32.547421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656632+0000","last_change":"2024-03-22T04:11:52.646886+0000","last_active":"2024-03-22T04:11:53.656632+0000","last_peered":"2024-03-22T04:11:53.656632+0000","last_clean":"2024-03-22T04:11:53.656632+0000","last_became_active":"2024-03-22T04:11:52.645310+0000","last_became_peered":"2024-03-22T04:11:52.645310+0000","last_unstale":"2024-03-22T04:11:53.656632+0000","last_undegraded":"2024-03-22T04:11:53.656632+0000","last_fullsized":"2024-03-22T04:11:53.656632+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:25:47.554962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131882+0000","last_change":"2024-03-22T04:11:51.638270+0000","last_active":"2024-03-22T04:11:54.131882+0000","last_peered":"2024-03-22T04:11:54.131882+0000","last_clean":"2024-03-22T04:11:54.131882+0000","last_became_active":"2024-03-22T04:11:51.637766+0000","last_became_peered":"2024-03-22T04:11:51.637766+0000","last_unstale":"2024-03-22T04:11:54.131882+0000","last_undegraded":"2024-03-22T04:11:54.131882+0000","last_fullsized":"2024-03-22T04:11:54.131882+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:03.834934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.657919+0000","last_change":"2024-03-22T04:11:52.652881+0000","last_active":"2024-03-22T04:11:52.657919+0000","last_peered":"2024-03-22T04:11:52.657919+0000","last_clean":"2024-03-22T04:11:52.657919+0000","last_became_active":"2024-03-22T04:11:52.652190+0000","last_became_peered":"2024-03-22T04:11:52.652190+0000","last_unstale":"2024-03-22T04:11:52.657919+0000","last_undegraded":"2024-03-22T04:11:52.657919+0000","last_fullsized":"2024-03-22T04:11:52.657919+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:08:26.601087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639424+0000","last_change":"2024-03-22T04:11:51.635044+0000","last_active":"2024-03-22T04:11:52.639424+0000","last_peered":"2024-03-22T04:11:52.639424+0000","last_clean":"2024-03-22T04:11:52.639424+0000","last_became_active":"2024-03-22T04:11:51.633837+0000","last_became_peered":"2024-03-22T04:11:51.633837+0000","last_unstale":"2024-03-22T04:11:52.639424+0000","last_undegraded":"2024-03-22T04:11:52.639424+0000","last_fullsized":"2024-03-22T04:11:52.639424+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:31:32.200080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131134+0000","last_change":"2024-03-22T04:11:52.659709+0000","last_active":"2024-03-22T04:11:54.131134+0000","last_peered":"2024-03-22T04:11:54.131134+0000","last_clean":"2024-03-22T04:11:54.131134+0000","last_became_active":"2024-03-22T04:11:52.659232+0000","last_became_peered":"2024-03-22T04:11:52.659232+0000","last_unstale":"2024-03-22T04:11:54.131134+0000","last_undegraded":"2024-03-22T04:11:54.131134+0000","last_fullsized":"2024-03-22T04:11:54.131134+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:11:25.368184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653983+0000","last_change":"2024-03-22T04:11:51.637015+0000","last_active":"2024-03-22T04:11:53.653983+0000","last_peered":"2024-03-22T04:11:53.653983+0000","last_clean":"2024-03-22T04:11:53.653983+0000","last_became_active":"2024-03-22T04:11:51.636733+0000","last_became_peered":"2024-03-22T04:11:51.636733+0000","last_unstale":"2024-03-22T04:11:53.653983+0000","last_undegraded":"2024-03-22T04:11:53.653983+0000","last_fullsized":"2024-03-22T04:11:53.653983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:23:50.026312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.665977+0000","last_change":"2024-03-22T04:11:52.656994+0000","last_active":"2024-03-22T04:11:52.665977+0000","last_peered":"2024-03-22T04:11:52.665977+0000","last_clean":"2024-03-22T04:11:52.665977+0000","last_became_active":"2024-03-22T04:11:52.656688+0000","last_became_peered":"2024-03-22T04:11:52.656688+0000","last_unstale":"2024-03-22T04:11:52.665977+0000","last_undegraded":"2024-03-22T04:11:52.665977+0000","last_fullsized":"2024-03-22T04:11:52.665977+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:18:30.069733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653844+0000","last_change":"2024-03-22T04:11:51.634042+0000","last_active":"2024-03-22T04:11:53.653844+0000","last_peered":"2024-03-22T04:11:53.653844+0000","last_clean":"2024-03-22T04:11:53.653844+0000","last_became_active":"2024-03-22T04:11:51.633780+0000","last_became_peered":"2024-03-22T04:11:51.633780+0000","last_unstale":"2024-03-22T04:11:53.653844+0000","last_undegraded":"2024-03-22T04:11:53.653844+0000","last_fullsized":"2024-03-22T04:11:53.653844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:44.700396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131921+0000","last_change":"2024-03-22T04:11:52.659065+0000","last_active":"2024-03-22T04:11:54.131921+0000","last_peered":"2024-03-22T04:11:54.131921+0000","last_clean":"2024-03-22T04:11:54.131921+0000","last_became_active":"2024-03-22T04:11:52.658028+0000","last_became_peered":"2024-03-22T04:11:52.658028+0000","last_unstale":"2024-03-22T04:11:54.131921+0000","last_undegraded":"2024-03-22T04:11:54.131921+0000","last_fullsized":"2024-03-22T04:11:54.131921+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:44:22.160540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.681884+0000","last_change":"2024-03-22T04:11:51.644715+0000","last_active":"2024-03-22T04:11:52.681884+0000","last_peered":"2024-03-22T04:11:52.681884+0000","last_clean":"2024-03-22T04:11:52.681884+0000","last_became_active":"2024-03-22T04:11:51.644365+0000","last_became_peered":"2024-03-22T04:11:51.644365+0000","last_unstale":"2024-03-22T04:11:52.681884+0000","last_undegraded":"2024-03-22T04:11:52.681884+0000","last_fullsized":"2024-03-22T04:11:52.681884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:49:10.202707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.656068+0000","last_change":"2024-03-22T04:11:52.646801+0000","last_active":"2024-03-22T04:11:52.656068+0000","last_peered":"2024-03-22T04:11:52.656068+0000","last_clean":"2024-03-22T04:11:52.656068+0000","last_became_active":"2024-03-22T04:11:52.646491+0000","last_became_peered":"2024-03-22T04:11:52.646491+0000","last_unstale":"2024-03-22T04:11:52.656068+0000","last_undegraded":"2024-03-22T04:11:52.656068+0000","last_fullsized":"2024-03-22T04:11:52.656068+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:44:29.455556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.679634+0000","last_change":"2024-03-22T04:11:51.635750+0000","last_active":"2024-03-22T04:11:52.679634+0000","last_peered":"2024-03-22T04:11:52.679634+0000","last_clean":"2024-03-22T04:11:52.679634+0000","last_became_active":"2024-03-22T04:11:51.634686+0000","last_became_peered":"2024-03-22T04:11:51.634686+0000","last_unstale":"2024-03-22T04:11:52.679634+0000","last_undegraded":"2024-03-22T04:11:52.679634+0000","last_fullsized":"2024-03-22T04:11:52.679634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:56:45.221222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655892+0000","last_change":"2024-03-22T04:11:52.651829+0000","last_active":"2024-03-22T04:11:53.655892+0000","last_peered":"2024-03-22T04:11:53.655892+0000","last_clean":"2024-03-22T04:11:53.655892+0000","last_became_active":"2024-03-22T04:11:52.651513+0000","last_became_peered":"2024-03-22T04:11:52.651513+0000","last_unstale":"2024-03-22T04:11:53.655892+0000","last_undegraded":"2024-03-22T04:11:53.655892+0000","last_fullsized":"2024-03-22T04:11:53.655892+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:18:38.492478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656151+0000","last_change":"2024-03-22T04:11:51.634149+0000","last_active":"2024-03-22T04:11:53.656151+0000","last_peered":"2024-03-22T04:11:53.656151+0000","last_clean":"2024-03-22T04:11:53.656151+0000","last_became_active":"2024-03-22T04:11:51.633789+0000","last_became_peered":"2024-03-22T04:11:51.633789+0000","last_unstale":"2024-03-22T04:11:53.656151+0000","last_undegraded":"2024-03-22T04:11:53.656151+0000","last_fullsized":"2024-03-22T04:11:53.656151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:48:58.697949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661610+0000","last_change":"2024-03-22T04:11:52.665785+0000","last_active":"2024-03-22T04:11:53.661610+0000","last_peered":"2024-03-22T04:11:53.661610+0000","last_clean":"2024-03-22T04:11:53.661610+0000","last_became_active":"2024-03-22T04:11:52.665484+0000","last_became_peered":"2024-03-22T04:11:52.665484+0000","last_unstale":"2024-03-22T04:11:53.661610+0000","last_undegraded":"2024-03-22T04:11:53.661610+0000","last_fullsized":"2024-03-22T04:11:53.661610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:01:57.811504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653648+0000","last_change":"2024-03-22T04:11:51.641650+0000","last_active":"2024-03-22T04:11:53.653648+0000","last_peered":"2024-03-22T04:11:53.653648+0000","last_clean":"2024-03-22T04:11:53.653648+0000","last_became_active":"2024-03-22T04:11:51.641303+0000","last_became_peered":"2024-03-22T04:11:51.641303+0000","last_unstale":"2024-03-22T04:11:53.653648+0000","last_undegraded":"2024-03-22T04:11:53.653648+0000","last_fullsized":"2024-03-22T04:11:53.653648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:32:52.323037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655566+0000","last_change":"2024-03-22T04:11:52.661358+0000","last_active":"2024-03-22T04:11:53.655566+0000","last_peered":"2024-03-22T04:11:53.655566+0000","last_clean":"2024-03-22T04:11:53.655566+0000","last_became_active":"2024-03-22T04:11:52.660998+0000","last_became_peered":"2024-03-22T04:11:52.660998+0000","last_unstale":"2024-03-22T04:11:53.655566+0000","last_undegraded":"2024-03-22T04:11:53.655566+0000","last_fullsized":"2024-03-22T04:11:53.655566+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:48:59.705079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131112+0000","last_change":"2024-03-22T04:11:51.635563+0000","last_active":"2024-03-22T04:11:54.131112+0000","last_peered":"2024-03-22T04:11:54.131112+0000","last_clean":"2024-03-22T04:11:54.131112+0000","last_became_active":"2024-03-22T04:11:51.635265+0000","last_became_peered":"2024-03-22T04:11:51.635265+0000","last_unstale":"2024-03-22T04:11:54.131112+0000","last_undegraded":"2024-03-22T04:11:54.131112+0000","last_fullsized":"2024-03-22T04:11:54.131112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:22:59.050620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656566+0000","last_change":"2024-03-22T04:11:52.664104+0000","last_active":"2024-03-22T04:11:53.656566+0000","last_peered":"2024-03-22T04:11:53.656566+0000","last_clean":"2024-03-22T04:11:53.656566+0000","last_became_active":"2024-03-22T04:11:52.663837+0000","last_became_peered":"2024-03-22T04:11:52.663837+0000","last_unstale":"2024-03-22T04:11:53.656566+0000","last_undegraded":"2024-03-22T04:11:53.656566+0000","last_fullsized":"2024-03-22T04:11:53.656566+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:13:24.369315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131741+0000","last_change":"2024-03-22T04:11:51.638377+0000","last_active":"2024-03-22T04:11:54.131741+0000","last_peered":"2024-03-22T04:11:54.131741+0000","last_clean":"2024-03-22T04:11:54.131741+0000","last_became_active":"2024-03-22T04:11:51.638093+0000","last_became_peered":"2024-03-22T04:11:51.638093+0000","last_unstale":"2024-03-22T04:11:54.131741+0000","last_undegraded":"2024-03-22T04:11:54.131741+0000","last_fullsized":"2024-03-22T04:11:54.131741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:42:02.600608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131790+0000","last_change":"2024-03-22T04:11:52.657458+0000","last_active":"2024-03-22T04:11:54.131790+0000","last_peered":"2024-03-22T04:11:54.131790+0000","last_clean":"2024-03-22T04:11:54.131790+0000","last_became_active":"2024-03-22T04:11:52.657203+0000","last_became_peered":"2024-03-22T04:11:52.657203+0000","last_unstale":"2024-03-22T04:11:54.131790+0000","last_undegraded":"2024-03-22T04:11:54.131790+0000","last_fullsized":"2024-03-22T04:11:54.131790+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:52:01.031151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.654068+0000","last_change":"2024-03-22T04:11:51.633320+0000","last_active":"2024-03-22T04:11:53.654068+0000","last_peered":"2024-03-22T04:11:53.654068+0000","last_clean":"2024-03-22T04:11:53.654068+0000","last_became_active":"2024-03-22T04:11:51.633063+0000","last_became_peered":"2024-03-22T04:11:51.633063+0000","last_unstale":"2024-03-22T04:11:53.654068+0000","last_undegraded":"2024-03-22T04:11:53.654068+0000","last_fullsized":"2024-03-22T04:11:53.654068+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:38:18.241111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131860+0000","last_change":"2024-03-22T04:11:52.663760+0000","last_active":"2024-03-22T04:11:54.131860+0000","last_peered":"2024-03-22T04:11:54.131860+0000","last_clean":"2024-03-22T04:11:54.131860+0000","last_became_active":"2024-03-22T04:11:52.663478+0000","last_became_peered":"2024-03-22T04:11:52.663478+0000","last_unstale":"2024-03-22T04:11:54.131860+0000","last_undegraded":"2024-03-22T04:11:54.131860+0000","last_fullsized":"2024-03-22T04:11:54.131860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:23:07.213903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655686+0000","last_change":"2024-03-22T04:11:51.644988+0000","last_active":"2024-03-22T04:11:53.655686+0000","last_peered":"2024-03-22T04:11:53.655686+0000","last_clean":"2024-03-22T04:11:53.655686+0000","last_became_active":"2024-03-22T04:11:51.644707+0000","last_became_peered":"2024-03-22T04:11:51.644707+0000","last_unstale":"2024-03-22T04:11:53.655686+0000","last_undegraded":"2024-03-22T04:11:53.655686+0000","last_fullsized":"2024-03-22T04:11:53.655686+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:53:29.162898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132835+0000","last_change":"2024-03-22T04:11:52.658468+0000","last_active":"2024-03-22T04:11:54.132835+0000","last_peered":"2024-03-22T04:11:54.132835+0000","last_clean":"2024-03-22T04:11:54.132835+0000","last_became_active":"2024-03-22T04:11:52.657183+0000","last_became_peered":"2024-03-22T04:11:52.657183+0000","last_unstale":"2024-03-22T04:11:54.132835+0000","last_undegraded":"2024-03-22T04:11:54.132835+0000","last_fullsized":"2024-03-22T04:11:54.132835+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:25:47.554962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132342+0000","last_change":"2024-03-22T04:11:51.634391+0000","last_active":"2024-03-22T04:11:54.132342+0000","last_peered":"2024-03-22T04:11:54.132342+0000","last_clean":"2024-03-22T04:11:54.132342+0000","last_became_active":"2024-03-22T04:11:51.634111+0000","last_became_peered":"2024-03-22T04:11:51.634111+0000","last_unstale":"2024-03-22T04:11:54.132342+0000","last_undegraded":"2024-03-22T04:11:54.132342+0000","last_fullsized":"2024-03-22T04:11:54.132342+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:03:04.053607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653525+0000","last_change":"2024-03-22T04:11:52.645037+0000","last_active":"2024-03-22T04:11:53.653525+0000","last_peered":"2024-03-22T04:11:53.653525+0000","last_clean":"2024-03-22T04:11:53.653525+0000","last_became_active":"2024-03-22T04:11:52.644771+0000","last_became_peered":"2024-03-22T04:11:52.644771+0000","last_unstale":"2024-03-22T04:11:53.653525+0000","last_undegraded":"2024-03-22T04:11:53.653525+0000","last_fullsized":"2024-03-22T04:11:53.653525+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:10.393113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132444+0000","last_change":"2024-03-22T04:11:51.634888+0000","last_active":"2024-03-22T04:11:54.132444+0000","last_peered":"2024-03-22T04:11:54.132444+0000","last_clean":"2024-03-22T04:11:54.132444+0000","last_became_active":"2024-03-22T04:11:51.633782+0000","last_became_peered":"2024-03-22T04:11:51.633782+0000","last_unstale":"2024-03-22T04:11:54.132444+0000","last_undegraded":"2024-03-22T04:11:54.132444+0000","last_fullsized":"2024-03-22T04:11:54.132444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:02:02.722846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.666825+0000","last_change":"2024-03-22T04:11:52.657565+0000","last_active":"2024-03-22T04:11:52.666825+0000","last_peered":"2024-03-22T04:11:52.666825+0000","last_clean":"2024-03-22T04:11:52.666825+0000","last_became_active":"2024-03-22T04:11:52.657329+0000","last_became_peered":"2024-03-22T04:11:52.657329+0000","last_unstale":"2024-03-22T04:11:52.666825+0000","last_undegraded":"2024-03-22T04:11:52.666825+0000","last_fullsized":"2024-03-22T04:11:52.666825+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:51.727928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656076+0000","last_change":"2024-03-22T04:11:51.640816+0000","last_active":"2024-03-22T04:11:53.656076+0000","last_peered":"2024-03-22T04:11:53.656076+0000","last_clean":"2024-03-22T04:11:53.656076+0000","last_became_active":"2024-03-22T04:11:51.640560+0000","last_became_peered":"2024-03-22T04:11:51.640560+0000","last_unstale":"2024-03-22T04:11:53.656076+0000","last_undegraded":"2024-03-22T04:11:53.656076+0000","last_fullsized":"2024-03-22T04:11:53.656076+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:22:59.050620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661348+0000","last_change":"2024-03-22T04:11:52.661321+0000","last_active":"2024-03-22T04:11:53.661348+0000","last_peered":"2024-03-22T04:11:53.661348+0000","last_clean":"2024-03-22T04:11:53.661348+0000","last_became_active":"2024-03-22T04:11:52.660848+0000","last_became_peered":"2024-03-22T04:11:52.660848+0000","last_unstale":"2024-03-22T04:11:53.661348+0000","last_undegraded":"2024-03-22T04:11:53.661348+0000","last_fullsized":"2024-03-22T04:11:53.661348+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:45:14.997775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131508+0000","last_change":"2024-03-22T04:11:51.638060+0000","last_active":"2024-03-22T04:11:54.131508+0000","last_peered":"2024-03-22T04:11:54.131508+0000","last_clean":"2024-03-22T04:11:54.131508+0000","last_became_active":"2024-03-22T04:11:51.637774+0000","last_became_peered":"2024-03-22T04:11:51.637774+0000","last_unstale":"2024-03-22T04:11:54.131508+0000","last_undegraded":"2024-03-22T04:11:54.131508+0000","last_fullsized":"2024-03-22T04:11:54.131508+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:48.280840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.667509+0000","last_change":"2024-03-22T04:11:52.657915+0000","last_active":"2024-03-22T04:11:52.667509+0000","last_peered":"2024-03-22T04:11:52.667509+0000","last_clean":"2024-03-22T04:11:52.667509+0000","last_became_active":"2024-03-22T04:11:52.657728+0000","last_became_peered":"2024-03-22T04:11:52.657728+0000","last_unstale":"2024-03-22T04:11:52.667509+0000","last_undegraded":"2024-03-22T04:11:52.667509+0000","last_fullsized":"2024-03-22T04:11:52.667509+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:53:28.532936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655308+0000","last_change":"2024-03-22T04:11:51.632186+0000","last_active":"2024-03-22T04:11:53.655308+0000","last_peered":"2024-03-22T04:11:53.655308+0000","last_clean":"2024-03-22T04:11:53.655308+0000","last_became_active":"2024-03-22T04:11:51.631907+0000","last_became_peered":"2024-03-22T04:11:51.631907+0000","last_unstale":"2024-03-22T04:11:53.655308+0000","last_undegraded":"2024-03-22T04:11:53.655308+0000","last_fullsized":"2024-03-22T04:11:53.655308+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:10:15.275541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655767+0000","last_change":"2024-03-22T04:11:52.661682+0000","last_active":"2024-03-22T04:11:53.655767+0000","last_peered":"2024-03-22T04:11:53.655767+0000","last_clean":"2024-03-22T04:11:53.655767+0000","last_became_active":"2024-03-22T04:11:52.661384+0000","last_became_peered":"2024-03-22T04:11:52.661384+0000","last_unstale":"2024-03-22T04:11:53.655767+0000","last_undegraded":"2024-03-22T04:11:53.655767+0000","last_fullsized":"2024-03-22T04:11:53.655767+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:52:01.031151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.654041+0000","last_change":"2024-03-22T04:11:51.633301+0000","last_active":"2024-03-22T04:11:53.654041+0000","last_peered":"2024-03-22T04:11:53.654041+0000","last_clean":"2024-03-22T04:11:53.654041+0000","last_became_active":"2024-03-22T04:11:51.633017+0000","last_became_peered":"2024-03-22T04:11:51.633017+0000","last_unstale":"2024-03-22T04:11:53.654041+0000","last_undegraded":"2024-03-22T04:11:53.654041+0000","last_fullsized":"2024-03-22T04:11:53.654041+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:15:18.759040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661989+0000","last_change":"2024-03-22T04:11:52.656332+0000","last_active":"2024-03-22T04:11:53.661989+0000","last_peered":"2024-03-22T04:11:53.661989+0000","last_clean":"2024-03-22T04:11:53.661989+0000","last_became_active":"2024-03-22T04:11:52.656036+0000","last_became_peered":"2024-03-22T04:11:52.656036+0000","last_unstale":"2024-03-22T04:11:53.661989+0000","last_undegraded":"2024-03-22T04:11:53.661989+0000","last_fullsized":"2024-03-22T04:11:53.661989+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:08.306240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655834+0000","last_change":"2024-03-22T04:11:51.644074+0000","last_active":"2024-03-22T04:11:53.655834+0000","last_peered":"2024-03-22T04:11:53.655834+0000","last_clean":"2024-03-22T04:11:53.655834+0000","last_became_active":"2024-03-22T04:11:51.643795+0000","last_became_peered":"2024-03-22T04:11:51.643795+0000","last_unstale":"2024-03-22T04:11:53.655834+0000","last_undegraded":"2024-03-22T04:11:53.655834+0000","last_fullsized":"2024-03-22T04:11:53.655834+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:28:33.417099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132503+0000","last_change":"2024-03-22T04:11:52.650532+0000","last_active":"2024-03-22T04:11:54.132503+0000","last_peered":"2024-03-22T04:11:54.132503+0000","last_clean":"2024-03-22T04:11:54.132503+0000","last_became_active":"2024-03-22T04:11:52.650306+0000","last_became_peered":"2024-03-22T04:11:52.650306+0000","last_unstale":"2024-03-22T04:11:54.132503+0000","last_undegraded":"2024-03-22T04:11:54.132503+0000","last_fullsized":"2024-03-22T04:11:54.132503+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:13:24.369315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655796+0000","last_change":"2024-03-22T04:11:51.639226+0000","last_active":"2024-03-22T04:11:53.655796+0000","last_peered":"2024-03-22T04:11:53.655796+0000","last_clean":"2024-03-22T04:11:53.655796+0000","last_became_active":"2024-03-22T04:11:51.638927+0000","last_became_peered":"2024-03-22T04:11:51.638927+0000","last_unstale":"2024-03-22T04:11:53.655796+0000","last_undegraded":"2024-03-22T04:11:53.655796+0000","last_fullsized":"2024-03-22T04:11:53.655796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:11:14.812727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655207+0000","last_change":"2024-03-22T04:11:52.663709+0000","last_active":"2024-03-22T04:11:53.655207+0000","last_peered":"2024-03-22T04:11:53.655207+0000","last_clean":"2024-03-22T04:11:53.655207+0000","last_became_active":"2024-03-22T04:11:52.663292+0000","last_became_peered":"2024-03-22T04:11:52.663292+0000","last_unstale":"2024-03-22T04:11:53.655207+0000","last_undegraded":"2024-03-22T04:11:53.655207+0000","last_fullsized":"2024-03-22T04:11:53.655207+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:10:16.282670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639215+0000","last_change":"2024-03-22T04:11:51.635806+0000","last_active":"2024-03-22T04:11:52.639215+0000","last_peered":"2024-03-22T04:11:52.639215+0000","last_clean":"2024-03-22T04:11:52.639215+0000","last_became_active":"2024-03-22T04:11:51.633614+0000","last_became_peered":"2024-03-22T04:11:51.633614+0000","last_unstale":"2024-03-22T04:11:52.639215+0000","last_undegraded":"2024-03-22T04:11:52.639215+0000","last_fullsized":"2024-03-22T04:11:52.639215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:14.289312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655915+0000","last_change":"2024-03-22T04:11:52.660518+0000","last_active":"2024-03-22T04:11:53.655915+0000","last_peered":"2024-03-22T04:11:53.655915+0000","last_clean":"2024-03-22T04:11:53.655915+0000","last_became_active":"2024-03-22T04:11:52.660262+0000","last_became_peered":"2024-03-22T04:11:52.660262+0000","last_unstale":"2024-03-22T04:11:53.655915+0000","last_undegraded":"2024-03-22T04:11:53.655915+0000","last_fullsized":"2024-03-22T04:11:53.655915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:05.323006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661486+0000","last_change":"2024-03-22T04:11:51.639131+0000","last_active":"2024-03-22T04:11:53.661486+0000","last_peered":"2024-03-22T04:11:53.661486+0000","last_clean":"2024-03-22T04:11:53.661486+0000","last_became_active":"2024-03-22T04:11:51.638728+0000","last_became_peered":"2024-03-22T04:11:51.638728+0000","last_unstale":"2024-03-22T04:11:53.661486+0000","last_undegraded":"2024-03-22T04:11:53.661486+0000","last_fullsized":"2024-03-22T04:11:53.661486+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:44:21.153411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653793+0000","last_change":"2024-03-22T04:11:52.643385+0000","last_active":"2024-03-22T04:11:53.653793+0000","last_peered":"2024-03-22T04:11:53.653793+0000","last_clean":"2024-03-22T04:11:53.653793+0000","last_became_active":"2024-03-22T04:11:52.643071+0000","last_became_peered":"2024-03-22T04:11:52.643071+0000","last_unstale":"2024-03-22T04:11:53.653793+0000","last_undegraded":"2024-03-22T04:11:53.653793+0000","last_fullsized":"2024-03-22T04:11:53.653793+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:37:02.554632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653744+0000","last_change":"2024-03-22T04:11:51.632239+0000","last_active":"2024-03-22T04:11:53.653744+0000","last_peered":"2024-03-22T04:11:53.653744+0000","last_clean":"2024-03-22T04:11:53.653744+0000","last_became_active":"2024-03-22T04:11:51.631969+0000","last_became_peered":"2024-03-22T04:11:51.631969+0000","last_unstale":"2024-03-22T04:11:53.653744+0000","last_undegraded":"2024-03-22T04:11:53.653744+0000","last_fullsized":"2024-03-22T04:11:53.653744+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:59.446983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.662175+0000","last_change":"2024-03-22T04:11:52.653989+0000","last_active":"2024-03-22T04:11:53.662175+0000","last_peered":"2024-03-22T04:11:53.662175+0000","last_clean":"2024-03-22T04:11:53.662175+0000","last_became_active":"2024-03-22T04:11:52.653722+0000","last_became_peered":"2024-03-22T04:11:52.653722+0000","last_unstale":"2024-03-22T04:11:53.662175+0000","last_undegraded":"2024-03-22T04:11:53.662175+0000","last_fullsized":"2024-03-22T04:11:53.662175+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:58:21.035354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T04:11:52.639907+0000","last_change":"2024-03-22T04:11:51.641613+0000","last_active":"2024-03-22T04:11:52.639907+0000","last_peered":"2024-03-22T04:11:52.639907+0000","last_clean":"2024-03-22T04:11:52.639907+0000","last_became_active":"2024-03-22T04:11:51.641256+0000","last_became_peered":"2024-03-22T04:11:51.641256+0000","last_unstale":"2024-03-22T04:11:52.639907+0000","last_undegraded":"2024-03-22T04:11:52.639907+0000","last_fullsized":"2024-03-22T04:11:52.639907+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:27.877295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653987+0000","last_change":"2024-03-22T04:11:52.658436+0000","last_active":"2024-03-22T04:11:53.653987+0000","last_peered":"2024-03-22T04:11:53.653987+0000","last_clean":"2024-03-22T04:11:53.653987+0000","last_became_active":"2024-03-22T04:11:52.658166+0000","last_became_peered":"2024-03-22T04:11:52.658166+0000","last_unstale":"2024-03-22T04:11:53.653987+0000","last_undegraded":"2024-03-22T04:11:53.653987+0000","last_fullsized":"2024-03-22T04:11:53.653987+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:23:31.979507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661483+0000","last_change":"2024-03-22T04:11:51.637773+0000","last_active":"2024-03-22T04:11:53.661483+0000","last_peered":"2024-03-22T04:11:53.661483+0000","last_clean":"2024-03-22T04:11:53.661483+0000","last_became_active":"2024-03-22T04:11:51.637519+0000","last_became_peered":"2024-03-22T04:11:51.637519+0000","last_unstale":"2024-03-22T04:11:53.661483+0000","last_undegraded":"2024-03-22T04:11:53.661483+0000","last_fullsized":"2024-03-22T04:11:53.661483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:22.199529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":202,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131908+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:11:54.131908+0000","last_peered":"2024-03-22T04:11:54.131908+0000","last_clean":"2024-03-22T04:11:54.131908+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:11:54.131908+0000","last_undegraded":"2024-03-22T04:11:54.131908+0000","last_fullsized":"2024-03-22T04:11:54.131908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":199921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":199885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1064960,"data_stored":790801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":199941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":202045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1064960,"data_stored":790241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":199943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":194895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:12:01.238 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:12:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e7800640 1 Processor -- start 2024-03-22T04:12:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e7800640 1 -- start start 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e7800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e015b3f0 0x7fb9e00729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e7800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9e00cc3e0 0x7fb9e0072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e7800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9e00b30f0 0x7fb9e00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e7800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb9e00716d0 con 0x7fb9e00b30f0 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e7800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb9e006d990 con 0x7fb9e00cc3e0 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e7800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e0071920 con 0x7fb9e015b3f0 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9e00b30f0 0x7fb9e00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e4e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e015b3f0 0x7fb9e00729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9e00b30f0 0x7fb9e00ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36744/0 (socket says 172.21.15.86:36744) 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9e5800640 1 -- 172.21.15.86:0/668379766 learned_addr learned my addr 172.21.15.86:0/668379766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.378+0000 7fb9d7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9e00cc3e0 0x7fb9e0072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9e5800640 1 -- 172.21.15.86:0/668379766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e015b3f0 msgr2=0x7fb9e00729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9e5800640 1 --2- 172.21.15.86:0/668379766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e015b3f0 0x7fb9e00729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9e5800640 1 -- 172.21.15.86:0/668379766 >> v2:172.21.15.196:3300/0 conn(0x7fb9e00cc3e0 msgr2=0x7fb9e0072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9e5800640 1 --2- 172.21.15.86:0/668379766 >> v2:172.21.15.196:3300/0 conn(0x7fb9e00cc3e0 0x7fb9e0072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9e5800640 1 -- 172.21.15.86:0/668379766 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e0071c00 con 0x7fb9e00b30f0 2024-03-22T04:12:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9d7e00640 1 --2- 172.21.15.86:0/668379766 >> v2:172.21.15.196:3300/0 conn(0x7fb9e00cc3e0 0x7fb9e0072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:01.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9e4e00640 1 --2- 172.21.15.86:0/668379766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e015b3f0 0x7fb9e00729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9e5800640 1 --2- 172.21.15.86:0/668379766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e00ade30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb9e006d080 tx=0x7fb9d801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1561ab65292e5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9dfe00640 1 -- 172.21.15.86:0/668379766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80a55e0 con 0x7fb9e00b30f0 2024-03-22T04:12:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9dfe00640 1 -- 172.21.15.86:0/668379766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d80a5780 con 0x7fb9e00b30f0 2024-03-22T04:12:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.382+0000 7fb9dfe00640 1 -- 172.21.15.86:0/668379766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80a5b50 con 0x7fb9e00b30f0 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- 172.21.15.86:0/668379766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 msgr2=0x7fb9e00ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 --2- 172.21.15.86:0/668379766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e00ade30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb9e006d080 tx=0x7fb9d801c960 comp rx=0 tx=0).stop 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- 172.21.15.86:0/668379766 shutdown_connections 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 --2- 172.21.15.86:0/668379766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e00ade30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 --2- 172.21.15.86:0/668379766 >> v2:172.21.15.196:3300/0 conn(0x7fb9e00cc3e0 0x7fb9e0072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 --2- 172.21.15.86:0/668379766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e015b3f0 0x7fb9e00729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- 172.21.15.86:0/668379766 >> 172.21.15.86:0/668379766 conn(0x7fb9e00b1df0 msgr2=0x7fb9e00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- 172.21.15.86:0/668379766 shutdown_connections 2024-03-22T04:12:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- 172.21.15.86:0/668379766 wait complete. 2024-03-22T04:12:01.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 Processor -- start 2024-03-22T04:12:01.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- start start 2024-03-22T04:12:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e0152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e00cc3e0 0x7fb9e0152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e015b3f0 0x7fb9e0153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e015f1e0 con 0x7fb9e00b30f0 2024-03-22T04:12:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e015f060 con 0x7fb9e015b3f0 2024-03-22T04:12:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e7800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e015f360 con 0x7fb9e00cc3e0 2024-03-22T04:12:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e0152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e0152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36758/0 (socket says 172.21.15.86:36758) 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 -- 172.21.15.86:0/3514621173 learned_addr learned my addr 172.21.15.86:0/3514621173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e5800640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e015b3f0 0x7fb9e0153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9d7e00640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e00cc3e0 0x7fb9e0152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 -- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e00cc3e0 msgr2=0x7fb9e0152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e00cc3e0 0x7fb9e0152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 -- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e015b3f0 msgr2=0x7fb9e0153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e015b3f0 0x7fb9e0153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 -- 172.21.15.86:0/3514621173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e0072810 con 0x7fb9e00b30f0 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e5800640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e015b3f0 0x7fb9e0153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9e4e00640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e0152860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e980 tx=0x7fb9d000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9dd600640 1 -- 172.21.15.86:0/3514621173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0017070 con 0x7fb9e00b30f0 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9dd600640 1 -- 172.21.15.86:0/3514621173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d000ca00 con 0x7fb9e00b30f0 2024-03-22T04:12:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.386+0000 7fb9dd600640 1 -- 172.21.15.86:0/3514621173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000cdd0 con 0x7fb9e00b30f0 2024-03-22T04:12:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.390+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e009e010 con 0x7fb9e00b30f0 2024-03-22T04:12:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.390+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e01fa1b0 con 0x7fb9e00b30f0 2024-03-22T04:12:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.390+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e0068650 con 0x7fb9e00b30f0 2024-03-22T04:12:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.390+0000 7fb9dd600640 1 -- 172.21.15.86:0/3514621173 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb9d001a070 con 0x7fb9e00b30f0 2024-03-22T04:12:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.390+0000 7fb9dd600640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9bc067b20 0x7fb9bc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.390+0000 7fb9dd600640 1 -- 172.21.15.86:0/3514621173 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fb9d0085ff0 con 0x7fb9e00b30f0 2024-03-22T04:12:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.390+0000 7fb9d7e00640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9bc067b20 0x7fb9bc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.390+0000 7fb9d7e00640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9bc067b20 0x7fb9bc069fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9e015ddc0 tx=0x7fb9c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.394+0000 7fb9dd600640 1 -- 172.21.15.86:0/3514621173 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb9e009e010 con 0x7fb9e00b30f0 2024-03-22T04:12:01.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.546+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9e00ad2d0 con 0x7fb9bc067b20 2024-03-22T04:12:01.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.554+0000 7fb9dd600640 1 -- 172.21.15.86:0/3514621173 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7fb9e00ad2d0 con 0x7fb9bc067b20 2024-03-22T04:12:01.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:01.563 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:12:01.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9bc067b20 msgr2=0x7fb9bc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:01.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9bc067b20 0x7fb9bc069fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9e015ddc0 tx=0x7fb9c8008040 comp rx=0 tx=0).stop 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 msgr2=0x7fb9e0152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e0152860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e980 tx=0x7fb9d000ee40 comp rx=0 tx=0).stop 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 shutdown_connections 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb9bc067b20 0x7fb9bc069fe0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e015b3f0 0x7fb9e0153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9e00cc3e0 0x7fb9e0152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 --2- 172.21.15.86:0/3514621173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9e00b30f0 0x7fb9e0152860 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 >> 172.21.15.86:0/3514621173 conn(0x7fb9e00b1df0 msgr2=0x7fb9e015b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 shutdown_connections 2024-03-22T04:12:01.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.558+0000 7fb9e7800640 1 -- 172.21.15.86:0/3514621173 wait complete. 2024-03-22T04:12:01.582 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-03-22T04:11:59.808922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221060,"kb_used_data":4872,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753660,"statfs":{"total":773094113280,"available":772867747840,"internally_reserved":0,"allocated":4988928,"data_stored":2784364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001527"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.133327+0000","last_change":"2024-03-22T04:11:52.662881+0000","last_active":"2024-03-22T04:11:54.133327+0000","last_peered":"2024-03-22T04:11:54.133327+0000","last_clean":"2024-03-22T04:11:54.133327+0000","last_became_active":"2024-03-22T04:11:52.662454+0000","last_became_peered":"2024-03-22T04:11:52.662454+0000","last_unstale":"2024-03-22T04:11:54.133327+0000","last_undegraded":"2024-03-22T04:11:54.133327+0000","last_fullsized":"2024-03-22T04:11:54.133327+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:09:46.925780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656654+0000","last_change":"2024-03-22T04:11:52.659739+0000","last_active":"2024-03-22T04:11:53.656654+0000","last_peered":"2024-03-22T04:11:53.656654+0000","last_clean":"2024-03-22T04:11:53.656654+0000","last_became_active":"2024-03-22T04:11:52.659450+0000","last_became_peered":"2024-03-22T04:11:52.659450+0000","last_unstale":"2024-03-22T04:11:53.656654+0000","last_undegraded":"2024-03-22T04:11:53.656654+0000","last_fullsized":"2024-03-22T04:11:53.656654+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.600899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132945+0000","last_change":"2024-03-22T04:11:52.662047+0000","last_active":"2024-03-22T04:11:54.132945+0000","last_peered":"2024-03-22T04:11:54.132945+0000","last_clean":"2024-03-22T04:11:54.132945+0000","last_became_active":"2024-03-22T04:11:52.661357+0000","last_became_peered":"2024-03-22T04:11:52.661357+0000","last_unstale":"2024-03-22T04:11:54.132945+0000","last_undegraded":"2024-03-22T04:11:54.132945+0000","last_fullsized":"2024-03-22T04:11:54.132945+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:24:52.307888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.133312+0000","last_change":"2024-03-22T04:11:52.655700+0000","last_active":"2024-03-22T04:11:54.133312+0000","last_peered":"2024-03-22T04:11:54.133312+0000","last_clean":"2024-03-22T04:11:54.133312+0000","last_became_active":"2024-03-22T04:11:52.655442+0000","last_became_peered":"2024-03-22T04:11:52.655442+0000","last_unstale":"2024-03-22T04:11:54.133312+0000","last_undegraded":"2024-03-22T04:11:54.133312+0000","last_fullsized":"2024-03-22T04:11:54.133312+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:10:08.316702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655387+0000","last_change":"2024-03-22T04:11:52.665671+0000","last_active":"2024-03-22T04:11:53.655387+0000","last_peered":"2024-03-22T04:11:53.655387+0000","last_clean":"2024-03-22T04:11:53.655387+0000","last_became_active":"2024-03-22T04:11:52.665384+0000","last_became_peered":"2024-03-22T04:11:52.665384+0000","last_unstale":"2024-03-22T04:11:53.655387+0000","last_undegraded":"2024-03-22T04:11:53.655387+0000","last_fullsized":"2024-03-22T04:11:53.655387+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:23:00.057749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656097+0000","last_change":"2024-03-22T04:11:52.653999+0000","last_active":"2024-03-22T04:11:53.656097+0000","last_peered":"2024-03-22T04:11:53.656097+0000","last_clean":"2024-03-22T04:11:53.656097+0000","last_became_active":"2024-03-22T04:11:52.653699+0000","last_became_peered":"2024-03-22T04:11:52.653699+0000","last_unstale":"2024-03-22T04:11:53.656097+0000","last_undegraded":"2024-03-22T04:11:53.656097+0000","last_fullsized":"2024-03-22T04:11:53.656097+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:47.474967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131452+0000","last_change":"2024-03-22T04:11:52.666658+0000","last_active":"2024-03-22T04:11:54.131452+0000","last_peered":"2024-03-22T04:11:54.131452+0000","last_clean":"2024-03-22T04:11:54.131452+0000","last_became_active":"2024-03-22T04:11:52.666408+0000","last_became_peered":"2024-03-22T04:11:52.666408+0000","last_unstale":"2024-03-22T04:11:54.131452+0000","last_undegraded":"2024-03-22T04:11:54.131452+0000","last_fullsized":"2024-03-22T04:11:54.131452+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:23:00.057749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132247+0000","last_change":"2024-03-22T04:11:52.658845+0000","last_active":"2024-03-22T04:11:54.132247+0000","last_peered":"2024-03-22T04:11:54.132247+0000","last_clean":"2024-03-22T04:11:54.132247+0000","last_became_active":"2024-03-22T04:11:52.657778+0000","last_became_peered":"2024-03-22T04:11:52.657778+0000","last_unstale":"2024-03-22T04:11:54.132247+0000","last_undegraded":"2024-03-22T04:11:54.132247+0000","last_fullsized":"2024-03-22T04:11:54.132247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:17:11.188981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656254+0000","last_change":"2024-03-22T04:11:52.665877+0000","last_active":"2024-03-22T04:11:53.656254+0000","last_peered":"2024-03-22T04:11:53.656254+0000","last_clean":"2024-03-22T04:11:53.656254+0000","last_became_active":"2024-03-22T04:11:52.665559+0000","last_became_peered":"2024-03-22T04:11:52.665559+0000","last_unstale":"2024-03-22T04:11:53.656254+0000","last_undegraded":"2024-03-22T04:11:53.656254+0000","last_fullsized":"2024-03-22T04:11:53.656254+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:48:52.128428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131943+0000","last_change":"2024-03-22T04:11:52.652350+0000","last_active":"2024-03-22T04:11:54.131943+0000","last_peered":"2024-03-22T04:11:54.131943+0000","last_clean":"2024-03-22T04:11:54.131943+0000","last_became_active":"2024-03-22T04:11:52.651964+0000","last_became_peered":"2024-03-22T04:11:52.651964+0000","last_unstale":"2024-03-22T04:11:54.131943+0000","last_undegraded":"2024-03-22T04:11:54.131943+0000","last_fullsized":"2024-03-22T04:11:54.131943+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:23.206659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661723+0000","last_change":"2024-03-22T04:11:52.656438+0000","last_active":"2024-03-22T04:11:53.661723+0000","last_peered":"2024-03-22T04:11:53.661723+0000","last_clean":"2024-03-22T04:11:53.661723+0000","last_became_active":"2024-03-22T04:11:52.655889+0000","last_became_peered":"2024-03-22T04:11:52.655889+0000","last_unstale":"2024-03-22T04:11:53.661723+0000","last_undegraded":"2024-03-22T04:11:53.661723+0000","last_fullsized":"2024-03-22T04:11:53.661723+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:45:11.033321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655653+0000","last_change":"2024-03-22T04:11:52.661339+0000","last_active":"2024-03-22T04:11:53.655653+0000","last_peered":"2024-03-22T04:11:53.655653+0000","last_clean":"2024-03-22T04:11:53.655653+0000","last_became_active":"2024-03-22T04:11:52.660933+0000","last_became_peered":"2024-03-22T04:11:52.660933+0000","last_unstale":"2024-03-22T04:11:53.655653+0000","last_undegraded":"2024-03-22T04:11:53.655653+0000","last_fullsized":"2024-03-22T04:11:53.655653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:47:59.233121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131386+0000","last_change":"2024-03-22T04:11:52.648731+0000","last_active":"2024-03-22T04:11:54.131386+0000","last_peered":"2024-03-22T04:11:54.131386+0000","last_clean":"2024-03-22T04:11:54.131386+0000","last_became_active":"2024-03-22T04:11:52.648451+0000","last_became_peered":"2024-03-22T04:11:52.648451+0000","last_unstale":"2024-03-22T04:11:54.131386+0000","last_undegraded":"2024-03-22T04:11:54.131386+0000","last_fullsized":"2024-03-22T04:11:54.131386+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:10:16.282670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131091+0000","last_change":"2024-03-22T04:11:52.645044+0000","last_active":"2024-03-22T04:11:54.131091+0000","last_peered":"2024-03-22T04:11:54.131091+0000","last_clean":"2024-03-22T04:11:54.131091+0000","last_became_active":"2024-03-22T04:11:52.644788+0000","last_became_peered":"2024-03-22T04:11:52.644788+0000","last_unstale":"2024-03-22T04:11:54.131091+0000","last_undegraded":"2024-03-22T04:11:54.131091+0000","last_fullsized":"2024-03-22T04:11:54.131091+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:15:20.514843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653534+0000","last_change":"2024-03-22T04:11:52.646944+0000","last_active":"2024-03-22T04:11:53.653534+0000","last_peered":"2024-03-22T04:11:53.653534+0000","last_clean":"2024-03-22T04:11:53.653534+0000","last_became_active":"2024-03-22T04:11:52.646677+0000","last_became_peered":"2024-03-22T04:11:52.646677+0000","last_unstale":"2024-03-22T04:11:53.653534+0000","last_undegraded":"2024-03-22T04:11:53.653534+0000","last_fullsized":"2024-03-22T04:11:53.653534+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:55:53.302385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131789+0000","last_change":"2024-03-22T04:11:52.665504+0000","last_active":"2024-03-22T04:11:54.131789+0000","last_peered":"2024-03-22T04:11:54.131789+0000","last_clean":"2024-03-22T04:11:54.131789+0000","last_became_active":"2024-03-22T04:11:52.665186+0000","last_became_peered":"2024-03-22T04:11:52.665186+0000","last_unstale":"2024-03-22T04:11:54.131789+0000","last_undegraded":"2024-03-22T04:11:54.131789+0000","last_fullsized":"2024-03-22T04:11:54.131789+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:48:52.128428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.654017+0000","last_change":"2024-03-22T04:11:52.658480+0000","last_active":"2024-03-22T04:11:53.654017+0000","last_peered":"2024-03-22T04:11:53.654017+0000","last_clean":"2024-03-22T04:11:53.654017+0000","last_became_active":"2024-03-22T04:11:52.658213+0000","last_became_peered":"2024-03-22T04:11:52.658213+0000","last_unstale":"2024-03-22T04:11:53.654017+0000","last_undegraded":"2024-03-22T04:11:53.654017+0000","last_fullsized":"2024-03-22T04:11:53.654017+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:07.174862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653915+0000","last_change":"2024-03-22T04:11:52.658663+0000","last_active":"2024-03-22T04:11:53.653915+0000","last_peered":"2024-03-22T04:11:53.653915+0000","last_clean":"2024-03-22T04:11:53.653915+0000","last_became_active":"2024-03-22T04:11:52.658389+0000","last_became_peered":"2024-03-22T04:11:52.658389+0000","last_unstale":"2024-03-22T04:11:53.653915+0000","last_undegraded":"2024-03-22T04:11:53.653915+0000","last_fullsized":"2024-03-22T04:11:53.653915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:01:24.998993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132674+0000","last_change":"2024-03-22T04:11:52.657515+0000","last_active":"2024-03-22T04:11:54.132674+0000","last_peered":"2024-03-22T04:11:54.132674+0000","last_clean":"2024-03-22T04:11:54.132674+0000","last_became_active":"2024-03-22T04:11:52.657199+0000","last_became_peered":"2024-03-22T04:11:52.657199+0000","last_unstale":"2024-03-22T04:11:54.132674+0000","last_undegraded":"2024-03-22T04:11:54.132674+0000","last_fullsized":"2024-03-22T04:11:54.132674+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:15:03.000496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661434+0000","last_change":"2024-03-22T04:11:52.659905+0000","last_active":"2024-03-22T04:11:53.661434+0000","last_peered":"2024-03-22T04:11:53.661434+0000","last_clean":"2024-03-22T04:11:53.661434+0000","last_became_active":"2024-03-22T04:11:52.659333+0000","last_became_peered":"2024-03-22T04:11:52.659333+0000","last_unstale":"2024-03-22T04:11:53.661434+0000","last_undegraded":"2024-03-22T04:11:53.661434+0000","last_fullsized":"2024-03-22T04:11:53.661434+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:37:01.810257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.662140+0000","last_change":"2024-03-22T04:11:52.663400+0000","last_active":"2024-03-22T04:11:53.662140+0000","last_peered":"2024-03-22T04:11:53.662140+0000","last_clean":"2024-03-22T04:11:53.662140+0000","last_became_active":"2024-03-22T04:11:52.662511+0000","last_became_peered":"2024-03-22T04:11:52.662511+0000","last_unstale":"2024-03-22T04:11:53.662140+0000","last_undegraded":"2024-03-22T04:11:53.662140+0000","last_fullsized":"2024-03-22T04:11:53.662140+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:43:40.916261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661358+0000","last_change":"2024-03-22T04:11:52.664517+0000","last_active":"2024-03-22T04:11:53.661358+0000","last_peered":"2024-03-22T04:11:53.661358+0000","last_clean":"2024-03-22T04:11:53.661358+0000","last_became_active":"2024-03-22T04:11:52.664245+0000","last_became_peered":"2024-03-22T04:11:52.664245+0000","last_unstale":"2024-03-22T04:11:53.661358+0000","last_undegraded":"2024-03-22T04:11:53.661358+0000","last_fullsized":"2024-03-22T04:11:53.661358+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:07:26.114489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653590+0000","last_change":"2024-03-22T04:11:52.655515+0000","last_active":"2024-03-22T04:11:53.653590+0000","last_peered":"2024-03-22T04:11:53.653590+0000","last_clean":"2024-03-22T04:11:53.653590+0000","last_became_active":"2024-03-22T04:11:52.655189+0000","last_became_peered":"2024-03-22T04:11:52.655189+0000","last_unstale":"2024-03-22T04:11:53.653590+0000","last_undegraded":"2024-03-22T04:11:53.653590+0000","last_fullsized":"2024-03-22T04:11:53.653590+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:16.270410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132267+0000","last_change":"2024-03-22T04:11:52.658420+0000","last_active":"2024-03-22T04:11:54.132267+0000","last_peered":"2024-03-22T04:11:54.132267+0000","last_clean":"2024-03-22T04:11:54.132267+0000","last_became_active":"2024-03-22T04:11:52.657062+0000","last_became_peered":"2024-03-22T04:11:52.657062+0000","last_unstale":"2024-03-22T04:11:54.132267+0000","last_undegraded":"2024-03-22T04:11:54.132267+0000","last_fullsized":"2024-03-22T04:11:54.132267+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:02:44.300883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131795+0000","last_change":"2024-03-22T04:11:52.650350+0000","last_active":"2024-03-22T04:11:54.131795+0000","last_peered":"2024-03-22T04:11:54.131795+0000","last_clean":"2024-03-22T04:11:54.131795+0000","last_became_active":"2024-03-22T04:11:52.650085+0000","last_became_peered":"2024-03-22T04:11:52.650085+0000","last_unstale":"2024-03-22T04:11:54.131795+0000","last_undegraded":"2024-03-22T04:11:54.131795+0000","last_fullsized":"2024-03-22T04:11:54.131795+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:03:05.060737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132847+0000","last_change":"2024-03-22T04:11:52.652956+0000","last_active":"2024-03-22T04:11:54.132847+0000","last_peered":"2024-03-22T04:11:54.132847+0000","last_clean":"2024-03-22T04:11:54.132847+0000","last_became_active":"2024-03-22T04:11:52.652568+0000","last_became_peered":"2024-03-22T04:11:52.652568+0000","last_unstale":"2024-03-22T04:11:54.132847+0000","last_undegraded":"2024-03-22T04:11:54.132847+0000","last_fullsized":"2024-03-22T04:11:54.132847+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:42:13.403797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661860+0000","last_change":"2024-03-22T04:11:52.665778+0000","last_active":"2024-03-22T04:11:53.661860+0000","last_peered":"2024-03-22T04:11:53.661860+0000","last_clean":"2024-03-22T04:11:53.661860+0000","last_became_active":"2024-03-22T04:11:52.665516+0000","last_became_peered":"2024-03-22T04:11:52.665516+0000","last_unstale":"2024-03-22T04:11:53.661860+0000","last_undegraded":"2024-03-22T04:11:53.661860+0000","last_fullsized":"2024-03-22T04:11:53.661860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:13:33.572221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131655+0000","last_change":"2024-03-22T04:11:52.652480+0000","last_active":"2024-03-22T04:11:54.131655+0000","last_peered":"2024-03-22T04:11:54.131655+0000","last_clean":"2024-03-22T04:11:54.131655+0000","last_became_active":"2024-03-22T04:11:52.652170+0000","last_became_peered":"2024-03-22T04:11:52.652170+0000","last_unstale":"2024-03-22T04:11:54.131655+0000","last_undegraded":"2024-03-22T04:11:54.131655+0000","last_fullsized":"2024-03-22T04:11:54.131655+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:47:59.233121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653702+0000","last_change":"2024-03-22T04:11:52.656760+0000","last_active":"2024-03-22T04:11:53.653702+0000","last_peered":"2024-03-22T04:11:53.653702+0000","last_clean":"2024-03-22T04:11:53.653702+0000","last_became_active":"2024-03-22T04:11:52.654908+0000","last_became_peered":"2024-03-22T04:11:52.654908+0000","last_unstale":"2024-03-22T04:11:53.653702+0000","last_undegraded":"2024-03-22T04:11:53.653702+0000","last_fullsized":"2024-03-22T04:11:53.653702+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:45:59.503409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653629+0000","last_change":"2024-03-22T04:11:52.655520+0000","last_active":"2024-03-22T04:11:53.653629+0000","last_peered":"2024-03-22T04:11:53.653629+0000","last_clean":"2024-03-22T04:11:53.653629+0000","last_became_active":"2024-03-22T04:11:52.655233+0000","last_became_peered":"2024-03-22T04:11:52.655233+0000","last_unstale":"2024-03-22T04:11:53.653629+0000","last_undegraded":"2024-03-22T04:11:53.653629+0000","last_fullsized":"2024-03-22T04:11:53.653629+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:35:20.592890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131533+0000","last_change":"2024-03-22T04:11:52.650033+0000","last_active":"2024-03-22T04:11:54.131533+0000","last_peered":"2024-03-22T04:11:54.131533+0000","last_clean":"2024-03-22T04:11:54.131533+0000","last_became_active":"2024-03-22T04:11:52.648288+0000","last_became_peered":"2024-03-22T04:11:52.648288+0000","last_unstale":"2024-03-22T04:11:54.131533+0000","last_undegraded":"2024-03-22T04:11:54.131533+0000","last_fullsized":"2024-03-22T04:11:54.131533+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:48:59.705079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132415+0000","last_change":"2024-03-22T04:11:52.658632+0000","last_active":"2024-03-22T04:11:54.132415+0000","last_peered":"2024-03-22T04:11:54.132415+0000","last_clean":"2024-03-22T04:11:54.132415+0000","last_became_active":"2024-03-22T04:11:52.657426+0000","last_became_peered":"2024-03-22T04:11:52.657426+0000","last_unstale":"2024-03-22T04:11:54.132415+0000","last_undegraded":"2024-03-22T04:11:54.132415+0000","last_fullsized":"2024-03-22T04:11:54.132415+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:37:39.339274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653910+0000","last_change":"2024-03-22T04:11:52.646488+0000","last_active":"2024-03-22T04:11:53.653910+0000","last_peered":"2024-03-22T04:11:53.653910+0000","last_clean":"2024-03-22T04:11:53.653910+0000","last_became_active":"2024-03-22T04:11:52.646297+0000","last_became_peered":"2024-03-22T04:11:52.646297+0000","last_unstale":"2024-03-22T04:11:53.653910+0000","last_undegraded":"2024-03-22T04:11:53.653910+0000","last_fullsized":"2024-03-22T04:11:53.653910+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:54:16.199170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131668+0000","last_change":"2024-03-22T04:11:51.636676+0000","last_active":"2024-03-22T04:11:54.131668+0000","last_peered":"2024-03-22T04:11:54.131668+0000","last_clean":"2024-03-22T04:11:54.131668+0000","last_became_active":"2024-03-22T04:11:51.636389+0000","last_became_peered":"2024-03-22T04:11:51.636389+0000","last_unstale":"2024-03-22T04:11:54.131668+0000","last_undegraded":"2024-03-22T04:11:54.131668+0000","last_fullsized":"2024-03-22T04:11:54.131668+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:48:58.697949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656600+0000","last_change":"2024-03-22T04:11:52.663434+0000","last_active":"2024-03-22T04:11:53.656600+0000","last_peered":"2024-03-22T04:11:53.656600+0000","last_clean":"2024-03-22T04:11:53.656600+0000","last_became_active":"2024-03-22T04:11:52.663176+0000","last_became_peered":"2024-03-22T04:11:52.663176+0000","last_unstale":"2024-03-22T04:11:53.656600+0000","last_undegraded":"2024-03-22T04:11:53.656600+0000","last_fullsized":"2024-03-22T04:11:53.656600+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:15:03.000496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656772+0000","last_change":"2024-03-22T04:11:51.630512+0000","last_active":"2024-03-22T04:11:53.656772+0000","last_peered":"2024-03-22T04:11:53.656772+0000","last_clean":"2024-03-22T04:11:53.656772+0000","last_became_active":"2024-03-22T04:11:51.628986+0000","last_became_peered":"2024-03-22T04:11:51.628986+0000","last_unstale":"2024-03-22T04:11:53.656772+0000","last_undegraded":"2024-03-22T04:11:53.656772+0000","last_fullsized":"2024-03-22T04:11:53.656772+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:47:58.225991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.654082+0000","last_change":"2024-03-22T04:11:52.645360+0000","last_active":"2024-03-22T04:11:53.654082+0000","last_peered":"2024-03-22T04:11:53.654082+0000","last_clean":"2024-03-22T04:11:53.654082+0000","last_became_active":"2024-03-22T04:11:52.645030+0000","last_became_peered":"2024-03-22T04:11:52.645030+0000","last_unstale":"2024-03-22T04:11:53.654082+0000","last_undegraded":"2024-03-22T04:11:53.654082+0000","last_fullsized":"2024-03-22T04:11:53.654082+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:50:44.542135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.133107+0000","last_change":"2024-03-22T04:11:51.636087+0000","last_active":"2024-03-22T04:11:54.133107+0000","last_peered":"2024-03-22T04:11:54.133107+0000","last_clean":"2024-03-22T04:11:54.133107+0000","last_became_active":"2024-03-22T04:11:51.635794+0000","last_became_peered":"2024-03-22T04:11:51.635794+0000","last_unstale":"2024-03-22T04:11:54.133107+0000","last_undegraded":"2024-03-22T04:11:54.133107+0000","last_fullsized":"2024-03-22T04:11:54.133107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:55:29.018276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656657+0000","last_change":"2024-03-22T04:11:52.662695+0000","last_active":"2024-03-22T04:11:53.656657+0000","last_peered":"2024-03-22T04:11:53.656657+0000","last_clean":"2024-03-22T04:11:53.656657+0000","last_became_active":"2024-03-22T04:11:52.662338+0000","last_became_peered":"2024-03-22T04:11:52.662338+0000","last_unstale":"2024-03-22T04:11:53.656657+0000","last_undegraded":"2024-03-22T04:11:53.656657+0000","last_fullsized":"2024-03-22T04:11:53.656657+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:45:14.997775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131226+0000","last_change":"2024-03-22T04:11:51.641684+0000","last_active":"2024-03-22T04:11:54.131226+0000","last_peered":"2024-03-22T04:11:54.131226+0000","last_clean":"2024-03-22T04:11:54.131226+0000","last_became_active":"2024-03-22T04:11:51.641520+0000","last_became_peered":"2024-03-22T04:11:51.641520+0000","last_unstale":"2024-03-22T04:11:54.131226+0000","last_undegraded":"2024-03-22T04:11:54.131226+0000","last_fullsized":"2024-03-22T04:11:54.131226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:47:58.225991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132580+0000","last_change":"2024-03-22T04:11:52.659045+0000","last_active":"2024-03-22T04:11:54.132580+0000","last_peered":"2024-03-22T04:11:54.132580+0000","last_clean":"2024-03-22T04:11:54.132580+0000","last_became_active":"2024-03-22T04:11:52.658165+0000","last_became_peered":"2024-03-22T04:11:52.658165+0000","last_unstale":"2024-03-22T04:11:54.132580+0000","last_undegraded":"2024-03-22T04:11:54.132580+0000","last_fullsized":"2024-03-22T04:11:54.132580+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:55.816845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131108+0000","last_change":"2024-03-22T04:11:51.640008+0000","last_active":"2024-03-22T04:11:54.131108+0000","last_peered":"2024-03-22T04:11:54.131108+0000","last_clean":"2024-03-22T04:11:54.131108+0000","last_became_active":"2024-03-22T04:11:51.637341+0000","last_became_peered":"2024-03-22T04:11:51.637341+0000","last_unstale":"2024-03-22T04:11:54.131108+0000","last_undegraded":"2024-03-22T04:11:54.131108+0000","last_fullsized":"2024-03-22T04:11:54.131108+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:10:15.275541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661326+0000","last_change":"2024-03-22T04:11:52.660384+0000","last_active":"2024-03-22T04:11:53.661326+0000","last_peered":"2024-03-22T04:11:53.661326+0000","last_clean":"2024-03-22T04:11:53.661326+0000","last_became_active":"2024-03-22T04:11:52.660042+0000","last_became_peered":"2024-03-22T04:11:52.660042+0000","last_unstale":"2024-03-22T04:11:53.661326+0000","last_undegraded":"2024-03-22T04:11:53.661326+0000","last_fullsized":"2024-03-22T04:11:53.661326+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:37.600899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131442+0000","last_change":"2024-03-22T04:11:51.638506+0000","last_active":"2024-03-22T04:11:54.131442+0000","last_peered":"2024-03-22T04:11:54.131442+0000","last_clean":"2024-03-22T04:11:54.131442+0000","last_became_active":"2024-03-22T04:11:51.638256+0000","last_became_peered":"2024-03-22T04:11:51.638256+0000","last_unstale":"2024-03-22T04:11:54.131442+0000","last_undegraded":"2024-03-22T04:11:54.131442+0000","last_fullsized":"2024-03-22T04:11:54.131442+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:16:50.683344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653699+0000","last_change":"2024-03-22T04:11:52.660266+0000","last_active":"2024-03-22T04:11:53.653699+0000","last_peered":"2024-03-22T04:11:53.653699+0000","last_clean":"2024-03-22T04:11:53.653699+0000","last_became_active":"2024-03-22T04:11:52.659895+0000","last_became_peered":"2024-03-22T04:11:52.659895+0000","last_unstale":"2024-03-22T04:11:53.653699+0000","last_undegraded":"2024-03-22T04:11:53.653699+0000","last_fullsized":"2024-03-22T04:11:53.653699+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:29:35.693232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132281+0000","last_change":"2024-03-22T04:11:51.645291+0000","last_active":"2024-03-22T04:11:54.132281+0000","last_peered":"2024-03-22T04:11:54.132281+0000","last_clean":"2024-03-22T04:11:54.132281+0000","last_became_active":"2024-03-22T04:11:51.644994+0000","last_became_peered":"2024-03-22T04:11:51.644994+0000","last_unstale":"2024-03-22T04:11:54.132281+0000","last_undegraded":"2024-03-22T04:11:54.132281+0000","last_fullsized":"2024-03-22T04:11:54.132281+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:36:32.547421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656632+0000","last_change":"2024-03-22T04:11:52.646886+0000","last_active":"2024-03-22T04:11:53.656632+0000","last_peered":"2024-03-22T04:11:53.656632+0000","last_clean":"2024-03-22T04:11:53.656632+0000","last_became_active":"2024-03-22T04:11:52.645310+0000","last_became_peered":"2024-03-22T04:11:52.645310+0000","last_unstale":"2024-03-22T04:11:53.656632+0000","last_undegraded":"2024-03-22T04:11:53.656632+0000","last_fullsized":"2024-03-22T04:11:53.656632+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:25:47.554962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131882+0000","last_change":"2024-03-22T04:11:51.638270+0000","last_active":"2024-03-22T04:11:54.131882+0000","last_peered":"2024-03-22T04:11:54.131882+0000","last_clean":"2024-03-22T04:11:54.131882+0000","last_became_active":"2024-03-22T04:11:51.637766+0000","last_became_peered":"2024-03-22T04:11:51.637766+0000","last_unstale":"2024-03-22T04:11:54.131882+0000","last_undegraded":"2024-03-22T04:11:54.131882+0000","last_fullsized":"2024-03-22T04:11:54.131882+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:03.834934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.133289+0000","last_change":"2024-03-22T04:11:52.652881+0000","last_active":"2024-03-22T04:11:54.133289+0000","last_peered":"2024-03-22T04:11:54.133289+0000","last_clean":"2024-03-22T04:11:54.133289+0000","last_became_active":"2024-03-22T04:11:52.652190+0000","last_became_peered":"2024-03-22T04:11:52.652190+0000","last_unstale":"2024-03-22T04:11:54.133289+0000","last_undegraded":"2024-03-22T04:11:54.133289+0000","last_fullsized":"2024-03-22T04:11:54.133289+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:27:00.332973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.133373+0000","last_change":"2024-03-22T04:11:51.635044+0000","last_active":"2024-03-22T04:11:54.133373+0000","last_peered":"2024-03-22T04:11:54.133373+0000","last_clean":"2024-03-22T04:11:54.133373+0000","last_became_active":"2024-03-22T04:11:51.633837+0000","last_became_peered":"2024-03-22T04:11:51.633837+0000","last_unstale":"2024-03-22T04:11:54.133373+0000","last_undegraded":"2024-03-22T04:11:54.133373+0000","last_fullsized":"2024-03-22T04:11:54.133373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:31:32.200080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131134+0000","last_change":"2024-03-22T04:11:52.659709+0000","last_active":"2024-03-22T04:11:54.131134+0000","last_peered":"2024-03-22T04:11:54.131134+0000","last_clean":"2024-03-22T04:11:54.131134+0000","last_became_active":"2024-03-22T04:11:52.659232+0000","last_became_peered":"2024-03-22T04:11:52.659232+0000","last_unstale":"2024-03-22T04:11:54.131134+0000","last_undegraded":"2024-03-22T04:11:54.131134+0000","last_fullsized":"2024-03-22T04:11:54.131134+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:11:25.368184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653983+0000","last_change":"2024-03-22T04:11:51.637015+0000","last_active":"2024-03-22T04:11:53.653983+0000","last_peered":"2024-03-22T04:11:53.653983+0000","last_clean":"2024-03-22T04:11:53.653983+0000","last_became_active":"2024-03-22T04:11:51.636733+0000","last_became_peered":"2024-03-22T04:11:51.636733+0000","last_unstale":"2024-03-22T04:11:53.653983+0000","last_undegraded":"2024-03-22T04:11:53.653983+0000","last_fullsized":"2024-03-22T04:11:53.653983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:23:50.026312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.133009+0000","last_change":"2024-03-22T04:11:52.656994+0000","last_active":"2024-03-22T04:11:54.133009+0000","last_peered":"2024-03-22T04:11:54.133009+0000","last_clean":"2024-03-22T04:11:54.133009+0000","last_became_active":"2024-03-22T04:11:52.656688+0000","last_became_peered":"2024-03-22T04:11:52.656688+0000","last_unstale":"2024-03-22T04:11:54.133009+0000","last_undegraded":"2024-03-22T04:11:54.133009+0000","last_fullsized":"2024-03-22T04:11:54.133009+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:45:26.372631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653844+0000","last_change":"2024-03-22T04:11:51.634042+0000","last_active":"2024-03-22T04:11:53.653844+0000","last_peered":"2024-03-22T04:11:53.653844+0000","last_clean":"2024-03-22T04:11:53.653844+0000","last_became_active":"2024-03-22T04:11:51.633780+0000","last_became_peered":"2024-03-22T04:11:51.633780+0000","last_unstale":"2024-03-22T04:11:53.653844+0000","last_undegraded":"2024-03-22T04:11:53.653844+0000","last_fullsized":"2024-03-22T04:11:53.653844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:56:44.700396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131921+0000","last_change":"2024-03-22T04:11:52.659065+0000","last_active":"2024-03-22T04:11:54.131921+0000","last_peered":"2024-03-22T04:11:54.131921+0000","last_clean":"2024-03-22T04:11:54.131921+0000","last_became_active":"2024-03-22T04:11:52.658028+0000","last_became_peered":"2024-03-22T04:11:52.658028+0000","last_unstale":"2024-03-22T04:11:54.131921+0000","last_undegraded":"2024-03-22T04:11:54.131921+0000","last_fullsized":"2024-03-22T04:11:54.131921+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:44:22.160540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.133375+0000","last_change":"2024-03-22T04:11:51.644715+0000","last_active":"2024-03-22T04:11:54.133375+0000","last_peered":"2024-03-22T04:11:54.133375+0000","last_clean":"2024-03-22T04:11:54.133375+0000","last_became_active":"2024-03-22T04:11:51.644365+0000","last_became_peered":"2024-03-22T04:11:51.644365+0000","last_unstale":"2024-03-22T04:11:54.133375+0000","last_undegraded":"2024-03-22T04:11:54.133375+0000","last_fullsized":"2024-03-22T04:11:54.133375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:49:10.202707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132580+0000","last_change":"2024-03-22T04:11:52.646801+0000","last_active":"2024-03-22T04:11:54.132580+0000","last_peered":"2024-03-22T04:11:54.132580+0000","last_clean":"2024-03-22T04:11:54.132580+0000","last_became_active":"2024-03-22T04:11:52.646491+0000","last_became_peered":"2024-03-22T04:11:52.646491+0000","last_unstale":"2024-03-22T04:11:54.132580+0000","last_undegraded":"2024-03-22T04:11:54.132580+0000","last_fullsized":"2024-03-22T04:11:54.132580+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:44:24.434846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132951+0000","last_change":"2024-03-22T04:11:51.635750+0000","last_active":"2024-03-22T04:11:54.132951+0000","last_peered":"2024-03-22T04:11:54.132951+0000","last_clean":"2024-03-22T04:11:54.132951+0000","last_became_active":"2024-03-22T04:11:51.634686+0000","last_became_peered":"2024-03-22T04:11:51.634686+0000","last_unstale":"2024-03-22T04:11:54.132951+0000","last_undegraded":"2024-03-22T04:11:54.132951+0000","last_fullsized":"2024-03-22T04:11:54.132951+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:56:45.221222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655892+0000","last_change":"2024-03-22T04:11:52.651829+0000","last_active":"2024-03-22T04:11:53.655892+0000","last_peered":"2024-03-22T04:11:53.655892+0000","last_clean":"2024-03-22T04:11:53.655892+0000","last_became_active":"2024-03-22T04:11:52.651513+0000","last_became_peered":"2024-03-22T04:11:52.651513+0000","last_unstale":"2024-03-22T04:11:53.655892+0000","last_undegraded":"2024-03-22T04:11:53.655892+0000","last_fullsized":"2024-03-22T04:11:53.655892+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:18:38.492478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656151+0000","last_change":"2024-03-22T04:11:51.634149+0000","last_active":"2024-03-22T04:11:53.656151+0000","last_peered":"2024-03-22T04:11:53.656151+0000","last_clean":"2024-03-22T04:11:53.656151+0000","last_became_active":"2024-03-22T04:11:51.633789+0000","last_became_peered":"2024-03-22T04:11:51.633789+0000","last_unstale":"2024-03-22T04:11:53.656151+0000","last_undegraded":"2024-03-22T04:11:53.656151+0000","last_fullsized":"2024-03-22T04:11:53.656151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:48:58.697949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661610+0000","last_change":"2024-03-22T04:11:52.665785+0000","last_active":"2024-03-22T04:11:53.661610+0000","last_peered":"2024-03-22T04:11:53.661610+0000","last_clean":"2024-03-22T04:11:53.661610+0000","last_became_active":"2024-03-22T04:11:52.665484+0000","last_became_peered":"2024-03-22T04:11:52.665484+0000","last_unstale":"2024-03-22T04:11:53.661610+0000","last_undegraded":"2024-03-22T04:11:53.661610+0000","last_fullsized":"2024-03-22T04:11:53.661610+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:01:57.811504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653648+0000","last_change":"2024-03-22T04:11:51.641650+0000","last_active":"2024-03-22T04:11:53.653648+0000","last_peered":"2024-03-22T04:11:53.653648+0000","last_clean":"2024-03-22T04:11:53.653648+0000","last_became_active":"2024-03-22T04:11:51.641303+0000","last_became_peered":"2024-03-22T04:11:51.641303+0000","last_unstale":"2024-03-22T04:11:53.653648+0000","last_undegraded":"2024-03-22T04:11:53.653648+0000","last_fullsized":"2024-03-22T04:11:53.653648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:32:52.323037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655566+0000","last_change":"2024-03-22T04:11:52.661358+0000","last_active":"2024-03-22T04:11:53.655566+0000","last_peered":"2024-03-22T04:11:53.655566+0000","last_clean":"2024-03-22T04:11:53.655566+0000","last_became_active":"2024-03-22T04:11:52.660998+0000","last_became_peered":"2024-03-22T04:11:52.660998+0000","last_unstale":"2024-03-22T04:11:53.655566+0000","last_undegraded":"2024-03-22T04:11:53.655566+0000","last_fullsized":"2024-03-22T04:11:53.655566+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:48:59.705079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131112+0000","last_change":"2024-03-22T04:11:51.635563+0000","last_active":"2024-03-22T04:11:54.131112+0000","last_peered":"2024-03-22T04:11:54.131112+0000","last_clean":"2024-03-22T04:11:54.131112+0000","last_became_active":"2024-03-22T04:11:51.635265+0000","last_became_peered":"2024-03-22T04:11:51.635265+0000","last_unstale":"2024-03-22T04:11:54.131112+0000","last_undegraded":"2024-03-22T04:11:54.131112+0000","last_fullsized":"2024-03-22T04:11:54.131112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:22:59.050620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656566+0000","last_change":"2024-03-22T04:11:52.664104+0000","last_active":"2024-03-22T04:11:53.656566+0000","last_peered":"2024-03-22T04:11:53.656566+0000","last_clean":"2024-03-22T04:11:53.656566+0000","last_became_active":"2024-03-22T04:11:52.663837+0000","last_became_peered":"2024-03-22T04:11:52.663837+0000","last_unstale":"2024-03-22T04:11:53.656566+0000","last_undegraded":"2024-03-22T04:11:53.656566+0000","last_fullsized":"2024-03-22T04:11:53.656566+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:13:24.369315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131741+0000","last_change":"2024-03-22T04:11:51.638377+0000","last_active":"2024-03-22T04:11:54.131741+0000","last_peered":"2024-03-22T04:11:54.131741+0000","last_clean":"2024-03-22T04:11:54.131741+0000","last_became_active":"2024-03-22T04:11:51.638093+0000","last_became_peered":"2024-03-22T04:11:51.638093+0000","last_unstale":"2024-03-22T04:11:54.131741+0000","last_undegraded":"2024-03-22T04:11:54.131741+0000","last_fullsized":"2024-03-22T04:11:54.131741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:42:02.600608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131790+0000","last_change":"2024-03-22T04:11:52.657458+0000","last_active":"2024-03-22T04:11:54.131790+0000","last_peered":"2024-03-22T04:11:54.131790+0000","last_clean":"2024-03-22T04:11:54.131790+0000","last_became_active":"2024-03-22T04:11:52.657203+0000","last_became_peered":"2024-03-22T04:11:52.657203+0000","last_unstale":"2024-03-22T04:11:54.131790+0000","last_undegraded":"2024-03-22T04:11:54.131790+0000","last_fullsized":"2024-03-22T04:11:54.131790+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:52:01.031151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.654068+0000","last_change":"2024-03-22T04:11:51.633320+0000","last_active":"2024-03-22T04:11:53.654068+0000","last_peered":"2024-03-22T04:11:53.654068+0000","last_clean":"2024-03-22T04:11:53.654068+0000","last_became_active":"2024-03-22T04:11:51.633063+0000","last_became_peered":"2024-03-22T04:11:51.633063+0000","last_unstale":"2024-03-22T04:11:53.654068+0000","last_undegraded":"2024-03-22T04:11:53.654068+0000","last_fullsized":"2024-03-22T04:11:53.654068+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:38:18.241111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131860+0000","last_change":"2024-03-22T04:11:52.663760+0000","last_active":"2024-03-22T04:11:54.131860+0000","last_peered":"2024-03-22T04:11:54.131860+0000","last_clean":"2024-03-22T04:11:54.131860+0000","last_became_active":"2024-03-22T04:11:52.663478+0000","last_became_peered":"2024-03-22T04:11:52.663478+0000","last_unstale":"2024-03-22T04:11:54.131860+0000","last_undegraded":"2024-03-22T04:11:54.131860+0000","last_fullsized":"2024-03-22T04:11:54.131860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:23:07.213903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655686+0000","last_change":"2024-03-22T04:11:51.644988+0000","last_active":"2024-03-22T04:11:53.655686+0000","last_peered":"2024-03-22T04:11:53.655686+0000","last_clean":"2024-03-22T04:11:53.655686+0000","last_became_active":"2024-03-22T04:11:51.644707+0000","last_became_peered":"2024-03-22T04:11:51.644707+0000","last_unstale":"2024-03-22T04:11:53.655686+0000","last_undegraded":"2024-03-22T04:11:53.655686+0000","last_fullsized":"2024-03-22T04:11:53.655686+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:53:29.162898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132835+0000","last_change":"2024-03-22T04:11:52.658468+0000","last_active":"2024-03-22T04:11:54.132835+0000","last_peered":"2024-03-22T04:11:54.132835+0000","last_clean":"2024-03-22T04:11:54.132835+0000","last_became_active":"2024-03-22T04:11:52.657183+0000","last_became_peered":"2024-03-22T04:11:52.657183+0000","last_unstale":"2024-03-22T04:11:54.132835+0000","last_undegraded":"2024-03-22T04:11:54.132835+0000","last_fullsized":"2024-03-22T04:11:54.132835+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:25:47.554962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132342+0000","last_change":"2024-03-22T04:11:51.634391+0000","last_active":"2024-03-22T04:11:54.132342+0000","last_peered":"2024-03-22T04:11:54.132342+0000","last_clean":"2024-03-22T04:11:54.132342+0000","last_became_active":"2024-03-22T04:11:51.634111+0000","last_became_peered":"2024-03-22T04:11:51.634111+0000","last_unstale":"2024-03-22T04:11:54.132342+0000","last_undegraded":"2024-03-22T04:11:54.132342+0000","last_fullsized":"2024-03-22T04:11:54.132342+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:03:04.053607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653525+0000","last_change":"2024-03-22T04:11:52.645037+0000","last_active":"2024-03-22T04:11:53.653525+0000","last_peered":"2024-03-22T04:11:53.653525+0000","last_clean":"2024-03-22T04:11:53.653525+0000","last_became_active":"2024-03-22T04:11:52.644771+0000","last_became_peered":"2024-03-22T04:11:52.644771+0000","last_unstale":"2024-03-22T04:11:53.653525+0000","last_undegraded":"2024-03-22T04:11:53.653525+0000","last_fullsized":"2024-03-22T04:11:53.653525+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:10.393113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132444+0000","last_change":"2024-03-22T04:11:51.634888+0000","last_active":"2024-03-22T04:11:54.132444+0000","last_peered":"2024-03-22T04:11:54.132444+0000","last_clean":"2024-03-22T04:11:54.132444+0000","last_became_active":"2024-03-22T04:11:51.633782+0000","last_became_peered":"2024-03-22T04:11:51.633782+0000","last_unstale":"2024-03-22T04:11:54.132444+0000","last_undegraded":"2024-03-22T04:11:54.132444+0000","last_fullsized":"2024-03-22T04:11:54.132444+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:02:02.722846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132307+0000","last_change":"2024-03-22T04:11:52.657565+0000","last_active":"2024-03-22T04:11:54.132307+0000","last_peered":"2024-03-22T04:11:54.132307+0000","last_clean":"2024-03-22T04:11:54.132307+0000","last_became_active":"2024-03-22T04:11:52.657329+0000","last_became_peered":"2024-03-22T04:11:52.657329+0000","last_unstale":"2024-03-22T04:11:54.132307+0000","last_undegraded":"2024-03-22T04:11:54.132307+0000","last_fullsized":"2024-03-22T04:11:54.132307+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:59:29.943728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.656076+0000","last_change":"2024-03-22T04:11:51.640816+0000","last_active":"2024-03-22T04:11:53.656076+0000","last_peered":"2024-03-22T04:11:53.656076+0000","last_clean":"2024-03-22T04:11:53.656076+0000","last_became_active":"2024-03-22T04:11:51.640560+0000","last_became_peered":"2024-03-22T04:11:51.640560+0000","last_unstale":"2024-03-22T04:11:53.656076+0000","last_undegraded":"2024-03-22T04:11:53.656076+0000","last_fullsized":"2024-03-22T04:11:53.656076+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:22:59.050620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661348+0000","last_change":"2024-03-22T04:11:52.661321+0000","last_active":"2024-03-22T04:11:53.661348+0000","last_peered":"2024-03-22T04:11:53.661348+0000","last_clean":"2024-03-22T04:11:53.661348+0000","last_became_active":"2024-03-22T04:11:52.660848+0000","last_became_peered":"2024-03-22T04:11:52.660848+0000","last_unstale":"2024-03-22T04:11:53.661348+0000","last_undegraded":"2024-03-22T04:11:53.661348+0000","last_fullsized":"2024-03-22T04:11:53.661348+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:45:14.997775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131508+0000","last_change":"2024-03-22T04:11:51.638060+0000","last_active":"2024-03-22T04:11:54.131508+0000","last_peered":"2024-03-22T04:11:54.131508+0000","last_clean":"2024-03-22T04:11:54.131508+0000","last_became_active":"2024-03-22T04:11:51.637774+0000","last_became_peered":"2024-03-22T04:11:51.637774+0000","last_unstale":"2024-03-22T04:11:54.131508+0000","last_undegraded":"2024-03-22T04:11:54.131508+0000","last_fullsized":"2024-03-22T04:11:54.131508+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:48.280840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132846+0000","last_change":"2024-03-22T04:11:52.657915+0000","last_active":"2024-03-22T04:11:54.132846+0000","last_peered":"2024-03-22T04:11:54.132846+0000","last_clean":"2024-03-22T04:11:54.132846+0000","last_became_active":"2024-03-22T04:11:52.657728+0000","last_became_peered":"2024-03-22T04:11:52.657728+0000","last_unstale":"2024-03-22T04:11:54.132846+0000","last_undegraded":"2024-03-22T04:11:54.132846+0000","last_fullsized":"2024-03-22T04:11:54.132846+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:10:46.917667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655308+0000","last_change":"2024-03-22T04:11:51.632186+0000","last_active":"2024-03-22T04:11:53.655308+0000","last_peered":"2024-03-22T04:11:53.655308+0000","last_clean":"2024-03-22T04:11:53.655308+0000","last_became_active":"2024-03-22T04:11:51.631907+0000","last_became_peered":"2024-03-22T04:11:51.631907+0000","last_unstale":"2024-03-22T04:11:53.655308+0000","last_undegraded":"2024-03-22T04:11:53.655308+0000","last_fullsized":"2024-03-22T04:11:53.655308+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:10:15.275541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655767+0000","last_change":"2024-03-22T04:11:52.661682+0000","last_active":"2024-03-22T04:11:53.655767+0000","last_peered":"2024-03-22T04:11:53.655767+0000","last_clean":"2024-03-22T04:11:53.655767+0000","last_became_active":"2024-03-22T04:11:52.661384+0000","last_became_peered":"2024-03-22T04:11:52.661384+0000","last_unstale":"2024-03-22T04:11:53.655767+0000","last_undegraded":"2024-03-22T04:11:53.655767+0000","last_fullsized":"2024-03-22T04:11:53.655767+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:52:01.031151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.654041+0000","last_change":"2024-03-22T04:11:51.633301+0000","last_active":"2024-03-22T04:11:53.654041+0000","last_peered":"2024-03-22T04:11:53.654041+0000","last_clean":"2024-03-22T04:11:53.654041+0000","last_became_active":"2024-03-22T04:11:51.633017+0000","last_became_peered":"2024-03-22T04:11:51.633017+0000","last_unstale":"2024-03-22T04:11:53.654041+0000","last_undegraded":"2024-03-22T04:11:53.654041+0000","last_fullsized":"2024-03-22T04:11:53.654041+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:15:18.759040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661989+0000","last_change":"2024-03-22T04:11:52.656332+0000","last_active":"2024-03-22T04:11:53.661989+0000","last_peered":"2024-03-22T04:11:53.661989+0000","last_clean":"2024-03-22T04:11:53.661989+0000","last_became_active":"2024-03-22T04:11:52.656036+0000","last_became_peered":"2024-03-22T04:11:52.656036+0000","last_unstale":"2024-03-22T04:11:53.661989+0000","last_undegraded":"2024-03-22T04:11:53.661989+0000","last_fullsized":"2024-03-22T04:11:53.661989+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:08.306240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655834+0000","last_change":"2024-03-22T04:11:51.644074+0000","last_active":"2024-03-22T04:11:53.655834+0000","last_peered":"2024-03-22T04:11:53.655834+0000","last_clean":"2024-03-22T04:11:53.655834+0000","last_became_active":"2024-03-22T04:11:51.643795+0000","last_became_peered":"2024-03-22T04:11:51.643795+0000","last_unstale":"2024-03-22T04:11:53.655834+0000","last_undegraded":"2024-03-22T04:11:53.655834+0000","last_fullsized":"2024-03-22T04:11:53.655834+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:28:33.417099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132503+0000","last_change":"2024-03-22T04:11:52.650532+0000","last_active":"2024-03-22T04:11:54.132503+0000","last_peered":"2024-03-22T04:11:54.132503+0000","last_clean":"2024-03-22T04:11:54.132503+0000","last_became_active":"2024-03-22T04:11:52.650306+0000","last_became_peered":"2024-03-22T04:11:52.650306+0000","last_unstale":"2024-03-22T04:11:54.132503+0000","last_undegraded":"2024-03-22T04:11:54.132503+0000","last_fullsized":"2024-03-22T04:11:54.132503+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:13:24.369315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655796+0000","last_change":"2024-03-22T04:11:51.639226+0000","last_active":"2024-03-22T04:11:53.655796+0000","last_peered":"2024-03-22T04:11:53.655796+0000","last_clean":"2024-03-22T04:11:53.655796+0000","last_became_active":"2024-03-22T04:11:51.638927+0000","last_became_peered":"2024-03-22T04:11:51.638927+0000","last_unstale":"2024-03-22T04:11:53.655796+0000","last_undegraded":"2024-03-22T04:11:53.655796+0000","last_fullsized":"2024-03-22T04:11:53.655796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:11:14.812727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655207+0000","last_change":"2024-03-22T04:11:52.663709+0000","last_active":"2024-03-22T04:11:53.655207+0000","last_peered":"2024-03-22T04:11:53.655207+0000","last_clean":"2024-03-22T04:11:53.655207+0000","last_became_active":"2024-03-22T04:11:52.663292+0000","last_became_peered":"2024-03-22T04:11:52.663292+0000","last_unstale":"2024-03-22T04:11:53.655207+0000","last_undegraded":"2024-03-22T04:11:53.655207+0000","last_fullsized":"2024-03-22T04:11:53.655207+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:10:16.282670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.133126+0000","last_change":"2024-03-22T04:11:51.635806+0000","last_active":"2024-03-22T04:11:54.133126+0000","last_peered":"2024-03-22T04:11:54.133126+0000","last_clean":"2024-03-22T04:11:54.133126+0000","last_became_active":"2024-03-22T04:11:51.633614+0000","last_became_peered":"2024-03-22T04:11:51.633614+0000","last_unstale":"2024-03-22T04:11:54.133126+0000","last_undegraded":"2024-03-22T04:11:54.133126+0000","last_fullsized":"2024-03-22T04:11:54.133126+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:29:14.289312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.655915+0000","last_change":"2024-03-22T04:11:52.660518+0000","last_active":"2024-03-22T04:11:53.655915+0000","last_peered":"2024-03-22T04:11:53.655915+0000","last_clean":"2024-03-22T04:11:53.655915+0000","last_became_active":"2024-03-22T04:11:52.660262+0000","last_became_peered":"2024-03-22T04:11:52.660262+0000","last_unstale":"2024-03-22T04:11:53.655915+0000","last_undegraded":"2024-03-22T04:11:53.655915+0000","last_fullsized":"2024-03-22T04:11:53.655915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:05.323006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661486+0000","last_change":"2024-03-22T04:11:51.639131+0000","last_active":"2024-03-22T04:11:53.661486+0000","last_peered":"2024-03-22T04:11:53.661486+0000","last_clean":"2024-03-22T04:11:53.661486+0000","last_became_active":"2024-03-22T04:11:51.638728+0000","last_became_peered":"2024-03-22T04:11:51.638728+0000","last_unstale":"2024-03-22T04:11:53.661486+0000","last_undegraded":"2024-03-22T04:11:53.661486+0000","last_fullsized":"2024-03-22T04:11:53.661486+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:44:21.153411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653793+0000","last_change":"2024-03-22T04:11:52.643385+0000","last_active":"2024-03-22T04:11:53.653793+0000","last_peered":"2024-03-22T04:11:53.653793+0000","last_clean":"2024-03-22T04:11:53.653793+0000","last_became_active":"2024-03-22T04:11:52.643071+0000","last_became_peered":"2024-03-22T04:11:52.643071+0000","last_unstale":"2024-03-22T04:11:53.653793+0000","last_undegraded":"2024-03-22T04:11:53.653793+0000","last_fullsized":"2024-03-22T04:11:53.653793+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:37:02.554632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653744+0000","last_change":"2024-03-22T04:11:51.632239+0000","last_active":"2024-03-22T04:11:53.653744+0000","last_peered":"2024-03-22T04:11:53.653744+0000","last_clean":"2024-03-22T04:11:53.653744+0000","last_became_active":"2024-03-22T04:11:51.631969+0000","last_became_peered":"2024-03-22T04:11:51.631969+0000","last_unstale":"2024-03-22T04:11:53.653744+0000","last_undegraded":"2024-03-22T04:11:53.653744+0000","last_fullsized":"2024-03-22T04:11:53.653744+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:59.446983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.662175+0000","last_change":"2024-03-22T04:11:52.653989+0000","last_active":"2024-03-22T04:11:53.662175+0000","last_peered":"2024-03-22T04:11:53.662175+0000","last_clean":"2024-03-22T04:11:53.662175+0000","last_became_active":"2024-03-22T04:11:52.653722+0000","last_became_peered":"2024-03-22T04:11:52.653722+0000","last_unstale":"2024-03-22T04:11:53.662175+0000","last_undegraded":"2024-03-22T04:11:53.662175+0000","last_fullsized":"2024-03-22T04:11:53.662175+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:58:21.035354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.132729+0000","last_change":"2024-03-22T04:11:51.641613+0000","last_active":"2024-03-22T04:11:54.132729+0000","last_peered":"2024-03-22T04:11:54.132729+0000","last_clean":"2024-03-22T04:11:54.132729+0000","last_became_active":"2024-03-22T04:11:51.641256+0000","last_became_peered":"2024-03-22T04:11:51.641256+0000","last_unstale":"2024-03-22T04:11:54.132729+0000","last_undegraded":"2024-03-22T04:11:54.132729+0000","last_fullsized":"2024-03-22T04:11:54.132729+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:58:27.877295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.653987+0000","last_change":"2024-03-22T04:11:52.658436+0000","last_active":"2024-03-22T04:11:53.653987+0000","last_peered":"2024-03-22T04:11:53.653987+0000","last_clean":"2024-03-22T04:11:53.653987+0000","last_became_active":"2024-03-22T04:11:52.658166+0000","last_became_peered":"2024-03-22T04:11:52.658166+0000","last_unstale":"2024-03-22T04:11:53.653987+0000","last_undegraded":"2024-03-22T04:11:53.653987+0000","last_fullsized":"2024-03-22T04:11:53.653987+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:51.610479+0000","last_clean_scrub_stamp":"2024-03-22T04:11:51.610479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:23:31.979507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:53.661483+0000","last_change":"2024-03-22T04:11:51.637773+0000","last_active":"2024-03-22T04:11:53.661483+0000","last_peered":"2024-03-22T04:11:53.661483+0000","last_clean":"2024-03-22T04:11:53.661483+0000","last_became_active":"2024-03-22T04:11:51.637519+0000","last_became_peered":"2024-03-22T04:11:51.637519+0000","last_unstale":"2024-03-22T04:11:53.661483+0000","last_undegraded":"2024-03-22T04:11:53.661483+0000","last_fullsized":"2024-03-22T04:11:53.661483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:11:50.603350+0000","last_clean_scrub_stamp":"2024-03-22T04:11:50.603350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:22.199529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":202,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-22T04:11:54.131908+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:11:54.131908+0000","last_peered":"2024-03-22T04:11:54.131908+0000","last_clean":"2024-03-22T04:11:54.131908+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:11:54.131908+0000","last_undegraded":"2024-03-22T04:11:54.131908+0000","last_fullsized":"2024-03-22T04:11:54.131908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":199921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":199885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1064960,"data_stored":790801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":199941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":202045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1064960,"data_stored":790241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":199943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":201587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:12:01.586 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T04:12:01.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-22T04:12:01.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccc6200640 1 Processor -- start 2024-03-22T04:12:01.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccc6200640 1 -- start start 2024-03-22T04:12:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccc6200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc015c010 0x7fccc00d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccc6200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fccc00d28f0 0x7fccc015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccc6200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fccc015e980 0x7fccc006dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccc6200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fccc0161f50 con 0x7fccc015e980 2024-03-22T04:12:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccc6200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fccc015ee60 con 0x7fccc00d28f0 2024-03-22T04:12:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccc6200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fccc015f140 con 0x7fccc015c010 2024-03-22T04:12:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fccc015e980 0x7fccc006dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fccc00d28f0 0x7fccc015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbf400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc015c010 0x7fccc00d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fccc00d28f0 0x7fccc015f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:42260/0 (socket says 172.21.15.86:42260) 2024-03-22T04:12:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbea00640 1 -- 172.21.15.86:0/2022929398 learned_addr learned my addr 172.21.15.86:0/2022929398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbea00640 1 -- 172.21.15.86:0/2022929398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc015c010 msgr2=0x7fccc00d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbea00640 1 --2- 172.21.15.86:0/2022929398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc015c010 0x7fccc00d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbea00640 1 -- 172.21.15.86:0/2022929398 >> v2:172.21.15.86:3300/0 conn(0x7fccc015e980 msgr2=0x7fccc006dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbea00640 1 --2- 172.21.15.86:0/2022929398 >> v2:172.21.15.86:3300/0 conn(0x7fccc015e980 0x7fccc006dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.766+0000 7fccbea00640 1 -- 172.21.15.86:0/2022929398 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc015f420 con 0x7fccc00d28f0 2024-03-22T04:12:01.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccbea00640 1 --2- 172.21.15.86:0/2022929398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc00d28f0 0x7fccc015f9e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fccc00cfc40 tx=0x7fccac01c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c904574453ec6456 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:01.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccbe000640 1 -- 172.21.15.86:0/2022929398 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0aa430 con 0x7fccc00d28f0 2024-03-22T04:12:01.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccbe000640 1 -- 172.21.15.86:0/2022929398 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccac0af070 con 0x7fccc00d28f0 2024-03-22T04:12:01.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccbe000640 1 -- 172.21.15.86:0/2022929398 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0aa7f0 con 0x7fccc00d28f0 2024-03-22T04:12:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 -- 172.21.15.86:0/2022929398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc00d28f0 msgr2=0x7fccc015f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 --2- 172.21.15.86:0/2022929398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc00d28f0 0x7fccc015f9e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fccc00cfc40 tx=0x7fccac01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 -- 172.21.15.86:0/2022929398 shutdown_connections 2024-03-22T04:12:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 --2- 172.21.15.86:0/2022929398 >> v2:172.21.15.86:3300/0 conn(0x7fccc015e980 0x7fccc006dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 --2- 172.21.15.86:0/2022929398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc00d28f0 0x7fccc015f9e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 --2- 172.21.15.86:0/2022929398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc015c010 0x7fccc00d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 -- 172.21.15.86:0/2022929398 >> 172.21.15.86:0/2022929398 conn(0x7fccc006b660 msgr2=0x7fccc006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 -- 172.21.15.86:0/2022929398 shutdown_connections 2024-03-22T04:12:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.770+0000 7fccc6200640 1 -- 172.21.15.86:0/2022929398 wait complete. 2024-03-22T04:12:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 Processor -- start 2024-03-22T04:12:01.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 -- start start 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccc015c010 0x7fccc00c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc015e980 0x7fccc00c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc01586a0 0x7fccc0158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fccc006cb00 con 0x7fccc015c010 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccc006c980 con 0x7fccc015e980 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fccc006cc80 con 0x7fccc01586a0 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccc015c010 0x7fccc00c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccc015c010 0x7fccc00c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36776/0 (socket says 172.21.15.86:36776) 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 -- 172.21.15.86:0/2618888033 learned_addr learned my addr 172.21.15.86:0/2618888033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbfe00640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc01586a0 0x7fccc0158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbea00640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc015e980 0x7fccc00c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 -- 172.21.15.86:0/2618888033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc01586a0 msgr2=0x7fccc0158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc01586a0 0x7fccc0158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 -- 172.21.15.86:0/2618888033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc015e980 msgr2=0x7fccc00c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc015e980 0x7fccc00c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 -- 172.21.15.86:0/2618888033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc01fb310 con 0x7fccc015c010 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccbf400640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccc015c010 0x7fccc00c5c50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e930 tx=0x7fccb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccb7400640 1 -- 172.21.15.86:0/2618888033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0017070 con 0x7fccc015c010 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccb7400640 1 -- 172.21.15.86:0/2618888033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb000ca00 con 0x7fccc015c010 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccb7400640 1 -- 172.21.15.86:0/2618888033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb000cdb0 con 0x7fccc015c010 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc01fb5f0 con 0x7fccc015c010 2024-03-22T04:12:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.774+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc01fbbb0 con 0x7fccc015c010 2024-03-22T04:12:01.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.778+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc0074470 con 0x7fccc015c010 2024-03-22T04:12:01.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.778+0000 7fccb7400640 1 -- 172.21.15.86:0/2618888033 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fccb001a070 con 0x7fccc015c010 2024-03-22T04:12:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.778+0000 7fccb7400640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcca0067b20 0x7fcca0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.778+0000 7fccb7400640 1 -- 172.21.15.86:0/2618888033 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fccb0085ff0 con 0x7fccc015c010 2024-03-22T04:12:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.778+0000 7fccbea00640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcca0067b20 0x7fcca0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.778+0000 7fccbea00640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcca0067b20 0x7fcca0069fe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccac0ae000 tx=0x7fccac0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:01.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.782+0000 7fccb7400640 1 -- 172.21.15.86:0/2618888033 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fccb0056310 con 0x7fccc015c010 2024-03-22T04:12:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.994+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fccc0070300 con 0x7fccc015c010 2024-03-22T04:12:02.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.994+0000 7fccb7400640 1 -- 172.21.15.86:0/2618888033 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fccb005a3b0 con 0x7fccc015c010 2024-03-22T04:12:02.002 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T04:12:02.002 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:12:02.002 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-03-22T04:12:02.002 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-03-22T04:12:02.002 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-22T04:12:02.002 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcca0067b20 msgr2=0x7fcca0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcca0067b20 0x7fcca0069fe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccac0ae000 tx=0x7fccac0a7000 comp rx=0 tx=0).stop 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccc015c010 msgr2=0x7fccc00c5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccc015c010 0x7fccc00c5c50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e930 tx=0x7fccb000edf0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 shutdown_connections 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcca0067b20 0x7fcca0069fe0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc015e980 0x7fccc00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccc01586a0 0x7fccc0158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 --2- 172.21.15.86:0/2618888033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccc015c010 0x7fccc00c5c50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 >> 172.21.15.86:0/2618888033 conn(0x7fccc006b660 msgr2=0x7fccc0161220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 shutdown_connections 2024-03-22T04:12:02.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:01.998+0000 7fccc6200640 1 -- 172.21.15.86:0/2618888033 wait complete. 2024-03-22T04:12:02.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:02.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.150+0000 7efe41a00640 1 Processor -- start 2024-03-22T04:12:02.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.150+0000 7efe41a00640 1 -- start start 2024-03-22T04:12:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe41a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c15b0d0 0x7efe3c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe41a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe3c0b5320 0x7efe3c15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe41a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3c15bd30 con 0x7efe3c0b2570 2024-03-22T04:12:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe41a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efe3c0b22e0 con 0x7efe3c0b5320 2024-03-22T04:12:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe41a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c15c010 con 0x7efe3c15b0d0 2024-03-22T04:12:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe3a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efe3c0b2570 0x7efe3c0b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40400/0 (socket says 172.21.15.86:40400) 2024-03-22T04:12:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe3a000640 1 -- 172.21.15.86:0/2596397948 learned_addr learned my addr 172.21.15.86:0/2596397948 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe3b400640 1 --2- 172.21.15.86:0/2596397948 >> v2:172.21.15.196:3300/0 conn(0x7efe3c0b5320 0x7efe3c15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe3aa00640 1 --2- 172.21.15.86:0/2596397948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c15b0d0 0x7efe3c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe39600640 1 -- 172.21.15.86:0/2596397948 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307878976 0 0) 0x7efe3c15bd30 con 0x7efe3c0b2570 2024-03-22T04:12:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe39600640 1 -- 172.21.15.86:0/2596397948 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe20005740 con 0x7efe3c0b2570 2024-03-22T04:12:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe3b400640 1 -- 172.21.15.86:0/2596397948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c15b0d0 msgr2=0x7efe3c15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe3b400640 1 --2- 172.21.15.86:0/2596397948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c15b0d0 0x7efe3c15b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe3b400640 1 -- 172.21.15.86:0/2596397948 >> v1:172.21.15.86:6789/0 conn(0x7efe3c0b2570 legacy=0x7efe3c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.154+0000 7efe3b400640 1 -- 172.21.15.86:0/2596397948 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c15c2f0 con 0x7efe3c0b5320 2024-03-22T04:12:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe3b400640 1 --2- 172.21.15.86:0/2596397948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c0b5320 0x7efe3c15cdd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efe340097c0 tx=0x7efe3401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d77e6661f8fcf7b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe39600640 1 -- 172.21.15.86:0/2596397948 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340b2070 con 0x7efe3c0b5320 2024-03-22T04:12:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe39600640 1 -- 172.21.15.86:0/2596397948 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe340ad640 con 0x7efe3c0b5320 2024-03-22T04:12:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe39600640 1 -- 172.21.15.86:0/2596397948 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340ada10 con 0x7efe3c0b5320 2024-03-22T04:12:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 -- 172.21.15.86:0/2596397948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c0b5320 msgr2=0x7efe3c15cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 --2- 172.21.15.86:0/2596397948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c0b5320 0x7efe3c15cdd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efe340097c0 tx=0x7efe3401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 -- 172.21.15.86:0/2596397948 shutdown_connections 2024-03-22T04:12:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 --2- 172.21.15.86:0/2596397948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c0b5320 0x7efe3c15cdd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 --2- 172.21.15.86:0/2596397948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c15b0d0 0x7efe3c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 -- 172.21.15.86:0/2596397948 >> 172.21.15.86:0/2596397948 conn(0x7efe3c0b6620 msgr2=0x7efe3c0b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 -- 172.21.15.86:0/2596397948 shutdown_connections 2024-03-22T04:12:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 -- 172.21.15.86:0/2596397948 wait complete. 2024-03-22T04:12:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 Processor -- start 2024-03-22T04:12:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.158+0000 7efe41a00640 1 -- start start 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3c0b2570 0x7efe3c14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c0b5320 0x7efe3c0b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c15b0d0 0x7efe3c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c15dca0 con 0x7efe3c0b2570 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c15db20 con 0x7efe3c15b0d0 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c15de20 con 0x7efe3c0b5320 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3c0b2570 0x7efe3c14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3c0b2570 0x7efe3c14b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36782/0 (socket says 172.21.15.86:36782) 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 -- 172.21.15.86:0/3770607362 learned_addr learned my addr 172.21.15.86:0/3770607362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3a000640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c0b5320 0x7efe3c0b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3b400640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c15b0d0 0x7efe3c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 -- 172.21.15.86:0/3770607362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c0b5320 msgr2=0x7efe3c0b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c0b5320 0x7efe3c0b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 -- 172.21.15.86:0/3770607362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c15b0d0 msgr2=0x7efe3c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c15b0d0 0x7efe3c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 -- 172.21.15.86:0/3770607362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c159120 con 0x7efe3c0b2570 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe3aa00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3c0b2570 0x7efe3c14b200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00efc0 tx=0x7efe2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe32a00640 1 -- 172.21.15.86:0/3770607362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c018020 con 0x7efe3c0b2570 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe32a00640 1 -- 172.21.15.86:0/3770607362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c014480 con 0x7efe3c0b2570 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c159400 con 0x7efe3c0b2570 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe32a00640 1 -- 172.21.15.86:0/3770607362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c014850 con 0x7efe3c0b2570 2024-03-22T04:12:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c159ca0 con 0x7efe3c0b2570 2024-03-22T04:12:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c1595d0 con 0x7efe3c0b2570 2024-03-22T04:12:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.162+0000 7efe32a00640 1 -- 172.21.15.86:0/3770607362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efe2c01b020 con 0x7efe3c0b2570 2024-03-22T04:12:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.166+0000 7efe32a00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe1c067b50 0x7efe1c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.166+0000 7efe32a00640 1 -- 172.21.15.86:0/3770607362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7efe2c086a10 con 0x7efe3c0b2570 2024-03-22T04:12:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.166+0000 7efe3a000640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe1c067b50 0x7efe1c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.166+0000 7efe3a000640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe1c067b50 0x7efe1c06a010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe28007fd0 tx=0x7efe28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.170+0000 7efe32a00640 1 -- 172.21.15.86:0/3770607362 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efe2c022070 con 0x7efe3c0b2570 2024-03-22T04:12:02.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.354+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe3c068650 con 0x7efe3c0b2570 2024-03-22T04:12:02.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe32a00640 1 -- 172.21.15.86:0/3770607362 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7efe2c010040 con 0x7efe3c0b2570 2024-03-22T04:12:02.363 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:02.363 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:11:51.996418+0000","modified":"2024-03-22T04:11:55.030984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74930":{"gid":74930,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1426902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1426902111},{"type":"v1","addr":"172.21.15.196:6835","nonce":1426902111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74930,"qdb_cluster":[74930]},"id":2}]} 2024-03-22T04:12:02.363 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-03-22T04:12:02.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe1c067b50 msgr2=0x7efe1c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe1c067b50 0x7efe1c06a010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe28007fd0 tx=0x7efe28009040 comp rx=0 tx=0).stop 2024-03-22T04:12:02.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3c0b2570 msgr2=0x7efe3c14b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3c0b2570 0x7efe3c14b200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00efc0 tx=0x7efe2c00c490 comp rx=0 tx=0).stop 2024-03-22T04:12:02.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 shutdown_connections 2024-03-22T04:12:02.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe1c067b50 0x7efe1c06a010 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3c15b0d0 0x7efe3c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe3c0b5320 0x7efe3c0b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 --2- 172.21.15.86:0/3770607362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3c0b2570 0x7efe3c14b200 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 >> 172.21.15.86:0/3770607362 conn(0x7efe3c0b6620 msgr2=0x7efe3c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 shutdown_connections 2024-03-22T04:12:02.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.358+0000 7efe41a00640 1 -- 172.21.15.86:0/3770607362 wait complete. 2024-03-22T04:12:02.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:11:51.996418+0000', 'modified': '2024-03-22T04:11:55.030984+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 74930}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_74930': {'gid': 74930, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/1426902111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6833', 'nonce': 1426902111}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1426902111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 74930, 'qdb_cluster': [74930]} 2024-03-22T04:12:02.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:12:02.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T04:12:02.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f360b800640 1 Processor -- start 2024-03-22T04:12:02.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f360b800640 1 -- start start 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f360b800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f36040730d0 0x7f36040ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f360b800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f36040cc3e0 0x7f36040ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f360b800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3604071920 con 0x7f360415b3f0 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f360b800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f36040716d0 con 0x7f36040730d0 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f360b800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f360406d990 con 0x7f36040cc3e0 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f3608e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f360415b3f0 0x7f36040729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40408/0 (socket says 172.21.15.86:40408) 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f3608e00640 1 -- 172.21.15.86:0/3787802846 learned_addr learned my addr 172.21.15.86:0/3787802846 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f35fbe00640 1 --2- 172.21.15.86:0/3787802846 >> v2:172.21.15.86:3300/0 conn(0x7f36040730d0 0x7f36040ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f3609800640 1 --2- 172.21.15.86:0/3787802846 >> v2:172.21.15.196:3300/0 conn(0x7f36040cc3e0 0x7f36040ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f35fbe00640 1 -- 172.21.15.86:0/3787802846 >> v2:172.21.15.196:3300/0 conn(0x7f36040cc3e0 msgr2=0x7f36040ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f35fbe00640 1 --2- 172.21.15.86:0/3787802846 >> v2:172.21.15.196:3300/0 conn(0x7f36040cc3e0 0x7f36040ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f35fbe00640 1 -- 172.21.15.86:0/3787802846 >> v1:172.21.15.86:6789/0 conn(0x7f360415b3f0 legacy=0x7f36040729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f35fbe00640 1 -- 172.21.15.86:0/3787802846 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3604071c00 con 0x7f36040730d0 2024-03-22T04:12:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.922+0000 7f3603e00640 1 -- 172.21.15.86:0/3787802846 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369715050 0 0) 0x7f3604071920 con 0x7f360415b3f0 2024-03-22T04:12:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f35fbe00640 1 --2- 172.21.15.86:0/3787802846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f36040ade30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3604073b20 tx=0x7f35ec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb17a923b09796 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f3603e00640 1 -- 172.21.15.86:0/3787802846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0b0070 con 0x7f36040730d0 2024-03-22T04:12:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f3603e00640 1 -- 172.21.15.86:0/3787802846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec0ab600 con 0x7f36040730d0 2024-03-22T04:12:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f3603e00640 1 -- 172.21.15.86:0/3787802846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0ab9d0 con 0x7f36040730d0 2024-03-22T04:12:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f360b800640 1 -- 172.21.15.86:0/3787802846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 msgr2=0x7f36040ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f360b800640 1 --2- 172.21.15.86:0/3787802846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f36040ade30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3604073b20 tx=0x7f35ec01c480 comp rx=0 tx=0).stop 2024-03-22T04:12:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f360b800640 1 -- 172.21.15.86:0/3787802846 shutdown_connections 2024-03-22T04:12:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f360b800640 1 --2- 172.21.15.86:0/3787802846 >> v2:172.21.15.196:3300/0 conn(0x7f36040cc3e0 0x7f36040ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f360b800640 1 --2- 172.21.15.86:0/3787802846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f36040ade30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f360b800640 1 -- 172.21.15.86:0/3787802846 >> 172.21.15.86:0/3787802846 conn(0x7f36040b1df0 msgr2=0x7f36040b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f360b800640 1 -- 172.21.15.86:0/3787802846 shutdown_connections 2024-03-22T04:12:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.926+0000 7f360b800640 1 -- 172.21.15.86:0/3787802846 wait complete. 2024-03-22T04:12:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 Processor -- start 2024-03-22T04:12:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 -- start start 2024-03-22T04:12:02.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f3604159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36040cc3e0 0x7f360414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f360415b3f0 0x7f360414a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3604168580 con 0x7f36040730d0 2024-03-22T04:12:02.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3604168400 con 0x7f360415b3f0 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3604168700 con 0x7f36040cc3e0 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f3604159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f3604159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36792/0 (socket says 172.21.15.86:36792) 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 -- 172.21.15.86:0/755663385 learned_addr learned my addr 172.21.15.86:0/755663385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f35fbe00640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36040cc3e0 0x7f360414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3609800640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f360415b3f0 0x7f360414a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 -- 172.21.15.86:0/755663385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36040cc3e0 msgr2=0x7f360414a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36040cc3e0 0x7f360414a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 -- 172.21.15.86:0/755663385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f360415b3f0 msgr2=0x7f360414a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f360415b3f0 0x7f360414a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 -- 172.21.15.86:0/755663385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3604099dd0 con 0x7f36040730d0 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3608e00640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f3604159a70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f35f40011f0 tx=0x7f35f400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3601600640 1 -- 172.21.15.86:0/755663385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4018070 con 0x7f36040730d0 2024-03-22T04:12:02.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3601600640 1 -- 172.21.15.86:0/755663385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f40074e0 con 0x7f36040730d0 2024-03-22T04:12:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f3601600640 1 -- 172.21.15.86:0/755663385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4004a10 con 0x7f36040730d0 2024-03-22T04:12:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 -- 172.21.15.86:0/755663385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3604150500 con 0x7f36040730d0 2024-03-22T04:12:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.930+0000 7f360b800640 1 -- 172.21.15.86:0/755663385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3604150ac0 con 0x7f36040730d0 2024-03-22T04:12:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f3601600640 1 -- 172.21.15.86:0/755663385 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f35f401b070 con 0x7f36040730d0 2024-03-22T04:12:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f3601600640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35e0067b20 0x7f35e0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f3601600640 1 -- 172.21.15.86:0/755663385 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f35f4085410 con 0x7f36040730d0 2024-03-22T04:12:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f35fbe00640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35e0067b20 0x7f35e0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f360ed7c000 1 Processor -- start 2024-03-22T04:12:02.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f360ed7c000 1 -- start start 2024-03-22T04:12:02.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f35fbe00640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35e0067b20 0x7f35e0069fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0ad000 tx=0x7f35ec0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90ec860 0x560ca90ecc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca90e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e9410 0x560ca9135470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f360ed7c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ca8ee2140 con 0x560ca90e8af0 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f360ed7c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560ca8f35b10 con 0x560ca90ec860 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f360ed7c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x560ca8ebcc50 con 0x560ca90e9410 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f35fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca90e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f35fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca90e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36800/0 (socket says 172.21.15.86:36800) 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f35fbe00640 1 -- 172.21.15.86:0/772567492 learned_addr learned my addr 172.21.15.86:0/772567492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f3609800640 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e9410 0x560ca9135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.934+0000 7f3608e00640 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90ec860 0x560ca90ecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35fbe00640 1 -- 172.21.15.86:0/772567492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e9410 msgr2=0x560ca9135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35fbe00640 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e9410 0x560ca9135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35fbe00640 1 -- 172.21.15.86:0/772567492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90ec860 msgr2=0x560ca90ecc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35fbe00640 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90ec860 0x560ca90ecc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35fbe00640 1 -- 172.21.15.86:0/772567492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca8f7af20 con 0x560ca90e8af0 2024-03-22T04:12:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35fbe00640 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca90e8ed0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f35ec01a0b0 tx=0x7f35ec004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da813e4f4c64c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35f9600640 1 -- 172.21.15.86:0/772567492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0b0040 con 0x560ca90e8af0 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35f9600640 1 -- 172.21.15.86:0/772567492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec0b1040 con 0x560ca90e8af0 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35f9600640 1 -- 172.21.15.86:0/772567492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0c2400 con 0x560ca90e8af0 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- 172.21.15.86:0/772567492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 msgr2=0x560ca90e8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca90e8ed0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f35ec01a0b0 tx=0x7f35ec004740 comp rx=0 tx=0).stop 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- 172.21.15.86:0/772567492 shutdown_connections 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e9410 0x560ca9135470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca90e8ed0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 --2- 172.21.15.86:0/772567492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90ec860 0x560ca90ecc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- 172.21.15.86:0/772567492 >> 172.21.15.86:0/772567492 conn(0x560ca9111c10 msgr2=0x560ca9114000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- 172.21.15.86:0/772567492 shutdown_connections 2024-03-22T04:12:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- 172.21.15.86:0/772567492 wait complete. 2024-03-22T04:12:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- start start 2024-03-22T04:12:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 Processor -- start 2024-03-22T04:12:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 0x560ca913c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90ec860 0x560ca90f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ca8eefab0 con 0x560ca90ec860 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560ca8f31ac0 con 0x560ca90e9410 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x560ca8edbe80 con 0x560ca90e8af0 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f360ed7c000 10 client.0 _ll_register_callbacks cb 0x560ca90d2380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3609800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90ec860 0x560ca90f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3609800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90ec860 0x560ca90f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36806/0 (socket says 172.21.15.86:36806) 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3609800640 1 -- 172.21.15.86:0/427044546 learned_addr learned my addr 172.21.15.86:0/427044546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3608e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 0x560ca913c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35fbe00640 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3609800640 1 -- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 msgr2=0x560ca913c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3609800640 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 0x560ca913c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3609800640 1 -- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 msgr2=0x560ca913cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3609800640 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f3609800640 1 -- 172.21.15.86:0/427044546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca8f0b570 con 0x560ca90ec860 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.938+0000 7f35fbe00640 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f3609800640 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90ec860 0x560ca90f3d30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f35fc014df0 tx=0x7f35fc012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b35edc81644c36f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 1 -- 172.21.15.86:0/427044546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc01e620 con 0x560ca90ec860 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 -- 172.21.15.86:0/427044546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ec0b1040 con 0x560ca90ec860 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 1 -- 172.21.15.86:0/427044546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f35fc01e7c0 con 0x560ca90ec860 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 1 -- 172.21.15.86:0/427044546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f35fc01e9c0 con 0x560ca90ec860 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 10 client.5484 fetch_fsmap learned FSMap version 17 2024-03-22T04:12:02.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 -- 172.21.15.86:0/427044546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560ca8f01880 con 0x560ca90ec860 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 1 -- 172.21.15.86:0/427044546 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec0b1040 con 0x560ca90ec860 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 1 -- 172.21.15.86:0/427044546 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f35fc02b080 con 0x560ca90ec860 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 10 client.5484 fetch_fsmap finished waiting for FSMap version 17 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 10 client.5484 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 20 client.5484 populate_metadata read hostname 'smithi086' 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] conn(0x560ca90f4aa0 0x560ca90f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 4 client.5484 mds_command: new command op to 74930 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 -- 172.21.15.86:0/427044546 --> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ca8ed0bc0 con 0x560ca90f4aa0 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f3608e00640 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] conn(0x560ca90f4aa0 0x560ca90f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f3608e00640 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] conn(0x560ca90f4aa0 0x560ca90f4e80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb60ef30c46e048f server_cookie=f882b4f2386436a5 in_seq=0 out_seq=0 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 10 client.5484 ms_handle_connect on v2:172.21.15.196:6833/1426902111 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 1 -- 172.21.15.86:0/427044546 <== mds.0 v2:172.21.15.196:6833/1426902111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560ca8ed0bc0 con 0x560ca90f4aa0 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 10 client.5484 handle_command_reply: tid=0 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 client.5484 shutdown 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 -- 172.21.15.86:0/427044546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90ec860 msgr2=0x560ca90f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90ec860 0x560ca90f3d30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f35fc014df0 tx=0x7f35fc012670 comp rx=0 tx=0).stop 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 -- 172.21.15.86:0/427044546 shutdown_connections 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] conn(0x560ca90f4aa0 0x560ca90f4e80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90ec860 0x560ca90f3d30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f35f2000640 0 client.5484 ms_handle_reset on v2:172.21.15.196:6833/1426902111 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 --2- 172.21.15.86:0/427044546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 0x560ca913c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 -- 172.21.15.86:0/427044546 >> 172.21.15.86:0/427044546 conn(0x560ca9111c10 msgr2=0x560ca9114000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 -- 172.21.15.86:0/427044546 shutdown_connections 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 1 -- 172.21.15.86:0/427044546 wait complete. 2024-03-22T04:12:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.942+0000 7f360ed7c000 20 client.5484 trim_cache size 0 max 16384 2024-03-22T04:12:02.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 Processor -- start 2024-03-22T04:12:02.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- start start 2024-03-22T04:12:02.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca9111c10 0x560ca90f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca913f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36041ffe70 con 0x560ca90e8af0 2024-03-22T04:12:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f360415e020 con 0x560ca90e9410 2024-03-22T04:12:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f360415d380 con 0x560ca9111c10 2024-03-22T04:12:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca913f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca913f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36814/0 (socket says 172.21.15.86:36814) 2024-03-22T04:12:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 -- 172.21.15.86:0/2134459327 learned_addr learned my addr 172.21.15.86:0/2134459327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f3608e00640 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca9111c10 0x560ca90f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f3609800640 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 -- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca9111c10 msgr2=0x560ca90f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca9111c10 0x560ca90f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 -- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 msgr2=0x560ca913f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 -- 172.21.15.86:0/2134459327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca8ebde80 con 0x560ca90e8af0 2024-03-22T04:12:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f3609800640 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35fbe00640 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca913f1f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f35ec008010 tx=0x7f35ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bae4f8f9350192 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35f8c00640 1 -- 172.21.15.86:0/2134459327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0c2790 con 0x560ca90e8af0 2024-03-22T04:12:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35f8c00640 1 -- 172.21.15.86:0/2134459327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec0c2930 con 0x560ca90e8af0 2024-03-22T04:12:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f35f8c00640 1 -- 172.21.15.86:0/2134459327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0c2b20 con 0x560ca90e8af0 2024-03-22T04:12:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- 172.21.15.86:0/2134459327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 msgr2=0x560ca913f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca913f1f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f35ec008010 tx=0x7f35ec008040 comp rx=0 tx=0).stop 2024-03-22T04:12:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- 172.21.15.86:0/2134459327 shutdown_connections 2024-03-22T04:12:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca90e9410 0x560ca913f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e8af0 0x560ca913f1f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 --2- 172.21.15.86:0/2134459327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca9111c10 0x560ca90f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- 172.21.15.86:0/2134459327 >> 172.21.15.86:0/2134459327 conn(0x560ca9111830 msgr2=0x560ca90e36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- 172.21.15.86:0/2134459327 shutdown_connections 2024-03-22T04:12:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.962+0000 7f360ed7c000 1 -- 172.21.15.86:0/2134459327 wait complete. 2024-03-22T04:12:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 -- start start 2024-03-22T04:12:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 Processor -- start 2024-03-22T04:12:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 0x560ca91a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e9410 0x560ca9140750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca9111c10 0x560ca9140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ca8f34e20 con 0x560ca90e9410 2024-03-22T04:12:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f360415d9d0 con 0x560ca9111c10 2024-03-22T04:12:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x560ca8efa710 con 0x560ca90e8af0 2024-03-22T04:12:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 10 client.0 _ll_register_callbacks cb 0x7f35fc0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:12:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e9410 0x560ca9140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f3608e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 0x560ca91a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e9410 0x560ca9140750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36822/0 (socket says 172.21.15.86:36822) 2024-03-22T04:12:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 -- 172.21.15.86:0/3041107912 learned_addr learned my addr 172.21.15.86:0/3041107912 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f3609800640 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca9111c10 0x560ca9140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 -- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 msgr2=0x560ca91a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 0x560ca91a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 -- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca9111c10 msgr2=0x560ca9140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca9111c10 0x560ca9140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 -- 172.21.15.86:0/3041107912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca8ec9750 con 0x560ca90e9410 2024-03-22T04:12:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35fbe00640 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e9410 0x560ca9140750 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009580 tx=0x7f35ec01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9c0fbc972e17dc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35f2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:12:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35f2000640 1 -- 172.21.15.86:0/3041107912 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0d7060 con 0x560ca90e9410 2024-03-22T04:12:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35f2000640 1 -- 172.21.15.86:0/3041107912 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f35ec0c7040 con 0x560ca90e9410 2024-03-22T04:12:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 -- 172.21.15.86:0/3041107912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ec0c2930 con 0x560ca90e9410 2024-03-22T04:12:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35f2000640 1 -- 172.21.15.86:0/3041107912 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f35ec0dd050 con 0x560ca90e9410 2024-03-22T04:12:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 10 client.5490 fetch_fsmap learned FSMap version 17 2024-03-22T04:12:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 -- 172.21.15.86:0/3041107912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560ca8edc450 con 0x560ca90e9410 2024-03-22T04:12:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35f2000640 1 -- 172.21.15.86:0/3041107912 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec0c2930 con 0x560ca90e9410 2024-03-22T04:12:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35f2000640 1 -- 172.21.15.86:0/3041107912 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f35ec0e0020 con 0x560ca90e9410 2024-03-22T04:12:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 10 client.5490 fetch_fsmap finished waiting for FSMap version 17 2024-03-22T04:12:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 10 client.5490 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:12:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 20 client.5490 populate_metadata read hostname 'smithi086' 2024-03-22T04:12:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] conn(0x560ca90ec860 0x560ca91411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 4 client.5490 mds_command: new command op to 74930 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:12:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f360ed7c000 1 -- 172.21.15.86:0/3041107912 --> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] -- command(tid 0: {"prefix": "status"}) -- 0x560ca8f43050 con 0x560ca90ec860 2024-03-22T04:12:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f3608e00640 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] conn(0x560ca90ec860 0x560ca91411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f35f2000640 10 client.5490 ms_handle_connect on v2:172.21.15.196:6833/1426902111 2024-03-22T04:12:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.966+0000 7f3608e00640 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] conn(0x560ca90ec860 0x560ca91411d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caffaf0f0480a80d server_cookie=13332c2a3c64d922 in_seq=0 out_seq=0 2024-03-22T04:12:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f35f2000640 1 -- 172.21.15.86:0/3041107912 <== mds.0 v2:172.21.15.196:6833/1426902111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x560ca8f43050 con 0x560ca90ec860 2024-03-22T04:12:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f35f2000640 10 client.5490 handle_command_reply: tid=0 2024-03-22T04:12:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 client.5490 shutdown 2024-03-22T04:12:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 -- 172.21.15.86:0/3041107912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e9410 msgr2=0x560ca9140750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e9410 0x560ca9140750 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009580 tx=0x7f35ec01acc0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 -- 172.21.15.86:0/3041107912 shutdown_connections 2024-03-22T04:12:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:6833/1426902111,v1:172.21.15.196:6835/1426902111] conn(0x560ca90ec860 0x560ca91411d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ca9111c10 0x560ca9140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ca90e9410 0x560ca9140750 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 --2- 172.21.15.86:0/3041107912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560ca90e8af0 0x560ca91a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 -- 172.21.15.86:0/3041107912 >> 172.21.15.86:0/3041107912 conn(0x560ca9111830 msgr2=0x560ca90e3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f35f2000640 0 client.5490 ms_handle_reset on v2:172.21.15.196:6833/1426902111 2024-03-22T04:12:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 -- 172.21.15.86:0/3041107912 shutdown_connections 2024-03-22T04:12:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 1 -- 172.21.15.86:0/3041107912 wait complete. 2024-03-22T04:12:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360ed7c000 20 client.5490 trim_cache size 0 max 16384 2024-03-22T04:12:02.991 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:12:02.991 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:12:02.991 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:12:02.991 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 74930, 2024-03-22T04:12:02.992 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:12:02.992 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:12:02.992 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:12:02.992 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.325881652, 2024-03-22T04:12:02.992 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 17, 2024-03-22T04:12:02.992 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 41, 2024-03-22T04:12:02.992 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 41, 2024-03-22T04:12:02.993 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 18.213324046 2024-03-22T04:12:02.993 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:12:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 -- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35e0067b20 msgr2=0x7f35e0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35e0067b20 0x7f35e0069fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0ad000 tx=0x7f35ec0a6000 comp rx=0 tx=0).stop 2024-03-22T04:12:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 -- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 msgr2=0x7f3604159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f3604159a70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f35f40011f0 tx=0x7f35f400c9b0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 -- 172.21.15.86:0/755663385 shutdown_connections 2024-03-22T04:12:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35e0067b20 0x7f35e0069fe0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f360415b3f0 0x7f360414a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36040cc3e0 0x7f360414a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 --2- 172.21.15.86:0/755663385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040730d0 0x7f3604159a70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 -- 172.21.15.86:0/755663385 >> 172.21.15.86:0/755663385 conn(0x7f36040b1df0 msgr2=0x7f36041552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:02.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 -- 172.21.15.86:0/755663385 shutdown_connections 2024-03-22T04:12:02.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:02.970+0000 7f360b800640 1 -- 172.21.15.86:0/755663385 wait complete. 2024-03-22T04:12:02.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:12:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.126+0000 7ff19bc00640 1 Processor -- start 2024-03-22T04:12:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.126+0000 7ff19bc00640 1 -- start start 2024-03-22T04:12:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.126+0000 7ff19bc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1940d0de0 0x7ff1940ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.126+0000 7ff19bc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff1940d19a0 0x7ff1940cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.126+0000 7ff19bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff19415fed0 0x7ff194160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.126+0000 7ff19bc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff194160860 con 0x7ff1940d0de0 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.126+0000 7ff19bc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff1940d11c0 con 0x7ff1940d19a0 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.126+0000 7ff19bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff1940cece0 con 0x7ff19415fed0 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1940d0de0 0x7ff1940ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1940d0de0 0x7ff1940ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36832/0 (socket says 172.21.15.86:36832) 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff18be00640 1 --2- 172.21.15.86:0/778794790 >> v2:172.21.15.196:3300/0 conn(0x7ff1940d19a0 0x7ff1940cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 -- 172.21.15.86:0/778794790 learned_addr learned my addr 172.21.15.86:0/778794790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199c00640 1 --2- 172.21.15.86:0/778794790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff19415fed0 0x7ff194160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 -- 172.21.15.86:0/778794790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff19415fed0 msgr2=0x7ff194160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 --2- 172.21.15.86:0/778794790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff19415fed0 0x7ff194160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 -- 172.21.15.86:0/778794790 >> v2:172.21.15.196:3300/0 conn(0x7ff1940d19a0 msgr2=0x7ff1940cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 --2- 172.21.15.86:0/778794790 >> v2:172.21.15.196:3300/0 conn(0x7ff1940d19a0 0x7ff1940cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 -- 172.21.15.86:0/778794790 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1940cefc0 con 0x7ff1940d0de0 2024-03-22T04:12:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff18be00640 1 --2- 172.21.15.86:0/778794790 >> v2:172.21.15.196:3300/0 conn(0x7ff1940d19a0 0x7ff1940cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff199200640 1 --2- 172.21.15.86:0/778794790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d0de0 0x7ff1940ce6b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff1940d32d0 tx=0x7ff18401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9388499f633edb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff193e00640 1 -- 172.21.15.86:0/778794790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1840b1070 con 0x7ff1940d0de0 2024-03-22T04:12:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff193e00640 1 -- 172.21.15.86:0/778794790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1840ac630 con 0x7ff1940d0de0 2024-03-22T04:12:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.130+0000 7ff193e00640 1 -- 172.21.15.86:0/778794790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1840a8050 con 0x7ff1940d0de0 2024-03-22T04:12:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- 172.21.15.86:0/778794790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d0de0 msgr2=0x7ff1940ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 --2- 172.21.15.86:0/778794790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d0de0 0x7ff1940ce6b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff1940d32d0 tx=0x7ff18401c520 comp rx=0 tx=0).stop 2024-03-22T04:12:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- 172.21.15.86:0/778794790 shutdown_connections 2024-03-22T04:12:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 --2- 172.21.15.86:0/778794790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff19415fed0 0x7ff194160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 --2- 172.21.15.86:0/778794790 >> v2:172.21.15.196:3300/0 conn(0x7ff1940d19a0 0x7ff1940cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 --2- 172.21.15.86:0/778794790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d0de0 0x7ff1940ce6b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- 172.21.15.86:0/778794790 >> 172.21.15.86:0/778794790 conn(0x7ff19406f910 msgr2=0x7ff19406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- 172.21.15.86:0/778794790 shutdown_connections 2024-03-22T04:12:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- 172.21.15.86:0/778794790 wait complete. 2024-03-22T04:12:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 Processor -- start 2024-03-22T04:12:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- start start 2024-03-22T04:12:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1940d0de0 0x7ff19406c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d19a0 0x7ff1940c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff19415fed0 0x7ff194141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1940d0530 con 0x7ff1940d19a0 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1940d03b0 con 0x7ff19415fed0 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff19bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff1940d1db0 con 0x7ff1940d0de0 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff18be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d19a0 0x7ff1940c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff199200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1940d0de0 0x7ff19406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff199200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1940d0de0 0x7ff19406c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:47874/0 (socket says 172.21.15.86:47874) 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff199200640 1 -- 172.21.15.86:0/3148550162 learned_addr learned my addr 172.21.15.86:0/3148550162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff199c00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff19415fed0 0x7ff194141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff18be00640 1 -- 172.21.15.86:0/3148550162 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1940d0de0 msgr2=0x7ff19406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff18be00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1940d0de0 0x7ff19406c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff18be00640 1 -- 172.21.15.86:0/3148550162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff19415fed0 msgr2=0x7ff194141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff18be00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff19415fed0 0x7ff194141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff18be00640 1 -- 172.21.15.86:0/3148550162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19406d030 con 0x7ff1940d19a0 2024-03-22T04:12:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.134+0000 7ff199200640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1940d0de0 0x7ff19406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff18be00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d19a0 0x7ff1940c95d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0093f0 tx=0x7ff17c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff191600640 1 -- 172.21.15.86:0/3148550162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c004030 con 0x7ff1940d19a0 2024-03-22T04:12:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff191600640 1 -- 172.21.15.86:0/3148550162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17c0041d0 con 0x7ff1940d19a0 2024-03-22T04:12:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff191600640 1 -- 172.21.15.86:0/3148550162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00c940 con 0x7ff1940d19a0 2024-03-22T04:12:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19406d310 con 0x7ff1940d19a0 2024-03-22T04:12:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19406d8d0 con 0x7ff1940d19a0 2024-03-22T04:12:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff194068650 con 0x7ff1940d19a0 2024-03-22T04:12:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff191600640 1 -- 172.21.15.86:0/3148550162 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff17c00cae0 con 0x7ff1940d19a0 2024-03-22T04:12:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff191600640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff170067b50 0x7ff17006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff191600640 1 -- 172.21.15.86:0/3148550162 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7ff17c086300 con 0x7ff1940d19a0 2024-03-22T04:12:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.138+0000 7ff199200640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff170067b50 0x7ff17006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.142+0000 7ff199200640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff170067b50 0x7ff17006a010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff18401c560 tx=0x7ff184009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.142+0000 7ff191600640 1 -- 172.21.15.86:0/3148550162 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff17c017020 con 0x7ff1940d19a0 2024-03-22T04:12:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff194071ea0 con 0x7ff1940d19a0 2024-03-22T04:12:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff191600640 1 -- 172.21.15.86:0/3148550162 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7ff17c0860c0 con 0x7ff1940d19a0 2024-03-22T04:12:03.336 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:12:03.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff170067b50 msgr2=0x7ff17006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:03.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff170067b50 0x7ff17006a010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff18401c560 tx=0x7ff184009040 comp rx=0 tx=0).stop 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d19a0 msgr2=0x7ff1940c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d19a0 0x7ff1940c95d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0093f0 tx=0x7ff17c00ed60 comp rx=0 tx=0).stop 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 shutdown_connections 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff170067b50 0x7ff17006a010 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff19415fed0 0x7ff194141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940d19a0 0x7ff1940c95d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 --2- 172.21.15.86:0/3148550162 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1940d0de0 0x7ff19406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.330+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 >> 172.21.15.86:0/3148550162 conn(0x7ff19406f910 msgr2=0x7ff1940c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.334+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 shutdown_connections 2024-03-22T04:12:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03.334+0000 7ff19bc00640 1 -- 172.21.15.86:0/3148550162 wait complete. 2024-03-22T04:12:03.358 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:12:03.358 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:12:03.358 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:12:03.358 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:12:03.358 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:03.358 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:12:03.358 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:12:03.358 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:12:03.359 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:12:03.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:03.359 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:12:03.369 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:12:03.369 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:12:03.369 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:12:03.369 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:03.369 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:12:03.369 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:03.369 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42408sec preferred_lft 42408sec 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:12:03.370 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:12:03.371 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:12:03.372 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:03.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:03.372 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:12:03.427 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:12:03.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:03.430 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:12:03.482 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:12:03.483 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:12:03.483 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:12:03.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:12:03.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:12:03.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:12:03.483 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:12:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:12:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:03.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:03.618 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:12:03.627 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:12:03.628 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:12:03.628 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:12:03.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:03.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:12:03.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:12:03.690 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:03.738 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:12:03.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:12:03.738 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:03.789 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:12:03.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:03.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:12:03.877 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:12:03.877 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:12:03.877 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:12:03.892 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:12:03.892 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:12:03.892 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:12:03.893 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:12:03.893 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:12:03.903 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:12:03.903 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:12:03.903 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:12:03.903 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:12:03.903 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:12:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:12:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:12:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:12:03.907 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:12:03.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:03.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:03.928 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:12:04.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:12:04.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:04.034 INFO:teuthology.orchestra.run.smithi086.stdout:client5499 2024-03-22T04:12:04.034 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:12:04.034 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:04.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:04.043 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-03-22T04:12:04.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:04.106 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-03-22T04:12:04.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:04.161 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-03-22T04:12:04.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:04.212 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-03-22T04:12:04.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:04.266 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-03-22T04:12:04.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:04.319 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-22T04:12:04.375 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-03-22T04:12:04.375 INFO:teuthology.orchestra.run.smithi086.stdout:./rfile 2024-03-22T04:12:04.375 INFO:teuthology.orchestra.run.smithi086.stdout:./parentdir 2024-03-22T04:12:04.375 INFO:teuthology.orchestra.run.smithi086.stdout:./parentdir/pfile 2024-03-22T04:12:04.375 INFO:teuthology.orchestra.run.smithi086.stdout:./parentdir/childdir 2024-03-22T04:12:04.376 INFO:teuthology.orchestra.run.smithi086.stdout:./parentdir/childdir/cfile 2024-03-22T04:12:04.377 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:12:04.377 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:12:04.377 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:12:04.377 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:12:04.377 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-22T04:12:04.377 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:12:04.481 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:12:04.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:04.490 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:12:04.490 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:12:04.490 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:12:04.490 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:12:04.490 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-03-22T04:12:04.491 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:12:04.598 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-03-22T04:12:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:04.607 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:12:04.608 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:12:04.608 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:12:04.608 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:12:04.608 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-03-22T04:12:04.608 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:12:04.713 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-03-22T04:12:04.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:04.722 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:12:04.722 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:12:04.722 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:12:04.722 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:12:04.722 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-03-22T04:12:04.722 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:12:04.834 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628279 2024-03-22T04:12:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:04.843 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:12:04.843 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:12:04.843 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:12:04.843 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:12:04.843 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-03-22T04:12:04.843 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:12:04.949 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-03-22T04:12:04.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:04.958 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:12:04.958 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:12:04.958 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:12:04.958 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:12:04.958 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-03-22T04:12:04.959 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:12:05.069 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628280 2024-03-22T04:12:05.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:05.078 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:12:05.078 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:05.130 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:12:05.130 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:05.182 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe68600640 1 Processor -- start 2024-03-22T04:12:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe68600640 1 -- start start 2024-03-22T04:12:05.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe68600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600aff50 0x7fbe6006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe68600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbe6006cbb0 0x7fbe60143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe68600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe600ae5a0 con 0x7fbe60144320 2024-03-22T04:12:05.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe68600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbe6006d010 con 0x7fbe6006cbb0 2024-03-22T04:12:05.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe68600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbe60162aa0 con 0x7fbe600aff50 2024-03-22T04:12:05.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe66600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbe60144320 0x7fbe6006e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40416/0 (socket says 172.21.15.86:40416) 2024-03-22T04:12:05.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe66600640 1 -- 172.21.15.86:0/2230863343 learned_addr learned my addr 172.21.15.86:0/2230863343 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:05.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe65200640 1 --2- 172.21.15.86:0/2230863343 >> v2:172.21.15.196:3300/0 conn(0x7fbe6006cbb0 0x7fbe60143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe65c00640 1 --2- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600aff50 0x7fbe6006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe5fe00640 1 -- 172.21.15.86:0/2230863343 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335305351 0 0) 0x7fbe580014a0 con 0x7fbe60144320 2024-03-22T04:12:05.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe5fe00640 1 -- 172.21.15.86:0/2230863343 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe48003530 con 0x7fbe60144320 2024-03-22T04:12:05.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe65200640 1 -- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600aff50 msgr2=0x7fbe6006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:05.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe65200640 1 --2- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600aff50 0x7fbe6006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe65200640 1 -- 172.21.15.86:0/2230863343 >> v1:172.21.15.86:6789/0 conn(0x7fbe60144320 legacy=0x7fbe6006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe65200640 1 -- 172.21.15.86:0/2230863343 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe60144b80 con 0x7fbe6006cbb0 2024-03-22T04:12:05.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe65c00640 1 --2- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600aff50 0x7fbe6006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:05.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe65200640 1 --2- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe6006cbb0 0x7fbe60143de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0097c0 tx=0x7fbe4c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa23a5a0ec11ff40 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:05.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe5fe00640 1 -- 172.21.15.86:0/2230863343 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0b0070 con 0x7fbe6006cbb0 2024-03-22T04:12:05.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe5fe00640 1 -- 172.21.15.86:0/2230863343 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4c0ab600 con 0x7fbe6006cbb0 2024-03-22T04:12:05.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.358+0000 7fbe5fe00640 1 -- 172.21.15.86:0/2230863343 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0ab9b0 con 0x7fbe6006cbb0 2024-03-22T04:12:05.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe6006cbb0 msgr2=0x7fbe60143de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 --2- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe6006cbb0 0x7fbe60143de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0097c0 tx=0x7fbe4c01c410 comp rx=0 tx=0).stop 2024-03-22T04:12:05.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- 172.21.15.86:0/2230863343 shutdown_connections 2024-03-22T04:12:05.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 --2- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe6006cbb0 0x7fbe60143de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 --2- 172.21.15.86:0/2230863343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600aff50 0x7fbe6006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- 172.21.15.86:0/2230863343 >> 172.21.15.86:0/2230863343 conn(0x7fbe60071fe0 msgr2=0x7fbe600723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:05.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- 172.21.15.86:0/2230863343 shutdown_connections 2024-03-22T04:12:05.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- 172.21.15.86:0/2230863343 wait complete. 2024-03-22T04:12:05.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 Processor -- start 2024-03-22T04:12:05.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- start start 2024-03-22T04:12:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe6006cbb0 0x7fbe601fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe600aff50 0x7fbe60140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60144320 0x7fbe600b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60163090 con 0x7fbe6006cbb0 2024-03-22T04:12:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60162f10 con 0x7fbe600aff50 2024-03-22T04:12:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe68600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbe600b0330 con 0x7fbe60144320 2024-03-22T04:12:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe6006cbb0 0x7fbe601fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe6006cbb0 0x7fbe601fb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36848/0 (socket says 172.21.15.86:36848) 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65c00640 1 -- 172.21.15.86:0/1608347969 learned_addr learned my addr 172.21.15.86:0/1608347969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65200640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe600aff50 0x7fbe60140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe66600640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60144320 0x7fbe600b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65c00640 1 -- 172.21.15.86:0/1608347969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60144320 msgr2=0x7fbe600b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65c00640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60144320 0x7fbe600b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65c00640 1 -- 172.21.15.86:0/1608347969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe600aff50 msgr2=0x7fbe60140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65c00640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe600aff50 0x7fbe60140b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.362+0000 7fbe65c00640 1 -- 172.21.15.86:0/1608347969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe601fba00 con 0x7fbe6006cbb0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe65c00640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe6006cbb0 0x7fbe601fb4c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400ec10 tx=0x7fbe5400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe5d600640 1 -- 172.21.15.86:0/1608347969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54017070 con 0x7fbe6006cbb0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe5d600640 1 -- 172.21.15.86:0/1608347969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5400cc30 con 0x7fbe6006cbb0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe5d600640 1 -- 172.21.15.86:0/1608347969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54013470 con 0x7fbe6006cbb0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe60141080 con 0x7fbe6006cbb0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe60157c80 con 0x7fbe6006cbb0 2024-03-22T04:12:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe60068650 con 0x7fbe6006cbb0 2024-03-22T04:12:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe5d600640 1 -- 172.21.15.86:0/1608347969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbe5401a070 con 0x7fbe6006cbb0 2024-03-22T04:12:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe5d600640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe40067b50 0x7fbe4006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe5d600640 1 -- 172.21.15.86:0/1608347969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fbe54085ff0 con 0x7fbe6006cbb0 2024-03-22T04:12:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.366+0000 7fbe65200640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe40067b50 0x7fbe4006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.370+0000 7fbe65200640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe40067b50 0x7fbe4006a010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0ad000 tx=0x7fbe4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.370+0000 7fbe5d600640 1 -- 172.21.15.86:0/1608347969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbe601fba00 con 0x7fbe6006cbb0 2024-03-22T04:12:05.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe601417e0 con 0x7fbe6006cbb0 2024-03-22T04:12:05.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe5d600640 1 -- 172.21.15.86:0/1608347969 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16942 (secure 0 0 0) 0x7fbe54056180 con 0x7fbe6006cbb0 2024-03-22T04:12:05.536 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:05.536 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":41,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:11:53.644770+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:11:49.644732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-22T04:11:50.961986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe40067b50 msgr2=0x7fbe4006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe40067b50 0x7fbe4006a010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0ad000 tx=0x7fbe4c008040 comp rx=0 tx=0).stop 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe6006cbb0 msgr2=0x7fbe601fb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe6006cbb0 0x7fbe601fb4c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400ec10 tx=0x7fbe5400c520 comp rx=0 tx=0).stop 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 shutdown_connections 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe40067b50 0x7fbe4006a010 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe600aff50 0x7fbe60140b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60144320 0x7fbe600b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 --2- 172.21.15.86:0/1608347969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe6006cbb0 0x7fbe601fb4c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 >> 172.21.15.86:0/1608347969 conn(0x7fbe60071fe0 msgr2=0x7fbe600b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 shutdown_connections 2024-03-22T04:12:05.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.530+0000 7fbe68600640 1 -- 172.21.15.86:0/1608347969 wait complete. 2024-03-22T04:12:05.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:11:49.644732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:11:50.961986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:12:05.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:12:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f242b200640 1 Processor -- start 2024-03-22T04:12:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f242b200640 1 -- start start 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f242b200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24240b2360 0x7f24240b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f242b200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f242415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f242b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242415d710 con 0x7f242415cee0 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f242b200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f24240b2740 con 0x7f24240b2360 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f242b200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f242415fe40 con 0x7f24240b3430 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f2429200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f242415cee0 0x7f242406d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40424/0 (socket says 172.21.15.86:40424) 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f2429200640 1 -- 172.21.15.86:0/4073004596 learned_addr learned my addr 172.21.15.86:0/4073004596 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.694+0000 7f241b400640 1 --2- 172.21.15.86:0/4073004596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f242415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f2423e00640 1 --2- 172.21.15.86:0/4073004596 >> v2:172.21.15.196:3300/0 conn(0x7f24240b2360 0x7f24240b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f2423400640 1 -- 172.21.15.86:0/4073004596 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4174105705 0 0) 0x7f242415d710 con 0x7f242415cee0 2024-03-22T04:12:05.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f2423400640 1 -- 172.21.15.86:0/4073004596 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2408003530 con 0x7f242415cee0 2024-03-22T04:12:05.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f241b400640 1 -- 172.21.15.86:0/4073004596 >> v2:172.21.15.196:3300/0 conn(0x7f24240b2360 msgr2=0x7f24240b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:05.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f241b400640 1 --2- 172.21.15.86:0/4073004596 >> v2:172.21.15.196:3300/0 conn(0x7f24240b2360 0x7f24240b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f241b400640 1 -- 172.21.15.86:0/4073004596 >> v1:172.21.15.86:6789/0 conn(0x7f242415cee0 legacy=0x7f242406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f241b400640 1 -- 172.21.15.86:0/4073004596 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2424160120 con 0x7f24240b3430 2024-03-22T04:12:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f2423e00640 1 --2- 172.21.15.86:0/4073004596 >> v2:172.21.15.196:3300/0 conn(0x7f24240b2360 0x7f24240b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:05.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f241b400640 1 --2- 172.21.15.86:0/4073004596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f242415c9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f240c0097c0 tx=0x7f240c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2662d937adbeb40 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:05.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f2423400640 1 -- 172.21.15.86:0/4073004596 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c0b0070 con 0x7f24240b3430 2024-03-22T04:12:05.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f2423400640 1 -- 172.21.15.86:0/4073004596 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240c0ab600 con 0x7f24240b3430 2024-03-22T04:12:05.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.698+0000 7f2423400640 1 -- 172.21.15.86:0/4073004596 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c0ab9d0 con 0x7f24240b3430 2024-03-22T04:12:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- 172.21.15.86:0/4073004596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 msgr2=0x7f242415c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 --2- 172.21.15.86:0/4073004596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f242415c9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f240c0097c0 tx=0x7f240c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- 172.21.15.86:0/4073004596 shutdown_connections 2024-03-22T04:12:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 --2- 172.21.15.86:0/4073004596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f242415c9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 --2- 172.21.15.86:0/4073004596 >> v2:172.21.15.196:3300/0 conn(0x7f24240b2360 0x7f24240b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- 172.21.15.86:0/4073004596 >> 172.21.15.86:0/4073004596 conn(0x7f24240cc940 msgr2=0x7f24240ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- 172.21.15.86:0/4073004596 shutdown_connections 2024-03-22T04:12:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- 172.21.15.86:0/4073004596 wait complete. 2024-03-22T04:12:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 Processor -- start 2024-03-22T04:12:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- start start 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24240b2360 0x7f2424146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f2424154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242415cee0 0x7f242414c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2424160c00 con 0x7f24240b2360 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f242415d2f0 con 0x7f242415cee0 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f242b200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2424146580 con 0x7f24240b3430 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24240b2360 0x7f2424146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24240b2360 0x7f2424146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36850/0 (socket says 172.21.15.86:36850) 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 -- 172.21.15.86:0/2137105454 learned_addr learned my addr 172.21.15.86:0/2137105454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f241b400640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f2424154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2429200640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242415cee0 0x7f242414c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 -- 172.21.15.86:0/2137105454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 msgr2=0x7f2424154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f2424154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 -- 172.21.15.86:0/2137105454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242415cee0 msgr2=0x7f242414c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242415cee0 0x7f242414c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 -- 172.21.15.86:0/2137105454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242414a210 con 0x7f24240b2360 2024-03-22T04:12:05.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.702+0000 7f2423e00640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24240b2360 0x7f2424146040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f241400e9b0 tx=0x7f241400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f2420c00640 1 -- 172.21.15.86:0/2137105454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414017070 con 0x7f24240b2360 2024-03-22T04:12:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f2420c00640 1 -- 172.21.15.86:0/2137105454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241400c9a0 con 0x7f24240b2360 2024-03-22T04:12:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f2420c00640 1 -- 172.21.15.86:0/2137105454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400cd70 con 0x7f24240b2360 2024-03-22T04:12:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242414a4f0 con 0x7f24240b2360 2024-03-22T04:12:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242414aab0 con 0x7f24240b2360 2024-03-22T04:12:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2424074470 con 0x7f24240b2360 2024-03-22T04:12:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f2420c00640 1 -- 172.21.15.86:0/2137105454 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f241401a070 con 0x7f24240b2360 2024-03-22T04:12:05.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f2420c00640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2400067b50 0x7f240006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:05.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f2420c00640 1 -- 172.21.15.86:0/2137105454 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f2414086250 con 0x7f24240b2360 2024-03-22T04:12:05.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f241b400640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2400067b50 0x7f240006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.706+0000 7f241b400640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2400067b50 0x7f240006a010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f240c0ad000 tx=0x7f240c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:05.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.710+0000 7f2420c00640 1 -- 172.21.15.86:0/2137105454 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f24140563e0 con 0x7f24240b2360 2024-03-22T04:12:05.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.866+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f242406e9f0 con 0x7f24240b2360 2024-03-22T04:12:05.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f2420c00640 1 -- 172.21.15.86:0/2137105454 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f241405a480 con 0x7f24240b2360 2024-03-22T04:12:05.871 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:12:05.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2400067b50 msgr2=0x7f240006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2400067b50 0x7f240006a010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f240c0ad000 tx=0x7f240c008040 comp rx=0 tx=0).stop 2024-03-22T04:12:05.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24240b2360 msgr2=0x7f2424146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:05.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24240b2360 0x7f2424146040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f241400e9b0 tx=0x7f241400ee70 comp rx=0 tx=0).stop 2024-03-22T04:12:05.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 shutdown_connections 2024-03-22T04:12:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2400067b50 0x7f240006a010 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242415cee0 0x7f242414c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24240b3430 0x7f2424154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 --2- 172.21.15.86:0/2137105454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24240b2360 0x7f2424146040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 >> 172.21.15.86:0/2137105454 conn(0x7f24240cc940 msgr2=0x7f24241536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 shutdown_connections 2024-03-22T04:12:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:05.870+0000 7f242b200640 1 -- 172.21.15.86:0/2137105454 wait complete. 2024-03-22T04:12:05.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:12:06.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800f800640 1 Processor -- start 2024-03-22T04:12:06.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800f800640 1 -- start start 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b3600 0x7f800815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800f800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f800815cee0 0x7f80080b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800f800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800815d710 con 0x7f80080b2360 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800f800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f80080b2740 con 0x7f800815cee0 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f800815fe40 con 0x7f80080b3600 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800d800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f800815cee0 0x7f80080b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800ce00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f80080b2360 0x7f80080b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:44138/0 (socket says 172.21.15.86:44138) 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f8007e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b3600 0x7f800815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800ce00640 1 -- 172.21.15.86:0/2020510592 learned_addr learned my addr 172.21.15.86:0/2020510592 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:06.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800d800640 1 -- 172.21.15.86:0/2020510592 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b3600 msgr2=0x7f800815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:06.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800d800640 1 --2- 172.21.15.86:0/2020510592 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b3600 0x7f800815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800d800640 1 -- 172.21.15.86:0/2020510592 >> v1:172.21.15.196:6789/0 conn(0x7f80080b2360 legacy=0x7f80080b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f8007400640 1 -- 172.21.15.86:0/2020510592 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1393788222 0 0) 0x7f800815d710 con 0x7f80080b2360 2024-03-22T04:12:06.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.030+0000 7f800d800640 1 -- 172.21.15.86:0/2020510592 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8008160120 con 0x7f800815cee0 2024-03-22T04:12:06.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800d800640 1 --2- 172.21.15.86:0/2020510592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 0x7f80080b4d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8000009d90 tx=0x7f800001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=240089faaec3ba23 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f8007400640 1 -- 172.21.15.86:0/2020510592 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000b2070 con 0x7f800815cee0 2024-03-22T04:12:06.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f8007400640 1 -- 172.21.15.86:0/2020510592 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80000a5a00 con 0x7f800815cee0 2024-03-22T04:12:06.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f8007400640 1 -- 172.21.15.86:0/2020510592 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000a5dd0 con 0x7f800815cee0 2024-03-22T04:12:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 -- 172.21.15.86:0/2020510592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 msgr2=0x7f80080b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 --2- 172.21.15.86:0/2020510592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 0x7f80080b4d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8000009d90 tx=0x7f800001c940 comp rx=0 tx=0).stop 2024-03-22T04:12:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 -- 172.21.15.86:0/2020510592 shutdown_connections 2024-03-22T04:12:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 --2- 172.21.15.86:0/2020510592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 0x7f80080b4d80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 --2- 172.21.15.86:0/2020510592 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b3600 0x7f800815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 -- 172.21.15.86:0/2020510592 >> 172.21.15.86:0/2020510592 conn(0x7f80080cc940 msgr2=0x7f80080ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:06.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 -- 172.21.15.86:0/2020510592 shutdown_connections 2024-03-22T04:12:06.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 -- 172.21.15.86:0/2020510592 wait complete. 2024-03-22T04:12:06.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 Processor -- start 2024-03-22T04:12:06.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.034+0000 7f800f800640 1 -- start start 2024-03-22T04:12:06.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b2360 0x7f80080729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80080b3600 0x7f800814f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 0x7f800815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f800815d490 con 0x7f80080b3600 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f800815d310 con 0x7f800815cee0 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f80080b5ad0 con 0x7f80080b2360 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80080b3600 0x7f800814f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80080b3600 0x7f800814f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39928/0 (socket says 172.21.15.86:39928) 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800ce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b2360 0x7f80080729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 -- 172.21.15.86:0/1288106573 learned_addr learned my addr 172.21.15.86:0/1288106573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800d800640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 0x7f800815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 -- 172.21.15.86:0/1288106573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b2360 msgr2=0x7f80080729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b2360 0x7f80080729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 -- 172.21.15.86:0/1288106573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 msgr2=0x7f800815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 0x7f800815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 -- 172.21.15.86:0/1288106573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080731f0 con 0x7f80080b3600 2024-03-22T04:12:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800ce00640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b2360 0x7f80080729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8007e00640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80080b3600 0x7f800814f280 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400e930 tx=0x7f7ff400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8004c00640 1 -- 172.21.15.86:0/1288106573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4017070 con 0x7f80080b3600 2024-03-22T04:12:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8004c00640 1 -- 172.21.15.86:0/1288106573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff400ca00 con 0x7f80080b3600 2024-03-22T04:12:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f8004c00640 1 -- 172.21.15.86:0/1288106573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400cdb0 con 0x7f80080b3600 2024-03-22T04:12:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8008162e60 con 0x7f80080b3600 2024-03-22T04:12:06.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8008163420 con 0x7f80080b3600 2024-03-22T04:12:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.038+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8008074470 con 0x7f80080b3600 2024-03-22T04:12:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.042+0000 7f8004c00640 1 -- 172.21.15.86:0/1288106573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7ff401a070 con 0x7f80080b3600 2024-03-22T04:12:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.042+0000 7f8004c00640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe8067b20 0x7f7fe8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.042+0000 7f8004c00640 1 -- 172.21.15.86:0/1288106573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f7ff40860c0 con 0x7f80080b3600 2024-03-22T04:12:06.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.042+0000 7f800ce00640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe8067b20 0x7f7fe8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.042+0000 7f800ce00640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe8067b20 0x7f7fe8069fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc007fd0 tx=0x7f7ffc007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.046+0000 7f8004c00640 1 -- 172.21.15.86:0/1288106573 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7ff4056310 con 0x7f80080b3600 2024-03-22T04:12:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.198+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f800806e9f0 con 0x7f80080b3600 2024-03-22T04:12:06.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.198+0000 7f8004c00640 1 -- 172.21.15.86:0/1288106573 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f7ff405a3b0 con 0x7f80080b3600 2024-03-22T04:12:06.202 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:12:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe8067b20 msgr2=0x7f7fe8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe8067b20 0x7f7fe8069fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc007fd0 tx=0x7f7ffc007d60 comp rx=0 tx=0).stop 2024-03-22T04:12:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80080b3600 msgr2=0x7f800814f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80080b3600 0x7f800814f280 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400e930 tx=0x7f7ff400edf0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 shutdown_connections 2024-03-22T04:12:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe8067b20 0x7f7fe8069fe0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f800815cee0 0x7f800815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80080b3600 0x7f800814f280 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 --2- 172.21.15.86:0/1288106573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080b2360 0x7f80080729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 >> 172.21.15.86:0/1288106573 conn(0x7f80080cc940 msgr2=0x7f80080b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 shutdown_connections 2024-03-22T04:12:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.202+0000 7f800f800640 1 -- 172.21.15.86:0/1288106573 wait complete. 2024-03-22T04:12:06.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:12:06.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.358+0000 7f512fe00640 1 Processor -- start 2024-03-22T04:12:06.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.358+0000 7f512fe00640 1 -- start start 2024-03-22T04:12:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f513015c410 0x7f51300cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 0x7f51300cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f51300d0080 0x7f51300d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f51300ce030 con 0x7f513015c410 2024-03-22T04:12:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512fe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f51300cdd50 con 0x7f51300d0080 2024-03-22T04:12:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f51300c5bd0 con 0x7f51300c86c0 2024-03-22T04:12:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f513015c410 0x7f51300cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f513015c410 0x7f51300cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39936/0 (socket says 172.21.15.86:39936) 2024-03-22T04:12:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 -- 172.21.15.86:0/1912714763 learned_addr learned my addr 172.21.15.86:0/1912714763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512f400640 1 --2- 172.21.15.86:0/1912714763 >> v2:172.21.15.196:3300/0 conn(0x7f51300d0080 0x7f51300d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f5126000640 1 --2- 172.21.15.86:0/1912714763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 0x7f51300cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 -- 172.21.15.86:0/1912714763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 msgr2=0x7f51300cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 --2- 172.21.15.86:0/1912714763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 0x7f51300cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 -- 172.21.15.86:0/1912714763 >> v2:172.21.15.196:3300/0 conn(0x7f51300d0080 msgr2=0x7f51300d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 --2- 172.21.15.86:0/1912714763 >> v2:172.21.15.196:3300/0 conn(0x7f51300d0080 0x7f51300d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 -- 172.21.15.86:0/1912714763 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51300c5eb0 con 0x7f513015c410 2024-03-22T04:12:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.362+0000 7f512ea00640 1 --2- 172.21.15.86:0/1912714763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f513015c410 0x7f51300cd620 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5130073410 tx=0x7f512001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd53c0aa95d001d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512e000640 1 -- 172.21.15.86:0/1912714763 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51200a5570 con 0x7f513015c410 2024-03-22T04:12:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512e000640 1 -- 172.21.15.86:0/1912714763 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51200a5710 con 0x7f513015c410 2024-03-22T04:12:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512e000640 1 -- 172.21.15.86:0/1912714763 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51200a5ae0 con 0x7f513015c410 2024-03-22T04:12:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- 172.21.15.86:0/1912714763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f513015c410 msgr2=0x7f51300cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 --2- 172.21.15.86:0/1912714763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f513015c410 0x7f51300cd620 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5130073410 tx=0x7f512001c9f0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- 172.21.15.86:0/1912714763 shutdown_connections 2024-03-22T04:12:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 --2- 172.21.15.86:0/1912714763 >> v2:172.21.15.196:3300/0 conn(0x7f51300d0080 0x7f51300d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 --2- 172.21.15.86:0/1912714763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 0x7f51300cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 --2- 172.21.15.86:0/1912714763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f513015c410 0x7f51300cd620 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- 172.21.15.86:0/1912714763 >> 172.21.15.86:0/1912714763 conn(0x7f51300d1f70 msgr2=0x7f51300d2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- 172.21.15.86:0/1912714763 shutdown_connections 2024-03-22T04:12:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- 172.21.15.86:0/1912714763 wait complete. 2024-03-22T04:12:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 Processor -- start 2024-03-22T04:12:06.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- start start 2024-03-22T04:12:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 0x7f5130154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51300d0080 0x7f51300cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f513015c410 0x7f51301545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51300cf500 con 0x7f51300d0080 2024-03-22T04:12:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51300cf380 con 0x7f513015c410 2024-03-22T04:12:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f51300cf680 con 0x7f51300c86c0 2024-03-22T04:12:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f5126000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51300d0080 0x7f51300cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f5126000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51300d0080 0x7f51300cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39938/0 (socket says 172.21.15.86:39938) 2024-03-22T04:12:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f5126000640 1 -- 172.21.15.86:0/398008411 learned_addr learned my addr 172.21.15.86:0/398008411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512ea00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 0x7f5130154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.366+0000 7f512f400640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f513015c410 0x7f51301545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5126000640 1 -- 172.21.15.86:0/398008411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 msgr2=0x7f5130154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5126000640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 0x7f5130154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5126000640 1 -- 172.21.15.86:0/398008411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f513015c410 msgr2=0x7f51301545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5126000640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f513015c410 0x7f51301545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5126000640 1 -- 172.21.15.86:0/398008411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51300cbbc0 con 0x7f51300d0080 2024-03-22T04:12:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5126000640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51300d0080 0x7f51300cb680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5118009fd0 tx=0x7f511800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5127400640 1 -- 172.21.15.86:0/398008411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118017070 con 0x7f51300d0080 2024-03-22T04:12:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5127400640 1 -- 172.21.15.86:0/398008411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511800c940 con 0x7f51300d0080 2024-03-22T04:12:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5127400640 1 -- 172.21.15.86:0/398008411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511800cd30 con 0x7f51300d0080 2024-03-22T04:12:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51300ca5e0 con 0x7f51300d0080 2024-03-22T04:12:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51300caba0 con 0x7f51300d0080 2024-03-22T04:12:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513004f900 con 0x7f51300d0080 2024-03-22T04:12:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5127400640 1 -- 172.21.15.86:0/398008411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f511801a070 con 0x7f51300d0080 2024-03-22T04:12:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5127400640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f510c067b20 0x7f510c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f5127400640 1 -- 172.21.15.86:0/398008411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f5118086060 con 0x7f51300d0080 2024-03-22T04:12:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.370+0000 7f512ea00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f510c067b20 0x7f510c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.374+0000 7f512ea00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f510c067b20 0x7f510c069fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f51200ae000 tx=0x7f51200a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.374+0000 7f5127400640 1 -- 172.21.15.86:0/398008411 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f51180561f0 con 0x7f51300d0080 2024-03-22T04:12:06.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f513015a4f0 con 0x7f51300d0080 2024-03-22T04:12:06.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f5127400640 1 -- 172.21.15.86:0/398008411 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f511805a290 con 0x7f51300d0080 2024-03-22T04:12:06.532 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f510c067b20 msgr2=0x7f510c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f510c067b20 0x7f510c069fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f51200ae000 tx=0x7f51200a7070 comp rx=0 tx=0).stop 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51300d0080 msgr2=0x7f51300cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51300d0080 0x7f51300cb680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5118009fd0 tx=0x7f511800ed40 comp rx=0 tx=0).stop 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 shutdown_connections 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f510c067b20 0x7f510c069fe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f513015c410 0x7f51301545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51300d0080 0x7f51300cb680 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 --2- 172.21.15.86:0/398008411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51300c86c0 0x7f5130154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 >> 172.21.15.86:0/398008411 conn(0x7f51300d1f70 msgr2=0x7f513015f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:06.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 shutdown_connections 2024-03-22T04:12:06.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.530+0000 7f512fe00640 1 -- 172.21.15.86:0/398008411 wait complete. 2024-03-22T04:12:06.553 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:12:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab18000640 1 Processor -- start 2024-03-22T04:12:06.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab18000640 1 -- start start 2024-03-22T04:12:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab18000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab1015b3f0 0x7fab100729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab18000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fab100cc3e0 0x7fab10072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab18000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab100ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab18000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fab100716d0 con 0x7fab1015b3f0 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab18000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fab1006d990 con 0x7fab100cc3e0 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab18000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab10071920 con 0x7fab100b30f0 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.690+0000 7fab15600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab1015b3f0 0x7fab100729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab15600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab1015b3f0 0x7fab100729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39948/0 (socket says 172.21.15.86:39948) 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab15600640 1 -- 172.21.15.86:0/3645439529 learned_addr learned my addr 172.21.15.86:0/3645439529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab14c00640 1 --2- 172.21.15.86:0/3645439529 >> v2:172.21.15.196:3300/0 conn(0x7fab100cc3e0 0x7fab10072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab16000640 1 --2- 172.21.15.86:0/3645439529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab100ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab15600640 1 -- 172.21.15.86:0/3645439529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 msgr2=0x7fab100ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab15600640 1 --2- 172.21.15.86:0/3645439529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab100ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab15600640 1 -- 172.21.15.86:0/3645439529 >> v2:172.21.15.196:3300/0 conn(0x7fab100cc3e0 msgr2=0x7fab10072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab15600640 1 --2- 172.21.15.86:0/3645439529 >> v2:172.21.15.196:3300/0 conn(0x7fab100cc3e0 0x7fab10072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab15600640 1 -- 172.21.15.86:0/3645439529 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab10071c00 con 0x7fab1015b3f0 2024-03-22T04:12:06.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab16000640 1 --2- 172.21.15.86:0/3645439529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab100ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:06.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab15600640 1 --2- 172.21.15.86:0/3645439529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 0x7fab100729c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fab10070ed0 tx=0x7fab0401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97512c92478aee server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab0fe00640 1 -- 172.21.15.86:0/3645439529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040a55e0 con 0x7fab1015b3f0 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab0fe00640 1 -- 172.21.15.86:0/3645439529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab040a5780 con 0x7fab1015b3f0 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.694+0000 7fab0fe00640 1 -- 172.21.15.86:0/3645439529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040a5b50 con 0x7fab1015b3f0 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- 172.21.15.86:0/3645439529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 msgr2=0x7fab100729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 --2- 172.21.15.86:0/3645439529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 0x7fab100729c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fab10070ed0 tx=0x7fab0401c960 comp rx=0 tx=0).stop 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- 172.21.15.86:0/3645439529 shutdown_connections 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 --2- 172.21.15.86:0/3645439529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab100ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 --2- 172.21.15.86:0/3645439529 >> v2:172.21.15.196:3300/0 conn(0x7fab100cc3e0 0x7fab10072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 --2- 172.21.15.86:0/3645439529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 0x7fab100729c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- 172.21.15.86:0/3645439529 >> 172.21.15.86:0/3645439529 conn(0x7fab100b1df0 msgr2=0x7fab100b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- 172.21.15.86:0/3645439529 shutdown_connections 2024-03-22T04:12:06.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- 172.21.15.86:0/3645439529 wait complete. 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 Processor -- start 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- start start 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab1009d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab100cc3e0 0x7fab1009a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 0x7fab10152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab1015f1e0 con 0x7fab1015b3f0 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab1015f060 con 0x7fab100cc3e0 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab18000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab1015f360 con 0x7fab100b30f0 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab16000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 0x7fab10152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab15600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab1009d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab14c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab100cc3e0 0x7fab1009a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab15600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab1009d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:46330/0 (socket says 172.21.15.86:46330) 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab15600640 1 -- 172.21.15.86:0/1113417331 learned_addr learned my addr 172.21.15.86:0/1113417331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab14c00640 1 -- 172.21.15.86:0/1113417331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 msgr2=0x7fab1009d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab14c00640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab1009d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab14c00640 1 -- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 msgr2=0x7fab10152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab14c00640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 0x7fab10152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab14c00640 1 -- 172.21.15.86:0/1113417331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab10154060 con 0x7fab100cc3e0 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.698+0000 7fab16000640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 0x7fab10152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab14c00640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab100cc3e0 0x7fab1009a270 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faafc00e870 tx=0x7faafc00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab0d600640 1 -- 172.21.15.86:0/1113417331 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc017070 con 0x7fab100cc3e0 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab0d600640 1 -- 172.21.15.86:0/1113417331 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc00c940 con 0x7fab100cc3e0 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab0d600640 1 -- 172.21.15.86:0/1113417331 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc00ccf0 con 0x7fab100cc3e0 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab10154340 con 0x7fab100cc3e0 2024-03-22T04:12:06.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1009ad30 con 0x7fab100cc3e0 2024-03-22T04:12:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab10154ae0 con 0x7fab100cc3e0 2024-03-22T04:12:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab0d600640 1 -- 172.21.15.86:0/1113417331 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7faafc01a070 con 0x7fab100cc3e0 2024-03-22T04:12:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab0d600640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faaf0067b50 0x7faaf006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab0d600640 1 -- 172.21.15.86:0/1113417331 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7faafc0862e0 con 0x7fab100cc3e0 2024-03-22T04:12:06.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab15600640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faaf0067b50 0x7faaf006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:06.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.702+0000 7fab15600640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faaf0067b50 0x7faaf006a010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab0401b010 tx=0x7fab040a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:06.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.706+0000 7fab0d600640 1 -- 172.21.15.86:0/1113417331 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7faafc056470 con 0x7fab100cc3e0 2024-03-22T04:12:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.874+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fab100b1860 con 0x7fab100cc3e0 2024-03-22T04:12:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.874+0000 7fab0d600640 1 -- 172.21.15.86:0/1113417331 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7faafc05a510 con 0x7fab100cc3e0 2024-03-22T04:12:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faaf0067b50 msgr2=0x7faaf006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faaf0067b50 0x7faaf006a010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab0401b010 tx=0x7fab040a7070 comp rx=0 tx=0).stop 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab100cc3e0 msgr2=0x7fab1009a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab100cc3e0 0x7fab1009a270 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faafc00e870 tx=0x7faafc00ec70 comp rx=0 tx=0).stop 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 shutdown_connections 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faaf0067b50 0x7faaf006a010 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab100cc3e0 0x7fab1009a270 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1015b3f0 0x7fab10152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 --2- 172.21.15.86:0/1113417331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab100b30f0 0x7fab1009d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 >> 172.21.15.86:0/1113417331 conn(0x7fab100b1df0 msgr2=0x7fab10155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 shutdown_connections 2024-03-22T04:12:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:06.878+0000 7fab18000640 1 -- 172.21.15.86:0/1113417331 wait complete. 2024-03-22T04:12:06.897 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:12:06.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:06.898 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:06.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:12:06.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:06.959 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:12:06.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.146+0000 7fcf7b000640 1 Processor -- start 2024-03-22T04:12:07.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.146+0000 7fcf7b000640 1 -- start start 2024-03-22T04:12:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf7b000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcf7406c270 0x7fcf74162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf7b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf740b2bd0 con 0x7fcf74162540 2024-03-22T04:12:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf7b000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf740b2610 con 0x7fcf74147e10 2024-03-22T04:12:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf7b000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcf74162990 con 0x7fcf7406c270 2024-03-22T04:12:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf79000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcf74162540 0x7fcf7415fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37946/0 (socket says 172.21.15.86:37946) 2024-03-22T04:12:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf79000640 1 -- 172.21.15.86:0/1844604021 learned_addr learned my addr 172.21.15.86:0/1844604021 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf6b400640 1 --2- 172.21.15.86:0/1844604021 >> v2:172.21.15.196:3300/0 conn(0x7fcf7406c270 0x7fcf74162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf73400640 1 -- 172.21.15.86:0/1844604021 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160396389 0 0) 0x7fcf740b2bd0 con 0x7fcf74162540 2024-03-22T04:12:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf73400640 1 -- 172.21.15.86:0/1844604021 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf58003530 con 0x7fcf74162540 2024-03-22T04:12:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf73400640 1 -- 172.21.15.86:0/1844604021 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4018998249 0 0) 0x7fcf740b2610 con 0x7fcf74147e10 2024-03-22T04:12:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf73400640 1 -- 172.21.15.86:0/1844604021 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf740b2bd0 con 0x7fcf74147e10 2024-03-22T04:12:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf6b400640 1 -- 172.21.15.86:0/1844604021 >> v1:172.21.15.196:6789/0 conn(0x7fcf74147e10 legacy=0x7fcf740b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf6b400640 1 -- 172.21.15.86:0/1844604021 >> v1:172.21.15.86:6789/0 conn(0x7fcf74162540 legacy=0x7fcf7415fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf6b400640 1 -- 172.21.15.86:0/1844604021 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf740b28f0 con 0x7fcf7406c270 2024-03-22T04:12:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.150+0000 7fcf6b400640 1 --2- 172.21.15.86:0/1844604021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7406c270 0x7fcf74162000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0097c0 tx=0x7fcf5c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d2eda58a9487161 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf73400640 1 -- 172.21.15.86:0/1844604021 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0b1070 con 0x7fcf7406c270 2024-03-22T04:12:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf73400640 1 -- 172.21.15.86:0/1844604021 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5c0ac410 con 0x7fcf7406c270 2024-03-22T04:12:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf73400640 1 -- 172.21.15.86:0/1844604021 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0ac7c0 con 0x7fcf7406c270 2024-03-22T04:12:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- 172.21.15.86:0/1844604021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7406c270 msgr2=0x7fcf74162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 --2- 172.21.15.86:0/1844604021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7406c270 0x7fcf74162000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0097c0 tx=0x7fcf5c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- 172.21.15.86:0/1844604021 shutdown_connections 2024-03-22T04:12:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 --2- 172.21.15.86:0/1844604021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7406c270 0x7fcf74162000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- 172.21.15.86:0/1844604021 >> 172.21.15.86:0/1844604021 conn(0x7fcf740b6620 msgr2=0x7fcf740b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- 172.21.15.86:0/1844604021 shutdown_connections 2024-03-22T04:12:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- 172.21.15.86:0/1844604021 wait complete. 2024-03-22T04:12:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 Processor -- start 2024-03-22T04:12:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- start start 2024-03-22T04:12:07.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7406c270 0x7fcf74139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74147e10 0x7fcf740cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74162540 0x7fcf74146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7416c160 con 0x7fcf74162540 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74160c90 con 0x7fcf74147e10 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf7b000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7416c2e0 con 0x7fcf7406c270 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf79000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74162540 0x7fcf74146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf79000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74162540 0x7fcf74146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39958/0 (socket says 172.21.15.86:39958) 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf79000640 1 -- 172.21.15.86:0/2263245078 learned_addr learned my addr 172.21.15.86:0/2263245078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf73e00640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7406c270 0x7fcf74139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.154+0000 7fcf6b400640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74147e10 0x7fcf740cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf79000640 1 -- 172.21.15.86:0/2263245078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7406c270 msgr2=0x7fcf74139f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf79000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7406c270 0x7fcf74139f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf79000640 1 -- 172.21.15.86:0/2263245078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74147e10 msgr2=0x7fcf740cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf79000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74147e10 0x7fcf740cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf79000640 1 -- 172.21.15.86:0/2263245078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf74158450 con 0x7fcf74162540 2024-03-22T04:12:07.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf79000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74162540 0x7fcf74146a00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00cad0 tx=0x7fcf6c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:07.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf70c00640 1 -- 172.21.15.86:0/2263245078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c017070 con 0x7fcf74162540 2024-03-22T04:12:07.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf70c00640 1 -- 172.21.15.86:0/2263245078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6c00ac50 con 0x7fcf74162540 2024-03-22T04:12:07.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf74158730 con 0x7fcf74162540 2024-03-22T04:12:07.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf70c00640 1 -- 172.21.15.86:0/2263245078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c012400 con 0x7fcf74162540 2024-03-22T04:12:07.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf740cba20 con 0x7fcf74162540 2024-03-22T04:12:07.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf74068650 con 0x7fcf74162540 2024-03-22T04:12:07.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf70c00640 1 -- 172.21.15.86:0/2263245078 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcf6c01a070 con 0x7fcf74162540 2024-03-22T04:12:07.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf70c00640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf50067b20 0x7fcf50069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf70c00640 1 -- 172.21.15.86:0/2263245078 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fcf6c085e80 con 0x7fcf74162540 2024-03-22T04:12:07.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.158+0000 7fcf73e00640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf50067b20 0x7fcf50069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.162+0000 7fcf73e00640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf50067b20 0x7fcf50069fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcf7413efa0 tx=0x7fcf64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:07.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.162+0000 7fcf70c00640 1 -- 172.21.15.86:0/2263245078 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcf6c056010 con 0x7fcf74162540 2024-03-22T04:12:07.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf7415a530 con 0x7fcf74162540 2024-03-22T04:12:07.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf70c00640 1 -- 172.21.15.86:0/2263245078 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7fcf6c05a0b0 con 0x7fcf74162540 2024-03-22T04:12:07.359 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:07.359 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:11:51.996418+0000","modified":"2024-03-22T04:11:55.030984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74930":{"gid":74930,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1426902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1426902111},{"type":"v1","addr":"172.21.15.196:6835","nonce":1426902111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74930,"qdb_cluster":[74930]},"id":2}]} 2024-03-22T04:12:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-03-22T04:12:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf50067b20 msgr2=0x7fcf50069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf50067b20 0x7fcf50069fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcf7413efa0 tx=0x7fcf64009040 comp rx=0 tx=0).stop 2024-03-22T04:12:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74162540 msgr2=0x7fcf74146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74162540 0x7fcf74146a00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00cad0 tx=0x7fcf6c00cf90 comp rx=0 tx=0).stop 2024-03-22T04:12:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 shutdown_connections 2024-03-22T04:12:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf50067b20 0x7fcf50069fe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf74147e10 0x7fcf740cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74162540 0x7fcf74146a00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 --2- 172.21.15.86:0/2263245078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7406c270 0x7fcf74139f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 >> 172.21.15.86:0/2263245078 conn(0x7fcf740b6620 msgr2=0x7fcf7406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 shutdown_connections 2024-03-22T04:12:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.358+0000 7fcf7b000640 1 -- 172.21.15.86:0/2263245078 wait complete. 2024-03-22T04:12:07.380 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-22T04:12:07.549 INFO:teuthology.orchestra.run.smithi196.stdout:{"message":"","return_code":0} 2024-03-22T04:12:07.550 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:12:07.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6769200640 1 Processor -- start 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6769200640 1 -- start start 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6769200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f676406ef40 0x7f6764142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6769200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676415c7c0 con 0x7f67640ad710 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6769200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676406dea0 con 0x7f67640ad330 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6769200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f676406e180 con 0x7f676406ef40 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6759400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f676406ef40 0x7f6764142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6762800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f67640ad710 0x7f67641402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37950/0 (socket says 172.21.15.86:37950) 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6762800640 1 -- 172.21.15.86:0/3230305808 learned_addr learned my addr 172.21.15.86:0/3230305808 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6761400640 1 -- 172.21.15.86:0/3230305808 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883213515 0 0) 0x7f676415c7c0 con 0x7f67640ad710 2024-03-22T04:12:07.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.686+0000 7f6761400640 1 -- 172.21.15.86:0/3230305808 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6748003530 con 0x7f67640ad710 2024-03-22T04:12:07.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6761400640 1 -- 172.21.15.86:0/3230305808 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009046792 0 0) 0x7f676406dea0 con 0x7f67640ad330 2024-03-22T04:12:07.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6759400640 1 -- 172.21.15.86:0/3230305808 >> v1:172.21.15.196:6789/0 conn(0x7f67640ad330 legacy=0x7f676413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6759400640 1 -- 172.21.15.86:0/3230305808 >> v1:172.21.15.86:6789/0 conn(0x7f67640ad710 legacy=0x7f67641402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6759400640 1 -- 172.21.15.86:0/3230305808 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676406e460 con 0x7f676406ef40 2024-03-22T04:12:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6759400640 1 --2- 172.21.15.86:0/3230305808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f676406ef40 0x7f6764142690 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f674c0097c0 tx=0x7f674c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1364ba79268a2785 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6761400640 1 -- 172.21.15.86:0/3230305808 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0aa5f0 con 0x7f676406ef40 2024-03-22T04:12:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6761400640 1 -- 172.21.15.86:0/3230305808 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674c0af070 con 0x7f676406ef40 2024-03-22T04:12:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6761400640 1 -- 172.21.15.86:0/3230305808 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0aa9b0 con 0x7f676406ef40 2024-03-22T04:12:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6769200640 1 -- 172.21.15.86:0/3230305808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f676406ef40 msgr2=0x7f6764142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6769200640 1 --2- 172.21.15.86:0/3230305808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f676406ef40 0x7f6764142690 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f674c0097c0 tx=0x7f674c01c470 comp rx=0 tx=0).stop 2024-03-22T04:12:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6769200640 1 -- 172.21.15.86:0/3230305808 shutdown_connections 2024-03-22T04:12:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6769200640 1 --2- 172.21.15.86:0/3230305808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f676406ef40 0x7f6764142690 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6769200640 1 -- 172.21.15.86:0/3230305808 >> 172.21.15.86:0/3230305808 conn(0x7f67640b1df0 msgr2=0x7f67640b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6769200640 1 -- 172.21.15.86:0/3230305808 shutdown_connections 2024-03-22T04:12:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.690+0000 7f6769200640 1 -- 172.21.15.86:0/3230305808 wait complete. 2024-03-22T04:12:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 Processor -- start 2024-03-22T04:12:07.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 -- start start 2024-03-22T04:12:07.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f676406ef40 0x7f676415b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67640ad330 0x7f676415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67640ad710 0x7f676415aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f676406db10 con 0x7f67640ad710 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67641409d0 con 0x7f67640ad330 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f676416a780 con 0x7f676406ef40 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67640ad710 0x7f676415aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6761e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f676406ef40 0x7f676415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67640ad710 0x7f676415aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39978/0 (socket says 172.21.15.86:39978) 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 -- 172.21.15.86:0/274967456 learned_addr learned my addr 172.21.15.86:0/274967456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6759400640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67640ad330 0x7f676415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 -- 172.21.15.86:0/274967456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f676406ef40 msgr2=0x7f676415b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f676406ef40 0x7f676415b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 -- 172.21.15.86:0/274967456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67640ad330 msgr2=0x7f676415a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67640ad330 0x7f676415a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 -- 172.21.15.86:0/274967456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6764166bf0 con 0x7f67640ad710 2024-03-22T04:12:07.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6762800640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67640ad710 0x7f676415aad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c430 tx=0x7f675c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:07.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f675aa00640 1 -- 172.21.15.86:0/274967456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c017070 con 0x7f67640ad710 2024-03-22T04:12:07.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f675aa00640 1 -- 172.21.15.86:0/274967456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675c005cf0 con 0x7f67640ad710 2024-03-22T04:12:07.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6764166ed0 con 0x7f67640ad710 2024-03-22T04:12:07.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.694+0000 7f675aa00640 1 -- 172.21.15.86:0/274967456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c0072d0 con 0x7f67640ad710 2024-03-22T04:12:07.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.698+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6764167490 con 0x7f67640ad710 2024-03-22T04:12:07.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.698+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6764068650 con 0x7f67640ad710 2024-03-22T04:12:07.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.698+0000 7f675aa00640 1 -- 172.21.15.86:0/274967456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f675c01a070 con 0x7f67640ad710 2024-03-22T04:12:07.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.698+0000 7f675aa00640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6740067b20 0x7f6740069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:07.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.698+0000 7f675aa00640 1 -- 172.21.15.86:0/274967456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f675c085460 con 0x7f67640ad710 2024-03-22T04:12:07.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.698+0000 7f6761e00640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6740067b20 0x7f6740069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:07.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.698+0000 7f6761e00640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6740067b20 0x7f6740069fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f67540012f0 tx=0x7f6754009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:07.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.702+0000 7f675aa00640 1 -- 172.21.15.86:0/274967456 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f675c0556b0 con 0x7f67640ad710 2024-03-22T04:12:07.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.890+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f676415fae0 con 0x7f67640ad710 2024-03-22T04:12:07.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f675aa00640 1 -- 172.21.15.86:0/274967456 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f675c059750 con 0x7f67640ad710 2024-03-22T04:12:07.895 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:07.895 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:11:51.996418+0000","modified":"2024-03-22T04:11:55.030984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74930":{"gid":74930,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1426902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1426902111},{"type":"v1","addr":"172.21.15.196:6835","nonce":1426902111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74930,"qdb_cluster":[74930]},"id":2}]} 2024-03-22T04:12:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6740067b20 msgr2=0x7f6740069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6740067b20 0x7f6740069fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f67540012f0 tx=0x7f6754009040 comp rx=0 tx=0).stop 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67640ad710 msgr2=0x7f676415aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67640ad710 0x7f676415aad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c430 tx=0x7f675c00c8f0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 shutdown_connections 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6740067b20 0x7f6740069fe0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67640ad330 0x7f676415a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67640ad710 0x7f676415aad0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 --2- 172.21.15.86:0/274967456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f676406ef40 0x7f676415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 >> 172.21.15.86:0/274967456 conn(0x7f67640b1df0 msgr2=0x7f67640aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 shutdown_connections 2024-03-22T04:12:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:07.894+0000 7f6769200640 1 -- 172.21.15.86:0/274967456 wait complete. 2024-03-22T04:12:07.916 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json tag path /parentdir mytag 2024-03-22T04:12:08.077 INFO:teuthology.orchestra.run.smithi196.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-03-22T04:12:08.078 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-03-22T04:12:18.079 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-03-22T04:12:18.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c18c3a080 1 Processor -- start 2024-03-22T04:12:18.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c18c3a080 1 -- start start 2024-03-22T04:12:18.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c18c3a080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb319410 0x5565eb31bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c18c3a080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb31c8d0 0x5565eb3bc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c18c3a080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 0x5565eb3bce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c18c3a080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5565eb335740 con 0x5565eb31c8d0 2024-03-22T04:12:18.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c18c3a080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565eb290620 con 0x5565eb319410 2024-03-22T04:12:18.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c18c3a080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5565eb3359e0 con 0x5565eb3c1610 2024-03-22T04:12:18.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb31c8d0 0x5565eb3bc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb31c8d0 0x5565eb3bc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53406/0 (socket says 172.21.15.86:53406) 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c0fe00640 1 -- 172.21.15.86:0/4078637667 learned_addr learned my addr 172.21.15.86:0/4078637667 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c15e00640 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb319410 0x5565eb31bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c16800640 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 0x5565eb3bce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c0fe00640 1 -- 172.21.15.86:0/4078637667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 msgr2=0x5565eb3bce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c0fe00640 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 0x5565eb3bce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c0fe00640 1 -- 172.21.15.86:0/4078637667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb319410 msgr2=0x5565eb31bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c0fe00640 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb319410 0x5565eb31bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.130+0000 7f7c0fe00640 1 -- 172.21.15.86:0/4078637667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565eb334760 con 0x5565eb31c8d0 2024-03-22T04:12:18.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c0fe00640 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb31c8d0 0x5565eb3bc930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7c00018f50 tx=0x7f7c00003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01c6ae7f2ee60e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c14a00640 1 -- 172.21.15.86:0/4078637667 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c000af040 con 0x5565eb31c8d0 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c14a00640 1 -- 172.21.15.86:0/4078637667 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c000b9070 con 0x5565eb31c8d0 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c14a00640 1 -- 172.21.15.86:0/4078637667 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c000aadf0 con 0x5565eb31c8d0 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4078637667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb31c8d0 msgr2=0x5565eb3bc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb31c8d0 0x5565eb3bc930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7c00018f50 tx=0x7f7c00003030 comp rx=0 tx=0).stop 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4078637667 shutdown_connections 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 0x5565eb3bce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb31c8d0 0x5565eb3bc930 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 --2- 172.21.15.86:0/4078637667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb319410 0x5565eb31bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4078637667 >> 172.21.15.86:0/4078637667 conn(0x5565eb317ff0 msgr2=0x5565eb2ea830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4078637667 shutdown_connections 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4078637667 wait complete. 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 Processor -- start 2024-03-22T04:12:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- start start 2024-03-22T04:12:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb319410 0x5565eb3a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb31c8d0 0x5565eb3a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 0x5565eb3314f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5565eb2e8b70 con 0x5565eb319410 2024-03-22T04:12:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565eb3be110 con 0x5565eb31c8d0 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c18c3a080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5565eb332260 con 0x5565eb3c1610 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c15e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb319410 0x5565eb3a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c15e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb319410 0x5565eb3a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53418/0 (socket says 172.21.15.86:53418) 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb31c8d0 0x5565eb3a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c15e00640 1 -- 172.21.15.86:0/4201984934 learned_addr learned my addr 172.21.15.86:0/4201984934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c16800640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 0x5565eb3314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c15e00640 1 -- 172.21.15.86:0/4201984934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 msgr2=0x5565eb3314f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c15e00640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565eb3c1610 0x5565eb3314f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c15e00640 1 -- 172.21.15.86:0/4201984934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb31c8d0 msgr2=0x5565eb3a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c15e00640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565eb31c8d0 0x5565eb3a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.134+0000 7f7c15e00640 1 -- 172.21.15.86:0/4201984934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565eb332500 con 0x5565eb319410 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c15e00640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb319410 0x5565eb3a0ca0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7c080087a0 tx=0x7f7c08008c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c0ea00640 1 -- 172.21.15.86:0/4201984934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08019020 con 0x5565eb319410 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c0ea00640 1 -- 172.21.15.86:0/4201984934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c08010ba0 con 0x5565eb319410 2024-03-22T04:12:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c0ea00640 1 -- 172.21.15.86:0/4201984934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08015450 con 0x5565eb319410 2024-03-22T04:12:18.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5565eb3327a0 con 0x5565eb319410 2024-03-22T04:12:18.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565eb332ce0 con 0x5565eb319410 2024-03-22T04:12:18.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c0ea00640 1 -- 172.21.15.86:0/4201984934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7c0801c020 con 0x5565eb319410 2024-03-22T04:12:18.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c0ea00640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bf4066500 0x7f7bf40689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c0ea00640 1 -- 172.21.15.86:0/4201984934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f7c08085b30 con 0x5565eb319410 2024-03-22T04:12:18.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c0fe00640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bf4066500 0x7f7bf40689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c18c3a080 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5565eb32ed60 0x5565eb32f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.138+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x5565eb32f680 con 0x5565eb32ed60 2024-03-22T04:12:18.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c16800640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5565eb32ed60 0x5565eb32f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c0fe00640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bf4066500 0x7f7bf40689c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c000b8000 tx=0x7f7c000a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c16800640 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5565eb32ed60 0x5565eb32f140 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c16800640 1 -- 172.21.15.86:0/4201984934 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f7c10001a30 con 0x5565eb32ed60 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5565eb32ed60 msgr2=0x5565eb32f140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5565eb32ed60 0x5565eb32f140 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bf4066500 msgr2=0x7f7bf40689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bf4066500 0x7f7bf40689c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c000b8000 tx=0x7f7c000a7000 comp rx=0 tx=0).stop 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb319410 msgr2=0x5565eb3a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 --2- 172.21.15.86:0/4201984934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565eb319410 0x5565eb3a0ca0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7c080087a0 tx=0x7f7c08008c60 comp rx=0 tx=0).stop 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c16800640 1 -- 172.21.15.86:0/4201984934 reap_dead start 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 shutdown_connections 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 >> 172.21.15.86:0/4201984934 conn(0x5565eb317ff0 msgr2=0x5565eb2ea7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 shutdown_connections 2024-03-22T04:12:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.142+0000 7f7c18c3a080 1 -- 172.21.15.86:0/4201984934 wait complete. 2024-03-22T04:12:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:mytag 2024-03-22T04:12:18.148 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-03-22T04:12:18.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 Processor -- start 2024-03-22T04:12:18.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 -- start start 2024-03-22T04:12:18.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266b7cb0 0x55c326787c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 0x55c326790640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266e6900 0x55c326790b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c32665d620 con 0x55c3266b7cb0 2024-03-22T04:12:18.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c326661670 con 0x55c3266e6900 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55c3266b8a50 con 0x55c3266e6520 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266b7cb0 0x55c326787c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266b7cb0 0x55c326787c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53422/0 (socket says 172.21.15.86:53422) 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 -- 172.21.15.86:0/2539333984 learned_addr learned my addr 172.21.15.86:0/2539333984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecc000640 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266e6900 0x55c326790b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecac00640 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 0x55c326790640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 -- 172.21.15.86:0/2539333984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 msgr2=0x55c326790640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 0x55c326790640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 -- 172.21.15.86:0/2539333984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266e6900 msgr2=0x55c326790b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266e6900 0x55c326790b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 -- 172.21.15.86:0/2539333984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3266b9f00 con 0x55c3266b7cb0 2024-03-22T04:12:18.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ecb600640 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266b7cb0 0x55c326787c50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc019520 tx=0x7f6ebc003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f62d50905ba2cd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ec9800640 1 -- 172.21.15.86:0/2539333984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0ae040 con 0x55c3266b7cb0 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ec9800640 1 -- 172.21.15.86:0/2539333984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc0b8070 con 0x55c3266b7cb0 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ec9800640 1 -- 172.21.15.86:0/2539333984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b3410 con 0x55c3266b7cb0 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 -- 172.21.15.86:0/2539333984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266b7cb0 msgr2=0x55c326787c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266b7cb0 0x55c326787c50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc019520 tx=0x7f6ebc003030 comp rx=0 tx=0).stop 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 -- 172.21.15.86:0/2539333984 shutdown_connections 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266e6900 0x55c326790b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 0x55c326790640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 --2- 172.21.15.86:0/2539333984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266b7cb0 0x55c326787c50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.194+0000 7f6ece2e2080 1 -- 172.21.15.86:0/2539333984 >> 172.21.15.86:0/2539333984 conn(0x55c3266e3ec0 msgr2=0x55c3266e8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 -- 172.21.15.86:0/2539333984 shutdown_connections 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 -- 172.21.15.86:0/2539333984 wait complete. 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 Processor -- start 2024-03-22T04:12:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 -- start start 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266b7cb0 0x55c32678c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 0x55c3266ba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266e6900 0x55c32678acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c3266ba1a0 con 0x55c3266e6900 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3266fd010 con 0x55c3266b7cb0 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55c3266fc630 con 0x55c3266e6520 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266e6900 0x55c32678acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecb600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266b7cb0 0x55c32678c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecb600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266b7cb0 0x55c32678c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:47734/0 (socket says 172.21.15.86:47734) 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecb600640 1 -- 172.21.15.86:0/3403899309 learned_addr learned my addr 172.21.15.86:0/3403899309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266e6900 0x55c32678acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53438/0 (socket says 172.21.15.86:53438) 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecac00640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 0x55c3266ba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecc000640 1 -- 172.21.15.86:0/3403899309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 msgr2=0x55c3266ba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecc000640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c3266e6520 0x55c3266ba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecc000640 1 -- 172.21.15.86:0/3403899309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266b7cb0 msgr2=0x55c32678c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecc000640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3266b7cb0 0x55c32678c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecc000640 1 -- 172.21.15.86:0/3403899309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32678b9a0 con 0x55c3266e6900 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ecc000640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266e6900 0x55c32678acd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400d530 tx=0x7f6ec400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ec3e00640 1 -- 172.21.15.86:0/3403899309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4006260 con 0x55c3266e6900 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ec3e00640 1 -- 172.21.15.86:0/3403899309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4018070 con 0x55c3266e6900 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ec3e00640 1 -- 172.21.15.86:0/3403899309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40136a0 con 0x55c3266e6900 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c32678bc40 con 0x55c3266e6900 2024-03-22T04:12:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.198+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c32678c420 con 0x55c3266e6900 2024-03-22T04:12:18.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ec3e00640 1 -- 172.21.15.86:0/3403899309 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6ec4013840 con 0x55c3266e6900 2024-03-22T04:12:18.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ec3e00640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6eac066530 0x7f6eac0689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ec3e00640 1 -- 172.21.15.86:0/3403899309 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f6ec4084e60 con 0x55c3266e6900 2024-03-22T04:12:18.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ecb600640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6eac066530 0x7f6eac0689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c3266baf30 0x55c326770640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x55c3266fe180 con 0x55c3266baf30 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ecac00640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c3266baf30 0x55c326770640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ecb600640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6eac066530 0x7f6eac0689f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc01aeb0 tx=0x7f6ebc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ecac00640 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c3266baf30 0x55c326770640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ecac00640 1 -- 172.21.15.86:0/3403899309 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f6eb8011760 con 0x55c3266baf30 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c3266baf30 msgr2=0x55c326770640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c3266baf30 0x55c326770640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6eac066530 msgr2=0x7f6eac0689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6eac066530 0x7f6eac0689f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc01aeb0 tx=0x7f6ebc0a6000 comp rx=0 tx=0).stop 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266e6900 msgr2=0x55c32678acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 --2- 172.21.15.86:0/3403899309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c3266e6900 0x55c32678acd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400d530 tx=0x7f6ec400d9f0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ecc000640 1 -- 172.21.15.86:0/3403899309 reap_dead start 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 shutdown_connections 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 >> 172.21.15.86:0/3403899309 conn(0x55c3266e3ec0 msgr2=0x55c326702380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 shutdown_connections 2024-03-22T04:12:18.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.202+0000 7f6ece2e2080 1 -- 172.21.15.86:0/3403899309 wait complete. 2024-03-22T04:12:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:mytag 2024-03-22T04:12:18.211 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-22T04:12:18.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 Processor -- start 2024-03-22T04:12:18.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- start start 2024-03-22T04:12:18.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 0x5617db708060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db791770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 0x5617db8251e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5617db6ef250 con 0x5617db793460 2024-03-22T04:12:18.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5617db664620 con 0x5617db793840 2024-03-22T04:12:18.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5617db6ef4f0 con 0x5617db6eecd0 2024-03-22T04:12:18.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db791770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271f600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 0x5617db8251e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 0x5617db708060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db791770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53452/0 (socket says 172.21.15.86:53452) 2024-03-22T04:12:18.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 -- 172.21.15.86:0/2796541117 learned_addr learned my addr 172.21.15.86:0/2796541117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 -- 172.21.15.86:0/2796541117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 msgr2=0x5617db708060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 --2- 172.21.15.86:0/2796541117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 0x5617db708060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 -- 172.21.15.86:0/2796541117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 msgr2=0x5617db8251e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 --2- 172.21.15.86:0/2796541117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 0x5617db8251e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 -- 172.21.15.86:0/2796541117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617db7070a0 con 0x5617db793460 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271e200640 1 --2- 172.21.15.86:0/2796541117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db791770 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f270c0ae040 tx=0x7f270c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c058d801903a8ab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271ce00640 1 -- 172.21.15.86:0/2796541117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c0bc070 con 0x5617db793460 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271ce00640 1 -- 172.21.15.86:0/2796541117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270c0ab840 con 0x5617db793460 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f271ce00640 1 -- 172.21.15.86:0/2796541117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c0b0040 con 0x5617db793460 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- 172.21.15.86:0/2796541117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 msgr2=0x5617db791770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 --2- 172.21.15.86:0/2796541117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db791770 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f270c0ae040 tx=0x7f270c003030 comp rx=0 tx=0).stop 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- 172.21.15.86:0/2796541117 shutdown_connections 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 --2- 172.21.15.86:0/2796541117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 0x5617db8251e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 --2- 172.21.15.86:0/2796541117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db791770 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 --2- 172.21.15.86:0/2796541117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 0x5617db708060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- 172.21.15.86:0/2796541117 >> 172.21.15.86:0/2796541117 conn(0x5617db6eaec0 msgr2=0x5617db6c1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- 172.21.15.86:0/2796541117 shutdown_connections 2024-03-22T04:12:18.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.298+0000 7f2721a4e080 1 -- 172.21.15.86:0/2796541117 wait complete. 2024-03-22T04:12:18.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 Processor -- start 2024-03-22T04:12:18.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 -- start start 2024-03-22T04:12:18.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 0x5617db78aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db704190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 0x5617db7046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5617db705420 con 0x5617db793460 2024-03-22T04:12:18.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5617db7056c0 con 0x5617db793840 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5617db706e00 con 0x5617db6eecd0 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db704190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db704190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53462/0 (socket says 172.21.15.86:53462) 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 -- 172.21.15.86:0/400269248 learned_addr learned my addr 172.21.15.86:0/400269248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271ec00640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 0x5617db78aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271f600640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 0x5617db7046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 -- 172.21.15.86:0/400269248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 msgr2=0x5617db78aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5617db6eecd0 0x5617db78aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 -- 172.21.15.86:0/400269248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 msgr2=0x5617db7046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5617db793840 0x5617db7046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 -- 172.21.15.86:0/400269248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617db78ca80 con 0x5617db793460 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f271e200640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db704190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f270c01b8a0 tx=0x7f270c018160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2717400640 1 -- 172.21.15.86:0/400269248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c0c6070 con 0x5617db793460 2024-03-22T04:12:18.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2717400640 1 -- 172.21.15.86:0/400269248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270c0abd50 con 0x5617db793460 2024-03-22T04:12:18.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2717400640 1 -- 172.21.15.86:0/400269248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c0bc040 con 0x5617db793460 2024-03-22T04:12:18.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5617db78cd20 con 0x5617db793460 2024-03-22T04:12:18.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.302+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617db792550 con 0x5617db793460 2024-03-22T04:12:18.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2717400640 1 -- 172.21.15.86:0/400269248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f270c0c9070 con 0x5617db793460 2024-03-22T04:12:18.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2717400640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2700066500 0x7f27000689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2717400640 1 -- 172.21.15.86:0/400269248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f270c1201d0 con 0x5617db793460 2024-03-22T04:12:18.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f271ec00640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2700066500 0x7f27000689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5617db78f730 0x5617db6be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x5617db6be5e0 con 0x5617db78f730 2024-03-22T04:12:18.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f271f600640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5617db78f730 0x5617db6be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f271ec00640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2700066500 0x7f27000689c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2710005410 tx=0x7f271000c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f271f600640 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5617db78f730 0x5617db6be0a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f271f600640 1 -- 172.21.15.86:0/400269248 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f27180034c0 con 0x5617db78f730 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5617db78f730 msgr2=0x5617db6be0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5617db78f730 0x5617db6be0a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2700066500 msgr2=0x7f27000689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2700066500 0x7f27000689c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2710005410 tx=0x7f271000c040 comp rx=0 tx=0).stop 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 msgr2=0x5617db704190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 --2- 172.21.15.86:0/400269248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617db793460 0x5617db704190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f270c01b8a0 tx=0x7f270c018160 comp rx=0 tx=0).stop 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f271f600640 1 -- 172.21.15.86:0/400269248 reap_dead start 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 shutdown_connections 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 >> 172.21.15.86:0/400269248 conn(0x5617db6eaec0 msgr2=0x5617db6c1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 shutdown_connections 2024-03-22T04:12:18.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.306+0000 7f2721a4e080 1 -- 172.21.15.86:0/400269248 wait complete. 2024-03-22T04:12:18.313 INFO:teuthology.orchestra.run.smithi086.stdout:mytag 2024-03-22T04:12:18.314 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-22T04:12:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 Processor -- start 2024-03-22T04:12:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- start start 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 0x561d4e3f3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5460 0x561d4e4c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5840 0x561d4e4c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561d4e4c6ab0 con 0x561d4e4c5840 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d4e396620 con 0x561d4e4c5460 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561d4e4c6c30 con 0x561d4e420cd0 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5840 0x561d4e4c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe87c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 0x561d4e3f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5840 0x561d4e4c2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53466/0 (socket says 172.21.15.86:53466) 2024-03-22T04:12:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 -- 172.21.15.86:0/2838910955 learned_addr learned my addr 172.21.15.86:0/2838910955 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe87200640 1 --2- 172.21.15.86:0/2838910955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5460 0x561d4e4c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 -- 172.21.15.86:0/2838910955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 msgr2=0x561d4e3f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 --2- 172.21.15.86:0/2838910955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 0x561d4e3f3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 -- 172.21.15.86:0/2838910955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5460 msgr2=0x561d4e4c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 --2- 172.21.15.86:0/2838910955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5460 0x561d4e4c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 -- 172.21.15.86:0/2838910955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4e431530 con 0x561d4e4c5840 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe88600640 1 --2- 172.21.15.86:0/2838910955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5840 0x561d4e4c2500 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efe8000b760 tx=0x7efe8000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d243927b5bd9ef server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe85e00640 1 -- 172.21.15.86:0/2838910955 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80016020 con 0x561d4e4c5840 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe85e00640 1 -- 172.21.15.86:0/2838910955 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8000d750 con 0x561d4e4c5840 2024-03-22T04:12:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe85e00640 1 -- 172.21.15.86:0/2838910955 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8000db20 con 0x561d4e4c5840 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- 172.21.15.86:0/2838910955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5840 msgr2=0x561d4e4c2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 --2- 172.21.15.86:0/2838910955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5840 0x561d4e4c2500 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efe8000b760 tx=0x7efe8000bc20 comp rx=0 tx=0).stop 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- 172.21.15.86:0/2838910955 shutdown_connections 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 --2- 172.21.15.86:0/2838910955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5460 0x561d4e4c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 --2- 172.21.15.86:0/2838910955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5840 0x561d4e4c2500 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 --2- 172.21.15.86:0/2838910955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 0x561d4e3f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- 172.21.15.86:0/2838910955 >> 172.21.15.86:0/2838910955 conn(0x561d4e41cec0 msgr2=0x561d4e4c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- 172.21.15.86:0/2838910955 shutdown_connections 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.398+0000 7efe8aa55080 1 -- 172.21.15.86:0/2838910955 wait complete. 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 Processor -- start 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 -- start start 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 0x561d4e4c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5460 0x561d4e4beb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5840 0x561d4e4c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561d4e431a70 con 0x561d4e4c5460 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d4e4317d0 con 0x561d4e4c5840 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561d4e4c4a90 con 0x561d4e420cd0 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5460 0x561d4e4beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5460 0x561d4e4beb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53478/0 (socket says 172.21.15.86:53478) 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 -- 172.21.15.86:0/3345542512 learned_addr learned my addr 172.21.15.86:0/3345542512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87c00640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 0x561d4e4c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe88600640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5840 0x561d4e4c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 -- 172.21.15.86:0/3345542512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 msgr2=0x561d4e4c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561d4e420cd0 0x561d4e4c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 -- 172.21.15.86:0/3345542512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5840 msgr2=0x561d4e4c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4e4c5840 0x561d4e4c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 -- 172.21.15.86:0/3345542512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4e4c1b90 con 0x561d4e4c5460 2024-03-22T04:12:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe87200640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5460 0x561d4e4beb30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efe7400a500 tx=0x7efe740180c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe84a00640 1 -- 172.21.15.86:0/3345542512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe740af040 con 0x561d4e4c5460 2024-03-22T04:12:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe84a00640 1 -- 172.21.15.86:0/3345542512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe740b4910 con 0x561d4e4c5460 2024-03-22T04:12:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d4e4c1e30 con 0x561d4e4c5460 2024-03-22T04:12:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe84a00640 1 -- 172.21.15.86:0/3345542512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe740b4ce0 con 0x561d4e4c5460 2024-03-22T04:12:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.402+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4e4c2370 con 0x561d4e4c5460 2024-03-22T04:12:18.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe84a00640 1 -- 172.21.15.86:0/3345542512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efe740b2040 con 0x561d4e4c5460 2024-03-22T04:12:18.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe84a00640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe68066500 0x7efe680689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe84a00640 1 -- 172.21.15.86:0/3345542512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7efe7411f8a0 con 0x561d4e4c5460 2024-03-22T04:12:18.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe87c00640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe68066500 0x7efe680689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561d4e4c14b0 0x561d4e4cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x561d4e4bf070 con 0x561d4e4c14b0 2024-03-22T04:12:18.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe88600640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561d4e4c14b0 0x561d4e4cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe87c00640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe68066500 0x7efe680689c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe78000c10 tx=0x7efe78001800 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe88600640 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561d4e4c14b0 0x561d4e4cb010 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe88600640 1 -- 172.21.15.86:0/3345542512 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7efe8000dc30 con 0x561d4e4c14b0 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561d4e4c14b0 msgr2=0x561d4e4cb010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561d4e4c14b0 0x561d4e4cb010 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe68066500 msgr2=0x7efe680689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efe68066500 0x7efe680689c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe78000c10 tx=0x7efe78001800 comp rx=0 tx=0).stop 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5460 msgr2=0x561d4e4beb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 --2- 172.21.15.86:0/3345542512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4e4c5460 0x561d4e4beb30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efe7400a500 tx=0x7efe740180c0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe88600640 1 -- 172.21.15.86:0/3345542512 reap_dead start 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 shutdown_connections 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 >> 172.21.15.86:0/3345542512 conn(0x561d4e41cec0 msgr2=0x561d4e3f3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 shutdown_connections 2024-03-22T04:12:18.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.406+0000 7efe8aa55080 1 -- 172.21.15.86:0/3345542512 wait complete. 2024-03-22T04:12:18.413 INFO:teuthology.orchestra.run.smithi086.stdout:mytag 2024-03-22T04:12:18.414 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-22T04:12:18.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 Processor -- start 2024-03-22T04:12:18.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- start start 2024-03-22T04:12:18.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818a8bcd0 0x55b818aa2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30460 0x55b818aa1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818b30840 0x55b818b297f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b818a01620 con 0x55b818b30840 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b818a05670 con 0x55b818b30460 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b818b321c0 con 0x55b818a8bcd0 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818b30840 0x55b818b297f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818b30840 0x55b818b297f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53502/0 (socket says 172.21.15.86:53502) 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 -- 172.21.15.86:0/1712160938 learned_addr learned my addr 172.21.15.86:0/1712160938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2210800640 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818a8bcd0 0x55b818aa2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f220fe00640 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30460 0x55b818aa1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 -- 172.21.15.86:0/1712160938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818a8bcd0 msgr2=0x55b818aa2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818a8bcd0 0x55b818aa2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 -- 172.21.15.86:0/1712160938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30460 msgr2=0x55b818aa1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30460 0x55b818aa1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 -- 172.21.15.86:0/1712160938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b818b2fb40 con 0x55b818b30840 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2211200640 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818b30840 0x55b818b297f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2208001650 tx=0x7f2208001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3fd86296330a80 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f220ea00640 1 -- 172.21.15.86:0/1712160938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080062f0 con 0x55b818b30840 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f220ea00640 1 -- 172.21.15.86:0/1712160938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208003660 con 0x55b818b30840 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f220ea00640 1 -- 172.21.15.86:0/1712160938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208003a50 con 0x55b818b30840 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- 172.21.15.86:0/1712160938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818b30840 msgr2=0x55b818b297f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818b30840 0x55b818b297f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2208001650 tx=0x7f2208001b10 comp rx=0 tx=0).stop 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- 172.21.15.86:0/1712160938 shutdown_connections 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30460 0x55b818aa1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818b30840 0x55b818b297f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 --2- 172.21.15.86:0/1712160938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818a8bcd0 0x55b818aa2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- 172.21.15.86:0/1712160938 >> 172.21.15.86:0/1712160938 conn(0x55b818a87ec0 msgr2=0x55b818a8ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- 172.21.15.86:0/1712160938 shutdown_connections 2024-03-22T04:12:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 -- 172.21.15.86:0/1712160938 wait complete. 2024-03-22T04:12:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.458+0000 7f2213526080 1 Processor -- start 2024-03-22T04:12:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 -- start start 2024-03-22T04:12:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818a8bcd0 0x55b818b0f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818b30460 0x55b818b10e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30840 0x55b818a5e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b818a9d260 con 0x55b818a8bcd0 2024-03-22T04:12:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b818aa6cb0 con 0x55b818b30840 2024-03-22T04:12:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b818b0ff20 con 0x55b818b30460 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818a8bcd0 0x55b818b0f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818a8bcd0 0x55b818b0f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53512/0 (socket says 172.21.15.86:53512) 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 -- 172.21.15.86:0/1350264284 learned_addr learned my addr 172.21.15.86:0/1350264284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f220fe00640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818b30460 0x55b818b10e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2211200640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30840 0x55b818a5e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 -- 172.21.15.86:0/1350264284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818b30460 msgr2=0x55b818b10e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b818b30460 0x55b818b10e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 -- 172.21.15.86:0/1350264284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30840 msgr2=0x55b818a5e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30840 0x55b818a5e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 -- 172.21.15.86:0/1350264284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b818b101c0 con 0x55b818a8bcd0 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2211200640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b818b30840 0x55b818a5e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2210800640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818a8bcd0 0x55b818b0f730 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f220400efd0 tx=0x7f2204002f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f220d600640 1 -- 172.21.15.86:0/1350264284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220400a040 con 0x55b818a8bcd0 2024-03-22T04:12:18.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f220d600640 1 -- 172.21.15.86:0/1350264284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204001c60 con 0x55b818a8bcd0 2024-03-22T04:12:18.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b818b10460 con 0x55b818a8bcd0 2024-03-22T04:12:18.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f220d600640 1 -- 172.21.15.86:0/1350264284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204002250 con 0x55b818a8bcd0 2024-03-22T04:12:18.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b818b109a0 con 0x55b818a8bcd0 2024-03-22T04:12:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.462+0000 7f220d600640 1 -- 172.21.15.86:0/1350264284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f22040023f0 con 0x55b818a8bcd0 2024-03-22T04:12:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f220d600640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0066530 0x7f21f00689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f220d600640 1 -- 172.21.15.86:0/1350264284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f2204084ff0 con 0x55b818a8bcd0 2024-03-22T04:12:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f220fe00640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0066530 0x7f21f00689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55b818b34b70 0x55b818b399e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x55b818b39f20 con 0x55b818b34b70 2024-03-22T04:12:18.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2211200640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55b818b34b70 0x55b818b399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f220fe00640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0066530 0x7f21f00689f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0ac000 tx=0x7f21fc0a5000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2211200640 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55b818b34b70 0x55b818b399e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2211200640 1 -- 172.21.15.86:0/1350264284 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f220800f030 con 0x55b818b34b70 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55b818b34b70 msgr2=0x55b818b399e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55b818b34b70 0x55b818b399e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0066530 msgr2=0x7f21f00689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0066530 0x7f21f00689f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0ac000 tx=0x7f21fc0a5000 comp rx=0 tx=0).stop 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818a8bcd0 msgr2=0x55b818b0f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 --2- 172.21.15.86:0/1350264284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b818a8bcd0 0x55b818b0f730 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f220400efd0 tx=0x7f2204002f90 comp rx=0 tx=0).stop 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2211200640 1 -- 172.21.15.86:0/1350264284 reap_dead start 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 shutdown_connections 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 >> 172.21.15.86:0/1350264284 conn(0x55b818a87ec0 msgr2=0x55b818a8ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 shutdown_connections 2024-03-22T04:12:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.466+0000 7f2213526080 1 -- 172.21.15.86:0/1350264284 wait complete. 2024-03-22T04:12:18.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:12:18.473 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:12:18.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:12:18.569 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:12:18.656 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:12:18.656 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:12:18.656 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:12:18.656 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:18.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:12:18.665 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:12:18.665 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:12:18.665 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:18.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:12:18.674 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:12:18.674 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:12:18.674 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:18.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:12:18.713 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:12:18.713 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:12:18.713 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:18.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:12:18.725 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:18.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.886+0000 7ffa94c00640 1 Processor -- start 2024-03-22T04:12:18.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.886+0000 7ffa94c00640 1 -- start start 2024-03-22T04:12:18.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa94c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa9006c270 0x7ffa90162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa94c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa90162540 0x7ffa9015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa94c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa900b2bd0 con 0x7ffa90147e10 2024-03-22T04:12:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa94c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffa90162990 con 0x7ffa9006c270 2024-03-22T04:12:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa94c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ffa900b2610 con 0x7ffa90162540 2024-03-22T04:12:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa85400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa9006c270 0x7ffa90162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa8de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ffa90147e10 0x7ffa900b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34672/0 (socket says 172.21.15.86:34672) 2024-03-22T04:12:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa8de00640 1 -- 172.21.15.86:0/1965979165 learned_addr learned my addr 172.21.15.86:0/1965979165 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa8e800640 1 --2- 172.21.15.86:0/1965979165 >> v2:172.21.15.196:3300/0 conn(0x7ffa90162540 0x7ffa9015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa85400640 1 -- 172.21.15.86:0/1965979165 >> v2:172.21.15.196:3300/0 conn(0x7ffa90162540 msgr2=0x7ffa9015fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:18.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa85400640 1 --2- 172.21.15.86:0/1965979165 >> v2:172.21.15.196:3300/0 conn(0x7ffa90162540 0x7ffa9015fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa85400640 1 -- 172.21.15.86:0/1965979165 >> v1:172.21.15.86:6789/0 conn(0x7ffa90147e10 legacy=0x7ffa900b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa85400640 1 -- 172.21.15.86:0/1965979165 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa900b28f0 con 0x7ffa9006c270 2024-03-22T04:12:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa85400640 1 --2- 172.21.15.86:0/1965979165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9006c270 0x7ffa90162000 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ffa90072c20 tx=0x7ffa7801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce45950b3b7aa4c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.890+0000 7ffa8d400640 1 -- 172.21.15.86:0/1965979165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780b1070 con 0x7ffa9006c270 2024-03-22T04:12:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa8d400640 1 -- 172.21.15.86:0/1965979165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa780ac5f0 con 0x7ffa9006c270 2024-03-22T04:12:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa8d400640 1 -- 172.21.15.86:0/1965979165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780ac9c0 con 0x7ffa9006c270 2024-03-22T04:12:18.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- 172.21.15.86:0/1965979165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9006c270 msgr2=0x7ffa90162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1965979165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9006c270 0x7ffa90162000 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ffa90072c20 tx=0x7ffa7801c480 comp rx=0 tx=0).stop 2024-03-22T04:12:18.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- 172.21.15.86:0/1965979165 shutdown_connections 2024-03-22T04:12:18.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1965979165 >> v2:172.21.15.196:3300/0 conn(0x7ffa90162540 0x7ffa9015fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1965979165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9006c270 0x7ffa90162000 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- 172.21.15.86:0/1965979165 >> 172.21.15.86:0/1965979165 conn(0x7ffa900b6620 msgr2=0x7ffa900b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:18.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- 172.21.15.86:0/1965979165 shutdown_connections 2024-03-22T04:12:18.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- 172.21.15.86:0/1965979165 wait complete. 2024-03-22T04:12:18.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 Processor -- start 2024-03-22T04:12:18.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- start start 2024-03-22T04:12:18.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9006c270 0x7ffa900cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa90147e10 0x7ffa9013d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa90162540 0x7ffa9015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90160c00 con 0x7ffa90147e10 2024-03-22T04:12:18.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90160a80 con 0x7ffa9006c270 2024-03-22T04:12:18.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa94c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ffa90160d80 con 0x7ffa90162540 2024-03-22T04:12:18.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa90147e10 0x7ffa9013d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa90147e10 0x7ffa9013d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53540/0 (socket says 172.21.15.86:53540) 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa85400640 1 -- 172.21.15.86:0/1809242465 learned_addr learned my addr 172.21.15.86:0/1809242465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa8de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9006c270 0x7ffa900cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.894+0000 7ffa8e800640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa90162540 0x7ffa9015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa85400640 1 -- 172.21.15.86:0/1809242465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa90162540 msgr2=0x7ffa9015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa85400640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa90162540 0x7ffa9015c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa85400640 1 -- 172.21.15.86:0/1809242465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9006c270 msgr2=0x7ffa900cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa85400640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9006c270 0x7ffa900cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa85400640 1 -- 172.21.15.86:0/1809242465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9013d7c0 con 0x7ffa90147e10 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa85400640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa90147e10 0x7ffa9013d040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffa7801ab30 tx=0x7ffa78019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa86a00640 1 -- 172.21.15.86:0/1809242465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780b1040 con 0x7ffa90147e10 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa86a00640 1 -- 172.21.15.86:0/1809242465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa780bb070 con 0x7ffa90147e10 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa86a00640 1 -- 172.21.15.86:0/1809242465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78006e20 con 0x7ffa90147e10 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9015cc30 con 0x7ffa90147e10 2024-03-22T04:12:18.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9015d1f0 con 0x7ffa90147e10 2024-03-22T04:12:18.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9015ca20 con 0x7ffa90147e10 2024-03-22T04:12:18.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa86a00640 1 -- 172.21.15.86:0/1809242465 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ffa780b5650 con 0x7ffa90147e10 2024-03-22T04:12:18.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa86a00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffa6c067b20 0x7ffa6c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:18.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa86a00640 1 -- 172.21.15.86:0/1809242465 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7ffa7811fa90 con 0x7ffa90147e10 2024-03-22T04:12:18.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.898+0000 7ffa8de00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffa6c067b20 0x7ffa6c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:18.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.902+0000 7ffa8de00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffa6c067b20 0x7ffa6c069fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000f70 tx=0x7ffa80007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:18.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:18.906+0000 7ffa86a00640 1 -- 172.21.15.86:0/1809242465 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ffa780a9070 con 0x7ffa90147e10 2024-03-22T04:12:19.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.090+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa90068650 con 0x7ffa90147e10 2024-03-22T04:12:19.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.090+0000 7ffa86a00640 1 -- 172.21.15.86:0/1809242465 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7ffa78016070 con 0x7ffa90147e10 2024-03-22T04:12:19.093 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:19.093 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:11:51.996418+0000","modified":"2024-03-22T04:11:55.030984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74930":{"gid":74930,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1426902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1426902111},{"type":"v1","addr":"172.21.15.196:6835","nonce":1426902111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74930,"qdb_cluster":[74930]},"id":2}]} 2024-03-22T04:12:19.094 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-03-22T04:12:19.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffa6c067b20 msgr2=0x7ffa6c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffa6c067b20 0x7ffa6c069fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000f70 tx=0x7ffa80007e90 comp rx=0 tx=0).stop 2024-03-22T04:12:19.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa90147e10 msgr2=0x7ffa9013d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa90147e10 0x7ffa9013d040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffa7801ab30 tx=0x7ffa78019220 comp rx=0 tx=0).stop 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 shutdown_connections 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffa6c067b20 0x7ffa6c069fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa90162540 0x7ffa9015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa90147e10 0x7ffa9013d040 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 --2- 172.21.15.86:0/1809242465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9006c270 0x7ffa900cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 >> 172.21.15.86:0/1809242465 conn(0x7ffa900b6620 msgr2=0x7ffa9006ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 shutdown_connections 2024-03-22T04:12:19.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.094+0000 7ffa94c00640 1 -- 172.21.15.86:0/1809242465 wait complete. 2024-03-22T04:12:19.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:19.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.246+0000 7f8afee00640 1 Processor -- start 2024-03-22T04:12:19.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8afee00640 1 -- start start 2024-03-22T04:12:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8afee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8af8161880 0x7f8af80b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8afee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af806e350 con 0x7f8af8161c60 2024-03-22T04:12:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8afee00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af806e630 con 0x7f8af806de10 2024-03-22T04:12:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8afee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8af815ee80 con 0x7f8af8161880 2024-03-22T04:12:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8af7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8af8161880 0x7f8af80b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8af7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8af8161c60 0x7f8af806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34676/0 (socket says 172.21.15.86:34676) 2024-03-22T04:12:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8af7400640 1 -- 172.21.15.86:0/2136028181 learned_addr learned my addr 172.21.15.86:0/2136028181 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8af7e00640 1 -- 172.21.15.86:0/2136028181 >> v1:172.21.15.196:6789/0 conn(0x7f8af806de10 legacy=0x7f8af815fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8af7e00640 1 -- 172.21.15.86:0/2136028181 >> v1:172.21.15.86:6789/0 conn(0x7f8af8161c60 legacy=0x7f8af806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8af7e00640 1 -- 172.21.15.86:0/2136028181 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af806e910 con 0x7f8af8161880 2024-03-22T04:12:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.250+0000 7f8af6a00640 1 -- 172.21.15.86:0/2136028181 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881772206 0 0) 0x7f8af806e350 con 0x7f8af8161c60 2024-03-22T04:12:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8af7e00640 1 --2- 172.21.15.86:0/2136028181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8161880 0x7f8af80b30f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00097c0 tx=0x7f8ae001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9f2d2da621f642 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8af6a00640 1 -- 172.21.15.86:0/2136028181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00b0070 con 0x7f8af8161880 2024-03-22T04:12:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8af6a00640 1 -- 172.21.15.86:0/2136028181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae00ab600 con 0x7f8af8161880 2024-03-22T04:12:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8af6a00640 1 -- 172.21.15.86:0/2136028181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00ab9d0 con 0x7f8af8161880 2024-03-22T04:12:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 -- 172.21.15.86:0/2136028181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8161880 msgr2=0x7f8af80b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 --2- 172.21.15.86:0/2136028181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8161880 0x7f8af80b30f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00097c0 tx=0x7f8ae001c3e0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 -- 172.21.15.86:0/2136028181 shutdown_connections 2024-03-22T04:12:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 --2- 172.21.15.86:0/2136028181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af8161880 0x7f8af80b30f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 -- 172.21.15.86:0/2136028181 >> 172.21.15.86:0/2136028181 conn(0x7f8af80b1df0 msgr2=0x7f8af80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 -- 172.21.15.86:0/2136028181 shutdown_connections 2024-03-22T04:12:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 -- 172.21.15.86:0/2136028181 wait complete. 2024-03-22T04:12:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 Processor -- start 2024-03-22T04:12:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.254+0000 7f8afee00640 1 -- start start 2024-03-22T04:12:19.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af806de10 0x7f8af81576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8161880 0x7f8af8155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af8161c60 0x7f8af815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8161060 con 0x7f8af806de10 2024-03-22T04:12:19.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8160ee0 con 0x7f8af8161880 2024-03-22T04:12:19.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8af816c480 con 0x7f8af8161c60 2024-03-22T04:12:19.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af806de10 0x7f8af81576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af806de10 0x7f8af81576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53564/0 (socket says 172.21.15.86:53564) 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 -- 172.21.15.86:0/3410410030 learned_addr learned my addr 172.21.15.86:0/3410410030 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7400640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8161880 0x7f8af8155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afce00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af8161c60 0x7f8af815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 -- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af8161c60 msgr2=0x7f8af815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af8161c60 0x7f8af815ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 -- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8161880 msgr2=0x7f8af8155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8161880 0x7f8af8155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 -- 172.21.15.86:0/3410410030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af8073620 con 0x7f8af806de10 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afce00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af8161c60 0x7f8af815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8af7e00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af806de10 0x7f8af81576a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00097c0 tx=0x7f8ae001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8aefe00640 1 -- 172.21.15.86:0/3410410030 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00bc070 con 0x7f8af806de10 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8aefe00640 1 -- 172.21.15.86:0/3410410030 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae00b1040 con 0x7f8af806de10 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af8158240 con 0x7f8af806de10 2024-03-22T04:12:19.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8aefe00640 1 -- 172.21.15.86:0/3410410030 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00b0050 con 0x7f8af806de10 2024-03-22T04:12:19.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af8152700 con 0x7f8af806de10 2024-03-22T04:12:19.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af8068650 con 0x7f8af806de10 2024-03-22T04:12:19.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.258+0000 7f8aefe00640 1 -- 172.21.15.86:0/3410410030 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8ae00bf070 con 0x7f8af806de10 2024-03-22T04:12:19.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.262+0000 7f8aefe00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad4067b20 0x7f8ad4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.262+0000 7f8aefe00640 1 -- 172.21.15.86:0/3410410030 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f8ae011fc20 con 0x7f8af806de10 2024-03-22T04:12:19.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.262+0000 7f8af7400640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad4067b20 0x7f8ad4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.262+0000 7f8af7400640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad4067b20 0x7f8ad4069fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8af80cfc50 tx=0x7f8ae8007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.266+0000 7f8aefe00640 1 -- 172.21.15.86:0/3410410030 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8ae0016090 con 0x7f8af806de10 2024-03-22T04:12:19.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.418+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8af80d0570 con 0x7f8af806de10 2024-03-22T04:12:19.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8aefe00640 1 -- 172.21.15.86:0/3410410030 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17085 (secure 0 0 0) 0x7f8ae00efdf0 con 0x7f8af806de10 2024-03-22T04:12:19.423 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:19.423 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:12:15.901464+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:11:49.644732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-22T04:11:50.961986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.23","mappings":[{"from":3,"to":2}]},{"pgid":"6.28","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad4067b20 msgr2=0x7f8ad4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad4067b20 0x7f8ad4069fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8af80cfc50 tx=0x7f8ae8007240 comp rx=0 tx=0).stop 2024-03-22T04:12:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af806de10 msgr2=0x7f8af81576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af806de10 0x7f8af81576a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00097c0 tx=0x7f8ae001ac20 comp rx=0 tx=0).stop 2024-03-22T04:12:19.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 shutdown_connections 2024-03-22T04:12:19.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ad4067b20 0x7f8ad4069fe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8af8161880 0x7f8af8155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8af8161c60 0x7f8af815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 --2- 172.21.15.86:0/3410410030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af806de10 0x7f8af81576a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 >> 172.21.15.86:0/3410410030 conn(0x7f8af80b1df0 msgr2=0x7f8af80717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:19.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 shutdown_connections 2024-03-22T04:12:19.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.422+0000 7f8afee00640 1 -- 172.21.15.86:0/3410410030 wait complete. 2024-03-22T04:12:19.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:11:49.644732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:11:50.961986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:12:19.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:12:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11afa00640 1 Processor -- start 2024-03-22T04:12:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11afa00640 1 -- start start 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11afa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80ce000 0x7f11a81608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11afa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11a80b6380 0x7f11a80b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11afa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a8160df0 con 0x7f11a80ce3e0 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11afa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f11a80b6d30 con 0x7f11a80b6380 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11afa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f11a815d3e0 con 0x7f11a80ce000 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11ada00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11a80b6380 0x7f11a80b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11ada00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11a80b6380 0x7f11a80b67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53566/0 (socket says 172.21.15.86:53566) 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11ada00640 1 -- 172.21.15.86:0/1581048451 learned_addr learned my addr 172.21.15.86:0/1581048451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11ad000640 1 --2- 172.21.15.86:0/1581048451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80ce000 0x7f11a81608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11ada00640 1 -- 172.21.15.86:0/1581048451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80ce000 msgr2=0x7f11a81608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11ada00640 1 --2- 172.21.15.86:0/1581048451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80ce000 0x7f11a81608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11ada00640 1 -- 172.21.15.86:0/1581048451 >> v1:172.21.15.196:6789/0 conn(0x7f11a80ce3e0 legacy=0x7f11a815c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11ada00640 1 -- 172.21.15.86:0/1581048451 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a815d6c0 con 0x7f11a80b6380 2024-03-22T04:12:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.582+0000 7f11a7400640 1 -- 172.21.15.86:0/1581048451 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692219255 0 0) 0x7f11a8160df0 con 0x7f11a80ce3e0 2024-03-22T04:12:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11ada00640 1 --2- 172.21.15.86:0/1581048451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80b6380 0x7f11a80b67f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009e50 tx=0x7f11a001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79c76b5c3fc8e34 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11a7400640 1 -- 172.21.15.86:0/1581048451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a00a5660 con 0x7f11a80b6380 2024-03-22T04:12:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11a7400640 1 -- 172.21.15.86:0/1581048451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a00a5800 con 0x7f11a80b6380 2024-03-22T04:12:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11a7400640 1 -- 172.21.15.86:0/1581048451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a00a5bd0 con 0x7f11a80b6380 2024-03-22T04:12:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 -- 172.21.15.86:0/1581048451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80b6380 msgr2=0x7f11a80b67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 --2- 172.21.15.86:0/1581048451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80b6380 0x7f11a80b67f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009e50 tx=0x7f11a001c9a0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 -- 172.21.15.86:0/1581048451 shutdown_connections 2024-03-22T04:12:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 --2- 172.21.15.86:0/1581048451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80b6380 0x7f11a80b67f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 --2- 172.21.15.86:0/1581048451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80ce000 0x7f11a81608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 -- 172.21.15.86:0/1581048451 >> 172.21.15.86:0/1581048451 conn(0x7f11a80d0fe0 msgr2=0x7f11a80d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 -- 172.21.15.86:0/1581048451 shutdown_connections 2024-03-22T04:12:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 -- 172.21.15.86:0/1581048451 wait complete. 2024-03-22T04:12:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.586+0000 7f11afa00640 1 Processor -- start 2024-03-22T04:12:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 -- start start 2024-03-22T04:12:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80b6380 0x7f11a8143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80ce000 0x7f11a815e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a80ce3e0 0x7f11a815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80d0400 con 0x7f11a80ce000 2024-03-22T04:12:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80d0280 con 0x7f11a80ce3e0 2024-03-22T04:12:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f11a80d0580 con 0x7f11a80b6380 2024-03-22T04:12:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80ce000 0x7f11a815e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ad000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80b6380 0x7f11a8143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ada00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a80ce3e0 0x7f11a815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80ce000 0x7f11a815e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53578/0 (socket says 172.21.15.86:53578) 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ada00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a80ce3e0 0x7f11a815e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:47840/0 (socket says 172.21.15.86:47840) 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11a7e00640 1 -- 172.21.15.86:0/2055135919 learned_addr learned my addr 172.21.15.86:0/2055135919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ad000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80b6380 0x7f11a8143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:50602/0 (socket says 172.21.15.86:50602) 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ada00640 1 -- 172.21.15.86:0/2055135919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80b6380 msgr2=0x7f11a8143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ada00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80b6380 0x7f11a8143b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ada00640 1 -- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80ce000 msgr2=0x7f11a815e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ada00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80ce000 0x7f11a815e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ada00640 1 -- 172.21.15.86:0/2055135919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a8144090 con 0x7f11a80ce3e0 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11a7e00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80ce000 0x7f11a815e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11ada00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a80ce3e0 0x7f11a815e6c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f11a00ab000 tx=0x7f11a0019740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11a4c00640 1 -- 172.21.15.86:0/2055135919 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a00bc030 con 0x7f11a80ce3e0 2024-03-22T04:12:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11a4c00640 1 -- 172.21.15.86:0/2055135919 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a00b8470 con 0x7f11a80ce3e0 2024-03-22T04:12:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a815b2c0 con 0x7f11a80ce3e0 2024-03-22T04:12:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11a4c00640 1 -- 172.21.15.86:0/2055135919 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0009510 con 0x7f11a80ce3e0 2024-03-22T04:12:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a815b880 con 0x7f11a80ce3e0 2024-03-22T04:12:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.590+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a8074470 con 0x7f11a80ce3e0 2024-03-22T04:12:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.594+0000 7f11a4c00640 1 -- 172.21.15.86:0/2055135919 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f11a00bf070 con 0x7f11a80ce3e0 2024-03-22T04:12:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.594+0000 7f11a4c00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1188067b50 0x7f118806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.594+0000 7f11a4c00640 1 -- 172.21.15.86:0/2055135919 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f11a0120c40 con 0x7f11a80ce3e0 2024-03-22T04:12:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.594+0000 7f11ad000640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1188067b50 0x7f118806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.594+0000 7f11ad000640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1188067b50 0x7f118806a010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f119c009ac0 tx=0x7f119c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.598+0000 7f11a4c00640 1 -- 172.21.15.86:0/2055135919 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f11a0016620 con 0x7f11a80ce3e0 2024-03-22T04:12:19.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f11a806e650 con 0x7f11a80ce3e0 2024-03-22T04:12:19.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11a4c00640 1 -- 172.21.15.86:0/2055135919 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7f11a00f0e10 con 0x7f11a80ce3e0 2024-03-22T04:12:19.755 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1188067b50 msgr2=0x7f118806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1188067b50 0x7f118806a010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f119c009ac0 tx=0x7f119c009190 comp rx=0 tx=0).stop 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a80ce3e0 msgr2=0x7f11a815e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a80ce3e0 0x7f11a815e6c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f11a00ab000 tx=0x7f11a0019740 comp rx=0 tx=0).stop 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 shutdown_connections 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1188067b50 0x7f118806a010 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a80ce3e0 0x7f11a815e6c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80ce000 0x7f11a815e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 --2- 172.21.15.86:0/2055135919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a80b6380 0x7f11a8143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 >> 172.21.15.86:0/2055135919 conn(0x7f11a80d0fe0 msgr2=0x7f11a80b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:19.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 shutdown_connections 2024-03-22T04:12:19.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.754+0000 7f11afa00640 1 -- 172.21.15.86:0/2055135919 wait complete. 2024-03-22T04:12:19.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:12:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba67c00640 1 Processor -- start 2024-03-22T04:12:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba67c00640 1 -- start start 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba67c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fba6006eb40 0x7fba600b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba67c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba60072460 0x7fba60072890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba67c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba60073190 con 0x7fba60144b30 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba67c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fba600b64f0 con 0x7fba60072460 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba67c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fba600b5ea0 con 0x7fba6006eb40 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba65c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba60072460 0x7fba60072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba65c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba60072460 0x7fba60072890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53586/0 (socket says 172.21.15.86:53586) 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba65c00640 1 -- 172.21.15.86:0/1174408425 learned_addr learned my addr 172.21.15.86:0/1174408425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba57e00640 1 --2- 172.21.15.86:0/1174408425 >> v2:172.21.15.196:3300/0 conn(0x7fba6006eb40 0x7fba600b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba65c00640 1 -- 172.21.15.86:0/1174408425 >> v2:172.21.15.196:3300/0 conn(0x7fba6006eb40 msgr2=0x7fba600b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba65c00640 1 --2- 172.21.15.86:0/1174408425 >> v2:172.21.15.196:3300/0 conn(0x7fba6006eb40 0x7fba600b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba65c00640 1 -- 172.21.15.86:0/1174408425 >> v1:172.21.15.196:6789/0 conn(0x7fba60144b30 legacy=0x7fba6006e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.914+0000 7fba65c00640 1 -- 172.21.15.86:0/1174408425 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba600b6180 con 0x7fba60072460 2024-03-22T04:12:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba65c00640 1 --2- 172.21.15.86:0/1174408425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba60072890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fba6006d720 tx=0x7fba5801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a07f200d723361d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba5fe00640 1 -- 172.21.15.86:0/1174408425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba580a56c0 con 0x7fba60072460 2024-03-22T04:12:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba5fe00640 1 -- 172.21.15.86:0/1174408425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba580a5860 con 0x7fba60072460 2024-03-22T04:12:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba5fe00640 1 -- 172.21.15.86:0/1174408425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba580a5c30 con 0x7fba60072460 2024-03-22T04:12:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba67c00640 1 -- 172.21.15.86:0/1174408425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 msgr2=0x7fba60072890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba67c00640 1 --2- 172.21.15.86:0/1174408425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba60072890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fba6006d720 tx=0x7fba5801c9f0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba67c00640 1 -- 172.21.15.86:0/1174408425 shutdown_connections 2024-03-22T04:12:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba67c00640 1 --2- 172.21.15.86:0/1174408425 >> v2:172.21.15.196:3300/0 conn(0x7fba6006eb40 0x7fba600b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba67c00640 1 --2- 172.21.15.86:0/1174408425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba60072890 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba67c00640 1 -- 172.21.15.86:0/1174408425 >> 172.21.15.86:0/1174408425 conn(0x7fba600d0fe0 msgr2=0x7fba600d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba67c00640 1 -- 172.21.15.86:0/1174408425 shutdown_connections 2024-03-22T04:12:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.918+0000 7fba67c00640 1 -- 172.21.15.86:0/1174408425 wait complete. 2024-03-22T04:12:19.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 Processor -- start 2024-03-22T04:12:19.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 -- start start 2024-03-22T04:12:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6006eb40 0x7fba60159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba6015e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba60144b30 0x7fba6015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba600b4c10 con 0x7fba60072460 2024-03-22T04:12:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba600b4a90 con 0x7fba6006eb40 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fba600b4d90 con 0x7fba60144b30 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba6015e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba6015e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53592/0 (socket says 172.21.15.86:53592) 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 -- 172.21.15.86:0/3302437571 learned_addr learned my addr 172.21.15.86:0/3302437571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba65200640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6006eb40 0x7fba60159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba65c00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba60144b30 0x7fba6015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 -- 172.21.15.86:0/3302437571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba60144b30 msgr2=0x7fba6015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba60144b30 0x7fba6015ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 -- 172.21.15.86:0/3302437571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6006eb40 msgr2=0x7fba60159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6006eb40 0x7fba60159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 -- 172.21.15.86:0/3302437571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba60156a80 con 0x7fba60072460 2024-03-22T04:12:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba57e00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba6015e830 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fba4800e8a0 tx=0x7fba4800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba5d600640 1 -- 172.21.15.86:0/3302437571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48017070 con 0x7fba60072460 2024-03-22T04:12:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba5d600640 1 -- 172.21.15.86:0/3302437571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba4800c940 con 0x7fba60072460 2024-03-22T04:12:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba5d600640 1 -- 172.21.15.86:0/3302437571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4800ccf0 con 0x7fba60072460 2024-03-22T04:12:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba60156d60 con 0x7fba60072460 2024-03-22T04:12:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba6012eda0 con 0x7fba60072460 2024-03-22T04:12:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.922+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba60074470 con 0x7fba60072460 2024-03-22T04:12:19.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.926+0000 7fba5d600640 1 -- 172.21.15.86:0/3302437571 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fba4801a070 con 0x7fba60072460 2024-03-22T04:12:19.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.926+0000 7fba5d600640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba3c067b20 0x7fba3c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:19.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.926+0000 7fba5d600640 1 -- 172.21.15.86:0/3302437571 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7fba480862d0 con 0x7fba60072460 2024-03-22T04:12:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.926+0000 7fba65200640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba3c067b20 0x7fba3c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.926+0000 7fba65200640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba3c067b20 0x7fba3c069fe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba60144550 tx=0x7fba50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:19.930+0000 7fba5d600640 1 -- 172.21.15.86:0/3302437571 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fba480564a0 con 0x7fba60072460 2024-03-22T04:12:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.082+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fba6006efc0 con 0x7fba60072460 2024-03-22T04:12:20.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.082+0000 7fba5d600640 1 -- 172.21.15.86:0/3302437571 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) ==== 104+0+11 (secure 0 0 0) 0x7fba4805a540 con 0x7fba60072460 2024-03-22T04:12:20.086 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba3c067b20 msgr2=0x7fba3c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba3c067b20 0x7fba3c069fe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba60144550 tx=0x7fba50009040 comp rx=0 tx=0).stop 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 msgr2=0x7fba6015e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba6015e830 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fba4800e8a0 tx=0x7fba4800eca0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 shutdown_connections 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba3c067b20 0x7fba3c069fe0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba60144b30 0x7fba6015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba60072460 0x7fba6015e830 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 --2- 172.21.15.86:0/3302437571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6006eb40 0x7fba60159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 >> 172.21.15.86:0/3302437571 conn(0x7fba600d0fe0 msgr2=0x7fba600b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 shutdown_connections 2024-03-22T04:12:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.086+0000 7fba67c00640 1 -- 172.21.15.86:0/3302437571 wait complete. 2024-03-22T04:12:20.107 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:12:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.242+0000 7f3fe9000640 1 Processor -- start 2024-03-22T04:12:20.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.242+0000 7f3fe9000640 1 -- start start 2024-03-22T04:12:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe9000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe40b2360 0x7f3fe40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe9000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe415d710 con 0x7f3fe40b3430 2024-03-22T04:12:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe9000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe415fe40 con 0x7f3fe415d0b0 2024-03-22T04:12:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe9000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe40b3890 con 0x7f3fe40b2360 2024-03-22T04:12:20.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fd9400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3fe40b3430 0x7f3fe415c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34684/0 (socket says 172.21.15.86:34684) 2024-03-22T04:12:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fd9400640 1 -- 172.21.15.86:0/1758380752 learned_addr learned my addr 172.21.15.86:0/1758380752 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:20.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe1e00640 1 --2- 172.21.15.86:0/1758380752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe40b2360 0x7f3fe40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe1400640 1 -- 172.21.15.86:0/1758380752 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186971777 0 0) 0x7f3fe415d710 con 0x7f3fe40b3430 2024-03-22T04:12:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe1400640 1 -- 172.21.15.86:0/1758380752 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fc8003530 con 0x7f3fe40b3430 2024-03-22T04:12:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe1400640 1 -- 172.21.15.86:0/1758380752 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 611888940 0 0) 0x7f3fe415fe40 con 0x7f3fe415d0b0 2024-03-22T04:12:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe1400640 1 -- 172.21.15.86:0/1758380752 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fe415d710 con 0x7f3fe415d0b0 2024-03-22T04:12:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe1e00640 1 -- 172.21.15.86:0/1758380752 >> v1:172.21.15.196:6789/0 conn(0x7f3fe415d0b0 legacy=0x7f3fe40b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe1e00640 1 -- 172.21.15.86:0/1758380752 >> v1:172.21.15.86:6789/0 conn(0x7f3fe40b3430 legacy=0x7f3fe415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.246+0000 7f3fe1e00640 1 -- 172.21.15.86:0/1758380752 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe4160120 con 0x7f3fe40b2360 2024-03-22T04:12:20.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe1e00640 1 --2- 172.21.15.86:0/1758380752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe40b2360 0x7f3fe40b2ef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40097f0 tx=0x7f3fd401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0f4726d3d39dedb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe1400640 1 -- 172.21.15.86:0/1758380752 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40b0070 con 0x7f3fe40b2360 2024-03-22T04:12:20.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe1400640 1 -- 172.21.15.86:0/1758380752 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd40ab600 con 0x7f3fe40b2360 2024-03-22T04:12:20.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe1400640 1 -- 172.21.15.86:0/1758380752 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40ab9b0 con 0x7f3fe40b2360 2024-03-22T04:12:20.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 -- 172.21.15.86:0/1758380752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe40b2360 msgr2=0x7f3fe40b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 --2- 172.21.15.86:0/1758380752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe40b2360 0x7f3fe40b2ef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40097f0 tx=0x7f3fd401c4e0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 -- 172.21.15.86:0/1758380752 shutdown_connections 2024-03-22T04:12:20.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 --2- 172.21.15.86:0/1758380752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe40b2360 0x7f3fe40b2ef0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 -- 172.21.15.86:0/1758380752 >> 172.21.15.86:0/1758380752 conn(0x7f3fe40cc940 msgr2=0x7f3fe40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:20.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 -- 172.21.15.86:0/1758380752 shutdown_connections 2024-03-22T04:12:20.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 -- 172.21.15.86:0/1758380752 wait complete. 2024-03-22T04:12:20.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 Processor -- start 2024-03-22T04:12:20.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.250+0000 7f3fe9000640 1 -- start start 2024-03-22T04:12:20.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fe40b2360 0x7f3fe4153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe40b3430 0x7f3fe4149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe415d0b0 0x7f3fe414f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe416a370 con 0x7f3fe40b3430 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe416a1f0 con 0x7f3fe40b2360 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe416a4f0 con 0x7f3fe415d0b0 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe40b3430 0x7f3fe4149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe40b3430 0x7f3fe4149120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53600/0 (socket says 172.21.15.86:53600) 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 -- 172.21.15.86:0/3653584878 learned_addr learned my addr 172.21.15.86:0/3653584878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe1e00640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fe40b2360 0x7f3fe4153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe2800640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe415d0b0 0x7f3fe414f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 -- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe415d0b0 msgr2=0x7f3fe414f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe415d0b0 0x7f3fe414f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 -- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fe40b2360 msgr2=0x7f3fe4153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fe40b2360 0x7f3fe4153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 -- 172.21.15.86:0/3653584878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe4149660 con 0x7f3fe40b3430 2024-03-22T04:12:20.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe2800640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe415d0b0 0x7f3fe414f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:20.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fd9400640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe40b3430 0x7f3fe4149120 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c4d0 tx=0x7f3fcc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fdaa00640 1 -- 172.21.15.86:0/3653584878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc017070 con 0x7f3fe40b3430 2024-03-22T04:12:20.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fdaa00640 1 -- 172.21.15.86:0/3653584878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fcc005cf0 con 0x7f3fe40b3430 2024-03-22T04:12:20.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fdaa00640 1 -- 172.21.15.86:0/3653584878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc0072b0 con 0x7f3fe40b3430 2024-03-22T04:12:20.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe4150b10 con 0x7f3fe40b3430 2024-03-22T04:12:20.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe4150270 con 0x7f3fe40b3430 2024-03-22T04:12:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe4074470 con 0x7f3fe40b3430 2024-03-22T04:12:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.254+0000 7f3fdaa00640 1 -- 172.21.15.86:0/3653584878 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3fcc01a070 con 0x7f3fe40b3430 2024-03-22T04:12:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.258+0000 7f3fdaa00640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3fc0067b20 0x7f3fc0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.258+0000 7f3fdaa00640 1 -- 172.21.15.86:0/3653584878 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f3fcc085430 con 0x7f3fe40b3430 2024-03-22T04:12:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.258+0000 7f3fe1e00640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3fc0067b20 0x7f3fc0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.258+0000 7f3fe1e00640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3fc0067b20 0x7f3fc0069fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40ad000 tx=0x7f3fd40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.262+0000 7f3fdaa00640 1 -- 172.21.15.86:0/3653584878 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3fcc055600 con 0x7f3fe40b3430 2024-03-22T04:12:20.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.414+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3fe406e9f0 con 0x7f3fe40b3430 2024-03-22T04:12:20.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.414+0000 7f3fdaa00640 1 -- 172.21.15.86:0/3653584878 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) ==== 100+0+11 (secure 0 0 0) 0x7f3fcc0596a0 con 0x7f3fe40b3430 2024-03-22T04:12:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:12:20.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3fc0067b20 msgr2=0x7f3fc0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3fc0067b20 0x7f3fc0069fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40ad000 tx=0x7f3fd40a6000 comp rx=0 tx=0).stop 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe40b3430 msgr2=0x7f3fe4149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe40b3430 0x7f3fe4149120 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c4d0 tx=0x7f3fcc00c990 comp rx=0 tx=0).stop 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 shutdown_connections 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3fc0067b20 0x7f3fc0069fe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fe415d0b0 0x7f3fe414f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe40b3430 0x7f3fe4149120 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 --2- 172.21.15.86:0/3653584878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fe40b2360 0x7f3fe4153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 >> 172.21.15.86:0/3653584878 conn(0x7f3fe40cc940 msgr2=0x7f3fe40ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 shutdown_connections 2024-03-22T04:12:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.418+0000 7f3fe9000640 1 -- 172.21.15.86:0/3653584878 wait complete. 2024-03-22T04:12:20.438 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.570+0000 7fb33e400640 1 Processor -- start 2024-03-22T04:12:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.570+0000 7fb33e400640 1 -- start start 2024-03-22T04:12:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb33e400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb33815b3f0 0x7fb3380729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb33e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb338071920 con 0x7fb3380cc3e0 2024-03-22T04:12:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb33e400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb338071c00 con 0x7fb3380ae540 2024-03-22T04:12:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb33e400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb3380716d0 con 0x7fb33815b3f0 2024-03-22T04:12:20.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb336a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb3380cc3e0 0x7fb3380ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34698/0 (socket says 172.21.15.86:34698) 2024-03-22T04:12:20.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb336a00640 1 -- 172.21.15.86:0/2792604764 learned_addr learned my addr 172.21.15.86:0/2792604764 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:20.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb337400640 1 --2- 172.21.15.86:0/2792604764 >> v2:172.21.15.196:3300/0 conn(0x7fb33815b3f0 0x7fb3380729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb336000640 1 -- 172.21.15.86:0/2792604764 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078600632 0 0) 0x7fb338071920 con 0x7fb3380cc3e0 2024-03-22T04:12:20.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb336000640 1 -- 172.21.15.86:0/2792604764 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb31c003530 con 0x7fb3380cc3e0 2024-03-22T04:12:20.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb337400640 1 -- 172.21.15.86:0/2792604764 >> v1:172.21.15.196:6789/0 conn(0x7fb3380ae540 legacy=0x7fb3380b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb337400640 1 -- 172.21.15.86:0/2792604764 >> v1:172.21.15.86:6789/0 conn(0x7fb3380cc3e0 legacy=0x7fb3380ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.574+0000 7fb337400640 1 -- 172.21.15.86:0/2792604764 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb338071ee0 con 0x7fb33815b3f0 2024-03-22T04:12:20.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb337400640 1 --2- 172.21.15.86:0/2792604764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 0x7fb3380729c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb328009d90 tx=0x7fb32801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4411bd9ec1667b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb336000640 1 -- 172.21.15.86:0/2792604764 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3280b2070 con 0x7fb33815b3f0 2024-03-22T04:12:20.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb336000640 1 -- 172.21.15.86:0/2792604764 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3280a5830 con 0x7fb33815b3f0 2024-03-22T04:12:20.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb336000640 1 -- 172.21.15.86:0/2792604764 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3280a5c00 con 0x7fb33815b3f0 2024-03-22T04:12:20.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 -- 172.21.15.86:0/2792604764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 msgr2=0x7fb3380729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 --2- 172.21.15.86:0/2792604764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 0x7fb3380729c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb328009d90 tx=0x7fb32801c970 comp rx=0 tx=0).stop 2024-03-22T04:12:20.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 -- 172.21.15.86:0/2792604764 shutdown_connections 2024-03-22T04:12:20.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 --2- 172.21.15.86:0/2792604764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 0x7fb3380729c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 -- 172.21.15.86:0/2792604764 >> 172.21.15.86:0/2792604764 conn(0x7fb3380b1df0 msgr2=0x7fb3380b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:20.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 -- 172.21.15.86:0/2792604764 shutdown_connections 2024-03-22T04:12:20.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 -- 172.21.15.86:0/2792604764 wait complete. 2024-03-22T04:12:20.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 Processor -- start 2024-03-22T04:12:20.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.578+0000 7fb33e400640 1 -- start start 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3380ae540 0x7fb3381461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3380cc3e0 0x7fb33814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 0x7fb33814cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3380b30f0 con 0x7fb3380cc3e0 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb3380b4c20 con 0x7fb33815b3f0 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb3380b3270 con 0x7fb3380ae540 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3380cc3e0 0x7fb33814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb337400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3380ae540 0x7fb3381461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3380cc3e0 0x7fb33814c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53608/0 (socket says 172.21.15.86:53608) 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 -- 172.21.15.86:0/3373636981 learned_addr learned my addr 172.21.15.86:0/3373636981 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:20.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb337e00640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 0x7fb33814cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 -- 172.21.15.86:0/3373636981 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3380ae540 msgr2=0x7fb3381461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3380ae540 0x7fb3381461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 -- 172.21.15.86:0/3373636981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 msgr2=0x7fb33814cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 0x7fb33814cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 -- 172.21.15.86:0/3373636981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33814ad30 con 0x7fb3380cc3e0 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb337e00640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 0x7fb33814cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb336a00640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3380cc3e0 0x7fb33814c830 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c4d0 tx=0x7fb32400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb32f400640 1 -- 172.21.15.86:0/3373636981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324017070 con 0x7fb3380cc3e0 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb32f400640 1 -- 172.21.15.86:0/3373636981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb324005cf0 con 0x7fb3380cc3e0 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb32f400640 1 -- 172.21.15.86:0/3373636981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324007270 con 0x7fb3380cc3e0 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33814b010 con 0x7fb3380cc3e0 2024-03-22T04:12:20.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33814ff50 con 0x7fb3380cc3e0 2024-03-22T04:12:20.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33814b530 con 0x7fb3380cc3e0 2024-03-22T04:12:20.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.582+0000 7fb32f400640 1 -- 172.21.15.86:0/3373636981 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb32401a070 con 0x7fb3380cc3e0 2024-03-22T04:12:20.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.586+0000 7fb32f400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb318067b20 0x7fb318069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.586+0000 7fb32f400640 1 -- 172.21.15.86:0/3373636981 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7fb324085410 con 0x7fb3380cc3e0 2024-03-22T04:12:20.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.586+0000 7fb337400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb318067b20 0x7fb318069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.586+0000 7fb337400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb318067b20 0x7fb318069fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb3280af000 tx=0x7fb3280a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.590+0000 7fb32f400640 1 -- 172.21.15.86:0/3373636981 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb33814b530 con 0x7fb3380cc3e0 2024-03-22T04:12:20.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb338150150 con 0x7fb3380cc3e0 2024-03-22T04:12:20.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb32f400640 1 -- 172.21.15.86:0/3373636981 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7fb338150150 con 0x7fb3380cc3e0 2024-03-22T04:12:20.780 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:20.780 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:11:51.996418+0000","modified":"2024-03-22T04:11:55.030984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74930":{"gid":74930,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1426902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1426902111},{"type":"v1","addr":"172.21.15.196:6835","nonce":1426902111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74930,"qdb_cluster":[74930]},"id":2}]} 2024-03-22T04:12:20.780 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-03-22T04:12:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb318067b20 msgr2=0x7fb318069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb318067b20 0x7fb318069fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb3280af000 tx=0x7fb3280a7070 comp rx=0 tx=0).stop 2024-03-22T04:12:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3380cc3e0 msgr2=0x7fb33814c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3380cc3e0 0x7fb33814c830 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c4d0 tx=0x7fb32400c990 comp rx=0 tx=0).stop 2024-03-22T04:12:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 shutdown_connections 2024-03-22T04:12:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb318067b20 0x7fb318069fe0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33815b3f0 0x7fb33814cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3380cc3e0 0x7fb33814c830 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 --2- 172.21.15.86:0/3373636981 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3380ae540 0x7fb3381461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 >> 172.21.15.86:0/3373636981 conn(0x7fb3380b1df0 msgr2=0x7fb33815e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 shutdown_connections 2024-03-22T04:12:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.778+0000 7fb33e400640 1 -- 172.21.15.86:0/3373636981 wait complete. 2024-03-22T04:12:20.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd581600640 1 Processor -- start 2024-03-22T04:12:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd581600640 1 -- start start 2024-03-22T04:12:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd581600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd57c0d0de0 0x7fd57c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd581600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd57c0d19a0 0x7fd57c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd581600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57c0d11c0 con 0x7fd57c15fed0 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd581600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd57c0cece0 con 0x7fd57c0d19a0 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd581600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd57c0cefc0 con 0x7fd57c0d0de0 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd572000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd57c0d19a0 0x7fd57c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd572000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd57c0d19a0 0x7fd57c0cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53610/0 (socket says 172.21.15.86:53610) 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd57b400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd57c15fed0 0x7fd57c1602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:56166/0 (socket says 172.21.15.86:56166) 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd572000640 1 -- 172.21.15.86:0/679766183 learned_addr learned my addr 172.21.15.86:0/679766183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd57aa00640 1 --2- 172.21.15.86:0/679766183 >> v2:172.21.15.196:3300/0 conn(0x7fd57c0d0de0 0x7fd57c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd572000640 1 -- 172.21.15.86:0/679766183 >> v2:172.21.15.196:3300/0 conn(0x7fd57c0d0de0 msgr2=0x7fd57c0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd572000640 1 --2- 172.21.15.86:0/679766183 >> v2:172.21.15.196:3300/0 conn(0x7fd57c0d0de0 0x7fd57c0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd572000640 1 -- 172.21.15.86:0/679766183 >> v1:172.21.15.196:6789/0 conn(0x7fd57c15fed0 legacy=0x7fd57c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.934+0000 7fd572000640 1 -- 172.21.15.86:0/679766183 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c0cf2a0 con 0x7fd57c0d19a0 2024-03-22T04:12:20.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd572000640 1 --2- 172.21.15.86:0/679766183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c0d19a0 0x7fd57c0cfe70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd57c06f4c0 tx=0x7fd56401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9143984f947fa69 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd57a000640 1 -- 172.21.15.86:0/679766183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640ab610 con 0x7fd57c0d19a0 2024-03-22T04:12:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd57a000640 1 -- 172.21.15.86:0/679766183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5640b0070 con 0x7fd57c0d19a0 2024-03-22T04:12:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd57a000640 1 -- 172.21.15.86:0/679766183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640ab9d0 con 0x7fd57c0d19a0 2024-03-22T04:12:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd581600640 1 -- 172.21.15.86:0/679766183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c0d19a0 msgr2=0x7fd57c0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd581600640 1 --2- 172.21.15.86:0/679766183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c0d19a0 0x7fd57c0cfe70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd57c06f4c0 tx=0x7fd56401c520 comp rx=0 tx=0).stop 2024-03-22T04:12:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd581600640 1 -- 172.21.15.86:0/679766183 shutdown_connections 2024-03-22T04:12:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd581600640 1 --2- 172.21.15.86:0/679766183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c0d19a0 0x7fd57c0cfe70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd581600640 1 --2- 172.21.15.86:0/679766183 >> v2:172.21.15.196:3300/0 conn(0x7fd57c0d0de0 0x7fd57c0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd581600640 1 -- 172.21.15.86:0/679766183 >> 172.21.15.86:0/679766183 conn(0x7fd57c06f910 msgr2=0x7fd57c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd581600640 1 -- 172.21.15.86:0/679766183 shutdown_connections 2024-03-22T04:12:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.938+0000 7fd581600640 1 -- 172.21.15.86:0/679766183 wait complete. 2024-03-22T04:12:20.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 Processor -- start 2024-03-22T04:12:20.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 -- start start 2024-03-22T04:12:20.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd57c0d0de0 0x7fd57c14b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd57c0d19a0 0x7fd57c157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c15fed0 0x7fd57c157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c0d1f70 con 0x7fd57c15fed0 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c0d1df0 con 0x7fd57c0d19a0 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c071e30 con 0x7fd57c0d0de0 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c15fed0 0x7fd57c157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57aa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd57c0d0de0 0x7fd57c14b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c15fed0 0x7fd57c157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53616/0 (socket says 172.21.15.86:53616) 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 -- 172.21.15.86:0/1981171830 learned_addr learned my addr 172.21.15.86:0/1981171830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd572000640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd57c0d19a0 0x7fd57c157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 -- 172.21.15.86:0/1981171830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd57c0d0de0 msgr2=0x7fd57c14b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd57c0d0de0 0x7fd57c14b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 -- 172.21.15.86:0/1981171830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd57c0d19a0 msgr2=0x7fd57c157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd57c0d19a0 0x7fd57c157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 -- 172.21.15.86:0/1981171830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c14b550 con 0x7fd57c15fed0 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd57b400640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c15fed0 0x7fd57c157aa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c710 tx=0x7fd57400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd573400640 1 -- 172.21.15.86:0/1981171830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574017070 con 0x7fd57c15fed0 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd573400640 1 -- 172.21.15.86:0/1981171830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57400ce20 con 0x7fd57c15fed0 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c153bd0 con 0x7fd57c15fed0 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd573400640 1 -- 172.21.15.86:0/1981171830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57400a860 con 0x7fd57c15fed0 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.942+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c0cab50 con 0x7fd57c15fed0 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.946+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57c068650 con 0x7fd57c15fed0 2024-03-22T04:12:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.946+0000 7fd573400640 1 -- 172.21.15.86:0/1981171830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd57401a070 con 0x7fd57c15fed0 2024-03-22T04:12:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.946+0000 7fd573400640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd558067b20 0x7fd558069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.946+0000 7fd573400640 1 -- 172.21.15.86:0/1981171830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7fd574085700 con 0x7fd57c15fed0 2024-03-22T04:12:20.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.946+0000 7fd57aa00640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd558067b20 0x7fd558069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:20.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.946+0000 7fd57aa00640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd558067b20 0x7fd558069fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009ac0 tx=0x7fd56c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:20.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:20.950+0000 7fd573400640 1 -- 172.21.15.86:0/1981171830 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd574055840 con 0x7fd57c15fed0 2024-03-22T04:12:21.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.106+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd57c0ce920 con 0x7fd57c15fed0 2024-03-22T04:12:21.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.106+0000 7fd573400640 1 -- 172.21.15.86:0/1981171830 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17085 (secure 0 0 0) 0x7fd5740598e0 con 0x7fd57c15fed0 2024-03-22T04:12:21.109 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:21.109 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:12:15.901464+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:11:49.644732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-22T04:11:50.961986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.23","mappings":[{"from":3,"to":2}]},{"pgid":"6.28","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:21.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd558067b20 msgr2=0x7fd558069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd558067b20 0x7fd558069fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009ac0 tx=0x7fd56c009190 comp rx=0 tx=0).stop 2024-03-22T04:12:21.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c15fed0 msgr2=0x7fd57c157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c15fed0 0x7fd57c157aa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c710 tx=0x7fd57400cbd0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 shutdown_connections 2024-03-22T04:12:21.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd558067b20 0x7fd558069fe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd57c0d19a0 0x7fd57c157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c15fed0 0x7fd57c157aa0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 --2- 172.21.15.86:0/1981171830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd57c0d0de0 0x7fd57c14b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 >> 172.21.15.86:0/1981171830 conn(0x7fd57c06f910 msgr2=0x7fd57c15e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:21.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 shutdown_connections 2024-03-22T04:12:21.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.110+0000 7fd581600640 1 -- 172.21.15.86:0/1981171830 wait complete. 2024-03-22T04:12:21.129 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:12:21.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.262+0000 7f28baa00640 1 Processor -- start 2024-03-22T04:12:21.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.262+0000 7f28baa00640 1 -- start start 2024-03-22T04:12:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28baa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28b415b3f0 0x7f28b40729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28baa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f28b40cc3e0 0x7f28b4072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28baa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b4071920 con 0x7f28b40b30f0 2024-03-22T04:12:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28baa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f28b40716d0 con 0x7f28b415b3f0 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28baa00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f28b406d990 con 0x7f28b40cc3e0 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28b415b3f0 0x7f28b40729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28b415b3f0 0x7f28b40729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53620/0 (socket says 172.21.15.86:53620) 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3e00640 1 -- 172.21.15.86:0/3050598408 learned_addr learned my addr 172.21.15.86:0/3050598408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3400640 1 --2- 172.21.15.86:0/3050598408 >> v2:172.21.15.196:3300/0 conn(0x7f28b40cc3e0 0x7f28b4072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3e00640 1 -- 172.21.15.86:0/3050598408 >> v2:172.21.15.196:3300/0 conn(0x7f28b40cc3e0 msgr2=0x7f28b4072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3e00640 1 --2- 172.21.15.86:0/3050598408 >> v2:172.21.15.196:3300/0 conn(0x7f28b40cc3e0 0x7f28b4072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3e00640 1 -- 172.21.15.86:0/3050598408 >> v1:172.21.15.196:6789/0 conn(0x7f28b40b30f0 legacy=0x7f28b40ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3e00640 1 -- 172.21.15.86:0/3050598408 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b4071c00 con 0x7f28b415b3f0 2024-03-22T04:12:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3400640 1 --2- 172.21.15.86:0/3050598408 >> v2:172.21.15.196:3300/0 conn(0x7f28b40cc3e0 0x7f28b4072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.266+0000 7f28b3e00640 1 --2- 172.21.15.86:0/3050598408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b415b3f0 0x7f28b40729c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f28b4070ed0 tx=0x7f28a401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bd811255d420da server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28b2a00640 1 -- 172.21.15.86:0/3050598408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40b2070 con 0x7f28b415b3f0 2024-03-22T04:12:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28b2a00640 1 -- 172.21.15.86:0/3050598408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a40ad680 con 0x7f28b415b3f0 2024-03-22T04:12:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28b2a00640 1 -- 172.21.15.86:0/3050598408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40ada50 con 0x7f28b415b3f0 2024-03-22T04:12:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- 172.21.15.86:0/3050598408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b415b3f0 msgr2=0x7f28b40729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 --2- 172.21.15.86:0/3050598408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b415b3f0 0x7f28b40729c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f28b4070ed0 tx=0x7f28a401c490 comp rx=0 tx=0).stop 2024-03-22T04:12:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- 172.21.15.86:0/3050598408 shutdown_connections 2024-03-22T04:12:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 --2- 172.21.15.86:0/3050598408 >> v2:172.21.15.196:3300/0 conn(0x7f28b40cc3e0 0x7f28b4072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 --2- 172.21.15.86:0/3050598408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b415b3f0 0x7f28b40729c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- 172.21.15.86:0/3050598408 >> 172.21.15.86:0/3050598408 conn(0x7f28b40b1df0 msgr2=0x7f28b40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- 172.21.15.86:0/3050598408 shutdown_connections 2024-03-22T04:12:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- 172.21.15.86:0/3050598408 wait complete. 2024-03-22T04:12:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 Processor -- start 2024-03-22T04:12:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- start start 2024-03-22T04:12:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b40b30f0 0x7f28b414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28b40cc3e0 0x7f28b4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b415b3f0 0x7f28b414c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28b406e0c0 con 0x7f28b40b30f0 2024-03-22T04:12:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28b406df40 con 0x7f28b415b3f0 2024-03-22T04:12:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28baa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f28b406e240 con 0x7f28b40cc3e0 2024-03-22T04:12:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b40b30f0 0x7f28b414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b40b30f0 0x7f28b414fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53628/0 (socket says 172.21.15.86:53628) 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28b3e00640 1 -- 172.21.15.86:0/3597589534 learned_addr learned my addr 172.21.15.86:0/3597589534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28b3400640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28b40cc3e0 0x7f28b4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.270+0000 7f28b8a00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b415b3f0 0x7f28b414c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b3e00640 1 -- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28b40cc3e0 msgr2=0x7f28b4156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b3e00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28b40cc3e0 0x7f28b4156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b3e00640 1 -- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b415b3f0 msgr2=0x7f28b414c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b3e00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b415b3f0 0x7f28b414c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b3e00640 1 -- 172.21.15.86:0/3597589534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b4155b50 con 0x7f28b40b30f0 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b3400640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28b40cc3e0 0x7f28b4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b8a00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b415b3f0 0x7f28b414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b3e00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b40b30f0 0x7f28b414fd30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f28a40190a0 tx=0x7f28a401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28abe00640 1 -- 172.21.15.86:0/3597589534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40b2040 con 0x7f28b40b30f0 2024-03-22T04:12:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28abe00640 1 -- 172.21.15.86:0/3597589534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a40ada70 con 0x7f28b40b30f0 2024-03-22T04:12:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b4155df0 con 0x7f28b40b30f0 2024-03-22T04:12:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28abe00640 1 -- 172.21.15.86:0/3597589534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40ade20 con 0x7f28b40b30f0 2024-03-22T04:12:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41563b0 con 0x7f28b40b30f0 2024-03-22T04:12:21.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b4068650 con 0x7f28b40b30f0 2024-03-22T04:12:21.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28abe00640 1 -- 172.21.15.86:0/3597589534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f28a40bf030 con 0x7f28b40b30f0 2024-03-22T04:12:21.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28abe00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2894067b20 0x7f2894069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28abe00640 1 -- 172.21.15.86:0/3597589534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f28a411fa90 con 0x7f28b40b30f0 2024-03-22T04:12:21.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.274+0000 7f28b3400640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2894067b20 0x7f2894069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.278+0000 7f28b3400640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2894067b20 0x7f2894069fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28b4166130 tx=0x7f28a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:21.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.278+0000 7f28abe00640 1 -- 172.21.15.86:0/3597589534 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f28a40c6020 con 0x7f28b40b30f0 2024-03-22T04:12:21.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.466+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b40b40e0 con 0x7f28b40b30f0 2024-03-22T04:12:21.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.466+0000 7f28abe00640 1 -- 172.21.15.86:0/3597589534 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f28a4016110 con 0x7f28b40b30f0 2024-03-22T04:12:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:21.470 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:11:51.996418+0000","modified":"2024-03-22T04:11:55.030984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74930":{"gid":74930,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1426902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1426902111},{"type":"v1","addr":"172.21.15.196:6835","nonce":1426902111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74930,"qdb_cluster":[74930]},"id":2}]} 2024-03-22T04:12:21.471 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-03-22T04:12:21.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2894067b20 msgr2=0x7f2894069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2894067b20 0x7f2894069fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28b4166130 tx=0x7f28a0008040 comp rx=0 tx=0).stop 2024-03-22T04:12:21.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b40b30f0 msgr2=0x7f28b414fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b40b30f0 0x7f28b414fd30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f28a40190a0 tx=0x7f28a401acd0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 shutdown_connections 2024-03-22T04:12:21.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2894067b20 0x7f2894069fe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b415b3f0 0x7f28b414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28b40cc3e0 0x7f28b4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 --2- 172.21.15.86:0/3597589534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b40b30f0 0x7f28b414fd30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 >> 172.21.15.86:0/3597589534 conn(0x7f28b40b1df0 msgr2=0x7f28b4152e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 shutdown_connections 2024-03-22T04:12:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.470+0000 7f28baa00640 1 -- 172.21.15.86:0/3597589534 wait complete. 2024-03-22T04:12:21.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.622+0000 7f7bbce00640 1 Processor -- start 2024-03-22T04:12:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.622+0000 7f7bbce00640 1 -- start start 2024-03-22T04:12:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bbce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7bb815b3f0 0x7f7bb80729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bbce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bb80cc3e0 0x7f7bb8072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bbce00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb8071920 con 0x7f7bb80b30f0 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bbce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb80716d0 con 0x7f7bb80cc3e0 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bbce00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb806d990 con 0x7f7bb815b3f0 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bb80cc3e0 0x7f7bb8072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bb80cc3e0 0x7f7bb8072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53632/0 (socket says 172.21.15.86:53632) 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5400640 1 -- 172.21.15.86:0/3447244089 learned_addr learned my addr 172.21.15.86:0/3447244089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5e00640 1 --2- 172.21.15.86:0/3447244089 >> v2:172.21.15.196:3300/0 conn(0x7f7bb815b3f0 0x7f7bb80729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5400640 1 -- 172.21.15.86:0/3447244089 >> v2:172.21.15.196:3300/0 conn(0x7f7bb815b3f0 msgr2=0x7f7bb80729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5400640 1 --2- 172.21.15.86:0/3447244089 >> v2:172.21.15.196:3300/0 conn(0x7f7bb815b3f0 0x7f7bb80729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5400640 1 -- 172.21.15.86:0/3447244089 >> v1:172.21.15.196:6789/0 conn(0x7f7bb80b30f0 legacy=0x7f7bb80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5400640 1 -- 172.21.15.86:0/3447244089 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb8071c00 con 0x7f7bb80cc3e0 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5e00640 1 --2- 172.21.15.86:0/3447244089 >> v2:172.21.15.196:3300/0 conn(0x7f7bb815b3f0 0x7f7bb80729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb5400640 1 --2- 172.21.15.86:0/3447244089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cc3e0 0x7f7bb8072f00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7bb806c510 tx=0x7f7ba001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66d2a6298417049 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb4a00640 1 -- 172.21.15.86:0/3447244089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00b2070 con 0x7f7bb80cc3e0 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.626+0000 7f7bb4a00640 1 -- 172.21.15.86:0/3447244089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba00ad680 con 0x7f7bb80cc3e0 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bb4a00640 1 -- 172.21.15.86:0/3447244089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00ada50 con 0x7f7bb80cc3e0 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- 172.21.15.86:0/3447244089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cc3e0 msgr2=0x7f7bb8072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 --2- 172.21.15.86:0/3447244089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cc3e0 0x7f7bb8072f00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7bb806c510 tx=0x7f7ba001c490 comp rx=0 tx=0).stop 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- 172.21.15.86:0/3447244089 shutdown_connections 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 --2- 172.21.15.86:0/3447244089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cc3e0 0x7f7bb8072f00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 --2- 172.21.15.86:0/3447244089 >> v2:172.21.15.196:3300/0 conn(0x7f7bb815b3f0 0x7f7bb80729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- 172.21.15.86:0/3447244089 >> 172.21.15.86:0/3447244089 conn(0x7f7bb80b1df0 msgr2=0x7f7bb80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- 172.21.15.86:0/3447244089 shutdown_connections 2024-03-22T04:12:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- 172.21.15.86:0/3447244089 wait complete. 2024-03-22T04:12:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 Processor -- start 2024-03-22T04:12:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- start start 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bb80b30f0 0x7f7bb8146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bb80cc3e0 0x7f7bb81555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb815b3f0 0x7f7bb8143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb806e2d0 con 0x7f7bb815b3f0 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb806e150 con 0x7f7bb80cc3e0 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bbce00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb806e450 con 0x7f7bb80b30f0 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bb6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb815b3f0 0x7f7bb8143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bb5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bb80b30f0 0x7f7bb8146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bb6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb815b3f0 0x7f7bb8143d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53648/0 (socket says 172.21.15.86:53648) 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bb6800640 1 -- 172.21.15.86:0/1141060100 learned_addr learned my addr 172.21.15.86:0/1141060100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.630+0000 7f7bb5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bb80cc3e0 0x7f7bb81555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bb6800640 1 -- 172.21.15.86:0/1141060100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bb80b30f0 msgr2=0x7f7bb8146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bb6800640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bb80b30f0 0x7f7bb8146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bb6800640 1 -- 172.21.15.86:0/1141060100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bb80cc3e0 msgr2=0x7f7bb81555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bb6800640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bb80cc3e0 0x7f7bb81555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bb6800640 1 -- 172.21.15.86:0/1141060100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb8147100 con 0x7f7bb815b3f0 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bb6800640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb815b3f0 0x7f7bb8143d20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000c770 tx=0x7f7bb000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bae000640 1 -- 172.21.15.86:0/1141060100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0003b80 con 0x7f7bb815b3f0 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bae000640 1 -- 172.21.15.86:0/1141060100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb0003d20 con 0x7f7bb815b3f0 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bae000640 1 -- 172.21.15.86:0/1141060100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000aa90 con 0x7f7bb815b3f0 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb8154fe0 con 0x7f7bb815b3f0 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb814a090 con 0x7f7bb815b3f0 2024-03-22T04:12:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb8068650 con 0x7f7bb815b3f0 2024-03-22T04:12:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bae000640 1 -- 172.21.15.86:0/1141060100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7bb000ac30 con 0x7f7bb815b3f0 2024-03-22T04:12:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bae000640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7b94067b20 0x7f7b94069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bae000640 1 -- 172.21.15.86:0/1141060100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f7bb0085ed0 con 0x7f7bb815b3f0 2024-03-22T04:12:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.634+0000 7f7bb5e00640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7b94067b20 0x7f7b94069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.638+0000 7f7bb5e00640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7b94067b20 0x7f7b94069fe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7bb809a0b0 tx=0x7f7ba8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.638+0000 7f7bae000640 1 -- 172.21.15.86:0/1141060100 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7bb0056090 con 0x7f7bb815b3f0 2024-03-22T04:12:21.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.826+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bb8153f80 con 0x7f7bb815b3f0 2024-03-22T04:12:21.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bae000640 1 -- 172.21.15.86:0/1141060100 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f7bb005a130 con 0x7f7bb815b3f0 2024-03-22T04:12:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:21.831 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:11:51.996418+0000","modified":"2024-03-22T04:11:55.030984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":74930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_74930":{"gid":74930,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1426902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1426902111},{"type":"v1","addr":"172.21.15.196:6835","nonce":1426902111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":74930,"qdb_cluster":[74930]},"id":2}]} 2024-03-22T04:12:21.831 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-03-22T04:12:21.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7b94067b20 msgr2=0x7f7b94069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7b94067b20 0x7f7b94069fe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7bb809a0b0 tx=0x7f7ba8008040 comp rx=0 tx=0).stop 2024-03-22T04:12:21.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb815b3f0 msgr2=0x7f7bb8143d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb815b3f0 0x7f7bb8143d20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000c770 tx=0x7f7bb000cc30 comp rx=0 tx=0).stop 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 shutdown_connections 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7b94067b20 0x7f7b94069fe0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bb80cc3e0 0x7f7bb81555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb815b3f0 0x7f7bb8143d20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 --2- 172.21.15.86:0/1141060100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bb80b30f0 0x7f7bb8146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 >> 172.21.15.86:0/1141060100 conn(0x7f7bb80b1df0 msgr2=0x7f7bb815c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 shutdown_connections 2024-03-22T04:12:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.830+0000 7f7bbce00640 1 -- 172.21.15.86:0/1141060100 wait complete. 2024-03-22T04:12:21.852 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:21.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.982+0000 7f097c400640 1 Processor -- start 2024-03-22T04:12:21.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.982+0000 7f097c400640 1 -- start start 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f097c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f097406ef40 0x7f0974142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f097c400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f097415c7c0 con 0x7f09740ad330 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f097c400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f097406dea0 con 0x7f09740ad710 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f097c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f097406e180 con 0x7f097406ef40 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0979a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f09740ad330 0x7f097413fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34714/0 (socket says 172.21.15.86:34714) 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0979a00640 1 -- 172.21.15.86:0/1014541812 learned_addr learned my addr 172.21.15.86:0/1014541812 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0979000640 1 --2- 172.21.15.86:0/1014541812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f097406ef40 0x7f0974142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0973e00640 1 -- 172.21.15.86:0/1014541812 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1484443045 0 0) 0x7f097415c7c0 con 0x7f09740ad330 2024-03-22T04:12:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0973e00640 1 -- 172.21.15.86:0/1014541812 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f095c003530 con 0x7f09740ad330 2024-03-22T04:12:21.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0973e00640 1 -- 172.21.15.86:0/1014541812 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299710318 0 0) 0x7f097406dea0 con 0x7f09740ad710 2024-03-22T04:12:21.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0973e00640 1 -- 172.21.15.86:0/1014541812 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f097415c7c0 con 0x7f09740ad710 2024-03-22T04:12:21.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0979000640 1 -- 172.21.15.86:0/1014541812 >> v1:172.21.15.196:6789/0 conn(0x7f09740ad710 legacy=0x7f09741402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0979000640 1 -- 172.21.15.86:0/1014541812 >> v1:172.21.15.86:6789/0 conn(0x7f09740ad330 legacy=0x7f097413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.986+0000 7f0979000640 1 -- 172.21.15.86:0/1014541812 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097406e460 con 0x7f097406ef40 2024-03-22T04:12:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f0979000640 1 --2- 172.21.15.86:0/1014541812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f097406ef40 0x7f0974142690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f09600097f0 tx=0x7f096001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ee76398b6f44cb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f0973e00640 1 -- 172.21.15.86:0/1014541812 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600aa430 con 0x7f097406ef40 2024-03-22T04:12:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f0973e00640 1 -- 172.21.15.86:0/1014541812 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09600af070 con 0x7f097406ef40 2024-03-22T04:12:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f0973e00640 1 -- 172.21.15.86:0/1014541812 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600aa7f0 con 0x7f097406ef40 2024-03-22T04:12:21.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 -- 172.21.15.86:0/1014541812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f097406ef40 msgr2=0x7f0974142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 --2- 172.21.15.86:0/1014541812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f097406ef40 0x7f0974142690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f09600097f0 tx=0x7f096001c4b0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 -- 172.21.15.86:0/1014541812 shutdown_connections 2024-03-22T04:12:21.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 --2- 172.21.15.86:0/1014541812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f097406ef40 0x7f0974142690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 -- 172.21.15.86:0/1014541812 >> 172.21.15.86:0/1014541812 conn(0x7f09740b1df0 msgr2=0x7f09740b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:21.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 -- 172.21.15.86:0/1014541812 shutdown_connections 2024-03-22T04:12:21.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 -- 172.21.15.86:0/1014541812 wait complete. 2024-03-22T04:12:21.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 Processor -- start 2024-03-22T04:12:21.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.990+0000 7f097c400640 1 -- start start 2024-03-22T04:12:21.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097406ef40 0x7f09740964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09740ad330 0x7f0974155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09740ad710 0x7f09741535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f097406db10 con 0x7f09740ad330 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09741409d0 con 0x7f097406ef40 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f097416a780 con 0x7f09740ad710 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09740ad330 0x7f0974155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09740ad330 0x7f0974155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53662/0 (socket says 172.21.15.86:53662) 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 -- 172.21.15.86:0/3108172438 learned_addr learned my addr 172.21.15.86:0/3108172438 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979a00640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097406ef40 0x7f09740964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097a400640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09740ad710 0x7f09741535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 -- 172.21.15.86:0/3108172438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09740ad710 msgr2=0x7f09741535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09740ad710 0x7f09741535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 -- 172.21.15.86:0/3108172438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097406ef40 msgr2=0x7f09740964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097406ef40 0x7f09740964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:21.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 -- 172.21.15.86:0/3108172438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0974096c60 con 0x7f09740ad330 2024-03-22T04:12:21.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0979000640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09740ad330 0x7f0974155740 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f09600190c0 tx=0x7f096001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:21.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0971600640 1 -- 172.21.15.86:0/3108172438 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600be610 con 0x7f09740ad330 2024-03-22T04:12:21.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0971600640 1 -- 172.21.15.86:0/3108172438 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09600be7b0 con 0x7f09740ad330 2024-03-22T04:12:21.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0974096f40 con 0x7f09740ad330 2024-03-22T04:12:21.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0971600640 1 -- 172.21.15.86:0/3108172438 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600beba0 con 0x7f09740ad330 2024-03-22T04:12:21.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0974097500 con 0x7f09740ad330 2024-03-22T04:12:21.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0974068650 con 0x7f09740ad330 2024-03-22T04:12:21.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.994+0000 7f0971600640 1 -- 172.21.15.86:0/3108172438 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f09600bc020 con 0x7f09740ad330 2024-03-22T04:12:21.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.998+0000 7f0971600640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0954067b20 0x7f0954069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:21.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.998+0000 7f0971600640 1 -- 172.21.15.86:0/3108172438 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7f0960120470 con 0x7f09740ad330 2024-03-22T04:12:21.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.998+0000 7f0979a00640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0954067b20 0x7f0954069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:21.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:21.998+0000 7f0979a00640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0954067b20 0x7f0954069fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0968007fd0 tx=0x7f0968007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.002+0000 7f0971600640 1 -- 172.21.15.86:0/3108172438 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f09600160d0 con 0x7f09740ad330 2024-03-22T04:12:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.162+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09740b2510 con 0x7f09740ad330 2024-03-22T04:12:22.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.162+0000 7f0971600640 1 -- 172.21.15.86:0/3108172438 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17085 (secure 0 0 0) 0x7f09600f0640 con 0x7f09740ad330 2024-03-22T04:12:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:22.164 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:12:15.901464+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:11:49.644732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-22T04:11:50.961986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.23","mappings":[{"from":3,"to":2}]},{"pgid":"6.28","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:22.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.162+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0954067b20 msgr2=0x7f0954069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.162+0000 7f097c400640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0954067b20 0x7f0954069fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0968007fd0 tx=0x7f0968007d80 comp rx=0 tx=0).stop 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09740ad330 msgr2=0x7f0974155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09740ad330 0x7f0974155740 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f09600190c0 tx=0x7f096001acf0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 shutdown_connections 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0954067b20 0x7f0954069fe0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09740ad710 0x7f09741535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09740ad330 0x7f0974155740 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 --2- 172.21.15.86:0/3108172438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097406ef40 0x7f09740964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 >> 172.21.15.86:0/3108172438 conn(0x7f09740b1df0 msgr2=0x7f09740afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 shutdown_connections 2024-03-22T04:12:22.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.166+0000 7f097c400640 1 -- 172.21.15.86:0/3108172438 wait complete. 2024-03-22T04:12:22.186 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:12:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.318+0000 7ff86e800640 1 Processor -- start 2024-03-22T04:12:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff86e800640 1 -- start start 2024-03-22T04:12:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff86e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad330 0x7ff86815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff86e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff86815c7c0 con 0x7ff8680ad710 2024-03-22T04:12:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff86e800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff86806dea0 con 0x7ff86806ef40 2024-03-22T04:12:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff86e800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff86806e180 con 0x7ff8680ad330 2024-03-22T04:12:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff867e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff8680ad710 0x7ff8681402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34722/0 (socket says 172.21.15.86:34722) 2024-03-22T04:12:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff867e00640 1 -- 172.21.15.86:0/1830388966 learned_addr learned my addr 172.21.15.86:0/1830388966 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff867400640 1 --2- 172.21.15.86:0/1830388966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad330 0x7ff86815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff866000640 1 -- 172.21.15.86:0/1830388966 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1048856383 0 0) 0x7ff86815c7c0 con 0x7ff8680ad710 2024-03-22T04:12:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff866000640 1 -- 172.21.15.86:0/1830388966 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff84c003530 con 0x7ff8680ad710 2024-03-22T04:12:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff866000640 1 -- 172.21.15.86:0/1830388966 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 317005763 0 0) 0x7ff86806dea0 con 0x7ff86806ef40 2024-03-22T04:12:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff866000640 1 -- 172.21.15.86:0/1830388966 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff86815c7c0 con 0x7ff86806ef40 2024-03-22T04:12:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff867400640 1 -- 172.21.15.86:0/1830388966 >> v1:172.21.15.196:6789/0 conn(0x7ff86806ef40 legacy=0x7ff86813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff867400640 1 -- 172.21.15.86:0/1830388966 >> v1:172.21.15.86:6789/0 conn(0x7ff8680ad710 legacy=0x7ff8681402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.322+0000 7ff867400640 1 -- 172.21.15.86:0/1830388966 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86806e460 con 0x7ff8680ad330 2024-03-22T04:12:22.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff867400640 1 --2- 172.21.15.86:0/1830388966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad330 0x7ff86815d4a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097f0 tx=0x7ff85801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbc396adfca7f9cc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff866000640 1 -- 172.21.15.86:0/1830388966 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580b1070 con 0x7ff8680ad330 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff866000640 1 -- 172.21.15.86:0/1830388966 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8580ac620 con 0x7ff8680ad330 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff866000640 1 -- 172.21.15.86:0/1830388966 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580ac9f0 con 0x7ff8680ad330 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 -- 172.21.15.86:0/1830388966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad330 msgr2=0x7ff86815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 --2- 172.21.15.86:0/1830388966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad330 0x7ff86815d4a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097f0 tx=0x7ff85801c4b0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 -- 172.21.15.86:0/1830388966 shutdown_connections 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 --2- 172.21.15.86:0/1830388966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad330 0x7ff86815d4a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 -- 172.21.15.86:0/1830388966 >> 172.21.15.86:0/1830388966 conn(0x7ff8680b1df0 msgr2=0x7ff8680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 -- 172.21.15.86:0/1830388966 shutdown_connections 2024-03-22T04:12:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 -- 172.21.15.86:0/1830388966 wait complete. 2024-03-22T04:12:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 Processor -- start 2024-03-22T04:12:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.326+0000 7ff86e800640 1 -- start start 2024-03-22T04:12:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86806ef40 0x7ff868072910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8680ad330 0x7ff86814e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad710 0x7ff86814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff86816a780 con 0x7ff86806ef40 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff86806db10 con 0x7ff8680ad330 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff86816a900 con 0x7ff8680ad710 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86806ef40 0x7ff868072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86806ef40 0x7ff868072910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53666/0 (socket says 172.21.15.86:53666) 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 -- 172.21.15.86:0/2330030876 learned_addr learned my addr 172.21.15.86:0/2330030876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff866a00640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8680ad330 0x7ff86814e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867e00640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad710 0x7ff86814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 -- 172.21.15.86:0/2330030876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad710 msgr2=0x7ff86814ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad710 0x7ff86814ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 -- 172.21.15.86:0/2330030876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8680ad330 msgr2=0x7ff86814e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8680ad330 0x7ff86814e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 -- 172.21.15.86:0/2330030876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff868073220 con 0x7ff86806ef40 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff867400640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86806ef40 0x7ff868072910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097f0 tx=0x7ff85801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff85f400640 1 -- 172.21.15.86:0/2330030876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580b1050 con 0x7ff86806ef40 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff85f400640 1 -- 172.21.15.86:0/2330030876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8580bd070 con 0x7ff86806ef40 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff868073500 con 0x7ff86806ef40 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff85f400640 1 -- 172.21.15.86:0/2330030876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858006e20 con 0x7ff86806ef40 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff868099660 con 0x7ff86806ef40 2024-03-22T04:12:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff868068650 con 0x7ff86806ef40 2024-03-22T04:12:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.330+0000 7ff85f400640 1 -- 172.21.15.86:0/2330030876 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff8580b86b0 con 0x7ff86806ef40 2024-03-22T04:12:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.334+0000 7ff85f400640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff848067b20 0x7ff848069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.334+0000 7ff85f400640 1 -- 172.21.15.86:0/2330030876 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6188+0+0 (secure 0 0 0) 0x7ff858120640 con 0x7ff86806ef40 2024-03-22T04:12:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.334+0000 7ff866a00640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff848067b20 0x7ff848069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.334+0000 7ff866a00640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff848067b20 0x7ff848069fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff8680b5410 tx=0x7ff854009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:22.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.338+0000 7ff85f400640 1 -- 172.21.15.86:0/2330030876 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff8580160d0 con 0x7ff86806ef40 2024-03-22T04:12:22.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.522+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff8680736d0 con 0x7ff86806ef40 2024-03-22T04:12:22.525 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:12:22.522+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:12:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff85f400640 1 -- 172.21.15.86:0/2330030876 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7ff8580f0780 con 0x7ff86806ef40 2024-03-22T04:12:22.527 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:12:22.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff848067b20 msgr2=0x7ff848069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff848067b20 0x7ff848069fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff8680b5410 tx=0x7ff854009040 comp rx=0 tx=0).stop 2024-03-22T04:12:22.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86806ef40 msgr2=0x7ff868072910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86806ef40 0x7ff868072910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097f0 tx=0x7ff85801acf0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 shutdown_connections 2024-03-22T04:12:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff848067b20 0x7ff848069fe0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8680ad330 0x7ff86814e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8680ad710 0x7ff86814ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 --2- 172.21.15.86:0/2330030876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86806ef40 0x7ff868072910 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.526+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 >> 172.21.15.86:0/2330030876 conn(0x7ff8680b1df0 msgr2=0x7ff8680cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.530+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 shutdown_connections 2024-03-22T04:12:22.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.530+0000 7ff86e800640 1 -- 172.21.15.86:0/2330030876 wait complete. 2024-03-22T04:12:22.542 INFO:tasks.ceph.mds.b.smithi196.stderr: -6720> 2024-03-22T04:11:56.594+0000 7f9eac000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:12:22.549 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T04:12:22.591 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:12:22.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.686+0000 7f796c400640 1 Processor -- start 2024-03-22T04:12:22.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.686+0000 7f796c400640 1 -- start start 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f796415c390 0x7f7964160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796c400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79640c6930 0x7f79640c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796c400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796415c770 con 0x7f79640af910 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796c400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f79640c72c0 con 0x7f79640c6930 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7964160d90 con 0x7f796415c390 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79640c6930 0x7f79640c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f7969000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f79640af910 0x7f796406df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34726/0 (socket says 172.21.15.86:34726) 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f7969000640 1 -- 172.21.15.86:0/4010482940 learned_addr learned my addr 172.21.15.86:0/4010482940 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:22.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f7969a00640 1 --2- 172.21.15.86:0/4010482940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f796415c390 0x7f7964160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796a400640 1 -- 172.21.15.86:0/4010482940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f796415c390 msgr2=0x7f7964160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796a400640 1 --2- 172.21.15.86:0/4010482940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f796415c390 0x7f7964160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796a400640 1 -- 172.21.15.86:0/4010482940 >> v1:172.21.15.86:6789/0 conn(0x7f79640af910 legacy=0x7f796406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f796a400640 1 -- 172.21.15.86:0/4010482940 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79640cb7a0 con 0x7f79640c6930 2024-03-22T04:12:22.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.690+0000 7f7963e00640 1 -- 172.21.15.86:0/4010482940 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2174991000 0 0) 0x7f796415c770 con 0x7f79640af910 2024-03-22T04:12:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796a400640 1 --2- 172.21.15.86:0/4010482940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640c6930 0x7f79640c6d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f795c009d90 tx=0x7f795c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688c74ad98503623 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f7963e00640 1 -- 172.21.15.86:0/4010482940 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c0b2070 con 0x7f79640c6930 2024-03-22T04:12:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f7963e00640 1 -- 172.21.15.86:0/4010482940 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c0a5910 con 0x7f79640c6930 2024-03-22T04:12:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f7963e00640 1 -- 172.21.15.86:0/4010482940 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c0a5ce0 con 0x7f79640c6930 2024-03-22T04:12:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- 172.21.15.86:0/4010482940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640c6930 msgr2=0x7f79640c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 --2- 172.21.15.86:0/4010482940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640c6930 0x7f79640c6d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f795c009d90 tx=0x7f795c01c940 comp rx=0 tx=0).stop 2024-03-22T04:12:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- 172.21.15.86:0/4010482940 shutdown_connections 2024-03-22T04:12:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 --2- 172.21.15.86:0/4010482940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640c6930 0x7f79640c6d80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 --2- 172.21.15.86:0/4010482940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f796415c390 0x7f7964160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- 172.21.15.86:0/4010482940 >> 172.21.15.86:0/4010482940 conn(0x7f79640d0730 msgr2=0x7f79640d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- 172.21.15.86:0/4010482940 shutdown_connections 2024-03-22T04:12:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- 172.21.15.86:0/4010482940 wait complete. 2024-03-22T04:12:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 Processor -- start 2024-03-22T04:12:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- start start 2024-03-22T04:12:22.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640af910 0x7f796413de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79640c6930 0x7f7964153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f796415c390 0x7f79640b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f796406e630 con 0x7f79640af910 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7964072060 con 0x7f796415c390 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.694+0000 7f796c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7964072d90 con 0x7f79640c6930 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640af910 0x7f796413de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640af910 0x7f796413de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53696/0 (socket says 172.21.15.86:53696) 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 -- 172.21.15.86:0/1604485660 learned_addr learned my addr 172.21.15.86:0/1604485660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f796a400640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f796415c390 0x7f79640b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969000640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79640c6930 0x7f7964153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 -- 172.21.15.86:0/1604485660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79640c6930 msgr2=0x7f7964153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79640c6930 0x7f7964153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 -- 172.21.15.86:0/1604485660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f796415c390 msgr2=0x7f79640b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f796415c390 0x7f79640b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 -- 172.21.15.86:0/1604485660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7964140590 con 0x7f79640af910 2024-03-22T04:12:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7969a00640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640af910 0x7f796413de60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f795800e8a0 tx=0x7f795800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7961600640 1 -- 172.21.15.86:0/1604485660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958017070 con 0x7f79640af910 2024-03-22T04:12:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7961600640 1 -- 172.21.15.86:0/1604485660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795800c940 con 0x7f79640af910 2024-03-22T04:12:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7961600640 1 -- 172.21.15.86:0/1604485660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795800ccf0 con 0x7f79640af910 2024-03-22T04:12:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79640b5b00 con 0x7f79640af910 2024-03-22T04:12:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79640b60c0 con 0x7f79640af910 2024-03-22T04:12:22.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7964074470 con 0x7f79640af910 2024-03-22T04:12:22.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7961600640 1 -- 172.21.15.86:0/1604485660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f795801a070 con 0x7f79640af910 2024-03-22T04:12:22.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7961600640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7944067b20 0x7f7944069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:22.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.698+0000 7f7961600640 1 -- 172.21.15.86:0/1604485660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6274+0+0 (secure 0 0 0) 0x7f7958086080 con 0x7f79640af910 2024-03-22T04:12:22.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.702+0000 7f7969000640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7944067b20 0x7f7944069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:22.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.702+0000 7f7969000640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7944067b20 0x7f7944069fe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7964070510 tx=0x7f7950007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.702+0000 7f7961600640 1 -- 172.21.15.86:0/1604485660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7958056160 con 0x7f79640af910 2024-03-22T04:12:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:22.894+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f79640cd740 con 0x7f79640af910 2024-03-22T04:12:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.526+0000 7f7961600640 1 -- 172.21.15.86:0/1604485660 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f795805a200 con 0x7f79640af910 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7944067b20 msgr2=0x7f7944069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7944067b20 0x7f7944069fe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7964070510 tx=0x7f7950007d60 comp rx=0 tx=0).stop 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640af910 msgr2=0x7f796413de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640af910 0x7f796413de60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f795800e8a0 tx=0x7f795800eca0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 shutdown_connections 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7944067b20 0x7f7944069fe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f796415c390 0x7f79640b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79640c6930 0x7f7964153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 --2- 172.21.15.86:0/1604485660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79640af910 0x7f796413de60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 >> 172.21.15.86:0/1604485660 conn(0x7f79640d0730 msgr2=0x7f79640d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:23.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 shutdown_connections 2024-03-22T04:12:23.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.530+0000 7f796c400640 1 -- 172.21.15.86:0/1604485660 wait complete. 2024-03-22T04:12:23.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T04:12:23.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca5da00640 1 Processor -- start 2024-03-22T04:12:23.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca5da00640 1 -- start start 2024-03-22T04:12:23.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca5da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca58161db0 0x7fca5809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:23.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca5da00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca580b4a40 0x7fca5809afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:23.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca5da00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca58070d60 0x7fca5806f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:23.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca5da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fca58071460 con 0x7fca58161db0 2024-03-22T04:12:23.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca5da00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fca580d0610 con 0x7fca580b4a40 2024-03-22T04:12:23.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca5da00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca5806dcd0 con 0x7fca58070d60 2024-03-22T04:12:23.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca58161db0 0x7fca5809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:23.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca58161db0 0x7fca5809aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53704/0 (socket says 172.21.15.86:53704) 2024-03-22T04:12:23.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56a00640 1 -- 172.21.15.86:0/1171516974 learned_addr learned my addr 172.21.15.86:0/1171516974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:23.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56a00640 1 -- 172.21.15.86:0/1171516974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca58070d60 msgr2=0x7fca5806f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:23.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56a00640 1 --2- 172.21.15.86:0/1171516974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca58070d60 0x7fca5806f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56a00640 1 -- 172.21.15.86:0/1171516974 >> v2:172.21.15.196:3300/0 conn(0x7fca580b4a40 msgr2=0x7fca5809afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:23.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56000640 1 --2- 172.21.15.86:0/1171516974 >> v2:172.21.15.196:3300/0 conn(0x7fca580b4a40 0x7fca5809afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:23.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56a00640 1 --2- 172.21.15.86:0/1171516974 >> v2:172.21.15.196:3300/0 conn(0x7fca580b4a40 0x7fca5809afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56a00640 1 -- 172.21.15.86:0/1171516974 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca580707f0 con 0x7fca58161db0 2024-03-22T04:12:23.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.698+0000 7fca56000640 1 --2- 172.21.15.86:0/1171516974 >> v2:172.21.15.196:3300/0 conn(0x7fca580b4a40 0x7fca5809afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:23.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca56a00640 1 --2- 172.21.15.86:0/1171516974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58161db0 0x7fca5809aa70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca58073be0 tx=0x7fca4801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484a967b1091c14e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:23.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca55600640 1 -- 172.21.15.86:0/1171516974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480b0070 con 0x7fca58161db0 2024-03-22T04:12:23.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca55600640 1 -- 172.21.15.86:0/1171516974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca480ab600 con 0x7fca58161db0 2024-03-22T04:12:23.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca55600640 1 -- 172.21.15.86:0/1171516974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480ab9d0 con 0x7fca58161db0 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 -- 172.21.15.86:0/1171516974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58161db0 msgr2=0x7fca5809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 --2- 172.21.15.86:0/1171516974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58161db0 0x7fca5809aa70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca58073be0 tx=0x7fca4801c260 comp rx=0 tx=0).stop 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 -- 172.21.15.86:0/1171516974 shutdown_connections 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 --2- 172.21.15.86:0/1171516974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca58070d60 0x7fca5806f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 --2- 172.21.15.86:0/1171516974 >> v2:172.21.15.196:3300/0 conn(0x7fca580b4a40 0x7fca5809afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 --2- 172.21.15.86:0/1171516974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58161db0 0x7fca5809aa70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 -- 172.21.15.86:0/1171516974 >> 172.21.15.86:0/1171516974 conn(0x7fca580b3d40 msgr2=0x7fca580b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 -- 172.21.15.86:0/1171516974 shutdown_connections 2024-03-22T04:12:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 -- 172.21.15.86:0/1171516974 wait complete. 2024-03-22T04:12:23.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 Processor -- start 2024-03-22T04:12:23.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.702+0000 7fca5da00640 1 -- start start 2024-03-22T04:12:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58070d60 0x7fca580b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca580b4a40 0x7fca581571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca58161db0 0x7fca58155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca580d0ab0 con 0x7fca58070d60 2024-03-22T04:12:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca5806d9f0 con 0x7fca580b4a40 2024-03-22T04:12:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca580d0c30 con 0x7fca58161db0 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58070d60 0x7fca580b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58070d60 0x7fca580b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53714/0 (socket says 172.21.15.86:53714) 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 -- 172.21.15.86:0/3808242657 learned_addr learned my addr 172.21.15.86:0/3808242657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 -- 172.21.15.86:0/3808242657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca58161db0 msgr2=0x7fca58155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca58161db0 0x7fca58155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 -- 172.21.15.86:0/3808242657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca580b4a40 msgr2=0x7fca581571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca580b4a40 0x7fca581571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 -- 172.21.15.86:0/3808242657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca58159fd0 con 0x7fca58070d60 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca56a00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58070d60 0x7fca580b7160 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fca480ae040 tx=0x7fca48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca4ea00640 1 -- 172.21.15.86:0/3808242657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480b0040 con 0x7fca58070d60 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca4ea00640 1 -- 172.21.15.86:0/3808242657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca480b9070 con 0x7fca58070d60 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca4ea00640 1 -- 172.21.15.86:0/3808242657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480b48a0 con 0x7fca58070d60 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5815a2b0 con 0x7fca58070d60 2024-03-22T04:12:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca580b76a0 con 0x7fca58070d60 2024-03-22T04:12:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca58074580 con 0x7fca58070d60 2024-03-22T04:12:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.706+0000 7fca4ea00640 1 -- 172.21.15.86:0/3808242657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fca480ab4c0 con 0x7fca58070d60 2024-03-22T04:12:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.710+0000 7fca4ea00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca38067b20 0x7fca38069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.710+0000 7fca4ea00640 1 -- 172.21.15.86:0/3808242657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6274+0+0 (secure 0 0 0) 0x7fca4811ffe0 con 0x7fca58070d60 2024-03-22T04:12:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.710+0000 7fca56000640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca38067b20 0x7fca38069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.710+0000 7fca56000640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca38067b20 0x7fca38069fe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca5815b550 tx=0x7fca44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.714+0000 7fca4ea00640 1 -- 172.21.15.86:0/3808242657 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fca48016110 con 0x7fca58070d60 2024-03-22T04:12:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:23.870+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fca58158ab0 con 0x7fca58070d60 2024-03-22T04:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca4ea00640 1 -- 172.21.15.86:0/3808242657 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) ==== 181+0+0 (secure 0 0 0) 0x7fca480f0710 con 0x7fca58070d60 2024-03-22T04:12:24.540 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T04:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca38067b20 msgr2=0x7fca38069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca38067b20 0x7fca38069fe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca5815b550 tx=0x7fca44008040 comp rx=0 tx=0).stop 2024-03-22T04:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58070d60 msgr2=0x7fca580b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58070d60 0x7fca580b7160 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fca480ae040 tx=0x7fca48019000 comp rx=0 tx=0).stop 2024-03-22T04:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 shutdown_connections 2024-03-22T04:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca38067b20 0x7fca38069fe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca580b4a40 0x7fca581571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca58161db0 0x7fca58155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 --2- 172.21.15.86:0/3808242657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca58070d60 0x7fca580b7160 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 >> 172.21.15.86:0/3808242657 conn(0x7fca580b3d40 msgr2=0x7fca5806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.538+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 shutdown_connections 2024-03-22T04:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.542+0000 7fca5da00640 1 -- 172.21.15.86:0/3808242657 wait complete. 2024-03-22T04:12:24.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T04:12:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61f2400640 1 Processor -- start 2024-03-22T04:12:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61f2400640 1 -- start start 2024-03-22T04:12:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61f2400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f61ec0b1cd0 0x7f61ec0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61f2400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61ec0b25f0 con 0x7f61ec06de70 2024-03-22T04:12:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61f2400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61ec0cecb0 con 0x7f61ec0cfa70 2024-03-22T04:12:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61f2400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f61ec0cea80 con 0x7f61ec0b1cd0 2024-03-22T04:12:24.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61ebe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f61ec06de70 0x7f61ec13f8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34738/0 (socket says 172.21.15.86:34738) 2024-03-22T04:12:24.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61ebe00640 1 -- 172.21.15.86:0/3568625730 learned_addr learned my addr 172.21.15.86:0/3568625730 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:24.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61eb400640 1 --2- 172.21.15.86:0/3568625730 >> v2:172.21.15.196:3300/0 conn(0x7f61ec0b1cd0 0x7f61ec0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:24.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61eaa00640 1 -- 172.21.15.86:0/3568625730 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 543845567 0 0) 0x7f61ec0b25f0 con 0x7f61ec06de70 2024-03-22T04:12:24.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61eaa00640 1 -- 172.21.15.86:0/3568625730 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61d0003530 con 0x7f61ec06de70 2024-03-22T04:12:24.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61eb400640 1 -- 172.21.15.86:0/3568625730 >> v1:172.21.15.196:6789/0 conn(0x7f61ec0cfa70 legacy=0x7f61ec06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:24.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61eb400640 1 -- 172.21.15.86:0/3568625730 >> v1:172.21.15.86:6789/0 conn(0x7f61ec06de70 legacy=0x7f61ec13f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:24.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.710+0000 7f61eb400640 1 -- 172.21.15.86:0/3568625730 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec0cef90 con 0x7f61ec0b1cd0 2024-03-22T04:12:24.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61eb400640 1 --2- 172.21.15.86:0/3568625730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 0x7f61ec0b20b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0097c0 tx=0x7f61dc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=261af285cde51579 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61eaa00640 1 -- 172.21.15.86:0/3568625730 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0b0070 con 0x7f61ec0b1cd0 2024-03-22T04:12:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61eaa00640 1 -- 172.21.15.86:0/3568625730 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc0ab600 con 0x7f61ec0b1cd0 2024-03-22T04:12:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61eaa00640 1 -- 172.21.15.86:0/3568625730 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0ab9b0 con 0x7f61ec0b1cd0 2024-03-22T04:12:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61f2400640 1 -- 172.21.15.86:0/3568625730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 msgr2=0x7f61ec0b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:24.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61f2400640 1 --2- 172.21.15.86:0/3568625730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 0x7f61ec0b20b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0097c0 tx=0x7f61dc01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:12:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61f2400640 1 -- 172.21.15.86:0/3568625730 shutdown_connections 2024-03-22T04:12:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61f2400640 1 --2- 172.21.15.86:0/3568625730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 0x7f61ec0b20b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61f2400640 1 -- 172.21.15.86:0/3568625730 >> 172.21.15.86:0/3568625730 conn(0x7f61ec06b660 msgr2=0x7f61ec06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61f2400640 1 -- 172.21.15.86:0/3568625730 shutdown_connections 2024-03-22T04:12:24.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.714+0000 7f61f2400640 1 -- 172.21.15.86:0/3568625730 wait complete. 2024-03-22T04:12:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 Processor -- start 2024-03-22T04:12:24.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 -- start start 2024-03-22T04:12:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec06de70 0x7f61ec155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 0x7f61ec0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:24.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61ec0cfa70 0x7f61ec0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec169f10 con 0x7f61ec06de70 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec169d90 con 0x7f61ec0b1cd0 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f61ec16a090 con 0x7f61ec0cfa70 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec06de70 0x7f61ec155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec06de70 0x7f61ec155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53722/0 (socket says 172.21.15.86:53722) 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 -- 172.21.15.86:0/378076996 learned_addr learned my addr 172.21.15.86:0/378076996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61ebe00640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61ec0cfa70 0x7f61ec0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61e2a00640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 0x7f61ec0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 -- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61ec0cfa70 msgr2=0x7f61ec0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61ec0cfa70 0x7f61ec0b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:24.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 -- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 msgr2=0x7f61ec0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 0x7f61ec0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 -- 172.21.15.86:0/378076996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec157400 con 0x7f61ec06de70 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61e2a00640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 0x7f61ec0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61eb400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec06de70 0x7f61ec155b40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0190b0 tx=0x7f61dc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61e3e00640 1 -- 172.21.15.86:0/378076996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0b0070 con 0x7f61ec06de70 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61e3e00640 1 -- 172.21.15.86:0/378076996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc0b1040 con 0x7f61ec06de70 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec1576e0 con 0x7f61ec06de70 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61e3e00640 1 -- 172.21.15.86:0/378076996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0abde0 con 0x7f61ec06de70 2024-03-22T04:12:24.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.718+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec157ca0 con 0x7f61ec06de70 2024-03-22T04:12:24.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.722+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ec074580 con 0x7f61ec06de70 2024-03-22T04:12:24.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.722+0000 7f61e3e00640 1 -- 172.21.15.86:0/378076996 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f61dc0bc060 con 0x7f61ec06de70 2024-03-22T04:12:24.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.722+0000 7f61e3e00640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f61c8067b20 0x7f61c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:24.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.722+0000 7f61e3e00640 1 -- 172.21.15.86:0/378076996 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5836+0+0 (secure 0 0 0) 0x7f61dc11f140 con 0x7f61ec06de70 2024-03-22T04:12:24.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.722+0000 7f61e2a00640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f61c8067b20 0x7f61c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:24.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.722+0000 7f61e2a00640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f61c8067b20 0x7f61c8069fe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0af600 tx=0x7f61d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:24.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.726+0000 7f61e3e00640 1 -- 172.21.15.86:0/378076996 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f61dc016090 con 0x7f61ec06de70 2024-03-22T04:12:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:24.890+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f61ec1570b0 con 0x7f61ec06de70 2024-03-22T04:12:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.538+0000 7f61e3e00640 1 -- 172.21.15.86:0/378076996 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) ==== 169+0+0 (secure 0 0 0) 0x7f61dc0b3300 con 0x7f61ec06de70 2024-03-22T04:12:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T04:12:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f61c8067b20 msgr2=0x7f61c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f61c8067b20 0x7f61c8069fe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0af600 tx=0x7f61d4009040 comp rx=0 tx=0).stop 2024-03-22T04:12:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec06de70 msgr2=0x7f61ec155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec06de70 0x7f61ec155b40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0190b0 tx=0x7f61dc01ace0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 shutdown_connections 2024-03-22T04:12:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f61c8067b20 0x7f61c8069fe0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec0b1cd0 0x7f61ec0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61ec0cfa70 0x7f61ec0b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 --2- 172.21.15.86:0/378076996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec06de70 0x7f61ec155b40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 >> 172.21.15.86:0/378076996 conn(0x7f61ec06b660 msgr2=0x7f61ec15ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.542+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 shutdown_connections 2024-03-22T04:12:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.546+0000 7f61f2400640 1 -- 172.21.15.86:0/378076996 wait complete. 2024-03-22T04:12:25.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-22T04:12:25.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0ce600640 1 Processor -- start 2024-03-22T04:12:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0ce600640 1 -- start start 2024-03-22T04:12:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0ce600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0ce600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe0c8160150 0x7fe0c80ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0ce600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c80d0440 con 0x7fe0c806d330 2024-03-22T04:12:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0ce600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c80d0720 con 0x7fe0c8160150 2024-03-22T04:12:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0ce600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c80d0a00 con 0x7fe0c815e250 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0cdc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe0c8160150 0x7fe0c80ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0cd200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0c7e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe0c806d330 0x7fe0c815fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:56230/0 (socket says 172.21.15.86:56230) 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0c7e00640 1 -- 172.21.15.86:0/1301315923 learned_addr learned my addr 172.21.15.86:0/1301315923 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0cdc00640 1 -- 172.21.15.86:0/1301315923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 msgr2=0x7fe0c80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0cdc00640 1 --2- 172.21.15.86:0/1301315923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c80d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0cdc00640 1 -- 172.21.15.86:0/1301315923 >> v1:172.21.15.196:6789/0 conn(0x7fe0c806d330 legacy=0x7fe0c815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0cdc00640 1 -- 172.21.15.86:0/1301315923 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c80d0ce0 con 0x7fe0c8160150 2024-03-22T04:12:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.710+0000 7fe0cd200640 1 --2- 172.21.15.86:0/1301315923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c80d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0cdc00640 1 --2- 172.21.15.86:0/1301315923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 0x7fe0c80ce5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009d90 tx=0x7fe0c001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a7c6139ede944b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0c7400640 1 -- 172.21.15.86:0/1301315923 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00b2070 con 0x7fe0c8160150 2024-03-22T04:12:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0c7400640 1 -- 172.21.15.86:0/1301315923 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c00a57a0 con 0x7fe0c8160150 2024-03-22T04:12:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0c7400640 1 -- 172.21.15.86:0/1301315923 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00a5b70 con 0x7fe0c8160150 2024-03-22T04:12:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0ce600640 1 -- 172.21.15.86:0/1301315923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 msgr2=0x7fe0c80ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1301315923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 0x7fe0c80ce5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009d90 tx=0x7fe0c001c8e0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0ce600640 1 -- 172.21.15.86:0/1301315923 shutdown_connections 2024-03-22T04:12:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1301315923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 0x7fe0c80ce5a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1301315923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c80d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0ce600640 1 -- 172.21.15.86:0/1301315923 >> 172.21.15.86:0/1301315923 conn(0x7fe0c80d1f30 msgr2=0x7fe0c80d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:25.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0ce600640 1 -- 172.21.15.86:0/1301315923 shutdown_connections 2024-03-22T04:12:25.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.714+0000 7fe0ce600640 1 -- 172.21.15.86:0/1301315923 wait complete. 2024-03-22T04:12:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 Processor -- start 2024-03-22T04:12:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 -- start start 2024-03-22T04:12:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c806d330 0x7fe0c81594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 0x7fe0c815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c815e7b0 con 0x7fe0c806d330 2024-03-22T04:12:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c815e630 con 0x7fe0c8160150 2024-03-22T04:12:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c815e930 con 0x7fe0c815e250 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c806d330 0x7fe0c81594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c806d330 0x7fe0c81594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53726/0 (socket says 172.21.15.86:53726) 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 -- 172.21.15.86:0/1505146846 learned_addr learned my addr 172.21.15.86:0/1505146846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0c7e00640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cdc00640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 0x7fe0c815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 -- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 msgr2=0x7fe0c815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 -- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 msgr2=0x7fe0c815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 0x7fe0c815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 -- 172.21.15.86:0/1505146846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c8158400 con 0x7fe0c806d330 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cdc00640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 0x7fe0c815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0c7e00640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0cd200640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c806d330 0x7fe0c81594a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800ebe0 tx=0x7fe0b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0c4c00640 1 -- 172.21.15.86:0/1505146846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800cc30 con 0x7fe0c806d330 2024-03-22T04:12:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0c4c00640 1 -- 172.21.15.86:0/1505146846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b800cdd0 con 0x7fe0c806d330 2024-03-22T04:12:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0c4c00640 1 -- 172.21.15.86:0/1505146846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8005bc0 con 0x7fe0c806d330 2024-03-22T04:12:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c81586e0 con 0x7fe0c806d330 2024-03-22T04:12:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c8158ca0 con 0x7fe0c806d330 2024-03-22T04:12:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.718+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c804f900 con 0x7fe0c806d330 2024-03-22T04:12:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.722+0000 7fe0c4c00640 1 -- 172.21.15.86:0/1505146846 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe0b8006030 con 0x7fe0c806d330 2024-03-22T04:12:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.722+0000 7fe0c4c00640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe0a8067b20 0x7fe0a8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.722+0000 7fe0c4c00640 1 -- 172.21.15.86:0/1505146846 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7fe0b80861f0 con 0x7fe0c806d330 2024-03-22T04:12:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.722+0000 7fe0c7e00640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe0a8067b20 0x7fe0a8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.722+0000 7fe0c7e00640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe0a8067b20 0x7fe0a8069fe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4007fd0 tx=0x7fe0b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.726+0000 7fe0c4c00640 1 -- 172.21.15.86:0/1505146846 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe0b8017020 con 0x7fe0c806d330 2024-03-22T04:12:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.882+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe0c80b7eb0 con 0x7fe0c806d330 2024-03-22T04:12:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.886+0000 7fe0c4c00640 1 -- 172.21.15.86:0/1505146846 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fe0b8056d50 con 0x7fe0c806d330 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe0a8067b20 msgr2=0x7fe0a8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe0a8067b20 0x7fe0a8069fe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4007fd0 tx=0x7fe0b4009040 comp rx=0 tx=0).stop 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c806d330 msgr2=0x7fe0c81594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c806d330 0x7fe0c81594a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800ebe0 tx=0x7fe0b800c520 comp rx=0 tx=0).stop 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 shutdown_connections 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe0a8067b20 0x7fe0a8069fe0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c8160150 0x7fe0c815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c815e250 0x7fe0c815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 --2- 172.21.15.86:0/1505146846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c806d330 0x7fe0c81594a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 >> 172.21.15.86:0/1505146846 conn(0x7fe0c80d1f30 msgr2=0x7fe0c80d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 shutdown_connections 2024-03-22T04:12:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:25.890+0000 7fe0ce600640 1 -- 172.21.15.86:0/1505146846 wait complete. 2024-03-22T04:12:25.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-22T04:12:26.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.042+0000 7f1231000640 1 Processor -- start 2024-03-22T04:12:26.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1231000640 1 -- start start 2024-03-22T04:12:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1231000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f122c06e220 0x7f122c15f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1231000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122c06e890 con 0x7f122c158a50 2024-03-22T04:12:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1231000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122c06eb70 con 0x7f122c06cd80 2024-03-22T04:12:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1231000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f122c06ee50 con 0x7f122c06e220 2024-03-22T04:12:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1229e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f122c158a50 0x7f122c06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55808/0 (socket says 172.21.15.86:55808) 2024-03-22T04:12:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1229e00640 1 -- 172.21.15.86:0/1682233784 learned_addr learned my addr 172.21.15.86:0/1682233784 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f122a800640 1 --2- 172.21.15.86:0/1682233784 >> v2:172.21.15.196:3300/0 conn(0x7f122c06e220 0x7f122c15f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1229400640 1 -- 172.21.15.86:0/1682233784 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110717761 0 0) 0x7f122c06e890 con 0x7f122c158a50 2024-03-22T04:12:26.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1229400640 1 -- 172.21.15.86:0/1682233784 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1210003530 con 0x7f122c158a50 2024-03-22T04:12:26.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f1229400640 1 -- 172.21.15.86:0/1682233784 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2246751549 0 0) 0x7f1210003530 con 0x7f122c158a50 2024-03-22T04:12:26.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f122a800640 1 -- 172.21.15.86:0/1682233784 >> v1:172.21.15.196:6789/0 conn(0x7f122c06cd80 legacy=0x7f122c0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f122a800640 1 -- 172.21.15.86:0/1682233784 >> v1:172.21.15.86:6789/0 conn(0x7f122c158a50 legacy=0x7f122c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.046+0000 7f122a800640 1 -- 172.21.15.86:0/1682233784 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c06f130 con 0x7f122c06e220 2024-03-22T04:12:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f122a800640 1 --2- 172.21.15.86:0/1682233784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c06e220 0x7f122c15f500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1224009d90 tx=0x7f122401c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ed586d6cdbed38 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1229400640 1 -- 172.21.15.86:0/1682233784 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c001680 con 0x7f122c06e220 2024-03-22T04:12:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1229400640 1 -- 172.21.15.86:0/1682233784 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12240a6450 con 0x7f122c06e220 2024-03-22T04:12:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1229400640 1 -- 172.21.15.86:0/1682233784 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240a6820 con 0x7f122c06e220 2024-03-22T04:12:26.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1231000640 1 -- 172.21.15.86:0/1682233784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c06e220 msgr2=0x7f122c15f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1231000640 1 --2- 172.21.15.86:0/1682233784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c06e220 0x7f122c15f500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1224009d90 tx=0x7f122401c800 comp rx=0 tx=0).stop 2024-03-22T04:12:26.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1231000640 1 -- 172.21.15.86:0/1682233784 shutdown_connections 2024-03-22T04:12:26.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1231000640 1 --2- 172.21.15.86:0/1682233784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c06e220 0x7f122c15f500 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1231000640 1 -- 172.21.15.86:0/1682233784 >> 172.21.15.86:0/1682233784 conn(0x7f122c071fe0 msgr2=0x7f122c0723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:26.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1231000640 1 -- 172.21.15.86:0/1682233784 shutdown_connections 2024-03-22T04:12:26.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1231000640 1 -- 172.21.15.86:0/1682233784 wait complete. 2024-03-22T04:12:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.050+0000 7f1231000640 1 Processor -- start 2024-03-22T04:12:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 -- start start 2024-03-22T04:12:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c06cd80 0x7f122c161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f122c06e220 0x7f122c1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c158a50 0x7f122c159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f122c0afdc0 con 0x7f122c06cd80 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f122c0afc40 con 0x7f122c158a50 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f122c16c1d0 con 0x7f122c06e220 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c06cd80 0x7f122c161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c06cd80 0x7f122c161d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58056/0 (socket says 172.21.15.86:58056) 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 -- 172.21.15.86:0/1692679737 learned_addr learned my addr 172.21.15.86:0/1692679737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1223e00640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f122c06e220 0x7f122c1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f122a800640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c158a50 0x7f122c159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 -- 172.21.15.86:0/1692679737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f122c06e220 msgr2=0x7f122c1622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f122c06e220 0x7f122c1622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 -- 172.21.15.86:0/1692679737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c158a50 msgr2=0x7f122c159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c158a50 0x7f122c159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 -- 172.21.15.86:0/1692679737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c141400 con 0x7f122c06cd80 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1229e00640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c06cd80 0x7f122c161d80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f121c00c730 tx=0x7f121c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1222000640 1 -- 172.21.15.86:0/1692679737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c017070 con 0x7f122c06cd80 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1222000640 1 -- 172.21.15.86:0/1692679737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121c005b00 con 0x7f122c06cd80 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1222000640 1 -- 172.21.15.86:0/1692679737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c007070 con 0x7f122c06cd80 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c1f79b0 con 0x7f122c06cd80 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c159580 con 0x7f122c06cd80 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.054+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c1f77a0 con 0x7f122c06cd80 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.058+0000 7f1222000640 1 -- 172.21.15.86:0/1692679737 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f121c01a070 con 0x7f122c06cd80 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.058+0000 7f1222000640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1208067b20 0x7f1208069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.058+0000 7f1222000640 1 -- 172.21.15.86:0/1692679737 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f121c085410 con 0x7f122c06cd80 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.058+0000 7f1223e00640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1208067b20 0x7f1208069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.058+0000 7f1223e00640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1208067b20 0x7f1208069fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f122c15a100 tx=0x7f1214009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.062+0000 7f1222000640 1 -- 172.21.15.86:0/1692679737 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f121c055e60 con 0x7f122c06cd80 2024-03-22T04:12:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.270+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f122c068650 con 0x7f122c06cd80 2024-03-22T04:12:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.554+0000 7f1222000640 1 -- 172.21.15.86:0/1692679737 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v132) ==== 142+0+0 (secure 0 0 0) 0x7f121c055760 con 0x7f122c06cd80 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1208067b20 msgr2=0x7f1208069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1208067b20 0x7f1208069fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f122c15a100 tx=0x7f1214009040 comp rx=0 tx=0).stop 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c06cd80 msgr2=0x7f122c161d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c06cd80 0x7f122c161d80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f121c00c730 tx=0x7f121c00cbf0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 shutdown_connections 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1208067b20 0x7f1208069fe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f122c158a50 0x7f122c159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f122c06e220 0x7f122c1622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 --2- 172.21.15.86:0/1692679737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c06cd80 0x7f122c161d80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 >> 172.21.15.86:0/1692679737 conn(0x7f122c071fe0 msgr2=0x7f122c0b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 shutdown_connections 2024-03-22T04:12:26.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.558+0000 7f1231000640 1 -- 172.21.15.86:0/1692679737 wait complete. 2024-03-22T04:12:26.578 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-22T04:12:26.578 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-03-22T04:12:26.578 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-22T04:12:26.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.718+0000 7fed39800640 1 Processor -- start 2024-03-22T04:12:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.718+0000 7fed39800640 1 -- start start 2024-03-22T04:12:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed39800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed34147e90 0x7fed3406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed39800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed3406e080 con 0x7fed3406ff10 2024-03-22T04:12:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed39800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed34162c70 con 0x7fed34147ab0 2024-03-22T04:12:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed39800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fed34070380 con 0x7fed34147e90 2024-03-22T04:12:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed2b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed34147e90 0x7fed3406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed38e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fed3406ff10 0x7fed34161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55816/0 (socket says 172.21.15.86:55816) 2024-03-22T04:12:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed38e00640 1 -- 172.21.15.86:0/350899481 learned_addr learned my addr 172.21.15.86:0/350899481 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed2b400640 1 -- 172.21.15.86:0/350899481 >> v1:172.21.15.196:6789/0 conn(0x7fed34147ab0 legacy=0x7fed3406d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed2b400640 1 -- 172.21.15.86:0/350899481 >> v1:172.21.15.86:6789/0 conn(0x7fed3406ff10 legacy=0x7fed34161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed2b400640 1 -- 172.21.15.86:0/350899481 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3406ce90 con 0x7fed34147e90 2024-03-22T04:12:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.722+0000 7fed33400640 1 -- 172.21.15.86:0/350899481 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549838455 0 0) 0x7fed3406e080 con 0x7fed3406ff10 2024-03-22T04:12:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed2b400640 1 --2- 172.21.15.86:0/350899481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147e90 0x7fed3406f9d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0097c0 tx=0x7fed1c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc601df043509ff server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed33400640 1 -- 172.21.15.86:0/350899481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0b0070 con 0x7fed34147e90 2024-03-22T04:12:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed33400640 1 -- 172.21.15.86:0/350899481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c0ab600 con 0x7fed34147e90 2024-03-22T04:12:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed33400640 1 -- 172.21.15.86:0/350899481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0ab9d0 con 0x7fed34147e90 2024-03-22T04:12:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- 172.21.15.86:0/350899481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147e90 msgr2=0x7fed3406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 --2- 172.21.15.86:0/350899481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147e90 0x7fed3406f9d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0097c0 tx=0x7fed1c01c410 comp rx=0 tx=0).stop 2024-03-22T04:12:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- 172.21.15.86:0/350899481 shutdown_connections 2024-03-22T04:12:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 --2- 172.21.15.86:0/350899481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147e90 0x7fed3406f9d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- 172.21.15.86:0/350899481 >> 172.21.15.86:0/350899481 conn(0x7fed3406bc60 msgr2=0x7fed340b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- 172.21.15.86:0/350899481 shutdown_connections 2024-03-22T04:12:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- 172.21.15.86:0/350899481 wait complete. 2024-03-22T04:12:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 Processor -- start 2024-03-22T04:12:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- start start 2024-03-22T04:12:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed3406ff10 0x7fed340cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147ab0 0x7fed3415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed34147e90 0x7fed341459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed3416c2f0 con 0x7fed34147ab0 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed3415f420 con 0x7fed34147e90 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.726+0000 7fed39800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fed3416c470 con 0x7fed3406ff10 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147ab0 0x7fed3415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147ab0 0x7fed3415d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58068/0 (socket says 172.21.15.86:58068) 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 -- 172.21.15.86:0/1810375361 learned_addr learned my addr 172.21.15.86:0/1810375361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed33e00640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed3406ff10 0x7fed340cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 -- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed3406ff10 msgr2=0x7fed340cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed38e00640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed34147e90 0x7fed341459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed3406ff10 0x7fed340cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 -- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed34147e90 msgr2=0x7fed341459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed34147e90 0x7fed341459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 -- 172.21.15.86:0/1810375361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3415dc10 con 0x7fed34147ab0 2024-03-22T04:12:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed33e00640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed3406ff10 0x7fed340cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed38e00640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed34147e90 0x7fed341459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed2b400640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147ab0 0x7fed3415d6d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0097c0 tx=0x7fed1c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed30c00640 1 -- 172.21.15.86:0/1810375361 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0b0050 con 0x7fed34147ab0 2024-03-22T04:12:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed30c00640 1 -- 172.21.15.86:0/1810375361 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c0bc070 con 0x7fed34147ab0 2024-03-22T04:12:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed30c00640 1 -- 172.21.15.86:0/1810375361 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0b5a30 con 0x7fed34147ab0 2024-03-22T04:12:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed340cc0d0 con 0x7fed34147ab0 2024-03-22T04:12:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed340cc690 con 0x7fed34147ab0 2024-03-22T04:12:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed34074470 con 0x7fed34147ab0 2024-03-22T04:12:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed30c00640 1 -- 172.21.15.86:0/1810375361 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fed1c0ab4c0 con 0x7fed34147ab0 2024-03-22T04:12:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed30c00640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed10067b50 0x7fed1006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.730+0000 7fed30c00640 1 -- 172.21.15.86:0/1810375361 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7fed1c120690 con 0x7fed34147ab0 2024-03-22T04:12:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.734+0000 7fed33e00640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed10067b50 0x7fed1006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.734+0000 7fed33e00640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed10067b50 0x7fed1006a010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fed341fa930 tx=0x7fed24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.734+0000 7fed30c00640 1 -- 172.21.15.86:0/1810375361 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fed1c016090 con 0x7fed34147ab0 2024-03-22T04:12:26.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.890+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed340b4910 con 0x7fed34147ab0 2024-03-22T04:12:26.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed30c00640 1 -- 172.21.15.86:0/1810375361 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+13295 (secure 0 0 0) 0x7fed1c0f1170 con 0x7fed34147ab0 2024-03-22T04:12:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:26.895 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:12:25.539697+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1426902111":"2024-03-23T04:12:22.525601+0000","172.21.15.196:6833/1426902111":"2024-03-23T04:12:22.525601+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:26.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed10067b50 msgr2=0x7fed1006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed10067b50 0x7fed1006a010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fed341fa930 tx=0x7fed24009040 comp rx=0 tx=0).stop 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147ab0 msgr2=0x7fed3415d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147ab0 0x7fed3415d6d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0097c0 tx=0x7fed1c01ac50 comp rx=0 tx=0).stop 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 shutdown_connections 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fed10067b50 0x7fed1006a010 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed34147e90 0x7fed341459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed34147ab0 0x7fed3415d6d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 --2- 172.21.15.86:0/1810375361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed3406ff10 0x7fed340cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 >> 172.21.15.86:0/1810375361 conn(0x7fed3406bc60 msgr2=0x7fed340b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 shutdown_connections 2024-03-22T04:12:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:26.894+0000 7fed39800640 1 -- 172.21.15.86:0/1810375361 wait complete. 2024-03-22T04:12:26.916 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:12:26.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:12:27.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a3e00640 1 Processor -- start 2024-03-22T04:12:27.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a3e00640 1 -- start start 2024-03-22T04:12:27.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a3e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c072650 0x7fe49c072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a3e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49c073150 con 0x7fe49c15b760 2024-03-22T04:12:27.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49c0715a0 con 0x7fe49c06e7e0 2024-03-22T04:12:27.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a3e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c071350 con 0x7fe49c072650 2024-03-22T04:12:27.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe49c15b760 0x7fe49c15bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55818/0 (socket says 172.21.15.86:55818) 2024-03-22T04:12:27.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1e00640 1 -- 172.21.15.86:0/2431544506 learned_addr learned my addr 172.21.15.86:0/2431544506 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:27.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a0a00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c072650 0x7fe49c072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 721601903 0 0) 0x7fe49c073150 con 0x7fe49c15b760 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe484005740 con 0x7fe49c15b760 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480552781 0 0) 0x7fe49c0715a0 con 0x7fe49c06e7e0 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe49c073150 con 0x7fe49c06e7e0 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4190704212 0 0) 0x7fe484005740 con 0x7fe49c15b760 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c072650 msgr2=0x7fe49c072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c072650 0x7fe49c072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 >> v1:172.21.15.196:6789/0 conn(0x7fe49c06e7e0 legacy=0x7fe49c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c071880 con 0x7fe49c15b760 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fe494001a00 con 0x7fe49c15b760 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 >> v1:172.21.15.86:6789/0 conn(0x7fe49c15b760 legacy=0x7fe49c15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe484005d90 0x7fe484006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484006a20 0x7fe484006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c072650 0x7fe484007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c071350 con 0x7fe484005d90 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe48400dd70 con 0x7fe484006a20 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe48400df40 con 0x7fe49c072650 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1400640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe484005d90 0x7fe484006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a0a00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484006a20 0x7fe484006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1e00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c072650 0x7fe484007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1400640 1 -- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c072650 msgr2=0x7fe484007440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1400640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c072650 0x7fe484007440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1400640 1 -- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484006a20 msgr2=0x7fe484006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1400640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484006a20 0x7fe484006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a1400640 1 -- 172.21.15.86:0/2431544506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe484005740 con 0x7fe484005d90 2024-03-22T04:12:27.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.054+0000 7fe4a0a00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484006a20 0x7fe484006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe4a1400640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe484005d90 0x7fe484006190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe490001900 tx=0x7fe49001c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6315936f2eb018f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490001520 con 0x7fe484005d90 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4900052f0 con 0x7fe484005d90 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe49be00640 1 -- 172.21.15.86:0/2431544506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490003840 con 0x7fe484005d90 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe4a3e00640 1 -- 172.21.15.86:0/2431544506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe484005d90 msgr2=0x7fe484006190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe4a3e00640 1 --2- 172.21.15.86:0/2431544506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe484005d90 0x7fe484006190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe490001900 tx=0x7fe49001c550 comp rx=0 tx=0).stop 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe4a1e00640 1 -- 172.21.15.86:0/2431544506 reap_dead start 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe4a3e00640 1 -- 172.21.15.86:0/2431544506 shutdown_connections 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe4a3e00640 1 -- 172.21.15.86:0/2431544506 >> 172.21.15.86:0/2431544506 conn(0x7fe49c0cc940 msgr2=0x7fe49c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe4a3e00640 1 -- 172.21.15.86:0/2431544506 shutdown_connections 2024-03-22T04:12:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.058+0000 7fe4a3e00640 1 -- 172.21.15.86:0/2431544506 wait complete. 2024-03-22T04:12:27.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 Processor -- start 2024-03-22T04:12:27.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 -- start start 2024-03-22T04:12:27.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484005d90 0x7fe49c15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c149a00 0x7fe49c157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49c1f7740 0x7fe49c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c072080 con 0x7fe49c1f7740 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c072e40 con 0x7fe484005d90 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c160120 con 0x7fe49c149a00 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49c1f7740 0x7fe49c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49c1f7740 0x7fe49c157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58090/0 (socket says 172.21.15.86:58090) 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 -- 172.21.15.86:0/1333323071 learned_addr learned my addr 172.21.15.86:0/1333323071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a0a00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c149a00 0x7fe49c157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1400640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484005d90 0x7fe49c15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 -- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c149a00 msgr2=0x7fe49c157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c149a00 0x7fe49c157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 -- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484005d90 msgr2=0x7fe49c15eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484005d90 0x7fe49c15eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 -- 172.21.15.86:0/1333323071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c1605b0 con 0x7fe49c1f7740 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1400640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484005d90 0x7fe49c15eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a0a00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c149a00 0x7fe49c157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a1e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49c1f7740 0x7fe49c157ba0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0b6130 tx=0x7fe494007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe499600640 1 -- 172.21.15.86:0/1333323071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49400ea50 con 0x7fe49c1f7740 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe499600640 1 -- 172.21.15.86:0/1333323071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49400ebf0 con 0x7fe49c1f7740 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe499600640 1 -- 172.21.15.86:0/1333323071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494014430 con 0x7fe49c1f7740 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c15f510 con 0x7fe49c1f7740 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.062+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c15fad0 con 0x7fe49c1f7740 2024-03-22T04:12:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.066+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c074470 con 0x7fe49c1f7740 2024-03-22T04:12:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.066+0000 7fe499600640 1 -- 172.21.15.86:0/1333323071 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe4940145d0 con 0x7fe49c1f7740 2024-03-22T04:12:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.066+0000 7fe499600640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe47c067b20 0x7fe47c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.066+0000 7fe499600640 1 -- 172.21.15.86:0/1333323071 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7fe494085ab0 con 0x7fe49c1f7740 2024-03-22T04:12:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.066+0000 7fe4a1400640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe47c067b20 0x7fe47c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.066+0000 7fe4a1400640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe47c067b20 0x7fe47c069fe0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe4900ac000 tx=0x7fe4900a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:27.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.070+0000 7fe499600640 1 -- 172.21.15.86:0/1333323071 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe4940565a0 con 0x7fe49c1f7740 2024-03-22T04:12:27.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.226+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe49c15c9f0 con 0x7fe49c1f7740 2024-03-22T04:12:27.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.226+0000 7fe499600640 1 -- 172.21.15.86:0/1333323071 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7fe494055ea0 con 0x7fe49c1f7740 2024-03-22T04:12:27.230 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:12:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe47c067b20 msgr2=0x7fe47c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe47c067b20 0x7fe47c069fe0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe4900ac000 tx=0x7fe4900a6000 comp rx=0 tx=0).stop 2024-03-22T04:12:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49c1f7740 msgr2=0x7fe49c157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49c1f7740 0x7fe49c157ba0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0b6130 tx=0x7fe494007bc0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 shutdown_connections 2024-03-22T04:12:27.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe47c067b20 0x7fe47c069fe0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe49c1f7740 0x7fe49c157ba0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe49c149a00 0x7fe49c157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 --2- 172.21.15.86:0/1333323071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe484005d90 0x7fe49c15eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 >> 172.21.15.86:0/1333323071 conn(0x7fe49c0cc940 msgr2=0x7fe49c0b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 shutdown_connections 2024-03-22T04:12:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.230+0000 7fe4a3e00640 1 -- 172.21.15.86:0/1333323071 wait complete. 2024-03-22T04:12:27.251 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-22T04:12:27.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:12:27.301 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:12:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f572a400640 1 Processor -- start 2024-03-22T04:12:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f572a400640 1 -- start start 2024-03-22T04:12:27.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f572a400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f57240ad330 0x7f572415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f572a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f57240ad710 0x7f5724142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f572a400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f572415c7c0 con 0x7f572406ef40 2024-03-22T04:12:27.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f572a400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f572406dea0 con 0x7f57240ad710 2024-03-22T04:12:27.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f572a400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f572406e180 con 0x7f57240ad330 2024-03-22T04:12:27.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f5722a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f572406ef40 0x7f572413fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55834/0 (socket says 172.21.15.86:55834) 2024-03-22T04:12:27.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f5722a00640 1 -- 172.21.15.86:0/1134917067 learned_addr learned my addr 172.21.15.86:0/1134917067 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:27.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f5723e00640 1 --2- 172.21.15.86:0/1134917067 >> v2:172.21.15.86:3300/0 conn(0x7f57240ad710 0x7f5724142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f5723400640 1 --2- 172.21.15.86:0/1134917067 >> v2:172.21.15.196:3300/0 conn(0x7f57240ad330 0x7f572415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f5723e00640 1 -- 172.21.15.86:0/1134917067 >> v2:172.21.15.196:3300/0 conn(0x7f57240ad330 msgr2=0x7f572415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f5723e00640 1 --2- 172.21.15.86:0/1134917067 >> v2:172.21.15.196:3300/0 conn(0x7f57240ad330 0x7f572415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f5723e00640 1 -- 172.21.15.86:0/1134917067 >> v1:172.21.15.86:6789/0 conn(0x7f572406ef40 legacy=0x7f572413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.386+0000 7f5723e00640 1 -- 172.21.15.86:0/1134917067 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572406e460 con 0x7f57240ad710 2024-03-22T04:12:27.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f5723e00640 1 --2- 172.21.15.86:0/1134917067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad710 0x7f5724142690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f572406d080 tx=0x7f571c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fc5c97d2564f42 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:27.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f5722000640 1 -- 172.21.15.86:0/1134917067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0b2070 con 0x7f57240ad710 2024-03-22T04:12:27.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f5722000640 1 -- 172.21.15.86:0/1134917067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c0a59c0 con 0x7f57240ad710 2024-03-22T04:12:27.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f5722000640 1 -- 172.21.15.86:0/1134917067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0a5d90 con 0x7f57240ad710 2024-03-22T04:12:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f572a400640 1 -- 172.21.15.86:0/1134917067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad710 msgr2=0x7f5724142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f572a400640 1 --2- 172.21.15.86:0/1134917067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad710 0x7f5724142690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f572406d080 tx=0x7f571c01c900 comp rx=0 tx=0).stop 2024-03-22T04:12:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f572a400640 1 -- 172.21.15.86:0/1134917067 shutdown_connections 2024-03-22T04:12:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f572a400640 1 --2- 172.21.15.86:0/1134917067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad710 0x7f5724142690 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f572a400640 1 --2- 172.21.15.86:0/1134917067 >> v2:172.21.15.196:3300/0 conn(0x7f57240ad330 0x7f572415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f572a400640 1 -- 172.21.15.86:0/1134917067 >> 172.21.15.86:0/1134917067 conn(0x7f57240b1df0 msgr2=0x7f57240b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f572a400640 1 -- 172.21.15.86:0/1134917067 shutdown_connections 2024-03-22T04:12:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.390+0000 7f572a400640 1 -- 172.21.15.86:0/1134917067 wait complete. 2024-03-22T04:12:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 Processor -- start 2024-03-22T04:12:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 -- start start 2024-03-22T04:12:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f572406ef40 0x7f57241507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad330 0x7f57240b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57240ad710 0x7f57240b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57241690e0 con 0x7f57240ad330 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5724168f60 con 0x7f57240ad710 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5724169260 con 0x7f572406ef40 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad330 0x7f57240b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad330 0x7f57240b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58110/0 (socket says 172.21.15.86:58110) 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 -- 172.21.15.86:0/2737460141 learned_addr learned my addr 172.21.15.86:0/2737460141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5723e00640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57240ad710 0x7f57240b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5723400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f572406ef40 0x7f57241507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 -- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f572406ef40 msgr2=0x7f57241507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f572406ef40 0x7f57241507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 -- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57240ad710 msgr2=0x7f57240b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:27.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57240ad710 0x7f57240b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:27.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 -- 172.21.15.86:0/2737460141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572415c1c0 con 0x7f57240ad330 2024-03-22T04:12:27.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5723400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f572406ef40 0x7f57241507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:27.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f5722a00640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad330 0x7f57240b3960 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f571400c450 tx=0x7f571400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f571b400640 1 -- 172.21.15.86:0/2737460141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714017070 con 0x7f57240ad330 2024-03-22T04:12:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f571b400640 1 -- 172.21.15.86:0/2737460141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5714005cf0 con 0x7f57240ad330 2024-03-22T04:12:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f571b400640 1 -- 172.21.15.86:0/2737460141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714007290 con 0x7f57240ad330 2024-03-22T04:12:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572415b260 con 0x7f57240ad330 2024-03-22T04:12:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5724150d10 con 0x7f57240ad330 2024-03-22T04:12:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.394+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5724068650 con 0x7f57240ad330 2024-03-22T04:12:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.398+0000 7f571b400640 1 -- 172.21.15.86:0/2737460141 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f571401a070 con 0x7f57240ad330 2024-03-22T04:12:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.398+0000 7f571b400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f570006bf30 0x7f570006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.398+0000 7f571b400640 1 -- 172.21.15.86:0/2737460141 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f5714084fd0 con 0x7f57240ad330 2024-03-22T04:12:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.398+0000 7f5723400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f570006bf30 0x7f570006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.398+0000 7f5723400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f570006bf30 0x7f570006e3f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f570c007a70 tx=0x7f570c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.402+0000 7f571b400640 1 -- 172.21.15.86:0/2737460141 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f57140554f0 con 0x7f57240ad330 2024-03-22T04:12:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:27.614+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f57240af8a0 con 0x7f57240ad330 2024-03-22T04:12:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.562+0000 7f571b400640 1 -- 172.21.15.86:0/2737460141 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v134) ==== 152+0+0 (secure 0 0 0) 0x7f5714059590 con 0x7f57240ad330 2024-03-22T04:12:28.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f570006bf30 msgr2=0x7f570006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:28.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f570006bf30 0x7f570006e3f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f570c007a70 tx=0x7f570c008040 comp rx=0 tx=0).stop 2024-03-22T04:12:28.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad330 msgr2=0x7f57240b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:28.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad330 0x7f57240b3960 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f571400c450 tx=0x7f571400c910 comp rx=0 tx=0).stop 2024-03-22T04:12:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 shutdown_connections 2024-03-22T04:12:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f570006bf30 0x7f570006e3f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57240ad710 0x7f57240b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57240ad330 0x7f57240b3960 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 --2- 172.21.15.86:0/2737460141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f572406ef40 0x7f57241507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 >> 172.21.15.86:0/2737460141 conn(0x7f57240b1df0 msgr2=0x7f57241558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 shutdown_connections 2024-03-22T04:12:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.566+0000 7f572a400640 1 -- 172.21.15.86:0/2737460141 wait complete. 2024-03-22T04:12:28.594 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-22T04:12:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.778+0000 7f55a0600640 1 Processor -- start 2024-03-22T04:12:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.778+0000 7f55a0600640 1 -- start start 2024-03-22T04:12:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.778+0000 7f55a0600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f55980b2360 0x7f55980b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.778+0000 7f55a0600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f559815d0b0 0x7f55980b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.778+0000 7f55a0600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559815d710 con 0x7f55980b3430 2024-03-22T04:12:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.778+0000 7f55a0600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f55980b2740 con 0x7f559815d0b0 2024-03-22T04:12:28.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.778+0000 7f55a0600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f559815d4b0 con 0x7f55980b2360 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f559815d0b0 0x7f55980b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559d200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f55980b3430 0x7f559815c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55846/0 (socket says 172.21.15.86:55846) 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559d200640 1 -- 172.21.15.86:0/3271861359 learned_addr learned my addr 172.21.15.86:0/3271861359 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559dc00640 1 --2- 172.21.15.86:0/3271861359 >> v2:172.21.15.196:3300/0 conn(0x7f55980b2360 0x7f55980b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559e600640 1 -- 172.21.15.86:0/3271861359 >> v2:172.21.15.196:3300/0 conn(0x7f55980b2360 msgr2=0x7f55980b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559e600640 1 --2- 172.21.15.86:0/3271861359 >> v2:172.21.15.196:3300/0 conn(0x7f55980b2360 0x7f55980b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559e600640 1 -- 172.21.15.86:0/3271861359 >> v1:172.21.15.86:6789/0 conn(0x7f55980b3430 legacy=0x7f559815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f5597e00640 1 -- 172.21.15.86:0/3271861359 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 784969808 0 0) 0x7f559815d710 con 0x7f55980b3430 2024-03-22T04:12:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559e600640 1 -- 172.21.15.86:0/3271861359 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559815fe40 con 0x7f559815d0b0 2024-03-22T04:12:28.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f559e600640 1 --2- 172.21.15.86:0/3271861359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559815d0b0 0x7f55980b4d80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f55980683e0 tx=0x7f559001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d581c15a1d765d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:28.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f5597e00640 1 -- 172.21.15.86:0/3271861359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900b2070 con 0x7f559815d0b0 2024-03-22T04:12:28.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f5597e00640 1 -- 172.21.15.86:0/3271861359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55900a58e0 con 0x7f559815d0b0 2024-03-22T04:12:28.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.782+0000 7f5597e00640 1 -- 172.21.15.86:0/3271861359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900a5cb0 con 0x7f559815d0b0 2024-03-22T04:12:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- 172.21.15.86:0/3271861359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559815d0b0 msgr2=0x7f55980b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 --2- 172.21.15.86:0/3271861359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559815d0b0 0x7f55980b4d80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f55980683e0 tx=0x7f559001c930 comp rx=0 tx=0).stop 2024-03-22T04:12:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- 172.21.15.86:0/3271861359 shutdown_connections 2024-03-22T04:12:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 --2- 172.21.15.86:0/3271861359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f559815d0b0 0x7f55980b4d80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 --2- 172.21.15.86:0/3271861359 >> v2:172.21.15.196:3300/0 conn(0x7f55980b2360 0x7f55980b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- 172.21.15.86:0/3271861359 >> 172.21.15.86:0/3271861359 conn(0x7f55980cc940 msgr2=0x7f55980ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- 172.21.15.86:0/3271861359 shutdown_connections 2024-03-22T04:12:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- 172.21.15.86:0/3271861359 wait complete. 2024-03-22T04:12:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 Processor -- start 2024-03-22T04:12:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- start start 2024-03-22T04:12:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55980b2360 0x7f559814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55980b3430 0x7f559814d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559815d0b0 0x7f5598072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5598160c00 con 0x7f55980b3430 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55980b5ad0 con 0x7f559815d0b0 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f55a0600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f559814f360 con 0x7f55980b2360 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55980b3430 0x7f559814d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55980b3430 0x7f559814d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58126/0 (socket says 172.21.15.86:58126) 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 -- 172.21.15.86:0/79132159 learned_addr learned my addr 172.21.15.86:0/79132159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559e600640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559815d0b0 0x7f5598072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559dc00640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55980b2360 0x7f559814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 -- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55980b2360 msgr2=0x7f559814d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55980b2360 0x7f559814d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 -- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559815d0b0 msgr2=0x7f5598072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559815d0b0 0x7f5598072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 -- 172.21.15.86:0/79132159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559814f080 con 0x7f55980b3430 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559dc00640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55980b2360 0x7f559814d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.786+0000 7f559d200640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55980b3430 0x7f559814d8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f55840011f0 tx=0x7f558400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f5595600640 1 -- 172.21.15.86:0/79132159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584018070 con 0x7f55980b3430 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f5595600640 1 -- 172.21.15.86:0/79132159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55840074e0 con 0x7f55980b3430 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559815ea10 con 0x7f55980b3430 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f5595600640 1 -- 172.21.15.86:0/79132159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584004a10 con 0x7f55980b3430 2024-03-22T04:12:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559814ed10 con 0x7f55980b3430 2024-03-22T04:12:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5598074470 con 0x7f55980b3430 2024-03-22T04:12:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f5595600640 1 -- 172.21.15.86:0/79132159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f558401b070 con 0x7f55980b3430 2024-03-22T04:12:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f5595600640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5578067b20 0x7f5578069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f5595600640 1 -- 172.21.15.86:0/79132159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f5584085080 con 0x7f55980b3430 2024-03-22T04:12:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f559dc00640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5578067b20 0x7f5578069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.790+0000 7f559dc00640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5578067b20 0x7f5578069fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f558c007a70 tx=0x7f558c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:28.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.794+0000 7f5595600640 1 -- 172.21.15.86:0/79132159 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f55840555a0 con 0x7f55980b3430 2024-03-22T04:12:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.950+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f55980af310 con 0x7f55980b3430 2024-03-22T04:12:28.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.954+0000 7f5595600640 1 -- 172.21.15.86:0/79132159 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f5584059640 con 0x7f55980b3430 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5578067b20 msgr2=0x7f5578069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5578067b20 0x7f5578069fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f558c007a70 tx=0x7f558c008040 comp rx=0 tx=0).stop 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55980b3430 msgr2=0x7f559814d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55980b3430 0x7f559814d8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f55840011f0 tx=0x7f558400c9b0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 shutdown_connections 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5578067b20 0x7f5578069fe0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f559815d0b0 0x7f5598072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55980b3430 0x7f559814d8f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 --2- 172.21.15.86:0/79132159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55980b2360 0x7f559814d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 >> 172.21.15.86:0/79132159 conn(0x7f55980cc940 msgr2=0x7f5598155710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 shutdown_connections 2024-03-22T04:12:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:28.958+0000 7f55a0600640 1 -- 172.21.15.86:0/79132159 wait complete. 2024-03-22T04:12:28.979 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:12:28.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:12:29.065 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:12:29.164 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:12:29.164 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:12:29.164 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:29.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:12:29.173 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:12:29.173 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:29.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:12:29.182 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:12:29.182 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:29.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:12:29.221 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:12:29.222 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:29.237 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:12:29.237 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-22T04:12:29.238 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.394+0000 7f1e57400640 1 Processor -- start 2024-03-22T04:12:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.394+0000 7f1e57400640 1 -- start start 2024-03-22T04:12:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.394+0000 7f1e57400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e50161880 0x7f1e500b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.394+0000 7f1e57400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e500b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.394+0000 7f1e57400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e5006e350 con 0x7f1e5006d700 2024-03-22T04:12:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.394+0000 7f1e57400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5015ee80 con 0x7f1e50161880 2024-03-22T04:12:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.394+0000 7f1e57400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5006e630 con 0x7f1e50161c60 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e47e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e500b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e54a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e50161880 0x7f1e500b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e47e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e500b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:51000/0 (socket says 172.21.15.86:51000) 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e54a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e50161880 0x7f1e500b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:57990/0 (socket says 172.21.15.86:57990) 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e55400640 1 --1- 172.21.15.86:0/3986395530 >> v1:172.21.15.196:6789/0 conn(0x7f1e5006d700 0x7f1e5006db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:56454/0 (socket says 172.21.15.86:56454) 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e47e00640 1 -- 172.21.15.86:0/3986395530 learned_addr learned my addr 172.21.15.86:0/3986395530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e47e00640 1 -- 172.21.15.86:0/3986395530 >> v2:172.21.15.196:3300/0 conn(0x7f1e50161880 msgr2=0x7f1e500b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e47e00640 1 --2- 172.21.15.86:0/3986395530 >> v2:172.21.15.196:3300/0 conn(0x7f1e50161880 0x7f1e500b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e47e00640 1 -- 172.21.15.86:0/3986395530 >> v1:172.21.15.196:6789/0 conn(0x7f1e5006d700 legacy=0x7f1e5006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e47e00640 1 -- 172.21.15.86:0/3986395530 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5006e910 con 0x7f1e50161c60 2024-03-22T04:12:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e47e00640 1 --2- 172.21.15.86:0/3986395530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e500b3630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009870 tx=0x7f1e3801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66f3ef629631f14 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.398+0000 7f1e4fe00640 1 -- 172.21.15.86:0/3986395530 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380aa600 con 0x7f1e50161c60 2024-03-22T04:12:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e4fe00640 1 -- 172.21.15.86:0/3986395530 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e380af070 con 0x7f1e50161c60 2024-03-22T04:12:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e4fe00640 1 -- 172.21.15.86:0/3986395530 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380aa9c0 con 0x7f1e50161c60 2024-03-22T04:12:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- 172.21.15.86:0/3986395530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 msgr2=0x7f1e500b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 --2- 172.21.15.86:0/3986395530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e500b3630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009870 tx=0x7f1e3801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- 172.21.15.86:0/3986395530 shutdown_connections 2024-03-22T04:12:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 --2- 172.21.15.86:0/3986395530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e500b3630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 --2- 172.21.15.86:0/3986395530 >> v2:172.21.15.196:3300/0 conn(0x7f1e50161880 0x7f1e500b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- 172.21.15.86:0/3986395530 >> 172.21.15.86:0/3986395530 conn(0x7f1e500b1df0 msgr2=0x7f1e500b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- 172.21.15.86:0/3986395530 shutdown_connections 2024-03-22T04:12:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- 172.21.15.86:0/3986395530 wait complete. 2024-03-22T04:12:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 Processor -- start 2024-03-22T04:12:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- start start 2024-03-22T04:12:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5006d700 0x7f1e50099b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e50161880 0x7f1e500ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e50152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50160950 con 0x7f1e5006d700 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501607d0 con 0x7f1e50161880 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e57400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1e50160ad0 con 0x7f1e50161c60 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e54a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5006d700 0x7f1e50099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e54a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5006d700 0x7f1e50099b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58140/0 (socket says 172.21.15.86:58140) 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e54a00640 1 -- 172.21.15.86:0/1139952406 learned_addr learned my addr 172.21.15.86:0/1139952406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e47e00640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e50161880 0x7f1e500ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e55400640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e50152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e54a00640 1 -- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 msgr2=0x7f1e50152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e54a00640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e50152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e54a00640 1 -- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e50161880 msgr2=0x7f1e500ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e54a00640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e50161880 0x7f1e500ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.402+0000 7f1e54a00640 1 -- 172.21.15.86:0/1139952406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e500b4110 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e47e00640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e50161880 0x7f1e500ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e54a00640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5006d700 0x7f1e50099b40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000e960 tx=0x7f1e4000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e4d600640 1 -- 172.21.15.86:0/1139952406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40017070 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e4d600640 1 -- 172.21.15.86:0/1139952406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4000ca00 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e500adc10 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e4d600640 1 -- 172.21.15.86:0/1139952406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000cdb0 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e500ae1d0 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e50068650 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e4d600640 1 -- 172.21.15.86:0/1139952406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1e4001a070 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e4d600640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e2c067b20 0x7f1e2c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e4d600640 1 -- 172.21.15.86:0/1139952406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f1e40085ef0 con 0x7f1e5006d700 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.406+0000 7f1e47e00640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e2c067b20 0x7f1e2c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.410+0000 7f1e47e00640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e2c067b20 0x7f1e2c069fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ae000 tx=0x7f1e380a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:29.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.410+0000 7f1e4d600640 1 -- 172.21.15.86:0/1139952406 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1e40056950 con 0x7f1e5006d700 2024-03-22T04:12:29.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.598+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e500b4a80 con 0x7f1e5006d700 2024-03-22T04:12:29.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.598+0000 7f1e4d600640 1 -- 172.21.15.86:0/1139952406 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3875 (secure 0 0 0) 0x7f1e40056250 con 0x7f1e5006d700 2024-03-22T04:12:29.600 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:29.601 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3239982750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3239982750},{"type":"v1","addr":"172.21.15.86:6836","nonce":3239982750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3724823400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3724823400},{"type":"v1","addr":"172.21.15.196:6834","nonce":3724823400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3153694956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3153694956},{"type":"v1","addr":"172.21.15.86:6837","nonce":3153694956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5641,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2884729845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2884729845},{"type":"v1","addr":"172.21.15.196:6835","nonce":2884729845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-03-22T04:12:29.601 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 19 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.598+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e2c067b20 msgr2=0x7f1e2c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.598+0000 7f1e57400640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e2c067b20 0x7f1e2c069fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ae000 tx=0x7f1e380a7000 comp rx=0 tx=0).stop 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5006d700 msgr2=0x7f1e50099b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5006d700 0x7f1e50099b40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000e960 tx=0x7f1e4000ee20 comp rx=0 tx=0).stop 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 shutdown_connections 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e2c067b20 0x7f1e2c069fe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e50161880 0x7f1e500ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e50161c60 0x7f1e50152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 --2- 172.21.15.86:0/1139952406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5006d700 0x7f1e50099b40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 >> 172.21.15.86:0/1139952406 conn(0x7f1e500b1df0 msgr2=0x7f1e500cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 shutdown_connections 2024-03-22T04:12:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.602+0000 7f1e57400640 1 -- 172.21.15.86:0/1139952406 wait complete. 2024-03-22T04:12:29.622 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-22T04:12:29.622 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-22T04:12:29.622 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-22T04:12:29.622 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:12:29.623 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-22T04:12:29.623 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-22T04:12:29.623 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-22T04:12:29.623 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:12:29.623 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-22T04:12:29.623 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-22T04:12:29.624 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-22T04:12:29.624 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:12:29.624 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-22T04:12:29.624 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-22T04:12:29.624 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-22T04:12:29.624 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:12:29.634 INFO:tasks.ceph.mds.a:Stopped 2024-03-22T04:12:29.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-22T04:12:29.636 INFO:tasks.ceph.mds.c:Stopped 2024-03-22T04:12:29.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-22T04:12:29.644 INFO:tasks.ceph.mds.b:Stopped 2024-03-22T04:12:29.645 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-22T04:12:29.645 INFO:tasks.ceph.mds.d:Stopped 2024-03-22T04:12:29.645 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-22T04:12:29.680 INFO:tasks.ceph.mds.a:Started 2024-03-22T04:12:29.681 DEBUG:teuthology.parallel:result is None 2024-03-22T04:12:29.682 INFO:tasks.ceph.mds.c:Started 2024-03-22T04:12:29.683 DEBUG:teuthology.parallel:result is None 2024-03-22T04:12:29.688 INFO:tasks.ceph.mds.b:Started 2024-03-22T04:12:29.689 DEBUG:teuthology.parallel:result is None 2024-03-22T04:12:29.690 INFO:tasks.ceph.mds.d:Started 2024-03-22T04:12:29.690 DEBUG:teuthology.parallel:result is None 2024-03-22T04:12:29.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-22T04:12:29.766 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:12:29.768 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-03-22T04:12:29.780 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:12:29.780 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:12:29.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895c00640 1 Processor -- start 2024-03-22T04:12:29.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895c00640 1 -- start start 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f28900affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f289006bd70 0x7f289006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f289015ebb0 con 0x7f28900b0680 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f28900b04e0 con 0x7f289006bd70 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f289015ee90 con 0x7f289015e3a0 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f288f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f28900b0680 0x7f289006b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55864/0 (socket says 172.21.15.86:55864) 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f288f400640 1 -- 172.21.15.86:0/2721939712 learned_addr learned my addr 172.21.15.86:0/2721939712 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895200640 1 --2- 172.21.15.86:0/2721939712 >> v2:172.21.15.196:3300/0 conn(0x7f289006bd70 0x7f289006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f288fe00640 1 --2- 172.21.15.86:0/2721939712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f28900affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f288ea00640 1 -- 172.21.15.86:0/2721939712 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 134035619 0 0) 0x7f289015ebb0 con 0x7f28900b0680 2024-03-22T04:12:29.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895200640 1 -- 172.21.15.86:0/2721939712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 msgr2=0x7f28900affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:29.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895200640 1 --2- 172.21.15.86:0/2721939712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f28900affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895200640 1 -- 172.21.15.86:0/2721939712 >> v1:172.21.15.86:6789/0 conn(0x7f28900b0680 legacy=0x7f289006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.834+0000 7f2895200640 1 -- 172.21.15.86:0/2721939712 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289015f170 con 0x7f289006bd70 2024-03-22T04:12:29.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895200640 1 --2- 172.21.15.86:0/2721939712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f289006bd70 0x7f289006fb40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2888009d90 tx=0x7f288801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a31baa564efc2e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:29.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f288ea00640 1 -- 172.21.15.86:0/2721939712 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880b2070 con 0x7f289006bd70 2024-03-22T04:12:29.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f288ea00640 1 -- 172.21.15.86:0/2721939712 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28880a57a0 con 0x7f289006bd70 2024-03-22T04:12:29.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f288ea00640 1 -- 172.21.15.86:0/2721939712 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880a5b70 con 0x7f289006bd70 2024-03-22T04:12:29.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895c00640 1 -- 172.21.15.86:0/2721939712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f289006bd70 msgr2=0x7f289006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895c00640 1 --2- 172.21.15.86:0/2721939712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f289006bd70 0x7f289006fb40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2888009d90 tx=0x7f288801c8e0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895c00640 1 -- 172.21.15.86:0/2721939712 shutdown_connections 2024-03-22T04:12:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895c00640 1 --2- 172.21.15.86:0/2721939712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f289006bd70 0x7f289006fb40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895c00640 1 --2- 172.21.15.86:0/2721939712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f28900affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895c00640 1 -- 172.21.15.86:0/2721939712 >> 172.21.15.86:0/2721939712 conn(0x7f28900d1000 msgr2=0x7f28900d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895c00640 1 -- 172.21.15.86:0/2721939712 shutdown_connections 2024-03-22T04:12:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.838+0000 7f2895c00640 1 -- 172.21.15.86:0/2721939712 wait complete. 2024-03-22T04:12:29.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 Processor -- start 2024-03-22T04:12:29.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 -- start start 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289006bd70 0x7f2890160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28900b0680 0x7f2890156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f2890157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2890070ae0 con 0x7f289006bd70 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f289015e990 con 0x7f28900b0680 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2890070c60 con 0x7f289015e3a0 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289006bd70 0x7f2890160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289006bd70 0x7f2890160760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58186/0 (socket says 172.21.15.86:58186) 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 -- 172.21.15.86:0/1079277210 learned_addr learned my addr 172.21.15.86:0/1079277210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895200640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f2890157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 -- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 msgr2=0x7f2890157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288f400640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28900b0680 0x7f2890156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f2890157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 -- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28900b0680 msgr2=0x7f2890156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:29.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28900b0680 0x7f2890156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:29.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 -- 172.21.15.86:0/1079277210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28900b5880 con 0x7f289006bd70 2024-03-22T04:12:29.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895200640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f2890157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:29.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288f400640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28900b0680 0x7f2890156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:12:29.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f288fe00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289006bd70 0x7f2890160760 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f288000e870 tx=0x7f288000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2887e00640 1 -- 172.21.15.86:0/1079277210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880017070 con 0x7f289006bd70 2024-03-22T04:12:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2887e00640 1 -- 172.21.15.86:0/1079277210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f288000c940 con 0x7f289006bd70 2024-03-22T04:12:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28901556b0 con 0x7f289006bd70 2024-03-22T04:12:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2887e00640 1 -- 172.21.15.86:0/1079277210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288000ccf0 con 0x7f289006bd70 2024-03-22T04:12:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.842+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2890155c70 con 0x7f289006bd70 2024-03-22T04:12:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.846+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2890074470 con 0x7f289006bd70 2024-03-22T04:12:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.846+0000 7f2887e00640 1 -- 172.21.15.86:0/1079277210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f288001a070 con 0x7f289006bd70 2024-03-22T04:12:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.846+0000 7f2887e00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2870067b20 0x7f2870069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.846+0000 7f2887e00640 1 -- 172.21.15.86:0/1079277210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f2880085c10 con 0x7f289006bd70 2024-03-22T04:12:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.846+0000 7f288f400640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2870067b20 0x7f2870069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:29.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.846+0000 7f288f400640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2870067b20 0x7f2870069fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28901fa4a0 tx=0x7f287c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:29.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:29.850+0000 7f2887e00640 1 -- 172.21.15.86:0/1079277210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2880056820 con 0x7f289006bd70 2024-03-22T04:12:30.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.018+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f28900d1b20 con 0x7f289006bd70 2024-03-22T04:12:30.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.582+0000 7f2887e00640 1 -- 172.21.15.86:0/1079277210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) ==== 97+0+0 (secure 0 0 0) 0x7f2880056120 con 0x7f289006bd70 2024-03-22T04:12:30.585 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-03-22T04:12:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2870067b20 msgr2=0x7f2870069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2870067b20 0x7f2870069fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28901fa4a0 tx=0x7f287c009040 comp rx=0 tx=0).stop 2024-03-22T04:12:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289006bd70 msgr2=0x7f2890160760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289006bd70 0x7f2890160760 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f288000e870 tx=0x7f288000ec70 comp rx=0 tx=0).stop 2024-03-22T04:12:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 shutdown_connections 2024-03-22T04:12:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2870067b20 0x7f2870069fe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28900b0680 0x7f2890156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f289015e3a0 0x7f2890157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 --2- 172.21.15.86:0/1079277210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289006bd70 0x7f2890160760 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 >> 172.21.15.86:0/1079277210 conn(0x7f28900d1000 msgr2=0x7f289006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 shutdown_connections 2024-03-22T04:12:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.586+0000 7f2895c00640 1 -- 172.21.15.86:0/1079277210 wait complete. 2024-03-22T04:12:30.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-22T04:12:30.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.742+0000 7fb225c00640 1 Processor -- start 2024-03-22T04:12:30.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb225c00640 1 -- start start 2024-03-22T04:12:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb225c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb22015e630 0x7fb2200cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb225c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb22006e2f0 0x7fb2200d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb225c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2201623d0 con 0x7fb22006cc80 2024-03-22T04:12:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb225c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb2201620f0 con 0x7fb22006e2f0 2024-03-22T04:12:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb225c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb22015ec10 con 0x7fb22015e630 2024-03-22T04:12:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb216000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb22006e2f0 0x7fb2200d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb21f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb22006cc80 0x7fb22006d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55870/0 (socket says 172.21.15.86:55870) 2024-03-22T04:12:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb21f400640 1 -- 172.21.15.86:0/1939519449 learned_addr learned my addr 172.21.15.86:0/1939519449 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:30.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb21ea00640 1 --2- 172.21.15.86:0/1939519449 >> v2:172.21.15.196:3300/0 conn(0x7fb22015e630 0x7fb2200cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:30.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb216000640 1 -- 172.21.15.86:0/1939519449 >> v2:172.21.15.196:3300/0 conn(0x7fb22015e630 msgr2=0x7fb2200cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:30.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb216000640 1 --2- 172.21.15.86:0/1939519449 >> v2:172.21.15.196:3300/0 conn(0x7fb22015e630 0x7fb2200cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb216000640 1 -- 172.21.15.86:0/1939519449 >> v1:172.21.15.86:6789/0 conn(0x7fb22006cc80 legacy=0x7fb22006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:30.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb216000640 1 -- 172.21.15.86:0/1939519449 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22015eef0 con 0x7fb22006e2f0 2024-03-22T04:12:30.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.746+0000 7fb21e000640 1 -- 172.21.15.86:0/1939519449 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894308453 0 0) 0x7fb2201623d0 con 0x7fb22006cc80 2024-03-22T04:12:30.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb216000640 1 --2- 172.21.15.86:0/1939519449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006e2f0 0x7fb2200d03b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb220072430 tx=0x7fb20801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e8e949ccb4ba08 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:30.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb21e000640 1 -- 172.21.15.86:0/1939519449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2080b0070 con 0x7fb22006e2f0 2024-03-22T04:12:30.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb21e000640 1 -- 172.21.15.86:0/1939519449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2080ab600 con 0x7fb22006e2f0 2024-03-22T04:12:30.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb21e000640 1 -- 172.21.15.86:0/1939519449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2080ab9d0 con 0x7fb22006e2f0 2024-03-22T04:12:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 -- 172.21.15.86:0/1939519449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006e2f0 msgr2=0x7fb2200d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 --2- 172.21.15.86:0/1939519449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006e2f0 0x7fb2200d03b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb220072430 tx=0x7fb20801c480 comp rx=0 tx=0).stop 2024-03-22T04:12:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 -- 172.21.15.86:0/1939519449 shutdown_connections 2024-03-22T04:12:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 --2- 172.21.15.86:0/1939519449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006e2f0 0x7fb2200d03b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 --2- 172.21.15.86:0/1939519449 >> v2:172.21.15.196:3300/0 conn(0x7fb22015e630 0x7fb2200cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 -- 172.21.15.86:0/1939519449 >> 172.21.15.86:0/1939519449 conn(0x7fb22006f910 msgr2=0x7fb22006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 -- 172.21.15.86:0/1939519449 shutdown_connections 2024-03-22T04:12:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 -- 172.21.15.86:0/1939519449 wait complete. 2024-03-22T04:12:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 Processor -- start 2024-03-22T04:12:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.750+0000 7fb225c00640 1 -- start start 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cc80 0x7fb2201614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006e2f0 0x7fb2201619e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22015e630 0x7fb22015a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200cfaf0 con 0x7fb22006cc80 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb22015f9d0 con 0x7fb22015e630 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb2200cfc70 con 0x7fb22006e2f0 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cc80 0x7fb2201614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cc80 0x7fb2201614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58198/0 (socket says 172.21.15.86:58198) 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 -- 172.21.15.86:0/1356235533 learned_addr learned my addr 172.21.15.86:0/1356235533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21f400640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22015e630 0x7fb22015a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb216000640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006e2f0 0x7fb2201619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 -- 172.21.15.86:0/1356235533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006e2f0 msgr2=0x7fb2201619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006e2f0 0x7fb2201619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 -- 172.21.15.86:0/1356235533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22015e630 msgr2=0x7fb22015a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22015e630 0x7fb22015a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 -- 172.21.15.86:0/1356235533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200b5630 con 0x7fb22006cc80 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb21ea00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cc80 0x7fb2201614a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb210007e40 tx=0x7fb21000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb217400640 1 -- 172.21.15.86:0/1356235533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210014620 con 0x7fb22006cc80 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb217400640 1 -- 172.21.15.86:0/1356235533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210019070 con 0x7fb22006cc80 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb217400640 1 -- 172.21.15.86:0/1356235533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210014a00 con 0x7fb22006cc80 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2200b58d0 con 0x7fb22006cc80 2024-03-22T04:12:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2200b5e90 con 0x7fb22006cc80 2024-03-22T04:12:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb220068650 con 0x7fb22006cc80 2024-03-22T04:12:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.754+0000 7fb217400640 1 -- 172.21.15.86:0/1356235533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb210014ba0 con 0x7fb22006cc80 2024-03-22T04:12:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.758+0000 7fb217400640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1fc067b20 0x7fb1fc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:30.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.758+0000 7fb217400640 1 -- 172.21.15.86:0/1356235533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7fb210086690 con 0x7fb22006cc80 2024-03-22T04:12:30.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.758+0000 7fb216000640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1fc067b20 0x7fb1fc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:30.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.758+0000 7fb216000640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1fc067b20 0x7fb1fc069fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb2080ad000 tx=0x7fb2080a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.762+0000 7fb217400640 1 -- 172.21.15.86:0/1356235533 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb2100571c0 con 0x7fb22006cc80 2024-03-22T04:12:30.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.970+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb2200b6550 con 0x7fb22006cc80 2024-03-22T04:12:30.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.970+0000 7fb217400640 1 -- 172.21.15.86:0/1356235533 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fb210056ac0 con 0x7fb22006cc80 2024-03-22T04:12:30.972 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA==", 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw==", 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q==", 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:12:30.974 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg==", 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA==", 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA==", 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:12:30.976 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg==", 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA==", 2024-03-22T04:12:30.977 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ==", 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow==", 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg==", 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:12:30.979 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg==", 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-03-22T04:12:30.980 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA==", 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg==", 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w==", 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w==", 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.982 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB4BP1lvLoQORAADwVEr78V9aUc1YblO47Gaw==", 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:12:30.983 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g==", 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ==", 2024-03-22T04:12:30.984 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:12:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:12:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:12:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:12:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:30.985 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T04:12:30.985 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:12:30.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.970+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1fc067b20 msgr2=0x7fb1fc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:30.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.970+0000 7fb225c00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1fc067b20 0x7fb1fc069fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb2080ad000 tx=0x7fb2080a6000 comp rx=0 tx=0).stop 2024-03-22T04:12:30.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.970+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cc80 msgr2=0x7fb2201614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:30.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.970+0000 7fb225c00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cc80 0x7fb2201614a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb210007e40 tx=0x7fb21000e4b0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.974+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 shutdown_connections 2024-03-22T04:12:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.974+0000 7fb225c00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1fc067b20 0x7fb1fc069fe0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.974+0000 7fb225c00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22015e630 0x7fb22015a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.974+0000 7fb225c00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006e2f0 0x7fb2201619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.974+0000 7fb225c00640 1 --2- 172.21.15.86:0/1356235533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cc80 0x7fb2201614a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.974+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 >> 172.21.15.86:0/1356235533 conn(0x7fb22006f910 msgr2=0x7fb2200cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.974+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 shutdown_connections 2024-03-22T04:12:30.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:30.974+0000 7fb225c00640 1 -- 172.21.15.86:0/1356235533 wait complete. 2024-03-22T04:12:30.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.126+0000 7f75f4a00640 1 Processor -- start 2024-03-22T04:12:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.126+0000 7f75f4a00640 1 -- start start 2024-03-22T04:12:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.126+0000 7f75f4a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75f00ad330 0x7f75f015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.126+0000 7f75f4a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75f015c7c0 con 0x7f75f00ad710 2024-03-22T04:12:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.126+0000 7f75f4a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75f006dea0 con 0x7f75f006ef40 2024-03-22T04:12:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.126+0000 7f75f4a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f75f006e180 con 0x7f75f00ad330 2024-03-22T04:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.126+0000 7f75ede00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75f00ad330 0x7f75f015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ee800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f75f00ad710 0x7f75f01402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55874/0 (socket says 172.21.15.86:55874) 2024-03-22T04:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ee800640 1 -- 172.21.15.86:0/3677131227 learned_addr learned my addr 172.21.15.86:0/3677131227 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ed400640 1 -- 172.21.15.86:0/3677131227 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2400984013 0 0) 0x7f75f015c7c0 con 0x7f75f00ad710 2024-03-22T04:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ed400640 1 -- 172.21.15.86:0/3677131227 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75d4003530 con 0x7f75f00ad710 2024-03-22T04:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ede00640 1 -- 172.21.15.86:0/3677131227 >> v1:172.21.15.196:6789/0 conn(0x7f75f006ef40 legacy=0x7f75f013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ede00640 1 -- 172.21.15.86:0/3677131227 >> v1:172.21.15.86:6789/0 conn(0x7f75f00ad710 legacy=0x7f75f01402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ede00640 1 -- 172.21.15.86:0/3677131227 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f006e460 con 0x7f75f00ad330 2024-03-22T04:12:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ede00640 1 --2- 172.21.15.86:0/3677131227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f015d4a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097c0 tx=0x7f75e001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91c3ce675565932 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ed400640 1 -- 172.21.15.86:0/3677131227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e00b2070 con 0x7f75f00ad330 2024-03-22T04:12:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ed400640 1 -- 172.21.15.86:0/3677131227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e00ad680 con 0x7f75f00ad330 2024-03-22T04:12:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.130+0000 7f75ed400640 1 -- 172.21.15.86:0/3677131227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e00ada50 con 0x7f75f00ad330 2024-03-22T04:12:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- 172.21.15.86:0/3677131227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 msgr2=0x7f75f015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 --2- 172.21.15.86:0/3677131227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f015d4a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097c0 tx=0x7f75e001c470 comp rx=0 tx=0).stop 2024-03-22T04:12:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- 172.21.15.86:0/3677131227 shutdown_connections 2024-03-22T04:12:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 --2- 172.21.15.86:0/3677131227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f015d4a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- 172.21.15.86:0/3677131227 >> 172.21.15.86:0/3677131227 conn(0x7f75f00b1df0 msgr2=0x7f75f00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- 172.21.15.86:0/3677131227 shutdown_connections 2024-03-22T04:12:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- 172.21.15.86:0/3677131227 wait complete. 2024-03-22T04:12:31.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 Processor -- start 2024-03-22T04:12:31.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- start start 2024-03-22T04:12:31.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75f006ef40 0x7f75f015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f009ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75f00ad710 0x7f75f009b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75f016a780 con 0x7f75f00ad330 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75f006db10 con 0x7f75f006ef40 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75f4a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f75f016a900 con 0x7f75f00ad710 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f009ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75ede00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75f006ef40 0x7f75f015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f009ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58210/0 (socket says 172.21.15.86:58210) 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 -- 172.21.15.86:0/566800177 learned_addr learned my addr 172.21.15.86:0/566800177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75ee800640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75f00ad710 0x7f75f009b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 -- 172.21.15.86:0/566800177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75f00ad710 msgr2=0x7f75f009b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75f00ad710 0x7f75f009b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 -- 172.21.15.86:0/566800177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75f006ef40 msgr2=0x7f75f015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75f006ef40 0x7f75f015b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 -- 172.21.15.86:0/566800177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f0166e40 con 0x7f75f00ad330 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.134+0000 7f75e5400640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f009ccc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f75d800c420 tx=0x7f75d800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75e6a00640 1 -- 172.21.15.86:0/566800177 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d80052d0 con 0x7f75f00ad330 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75e6a00640 1 -- 172.21.15.86:0/566800177 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d8005470 con 0x7f75f00ad330 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75e6a00640 1 -- 172.21.15.86:0/566800177 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8007260 con 0x7f75f00ad330 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f009a830 con 0x7f75f00ad330 2024-03-22T04:12:31.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f009adf0 con 0x7f75f00ad330 2024-03-22T04:12:31.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f0068650 con 0x7f75f00ad330 2024-03-22T04:12:31.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75e6a00640 1 -- 172.21.15.86:0/566800177 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f75d8007400 con 0x7f75f00ad330 2024-03-22T04:12:31.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75e6a00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f75cc067b50 0x7f75cc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75e6a00640 1 -- 172.21.15.86:0/566800177 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f75d80851c0 con 0x7f75f00ad330 2024-03-22T04:12:31.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.138+0000 7f75ede00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f75cc067b50 0x7f75cc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.142+0000 7f75ede00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f75cc067b50 0x7f75cc06a010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f75e00af000 tx=0x7f75e00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.142+0000 7f75e6a00640 1 -- 172.21.15.86:0/566800177 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f75d80556a0 con 0x7f75f00ad330 2024-03-22T04:12:31.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.298+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75f0167d20 con 0x7f75f00ad330 2024-03-22T04:12:31.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.298+0000 7f75e6a00640 1 -- 172.21.15.86:0/566800177 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13164 (secure 0 0 0) 0x7f75d8059740 con 0x7f75f00ad330 2024-03-22T04:12:31.301 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:31.301 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:12:30.591956+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f75cc067b50 msgr2=0x7f75cc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f75cc067b50 0x7f75cc06a010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f75e00af000 tx=0x7f75e00a7000 comp rx=0 tx=0).stop 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 msgr2=0x7f75f009ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f009ccc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f75d800c420 tx=0x7f75d800c8e0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 shutdown_connections 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f75cc067b50 0x7f75cc06a010 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75f00ad710 0x7f75f009b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f00ad330 0x7f75f009ccc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 --2- 172.21.15.86:0/566800177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75f006ef40 0x7f75f015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 >> 172.21.15.86:0/566800177 conn(0x7f75f00b1df0 msgr2=0x7f75f00731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 shutdown_connections 2024-03-22T04:12:31.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.302+0000 7f75f4a00640 1 -- 172.21.15.86:0/566800177 wait complete. 2024-03-22T04:12:31.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:12:31.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:12:31.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.458+0000 7f8b19400640 1 Processor -- start 2024-03-22T04:12:31.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.458+0000 7f8b19400640 1 -- start start 2024-03-22T04:12:31.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.458+0000 7f8b19400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8b140c6f20 0x7f8b14072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.458+0000 7f8b19400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.458+0000 7f8b19400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b140cc7c0 con 0x7f8b140cb1d0 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.458+0000 7f8b19400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8b140cb720 con 0x7f8b140c6f20 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.458+0000 7f8b19400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8b140cba00 con 0x7f8b14162740 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b09e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8b140cb1d0 0x7f8b14162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55878/0 (socket says 172.21.15.86:55878) 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b09e00640 1 -- 172.21.15.86:0/2178106721 learned_addr learned my addr 172.21.15.86:0/2178106721 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b13200640 1 --2- 172.21.15.86:0/2178106721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b12800640 1 --2- 172.21.15.86:0/2178106721 >> v2:172.21.15.196:3300/0 conn(0x7f8b140c6f20 0x7f8b14072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b11e00640 1 -- 172.21.15.86:0/2178106721 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011985189 0 0) 0x7f8b140cc7c0 con 0x7f8b140cb1d0 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b11e00640 1 -- 172.21.15.86:0/2178106721 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8af8003530 con 0x7f8b140cb1d0 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b11e00640 1 -- 172.21.15.86:0/2178106721 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 793521373 0 0) 0x7f8af8003530 con 0x7f8b140cb1d0 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b13200640 1 -- 172.21.15.86:0/2178106721 >> v2:172.21.15.196:3300/0 conn(0x7f8b140c6f20 msgr2=0x7f8b14072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b13200640 1 --2- 172.21.15.86:0/2178106721 >> v2:172.21.15.196:3300/0 conn(0x7f8b140c6f20 0x7f8b14072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b13200640 1 -- 172.21.15.86:0/2178106721 >> v1:172.21.15.86:6789/0 conn(0x7f8b140cb1d0 legacy=0x7f8b14162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b13200640 1 -- 172.21.15.86:0/2178106721 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b140cbce0 con 0x7f8b14162740 2024-03-22T04:12:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.462+0000 7f8b13200640 1 --2- 172.21.15.86:0/2178106721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140c55f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0097c0 tx=0x7f8b0c01c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28305b30dca4da0a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b11e00640 1 -- 172.21.15.86:0/2178106721 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0b0070 con 0x7f8b14162740 2024-03-22T04:12:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b11e00640 1 -- 172.21.15.86:0/2178106721 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0c0ab400 con 0x7f8b14162740 2024-03-22T04:12:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b11e00640 1 -- 172.21.15.86:0/2178106721 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc001680 con 0x7f8b14162740 2024-03-22T04:12:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- 172.21.15.86:0/2178106721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 msgr2=0x7f8b140c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 --2- 172.21.15.86:0/2178106721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140c55f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0097c0 tx=0x7f8b0c01c390 comp rx=0 tx=0).stop 2024-03-22T04:12:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- 172.21.15.86:0/2178106721 shutdown_connections 2024-03-22T04:12:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 --2- 172.21.15.86:0/2178106721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140c55f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 --2- 172.21.15.86:0/2178106721 >> v2:172.21.15.196:3300/0 conn(0x7f8b140c6f20 0x7f8b14072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- 172.21.15.86:0/2178106721 >> 172.21.15.86:0/2178106721 conn(0x7f8b140d0730 msgr2=0x7f8b140d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:31.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- 172.21.15.86:0/2178106721 shutdown_connections 2024-03-22T04:12:31.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- 172.21.15.86:0/2178106721 wait complete. 2024-03-22T04:12:31.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 Processor -- start 2024-03-22T04:12:31.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- start start 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b140c6f20 0x7f8b1415e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140cb1d0 0x7f8b141f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b140c64c0 con 0x7f8b140c6f20 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b140c6340 con 0x7f8b140cb1d0 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b19400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8b140c6640 con 0x7f8b14162740 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b12800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b140c6f20 0x7f8b1415e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b12800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b140c6f20 0x7f8b1415e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58226/0 (socket says 172.21.15.86:58226) 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b12800640 1 -- 172.21.15.86:0/2448816681 learned_addr learned my addr 172.21.15.86:0/2448816681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b13200640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b09e00640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140cb1d0 0x7f8b141f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b12800640 1 -- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 msgr2=0x7f8b140cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b12800640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b12800640 1 -- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140cb1d0 msgr2=0x7f8b141f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b12800640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140cb1d0 0x7f8b141f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b12800640 1 -- 172.21.15.86:0/2448816681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b141442b0 con 0x7f8b140c6f20 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.466+0000 7f8b13200640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b12800640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b140c6f20 0x7f8b1415e440 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009db0 tx=0x7f8b0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b0b400640 1 -- 172.21.15.86:0/2448816681 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04018020 con 0x7f8b140c6f20 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b0b400640 1 -- 172.21.15.86:0/2448816681 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b04014480 con 0x7f8b140c6f20 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b14144590 con 0x7f8b140c6f20 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b0b400640 1 -- 172.21.15.86:0/2448816681 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04014850 con 0x7f8b140c6f20 2024-03-22T04:12:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b141f7300 con 0x7f8b140c6f20 2024-03-22T04:12:31.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b141f7110 con 0x7f8b140c6f20 2024-03-22T04:12:31.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b0b400640 1 -- 172.21.15.86:0/2448816681 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8b0401b020 con 0x7f8b140c6f20 2024-03-22T04:12:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b0b400640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b0b400640 1 -- 172.21.15.86:0/2448816681 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f8b04086930 con 0x7f8b140c6f20 2024-03-22T04:12:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.470+0000 7f8b09e00640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.474+0000 7f8b09e00640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8b14157ef0 tx=0x7f8afc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.474+0000 7f8b0b400640 1 -- 172.21.15.86:0/2448816681 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8b04022070 con 0x7f8b140c6f20 2024-03-22T04:12:31.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.638+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8b14074470 con 0x7f8b140c6f20 2024-03-22T04:12:31.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.638+0000 7f8b0b400640 1 -- 172.21.15.86:0/2448816681 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f8b04010040 con 0x7f8b140c6f20 2024-03-22T04:12:31.641 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:12:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 msgr2=0x7f8af0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8b14157ef0 tx=0x7f8afc009040 comp rx=0 tx=0).stop 2024-03-22T04:12:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b140c6f20 msgr2=0x7f8b1415e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b140c6f20 0x7f8b1415e440 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009db0 tx=0x7f8b0400c490 comp rx=0 tx=0).stop 2024-03-22T04:12:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 shutdown_connections 2024-03-22T04:12:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8af0067b20 0x7f8af0069fe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b140cb1d0 0x7f8b141f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b14162740 0x7f8b140cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 --2- 172.21.15.86:0/2448816681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b140c6f20 0x7f8b1415e440 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 >> 172.21.15.86:0/2448816681 conn(0x7f8b140d0730 msgr2=0x7f8b140d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 shutdown_connections 2024-03-22T04:12:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.642+0000 7f8b19400640 1 -- 172.21.15.86:0/2448816681 wait complete. 2024-03-22T04:12:31.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T04:12:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facfda00640 1 Processor -- start 2024-03-22T04:12:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facfda00640 1 -- start start 2024-03-22T04:12:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facfda00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7facf815b3f0 0x7facf80729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facfda00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7facf80cc3e0 0x7facf8072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facfda00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf8071920 con 0x7facf80b30f0 2024-03-22T04:12:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facfda00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7facf80716d0 con 0x7facf815b3f0 2024-03-22T04:12:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facfda00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7facf806d990 con 0x7facf80cc3e0 2024-03-22T04:12:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facf6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7facf815b3f0 0x7facf80729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facf6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7facf815b3f0 0x7facf80729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58242/0 (socket says 172.21.15.86:58242) 2024-03-22T04:12:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facf6a00640 1 -- 172.21.15.86:0/625931658 learned_addr learned my addr 172.21.15.86:0/625931658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facefe00640 1 --2- 172.21.15.86:0/625931658 >> v2:172.21.15.196:3300/0 conn(0x7facf80cc3e0 0x7facf8072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facf6a00640 1 -- 172.21.15.86:0/625931658 >> v2:172.21.15.196:3300/0 conn(0x7facf80cc3e0 msgr2=0x7facf8072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facf6a00640 1 --2- 172.21.15.86:0/625931658 >> v2:172.21.15.196:3300/0 conn(0x7facf80cc3e0 0x7facf8072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facf6a00640 1 -- 172.21.15.86:0/625931658 >> v1:172.21.15.196:6789/0 conn(0x7facf80b30f0 legacy=0x7facf80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.794+0000 7facf6a00640 1 -- 172.21.15.86:0/625931658 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf8071c00 con 0x7facf815b3f0 2024-03-22T04:12:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facf6a00640 1 --2- 172.21.15.86:0/625931658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf815b3f0 0x7facf80729c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7facf8070ed0 tx=0x7face801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c3cfcf85f292be server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facf6000640 1 -- 172.21.15.86:0/625931658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80b2070 con 0x7facf815b3f0 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facf6000640 1 -- 172.21.15.86:0/625931658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face80ad680 con 0x7facf815b3f0 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facf6000640 1 -- 172.21.15.86:0/625931658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80ada50 con 0x7facf815b3f0 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facfda00640 1 -- 172.21.15.86:0/625931658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf815b3f0 msgr2=0x7facf80729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facfda00640 1 --2- 172.21.15.86:0/625931658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf815b3f0 0x7facf80729c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7facf8070ed0 tx=0x7face801c490 comp rx=0 tx=0).stop 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facfda00640 1 -- 172.21.15.86:0/625931658 shutdown_connections 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facfda00640 1 --2- 172.21.15.86:0/625931658 >> v2:172.21.15.196:3300/0 conn(0x7facf80cc3e0 0x7facf8072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facfda00640 1 --2- 172.21.15.86:0/625931658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf815b3f0 0x7facf80729c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facfda00640 1 -- 172.21.15.86:0/625931658 >> 172.21.15.86:0/625931658 conn(0x7facf80b1df0 msgr2=0x7facf80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facfda00640 1 -- 172.21.15.86:0/625931658 shutdown_connections 2024-03-22T04:12:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.798+0000 7facfda00640 1 -- 172.21.15.86:0/625931658 wait complete. 2024-03-22T04:12:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 Processor -- start 2024-03-22T04:12:31.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 -- start start 2024-03-22T04:12:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7facf80b30f0 0x7facf809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf80cc3e0 0x7facf814ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facf815b3f0 0x7facf8151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facf8168580 con 0x7facf80cc3e0 2024-03-22T04:12:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7facf8168400 con 0x7facf815b3f0 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7facf8168700 con 0x7facf80b30f0 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf80cc3e0 0x7facf814ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf80cc3e0 0x7facf814ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58256/0 (socket says 172.21.15.86:58256) 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 -- 172.21.15.86:0/3921142130 learned_addr learned my addr 172.21.15.86:0/3921142130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facf6a00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7facf80b30f0 0x7facf809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facf7400640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facf815b3f0 0x7facf8151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 -- 172.21.15.86:0/3921142130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7facf80b30f0 msgr2=0x7facf809cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7facf80b30f0 0x7facf809cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 -- 172.21.15.86:0/3921142130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facf815b3f0 msgr2=0x7facf8151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facf815b3f0 0x7facf8151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 -- 172.21.15.86:0/3921142130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf809d4e0 con 0x7facf80cc3e0 2024-03-22T04:12:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facefe00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf80cc3e0 0x7facf814ab60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7face000e930 tx=0x7face000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7faceea00640 1 -- 172.21.15.86:0/3921142130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0017070 con 0x7facf80cc3e0 2024-03-22T04:12:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7faceea00640 1 -- 172.21.15.86:0/3921142130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face000ca00 con 0x7facf80cc3e0 2024-03-22T04:12:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf809d200 con 0x7facf80cc3e0 2024-03-22T04:12:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7faceea00640 1 -- 172.21.15.86:0/3921142130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face000cdb0 con 0x7facf80cc3e0 2024-03-22T04:12:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.802+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf81f8310 con 0x7facf80cc3e0 2024-03-22T04:12:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.806+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf8068650 con 0x7facf80cc3e0 2024-03-22T04:12:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.806+0000 7faceea00640 1 -- 172.21.15.86:0/3921142130 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7face001a070 con 0x7facf80cc3e0 2024-03-22T04:12:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.806+0000 7faceea00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facd4067b20 0x7facd4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.806+0000 7faceea00640 1 -- 172.21.15.86:0/3921142130 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7face0085d70 con 0x7facf80cc3e0 2024-03-22T04:12:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.806+0000 7facf6a00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facd4067b20 0x7facd4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.810+0000 7facf6a00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facd4067b20 0x7facd4069fe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7face80af000 tx=0x7face80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.810+0000 7faceea00640 1 -- 172.21.15.86:0/3921142130 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7face0056310 con 0x7facf80cc3e0 2024-03-22T04:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.966+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7facf809e180 con 0x7facf80cc3e0 2024-03-22T04:12:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7faceea00640 1 -- 172.21.15.86:0/3921142130 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+65 (secure 0 0 0) 0x7face005a3b0 con 0x7facf80cc3e0 2024-03-22T04:12:31.971 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:31.971 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:12:31.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:31.971 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T04:12:31.971 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T04:12:31.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:31.971 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facd4067b20 msgr2=0x7facd4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facd4067b20 0x7facd4069fe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7face80af000 tx=0x7face80a7000 comp rx=0 tx=0).stop 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf80cc3e0 msgr2=0x7facf814ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf80cc3e0 0x7facf814ab60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7face000e930 tx=0x7face000edf0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 shutdown_connections 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facd4067b20 0x7facd4069fe0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7facf815b3f0 0x7facf8151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf80cc3e0 0x7facf814ab60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 --2- 172.21.15.86:0/3921142130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7facf80b30f0 0x7facf809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 >> 172.21.15.86:0/3921142130 conn(0x7facf80b1df0 msgr2=0x7facf8158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 shutdown_connections 2024-03-22T04:12:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:31.970+0000 7facfda00640 1 -- 172.21.15.86:0/3921142130 wait complete. 2024-03-22T04:12:31.991 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T04:12:31.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T04:12:32.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.126+0000 7f39b0400640 1 Processor -- start 2024-03-22T04:12:32.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.126+0000 7f39b0400640 1 -- start start 2024-03-22T04:12:32.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39b0400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a80b1cd0 0x7f39a80b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39b0400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39a80cfa70 0x7f39a806d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39b0400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a80b25f0 con 0x7f39a806dca0 2024-03-22T04:12:32.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39b0400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f39a80cea80 con 0x7f39a80cfa70 2024-03-22T04:12:32.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39b0400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f39a806e4d0 con 0x7f39a80b1cd0 2024-03-22T04:12:32.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39ae400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f39a806dca0 0x7f39a8141960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55882/0 (socket says 172.21.15.86:55882) 2024-03-22T04:12:32.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39ae400640 1 -- 172.21.15.86:0/204385948 learned_addr learned my addr 172.21.15.86:0/204385948 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:32.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39ada00640 1 --2- 172.21.15.86:0/204385948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a80b1cd0 0x7f39a80b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39ad000640 1 --2- 172.21.15.86:0/204385948 >> v2:172.21.15.196:3300/0 conn(0x7f39a80cfa70 0x7f39a806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39a7e00640 1 -- 172.21.15.86:0/204385948 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3833560975 0 0) 0x7f39a80b25f0 con 0x7f39a806dca0 2024-03-22T04:12:32.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39a7e00640 1 -- 172.21.15.86:0/204385948 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3990003530 con 0x7f39a806dca0 2024-03-22T04:12:32.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39ada00640 1 -- 172.21.15.86:0/204385948 >> v2:172.21.15.196:3300/0 conn(0x7f39a80cfa70 msgr2=0x7f39a806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:32.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39ada00640 1 --2- 172.21.15.86:0/204385948 >> v2:172.21.15.196:3300/0 conn(0x7f39a80cfa70 0x7f39a806d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39a7e00640 1 -- 172.21.15.86:0/204385948 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2162231855 0 0) 0x7f3990003530 con 0x7f39a806dca0 2024-03-22T04:12:32.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39ada00640 1 -- 172.21.15.86:0/204385948 >> v1:172.21.15.86:6789/0 conn(0x7f39a806dca0 legacy=0x7f39a8141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.130+0000 7f39ada00640 1 -- 172.21.15.86:0/204385948 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80cecb0 con 0x7f39a80b1cd0 2024-03-22T04:12:32.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39ada00640 1 --2- 172.21.15.86:0/204385948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a80b1cd0 0x7f39a80b20b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f399c0097c0 tx=0x7f399c01c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81784cc07c0f017e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:32.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39a7e00640 1 -- 172.21.15.86:0/204385948 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c0b0070 con 0x7f39a80b1cd0 2024-03-22T04:12:32.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39a7e00640 1 -- 172.21.15.86:0/204385948 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f399c0ab400 con 0x7f39a80b1cd0 2024-03-22T04:12:32.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39a7e00640 1 -- 172.21.15.86:0/204385948 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0001a00 con 0x7f39a80b1cd0 2024-03-22T04:12:32.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 -- 172.21.15.86:0/204385948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a80b1cd0 msgr2=0x7f39a80b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 --2- 172.21.15.86:0/204385948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a80b1cd0 0x7f39a80b20b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f399c0097c0 tx=0x7f399c01c390 comp rx=0 tx=0).stop 2024-03-22T04:12:32.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 -- 172.21.15.86:0/204385948 shutdown_connections 2024-03-22T04:12:32.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 --2- 172.21.15.86:0/204385948 >> v2:172.21.15.196:3300/0 conn(0x7f39a80cfa70 0x7f39a806d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 --2- 172.21.15.86:0/204385948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a80b1cd0 0x7f39a80b20b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 -- 172.21.15.86:0/204385948 >> 172.21.15.86:0/204385948 conn(0x7f39a806b660 msgr2=0x7f39a806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:32.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 -- 172.21.15.86:0/204385948 shutdown_connections 2024-03-22T04:12:32.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 -- 172.21.15.86:0/204385948 wait complete. 2024-03-22T04:12:32.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 Processor -- start 2024-03-22T04:12:32.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.134+0000 7f39b0400640 1 -- start start 2024-03-22T04:12:32.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a806dca0 0x7f39a8141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80b1cd0 0x7f39a80b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a80cfa70 0x7f39a81534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39a806e270 con 0x7f39a80cfa70 2024-03-22T04:12:32.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39a806e0f0 con 0x7f39a80b1cd0 2024-03-22T04:12:32.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f39a813f460 con 0x7f39a806dca0 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a80cfa70 0x7f39a81534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a80cfa70 0x7f39a81534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58270/0 (socket says 172.21.15.86:58270) 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 -- 172.21.15.86:0/1099947132 learned_addr learned my addr 172.21.15.86:0/1099947132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ad000640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80b1cd0 0x7f39a80b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ada00640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a806dca0 0x7f39a8141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 -- 172.21.15.86:0/1099947132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a806dca0 msgr2=0x7f39a8141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a806dca0 0x7f39a8141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 -- 172.21.15.86:0/1099947132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80b1cd0 msgr2=0x7f39a80b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80b1cd0 0x7f39a80b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 -- 172.21.15.86:0/1099947132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80b4280 con 0x7f39a80cfa70 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39ae400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a80cfa70 0x7f39a81534b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f39a00018a0 tx=0x7f39a000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39a5600640 1 -- 172.21.15.86:0/1099947132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0012400 con 0x7f39a80cfa70 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39a5600640 1 -- 172.21.15.86:0/1099947132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a00125c0 con 0x7f39a80cfa70 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39a5600640 1 -- 172.21.15.86:0/1099947132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a00129b0 con 0x7f39a80cfa70 2024-03-22T04:12:32.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a80b4560 con 0x7f39a80cfa70 2024-03-22T04:12:32.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a80b4b20 con 0x7f39a80cfa70 2024-03-22T04:12:32.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a8074580 con 0x7f39a80cfa70 2024-03-22T04:12:32.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.138+0000 7f39a5600640 1 -- 172.21.15.86:0/1099947132 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f39a0012b50 con 0x7f39a80cfa70 2024-03-22T04:12:32.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.142+0000 7f39a5600640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3988067b20 0x7f3988069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.142+0000 7f39a5600640 1 -- 172.21.15.86:0/1099947132 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f39a0086640 con 0x7f39a80cfa70 2024-03-22T04:12:32.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.142+0000 7f39ada00640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3988067b20 0x7f3988069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.142+0000 7f39ada00640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3988067b20 0x7f3988069fe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f399c0ad000 tx=0x7f399c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:32.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.146+0000 7f39a5600640 1 -- 172.21.15.86:0/1099947132 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f39a0056a90 con 0x7f39a80cfa70 2024-03-22T04:12:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.302+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f39a80731e0 con 0x7f39a80cfa70 2024-03-22T04:12:32.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39a5600640 1 -- 172.21.15.86:0/1099947132 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) ==== 138+0+0 (secure 0 0 0) 0x7f39a005ab30 con 0x7f39a80cfa70 2024-03-22T04:12:32.592 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T04:12:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3988067b20 msgr2=0x7f3988069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3988067b20 0x7f3988069fe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f399c0ad000 tx=0x7f399c0a6000 comp rx=0 tx=0).stop 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a80cfa70 msgr2=0x7f39a81534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a80cfa70 0x7f39a81534b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f39a00018a0 tx=0x7f39a000a430 comp rx=0 tx=0).stop 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 shutdown_connections 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3988067b20 0x7f3988069fe0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80b1cd0 0x7f39a80b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a80cfa70 0x7f39a81534b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 --2- 172.21.15.86:0/1099947132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a806dca0 0x7f39a8141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.590+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 >> 172.21.15.86:0/1099947132 conn(0x7f39a806b660 msgr2=0x7f39a8071620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.594+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 shutdown_connections 2024-03-22T04:12:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.594+0000 7f39b0400640 1 -- 172.21.15.86:0/1099947132 wait complete. 2024-03-22T04:12:32.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T04:12:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.758+0000 7fce13200640 1 Processor -- start 2024-03-22T04:12:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce13200640 1 -- start start 2024-03-22T04:12:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce13200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce0c13f330 0x7fce0c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce13200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 0x7fce0c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce13200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fce0c0cf030 0x7fce0c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce13200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fce0c15ff50 con 0x7fce0c13f330 2024-03-22T04:12:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce13200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fce0c160230 con 0x7fce0c0cf030 2024-03-22T04:12:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce13200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c160510 con 0x7fce0c15c920 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce0c13f330 0x7fce0c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce0c13f330 0x7fce0c0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58282/0 (socket says 172.21.15.86:58282) 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0be00640 1 -- 172.21.15.86:0/2529991790 learned_addr learned my addr 172.21.15.86:0/2529991790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0b400640 1 --2- 172.21.15.86:0/2529991790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 0x7fce0c0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce11200640 1 --2- 172.21.15.86:0/2529991790 >> v2:172.21.15.196:3300/0 conn(0x7fce0c0cf030 0x7fce0c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0be00640 1 -- 172.21.15.86:0/2529991790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 msgr2=0x7fce0c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0be00640 1 --2- 172.21.15.86:0/2529991790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 0x7fce0c0d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0be00640 1 -- 172.21.15.86:0/2529991790 >> v2:172.21.15.196:3300/0 conn(0x7fce0c0cf030 msgr2=0x7fce0c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0be00640 1 --2- 172.21.15.86:0/2529991790 >> v2:172.21.15.196:3300/0 conn(0x7fce0c0cf030 0x7fce0c0cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.762+0000 7fce0be00640 1 -- 172.21.15.86:0/2529991790 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c1607f0 con 0x7fce0c13f330 2024-03-22T04:12:32.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce0be00640 1 --2- 172.21.15.86:0/2529991790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c0cfda0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fce0c153a20 tx=0x7fcdfc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4591038c50bcf86c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:32.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce0aa00640 1 -- 172.21.15.86:0/2529991790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0a5670 con 0x7fce0c13f330 2024-03-22T04:12:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce0aa00640 1 -- 172.21.15.86:0/2529991790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0a5810 con 0x7fce0c13f330 2024-03-22T04:12:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce0aa00640 1 -- 172.21.15.86:0/2529991790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0a5be0 con 0x7fce0c13f330 2024-03-22T04:12:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 -- 172.21.15.86:0/2529991790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 msgr2=0x7fce0c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 --2- 172.21.15.86:0/2529991790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c0cfda0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fce0c153a20 tx=0x7fcdfc01c9f0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 -- 172.21.15.86:0/2529991790 shutdown_connections 2024-03-22T04:12:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 --2- 172.21.15.86:0/2529991790 >> v2:172.21.15.196:3300/0 conn(0x7fce0c0cf030 0x7fce0c0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 --2- 172.21.15.86:0/2529991790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 0x7fce0c0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 --2- 172.21.15.86:0/2529991790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c0cfda0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 -- 172.21.15.86:0/2529991790 >> 172.21.15.86:0/2529991790 conn(0x7fce0c06ef50 msgr2=0x7fce0c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 -- 172.21.15.86:0/2529991790 shutdown_connections 2024-03-22T04:12:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 -- 172.21.15.86:0/2529991790 wait complete. 2024-03-22T04:12:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 Processor -- start 2024-03-22T04:12:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.766+0000 7fce13200640 1 -- start start 2024-03-22T04:12:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cf030 0x7fce0c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c13cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 0x7fce0c13d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c06dda0 con 0x7fce0c13f330 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c06dc20 con 0x7fce0c0cf030 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c06df20 con 0x7fce0c15c920 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c13cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c13cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58286/0 (socket says 172.21.15.86:58286) 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 -- 172.21.15.86:0/739811529 learned_addr learned my addr 172.21.15.86:0/739811529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0be00640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cf030 0x7fce0c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce11200640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 0x7fce0c13d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 -- 172.21.15.86:0/739811529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 msgr2=0x7fce0c13d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 0x7fce0c13d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 -- 172.21.15.86:0/739811529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cf030 msgr2=0x7fce0c161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cf030 0x7fce0c161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:32.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 -- 172.21.15.86:0/739811529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c158d90 con 0x7fce0c13f330 2024-03-22T04:12:32.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce0b400640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c13cda0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800d5f0 tx=0x7fcdf800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:32.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce03e00640 1 -- 172.21.15.86:0/739811529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80125f0 con 0x7fce0c13f330 2024-03-22T04:12:32.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce03e00640 1 -- 172.21.15.86:0/739811529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf8019070 con 0x7fce0c13f330 2024-03-22T04:12:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce03e00640 1 -- 172.21.15.86:0/739811529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80129b0 con 0x7fce0c13f330 2024-03-22T04:12:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c15a710 con 0x7fce0c13f330 2024-03-22T04:12:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c15acd0 con 0x7fce0c13f330 2024-03-22T04:12:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.770+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c074470 con 0x7fce0c13f330 2024-03-22T04:12:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.774+0000 7fce03e00640 1 -- 172.21.15.86:0/739811529 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcdf8012b50 con 0x7fce0c13f330 2024-03-22T04:12:32.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.774+0000 7fce03e00640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:32.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.774+0000 7fce03e00640 1 -- 172.21.15.86:0/739811529 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5619+0+0 (secure 0 0 0) 0x7fcdf8086110 con 0x7fce0c13f330 2024-03-22T04:12:32.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.774+0000 7fce0be00640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.774+0000 7fce0be00640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0ae000 tx=0x7fcdfc0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:32.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.778+0000 7fce03e00640 1 -- 172.21.15.86:0/739811529 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcdf8056510 con 0x7fce0c13f330 2024-03-22T04:12:32.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:32.934+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fce0c1670c0 con 0x7fce0c13f330 2024-03-22T04:12:33.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.594+0000 7fce03e00640 1 -- 172.21.15.86:0/739811529 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) ==== 170+0+0 (secure 0 0 0) 0x7fcdf805a5b0 con 0x7fce0c13f330 2024-03-22T04:12:33.595 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T04:12:33.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.594+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 msgr2=0x7fcdec069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.594+0000 7fce13200640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0ae000 tx=0x7fcdfc0a7070 comp rx=0 tx=0).stop 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.594+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 msgr2=0x7fce0c13cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.594+0000 7fce13200640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c13cda0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800d5f0 tx=0x7fcdf800dab0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.598+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 shutdown_connections 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.598+0000 7fce13200640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.598+0000 7fce13200640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c15c920 0x7fce0c13d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.598+0000 7fce13200640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c13f330 0x7fce0c13cda0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.598+0000 7fce13200640 1 --2- 172.21.15.86:0/739811529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cf030 0x7fce0c161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.598+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 >> 172.21.15.86:0/739811529 conn(0x7fce0c06ef50 msgr2=0x7fce0c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.598+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 shutdown_connections 2024-03-22T04:12:33.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.598+0000 7fce13200640 1 -- 172.21.15.86:0/739811529 wait complete. 2024-03-22T04:12:33.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T04:12:33.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.770+0000 7f440f800640 1 Processor -- start 2024-03-22T04:12:33.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.770+0000 7f440f800640 1 -- start start 2024-03-22T04:12:33.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4408145c90 0x7f4408160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:33.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 0x7f4408070150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:33.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4408072e10 con 0x7f440806b660 2024-03-22T04:12:33.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f44080730f0 con 0x7f4408145c90 2024-03-22T04:12:33.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44080cb720 con 0x7f440806bac0 2024-03-22T04:12:33.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4408145c90 0x7f4408160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:33.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f4407e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f440806b660 0x7f440806fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55896/0 (socket says 172.21.15.86:55896) 2024-03-22T04:12:33.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4408145c90 0x7f4408160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58298/0 (socket says 172.21.15.86:58298) 2024-03-22T04:12:33.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f4407e00640 1 -- 172.21.15.86:0/3682868216 learned_addr learned my addr 172.21.15.86:0/3682868216 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440d800640 1 --2- 172.21.15.86:0/3682868216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 0x7f4408070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440ce00640 1 -- 172.21.15.86:0/3682868216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 msgr2=0x7f4408070150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440ce00640 1 --2- 172.21.15.86:0/3682868216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 0x7f4408070150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440ce00640 1 -- 172.21.15.86:0/3682868216 >> v1:172.21.15.86:6789/0 conn(0x7f440806b660 legacy=0x7f440806fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f440ce00640 1 -- 172.21.15.86:0/3682868216 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44080cba00 con 0x7f4408145c90 2024-03-22T04:12:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.774+0000 7f4407400640 1 -- 172.21.15.86:0/3682868216 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635357167 0 0) 0x7f4408072e10 con 0x7f440806b660 2024-03-22T04:12:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440ce00640 1 --2- 172.21.15.86:0/3682868216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f4408160710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0097c0 tx=0x7f43fc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8fa7fc847e8c80 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f4407400640 1 -- 172.21.15.86:0/3682868216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0b0070 con 0x7f4408145c90 2024-03-22T04:12:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f4407400640 1 -- 172.21.15.86:0/3682868216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43fc0ab600 con 0x7f4408145c90 2024-03-22T04:12:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f4407400640 1 -- 172.21.15.86:0/3682868216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0ab9d0 con 0x7f4408145c90 2024-03-22T04:12:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- 172.21.15.86:0/3682868216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 msgr2=0x7f4408160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 --2- 172.21.15.86:0/3682868216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f4408160710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0097c0 tx=0x7f43fc01c410 comp rx=0 tx=0).stop 2024-03-22T04:12:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- 172.21.15.86:0/3682868216 shutdown_connections 2024-03-22T04:12:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 --2- 172.21.15.86:0/3682868216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 0x7f4408070150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 --2- 172.21.15.86:0/3682868216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f4408160710 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- 172.21.15.86:0/3682868216 >> 172.21.15.86:0/3682868216 conn(0x7f44080d0fa0 msgr2=0x7f44080d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- 172.21.15.86:0/3682868216 shutdown_connections 2024-03-22T04:12:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- 172.21.15.86:0/3682868216 wait complete. 2024-03-22T04:12:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 Processor -- start 2024-03-22T04:12:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- start start 2024-03-22T04:12:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440806b660 0x7f44081f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 0x7f440815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f44081fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44080ce6c0 con 0x7f4408145c90 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44080b08d0 con 0x7f440806b660 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.778+0000 7f440f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44080ce840 con 0x7f440806bac0 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f44081fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f44081fa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58300/0 (socket says 172.21.15.86:58300) 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 -- 172.21.15.86:0/1901088590 learned_addr learned my addr 172.21.15.86:0/1901088590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440ce00640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440806b660 0x7f44081f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f4407e00640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 0x7f440815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 -- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 msgr2=0x7f440815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 0x7f440815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 -- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440806b660 msgr2=0x7f44081f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440806b660 0x7f44081f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 -- 172.21.15.86:0/1901088590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44081fb3f0 con 0x7f4408145c90 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440ce00640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440806b660 0x7f44081f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440d800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f44081fa7e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4400007e60 tx=0x7f440000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f4404c00640 1 -- 172.21.15.86:0/1901088590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400014420 con 0x7f4408145c90 2024-03-22T04:12:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f4404c00640 1 -- 172.21.15.86:0/1901088590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4400019070 con 0x7f4408145c90 2024-03-22T04:12:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f4404c00640 1 -- 172.21.15.86:0/1901088590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400014820 con 0x7f4408145c90 2024-03-22T04:12:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44081fb6d0 con 0x7f4408145c90 2024-03-22T04:12:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44081f75e0 con 0x7f4408145c90 2024-03-22T04:12:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4408074470 con 0x7f4408145c90 2024-03-22T04:12:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.782+0000 7f4404c00640 1 -- 172.21.15.86:0/1901088590 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f44000149c0 con 0x7f4408145c90 2024-03-22T04:12:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.786+0000 7f4404c00640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43e8067b20 0x7f43e8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.786+0000 7f4404c00640 1 -- 172.21.15.86:0/1901088590 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6001+0+0 (secure 0 0 0) 0x7f4400086950 con 0x7f4408145c90 2024-03-22T04:12:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.786+0000 7f440ce00640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43e8067b20 0x7f43e8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.786+0000 7f440ce00640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43e8067b20 0x7f43e8069fe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0ad000 tx=0x7f43fc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.790+0000 7f4404c00640 1 -- 172.21.15.86:0/1901088590 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4400057190 con 0x7f4408145c90 2024-03-22T04:12:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:33.978+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f44080726f0 con 0x7f4408145c90 2024-03-22T04:12:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.621+0000 7f4404c00640 1 -- 172.21.15.86:0/1901088590 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7f4400056a90 con 0x7f4408145c90 2024-03-22T04:12:34.628 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 7 and data pool 8 2024-03-22T04:12:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.629+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43e8067b20 msgr2=0x7f43e8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.629+0000 7f440f800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43e8067b20 0x7f43e8069fe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0ad000 tx=0x7f43fc0a6000 comp rx=0 tx=0).stop 2024-03-22T04:12:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.629+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 msgr2=0x7f44081fa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.629+0000 7f440f800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f44081fa7e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4400007e60 tx=0x7f440000e4b0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.633+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 shutdown_connections 2024-03-22T04:12:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.633+0000 7f440f800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43e8067b20 0x7f43e8069fe0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.633+0000 7f440f800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4408145c90 0x7f44081fa7e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.633+0000 7f440f800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f440806bac0 0x7f440815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.633+0000 7f440f800640 1 --2- 172.21.15.86:0/1901088590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f440806b660 0x7f44081f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.633+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 >> 172.21.15.86:0/1901088590 conn(0x7f44080d0fa0 msgr2=0x7f440815ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:34.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.633+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 shutdown_connections 2024-03-22T04:12:34.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.633+0000 7f440f800640 1 -- 172.21.15.86:0/1901088590 wait complete. 2024-03-22T04:12:34.665 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:34.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4f4c00640 1 Processor -- start 2024-03-22T04:12:34.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4f4c00640 1 -- start start 2024-03-22T04:12:34.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4f4c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f0070390 0x7fc4f00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:34.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4f4c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc4f00ae5e0 0x7fc4f006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:34.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4f4c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f015f9e0 con 0x7fc4f015f560 2024-03-22T04:12:34.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4f4c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f00b4810 con 0x7fc4f00ae5e0 2024-03-22T04:12:34.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4f4c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f00b4af0 con 0x7fc4f0070390 2024-03-22T04:12:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ee800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc4f00ae5e0 0x7fc4f006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ede00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f0070390 0x7fc4f00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ede00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f0070390 0x7fc4f00b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:51150/0 (socket says 172.21.15.86:51150) 2024-03-22T04:12:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ee800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc4f00ae5e0 0x7fc4f006b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:58186/0 (socket says 172.21.15.86:58186) 2024-03-22T04:12:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ede00640 1 -- 172.21.15.86:0/1687618549 learned_addr learned my addr 172.21.15.86:0/1687618549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ede00640 1 -- 172.21.15.86:0/1687618549 >> v2:172.21.15.196:3300/0 conn(0x7fc4f00ae5e0 msgr2=0x7fc4f006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ede00640 1 --2- 172.21.15.86:0/1687618549 >> v2:172.21.15.196:3300/0 conn(0x7fc4f00ae5e0 0x7fc4f006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ede00640 1 -- 172.21.15.86:0/1687618549 >> v1:172.21.15.196:6789/0 conn(0x7fc4f015f560 legacy=0x7fc4f00aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:34.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ede00640 1 -- 172.21.15.86:0/1687618549 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00b4dd0 con 0x7fc4f0070390 2024-03-22T04:12:34.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.801+0000 7fc4ee800640 1 --2- 172.21.15.86:0/1687618549 >> v2:172.21.15.196:3300/0 conn(0x7fc4f00ae5e0 0x7fc4f006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:34.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4ede00640 1 --2- 172.21.15.86:0/1687618549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f0070390 0x7fc4f00b4150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009870 tx=0x7fc4e001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3221bee2426fcbb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:34.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4eca00640 1 -- 172.21.15.86:0/1687618549 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00a95f0 con 0x7fc4f0070390 2024-03-22T04:12:34.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4eca00640 1 -- 172.21.15.86:0/1687618549 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e00a9790 con 0x7fc4f0070390 2024-03-22T04:12:34.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4eca00640 1 -- 172.21.15.86:0/1687618549 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00a9b60 con 0x7fc4f0070390 2024-03-22T04:12:34.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4f4c00640 1 -- 172.21.15.86:0/1687618549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f0070390 msgr2=0x7fc4f00b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:34.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/1687618549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f0070390 0x7fc4f00b4150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009870 tx=0x7fc4e001c530 comp rx=0 tx=0).stop 2024-03-22T04:12:34.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4f4c00640 1 -- 172.21.15.86:0/1687618549 shutdown_connections 2024-03-22T04:12:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/1687618549 >> v2:172.21.15.196:3300/0 conn(0x7fc4f00ae5e0 0x7fc4f006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/1687618549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f0070390 0x7fc4f00b4150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4f4c00640 1 -- 172.21.15.86:0/1687618549 >> 172.21.15.86:0/1687618549 conn(0x7fc4f00d0860 msgr2=0x7fc4f00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4f4c00640 1 -- 172.21.15.86:0/1687618549 shutdown_connections 2024-03-22T04:12:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.805+0000 7fc4f4c00640 1 -- 172.21.15.86:0/1687618549 wait complete. 2024-03-22T04:12:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 Processor -- start 2024-03-22T04:12:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 -- start start 2024-03-22T04:12:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4f0070390 0x7fc4f014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f00ae5e0 0x7fc4f014ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f015f560 0x7fc4f014f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00708f0 con 0x7fc4f0070390 2024-03-22T04:12:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0070770 con 0x7fc4f00ae5e0 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f006c3b0 con 0x7fc4f015f560 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4f0070390 0x7fc4f014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4f0070390 0x7fc4f014aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58310/0 (socket says 172.21.15.86:58310) 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 -- 172.21.15.86:0/2513033084 learned_addr learned my addr 172.21.15.86:0/2513033084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ed400640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f00ae5e0 0x7fc4f014ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ee800640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f015f560 0x7fc4f014f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 -- 172.21.15.86:0/2513033084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f015f560 msgr2=0x7fc4f014f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f015f560 0x7fc4f014f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 -- 172.21.15.86:0/2513033084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f00ae5e0 msgr2=0x7fc4f014ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f00ae5e0 0x7fc4f014ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 -- 172.21.15.86:0/2513033084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f0156f90 con 0x7fc4f0070390 2024-03-22T04:12:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4ede00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4f0070390 0x7fc4f014aed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc4e001cdd0 tx=0x7fc4e00192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4e6000640 1 -- 172.21.15.86:0/2513033084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00b15f0 con 0x7fc4f0070390 2024-03-22T04:12:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4e6000640 1 -- 172.21.15.86:0/2513033084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e00b17b0 con 0x7fc4f0070390 2024-03-22T04:12:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4e6000640 1 -- 172.21.15.86:0/2513033084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00b1ba0 con 0x7fc4f0070390 2024-03-22T04:12:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f0153c00 con 0x7fc4f0070390 2024-03-22T04:12:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f0157270 con 0x7fc4f0070390 2024-03-22T04:12:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.809+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f0068650 con 0x7fc4f0070390 2024-03-22T04:12:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.813+0000 7fc4e6000640 1 -- 172.21.15.86:0/2513033084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc4e00a8350 con 0x7fc4f0070390 2024-03-22T04:12:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.813+0000 7fc4e6000640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc4d0067b20 0x7fc4d0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.813+0000 7fc4e6000640 1 -- 172.21.15.86:0/2513033084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc4e01203b0 con 0x7fc4f0070390 2024-03-22T04:12:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.813+0000 7fc4ed400640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc4d0067b20 0x7fc4d0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.813+0000 7fc4ed400640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc4d0067b20 0x7fc4d0069fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0150220 tx=0x7fc4dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.817+0000 7fc4e6000640 1 -- 172.21.15.86:0/2513033084 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc4e00c0070 con 0x7fc4f0070390 2024-03-22T04:12:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.973+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4f00b3ed0 con 0x7fc4f0070390 2024-03-22T04:12:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4e6000640 1 -- 172.21.15.86:0/2513033084 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16895 (secure 0 0 0) 0x7fc4e00ad020 con 0x7fc4f0070390 2024-03-22T04:12:34.980 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:34.980 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:12:33.981433+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:12:32.314387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-22T04:12:32.945472+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc4d0067b20 msgr2=0x7fc4d0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc4d0067b20 0x7fc4d0069fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0150220 tx=0x7fc4dc009040 comp rx=0 tx=0).stop 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4f0070390 msgr2=0x7fc4f014aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4f0070390 0x7fc4f014aed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc4e001cdd0 tx=0x7fc4e00192d0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 shutdown_connections 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc4d0067b20 0x7fc4d0069fe0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f00ae5e0 0x7fc4f014ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f015f560 0x7fc4f014f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 --2- 172.21.15.86:0/2513033084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4f0070390 0x7fc4f014aed0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 >> 172.21.15.86:0/2513033084 conn(0x7fc4f00d0860 msgr2=0x7fc4f015fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 shutdown_connections 2024-03-22T04:12:34.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:34.977+0000 7fc4f4c00640 1 -- 172.21.15.86:0/2513033084 wait complete. 2024-03-22T04:12:35.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b46c00640 1 Processor -- start 2024-03-22T04:12:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b46c00640 1 -- start start 2024-03-22T04:12:35.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b46c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b4015b3f0 0x7f1b400729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b46c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b400cc3e0 0x7f1b40072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b46c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b40071920 con 0x7f1b400b30f0 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b46c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1b400716d0 con 0x7f1b400cc3e0 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b46c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4006d990 con 0x7f1b4015b3f0 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b37400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b400cc3e0 0x7f1b40072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b37400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b400cc3e0 0x7f1b40072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58316/0 (socket says 172.21.15.86:58316) 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.133+0000 7f1b37400640 1 -- 172.21.15.86:0/3781005821 learned_addr learned my addr 172.21.15.86:0/3781005821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b3fe00640 1 --2- 172.21.15.86:0/3781005821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b4015b3f0 0x7f1b400729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b37400640 1 -- 172.21.15.86:0/3781005821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b4015b3f0 msgr2=0x7f1b400729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b37400640 1 --2- 172.21.15.86:0/3781005821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b4015b3f0 0x7f1b400729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b37400640 1 -- 172.21.15.86:0/3781005821 >> v1:172.21.15.196:6789/0 conn(0x7f1b400b30f0 legacy=0x7f1b400ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b37400640 1 -- 172.21.15.86:0/3781005821 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b40071c00 con 0x7f1b400cc3e0 2024-03-22T04:12:35.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b37400640 1 --2- 172.21.15.86:0/3781005821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b400cc3e0 0x7f1b40072f00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009870 tx=0x7f1b2801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe8f04fec51dd4a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:35.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b3f400640 1 -- 172.21.15.86:0/3781005821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b280b1070 con 0x7f1b400cc3e0 2024-03-22T04:12:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b3f400640 1 -- 172.21.15.86:0/3781005821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b280ac620 con 0x7f1b400cc3e0 2024-03-22T04:12:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b3f400640 1 -- 172.21.15.86:0/3781005821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b280ac9f0 con 0x7f1b400cc3e0 2024-03-22T04:12:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b46c00640 1 -- 172.21.15.86:0/3781005821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b400cc3e0 msgr2=0x7f1b40072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.137+0000 7f1b46c00640 1 --2- 172.21.15.86:0/3781005821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b400cc3e0 0x7f1b40072f00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009870 tx=0x7f1b2801c4d0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 -- 172.21.15.86:0/3781005821 shutdown_connections 2024-03-22T04:12:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 --2- 172.21.15.86:0/3781005821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b400cc3e0 0x7f1b40072f00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 --2- 172.21.15.86:0/3781005821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b4015b3f0 0x7f1b400729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 -- 172.21.15.86:0/3781005821 >> 172.21.15.86:0/3781005821 conn(0x7f1b400b1df0 msgr2=0x7f1b400b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 -- 172.21.15.86:0/3781005821 shutdown_connections 2024-03-22T04:12:35.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 -- 172.21.15.86:0/3781005821 wait complete. 2024-03-22T04:12:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 Processor -- start 2024-03-22T04:12:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 -- start start 2024-03-22T04:12:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b400b30f0 0x7f1b40149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b400cc3e0 0x7f1b401518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4015b3f0 0x7f1b40151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40168580 con 0x7f1b4015b3f0 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40168400 con 0x7f1b400cc3e0 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b46c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1b40168700 con 0x7f1b400b30f0 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4015b3f0 0x7f1b40151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4015b3f0 0x7f1b40151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58330/0 (socket says 172.21.15.86:58330) 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 -- 172.21.15.86:0/2781418862 learned_addr learned my addr 172.21.15.86:0/2781418862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b3fe00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b400b30f0 0x7f1b40149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b37400640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b400cc3e0 0x7f1b401518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 -- 172.21.15.86:0/2781418862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b400b30f0 msgr2=0x7f1b40149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b400b30f0 0x7f1b40149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 -- 172.21.15.86:0/2781418862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b400cc3e0 msgr2=0x7f1b401518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b400cc3e0 0x7f1b401518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 -- 172.21.15.86:0/2781418862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b40156640 con 0x7f1b4015b3f0 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.141+0000 7f1b44c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4015b3f0 0x7f1b40151e30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c760 tx=0x7f1b3800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b3cc00640 1 -- 172.21.15.86:0/2781418862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38003b80 con 0x7f1b4015b3f0 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b3cc00640 1 -- 172.21.15.86:0/2781418862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38003d20 con 0x7f1b4015b3f0 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b40156920 con 0x7f1b4015b3f0 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b3cc00640 1 -- 172.21.15.86:0/2781418862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800aa30 con 0x7f1b4015b3f0 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b40163ac0 con 0x7f1b4015b3f0 2024-03-22T04:12:35.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b40068650 con 0x7f1b4015b3f0 2024-03-22T04:12:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b3cc00640 1 -- 172.21.15.86:0/2781418862 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1b3800abd0 con 0x7f1b4015b3f0 2024-03-22T04:12:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b3cc00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b1c067b20 0x7f1b1c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b3cc00640 1 -- 172.21.15.86:0/2781418862 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f1b38085f40 con 0x7f1b4015b3f0 2024-03-22T04:12:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b3fe00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b1c067b20 0x7f1b1c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.145+0000 7f1b3fe00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b1c067b20 0x7f1b1c069fe0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1b30006aa0 tx=0x7f1b30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:35.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.149+0000 7f1b3cc00640 1 -- 172.21.15.86:0/2781418862 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1b38056040 con 0x7f1b4015b3f0 2024-03-22T04:12:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.305+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b4006f2c0 con 0x7f1b4015b3f0 2024-03-22T04:12:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.305+0000 7f1b3cc00640 1 -- 172.21.15.86:0/2781418862 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16895 (secure 0 0 0) 0x7f1b3805a0e0 con 0x7f1b4015b3f0 2024-03-22T04:12:35.310 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:35.310 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:12:33.981433+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:12:32.314387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-22T04:12:32.945472+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b1c067b20 msgr2=0x7f1b1c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b1c067b20 0x7f1b1c069fe0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1b30006aa0 tx=0x7f1b30008040 comp rx=0 tx=0).stop 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4015b3f0 msgr2=0x7f1b40151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4015b3f0 0x7f1b40151e30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c760 tx=0x7f1b3800cc20 comp rx=0 tx=0).stop 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 shutdown_connections 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b1c067b20 0x7f1b1c069fe0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b400cc3e0 0x7f1b401518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4015b3f0 0x7f1b40151e30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 --2- 172.21.15.86:0/2781418862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b400b30f0 0x7f1b40149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 >> 172.21.15.86:0/2781418862 conn(0x7f1b400b1df0 msgr2=0x7f1b40157510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 shutdown_connections 2024-03-22T04:12:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.309+0000 7f1b46c00640 1 -- 172.21.15.86:0/2781418862 wait complete. 2024-03-22T04:12:35.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T04:12:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.465+0000 7f0dae200640 1 Processor -- start 2024-03-22T04:12:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.465+0000 7f0dae200640 1 -- start start 2024-03-22T04:12:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.465+0000 7f0dae200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da80b2360 0x7f0da80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.465+0000 7f0dae200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0da815d0b0 0x7f0da80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.465+0000 7f0dae200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da815d710 con 0x7f0da80b3430 2024-03-22T04:12:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.465+0000 7f0dae200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0da80b3890 con 0x7f0da815d0b0 2024-03-22T04:12:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.465+0000 7f0dae200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0da80b2740 con 0x7f0da80b2360 2024-03-22T04:12:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0da815d0b0 0x7f0da80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da80b2360 0x7f0da80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0da815d0b0 0x7f0da80b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:58226/0 (socket says 172.21.15.86:58226) 2024-03-22T04:12:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7e00640 1 -- 172.21.15.86:0/3322460531 learned_addr learned my addr 172.21.15.86:0/3322460531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0d9ea00640 1 --1- 172.21.15.86:0/3322460531 >> v1:172.21.15.196:6789/0 conn(0x7f0da80b3430 0x7f0da815c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:56508/0 (socket says 172.21.15.86:56508) 2024-03-22T04:12:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7e00640 1 -- 172.21.15.86:0/3322460531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da80b2360 msgr2=0x7f0da80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7e00640 1 --2- 172.21.15.86:0/3322460531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da80b2360 0x7f0da80b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7e00640 1 -- 172.21.15.86:0/3322460531 >> v1:172.21.15.196:6789/0 conn(0x7f0da80b3430 legacy=0x7f0da815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7e00640 1 -- 172.21.15.86:0/3322460531 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da815fe40 con 0x7f0da815d0b0 2024-03-22T04:12:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da7e00640 1 --2- 172.21.15.86:0/3322460531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da815d0b0 0x7f0da80b4d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009e50 tx=0x7f0da001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565ae847fb255bfd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.469+0000 7f0da6a00640 1 -- 172.21.15.86:0/3322460531 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da00a57f0 con 0x7f0da815d0b0 2024-03-22T04:12:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0da6a00640 1 -- 172.21.15.86:0/3322460531 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da00a5990 con 0x7f0da815d0b0 2024-03-22T04:12:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0da6a00640 1 -- 172.21.15.86:0/3322460531 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da00a5d60 con 0x7f0da815d0b0 2024-03-22T04:12:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- 172.21.15.86:0/3322460531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da815d0b0 msgr2=0x7f0da80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 --2- 172.21.15.86:0/3322460531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da815d0b0 0x7f0da80b4d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009e50 tx=0x7f0da001ca30 comp rx=0 tx=0).stop 2024-03-22T04:12:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- 172.21.15.86:0/3322460531 shutdown_connections 2024-03-22T04:12:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 --2- 172.21.15.86:0/3322460531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da815d0b0 0x7f0da80b4d80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 --2- 172.21.15.86:0/3322460531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da80b2360 0x7f0da80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- 172.21.15.86:0/3322460531 >> 172.21.15.86:0/3322460531 conn(0x7f0da80cc940 msgr2=0x7f0da80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- 172.21.15.86:0/3322460531 shutdown_connections 2024-03-22T04:12:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- 172.21.15.86:0/3322460531 wait complete. 2024-03-22T04:12:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 Processor -- start 2024-03-22T04:12:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- start start 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80b2360 0x7f0da814b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da80b3430 0x7f0da8155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815d0b0 0x7f0da8149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8160c00 con 0x7f0da80b3430 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0da80b5ad0 con 0x7f0da80b2360 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0dae200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0da814b530 con 0x7f0da815d0b0 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0d9ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da80b3430 0x7f0da8155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0da7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815d0b0 0x7f0da8149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0d9ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da80b3430 0x7f0da8155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58340/0 (socket says 172.21.15.86:58340) 2024-03-22T04:12:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0d9ea00640 1 -- 172.21.15.86:0/2065221294 learned_addr learned my addr 172.21.15.86:0/2065221294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:35.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.473+0000 7f0da7400640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80b2360 0x7f0da814b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9ea00640 1 -- 172.21.15.86:0/2065221294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815d0b0 msgr2=0x7f0da8149b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9ea00640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815d0b0 0x7f0da8149b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9ea00640 1 -- 172.21.15.86:0/2065221294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80b2360 msgr2=0x7f0da814b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9ea00640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80b2360 0x7f0da814b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9ea00640 1 -- 172.21.15.86:0/2065221294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da80b16e0 con 0x7f0da80b3430 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9ea00640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da80b3430 0x7f0da8155fa0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c420 tx=0x7f0d9000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9fe00640 1 -- 172.21.15.86:0/2065221294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d900074e0 con 0x7f0da80b3430 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9fe00640 1 -- 172.21.15.86:0/2065221294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d90007680 con 0x7f0da80b3430 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9fe00640 1 -- 172.21.15.86:0/2065221294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90004a30 con 0x7f0da80b3430 2024-03-22T04:12:35.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da8154bb0 con 0x7f0da80b3430 2024-03-22T04:12:35.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da8155170 con 0x7f0da80b3430 2024-03-22T04:12:35.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da8074470 con 0x7f0da80b3430 2024-03-22T04:12:35.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9fe00640 1 -- 172.21.15.86:0/2065221294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0d9001a020 con 0x7f0da80b3430 2024-03-22T04:12:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9fe00640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d84067b20 0x7f0d84069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0d9fe00640 1 -- 172.21.15.86:0/2065221294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f0d90085750 con 0x7f0da80b3430 2024-03-22T04:12:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.477+0000 7f0da7400640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d84067b20 0x7f0d84069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.481+0000 7f0da7400640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d84067b20 0x7f0d84069fe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0da8155d60 tx=0x7f0d98009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:35.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.481+0000 7f0d9fe00640 1 -- 172.21.15.86:0/2065221294 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0d90055970 con 0x7f0da80b3430 2024-03-22T04:12:35.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:35.673+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0da806e9f0 con 0x7f0da80b3430 2024-03-22T04:12:36.591 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:12:36.589+0000 7f5545400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:12:36.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.629+0000 7f0d9fe00640 1 -- 172.21.15.86:0/2065221294 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f0d90059a10 con 0x7f0da80b3430 2024-03-22T04:12:36.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d84067b20 msgr2=0x7f0d84069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:36.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d84067b20 0x7f0d84069fe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0da8155d60 tx=0x7f0d98009190 comp rx=0 tx=0).stop 2024-03-22T04:12:36.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da80b3430 msgr2=0x7f0da8155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:36.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da80b3430 0x7f0da8155fa0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c420 tx=0x7f0d9000c8e0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 shutdown_connections 2024-03-22T04:12:36.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d84067b20 0x7f0d84069fe0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815d0b0 0x7f0da8149b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da80b3430 0x7f0da8155fa0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 --2- 172.21.15.86:0/2065221294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80b2360 0x7f0da814b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 >> 172.21.15.86:0/2065221294 conn(0x7f0da80cc940 msgr2=0x7f0da80cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:36.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 shutdown_connections 2024-03-22T04:12:36.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.633+0000 7f0dae200640 1 -- 172.21.15.86:0/2065221294 wait complete. 2024-03-22T04:12:36.658 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-22T04:12:36.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-22T04:12:36.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.793+0000 7fb514800640 1 Processor -- start 2024-03-22T04:12:36.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb514800640 1 -- start start 2024-03-22T04:12:36.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb514800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:36.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb514800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb50c15d0b0 0x7fb50c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:36.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb514800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb50c15d710 con 0x7fb50c0b3430 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb514800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb50c0b3890 con 0x7fb50c15d0b0 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb514800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb50c0b2740 con 0x7fb50c0b2360 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb511400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb50c0b3430 0x7fb50c15c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57212/0 (socket says 172.21.15.86:57212) 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb511400640 1 -- 172.21.15.86:0/1841243631 learned_addr learned my addr 172.21.15.86:0/1841243631 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb512800640 1 --2- 172.21.15.86:0/1841243631 >> v2:172.21.15.196:3300/0 conn(0x7fb50c15d0b0 0x7fb50c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb511e00640 1 --2- 172.21.15.86:0/1841243631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb510a00640 1 -- 172.21.15.86:0/1841243631 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584572899 0 0) 0x7fb50c15d710 con 0x7fb50c0b3430 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb510a00640 1 -- 172.21.15.86:0/1841243631 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4f4003530 con 0x7fb50c0b3430 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb511e00640 1 -- 172.21.15.86:0/1841243631 >> v2:172.21.15.196:3300/0 conn(0x7fb50c15d0b0 msgr2=0x7fb50c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb511e00640 1 --2- 172.21.15.86:0/1841243631 >> v2:172.21.15.196:3300/0 conn(0x7fb50c15d0b0 0x7fb50c0b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb511e00640 1 -- 172.21.15.86:0/1841243631 >> v1:172.21.15.86:6789/0 conn(0x7fb50c0b3430 legacy=0x7fb50c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:36.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb511e00640 1 -- 172.21.15.86:0/1841243631 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c15fe40 con 0x7fb50c0b2360 2024-03-22T04:12:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.797+0000 7fb512800640 1 --2- 172.21.15.86:0/1841243631 >> v2:172.21.15.196:3300/0 conn(0x7fb50c15d0b0 0x7fb50c0b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb511e00640 1 --2- 172.21.15.86:0/1841243631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c0b2ef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5000097c0 tx=0x7fb50001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a0be5e5be1699dc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb510a00640 1 -- 172.21.15.86:0/1841243631 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5000b2070 con 0x7fb50c0b2360 2024-03-22T04:12:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb510a00640 1 -- 172.21.15.86:0/1841243631 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5000ad640 con 0x7fb50c0b2360 2024-03-22T04:12:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb510a00640 1 -- 172.21.15.86:0/1841243631 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5000ada10 con 0x7fb50c0b2360 2024-03-22T04:12:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 -- 172.21.15.86:0/1841243631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 msgr2=0x7fb50c0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 --2- 172.21.15.86:0/1841243631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c0b2ef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5000097c0 tx=0x7fb50001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 -- 172.21.15.86:0/1841243631 shutdown_connections 2024-03-22T04:12:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 --2- 172.21.15.86:0/1841243631 >> v2:172.21.15.196:3300/0 conn(0x7fb50c15d0b0 0x7fb50c0b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 --2- 172.21.15.86:0/1841243631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c0b2ef0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 -- 172.21.15.86:0/1841243631 >> 172.21.15.86:0/1841243631 conn(0x7fb50c0cc940 msgr2=0x7fb50c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 -- 172.21.15.86:0/1841243631 shutdown_connections 2024-03-22T04:12:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 -- 172.21.15.86:0/1841243631 wait complete. 2024-03-22T04:12:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 Processor -- start 2024-03-22T04:12:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.801+0000 7fb514800640 1 -- start start 2024-03-22T04:12:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c14b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c0b3430 0x7fb50c15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb50c15d0b0 0x7fb50c15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c160c00 con 0x7fb50c15d0b0 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c0b5ad0 con 0x7fb50c0b3430 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb50c1503a0 con 0x7fb50c0b2360 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb50c15d0b0 0x7fb50c15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb50c15d0b0 0x7fb50c15e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51362/0 (socket says 172.21.15.86:51362) 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 -- 172.21.15.86:0/993927082 learned_addr learned my addr 172.21.15.86:0/993927082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb511e00640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c14b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb511400640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c0b3430 0x7fb50c15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 -- 172.21.15.86:0/993927082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 msgr2=0x7fb50c14b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c14b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 -- 172.21.15.86:0/993927082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c0b3430 msgr2=0x7fb50c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:36.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c0b3430 0x7fb50c15df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 -- 172.21.15.86:0/993927082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c14fa70 con 0x7fb50c15d0b0 2024-03-22T04:12:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb512800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb50c15d0b0 0x7fb50c15e490 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb508009e60 tx=0x7fb50800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb506000640 1 -- 172.21.15.86:0/993927082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508018030 con 0x7fb50c15d0b0 2024-03-22T04:12:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb506000640 1 -- 172.21.15.86:0/993927082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50800cdf0 con 0x7fb50c15d0b0 2024-03-22T04:12:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb506000640 1 -- 172.21.15.86:0/993927082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508003bc0 con 0x7fb50c15d0b0 2024-03-22T04:12:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c14fd50 con 0x7fb50c15d0b0 2024-03-22T04:12:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c14b780 con 0x7fb50c15d0b0 2024-03-22T04:12:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.805+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50c074470 con 0x7fb50c15d0b0 2024-03-22T04:12:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.809+0000 7fb506000640 1 -- 172.21.15.86:0/993927082 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb50801b070 con 0x7fb50c15d0b0 2024-03-22T04:12:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.809+0000 7fb506000640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4f0067b20 0x7fb4f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.809+0000 7fb506000640 1 -- 172.21.15.86:0/993927082 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fb5080869a0 con 0x7fb50c15d0b0 2024-03-22T04:12:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.809+0000 7fb511e00640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4f0067b20 0x7fb4f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.809+0000 7fb511e00640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4f0067b20 0x7fb4f0069fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5000af000 tx=0x7fb500008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.813+0000 7fb506000640 1 -- 172.21.15.86:0/993927082 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb508010040 con 0x7fb50c15d0b0 2024-03-22T04:12:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:36.997+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb50c06e9f0 con 0x7fb50c15d0b0 2024-03-22T04:12:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.633+0000 7fb506000640 1 -- 172.21.15.86:0/993927082 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fb508056b30 con 0x7fb50c15d0b0 2024-03-22T04:12:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4f0067b20 msgr2=0x7fb4f0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4f0067b20 0x7fb4f0069fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5000af000 tx=0x7fb500008040 comp rx=0 tx=0).stop 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb50c15d0b0 msgr2=0x7fb50c15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb50c15d0b0 0x7fb50c15e490 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb508009e60 tx=0x7fb50800c490 comp rx=0 tx=0).stop 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 shutdown_connections 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4f0067b20 0x7fb4f0069fe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c0b3430 0x7fb50c15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb50c15d0b0 0x7fb50c15e490 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 --2- 172.21.15.86:0/993927082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb50c0b2360 0x7fb50c14b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 >> 172.21.15.86:0/993927082 conn(0x7fb50c0cc940 msgr2=0x7fb50c0b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 shutdown_connections 2024-03-22T04:12:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.637+0000 7fb514800640 1 -- 172.21.15.86:0/993927082 wait complete. 2024-03-22T04:12:37.661 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-22T04:12:37.661 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.793+0000 7f62c8800640 1 Processor -- start 2024-03-22T04:12:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.793+0000 7f62c8800640 1 -- start start 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c8800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 0x7f62c006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c8800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62c006cbb0 0x7f62c0143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c8800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f62c0144320 0x7f62c006e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c8800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f62c006d010 con 0x7f62c006cbb0 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c8800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f62c0162aa0 con 0x7f62c0144320 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c8800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62c00ae5a0 con 0x7f62c00aff50 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62c006cbb0 0x7f62c0143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62c006cbb0 0x7f62c0143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51378/0 (socket says 172.21.15.86:51378) 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5400640 1 -- 172.21.15.86:0/1410761224 learned_addr learned my addr 172.21.15.86:0/1410761224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5e00640 1 --2- 172.21.15.86:0/1410761224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 0x7f62c006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c6800640 1 --2- 172.21.15.86:0/1410761224 >> v2:172.21.15.196:3300/0 conn(0x7f62c0144320 0x7f62c006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5400640 1 -- 172.21.15.86:0/1410761224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 msgr2=0x7f62c006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5400640 1 --2- 172.21.15.86:0/1410761224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 0x7f62c006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5400640 1 -- 172.21.15.86:0/1410761224 >> v2:172.21.15.196:3300/0 conn(0x7f62c0144320 msgr2=0x7f62c006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5400640 1 --2- 172.21.15.86:0/1410761224 >> v2:172.21.15.196:3300/0 conn(0x7f62c0144320 0x7f62c006e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.797+0000 7f62c5400640 1 -- 172.21.15.86:0/1410761224 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c0144b80 con 0x7f62c006cbb0 2024-03-22T04:12:37.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c5400640 1 --2- 172.21.15.86:0/1410761224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c0143de0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ae750 tx=0x7f62b001c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b412896809aeb5f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c4a00640 1 -- 172.21.15.86:0/1410761224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00b0070 con 0x7f62c006cbb0 2024-03-22T04:12:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c4a00640 1 -- 172.21.15.86:0/1410761224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b00ab600 con 0x7f62c006cbb0 2024-03-22T04:12:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c4a00640 1 -- 172.21.15.86:0/1410761224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00ab9d0 con 0x7f62c006cbb0 2024-03-22T04:12:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 -- 172.21.15.86:0/1410761224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 msgr2=0x7f62c0143de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 --2- 172.21.15.86:0/1410761224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c0143de0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ae750 tx=0x7f62b001c1a0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 -- 172.21.15.86:0/1410761224 shutdown_connections 2024-03-22T04:12:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 --2- 172.21.15.86:0/1410761224 >> v2:172.21.15.196:3300/0 conn(0x7f62c0144320 0x7f62c006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 --2- 172.21.15.86:0/1410761224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c0143de0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 --2- 172.21.15.86:0/1410761224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 0x7f62c006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 -- 172.21.15.86:0/1410761224 >> 172.21.15.86:0/1410761224 conn(0x7f62c0071fe0 msgr2=0x7f62c00723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 -- 172.21.15.86:0/1410761224 shutdown_connections 2024-03-22T04:12:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 -- 172.21.15.86:0/1410761224 wait complete. 2024-03-22T04:12:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 Processor -- start 2024-03-22T04:12:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.801+0000 7f62c8800640 1 -- start start 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c00af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 0x7f62c00af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c0144320 0x7f62c013afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0163090 con 0x7f62c006cbb0 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0162f10 con 0x7f62c0144320 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62c00b0330 con 0x7f62c00aff50 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c00af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c00af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51392/0 (socket says 172.21.15.86:51392) 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 -- 172.21.15.86:0/2005671294 learned_addr learned my addr 172.21.15.86:0/2005671294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c6800640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c0144320 0x7f62c013afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5400640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 0x7f62c00af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 -- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 msgr2=0x7f62c00af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 0x7f62c00af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 -- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c0144320 msgr2=0x7f62c013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c0144320 0x7f62c013afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 -- 172.21.15.86:0/2005671294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c00b3990 con 0x7f62c006cbb0 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c6800640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c0144320 0x7f62c013afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c5e00640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c00af000 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f62b400d830 tx=0x7f62b400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62ba000640 1 -- 172.21.15.86:0/2005671294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b400b6c0 con 0x7f62c006cbb0 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62ba000640 1 -- 172.21.15.86:0/2005671294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b4019070 con 0x7f62c006cbb0 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c00b3c70 con 0x7f62c006cbb0 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62ba000640 1 -- 172.21.15.86:0/2005671294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b400ba60 con 0x7f62c006cbb0 2024-03-22T04:12:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c00b4230 con 0x7f62c006cbb0 2024-03-22T04:12:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c0068650 con 0x7f62c006cbb0 2024-03-22T04:12:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.805+0000 7f62ba000640 1 -- 172.21.15.86:0/2005671294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f62b401e460 con 0x7f62c006cbb0 2024-03-22T04:12:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.809+0000 7f62ba000640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f62a4067b20 0x7f62a4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.809+0000 7f62ba000640 1 -- 172.21.15.86:0/2005671294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f62b4086780 con 0x7f62c006cbb0 2024-03-22T04:12:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.809+0000 7f62c5400640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f62a4067b20 0x7f62a4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.809+0000 7f62c5400640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f62a4067b20 0x7f62a4069fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f62b00ad000 tx=0x7f62b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:37.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.813+0000 7f62ba000640 1 -- 172.21.15.86:0/2005671294 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f62b4057000 con 0x7f62c006cbb0 2024-03-22T04:12:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:37.997+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62c00b69b0 con 0x7f62c006cbb0 2024-03-22T04:12:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62ba000640 1 -- 172.21.15.86:0/2005671294 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5189 (secure 0 0 0) 0x7f62b4056900 con 0x7f62c006cbb0 2024-03-22T04:12:38.003 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:38.003 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:12:33.981773+0000","modified":"2024-03-22T04:12:37.002106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75245":{"gid":75245,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6834/2728972599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2728972599},{"type":"v1","addr":"172.21.15.196:6834","nonce":2728972599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75245,"qdb_cluster":[75245]},"id":3}]} 2024-03-22T04:12:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f62a4067b20 msgr2=0x7f62a4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f62a4067b20 0x7f62a4069fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f62b00ad000 tx=0x7f62b0008040 comp rx=0 tx=0).stop 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 msgr2=0x7f62c00af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c00af000 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f62b400d830 tx=0x7f62b400dcf0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 shutdown_connections 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f62a4067b20 0x7f62a4069fe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c0144320 0x7f62c013afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c00aff50 0x7f62c00af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 --2- 172.21.15.86:0/2005671294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c006cbb0 0x7f62c00af000 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 >> 172.21.15.86:0/2005671294 conn(0x7f62c0071fe0 msgr2=0x7f62c00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 shutdown_connections 2024-03-22T04:12:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.001+0000 7f62c8800640 1 -- 172.21.15.86:0/2005671294 wait complete. 2024-03-22T04:12:38.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:12:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851be00640 1 Processor -- start 2024-03-22T04:12:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851be00640 1 -- start start 2024-03-22T04:12:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f851c161cd0 0x7f851c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0b4a90 0x7f851c0d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f851c09aa70 0x7f851c09af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f851c0b33b0 con 0x7f851c161cd0 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851be00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f851c0d1cc0 con 0x7f851c09aa70 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f851c0d0dd0 con 0x7f851c0b4a90 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f851c161cd0 0x7f851c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f851c161cd0 0x7f851c0b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51396/0 (socket says 172.21.15.86:51396) 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851aa00640 1 -- 172.21.15.86:0/4129256473 learned_addr learned my addr 172.21.15.86:0/4129256473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f8513e00640 1 --2- 172.21.15.86:0/4129256473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0b4a90 0x7f851c0d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851b400640 1 --2- 172.21.15.86:0/4129256473 >> v2:172.21.15.196:3300/0 conn(0x7f851c09aa70 0x7f851c09af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851aa00640 1 -- 172.21.15.86:0/4129256473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0b4a90 msgr2=0x7f851c0d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851aa00640 1 --2- 172.21.15.86:0/4129256473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0b4a90 0x7f851c0d12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851aa00640 1 -- 172.21.15.86:0/4129256473 >> v2:172.21.15.196:3300/0 conn(0x7f851c09aa70 msgr2=0x7f851c09af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851aa00640 1 --2- 172.21.15.86:0/4129256473 >> v2:172.21.15.196:3300/0 conn(0x7f851c09aa70 0x7f851c09af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.157+0000 7f851aa00640 1 -- 172.21.15.86:0/4129256473 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c0d1fa0 con 0x7f851c161cd0 2024-03-22T04:12:38.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851aa00640 1 --2- 172.21.15.86:0/4129256473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c161cd0 0x7f851c0b3690 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f851c09b7a0 tx=0x7f850c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f774a84a3a00890 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851a000640 1 -- 172.21.15.86:0/4129256473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0b0070 con 0x7f851c161cd0 2024-03-22T04:12:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851a000640 1 -- 172.21.15.86:0/4129256473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c0ab600 con 0x7f851c161cd0 2024-03-22T04:12:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851a000640 1 -- 172.21.15.86:0/4129256473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0ab9d0 con 0x7f851c161cd0 2024-03-22T04:12:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 -- 172.21.15.86:0/4129256473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c161cd0 msgr2=0x7f851c0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 --2- 172.21.15.86:0/4129256473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c161cd0 0x7f851c0b3690 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f851c09b7a0 tx=0x7f850c01c520 comp rx=0 tx=0).stop 2024-03-22T04:12:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 -- 172.21.15.86:0/4129256473 shutdown_connections 2024-03-22T04:12:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 --2- 172.21.15.86:0/4129256473 >> v2:172.21.15.196:3300/0 conn(0x7f851c09aa70 0x7f851c09af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 --2- 172.21.15.86:0/4129256473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0b4a90 0x7f851c0d12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 --2- 172.21.15.86:0/4129256473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c161cd0 0x7f851c0b3690 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 -- 172.21.15.86:0/4129256473 >> 172.21.15.86:0/4129256473 conn(0x7f851c0b3d60 msgr2=0x7f851c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 -- 172.21.15.86:0/4129256473 shutdown_connections 2024-03-22T04:12:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 -- 172.21.15.86:0/4129256473 wait complete. 2024-03-22T04:12:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 Processor -- start 2024-03-22T04:12:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.161+0000 7f851be00640 1 -- start start 2024-03-22T04:12:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c09aa70 0x7f851c09fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c0b4a90 0x7f851c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c161cd0 0x7f851c15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0b2770 con 0x7f851c09aa70 2024-03-22T04:12:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0b25f0 con 0x7f851c0b4a90 2024-03-22T04:12:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f851c06c770 con 0x7f851c161cd0 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c09aa70 0x7f851c09fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c09aa70 0x7f851c09fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51404/0 (socket says 172.21.15.86:51404) 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 -- 172.21.15.86:0/3092968206 learned_addr learned my addr 172.21.15.86:0/3092968206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f8513e00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c0b4a90 0x7f851c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851b400640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c161cd0 0x7f851c15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 -- 172.21.15.86:0/3092968206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c161cd0 msgr2=0x7f851c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c161cd0 0x7f851c15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 -- 172.21.15.86:0/3092968206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c0b4a90 msgr2=0x7f851c1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c0b4a90 0x7f851c1596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 -- 172.21.15.86:0/3092968206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c159c20 con 0x7f851c09aa70 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851aa00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c09aa70 0x7f851c09fc60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f850c0097c0 tx=0x7f850c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f8512a00640 1 -- 172.21.15.86:0/3092968206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0b0040 con 0x7f851c09aa70 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f8512a00640 1 -- 172.21.15.86:0/3092968206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c0bc070 con 0x7f851c09aa70 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c09e380 con 0x7f851c09aa70 2024-03-22T04:12:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f8512a00640 1 -- 172.21.15.86:0/3092968206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0b6880 con 0x7f851c09aa70 2024-03-22T04:12:38.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c159e10 con 0x7f851c09aa70 2024-03-22T04:12:38.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c074470 con 0x7f851c09aa70 2024-03-22T04:12:38.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.165+0000 7f8512a00640 1 -- 172.21.15.86:0/3092968206 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f850c0ab4c0 con 0x7f851c09aa70 2024-03-22T04:12:38.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.169+0000 7f8512a00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84f8067b20 0x7f84f8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.169+0000 7f8512a00640 1 -- 172.21.15.86:0/3092968206 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f850c1202e0 con 0x7f851c09aa70 2024-03-22T04:12:38.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.169+0000 7f8513e00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84f8067b20 0x7f84f8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.169+0000 7f8513e00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84f8067b20 0x7f84f8069fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f851c160ec0 tx=0x7f8504009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.173+0000 7f8512a00640 1 -- 172.21.15.86:0/3092968206 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f850c016110 con 0x7f851c09aa70 2024-03-22T04:12:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.329+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f851c09b6c0 con 0x7f851c09aa70 2024-03-22T04:12:38.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.329+0000 7f8512a00640 1 -- 172.21.15.86:0/3092968206 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16882 (secure 0 0 0) 0x7f850c0f0470 con 0x7f851c09aa70 2024-03-22T04:12:38.332 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:38.333 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":54,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:12:35.607984+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:12:32.314387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-22T04:12:32.945472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:12:38.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.329+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84f8067b20 msgr2=0x7f84f8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.329+0000 7f851be00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84f8067b20 0x7f84f8069fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f851c160ec0 tx=0x7f8504009190 comp rx=0 tx=0).stop 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.329+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c09aa70 msgr2=0x7f851c09fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.329+0000 7f851be00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c09aa70 0x7f851c09fc60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f850c0097c0 tx=0x7f850c01ad60 comp rx=0 tx=0).stop 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.333+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 shutdown_connections 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.333+0000 7f851be00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84f8067b20 0x7f84f8069fe0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.333+0000 7f851be00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c0b4a90 0x7f851c1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.333+0000 7f851be00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c161cd0 0x7f851c15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.333+0000 7f851be00640 1 --2- 172.21.15.86:0/3092968206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c09aa70 0x7f851c09fc60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.333+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 >> 172.21.15.86:0/3092968206 conn(0x7f851c0b3d60 msgr2=0x7f851c0b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.333+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 shutdown_connections 2024-03-22T04:12:38.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.333+0000 7f851be00640 1 -- 172.21.15.86:0/3092968206 wait complete. 2024-03-22T04:12:38.354 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T04:12:38.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:12:38.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.485+0000 7f0e5ce00640 1 Processor -- start 2024-03-22T04:12:38.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.485+0000 7f0e5ce00640 1 -- start start 2024-03-22T04:12:38.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.485+0000 7f0e5ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e5815b3f0 0x7f0e580729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.485+0000 7f0e5ce00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e58071920 con 0x7f0e580cc3e0 2024-03-22T04:12:38.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.485+0000 7f0e5ce00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e58071c00 con 0x7f0e580ae540 2024-03-22T04:12:38.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.485+0000 7f0e5ce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0e580716d0 con 0x7f0e5815b3f0 2024-03-22T04:12:38.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e5815b3f0 0x7f0e580729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e4d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0e580cc3e0 0x7f0e580ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57224/0 (socket says 172.21.15.86:57224) 2024-03-22T04:12:38.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e4d400640 1 -- 172.21.15.86:0/1240020860 learned_addr learned my addr 172.21.15.86:0/1240020860 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:38.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55e00640 1 -- 172.21.15.86:0/1240020860 >> v1:172.21.15.196:6789/0 conn(0x7f0e580ae540 legacy=0x7f0e580b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55e00640 1 -- 172.21.15.86:0/1240020860 >> v1:172.21.15.86:6789/0 conn(0x7f0e580cc3e0 legacy=0x7f0e580ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55e00640 1 -- 172.21.15.86:0/1240020860 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e58071ee0 con 0x7f0e5815b3f0 2024-03-22T04:12:38.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55400640 1 -- 172.21.15.86:0/1240020860 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3838892463 0 0) 0x7f0e58071920 con 0x7f0e580cc3e0 2024-03-22T04:12:38.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55e00640 1 --2- 172.21.15.86:0/1240020860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5815b3f0 0x7f0e580729c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009740 tx=0x7f0e4801c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b77c131f88e6a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55400640 1 -- 172.21.15.86:0/1240020860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e480b0070 con 0x7f0e5815b3f0 2024-03-22T04:12:38.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55400640 1 -- 172.21.15.86:0/1240020860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e480ab600 con 0x7f0e5815b3f0 2024-03-22T04:12:38.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.489+0000 7f0e55400640 1 -- 172.21.15.86:0/1240020860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e480ab9d0 con 0x7f0e5815b3f0 2024-03-22T04:12:38.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1240020860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5815b3f0 msgr2=0x7f0e580729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 --2- 172.21.15.86:0/1240020860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5815b3f0 0x7f0e580729c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009740 tx=0x7f0e4801c2d0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1240020860 shutdown_connections 2024-03-22T04:12:38.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 --2- 172.21.15.86:0/1240020860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5815b3f0 0x7f0e580729c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1240020860 >> 172.21.15.86:0/1240020860 conn(0x7f0e580b1df0 msgr2=0x7f0e580b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:38.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1240020860 shutdown_connections 2024-03-22T04:12:38.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1240020860 wait complete. 2024-03-22T04:12:38.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 Processor -- start 2024-03-22T04:12:38.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- start start 2024-03-22T04:12:38.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580ae540 0x7f0e581f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cc3e0 0x7f0e58154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5815b3f0 0x7f0e5815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e580b4a10 con 0x7f0e580cc3e0 2024-03-22T04:12:38.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5806d990 con 0x7f0e5815b3f0 2024-03-22T04:12:38.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e5ce00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0e580b4b90 con 0x7f0e580ae540 2024-03-22T04:12:38.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cc3e0 0x7f0e58154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cc3e0 0x7f0e58154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51424/0 (socket says 172.21.15.86:51424) 2024-03-22T04:12:38.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 -- 172.21.15.86:0/1869143318 learned_addr learned my addr 172.21.15.86:0/1869143318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:38.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e56800640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5815b3f0 0x7f0e5815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e55e00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580ae540 0x7f0e581f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 -- 172.21.15.86:0/1869143318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580ae540 msgr2=0x7f0e581f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580ae540 0x7f0e581f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 -- 172.21.15.86:0/1869143318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5815b3f0 msgr2=0x7f0e5815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5815b3f0 0x7f0e5815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 -- 172.21.15.86:0/1869143318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e581f24f0 con 0x7f0e580cc3e0 2024-03-22T04:12:38.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4d400640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cc3e0 0x7f0e58154840 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000c520 tx=0x7f0e4000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.493+0000 7f0e4ea00640 1 -- 172.21.15.86:0/1869143318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40018070 con 0x7f0e580cc3e0 2024-03-22T04:12:38.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e4ea00640 1 -- 172.21.15.86:0/1869143318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e40007500 con 0x7f0e580cc3e0 2024-03-22T04:12:38.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e4ea00640 1 -- 172.21.15.86:0/1869143318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40004a30 con 0x7f0e580cc3e0 2024-03-22T04:12:38.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e58155090 con 0x7f0e580cc3e0 2024-03-22T04:12:38.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e58155650 con 0x7f0e580cc3e0 2024-03-22T04:12:38.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e58068650 con 0x7f0e580cc3e0 2024-03-22T04:12:38.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e4ea00640 1 -- 172.21.15.86:0/1869143318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0e4001b070 con 0x7f0e580cc3e0 2024-03-22T04:12:38.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e4ea00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e34067b50 0x7f0e3406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e4ea00640 1 -- 172.21.15.86:0/1869143318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0e400852f0 con 0x7f0e580cc3e0 2024-03-22T04:12:38.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e55e00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e34067b50 0x7f0e3406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.497+0000 7f0e55e00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e34067b50 0x7f0e3406a010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e480ad000 tx=0x7f0e480a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.501+0000 7f0e4ea00640 1 -- 172.21.15.86:0/1869143318 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0e40055510 con 0x7f0e580cc3e0 2024-03-22T04:12:38.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.661+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e5815ca70 con 0x7f0e34067b50 2024-03-22T04:12:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.665+0000 7f0e4ea00640 1 -- 172.21.15.86:0/1869143318 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251107 (secure 0 0 0) 0x7f0e5815ca70 con 0x7f0e34067b50 2024-03-22T04:12:38.673 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:12:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e34067b50 msgr2=0x7f0e3406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e34067b50 0x7f0e3406a010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e480ad000 tx=0x7f0e480a6000 comp rx=0 tx=0).stop 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cc3e0 msgr2=0x7f0e58154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cc3e0 0x7f0e58154840 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000c520 tx=0x7f0e4000c9e0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 shutdown_connections 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0e34067b50 0x7f0e3406a010 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e5815b3f0 0x7f0e5815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e580cc3e0 0x7f0e58154840 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 --2- 172.21.15.86:0/1869143318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e580ae540 0x7f0e581f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 >> 172.21.15.86:0/1869143318 conn(0x7f0e580b1df0 msgr2=0x7f0e5815c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 shutdown_connections 2024-03-22T04:12:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.669+0000 7f0e5ce00640 1 -- 172.21.15.86:0/1869143318 wait complete. 2024-03-22T04:12:38.693 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-03-22T04:12:37.816229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222236,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752484,"statfs":{"total":773094113280,"available":772866543616,"internally_reserved":0,"allocated":6201344,"data_stored":3366910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220955"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616060+0000","last_change":"2024-03-22T04:12:34.628936+0000","last_active":"2024-03-22T04:12:35.616060+0000","last_peered":"2024-03-22T04:12:35.616060+0000","last_clean":"2024-03-22T04:12:35.616060+0000","last_became_active":"2024-03-22T04:12:34.628641+0000","last_became_peered":"2024-03-22T04:12:34.628641+0000","last_unstale":"2024-03-22T04:12:35.616060+0000","last_undegraded":"2024-03-22T04:12:35.616060+0000","last_fullsized":"2024-03-22T04:12:35.616060+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639641+0000","last_change":"2024-03-22T04:12:34.637267+0000","last_active":"2024-03-22T04:12:34.639641+0000","last_peered":"2024-03-22T04:12:34.639641+0000","last_clean":"2024-03-22T04:12:34.639641+0000","last_became_active":"2024-03-22T04:12:34.637024+0000","last_became_peered":"2024-03-22T04:12:34.637024+0000","last_unstale":"2024-03-22T04:12:34.639641+0000","last_undegraded":"2024-03-22T04:12:34.639641+0000","last_fullsized":"2024-03-22T04:12:34.639641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603940+0000","last_change":"2024-03-22T04:12:33.603940+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603940+0000","last_undegraded":"2024-03-22T04:12:33.603940+0000","last_fullsized":"2024-03-22T04:12:33.603940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612444+0000","last_change":"2024-03-22T04:12:34.625832+0000","last_active":"2024-03-22T04:12:35.612444+0000","last_peered":"2024-03-22T04:12:35.612444+0000","last_clean":"2024-03-22T04:12:35.612444+0000","last_became_active":"2024-03-22T04:12:34.625533+0000","last_became_peered":"2024-03-22T04:12:34.625533+0000","last_unstale":"2024-03-22T04:12:35.612444+0000","last_undegraded":"2024-03-22T04:12:35.612444+0000","last_fullsized":"2024-03-22T04:12:35.612444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:00.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.635523+0000","last_change":"2024-03-22T04:12:34.627773+0000","last_active":"2024-03-22T04:12:34.635523+0000","last_peered":"2024-03-22T04:12:34.635523+0000","last_clean":"2024-03-22T04:12:34.635523+0000","last_became_active":"2024-03-22T04:12:34.624749+0000","last_became_peered":"2024-03-22T04:12:34.624749+0000","last_unstale":"2024-03-22T04:12:34.635523+0000","last_undegraded":"2024-03-22T04:12:34.635523+0000","last_fullsized":"2024-03-22T04:12:34.635523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:01.747862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643345+0000","last_change":"2024-03-22T04:12:34.635777+0000","last_active":"2024-03-22T04:12:34.643345+0000","last_peered":"2024-03-22T04:12:34.643345+0000","last_clean":"2024-03-22T04:12:34.643345+0000","last_became_active":"2024-03-22T04:12:34.635528+0000","last_became_peered":"2024-03-22T04:12:34.635528+0000","last_unstale":"2024-03-22T04:12:34.643345+0000","last_undegraded":"2024-03-22T04:12:34.643345+0000","last_fullsized":"2024-03-22T04:12:34.643345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:01:53.135313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614543+0000","last_change":"2024-03-22T04:12:34.636516+0000","last_active":"2024-03-22T04:12:35.614543+0000","last_peered":"2024-03-22T04:12:35.614543+0000","last_clean":"2024-03-22T04:12:35.614543+0000","last_became_active":"2024-03-22T04:12:34.636312+0000","last_became_peered":"2024-03-22T04:12:34.636312+0000","last_unstale":"2024-03-22T04:12:35.614543+0000","last_undegraded":"2024-03-22T04:12:35.614543+0000","last_fullsized":"2024-03-22T04:12:35.614543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657816+0000","last_change":"2024-03-22T04:12:34.650010+0000","last_active":"2024-03-22T04:12:34.657816+0000","last_peered":"2024-03-22T04:12:34.657816+0000","last_clean":"2024-03-22T04:12:34.657816+0000","last_became_active":"2024-03-22T04:12:34.649709+0000","last_became_peered":"2024-03-22T04:12:34.649709+0000","last_unstale":"2024-03-22T04:12:34.657816+0000","last_undegraded":"2024-03-22T04:12:34.657816+0000","last_fullsized":"2024-03-22T04:12:34.657816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:50.915283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603153+0000","last_change":"2024-03-22T04:12:33.603153+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603153+0000","last_undegraded":"2024-03-22T04:12:33.603153+0000","last_fullsized":"2024-03-22T04:12:33.603153+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615323+0000","last_change":"2024-03-22T04:12:34.641332+0000","last_active":"2024-03-22T04:12:35.615323+0000","last_peered":"2024-03-22T04:12:35.615323+0000","last_clean":"2024-03-22T04:12:35.615323+0000","last_became_active":"2024-03-22T04:12:34.641059+0000","last_became_peered":"2024-03-22T04:12:34.641059+0000","last_unstale":"2024-03-22T04:12:35.615323+0000","last_undegraded":"2024-03-22T04:12:35.615323+0000","last_fullsized":"2024-03-22T04:12:35.615323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615824+0000","last_change":"2024-03-22T04:12:34.627929+0000","last_active":"2024-03-22T04:12:35.615824+0000","last_peered":"2024-03-22T04:12:35.615824+0000","last_clean":"2024-03-22T04:12:35.615824+0000","last_became_active":"2024-03-22T04:12:34.627640+0000","last_became_peered":"2024-03-22T04:12:34.627640+0000","last_unstale":"2024-03-22T04:12:35.615824+0000","last_undegraded":"2024-03-22T04:12:35.615824+0000","last_fullsized":"2024-03-22T04:12:35.615824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643682+0000","last_change":"2024-03-22T04:12:34.639709+0000","last_active":"2024-03-22T04:12:34.643682+0000","last_peered":"2024-03-22T04:12:34.643682+0000","last_clean":"2024-03-22T04:12:34.643682+0000","last_became_active":"2024-03-22T04:12:34.638936+0000","last_became_peered":"2024-03-22T04:12:34.638936+0000","last_unstale":"2024-03-22T04:12:34.643682+0000","last_undegraded":"2024-03-22T04:12:34.643682+0000","last_fullsized":"2024-03-22T04:12:34.643682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:33.015134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.647737+0000","last_change":"2024-03-22T04:12:34.644837+0000","last_active":"2024-03-22T04:12:34.647737+0000","last_peered":"2024-03-22T04:12:34.647737+0000","last_clean":"2024-03-22T04:12:34.647737+0000","last_became_active":"2024-03-22T04:12:34.644507+0000","last_became_peered":"2024-03-22T04:12:34.644507+0000","last_unstale":"2024-03-22T04:12:34.647737+0000","last_undegraded":"2024-03-22T04:12:34.647737+0000","last_fullsized":"2024-03-22T04:12:34.647737+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:35.311859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614973+0000","last_change":"2024-03-22T04:12:34.641248+0000","last_active":"2024-03-22T04:12:35.614973+0000","last_peered":"2024-03-22T04:12:35.614973+0000","last_clean":"2024-03-22T04:12:35.614973+0000","last_became_active":"2024-03-22T04:12:34.640953+0000","last_became_peered":"2024-03-22T04:12:34.640953+0000","last_unstale":"2024-03-22T04:12:35.614973+0000","last_undegraded":"2024-03-22T04:12:35.614973+0000","last_fullsized":"2024-03-22T04:12:35.614973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612098+0000","last_change":"2024-03-22T04:12:34.632175+0000","last_active":"2024-03-22T04:12:35.612098+0000","last_peered":"2024-03-22T04:12:35.612098+0000","last_clean":"2024-03-22T04:12:35.612098+0000","last_became_active":"2024-03-22T04:12:34.631916+0000","last_became_peered":"2024-03-22T04:12:34.631916+0000","last_unstale":"2024-03-22T04:12:35.612098+0000","last_undegraded":"2024-03-22T04:12:35.612098+0000","last_fullsized":"2024-03-22T04:12:35.612098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643732+0000","last_change":"2024-03-22T04:12:34.639122+0000","last_active":"2024-03-22T04:12:34.643732+0000","last_peered":"2024-03-22T04:12:34.643732+0000","last_clean":"2024-03-22T04:12:34.643732+0000","last_became_active":"2024-03-22T04:12:34.638788+0000","last_became_peered":"2024-03-22T04:12:34.638788+0000","last_unstale":"2024-03-22T04:12:34.643732+0000","last_undegraded":"2024-03-22T04:12:34.643732+0000","last_fullsized":"2024-03-22T04:12:34.643732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:27.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616261+0000","last_change":"2024-03-22T04:12:34.632974+0000","last_active":"2024-03-22T04:12:35.616261+0000","last_peered":"2024-03-22T04:12:35.616261+0000","last_clean":"2024-03-22T04:12:35.616261+0000","last_became_active":"2024-03-22T04:12:34.627665+0000","last_became_peered":"2024-03-22T04:12:34.627665+0000","last_unstale":"2024-03-22T04:12:35.616261+0000","last_undegraded":"2024-03-22T04:12:35.616261+0000","last_fullsized":"2024-03-22T04:12:35.616261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612642+0000","last_change":"2024-03-22T04:12:34.626077+0000","last_active":"2024-03-22T04:12:35.612642+0000","last_peered":"2024-03-22T04:12:35.612642+0000","last_clean":"2024-03-22T04:12:35.612642+0000","last_became_active":"2024-03-22T04:12:34.625837+0000","last_became_peered":"2024-03-22T04:12:34.625837+0000","last_unstale":"2024-03-22T04:12:35.612642+0000","last_undegraded":"2024-03-22T04:12:35.612642+0000","last_fullsized":"2024-03-22T04:12:35.612642+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603525+0000","last_change":"2024-03-22T04:12:33.603525+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603525+0000","last_undegraded":"2024-03-22T04:12:33.603525+0000","last_fullsized":"2024-03-22T04:12:33.603525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634836+0000","last_change":"2024-03-22T04:12:34.630270+0000","last_active":"2024-03-22T04:12:34.634836+0000","last_peered":"2024-03-22T04:12:34.634836+0000","last_clean":"2024-03-22T04:12:34.634836+0000","last_became_active":"2024-03-22T04:12:34.629989+0000","last_became_peered":"2024-03-22T04:12:34.629989+0000","last_unstale":"2024-03-22T04:12:34.634836+0000","last_undegraded":"2024-03-22T04:12:34.634836+0000","last_fullsized":"2024-03-22T04:12:34.634836+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:50.915283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612164+0000","last_change":"2024-03-22T04:12:34.620830+0000","last_active":"2024-03-22T04:12:35.612164+0000","last_peered":"2024-03-22T04:12:35.612164+0000","last_clean":"2024-03-22T04:12:35.612164+0000","last_became_active":"2024-03-22T04:12:34.620520+0000","last_became_peered":"2024-03-22T04:12:34.620520+0000","last_unstale":"2024-03-22T04:12:35.612164+0000","last_undegraded":"2024-03-22T04:12:35.612164+0000","last_fullsized":"2024-03-22T04:12:35.612164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.604396+0000","last_change":"2024-03-22T04:12:33.604396+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.604396+0000","last_undegraded":"2024-03-22T04:12:33.604396+0000","last_fullsized":"2024-03-22T04:12:33.604396+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602445+0000","last_change":"2024-03-22T04:12:33.602445+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602445+0000","last_undegraded":"2024-03-22T04:12:33.602445+0000","last_fullsized":"2024-03-22T04:12:33.602445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603931+0000","last_change":"2024-03-22T04:12:33.603931+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603931+0000","last_undegraded":"2024-03-22T04:12:33.603931+0000","last_fullsized":"2024-03-22T04:12:33.603931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616176+0000","last_change":"2024-03-22T04:12:34.635548+0000","last_active":"2024-03-22T04:12:35.616176+0000","last_peered":"2024-03-22T04:12:35.616176+0000","last_clean":"2024-03-22T04:12:35.616176+0000","last_became_active":"2024-03-22T04:12:34.635270+0000","last_became_peered":"2024-03-22T04:12:34.635270+0000","last_unstale":"2024-03-22T04:12:35.616176+0000","last_undegraded":"2024-03-22T04:12:35.616176+0000","last_fullsized":"2024-03-22T04:12:35.616176+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.604463+0000","last_change":"2024-03-22T04:12:33.604463+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.604463+0000","last_undegraded":"2024-03-22T04:12:33.604463+0000","last_fullsized":"2024-03-22T04:12:33.604463+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.659926+0000","last_change":"2024-03-22T04:12:34.655920+0000","last_active":"2024-03-22T04:12:34.659926+0000","last_peered":"2024-03-22T04:12:34.659926+0000","last_clean":"2024-03-22T04:12:34.659926+0000","last_became_active":"2024-03-22T04:12:34.655580+0000","last_became_peered":"2024-03-22T04:12:34.655580+0000","last_unstale":"2024-03-22T04:12:34.659926+0000","last_undegraded":"2024-03-22T04:12:34.659926+0000","last_fullsized":"2024-03-22T04:12:34.659926+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:33.344021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634856+0000","last_change":"2024-03-22T04:12:34.633424+0000","last_active":"2024-03-22T04:12:34.634856+0000","last_peered":"2024-03-22T04:12:34.634856+0000","last_clean":"2024-03-22T04:12:34.634856+0000","last_became_active":"2024-03-22T04:12:34.632797+0000","last_became_peered":"2024-03-22T04:12:34.632797+0000","last_unstale":"2024-03-22T04:12:34.634856+0000","last_undegraded":"2024-03-22T04:12:34.634856+0000","last_fullsized":"2024-03-22T04:12:34.634856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:12.985866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615318+0000","last_change":"2024-03-22T04:12:34.630952+0000","last_active":"2024-03-22T04:12:35.615318+0000","last_peered":"2024-03-22T04:12:35.615318+0000","last_clean":"2024-03-22T04:12:35.615318+0000","last_became_active":"2024-03-22T04:12:34.630765+0000","last_became_peered":"2024-03-22T04:12:34.630765+0000","last_unstale":"2024-03-22T04:12:35.615318+0000","last_undegraded":"2024-03-22T04:12:35.615318+0000","last_fullsized":"2024-03-22T04:12:35.615318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.625106+0000","last_change":"2024-03-22T04:12:34.621885+0000","last_active":"2024-03-22T04:12:34.625106+0000","last_peered":"2024-03-22T04:12:34.625106+0000","last_clean":"2024-03-22T04:12:34.625106+0000","last_became_active":"2024-03-22T04:12:34.621626+0000","last_became_peered":"2024-03-22T04:12:34.621626+0000","last_unstale":"2024-03-22T04:12:34.625106+0000","last_undegraded":"2024-03-22T04:12:34.625106+0000","last_fullsized":"2024-03-22T04:12:34.625106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:12:20.702700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.601918+0000","last_change":"2024-03-22T04:12:33.601918+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.601918+0000","last_undegraded":"2024-03-22T04:12:33.601918+0000","last_fullsized":"2024-03-22T04:12:33.601918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.630607+0000","last_change":"2024-03-22T04:12:34.624213+0000","last_active":"2024-03-22T04:12:34.630607+0000","last_peered":"2024-03-22T04:12:34.630607+0000","last_clean":"2024-03-22T04:12:34.630607+0000","last_became_active":"2024-03-22T04:12:34.623954+0000","last_became_peered":"2024-03-22T04:12:34.623954+0000","last_unstale":"2024-03-22T04:12:34.630607+0000","last_undegraded":"2024-03-22T04:12:34.630607+0000","last_fullsized":"2024-03-22T04:12:34.630607+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:42.435805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.629445+0000","last_change":"2024-03-22T04:12:34.623255+0000","last_active":"2024-03-22T04:12:34.629445+0000","last_peered":"2024-03-22T04:12:34.629445+0000","last_clean":"2024-03-22T04:12:34.629445+0000","last_became_active":"2024-03-22T04:12:34.622995+0000","last_became_peered":"2024-03-22T04:12:34.622995+0000","last_unstale":"2024-03-22T04:12:34.629445+0000","last_undegraded":"2024-03-22T04:12:34.629445+0000","last_fullsized":"2024-03-22T04:12:34.629445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:50.075295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637402+0000","last_change":"2024-03-22T04:12:33.618251+0000","last_active":"2024-03-22T04:12:35.637402+0000","last_peered":"2024-03-22T04:12:35.637402+0000","last_clean":"2024-03-22T04:12:35.637402+0000","last_became_active":"2024-03-22T04:12:33.617405+0000","last_became_peered":"2024-03-22T04:12:33.617405+0000","last_unstale":"2024-03-22T04:12:35.637402+0000","last_undegraded":"2024-03-22T04:12:35.637402+0000","last_fullsized":"2024-03-22T04:12:35.637402+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:45.786716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.636934+0000","last_change":"2024-03-22T04:12:34.633188+0000","last_active":"2024-03-22T04:12:34.636934+0000","last_peered":"2024-03-22T04:12:34.636934+0000","last_clean":"2024-03-22T04:12:34.636934+0000","last_became_active":"2024-03-22T04:12:34.632933+0000","last_became_peered":"2024-03-22T04:12:34.632933+0000","last_unstale":"2024-03-22T04:12:34.636934+0000","last_undegraded":"2024-03-22T04:12:34.636934+0000","last_fullsized":"2024-03-22T04:12:34.636934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:30:04.785433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.660378+0000","last_change":"2024-03-22T04:12:33.623741+0000","last_active":"2024-03-22T04:12:34.660378+0000","last_peered":"2024-03-22T04:12:34.660378+0000","last_clean":"2024-03-22T04:12:34.660378+0000","last_became_active":"2024-03-22T04:12:33.623407+0000","last_became_peered":"2024-03-22T04:12:33.623407+0000","last_unstale":"2024-03-22T04:12:34.660378+0000","last_undegraded":"2024-03-22T04:12:34.660378+0000","last_fullsized":"2024-03-22T04:12:34.660378+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.625128+0000","last_change":"2024-03-22T04:12:34.622229+0000","last_active":"2024-03-22T04:12:34.625128+0000","last_peered":"2024-03-22T04:12:34.625128+0000","last_clean":"2024-03-22T04:12:34.625128+0000","last_became_active":"2024-03-22T04:12:34.621209+0000","last_became_peered":"2024-03-22T04:12:34.621209+0000","last_unstale":"2024-03-22T04:12:34.625128+0000","last_undegraded":"2024-03-22T04:12:34.625128+0000","last_fullsized":"2024-03-22T04:12:34.625128+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:14:10.558919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615885+0000","last_change":"2024-03-22T04:12:33.621633+0000","last_active":"2024-03-22T04:12:35.615885+0000","last_peered":"2024-03-22T04:12:35.615885+0000","last_clean":"2024-03-22T04:12:35.615885+0000","last_became_active":"2024-03-22T04:12:33.621320+0000","last_became_peered":"2024-03-22T04:12:33.621320+0000","last_unstale":"2024-03-22T04:12:35.615885+0000","last_undegraded":"2024-03-22T04:12:35.615885+0000","last_fullsized":"2024-03-22T04:12:35.615885+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:00.740320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615027+0000","last_change":"2024-03-22T04:12:34.638992+0000","last_active":"2024-03-22T04:12:35.615027+0000","last_peered":"2024-03-22T04:12:35.615027+0000","last_clean":"2024-03-22T04:12:35.615027+0000","last_became_active":"2024-03-22T04:12:34.638706+0000","last_became_peered":"2024-03-22T04:12:34.638706+0000","last_unstale":"2024-03-22T04:12:35.615027+0000","last_undegraded":"2024-03-22T04:12:35.615027+0000","last_fullsized":"2024-03-22T04:12:35.615027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657480+0000","last_change":"2024-03-22T04:12:33.619503+0000","last_active":"2024-03-22T04:12:34.657480+0000","last_peered":"2024-03-22T04:12:34.657480+0000","last_clean":"2024-03-22T04:12:34.657480+0000","last_became_active":"2024-03-22T04:12:33.619218+0000","last_became_peered":"2024-03-22T04:12:33.619218+0000","last_unstale":"2024-03-22T04:12:34.657480+0000","last_undegraded":"2024-03-22T04:12:34.657480+0000","last_fullsized":"2024-03-22T04:12:34.657480+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:26.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637412+0000","last_change":"2024-03-22T04:12:34.638372+0000","last_active":"2024-03-22T04:12:35.637412+0000","last_peered":"2024-03-22T04:12:35.637412+0000","last_clean":"2024-03-22T04:12:35.637412+0000","last_became_active":"2024-03-22T04:12:34.636653+0000","last_became_peered":"2024-03-22T04:12:34.636653+0000","last_unstale":"2024-03-22T04:12:35.637412+0000","last_undegraded":"2024-03-22T04:12:35.637412+0000","last_fullsized":"2024-03-22T04:12:35.637412+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.625947+0000","last_change":"2024-03-22T04:12:33.621816+0000","last_active":"2024-03-22T04:12:33.625947+0000","last_peered":"2024-03-22T04:12:33.625947+0000","last_clean":"2024-03-22T04:12:33.625947+0000","last_became_active":"2024-03-22T04:12:33.621560+0000","last_became_peered":"2024-03-22T04:12:33.621560+0000","last_unstale":"2024-03-22T04:12:33.625947+0000","last_undegraded":"2024-03-22T04:12:33.625947+0000","last_fullsized":"2024-03-22T04:12:33.625947+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:49.010265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639031+0000","last_change":"2024-03-22T04:12:34.633399+0000","last_active":"2024-03-22T04:12:34.639031+0000","last_peered":"2024-03-22T04:12:34.639031+0000","last_clean":"2024-03-22T04:12:34.639031+0000","last_became_active":"2024-03-22T04:12:34.633082+0000","last_became_peered":"2024-03-22T04:12:34.633082+0000","last_unstale":"2024-03-22T04:12:34.639031+0000","last_undegraded":"2024-03-22T04:12:34.639031+0000","last_fullsized":"2024-03-22T04:12:34.639031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:39:21.503071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.651676+0000","last_change":"2024-03-22T04:12:33.615641+0000","last_active":"2024-03-22T04:12:34.651676+0000","last_peered":"2024-03-22T04:12:34.651676+0000","last_clean":"2024-03-22T04:12:34.651676+0000","last_became_active":"2024-03-22T04:12:33.615372+0000","last_became_peered":"2024-03-22T04:12:33.615372+0000","last_unstale":"2024-03-22T04:12:34.651676+0000","last_undegraded":"2024-03-22T04:12:34.651676+0000","last_fullsized":"2024-03-22T04:12:34.651676+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:39.482917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615462+0000","last_change":"2024-03-22T04:12:34.637539+0000","last_active":"2024-03-22T04:12:35.615462+0000","last_peered":"2024-03-22T04:12:35.615462+0000","last_clean":"2024-03-22T04:12:35.615462+0000","last_became_active":"2024-03-22T04:12:34.636918+0000","last_became_peered":"2024-03-22T04:12:34.636918+0000","last_unstale":"2024-03-22T04:12:35.615462+0000","last_undegraded":"2024-03-22T04:12:35.615462+0000","last_fullsized":"2024-03-22T04:12:35.615462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.618429+0000","last_change":"2024-03-22T04:12:33.615588+0000","last_active":"2024-03-22T04:12:34.618429+0000","last_peered":"2024-03-22T04:12:34.618429+0000","last_clean":"2024-03-22T04:12:34.618429+0000","last_became_active":"2024-03-22T04:12:33.615259+0000","last_became_peered":"2024-03-22T04:12:33.615259+0000","last_unstale":"2024-03-22T04:12:34.618429+0000","last_undegraded":"2024-03-22T04:12:34.618429+0000","last_fullsized":"2024-03-22T04:12:34.618429+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:27:55.035247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612760+0000","last_change":"2024-03-22T04:12:34.635762+0000","last_active":"2024-03-22T04:12:35.612760+0000","last_peered":"2024-03-22T04:12:35.612760+0000","last_clean":"2024-03-22T04:12:35.612760+0000","last_became_active":"2024-03-22T04:12:34.635493+0000","last_became_peered":"2024-03-22T04:12:34.635493+0000","last_unstale":"2024-03-22T04:12:35.612760+0000","last_undegraded":"2024-03-22T04:12:35.612760+0000","last_fullsized":"2024-03-22T04:12:35.612760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.622856+0000","last_change":"2024-03-22T04:12:33.618851+0000","last_active":"2024-03-22T04:12:33.622856+0000","last_peered":"2024-03-22T04:12:33.622856+0000","last_clean":"2024-03-22T04:12:33.622856+0000","last_became_active":"2024-03-22T04:12:33.618525+0000","last_became_peered":"2024-03-22T04:12:33.618525+0000","last_unstale":"2024-03-22T04:12:33.622856+0000","last_undegraded":"2024-03-22T04:12:33.622856+0000","last_fullsized":"2024-03-22T04:12:33.622856+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:46.571387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637090+0000","last_change":"2024-03-22T04:12:34.626850+0000","last_active":"2024-03-22T04:12:35.637090+0000","last_peered":"2024-03-22T04:12:35.637090+0000","last_clean":"2024-03-22T04:12:35.637090+0000","last_became_active":"2024-03-22T04:12:34.626505+0000","last_became_peered":"2024-03-22T04:12:34.626505+0000","last_unstale":"2024-03-22T04:12:35.637090+0000","last_undegraded":"2024-03-22T04:12:35.637090+0000","last_fullsized":"2024-03-22T04:12:35.637090+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612750+0000","last_change":"2024-03-22T04:12:33.618147+0000","last_active":"2024-03-22T04:12:35.612750+0000","last_peered":"2024-03-22T04:12:35.612750+0000","last_clean":"2024-03-22T04:12:35.612750+0000","last_became_active":"2024-03-22T04:12:33.617867+0000","last_became_peered":"2024-03-22T04:12:33.617867+0000","last_unstale":"2024-03-22T04:12:35.612750+0000","last_undegraded":"2024-03-22T04:12:35.612750+0000","last_fullsized":"2024-03-22T04:12:35.612750+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612131+0000","last_change":"2024-03-22T04:12:34.624495+0000","last_active":"2024-03-22T04:12:35.612131+0000","last_peered":"2024-03-22T04:12:35.612131+0000","last_clean":"2024-03-22T04:12:35.612131+0000","last_became_active":"2024-03-22T04:12:34.624243+0000","last_became_peered":"2024-03-22T04:12:34.624243+0000","last_unstale":"2024-03-22T04:12:35.612131+0000","last_undegraded":"2024-03-22T04:12:35.612131+0000","last_fullsized":"2024-03-22T04:12:35.612131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612650+0000","last_change":"2024-03-22T04:12:33.617534+0000","last_active":"2024-03-22T04:12:35.612650+0000","last_peered":"2024-03-22T04:12:35.612650+0000","last_clean":"2024-03-22T04:12:35.612650+0000","last_became_active":"2024-03-22T04:12:33.617265+0000","last_became_peered":"2024-03-22T04:12:33.617265+0000","last_unstale":"2024-03-22T04:12:35.612650+0000","last_undegraded":"2024-03-22T04:12:35.612650+0000","last_fullsized":"2024-03-22T04:12:35.612650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615181+0000","last_change":"2024-03-22T04:12:34.635018+0000","last_active":"2024-03-22T04:12:35.615181+0000","last_peered":"2024-03-22T04:12:35.615181+0000","last_clean":"2024-03-22T04:12:35.615181+0000","last_became_active":"2024-03-22T04:12:34.634716+0000","last_became_peered":"2024-03-22T04:12:34.634716+0000","last_unstale":"2024-03-22T04:12:35.615181+0000","last_undegraded":"2024-03-22T04:12:35.615181+0000","last_fullsized":"2024-03-22T04:12:35.615181+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.658426+0000","last_change":"2024-03-22T04:12:33.623030+0000","last_active":"2024-03-22T04:12:34.658426+0000","last_peered":"2024-03-22T04:12:34.658426+0000","last_clean":"2024-03-22T04:12:34.658426+0000","last_became_active":"2024-03-22T04:12:33.622806+0000","last_became_peered":"2024-03-22T04:12:33.622806+0000","last_unstale":"2024-03-22T04:12:34.658426+0000","last_undegraded":"2024-03-22T04:12:34.658426+0000","last_fullsized":"2024-03-22T04:12:34.658426+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:50:40.568966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.641095+0000","last_change":"2024-03-22T04:12:34.635387+0000","last_active":"2024-03-22T04:12:34.641095+0000","last_peered":"2024-03-22T04:12:34.641095+0000","last_clean":"2024-03-22T04:12:34.641095+0000","last_became_active":"2024-03-22T04:12:34.632095+0000","last_became_peered":"2024-03-22T04:12:34.632095+0000","last_unstale":"2024-03-22T04:12:34.641095+0000","last_undegraded":"2024-03-22T04:12:34.641095+0000","last_fullsized":"2024-03-22T04:12:34.641095+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615307+0000","last_change":"2024-03-22T04:12:33.616617+0000","last_active":"2024-03-22T04:12:35.615307+0000","last_peered":"2024-03-22T04:12:35.615307+0000","last_clean":"2024-03-22T04:12:35.615307+0000","last_became_active":"2024-03-22T04:12:33.616088+0000","last_became_peered":"2024-03-22T04:12:33.616088+0000","last_unstale":"2024-03-22T04:12:35.615307+0000","last_undegraded":"2024-03-22T04:12:35.615307+0000","last_fullsized":"2024-03-22T04:12:35.615307+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:49.907740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.640577+0000","last_change":"2024-03-22T04:12:34.638920+0000","last_active":"2024-03-22T04:12:34.640577+0000","last_peered":"2024-03-22T04:12:34.640577+0000","last_clean":"2024-03-22T04:12:34.640577+0000","last_became_active":"2024-03-22T04:12:34.638167+0000","last_became_peered":"2024-03-22T04:12:34.638167+0000","last_unstale":"2024-03-22T04:12:34.640577+0000","last_undegraded":"2024-03-22T04:12:34.640577+0000","last_fullsized":"2024-03-22T04:12:34.640577+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:21:14.663985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.659863+0000","last_change":"2024-03-22T04:12:33.625471+0000","last_active":"2024-03-22T04:12:34.659863+0000","last_peered":"2024-03-22T04:12:34.659863+0000","last_clean":"2024-03-22T04:12:34.659863+0000","last_became_active":"2024-03-22T04:12:33.625189+0000","last_became_peered":"2024-03-22T04:12:33.625189+0000","last_unstale":"2024-03-22T04:12:34.659863+0000","last_undegraded":"2024-03-22T04:12:34.659863+0000","last_fullsized":"2024-03-22T04:12:34.659863+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615959+0000","last_change":"2024-03-22T04:12:34.642230+0000","last_active":"2024-03-22T04:12:35.615959+0000","last_peered":"2024-03-22T04:12:35.615959+0000","last_clean":"2024-03-22T04:12:35.615959+0000","last_became_active":"2024-03-22T04:12:34.642038+0000","last_became_peered":"2024-03-22T04:12:34.642038+0000","last_unstale":"2024-03-22T04:12:35.615959+0000","last_undegraded":"2024-03-22T04:12:35.615959+0000","last_fullsized":"2024-03-22T04:12:35.615959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612534+0000","last_change":"2024-03-22T04:12:33.620764+0000","last_active":"2024-03-22T04:12:35.612534+0000","last_peered":"2024-03-22T04:12:35.612534+0000","last_clean":"2024-03-22T04:12:35.612534+0000","last_became_active":"2024-03-22T04:12:33.620498+0000","last_became_peered":"2024-03-22T04:12:33.620498+0000","last_unstale":"2024-03-22T04:12:35.612534+0000","last_undegraded":"2024-03-22T04:12:35.612534+0000","last_fullsized":"2024-03-22T04:12:35.612534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.646765+0000","last_change":"2024-03-22T04:12:34.641493+0000","last_active":"2024-03-22T04:12:34.646765+0000","last_peered":"2024-03-22T04:12:34.646765+0000","last_clean":"2024-03-22T04:12:34.646765+0000","last_became_active":"2024-03-22T04:12:34.636610+0000","last_became_peered":"2024-03-22T04:12:34.636610+0000","last_unstale":"2024-03-22T04:12:34.646765+0000","last_undegraded":"2024-03-22T04:12:34.646765+0000","last_fullsized":"2024-03-22T04:12:34.646765+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:01.229933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.617067+0000","last_change":"2024-03-22T04:12:33.618015+0000","last_active":"2024-03-22T04:12:34.617067+0000","last_peered":"2024-03-22T04:12:34.617067+0000","last_clean":"2024-03-22T04:12:34.617067+0000","last_became_active":"2024-03-22T04:12:33.617794+0000","last_became_peered":"2024-03-22T04:12:33.617794+0000","last_unstale":"2024-03-22T04:12:34.617067+0000","last_undegraded":"2024-03-22T04:12:34.617067+0000","last_fullsized":"2024-03-22T04:12:34.617067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:00.222391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602266+0000","last_change":"2024-03-22T04:12:33.602266+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602266+0000","last_undegraded":"2024-03-22T04:12:33.602266+0000","last_fullsized":"2024-03-22T04:12:33.602266+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.661009+0000","last_change":"2024-03-22T04:12:33.621891+0000","last_active":"2024-03-22T04:12:34.661009+0000","last_peered":"2024-03-22T04:12:34.661009+0000","last_clean":"2024-03-22T04:12:34.661009+0000","last_became_active":"2024-03-22T04:12:33.621626+0000","last_became_peered":"2024-03-22T04:12:33.621626+0000","last_unstale":"2024-03-22T04:12:34.661009+0000","last_undegraded":"2024-03-22T04:12:34.661009+0000","last_fullsized":"2024-03-22T04:12:34.661009+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614700+0000","last_change":"2024-03-22T04:12:34.634352+0000","last_active":"2024-03-22T04:12:35.614700+0000","last_peered":"2024-03-22T04:12:35.614700+0000","last_clean":"2024-03-22T04:12:35.614700+0000","last_became_active":"2024-03-22T04:12:34.633729+0000","last_became_peered":"2024-03-22T04:12:34.633729+0000","last_unstale":"2024-03-22T04:12:35.614700+0000","last_undegraded":"2024-03-22T04:12:35.614700+0000","last_fullsized":"2024-03-22T04:12:35.614700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.660467+0000","last_change":"2024-03-22T04:12:33.623382+0000","last_active":"2024-03-22T04:12:34.660467+0000","last_peered":"2024-03-22T04:12:34.660467+0000","last_clean":"2024-03-22T04:12:34.660467+0000","last_became_active":"2024-03-22T04:12:33.622982+0000","last_became_peered":"2024-03-22T04:12:33.622982+0000","last_unstale":"2024-03-22T04:12:34.660467+0000","last_undegraded":"2024-03-22T04:12:34.660467+0000","last_fullsized":"2024-03-22T04:12:34.660467+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614924+0000","last_change":"2024-03-22T04:12:34.627065+0000","last_active":"2024-03-22T04:12:35.614924+0000","last_peered":"2024-03-22T04:12:35.614924+0000","last_clean":"2024-03-22T04:12:35.614924+0000","last_became_active":"2024-03-22T04:12:34.626777+0000","last_became_peered":"2024-03-22T04:12:34.626777+0000","last_unstale":"2024-03-22T04:12:35.614924+0000","last_undegraded":"2024-03-22T04:12:35.614924+0000","last_fullsized":"2024-03-22T04:12:35.614924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.658788+0000","last_change":"2024-03-22T04:12:33.622321+0000","last_active":"2024-03-22T04:12:34.658788+0000","last_peered":"2024-03-22T04:12:34.658788+0000","last_clean":"2024-03-22T04:12:34.658788+0000","last_became_active":"2024-03-22T04:12:33.622058+0000","last_became_peered":"2024-03-22T04:12:33.622058+0000","last_unstale":"2024-03-22T04:12:34.658788+0000","last_undegraded":"2024-03-22T04:12:34.658788+0000","last_fullsized":"2024-03-22T04:12:34.658788+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639497+0000","last_change":"2024-03-22T04:12:34.636289+0000","last_active":"2024-03-22T04:12:34.639497+0000","last_peered":"2024-03-22T04:12:34.639497+0000","last_clean":"2024-03-22T04:12:34.639497+0000","last_became_active":"2024-03-22T04:12:34.636025+0000","last_became_peered":"2024-03-22T04:12:34.636025+0000","last_unstale":"2024-03-22T04:12:34.639497+0000","last_undegraded":"2024-03-22T04:12:34.639497+0000","last_fullsized":"2024-03-22T04:12:34.639497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612548+0000","last_change":"2024-03-22T04:12:33.611878+0000","last_active":"2024-03-22T04:12:35.612548+0000","last_peered":"2024-03-22T04:12:35.612548+0000","last_clean":"2024-03-22T04:12:35.612548+0000","last_became_active":"2024-03-22T04:12:33.611537+0000","last_became_peered":"2024-03-22T04:12:33.611537+0000","last_unstale":"2024-03-22T04:12:35.612548+0000","last_undegraded":"2024-03-22T04:12:35.612548+0000","last_fullsized":"2024-03-22T04:12:35.612548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.653492+0000","last_change":"2024-03-22T04:12:34.650207+0000","last_active":"2024-03-22T04:12:34.653492+0000","last_peered":"2024-03-22T04:12:34.653492+0000","last_clean":"2024-03-22T04:12:34.653492+0000","last_became_active":"2024-03-22T04:12:34.649927+0000","last_became_peered":"2024-03-22T04:12:34.649927+0000","last_unstale":"2024-03-22T04:12:34.653492+0000","last_undegraded":"2024-03-22T04:12:34.653492+0000","last_fullsized":"2024-03-22T04:12:34.653492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:12.985866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.616557+0000","last_change":"2024-03-22T04:12:33.622557+0000","last_active":"2024-03-22T04:12:34.616557+0000","last_peered":"2024-03-22T04:12:34.616557+0000","last_clean":"2024-03-22T04:12:34.616557+0000","last_became_active":"2024-03-22T04:12:33.622240+0000","last_became_peered":"2024-03-22T04:12:33.622240+0000","last_unstale":"2024-03-22T04:12:34.616557+0000","last_undegraded":"2024-03-22T04:12:34.616557+0000","last_fullsized":"2024-03-22T04:12:34.616557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:32.007591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634871+0000","last_change":"2024-03-22T04:12:34.633656+0000","last_active":"2024-03-22T04:12:34.634871+0000","last_peered":"2024-03-22T04:12:34.634871+0000","last_clean":"2024-03-22T04:12:34.634871+0000","last_became_active":"2024-03-22T04:12:34.633150+0000","last_became_peered":"2024-03-22T04:12:34.633150+0000","last_unstale":"2024-03-22T04:12:34.634871+0000","last_undegraded":"2024-03-22T04:12:34.634871+0000","last_fullsized":"2024-03-22T04:12:34.634871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:33.344021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615428+0000","last_change":"2024-03-22T04:12:33.624581+0000","last_active":"2024-03-22T04:12:35.615428+0000","last_peered":"2024-03-22T04:12:35.615428+0000","last_clean":"2024-03-22T04:12:35.615428+0000","last_became_active":"2024-03-22T04:12:33.624324+0000","last_became_peered":"2024-03-22T04:12:33.624324+0000","last_unstale":"2024-03-22T04:12:35.615428+0000","last_undegraded":"2024-03-22T04:12:35.615428+0000","last_fullsized":"2024-03-22T04:12:35.615428+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:41.428263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612405+0000","last_change":"2024-03-22T04:12:34.631558+0000","last_active":"2024-03-22T04:12:35.612405+0000","last_peered":"2024-03-22T04:12:35.612405+0000","last_clean":"2024-03-22T04:12:35.612405+0000","last_became_active":"2024-03-22T04:12:34.621598+0000","last_became_peered":"2024-03-22T04:12:34.621598+0000","last_unstale":"2024-03-22T04:12:35.612405+0000","last_undegraded":"2024-03-22T04:12:35.612405+0000","last_fullsized":"2024-03-22T04:12:35.612405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:43:33.625589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.624342+0000","last_change":"2024-03-22T04:12:33.619281+0000","last_active":"2024-03-22T04:12:33.624342+0000","last_peered":"2024-03-22T04:12:33.624342+0000","last_clean":"2024-03-22T04:12:33.624342+0000","last_became_active":"2024-03-22T04:12:33.618935+0000","last_became_peered":"2024-03-22T04:12:33.618935+0000","last_unstale":"2024-03-22T04:12:33.624342+0000","last_undegraded":"2024-03-22T04:12:33.624342+0000","last_fullsized":"2024-03-22T04:12:33.624342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:11:53.935122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615777+0000","last_change":"2024-03-22T04:12:34.642768+0000","last_active":"2024-03-22T04:12:35.615777+0000","last_peered":"2024-03-22T04:12:35.615777+0000","last_clean":"2024-03-22T04:12:35.615777+0000","last_became_active":"2024-03-22T04:12:34.642534+0000","last_became_peered":"2024-03-22T04:12:34.642534+0000","last_unstale":"2024-03-22T04:12:35.615777+0000","last_undegraded":"2024-03-22T04:12:35.615777+0000","last_fullsized":"2024-03-22T04:12:35.615777+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615251+0000","last_change":"2024-03-22T04:12:33.626234+0000","last_active":"2024-03-22T04:12:35.615251+0000","last_peered":"2024-03-22T04:12:35.615251+0000","last_clean":"2024-03-22T04:12:35.615251+0000","last_became_active":"2024-03-22T04:12:33.623978+0000","last_became_peered":"2024-03-22T04:12:33.623978+0000","last_unstale":"2024-03-22T04:12:35.615251+0000","last_undegraded":"2024-03-22T04:12:35.615251+0000","last_fullsized":"2024-03-22T04:12:35.615251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:45.736028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.648376+0000","last_change":"2024-03-22T04:12:34.646431+0000","last_active":"2024-03-22T04:12:34.648376+0000","last_peered":"2024-03-22T04:12:34.648376+0000","last_clean":"2024-03-22T04:12:34.648376+0000","last_became_active":"2024-03-22T04:12:34.646176+0000","last_became_peered":"2024-03-22T04:12:34.646176+0000","last_unstale":"2024-03-22T04:12:34.648376+0000","last_undegraded":"2024-03-22T04:12:34.648376+0000","last_fullsized":"2024-03-22T04:12:34.648376+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:42.435805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614477+0000","last_change":"2024-03-22T04:12:33.610526+0000","last_active":"2024-03-22T04:12:35.614477+0000","last_peered":"2024-03-22T04:12:35.614477+0000","last_clean":"2024-03-22T04:12:35.614477+0000","last_became_active":"2024-03-22T04:12:33.609782+0000","last_became_peered":"2024-03-22T04:12:33.609782+0000","last_unstale":"2024-03-22T04:12:35.614477+0000","last_undegraded":"2024-03-22T04:12:35.614477+0000","last_fullsized":"2024-03-22T04:12:35.614477+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:11.978324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.611982+0000","last_change":"2024-03-22T04:12:34.622644+0000","last_active":"2024-03-22T04:12:35.611982+0000","last_peered":"2024-03-22T04:12:35.611982+0000","last_clean":"2024-03-22T04:12:35.611982+0000","last_became_active":"2024-03-22T04:12:34.622112+0000","last_became_peered":"2024-03-22T04:12:34.622112+0000","last_unstale":"2024-03-22T04:12:35.611982+0000","last_undegraded":"2024-03-22T04:12:35.611982+0000","last_fullsized":"2024-03-22T04:12:35.611982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657519+0000","last_change":"2024-03-22T04:12:33.621421+0000","last_active":"2024-03-22T04:12:34.657519+0000","last_peered":"2024-03-22T04:12:34.657519+0000","last_clean":"2024-03-22T04:12:34.657519+0000","last_became_active":"2024-03-22T04:12:33.621140+0000","last_became_peered":"2024-03-22T04:12:33.621140+0000","last_unstale":"2024-03-22T04:12:34.657519+0000","last_undegraded":"2024-03-22T04:12:34.657519+0000","last_fullsized":"2024-03-22T04:12:34.657519+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:21.786333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615749+0000","last_change":"2024-03-22T04:12:34.638291+0000","last_active":"2024-03-22T04:12:35.615749+0000","last_peered":"2024-03-22T04:12:35.615749+0000","last_clean":"2024-03-22T04:12:35.615749+0000","last_became_active":"2024-03-22T04:12:34.637953+0000","last_became_peered":"2024-03-22T04:12:34.637953+0000","last_unstale":"2024-03-22T04:12:35.615749+0000","last_undegraded":"2024-03-22T04:12:35.615749+0000","last_fullsized":"2024-03-22T04:12:35.615749+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:37:44.536324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.618566+0000","last_change":"2024-03-22T04:12:33.623102+0000","last_active":"2024-03-22T04:12:34.618566+0000","last_peered":"2024-03-22T04:12:34.618566+0000","last_clean":"2024-03-22T04:12:34.618566+0000","last_became_active":"2024-03-22T04:12:33.622803+0000","last_became_peered":"2024-03-22T04:12:33.622803+0000","last_unstale":"2024-03-22T04:12:34.618566+0000","last_undegraded":"2024-03-22T04:12:34.618566+0000","last_fullsized":"2024-03-22T04:12:34.618566+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:04:41.174822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602478+0000","last_change":"2024-03-22T04:12:33.602478+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602478+0000","last_undegraded":"2024-03-22T04:12:33.602478+0000","last_fullsized":"2024-03-22T04:12:33.602478+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.622958+0000","last_change":"2024-03-22T04:12:33.618302+0000","last_active":"2024-03-22T04:12:33.622958+0000","last_peered":"2024-03-22T04:12:33.622958+0000","last_clean":"2024-03-22T04:12:33.622958+0000","last_became_active":"2024-03-22T04:12:33.617971+0000","last_became_peered":"2024-03-22T04:12:33.617971+0000","last_unstale":"2024-03-22T04:12:33.622958+0000","last_undegraded":"2024-03-22T04:12:33.622958+0000","last_fullsized":"2024-03-22T04:12:33.622958+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:42.915758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615191+0000","last_change":"2024-03-22T04:12:34.644894+0000","last_active":"2024-03-22T04:12:35.615191+0000","last_peered":"2024-03-22T04:12:35.615191+0000","last_clean":"2024-03-22T04:12:35.615191+0000","last_became_active":"2024-03-22T04:12:34.644526+0000","last_became_peered":"2024-03-22T04:12:34.644526+0000","last_unstale":"2024-03-22T04:12:35.615191+0000","last_undegraded":"2024-03-22T04:12:35.615191+0000","last_fullsized":"2024-03-22T04:12:35.615191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615131+0000","last_change":"2024-03-22T04:12:33.620091+0000","last_active":"2024-03-22T04:12:35.615131+0000","last_peered":"2024-03-22T04:12:35.615131+0000","last_clean":"2024-03-22T04:12:35.615131+0000","last_became_active":"2024-03-22T04:12:33.619790+0000","last_became_peered":"2024-03-22T04:12:33.619790+0000","last_unstale":"2024-03-22T04:12:35.615131+0000","last_undegraded":"2024-03-22T04:12:35.615131+0000","last_fullsized":"2024-03-22T04:12:35.615131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:32.336479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639091+0000","last_change":"2024-03-22T04:12:34.636152+0000","last_active":"2024-03-22T04:12:34.639091+0000","last_peered":"2024-03-22T04:12:34.639091+0000","last_clean":"2024-03-22T04:12:34.639091+0000","last_became_active":"2024-03-22T04:12:34.635865+0000","last_became_peered":"2024-03-22T04:12:34.635865+0000","last_unstale":"2024-03-22T04:12:34.639091+0000","last_undegraded":"2024-03-22T04:12:34.639091+0000","last_fullsized":"2024-03-22T04:12:34.639091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.616064+0000","last_change":"2024-03-22T04:12:33.612435+0000","last_active":"2024-03-22T04:12:33.616064+0000","last_peered":"2024-03-22T04:12:33.616064+0000","last_clean":"2024-03-22T04:12:33.616064+0000","last_became_active":"2024-03-22T04:12:33.611642+0000","last_became_peered":"2024-03-22T04:12:33.611642+0000","last_unstale":"2024-03-22T04:12:33.616064+0000","last_undegraded":"2024-03-22T04:12:33.616064+0000","last_fullsized":"2024-03-22T04:12:33.616064+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:44:21.610192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615970+0000","last_change":"2024-03-22T04:12:34.637750+0000","last_active":"2024-03-22T04:12:35.615970+0000","last_peered":"2024-03-22T04:12:35.615970+0000","last_clean":"2024-03-22T04:12:35.615970+0000","last_became_active":"2024-03-22T04:12:34.637269+0000","last_became_peered":"2024-03-22T04:12:34.637269+0000","last_unstale":"2024-03-22T04:12:35.615970+0000","last_undegraded":"2024-03-22T04:12:35.615970+0000","last_fullsized":"2024-03-22T04:12:35.615970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:54:58.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.617221+0000","last_change":"2024-03-22T04:12:33.613711+0000","last_active":"2024-03-22T04:12:34.617221+0000","last_peered":"2024-03-22T04:12:34.617221+0000","last_clean":"2024-03-22T04:12:34.617221+0000","last_became_active":"2024-03-22T04:12:33.613398+0000","last_became_peered":"2024-03-22T04:12:33.613398+0000","last_unstale":"2024-03-22T04:12:34.617221+0000","last_undegraded":"2024-03-22T04:12:34.617221+0000","last_fullsized":"2024-03-22T04:12:34.617221+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:34.304317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.654950+0000","last_change":"2024-03-22T04:12:34.649939+0000","last_active":"2024-03-22T04:12:34.654950+0000","last_peered":"2024-03-22T04:12:34.654950+0000","last_clean":"2024-03-22T04:12:34.654950+0000","last_became_active":"2024-03-22T04:12:34.649639+0000","last_became_peered":"2024-03-22T04:12:34.649639+0000","last_unstale":"2024-03-22T04:12:34.654950+0000","last_undegraded":"2024-03-22T04:12:34.654950+0000","last_fullsized":"2024-03-22T04:12:34.654950+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:01.747862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612050+0000","last_change":"2024-03-22T04:12:33.613244+0000","last_active":"2024-03-22T04:12:35.612050+0000","last_peered":"2024-03-22T04:12:35.612050+0000","last_clean":"2024-03-22T04:12:35.612050+0000","last_became_active":"2024-03-22T04:12:33.612972+0000","last_became_peered":"2024-03-22T04:12:33.612972+0000","last_unstale":"2024-03-22T04:12:35.612050+0000","last_undegraded":"2024-03-22T04:12:35.612050+0000","last_fullsized":"2024-03-22T04:12:35.612050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637167+0000","last_change":"2024-03-22T04:12:34.638813+0000","last_active":"2024-03-22T04:12:35.637167+0000","last_peered":"2024-03-22T04:12:35.637167+0000","last_clean":"2024-03-22T04:12:35.637167+0000","last_became_active":"2024-03-22T04:12:34.638487+0000","last_became_peered":"2024-03-22T04:12:34.638487+0000","last_unstale":"2024-03-22T04:12:35.637167+0000","last_undegraded":"2024-03-22T04:12:35.637167+0000","last_fullsized":"2024-03-22T04:12:35.637167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637130+0000","last_change":"2024-03-22T04:12:33.624669+0000","last_active":"2024-03-22T04:12:35.637130+0000","last_peered":"2024-03-22T04:12:35.637130+0000","last_clean":"2024-03-22T04:12:35.637130+0000","last_became_active":"2024-03-22T04:12:33.624416+0000","last_became_peered":"2024-03-22T04:12:33.624416+0000","last_unstale":"2024-03-22T04:12:35.637130+0000","last_undegraded":"2024-03-22T04:12:35.637130+0000","last_fullsized":"2024-03-22T04:12:35.637130+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:18.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":227,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612039+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:12:35.612039+0000","last_peered":"2024-03-22T04:12:35.612039+0000","last_clean":"2024-03-22T04:12:35.612039+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:12:35.612039+0000","last_undegraded":"2024-03-22T04:12:35.612039+0000","last_fullsized":"2024-03-22T04:12:35.612039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":273831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":274357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":865845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":273865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":273809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":870905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":274357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":598016,"data_stored":259941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:12:38.698 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:12:38.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.865+0000 7f152a000640 1 Processor -- start 2024-03-22T04:12:38.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.865+0000 7f152a000640 1 -- start start 2024-03-22T04:12:38.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f152a000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152415fed0 0x7f1524160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f152a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15240d11c0 con 0x7f15240d0de0 2024-03-22T04:12:38.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f152a000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15240cece0 con 0x7f15240ce6b0 2024-03-22T04:12:38.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f152a000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15240d0580 con 0x7f152415fed0 2024-03-22T04:12:38.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1523400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f15240d0de0 0x7f15240d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57228/0 (socket says 172.21.15.86:57228) 2024-03-22T04:12:38.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1523400640 1 -- 172.21.15.86:0/247955637 learned_addr learned my addr 172.21.15.86:0/247955637 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:38.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1523e00640 1 --2- 172.21.15.86:0/247955637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152415fed0 0x7f1524160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1522a00640 1 -- 172.21.15.86:0/247955637 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2537295617 0 0) 0x7f15240d11c0 con 0x7f15240d0de0 2024-03-22T04:12:38.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1522a00640 1 -- 172.21.15.86:0/247955637 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1508003530 con 0x7f15240d0de0 2024-03-22T04:12:38.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1522a00640 1 -- 172.21.15.86:0/247955637 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037781632 0 0) 0x7f15240cece0 con 0x7f15240ce6b0 2024-03-22T04:12:38.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1523e00640 1 -- 172.21.15.86:0/247955637 >> v1:172.21.15.196:6789/0 conn(0x7f15240ce6b0 legacy=0x7f15240cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1523e00640 1 -- 172.21.15.86:0/247955637 >> v1:172.21.15.86:6789/0 conn(0x7f15240d0de0 legacy=0x7f15240d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.869+0000 7f1523e00640 1 -- 172.21.15.86:0/247955637 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15240cefc0 con 0x7f152415fed0 2024-03-22T04:12:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f1523e00640 1 --2- 172.21.15.86:0/247955637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152415fed0 0x7f1524160320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f151c009dc0 tx=0x7f151c01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d3642fd35f77565 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f1522a00640 1 -- 172.21.15.86:0/247955637 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0b2070 con 0x7f152415fed0 2024-03-22T04:12:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f1522a00640 1 -- 172.21.15.86:0/247955637 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151c0a5850 con 0x7f152415fed0 2024-03-22T04:12:38.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f1522a00640 1 -- 172.21.15.86:0/247955637 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0a5c20 con 0x7f152415fed0 2024-03-22T04:12:38.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 -- 172.21.15.86:0/247955637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152415fed0 msgr2=0x7f1524160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 --2- 172.21.15.86:0/247955637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152415fed0 0x7f1524160320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f151c009dc0 tx=0x7f151c01c990 comp rx=0 tx=0).stop 2024-03-22T04:12:38.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 -- 172.21.15.86:0/247955637 shutdown_connections 2024-03-22T04:12:38.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 --2- 172.21.15.86:0/247955637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152415fed0 0x7f1524160320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 -- 172.21.15.86:0/247955637 >> 172.21.15.86:0/247955637 conn(0x7f152406f910 msgr2=0x7f152406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:38.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 -- 172.21.15.86:0/247955637 shutdown_connections 2024-03-22T04:12:38.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 -- 172.21.15.86:0/247955637 wait complete. 2024-03-22T04:12:38.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 Processor -- start 2024-03-22T04:12:38.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.873+0000 7f152a000640 1 -- start start 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15240ce6b0 0x7f1524142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15240d0de0 0x7f15240c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152415fed0 0x7f1524156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1524071e30 con 0x7f15240ce6b0 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1524160860 con 0x7f152415fed0 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1524071fb0 con 0x7f15240d0de0 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15240ce6b0 0x7f1524142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15240ce6b0 0x7f1524142070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51432/0 (socket says 172.21.15.86:51432) 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 -- 172.21.15.86:0/1474879336 learned_addr learned my addr 172.21.15.86:0/1474879336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f151aa00640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15240d0de0 0x7f15240c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523e00640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152415fed0 0x7f1524156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 -- 172.21.15.86:0/1474879336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15240d0de0 msgr2=0x7f15240c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15240d0de0 0x7f15240c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 -- 172.21.15.86:0/1474879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152415fed0 msgr2=0x7f1524156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152415fed0 0x7f1524156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 -- 172.21.15.86:0/1474879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1524155240 con 0x7f15240ce6b0 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f1523400640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15240ce6b0 0x7f1524142070 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f151400c450 tx=0x7f151400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f151be00640 1 -- 172.21.15.86:0/1474879336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514017070 con 0x7f15240ce6b0 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f151be00640 1 -- 172.21.15.86:0/1474879336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1514005cf0 con 0x7f15240ce6b0 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1524155520 con 0x7f15240ce6b0 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f151be00640 1 -- 172.21.15.86:0/1474879336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514007290 con 0x7f15240ce6b0 2024-03-22T04:12:38.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152406e080 con 0x7f15240ce6b0 2024-03-22T04:12:38.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152406de70 con 0x7f15240ce6b0 2024-03-22T04:12:38.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.877+0000 7f151be00640 1 -- 172.21.15.86:0/1474879336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f151401a070 con 0x7f15240ce6b0 2024-03-22T04:12:38.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.881+0000 7f151be00640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1500067b20 0x7f1500069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:38.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.881+0000 7f151be00640 1 -- 172.21.15.86:0/1474879336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f1514085380 con 0x7f15240ce6b0 2024-03-22T04:12:38.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.881+0000 7f151aa00640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1500067b20 0x7f1500069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:38.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.881+0000 7f151aa00640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1500067b20 0x7f1500069fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f15240b17d0 tx=0x7f150c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:38.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:38.885+0000 7f151be00640 1 -- 172.21.15.86:0/1474879336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f15140555b0 con 0x7f15240ce6b0 2024-03-22T04:12:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.037+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1524068650 con 0x7f1500067b20 2024-03-22T04:12:39.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.041+0000 7f151be00640 1 -- 172.21.15.86:0/1474879336 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251107 (secure 0 0 0) 0x7f1524068650 con 0x7f1500067b20 2024-03-22T04:12:39.048 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:39.049 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:12:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1500067b20 msgr2=0x7f1500069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1500067b20 0x7f1500069fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f15240b17d0 tx=0x7f150c009040 comp rx=0 tx=0).stop 2024-03-22T04:12:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15240ce6b0 msgr2=0x7f1524142070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15240ce6b0 0x7f1524142070 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f151400c450 tx=0x7f151400c910 comp rx=0 tx=0).stop 2024-03-22T04:12:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 shutdown_connections 2024-03-22T04:12:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1500067b20 0x7f1500069fe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152415fed0 0x7f1524156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15240d0de0 0x7f15240c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 --2- 172.21.15.86:0/1474879336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15240ce6b0 0x7f1524142070 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 >> 172.21.15.86:0/1474879336 conn(0x7f152406f910 msgr2=0x7f152415a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 shutdown_connections 2024-03-22T04:12:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.045+0000 7f152a000640 1 -- 172.21.15.86:0/1474879336 wait complete. 2024-03-22T04:12:39.068 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-03-22T04:12:37.816229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222236,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752484,"statfs":{"total":773094113280,"available":772866543616,"internally_reserved":0,"allocated":6201344,"data_stored":3366910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220955"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616060+0000","last_change":"2024-03-22T04:12:34.628936+0000","last_active":"2024-03-22T04:12:35.616060+0000","last_peered":"2024-03-22T04:12:35.616060+0000","last_clean":"2024-03-22T04:12:35.616060+0000","last_became_active":"2024-03-22T04:12:34.628641+0000","last_became_peered":"2024-03-22T04:12:34.628641+0000","last_unstale":"2024-03-22T04:12:35.616060+0000","last_undegraded":"2024-03-22T04:12:35.616060+0000","last_fullsized":"2024-03-22T04:12:35.616060+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639641+0000","last_change":"2024-03-22T04:12:34.637267+0000","last_active":"2024-03-22T04:12:34.639641+0000","last_peered":"2024-03-22T04:12:34.639641+0000","last_clean":"2024-03-22T04:12:34.639641+0000","last_became_active":"2024-03-22T04:12:34.637024+0000","last_became_peered":"2024-03-22T04:12:34.637024+0000","last_unstale":"2024-03-22T04:12:34.639641+0000","last_undegraded":"2024-03-22T04:12:34.639641+0000","last_fullsized":"2024-03-22T04:12:34.639641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603940+0000","last_change":"2024-03-22T04:12:33.603940+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603940+0000","last_undegraded":"2024-03-22T04:12:33.603940+0000","last_fullsized":"2024-03-22T04:12:33.603940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612444+0000","last_change":"2024-03-22T04:12:34.625832+0000","last_active":"2024-03-22T04:12:35.612444+0000","last_peered":"2024-03-22T04:12:35.612444+0000","last_clean":"2024-03-22T04:12:35.612444+0000","last_became_active":"2024-03-22T04:12:34.625533+0000","last_became_peered":"2024-03-22T04:12:34.625533+0000","last_unstale":"2024-03-22T04:12:35.612444+0000","last_undegraded":"2024-03-22T04:12:35.612444+0000","last_fullsized":"2024-03-22T04:12:35.612444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:00.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.635523+0000","last_change":"2024-03-22T04:12:34.627773+0000","last_active":"2024-03-22T04:12:34.635523+0000","last_peered":"2024-03-22T04:12:34.635523+0000","last_clean":"2024-03-22T04:12:34.635523+0000","last_became_active":"2024-03-22T04:12:34.624749+0000","last_became_peered":"2024-03-22T04:12:34.624749+0000","last_unstale":"2024-03-22T04:12:34.635523+0000","last_undegraded":"2024-03-22T04:12:34.635523+0000","last_fullsized":"2024-03-22T04:12:34.635523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:01.747862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643345+0000","last_change":"2024-03-22T04:12:34.635777+0000","last_active":"2024-03-22T04:12:34.643345+0000","last_peered":"2024-03-22T04:12:34.643345+0000","last_clean":"2024-03-22T04:12:34.643345+0000","last_became_active":"2024-03-22T04:12:34.635528+0000","last_became_peered":"2024-03-22T04:12:34.635528+0000","last_unstale":"2024-03-22T04:12:34.643345+0000","last_undegraded":"2024-03-22T04:12:34.643345+0000","last_fullsized":"2024-03-22T04:12:34.643345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:01:53.135313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614543+0000","last_change":"2024-03-22T04:12:34.636516+0000","last_active":"2024-03-22T04:12:35.614543+0000","last_peered":"2024-03-22T04:12:35.614543+0000","last_clean":"2024-03-22T04:12:35.614543+0000","last_became_active":"2024-03-22T04:12:34.636312+0000","last_became_peered":"2024-03-22T04:12:34.636312+0000","last_unstale":"2024-03-22T04:12:35.614543+0000","last_undegraded":"2024-03-22T04:12:35.614543+0000","last_fullsized":"2024-03-22T04:12:35.614543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657816+0000","last_change":"2024-03-22T04:12:34.650010+0000","last_active":"2024-03-22T04:12:34.657816+0000","last_peered":"2024-03-22T04:12:34.657816+0000","last_clean":"2024-03-22T04:12:34.657816+0000","last_became_active":"2024-03-22T04:12:34.649709+0000","last_became_peered":"2024-03-22T04:12:34.649709+0000","last_unstale":"2024-03-22T04:12:34.657816+0000","last_undegraded":"2024-03-22T04:12:34.657816+0000","last_fullsized":"2024-03-22T04:12:34.657816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:50.915283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603153+0000","last_change":"2024-03-22T04:12:33.603153+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603153+0000","last_undegraded":"2024-03-22T04:12:33.603153+0000","last_fullsized":"2024-03-22T04:12:33.603153+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615323+0000","last_change":"2024-03-22T04:12:34.641332+0000","last_active":"2024-03-22T04:12:35.615323+0000","last_peered":"2024-03-22T04:12:35.615323+0000","last_clean":"2024-03-22T04:12:35.615323+0000","last_became_active":"2024-03-22T04:12:34.641059+0000","last_became_peered":"2024-03-22T04:12:34.641059+0000","last_unstale":"2024-03-22T04:12:35.615323+0000","last_undegraded":"2024-03-22T04:12:35.615323+0000","last_fullsized":"2024-03-22T04:12:35.615323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615824+0000","last_change":"2024-03-22T04:12:34.627929+0000","last_active":"2024-03-22T04:12:35.615824+0000","last_peered":"2024-03-22T04:12:35.615824+0000","last_clean":"2024-03-22T04:12:35.615824+0000","last_became_active":"2024-03-22T04:12:34.627640+0000","last_became_peered":"2024-03-22T04:12:34.627640+0000","last_unstale":"2024-03-22T04:12:35.615824+0000","last_undegraded":"2024-03-22T04:12:35.615824+0000","last_fullsized":"2024-03-22T04:12:35.615824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643682+0000","last_change":"2024-03-22T04:12:34.639709+0000","last_active":"2024-03-22T04:12:34.643682+0000","last_peered":"2024-03-22T04:12:34.643682+0000","last_clean":"2024-03-22T04:12:34.643682+0000","last_became_active":"2024-03-22T04:12:34.638936+0000","last_became_peered":"2024-03-22T04:12:34.638936+0000","last_unstale":"2024-03-22T04:12:34.643682+0000","last_undegraded":"2024-03-22T04:12:34.643682+0000","last_fullsized":"2024-03-22T04:12:34.643682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:33.015134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.647737+0000","last_change":"2024-03-22T04:12:34.644837+0000","last_active":"2024-03-22T04:12:34.647737+0000","last_peered":"2024-03-22T04:12:34.647737+0000","last_clean":"2024-03-22T04:12:34.647737+0000","last_became_active":"2024-03-22T04:12:34.644507+0000","last_became_peered":"2024-03-22T04:12:34.644507+0000","last_unstale":"2024-03-22T04:12:34.647737+0000","last_undegraded":"2024-03-22T04:12:34.647737+0000","last_fullsized":"2024-03-22T04:12:34.647737+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:35.311859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614973+0000","last_change":"2024-03-22T04:12:34.641248+0000","last_active":"2024-03-22T04:12:35.614973+0000","last_peered":"2024-03-22T04:12:35.614973+0000","last_clean":"2024-03-22T04:12:35.614973+0000","last_became_active":"2024-03-22T04:12:34.640953+0000","last_became_peered":"2024-03-22T04:12:34.640953+0000","last_unstale":"2024-03-22T04:12:35.614973+0000","last_undegraded":"2024-03-22T04:12:35.614973+0000","last_fullsized":"2024-03-22T04:12:35.614973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612098+0000","last_change":"2024-03-22T04:12:34.632175+0000","last_active":"2024-03-22T04:12:35.612098+0000","last_peered":"2024-03-22T04:12:35.612098+0000","last_clean":"2024-03-22T04:12:35.612098+0000","last_became_active":"2024-03-22T04:12:34.631916+0000","last_became_peered":"2024-03-22T04:12:34.631916+0000","last_unstale":"2024-03-22T04:12:35.612098+0000","last_undegraded":"2024-03-22T04:12:35.612098+0000","last_fullsized":"2024-03-22T04:12:35.612098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643732+0000","last_change":"2024-03-22T04:12:34.639122+0000","last_active":"2024-03-22T04:12:34.643732+0000","last_peered":"2024-03-22T04:12:34.643732+0000","last_clean":"2024-03-22T04:12:34.643732+0000","last_became_active":"2024-03-22T04:12:34.638788+0000","last_became_peered":"2024-03-22T04:12:34.638788+0000","last_unstale":"2024-03-22T04:12:34.643732+0000","last_undegraded":"2024-03-22T04:12:34.643732+0000","last_fullsized":"2024-03-22T04:12:34.643732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:27.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616261+0000","last_change":"2024-03-22T04:12:34.632974+0000","last_active":"2024-03-22T04:12:35.616261+0000","last_peered":"2024-03-22T04:12:35.616261+0000","last_clean":"2024-03-22T04:12:35.616261+0000","last_became_active":"2024-03-22T04:12:34.627665+0000","last_became_peered":"2024-03-22T04:12:34.627665+0000","last_unstale":"2024-03-22T04:12:35.616261+0000","last_undegraded":"2024-03-22T04:12:35.616261+0000","last_fullsized":"2024-03-22T04:12:35.616261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612642+0000","last_change":"2024-03-22T04:12:34.626077+0000","last_active":"2024-03-22T04:12:35.612642+0000","last_peered":"2024-03-22T04:12:35.612642+0000","last_clean":"2024-03-22T04:12:35.612642+0000","last_became_active":"2024-03-22T04:12:34.625837+0000","last_became_peered":"2024-03-22T04:12:34.625837+0000","last_unstale":"2024-03-22T04:12:35.612642+0000","last_undegraded":"2024-03-22T04:12:35.612642+0000","last_fullsized":"2024-03-22T04:12:35.612642+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603525+0000","last_change":"2024-03-22T04:12:33.603525+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603525+0000","last_undegraded":"2024-03-22T04:12:33.603525+0000","last_fullsized":"2024-03-22T04:12:33.603525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634836+0000","last_change":"2024-03-22T04:12:34.630270+0000","last_active":"2024-03-22T04:12:34.634836+0000","last_peered":"2024-03-22T04:12:34.634836+0000","last_clean":"2024-03-22T04:12:34.634836+0000","last_became_active":"2024-03-22T04:12:34.629989+0000","last_became_peered":"2024-03-22T04:12:34.629989+0000","last_unstale":"2024-03-22T04:12:34.634836+0000","last_undegraded":"2024-03-22T04:12:34.634836+0000","last_fullsized":"2024-03-22T04:12:34.634836+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:50.915283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612164+0000","last_change":"2024-03-22T04:12:34.620830+0000","last_active":"2024-03-22T04:12:35.612164+0000","last_peered":"2024-03-22T04:12:35.612164+0000","last_clean":"2024-03-22T04:12:35.612164+0000","last_became_active":"2024-03-22T04:12:34.620520+0000","last_became_peered":"2024-03-22T04:12:34.620520+0000","last_unstale":"2024-03-22T04:12:35.612164+0000","last_undegraded":"2024-03-22T04:12:35.612164+0000","last_fullsized":"2024-03-22T04:12:35.612164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.604396+0000","last_change":"2024-03-22T04:12:33.604396+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.604396+0000","last_undegraded":"2024-03-22T04:12:33.604396+0000","last_fullsized":"2024-03-22T04:12:33.604396+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602445+0000","last_change":"2024-03-22T04:12:33.602445+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602445+0000","last_undegraded":"2024-03-22T04:12:33.602445+0000","last_fullsized":"2024-03-22T04:12:33.602445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603931+0000","last_change":"2024-03-22T04:12:33.603931+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603931+0000","last_undegraded":"2024-03-22T04:12:33.603931+0000","last_fullsized":"2024-03-22T04:12:33.603931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616176+0000","last_change":"2024-03-22T04:12:34.635548+0000","last_active":"2024-03-22T04:12:35.616176+0000","last_peered":"2024-03-22T04:12:35.616176+0000","last_clean":"2024-03-22T04:12:35.616176+0000","last_became_active":"2024-03-22T04:12:34.635270+0000","last_became_peered":"2024-03-22T04:12:34.635270+0000","last_unstale":"2024-03-22T04:12:35.616176+0000","last_undegraded":"2024-03-22T04:12:35.616176+0000","last_fullsized":"2024-03-22T04:12:35.616176+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.604463+0000","last_change":"2024-03-22T04:12:33.604463+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.604463+0000","last_undegraded":"2024-03-22T04:12:33.604463+0000","last_fullsized":"2024-03-22T04:12:33.604463+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.659926+0000","last_change":"2024-03-22T04:12:34.655920+0000","last_active":"2024-03-22T04:12:34.659926+0000","last_peered":"2024-03-22T04:12:34.659926+0000","last_clean":"2024-03-22T04:12:34.659926+0000","last_became_active":"2024-03-22T04:12:34.655580+0000","last_became_peered":"2024-03-22T04:12:34.655580+0000","last_unstale":"2024-03-22T04:12:34.659926+0000","last_undegraded":"2024-03-22T04:12:34.659926+0000","last_fullsized":"2024-03-22T04:12:34.659926+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:33.344021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634856+0000","last_change":"2024-03-22T04:12:34.633424+0000","last_active":"2024-03-22T04:12:34.634856+0000","last_peered":"2024-03-22T04:12:34.634856+0000","last_clean":"2024-03-22T04:12:34.634856+0000","last_became_active":"2024-03-22T04:12:34.632797+0000","last_became_peered":"2024-03-22T04:12:34.632797+0000","last_unstale":"2024-03-22T04:12:34.634856+0000","last_undegraded":"2024-03-22T04:12:34.634856+0000","last_fullsized":"2024-03-22T04:12:34.634856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:12.985866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615318+0000","last_change":"2024-03-22T04:12:34.630952+0000","last_active":"2024-03-22T04:12:35.615318+0000","last_peered":"2024-03-22T04:12:35.615318+0000","last_clean":"2024-03-22T04:12:35.615318+0000","last_became_active":"2024-03-22T04:12:34.630765+0000","last_became_peered":"2024-03-22T04:12:34.630765+0000","last_unstale":"2024-03-22T04:12:35.615318+0000","last_undegraded":"2024-03-22T04:12:35.615318+0000","last_fullsized":"2024-03-22T04:12:35.615318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.625106+0000","last_change":"2024-03-22T04:12:34.621885+0000","last_active":"2024-03-22T04:12:34.625106+0000","last_peered":"2024-03-22T04:12:34.625106+0000","last_clean":"2024-03-22T04:12:34.625106+0000","last_became_active":"2024-03-22T04:12:34.621626+0000","last_became_peered":"2024-03-22T04:12:34.621626+0000","last_unstale":"2024-03-22T04:12:34.625106+0000","last_undegraded":"2024-03-22T04:12:34.625106+0000","last_fullsized":"2024-03-22T04:12:34.625106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:12:20.702700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.601918+0000","last_change":"2024-03-22T04:12:33.601918+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.601918+0000","last_undegraded":"2024-03-22T04:12:33.601918+0000","last_fullsized":"2024-03-22T04:12:33.601918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.630607+0000","last_change":"2024-03-22T04:12:34.624213+0000","last_active":"2024-03-22T04:12:34.630607+0000","last_peered":"2024-03-22T04:12:34.630607+0000","last_clean":"2024-03-22T04:12:34.630607+0000","last_became_active":"2024-03-22T04:12:34.623954+0000","last_became_peered":"2024-03-22T04:12:34.623954+0000","last_unstale":"2024-03-22T04:12:34.630607+0000","last_undegraded":"2024-03-22T04:12:34.630607+0000","last_fullsized":"2024-03-22T04:12:34.630607+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:42.435805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.629445+0000","last_change":"2024-03-22T04:12:34.623255+0000","last_active":"2024-03-22T04:12:34.629445+0000","last_peered":"2024-03-22T04:12:34.629445+0000","last_clean":"2024-03-22T04:12:34.629445+0000","last_became_active":"2024-03-22T04:12:34.622995+0000","last_became_peered":"2024-03-22T04:12:34.622995+0000","last_unstale":"2024-03-22T04:12:34.629445+0000","last_undegraded":"2024-03-22T04:12:34.629445+0000","last_fullsized":"2024-03-22T04:12:34.629445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:50.075295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637402+0000","last_change":"2024-03-22T04:12:33.618251+0000","last_active":"2024-03-22T04:12:35.637402+0000","last_peered":"2024-03-22T04:12:35.637402+0000","last_clean":"2024-03-22T04:12:35.637402+0000","last_became_active":"2024-03-22T04:12:33.617405+0000","last_became_peered":"2024-03-22T04:12:33.617405+0000","last_unstale":"2024-03-22T04:12:35.637402+0000","last_undegraded":"2024-03-22T04:12:35.637402+0000","last_fullsized":"2024-03-22T04:12:35.637402+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:45.786716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.636934+0000","last_change":"2024-03-22T04:12:34.633188+0000","last_active":"2024-03-22T04:12:34.636934+0000","last_peered":"2024-03-22T04:12:34.636934+0000","last_clean":"2024-03-22T04:12:34.636934+0000","last_became_active":"2024-03-22T04:12:34.632933+0000","last_became_peered":"2024-03-22T04:12:34.632933+0000","last_unstale":"2024-03-22T04:12:34.636934+0000","last_undegraded":"2024-03-22T04:12:34.636934+0000","last_fullsized":"2024-03-22T04:12:34.636934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:30:04.785433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.660378+0000","last_change":"2024-03-22T04:12:33.623741+0000","last_active":"2024-03-22T04:12:34.660378+0000","last_peered":"2024-03-22T04:12:34.660378+0000","last_clean":"2024-03-22T04:12:34.660378+0000","last_became_active":"2024-03-22T04:12:33.623407+0000","last_became_peered":"2024-03-22T04:12:33.623407+0000","last_unstale":"2024-03-22T04:12:34.660378+0000","last_undegraded":"2024-03-22T04:12:34.660378+0000","last_fullsized":"2024-03-22T04:12:34.660378+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.625128+0000","last_change":"2024-03-22T04:12:34.622229+0000","last_active":"2024-03-22T04:12:34.625128+0000","last_peered":"2024-03-22T04:12:34.625128+0000","last_clean":"2024-03-22T04:12:34.625128+0000","last_became_active":"2024-03-22T04:12:34.621209+0000","last_became_peered":"2024-03-22T04:12:34.621209+0000","last_unstale":"2024-03-22T04:12:34.625128+0000","last_undegraded":"2024-03-22T04:12:34.625128+0000","last_fullsized":"2024-03-22T04:12:34.625128+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:14:10.558919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615885+0000","last_change":"2024-03-22T04:12:33.621633+0000","last_active":"2024-03-22T04:12:35.615885+0000","last_peered":"2024-03-22T04:12:35.615885+0000","last_clean":"2024-03-22T04:12:35.615885+0000","last_became_active":"2024-03-22T04:12:33.621320+0000","last_became_peered":"2024-03-22T04:12:33.621320+0000","last_unstale":"2024-03-22T04:12:35.615885+0000","last_undegraded":"2024-03-22T04:12:35.615885+0000","last_fullsized":"2024-03-22T04:12:35.615885+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:00.740320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615027+0000","last_change":"2024-03-22T04:12:34.638992+0000","last_active":"2024-03-22T04:12:35.615027+0000","last_peered":"2024-03-22T04:12:35.615027+0000","last_clean":"2024-03-22T04:12:35.615027+0000","last_became_active":"2024-03-22T04:12:34.638706+0000","last_became_peered":"2024-03-22T04:12:34.638706+0000","last_unstale":"2024-03-22T04:12:35.615027+0000","last_undegraded":"2024-03-22T04:12:35.615027+0000","last_fullsized":"2024-03-22T04:12:35.615027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657480+0000","last_change":"2024-03-22T04:12:33.619503+0000","last_active":"2024-03-22T04:12:34.657480+0000","last_peered":"2024-03-22T04:12:34.657480+0000","last_clean":"2024-03-22T04:12:34.657480+0000","last_became_active":"2024-03-22T04:12:33.619218+0000","last_became_peered":"2024-03-22T04:12:33.619218+0000","last_unstale":"2024-03-22T04:12:34.657480+0000","last_undegraded":"2024-03-22T04:12:34.657480+0000","last_fullsized":"2024-03-22T04:12:34.657480+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:26.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637412+0000","last_change":"2024-03-22T04:12:34.638372+0000","last_active":"2024-03-22T04:12:35.637412+0000","last_peered":"2024-03-22T04:12:35.637412+0000","last_clean":"2024-03-22T04:12:35.637412+0000","last_became_active":"2024-03-22T04:12:34.636653+0000","last_became_peered":"2024-03-22T04:12:34.636653+0000","last_unstale":"2024-03-22T04:12:35.637412+0000","last_undegraded":"2024-03-22T04:12:35.637412+0000","last_fullsized":"2024-03-22T04:12:35.637412+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.625947+0000","last_change":"2024-03-22T04:12:33.621816+0000","last_active":"2024-03-22T04:12:33.625947+0000","last_peered":"2024-03-22T04:12:33.625947+0000","last_clean":"2024-03-22T04:12:33.625947+0000","last_became_active":"2024-03-22T04:12:33.621560+0000","last_became_peered":"2024-03-22T04:12:33.621560+0000","last_unstale":"2024-03-22T04:12:33.625947+0000","last_undegraded":"2024-03-22T04:12:33.625947+0000","last_fullsized":"2024-03-22T04:12:33.625947+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:49.010265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639031+0000","last_change":"2024-03-22T04:12:34.633399+0000","last_active":"2024-03-22T04:12:34.639031+0000","last_peered":"2024-03-22T04:12:34.639031+0000","last_clean":"2024-03-22T04:12:34.639031+0000","last_became_active":"2024-03-22T04:12:34.633082+0000","last_became_peered":"2024-03-22T04:12:34.633082+0000","last_unstale":"2024-03-22T04:12:34.639031+0000","last_undegraded":"2024-03-22T04:12:34.639031+0000","last_fullsized":"2024-03-22T04:12:34.639031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:39:21.503071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.651676+0000","last_change":"2024-03-22T04:12:33.615641+0000","last_active":"2024-03-22T04:12:34.651676+0000","last_peered":"2024-03-22T04:12:34.651676+0000","last_clean":"2024-03-22T04:12:34.651676+0000","last_became_active":"2024-03-22T04:12:33.615372+0000","last_became_peered":"2024-03-22T04:12:33.615372+0000","last_unstale":"2024-03-22T04:12:34.651676+0000","last_undegraded":"2024-03-22T04:12:34.651676+0000","last_fullsized":"2024-03-22T04:12:34.651676+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:39.482917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615462+0000","last_change":"2024-03-22T04:12:34.637539+0000","last_active":"2024-03-22T04:12:35.615462+0000","last_peered":"2024-03-22T04:12:35.615462+0000","last_clean":"2024-03-22T04:12:35.615462+0000","last_became_active":"2024-03-22T04:12:34.636918+0000","last_became_peered":"2024-03-22T04:12:34.636918+0000","last_unstale":"2024-03-22T04:12:35.615462+0000","last_undegraded":"2024-03-22T04:12:35.615462+0000","last_fullsized":"2024-03-22T04:12:35.615462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.618429+0000","last_change":"2024-03-22T04:12:33.615588+0000","last_active":"2024-03-22T04:12:34.618429+0000","last_peered":"2024-03-22T04:12:34.618429+0000","last_clean":"2024-03-22T04:12:34.618429+0000","last_became_active":"2024-03-22T04:12:33.615259+0000","last_became_peered":"2024-03-22T04:12:33.615259+0000","last_unstale":"2024-03-22T04:12:34.618429+0000","last_undegraded":"2024-03-22T04:12:34.618429+0000","last_fullsized":"2024-03-22T04:12:34.618429+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:27:55.035247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612760+0000","last_change":"2024-03-22T04:12:34.635762+0000","last_active":"2024-03-22T04:12:35.612760+0000","last_peered":"2024-03-22T04:12:35.612760+0000","last_clean":"2024-03-22T04:12:35.612760+0000","last_became_active":"2024-03-22T04:12:34.635493+0000","last_became_peered":"2024-03-22T04:12:34.635493+0000","last_unstale":"2024-03-22T04:12:35.612760+0000","last_undegraded":"2024-03-22T04:12:35.612760+0000","last_fullsized":"2024-03-22T04:12:35.612760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.622856+0000","last_change":"2024-03-22T04:12:33.618851+0000","last_active":"2024-03-22T04:12:33.622856+0000","last_peered":"2024-03-22T04:12:33.622856+0000","last_clean":"2024-03-22T04:12:33.622856+0000","last_became_active":"2024-03-22T04:12:33.618525+0000","last_became_peered":"2024-03-22T04:12:33.618525+0000","last_unstale":"2024-03-22T04:12:33.622856+0000","last_undegraded":"2024-03-22T04:12:33.622856+0000","last_fullsized":"2024-03-22T04:12:33.622856+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:46.571387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637090+0000","last_change":"2024-03-22T04:12:34.626850+0000","last_active":"2024-03-22T04:12:35.637090+0000","last_peered":"2024-03-22T04:12:35.637090+0000","last_clean":"2024-03-22T04:12:35.637090+0000","last_became_active":"2024-03-22T04:12:34.626505+0000","last_became_peered":"2024-03-22T04:12:34.626505+0000","last_unstale":"2024-03-22T04:12:35.637090+0000","last_undegraded":"2024-03-22T04:12:35.637090+0000","last_fullsized":"2024-03-22T04:12:35.637090+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612750+0000","last_change":"2024-03-22T04:12:33.618147+0000","last_active":"2024-03-22T04:12:35.612750+0000","last_peered":"2024-03-22T04:12:35.612750+0000","last_clean":"2024-03-22T04:12:35.612750+0000","last_became_active":"2024-03-22T04:12:33.617867+0000","last_became_peered":"2024-03-22T04:12:33.617867+0000","last_unstale":"2024-03-22T04:12:35.612750+0000","last_undegraded":"2024-03-22T04:12:35.612750+0000","last_fullsized":"2024-03-22T04:12:35.612750+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612131+0000","last_change":"2024-03-22T04:12:34.624495+0000","last_active":"2024-03-22T04:12:35.612131+0000","last_peered":"2024-03-22T04:12:35.612131+0000","last_clean":"2024-03-22T04:12:35.612131+0000","last_became_active":"2024-03-22T04:12:34.624243+0000","last_became_peered":"2024-03-22T04:12:34.624243+0000","last_unstale":"2024-03-22T04:12:35.612131+0000","last_undegraded":"2024-03-22T04:12:35.612131+0000","last_fullsized":"2024-03-22T04:12:35.612131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612650+0000","last_change":"2024-03-22T04:12:33.617534+0000","last_active":"2024-03-22T04:12:35.612650+0000","last_peered":"2024-03-22T04:12:35.612650+0000","last_clean":"2024-03-22T04:12:35.612650+0000","last_became_active":"2024-03-22T04:12:33.617265+0000","last_became_peered":"2024-03-22T04:12:33.617265+0000","last_unstale":"2024-03-22T04:12:35.612650+0000","last_undegraded":"2024-03-22T04:12:35.612650+0000","last_fullsized":"2024-03-22T04:12:35.612650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615181+0000","last_change":"2024-03-22T04:12:34.635018+0000","last_active":"2024-03-22T04:12:35.615181+0000","last_peered":"2024-03-22T04:12:35.615181+0000","last_clean":"2024-03-22T04:12:35.615181+0000","last_became_active":"2024-03-22T04:12:34.634716+0000","last_became_peered":"2024-03-22T04:12:34.634716+0000","last_unstale":"2024-03-22T04:12:35.615181+0000","last_undegraded":"2024-03-22T04:12:35.615181+0000","last_fullsized":"2024-03-22T04:12:35.615181+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.658426+0000","last_change":"2024-03-22T04:12:33.623030+0000","last_active":"2024-03-22T04:12:34.658426+0000","last_peered":"2024-03-22T04:12:34.658426+0000","last_clean":"2024-03-22T04:12:34.658426+0000","last_became_active":"2024-03-22T04:12:33.622806+0000","last_became_peered":"2024-03-22T04:12:33.622806+0000","last_unstale":"2024-03-22T04:12:34.658426+0000","last_undegraded":"2024-03-22T04:12:34.658426+0000","last_fullsized":"2024-03-22T04:12:34.658426+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:50:40.568966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.641095+0000","last_change":"2024-03-22T04:12:34.635387+0000","last_active":"2024-03-22T04:12:34.641095+0000","last_peered":"2024-03-22T04:12:34.641095+0000","last_clean":"2024-03-22T04:12:34.641095+0000","last_became_active":"2024-03-22T04:12:34.632095+0000","last_became_peered":"2024-03-22T04:12:34.632095+0000","last_unstale":"2024-03-22T04:12:34.641095+0000","last_undegraded":"2024-03-22T04:12:34.641095+0000","last_fullsized":"2024-03-22T04:12:34.641095+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615307+0000","last_change":"2024-03-22T04:12:33.616617+0000","last_active":"2024-03-22T04:12:35.615307+0000","last_peered":"2024-03-22T04:12:35.615307+0000","last_clean":"2024-03-22T04:12:35.615307+0000","last_became_active":"2024-03-22T04:12:33.616088+0000","last_became_peered":"2024-03-22T04:12:33.616088+0000","last_unstale":"2024-03-22T04:12:35.615307+0000","last_undegraded":"2024-03-22T04:12:35.615307+0000","last_fullsized":"2024-03-22T04:12:35.615307+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:49.907740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.640577+0000","last_change":"2024-03-22T04:12:34.638920+0000","last_active":"2024-03-22T04:12:34.640577+0000","last_peered":"2024-03-22T04:12:34.640577+0000","last_clean":"2024-03-22T04:12:34.640577+0000","last_became_active":"2024-03-22T04:12:34.638167+0000","last_became_peered":"2024-03-22T04:12:34.638167+0000","last_unstale":"2024-03-22T04:12:34.640577+0000","last_undegraded":"2024-03-22T04:12:34.640577+0000","last_fullsized":"2024-03-22T04:12:34.640577+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:21:14.663985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.659863+0000","last_change":"2024-03-22T04:12:33.625471+0000","last_active":"2024-03-22T04:12:34.659863+0000","last_peered":"2024-03-22T04:12:34.659863+0000","last_clean":"2024-03-22T04:12:34.659863+0000","last_became_active":"2024-03-22T04:12:33.625189+0000","last_became_peered":"2024-03-22T04:12:33.625189+0000","last_unstale":"2024-03-22T04:12:34.659863+0000","last_undegraded":"2024-03-22T04:12:34.659863+0000","last_fullsized":"2024-03-22T04:12:34.659863+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615959+0000","last_change":"2024-03-22T04:12:34.642230+0000","last_active":"2024-03-22T04:12:35.615959+0000","last_peered":"2024-03-22T04:12:35.615959+0000","last_clean":"2024-03-22T04:12:35.615959+0000","last_became_active":"2024-03-22T04:12:34.642038+0000","last_became_peered":"2024-03-22T04:12:34.642038+0000","last_unstale":"2024-03-22T04:12:35.615959+0000","last_undegraded":"2024-03-22T04:12:35.615959+0000","last_fullsized":"2024-03-22T04:12:35.615959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612534+0000","last_change":"2024-03-22T04:12:33.620764+0000","last_active":"2024-03-22T04:12:35.612534+0000","last_peered":"2024-03-22T04:12:35.612534+0000","last_clean":"2024-03-22T04:12:35.612534+0000","last_became_active":"2024-03-22T04:12:33.620498+0000","last_became_peered":"2024-03-22T04:12:33.620498+0000","last_unstale":"2024-03-22T04:12:35.612534+0000","last_undegraded":"2024-03-22T04:12:35.612534+0000","last_fullsized":"2024-03-22T04:12:35.612534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.646765+0000","last_change":"2024-03-22T04:12:34.641493+0000","last_active":"2024-03-22T04:12:34.646765+0000","last_peered":"2024-03-22T04:12:34.646765+0000","last_clean":"2024-03-22T04:12:34.646765+0000","last_became_active":"2024-03-22T04:12:34.636610+0000","last_became_peered":"2024-03-22T04:12:34.636610+0000","last_unstale":"2024-03-22T04:12:34.646765+0000","last_undegraded":"2024-03-22T04:12:34.646765+0000","last_fullsized":"2024-03-22T04:12:34.646765+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:01.229933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.617067+0000","last_change":"2024-03-22T04:12:33.618015+0000","last_active":"2024-03-22T04:12:34.617067+0000","last_peered":"2024-03-22T04:12:34.617067+0000","last_clean":"2024-03-22T04:12:34.617067+0000","last_became_active":"2024-03-22T04:12:33.617794+0000","last_became_peered":"2024-03-22T04:12:33.617794+0000","last_unstale":"2024-03-22T04:12:34.617067+0000","last_undegraded":"2024-03-22T04:12:34.617067+0000","last_fullsized":"2024-03-22T04:12:34.617067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:00.222391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602266+0000","last_change":"2024-03-22T04:12:33.602266+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602266+0000","last_undegraded":"2024-03-22T04:12:33.602266+0000","last_fullsized":"2024-03-22T04:12:33.602266+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.661009+0000","last_change":"2024-03-22T04:12:33.621891+0000","last_active":"2024-03-22T04:12:34.661009+0000","last_peered":"2024-03-22T04:12:34.661009+0000","last_clean":"2024-03-22T04:12:34.661009+0000","last_became_active":"2024-03-22T04:12:33.621626+0000","last_became_peered":"2024-03-22T04:12:33.621626+0000","last_unstale":"2024-03-22T04:12:34.661009+0000","last_undegraded":"2024-03-22T04:12:34.661009+0000","last_fullsized":"2024-03-22T04:12:34.661009+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614700+0000","last_change":"2024-03-22T04:12:34.634352+0000","last_active":"2024-03-22T04:12:35.614700+0000","last_peered":"2024-03-22T04:12:35.614700+0000","last_clean":"2024-03-22T04:12:35.614700+0000","last_became_active":"2024-03-22T04:12:34.633729+0000","last_became_peered":"2024-03-22T04:12:34.633729+0000","last_unstale":"2024-03-22T04:12:35.614700+0000","last_undegraded":"2024-03-22T04:12:35.614700+0000","last_fullsized":"2024-03-22T04:12:35.614700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.660467+0000","last_change":"2024-03-22T04:12:33.623382+0000","last_active":"2024-03-22T04:12:34.660467+0000","last_peered":"2024-03-22T04:12:34.660467+0000","last_clean":"2024-03-22T04:12:34.660467+0000","last_became_active":"2024-03-22T04:12:33.622982+0000","last_became_peered":"2024-03-22T04:12:33.622982+0000","last_unstale":"2024-03-22T04:12:34.660467+0000","last_undegraded":"2024-03-22T04:12:34.660467+0000","last_fullsized":"2024-03-22T04:12:34.660467+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614924+0000","last_change":"2024-03-22T04:12:34.627065+0000","last_active":"2024-03-22T04:12:35.614924+0000","last_peered":"2024-03-22T04:12:35.614924+0000","last_clean":"2024-03-22T04:12:35.614924+0000","last_became_active":"2024-03-22T04:12:34.626777+0000","last_became_peered":"2024-03-22T04:12:34.626777+0000","last_unstale":"2024-03-22T04:12:35.614924+0000","last_undegraded":"2024-03-22T04:12:35.614924+0000","last_fullsized":"2024-03-22T04:12:35.614924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.658788+0000","last_change":"2024-03-22T04:12:33.622321+0000","last_active":"2024-03-22T04:12:34.658788+0000","last_peered":"2024-03-22T04:12:34.658788+0000","last_clean":"2024-03-22T04:12:34.658788+0000","last_became_active":"2024-03-22T04:12:33.622058+0000","last_became_peered":"2024-03-22T04:12:33.622058+0000","last_unstale":"2024-03-22T04:12:34.658788+0000","last_undegraded":"2024-03-22T04:12:34.658788+0000","last_fullsized":"2024-03-22T04:12:34.658788+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639497+0000","last_change":"2024-03-22T04:12:34.636289+0000","last_active":"2024-03-22T04:12:34.639497+0000","last_peered":"2024-03-22T04:12:34.639497+0000","last_clean":"2024-03-22T04:12:34.639497+0000","last_became_active":"2024-03-22T04:12:34.636025+0000","last_became_peered":"2024-03-22T04:12:34.636025+0000","last_unstale":"2024-03-22T04:12:34.639497+0000","last_undegraded":"2024-03-22T04:12:34.639497+0000","last_fullsized":"2024-03-22T04:12:34.639497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612548+0000","last_change":"2024-03-22T04:12:33.611878+0000","last_active":"2024-03-22T04:12:35.612548+0000","last_peered":"2024-03-22T04:12:35.612548+0000","last_clean":"2024-03-22T04:12:35.612548+0000","last_became_active":"2024-03-22T04:12:33.611537+0000","last_became_peered":"2024-03-22T04:12:33.611537+0000","last_unstale":"2024-03-22T04:12:35.612548+0000","last_undegraded":"2024-03-22T04:12:35.612548+0000","last_fullsized":"2024-03-22T04:12:35.612548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.653492+0000","last_change":"2024-03-22T04:12:34.650207+0000","last_active":"2024-03-22T04:12:34.653492+0000","last_peered":"2024-03-22T04:12:34.653492+0000","last_clean":"2024-03-22T04:12:34.653492+0000","last_became_active":"2024-03-22T04:12:34.649927+0000","last_became_peered":"2024-03-22T04:12:34.649927+0000","last_unstale":"2024-03-22T04:12:34.653492+0000","last_undegraded":"2024-03-22T04:12:34.653492+0000","last_fullsized":"2024-03-22T04:12:34.653492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:12.985866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.616557+0000","last_change":"2024-03-22T04:12:33.622557+0000","last_active":"2024-03-22T04:12:34.616557+0000","last_peered":"2024-03-22T04:12:34.616557+0000","last_clean":"2024-03-22T04:12:34.616557+0000","last_became_active":"2024-03-22T04:12:33.622240+0000","last_became_peered":"2024-03-22T04:12:33.622240+0000","last_unstale":"2024-03-22T04:12:34.616557+0000","last_undegraded":"2024-03-22T04:12:34.616557+0000","last_fullsized":"2024-03-22T04:12:34.616557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:32.007591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634871+0000","last_change":"2024-03-22T04:12:34.633656+0000","last_active":"2024-03-22T04:12:34.634871+0000","last_peered":"2024-03-22T04:12:34.634871+0000","last_clean":"2024-03-22T04:12:34.634871+0000","last_became_active":"2024-03-22T04:12:34.633150+0000","last_became_peered":"2024-03-22T04:12:34.633150+0000","last_unstale":"2024-03-22T04:12:34.634871+0000","last_undegraded":"2024-03-22T04:12:34.634871+0000","last_fullsized":"2024-03-22T04:12:34.634871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:33.344021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615428+0000","last_change":"2024-03-22T04:12:33.624581+0000","last_active":"2024-03-22T04:12:35.615428+0000","last_peered":"2024-03-22T04:12:35.615428+0000","last_clean":"2024-03-22T04:12:35.615428+0000","last_became_active":"2024-03-22T04:12:33.624324+0000","last_became_peered":"2024-03-22T04:12:33.624324+0000","last_unstale":"2024-03-22T04:12:35.615428+0000","last_undegraded":"2024-03-22T04:12:35.615428+0000","last_fullsized":"2024-03-22T04:12:35.615428+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:41.428263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612405+0000","last_change":"2024-03-22T04:12:34.631558+0000","last_active":"2024-03-22T04:12:35.612405+0000","last_peered":"2024-03-22T04:12:35.612405+0000","last_clean":"2024-03-22T04:12:35.612405+0000","last_became_active":"2024-03-22T04:12:34.621598+0000","last_became_peered":"2024-03-22T04:12:34.621598+0000","last_unstale":"2024-03-22T04:12:35.612405+0000","last_undegraded":"2024-03-22T04:12:35.612405+0000","last_fullsized":"2024-03-22T04:12:35.612405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:43:33.625589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.624342+0000","last_change":"2024-03-22T04:12:33.619281+0000","last_active":"2024-03-22T04:12:33.624342+0000","last_peered":"2024-03-22T04:12:33.624342+0000","last_clean":"2024-03-22T04:12:33.624342+0000","last_became_active":"2024-03-22T04:12:33.618935+0000","last_became_peered":"2024-03-22T04:12:33.618935+0000","last_unstale":"2024-03-22T04:12:33.624342+0000","last_undegraded":"2024-03-22T04:12:33.624342+0000","last_fullsized":"2024-03-22T04:12:33.624342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:11:53.935122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615777+0000","last_change":"2024-03-22T04:12:34.642768+0000","last_active":"2024-03-22T04:12:35.615777+0000","last_peered":"2024-03-22T04:12:35.615777+0000","last_clean":"2024-03-22T04:12:35.615777+0000","last_became_active":"2024-03-22T04:12:34.642534+0000","last_became_peered":"2024-03-22T04:12:34.642534+0000","last_unstale":"2024-03-22T04:12:35.615777+0000","last_undegraded":"2024-03-22T04:12:35.615777+0000","last_fullsized":"2024-03-22T04:12:35.615777+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615251+0000","last_change":"2024-03-22T04:12:33.626234+0000","last_active":"2024-03-22T04:12:35.615251+0000","last_peered":"2024-03-22T04:12:35.615251+0000","last_clean":"2024-03-22T04:12:35.615251+0000","last_became_active":"2024-03-22T04:12:33.623978+0000","last_became_peered":"2024-03-22T04:12:33.623978+0000","last_unstale":"2024-03-22T04:12:35.615251+0000","last_undegraded":"2024-03-22T04:12:35.615251+0000","last_fullsized":"2024-03-22T04:12:35.615251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:45.736028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.648376+0000","last_change":"2024-03-22T04:12:34.646431+0000","last_active":"2024-03-22T04:12:34.648376+0000","last_peered":"2024-03-22T04:12:34.648376+0000","last_clean":"2024-03-22T04:12:34.648376+0000","last_became_active":"2024-03-22T04:12:34.646176+0000","last_became_peered":"2024-03-22T04:12:34.646176+0000","last_unstale":"2024-03-22T04:12:34.648376+0000","last_undegraded":"2024-03-22T04:12:34.648376+0000","last_fullsized":"2024-03-22T04:12:34.648376+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:42.435805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614477+0000","last_change":"2024-03-22T04:12:33.610526+0000","last_active":"2024-03-22T04:12:35.614477+0000","last_peered":"2024-03-22T04:12:35.614477+0000","last_clean":"2024-03-22T04:12:35.614477+0000","last_became_active":"2024-03-22T04:12:33.609782+0000","last_became_peered":"2024-03-22T04:12:33.609782+0000","last_unstale":"2024-03-22T04:12:35.614477+0000","last_undegraded":"2024-03-22T04:12:35.614477+0000","last_fullsized":"2024-03-22T04:12:35.614477+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:11.978324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.611982+0000","last_change":"2024-03-22T04:12:34.622644+0000","last_active":"2024-03-22T04:12:35.611982+0000","last_peered":"2024-03-22T04:12:35.611982+0000","last_clean":"2024-03-22T04:12:35.611982+0000","last_became_active":"2024-03-22T04:12:34.622112+0000","last_became_peered":"2024-03-22T04:12:34.622112+0000","last_unstale":"2024-03-22T04:12:35.611982+0000","last_undegraded":"2024-03-22T04:12:35.611982+0000","last_fullsized":"2024-03-22T04:12:35.611982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657519+0000","last_change":"2024-03-22T04:12:33.621421+0000","last_active":"2024-03-22T04:12:34.657519+0000","last_peered":"2024-03-22T04:12:34.657519+0000","last_clean":"2024-03-22T04:12:34.657519+0000","last_became_active":"2024-03-22T04:12:33.621140+0000","last_became_peered":"2024-03-22T04:12:33.621140+0000","last_unstale":"2024-03-22T04:12:34.657519+0000","last_undegraded":"2024-03-22T04:12:34.657519+0000","last_fullsized":"2024-03-22T04:12:34.657519+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:21.786333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615749+0000","last_change":"2024-03-22T04:12:34.638291+0000","last_active":"2024-03-22T04:12:35.615749+0000","last_peered":"2024-03-22T04:12:35.615749+0000","last_clean":"2024-03-22T04:12:35.615749+0000","last_became_active":"2024-03-22T04:12:34.637953+0000","last_became_peered":"2024-03-22T04:12:34.637953+0000","last_unstale":"2024-03-22T04:12:35.615749+0000","last_undegraded":"2024-03-22T04:12:35.615749+0000","last_fullsized":"2024-03-22T04:12:35.615749+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:37:44.536324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.618566+0000","last_change":"2024-03-22T04:12:33.623102+0000","last_active":"2024-03-22T04:12:34.618566+0000","last_peered":"2024-03-22T04:12:34.618566+0000","last_clean":"2024-03-22T04:12:34.618566+0000","last_became_active":"2024-03-22T04:12:33.622803+0000","last_became_peered":"2024-03-22T04:12:33.622803+0000","last_unstale":"2024-03-22T04:12:34.618566+0000","last_undegraded":"2024-03-22T04:12:34.618566+0000","last_fullsized":"2024-03-22T04:12:34.618566+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:04:41.174822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602478+0000","last_change":"2024-03-22T04:12:33.602478+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602478+0000","last_undegraded":"2024-03-22T04:12:33.602478+0000","last_fullsized":"2024-03-22T04:12:33.602478+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.622958+0000","last_change":"2024-03-22T04:12:33.618302+0000","last_active":"2024-03-22T04:12:33.622958+0000","last_peered":"2024-03-22T04:12:33.622958+0000","last_clean":"2024-03-22T04:12:33.622958+0000","last_became_active":"2024-03-22T04:12:33.617971+0000","last_became_peered":"2024-03-22T04:12:33.617971+0000","last_unstale":"2024-03-22T04:12:33.622958+0000","last_undegraded":"2024-03-22T04:12:33.622958+0000","last_fullsized":"2024-03-22T04:12:33.622958+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:42.915758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615191+0000","last_change":"2024-03-22T04:12:34.644894+0000","last_active":"2024-03-22T04:12:35.615191+0000","last_peered":"2024-03-22T04:12:35.615191+0000","last_clean":"2024-03-22T04:12:35.615191+0000","last_became_active":"2024-03-22T04:12:34.644526+0000","last_became_peered":"2024-03-22T04:12:34.644526+0000","last_unstale":"2024-03-22T04:12:35.615191+0000","last_undegraded":"2024-03-22T04:12:35.615191+0000","last_fullsized":"2024-03-22T04:12:35.615191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615131+0000","last_change":"2024-03-22T04:12:33.620091+0000","last_active":"2024-03-22T04:12:35.615131+0000","last_peered":"2024-03-22T04:12:35.615131+0000","last_clean":"2024-03-22T04:12:35.615131+0000","last_became_active":"2024-03-22T04:12:33.619790+0000","last_became_peered":"2024-03-22T04:12:33.619790+0000","last_unstale":"2024-03-22T04:12:35.615131+0000","last_undegraded":"2024-03-22T04:12:35.615131+0000","last_fullsized":"2024-03-22T04:12:35.615131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:32.336479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639091+0000","last_change":"2024-03-22T04:12:34.636152+0000","last_active":"2024-03-22T04:12:34.639091+0000","last_peered":"2024-03-22T04:12:34.639091+0000","last_clean":"2024-03-22T04:12:34.639091+0000","last_became_active":"2024-03-22T04:12:34.635865+0000","last_became_peered":"2024-03-22T04:12:34.635865+0000","last_unstale":"2024-03-22T04:12:34.639091+0000","last_undegraded":"2024-03-22T04:12:34.639091+0000","last_fullsized":"2024-03-22T04:12:34.639091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.616064+0000","last_change":"2024-03-22T04:12:33.612435+0000","last_active":"2024-03-22T04:12:33.616064+0000","last_peered":"2024-03-22T04:12:33.616064+0000","last_clean":"2024-03-22T04:12:33.616064+0000","last_became_active":"2024-03-22T04:12:33.611642+0000","last_became_peered":"2024-03-22T04:12:33.611642+0000","last_unstale":"2024-03-22T04:12:33.616064+0000","last_undegraded":"2024-03-22T04:12:33.616064+0000","last_fullsized":"2024-03-22T04:12:33.616064+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:44:21.610192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615970+0000","last_change":"2024-03-22T04:12:34.637750+0000","last_active":"2024-03-22T04:12:35.615970+0000","last_peered":"2024-03-22T04:12:35.615970+0000","last_clean":"2024-03-22T04:12:35.615970+0000","last_became_active":"2024-03-22T04:12:34.637269+0000","last_became_peered":"2024-03-22T04:12:34.637269+0000","last_unstale":"2024-03-22T04:12:35.615970+0000","last_undegraded":"2024-03-22T04:12:35.615970+0000","last_fullsized":"2024-03-22T04:12:35.615970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:54:58.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.617221+0000","last_change":"2024-03-22T04:12:33.613711+0000","last_active":"2024-03-22T04:12:34.617221+0000","last_peered":"2024-03-22T04:12:34.617221+0000","last_clean":"2024-03-22T04:12:34.617221+0000","last_became_active":"2024-03-22T04:12:33.613398+0000","last_became_peered":"2024-03-22T04:12:33.613398+0000","last_unstale":"2024-03-22T04:12:34.617221+0000","last_undegraded":"2024-03-22T04:12:34.617221+0000","last_fullsized":"2024-03-22T04:12:34.617221+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:34.304317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.654950+0000","last_change":"2024-03-22T04:12:34.649939+0000","last_active":"2024-03-22T04:12:34.654950+0000","last_peered":"2024-03-22T04:12:34.654950+0000","last_clean":"2024-03-22T04:12:34.654950+0000","last_became_active":"2024-03-22T04:12:34.649639+0000","last_became_peered":"2024-03-22T04:12:34.649639+0000","last_unstale":"2024-03-22T04:12:34.654950+0000","last_undegraded":"2024-03-22T04:12:34.654950+0000","last_fullsized":"2024-03-22T04:12:34.654950+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:01.747862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612050+0000","last_change":"2024-03-22T04:12:33.613244+0000","last_active":"2024-03-22T04:12:35.612050+0000","last_peered":"2024-03-22T04:12:35.612050+0000","last_clean":"2024-03-22T04:12:35.612050+0000","last_became_active":"2024-03-22T04:12:33.612972+0000","last_became_peered":"2024-03-22T04:12:33.612972+0000","last_unstale":"2024-03-22T04:12:35.612050+0000","last_undegraded":"2024-03-22T04:12:35.612050+0000","last_fullsized":"2024-03-22T04:12:35.612050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637167+0000","last_change":"2024-03-22T04:12:34.638813+0000","last_active":"2024-03-22T04:12:35.637167+0000","last_peered":"2024-03-22T04:12:35.637167+0000","last_clean":"2024-03-22T04:12:35.637167+0000","last_became_active":"2024-03-22T04:12:34.638487+0000","last_became_peered":"2024-03-22T04:12:34.638487+0000","last_unstale":"2024-03-22T04:12:35.637167+0000","last_undegraded":"2024-03-22T04:12:35.637167+0000","last_fullsized":"2024-03-22T04:12:35.637167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637130+0000","last_change":"2024-03-22T04:12:33.624669+0000","last_active":"2024-03-22T04:12:35.637130+0000","last_peered":"2024-03-22T04:12:35.637130+0000","last_clean":"2024-03-22T04:12:35.637130+0000","last_became_active":"2024-03-22T04:12:33.624416+0000","last_became_peered":"2024-03-22T04:12:33.624416+0000","last_unstale":"2024-03-22T04:12:35.637130+0000","last_undegraded":"2024-03-22T04:12:35.637130+0000","last_fullsized":"2024-03-22T04:12:35.637130+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:18.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":227,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612039+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:12:35.612039+0000","last_peered":"2024-03-22T04:12:35.612039+0000","last_clean":"2024-03-22T04:12:35.612039+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:12:35.612039+0000","last_undegraded":"2024-03-22T04:12:35.612039+0000","last_fullsized":"2024-03-22T04:12:35.612039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":273831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":274357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":865845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":273865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":273809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":870905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":274357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":598016,"data_stored":259941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:12:39.072 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:12:39.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc330e00640 1 Processor -- start 2024-03-22T04:12:39.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc330e00640 1 -- start start 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc330e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc330e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc32c15bc80 0x7fc32c15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc330e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32c15c550 con 0x7fc32c15eb10 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc330e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc32c15cb60 con 0x7fc32c15bc80 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc330e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c0b4750 con 0x7fc32c15ac10 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc321400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc32c15bc80 0x7fc32c15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc329e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc321400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc32c15bc80 0x7fc32c15e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51440/0 (socket says 172.21.15.86:51440) 2024-03-22T04:12:39.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.241+0000 7fc321400640 1 -- 172.21.15.86:0/772066840 learned_addr learned my addr 172.21.15.86:0/772066840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:39.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc321400640 1 -- 172.21.15.86:0/772066840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 msgr2=0x7fc32c0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:39.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc321400640 1 --2- 172.21.15.86:0/772066840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c0b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc321400640 1 -- 172.21.15.86:0/772066840 >> v1:172.21.15.196:6789/0 conn(0x7fc32c15eb10 legacy=0x7fc32c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc321400640 1 -- 172.21.15.86:0/772066840 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c0b4470 con 0x7fc32c15bc80 2024-03-22T04:12:39.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc321400640 1 --2- 172.21.15.86:0/772066840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c15e5d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc314009e50 tx=0x7fc31401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305bfd618d40d796 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:39.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc329400640 1 -- 172.21.15.86:0/772066840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140a5870 con 0x7fc32c15bc80 2024-03-22T04:12:39.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc329400640 1 -- 172.21.15.86:0/772066840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3140a5a10 con 0x7fc32c15bc80 2024-03-22T04:12:39.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc329400640 1 -- 172.21.15.86:0/772066840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140a5de0 con 0x7fc32c15bc80 2024-03-22T04:12:39.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc330e00640 1 -- 172.21.15.86:0/772066840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 msgr2=0x7fc32c15e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc330e00640 1 --2- 172.21.15.86:0/772066840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c15e5d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc314009e50 tx=0x7fc31401ca00 comp rx=0 tx=0).stop 2024-03-22T04:12:39.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc330e00640 1 -- 172.21.15.86:0/772066840 shutdown_connections 2024-03-22T04:12:39.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc330e00640 1 --2- 172.21.15.86:0/772066840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c15e5d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc330e00640 1 --2- 172.21.15.86:0/772066840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c0b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc330e00640 1 -- 172.21.15.86:0/772066840 >> 172.21.15.86:0/772066840 conn(0x7fc32c0b1df0 msgr2=0x7fc32c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:39.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc330e00640 1 -- 172.21.15.86:0/772066840 shutdown_connections 2024-03-22T04:12:39.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.245+0000 7fc330e00640 1 -- 172.21.15.86:0/772066840 wait complete. 2024-03-22T04:12:39.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 Processor -- start 2024-03-22T04:12:39.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 -- start start 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c165af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c15eb10 0x7fc32c0ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c15e280 con 0x7fc32c15bc80 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c15e100 con 0x7fc32c15eb10 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c06e620 con 0x7fc32c15ac10 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c0ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51448/0 (socket says 172.21.15.86:51448) 2024-03-22T04:12:39.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 -- 172.21.15.86:0/4043906112 learned_addr learned my addr 172.21.15.86:0/4043906112 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc329e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c165af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc32a800640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c15eb10 0x7fc32c0ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 -- 172.21.15.86:0/4043906112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 msgr2=0x7fc32c165af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c165af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 -- 172.21.15.86:0/4043906112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c15eb10 msgr2=0x7fc32c0ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c15eb10 0x7fc32c0ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 -- 172.21.15.86:0/4043906112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c0cf2e0 con 0x7fc32c15bc80 2024-03-22T04:12:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc329e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c165af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc321400640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c0ce860 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc31401a630 tx=0x7fc3140197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc322a00640 1 -- 172.21.15.86:0/4043906112 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140bc020 con 0x7fc32c15bc80 2024-03-22T04:12:39.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc322a00640 1 -- 172.21.15.86:0/4043906112 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3140b8670 con 0x7fc32c15bc80 2024-03-22T04:12:39.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c0cf830 con 0x7fc32c15bc80 2024-03-22T04:12:39.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.249+0000 7fc322a00640 1 -- 172.21.15.86:0/4043906112 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140b8a60 con 0x7fc32c15bc80 2024-03-22T04:12:39.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.253+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c0ad880 con 0x7fc32c15bc80 2024-03-22T04:12:39.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.253+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c068650 con 0x7fc32c15bc80 2024-03-22T04:12:39.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.253+0000 7fc322a00640 1 -- 172.21.15.86:0/4043906112 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc3140bf020 con 0x7fc32c15bc80 2024-03-22T04:12:39.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.253+0000 7fc322a00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc308067b20 0x7fc308069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.253+0000 7fc322a00640 1 -- 172.21.15.86:0/4043906112 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc3141210d0 con 0x7fc32c15bc80 2024-03-22T04:12:39.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.253+0000 7fc329e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc308067b20 0x7fc308069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.253+0000 7fc329e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc308067b20 0x7fc308069fe0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009f50 tx=0x7fc31c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:39.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.257+0000 7fc322a00640 1 -- 172.21.15.86:0/4043906112 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc3140c6030 con 0x7fc32c15bc80 2024-03-22T04:12:39.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.409+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc32c06ed60 con 0x7fc308067b20 2024-03-22T04:12:39.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.413+0000 7fc322a00640 1 -- 172.21.15.86:0/4043906112 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251107 (secure 0 0 0) 0x7fc32c06ed60 con 0x7fc308067b20 2024-03-22T04:12:39.421 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc308067b20 msgr2=0x7fc308069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc308067b20 0x7fc308069fe0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009f50 tx=0x7fc31c009190 comp rx=0 tx=0).stop 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 msgr2=0x7fc32c0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c0ce860 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc31401a630 tx=0x7fc3140197a0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 shutdown_connections 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc308067b20 0x7fc308069fe0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c15eb10 0x7fc32c0ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc32c15bc80 0x7fc32c0ce860 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 --2- 172.21.15.86:0/4043906112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc32c15ac10 0x7fc32c165af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 >> 172.21.15.86:0/4043906112 conn(0x7fc32c0b1df0 msgr2=0x7fc32c072590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 shutdown_connections 2024-03-22T04:12:39.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.417+0000 7fc330e00640 1 -- 172.21.15.86:0/4043906112 wait complete. 2024-03-22T04:12:39.440 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-03-22T04:12:37.816229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222236,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752484,"statfs":{"total":773094113280,"available":772866543616,"internally_reserved":0,"allocated":6201344,"data_stored":3366910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220955"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616060+0000","last_change":"2024-03-22T04:12:34.628936+0000","last_active":"2024-03-22T04:12:35.616060+0000","last_peered":"2024-03-22T04:12:35.616060+0000","last_clean":"2024-03-22T04:12:35.616060+0000","last_became_active":"2024-03-22T04:12:34.628641+0000","last_became_peered":"2024-03-22T04:12:34.628641+0000","last_unstale":"2024-03-22T04:12:35.616060+0000","last_undegraded":"2024-03-22T04:12:35.616060+0000","last_fullsized":"2024-03-22T04:12:35.616060+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639641+0000","last_change":"2024-03-22T04:12:34.637267+0000","last_active":"2024-03-22T04:12:34.639641+0000","last_peered":"2024-03-22T04:12:34.639641+0000","last_clean":"2024-03-22T04:12:34.639641+0000","last_became_active":"2024-03-22T04:12:34.637024+0000","last_became_peered":"2024-03-22T04:12:34.637024+0000","last_unstale":"2024-03-22T04:12:34.639641+0000","last_undegraded":"2024-03-22T04:12:34.639641+0000","last_fullsized":"2024-03-22T04:12:34.639641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603940+0000","last_change":"2024-03-22T04:12:33.603940+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603940+0000","last_undegraded":"2024-03-22T04:12:33.603940+0000","last_fullsized":"2024-03-22T04:12:33.603940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612444+0000","last_change":"2024-03-22T04:12:34.625832+0000","last_active":"2024-03-22T04:12:35.612444+0000","last_peered":"2024-03-22T04:12:35.612444+0000","last_clean":"2024-03-22T04:12:35.612444+0000","last_became_active":"2024-03-22T04:12:34.625533+0000","last_became_peered":"2024-03-22T04:12:34.625533+0000","last_unstale":"2024-03-22T04:12:35.612444+0000","last_undegraded":"2024-03-22T04:12:35.612444+0000","last_fullsized":"2024-03-22T04:12:35.612444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:00.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.635523+0000","last_change":"2024-03-22T04:12:34.627773+0000","last_active":"2024-03-22T04:12:34.635523+0000","last_peered":"2024-03-22T04:12:34.635523+0000","last_clean":"2024-03-22T04:12:34.635523+0000","last_became_active":"2024-03-22T04:12:34.624749+0000","last_became_peered":"2024-03-22T04:12:34.624749+0000","last_unstale":"2024-03-22T04:12:34.635523+0000","last_undegraded":"2024-03-22T04:12:34.635523+0000","last_fullsized":"2024-03-22T04:12:34.635523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:01.747862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643345+0000","last_change":"2024-03-22T04:12:34.635777+0000","last_active":"2024-03-22T04:12:34.643345+0000","last_peered":"2024-03-22T04:12:34.643345+0000","last_clean":"2024-03-22T04:12:34.643345+0000","last_became_active":"2024-03-22T04:12:34.635528+0000","last_became_peered":"2024-03-22T04:12:34.635528+0000","last_unstale":"2024-03-22T04:12:34.643345+0000","last_undegraded":"2024-03-22T04:12:34.643345+0000","last_fullsized":"2024-03-22T04:12:34.643345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:01:53.135313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614543+0000","last_change":"2024-03-22T04:12:34.636516+0000","last_active":"2024-03-22T04:12:35.614543+0000","last_peered":"2024-03-22T04:12:35.614543+0000","last_clean":"2024-03-22T04:12:35.614543+0000","last_became_active":"2024-03-22T04:12:34.636312+0000","last_became_peered":"2024-03-22T04:12:34.636312+0000","last_unstale":"2024-03-22T04:12:35.614543+0000","last_undegraded":"2024-03-22T04:12:35.614543+0000","last_fullsized":"2024-03-22T04:12:35.614543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657816+0000","last_change":"2024-03-22T04:12:34.650010+0000","last_active":"2024-03-22T04:12:34.657816+0000","last_peered":"2024-03-22T04:12:34.657816+0000","last_clean":"2024-03-22T04:12:34.657816+0000","last_became_active":"2024-03-22T04:12:34.649709+0000","last_became_peered":"2024-03-22T04:12:34.649709+0000","last_unstale":"2024-03-22T04:12:34.657816+0000","last_undegraded":"2024-03-22T04:12:34.657816+0000","last_fullsized":"2024-03-22T04:12:34.657816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:50.915283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603153+0000","last_change":"2024-03-22T04:12:33.603153+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603153+0000","last_undegraded":"2024-03-22T04:12:33.603153+0000","last_fullsized":"2024-03-22T04:12:33.603153+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615323+0000","last_change":"2024-03-22T04:12:34.641332+0000","last_active":"2024-03-22T04:12:35.615323+0000","last_peered":"2024-03-22T04:12:35.615323+0000","last_clean":"2024-03-22T04:12:35.615323+0000","last_became_active":"2024-03-22T04:12:34.641059+0000","last_became_peered":"2024-03-22T04:12:34.641059+0000","last_unstale":"2024-03-22T04:12:35.615323+0000","last_undegraded":"2024-03-22T04:12:35.615323+0000","last_fullsized":"2024-03-22T04:12:35.615323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615824+0000","last_change":"2024-03-22T04:12:34.627929+0000","last_active":"2024-03-22T04:12:35.615824+0000","last_peered":"2024-03-22T04:12:35.615824+0000","last_clean":"2024-03-22T04:12:35.615824+0000","last_became_active":"2024-03-22T04:12:34.627640+0000","last_became_peered":"2024-03-22T04:12:34.627640+0000","last_unstale":"2024-03-22T04:12:35.615824+0000","last_undegraded":"2024-03-22T04:12:35.615824+0000","last_fullsized":"2024-03-22T04:12:35.615824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643682+0000","last_change":"2024-03-22T04:12:34.639709+0000","last_active":"2024-03-22T04:12:34.643682+0000","last_peered":"2024-03-22T04:12:34.643682+0000","last_clean":"2024-03-22T04:12:34.643682+0000","last_became_active":"2024-03-22T04:12:34.638936+0000","last_became_peered":"2024-03-22T04:12:34.638936+0000","last_unstale":"2024-03-22T04:12:34.643682+0000","last_undegraded":"2024-03-22T04:12:34.643682+0000","last_fullsized":"2024-03-22T04:12:34.643682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:33.015134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.647737+0000","last_change":"2024-03-22T04:12:34.644837+0000","last_active":"2024-03-22T04:12:34.647737+0000","last_peered":"2024-03-22T04:12:34.647737+0000","last_clean":"2024-03-22T04:12:34.647737+0000","last_became_active":"2024-03-22T04:12:34.644507+0000","last_became_peered":"2024-03-22T04:12:34.644507+0000","last_unstale":"2024-03-22T04:12:34.647737+0000","last_undegraded":"2024-03-22T04:12:34.647737+0000","last_fullsized":"2024-03-22T04:12:34.647737+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:35.311859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614973+0000","last_change":"2024-03-22T04:12:34.641248+0000","last_active":"2024-03-22T04:12:35.614973+0000","last_peered":"2024-03-22T04:12:35.614973+0000","last_clean":"2024-03-22T04:12:35.614973+0000","last_became_active":"2024-03-22T04:12:34.640953+0000","last_became_peered":"2024-03-22T04:12:34.640953+0000","last_unstale":"2024-03-22T04:12:35.614973+0000","last_undegraded":"2024-03-22T04:12:35.614973+0000","last_fullsized":"2024-03-22T04:12:35.614973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612098+0000","last_change":"2024-03-22T04:12:34.632175+0000","last_active":"2024-03-22T04:12:35.612098+0000","last_peered":"2024-03-22T04:12:35.612098+0000","last_clean":"2024-03-22T04:12:35.612098+0000","last_became_active":"2024-03-22T04:12:34.631916+0000","last_became_peered":"2024-03-22T04:12:34.631916+0000","last_unstale":"2024-03-22T04:12:35.612098+0000","last_undegraded":"2024-03-22T04:12:35.612098+0000","last_fullsized":"2024-03-22T04:12:35.612098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643732+0000","last_change":"2024-03-22T04:12:34.639122+0000","last_active":"2024-03-22T04:12:34.643732+0000","last_peered":"2024-03-22T04:12:34.643732+0000","last_clean":"2024-03-22T04:12:34.643732+0000","last_became_active":"2024-03-22T04:12:34.638788+0000","last_became_peered":"2024-03-22T04:12:34.638788+0000","last_unstale":"2024-03-22T04:12:34.643732+0000","last_undegraded":"2024-03-22T04:12:34.643732+0000","last_fullsized":"2024-03-22T04:12:34.643732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:27.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616261+0000","last_change":"2024-03-22T04:12:34.632974+0000","last_active":"2024-03-22T04:12:35.616261+0000","last_peered":"2024-03-22T04:12:35.616261+0000","last_clean":"2024-03-22T04:12:35.616261+0000","last_became_active":"2024-03-22T04:12:34.627665+0000","last_became_peered":"2024-03-22T04:12:34.627665+0000","last_unstale":"2024-03-22T04:12:35.616261+0000","last_undegraded":"2024-03-22T04:12:35.616261+0000","last_fullsized":"2024-03-22T04:12:35.616261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612642+0000","last_change":"2024-03-22T04:12:34.626077+0000","last_active":"2024-03-22T04:12:35.612642+0000","last_peered":"2024-03-22T04:12:35.612642+0000","last_clean":"2024-03-22T04:12:35.612642+0000","last_became_active":"2024-03-22T04:12:34.625837+0000","last_became_peered":"2024-03-22T04:12:34.625837+0000","last_unstale":"2024-03-22T04:12:35.612642+0000","last_undegraded":"2024-03-22T04:12:35.612642+0000","last_fullsized":"2024-03-22T04:12:35.612642+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603525+0000","last_change":"2024-03-22T04:12:33.603525+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603525+0000","last_undegraded":"2024-03-22T04:12:33.603525+0000","last_fullsized":"2024-03-22T04:12:33.603525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634836+0000","last_change":"2024-03-22T04:12:34.630270+0000","last_active":"2024-03-22T04:12:34.634836+0000","last_peered":"2024-03-22T04:12:34.634836+0000","last_clean":"2024-03-22T04:12:34.634836+0000","last_became_active":"2024-03-22T04:12:34.629989+0000","last_became_peered":"2024-03-22T04:12:34.629989+0000","last_unstale":"2024-03-22T04:12:34.634836+0000","last_undegraded":"2024-03-22T04:12:34.634836+0000","last_fullsized":"2024-03-22T04:12:34.634836+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:50.915283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612164+0000","last_change":"2024-03-22T04:12:34.620830+0000","last_active":"2024-03-22T04:12:35.612164+0000","last_peered":"2024-03-22T04:12:35.612164+0000","last_clean":"2024-03-22T04:12:35.612164+0000","last_became_active":"2024-03-22T04:12:34.620520+0000","last_became_peered":"2024-03-22T04:12:34.620520+0000","last_unstale":"2024-03-22T04:12:35.612164+0000","last_undegraded":"2024-03-22T04:12:35.612164+0000","last_fullsized":"2024-03-22T04:12:35.612164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.604396+0000","last_change":"2024-03-22T04:12:33.604396+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.604396+0000","last_undegraded":"2024-03-22T04:12:33.604396+0000","last_fullsized":"2024-03-22T04:12:33.604396+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602445+0000","last_change":"2024-03-22T04:12:33.602445+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602445+0000","last_undegraded":"2024-03-22T04:12:33.602445+0000","last_fullsized":"2024-03-22T04:12:33.602445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.603931+0000","last_change":"2024-03-22T04:12:33.603931+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.603931+0000","last_undegraded":"2024-03-22T04:12:33.603931+0000","last_fullsized":"2024-03-22T04:12:33.603931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616176+0000","last_change":"2024-03-22T04:12:34.635548+0000","last_active":"2024-03-22T04:12:35.616176+0000","last_peered":"2024-03-22T04:12:35.616176+0000","last_clean":"2024-03-22T04:12:35.616176+0000","last_became_active":"2024-03-22T04:12:34.635270+0000","last_became_peered":"2024-03-22T04:12:34.635270+0000","last_unstale":"2024-03-22T04:12:35.616176+0000","last_undegraded":"2024-03-22T04:12:35.616176+0000","last_fullsized":"2024-03-22T04:12:35.616176+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.604463+0000","last_change":"2024-03-22T04:12:33.604463+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.604463+0000","last_undegraded":"2024-03-22T04:12:33.604463+0000","last_fullsized":"2024-03-22T04:12:33.604463+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.659926+0000","last_change":"2024-03-22T04:12:34.655920+0000","last_active":"2024-03-22T04:12:34.659926+0000","last_peered":"2024-03-22T04:12:34.659926+0000","last_clean":"2024-03-22T04:12:34.659926+0000","last_became_active":"2024-03-22T04:12:34.655580+0000","last_became_peered":"2024-03-22T04:12:34.655580+0000","last_unstale":"2024-03-22T04:12:34.659926+0000","last_undegraded":"2024-03-22T04:12:34.659926+0000","last_fullsized":"2024-03-22T04:12:34.659926+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:33.344021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634856+0000","last_change":"2024-03-22T04:12:34.633424+0000","last_active":"2024-03-22T04:12:34.634856+0000","last_peered":"2024-03-22T04:12:34.634856+0000","last_clean":"2024-03-22T04:12:34.634856+0000","last_became_active":"2024-03-22T04:12:34.632797+0000","last_became_peered":"2024-03-22T04:12:34.632797+0000","last_unstale":"2024-03-22T04:12:34.634856+0000","last_undegraded":"2024-03-22T04:12:34.634856+0000","last_fullsized":"2024-03-22T04:12:34.634856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:12.985866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615318+0000","last_change":"2024-03-22T04:12:34.630952+0000","last_active":"2024-03-22T04:12:35.615318+0000","last_peered":"2024-03-22T04:12:35.615318+0000","last_clean":"2024-03-22T04:12:35.615318+0000","last_became_active":"2024-03-22T04:12:34.630765+0000","last_became_peered":"2024-03-22T04:12:34.630765+0000","last_unstale":"2024-03-22T04:12:35.615318+0000","last_undegraded":"2024-03-22T04:12:35.615318+0000","last_fullsized":"2024-03-22T04:12:35.615318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.625106+0000","last_change":"2024-03-22T04:12:34.621885+0000","last_active":"2024-03-22T04:12:34.625106+0000","last_peered":"2024-03-22T04:12:34.625106+0000","last_clean":"2024-03-22T04:12:34.625106+0000","last_became_active":"2024-03-22T04:12:34.621626+0000","last_became_peered":"2024-03-22T04:12:34.621626+0000","last_unstale":"2024-03-22T04:12:34.625106+0000","last_undegraded":"2024-03-22T04:12:34.625106+0000","last_fullsized":"2024-03-22T04:12:34.625106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:12:20.702700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.601918+0000","last_change":"2024-03-22T04:12:33.601918+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.601918+0000","last_undegraded":"2024-03-22T04:12:33.601918+0000","last_fullsized":"2024-03-22T04:12:33.601918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.630607+0000","last_change":"2024-03-22T04:12:34.624213+0000","last_active":"2024-03-22T04:12:34.630607+0000","last_peered":"2024-03-22T04:12:34.630607+0000","last_clean":"2024-03-22T04:12:34.630607+0000","last_became_active":"2024-03-22T04:12:34.623954+0000","last_became_peered":"2024-03-22T04:12:34.623954+0000","last_unstale":"2024-03-22T04:12:34.630607+0000","last_undegraded":"2024-03-22T04:12:34.630607+0000","last_fullsized":"2024-03-22T04:12:34.630607+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:42.435805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.629445+0000","last_change":"2024-03-22T04:12:34.623255+0000","last_active":"2024-03-22T04:12:34.629445+0000","last_peered":"2024-03-22T04:12:34.629445+0000","last_clean":"2024-03-22T04:12:34.629445+0000","last_became_active":"2024-03-22T04:12:34.622995+0000","last_became_peered":"2024-03-22T04:12:34.622995+0000","last_unstale":"2024-03-22T04:12:34.629445+0000","last_undegraded":"2024-03-22T04:12:34.629445+0000","last_fullsized":"2024-03-22T04:12:34.629445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:50.075295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637402+0000","last_change":"2024-03-22T04:12:33.618251+0000","last_active":"2024-03-22T04:12:35.637402+0000","last_peered":"2024-03-22T04:12:35.637402+0000","last_clean":"2024-03-22T04:12:35.637402+0000","last_became_active":"2024-03-22T04:12:33.617405+0000","last_became_peered":"2024-03-22T04:12:33.617405+0000","last_unstale":"2024-03-22T04:12:35.637402+0000","last_undegraded":"2024-03-22T04:12:35.637402+0000","last_fullsized":"2024-03-22T04:12:35.637402+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:45.786716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.636934+0000","last_change":"2024-03-22T04:12:34.633188+0000","last_active":"2024-03-22T04:12:34.636934+0000","last_peered":"2024-03-22T04:12:34.636934+0000","last_clean":"2024-03-22T04:12:34.636934+0000","last_became_active":"2024-03-22T04:12:34.632933+0000","last_became_peered":"2024-03-22T04:12:34.632933+0000","last_unstale":"2024-03-22T04:12:34.636934+0000","last_undegraded":"2024-03-22T04:12:34.636934+0000","last_fullsized":"2024-03-22T04:12:34.636934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:30:04.785433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.660378+0000","last_change":"2024-03-22T04:12:33.623741+0000","last_active":"2024-03-22T04:12:34.660378+0000","last_peered":"2024-03-22T04:12:34.660378+0000","last_clean":"2024-03-22T04:12:34.660378+0000","last_became_active":"2024-03-22T04:12:33.623407+0000","last_became_peered":"2024-03-22T04:12:33.623407+0000","last_unstale":"2024-03-22T04:12:34.660378+0000","last_undegraded":"2024-03-22T04:12:34.660378+0000","last_fullsized":"2024-03-22T04:12:34.660378+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.625128+0000","last_change":"2024-03-22T04:12:34.622229+0000","last_active":"2024-03-22T04:12:34.625128+0000","last_peered":"2024-03-22T04:12:34.625128+0000","last_clean":"2024-03-22T04:12:34.625128+0000","last_became_active":"2024-03-22T04:12:34.621209+0000","last_became_peered":"2024-03-22T04:12:34.621209+0000","last_unstale":"2024-03-22T04:12:34.625128+0000","last_undegraded":"2024-03-22T04:12:34.625128+0000","last_fullsized":"2024-03-22T04:12:34.625128+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:14:10.558919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615885+0000","last_change":"2024-03-22T04:12:33.621633+0000","last_active":"2024-03-22T04:12:35.615885+0000","last_peered":"2024-03-22T04:12:35.615885+0000","last_clean":"2024-03-22T04:12:35.615885+0000","last_became_active":"2024-03-22T04:12:33.621320+0000","last_became_peered":"2024-03-22T04:12:33.621320+0000","last_unstale":"2024-03-22T04:12:35.615885+0000","last_undegraded":"2024-03-22T04:12:35.615885+0000","last_fullsized":"2024-03-22T04:12:35.615885+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:00.740320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615027+0000","last_change":"2024-03-22T04:12:34.638992+0000","last_active":"2024-03-22T04:12:35.615027+0000","last_peered":"2024-03-22T04:12:35.615027+0000","last_clean":"2024-03-22T04:12:35.615027+0000","last_became_active":"2024-03-22T04:12:34.638706+0000","last_became_peered":"2024-03-22T04:12:34.638706+0000","last_unstale":"2024-03-22T04:12:35.615027+0000","last_undegraded":"2024-03-22T04:12:35.615027+0000","last_fullsized":"2024-03-22T04:12:35.615027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657480+0000","last_change":"2024-03-22T04:12:33.619503+0000","last_active":"2024-03-22T04:12:34.657480+0000","last_peered":"2024-03-22T04:12:34.657480+0000","last_clean":"2024-03-22T04:12:34.657480+0000","last_became_active":"2024-03-22T04:12:33.619218+0000","last_became_peered":"2024-03-22T04:12:33.619218+0000","last_unstale":"2024-03-22T04:12:34.657480+0000","last_undegraded":"2024-03-22T04:12:34.657480+0000","last_fullsized":"2024-03-22T04:12:34.657480+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:26.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637412+0000","last_change":"2024-03-22T04:12:34.638372+0000","last_active":"2024-03-22T04:12:35.637412+0000","last_peered":"2024-03-22T04:12:35.637412+0000","last_clean":"2024-03-22T04:12:35.637412+0000","last_became_active":"2024-03-22T04:12:34.636653+0000","last_became_peered":"2024-03-22T04:12:34.636653+0000","last_unstale":"2024-03-22T04:12:35.637412+0000","last_undegraded":"2024-03-22T04:12:35.637412+0000","last_fullsized":"2024-03-22T04:12:35.637412+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.625947+0000","last_change":"2024-03-22T04:12:33.621816+0000","last_active":"2024-03-22T04:12:33.625947+0000","last_peered":"2024-03-22T04:12:33.625947+0000","last_clean":"2024-03-22T04:12:33.625947+0000","last_became_active":"2024-03-22T04:12:33.621560+0000","last_became_peered":"2024-03-22T04:12:33.621560+0000","last_unstale":"2024-03-22T04:12:33.625947+0000","last_undegraded":"2024-03-22T04:12:33.625947+0000","last_fullsized":"2024-03-22T04:12:33.625947+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:49.010265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639031+0000","last_change":"2024-03-22T04:12:34.633399+0000","last_active":"2024-03-22T04:12:34.639031+0000","last_peered":"2024-03-22T04:12:34.639031+0000","last_clean":"2024-03-22T04:12:34.639031+0000","last_became_active":"2024-03-22T04:12:34.633082+0000","last_became_peered":"2024-03-22T04:12:34.633082+0000","last_unstale":"2024-03-22T04:12:34.639031+0000","last_undegraded":"2024-03-22T04:12:34.639031+0000","last_fullsized":"2024-03-22T04:12:34.639031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:39:21.503071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.651676+0000","last_change":"2024-03-22T04:12:33.615641+0000","last_active":"2024-03-22T04:12:34.651676+0000","last_peered":"2024-03-22T04:12:34.651676+0000","last_clean":"2024-03-22T04:12:34.651676+0000","last_became_active":"2024-03-22T04:12:33.615372+0000","last_became_peered":"2024-03-22T04:12:33.615372+0000","last_unstale":"2024-03-22T04:12:34.651676+0000","last_undegraded":"2024-03-22T04:12:34.651676+0000","last_fullsized":"2024-03-22T04:12:34.651676+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:39.482917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615462+0000","last_change":"2024-03-22T04:12:34.637539+0000","last_active":"2024-03-22T04:12:35.615462+0000","last_peered":"2024-03-22T04:12:35.615462+0000","last_clean":"2024-03-22T04:12:35.615462+0000","last_became_active":"2024-03-22T04:12:34.636918+0000","last_became_peered":"2024-03-22T04:12:34.636918+0000","last_unstale":"2024-03-22T04:12:35.615462+0000","last_undegraded":"2024-03-22T04:12:35.615462+0000","last_fullsized":"2024-03-22T04:12:35.615462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.618429+0000","last_change":"2024-03-22T04:12:33.615588+0000","last_active":"2024-03-22T04:12:34.618429+0000","last_peered":"2024-03-22T04:12:34.618429+0000","last_clean":"2024-03-22T04:12:34.618429+0000","last_became_active":"2024-03-22T04:12:33.615259+0000","last_became_peered":"2024-03-22T04:12:33.615259+0000","last_unstale":"2024-03-22T04:12:34.618429+0000","last_undegraded":"2024-03-22T04:12:34.618429+0000","last_fullsized":"2024-03-22T04:12:34.618429+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:27:55.035247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612760+0000","last_change":"2024-03-22T04:12:34.635762+0000","last_active":"2024-03-22T04:12:35.612760+0000","last_peered":"2024-03-22T04:12:35.612760+0000","last_clean":"2024-03-22T04:12:35.612760+0000","last_became_active":"2024-03-22T04:12:34.635493+0000","last_became_peered":"2024-03-22T04:12:34.635493+0000","last_unstale":"2024-03-22T04:12:35.612760+0000","last_undegraded":"2024-03-22T04:12:35.612760+0000","last_fullsized":"2024-03-22T04:12:35.612760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.622856+0000","last_change":"2024-03-22T04:12:33.618851+0000","last_active":"2024-03-22T04:12:33.622856+0000","last_peered":"2024-03-22T04:12:33.622856+0000","last_clean":"2024-03-22T04:12:33.622856+0000","last_became_active":"2024-03-22T04:12:33.618525+0000","last_became_peered":"2024-03-22T04:12:33.618525+0000","last_unstale":"2024-03-22T04:12:33.622856+0000","last_undegraded":"2024-03-22T04:12:33.622856+0000","last_fullsized":"2024-03-22T04:12:33.622856+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:46.571387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637090+0000","last_change":"2024-03-22T04:12:34.626850+0000","last_active":"2024-03-22T04:12:35.637090+0000","last_peered":"2024-03-22T04:12:35.637090+0000","last_clean":"2024-03-22T04:12:35.637090+0000","last_became_active":"2024-03-22T04:12:34.626505+0000","last_became_peered":"2024-03-22T04:12:34.626505+0000","last_unstale":"2024-03-22T04:12:35.637090+0000","last_undegraded":"2024-03-22T04:12:35.637090+0000","last_fullsized":"2024-03-22T04:12:35.637090+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612750+0000","last_change":"2024-03-22T04:12:33.618147+0000","last_active":"2024-03-22T04:12:35.612750+0000","last_peered":"2024-03-22T04:12:35.612750+0000","last_clean":"2024-03-22T04:12:35.612750+0000","last_became_active":"2024-03-22T04:12:33.617867+0000","last_became_peered":"2024-03-22T04:12:33.617867+0000","last_unstale":"2024-03-22T04:12:35.612750+0000","last_undegraded":"2024-03-22T04:12:35.612750+0000","last_fullsized":"2024-03-22T04:12:35.612750+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612131+0000","last_change":"2024-03-22T04:12:34.624495+0000","last_active":"2024-03-22T04:12:35.612131+0000","last_peered":"2024-03-22T04:12:35.612131+0000","last_clean":"2024-03-22T04:12:35.612131+0000","last_became_active":"2024-03-22T04:12:34.624243+0000","last_became_peered":"2024-03-22T04:12:34.624243+0000","last_unstale":"2024-03-22T04:12:35.612131+0000","last_undegraded":"2024-03-22T04:12:35.612131+0000","last_fullsized":"2024-03-22T04:12:35.612131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612650+0000","last_change":"2024-03-22T04:12:33.617534+0000","last_active":"2024-03-22T04:12:35.612650+0000","last_peered":"2024-03-22T04:12:35.612650+0000","last_clean":"2024-03-22T04:12:35.612650+0000","last_became_active":"2024-03-22T04:12:33.617265+0000","last_became_peered":"2024-03-22T04:12:33.617265+0000","last_unstale":"2024-03-22T04:12:35.612650+0000","last_undegraded":"2024-03-22T04:12:35.612650+0000","last_fullsized":"2024-03-22T04:12:35.612650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615181+0000","last_change":"2024-03-22T04:12:34.635018+0000","last_active":"2024-03-22T04:12:35.615181+0000","last_peered":"2024-03-22T04:12:35.615181+0000","last_clean":"2024-03-22T04:12:35.615181+0000","last_became_active":"2024-03-22T04:12:34.634716+0000","last_became_peered":"2024-03-22T04:12:34.634716+0000","last_unstale":"2024-03-22T04:12:35.615181+0000","last_undegraded":"2024-03-22T04:12:35.615181+0000","last_fullsized":"2024-03-22T04:12:35.615181+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.658426+0000","last_change":"2024-03-22T04:12:33.623030+0000","last_active":"2024-03-22T04:12:34.658426+0000","last_peered":"2024-03-22T04:12:34.658426+0000","last_clean":"2024-03-22T04:12:34.658426+0000","last_became_active":"2024-03-22T04:12:33.622806+0000","last_became_peered":"2024-03-22T04:12:33.622806+0000","last_unstale":"2024-03-22T04:12:34.658426+0000","last_undegraded":"2024-03-22T04:12:34.658426+0000","last_fullsized":"2024-03-22T04:12:34.658426+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:50:40.568966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.641095+0000","last_change":"2024-03-22T04:12:34.635387+0000","last_active":"2024-03-22T04:12:34.641095+0000","last_peered":"2024-03-22T04:12:34.641095+0000","last_clean":"2024-03-22T04:12:34.641095+0000","last_became_active":"2024-03-22T04:12:34.632095+0000","last_became_peered":"2024-03-22T04:12:34.632095+0000","last_unstale":"2024-03-22T04:12:34.641095+0000","last_undegraded":"2024-03-22T04:12:34.641095+0000","last_fullsized":"2024-03-22T04:12:34.641095+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615307+0000","last_change":"2024-03-22T04:12:33.616617+0000","last_active":"2024-03-22T04:12:35.615307+0000","last_peered":"2024-03-22T04:12:35.615307+0000","last_clean":"2024-03-22T04:12:35.615307+0000","last_became_active":"2024-03-22T04:12:33.616088+0000","last_became_peered":"2024-03-22T04:12:33.616088+0000","last_unstale":"2024-03-22T04:12:35.615307+0000","last_undegraded":"2024-03-22T04:12:35.615307+0000","last_fullsized":"2024-03-22T04:12:35.615307+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:49.907740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.640577+0000","last_change":"2024-03-22T04:12:34.638920+0000","last_active":"2024-03-22T04:12:34.640577+0000","last_peered":"2024-03-22T04:12:34.640577+0000","last_clean":"2024-03-22T04:12:34.640577+0000","last_became_active":"2024-03-22T04:12:34.638167+0000","last_became_peered":"2024-03-22T04:12:34.638167+0000","last_unstale":"2024-03-22T04:12:34.640577+0000","last_undegraded":"2024-03-22T04:12:34.640577+0000","last_fullsized":"2024-03-22T04:12:34.640577+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:21:14.663985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.659863+0000","last_change":"2024-03-22T04:12:33.625471+0000","last_active":"2024-03-22T04:12:34.659863+0000","last_peered":"2024-03-22T04:12:34.659863+0000","last_clean":"2024-03-22T04:12:34.659863+0000","last_became_active":"2024-03-22T04:12:33.625189+0000","last_became_peered":"2024-03-22T04:12:33.625189+0000","last_unstale":"2024-03-22T04:12:34.659863+0000","last_undegraded":"2024-03-22T04:12:34.659863+0000","last_fullsized":"2024-03-22T04:12:34.659863+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615959+0000","last_change":"2024-03-22T04:12:34.642230+0000","last_active":"2024-03-22T04:12:35.615959+0000","last_peered":"2024-03-22T04:12:35.615959+0000","last_clean":"2024-03-22T04:12:35.615959+0000","last_became_active":"2024-03-22T04:12:34.642038+0000","last_became_peered":"2024-03-22T04:12:34.642038+0000","last_unstale":"2024-03-22T04:12:35.615959+0000","last_undegraded":"2024-03-22T04:12:35.615959+0000","last_fullsized":"2024-03-22T04:12:35.615959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612534+0000","last_change":"2024-03-22T04:12:33.620764+0000","last_active":"2024-03-22T04:12:35.612534+0000","last_peered":"2024-03-22T04:12:35.612534+0000","last_clean":"2024-03-22T04:12:35.612534+0000","last_became_active":"2024-03-22T04:12:33.620498+0000","last_became_peered":"2024-03-22T04:12:33.620498+0000","last_unstale":"2024-03-22T04:12:35.612534+0000","last_undegraded":"2024-03-22T04:12:35.612534+0000","last_fullsized":"2024-03-22T04:12:35.612534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.646765+0000","last_change":"2024-03-22T04:12:34.641493+0000","last_active":"2024-03-22T04:12:34.646765+0000","last_peered":"2024-03-22T04:12:34.646765+0000","last_clean":"2024-03-22T04:12:34.646765+0000","last_became_active":"2024-03-22T04:12:34.636610+0000","last_became_peered":"2024-03-22T04:12:34.636610+0000","last_unstale":"2024-03-22T04:12:34.646765+0000","last_undegraded":"2024-03-22T04:12:34.646765+0000","last_fullsized":"2024-03-22T04:12:34.646765+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:01.229933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.617067+0000","last_change":"2024-03-22T04:12:33.618015+0000","last_active":"2024-03-22T04:12:34.617067+0000","last_peered":"2024-03-22T04:12:34.617067+0000","last_clean":"2024-03-22T04:12:34.617067+0000","last_became_active":"2024-03-22T04:12:33.617794+0000","last_became_peered":"2024-03-22T04:12:33.617794+0000","last_unstale":"2024-03-22T04:12:34.617067+0000","last_undegraded":"2024-03-22T04:12:34.617067+0000","last_fullsized":"2024-03-22T04:12:34.617067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:00.222391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602266+0000","last_change":"2024-03-22T04:12:33.602266+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602266+0000","last_undegraded":"2024-03-22T04:12:33.602266+0000","last_fullsized":"2024-03-22T04:12:33.602266+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.661009+0000","last_change":"2024-03-22T04:12:33.621891+0000","last_active":"2024-03-22T04:12:34.661009+0000","last_peered":"2024-03-22T04:12:34.661009+0000","last_clean":"2024-03-22T04:12:34.661009+0000","last_became_active":"2024-03-22T04:12:33.621626+0000","last_became_peered":"2024-03-22T04:12:33.621626+0000","last_unstale":"2024-03-22T04:12:34.661009+0000","last_undegraded":"2024-03-22T04:12:34.661009+0000","last_fullsized":"2024-03-22T04:12:34.661009+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614700+0000","last_change":"2024-03-22T04:12:34.634352+0000","last_active":"2024-03-22T04:12:35.614700+0000","last_peered":"2024-03-22T04:12:35.614700+0000","last_clean":"2024-03-22T04:12:35.614700+0000","last_became_active":"2024-03-22T04:12:34.633729+0000","last_became_peered":"2024-03-22T04:12:34.633729+0000","last_unstale":"2024-03-22T04:12:35.614700+0000","last_undegraded":"2024-03-22T04:12:35.614700+0000","last_fullsized":"2024-03-22T04:12:35.614700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.660467+0000","last_change":"2024-03-22T04:12:33.623382+0000","last_active":"2024-03-22T04:12:34.660467+0000","last_peered":"2024-03-22T04:12:34.660467+0000","last_clean":"2024-03-22T04:12:34.660467+0000","last_became_active":"2024-03-22T04:12:33.622982+0000","last_became_peered":"2024-03-22T04:12:33.622982+0000","last_unstale":"2024-03-22T04:12:34.660467+0000","last_undegraded":"2024-03-22T04:12:34.660467+0000","last_fullsized":"2024-03-22T04:12:34.660467+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614924+0000","last_change":"2024-03-22T04:12:34.627065+0000","last_active":"2024-03-22T04:12:35.614924+0000","last_peered":"2024-03-22T04:12:35.614924+0000","last_clean":"2024-03-22T04:12:35.614924+0000","last_became_active":"2024-03-22T04:12:34.626777+0000","last_became_peered":"2024-03-22T04:12:34.626777+0000","last_unstale":"2024-03-22T04:12:35.614924+0000","last_undegraded":"2024-03-22T04:12:35.614924+0000","last_fullsized":"2024-03-22T04:12:35.614924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.658788+0000","last_change":"2024-03-22T04:12:33.622321+0000","last_active":"2024-03-22T04:12:34.658788+0000","last_peered":"2024-03-22T04:12:34.658788+0000","last_clean":"2024-03-22T04:12:34.658788+0000","last_became_active":"2024-03-22T04:12:33.622058+0000","last_became_peered":"2024-03-22T04:12:33.622058+0000","last_unstale":"2024-03-22T04:12:34.658788+0000","last_undegraded":"2024-03-22T04:12:34.658788+0000","last_fullsized":"2024-03-22T04:12:34.658788+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639497+0000","last_change":"2024-03-22T04:12:34.636289+0000","last_active":"2024-03-22T04:12:34.639497+0000","last_peered":"2024-03-22T04:12:34.639497+0000","last_clean":"2024-03-22T04:12:34.639497+0000","last_became_active":"2024-03-22T04:12:34.636025+0000","last_became_peered":"2024-03-22T04:12:34.636025+0000","last_unstale":"2024-03-22T04:12:34.639497+0000","last_undegraded":"2024-03-22T04:12:34.639497+0000","last_fullsized":"2024-03-22T04:12:34.639497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612548+0000","last_change":"2024-03-22T04:12:33.611878+0000","last_active":"2024-03-22T04:12:35.612548+0000","last_peered":"2024-03-22T04:12:35.612548+0000","last_clean":"2024-03-22T04:12:35.612548+0000","last_became_active":"2024-03-22T04:12:33.611537+0000","last_became_peered":"2024-03-22T04:12:33.611537+0000","last_unstale":"2024-03-22T04:12:35.612548+0000","last_undegraded":"2024-03-22T04:12:35.612548+0000","last_fullsized":"2024-03-22T04:12:35.612548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.653492+0000","last_change":"2024-03-22T04:12:34.650207+0000","last_active":"2024-03-22T04:12:34.653492+0000","last_peered":"2024-03-22T04:12:34.653492+0000","last_clean":"2024-03-22T04:12:34.653492+0000","last_became_active":"2024-03-22T04:12:34.649927+0000","last_became_peered":"2024-03-22T04:12:34.649927+0000","last_unstale":"2024-03-22T04:12:34.653492+0000","last_undegraded":"2024-03-22T04:12:34.653492+0000","last_fullsized":"2024-03-22T04:12:34.653492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:12.985866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.616557+0000","last_change":"2024-03-22T04:12:33.622557+0000","last_active":"2024-03-22T04:12:34.616557+0000","last_peered":"2024-03-22T04:12:34.616557+0000","last_clean":"2024-03-22T04:12:34.616557+0000","last_became_active":"2024-03-22T04:12:33.622240+0000","last_became_peered":"2024-03-22T04:12:33.622240+0000","last_unstale":"2024-03-22T04:12:34.616557+0000","last_undegraded":"2024-03-22T04:12:34.616557+0000","last_fullsized":"2024-03-22T04:12:34.616557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:32.007591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634871+0000","last_change":"2024-03-22T04:12:34.633656+0000","last_active":"2024-03-22T04:12:34.634871+0000","last_peered":"2024-03-22T04:12:34.634871+0000","last_clean":"2024-03-22T04:12:34.634871+0000","last_became_active":"2024-03-22T04:12:34.633150+0000","last_became_peered":"2024-03-22T04:12:34.633150+0000","last_unstale":"2024-03-22T04:12:34.634871+0000","last_undegraded":"2024-03-22T04:12:34.634871+0000","last_fullsized":"2024-03-22T04:12:34.634871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:33.344021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615428+0000","last_change":"2024-03-22T04:12:33.624581+0000","last_active":"2024-03-22T04:12:35.615428+0000","last_peered":"2024-03-22T04:12:35.615428+0000","last_clean":"2024-03-22T04:12:35.615428+0000","last_became_active":"2024-03-22T04:12:33.624324+0000","last_became_peered":"2024-03-22T04:12:33.624324+0000","last_unstale":"2024-03-22T04:12:35.615428+0000","last_undegraded":"2024-03-22T04:12:35.615428+0000","last_fullsized":"2024-03-22T04:12:35.615428+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:41.428263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612405+0000","last_change":"2024-03-22T04:12:34.631558+0000","last_active":"2024-03-22T04:12:35.612405+0000","last_peered":"2024-03-22T04:12:35.612405+0000","last_clean":"2024-03-22T04:12:35.612405+0000","last_became_active":"2024-03-22T04:12:34.621598+0000","last_became_peered":"2024-03-22T04:12:34.621598+0000","last_unstale":"2024-03-22T04:12:35.612405+0000","last_undegraded":"2024-03-22T04:12:35.612405+0000","last_fullsized":"2024-03-22T04:12:35.612405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:43:33.625589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.624342+0000","last_change":"2024-03-22T04:12:33.619281+0000","last_active":"2024-03-22T04:12:33.624342+0000","last_peered":"2024-03-22T04:12:33.624342+0000","last_clean":"2024-03-22T04:12:33.624342+0000","last_became_active":"2024-03-22T04:12:33.618935+0000","last_became_peered":"2024-03-22T04:12:33.618935+0000","last_unstale":"2024-03-22T04:12:33.624342+0000","last_undegraded":"2024-03-22T04:12:33.624342+0000","last_fullsized":"2024-03-22T04:12:33.624342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:11:53.935122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615777+0000","last_change":"2024-03-22T04:12:34.642768+0000","last_active":"2024-03-22T04:12:35.615777+0000","last_peered":"2024-03-22T04:12:35.615777+0000","last_clean":"2024-03-22T04:12:35.615777+0000","last_became_active":"2024-03-22T04:12:34.642534+0000","last_became_peered":"2024-03-22T04:12:34.642534+0000","last_unstale":"2024-03-22T04:12:35.615777+0000","last_undegraded":"2024-03-22T04:12:35.615777+0000","last_fullsized":"2024-03-22T04:12:35.615777+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615251+0000","last_change":"2024-03-22T04:12:33.626234+0000","last_active":"2024-03-22T04:12:35.615251+0000","last_peered":"2024-03-22T04:12:35.615251+0000","last_clean":"2024-03-22T04:12:35.615251+0000","last_became_active":"2024-03-22T04:12:33.623978+0000","last_became_peered":"2024-03-22T04:12:33.623978+0000","last_unstale":"2024-03-22T04:12:35.615251+0000","last_undegraded":"2024-03-22T04:12:35.615251+0000","last_fullsized":"2024-03-22T04:12:35.615251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:45.736028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.648376+0000","last_change":"2024-03-22T04:12:34.646431+0000","last_active":"2024-03-22T04:12:34.648376+0000","last_peered":"2024-03-22T04:12:34.648376+0000","last_clean":"2024-03-22T04:12:34.648376+0000","last_became_active":"2024-03-22T04:12:34.646176+0000","last_became_peered":"2024-03-22T04:12:34.646176+0000","last_unstale":"2024-03-22T04:12:34.648376+0000","last_undegraded":"2024-03-22T04:12:34.648376+0000","last_fullsized":"2024-03-22T04:12:34.648376+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:42.435805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614477+0000","last_change":"2024-03-22T04:12:33.610526+0000","last_active":"2024-03-22T04:12:35.614477+0000","last_peered":"2024-03-22T04:12:35.614477+0000","last_clean":"2024-03-22T04:12:35.614477+0000","last_became_active":"2024-03-22T04:12:33.609782+0000","last_became_peered":"2024-03-22T04:12:33.609782+0000","last_unstale":"2024-03-22T04:12:35.614477+0000","last_undegraded":"2024-03-22T04:12:35.614477+0000","last_fullsized":"2024-03-22T04:12:35.614477+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:11.978324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.611982+0000","last_change":"2024-03-22T04:12:34.622644+0000","last_active":"2024-03-22T04:12:35.611982+0000","last_peered":"2024-03-22T04:12:35.611982+0000","last_clean":"2024-03-22T04:12:35.611982+0000","last_became_active":"2024-03-22T04:12:34.622112+0000","last_became_peered":"2024-03-22T04:12:34.622112+0000","last_unstale":"2024-03-22T04:12:35.611982+0000","last_undegraded":"2024-03-22T04:12:35.611982+0000","last_fullsized":"2024-03-22T04:12:35.611982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657519+0000","last_change":"2024-03-22T04:12:33.621421+0000","last_active":"2024-03-22T04:12:34.657519+0000","last_peered":"2024-03-22T04:12:34.657519+0000","last_clean":"2024-03-22T04:12:34.657519+0000","last_became_active":"2024-03-22T04:12:33.621140+0000","last_became_peered":"2024-03-22T04:12:33.621140+0000","last_unstale":"2024-03-22T04:12:34.657519+0000","last_undegraded":"2024-03-22T04:12:34.657519+0000","last_fullsized":"2024-03-22T04:12:34.657519+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:21.786333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615749+0000","last_change":"2024-03-22T04:12:34.638291+0000","last_active":"2024-03-22T04:12:35.615749+0000","last_peered":"2024-03-22T04:12:35.615749+0000","last_clean":"2024-03-22T04:12:35.615749+0000","last_became_active":"2024-03-22T04:12:34.637953+0000","last_became_peered":"2024-03-22T04:12:34.637953+0000","last_unstale":"2024-03-22T04:12:35.615749+0000","last_undegraded":"2024-03-22T04:12:35.615749+0000","last_fullsized":"2024-03-22T04:12:35.615749+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:37:44.536324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.618566+0000","last_change":"2024-03-22T04:12:33.623102+0000","last_active":"2024-03-22T04:12:34.618566+0000","last_peered":"2024-03-22T04:12:34.618566+0000","last_clean":"2024-03-22T04:12:34.618566+0000","last_became_active":"2024-03-22T04:12:33.622803+0000","last_became_peered":"2024-03-22T04:12:33.622803+0000","last_unstale":"2024-03-22T04:12:34.618566+0000","last_undegraded":"2024-03-22T04:12:34.618566+0000","last_fullsized":"2024-03-22T04:12:34.618566+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:04:41.174822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":52,"state":"creating+peering","last_fresh":"2024-03-22T04:12:33.602478+0000","last_change":"2024-03-22T04:12:33.602478+0000","last_active":"2024-03-22T04:12:33.592171+0000","last_peered":"2024-03-22T04:12:33.592171+0000","last_clean":"2024-03-22T04:12:33.592171+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:12:33.602478+0000","last_undegraded":"2024-03-22T04:12:33.602478+0000","last_fullsized":"2024-03-22T04:12:33.602478+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.622958+0000","last_change":"2024-03-22T04:12:33.618302+0000","last_active":"2024-03-22T04:12:33.622958+0000","last_peered":"2024-03-22T04:12:33.622958+0000","last_clean":"2024-03-22T04:12:33.622958+0000","last_became_active":"2024-03-22T04:12:33.617971+0000","last_became_peered":"2024-03-22T04:12:33.617971+0000","last_unstale":"2024-03-22T04:12:33.622958+0000","last_undegraded":"2024-03-22T04:12:33.622958+0000","last_fullsized":"2024-03-22T04:12:33.622958+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:42.915758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615191+0000","last_change":"2024-03-22T04:12:34.644894+0000","last_active":"2024-03-22T04:12:35.615191+0000","last_peered":"2024-03-22T04:12:35.615191+0000","last_clean":"2024-03-22T04:12:35.615191+0000","last_became_active":"2024-03-22T04:12:34.644526+0000","last_became_peered":"2024-03-22T04:12:34.644526+0000","last_unstale":"2024-03-22T04:12:35.615191+0000","last_undegraded":"2024-03-22T04:12:35.615191+0000","last_fullsized":"2024-03-22T04:12:35.615191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615131+0000","last_change":"2024-03-22T04:12:33.620091+0000","last_active":"2024-03-22T04:12:35.615131+0000","last_peered":"2024-03-22T04:12:35.615131+0000","last_clean":"2024-03-22T04:12:35.615131+0000","last_became_active":"2024-03-22T04:12:33.619790+0000","last_became_peered":"2024-03-22T04:12:33.619790+0000","last_unstale":"2024-03-22T04:12:35.615131+0000","last_undegraded":"2024-03-22T04:12:35.615131+0000","last_fullsized":"2024-03-22T04:12:35.615131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:32.336479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639091+0000","last_change":"2024-03-22T04:12:34.636152+0000","last_active":"2024-03-22T04:12:34.639091+0000","last_peered":"2024-03-22T04:12:34.639091+0000","last_clean":"2024-03-22T04:12:34.639091+0000","last_became_active":"2024-03-22T04:12:34.635865+0000","last_became_peered":"2024-03-22T04:12:34.635865+0000","last_unstale":"2024-03-22T04:12:34.639091+0000","last_undegraded":"2024-03-22T04:12:34.639091+0000","last_fullsized":"2024-03-22T04:12:34.639091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-22T04:12:33.616064+0000","last_change":"2024-03-22T04:12:33.612435+0000","last_active":"2024-03-22T04:12:33.616064+0000","last_peered":"2024-03-22T04:12:33.616064+0000","last_clean":"2024-03-22T04:12:33.616064+0000","last_became_active":"2024-03-22T04:12:33.611642+0000","last_became_peered":"2024-03-22T04:12:33.611642+0000","last_unstale":"2024-03-22T04:12:33.616064+0000","last_undegraded":"2024-03-22T04:12:33.616064+0000","last_fullsized":"2024-03-22T04:12:33.616064+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:44:21.610192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615970+0000","last_change":"2024-03-22T04:12:34.637750+0000","last_active":"2024-03-22T04:12:35.615970+0000","last_peered":"2024-03-22T04:12:35.615970+0000","last_clean":"2024-03-22T04:12:35.615970+0000","last_became_active":"2024-03-22T04:12:34.637269+0000","last_became_peered":"2024-03-22T04:12:34.637269+0000","last_unstale":"2024-03-22T04:12:35.615970+0000","last_undegraded":"2024-03-22T04:12:35.615970+0000","last_fullsized":"2024-03-22T04:12:35.615970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:54:58.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.617221+0000","last_change":"2024-03-22T04:12:33.613711+0000","last_active":"2024-03-22T04:12:34.617221+0000","last_peered":"2024-03-22T04:12:34.617221+0000","last_clean":"2024-03-22T04:12:34.617221+0000","last_became_active":"2024-03-22T04:12:33.613398+0000","last_became_peered":"2024-03-22T04:12:33.613398+0000","last_unstale":"2024-03-22T04:12:34.617221+0000","last_undegraded":"2024-03-22T04:12:34.617221+0000","last_fullsized":"2024-03-22T04:12:34.617221+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:34.304317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.654950+0000","last_change":"2024-03-22T04:12:34.649939+0000","last_active":"2024-03-22T04:12:34.654950+0000","last_peered":"2024-03-22T04:12:34.654950+0000","last_clean":"2024-03-22T04:12:34.654950+0000","last_became_active":"2024-03-22T04:12:34.649639+0000","last_became_peered":"2024-03-22T04:12:34.649639+0000","last_unstale":"2024-03-22T04:12:34.654950+0000","last_undegraded":"2024-03-22T04:12:34.654950+0000","last_fullsized":"2024-03-22T04:12:34.654950+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:01.747862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612050+0000","last_change":"2024-03-22T04:12:33.613244+0000","last_active":"2024-03-22T04:12:35.612050+0000","last_peered":"2024-03-22T04:12:35.612050+0000","last_clean":"2024-03-22T04:12:35.612050+0000","last_became_active":"2024-03-22T04:12:33.612972+0000","last_became_peered":"2024-03-22T04:12:33.612972+0000","last_unstale":"2024-03-22T04:12:35.612050+0000","last_undegraded":"2024-03-22T04:12:35.612050+0000","last_fullsized":"2024-03-22T04:12:35.612050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637167+0000","last_change":"2024-03-22T04:12:34.638813+0000","last_active":"2024-03-22T04:12:35.637167+0000","last_peered":"2024-03-22T04:12:35.637167+0000","last_clean":"2024-03-22T04:12:35.637167+0000","last_became_active":"2024-03-22T04:12:34.638487+0000","last_became_peered":"2024-03-22T04:12:34.638487+0000","last_unstale":"2024-03-22T04:12:35.637167+0000","last_undegraded":"2024-03-22T04:12:35.637167+0000","last_fullsized":"2024-03-22T04:12:35.637167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637130+0000","last_change":"2024-03-22T04:12:33.624669+0000","last_active":"2024-03-22T04:12:35.637130+0000","last_peered":"2024-03-22T04:12:35.637130+0000","last_clean":"2024-03-22T04:12:35.637130+0000","last_became_active":"2024-03-22T04:12:33.624416+0000","last_became_peered":"2024-03-22T04:12:33.624416+0000","last_unstale":"2024-03-22T04:12:35.637130+0000","last_undegraded":"2024-03-22T04:12:35.637130+0000","last_fullsized":"2024-03-22T04:12:35.637130+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:18.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":227,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612039+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:12:35.612039+0000","last_peered":"2024-03-22T04:12:35.612039+0000","last_clean":"2024-03-22T04:12:35.612039+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:12:35.612039+0000","last_undegraded":"2024-03-22T04:12:35.612039+0000","last_fullsized":"2024-03-22T04:12:35.612039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":273831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":274357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":865845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":273865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":273809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":870905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":274357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":598016,"data_stored":259941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.603940+0000', 'last_change': '2024-03-22T04:12:33.603940+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.603940+0000', 'last_undegraded': '2024-03-22T04:12:33.603940+0000', 'last_fullsized': '2024-03-22T04:12:33.603940+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.603153+0000', 'last_change': '2024-03-22T04:12:33.603153+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.603153+0000', 'last_undegraded': '2024-03-22T04:12:33.603153+0000', 'last_fullsized': '2024-03-22T04:12:33.603153+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.603525+0000', 'last_change': '2024-03-22T04:12:33.603525+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.603525+0000', 'last_undegraded': '2024-03-22T04:12:33.603525+0000', 'last_fullsized': '2024-03-22T04:12:33.603525+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.604396+0000', 'last_change': '2024-03-22T04:12:33.604396+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.604396+0000', 'last_undegraded': '2024-03-22T04:12:33.604396+0000', 'last_fullsized': '2024-03-22T04:12:33.604396+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-03-22T04:12:39.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.602445+0000', 'last_change': '2024-03-22T04:12:33.602445+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.602445+0000', 'last_undegraded': '2024-03-22T04:12:33.602445+0000', 'last_fullsized': '2024-03-22T04:12:33.602445+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.603931+0000', 'last_change': '2024-03-22T04:12:33.603931+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.603931+0000', 'last_undegraded': '2024-03-22T04:12:33.603931+0000', 'last_fullsized': '2024-03-22T04:12:33.603931+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.604463+0000', 'last_change': '2024-03-22T04:12:33.604463+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.604463+0000', 'last_undegraded': '2024-03-22T04:12:33.604463+0000', 'last_fullsized': '2024-03-22T04:12:33.604463+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.601918+0000', 'last_change': '2024-03-22T04:12:33.601918+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.601918+0000', 'last_undegraded': '2024-03-22T04:12:33.601918+0000', 'last_fullsized': '2024-03-22T04:12:33.601918+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.602266+0000', 'last_change': '2024-03-22T04:12:33.602266+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.602266+0000', 'last_undegraded': '2024-03-22T04:12:33.602266+0000', 'last_fullsized': '2024-03-22T04:12:33.602266+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-03-22T04:12:39.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 52, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:12:33.602478+0000', 'last_change': '2024-03-22T04:12:33.602478+0000', 'last_active': '2024-03-22T04:12:33.592171+0000', 'last_peered': '2024-03-22T04:12:33.592171+0000', 'last_clean': '2024-03-22T04:12:33.592171+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:12:33.602478+0000', 'last_undegraded': '2024-03-22T04:12:33.602478+0000', 'last_fullsized': '2024-03-22T04:12:33.602478+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 52, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'last_clean_scrub_stamp': '2024-03-22T04:12:33.592171+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-22T04:12:39.450 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-22T04:12:39.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4fe00640 1 Processor -- start 2024-03-22T04:12:39.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4fe00640 1 -- start start 2024-03-22T04:12:39.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5015e4b0 0x7f9c50070380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c500b4390 0x7f9c5006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4fe00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c50070960 con 0x7f9c500cf2c0 2024-03-22T04:12:39.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5015e890 con 0x7f9c500b4390 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9c50161fa0 con 0x7f9c5015e4b0 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c500b4390 0x7f9c5006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5015e4b0 0x7f9c50070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4e000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f9c500cf2c0 0x7f9c500b3c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:48456/0 (socket says 172.21.15.86:48456) 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c500b4390 0x7f9c5006d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51456/0 (socket says 172.21.15.86:51456) 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4e000640 1 -- 172.21.15.86:0/2725325970 learned_addr learned my addr 172.21.15.86:0/2725325970 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4f400640 1 -- 172.21.15.86:0/2725325970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5015e4b0 msgr2=0x7f9c50070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4f400640 1 --2- 172.21.15.86:0/2725325970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5015e4b0 0x7f9c50070380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4f400640 1 -- 172.21.15.86:0/2725325970 >> v1:172.21.15.196:6789/0 conn(0x7f9c500cf2c0 legacy=0x7f9c500b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4f400640 1 -- 172.21.15.86:0/2725325970 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c501621f0 con 0x7f9c500b4390 2024-03-22T04:12:39.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.613+0000 7f9c4d600640 1 -- 172.21.15.86:0/2725325970 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320297221 0 0) 0x7f9c50070960 con 0x7f9c500cf2c0 2024-03-22T04:12:39.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4f400640 1 --2- 172.21.15.86:0/2725325970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5006d4d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c480097c0 tx=0x7f9c4801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9200b4f6928e02 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:39.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4d600640 1 -- 172.21.15.86:0/2725325970 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c480b0070 con 0x7f9c500b4390 2024-03-22T04:12:39.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4d600640 1 -- 172.21.15.86:0/2725325970 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c480ab600 con 0x7f9c500b4390 2024-03-22T04:12:39.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4d600640 1 -- 172.21.15.86:0/2725325970 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c480ab9d0 con 0x7f9c500b4390 2024-03-22T04:12:39.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4fe00640 1 -- 172.21.15.86:0/2725325970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 msgr2=0x7f9c5006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/2725325970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5006d4d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c480097c0 tx=0x7f9c4801c410 comp rx=0 tx=0).stop 2024-03-22T04:12:39.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4fe00640 1 -- 172.21.15.86:0/2725325970 shutdown_connections 2024-03-22T04:12:39.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/2725325970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5006d4d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/2725325970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5015e4b0 0x7f9c50070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4fe00640 1 -- 172.21.15.86:0/2725325970 >> 172.21.15.86:0/2725325970 conn(0x7f9c50067970 msgr2=0x7f9c500b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:39.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4fe00640 1 -- 172.21.15.86:0/2725325970 shutdown_connections 2024-03-22T04:12:39.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.617+0000 7f9c4fe00640 1 -- 172.21.15.86:0/2725325970 wait complete. 2024-03-22T04:12:39.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 Processor -- start 2024-03-22T04:12:39.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 -- start start 2024-03-22T04:12:39.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5014ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c500cf2c0 0x7f9c500b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5015e4b0 0x7f9c500b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5006e3a0 con 0x7f9c500b4390 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5006e220 con 0x7f9c5015e4b0 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5006e520 con 0x7f9c500cf2c0 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5014ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5014ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51460/0 (socket says 172.21.15.86:51460) 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 -- 172.21.15.86:0/674871704 learned_addr learned my addr 172.21.15.86:0/674871704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4e000640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c500cf2c0 0x7f9c500b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4f400640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5015e4b0 0x7f9c500b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 -- 172.21.15.86:0/674871704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c500cf2c0 msgr2=0x7f9c500b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c500cf2c0 0x7f9c500b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 -- 172.21.15.86:0/674871704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5015e4b0 msgr2=0x7f9c500b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5015e4b0 0x7f9c500b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 -- 172.21.15.86:0/674871704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c501607c0 con 0x7f9c500b4390 2024-03-22T04:12:39.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4ea00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5014ebc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009d10 tx=0x7f9c4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:39.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c46a00640 1 -- 172.21.15.86:0/674871704 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40018020 con 0x7f9c500b4390 2024-03-22T04:12:39.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c46a00640 1 -- 172.21.15.86:0/674871704 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c40014480 con 0x7f9c500b4390 2024-03-22T04:12:39.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c46a00640 1 -- 172.21.15.86:0/674871704 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40014830 con 0x7f9c500b4390 2024-03-22T04:12:39.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5015ff20 con 0x7f9c500b4390 2024-03-22T04:12:39.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c501604e0 con 0x7f9c500b4390 2024-03-22T04:12:39.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.621+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c50074470 con 0x7f9c500b4390 2024-03-22T04:12:39.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.625+0000 7f9c46a00640 1 -- 172.21.15.86:0/674871704 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9c4001b020 con 0x7f9c500b4390 2024-03-22T04:12:39.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.625+0000 7f9c46a00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c30067b20 0x7f9c30069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:39.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.625+0000 7f9c46a00640 1 -- 172.21.15.86:0/674871704 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9c40086990 con 0x7f9c500b4390 2024-03-22T04:12:39.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.625+0000 7f9c4e000640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c30067b20 0x7f9c30069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:39.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.625+0000 7f9c4e000640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c30067b20 0x7f9c30069fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c50155f50 tx=0x7f9c3c0079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:39.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.629+0000 7f9c46a00640 1 -- 172.21.15.86:0/674871704 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9c40022070 con 0x7f9c500b4390 2024-03-22T04:12:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.837+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9c50168c50 con 0x7f9c500b4390 2024-03-22T04:12:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.837+0000 7f9c46a00640 1 -- 172.21.15.86:0/674871704 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1100 (secure 0 0 0) 0x7f9c40010040 con 0x7f9c500b4390 2024-03-22T04:12:39.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:39.843 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":150,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1711080620,"num_in_osds":8,"osd_in_since":1711080611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":87},{"state_name":"creating+peering","count":10}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":227569664,"bytes_avail":772866543616,"bytes_total":773094113280,"inactive_pgs_ratio":0.10309278219938278,"write_bytes_sec":2745,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":75245}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-03-22T04:11:39.805364+0000","services":{}},"progress_events":{}} 2024-03-22T04:12:39.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c30067b20 msgr2=0x7f9c30069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c30067b20 0x7f9c30069fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c50155f50 tx=0x7f9c3c0079e0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 msgr2=0x7f9c5014ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5014ebc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009d10 tx=0x7f9c4000c490 comp rx=0 tx=0).stop 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 shutdown_connections 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c30067b20 0x7f9c30069fe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5015e4b0 0x7f9c500b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c500cf2c0 0x7f9c500b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/674871704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c500b4390 0x7f9c5014ebc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 >> 172.21.15.86:0/674871704 conn(0x7f9c50067970 msgr2=0x7f9c5015eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 shutdown_connections 2024-03-22T04:12:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.841+0000 7f9c4fe00640 1 -- 172.21.15.86:0/674871704 wait complete. 2024-03-22T04:12:39.864 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-22T04:12:39.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:12:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5dee600640 1 Processor -- start 2024-03-22T04:12:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5dee600640 1 -- start start 2024-03-22T04:12:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5dee600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de815e630 0x7f5de80cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5dee600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5de806e2f0 0x7f5de80d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5dee600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5de806cc80 0x7f5de806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5dee600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5de81623d0 con 0x7f5de806cc80 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5dee600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5de81620f0 con 0x7f5de806e2f0 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5dee600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5de815ec10 con 0x7f5de815e630 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5de806cc80 0x7f5de806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de6a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5de806e2f0 0x7f5de80d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de815e630 0x7f5de80cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5de806cc80 0x7f5de806d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51462/0 (socket says 172.21.15.86:51462) 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de815e630 0x7f5de80cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42382/0 (socket says 172.21.15.86:42382) 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7e00640 1 -- 172.21.15.86:0/454542656 learned_addr learned my addr 172.21.15.86:0/454542656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7e00640 1 -- 172.21.15.86:0/454542656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de815e630 msgr2=0x7f5de80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7e00640 1 --2- 172.21.15.86:0/454542656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de815e630 0x7f5de80cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7e00640 1 -- 172.21.15.86:0/454542656 >> v2:172.21.15.196:3300/0 conn(0x7f5de806e2f0 msgr2=0x7f5de80d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7e00640 1 --2- 172.21.15.86:0/454542656 >> v2:172.21.15.196:3300/0 conn(0x7f5de806e2f0 0x7f5de80d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:39.997+0000 7f5de7e00640 1 -- 172.21.15.86:0/454542656 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de815eef0 con 0x7f5de806cc80 2024-03-22T04:12:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5de7e00640 1 --2- 172.21.15.86:0/454542656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de806cc80 0x7f5de806d0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5de806c640 tx=0x7f5de001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f038c85c82cdc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5de6000640 1 -- 172.21.15.86:0/454542656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00b0070 con 0x7f5de806cc80 2024-03-22T04:12:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5de6000640 1 -- 172.21.15.86:0/454542656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de00ab600 con 0x7f5de806cc80 2024-03-22T04:12:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5de6000640 1 -- 172.21.15.86:0/454542656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00ab9d0 con 0x7f5de806cc80 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 -- 172.21.15.86:0/454542656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de806cc80 msgr2=0x7f5de806d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 --2- 172.21.15.86:0/454542656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de806cc80 0x7f5de806d0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5de806c640 tx=0x7f5de001c520 comp rx=0 tx=0).stop 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 -- 172.21.15.86:0/454542656 shutdown_connections 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 --2- 172.21.15.86:0/454542656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de806cc80 0x7f5de806d0d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 --2- 172.21.15.86:0/454542656 >> v2:172.21.15.196:3300/0 conn(0x7f5de806e2f0 0x7f5de80d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 --2- 172.21.15.86:0/454542656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de815e630 0x7f5de80cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 -- 172.21.15.86:0/454542656 >> 172.21.15.86:0/454542656 conn(0x7f5de806f910 msgr2=0x7f5de806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 -- 172.21.15.86:0/454542656 shutdown_connections 2024-03-22T04:12:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.001+0000 7f5dee600640 1 -- 172.21.15.86:0/454542656 wait complete. 2024-03-22T04:12:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 Processor -- start 2024-03-22T04:12:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 -- start start 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de806cc80 0x7f5de80cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de806e2f0 0x7f5de80cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de815e630 0x7f5de8145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5de815f9d0 con 0x7f5de815e630 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5de806da20 con 0x7f5de806e2f0 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5de80cfaf0 con 0x7f5de806cc80 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de815e630 0x7f5de8145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de815e630 0x7f5de8145850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51478/0 (socket says 172.21.15.86:51478) 2024-03-22T04:12:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 -- 172.21.15.86:0/1083693736 learned_addr learned my addr 172.21.15.86:0/1083693736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de6a00640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de806e2f0 0x7f5de80cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7400640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de806cc80 0x7f5de80cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 -- 172.21.15.86:0/1083693736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de806cc80 msgr2=0x7f5de80cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de806cc80 0x7f5de80cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 -- 172.21.15.86:0/1083693736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de806e2f0 msgr2=0x7f5de80cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de806e2f0 0x7f5de80cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 -- 172.21.15.86:0/1083693736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de80cc490 con 0x7f5de815e630 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5de7e00640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de815e630 0x7f5de8145850 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5de00ae040 tx=0x7f5de00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5ddf400640 1 -- 172.21.15.86:0/1083693736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00b0040 con 0x7f5de815e630 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5ddf400640 1 -- 172.21.15.86:0/1083693736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de00b9070 con 0x7f5de815e630 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de8157920 con 0x7f5de815e630 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5ddf400640 1 -- 172.21.15.86:0/1083693736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00b4670 con 0x7f5de815e630 2024-03-22T04:12:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.005+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de8157ee0 con 0x7f5de815e630 2024-03-22T04:12:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.009+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de8068650 con 0x7f5de815e630 2024-03-22T04:12:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.009+0000 7f5ddf400640 1 -- 172.21.15.86:0/1083693736 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5de00b4810 con 0x7f5de815e630 2024-03-22T04:12:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.009+0000 7f5ddf400640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dc8067b20 0x7f5dc8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.009+0000 7f5ddf400640 1 -- 172.21.15.86:0/1083693736 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f5de011f6b0 con 0x7f5de815e630 2024-03-22T04:12:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.009+0000 7f5de7400640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dc8067b20 0x7f5dc8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.009+0000 7f5de7400640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dc8067b20 0x7f5dc8069fe0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8011340 tx=0x7f5dd800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.013+0000 7f5ddf400640 1 -- 172.21.15.86:0/1083693736 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5de00c2020 con 0x7f5de815e630 2024-03-22T04:12:40.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.165+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5de8162a40 con 0x7f5dc8067b20 2024-03-22T04:12:40.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.173+0000 7f5ddf400640 1 -- 172.21.15.86:0/1083693736 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251931 (secure 0 0 0) 0x7f5de8162a40 con 0x7f5dc8067b20 2024-03-22T04:12:40.177 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dc8067b20 msgr2=0x7f5dc8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dc8067b20 0x7f5dc8069fe0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8011340 tx=0x7f5dd800b040 comp rx=0 tx=0).stop 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de815e630 msgr2=0x7f5de8145850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de815e630 0x7f5de8145850 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5de00ae040 tx=0x7f5de00192c0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 shutdown_connections 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dc8067b20 0x7f5dc8069fe0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de806e2f0 0x7f5de80cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de815e630 0x7f5de8145850 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 --2- 172.21.15.86:0/1083693736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de806cc80 0x7f5de80cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 >> 172.21.15.86:0/1083693736 conn(0x7f5de806f910 msgr2=0x7f5de80cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 shutdown_connections 2024-03-22T04:12:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:40.177+0000 7f5dee600640 1 -- 172.21.15.86:0/1083693736 wait complete. 2024-03-22T04:12:40.198 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-03-22T04:12:39.816614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222284,"kb_used_data":6112,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752436,"statfs":{"total":773094113280,"available":772866494464,"internally_reserved":0,"allocated":6258688,"data_stored":3389140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.220461"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616060+0000","last_change":"2024-03-22T04:12:34.628936+0000","last_active":"2024-03-22T04:12:35.616060+0000","last_peered":"2024-03-22T04:12:35.616060+0000","last_clean":"2024-03-22T04:12:35.616060+0000","last_became_active":"2024-03-22T04:12:34.628641+0000","last_became_peered":"2024-03-22T04:12:34.628641+0000","last_unstale":"2024-03-22T04:12:35.616060+0000","last_undegraded":"2024-03-22T04:12:35.616060+0000","last_fullsized":"2024-03-22T04:12:35.616060+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639641+0000","last_change":"2024-03-22T04:12:34.637267+0000","last_active":"2024-03-22T04:12:34.639641+0000","last_peered":"2024-03-22T04:12:34.639641+0000","last_clean":"2024-03-22T04:12:34.639641+0000","last_became_active":"2024-03-22T04:12:34.637024+0000","last_became_peered":"2024-03-22T04:12:34.637024+0000","last_unstale":"2024-03-22T04:12:34.639641+0000","last_undegraded":"2024-03-22T04:12:34.639641+0000","last_fullsized":"2024-03-22T04:12:34.639641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951210+0000","last_change":"2024-03-22T04:12:34.624650+0000","last_active":"2024-03-22T04:12:35.951210+0000","last_peered":"2024-03-22T04:12:35.951210+0000","last_clean":"2024-03-22T04:12:35.951210+0000","last_became_active":"2024-03-22T04:12:34.624347+0000","last_became_peered":"2024-03-22T04:12:34.624347+0000","last_unstale":"2024-03-22T04:12:35.951210+0000","last_undegraded":"2024-03-22T04:12:35.951210+0000","last_fullsized":"2024-03-22T04:12:35.951210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:16:52.531907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612444+0000","last_change":"2024-03-22T04:12:34.625832+0000","last_active":"2024-03-22T04:12:35.612444+0000","last_peered":"2024-03-22T04:12:35.612444+0000","last_clean":"2024-03-22T04:12:35.612444+0000","last_became_active":"2024-03-22T04:12:34.625533+0000","last_became_peered":"2024-03-22T04:12:34.625533+0000","last_unstale":"2024-03-22T04:12:35.612444+0000","last_undegraded":"2024-03-22T04:12:35.612444+0000","last_fullsized":"2024-03-22T04:12:35.612444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:00.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.635523+0000","last_change":"2024-03-22T04:12:34.627773+0000","last_active":"2024-03-22T04:12:34.635523+0000","last_peered":"2024-03-22T04:12:34.635523+0000","last_clean":"2024-03-22T04:12:34.635523+0000","last_became_active":"2024-03-22T04:12:34.624749+0000","last_became_peered":"2024-03-22T04:12:34.624749+0000","last_unstale":"2024-03-22T04:12:34.635523+0000","last_undegraded":"2024-03-22T04:12:34.635523+0000","last_fullsized":"2024-03-22T04:12:34.635523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:01.747862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643345+0000","last_change":"2024-03-22T04:12:34.635777+0000","last_active":"2024-03-22T04:12:34.643345+0000","last_peered":"2024-03-22T04:12:34.643345+0000","last_clean":"2024-03-22T04:12:34.643345+0000","last_became_active":"2024-03-22T04:12:34.635528+0000","last_became_peered":"2024-03-22T04:12:34.635528+0000","last_unstale":"2024-03-22T04:12:34.643345+0000","last_undegraded":"2024-03-22T04:12:34.643345+0000","last_fullsized":"2024-03-22T04:12:34.643345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:01:53.135313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614543+0000","last_change":"2024-03-22T04:12:34.636516+0000","last_active":"2024-03-22T04:12:35.614543+0000","last_peered":"2024-03-22T04:12:35.614543+0000","last_clean":"2024-03-22T04:12:35.614543+0000","last_became_active":"2024-03-22T04:12:34.636312+0000","last_became_peered":"2024-03-22T04:12:34.636312+0000","last_unstale":"2024-03-22T04:12:35.614543+0000","last_undegraded":"2024-03-22T04:12:35.614543+0000","last_fullsized":"2024-03-22T04:12:35.614543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657816+0000","last_change":"2024-03-22T04:12:34.650010+0000","last_active":"2024-03-22T04:12:34.657816+0000","last_peered":"2024-03-22T04:12:34.657816+0000","last_clean":"2024-03-22T04:12:34.657816+0000","last_became_active":"2024-03-22T04:12:34.649709+0000","last_became_peered":"2024-03-22T04:12:34.649709+0000","last_unstale":"2024-03-22T04:12:34.657816+0000","last_undegraded":"2024-03-22T04:12:34.657816+0000","last_fullsized":"2024-03-22T04:12:34.657816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:50.915283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951328+0000","last_change":"2024-03-22T04:12:34.625212+0000","last_active":"2024-03-22T04:12:35.951328+0000","last_peered":"2024-03-22T04:12:35.951328+0000","last_clean":"2024-03-22T04:12:35.951328+0000","last_became_active":"2024-03-22T04:12:34.624758+0000","last_became_peered":"2024-03-22T04:12:34.624758+0000","last_unstale":"2024-03-22T04:12:35.951328+0000","last_undegraded":"2024-03-22T04:12:35.951328+0000","last_fullsized":"2024-03-22T04:12:35.951328+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:23:14.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615323+0000","last_change":"2024-03-22T04:12:34.641332+0000","last_active":"2024-03-22T04:12:35.615323+0000","last_peered":"2024-03-22T04:12:35.615323+0000","last_clean":"2024-03-22T04:12:35.615323+0000","last_became_active":"2024-03-22T04:12:34.641059+0000","last_became_peered":"2024-03-22T04:12:34.641059+0000","last_unstale":"2024-03-22T04:12:35.615323+0000","last_undegraded":"2024-03-22T04:12:35.615323+0000","last_fullsized":"2024-03-22T04:12:35.615323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615824+0000","last_change":"2024-03-22T04:12:34.627929+0000","last_active":"2024-03-22T04:12:35.615824+0000","last_peered":"2024-03-22T04:12:35.615824+0000","last_clean":"2024-03-22T04:12:35.615824+0000","last_became_active":"2024-03-22T04:12:34.627640+0000","last_became_peered":"2024-03-22T04:12:34.627640+0000","last_unstale":"2024-03-22T04:12:35.615824+0000","last_undegraded":"2024-03-22T04:12:35.615824+0000","last_fullsized":"2024-03-22T04:12:35.615824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643682+0000","last_change":"2024-03-22T04:12:34.639709+0000","last_active":"2024-03-22T04:12:34.643682+0000","last_peered":"2024-03-22T04:12:34.643682+0000","last_clean":"2024-03-22T04:12:34.643682+0000","last_became_active":"2024-03-22T04:12:34.638936+0000","last_became_peered":"2024-03-22T04:12:34.638936+0000","last_unstale":"2024-03-22T04:12:34.643682+0000","last_undegraded":"2024-03-22T04:12:34.643682+0000","last_fullsized":"2024-03-22T04:12:34.643682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:33.015134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.647737+0000","last_change":"2024-03-22T04:12:34.644837+0000","last_active":"2024-03-22T04:12:34.647737+0000","last_peered":"2024-03-22T04:12:34.647737+0000","last_clean":"2024-03-22T04:12:34.647737+0000","last_became_active":"2024-03-22T04:12:34.644507+0000","last_became_peered":"2024-03-22T04:12:34.644507+0000","last_unstale":"2024-03-22T04:12:34.647737+0000","last_undegraded":"2024-03-22T04:12:34.647737+0000","last_fullsized":"2024-03-22T04:12:34.647737+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:35.311859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614973+0000","last_change":"2024-03-22T04:12:34.641248+0000","last_active":"2024-03-22T04:12:35.614973+0000","last_peered":"2024-03-22T04:12:35.614973+0000","last_clean":"2024-03-22T04:12:35.614973+0000","last_became_active":"2024-03-22T04:12:34.640953+0000","last_became_peered":"2024-03-22T04:12:34.640953+0000","last_unstale":"2024-03-22T04:12:35.614973+0000","last_undegraded":"2024-03-22T04:12:35.614973+0000","last_fullsized":"2024-03-22T04:12:35.614973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612098+0000","last_change":"2024-03-22T04:12:34.632175+0000","last_active":"2024-03-22T04:12:35.612098+0000","last_peered":"2024-03-22T04:12:35.612098+0000","last_clean":"2024-03-22T04:12:35.612098+0000","last_became_active":"2024-03-22T04:12:34.631916+0000","last_became_peered":"2024-03-22T04:12:34.631916+0000","last_unstale":"2024-03-22T04:12:35.612098+0000","last_undegraded":"2024-03-22T04:12:35.612098+0000","last_fullsized":"2024-03-22T04:12:35.612098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.643732+0000","last_change":"2024-03-22T04:12:34.639122+0000","last_active":"2024-03-22T04:12:34.643732+0000","last_peered":"2024-03-22T04:12:34.643732+0000","last_clean":"2024-03-22T04:12:34.643732+0000","last_became_active":"2024-03-22T04:12:34.638788+0000","last_became_peered":"2024-03-22T04:12:34.638788+0000","last_unstale":"2024-03-22T04:12:34.643732+0000","last_undegraded":"2024-03-22T04:12:34.643732+0000","last_fullsized":"2024-03-22T04:12:34.643732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:27.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616261+0000","last_change":"2024-03-22T04:12:34.632974+0000","last_active":"2024-03-22T04:12:35.616261+0000","last_peered":"2024-03-22T04:12:35.616261+0000","last_clean":"2024-03-22T04:12:35.616261+0000","last_became_active":"2024-03-22T04:12:34.627665+0000","last_became_peered":"2024-03-22T04:12:34.627665+0000","last_unstale":"2024-03-22T04:12:35.616261+0000","last_undegraded":"2024-03-22T04:12:35.616261+0000","last_fullsized":"2024-03-22T04:12:35.616261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612642+0000","last_change":"2024-03-22T04:12:34.626077+0000","last_active":"2024-03-22T04:12:35.612642+0000","last_peered":"2024-03-22T04:12:35.612642+0000","last_clean":"2024-03-22T04:12:35.612642+0000","last_became_active":"2024-03-22T04:12:34.625837+0000","last_became_peered":"2024-03-22T04:12:34.625837+0000","last_unstale":"2024-03-22T04:12:35.612642+0000","last_undegraded":"2024-03-22T04:12:35.612642+0000","last_fullsized":"2024-03-22T04:12:35.612642+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951555+0000","last_change":"2024-03-22T04:12:34.622480+0000","last_active":"2024-03-22T04:12:35.951555+0000","last_peered":"2024-03-22T04:12:35.951555+0000","last_clean":"2024-03-22T04:12:35.951555+0000","last_became_active":"2024-03-22T04:12:34.622192+0000","last_became_peered":"2024-03-22T04:12:34.622192+0000","last_unstale":"2024-03-22T04:12:35.951555+0000","last_undegraded":"2024-03-22T04:12:35.951555+0000","last_fullsized":"2024-03-22T04:12:35.951555+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:04:24.850239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634836+0000","last_change":"2024-03-22T04:12:34.630270+0000","last_active":"2024-03-22T04:12:34.634836+0000","last_peered":"2024-03-22T04:12:34.634836+0000","last_clean":"2024-03-22T04:12:34.634836+0000","last_became_active":"2024-03-22T04:12:34.629989+0000","last_became_peered":"2024-03-22T04:12:34.629989+0000","last_unstale":"2024-03-22T04:12:34.634836+0000","last_undegraded":"2024-03-22T04:12:34.634836+0000","last_fullsized":"2024-03-22T04:12:34.634836+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:50.915283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612164+0000","last_change":"2024-03-22T04:12:34.620830+0000","last_active":"2024-03-22T04:12:35.612164+0000","last_peered":"2024-03-22T04:12:35.612164+0000","last_clean":"2024-03-22T04:12:35.612164+0000","last_became_active":"2024-03-22T04:12:34.620520+0000","last_became_peered":"2024-03-22T04:12:34.620520+0000","last_unstale":"2024-03-22T04:12:35.612164+0000","last_undegraded":"2024-03-22T04:12:35.612164+0000","last_fullsized":"2024-03-22T04:12:35.612164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951264+0000","last_change":"2024-03-22T04:12:34.640293+0000","last_active":"2024-03-22T04:12:35.951264+0000","last_peered":"2024-03-22T04:12:35.951264+0000","last_clean":"2024-03-22T04:12:35.951264+0000","last_became_active":"2024-03-22T04:12:34.639924+0000","last_became_peered":"2024-03-22T04:12:34.639924+0000","last_unstale":"2024-03-22T04:12:35.951264+0000","last_undegraded":"2024-03-22T04:12:35.951264+0000","last_fullsized":"2024-03-22T04:12:35.951264+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:17.196299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951146+0000","last_change":"2024-03-22T04:12:34.621196+0000","last_active":"2024-03-22T04:12:35.951146+0000","last_peered":"2024-03-22T04:12:35.951146+0000","last_clean":"2024-03-22T04:12:35.951146+0000","last_became_active":"2024-03-22T04:12:34.620932+0000","last_became_peered":"2024-03-22T04:12:34.620932+0000","last_unstale":"2024-03-22T04:12:35.951146+0000","last_undegraded":"2024-03-22T04:12:35.951146+0000","last_fullsized":"2024-03-22T04:12:35.951146+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:18:25.483759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951097+0000","last_change":"2024-03-22T04:12:34.638359+0000","last_active":"2024-03-22T04:12:35.951097+0000","last_peered":"2024-03-22T04:12:35.951097+0000","last_clean":"2024-03-22T04:12:35.951097+0000","last_became_active":"2024-03-22T04:12:34.638069+0000","last_became_peered":"2024-03-22T04:12:34.638069+0000","last_unstale":"2024-03-22T04:12:35.951097+0000","last_undegraded":"2024-03-22T04:12:35.951097+0000","last_fullsized":"2024-03-22T04:12:35.951097+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:33:19.884683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616176+0000","last_change":"2024-03-22T04:12:34.635548+0000","last_active":"2024-03-22T04:12:35.616176+0000","last_peered":"2024-03-22T04:12:35.616176+0000","last_clean":"2024-03-22T04:12:35.616176+0000","last_became_active":"2024-03-22T04:12:34.635270+0000","last_became_peered":"2024-03-22T04:12:34.635270+0000","last_unstale":"2024-03-22T04:12:35.616176+0000","last_undegraded":"2024-03-22T04:12:35.616176+0000","last_fullsized":"2024-03-22T04:12:35.616176+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951282+0000","last_change":"2024-03-22T04:12:34.637119+0000","last_active":"2024-03-22T04:12:35.951282+0000","last_peered":"2024-03-22T04:12:35.951282+0000","last_clean":"2024-03-22T04:12:35.951282+0000","last_became_active":"2024-03-22T04:12:34.636839+0000","last_became_peered":"2024-03-22T04:12:34.636839+0000","last_unstale":"2024-03-22T04:12:35.951282+0000","last_undegraded":"2024-03-22T04:12:35.951282+0000","last_fullsized":"2024-03-22T04:12:35.951282+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:02:15.918731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.659926+0000","last_change":"2024-03-22T04:12:34.655920+0000","last_active":"2024-03-22T04:12:34.659926+0000","last_peered":"2024-03-22T04:12:34.659926+0000","last_clean":"2024-03-22T04:12:34.659926+0000","last_became_active":"2024-03-22T04:12:34.655580+0000","last_became_peered":"2024-03-22T04:12:34.655580+0000","last_unstale":"2024-03-22T04:12:34.659926+0000","last_undegraded":"2024-03-22T04:12:34.659926+0000","last_fullsized":"2024-03-22T04:12:34.659926+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:33.344021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634856+0000","last_change":"2024-03-22T04:12:34.633424+0000","last_active":"2024-03-22T04:12:34.634856+0000","last_peered":"2024-03-22T04:12:34.634856+0000","last_clean":"2024-03-22T04:12:34.634856+0000","last_became_active":"2024-03-22T04:12:34.632797+0000","last_became_peered":"2024-03-22T04:12:34.632797+0000","last_unstale":"2024-03-22T04:12:34.634856+0000","last_undegraded":"2024-03-22T04:12:34.634856+0000","last_fullsized":"2024-03-22T04:12:34.634856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:12.985866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615318+0000","last_change":"2024-03-22T04:12:34.630952+0000","last_active":"2024-03-22T04:12:35.615318+0000","last_peered":"2024-03-22T04:12:35.615318+0000","last_clean":"2024-03-22T04:12:35.615318+0000","last_became_active":"2024-03-22T04:12:34.630765+0000","last_became_peered":"2024-03-22T04:12:34.630765+0000","last_unstale":"2024-03-22T04:12:35.615318+0000","last_undegraded":"2024-03-22T04:12:35.615318+0000","last_fullsized":"2024-03-22T04:12:35.615318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.625106+0000","last_change":"2024-03-22T04:12:34.621885+0000","last_active":"2024-03-22T04:12:34.625106+0000","last_peered":"2024-03-22T04:12:34.625106+0000","last_clean":"2024-03-22T04:12:34.625106+0000","last_became_active":"2024-03-22T04:12:34.621626+0000","last_became_peered":"2024-03-22T04:12:34.621626+0000","last_unstale":"2024-03-22T04:12:34.625106+0000","last_undegraded":"2024-03-22T04:12:34.625106+0000","last_fullsized":"2024-03-22T04:12:34.625106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:12:20.702700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951012+0000","last_change":"2024-03-22T04:12:34.640876+0000","last_active":"2024-03-22T04:12:35.951012+0000","last_peered":"2024-03-22T04:12:35.951012+0000","last_clean":"2024-03-22T04:12:35.951012+0000","last_became_active":"2024-03-22T04:12:34.640622+0000","last_became_peered":"2024-03-22T04:12:34.640622+0000","last_unstale":"2024-03-22T04:12:35.951012+0000","last_undegraded":"2024-03-22T04:12:35.951012+0000","last_fullsized":"2024-03-22T04:12:35.951012+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:52:07.467181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.630607+0000","last_change":"2024-03-22T04:12:34.624213+0000","last_active":"2024-03-22T04:12:34.630607+0000","last_peered":"2024-03-22T04:12:34.630607+0000","last_clean":"2024-03-22T04:12:34.630607+0000","last_became_active":"2024-03-22T04:12:34.623954+0000","last_became_peered":"2024-03-22T04:12:34.623954+0000","last_unstale":"2024-03-22T04:12:34.630607+0000","last_undegraded":"2024-03-22T04:12:34.630607+0000","last_fullsized":"2024-03-22T04:12:34.630607+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:42.435805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.629445+0000","last_change":"2024-03-22T04:12:34.623255+0000","last_active":"2024-03-22T04:12:34.629445+0000","last_peered":"2024-03-22T04:12:34.629445+0000","last_clean":"2024-03-22T04:12:34.629445+0000","last_became_active":"2024-03-22T04:12:34.622995+0000","last_became_peered":"2024-03-22T04:12:34.622995+0000","last_unstale":"2024-03-22T04:12:34.629445+0000","last_undegraded":"2024-03-22T04:12:34.629445+0000","last_fullsized":"2024-03-22T04:12:34.629445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:50.075295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637402+0000","last_change":"2024-03-22T04:12:33.618251+0000","last_active":"2024-03-22T04:12:35.637402+0000","last_peered":"2024-03-22T04:12:35.637402+0000","last_clean":"2024-03-22T04:12:35.637402+0000","last_became_active":"2024-03-22T04:12:33.617405+0000","last_became_peered":"2024-03-22T04:12:33.617405+0000","last_unstale":"2024-03-22T04:12:35.637402+0000","last_undegraded":"2024-03-22T04:12:35.637402+0000","last_fullsized":"2024-03-22T04:12:35.637402+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:45.786716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.636934+0000","last_change":"2024-03-22T04:12:34.633188+0000","last_active":"2024-03-22T04:12:34.636934+0000","last_peered":"2024-03-22T04:12:34.636934+0000","last_clean":"2024-03-22T04:12:34.636934+0000","last_became_active":"2024-03-22T04:12:34.632933+0000","last_became_peered":"2024-03-22T04:12:34.632933+0000","last_unstale":"2024-03-22T04:12:34.636934+0000","last_undegraded":"2024-03-22T04:12:34.636934+0000","last_fullsized":"2024-03-22T04:12:34.636934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:30:04.785433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.660378+0000","last_change":"2024-03-22T04:12:33.623741+0000","last_active":"2024-03-22T04:12:34.660378+0000","last_peered":"2024-03-22T04:12:34.660378+0000","last_clean":"2024-03-22T04:12:34.660378+0000","last_became_active":"2024-03-22T04:12:33.623407+0000","last_became_peered":"2024-03-22T04:12:33.623407+0000","last_unstale":"2024-03-22T04:12:34.660378+0000","last_undegraded":"2024-03-22T04:12:34.660378+0000","last_fullsized":"2024-03-22T04:12:34.660378+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.625128+0000","last_change":"2024-03-22T04:12:34.622229+0000","last_active":"2024-03-22T04:12:34.625128+0000","last_peered":"2024-03-22T04:12:34.625128+0000","last_clean":"2024-03-22T04:12:34.625128+0000","last_became_active":"2024-03-22T04:12:34.621209+0000","last_became_peered":"2024-03-22T04:12:34.621209+0000","last_unstale":"2024-03-22T04:12:34.625128+0000","last_undegraded":"2024-03-22T04:12:34.625128+0000","last_fullsized":"2024-03-22T04:12:34.625128+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:14:10.558919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615885+0000","last_change":"2024-03-22T04:12:33.621633+0000","last_active":"2024-03-22T04:12:35.615885+0000","last_peered":"2024-03-22T04:12:35.615885+0000","last_clean":"2024-03-22T04:12:35.615885+0000","last_became_active":"2024-03-22T04:12:33.621320+0000","last_became_peered":"2024-03-22T04:12:33.621320+0000","last_unstale":"2024-03-22T04:12:35.615885+0000","last_undegraded":"2024-03-22T04:12:35.615885+0000","last_fullsized":"2024-03-22T04:12:35.615885+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:00.740320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615027+0000","last_change":"2024-03-22T04:12:34.638992+0000","last_active":"2024-03-22T04:12:35.615027+0000","last_peered":"2024-03-22T04:12:35.615027+0000","last_clean":"2024-03-22T04:12:35.615027+0000","last_became_active":"2024-03-22T04:12:34.638706+0000","last_became_peered":"2024-03-22T04:12:34.638706+0000","last_unstale":"2024-03-22T04:12:35.615027+0000","last_undegraded":"2024-03-22T04:12:35.615027+0000","last_fullsized":"2024-03-22T04:12:35.615027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657480+0000","last_change":"2024-03-22T04:12:33.619503+0000","last_active":"2024-03-22T04:12:34.657480+0000","last_peered":"2024-03-22T04:12:34.657480+0000","last_clean":"2024-03-22T04:12:34.657480+0000","last_became_active":"2024-03-22T04:12:33.619218+0000","last_became_peered":"2024-03-22T04:12:33.619218+0000","last_unstale":"2024-03-22T04:12:34.657480+0000","last_undegraded":"2024-03-22T04:12:34.657480+0000","last_fullsized":"2024-03-22T04:12:34.657480+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:26.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637412+0000","last_change":"2024-03-22T04:12:34.638372+0000","last_active":"2024-03-22T04:12:35.637412+0000","last_peered":"2024-03-22T04:12:35.637412+0000","last_clean":"2024-03-22T04:12:35.637412+0000","last_became_active":"2024-03-22T04:12:34.636653+0000","last_became_peered":"2024-03-22T04:12:34.636653+0000","last_unstale":"2024-03-22T04:12:35.637412+0000","last_undegraded":"2024-03-22T04:12:35.637412+0000","last_fullsized":"2024-03-22T04:12:35.637412+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951462+0000","last_change":"2024-03-22T04:12:33.621816+0000","last_active":"2024-03-22T04:12:35.951462+0000","last_peered":"2024-03-22T04:12:35.951462+0000","last_clean":"2024-03-22T04:12:35.951462+0000","last_became_active":"2024-03-22T04:12:33.621560+0000","last_became_peered":"2024-03-22T04:12:33.621560+0000","last_unstale":"2024-03-22T04:12:35.951462+0000","last_undegraded":"2024-03-22T04:12:35.951462+0000","last_fullsized":"2024-03-22T04:12:35.951462+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:50:45.907035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639031+0000","last_change":"2024-03-22T04:12:34.633399+0000","last_active":"2024-03-22T04:12:34.639031+0000","last_peered":"2024-03-22T04:12:34.639031+0000","last_clean":"2024-03-22T04:12:34.639031+0000","last_became_active":"2024-03-22T04:12:34.633082+0000","last_became_peered":"2024-03-22T04:12:34.633082+0000","last_unstale":"2024-03-22T04:12:34.639031+0000","last_undegraded":"2024-03-22T04:12:34.639031+0000","last_fullsized":"2024-03-22T04:12:34.639031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:39:21.503071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.651676+0000","last_change":"2024-03-22T04:12:33.615641+0000","last_active":"2024-03-22T04:12:34.651676+0000","last_peered":"2024-03-22T04:12:34.651676+0000","last_clean":"2024-03-22T04:12:34.651676+0000","last_became_active":"2024-03-22T04:12:33.615372+0000","last_became_peered":"2024-03-22T04:12:33.615372+0000","last_unstale":"2024-03-22T04:12:34.651676+0000","last_undegraded":"2024-03-22T04:12:34.651676+0000","last_fullsized":"2024-03-22T04:12:34.651676+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:39.482917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615462+0000","last_change":"2024-03-22T04:12:34.637539+0000","last_active":"2024-03-22T04:12:35.615462+0000","last_peered":"2024-03-22T04:12:35.615462+0000","last_clean":"2024-03-22T04:12:35.615462+0000","last_became_active":"2024-03-22T04:12:34.636918+0000","last_became_peered":"2024-03-22T04:12:34.636918+0000","last_unstale":"2024-03-22T04:12:35.615462+0000","last_undegraded":"2024-03-22T04:12:35.615462+0000","last_fullsized":"2024-03-22T04:12:35.615462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.618429+0000","last_change":"2024-03-22T04:12:33.615588+0000","last_active":"2024-03-22T04:12:34.618429+0000","last_peered":"2024-03-22T04:12:34.618429+0000","last_clean":"2024-03-22T04:12:34.618429+0000","last_became_active":"2024-03-22T04:12:33.615259+0000","last_became_peered":"2024-03-22T04:12:33.615259+0000","last_unstale":"2024-03-22T04:12:34.618429+0000","last_undegraded":"2024-03-22T04:12:34.618429+0000","last_fullsized":"2024-03-22T04:12:34.618429+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:27:55.035247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612760+0000","last_change":"2024-03-22T04:12:34.635762+0000","last_active":"2024-03-22T04:12:35.612760+0000","last_peered":"2024-03-22T04:12:35.612760+0000","last_clean":"2024-03-22T04:12:35.612760+0000","last_became_active":"2024-03-22T04:12:34.635493+0000","last_became_peered":"2024-03-22T04:12:34.635493+0000","last_unstale":"2024-03-22T04:12:35.612760+0000","last_undegraded":"2024-03-22T04:12:35.612760+0000","last_fullsized":"2024-03-22T04:12:35.612760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.950944+0000","last_change":"2024-03-22T04:12:33.618851+0000","last_active":"2024-03-22T04:12:35.950944+0000","last_peered":"2024-03-22T04:12:35.950944+0000","last_clean":"2024-03-22T04:12:35.950944+0000","last_became_active":"2024-03-22T04:12:33.618525+0000","last_became_peered":"2024-03-22T04:12:33.618525+0000","last_unstale":"2024-03-22T04:12:35.950944+0000","last_undegraded":"2024-03-22T04:12:35.950944+0000","last_fullsized":"2024-03-22T04:12:35.950944+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:59:32.268351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637090+0000","last_change":"2024-03-22T04:12:34.626850+0000","last_active":"2024-03-22T04:12:35.637090+0000","last_peered":"2024-03-22T04:12:35.637090+0000","last_clean":"2024-03-22T04:12:35.637090+0000","last_became_active":"2024-03-22T04:12:34.626505+0000","last_became_peered":"2024-03-22T04:12:34.626505+0000","last_unstale":"2024-03-22T04:12:35.637090+0000","last_undegraded":"2024-03-22T04:12:35.637090+0000","last_fullsized":"2024-03-22T04:12:35.637090+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612750+0000","last_change":"2024-03-22T04:12:33.618147+0000","last_active":"2024-03-22T04:12:35.612750+0000","last_peered":"2024-03-22T04:12:35.612750+0000","last_clean":"2024-03-22T04:12:35.612750+0000","last_became_active":"2024-03-22T04:12:33.617867+0000","last_became_peered":"2024-03-22T04:12:33.617867+0000","last_unstale":"2024-03-22T04:12:35.612750+0000","last_undegraded":"2024-03-22T04:12:35.612750+0000","last_fullsized":"2024-03-22T04:12:35.612750+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612131+0000","last_change":"2024-03-22T04:12:34.624495+0000","last_active":"2024-03-22T04:12:35.612131+0000","last_peered":"2024-03-22T04:12:35.612131+0000","last_clean":"2024-03-22T04:12:35.612131+0000","last_became_active":"2024-03-22T04:12:34.624243+0000","last_became_peered":"2024-03-22T04:12:34.624243+0000","last_unstale":"2024-03-22T04:12:35.612131+0000","last_undegraded":"2024-03-22T04:12:35.612131+0000","last_fullsized":"2024-03-22T04:12:35.612131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612650+0000","last_change":"2024-03-22T04:12:33.617534+0000","last_active":"2024-03-22T04:12:35.612650+0000","last_peered":"2024-03-22T04:12:35.612650+0000","last_clean":"2024-03-22T04:12:35.612650+0000","last_became_active":"2024-03-22T04:12:33.617265+0000","last_became_peered":"2024-03-22T04:12:33.617265+0000","last_unstale":"2024-03-22T04:12:35.612650+0000","last_undegraded":"2024-03-22T04:12:35.612650+0000","last_fullsized":"2024-03-22T04:12:35.612650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615181+0000","last_change":"2024-03-22T04:12:34.635018+0000","last_active":"2024-03-22T04:12:35.615181+0000","last_peered":"2024-03-22T04:12:35.615181+0000","last_clean":"2024-03-22T04:12:35.615181+0000","last_became_active":"2024-03-22T04:12:34.634716+0000","last_became_peered":"2024-03-22T04:12:34.634716+0000","last_unstale":"2024-03-22T04:12:35.615181+0000","last_undegraded":"2024-03-22T04:12:35.615181+0000","last_fullsized":"2024-03-22T04:12:35.615181+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.658426+0000","last_change":"2024-03-22T04:12:33.623030+0000","last_active":"2024-03-22T04:12:34.658426+0000","last_peered":"2024-03-22T04:12:34.658426+0000","last_clean":"2024-03-22T04:12:34.658426+0000","last_became_active":"2024-03-22T04:12:33.622806+0000","last_became_peered":"2024-03-22T04:12:33.622806+0000","last_unstale":"2024-03-22T04:12:34.658426+0000","last_undegraded":"2024-03-22T04:12:34.658426+0000","last_fullsized":"2024-03-22T04:12:34.658426+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:50:40.568966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.641095+0000","last_change":"2024-03-22T04:12:34.635387+0000","last_active":"2024-03-22T04:12:34.641095+0000","last_peered":"2024-03-22T04:12:34.641095+0000","last_clean":"2024-03-22T04:12:34.641095+0000","last_became_active":"2024-03-22T04:12:34.632095+0000","last_became_peered":"2024-03-22T04:12:34.632095+0000","last_unstale":"2024-03-22T04:12:34.641095+0000","last_undegraded":"2024-03-22T04:12:34.641095+0000","last_fullsized":"2024-03-22T04:12:34.641095+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615307+0000","last_change":"2024-03-22T04:12:33.616617+0000","last_active":"2024-03-22T04:12:35.615307+0000","last_peered":"2024-03-22T04:12:35.615307+0000","last_clean":"2024-03-22T04:12:35.615307+0000","last_became_active":"2024-03-22T04:12:33.616088+0000","last_became_peered":"2024-03-22T04:12:33.616088+0000","last_unstale":"2024-03-22T04:12:35.615307+0000","last_undegraded":"2024-03-22T04:12:35.615307+0000","last_fullsized":"2024-03-22T04:12:35.615307+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:49.907740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.640577+0000","last_change":"2024-03-22T04:12:34.638920+0000","last_active":"2024-03-22T04:12:34.640577+0000","last_peered":"2024-03-22T04:12:34.640577+0000","last_clean":"2024-03-22T04:12:34.640577+0000","last_became_active":"2024-03-22T04:12:34.638167+0000","last_became_peered":"2024-03-22T04:12:34.638167+0000","last_unstale":"2024-03-22T04:12:34.640577+0000","last_undegraded":"2024-03-22T04:12:34.640577+0000","last_fullsized":"2024-03-22T04:12:34.640577+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:21:14.663985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.659863+0000","last_change":"2024-03-22T04:12:33.625471+0000","last_active":"2024-03-22T04:12:34.659863+0000","last_peered":"2024-03-22T04:12:34.659863+0000","last_clean":"2024-03-22T04:12:34.659863+0000","last_became_active":"2024-03-22T04:12:33.625189+0000","last_became_peered":"2024-03-22T04:12:33.625189+0000","last_unstale":"2024-03-22T04:12:34.659863+0000","last_undegraded":"2024-03-22T04:12:34.659863+0000","last_fullsized":"2024-03-22T04:12:34.659863+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615959+0000","last_change":"2024-03-22T04:12:34.642230+0000","last_active":"2024-03-22T04:12:35.615959+0000","last_peered":"2024-03-22T04:12:35.615959+0000","last_clean":"2024-03-22T04:12:35.615959+0000","last_became_active":"2024-03-22T04:12:34.642038+0000","last_became_peered":"2024-03-22T04:12:34.642038+0000","last_unstale":"2024-03-22T04:12:35.615959+0000","last_undegraded":"2024-03-22T04:12:35.615959+0000","last_fullsized":"2024-03-22T04:12:35.615959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612534+0000","last_change":"2024-03-22T04:12:33.620764+0000","last_active":"2024-03-22T04:12:35.612534+0000","last_peered":"2024-03-22T04:12:35.612534+0000","last_clean":"2024-03-22T04:12:35.612534+0000","last_became_active":"2024-03-22T04:12:33.620498+0000","last_became_peered":"2024-03-22T04:12:33.620498+0000","last_unstale":"2024-03-22T04:12:35.612534+0000","last_undegraded":"2024-03-22T04:12:35.612534+0000","last_fullsized":"2024-03-22T04:12:35.612534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.646765+0000","last_change":"2024-03-22T04:12:34.641493+0000","last_active":"2024-03-22T04:12:34.646765+0000","last_peered":"2024-03-22T04:12:34.646765+0000","last_clean":"2024-03-22T04:12:34.646765+0000","last_became_active":"2024-03-22T04:12:34.636610+0000","last_became_peered":"2024-03-22T04:12:34.636610+0000","last_unstale":"2024-03-22T04:12:34.646765+0000","last_undegraded":"2024-03-22T04:12:34.646765+0000","last_fullsized":"2024-03-22T04:12:34.646765+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:01.229933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.617067+0000","last_change":"2024-03-22T04:12:33.618015+0000","last_active":"2024-03-22T04:12:34.617067+0000","last_peered":"2024-03-22T04:12:34.617067+0000","last_clean":"2024-03-22T04:12:34.617067+0000","last_became_active":"2024-03-22T04:12:33.617794+0000","last_became_peered":"2024-03-22T04:12:33.617794+0000","last_unstale":"2024-03-22T04:12:34.617067+0000","last_undegraded":"2024-03-22T04:12:34.617067+0000","last_fullsized":"2024-03-22T04:12:34.617067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:00.222391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951435+0000","last_change":"2024-03-22T04:12:34.626431+0000","last_active":"2024-03-22T04:12:35.951435+0000","last_peered":"2024-03-22T04:12:35.951435+0000","last_clean":"2024-03-22T04:12:35.951435+0000","last_became_active":"2024-03-22T04:12:34.626084+0000","last_became_peered":"2024-03-22T04:12:34.626084+0000","last_unstale":"2024-03-22T04:12:35.951435+0000","last_undegraded":"2024-03-22T04:12:35.951435+0000","last_fullsized":"2024-03-22T04:12:35.951435+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:34:04.944294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.661009+0000","last_change":"2024-03-22T04:12:33.621891+0000","last_active":"2024-03-22T04:12:34.661009+0000","last_peered":"2024-03-22T04:12:34.661009+0000","last_clean":"2024-03-22T04:12:34.661009+0000","last_became_active":"2024-03-22T04:12:33.621626+0000","last_became_peered":"2024-03-22T04:12:33.621626+0000","last_unstale":"2024-03-22T04:12:34.661009+0000","last_undegraded":"2024-03-22T04:12:34.661009+0000","last_fullsized":"2024-03-22T04:12:34.661009+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614700+0000","last_change":"2024-03-22T04:12:34.634352+0000","last_active":"2024-03-22T04:12:35.614700+0000","last_peered":"2024-03-22T04:12:35.614700+0000","last_clean":"2024-03-22T04:12:35.614700+0000","last_became_active":"2024-03-22T04:12:34.633729+0000","last_became_peered":"2024-03-22T04:12:34.633729+0000","last_unstale":"2024-03-22T04:12:35.614700+0000","last_undegraded":"2024-03-22T04:12:35.614700+0000","last_fullsized":"2024-03-22T04:12:35.614700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.660467+0000","last_change":"2024-03-22T04:12:33.623382+0000","last_active":"2024-03-22T04:12:34.660467+0000","last_peered":"2024-03-22T04:12:34.660467+0000","last_clean":"2024-03-22T04:12:34.660467+0000","last_became_active":"2024-03-22T04:12:33.622982+0000","last_became_peered":"2024-03-22T04:12:33.622982+0000","last_unstale":"2024-03-22T04:12:34.660467+0000","last_undegraded":"2024-03-22T04:12:34.660467+0000","last_fullsized":"2024-03-22T04:12:34.660467+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614924+0000","last_change":"2024-03-22T04:12:34.627065+0000","last_active":"2024-03-22T04:12:35.614924+0000","last_peered":"2024-03-22T04:12:35.614924+0000","last_clean":"2024-03-22T04:12:35.614924+0000","last_became_active":"2024-03-22T04:12:34.626777+0000","last_became_peered":"2024-03-22T04:12:34.626777+0000","last_unstale":"2024-03-22T04:12:35.614924+0000","last_undegraded":"2024-03-22T04:12:35.614924+0000","last_fullsized":"2024-03-22T04:12:35.614924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.658788+0000","last_change":"2024-03-22T04:12:33.622321+0000","last_active":"2024-03-22T04:12:34.658788+0000","last_peered":"2024-03-22T04:12:34.658788+0000","last_clean":"2024-03-22T04:12:34.658788+0000","last_became_active":"2024-03-22T04:12:33.622058+0000","last_became_peered":"2024-03-22T04:12:33.622058+0000","last_unstale":"2024-03-22T04:12:34.658788+0000","last_undegraded":"2024-03-22T04:12:34.658788+0000","last_fullsized":"2024-03-22T04:12:34.658788+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639497+0000","last_change":"2024-03-22T04:12:34.636289+0000","last_active":"2024-03-22T04:12:34.639497+0000","last_peered":"2024-03-22T04:12:34.639497+0000","last_clean":"2024-03-22T04:12:34.639497+0000","last_became_active":"2024-03-22T04:12:34.636025+0000","last_became_peered":"2024-03-22T04:12:34.636025+0000","last_unstale":"2024-03-22T04:12:34.639497+0000","last_undegraded":"2024-03-22T04:12:34.639497+0000","last_fullsized":"2024-03-22T04:12:34.639497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612548+0000","last_change":"2024-03-22T04:12:33.611878+0000","last_active":"2024-03-22T04:12:35.612548+0000","last_peered":"2024-03-22T04:12:35.612548+0000","last_clean":"2024-03-22T04:12:35.612548+0000","last_became_active":"2024-03-22T04:12:33.611537+0000","last_became_peered":"2024-03-22T04:12:33.611537+0000","last_unstale":"2024-03-22T04:12:35.612548+0000","last_undegraded":"2024-03-22T04:12:35.612548+0000","last_fullsized":"2024-03-22T04:12:35.612548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.653492+0000","last_change":"2024-03-22T04:12:34.650207+0000","last_active":"2024-03-22T04:12:34.653492+0000","last_peered":"2024-03-22T04:12:34.653492+0000","last_clean":"2024-03-22T04:12:34.653492+0000","last_became_active":"2024-03-22T04:12:34.649927+0000","last_became_peered":"2024-03-22T04:12:34.649927+0000","last_unstale":"2024-03-22T04:12:34.653492+0000","last_undegraded":"2024-03-22T04:12:34.653492+0000","last_fullsized":"2024-03-22T04:12:34.653492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:12.985866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.616557+0000","last_change":"2024-03-22T04:12:33.622557+0000","last_active":"2024-03-22T04:12:34.616557+0000","last_peered":"2024-03-22T04:12:34.616557+0000","last_clean":"2024-03-22T04:12:34.616557+0000","last_became_active":"2024-03-22T04:12:33.622240+0000","last_became_peered":"2024-03-22T04:12:33.622240+0000","last_unstale":"2024-03-22T04:12:34.616557+0000","last_undegraded":"2024-03-22T04:12:34.616557+0000","last_fullsized":"2024-03-22T04:12:34.616557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:32.007591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.634871+0000","last_change":"2024-03-22T04:12:34.633656+0000","last_active":"2024-03-22T04:12:34.634871+0000","last_peered":"2024-03-22T04:12:34.634871+0000","last_clean":"2024-03-22T04:12:34.634871+0000","last_became_active":"2024-03-22T04:12:34.633150+0000","last_became_peered":"2024-03-22T04:12:34.633150+0000","last_unstale":"2024-03-22T04:12:34.634871+0000","last_undegraded":"2024-03-22T04:12:34.634871+0000","last_fullsized":"2024-03-22T04:12:34.634871+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:33.344021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615428+0000","last_change":"2024-03-22T04:12:33.624581+0000","last_active":"2024-03-22T04:12:35.615428+0000","last_peered":"2024-03-22T04:12:35.615428+0000","last_clean":"2024-03-22T04:12:35.615428+0000","last_became_active":"2024-03-22T04:12:33.624324+0000","last_became_peered":"2024-03-22T04:12:33.624324+0000","last_unstale":"2024-03-22T04:12:35.615428+0000","last_undegraded":"2024-03-22T04:12:35.615428+0000","last_fullsized":"2024-03-22T04:12:35.615428+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:41.428263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612405+0000","last_change":"2024-03-22T04:12:34.631558+0000","last_active":"2024-03-22T04:12:35.612405+0000","last_peered":"2024-03-22T04:12:35.612405+0000","last_clean":"2024-03-22T04:12:35.612405+0000","last_became_active":"2024-03-22T04:12:34.621598+0000","last_became_peered":"2024-03-22T04:12:34.621598+0000","last_unstale":"2024-03-22T04:12:35.612405+0000","last_undegraded":"2024-03-22T04:12:35.612405+0000","last_fullsized":"2024-03-22T04:12:35.612405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:43:33.625589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951395+0000","last_change":"2024-03-22T04:12:33.619281+0000","last_active":"2024-03-22T04:12:35.951395+0000","last_peered":"2024-03-22T04:12:35.951395+0000","last_clean":"2024-03-22T04:12:35.951395+0000","last_became_active":"2024-03-22T04:12:33.618935+0000","last_became_peered":"2024-03-22T04:12:33.618935+0000","last_unstale":"2024-03-22T04:12:35.951395+0000","last_undegraded":"2024-03-22T04:12:35.951395+0000","last_fullsized":"2024-03-22T04:12:35.951395+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:48:41.700689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615777+0000","last_change":"2024-03-22T04:12:34.642768+0000","last_active":"2024-03-22T04:12:35.615777+0000","last_peered":"2024-03-22T04:12:35.615777+0000","last_clean":"2024-03-22T04:12:35.615777+0000","last_became_active":"2024-03-22T04:12:34.642534+0000","last_became_peered":"2024-03-22T04:12:34.642534+0000","last_unstale":"2024-03-22T04:12:35.615777+0000","last_undegraded":"2024-03-22T04:12:35.615777+0000","last_fullsized":"2024-03-22T04:12:35.615777+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615251+0000","last_change":"2024-03-22T04:12:33.626234+0000","last_active":"2024-03-22T04:12:35.615251+0000","last_peered":"2024-03-22T04:12:35.615251+0000","last_clean":"2024-03-22T04:12:35.615251+0000","last_became_active":"2024-03-22T04:12:33.623978+0000","last_became_peered":"2024-03-22T04:12:33.623978+0000","last_unstale":"2024-03-22T04:12:35.615251+0000","last_undegraded":"2024-03-22T04:12:35.615251+0000","last_fullsized":"2024-03-22T04:12:35.615251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:45.736028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.648376+0000","last_change":"2024-03-22T04:12:34.646431+0000","last_active":"2024-03-22T04:12:34.648376+0000","last_peered":"2024-03-22T04:12:34.648376+0000","last_clean":"2024-03-22T04:12:34.648376+0000","last_became_active":"2024-03-22T04:12:34.646176+0000","last_became_peered":"2024-03-22T04:12:34.646176+0000","last_unstale":"2024-03-22T04:12:34.648376+0000","last_undegraded":"2024-03-22T04:12:34.648376+0000","last_fullsized":"2024-03-22T04:12:34.648376+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:42.435805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614477+0000","last_change":"2024-03-22T04:12:33.610526+0000","last_active":"2024-03-22T04:12:35.614477+0000","last_peered":"2024-03-22T04:12:35.614477+0000","last_clean":"2024-03-22T04:12:35.614477+0000","last_became_active":"2024-03-22T04:12:33.609782+0000","last_became_peered":"2024-03-22T04:12:33.609782+0000","last_unstale":"2024-03-22T04:12:35.614477+0000","last_undegraded":"2024-03-22T04:12:35.614477+0000","last_fullsized":"2024-03-22T04:12:35.614477+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:11.978324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.611982+0000","last_change":"2024-03-22T04:12:34.622644+0000","last_active":"2024-03-22T04:12:35.611982+0000","last_peered":"2024-03-22T04:12:35.611982+0000","last_clean":"2024-03-22T04:12:35.611982+0000","last_became_active":"2024-03-22T04:12:34.622112+0000","last_became_peered":"2024-03-22T04:12:34.622112+0000","last_unstale":"2024-03-22T04:12:35.611982+0000","last_undegraded":"2024-03-22T04:12:35.611982+0000","last_fullsized":"2024-03-22T04:12:35.611982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.657519+0000","last_change":"2024-03-22T04:12:33.621421+0000","last_active":"2024-03-22T04:12:34.657519+0000","last_peered":"2024-03-22T04:12:34.657519+0000","last_clean":"2024-03-22T04:12:34.657519+0000","last_became_active":"2024-03-22T04:12:33.621140+0000","last_became_peered":"2024-03-22T04:12:33.621140+0000","last_unstale":"2024-03-22T04:12:34.657519+0000","last_undegraded":"2024-03-22T04:12:34.657519+0000","last_fullsized":"2024-03-22T04:12:34.657519+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:21.786333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615749+0000","last_change":"2024-03-22T04:12:34.638291+0000","last_active":"2024-03-22T04:12:35.615749+0000","last_peered":"2024-03-22T04:12:35.615749+0000","last_clean":"2024-03-22T04:12:35.615749+0000","last_became_active":"2024-03-22T04:12:34.637953+0000","last_became_peered":"2024-03-22T04:12:34.637953+0000","last_unstale":"2024-03-22T04:12:35.615749+0000","last_undegraded":"2024-03-22T04:12:35.615749+0000","last_fullsized":"2024-03-22T04:12:35.615749+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:37:44.536324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.618566+0000","last_change":"2024-03-22T04:12:33.623102+0000","last_active":"2024-03-22T04:12:34.618566+0000","last_peered":"2024-03-22T04:12:34.618566+0000","last_clean":"2024-03-22T04:12:34.618566+0000","last_became_active":"2024-03-22T04:12:33.622803+0000","last_became_peered":"2024-03-22T04:12:33.622803+0000","last_unstale":"2024-03-22T04:12:34.618566+0000","last_undegraded":"2024-03-22T04:12:34.618566+0000","last_fullsized":"2024-03-22T04:12:34.618566+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:04:41.174822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951365+0000","last_change":"2024-03-22T04:12:34.623839+0000","last_active":"2024-03-22T04:12:35.951365+0000","last_peered":"2024-03-22T04:12:35.951365+0000","last_clean":"2024-03-22T04:12:35.951365+0000","last_became_active":"2024-03-22T04:12:34.623357+0000","last_became_peered":"2024-03-22T04:12:34.623357+0000","last_unstale":"2024-03-22T04:12:35.951365+0000","last_undegraded":"2024-03-22T04:12:35.951365+0000","last_fullsized":"2024-03-22T04:12:35.951365+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:27:10.460890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951051+0000","last_change":"2024-03-22T04:12:33.618302+0000","last_active":"2024-03-22T04:12:35.951051+0000","last_peered":"2024-03-22T04:12:35.951051+0000","last_clean":"2024-03-22T04:12:35.951051+0000","last_became_active":"2024-03-22T04:12:33.617971+0000","last_became_peered":"2024-03-22T04:12:33.617971+0000","last_unstale":"2024-03-22T04:12:35.951051+0000","last_undegraded":"2024-03-22T04:12:35.951051+0000","last_fullsized":"2024-03-22T04:12:35.951051+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:32:05.092737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615191+0000","last_change":"2024-03-22T04:12:34.644894+0000","last_active":"2024-03-22T04:12:35.615191+0000","last_peered":"2024-03-22T04:12:35.615191+0000","last_clean":"2024-03-22T04:12:35.615191+0000","last_became_active":"2024-03-22T04:12:34.644526+0000","last_became_peered":"2024-03-22T04:12:34.644526+0000","last_unstale":"2024-03-22T04:12:35.615191+0000","last_undegraded":"2024-03-22T04:12:35.615191+0000","last_fullsized":"2024-03-22T04:12:35.615191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615131+0000","last_change":"2024-03-22T04:12:33.620091+0000","last_active":"2024-03-22T04:12:35.615131+0000","last_peered":"2024-03-22T04:12:35.615131+0000","last_clean":"2024-03-22T04:12:35.615131+0000","last_became_active":"2024-03-22T04:12:33.619790+0000","last_became_peered":"2024-03-22T04:12:33.619790+0000","last_unstale":"2024-03-22T04:12:35.615131+0000","last_undegraded":"2024-03-22T04:12:35.615131+0000","last_fullsized":"2024-03-22T04:12:35.615131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:32.336479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.639091+0000","last_change":"2024-03-22T04:12:34.636152+0000","last_active":"2024-03-22T04:12:34.639091+0000","last_peered":"2024-03-22T04:12:34.639091+0000","last_clean":"2024-03-22T04:12:34.639091+0000","last_became_active":"2024-03-22T04:12:34.635865+0000","last_became_peered":"2024-03-22T04:12:34.635865+0000","last_unstale":"2024-03-22T04:12:34.639091+0000","last_undegraded":"2024-03-22T04:12:34.639091+0000","last_fullsized":"2024-03-22T04:12:34.639091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951408+0000","last_change":"2024-03-22T04:12:33.612435+0000","last_active":"2024-03-22T04:12:35.951408+0000","last_peered":"2024-03-22T04:12:35.951408+0000","last_clean":"2024-03-22T04:12:35.951408+0000","last_became_active":"2024-03-22T04:12:33.611642+0000","last_became_peered":"2024-03-22T04:12:33.611642+0000","last_unstale":"2024-03-22T04:12:35.951408+0000","last_undegraded":"2024-03-22T04:12:35.951408+0000","last_fullsized":"2024-03-22T04:12:35.951408+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:18:53.878663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615970+0000","last_change":"2024-03-22T04:12:34.637750+0000","last_active":"2024-03-22T04:12:35.615970+0000","last_peered":"2024-03-22T04:12:35.615970+0000","last_clean":"2024-03-22T04:12:35.615970+0000","last_became_active":"2024-03-22T04:12:34.637269+0000","last_became_peered":"2024-03-22T04:12:34.637269+0000","last_unstale":"2024-03-22T04:12:35.615970+0000","last_undegraded":"2024-03-22T04:12:35.615970+0000","last_fullsized":"2024-03-22T04:12:35.615970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:54:58.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.617221+0000","last_change":"2024-03-22T04:12:33.613711+0000","last_active":"2024-03-22T04:12:34.617221+0000","last_peered":"2024-03-22T04:12:34.617221+0000","last_clean":"2024-03-22T04:12:34.617221+0000","last_became_active":"2024-03-22T04:12:33.613398+0000","last_became_peered":"2024-03-22T04:12:33.613398+0000","last_unstale":"2024-03-22T04:12:34.617221+0000","last_undegraded":"2024-03-22T04:12:34.617221+0000","last_fullsized":"2024-03-22T04:12:34.617221+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:34.304317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-22T04:12:34.654950+0000","last_change":"2024-03-22T04:12:34.649939+0000","last_active":"2024-03-22T04:12:34.654950+0000","last_peered":"2024-03-22T04:12:34.654950+0000","last_clean":"2024-03-22T04:12:34.654950+0000","last_became_active":"2024-03-22T04:12:34.649639+0000","last_became_peered":"2024-03-22T04:12:34.649639+0000","last_unstale":"2024-03-22T04:12:34.654950+0000","last_undegraded":"2024-03-22T04:12:34.654950+0000","last_fullsized":"2024-03-22T04:12:34.654950+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:01.747862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612050+0000","last_change":"2024-03-22T04:12:33.613244+0000","last_active":"2024-03-22T04:12:35.612050+0000","last_peered":"2024-03-22T04:12:35.612050+0000","last_clean":"2024-03-22T04:12:35.612050+0000","last_became_active":"2024-03-22T04:12:33.612972+0000","last_became_peered":"2024-03-22T04:12:33.612972+0000","last_unstale":"2024-03-22T04:12:35.612050+0000","last_undegraded":"2024-03-22T04:12:35.612050+0000","last_fullsized":"2024-03-22T04:12:35.612050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637167+0000","last_change":"2024-03-22T04:12:34.638813+0000","last_active":"2024-03-22T04:12:35.637167+0000","last_peered":"2024-03-22T04:12:35.637167+0000","last_clean":"2024-03-22T04:12:35.637167+0000","last_became_active":"2024-03-22T04:12:34.638487+0000","last_became_peered":"2024-03-22T04:12:34.638487+0000","last_unstale":"2024-03-22T04:12:35.637167+0000","last_undegraded":"2024-03-22T04:12:35.637167+0000","last_fullsized":"2024-03-22T04:12:35.637167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637130+0000","last_change":"2024-03-22T04:12:33.624669+0000","last_active":"2024-03-22T04:12:35.637130+0000","last_peered":"2024-03-22T04:12:35.637130+0000","last_clean":"2024-03-22T04:12:35.637130+0000","last_became_active":"2024-03-22T04:12:33.624416+0000","last_became_peered":"2024-03-22T04:12:33.624416+0000","last_unstale":"2024-03-22T04:12:35.637130+0000","last_undegraded":"2024-03-22T04:12:35.637130+0000","last_fullsized":"2024-03-22T04:12:35.637130+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:18.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":227,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612039+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:12:35.612039+0000","last_peered":"2024-03-22T04:12:35.612039+0000","last_clean":"2024-03-22T04:12:35.612039+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:12:35.612039+0000","last_undegraded":"2024-03-22T04:12:35.612039+0000","last_fullsized":"2024-03-22T04:12:35.612039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":273831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":274357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":865845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":273865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":273809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":870905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":274357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":282171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:12:43.203 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:12:43.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.345+0000 7f8b26000640 1 Processor -- start 2024-03-22T04:12:43.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.345+0000 7f8b26000640 1 -- start start 2024-03-22T04:12:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.345+0000 7f8b26000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8b2015ac10 0x7f8b200b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.345+0000 7f8b26000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b2015bc80 0x7f8b2015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.345+0000 7f8b26000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2015c550 con 0x7f8b2006df90 2024-03-22T04:12:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b26000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8b2015cb60 con 0x7f8b2015bc80 2024-03-22T04:12:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b26000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8b200b4750 con 0x7f8b2015ac10 2024-03-22T04:12:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b2015bc80 0x7f8b2015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b2015bc80 0x7f8b2015cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51490/0 (socket says 172.21.15.86:51490) 2024-03-22T04:12:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8b2006df90 0x7f8b2006e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57236/0 (socket says 172.21.15.86:57236) 2024-03-22T04:12:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1fe00640 1 -- 172.21.15.86:0/1444654925 learned_addr learned my addr 172.21.15.86:0/1444654925 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1ea00640 1 --2- 172.21.15.86:0/1444654925 >> v2:172.21.15.196:3300/0 conn(0x7f8b2015ac10 0x7f8b200b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1fe00640 1 -- 172.21.15.86:0/1444654925 >> v2:172.21.15.196:3300/0 conn(0x7f8b2015ac10 msgr2=0x7f8b200b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1fe00640 1 --2- 172.21.15.86:0/1444654925 >> v2:172.21.15.196:3300/0 conn(0x7f8b2015ac10 0x7f8b200b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1fe00640 1 -- 172.21.15.86:0/1444654925 >> v1:172.21.15.86:6789/0 conn(0x7f8b2006df90 legacy=0x7f8b2006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1e000640 1 -- 172.21.15.86:0/1444654925 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 176558208 0 0) 0x7f8b2015c550 con 0x7f8b2006df90 2024-03-22T04:12:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1fe00640 1 -- 172.21.15.86:0/1444654925 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b200b4470 con 0x7f8b2015bc80 2024-03-22T04:12:43.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1fe00640 1 --2- 172.21.15.86:0/1444654925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2015bc80 0x7f8b2015cff0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8b2006fa40 tx=0x7f8b1801dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f5138ad5da2e98 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:43.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1e000640 1 -- 172.21.15.86:0/1444654925 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800a2d0 con 0x7f8b2015bc80 2024-03-22T04:12:43.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1e000640 1 -- 172.21.15.86:0/1444654925 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b180a68c0 con 0x7f8b2015bc80 2024-03-22T04:12:43.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.349+0000 7f8b1e000640 1 -- 172.21.15.86:0/1444654925 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b180a6c90 con 0x7f8b2015bc80 2024-03-22T04:12:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- 172.21.15.86:0/1444654925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2015bc80 msgr2=0x7f8b2015cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 --2- 172.21.15.86:0/1444654925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2015bc80 0x7f8b2015cff0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8b2006fa40 tx=0x7f8b1801dac0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- 172.21.15.86:0/1444654925 shutdown_connections 2024-03-22T04:12:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 --2- 172.21.15.86:0/1444654925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2015bc80 0x7f8b2015cff0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 --2- 172.21.15.86:0/1444654925 >> v2:172.21.15.196:3300/0 conn(0x7f8b2015ac10 0x7f8b200b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- 172.21.15.86:0/1444654925 >> 172.21.15.86:0/1444654925 conn(0x7f8b200b1df0 msgr2=0x7f8b200b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- 172.21.15.86:0/1444654925 shutdown_connections 2024-03-22T04:12:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- 172.21.15.86:0/1444654925 wait complete. 2024-03-22T04:12:43.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 Processor -- start 2024-03-22T04:12:43.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- start start 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2006df90 0x7f8b2014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b2015ac10 0x7f8b200ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2015bc80 0x7f8b200cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2015d8c0 con 0x7f8b2006df90 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2015d740 con 0x7f8b2015bc80 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b26000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2015da40 con 0x7f8b2015ac10 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2006df90 0x7f8b2014bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2006df90 0x7f8b2014bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51494/0 (socket says 172.21.15.86:51494) 2024-03-22T04:12:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1f400640 1 -- 172.21.15.86:0/1666510257 learned_addr learned my addr 172.21.15.86:0/1666510257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1fe00640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2015bc80 0x7f8b200cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1ea00640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b2015ac10 0x7f8b200ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1f400640 1 -- 172.21.15.86:0/1666510257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b2015ac10 msgr2=0x7f8b200ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1f400640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b2015ac10 0x7f8b200ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1f400640 1 -- 172.21.15.86:0/1666510257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2015bc80 msgr2=0x7f8b200cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1f400640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2015bc80 0x7f8b200cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.353+0000 7f8b1f400640 1 -- 172.21.15.86:0/1666510257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b200cf6f0 con 0x7f8b2006df90 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b1f400640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2006df90 0x7f8b2014bdd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c520 tx=0x7f8b0800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b17400640 1 -- 172.21.15.86:0/1666510257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08018070 con 0x7f8b2006df90 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b17400640 1 -- 172.21.15.86:0/1666510257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b080074e0 con 0x7f8b2006df90 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b200cd610 con 0x7f8b2006df90 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b17400640 1 -- 172.21.15.86:0/1666510257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b080049f0 con 0x7f8b2006df90 2024-03-22T04:12:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b200cdbd0 con 0x7f8b2006df90 2024-03-22T04:12:43.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b20068650 con 0x7f8b2006df90 2024-03-22T04:12:43.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b17400640 1 -- 172.21.15.86:0/1666510257 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8b0801b070 con 0x7f8b2006df90 2024-03-22T04:12:43.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b17400640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8afc067b20 0x7f8afc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b17400640 1 -- 172.21.15.86:0/1666510257 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8b08085500 con 0x7f8b2006df90 2024-03-22T04:12:43.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.357+0000 7f8b1ea00640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8afc067b20 0x7f8afc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.361+0000 7f8b1ea00640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8afc067b20 0x7f8afc069fe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8b2015ed90 tx=0x7f8b10009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:43.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.361+0000 7f8b17400640 1 -- 172.21.15.86:0/1666510257 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8b08055690 con 0x7f8b2006df90 2024-03-22T04:12:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.513+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b2006ed60 con 0x7f8afc067b20 2024-03-22T04:12:43.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.521+0000 7f8b17400640 1 -- 172.21.15.86:0/1666510257 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251895 (secure 0 0 0) 0x7f8b2006ed60 con 0x7f8afc067b20 2024-03-22T04:12:43.525 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:43.527 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.521+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8afc067b20 msgr2=0x7f8afc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.521+0000 7f8b26000640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8afc067b20 0x7f8afc069fe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8b2015ed90 tx=0x7f8b10009190 comp rx=0 tx=0).stop 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2006df90 msgr2=0x7f8b2014bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2006df90 0x7f8b2014bdd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c520 tx=0x7f8b0800c9e0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 shutdown_connections 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8afc067b20 0x7f8afc069fe0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2015bc80 0x7f8b200cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b2015ac10 0x7f8b200ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 --2- 172.21.15.86:0/1666510257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2006df90 0x7f8b2014bdd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 >> 172.21.15.86:0/1666510257 conn(0x7f8b200b1df0 msgr2=0x7f8b2006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 shutdown_connections 2024-03-22T04:12:43.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.525+0000 7f8b26000640 1 -- 172.21.15.86:0/1666510257 wait complete. 2024-03-22T04:12:43.546 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-03-22T04:12:41.816954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222380,"kb_used_data":6184,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752340,"statfs":{"total":773094113280,"available":772866396160,"internally_reserved":0,"allocated":6332416,"data_stored":3429292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001328"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616060+0000","last_change":"2024-03-22T04:12:34.628936+0000","last_active":"2024-03-22T04:12:35.616060+0000","last_peered":"2024-03-22T04:12:35.616060+0000","last_clean":"2024-03-22T04:12:35.616060+0000","last_became_active":"2024-03-22T04:12:34.628641+0000","last_became_peered":"2024-03-22T04:12:34.628641+0000","last_unstale":"2024-03-22T04:12:35.616060+0000","last_undegraded":"2024-03-22T04:12:35.616060+0000","last_fullsized":"2024-03-22T04:12:35.616060+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515895+0000","last_change":"2024-03-22T04:12:34.637267+0000","last_active":"2024-03-22T04:12:36.515895+0000","last_peered":"2024-03-22T04:12:36.515895+0000","last_clean":"2024-03-22T04:12:36.515895+0000","last_became_active":"2024-03-22T04:12:34.637024+0000","last_became_peered":"2024-03-22T04:12:34.637024+0000","last_unstale":"2024-03-22T04:12:36.515895+0000","last_undegraded":"2024-03-22T04:12:36.515895+0000","last_fullsized":"2024-03-22T04:12:36.515895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951210+0000","last_change":"2024-03-22T04:12:34.624650+0000","last_active":"2024-03-22T04:12:35.951210+0000","last_peered":"2024-03-22T04:12:35.951210+0000","last_clean":"2024-03-22T04:12:35.951210+0000","last_became_active":"2024-03-22T04:12:34.624347+0000","last_became_peered":"2024-03-22T04:12:34.624347+0000","last_unstale":"2024-03-22T04:12:35.951210+0000","last_undegraded":"2024-03-22T04:12:35.951210+0000","last_fullsized":"2024-03-22T04:12:35.951210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:16:52.531907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612444+0000","last_change":"2024-03-22T04:12:34.625832+0000","last_active":"2024-03-22T04:12:35.612444+0000","last_peered":"2024-03-22T04:12:35.612444+0000","last_clean":"2024-03-22T04:12:35.612444+0000","last_became_active":"2024-03-22T04:12:34.625533+0000","last_became_peered":"2024-03-22T04:12:34.625533+0000","last_unstale":"2024-03-22T04:12:35.612444+0000","last_undegraded":"2024-03-22T04:12:35.612444+0000","last_fullsized":"2024-03-22T04:12:35.612444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:00.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516090+0000","last_change":"2024-03-22T04:12:34.627773+0000","last_active":"2024-03-22T04:12:36.516090+0000","last_peered":"2024-03-22T04:12:36.516090+0000","last_clean":"2024-03-22T04:12:36.516090+0000","last_became_active":"2024-03-22T04:12:34.624749+0000","last_became_peered":"2024-03-22T04:12:34.624749+0000","last_unstale":"2024-03-22T04:12:36.516090+0000","last_undegraded":"2024-03-22T04:12:36.516090+0000","last_fullsized":"2024-03-22T04:12:36.516090+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.953982+0000","last_change":"2024-03-22T04:12:34.635777+0000","last_active":"2024-03-22T04:12:35.953982+0000","last_peered":"2024-03-22T04:12:35.953982+0000","last_clean":"2024-03-22T04:12:35.953982+0000","last_became_active":"2024-03-22T04:12:34.635528+0000","last_became_peered":"2024-03-22T04:12:34.635528+0000","last_unstale":"2024-03-22T04:12:35.953982+0000","last_undegraded":"2024-03-22T04:12:35.953982+0000","last_fullsized":"2024-03-22T04:12:35.953982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614543+0000","last_change":"2024-03-22T04:12:34.636516+0000","last_active":"2024-03-22T04:12:35.614543+0000","last_peered":"2024-03-22T04:12:35.614543+0000","last_clean":"2024-03-22T04:12:35.614543+0000","last_became_active":"2024-03-22T04:12:34.636312+0000","last_became_peered":"2024-03-22T04:12:34.636312+0000","last_unstale":"2024-03-22T04:12:35.614543+0000","last_undegraded":"2024-03-22T04:12:35.614543+0000","last_fullsized":"2024-03-22T04:12:35.614543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.953809+0000","last_change":"2024-03-22T04:12:34.650010+0000","last_active":"2024-03-22T04:12:35.953809+0000","last_peered":"2024-03-22T04:12:35.953809+0000","last_clean":"2024-03-22T04:12:35.953809+0000","last_became_active":"2024-03-22T04:12:34.649709+0000","last_became_peered":"2024-03-22T04:12:34.649709+0000","last_unstale":"2024-03-22T04:12:35.953809+0000","last_undegraded":"2024-03-22T04:12:35.953809+0000","last_fullsized":"2024-03-22T04:12:35.953809+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951328+0000","last_change":"2024-03-22T04:12:34.625212+0000","last_active":"2024-03-22T04:12:35.951328+0000","last_peered":"2024-03-22T04:12:35.951328+0000","last_clean":"2024-03-22T04:12:35.951328+0000","last_became_active":"2024-03-22T04:12:34.624758+0000","last_became_peered":"2024-03-22T04:12:34.624758+0000","last_unstale":"2024-03-22T04:12:35.951328+0000","last_undegraded":"2024-03-22T04:12:35.951328+0000","last_fullsized":"2024-03-22T04:12:35.951328+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:23:14.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615323+0000","last_change":"2024-03-22T04:12:34.641332+0000","last_active":"2024-03-22T04:12:35.615323+0000","last_peered":"2024-03-22T04:12:35.615323+0000","last_clean":"2024-03-22T04:12:35.615323+0000","last_became_active":"2024-03-22T04:12:34.641059+0000","last_became_peered":"2024-03-22T04:12:34.641059+0000","last_unstale":"2024-03-22T04:12:35.615323+0000","last_undegraded":"2024-03-22T04:12:35.615323+0000","last_fullsized":"2024-03-22T04:12:35.615323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615824+0000","last_change":"2024-03-22T04:12:34.627929+0000","last_active":"2024-03-22T04:12:35.615824+0000","last_peered":"2024-03-22T04:12:35.615824+0000","last_clean":"2024-03-22T04:12:35.615824+0000","last_became_active":"2024-03-22T04:12:34.627640+0000","last_became_peered":"2024-03-22T04:12:34.627640+0000","last_unstale":"2024-03-22T04:12:35.615824+0000","last_undegraded":"2024-03-22T04:12:35.615824+0000","last_fullsized":"2024-03-22T04:12:35.615824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954061+0000","last_change":"2024-03-22T04:12:34.639709+0000","last_active":"2024-03-22T04:12:35.954061+0000","last_peered":"2024-03-22T04:12:35.954061+0000","last_clean":"2024-03-22T04:12:35.954061+0000","last_became_active":"2024-03-22T04:12:34.638936+0000","last_became_peered":"2024-03-22T04:12:34.638936+0000","last_unstale":"2024-03-22T04:12:35.954061+0000","last_undegraded":"2024-03-22T04:12:35.954061+0000","last_fullsized":"2024-03-22T04:12:35.954061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954050+0000","last_change":"2024-03-22T04:12:34.644837+0000","last_active":"2024-03-22T04:12:35.954050+0000","last_peered":"2024-03-22T04:12:35.954050+0000","last_clean":"2024-03-22T04:12:35.954050+0000","last_became_active":"2024-03-22T04:12:34.644507+0000","last_became_peered":"2024-03-22T04:12:34.644507+0000","last_unstale":"2024-03-22T04:12:35.954050+0000","last_undegraded":"2024-03-22T04:12:35.954050+0000","last_fullsized":"2024-03-22T04:12:35.954050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:00.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614973+0000","last_change":"2024-03-22T04:12:34.641248+0000","last_active":"2024-03-22T04:12:35.614973+0000","last_peered":"2024-03-22T04:12:35.614973+0000","last_clean":"2024-03-22T04:12:35.614973+0000","last_became_active":"2024-03-22T04:12:34.640953+0000","last_became_peered":"2024-03-22T04:12:34.640953+0000","last_unstale":"2024-03-22T04:12:35.614973+0000","last_undegraded":"2024-03-22T04:12:35.614973+0000","last_fullsized":"2024-03-22T04:12:35.614973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612098+0000","last_change":"2024-03-22T04:12:34.632175+0000","last_active":"2024-03-22T04:12:35.612098+0000","last_peered":"2024-03-22T04:12:35.612098+0000","last_clean":"2024-03-22T04:12:35.612098+0000","last_became_active":"2024-03-22T04:12:34.631916+0000","last_became_peered":"2024-03-22T04:12:34.631916+0000","last_unstale":"2024-03-22T04:12:35.612098+0000","last_undegraded":"2024-03-22T04:12:35.612098+0000","last_fullsized":"2024-03-22T04:12:35.612098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.953811+0000","last_change":"2024-03-22T04:12:34.639122+0000","last_active":"2024-03-22T04:12:35.953811+0000","last_peered":"2024-03-22T04:12:35.953811+0000","last_clean":"2024-03-22T04:12:35.953811+0000","last_became_active":"2024-03-22T04:12:34.638788+0000","last_became_peered":"2024-03-22T04:12:34.638788+0000","last_unstale":"2024-03-22T04:12:35.953811+0000","last_undegraded":"2024-03-22T04:12:35.953811+0000","last_fullsized":"2024-03-22T04:12:35.953811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:46.743571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616261+0000","last_change":"2024-03-22T04:12:34.632974+0000","last_active":"2024-03-22T04:12:35.616261+0000","last_peered":"2024-03-22T04:12:35.616261+0000","last_clean":"2024-03-22T04:12:35.616261+0000","last_became_active":"2024-03-22T04:12:34.627665+0000","last_became_peered":"2024-03-22T04:12:34.627665+0000","last_unstale":"2024-03-22T04:12:35.616261+0000","last_undegraded":"2024-03-22T04:12:35.616261+0000","last_fullsized":"2024-03-22T04:12:35.616261+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612642+0000","last_change":"2024-03-22T04:12:34.626077+0000","last_active":"2024-03-22T04:12:35.612642+0000","last_peered":"2024-03-22T04:12:35.612642+0000","last_clean":"2024-03-22T04:12:35.612642+0000","last_became_active":"2024-03-22T04:12:34.625837+0000","last_became_peered":"2024-03-22T04:12:34.625837+0000","last_unstale":"2024-03-22T04:12:35.612642+0000","last_undegraded":"2024-03-22T04:12:35.612642+0000","last_fullsized":"2024-03-22T04:12:35.612642+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951555+0000","last_change":"2024-03-22T04:12:34.622480+0000","last_active":"2024-03-22T04:12:35.951555+0000","last_peered":"2024-03-22T04:12:35.951555+0000","last_clean":"2024-03-22T04:12:35.951555+0000","last_became_active":"2024-03-22T04:12:34.622192+0000","last_became_peered":"2024-03-22T04:12:34.622192+0000","last_unstale":"2024-03-22T04:12:35.951555+0000","last_undegraded":"2024-03-22T04:12:35.951555+0000","last_fullsized":"2024-03-22T04:12:35.951555+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:04:24.850239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516225+0000","last_change":"2024-03-22T04:12:34.630270+0000","last_active":"2024-03-22T04:12:36.516225+0000","last_peered":"2024-03-22T04:12:36.516225+0000","last_clean":"2024-03-22T04:12:36.516225+0000","last_became_active":"2024-03-22T04:12:34.629989+0000","last_became_peered":"2024-03-22T04:12:34.629989+0000","last_unstale":"2024-03-22T04:12:36.516225+0000","last_undegraded":"2024-03-22T04:12:36.516225+0000","last_fullsized":"2024-03-22T04:12:36.516225+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612164+0000","last_change":"2024-03-22T04:12:34.620830+0000","last_active":"2024-03-22T04:12:35.612164+0000","last_peered":"2024-03-22T04:12:35.612164+0000","last_clean":"2024-03-22T04:12:35.612164+0000","last_became_active":"2024-03-22T04:12:34.620520+0000","last_became_peered":"2024-03-22T04:12:34.620520+0000","last_unstale":"2024-03-22T04:12:35.612164+0000","last_undegraded":"2024-03-22T04:12:35.612164+0000","last_fullsized":"2024-03-22T04:12:35.612164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:38:31.939143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951264+0000","last_change":"2024-03-22T04:12:34.640293+0000","last_active":"2024-03-22T04:12:35.951264+0000","last_peered":"2024-03-22T04:12:35.951264+0000","last_clean":"2024-03-22T04:12:35.951264+0000","last_became_active":"2024-03-22T04:12:34.639924+0000","last_became_peered":"2024-03-22T04:12:34.639924+0000","last_unstale":"2024-03-22T04:12:35.951264+0000","last_undegraded":"2024-03-22T04:12:35.951264+0000","last_fullsized":"2024-03-22T04:12:35.951264+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:17.196299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951146+0000","last_change":"2024-03-22T04:12:34.621196+0000","last_active":"2024-03-22T04:12:35.951146+0000","last_peered":"2024-03-22T04:12:35.951146+0000","last_clean":"2024-03-22T04:12:35.951146+0000","last_became_active":"2024-03-22T04:12:34.620932+0000","last_became_peered":"2024-03-22T04:12:34.620932+0000","last_unstale":"2024-03-22T04:12:35.951146+0000","last_undegraded":"2024-03-22T04:12:35.951146+0000","last_fullsized":"2024-03-22T04:12:35.951146+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:18:25.483759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951097+0000","last_change":"2024-03-22T04:12:34.638359+0000","last_active":"2024-03-22T04:12:35.951097+0000","last_peered":"2024-03-22T04:12:35.951097+0000","last_clean":"2024-03-22T04:12:35.951097+0000","last_became_active":"2024-03-22T04:12:34.638069+0000","last_became_peered":"2024-03-22T04:12:34.638069+0000","last_unstale":"2024-03-22T04:12:35.951097+0000","last_undegraded":"2024-03-22T04:12:35.951097+0000","last_fullsized":"2024-03-22T04:12:35.951097+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:33:19.884683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.616176+0000","last_change":"2024-03-22T04:12:34.635548+0000","last_active":"2024-03-22T04:12:35.616176+0000","last_peered":"2024-03-22T04:12:35.616176+0000","last_clean":"2024-03-22T04:12:35.616176+0000","last_became_active":"2024-03-22T04:12:34.635270+0000","last_became_peered":"2024-03-22T04:12:34.635270+0000","last_unstale":"2024-03-22T04:12:35.616176+0000","last_undegraded":"2024-03-22T04:12:35.616176+0000","last_fullsized":"2024-03-22T04:12:35.616176+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.485101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951282+0000","last_change":"2024-03-22T04:12:34.637119+0000","last_active":"2024-03-22T04:12:35.951282+0000","last_peered":"2024-03-22T04:12:35.951282+0000","last_clean":"2024-03-22T04:12:35.951282+0000","last_became_active":"2024-03-22T04:12:34.636839+0000","last_became_peered":"2024-03-22T04:12:34.636839+0000","last_unstale":"2024-03-22T04:12:35.951282+0000","last_undegraded":"2024-03-22T04:12:35.951282+0000","last_fullsized":"2024-03-22T04:12:35.951282+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:02:15.918731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954205+0000","last_change":"2024-03-22T04:12:34.655920+0000","last_active":"2024-03-22T04:12:35.954205+0000","last_peered":"2024-03-22T04:12:35.954205+0000","last_clean":"2024-03-22T04:12:35.954205+0000","last_became_active":"2024-03-22T04:12:34.655580+0000","last_became_peered":"2024-03-22T04:12:34.655580+0000","last_unstale":"2024-03-22T04:12:35.954205+0000","last_undegraded":"2024-03-22T04:12:35.954205+0000","last_fullsized":"2024-03-22T04:12:35.954205+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516297+0000","last_change":"2024-03-22T04:12:34.633424+0000","last_active":"2024-03-22T04:12:36.516297+0000","last_peered":"2024-03-22T04:12:36.516297+0000","last_clean":"2024-03-22T04:12:36.516297+0000","last_became_active":"2024-03-22T04:12:34.632797+0000","last_became_peered":"2024-03-22T04:12:34.632797+0000","last_unstale":"2024-03-22T04:12:36.516297+0000","last_undegraded":"2024-03-22T04:12:36.516297+0000","last_fullsized":"2024-03-22T04:12:36.516297+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615318+0000","last_change":"2024-03-22T04:12:34.630952+0000","last_active":"2024-03-22T04:12:35.615318+0000","last_peered":"2024-03-22T04:12:35.615318+0000","last_clean":"2024-03-22T04:12:35.615318+0000","last_became_active":"2024-03-22T04:12:34.630765+0000","last_became_peered":"2024-03-22T04:12:34.630765+0000","last_unstale":"2024-03-22T04:12:35.615318+0000","last_undegraded":"2024-03-22T04:12:35.615318+0000","last_fullsized":"2024-03-22T04:12:35.615318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515691+0000","last_change":"2024-03-22T04:12:34.621885+0000","last_active":"2024-03-22T04:12:36.515691+0000","last_peered":"2024-03-22T04:12:36.515691+0000","last_clean":"2024-03-22T04:12:36.515691+0000","last_became_active":"2024-03-22T04:12:34.621626+0000","last_became_peered":"2024-03-22T04:12:34.621626+0000","last_unstale":"2024-03-22T04:12:36.515691+0000","last_undegraded":"2024-03-22T04:12:36.515691+0000","last_fullsized":"2024-03-22T04:12:36.515691+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:31.105602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951012+0000","last_change":"2024-03-22T04:12:34.640876+0000","last_active":"2024-03-22T04:12:35.951012+0000","last_peered":"2024-03-22T04:12:35.951012+0000","last_clean":"2024-03-22T04:12:35.951012+0000","last_became_active":"2024-03-22T04:12:34.640622+0000","last_became_peered":"2024-03-22T04:12:34.640622+0000","last_unstale":"2024-03-22T04:12:35.951012+0000","last_undegraded":"2024-03-22T04:12:35.951012+0000","last_fullsized":"2024-03-22T04:12:35.951012+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:52:07.467181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516358+0000","last_change":"2024-03-22T04:12:34.624213+0000","last_active":"2024-03-22T04:12:36.516358+0000","last_peered":"2024-03-22T04:12:36.516358+0000","last_clean":"2024-03-22T04:12:36.516358+0000","last_became_active":"2024-03-22T04:12:34.623954+0000","last_became_peered":"2024-03-22T04:12:34.623954+0000","last_unstale":"2024-03-22T04:12:36.516358+0000","last_undegraded":"2024-03-22T04:12:36.516358+0000","last_fullsized":"2024-03-22T04:12:36.516358+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515496+0000","last_change":"2024-03-22T04:12:34.623255+0000","last_active":"2024-03-22T04:12:36.515496+0000","last_peered":"2024-03-22T04:12:36.515496+0000","last_clean":"2024-03-22T04:12:36.515496+0000","last_became_active":"2024-03-22T04:12:34.622995+0000","last_became_peered":"2024-03-22T04:12:34.622995+0000","last_unstale":"2024-03-22T04:12:36.515496+0000","last_undegraded":"2024-03-22T04:12:36.515496+0000","last_fullsized":"2024-03-22T04:12:36.515496+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:19:38.173946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637402+0000","last_change":"2024-03-22T04:12:33.618251+0000","last_active":"2024-03-22T04:12:35.637402+0000","last_peered":"2024-03-22T04:12:35.637402+0000","last_clean":"2024-03-22T04:12:35.637402+0000","last_became_active":"2024-03-22T04:12:33.617405+0000","last_became_peered":"2024-03-22T04:12:33.617405+0000","last_unstale":"2024-03-22T04:12:35.637402+0000","last_undegraded":"2024-03-22T04:12:35.637402+0000","last_fullsized":"2024-03-22T04:12:35.637402+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:45.786716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515621+0000","last_change":"2024-03-22T04:12:34.633188+0000","last_active":"2024-03-22T04:12:36.515621+0000","last_peered":"2024-03-22T04:12:36.515621+0000","last_clean":"2024-03-22T04:12:36.515621+0000","last_became_active":"2024-03-22T04:12:34.632933+0000","last_became_peered":"2024-03-22T04:12:34.632933+0000","last_unstale":"2024-03-22T04:12:36.515621+0000","last_undegraded":"2024-03-22T04:12:36.515621+0000","last_fullsized":"2024-03-22T04:12:36.515621+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:30.228368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954268+0000","last_change":"2024-03-22T04:12:33.623741+0000","last_active":"2024-03-22T04:12:35.954268+0000","last_peered":"2024-03-22T04:12:35.954268+0000","last_clean":"2024-03-22T04:12:35.954268+0000","last_became_active":"2024-03-22T04:12:33.623407+0000","last_became_peered":"2024-03-22T04:12:33.623407+0000","last_unstale":"2024-03-22T04:12:35.954268+0000","last_undegraded":"2024-03-22T04:12:35.954268+0000","last_fullsized":"2024-03-22T04:12:35.954268+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515525+0000","last_change":"2024-03-22T04:12:34.622229+0000","last_active":"2024-03-22T04:12:36.515525+0000","last_peered":"2024-03-22T04:12:36.515525+0000","last_clean":"2024-03-22T04:12:36.515525+0000","last_became_active":"2024-03-22T04:12:34.621209+0000","last_became_peered":"2024-03-22T04:12:34.621209+0000","last_unstale":"2024-03-22T04:12:36.515525+0000","last_undegraded":"2024-03-22T04:12:36.515525+0000","last_fullsized":"2024-03-22T04:12:36.515525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:51:01.872078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615885+0000","last_change":"2024-03-22T04:12:33.621633+0000","last_active":"2024-03-22T04:12:35.615885+0000","last_peered":"2024-03-22T04:12:35.615885+0000","last_clean":"2024-03-22T04:12:35.615885+0000","last_became_active":"2024-03-22T04:12:33.621320+0000","last_became_peered":"2024-03-22T04:12:33.621320+0000","last_unstale":"2024-03-22T04:12:35.615885+0000","last_undegraded":"2024-03-22T04:12:35.615885+0000","last_fullsized":"2024-03-22T04:12:35.615885+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:00.740320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615027+0000","last_change":"2024-03-22T04:12:34.638992+0000","last_active":"2024-03-22T04:12:35.615027+0000","last_peered":"2024-03-22T04:12:35.615027+0000","last_clean":"2024-03-22T04:12:35.615027+0000","last_became_active":"2024-03-22T04:12:34.638706+0000","last_became_peered":"2024-03-22T04:12:34.638706+0000","last_unstale":"2024-03-22T04:12:35.615027+0000","last_undegraded":"2024-03-22T04:12:35.615027+0000","last_fullsized":"2024-03-22T04:12:35.615027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516406+0000","last_change":"2024-03-22T04:12:33.619503+0000","last_active":"2024-03-22T04:12:36.516406+0000","last_peered":"2024-03-22T04:12:36.516406+0000","last_clean":"2024-03-22T04:12:36.516406+0000","last_became_active":"2024-03-22T04:12:33.619218+0000","last_became_peered":"2024-03-22T04:12:33.619218+0000","last_unstale":"2024-03-22T04:12:36.516406+0000","last_undegraded":"2024-03-22T04:12:36.516406+0000","last_fullsized":"2024-03-22T04:12:36.516406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:57:26.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637412+0000","last_change":"2024-03-22T04:12:34.638372+0000","last_active":"2024-03-22T04:12:35.637412+0000","last_peered":"2024-03-22T04:12:35.637412+0000","last_clean":"2024-03-22T04:12:35.637412+0000","last_became_active":"2024-03-22T04:12:34.636653+0000","last_became_peered":"2024-03-22T04:12:34.636653+0000","last_unstale":"2024-03-22T04:12:35.637412+0000","last_undegraded":"2024-03-22T04:12:35.637412+0000","last_fullsized":"2024-03-22T04:12:35.637412+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951462+0000","last_change":"2024-03-22T04:12:33.621816+0000","last_active":"2024-03-22T04:12:35.951462+0000","last_peered":"2024-03-22T04:12:35.951462+0000","last_clean":"2024-03-22T04:12:35.951462+0000","last_became_active":"2024-03-22T04:12:33.621560+0000","last_became_peered":"2024-03-22T04:12:33.621560+0000","last_unstale":"2024-03-22T04:12:35.951462+0000","last_undegraded":"2024-03-22T04:12:35.951462+0000","last_fullsized":"2024-03-22T04:12:35.951462+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:50:45.907035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516002+0000","last_change":"2024-03-22T04:12:34.633399+0000","last_active":"2024-03-22T04:12:36.516002+0000","last_peered":"2024-03-22T04:12:36.516002+0000","last_clean":"2024-03-22T04:12:36.516002+0000","last_became_active":"2024-03-22T04:12:34.633082+0000","last_became_peered":"2024-03-22T04:12:34.633082+0000","last_unstale":"2024-03-22T04:12:36.516002+0000","last_undegraded":"2024-03-22T04:12:36.516002+0000","last_fullsized":"2024-03-22T04:12:36.516002+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:36.338131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515798+0000","last_change":"2024-03-22T04:12:33.615641+0000","last_active":"2024-03-22T04:12:36.515798+0000","last_peered":"2024-03-22T04:12:36.515798+0000","last_clean":"2024-03-22T04:12:36.515798+0000","last_became_active":"2024-03-22T04:12:33.615372+0000","last_became_peered":"2024-03-22T04:12:33.615372+0000","last_unstale":"2024-03-22T04:12:36.515798+0000","last_undegraded":"2024-03-22T04:12:36.515798+0000","last_fullsized":"2024-03-22T04:12:36.515798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:39.482917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615462+0000","last_change":"2024-03-22T04:12:34.637539+0000","last_active":"2024-03-22T04:12:35.615462+0000","last_peered":"2024-03-22T04:12:35.615462+0000","last_clean":"2024-03-22T04:12:35.615462+0000","last_became_active":"2024-03-22T04:12:34.636918+0000","last_became_peered":"2024-03-22T04:12:34.636918+0000","last_unstale":"2024-03-22T04:12:35.615462+0000","last_undegraded":"2024-03-22T04:12:35.615462+0000","last_fullsized":"2024-03-22T04:12:35.615462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:30:17.674924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515426+0000","last_change":"2024-03-22T04:12:33.615588+0000","last_active":"2024-03-22T04:12:36.515426+0000","last_peered":"2024-03-22T04:12:36.515426+0000","last_clean":"2024-03-22T04:12:36.515426+0000","last_became_active":"2024-03-22T04:12:33.615259+0000","last_became_peered":"2024-03-22T04:12:33.615259+0000","last_unstale":"2024-03-22T04:12:36.515426+0000","last_undegraded":"2024-03-22T04:12:36.515426+0000","last_fullsized":"2024-03-22T04:12:36.515426+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:27:55.035247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612760+0000","last_change":"2024-03-22T04:12:34.635762+0000","last_active":"2024-03-22T04:12:35.612760+0000","last_peered":"2024-03-22T04:12:35.612760+0000","last_clean":"2024-03-22T04:12:35.612760+0000","last_became_active":"2024-03-22T04:12:34.635493+0000","last_became_peered":"2024-03-22T04:12:34.635493+0000","last_unstale":"2024-03-22T04:12:35.612760+0000","last_undegraded":"2024-03-22T04:12:35.612760+0000","last_fullsized":"2024-03-22T04:12:35.612760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.950944+0000","last_change":"2024-03-22T04:12:33.618851+0000","last_active":"2024-03-22T04:12:35.950944+0000","last_peered":"2024-03-22T04:12:35.950944+0000","last_clean":"2024-03-22T04:12:35.950944+0000","last_became_active":"2024-03-22T04:12:33.618525+0000","last_became_peered":"2024-03-22T04:12:33.618525+0000","last_unstale":"2024-03-22T04:12:35.950944+0000","last_undegraded":"2024-03-22T04:12:35.950944+0000","last_fullsized":"2024-03-22T04:12:35.950944+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:59:32.268351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637090+0000","last_change":"2024-03-22T04:12:34.626850+0000","last_active":"2024-03-22T04:12:35.637090+0000","last_peered":"2024-03-22T04:12:35.637090+0000","last_clean":"2024-03-22T04:12:35.637090+0000","last_became_active":"2024-03-22T04:12:34.626505+0000","last_became_peered":"2024-03-22T04:12:34.626505+0000","last_unstale":"2024-03-22T04:12:35.637090+0000","last_undegraded":"2024-03-22T04:12:35.637090+0000","last_fullsized":"2024-03-22T04:12:35.637090+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:06.384405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612750+0000","last_change":"2024-03-22T04:12:33.618147+0000","last_active":"2024-03-22T04:12:35.612750+0000","last_peered":"2024-03-22T04:12:35.612750+0000","last_clean":"2024-03-22T04:12:35.612750+0000","last_became_active":"2024-03-22T04:12:33.617867+0000","last_became_peered":"2024-03-22T04:12:33.617867+0000","last_unstale":"2024-03-22T04:12:35.612750+0000","last_undegraded":"2024-03-22T04:12:35.612750+0000","last_fullsized":"2024-03-22T04:12:35.612750+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612131+0000","last_change":"2024-03-22T04:12:34.624495+0000","last_active":"2024-03-22T04:12:35.612131+0000","last_peered":"2024-03-22T04:12:35.612131+0000","last_clean":"2024-03-22T04:12:35.612131+0000","last_became_active":"2024-03-22T04:12:34.624243+0000","last_became_peered":"2024-03-22T04:12:34.624243+0000","last_unstale":"2024-03-22T04:12:35.612131+0000","last_undegraded":"2024-03-22T04:12:35.612131+0000","last_fullsized":"2024-03-22T04:12:35.612131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:58:46.794258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612650+0000","last_change":"2024-03-22T04:12:33.617534+0000","last_active":"2024-03-22T04:12:35.612650+0000","last_peered":"2024-03-22T04:12:35.612650+0000","last_clean":"2024-03-22T04:12:35.612650+0000","last_became_active":"2024-03-22T04:12:33.617265+0000","last_became_peered":"2024-03-22T04:12:33.617265+0000","last_unstale":"2024-03-22T04:12:35.612650+0000","last_undegraded":"2024-03-22T04:12:35.612650+0000","last_fullsized":"2024-03-22T04:12:35.612650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:02.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615181+0000","last_change":"2024-03-22T04:12:34.635018+0000","last_active":"2024-03-22T04:12:35.615181+0000","last_peered":"2024-03-22T04:12:35.615181+0000","last_clean":"2024-03-22T04:12:35.615181+0000","last_became_active":"2024-03-22T04:12:34.634716+0000","last_became_peered":"2024-03-22T04:12:34.634716+0000","last_unstale":"2024-03-22T04:12:35.615181+0000","last_undegraded":"2024-03-22T04:12:35.615181+0000","last_fullsized":"2024-03-22T04:12:35.615181+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516018+0000","last_change":"2024-03-22T04:12:33.623030+0000","last_active":"2024-03-22T04:12:36.516018+0000","last_peered":"2024-03-22T04:12:36.516018+0000","last_clean":"2024-03-22T04:12:36.516018+0000","last_became_active":"2024-03-22T04:12:33.622806+0000","last_became_peered":"2024-03-22T04:12:33.622806+0000","last_unstale":"2024-03-22T04:12:36.516018+0000","last_undegraded":"2024-03-22T04:12:36.516018+0000","last_fullsized":"2024-03-22T04:12:36.516018+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:50:40.568966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515991+0000","last_change":"2024-03-22T04:12:34.635387+0000","last_active":"2024-03-22T04:12:36.515991+0000","last_peered":"2024-03-22T04:12:36.515991+0000","last_clean":"2024-03-22T04:12:36.515991+0000","last_became_active":"2024-03-22T04:12:34.632095+0000","last_became_peered":"2024-03-22T04:12:34.632095+0000","last_unstale":"2024-03-22T04:12:36.515991+0000","last_undegraded":"2024-03-22T04:12:36.515991+0000","last_fullsized":"2024-03-22T04:12:36.515991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615307+0000","last_change":"2024-03-22T04:12:33.616617+0000","last_active":"2024-03-22T04:12:35.615307+0000","last_peered":"2024-03-22T04:12:35.615307+0000","last_clean":"2024-03-22T04:12:35.615307+0000","last_became_active":"2024-03-22T04:12:33.616088+0000","last_became_peered":"2024-03-22T04:12:33.616088+0000","last_unstale":"2024-03-22T04:12:35.615307+0000","last_undegraded":"2024-03-22T04:12:35.615307+0000","last_fullsized":"2024-03-22T04:12:35.615307+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:49.907740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516039+0000","last_change":"2024-03-22T04:12:34.638920+0000","last_active":"2024-03-22T04:12:36.516039+0000","last_peered":"2024-03-22T04:12:36.516039+0000","last_clean":"2024-03-22T04:12:36.516039+0000","last_became_active":"2024-03-22T04:12:34.638167+0000","last_became_peered":"2024-03-22T04:12:34.638167+0000","last_unstale":"2024-03-22T04:12:36.516039+0000","last_undegraded":"2024-03-22T04:12:36.516039+0000","last_fullsized":"2024-03-22T04:12:36.516039+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:59:45.215713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954115+0000","last_change":"2024-03-22T04:12:33.625471+0000","last_active":"2024-03-22T04:12:35.954115+0000","last_peered":"2024-03-22T04:12:35.954115+0000","last_clean":"2024-03-22T04:12:35.954115+0000","last_became_active":"2024-03-22T04:12:33.625189+0000","last_became_peered":"2024-03-22T04:12:33.625189+0000","last_unstale":"2024-03-22T04:12:35.954115+0000","last_undegraded":"2024-03-22T04:12:35.954115+0000","last_fullsized":"2024-03-22T04:12:35.954115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615959+0000","last_change":"2024-03-22T04:12:34.642230+0000","last_active":"2024-03-22T04:12:35.615959+0000","last_peered":"2024-03-22T04:12:35.615959+0000","last_clean":"2024-03-22T04:12:35.615959+0000","last_became_active":"2024-03-22T04:12:34.642038+0000","last_became_peered":"2024-03-22T04:12:34.642038+0000","last_unstale":"2024-03-22T04:12:35.615959+0000","last_undegraded":"2024-03-22T04:12:35.615959+0000","last_fullsized":"2024-03-22T04:12:35.615959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:56:58.252103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612534+0000","last_change":"2024-03-22T04:12:33.620764+0000","last_active":"2024-03-22T04:12:35.612534+0000","last_peered":"2024-03-22T04:12:35.612534+0000","last_clean":"2024-03-22T04:12:35.612534+0000","last_became_active":"2024-03-22T04:12:33.620498+0000","last_became_peered":"2024-03-22T04:12:33.620498+0000","last_unstale":"2024-03-22T04:12:35.612534+0000","last_undegraded":"2024-03-22T04:12:35.612534+0000","last_fullsized":"2024-03-22T04:12:35.612534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:49.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954370+0000","last_change":"2024-03-22T04:12:34.641493+0000","last_active":"2024-03-22T04:12:35.954370+0000","last_peered":"2024-03-22T04:12:35.954370+0000","last_clean":"2024-03-22T04:12:35.954370+0000","last_became_active":"2024-03-22T04:12:34.636610+0000","last_became_peered":"2024-03-22T04:12:34.636610+0000","last_unstale":"2024-03-22T04:12:35.954370+0000","last_undegraded":"2024-03-22T04:12:35.954370+0000","last_fullsized":"2024-03-22T04:12:35.954370+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:01:23.748073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515773+0000","last_change":"2024-03-22T04:12:33.618015+0000","last_active":"2024-03-22T04:12:36.515773+0000","last_peered":"2024-03-22T04:12:36.515773+0000","last_clean":"2024-03-22T04:12:36.515773+0000","last_became_active":"2024-03-22T04:12:33.617794+0000","last_became_peered":"2024-03-22T04:12:33.617794+0000","last_unstale":"2024-03-22T04:12:36.515773+0000","last_undegraded":"2024-03-22T04:12:36.515773+0000","last_fullsized":"2024-03-22T04:12:36.515773+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:00.222391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951435+0000","last_change":"2024-03-22T04:12:34.626431+0000","last_active":"2024-03-22T04:12:35.951435+0000","last_peered":"2024-03-22T04:12:35.951435+0000","last_clean":"2024-03-22T04:12:35.951435+0000","last_became_active":"2024-03-22T04:12:34.626084+0000","last_became_peered":"2024-03-22T04:12:34.626084+0000","last_unstale":"2024-03-22T04:12:35.951435+0000","last_undegraded":"2024-03-22T04:12:35.951435+0000","last_fullsized":"2024-03-22T04:12:35.951435+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:34:04.944294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.953882+0000","last_change":"2024-03-22T04:12:33.621891+0000","last_active":"2024-03-22T04:12:35.953882+0000","last_peered":"2024-03-22T04:12:35.953882+0000","last_clean":"2024-03-22T04:12:35.953882+0000","last_became_active":"2024-03-22T04:12:33.621626+0000","last_became_peered":"2024-03-22T04:12:33.621626+0000","last_unstale":"2024-03-22T04:12:35.953882+0000","last_undegraded":"2024-03-22T04:12:35.953882+0000","last_fullsized":"2024-03-22T04:12:35.953882+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614700+0000","last_change":"2024-03-22T04:12:34.634352+0000","last_active":"2024-03-22T04:12:35.614700+0000","last_peered":"2024-03-22T04:12:35.614700+0000","last_clean":"2024-03-22T04:12:35.614700+0000","last_became_active":"2024-03-22T04:12:34.633729+0000","last_became_peered":"2024-03-22T04:12:34.633729+0000","last_unstale":"2024-03-22T04:12:35.614700+0000","last_undegraded":"2024-03-22T04:12:35.614700+0000","last_fullsized":"2024-03-22T04:12:35.614700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:56:35.284077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954420+0000","last_change":"2024-03-22T04:12:33.623382+0000","last_active":"2024-03-22T04:12:35.954420+0000","last_peered":"2024-03-22T04:12:35.954420+0000","last_clean":"2024-03-22T04:12:35.954420+0000","last_became_active":"2024-03-22T04:12:33.622982+0000","last_became_peered":"2024-03-22T04:12:33.622982+0000","last_unstale":"2024-03-22T04:12:35.954420+0000","last_undegraded":"2024-03-22T04:12:35.954420+0000","last_fullsized":"2024-03-22T04:12:35.954420+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614924+0000","last_change":"2024-03-22T04:12:34.627065+0000","last_active":"2024-03-22T04:12:35.614924+0000","last_peered":"2024-03-22T04:12:35.614924+0000","last_clean":"2024-03-22T04:12:35.614924+0000","last_became_active":"2024-03-22T04:12:34.626777+0000","last_became_peered":"2024-03-22T04:12:34.626777+0000","last_unstale":"2024-03-22T04:12:35.614924+0000","last_undegraded":"2024-03-22T04:12:35.614924+0000","last_fullsized":"2024-03-22T04:12:35.614924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954333+0000","last_change":"2024-03-22T04:12:33.622321+0000","last_active":"2024-03-22T04:12:35.954333+0000","last_peered":"2024-03-22T04:12:35.954333+0000","last_clean":"2024-03-22T04:12:35.954333+0000","last_became_active":"2024-03-22T04:12:33.622058+0000","last_became_peered":"2024-03-22T04:12:33.622058+0000","last_unstale":"2024-03-22T04:12:35.954333+0000","last_undegraded":"2024-03-22T04:12:35.954333+0000","last_fullsized":"2024-03-22T04:12:35.954333+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:13:23.560322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515730+0000","last_change":"2024-03-22T04:12:34.636289+0000","last_active":"2024-03-22T04:12:36.515730+0000","last_peered":"2024-03-22T04:12:36.515730+0000","last_clean":"2024-03-22T04:12:36.515730+0000","last_became_active":"2024-03-22T04:12:34.636025+0000","last_became_peered":"2024-03-22T04:12:34.636025+0000","last_unstale":"2024-03-22T04:12:36.515730+0000","last_undegraded":"2024-03-22T04:12:36.515730+0000","last_fullsized":"2024-03-22T04:12:36.515730+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:47:27.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612548+0000","last_change":"2024-03-22T04:12:33.611878+0000","last_active":"2024-03-22T04:12:35.612548+0000","last_peered":"2024-03-22T04:12:35.612548+0000","last_clean":"2024-03-22T04:12:35.612548+0000","last_became_active":"2024-03-22T04:12:33.611537+0000","last_became_peered":"2024-03-22T04:12:33.611537+0000","last_unstale":"2024-03-22T04:12:35.612548+0000","last_undegraded":"2024-03-22T04:12:35.612548+0000","last_fullsized":"2024-03-22T04:12:35.612548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:44:21.890411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954042+0000","last_change":"2024-03-22T04:12:34.650207+0000","last_active":"2024-03-22T04:12:35.954042+0000","last_peered":"2024-03-22T04:12:35.954042+0000","last_clean":"2024-03-22T04:12:35.954042+0000","last_became_active":"2024-03-22T04:12:34.649927+0000","last_became_peered":"2024-03-22T04:12:34.649927+0000","last_unstale":"2024-03-22T04:12:35.954042+0000","last_undegraded":"2024-03-22T04:12:35.954042+0000","last_fullsized":"2024-03-22T04:12:35.954042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:43:33.625589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516450+0000","last_change":"2024-03-22T04:12:33.622557+0000","last_active":"2024-03-22T04:12:36.516450+0000","last_peered":"2024-03-22T04:12:36.516450+0000","last_clean":"2024-03-22T04:12:36.516450+0000","last_became_active":"2024-03-22T04:12:33.622240+0000","last_became_peered":"2024-03-22T04:12:33.622240+0000","last_unstale":"2024-03-22T04:12:36.516450+0000","last_undegraded":"2024-03-22T04:12:36.516450+0000","last_fullsized":"2024-03-22T04:12:36.516450+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:32.007591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516362+0000","last_change":"2024-03-22T04:12:34.633656+0000","last_active":"2024-03-22T04:12:36.516362+0000","last_peered":"2024-03-22T04:12:36.516362+0000","last_clean":"2024-03-22T04:12:36.516362+0000","last_became_active":"2024-03-22T04:12:34.633150+0000","last_became_peered":"2024-03-22T04:12:34.633150+0000","last_unstale":"2024-03-22T04:12:36.516362+0000","last_undegraded":"2024-03-22T04:12:36.516362+0000","last_fullsized":"2024-03-22T04:12:36.516362+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615428+0000","last_change":"2024-03-22T04:12:33.624581+0000","last_active":"2024-03-22T04:12:35.615428+0000","last_peered":"2024-03-22T04:12:35.615428+0000","last_clean":"2024-03-22T04:12:35.615428+0000","last_became_active":"2024-03-22T04:12:33.624324+0000","last_became_peered":"2024-03-22T04:12:33.624324+0000","last_unstale":"2024-03-22T04:12:35.615428+0000","last_undegraded":"2024-03-22T04:12:35.615428+0000","last_fullsized":"2024-03-22T04:12:35.615428+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:41.428263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612405+0000","last_change":"2024-03-22T04:12:34.631558+0000","last_active":"2024-03-22T04:12:35.612405+0000","last_peered":"2024-03-22T04:12:35.612405+0000","last_clean":"2024-03-22T04:12:35.612405+0000","last_became_active":"2024-03-22T04:12:34.621598+0000","last_became_peered":"2024-03-22T04:12:34.621598+0000","last_unstale":"2024-03-22T04:12:35.612405+0000","last_undegraded":"2024-03-22T04:12:35.612405+0000","last_fullsized":"2024-03-22T04:12:35.612405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:43:33.625589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951395+0000","last_change":"2024-03-22T04:12:33.619281+0000","last_active":"2024-03-22T04:12:35.951395+0000","last_peered":"2024-03-22T04:12:35.951395+0000","last_clean":"2024-03-22T04:12:35.951395+0000","last_became_active":"2024-03-22T04:12:33.618935+0000","last_became_peered":"2024-03-22T04:12:33.618935+0000","last_unstale":"2024-03-22T04:12:35.951395+0000","last_undegraded":"2024-03-22T04:12:35.951395+0000","last_fullsized":"2024-03-22T04:12:35.951395+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:48:41.700689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615777+0000","last_change":"2024-03-22T04:12:34.642768+0000","last_active":"2024-03-22T04:12:35.615777+0000","last_peered":"2024-03-22T04:12:35.615777+0000","last_clean":"2024-03-22T04:12:35.615777+0000","last_became_active":"2024-03-22T04:12:34.642534+0000","last_became_peered":"2024-03-22T04:12:34.642534+0000","last_unstale":"2024-03-22T04:12:35.615777+0000","last_undegraded":"2024-03-22T04:12:35.615777+0000","last_fullsized":"2024-03-22T04:12:35.615777+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615251+0000","last_change":"2024-03-22T04:12:33.626234+0000","last_active":"2024-03-22T04:12:35.615251+0000","last_peered":"2024-03-22T04:12:35.615251+0000","last_clean":"2024-03-22T04:12:35.615251+0000","last_became_active":"2024-03-22T04:12:33.623978+0000","last_became_peered":"2024-03-22T04:12:33.623978+0000","last_unstale":"2024-03-22T04:12:35.615251+0000","last_undegraded":"2024-03-22T04:12:35.615251+0000","last_fullsized":"2024-03-22T04:12:35.615251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:36:45.736028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954443+0000","last_change":"2024-03-22T04:12:34.646431+0000","last_active":"2024-03-22T04:12:35.954443+0000","last_peered":"2024-03-22T04:12:35.954443+0000","last_clean":"2024-03-22T04:12:35.954443+0000","last_became_active":"2024-03-22T04:12:34.646176+0000","last_became_peered":"2024-03-22T04:12:34.646176+0000","last_unstale":"2024-03-22T04:12:35.954443+0000","last_undegraded":"2024-03-22T04:12:35.954443+0000","last_fullsized":"2024-03-22T04:12:35.954443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:10:52.374805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.614477+0000","last_change":"2024-03-22T04:12:33.610526+0000","last_active":"2024-03-22T04:12:35.614477+0000","last_peered":"2024-03-22T04:12:35.614477+0000","last_clean":"2024-03-22T04:12:35.614477+0000","last_became_active":"2024-03-22T04:12:33.609782+0000","last_became_peered":"2024-03-22T04:12:33.609782+0000","last_unstale":"2024-03-22T04:12:35.614477+0000","last_undegraded":"2024-03-22T04:12:35.614477+0000","last_fullsized":"2024-03-22T04:12:35.614477+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:37:11.978324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.611982+0000","last_change":"2024-03-22T04:12:34.622644+0000","last_active":"2024-03-22T04:12:35.611982+0000","last_peered":"2024-03-22T04:12:35.611982+0000","last_clean":"2024-03-22T04:12:35.611982+0000","last_became_active":"2024-03-22T04:12:34.622112+0000","last_became_peered":"2024-03-22T04:12:34.622112+0000","last_unstale":"2024-03-22T04:12:35.611982+0000","last_undegraded":"2024-03-22T04:12:35.611982+0000","last_fullsized":"2024-03-22T04:12:35.611982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:19.819646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516202+0000","last_change":"2024-03-22T04:12:33.621421+0000","last_active":"2024-03-22T04:12:36.516202+0000","last_peered":"2024-03-22T04:12:36.516202+0000","last_clean":"2024-03-22T04:12:36.516202+0000","last_became_active":"2024-03-22T04:12:33.621140+0000","last_became_peered":"2024-03-22T04:12:33.621140+0000","last_unstale":"2024-03-22T04:12:36.516202+0000","last_undegraded":"2024-03-22T04:12:36.516202+0000","last_fullsized":"2024-03-22T04:12:36.516202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:52:21.786333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615749+0000","last_change":"2024-03-22T04:12:34.638291+0000","last_active":"2024-03-22T04:12:35.615749+0000","last_peered":"2024-03-22T04:12:35.615749+0000","last_clean":"2024-03-22T04:12:35.615749+0000","last_became_active":"2024-03-22T04:12:34.637953+0000","last_became_peered":"2024-03-22T04:12:34.637953+0000","last_unstale":"2024-03-22T04:12:35.615749+0000","last_undegraded":"2024-03-22T04:12:35.615749+0000","last_fullsized":"2024-03-22T04:12:35.615749+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:37:44.536324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.516091+0000","last_change":"2024-03-22T04:12:33.623102+0000","last_active":"2024-03-22T04:12:36.516091+0000","last_peered":"2024-03-22T04:12:36.516091+0000","last_clean":"2024-03-22T04:12:36.516091+0000","last_became_active":"2024-03-22T04:12:33.622803+0000","last_became_peered":"2024-03-22T04:12:33.622803+0000","last_unstale":"2024-03-22T04:12:36.516091+0000","last_undegraded":"2024-03-22T04:12:36.516091+0000","last_fullsized":"2024-03-22T04:12:36.516091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:04:41.174822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951365+0000","last_change":"2024-03-22T04:12:34.623839+0000","last_active":"2024-03-22T04:12:35.951365+0000","last_peered":"2024-03-22T04:12:35.951365+0000","last_clean":"2024-03-22T04:12:35.951365+0000","last_became_active":"2024-03-22T04:12:34.623357+0000","last_became_peered":"2024-03-22T04:12:34.623357+0000","last_unstale":"2024-03-22T04:12:35.951365+0000","last_undegraded":"2024-03-22T04:12:35.951365+0000","last_fullsized":"2024-03-22T04:12:35.951365+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:27:10.460890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951051+0000","last_change":"2024-03-22T04:12:33.618302+0000","last_active":"2024-03-22T04:12:35.951051+0000","last_peered":"2024-03-22T04:12:35.951051+0000","last_clean":"2024-03-22T04:12:35.951051+0000","last_became_active":"2024-03-22T04:12:33.617971+0000","last_became_peered":"2024-03-22T04:12:33.617971+0000","last_unstale":"2024-03-22T04:12:35.951051+0000","last_undegraded":"2024-03-22T04:12:35.951051+0000","last_fullsized":"2024-03-22T04:12:35.951051+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:32:05.092737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615191+0000","last_change":"2024-03-22T04:12:34.644894+0000","last_active":"2024-03-22T04:12:35.615191+0000","last_peered":"2024-03-22T04:12:35.615191+0000","last_clean":"2024-03-22T04:12:35.615191+0000","last_became_active":"2024-03-22T04:12:34.644526+0000","last_became_peered":"2024-03-22T04:12:34.644526+0000","last_unstale":"2024-03-22T04:12:35.615191+0000","last_undegraded":"2024-03-22T04:12:35.615191+0000","last_fullsized":"2024-03-22T04:12:35.615191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:36:02.574582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615131+0000","last_change":"2024-03-22T04:12:33.620091+0000","last_active":"2024-03-22T04:12:35.615131+0000","last_peered":"2024-03-22T04:12:35.615131+0000","last_clean":"2024-03-22T04:12:35.615131+0000","last_became_active":"2024-03-22T04:12:33.619790+0000","last_became_peered":"2024-03-22T04:12:33.619790+0000","last_unstale":"2024-03-22T04:12:35.615131+0000","last_undegraded":"2024-03-22T04:12:35.615131+0000","last_fullsized":"2024-03-22T04:12:35.615131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:32.336479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515808+0000","last_change":"2024-03-22T04:12:34.636152+0000","last_active":"2024-03-22T04:12:36.515808+0000","last_peered":"2024-03-22T04:12:36.515808+0000","last_clean":"2024-03-22T04:12:36.515808+0000","last_became_active":"2024-03-22T04:12:34.635865+0000","last_became_peered":"2024-03-22T04:12:34.635865+0000","last_unstale":"2024-03-22T04:12:36.515808+0000","last_undegraded":"2024-03-22T04:12:36.515808+0000","last_fullsized":"2024-03-22T04:12:36.515808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:00.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.951408+0000","last_change":"2024-03-22T04:12:33.612435+0000","last_active":"2024-03-22T04:12:35.951408+0000","last_peered":"2024-03-22T04:12:35.951408+0000","last_clean":"2024-03-22T04:12:35.951408+0000","last_became_active":"2024-03-22T04:12:33.611642+0000","last_became_peered":"2024-03-22T04:12:33.611642+0000","last_unstale":"2024-03-22T04:12:35.951408+0000","last_undegraded":"2024-03-22T04:12:35.951408+0000","last_fullsized":"2024-03-22T04:12:35.951408+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:18:53.878663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.615970+0000","last_change":"2024-03-22T04:12:34.637750+0000","last_active":"2024-03-22T04:12:35.615970+0000","last_peered":"2024-03-22T04:12:35.615970+0000","last_clean":"2024-03-22T04:12:35.615970+0000","last_became_active":"2024-03-22T04:12:34.637269+0000","last_became_peered":"2024-03-22T04:12:34.637269+0000","last_unstale":"2024-03-22T04:12:35.615970+0000","last_undegraded":"2024-03-22T04:12:35.615970+0000","last_fullsized":"2024-03-22T04:12:35.615970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:54:58.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:36.515652+0000","last_change":"2024-03-22T04:12:33.613711+0000","last_active":"2024-03-22T04:12:36.515652+0000","last_peered":"2024-03-22T04:12:36.515652+0000","last_clean":"2024-03-22T04:12:36.515652+0000","last_became_active":"2024-03-22T04:12:33.613398+0000","last_became_peered":"2024-03-22T04:12:33.613398+0000","last_unstale":"2024-03-22T04:12:36.515652+0000","last_undegraded":"2024-03-22T04:12:36.515652+0000","last_fullsized":"2024-03-22T04:12:36.515652+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:34.304317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.954305+0000","last_change":"2024-03-22T04:12:34.649939+0000","last_active":"2024-03-22T04:12:35.954305+0000","last_peered":"2024-03-22T04:12:35.954305+0000","last_clean":"2024-03-22T04:12:35.954305+0000","last_became_active":"2024-03-22T04:12:34.649639+0000","last_became_peered":"2024-03-22T04:12:34.649639+0000","last_unstale":"2024-03-22T04:12:35.954305+0000","last_undegraded":"2024-03-22T04:12:35.954305+0000","last_fullsized":"2024-03-22T04:12:35.954305+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612050+0000","last_change":"2024-03-22T04:12:33.613244+0000","last_active":"2024-03-22T04:12:35.612050+0000","last_peered":"2024-03-22T04:12:35.612050+0000","last_clean":"2024-03-22T04:12:35.612050+0000","last_became_active":"2024-03-22T04:12:33.612972+0000","last_became_peered":"2024-03-22T04:12:33.612972+0000","last_unstale":"2024-03-22T04:12:35.612050+0000","last_undegraded":"2024-03-22T04:12:35.612050+0000","last_fullsized":"2024-03-22T04:12:35.612050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:14:14.546371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637167+0000","last_change":"2024-03-22T04:12:34.638813+0000","last_active":"2024-03-22T04:12:35.637167+0000","last_peered":"2024-03-22T04:12:35.637167+0000","last_clean":"2024-03-22T04:12:35.637167+0000","last_became_active":"2024-03-22T04:12:34.638487+0000","last_became_peered":"2024-03-22T04:12:34.638487+0000","last_unstale":"2024-03-22T04:12:35.637167+0000","last_undegraded":"2024-03-22T04:12:35.637167+0000","last_fullsized":"2024-03-22T04:12:35.637167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:33.592171+0000","last_clean_scrub_stamp":"2024-03-22T04:12:33.592171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.896802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.637130+0000","last_change":"2024-03-22T04:12:33.624669+0000","last_active":"2024-03-22T04:12:35.637130+0000","last_peered":"2024-03-22T04:12:35.637130+0000","last_clean":"2024-03-22T04:12:35.637130+0000","last_became_active":"2024-03-22T04:12:33.624416+0000","last_became_peered":"2024-03-22T04:12:33.624416+0000","last_unstale":"2024-03-22T04:12:35.637130+0000","last_undegraded":"2024-03-22T04:12:35.637130+0000","last_fullsized":"2024-03-22T04:12:35.637130+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:12:32.584629+0000","last_clean_scrub_stamp":"2024-03-22T04:12:32.584629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:18.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":227,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-22T04:12:35.612039+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:12:35.612039+0000","last_peered":"2024-03-22T04:12:35.612039+0000","last_clean":"2024-03-22T04:12:35.612039+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:12:35.612039+0000","last_undegraded":"2024-03-22T04:12:35.612039+0000","last_fullsized":"2024-03-22T04:12:35.612039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":280523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":281049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1245184,"data_stored":872537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":280557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":280501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":870905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":281049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":282171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:12:43.550 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T04:12:43.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-22T04:12:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc64e800640 1 Processor -- start 2024-03-22T04:12:43.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc64e800640 1 -- start start 2024-03-22T04:12:43.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc64e800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc64815b3d0 0x7fc6480cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc64e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6480b4d60 con 0x7fc6480d03a0 2024-03-22T04:12:43.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc64e800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6480cca50 con 0x7fc6480b4920 2024-03-22T04:12:43.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc64e800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc6480cd380 con 0x7fc64815b3d0 2024-03-22T04:12:43.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc647e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc6480d03a0 0x7fc6480b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57246/0 (socket says 172.21.15.86:57246) 2024-03-22T04:12:43.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc647e00640 1 -- 172.21.15.86:0/2631185902 learned_addr learned my addr 172.21.15.86:0/2631185902 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:43.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc647400640 1 --2- 172.21.15.86:0/2631185902 >> v2:172.21.15.196:3300/0 conn(0x7fc64815b3d0 0x7fc6480cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc646a00640 1 -- 172.21.15.86:0/2631185902 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2774633057 0 0) 0x7fc6480b4d60 con 0x7fc6480d03a0 2024-03-22T04:12:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc646a00640 1 -- 172.21.15.86:0/2631185902 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc62c005740 con 0x7fc6480d03a0 2024-03-22T04:12:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc647400640 1 -- 172.21.15.86:0/2631185902 >> v1:172.21.15.196:6789/0 conn(0x7fc6480b4920 legacy=0x7fc6480cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc646a00640 1 -- 172.21.15.86:0/2631185902 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385779751 0 0) 0x7fc6480cca50 con 0x7fc6480b4920 2024-03-22T04:12:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc647400640 1 -- 172.21.15.86:0/2631185902 >> v1:172.21.15.86:6789/0 conn(0x7fc6480d03a0 legacy=0x7fc6480b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.729+0000 7fc647400640 1 -- 172.21.15.86:0/2631185902 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6480ccd30 con 0x7fc64815b3d0 2024-03-22T04:12:43.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc647400640 1 --2- 172.21.15.86:0/2631185902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc64815b3d0 0x7fc6480cf5d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc6480cd500 tx=0x7fc63801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81daebb2901e2df server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:43.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc646a00640 1 -- 172.21.15.86:0/2631185902 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6380b5070 con 0x7fc64815b3d0 2024-03-22T04:12:43.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc646a00640 1 -- 172.21.15.86:0/2631185902 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6380a8990 con 0x7fc64815b3d0 2024-03-22T04:12:43.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc646a00640 1 -- 172.21.15.86:0/2631185902 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6380a8d60 con 0x7fc64815b3d0 2024-03-22T04:12:43.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc64e800640 1 -- 172.21.15.86:0/2631185902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc64815b3d0 msgr2=0x7fc6480cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc64e800640 1 --2- 172.21.15.86:0/2631185902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc64815b3d0 0x7fc6480cf5d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc6480cd500 tx=0x7fc63801ec90 comp rx=0 tx=0).stop 2024-03-22T04:12:43.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc64e800640 1 -- 172.21.15.86:0/2631185902 shutdown_connections 2024-03-22T04:12:43.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc64e800640 1 --2- 172.21.15.86:0/2631185902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc64815b3d0 0x7fc6480cf5d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc64e800640 1 -- 172.21.15.86:0/2631185902 >> 172.21.15.86:0/2631185902 conn(0x7fc64806ef50 msgr2=0x7fc64806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:43.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc64e800640 1 -- 172.21.15.86:0/2631185902 shutdown_connections 2024-03-22T04:12:43.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.733+0000 7fc64e800640 1 -- 172.21.15.86:0/2631185902 wait complete. 2024-03-22T04:12:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 Processor -- start 2024-03-22T04:12:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 -- start start 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6480b4920 0x7fc6481447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6480d03a0 0x7fc648154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc64815b3d0 0x7fc648144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6480d07f0 con 0x7fc64815b3d0 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6480cd810 con 0x7fc6480d03a0 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc6480b3790 con 0x7fc6480b4920 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc64815b3d0 0x7fc648144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc64815b3d0 0x7fc648144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51510/0 (socket says 172.21.15.86:51510) 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6480b4920 0x7fc6481447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647e00640 1 -- 172.21.15.86:0/4228230282 learned_addr learned my addr 172.21.15.86:0/4228230282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc63ea00640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6480d03a0 0x7fc648154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647400640 1 -- 172.21.15.86:0/4228230282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6480d03a0 msgr2=0x7fc648154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647400640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6480d03a0 0x7fc648154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647400640 1 -- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc64815b3d0 msgr2=0x7fc648144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647400640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc64815b3d0 0x7fc648144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647400640 1 -- 172.21.15.86:0/4228230282 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc648147e80 con 0x7fc6480b4920 2024-03-22T04:12:43.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647e00640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc64815b3d0 0x7fc648144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:43.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc647400640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6480b4920 0x7fc6481447d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6380af000 tx=0x7fc63801ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc63fe00640 1 -- 172.21.15.86:0/4228230282 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6380be020 con 0x7fc6480b4920 2024-03-22T04:12:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc63fe00640 1 -- 172.21.15.86:0/4228230282 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc638004220 con 0x7fc6480b4920 2024-03-22T04:12:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc63fe00640 1 -- 172.21.15.86:0/4228230282 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6380b5040 con 0x7fc6480b4920 2024-03-22T04:12:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc648153e70 con 0x7fc6480b4920 2024-03-22T04:12:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.737+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc648154430 con 0x7fc6480b4920 2024-03-22T04:12:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.741+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc648074470 con 0x7fc6480b4920 2024-03-22T04:12:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.741+0000 7fc63fe00640 1 -- 172.21.15.86:0/4228230282 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc6380c1070 con 0x7fc6480b4920 2024-03-22T04:12:43.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.741+0000 7fc63fe00640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc624067b20 0x7fc624069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:43.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.741+0000 7fc63fe00640 1 -- 172.21.15.86:0/4228230282 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc6381226e0 con 0x7fc6480b4920 2024-03-22T04:12:43.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.741+0000 7fc63ea00640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc624067b20 0x7fc624069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:43.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.741+0000 7fc63ea00640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc624067b20 0x7fc624069fe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc6481f7500 tx=0x7fc630009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.745+0000 7fc63fe00640 1 -- 172.21.15.86:0/4228230282 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc6380187f0 con 0x7fc6480b4920 2024-03-22T04:12:43.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.961+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc648072ba0 con 0x7fc6480b4920 2024-03-22T04:12:43.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.965+0000 7fc63fe00640 1 -- 172.21.15.86:0/4228230282 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fc6380f2900 con 0x7fc6480b4920 2024-03-22T04:12:43.970 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T04:12:43.970 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:12:43.970 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-03-22T04:12:43.970 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-03-22T04:12:43.970 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-22T04:12:43.970 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-03-22T04:12:43.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc624067b20 msgr2=0x7fc624069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc624067b20 0x7fc624069fe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc6481f7500 tx=0x7fc630009040 comp rx=0 tx=0).stop 2024-03-22T04:12:43.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6480b4920 msgr2=0x7fc6481447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:43.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6480b4920 0x7fc6481447d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6380af000 tx=0x7fc63801ba30 comp rx=0 tx=0).stop 2024-03-22T04:12:43.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 shutdown_connections 2024-03-22T04:12:43.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc624067b20 0x7fc624069fe0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6480d03a0 0x7fc648154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc64815b3d0 0x7fc648144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 --2- 172.21.15.86:0/4228230282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6480b4920 0x7fc6481447d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:43.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 >> 172.21.15.86:0/4228230282 conn(0x7fc64806ef50 msgr2=0x7fc6480715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:43.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 shutdown_connections 2024-03-22T04:12:43.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:43.969+0000 7fc64e800640 1 -- 172.21.15.86:0/4228230282 wait complete. 2024-03-22T04:12:43.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8521c00640 1 Processor -- start 2024-03-22T04:12:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8521c00640 1 -- start start 2024-03-22T04:12:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8521c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f851c0aff50 0x7f851c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8521c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f851c1444f0 0x7f851c15eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8521c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f851c0ae5a0 con 0x7f851c06cbb0 2024-03-22T04:12:44.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8521c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f851c06d010 con 0x7f851c0aff50 2024-03-22T04:12:44.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8521c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f851c144940 con 0x7f851c1444f0 2024-03-22T04:12:44.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f851aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f851c0aff50 0x7f851c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f851a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f851c06cbb0 0x7f851c143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57254/0 (socket says 172.21.15.86:57254) 2024-03-22T04:12:44.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f851a000640 1 -- 172.21.15.86:0/3329147609 learned_addr learned my addr 172.21.15.86:0/3329147609 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:44.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f851b400640 1 --2- 172.21.15.86:0/3329147609 >> v2:172.21.15.196:3300/0 conn(0x7f851c1444f0 0x7f851c15eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8519600640 1 -- 172.21.15.86:0/3329147609 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367915254 0 0) 0x7f8508001120 con 0x7f851c06cbb0 2024-03-22T04:12:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f8519600640 1 -- 172.21.15.86:0/3329147609 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8500003530 con 0x7f851c06cbb0 2024-03-22T04:12:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f851aa00640 1 -- 172.21.15.86:0/3329147609 >> v2:172.21.15.196:3300/0 conn(0x7f851c1444f0 msgr2=0x7f851c15eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f851aa00640 1 --2- 172.21.15.86:0/3329147609 >> v2:172.21.15.196:3300/0 conn(0x7f851c1444f0 0x7f851c15eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f851aa00640 1 -- 172.21.15.86:0/3329147609 >> v1:172.21.15.86:6789/0 conn(0x7f851c06cbb0 legacy=0x7f851c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.125+0000 7f851aa00640 1 -- 172.21.15.86:0/3329147609 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c144b80 con 0x7f851c0aff50 2024-03-22T04:12:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f851aa00640 1 --2- 172.21.15.86:0/3329147609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0aff50 0x7f851c06c670 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f851c0702c0 tx=0x7f850c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fcf49000461489 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8519600640 1 -- 172.21.15.86:0/3329147609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0b2070 con 0x7f851c0aff50 2024-03-22T04:12:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8519600640 1 -- 172.21.15.86:0/3329147609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c0a5850 con 0x7f851c0aff50 2024-03-22T04:12:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8519600640 1 -- 172.21.15.86:0/3329147609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0a5c20 con 0x7f851c0aff50 2024-03-22T04:12:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8521c00640 1 -- 172.21.15.86:0/3329147609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0aff50 msgr2=0x7f851c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8521c00640 1 --2- 172.21.15.86:0/3329147609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0aff50 0x7f851c06c670 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f851c0702c0 tx=0x7f850c01c8a0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8521c00640 1 -- 172.21.15.86:0/3329147609 shutdown_connections 2024-03-22T04:12:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8521c00640 1 --2- 172.21.15.86:0/3329147609 >> v2:172.21.15.196:3300/0 conn(0x7f851c1444f0 0x7f851c15eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8521c00640 1 --2- 172.21.15.86:0/3329147609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0aff50 0x7f851c06c670 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8521c00640 1 -- 172.21.15.86:0/3329147609 >> 172.21.15.86:0/3329147609 conn(0x7f851c071fe0 msgr2=0x7f851c0723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8521c00640 1 -- 172.21.15.86:0/3329147609 shutdown_connections 2024-03-22T04:12:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.129+0000 7f8521c00640 1 -- 172.21.15.86:0/3329147609 wait complete. 2024-03-22T04:12:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 Processor -- start 2024-03-22T04:12:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 -- start start 2024-03-22T04:12:44.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c06cbb0 0x7f851c13a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0aff50 0x7f851c13c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c1444f0 0x7f851c13c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f851c162f30 con 0x7f851c1444f0 2024-03-22T04:12:44.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0b0330 con 0x7f851c06cbb0 2024-03-22T04:12:44.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f851c13b090 con 0x7f851c0aff50 2024-03-22T04:12:44.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c1444f0 0x7f851c13c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c1444f0 0x7f851c13c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51528/0 (socket says 172.21.15.86:51528) 2024-03-22T04:12:44.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c06cbb0 0x7f851c13a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 -- 172.21.15.86:0/1478385572 learned_addr learned my addr 172.21.15.86:0/1478385572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:44.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851a000640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0aff50 0x7f851c13c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 -- 172.21.15.86:0/1478385572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0aff50 msgr2=0x7f851c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0aff50 0x7f851c13c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 -- 172.21.15.86:0/1478385572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c06cbb0 msgr2=0x7f851c13a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c06cbb0 0x7f851c13a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 -- 172.21.15.86:0/1478385572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c13b370 con 0x7f851c1444f0 2024-03-22T04:12:44.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f851b400640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c1444f0 0x7f851c13c8f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f851400e7f0 tx=0x7f851400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8512a00640 1 -- 172.21.15.86:0/1478385572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514017070 con 0x7f851c1444f0 2024-03-22T04:12:44.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8512a00640 1 -- 172.21.15.86:0/1478385572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f851400c940 con 0x7f851c1444f0 2024-03-22T04:12:44.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c13b650 con 0x7f851c1444f0 2024-03-22T04:12:44.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8512a00640 1 -- 172.21.15.86:0/1478385572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851400eea0 con 0x7f851c1444f0 2024-03-22T04:12:44.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c13ada0 con 0x7f851c1444f0 2024-03-22T04:12:44.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.133+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c13bb70 con 0x7f851c1444f0 2024-03-22T04:12:44.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.137+0000 7f8512a00640 1 -- 172.21.15.86:0/1478385572 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f851401a070 con 0x7f851c1444f0 2024-03-22T04:12:44.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.137+0000 7f8512a00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84fc067b20 0x7f84fc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.137+0000 7f8512a00640 1 -- 172.21.15.86:0/1478385572 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f85140861f0 con 0x7f851c1444f0 2024-03-22T04:12:44.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.137+0000 7f851aa00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84fc067b20 0x7f84fc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.137+0000 7f851aa00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84fc067b20 0x7f84fc069fe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f850c0af000 tx=0x7f850c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.141+0000 7f8512a00640 1 -- 172.21.15.86:0/1478385572 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f851c13bb70 con 0x7f851c1444f0 2024-03-22T04:12:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.325+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f851c068650 con 0x7f851c1444f0 2024-03-22T04:12:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.325+0000 7f8512a00640 1 -- 172.21.15.86:0/1478385572 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5189 (secure 0 0 0) 0x7f8514056410 con 0x7f851c1444f0 2024-03-22T04:12:44.330 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:44.330 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:12:33.981773+0000","modified":"2024-03-22T04:12:37.002106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75245":{"gid":75245,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6834/2728972599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2728972599},{"type":"v1","addr":"172.21.15.196:6834","nonce":2728972599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75245,"qdb_cluster":[75245]},"id":3}]} 2024-03-22T04:12:44.330 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84fc067b20 msgr2=0x7f84fc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84fc067b20 0x7f84fc069fe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f850c0af000 tx=0x7f850c0a7070 comp rx=0 tx=0).stop 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c1444f0 msgr2=0x7f851c13c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c1444f0 0x7f851c13c8f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f851400e7f0 tx=0x7f851400ecb0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 shutdown_connections 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f84fc067b20 0x7f84fc069fe0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c1444f0 0x7f851c13c8f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f851c0aff50 0x7f851c13c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 --2- 172.21.15.86:0/1478385572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f851c06cbb0 0x7f851c13a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 >> 172.21.15.86:0/1478385572 conn(0x7f851c071fe0 msgr2=0x7f851c06ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 shutdown_connections 2024-03-22T04:12:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.329+0000 7f8521c00640 1 -- 172.21.15.86:0/1478385572 wait complete. 2024-03-22T04:12:44.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:12:33.981773+0000', 'modified': '2024-03-22T04:12:37.002106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 75245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_75245': {'gid': 75245, 'name': 'd', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6834/2728972599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 2728972599}, {'type': 'v1', 'addr': '172.21.15.196:6834', 'nonce': 2728972599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 75245, 'qdb_cluster': [75245]} 2024-03-22T04:12:44.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:12:44.756 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-22T04:12:44.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.889+0000 7fe666c00640 1 Processor -- start 2024-03-22T04:12:44.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.889+0000 7fe666c00640 1 -- start start 2024-03-22T04:12:44.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe666c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe660144420 0x7fe66006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe666c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe66006cbb0 0x7fe66006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe666c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe660070210 0x7fe66015e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe666c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe66006d010 con 0x7fe66006cbb0 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe666c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe660144800 con 0x7fe660070210 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe666c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe66006f330 con 0x7fe660144420 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe66006cbb0 0x7fe66006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe66006cbb0 0x7fe66006fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51532/0 (socket says 172.21.15.86:51532) 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65f400640 1 -- 172.21.15.86:0/266436567 learned_addr learned my addr 172.21.15.86:0/266436567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe664c00640 1 --2- 172.21.15.86:0/266436567 >> v2:172.21.15.196:3300/0 conn(0x7fe660070210 0x7fe66015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65fe00640 1 --2- 172.21.15.86:0/266436567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe660144420 0x7fe66006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65f400640 1 -- 172.21.15.86:0/266436567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe660144420 msgr2=0x7fe66006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65f400640 1 --2- 172.21.15.86:0/266436567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe660144420 0x7fe66006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65f400640 1 -- 172.21.15.86:0/266436567 >> v2:172.21.15.196:3300/0 conn(0x7fe660070210 msgr2=0x7fe66015e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65f400640 1 --2- 172.21.15.86:0/266436567 >> v2:172.21.15.196:3300/0 conn(0x7fe660070210 0x7fe66015e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.893+0000 7fe65f400640 1 -- 172.21.15.86:0/266436567 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6600cf7e0 con 0x7fe66006cbb0 2024-03-22T04:12:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe65f400640 1 --2- 172.21.15.86:0/266436567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66006cbb0 0x7fe66006fcd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe660071f70 tx=0x7fe64c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33514ad16efd9bb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe65ea00640 1 -- 172.21.15.86:0/266436567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0b0070 con 0x7fe66006cbb0 2024-03-22T04:12:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe65ea00640 1 -- 172.21.15.86:0/266436567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe64c004230 con 0x7fe66006cbb0 2024-03-22T04:12:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe65ea00640 1 -- 172.21.15.86:0/266436567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c003620 con 0x7fe66006cbb0 2024-03-22T04:12:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 -- 172.21.15.86:0/266436567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66006cbb0 msgr2=0x7fe66006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 --2- 172.21.15.86:0/266436567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66006cbb0 0x7fe66006fcd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe660071f70 tx=0x7fe64c01c520 comp rx=0 tx=0).stop 2024-03-22T04:12:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 -- 172.21.15.86:0/266436567 shutdown_connections 2024-03-22T04:12:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 --2- 172.21.15.86:0/266436567 >> v2:172.21.15.196:3300/0 conn(0x7fe660070210 0x7fe66015e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 --2- 172.21.15.86:0/266436567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66006cbb0 0x7fe66006fcd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 --2- 172.21.15.86:0/266436567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe660144420 0x7fe66006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 -- 172.21.15.86:0/266436567 >> 172.21.15.86:0/266436567 conn(0x7fe660071fe0 msgr2=0x7fe6600723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 -- 172.21.15.86:0/266436567 shutdown_connections 2024-03-22T04:12:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 -- 172.21.15.86:0/266436567 wait complete. 2024-03-22T04:12:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 Processor -- start 2024-03-22T04:12:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.897+0000 7fe666c00640 1 -- start start 2024-03-22T04:12:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe666c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66006cbb0 0x7fe66013b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe666c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660070210 0x7fe6600b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe666c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe660144420 0x7fe6600b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe666c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe660070620 con 0x7fe660144420 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe666c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600cf2c0 con 0x7fe660070210 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe666c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600707a0 con 0x7fe66006cbb0 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe660144420 0x7fe6600b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe660144420 0x7fe6600b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51546/0 (socket says 172.21.15.86:51546) 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 -- 172.21.15.86:0/3261902306 learned_addr learned my addr 172.21.15.86:0/3261902306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe65f400640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660070210 0x7fe6600b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe65fe00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66006cbb0 0x7fe66013b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 -- 172.21.15.86:0/3261902306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66006cbb0 msgr2=0x7fe66013b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66006cbb0 0x7fe66013b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 -- 172.21.15.86:0/3261902306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660070210 msgr2=0x7fe6600b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660070210 0x7fe6600b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 -- 172.21.15.86:0/3261902306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6601407c0 con 0x7fe660144420 2024-03-22T04:12:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe664c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe660144420 0x7fe6600b1af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e850 tx=0x7fe65800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe657e00640 1 -- 172.21.15.86:0/3261902306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658017070 con 0x7fe660144420 2024-03-22T04:12:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe657e00640 1 -- 172.21.15.86:0/3261902306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65800c940 con 0x7fe660144420 2024-03-22T04:12:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe666c00640 1 -- 172.21.15.86:0/3261902306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe660140aa0 con 0x7fe660144420 2024-03-22T04:12:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe657e00640 1 -- 172.21.15.86:0/3261902306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65800cd10 con 0x7fe660144420 2024-03-22T04:12:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.901+0000 7fe666c00640 1 -- 172.21.15.86:0/3261902306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe660141060 con 0x7fe660144420 2024-03-22T04:12:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe657e00640 1 -- 172.21.15.86:0/3261902306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe65801a070 con 0x7fe660144420 2024-03-22T04:12:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe657e00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe657e00640 1 -- 172.21.15.86:0/3261902306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe658086290 con 0x7fe660144420 2024-03-22T04:12:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe65fe00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe66a04c000 1 Processor -- start 2024-03-22T04:12:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe66a04c000 1 -- start start 2024-03-22T04:12:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe65fe00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe6600b4560 tx=0x7fe65000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804555e70 0x555804556250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 0x5558045096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x55580450a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe66a04c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555804307140 con 0x555804509c20 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe66a04c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55580435ab10 con 0x555804555e70 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe66a04c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5558042e1c50 con 0x555804556790 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x55580450a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x55580450a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51556/0 (socket says 172.21.15.86:51556) 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804555e70 0x555804556250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 -- 172.21.15.86:0/297637807 learned_addr learned my addr 172.21.15.86:0/297637807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe65f400640 1 --2- 172.21.15.86:0/297637807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 0x5558045096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 -- 172.21.15.86:0/297637807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 msgr2=0x5558045096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 --2- 172.21.15.86:0/297637807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 0x5558045096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 -- 172.21.15.86:0/297637807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804555e70 msgr2=0x555804556250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 --2- 172.21.15.86:0/297637807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804555e70 0x555804556250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 -- 172.21.15.86:0/297637807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55580439ff20 con 0x555804509c20 2024-03-22T04:12:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe664c00640 1 --2- 172.21.15.86:0/297637807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x55580450a000 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe65805e8f0 tx=0x7fe658055fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598822230ad3298d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe654c00640 1 -- 172.21.15.86:0/297637807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658021040 con 0x555804509c20 2024-03-22T04:12:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe654c00640 1 -- 172.21.15.86:0/297637807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580545f0 con 0x555804509c20 2024-03-22T04:12:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.905+0000 7fe654c00640 1 -- 172.21.15.86:0/297637807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658052bc0 con 0x555804509c20 2024-03-22T04:12:44.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- 172.21.15.86:0/297637807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 msgr2=0x55580450a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 --2- 172.21.15.86:0/297637807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x55580450a000 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe65805e8f0 tx=0x7fe658055fb0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- 172.21.15.86:0/297637807 shutdown_connections 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 --2- 172.21.15.86:0/297637807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x55580450a000 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 --2- 172.21.15.86:0/297637807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 0x5558045096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 --2- 172.21.15.86:0/297637807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804555e70 0x555804556250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- 172.21.15.86:0/297637807 >> 172.21.15.86:0/297637807 conn(0x555804536c10 msgr2=0x555804539000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- 172.21.15.86:0/297637807 shutdown_connections 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- 172.21.15.86:0/297637807 wait complete. 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- start start 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 Processor -- start 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 0x555804511d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804555e70 0x55580450e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 0x55580450ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555804314ab0 con 0x555804555e70 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555804356ac0 con 0x555804509c20 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x555804300e80 con 0x555804556790 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 10 client.0 _ll_register_callbacks cb 0x5558044f7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804555e70 0x55580450e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804555e70 0x55580450e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51560/0 (socket says 172.21.15.86:51560) 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 -- 172.21.15.86:0/3706696532 learned_addr learned my addr 172.21.15.86:0/3706696532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65fe00640 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 0x555804511d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe664c00640 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 0x55580450ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 -- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 msgr2=0x55580450ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 0x55580450ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 -- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 msgr2=0x555804511d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 0x555804511d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 -- 172.21.15.86:0/3706696532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555804330570 con 0x555804555e70 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe65f400640 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804555e70 0x55580450e7e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009870 tx=0x7fe64c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476837e5f8bc2a54 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe649600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:12:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe649600640 1 -- 172.21.15.86:0/3706696532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0bc070 con 0x555804555e70 2024-03-22T04:12:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe649600640 1 -- 172.21.15.86:0/3706696532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe64c0043f0 con 0x555804555e70 2024-03-22T04:12:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- 172.21.15.86:0/3706696532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6580545f0 con 0x555804555e70 2024-03-22T04:12:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe649600640 1 -- 172.21.15.86:0/3706696532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe64c0c9050 con 0x555804555e70 2024-03-22T04:12:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 10 client.5907 fetch_fsmap learned FSMap version 26 2024-03-22T04:12:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- 172.21.15.86:0/3706696532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x555804326880 con 0x555804555e70 2024-03-22T04:12:44.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe649600640 1 -- 172.21.15.86:0/3706696532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580545f0 con 0x555804555e70 2024-03-22T04:12:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe649600640 1 -- 172.21.15.86:0/3706696532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fe64c0cc020 con 0x555804555e70 2024-03-22T04:12:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 10 client.5907 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T04:12:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 10 client.5907 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:12:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 20 client.5907 populate_metadata read hostname 'smithi086' 2024-03-22T04:12:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55580450fac0 0x555804567120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 4 client.5907 mds_command: new command op to 75245 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:12:44.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.909+0000 7fe66a04c000 1 -- 172.21.15.86:0/3706696532 --> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558042f5bc0 con 0x55580450fac0 2024-03-22T04:12:44.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe65fe00640 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55580450fac0 0x555804567120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe649600640 10 client.5907 ms_handle_connect on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe65fe00640 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55580450fac0 0x555804567120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f01bbdfb9a504be server_cookie=2d4a5e40a5baf1e6 in_seq=0 out_seq=0 2024-03-22T04:12:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe649600640 1 -- 172.21.15.86:0/3706696532 <== mds.0 v2:172.21.15.196:6832/2728972599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5558042f5bc0 con 0x55580450fac0 2024-03-22T04:12:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe649600640 10 client.5907 handle_command_reply: tid=0 2024-03-22T04:12:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 client.5907 shutdown 2024-03-22T04:12:44.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 -- 172.21.15.86:0/3706696532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804555e70 msgr2=0x55580450e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804555e70 0x55580450e7e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009870 tx=0x7fe64c01ad60 comp rx=0 tx=0).stop 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 -- 172.21.15.86:0/3706696532 shutdown_connections 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55580450fac0 0x555804567120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804556790 0x55580450ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804555e70 0x55580450e7e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe649600640 0 client.5907 ms_handle_reset on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 --2- 172.21.15.86:0/3706696532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 0x555804511d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 -- 172.21.15.86:0/3706696532 >> 172.21.15.86:0/3706696532 conn(0x555804536c10 msgr2=0x555804539000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 -- 172.21.15.86:0/3706696532 shutdown_connections 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 1 -- 172.21.15.86:0/3706696532 wait complete. 2024-03-22T04:12:44.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.913+0000 7fe66a04c000 20 client.5907 trim_cache size 0 max 16384 2024-03-22T04:12:44.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 Processor -- start 2024-03-22T04:12:44.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- start start 2024-03-22T04:12:44.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804536c10 0x555804555350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 0x555804555890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x55580450e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe660203d10 con 0x555804536c10 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe660202d90 con 0x555804509c20 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe6601426c0 con 0x555804555e70 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804536c10 0x555804555350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804536c10 0x555804555350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51566/0 (socket says 172.21.15.86:51566) 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 -- 172.21.15.86:0/159464252 learned_addr learned my addr 172.21.15.86:0/159464252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65f400640 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 0x555804555890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe664c00640 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x55580450e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 -- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 msgr2=0x55580450e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x55580450e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 -- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 msgr2=0x555804555890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 0x555804555890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 -- 172.21.15.86:0/159464252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558042e2e80 con 0x555804536c10 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe664c00640 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x55580450e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe65fe00640 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804536c10 0x555804555350 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x555804358160 tx=0x7fe650018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aad60ba2e9f2b7e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe64be00640 1 -- 172.21.15.86:0/159464252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65001d040 con 0x555804536c10 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe64be00640 1 -- 172.21.15.86:0/159464252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe650031070 con 0x555804536c10 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe64be00640 1 -- 172.21.15.86:0/159464252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65002ca30 con 0x555804536c10 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- 172.21.15.86:0/159464252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804536c10 msgr2=0x555804555350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804536c10 0x555804555350 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x555804358160 tx=0x7fe650018000 comp rx=0 tx=0).stop 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- 172.21.15.86:0/159464252 shutdown_connections 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804509c20 0x555804555890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x55580450e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 --2- 172.21.15.86:0/159464252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804536c10 0x555804555350 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- 172.21.15.86:0/159464252 >> 172.21.15.86:0/159464252 conn(0x555804510f40 msgr2=0x555804508d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:44.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- 172.21.15.86:0/159464252 shutdown_connections 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- 172.21.15.86:0/159464252 wait complete. 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 -- start start 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.933+0000 7fe66a04c000 1 Processor -- start 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x5558045c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804536c10 0x5558045c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x555804567ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555804359e20 con 0x555804509c20 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe660202740 con 0x555804536c10 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55580439aa60 con 0x555804555e70 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 10 client.0 _ll_register_callbacks cb 0x7fe6500095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x5558045c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x5558045c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51570/0 (socket says 172.21.15.86:51570) 2024-03-22T04:12:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 -- 172.21.15.86:0/4198323566 learned_addr learned my addr 172.21.15.86:0/4198323566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe664c00640 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x555804567ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65f400640 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804536c10 0x5558045c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 -- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 msgr2=0x555804567ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x555804567ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 -- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804536c10 msgr2=0x5558045c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804536c10 0x5558045c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 -- 172.21.15.86:0/4198323566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558042ee750 con 0x555804509c20 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65fe00640 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x5558045c5940 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe65000b010 tx=0x7fe650004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac5a146fea8a2b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 1 -- 172.21.15.86:0/4198323566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65001d050 con 0x555804509c20 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 1 -- 172.21.15.86:0/4198323566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe65002cc40 con 0x555804509c20 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- 172.21.15.86:0/4198323566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe650031070 con 0x555804509c20 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 1 -- 172.21.15.86:0/4198323566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe650042050 con 0x555804509c20 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 10 client.5913 fetch_fsmap learned FSMap version 26 2024-03-22T04:12:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- 172.21.15.86:0/4198323566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x555804301450 con 0x555804509c20 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 1 -- 172.21.15.86:0/4198323566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe650031070 con 0x555804509c20 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 1 -- 172.21.15.86:0/4198323566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fe650045020 con 0x555804509c20 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 10 client.5913 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 10 client.5913 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 20 client.5913 populate_metadata read hostname 'smithi086' 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x555804556790 0x5558045686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 4 client.5913 mds_command: new command op to 75245 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- 172.21.15.86:0/4198323566 --> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] -- command(tid 0: {"prefix": "status"}) -- 0x555804368050 con 0x555804556790 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65f400640 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x555804556790 0x5558045686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe65f400640 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x555804556790 0x5558045686a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1865723076c22e32 server_cookie=ae692bad3bc2e719 in_seq=0 out_seq=0 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 10 client.5913 ms_handle_connect on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 1 -- 172.21.15.86:0/4198323566 <== mds.0 v2:172.21.15.196:6832/2728972599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x555804368050 con 0x555804556790 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 10 client.5913 handle_command_reply: tid=0 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 client.5913 shutdown 2024-03-22T04:12:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- 172.21.15.86:0/4198323566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 msgr2=0x5558045c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x5558045c5940 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe65000b010 tx=0x7fe650004a50 comp rx=0 tx=0).stop 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- 172.21.15.86:0/4198323566 shutdown_connections 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x555804556790 0x5558045686a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555804536c10 0x5558045c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555804555e70 0x555804567ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 --2- 172.21.15.86:0/4198323566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555804509c20 0x5558045c5940 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe649600640 0 client.5913 ms_handle_reset on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- 172.21.15.86:0/4198323566 >> 172.21.15.86:0/4198323566 conn(0x555804510f40 msgr2=0x555804553f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- 172.21.15.86:0/4198323566 shutdown_connections 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 1 -- 172.21.15.86:0/4198323566 wait complete. 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.937+0000 7fe66a04c000 20 client.5913 trim_cache size 0 max 16384 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 75245, 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:12:44.950 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.313313716, 2024-03-22T04:12:44.951 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 26, 2024-03-22T04:12:44.951 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 54, 2024-03-22T04:12:44.951 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 54, 2024-03-22T04:12:44.951 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.158431204999999 2024-03-22T04:12:44.951 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 -- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 msgr2=0x7fe640069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe6600b4560 tx=0x7fe65000b040 comp rx=0 tx=0).stop 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 -- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe660144420 msgr2=0x7fe6600b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe660144420 0x7fe6600b1af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e850 tx=0x7fe65800ed10 comp rx=0 tx=0).stop 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 -- 172.21.15.86:0/3261902306 shutdown_connections 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660070210 0x7fe6600b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe660144420 0x7fe6600b1af0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 --2- 172.21.15.86:0/3261902306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66006cbb0 0x7fe66013b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 -- 172.21.15.86:0/3261902306 >> 172.21.15.86:0/3261902306 conn(0x7fe660071fe0 msgr2=0x7fe66013f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 -- 172.21.15.86:0/3261902306 shutdown_connections 2024-03-22T04:12:44.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:44.941+0000 7fe666c00640 1 -- 172.21.15.86:0/3261902306 wait complete. 2024-03-22T04:12:44.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:12:45.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.093+0000 7efc1fe00640 1 Processor -- start 2024-03-22T04:12:45.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.093+0000 7efc1fe00640 1 -- start start 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc1806df90 0x7efc1806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1815ac10 0x7efc180b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efc1815bc80 0x7efc1815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efc1815cb60 con 0x7efc1806df90 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1fe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efc1815c550 con 0x7efc1815bc80 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efc1806e8b0 con 0x7efc1815ac10 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc1806df90 0x7efc1806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1de00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efc1815bc80 0x7efc1815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1ca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1815ac10 0x7efc180b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc1806df90 0x7efc1806e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51584/0 (socket says 172.21.15.86:51584) 2024-03-22T04:12:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1d400640 1 -- 172.21.15.86:0/3540219910 learned_addr learned my addr 172.21.15.86:0/3540219910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:45.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1d400640 1 -- 172.21.15.86:0/3540219910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1815ac10 msgr2=0x7efc180b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:45.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1d400640 1 --2- 172.21.15.86:0/3540219910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1815ac10 0x7efc180b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1d400640 1 -- 172.21.15.86:0/3540219910 >> v2:172.21.15.196:3300/0 conn(0x7efc1815bc80 msgr2=0x7efc1815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:45.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1d400640 1 --2- 172.21.15.86:0/3540219910 >> v2:172.21.15.196:3300/0 conn(0x7efc1815bc80 0x7efc1815e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.097+0000 7efc1d400640 1 -- 172.21.15.86:0/3540219910 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc180b4750 con 0x7efc1806df90 2024-03-22T04:12:45.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1d400640 1 --2- 172.21.15.86:0/3540219910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1806df90 0x7efc1806e370 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efc18073b80 tx=0x7efc0c01cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7f81aa7c3826db server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:45.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc17e00640 1 -- 172.21.15.86:0/3540219910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0ab070 con 0x7efc1806df90 2024-03-22T04:12:45.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc17e00640 1 -- 172.21.15.86:0/3540219910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc0c0a5be0 con 0x7efc1806df90 2024-03-22T04:12:45.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc17e00640 1 -- 172.21.15.86:0/3540219910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c004230 con 0x7efc1806df90 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- 172.21.15.86:0/3540219910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1806df90 msgr2=0x7efc1806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 --2- 172.21.15.86:0/3540219910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1806df90 0x7efc1806e370 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efc18073b80 tx=0x7efc0c01cbb0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- 172.21.15.86:0/3540219910 shutdown_connections 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 --2- 172.21.15.86:0/3540219910 >> v2:172.21.15.196:3300/0 conn(0x7efc1815bc80 0x7efc1815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 --2- 172.21.15.86:0/3540219910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1815ac10 0x7efc180b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 --2- 172.21.15.86:0/3540219910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1806df90 0x7efc1806e370 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- 172.21.15.86:0/3540219910 >> 172.21.15.86:0/3540219910 conn(0x7efc180b1df0 msgr2=0x7efc180b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- 172.21.15.86:0/3540219910 shutdown_connections 2024-03-22T04:12:45.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- 172.21.15.86:0/3540219910 wait complete. 2024-03-22T04:12:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 Processor -- start 2024-03-22T04:12:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- start start 2024-03-22T04:12:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1806df90 0x7efc180cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc1815ac10 0x7efc1809caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1815bc80 0x7efc1815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc1815ec90 con 0x7efc1815bc80 2024-03-22T04:12:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc1815eb10 con 0x7efc1815ac10 2024-03-22T04:12:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.101+0000 7efc1fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efc1815ee10 con 0x7efc1806df90 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1815bc80 0x7efc1815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1815bc80 0x7efc1815b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51592/0 (socket says 172.21.15.86:51592) 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 -- 172.21.15.86:0/913030574 learned_addr learned my addr 172.21.15.86:0/913030574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1ca00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc1815ac10 0x7efc1809caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1d400640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1806df90 0x7efc180cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 -- 172.21.15.86:0/913030574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1806df90 msgr2=0x7efc180cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1806df90 0x7efc180cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 -- 172.21.15.86:0/913030574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc1815ac10 msgr2=0x7efc1809caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc1815ac10 0x7efc1809caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 -- 172.21.15.86:0/913030574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1809de30 con 0x7efc1815bc80 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1de00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1815bc80 0x7efc1815b010 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efc10009d30 tx=0x7efc1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc15600640 1 -- 172.21.15.86:0/913030574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10018020 con 0x7efc1815bc80 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc15600640 1 -- 172.21.15.86:0/913030574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc10014480 con 0x7efc1815bc80 2024-03-22T04:12:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc15600640 1 -- 172.21.15.86:0/913030574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10014870 con 0x7efc1815bc80 2024-03-22T04:12:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1809d590 con 0x7efc1815bc80 2024-03-22T04:12:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1809db50 con 0x7efc1815bc80 2024-03-22T04:12:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc18068650 con 0x7efc1815bc80 2024-03-22T04:12:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc15600640 1 -- 172.21.15.86:0/913030574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efc1001b020 con 0x7efc1815bc80 2024-03-22T04:12:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc15600640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efbf8067b20 0x7efbf8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.105+0000 7efc15600640 1 -- 172.21.15.86:0/913030574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7efc10086950 con 0x7efc1815bc80 2024-03-22T04:12:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.109+0000 7efc1d400640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efbf8067b20 0x7efbf8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.109+0000 7efc1d400640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efbf8067b20 0x7efbf8069fe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0af000 tx=0x7efc0c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.113+0000 7efc15600640 1 -- 172.21.15.86:0/913030574 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efc10022070 con 0x7efc1815bc80 2024-03-22T04:12:45.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.297+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efc180b39e0 con 0x7efc1815bc80 2024-03-22T04:12:45.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.297+0000 7efc15600640 1 -- 172.21.15.86:0/913030574 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7efc10010040 con 0x7efc1815bc80 2024-03-22T04:12:45.302 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:12:45.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efbf8067b20 msgr2=0x7efbf8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:45.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efbf8067b20 0x7efbf8069fe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0af000 tx=0x7efc0c0a7070 comp rx=0 tx=0).stop 2024-03-22T04:12:45.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1815bc80 msgr2=0x7efc1815b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:45.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1815bc80 0x7efc1815b010 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efc10009d30 tx=0x7efc1000c490 comp rx=0 tx=0).stop 2024-03-22T04:12:45.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 shutdown_connections 2024-03-22T04:12:45.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efbf8067b20 0x7efbf8069fe0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc1815ac10 0x7efc1809caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc1815bc80 0x7efc1815b010 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 --2- 172.21.15.86:0/913030574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc1806df90 0x7efc180cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:45.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 >> 172.21.15.86:0/913030574 conn(0x7efc180b1df0 msgr2=0x7efc180af480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:45.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 shutdown_connections 2024-03-22T04:12:45.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45.301+0000 7efc1fe00640 1 -- 172.21.15.86:0/913030574 wait complete. 2024-03-22T04:12:45.323 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:12:45.323 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:12:45.323 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:12:45.323 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:12:45.323 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:45.323 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:12:45.323 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:12:45.323 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:12:45.324 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:12:45.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:45.324 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:12:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:12:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:12:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:12:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:12:45.334 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42366sec preferred_lft 42366sec 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:12:45.335 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:12:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:12:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:12:45.337 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:12:45.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:45.337 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:12:45.392 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:12:45.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:45.394 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:12:45.446 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:12:45.447 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:12:45.447 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:12:45.447 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:12:45.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:12:45.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:12:45.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:12:45.447 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:12:45.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:12:45.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:45.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:45.583 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:12:45.593 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:12:45.593 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:12:45.594 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:12:45.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:45.594 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:12:45.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:12:45.654 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:12:45.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:12:45.702 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:45.754 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:12:45.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:45.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:12:45.846 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:12:45.847 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:12:45.847 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:12:45.862 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:12:45.862 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:12:45.862 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:12:45.862 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:12:45.862 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:12:45.870 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:12:45.870 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:12:45.871 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:12:45.871 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:12:45.871 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:12:45.871 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:12:45.871 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:12:45.871 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:12:45.874 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:12:45.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:45.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:12:45.896 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:12:45.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:12:45.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:46.001 INFO:teuthology.orchestra.run.smithi086.stdout:client5922 2024-03-22T04:12:46.002 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:12:46.002 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:12:46.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:46.011 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-03-22T04:12:46.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:46.073 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-03-22T04:12:46.124 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:12:46.124 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:12:46.124 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:12:46.125 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:12:46.125 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-03-22T04:12:46.125 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:12:46.229 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-03-22T04:12:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:12:46.238 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:12:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e8c00640 1 Processor -- start 2024-03-22T04:12:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e8c00640 1 -- start start 2024-03-22T04:12:46.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e8c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e40d0580 0x7fa9e4071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e8c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa9e406e2f0 0x7fa9e4071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e41623d0 con 0x7fa9e415e630 2024-03-22T04:12:46.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e8c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e41620f0 con 0x7fa9e406e2f0 2024-03-22T04:12:46.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e8c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e415ec10 con 0x7fa9e40d0580 2024-03-22T04:12:46.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa9e415e630 0x7fa9e40cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39060/0 (socket says 172.21.15.86:39060) 2024-03-22T04:12:46.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e1e00640 1 -- 172.21.15.86:0/1164504335 learned_addr learned my addr 172.21.15.86:0/1164504335 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e2800640 1 --2- 172.21.15.86:0/1164504335 >> v2:172.21.15.196:3300/0 conn(0x7fa9e406e2f0 0x7fa9e4071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9dbe00640 1 --2- 172.21.15.86:0/1164504335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e40d0580 0x7fa9e4071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e1400640 1 -- 172.21.15.86:0/1164504335 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235933665 0 0) 0x7fa9e41623d0 con 0x7fa9e415e630 2024-03-22T04:12:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9e1400640 1 -- 172.21.15.86:0/1164504335 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9c8003530 con 0x7fa9e415e630 2024-03-22T04:12:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9dbe00640 1 -- 172.21.15.86:0/1164504335 >> v2:172.21.15.196:3300/0 conn(0x7fa9e406e2f0 msgr2=0x7fa9e4071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9dbe00640 1 --2- 172.21.15.86:0/1164504335 >> v2:172.21.15.196:3300/0 conn(0x7fa9e406e2f0 0x7fa9e4071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9dbe00640 1 -- 172.21.15.86:0/1164504335 >> v1:172.21.15.86:6789/0 conn(0x7fa9e415e630 legacy=0x7fa9e40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.413+0000 7fa9dbe00640 1 -- 172.21.15.86:0/1164504335 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e415eef0 con 0x7fa9e40d0580 2024-03-22T04:12:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9dbe00640 1 --2- 172.21.15.86:0/1164504335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e40d0580 0x7fa9e4071100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0097c0 tx=0x7fa9cc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d3bfc9925b51f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e1400640 1 -- 172.21.15.86:0/1164504335 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0b0070 con 0x7fa9e40d0580 2024-03-22T04:12:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e1400640 1 -- 172.21.15.86:0/1164504335 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0ab600 con 0x7fa9e40d0580 2024-03-22T04:12:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e1400640 1 -- 172.21.15.86:0/1164504335 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0ab9d0 con 0x7fa9e40d0580 2024-03-22T04:12:46.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1164504335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e40d0580 msgr2=0x7fa9e4071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1164504335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e40d0580 0x7fa9e4071100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0097c0 tx=0x7fa9cc01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1164504335 shutdown_connections 2024-03-22T04:12:46.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1164504335 >> v2:172.21.15.196:3300/0 conn(0x7fa9e406e2f0 0x7fa9e4071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1164504335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e40d0580 0x7fa9e4071100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1164504335 >> 172.21.15.86:0/1164504335 conn(0x7fa9e406f910 msgr2=0x7fa9e406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:46.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1164504335 shutdown_connections 2024-03-22T04:12:46.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.417+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1164504335 wait complete. 2024-03-22T04:12:46.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 Processor -- start 2024-03-22T04:12:46.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 -- start start 2024-03-22T04:12:46.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e406e2f0 0x7fa9e4155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e40d0580 0x7fa9e40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e415e630 0x7fa9e40b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4071f90 con 0x7fa9e40d0580 2024-03-22T04:12:46.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e415f9d0 con 0x7fa9e406e2f0 2024-03-22T04:12:46.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e40cfaf0 con 0x7fa9e415e630 2024-03-22T04:12:46.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e40d0580 0x7fa9e40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e40d0580 0x7fa9e40b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49484/0 (socket says 172.21.15.86:49484) 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 -- 172.21.15.86:0/1118703003 learned_addr learned my addr 172.21.15.86:0/1118703003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e1e00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e406e2f0 0x7fa9e4155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e2800640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e415e630 0x7fa9e40b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 -- 172.21.15.86:0/1118703003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e415e630 msgr2=0x7fa9e40b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e415e630 0x7fa9e40b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 -- 172.21.15.86:0/1118703003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e406e2f0 msgr2=0x7fa9e4155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e406e2f0 0x7fa9e4155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 -- 172.21.15.86:0/1118703003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e40b54a0 con 0x7fa9e40d0580 2024-03-22T04:12:46.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9dbe00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e40d0580 0x7fa9e40b4f60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc01a0d0 tx=0x7fa9cc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:46.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9da000640 1 -- 172.21.15.86:0/1118703003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0b0040 con 0x7fa9e40d0580 2024-03-22T04:12:46.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9da000640 1 -- 172.21.15.86:0/1118703003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0b9070 con 0x7fa9e40d0580 2024-03-22T04:12:46.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9da000640 1 -- 172.21.15.86:0/1118703003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0b4650 con 0x7fa9e40d0580 2024-03-22T04:12:46.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e40b5770 con 0x7fa9e40d0580 2024-03-22T04:12:46.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.421+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e40b5d30 con 0x7fa9e40d0580 2024-03-22T04:12:46.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.425+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e4068650 con 0x7fa9e40d0580 2024-03-22T04:12:46.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.425+0000 7fa9da000640 1 -- 172.21.15.86:0/1118703003 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa9cc0b47f0 con 0x7fa9e40d0580 2024-03-22T04:12:46.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.425+0000 7fa9da000640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa9c0067b20 0x7fa9c0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.425+0000 7fa9da000640 1 -- 172.21.15.86:0/1118703003 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa9cc11fc10 con 0x7fa9e40d0580 2024-03-22T04:12:46.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.425+0000 7fa9e1e00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa9c0067b20 0x7fa9c0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.425+0000 7fa9e1e00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa9c0067b20 0x7fa9c0069fe0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007fd0 tx=0x7fa9d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:46.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.429+0000 7fa9da000640 1 -- 172.21.15.86:0/1118703003 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa9cc016090 con 0x7fa9e40d0580 2024-03-22T04:12:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9e40715f0 con 0x7fa9e40d0580 2024-03-22T04:12:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9da000640 1 -- 172.21.15.86:0/1118703003 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5189 (secure 0 0 0) 0x7fa9cc0efe30 con 0x7fa9e40d0580 2024-03-22T04:12:46.620 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:46.620 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:12:33.981773+0000","modified":"2024-03-22T04:12:37.002106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75245":{"gid":75245,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6834/2728972599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2728972599},{"type":"v1","addr":"172.21.15.196:6834","nonce":2728972599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75245,"qdb_cluster":[75245]},"id":3}]} 2024-03-22T04:12:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T04:12:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa9c0067b20 msgr2=0x7fa9c0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa9c0067b20 0x7fa9c0069fe0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007fd0 tx=0x7fa9d4009040 comp rx=0 tx=0).stop 2024-03-22T04:12:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e40d0580 msgr2=0x7fa9e40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e40d0580 0x7fa9e40b4f60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc01a0d0 tx=0x7fa9cc019240 comp rx=0 tx=0).stop 2024-03-22T04:12:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 shutdown_connections 2024-03-22T04:12:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa9c0067b20 0x7fa9c0069fe0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9e415e630 0x7fa9e40b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e40d0580 0x7fa9e40b4f60 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 --2- 172.21.15.86:0/1118703003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e406e2f0 0x7fa9e4155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 >> 172.21.15.86:0/1118703003 conn(0x7fa9e406f910 msgr2=0x7fa9e41604f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 shutdown_connections 2024-03-22T04:12:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.617+0000 7fa9e8c00640 1 -- 172.21.15.86:0/1118703003 wait complete. 2024-03-22T04:12:46.641 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-03-22T04:12:46.811 INFO:teuthology.orchestra.run.smithi196.stdout:{"message":"","return_code":0} 2024-03-22T04:12:46.811 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:12:46.811 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-22T04:12:46.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.853+0000 7f305888e080 1 Processor -- start 2024-03-22T04:12:46.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.853+0000 7f305888e080 1 -- start start 2024-03-22T04:12:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.853+0000 7f305888e080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5a4f410 0x5597c5a1e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.853+0000 7f305888e080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 0x5597c5aed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.853+0000 7f305888e080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5af8a00 0x5597c5af5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.853+0000 7f305888e080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597c59c6620 con 0x5597c5a4f410 2024-03-22T04:12:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.853+0000 7f305888e080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5597c59ca670 con 0x5597c5af7610 2024-03-22T04:12:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.853+0000 7f305888e080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5597c5a6b740 con 0x5597c5af8a00 2024-03-22T04:12:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5a4f410 0x5597c5a1e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5a4f410 0x5597c5a1e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49500/0 (socket says 172.21.15.86:49500) 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 -- 172.21.15.86:0/490797428 learned_addr learned my addr 172.21.15.86:0/490797428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5af8a00 0x5597c5af5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055000640 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 0x5597c5aed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 -- 172.21.15.86:0/490797428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5af8a00 msgr2=0x5597c5af5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5af8a00 0x5597c5af5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 -- 172.21.15.86:0/490797428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 msgr2=0x5597c5aed570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 0x5597c5aed570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 -- 172.21.15.86:0/490797428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c5afa7d0 con 0x5597c5a4f410 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5a4f410 0x5597c5a1e7e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f304800b300 tx=0x7f304800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897eb01492bbbc01 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f304f400640 1 -- 172.21.15.86:0/490797428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304800a040 con 0x5597c5a4f410 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f304f400640 1 -- 172.21.15.86:0/490797428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304800f620 con 0x5597c5a4f410 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f304f400640 1 -- 172.21.15.86:0/490797428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304800f9f0 con 0x5597c5a4f410 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- 172.21.15.86:0/490797428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5a4f410 msgr2=0x5597c5a1e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5a4f410 0x5597c5a1e7e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f304800b300 tx=0x7f304800b7c0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- 172.21.15.86:0/490797428 shutdown_connections 2024-03-22T04:12:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 0x5597c5aed570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5af8a00 0x5597c5af5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 --2- 172.21.15.86:0/490797428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5a4f410 0x5597c5a1e7e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- 172.21.15.86:0/490797428 >> 172.21.15.86:0/490797428 conn(0x5597c5a4dff0 msgr2=0x5597c5a52fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- 172.21.15.86:0/490797428 shutdown_connections 2024-03-22T04:12:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- 172.21.15.86:0/490797428 wait complete. 2024-03-22T04:12:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 Processor -- start 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- start start 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5a4f410 0x5597c5ae7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 0x5597c5af50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5af8a00 0x5597c5af5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597c5afaa70 con 0x5597c5af8a00 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5597c5afad10 con 0x5597c5af7610 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f305888e080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5597c5af6a10 con 0x5597c5a4f410 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5af8a00 0x5597c5af5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5af8a00 0x5597c5af5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49516/0 (socket says 172.21.15.86:49516) 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 -- 172.21.15.86:0/2015213726 learned_addr learned my addr 172.21.15.86:0/2015213726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055000640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 0x5597c5af50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3055a00640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5a4f410 0x5597c5ae7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 -- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5a4f410 msgr2=0x5597c5ae7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5597c5a4f410 0x5597c5ae7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 -- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 msgr2=0x5597c5af50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597c5af7610 0x5597c5af50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.857+0000 7f3056400640 1 -- 172.21.15.86:0/2015213726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c5ad11c0 con 0x5597c5af8a00 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f3056400640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5af8a00 0x5597c5af5600 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f305000a4a0 tx=0x7f30500198b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f304e000640 1 -- 172.21.15.86:0/2015213726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500b0040 con 0x5597c5af8a00 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f304e000640 1 -- 172.21.15.86:0/2015213726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30500bc070 con 0x5597c5af8a00 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f304e000640 1 -- 172.21.15.86:0/2015213726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500abcd0 con 0x5597c5af8a00 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5597c5ad1460 con 0x5597c5af8a00 2024-03-22T04:12:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597c5ad19a0 con 0x5597c5af8a00 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f304e000640 1 -- 172.21.15.86:0/2015213726 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f30500c1460 con 0x5597c5af8a00 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f304e000640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3038066530 0x7f30380689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f304e000640 1 -- 172.21.15.86:0/2015213726 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f305011f270 con 0x5597c5af8a00 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f3055a00640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3038066530 0x7f30380689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f305888e080 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5597c5ae7ad0 0x5597c5ae7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.861+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) -- 0x5597c5ae83f0 con 0x5597c5ae7ad0 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f3055000640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5597c5ae7ad0 0x5597c5ae7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f3055a00640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3038066530 0x7f30380689f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f304800b300 tx=0x7f3048009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f3055000640 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5597c5ae7ad0 0x5597c5ae7eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:46.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f3055000640 1 -- 172.21.15.86:0/2015213726 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f304400e750 con 0x5597c5ae7ad0 2024-03-22T04:12:46.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5597c5ae7ad0 msgr2=0x5597c5ae7eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5597c5ae7ad0 0x5597c5ae7eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3038066530 msgr2=0x7f30380689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3038066530 0x7f30380689f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f304800b300 tx=0x7f3048009000 comp rx=0 tx=0).stop 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5af8a00 msgr2=0x5597c5af5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 --2- 172.21.15.86:0/2015213726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597c5af8a00 0x5597c5af5600 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f305000a4a0 tx=0x7f30500198b0 comp rx=0 tx=0).stop 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f3056400640 1 -- 172.21.15.86:0/2015213726 reap_dead start 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 shutdown_connections 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 >> 172.21.15.86:0/2015213726 conn(0x5597c5a4dff0 msgr2=0x5597c5a207f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 shutdown_connections 2024-03-22T04:12:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:46.865+0000 7f305888e080 1 -- 172.21.15.86:0/2015213726 wait complete. 2024-03-22T04:12:46.872 INFO:teuthology.orchestra.run.smithi086.stdout:Yalphaparent_a 2024-03-22T04:12:46.872 DEBUG:teuthology.orchestra.run.smithi086:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-22T04:12:46.999 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:12:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-03-22T04:12:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "ancestors": [ 2024-03-22T04:12:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1099511627776, 2024-03-22T04:12:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "alpha", 2024-03-22T04:12:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 2 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1, 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "parent_a", 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 8, 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: "old_pools": [] 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:12:47.000 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:47.003 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-03-22T04:12:47.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f824109b080 1 Processor -- start 2024-03-22T04:12:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f824109b080 1 -- start start 2024-03-22T04:12:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f824109b080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d99e400 0x55ae8da25860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f824109b080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8d9a18c0 0x55ae8d9a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f824109b080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8da46600 0x55ae8da491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f824109b080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae8d9ba730 con 0x55ae8d99e400 2024-03-22T04:12:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f824109b080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae8d915620 con 0x55ae8da46600 2024-03-22T04:12:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f824109b080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ae8d9ba9d0 con 0x55ae8d9a18c0 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d99e400 0x55ae8da25860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823d800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8d9a18c0 0x55ae8d9a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d99e400 0x55ae8da25860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49520/0 (socket says 172.21.15.86:49520) 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 -- 172.21.15.86:0/2116392268 learned_addr learned my addr 172.21.15.86:0/2116392268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823ec00640 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8da46600 0x55ae8da491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 -- 172.21.15.86:0/2116392268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8d9a18c0 msgr2=0x55ae8d9a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8d9a18c0 0x55ae8d9a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 -- 172.21.15.86:0/2116392268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8da46600 msgr2=0x55ae8da491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8da46600 0x55ae8da491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 -- 172.21.15.86:0/2116392268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8da482a0 con 0x55ae8d99e400 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823d800640 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8d9a18c0 0x55ae8d9a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.089+0000 7f823e200640 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d99e400 0x55ae8da25860 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f82300ad040 tx=0x7f8230003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c651b77f27dccc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f8237e00640 1 -- 172.21.15.86:0/2116392268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82300af040 con 0x55ae8d99e400 2024-03-22T04:12:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f8237e00640 1 -- 172.21.15.86:0/2116392268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82300b9070 con 0x55ae8d99e400 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f8237e00640 1 -- 172.21.15.86:0/2116392268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82300aad90 con 0x55ae8d99e400 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- 172.21.15.86:0/2116392268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d99e400 msgr2=0x55ae8da25860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d99e400 0x55ae8da25860 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f82300ad040 tx=0x7f8230003030 comp rx=0 tx=0).stop 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- 172.21.15.86:0/2116392268 shutdown_connections 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8da46600 0x55ae8da491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8d9a18c0 0x55ae8d9a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 --2- 172.21.15.86:0/2116392268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d99e400 0x55ae8da25860 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- 172.21.15.86:0/2116392268 >> 172.21.15.86:0/2116392268 conn(0x55ae8d99cfe0 msgr2=0x55ae8d96f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- 172.21.15.86:0/2116392268 shutdown_connections 2024-03-22T04:12:47.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- 172.21.15.86:0/2116392268 wait complete. 2024-03-22T04:12:47.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 Processor -- start 2024-03-22T04:12:47.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- start start 2024-03-22T04:12:47.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8d9a18c0 0x55ae8dae2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8da46600 0x55ae8d9b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d9b2c80 0x55ae8dadfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae8da487e0 con 0x55ae8d9b2c80 2024-03-22T04:12:47.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae8da485a0 con 0x55ae8d9a18c0 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ae8da48a80 con 0x55ae8da46600 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d9b2c80 0x55ae8dadfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8d9a18c0 0x55ae8dae2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8d9a18c0 0x55ae8dae2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:39174/0 (socket says 172.21.15.86:39174) 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d9b2c80 0x55ae8dadfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49528/0 (socket says 172.21.15.86:49528) 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823e200640 1 -- 172.21.15.86:0/2666226733 learned_addr learned my addr 172.21.15.86:0/2666226733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823d800640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8da46600 0x55ae8d9b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823ec00640 1 -- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8da46600 msgr2=0x55ae8d9b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823ec00640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae8da46600 0x55ae8d9b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823ec00640 1 -- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8d9a18c0 msgr2=0x55ae8dae2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823ec00640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae8d9a18c0 0x55ae8dae2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823ec00640 1 -- 172.21.15.86:0/2666226733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8da42b20 con 0x55ae8d9b2c80 2024-03-22T04:12:47.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f823ec00640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d9b2c80 0x55ae8dadfd00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8238003ec0 tx=0x7f82380018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f8236a00640 1 -- 172.21.15.86:0/2666226733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82380020c0 con 0x55ae8d9b2c80 2024-03-22T04:12:47.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f8236a00640 1 -- 172.21.15.86:0/2666226733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8238017070 con 0x55ae8d9b2c80 2024-03-22T04:12:47.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f8236a00640 1 -- 172.21.15.86:0/2666226733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82380117a0 con 0x55ae8d9b2c80 2024-03-22T04:12:47.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.093+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ae8da42dc0 con 0x55ae8d9b2c80 2024-03-22T04:12:47.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae8da43300 con 0x55ae8d9b2c80 2024-03-22T04:12:47.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f8236a00640 1 -- 172.21.15.86:0/2666226733 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8238011940 con 0x55ae8d9b2c80 2024-03-22T04:12:47.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f8236a00640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8220066500 0x7f82200689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f8236a00640 1 -- 172.21.15.86:0/2666226733 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8238084a80 con 0x55ae8d9b2c80 2024-03-22T04:12:47.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f823e200640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8220066500 0x7f82200689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f824109b080 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ae8dae0240 0x55ae8dae0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) -- 0x55ae8dae0b60 con 0x55ae8dae0240 2024-03-22T04:12:47.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f823d800640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ae8dae0240 0x55ae8dae0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.097+0000 7f823e200640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8220066500 0x7f82200689c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8230017f30 tx=0x7f82300a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f823d800640 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ae8dae0240 0x55ae8dae0620 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f823d800640 1 -- 172.21.15.86:0/2666226733 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v54'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f822c00ac30 con 0x55ae8dae0240 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ae8dae0240 msgr2=0x55ae8dae0620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ae8dae0240 0x55ae8dae0620 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8220066500 msgr2=0x7f82200689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8220066500 0x7f82200689c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8230017f30 tx=0x7f82300a7000 comp rx=0 tx=0).stop 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d9b2c80 msgr2=0x55ae8dadfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 --2- 172.21.15.86:0/2666226733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d9b2c80 0x55ae8dadfd00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8238003ec0 tx=0x7f82380018f0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f823ec00640 1 -- 172.21.15.86:0/2666226733 reap_dead start 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 shutdown_connections 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 >> 172.21.15.86:0/2666226733 conn(0x55ae8d99cfe0 msgr2=0x55ae8da43a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 shutdown_connections 2024-03-22T04:12:47.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.101+0000 7f824109b080 1 -- 172.21.15.86:0/2666226733 wait complete. 2024-03-22T04:12:47.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-22T04:12:47.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:47.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-03-22T04:12:47.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d8200640 1 Processor -- start 2024-03-22T04:12:47.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d8200640 1 -- start start 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d8200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24d00b0f90 0x7f24d00aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d8200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00cfe90 0x7f24d00af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d8200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24d00ce8f0 con 0x7f24d015c4b0 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d8200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f24d015c890 con 0x7f24d00b0f90 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d8200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f24d009b620 con 0x7f24d00cfe90 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d5800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f24d015c4b0 0x7f24d00cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39070/0 (socket says 172.21.15.86:39070) 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d5800640 1 -- 172.21.15.86:0/402185350 learned_addr learned my addr 172.21.15.86:0/402185350 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d4e00640 1 --2- 172.21.15.86:0/402185350 >> v2:172.21.15.86:3300/0 conn(0x7f24d00b0f90 0x7f24d00aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24d6200640 1 --2- 172.21.15.86:0/402185350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00cfe90 0x7f24d00af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.249+0000 7f24cfe00640 1 -- 172.21.15.86:0/402185350 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606253405 0 0) 0x7f24d00ce8f0 con 0x7f24d015c4b0 2024-03-22T04:12:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24cfe00640 1 -- 172.21.15.86:0/402185350 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24b8003530 con 0x7f24d015c4b0 2024-03-22T04:12:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24d6200640 1 -- 172.21.15.86:0/402185350 >> v2:172.21.15.86:3300/0 conn(0x7f24d00b0f90 msgr2=0x7f24d00aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24d6200640 1 --2- 172.21.15.86:0/402185350 >> v2:172.21.15.86:3300/0 conn(0x7f24d00b0f90 0x7f24d00aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24d6200640 1 -- 172.21.15.86:0/402185350 >> v1:172.21.15.86:6789/0 conn(0x7f24d015c4b0 legacy=0x7f24d00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24d6200640 1 -- 172.21.15.86:0/402185350 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d009b900 con 0x7f24d00cfe90 2024-03-22T04:12:47.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24d6200640 1 --2- 172.21.15.86:0/402185350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00cfe90 0x7f24d00af200 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009d90 tx=0x7f24c801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2904d295536c86 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24cfe00640 1 -- 172.21.15.86:0/402185350 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80b2070 con 0x7f24d00cfe90 2024-03-22T04:12:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24cfe00640 1 -- 172.21.15.86:0/402185350 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c80a5a00 con 0x7f24d00cfe90 2024-03-22T04:12:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.253+0000 7f24cfe00640 1 -- 172.21.15.86:0/402185350 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80a5dd0 con 0x7f24d00cfe90 2024-03-22T04:12:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- 172.21.15.86:0/402185350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00cfe90 msgr2=0x7f24d00af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 --2- 172.21.15.86:0/402185350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00cfe90 0x7f24d00af200 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009d90 tx=0x7f24c801c940 comp rx=0 tx=0).stop 2024-03-22T04:12:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- 172.21.15.86:0/402185350 shutdown_connections 2024-03-22T04:12:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 --2- 172.21.15.86:0/402185350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00cfe90 0x7f24d00af200 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 --2- 172.21.15.86:0/402185350 >> v2:172.21.15.86:3300/0 conn(0x7f24d00b0f90 0x7f24d00aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- 172.21.15.86:0/402185350 >> 172.21.15.86:0/402185350 conn(0x7f24d00b3ba0 msgr2=0x7f24d00b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- 172.21.15.86:0/402185350 shutdown_connections 2024-03-22T04:12:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- 172.21.15.86:0/402185350 wait complete. 2024-03-22T04:12:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 Processor -- start 2024-03-22T04:12:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- start start 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00b0f90 0x7f24d006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24d00cfe90 0x7f24d0159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d015c4b0 0x7f24d0159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24d00ccdb0 con 0x7f24d015c4b0 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24d00ccc30 con 0x7f24d00cfe90 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d8200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f24d01692d0 con 0x7f24d00b0f90 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d015c4b0 0x7f24d0159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d015c4b0 0x7f24d0159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49538/0 (socket says 172.21.15.86:49538) 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d6200640 1 -- 172.21.15.86:0/361688459 learned_addr learned my addr 172.21.15.86:0/361688459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d5800640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00b0f90 0x7f24d006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d4e00640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24d00cfe90 0x7f24d0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d6200640 1 -- 172.21.15.86:0/361688459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00b0f90 msgr2=0x7f24d006dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d6200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00b0f90 0x7f24d006dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d6200640 1 -- 172.21.15.86:0/361688459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24d00cfe90 msgr2=0x7f24d0159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d6200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24d00cfe90 0x7f24d0159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.257+0000 7f24d6200640 1 -- 172.21.15.86:0/361688459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d0165f70 con 0x7f24d015c4b0 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24d6200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d015c4b0 0x7f24d0159dd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f24c80b0040 tx=0x7f24c80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24cd600640 1 -- 172.21.15.86:0/361688459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80b2050 con 0x7f24d015c4b0 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24cd600640 1 -- 172.21.15.86:0/361688459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c80be070 con 0x7f24d015c4b0 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24d8200640 1 -- 172.21.15.86:0/361688459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d0166250 con 0x7f24d015c4b0 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24cd600640 1 -- 172.21.15.86:0/361688459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80ada70 con 0x7f24d015c4b0 2024-03-22T04:12:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24d8200640 1 -- 172.21.15.86:0/361688459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d0166810 con 0x7f24d015c4b0 2024-03-22T04:12:47.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24cd600640 1 -- 172.21.15.86:0/361688459 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f24c80adc10 con 0x7f24d015c4b0 2024-03-22T04:12:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24cd600640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f24b0067b20 0x7f24b0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24cd600640 1 -- 172.21.15.86:0/361688459 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f24c8121220 con 0x7f24d015c4b0 2024-03-22T04:12:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.261+0000 7f24d5800640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f24b0067b20 0x7f24b0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f24b0067b20 0x7f24b0069fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007fd0 tx=0x7f24c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24db6a3000 1 Processor -- start 2024-03-22T04:12:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24db6a3000 1 -- start start 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f2af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f2b0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24db6a3000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a5f196830 con 0x561a5f2f8e10 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24db6a3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a5f0fde10 con 0x561a5f2f9730 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24db6a3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561a5f09ae50 con 0x561a5f2afeb0 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2f91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49544/0 (socket says 172.21.15.86:49544) 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 -- 172.21.15.86:0/2779546847 learned_addr learned my addr 172.21.15.86:0/2779546847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d4e00640 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f2af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d6200640 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f2b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 -- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 msgr2=0x561a5f2b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f2b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 -- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 msgr2=0x561a5f2af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f2af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d5800640 1 -- 172.21.15.86:0/2779546847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a5f083510 con 0x561a5f2f8e10 2024-03-22T04:12:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.265+0000 7f24d4e00640 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f2af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24d5800640 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2f91f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f24c400fae0 tx=0x7f24c4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ed74d4676cd5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24c2000640 1 -- 172.21.15.86:0/2779546847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4025070 con 0x561a5f2f8e10 2024-03-22T04:12:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24c2000640 1 -- 172.21.15.86:0/2779546847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c4015d70 con 0x561a5f2f8e10 2024-03-22T04:12:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24c2000640 1 -- 172.21.15.86:0/2779546847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4020620 con 0x561a5f2f8e10 2024-03-22T04:12:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- 172.21.15.86:0/2779546847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 msgr2=0x561a5f2f91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2f91f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f24c400fae0 tx=0x7f24c4000ee0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- 172.21.15.86:0/2779546847 shutdown_connections 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f2af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f2b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 --2- 172.21.15.86:0/2779546847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2f91f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- 172.21.15.86:0/2779546847 >> 172.21.15.86:0/2779546847 conn(0x561a5f2abe60 msgr2=0x561a5f2ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- 172.21.15.86:0/2779546847 shutdown_connections 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- 172.21.15.86:0/2779546847 wait complete. 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- start start 2024-03-22T04:12:47.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 Processor -- start 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2afeb0 0x561a5f2b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2f9730 0x561a5f2ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a5f0bb640 con 0x561a5f2f8e10 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a5f0f9dc0 con 0x561a5f2afeb0 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561a5f0fd120 con 0x561a5f2f9730 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24db6a3000 10 client.0 _ll_register_callbacks cb 0x561a5f29ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24d5800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2afeb0 0x561a5f2b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24d4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24d6200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2f9730 0x561a5f2ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24d5800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2afeb0 0x561a5f2b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:39206/0 (socket says 172.21.15.86:39206) 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.269+0000 7f24d5800640 1 -- 172.21.15.86:0/655302963 learned_addr learned my addr 172.21.15.86:0/655302963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d5800640 1 -- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2f9730 msgr2=0x561a5f2ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d5800640 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2f9730 0x561a5f2ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d5800640 1 -- 172.21.15.86:0/655302963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 msgr2=0x561a5f2b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d5800640 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d5800640 1 -- 172.21.15.86:0/655302963 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5f0bba40 con 0x561a5f2afeb0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d6200640 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2f9730 0x561a5f2ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d5800640 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2afeb0 0x561a5f2b53c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f24c4000f70 tx=0x7f24c40010b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3cfa099dd0ac server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 1 -- 172.21.15.86:0/655302963 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4025050 con 0x561a5f2afeb0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 -- 172.21.15.86:0/655302963 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c4015d70 con 0x561a5f2afeb0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 1 -- 172.21.15.86:0/655302963 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f24c40047f0 con 0x561a5f2afeb0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 1 -- 172.21.15.86:0/655302963 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f24c4004a30 con 0x561a5f2afeb0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 10 client.5917 fetch_fsmap learned FSMap version 26 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 -- 172.21.15.86:0/655302963 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x561a5f0b6370 con 0x561a5f2afeb0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 1 -- 172.21.15.86:0/655302963 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c4015d70 con 0x561a5f2afeb0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 1 -- 172.21.15.86:0/655302963 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f24c4004bd0 con 0x561a5f2afeb0 2024-03-22T04:12:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 10 client.5917 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T04:12:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 10 client.5917 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-22T04:12:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 20 client.5917 populate_metadata read hostname 'smithi086' 2024-03-22T04:12:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x561a5f3080b0 0x561a5f30a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 4 client.5917 mds_command: new command op to 75245 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:12:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 -- 172.21.15.86:0/655302963 --> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a5f098ed0 con 0x561a5f3080b0 2024-03-22T04:12:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d4e00640 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x561a5f3080b0 0x561a5f30a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24d4e00640 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x561a5f3080b0 0x561a5f30a4a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45780cac836967 server_cookie=5351c1b9ebc589d4 in_seq=0 out_seq=0 2024-03-22T04:12:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 10 client.5917 ms_handle_connect on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 1 -- 172.21.15.86:0/655302963 <== mds.0 v2:172.21.15.196:6832/2728972599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561a5f098ed0 con 0x561a5f3080b0 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 10 client.5917 handle_command_reply: tid=0 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 client.5917 shutdown 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 -- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2afeb0 msgr2=0x561a5f2b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2afeb0 0x561a5f2b53c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f24c4000f70 tx=0x7f24c40010b0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 -- 172.21.15.86:0/655302963 shutdown_connections 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x561a5f3080b0 0x561a5f30a4a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2f9730 0x561a5f2ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f2b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 --2- 172.21.15.86:0/655302963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2afeb0 0x561a5f2b53c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24b6a00640 0 client.5917 ms_handle_reset on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 -- 172.21.15.86:0/655302963 >> 172.21.15.86:0/655302963 conn(0x561a5f2abe60 msgr2=0x561a5f2aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 -- 172.21.15.86:0/655302963 shutdown_connections 2024-03-22T04:12:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 1 -- 172.21.15.86:0/655302963 wait complete. 2024-03-22T04:12:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.273+0000 7f24db6a3000 20 client.5917 trim_cache size 0 max 16384 2024-03-22T04:12:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ec200640 1 Processor -- start 2024-03-22T04:12:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ec200640 1 -- start start 2024-03-22T04:12:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ec200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8e409acd0 0x7fc8e406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ec200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e415eb90 0x7fc8e40b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ec200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e406ded0 con 0x7fc8e409b0b0 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ec200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e40ccb60 con 0x7fc8e409acd0 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ec200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e406e1b0 con 0x7fc8e415eb90 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e9800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8e409acd0 0x7fc8e406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e8e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc8e409b0b0 0x7fc8e415e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39072/0 (socket says 172.21.15.86:39072) 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e8e00640 1 -- 172.21.15.86:0/4101472042 learned_addr learned my addr 172.21.15.86:0/4101472042 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ea200640 1 --2- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e415eb90 0x7fc8e40b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e3e00640 1 -- 172.21.15.86:0/4101472042 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309272016 0 0) 0x7fc8e406ded0 con 0x7fc8e409b0b0 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e3e00640 1 -- 172.21.15.86:0/4101472042 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8cc003530 con 0x7fc8e409b0b0 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e9800640 1 -- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e415eb90 msgr2=0x7fc8e40b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e9800640 1 --2- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e415eb90 0x7fc8e40b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e9800640 1 -- 172.21.15.86:0/4101472042 >> v1:172.21.15.86:6789/0 conn(0x7fc8e409b0b0 legacy=0x7fc8e415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8e9800640 1 -- 172.21.15.86:0/4101472042 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e406e490 con 0x7fc8e409acd0 2024-03-22T04:12:47.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.277+0000 7fc8ea200640 1 --2- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e415eb90 0x7fc8e40b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:47.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8e9800640 1 --2- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e409acd0 0x7fc8e406d8f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80097c0 tx=0x7fc8d801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7c140dc02101d45 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8e3e00640 1 -- 172.21.15.86:0/4101472042 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80b1070 con 0x7fc8e409acd0 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8e3e00640 1 -- 172.21.15.86:0/4101472042 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d80ac600 con 0x7fc8e409acd0 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8e3e00640 1 -- 172.21.15.86:0/4101472042 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80ac9d0 con 0x7fc8e409acd0 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8ec200640 1 -- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e409acd0 msgr2=0x7fc8e406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8ec200640 1 --2- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e409acd0 0x7fc8e406d8f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80097c0 tx=0x7fc8d801c380 comp rx=0 tx=0).stop 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8ec200640 1 -- 172.21.15.86:0/4101472042 shutdown_connections 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8ec200640 1 --2- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e415eb90 0x7fc8e40b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8ec200640 1 --2- 172.21.15.86:0/4101472042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e409acd0 0x7fc8e406d8f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8ec200640 1 -- 172.21.15.86:0/4101472042 >> 172.21.15.86:0/4101472042 conn(0x7fc8e40b3bf0 msgr2=0x7fc8e40b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8ec200640 1 -- 172.21.15.86:0/4101472042 shutdown_connections 2024-03-22T04:12:47.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.281+0000 7fc8ec200640 1 -- 172.21.15.86:0/4101472042 wait complete. 2024-03-22T04:12:47.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 Processor -- start 2024-03-22T04:12:47.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 -- start start 2024-03-22T04:12:47.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e409acd0 0x7fc8e40c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e409b0b0 0x7fc8e415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e415eb90 0x7fc8e40b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40cd980 con 0x7fc8e415eb90 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40cd800 con 0x7fc8e409acd0 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e40cdb00 con 0x7fc8e409b0b0 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e415eb90 0x7fc8e40b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e415eb90 0x7fc8e40b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49574/0 (socket says 172.21.15.86:49574) 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 -- 172.21.15.86:0/982843807 learned_addr learned my addr 172.21.15.86:0/982843807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8e8e00640 1 --2- 172.21.15.86:0/982843807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e409b0b0 0x7fc8e415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8e9800640 1 --2- 172.21.15.86:0/982843807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e409acd0 0x7fc8e40c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 -- 172.21.15.86:0/982843807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e409b0b0 msgr2=0x7fc8e415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 --2- 172.21.15.86:0/982843807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e409b0b0 0x7fc8e415a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 -- 172.21.15.86:0/982843807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e409acd0 msgr2=0x7fc8e40c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 --2- 172.21.15.86:0/982843807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e409acd0 0x7fc8e40c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 -- 172.21.15.86:0/982843807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e40c99c0 con 0x7fc8e415eb90 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ea200640 1 --2- 172.21.15.86:0/982843807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e415eb90 0x7fc8e40b1f40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009e60 tx=0x7fc8dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc018030 con 0x7fc8e415eb90 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8dc00cdf0 con 0x7fc8e415eb90 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc003bc0 con 0x7fc8e415eb90 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 -- 172.21.15.86:0/982843807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e40c9ca0 con 0x7fc8e415eb90 2024-03-22T04:12:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.285+0000 7fc8ec200640 1 -- 172.21.15.86:0/982843807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e40ca260 con 0x7fc8e415eb90 2024-03-22T04:12:47.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.289+0000 7fc8ec200640 1 -- 172.21.15.86:0/982843807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc8e4074470 con 0x7fc8e415eb90 2024-03-22T04:12:47.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.289+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc8dc01b070 con 0x7fc8e415eb90 2024-03-22T04:12:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.289+0000 7fc8e1600640 1 --2- 172.21.15.86:0/982843807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c4067b20 0x7fc8c4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.289+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc8dc087310 con 0x7fc8e415eb90 2024-03-22T04:12:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.289+0000 7fc8e9800640 1 --2- 172.21.15.86:0/982843807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c4067b20 0x7fc8c4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.289+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7fc8dc010040 con 0x7fc8e415eb90 2024-03-22T04:12:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.289+0000 7fc8ec200640 1 -- 172.21.15.86:0/982843807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fc8e4154b30 con 0x7fc8e415eb90 2024-03-22T04:12:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.289+0000 7fc8e9800640 1 --2- 172.21.15.86:0/982843807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8c4067b20 0x7fc8c4069fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80ae000 tx=0x7fc8d80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24db6a3000 1 Processor -- start 2024-03-22T04:12:47.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24db6a3000 1 -- start start 2024-03-22T04:12:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f363810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f8e10 0x561a5f363d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f9730 0x561a5f364290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24db6a3000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0203eb0 con 0x561a5f2f9730 2024-03-22T04:12:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24db6a3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0202f20 con 0x561a5f2f8e10 2024-03-22T04:12:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24db6a3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f24d006d7e0 con 0x561a5f2afeb0 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f9730 0x561a5f364290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f9730 0x561a5f364290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49580/0 (socket says 172.21.15.86:49580) 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 -- 172.21.15.86:0/1963874715 learned_addr learned my addr 172.21.15.86:0/1963874715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d4e00640 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f8e10 0x561a5f363d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d5800640 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f363810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 -- 172.21.15.86:0/1963874715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 msgr2=0x561a5f363810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f363810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 -- 172.21.15.86:0/1963874715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f8e10 msgr2=0x561a5f363d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f8e10 0x561a5f363d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 -- 172.21.15.86:0/1963874715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a5f09b0a0 con 0x561a5f2f9730 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.293+0000 7f24d6200640 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f9730 0x561a5f364290 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f24c801ceb0 tx=0x7f24c80c5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7b57cdb7983aae server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24c1600640 1 -- 172.21.15.86:0/1963874715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c811d070 con 0x561a5f2f9730 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24c1600640 1 -- 172.21.15.86:0/1963874715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c80b3040 con 0x561a5f2f9730 2024-03-22T04:12:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24c1600640 1 -- 172.21.15.86:0/1963874715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80f14d0 con 0x561a5f2f9730 2024-03-22T04:12:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- 172.21.15.86:0/1963874715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f9730 msgr2=0x561a5f364290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f9730 0x561a5f364290 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f24c801ceb0 tx=0x7f24c80c5040 comp rx=0 tx=0).stop 2024-03-22T04:12:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- 172.21.15.86:0/1963874715 shutdown_connections 2024-03-22T04:12:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f8e10 0x561a5f363d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f9730 0x561a5f364290 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1963874715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f363810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- 172.21.15.86:0/1963874715 >> 172.21.15.86:0/1963874715 conn(0x561a5f2abe60 msgr2=0x561a5f2f7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- 172.21.15.86:0/1963874715 shutdown_connections 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- 172.21.15.86:0/1963874715 wait complete. 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- start start 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 Processor -- start 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f30cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f30d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f364650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0156fc0 con 0x561a5f2f8e10 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a5f0f9dc0 con 0x561a5f2f9730 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561a5f0a60b0 con 0x561a5f2afeb0 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 10 client.0 _ll_register_callbacks cb 0x7f24c4023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f30d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f30d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49594/0 (socket says 172.21.15.86:49594) 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 -- 172.21.15.86:0/1098304934 learned_addr learned my addr 172.21.15.86:0/1098304934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d5800640 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f30cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d6200640 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f364650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 -- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 msgr2=0x561a5f30cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f30cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 -- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 msgr2=0x561a5f364650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f364650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 -- 172.21.15.86:0/1098304934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5f17f840 con 0x561a5f2f8e10 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d6200640 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f364650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24d4e00640 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f30d160 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x561a5f27e220 tx=0x7f24bc01f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99419831ddd9fd33 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24b6a00640 1 -- 172.21.15.86:0/1098304934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0180d0 con 0x561a5f2f8e10 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24b6a00640 1 -- 172.21.15.86:0/1098304934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f24bc029c80 con 0x561a5f2f8e10 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 1 -- 172.21.15.86:0/1098304934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c80b3040 con 0x561a5f2f8e10 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24b6a00640 1 -- 172.21.15.86:0/1098304934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f24bc018270 con 0x561a5f2f8e10 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.297+0000 7f24db6a3000 10 client.5970 fetch_fsmap learned FSMap version 26 2024-03-22T04:12:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 -- 172.21.15.86:0/1098304934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x561a5edb9470 con 0x561a5f2f8e10 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24b6a00640 1 -- 172.21.15.86:0/1098304934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c80b3040 con 0x561a5f2f8e10 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24b6a00640 1 -- 172.21.15.86:0/1098304934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f24bc026070 con 0x561a5f2f8e10 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 10 client.5970 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 10 client.5970 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 20 client.5970 populate_metadata read hostname 'smithi086' 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x561a5f364b90 0x561a5f364f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 4 client.5970 mds_command: new command op to 75245 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 -- 172.21.15.86:0/1098304934 --> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x561a5f0d76b0 con 0x561a5f364b90 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d5800640 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x561a5f364b90 0x561a5f364f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d5800640 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x561a5f364b90 0x561a5f364f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4482147e5e4f176 server_cookie=9b0931dd052eeb74 in_seq=0 out_seq=0 2024-03-22T04:12:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24b6a00640 10 client.5970 ms_handle_connect on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:47.307 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:12:47.300+0000 7f5540400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5922=0-4194304@1} caps={5922=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5922 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x558247a92c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24b6a00640 1 -- 172.21.15.86:0/1098304934 <== mds.0 v2:172.21.15.196:6832/2728972599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a5f0d76b0 con 0x561a5f364b90 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24b6a00640 10 client.5970 handle_command_reply: tid=0 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 client.5970 shutdown 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 -- 172.21.15.86:0/1098304934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 msgr2=0x561a5f30d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f30d160 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x561a5f27e220 tx=0x7f24bc01f660 comp rx=0 tx=0).stop 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 -- 172.21.15.86:0/1098304934 shutdown_connections 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x561a5f364b90 0x561a5f364f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a5f2f9730 0x561a5f364650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a5f2f8e10 0x561a5f30d160 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 --2- 172.21.15.86:0/1098304934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561a5f2afeb0 0x561a5f30cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 -- 172.21.15.86:0/1098304934 >> 172.21.15.86:0/1098304934 conn(0x561a5f2abe60 msgr2=0x561a5f2f79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24b6a00640 0 client.5970 ms_handle_reset on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 -- 172.21.15.86:0/1098304934 shutdown_connections 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 1 -- 172.21.15.86:0/1098304934 wait complete. 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24db6a3000 20 client.5970 trim_cache size 0 max 16384 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "8c0f63af-d93b-4a50-a2a6-7ed6253b5a4b", 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:12:47.309 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:12:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 -- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f24b0067b20 msgr2=0x7f24b0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f24b0067b20 0x7f24b0069fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007fd0 tx=0x7f24c4009040 comp rx=0 tx=0).stop 2024-03-22T04:12:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 -- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d015c4b0 msgr2=0x7f24d0159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d015c4b0 0x7f24d0159dd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f24c80b0040 tx=0x7f24c80196e0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 -- 172.21.15.86:0/361688459 shutdown_connections 2024-03-22T04:12:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f24b0067b20 0x7f24b0069fe0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24d00cfe90 0x7f24d0159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24d015c4b0 0x7f24d0159dd0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 --2- 172.21.15.86:0/361688459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24d00b0f90 0x7f24d006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 -- 172.21.15.86:0/361688459 >> 172.21.15.86:0/361688459 conn(0x7f24d00b3ba0 msgr2=0x7f24d00cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 -- 172.21.15.86:0/361688459 shutdown_connections 2024-03-22T04:12:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.301+0000 7f24d8200640 1 -- 172.21.15.86:0/361688459 wait complete. 2024-03-22T04:12:47.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:12:47.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-22T04:12:47.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca1800640 1 Processor -- start 2024-03-22T04:12:47.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca1800640 1 -- start start 2024-03-22T04:12:47.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca1800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3c9c15e250 0x7f3c9c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca1800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c9c06d330 0x7f3c9c15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c15ff80 0x7f3c9c1603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca1800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3c9c0d0440 con 0x7f3c9c06d330 2024-03-22T04:12:47.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca1800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3c9c0d0720 con 0x7f3c9c15e250 2024-03-22T04:12:47.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca1800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c0d0a00 con 0x7f3c9c15ff80 2024-03-22T04:12:47.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3c93400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c9c06d330 0x7f3c9c15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3c93400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c9c06d330 0x7f3c9c15fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49604/0 (socket says 172.21.15.86:49604) 2024-03-22T04:12:47.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c93400640 1 -- 172.21.15.86:0/1992547605 learned_addr learned my addr 172.21.15.86:0/1992547605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3ca0e00640 1 --2- 172.21.15.86:0/1992547605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c15ff80 0x7f3c9c1603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.457+0000 7f3c9be00640 1 --2- 172.21.15.86:0/1992547605 >> v2:172.21.15.196:3300/0 conn(0x7f3c9c15e250 0x7f3c9c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c93400640 1 -- 172.21.15.86:0/1992547605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c15ff80 msgr2=0x7f3c9c1603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c93400640 1 --2- 172.21.15.86:0/1992547605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c15ff80 0x7f3c9c1603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c93400640 1 -- 172.21.15.86:0/1992547605 >> v2:172.21.15.196:3300/0 conn(0x7f3c9c15e250 msgr2=0x7f3c9c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c93400640 1 --2- 172.21.15.86:0/1992547605 >> v2:172.21.15.196:3300/0 conn(0x7f3c9c15e250 0x7f3c9c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c93400640 1 -- 172.21.15.86:0/1992547605 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c0d0ce0 con 0x7f3c9c06d330 2024-03-22T04:12:47.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c9be00640 1 --2- 172.21.15.86:0/1992547605 >> v2:172.21.15.196:3300/0 conn(0x7f3c9c15e250 0x7f3c9c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:47.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c93400640 1 --2- 172.21.15.86:0/1992547605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c06d330 0x7f3c9c15fa40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c068b90 tx=0x7f3c8401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f956a33568ad880 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c9b400640 1 -- 172.21.15.86:0/1992547605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840b0070 con 0x7f3c9c06d330 2024-03-22T04:12:47.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c9b400640 1 -- 172.21.15.86:0/1992547605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c840ab600 con 0x7f3c9c06d330 2024-03-22T04:12:47.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3c9b400640 1 -- 172.21.15.86:0/1992547605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840ab9d0 con 0x7f3c9c06d330 2024-03-22T04:12:47.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3ca1800640 1 -- 172.21.15.86:0/1992547605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c06d330 msgr2=0x7f3c9c15fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.461+0000 7f3ca1800640 1 --2- 172.21.15.86:0/1992547605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c06d330 0x7f3c9c15fa40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c068b90 tx=0x7f3c8401c490 comp rx=0 tx=0).stop 2024-03-22T04:12:47.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- 172.21.15.86:0/1992547605 shutdown_connections 2024-03-22T04:12:47.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 --2- 172.21.15.86:0/1992547605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c15ff80 0x7f3c9c1603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 --2- 172.21.15.86:0/1992547605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c06d330 0x7f3c9c15fa40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 --2- 172.21.15.86:0/1992547605 >> v2:172.21.15.196:3300/0 conn(0x7f3c9c15e250 0x7f3c9c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- 172.21.15.86:0/1992547605 >> 172.21.15.86:0/1992547605 conn(0x7f3c9c0d1f30 msgr2=0x7f3c9c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- 172.21.15.86:0/1992547605 shutdown_connections 2024-03-22T04:12:47.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- 172.21.15.86:0/1992547605 wait complete. 2024-03-22T04:12:47.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 Processor -- start 2024-03-22T04:12:47.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- start start 2024-03-22T04:12:47.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c06d330 0x7f3c9c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c15e250 0x7f3c9c15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c9c15ff80 0x7f3c9c15d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c0cf140 con 0x7f3c9c15e250 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c0cefc0 con 0x7f3c9c15ff80 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c0cf2c0 con 0x7f3c9c06d330 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c15e250 0x7f3c9c15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c06d330 0x7f3c9c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c15e250 0x7f3c9c15cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49606/0 (socket says 172.21.15.86:49606) 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c06d330 0x7f3c9c155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:43358/0 (socket says 172.21.15.86:43358) 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 -- 172.21.15.86:0/3566217122 learned_addr learned my addr 172.21.15.86:0/3566217122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca0e00640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c9c15ff80 0x7f3c9c15d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 -- 172.21.15.86:0/3566217122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c06d330 msgr2=0x7f3c9c155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c06d330 0x7f3c9c155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 -- 172.21.15.86:0/3566217122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c9c15ff80 msgr2=0x7f3c9c15d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c9c15ff80 0x7f3c9c15d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 -- 172.21.15.86:0/3566217122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c155790 con 0x7f3c9c15e250 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c93400640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c15e250 0x7f3c9c15cd50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3c8401ab40 tx=0x7f3c84019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c98c00640 1 -- 172.21.15.86:0/3566217122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840b0040 con 0x7f3c9c15e250 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c98c00640 1 -- 172.21.15.86:0/3566217122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c840b4620 con 0x7f3c9c15e250 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3ca1800640 1 -- 172.21.15.86:0/3566217122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c15b090 con 0x7f3c9c15e250 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.465+0000 7f3c98c00640 1 -- 172.21.15.86:0/3566217122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840b4a10 con 0x7f3c9c15e250 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.469+0000 7f3ca1800640 1 -- 172.21.15.86:0/3566217122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c15b650 con 0x7f3c9c15e250 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.469+0000 7f3c98c00640 1 -- 172.21.15.86:0/3566217122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3c840ab4c0 con 0x7f3c9c15e250 2024-03-22T04:12:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.469+0000 7f3c98c00640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3c78067b20 0x7f3c78069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.469+0000 7f3c98c00640 1 -- 172.21.15.86:0/3566217122 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3c8411fb80 con 0x7f3c9c15e250 2024-03-22T04:12:47.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.469+0000 7f3c9be00640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3c78067b20 0x7f3c78069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.469+0000 7f3ca61de000 1 Processor -- start 2024-03-22T04:12:47.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.469+0000 7f3c9be00640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3c78067b20 0x7f3c78069fe0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c159c90 tx=0x7f3c8c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.469+0000 7f3ca61de000 1 -- start start 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec677ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec677f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dec65cfc40 con 0x55dec67caf80 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dec656cd90 con 0x55dec677ef50 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55dec65cbbf0 con 0x55dec67cb8a0 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49612/0 (socket says 172.21.15.86:49612) 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c9be00640 1 -- 172.21.15.86:0/935639868 learned_addr learned my addr 172.21.15.86:0/935639868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c93400640 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec677ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca0e00640 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec677f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c93400640 1 -- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 msgr2=0x55dec677f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c93400640 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec677f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c93400640 1 -- 172.21.15.86:0/935639868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 msgr2=0x55dec67cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c93400640 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c93400640 1 -- 172.21.15.86:0/935639868 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec6677130 con 0x55dec67cb8a0 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c9be00640 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c93400640 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec677ea10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c840efb30 tx=0x7f3c840ef850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e877e3f28c6646b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c90c00640 1 -- 172.21.15.86:0/935639868 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840f8700 con 0x55dec67cb8a0 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c90c00640 1 -- 172.21.15.86:0/935639868 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c840ee130 con 0x55dec67cb8a0 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3c90c00640 1 -- 172.21.15.86:0/935639868 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840f8270 con 0x55dec67cb8a0 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 msgr2=0x55dec677ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec677ea10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c840efb30 tx=0x7f3c840ef850 comp rx=0 tx=0).stop 2024-03-22T04:12:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- 172.21.15.86:0/935639868 shutdown_connections 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec677f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec677ea10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- 172.21.15.86:0/935639868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- 172.21.15.86:0/935639868 >> 172.21.15.86:0/935639868 conn(0x55dec6668990 msgr2=0x55dec67ae150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- 172.21.15.86:0/935639868 shutdown_connections 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- 172.21.15.86:0/935639868 wait complete. 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- start start 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 Processor -- start 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec6786a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec67af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dec658b710 con 0x55dec67caf80 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dec658d580 con 0x55dec677ef50 2024-03-22T04:12:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.473+0000 7f3ca61de000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55dec65d4270 con 0x55dec67cb8a0 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 10 client.0 _ll_register_callbacks cb 0x55dec6631200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49622/0 (socket says 172.21.15.86:49622) 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 -- 172.21.15.86:0/826837959 learned_addr learned my addr 172.21.15.86:0/826837959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c9be00640 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec6786a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca0e00640 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec67af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 -- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 msgr2=0x55dec67af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec67af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 -- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 msgr2=0x55dec6786a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec6786a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 -- 172.21.15.86:0/826837959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec65a8d30 con 0x55dec67caf80 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c93400640 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67af380 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3c8401ca00 tx=0x7f3c84000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9a1db19fc12b1f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 1 -- 172.21.15.86:0/826837959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840ec0d0 con 0x55dec67caf80 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 1 -- 172.21.15.86:0/826837959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3c840f8470 con 0x55dec67caf80 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 -- 172.21.15.86:0/826837959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c840ee130 con 0x55dec67caf80 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 1 -- 172.21.15.86:0/826837959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c8410e050 con 0x55dec67caf80 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 10 client.5982 fetch_fsmap learned FSMap version 26 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 -- 172.21.15.86:0/826837959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55dec6614fc0 con 0x55dec67caf80 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 1 -- 172.21.15.86:0/826837959 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c840ee130 con 0x55dec67caf80 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 1 -- 172.21.15.86:0/826837959 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f3c84111020 con 0x55dec67caf80 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 10 client.5982 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 10 client.5982 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 20 client.5982 populate_metadata read hostname 'smithi086' 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55dec67d15c0 0x55dec67d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 4 client.5982 mds_command: new command op to 75245 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 -- 172.21.15.86:0/826837959 --> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dec656add0 con 0x55dec67d15c0 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c9be00640 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55dec67d15c0 0x55dec67d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c9be00640 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55dec67d15c0 0x55dec67d19a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c15638e50227a37 server_cookie=64eefb32f7a16585 in_seq=0 out_seq=0 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 10 client.5982 ms_handle_connect on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 1 -- 172.21.15.86:0/826837959 <== mds.0 v2:172.21.15.196:6832/2728972599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dec656add0 con 0x55dec67d15c0 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 10 client.5982 handle_command_reply: tid=0 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 client.5982 shutdown 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 -- 172.21.15.86:0/826837959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 msgr2=0x55dec67af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67af380 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3c8401ca00 tx=0x7f3c84000bd0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 -- 172.21.15.86:0/826837959 shutdown_connections 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55dec67d15c0 0x55dec67d19a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67cb8a0 0x55dec67af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec67caf80 0x55dec67af380 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3c89600640 0 client.5982 ms_handle_reset on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 --2- 172.21.15.86:0/826837959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec6786a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 -- 172.21.15.86:0/826837959 >> 172.21.15.86:0/826837959 conn(0x55dec6668990 msgr2=0x55dec677e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 -- 172.21.15.86:0/826837959 shutdown_connections 2024-03-22T04:12:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.477+0000 7f3ca61de000 1 -- 172.21.15.86:0/826837959 wait complete. 2024-03-22T04:12:47.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.481+0000 7f3ca61de000 20 client.5982 trim_cache size 0 max 16384 2024-03-22T04:12:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 Processor -- start 2024-03-22T04:12:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- start start 2024-03-22T04:12:47.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 0x55dec67d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c203c50 con 0x55dec6668990 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c202c70 con 0x55dec677ef50 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c201fd0 con 0x55dec67caf80 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49636/0 (socket says 172.21.15.86:49636) 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 -- 172.21.15.86:0/15769012 learned_addr learned my addr 172.21.15.86:0/15769012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c93400640 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca0e00640 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 0x55dec67d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 -- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 msgr2=0x55dec67d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 0x55dec67d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 -- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 msgr2=0x55dec67e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67e1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 -- 172.21.15.86:0/15769012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec65a53a0 con 0x55dec6668990 2024-03-22T04:12:47.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c93400640 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:12:47.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c9be00640 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d79f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c003fd0 tx=0x7f3c8c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b410962189a15013 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c8be00640 1 -- 172.21.15.86:0/15769012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c02bc30 con 0x55dec6668990 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c8be00640 1 -- 172.21.15.86:0/15769012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c030070 con 0x55dec6668990 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3c8be00640 1 -- 172.21.15.86:0/15769012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c02b2c0 con 0x55dec6668990 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- 172.21.15.86:0/15769012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 msgr2=0x55dec67d79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d79f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c003fd0 tx=0x7f3c8c018000 comp rx=0 tx=0).stop 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- 172.21.15.86:0/15769012 shutdown_connections 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 0x55dec67d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 --2- 172.21.15.86:0/15769012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d79f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- 172.21.15.86:0/15769012 >> 172.21.15.86:0/15769012 conn(0x55dec67ae400 msgr2=0x55dec67cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- 172.21.15.86:0/15769012 shutdown_connections 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.497+0000 7f3ca61de000 1 -- 172.21.15.86:0/15769012 wait complete. 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 -- start start 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 Processor -- start 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 0x55dec67d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dec668b810 con 0x55dec6668990 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c202620 con 0x55dec677ef50 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55dec65cbbf0 con 0x55dec67caf80 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 10 client.0 _ll_register_callbacks cb 0x7f3c8c011ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49646/0 (socket says 172.21.15.86:49646) 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 -- 172.21.15.86:0/1845387272 learned_addr learned my addr 172.21.15.86:0/1845387272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:12:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca0e00640 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 0x55dec67d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c93400640 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 -- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 msgr2=0x55dec67d4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 0x55dec67d4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 -- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 msgr2=0x55dec67d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:12:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 -- 172.21.15.86:0/1845387272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec6555380 con 0x55dec6668990 2024-03-22T04:12:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c9be00640 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d7bc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00b010 tx=0x7f3c8c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3229e62b70e581d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:12:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:12:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 1 -- 172.21.15.86:0/1845387272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c03a020 con 0x55dec6668990 2024-03-22T04:12:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 1 -- 172.21.15.86:0/1845387272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3c8c02b4c0 con 0x55dec6668990 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 -- 172.21.15.86:0/1845387272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c8c030070 con 0x55dec6668990 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 1 -- 172.21.15.86:0/1845387272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c8c02b6c0 con 0x55dec6668990 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 10 client.5988 fetch_fsmap learned FSMap version 26 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 -- 172.21.15.86:0/1845387272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55dec6651460 con 0x55dec6668990 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 1 -- 172.21.15.86:0/1845387272 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c030070 con 0x55dec6668990 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 1 -- 172.21.15.86:0/1845387272 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f3c8c047080 con 0x55dec6668990 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 10 client.5988 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 10 client.5988 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 20 client.5988 populate_metadata read hostname 'smithi086' 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55dec67cb8a0 0x55dec67d5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 4 client.5988 mds_command: new command op to 75245 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3ca61de000 1 -- 172.21.15.86:0/1845387272 --> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dec656f160 con 0x55dec67cb8a0 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c93400640 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55dec67cb8a0 0x55dec67d5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:12:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c93400640 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55dec67cb8a0 0x55dec67d5480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ba108f94448bab server_cookie=4edd74b71ebeb964 in_seq=0 out_seq=0 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 10 client.5988 ms_handle_connect on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 1 -- 172.21.15.86:0/1845387272 <== mds.0 v2:172.21.15.196:6832/2728972599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55dec656f160 con 0x55dec67cb8a0 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.501+0000 7f3c89600640 10 client.5988 handle_command_reply: tid=0 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 client.5988 shutdown 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 -- 172.21.15.86:0/1845387272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 msgr2=0x55dec67d7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d7bc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00b010 tx=0x7f3c8c004a50 comp rx=0 tx=0).stop 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 -- 172.21.15.86:0/1845387272 shutdown_connections 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x55dec67cb8a0 0x55dec67d5480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dec677ef50 0x55dec67d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dec67caf80 0x55dec67d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3c89600640 0 client.5988 ms_handle_reset on v2:172.21.15.196:6832/2728972599 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 --2- 172.21.15.86:0/1845387272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dec6668990 0x55dec67d7bc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 -- 172.21.15.86:0/1845387272 >> 172.21.15.86:0/1845387272 conn(0x55dec67ae400 msgr2=0x55dec67d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 -- 172.21.15.86:0/1845387272 shutdown_connections 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 1 -- 172.21.15.86:0/1845387272 wait complete. 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca61de000 20 client.5988 trim_cache size 0 max 16384 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "8c0f63af-d93b-4a50-a2a6-7ed6253b5a4b": { 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "8c0f63af-d93b-4a50-a2a6-7ed6253b5a4b", 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:12:47.512 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:12:47.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 -- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3c78067b20 msgr2=0x7f3c78069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3c78067b20 0x7f3c78069fe0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c159c90 tx=0x7f3c8c00b040 comp rx=0 tx=0).stop 2024-03-22T04:12:47.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 -- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c15e250 msgr2=0x7f3c9c15cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:12:47.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c15e250 0x7f3c9c15cd50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3c8401ab40 tx=0x7f3c84019230 comp rx=0 tx=0).stop 2024-03-22T04:12:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 -- 172.21.15.86:0/3566217122 shutdown_connections 2024-03-22T04:12:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3c78067b20 0x7f3c78069fe0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c9c15ff80 0x7f3c9c15d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c15e250 0x7f3c9c15cd50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 --2- 172.21.15.86:0/3566217122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c9c06d330 0x7f3c9c155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:12:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 -- 172.21.15.86:0/3566217122 >> 172.21.15.86:0/3566217122 conn(0x7f3c9c0d1f30 msgr2=0x7f3c9c160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:12:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 -- 172.21.15.86:0/3566217122 shutdown_connections 2024-03-22T04:12:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.505+0000 7f3ca1800640 1 -- 172.21.15.86:0/3566217122 wait complete. 2024-03-22T04:12:47.525 INFO:tasks.cephfs.filesystem:scrub status for tag:8c0f63af-d93b-4a50-a2a6-7ed6253b5a4b - {'path': '/', 'tag': '8c0f63af-d93b-4a50-a2a6-7ed6253b5a4b', 'options': 'recursive,repair'} 2024-03-22T04:12:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:47.909+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 7 ==== log(6 entries from seq 1 at 2024-03-22T04:12:47.304672+0000) ==== 1078+0+0 (secure 0 0 0) 0x7fc8dc0555d0 con 0x7fc8e415eb90 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: cluster: 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: id: 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: health: HEALTH_OK 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: services: 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: mgr: y(active, since 2m), standbys: x 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: mds: 1/1 daemons up, 3 standby 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-03-22T04:12:47.915 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: data: 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: volumes: 1/1 healthy 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: pools: 3 pools, 97 pgs 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: objects: 24 objects, 579 KiB 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: usage: 217 MiB used, 720 GiB / 720 GiB avail 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: pgs: 97 active+clean 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: io: 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: client: 1.4 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:12:47.304672+0000 mds.d [INF] scrub queued for path: / 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:12:47.304675+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:12:47.304679+0000 mds.d [INF] scrub summary: active paths [/] 2024-03-22T04:12:47.916 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:12:47.305425+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-03-22T04:12:47.917 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:12:47.305433+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-03-22T04:12:47.917 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:12:47.305957+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-03-22T04:12:50.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:12:50.901+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 8 ==== log(1 entries from seq 7 at 2024-03-22T04:12:49.787229+0000) ==== 189+0+0 (secure 0 0 0) 0x7fc8dc055830 con 0x7fc8e415eb90 2024-03-22T04:12:50.907 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:12:49.787229+0000 mds.d [INF] scrub summary: idle 2024-03-22T04:12:57.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:12:57.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:13:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:14.969+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 9 ==== log(1 entries from seq 601 at 2024-03-22T04:13:14.914693+0000) ==== 321+0+0 (secure 0 0 0) 0x7fc8dc085720 con 0x7fc8e415eb90 2024-03-22T04:13:15.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:15.973+0000 7fc8e1600640 1 -- 172.21.15.86:0/982843807 <== mon.0 v2:172.21.15.86:3300/0 10 ==== log(1 entries from seq 602 at 2024-03-22T04:13:14.966685+0000) ==== 323+0+0 (secure 0 0 0) 0x7fc8dc053ba0 con 0x7fc8e415eb90 2024-03-22T04:13:17.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:17.527 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-22T04:13:17.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a4400640 1 Processor -- start 2024-03-22T04:13:17.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a4400640 1 -- start start 2024-03-22T04:13:17.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a4400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb29c06e7e0 0x7fb29c06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a4400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072b00 0x7fb29c0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a4400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb29c15c770 con 0x7fb29c072650 2024-03-22T04:13:17.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a4400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb29c071350 con 0x7fb29c06e7e0 2024-03-22T04:13:17.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a4400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c06f100 con 0x7fb29c072b00 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a1a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb29c06e7e0 0x7fb29c06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a2400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072b00 0x7fb29c0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a1000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb29c072650 0x7fb29c0b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:59816/0 (socket says 172.21.15.86:59816) 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb2a1000640 1 -- 172.21.15.86:0/2382540122 learned_addr learned my addr 172.21.15.86:0/2382540122 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb29be00640 1 -- 172.21.15.86:0/2382540122 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527128468 0 0) 0x7fb288001120 con 0x7fb29c072650 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.673+0000 7fb29be00640 1 -- 172.21.15.86:0/2382540122 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb284003530 con 0x7fb29c072650 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a1a00640 1 -- 172.21.15.86:0/2382540122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072b00 msgr2=0x7fb29c0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/2382540122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072b00 0x7fb29c0b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a1a00640 1 -- 172.21.15.86:0/2382540122 >> v1:172.21.15.196:6789/0 conn(0x7fb29c072650 legacy=0x7fb29c0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a1a00640 1 -- 172.21.15.86:0/2382540122 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c073150 con 0x7fb29c06e7e0 2024-03-22T04:13:17.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a2400640 1 --2- 172.21.15.86:0/2382540122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072b00 0x7fb29c0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:17.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/2382540122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 0x7fb29c06ebc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb290009d90 tx=0x7fb29001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6df85ec7e3cee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb29be00640 1 -- 172.21.15.86:0/2382540122 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900b2070 con 0x7fb29c06e7e0 2024-03-22T04:13:17.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb29be00640 1 -- 172.21.15.86:0/2382540122 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2900a59a0 con 0x7fb29c06e7e0 2024-03-22T04:13:17.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb29be00640 1 -- 172.21.15.86:0/2382540122 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900a5d70 con 0x7fb29c06e7e0 2024-03-22T04:13:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a4400640 1 -- 172.21.15.86:0/2382540122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 msgr2=0x7fb29c06ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2382540122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 0x7fb29c06ebc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb290009d90 tx=0x7fb29001c8e0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a4400640 1 -- 172.21.15.86:0/2382540122 shutdown_connections 2024-03-22T04:13:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2382540122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072b00 0x7fb29c0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2382540122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 0x7fb29c06ebc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a4400640 1 -- 172.21.15.86:0/2382540122 >> 172.21.15.86:0/2382540122 conn(0x7fb29c0cc940 msgr2=0x7fb29c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.677+0000 7fb2a4400640 1 -- 172.21.15.86:0/2382540122 shutdown_connections 2024-03-22T04:13:17.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 -- 172.21.15.86:0/2382540122 wait complete. 2024-03-22T04:13:17.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 Processor -- start 2024-03-22T04:13:17.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 -- start start 2024-03-22T04:13:17.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 0x7fb29c1fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072650 0x7fb29c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c072b00 0x7fb29c14ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c072080 con 0x7fb29c072b00 2024-03-22T04:13:17.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c0ce930 con 0x7fb29c06e7e0 2024-03-22T04:13:17.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a4400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c0c4f70 con 0x7fb29c072650 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c072b00 0x7fb29c14ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a1a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 0x7fb29c1fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c072b00 0x7fb29c14ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57226/0 (socket says 172.21.15.86:57226) 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 -- 172.21.15.86:0/2035662591 learned_addr learned my addr 172.21.15.86:0/2035662591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a1000640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072650 0x7fb29c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 -- 172.21.15.86:0/2035662591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072650 msgr2=0x7fb29c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072650 0x7fb29c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 -- 172.21.15.86:0/2035662591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 msgr2=0x7fb29c1fb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 0x7fb29c1fb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 -- 172.21.15.86:0/2035662591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c1fb830 con 0x7fb29c072b00 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 0x7fb29c1fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb2a2400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c072b00 0x7fb29c14ee10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb29400e930 tx=0x7fb29400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.681+0000 7fb299600640 1 -- 172.21.15.86:0/2035662591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294017070 con 0x7fb29c072b00 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.685+0000 7fb299600640 1 -- 172.21.15.86:0/2035662591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb294004020 con 0x7fb29c072b00 2024-03-22T04:13:17.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.685+0000 7fb2a4400640 1 -- 172.21.15.86:0/2035662591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c155e20 con 0x7fb29c072b00 2024-03-22T04:13:17.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.685+0000 7fb299600640 1 -- 172.21.15.86:0/2035662591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294004430 con 0x7fb29c072b00 2024-03-22T04:13:17.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.685+0000 7fb2a4400640 1 -- 172.21.15.86:0/2035662591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c1563e0 con 0x7fb29c072b00 2024-03-22T04:13:17.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.685+0000 7fb299600640 1 -- 172.21.15.86:0/2035662591 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb29400c940 con 0x7fb29c072b00 2024-03-22T04:13:17.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.685+0000 7fb299600640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb27c067b50 0x7fb27c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.685+0000 7fb299600640 1 -- 172.21.15.86:0/2035662591 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fb294085e70 con 0x7fb29c072b00 2024-03-22T04:13:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.685+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb27c067b50 0x7fb27c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 Processor -- start 2024-03-22T04:13:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- start start 2024-03-22T04:13:17.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb27c067b50 0x7fb27c06a010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb2900af000 tx=0x7fb2900a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 0x5575a637ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a637ef50 0x5575a637f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575a61cfc40 con 0x5575a63caf80 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575a616cd90 con 0x5575a637ef50 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5575a61cbbf0 con 0x5575a63cb8a0 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57234/0 (socket says 172.21.15.86:57234) 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 -- 172.21.15.86:0/1217797615 learned_addr learned my addr 172.21.15.86:0/1217797615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1000640 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 0x5575a637ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a2400640 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a637ef50 0x5575a637f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 -- 172.21.15.86:0/1217797615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 msgr2=0x5575a637ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 0x5575a637ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 -- 172.21.15.86:0/1217797615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a637ef50 msgr2=0x5575a637f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a637ef50 0x5575a637f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 -- 172.21.15.86:0/1217797615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a6277130 con 0x5575a63caf80 2024-03-22T04:13:17.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63cb360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb2900b0040 tx=0x7fb2900a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dec3485c5298a25 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb28e000640 1 -- 172.21.15.86:0/1217797615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900b2040 con 0x5575a63caf80 2024-03-22T04:13:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb28e000640 1 -- 172.21.15.86:0/1217797615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2900cb070 con 0x5575a63caf80 2024-03-22T04:13:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb28e000640 1 -- 172.21.15.86:0/1217797615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900b9bb0 con 0x5575a63caf80 2024-03-22T04:13:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1217797615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 msgr2=0x5575a63cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63cb360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb2900b0040 tx=0x7fb2900a7000 comp rx=0 tx=0).stop 2024-03-22T04:13:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1217797615 shutdown_connections 2024-03-22T04:13:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a637ef50 0x5575a637f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 0x5575a637ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1217797615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63cb360 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1217797615 >> 172.21.15.86:0/1217797615 conn(0x5575a6268990 msgr2=0x5575a63ae150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1217797615 shutdown_connections 2024-03-22T04:13:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1217797615 wait complete. 2024-03-22T04:13:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 -- start start 2024-03-22T04:13:17.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.689+0000 7fb2a77ce000 1 Processor -- start 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63869c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63cb8a0 0x5575a63af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575a618b710 con 0x5575a63caf80 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5575a618d580 con 0x5575a63cb8a0 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5575a61d4270 con 0x5575a637ef50 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 10 client.0 _ll_register_callbacks cb 0x5575a6231200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57250/0 (socket says 172.21.15.86:57250) 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 -- 172.21.15.86:0/3686770070 learned_addr learned my addr 172.21.15.86:0/3686770070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:17.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a2400640 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63cb8a0 0x5575a63af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 -- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 msgr2=0x5575a63869c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63869c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 -- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63cb8a0 msgr2=0x5575a63af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63cb8a0 0x5575a63af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 -- 172.21.15.86:0/3686770070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a61a8d30 con 0x5575a63caf80 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63869c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1000640 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63af380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb288005890 tx=0x7fb288001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840cc049b21784bd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb282a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb282a00640 1 -- 172.21.15.86:0/3686770070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288019020 con 0x5575a63caf80 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb282a00640 1 -- 172.21.15.86:0/3686770070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fb288005ad0 con 0x5575a63caf80 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3686770070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2900cb070 con 0x5575a63caf80 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb282a00640 1 -- 172.21.15.86:0/3686770070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb288023050 con 0x5575a63caf80 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 10 client.5997 fetch_fsmap learned FSMap version 26 2024-03-22T04:13:17.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3686770070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5575a6214fc0 con 0x5575a63caf80 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb282a00640 1 -- 172.21.15.86:0/3686770070 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2900cb070 con 0x5575a63caf80 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb282a00640 1 -- 172.21.15.86:0/3686770070 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fb288026020 con 0x5575a63caf80 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 10 client.5997 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 10 client.5997 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 20 client.5997 populate_metadata read hostname 'smithi086' 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x5575a63d15c0 0x5575a63d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 4 client.5997 mds_command: new command op to 75245 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3686770070 --> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575a616add0 con 0x5575a63d15c0 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x5575a63d15c0 0x5575a63d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb282a00640 10 client.5997 ms_handle_connect on v2:172.21.15.196:6832/2728972599 2024-03-22T04:13:17.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.693+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x5575a63d15c0 0x5575a63d19d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0ab16adfe36312 server_cookie=6ffa7f750da371ed in_seq=0 out_seq=0 2024-03-22T04:13:17.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb282a00640 1 -- 172.21.15.86:0/3686770070 <== mds.0 v2:172.21.15.196:6832/2728972599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5575a616add0 con 0x5575a63d15c0 2024-03-22T04:13:17.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb282a00640 10 client.5997 handle_command_reply: tid=0 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 client.5997 shutdown 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3686770070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 msgr2=0x5575a63af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63af380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb288005890 tx=0x7fb288001e80 comp rx=0 tx=0).stop 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3686770070 shutdown_connections 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x5575a63d15c0 0x5575a63d19d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63cb8a0 0x5575a63af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63caf80 0x5575a63af380 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb282a00640 0 client.5997 ms_handle_reset on v2:172.21.15.196:6832/2728972599 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3686770070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63869c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3686770070 >> 172.21.15.86:0/3686770070 conn(0x5575a6268990 msgr2=0x5575a637e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3686770070 shutdown_connections 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3686770070 wait complete. 2024-03-22T04:13:17.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.697+0000 7fb2a77ce000 20 client.5997 trim_cache size 0 max 16384 2024-03-22T04:13:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a77ce000 1 Processor -- start 2024-03-22T04:13:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a77ce000 1 -- start start 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 0x5575a63cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63cb8a0 0x5575a63d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c204030 con 0x5575a63cb8a0 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c0b40e0 con 0x5575a63caf80 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c157830 con 0x5575a637ef50 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63cb8a0 0x5575a63d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63cb8a0 0x5575a63d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57252/0 (socket says 172.21.15.86:57252) 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 -- 172.21.15.86:0/3390734766 learned_addr learned my addr 172.21.15.86:0/3390734766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a1000640 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 0x5575a63cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 -- 172.21.15.86:0/3390734766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 msgr2=0x5575a63e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63e1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 -- 172.21.15.86:0/3390734766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 msgr2=0x5575a63cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 0x5575a63cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 -- 172.21.15.86:0/3390734766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a61a53a0 con 0x5575a63cb8a0 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb2a2400640 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63cb8a0 0x5575a63d01e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb294055ef0 tx=0x7fb294056020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e51e06a0995af4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb28d600640 1 -- 172.21.15.86:0/3390734766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294020040 con 0x5575a63cb8a0 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb28d600640 1 -- 172.21.15.86:0/3390734766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb294054260 con 0x5575a63cb8a0 2024-03-22T04:13:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.713+0000 7fb28d600640 1 -- 172.21.15.86:0/3390734766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2940525d0 con 0x5575a63cb8a0 2024-03-22T04:13:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3390734766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63cb8a0 msgr2=0x5575a63d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63cb8a0 0x5575a63d01e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb294055ef0 tx=0x7fb294056020 comp rx=0 tx=0).stop 2024-03-22T04:13:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3390734766 shutdown_connections 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 0x5575a63cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a63cb8a0 0x5575a63d01e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/3390734766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a637ef50 0x5575a63e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3390734766 >> 172.21.15.86:0/3390734766 conn(0x5575a6268990 msgr2=0x5575a6387070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3390734766 shutdown_connections 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- 172.21.15.86:0/3390734766 wait complete. 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- start start 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 Processor -- start 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a637ef50 0x5575a63e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 0x5575a63d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 0x5575a63d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575a628b810 con 0x5575a637ef50 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1f7130 con 0x5575a63caf80 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5575a61cef50 con 0x5575a63cb8a0 2024-03-22T04:13:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 10 client.0 _ll_register_callbacks cb 0x7fb288007cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a637ef50 0x5575a63e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a637ef50 0x5575a63e13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57262/0 (socket says 172.21.15.86:57262) 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 -- 172.21.15.86:0/1782669035 learned_addr learned my addr 172.21.15.86:0/1782669035 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a2400640 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 0x5575a63d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1000640 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 0x5575a63d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 -- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 msgr2=0x5575a63d4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 0x5575a63d4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 -- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 msgr2=0x5575a63d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 0x5575a63d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 -- 172.21.15.86:0/1782669035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a6155380 con 0x5575a637ef50 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a1a00640 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a637ef50 0x5575a63e13d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb29001cdc0 tx=0x7fb290004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abe772792775 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb282a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb282a00640 1 -- 172.21.15.86:0/1782669035 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900b2040 con 0x5575a637ef50 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb282a00640 1 -- 172.21.15.86:0/1782669035 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fb2900b9c30 con 0x5575a637ef50 2024-03-22T04:13:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1782669035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb294054260 con 0x5575a637ef50 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb282a00640 1 -- 172.21.15.86:0/1782669035 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2900e7050 con 0x5575a637ef50 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 10 client.6003 fetch_fsmap learned FSMap version 26 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1782669035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5575a6251460 con 0x5575a637ef50 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb282a00640 1 -- 172.21.15.86:0/1782669035 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb294054260 con 0x5575a637ef50 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb282a00640 1 -- 172.21.15.86:0/1782669035 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fb2900ea020 con 0x5575a637ef50 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 10 client.6003 fetch_fsmap finished waiting for FSMap version 26 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 10 client.6003 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 20 client.6003 populate_metadata read hostname 'smithi086' 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x5575a6442bd0 0x5575a6442fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 4 client.6003 mds_command: new command op to 75245 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.717+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1782669035 --> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5575a616f160 con 0x5575a6442bd0 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a1000640 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x5575a6442bd0 0x5575a6442fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a1000640 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x5575a6442bd0 0x5575a6442fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ef5bb5ff2063c6 server_cookie=3c03653c95ca3b7e in_seq=0 out_seq=0 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb282a00640 10 client.6003 ms_handle_connect on v2:172.21.15.196:6832/2728972599 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb282a00640 1 -- 172.21.15.86:0/1782669035 <== mds.0 v2:172.21.15.196:6832/2728972599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5575a616f160 con 0x5575a6442bd0 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb282a00640 10 client.6003 handle_command_reply: tid=0 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 client.6003 shutdown 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1782669035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a637ef50 msgr2=0x5575a63e13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a637ef50 0x5575a63e13d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb29001cdc0 tx=0x7fb290004a50 comp rx=0 tx=0).stop 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1782669035 shutdown_connections 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:6832/2728972599,v1:172.21.15.196:6834/2728972599] conn(0x5575a6442bd0 0x5575a6442fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5575a63caf80 0x5575a63d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5575a63cb8a0 0x5575a63d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 --2- 172.21.15.86:0/1782669035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575a637ef50 0x5575a63e13d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb282a00640 0 client.6003 ms_handle_reset on v2:172.21.15.196:6832/2728972599 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1782669035 >> 172.21.15.86:0/1782669035 conn(0x5575a6268990 msgr2=0x5575a6385190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1782669035 shutdown_connections 2024-03-22T04:13:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 1 -- 172.21.15.86:0/1782669035 wait complete. 2024-03-22T04:13:17.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a77ce000 20 client.6003 trim_cache size 0 max 16384 2024-03-22T04:13:17.728 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:13:17.728 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:13:17.728 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:13:17.728 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:13:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 -- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb27c067b50 msgr2=0x7fb27c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb27c067b50 0x7fb27c06a010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb2900af000 tx=0x7fb2900a7070 comp rx=0 tx=0).stop 2024-03-22T04:13:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 -- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c072b00 msgr2=0x7fb29c14ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c072b00 0x7fb29c14ee10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb29400e930 tx=0x7fb29400edf0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 -- 172.21.15.86:0/2035662591 shutdown_connections 2024-03-22T04:13:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb27c067b50 0x7fb27c06a010 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c072b00 0x7fb29c14ee10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb29c072650 0x7fb29c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 --2- 172.21.15.86:0/2035662591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb29c06e7e0 0x7fb29c1fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 -- 172.21.15.86:0/2035662591 >> 172.21.15.86:0/2035662591 conn(0x7fb29c0cc940 msgr2=0x7fb29c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 -- 172.21.15.86:0/2035662591 shutdown_connections 2024-03-22T04:13:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.721+0000 7fb2a4400640 1 -- 172.21.15.86:0/2035662591 wait complete. 2024-03-22T04:13:17.743 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:13:17.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f3154a00640 1 Processor -- start 2024-03-22T04:13:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f3154a00640 1 -- start start 2024-03-22T04:13:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f3154a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3150161880 0x7f31500b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f3154a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f315006e350 con 0x7f315006de10 2024-03-22T04:13:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f3154a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f315006e630 con 0x7f3150161c60 2024-03-22T04:13:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f3154a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f315015ee80 con 0x7f3150161880 2024-03-22T04:13:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f314de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3150161880 0x7f31500b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f314e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f315006de10 0x7f315015fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47616/0 (socket says 172.21.15.86:47616) 2024-03-22T04:13:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f314e800640 1 -- 172.21.15.86:0/4275583982 learned_addr learned my addr 172.21.15.86:0/4275583982 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f314ca00640 1 -- 172.21.15.86:0/4275583982 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686350873 0 0) 0x7f315006e350 con 0x7f315006de10 2024-03-22T04:13:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f314ca00640 1 -- 172.21.15.86:0/4275583982 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3134003530 con 0x7f315006de10 2024-03-22T04:13:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f314de00640 1 -- 172.21.15.86:0/4275583982 >> v1:172.21.15.196:6789/0 conn(0x7f3150161c60 legacy=0x7f315006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f314de00640 1 -- 172.21.15.86:0/4275583982 >> v1:172.21.15.86:6789/0 conn(0x7f315006de10 legacy=0x7f315015fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.981+0000 7f314de00640 1 -- 172.21.15.86:0/4275583982 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315006e910 con 0x7f3150161880 2024-03-22T04:13:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.985+0000 7f314de00640 1 --2- 172.21.15.86:0/4275583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161880 0x7f31500b30f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f31400097c0 tx=0x7f314001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff593d087a12043 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.985+0000 7f314ca00640 1 -- 172.21.15.86:0/4275583982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31400b0070 con 0x7f3150161880 2024-03-22T04:13:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.985+0000 7f314ca00640 1 -- 172.21.15.86:0/4275583982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31400ab600 con 0x7f3150161880 2024-03-22T04:13:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.985+0000 7f314ca00640 1 -- 172.21.15.86:0/4275583982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31400ab9b0 con 0x7f3150161880 2024-03-22T04:13:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- 172.21.15.86:0/4275583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161880 msgr2=0x7f31500b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 --2- 172.21.15.86:0/4275583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161880 0x7f31500b30f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f31400097c0 tx=0x7f314001c3e0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- 172.21.15.86:0/4275583982 shutdown_connections 2024-03-22T04:13:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 --2- 172.21.15.86:0/4275583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161880 0x7f31500b30f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- 172.21.15.86:0/4275583982 >> 172.21.15.86:0/4275583982 conn(0x7f31500b1df0 msgr2=0x7f31500b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- 172.21.15.86:0/4275583982 shutdown_connections 2024-03-22T04:13:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- 172.21.15.86:0/4275583982 wait complete. 2024-03-22T04:13:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 Processor -- start 2024-03-22T04:13:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- start start 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315006de10 0x7f3150150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3150161880 0x7f31500ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161c60 0x7f315015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3150161020 con 0x7f3150161c60 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3150160ea0 con 0x7f3150161880 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f3154a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f315016c480 con 0x7f315006de10 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161c60 0x7f315015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161c60 0x7f315015b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57274/0 (socket says 172.21.15.86:57274) 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3150161880 0x7f31500ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314e800640 1 -- 172.21.15.86:0/4202525254 learned_addr learned my addr 172.21.15.86:0/4202525254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314de00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315006de10 0x7f3150150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314e800640 1 -- 172.21.15.86:0/4202525254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315006de10 msgr2=0x7f3150150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314e800640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315006de10 0x7f3150150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314e800640 1 -- 172.21.15.86:0/4202525254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3150161880 msgr2=0x7f31500ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314e800640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3150161880 0x7f31500ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.989+0000 7f314e800640 1 -- 172.21.15.86:0/4202525254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3150151190 con 0x7f3150161c60 2024-03-22T04:13:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f314e800640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161c60 0x7f315015b9e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f314800cad0 tx=0x7f314800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3146000640 1 -- 172.21.15.86:0/4202525254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148017070 con 0x7f3150161c60 2024-03-22T04:13:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3146000640 1 -- 172.21.15.86:0/4202525254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314800ac30 con 0x7f3150161c60 2024-03-22T04:13:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31500cf930 con 0x7f3150161c60 2024-03-22T04:13:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3146000640 1 -- 172.21.15.86:0/4202525254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148012400 con 0x7f3150161c60 2024-03-22T04:13:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3150158db0 con 0x7f3150161c60 2024-03-22T04:13:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3150068650 con 0x7f3150161c60 2024-03-22T04:13:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3146000640 1 -- 172.21.15.86:0/4202525254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f314801a070 con 0x7f3150161c60 2024-03-22T04:13:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3146000640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3130067b50 0x7f313006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f3146000640 1 -- 172.21.15.86:0/4202525254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f3148085dd0 con 0x7f3150161c60 2024-03-22T04:13:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.993+0000 7f314de00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3130067b50 0x7f313006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:17.997+0000 7f314de00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3130067b50 0x7f313006a010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f31400ad000 tx=0x7f31400a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:18.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.001+0000 7f3146000640 1 -- 172.21.15.86:0/4202525254 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3148055f40 con 0x7f3150161c60 2024-03-22T04:13:18.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.185+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3150161200 con 0x7f3150161c60 2024-03-22T04:13:18.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.185+0000 7f3146000640 1 -- 172.21.15.86:0/4202525254 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5189 (secure 0 0 0) 0x7f3148059fe0 con 0x7f3150161c60 2024-03-22T04:13:18.190 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:12:33.981773+0000","modified":"2024-03-22T04:12:37.002106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75245":{"gid":75245,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6834/2728972599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2728972599},{"type":"v1","addr":"172.21.15.196:6834","nonce":2728972599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75245,"qdb_cluster":[75245]},"id":3}]} 2024-03-22T04:13:18.190 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3130067b50 msgr2=0x7f313006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3130067b50 0x7f313006a010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f31400ad000 tx=0x7f31400a6000 comp rx=0 tx=0).stop 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161c60 msgr2=0x7f315015b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161c60 0x7f315015b9e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f314800cad0 tx=0x7f314800cf90 comp rx=0 tx=0).stop 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 shutdown_connections 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3130067b50 0x7f313006a010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3150161880 0x7f31500ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3150161c60 0x7f315015b9e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 --2- 172.21.15.86:0/4202525254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315006de10 0x7f3150150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 >> 172.21.15.86:0/4202525254 conn(0x7f31500b1df0 msgr2=0x7f31500af310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 shutdown_connections 2024-03-22T04:13:18.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.189+0000 7f3154a00640 1 -- 172.21.15.86:0/4202525254 wait complete. 2024-03-22T04:13:18.211 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-03-22T04:13:18.378 INFO:teuthology.orchestra.run.smithi196.stdout:{"message":"","return_code":0} 2024-03-22T04:13:18.378 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:13:18.378 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-22T04:13:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 Processor -- start 2024-03-22T04:13:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 -- start start 2024-03-22T04:13:18.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aee870d0 0x5568aeeb7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aeed1730 0x5568aef64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aef5b450 0x5568aef64d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568aeed0ab0 con 0x5568aee870d0 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5568aee2d620 con 0x5568aef5b450 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5568aeed0d50 con 0x5568aeed1730 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aee870d0 0x5568aeeb7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aee870d0 0x5568aeeb7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57286/0 (socket says 172.21.15.86:57286) 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 -- 172.21.15.86:0/978593840 learned_addr learned my addr 172.21.15.86:0/978593840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787b000640 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aeed1730 0x5568aef64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787c400640 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aef5b450 0x5568aef64d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 -- 172.21.15.86:0/978593840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aeed1730 msgr2=0x5568aef64830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aeed1730 0x5568aef64830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 -- 172.21.15.86:0/978593840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aef5b450 msgr2=0x5568aef64d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aef5b450 0x5568aef64d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 -- 172.21.15.86:0/978593840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568aee86da0 con 0x5568aee870d0 2024-03-22T04:13:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787ba00640 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aee870d0 0x5568aeeb7640 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f786c0ad040 tx=0x7f786c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9713bac3c448af2b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f7879c00640 1 -- 172.21.15.86:0/978593840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0af040 con 0x5568aee870d0 2024-03-22T04:13:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f7879c00640 1 -- 172.21.15.86:0/978593840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c0b9070 con 0x5568aee870d0 2024-03-22T04:13:18.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f7879c00640 1 -- 172.21.15.86:0/978593840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0aacf0 con 0x5568aee870d0 2024-03-22T04:13:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 -- 172.21.15.86:0/978593840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aee870d0 msgr2=0x5568aeeb7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.421+0000 7f787f247080 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aee870d0 0x5568aeeb7640 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f786c0ad040 tx=0x7f786c003030 comp rx=0 tx=0).stop 2024-03-22T04:13:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- 172.21.15.86:0/978593840 shutdown_connections 2024-03-22T04:13:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aef5b450 0x5568aef64d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aeed1730 0x5568aef64830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 --2- 172.21.15.86:0/978593840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aee870d0 0x5568aeeb7640 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- 172.21.15.86:0/978593840 >> 172.21.15.86:0/978593840 conn(0x5568aeeb5250 msgr2=0x5568aef5c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- 172.21.15.86:0/978593840 shutdown_connections 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- 172.21.15.86:0/978593840 wait complete. 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 Processor -- start 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- start start 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aee870d0 0x5568aef410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aeed1730 0x5568aef64830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aef5b450 0x5568aef64d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568aeeb7b80 con 0x5568aef5b450 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5568aef65600 con 0x5568aeed1730 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5568aef42e00 con 0x5568aee870d0 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aef5b450 0x5568aef64d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aef5b450 0x5568aef64d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57292/0 (socket says 172.21.15.86:57292) 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 -- 172.21.15.86:0/510334161 learned_addr learned my addr 172.21.15.86:0/510334161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787ba00640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aee870d0 0x5568aef410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787b000640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aeed1730 0x5568aef64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 -- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aee870d0 msgr2=0x5568aef410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5568aee870d0 0x5568aef410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 -- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aeed1730 msgr2=0x5568aef64830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568aeed1730 0x5568aef64830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 -- 172.21.15.86:0/510334161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568aef430a0 con 0x5568aef5b450 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787c400640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aef5b450 0x5568aef64d70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7874004eb0 tx=0x7f7874001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f7873e00640 1 -- 172.21.15.86:0/510334161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874018070 con 0x5568aef5b450 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f7873e00640 1 -- 172.21.15.86:0/510334161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78740020c0 con 0x5568aef5b450 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5568aef43340 con 0x5568aef5b450 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f7873e00640 1 -- 172.21.15.86:0/510334161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78740106f0 con 0x5568aef5b450 2024-03-22T04:13:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.425+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568aef41610 con 0x5568aef5b450 2024-03-22T04:13:18.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f7873e00640 1 -- 172.21.15.86:0/510334161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7874010890 con 0x5568aef5b450 2024-03-22T04:13:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f7873e00640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f785c066530 0x7f785c0689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f7873e00640 1 -- 172.21.15.86:0/510334161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f7874083e40 con 0x5568aef5b450 2024-03-22T04:13:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787ba00640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f785c066530 0x7f785c0689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5568aef451a0 0x5568aef455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56) -- 0x5568aef45af0 con 0x5568aef451a0 2024-03-22T04:13:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787b000640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5568aef451a0 0x5568aef455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787ba00640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f785c066530 0x7f785c0689f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b8000 tx=0x7f786c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:18.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787b000640 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5568aef451a0 0x5568aef455b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:18.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787b000640 1 -- 172.21.15.86:0/510334161 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f7868009960 con 0x5568aef451a0 2024-03-22T04:13:18.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5568aef451a0 msgr2=0x5568aef455b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5568aef451a0 0x5568aef455b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f785c066530 msgr2=0x7f785c0689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f785c066530 0x7f785c0689f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b8000 tx=0x7f786c0a7000 comp rx=0 tx=0).stop 2024-03-22T04:13:18.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aef5b450 msgr2=0x5568aef64d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 --2- 172.21.15.86:0/510334161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568aef5b450 0x5568aef64d70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7874004eb0 tx=0x7f7874001a10 comp rx=0 tx=0).stop 2024-03-22T04:13:18.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787c400640 1 -- 172.21.15.86:0/510334161 reap_dead start 2024-03-22T04:13:18.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 shutdown_connections 2024-03-22T04:13:18.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 >> 172.21.15.86:0/510334161 conn(0x5568aeeb5250 msgr2=0x5568aef54310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:18.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 shutdown_connections 2024-03-22T04:13:18.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.429+0000 7f787f247080 1 -- 172.21.15.86:0/510334161 wait complete. 2024-03-22T04:13:18.438 INFO:teuthology.orchestra.run.smithi086.stdout:Yalphaparent_a 2024-03-22T04:13:18.439 DEBUG:teuthology.orchestra.run.smithi086:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-22T04:13:18.530 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "ancestors": [ 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1099511627776, 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "alpha", 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 2 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1, 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "parent_a", 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 8, 2024-03-22T04:13:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "old_pools": [] 2024-03-22T04:13:18.532 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:13:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:18.536 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:13:18.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:13:18.648 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:13:18.724 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:13:18.724 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:13:18.724 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:13:18.724 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:18.733 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:13:18.733 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:18.782 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:18.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72eb200640 1 Processor -- start 2024-03-22T04:13:18.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72eb200640 1 -- start start 2024-03-22T04:13:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72eb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72ec0ced10 0x7f72ec0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72eb200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0cf0f0 0x7f72ec142190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72eb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72ec15c990 con 0x7f72ec0b1880 2024-03-22T04:13:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72eb200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f72ec13f550 con 0x7f72ec0ced10 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72eb200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f72ec1426d0 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72e9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72ec0ced10 0x7f72ec0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.957+0000 7f72e9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72ec0ced10 0x7f72ec0b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57306/0 (socket says 172.21.15.86:57306) 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e9e00640 1 -- 172.21.15.86:0/3094568705 learned_addr learned my addr 172.21.15.86:0/3094568705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e9400640 1 --1- 172.21.15.86:0/3094568705 >> v1:172.21.15.86:6789/0 conn(0x7f72ec0b1880 0x7f72ec13f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47628/0 (socket says 172.21.15.86:47628) 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72ea800640 1 --2- 172.21.15.86:0/3094568705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0cf0f0 0x7f72ec142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e9e00640 1 -- 172.21.15.86:0/3094568705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0cf0f0 msgr2=0x7f72ec142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e9e00640 1 --2- 172.21.15.86:0/3094568705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0cf0f0 0x7f72ec142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e9e00640 1 -- 172.21.15.86:0/3094568705 >> v1:172.21.15.86:6789/0 conn(0x7f72ec0b1880 legacy=0x7f72ec13f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e9e00640 1 -- 172.21.15.86:0/3094568705 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec15cc70 con 0x7f72ec0ced10 2024-03-22T04:13:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e8a00640 1 -- 172.21.15.86:0/3094568705 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15809000 0 0) 0x7f72ec15c990 con 0x7f72ec0b1880 2024-03-22T04:13:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e9e00640 1 --2- 172.21.15.86:0/3094568705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0ced10 0x7f72ec0b4410 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009870 tx=0x7f72dc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f011e7010e6611ad server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e8a00640 1 -- 172.21.15.86:0/3094568705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0ab640 con 0x7f72ec0ced10 2024-03-22T04:13:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e8a00640 1 -- 172.21.15.86:0/3094568705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72dc0b0070 con 0x7f72ec0ced10 2024-03-22T04:13:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.961+0000 7f72e8a00640 1 -- 172.21.15.86:0/3094568705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0aba00 con 0x7f72ec0ced10 2024-03-22T04:13:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- 172.21.15.86:0/3094568705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0ced10 msgr2=0x7f72ec0b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 --2- 172.21.15.86:0/3094568705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0ced10 0x7f72ec0b4410 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009870 tx=0x7f72dc01c530 comp rx=0 tx=0).stop 2024-03-22T04:13:18.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- 172.21.15.86:0/3094568705 shutdown_connections 2024-03-22T04:13:18.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 --2- 172.21.15.86:0/3094568705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0cf0f0 0x7f72ec142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 --2- 172.21.15.86:0/3094568705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0ced10 0x7f72ec0b4410 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- 172.21.15.86:0/3094568705 >> 172.21.15.86:0/3094568705 conn(0x7f72ec06b830 msgr2=0x7f72ec06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:18.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- 172.21.15.86:0/3094568705 shutdown_connections 2024-03-22T04:13:18.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- 172.21.15.86:0/3094568705 wait complete. 2024-03-22T04:13:18.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 Processor -- start 2024-03-22T04:13:18.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- start start 2024-03-22T04:13:18.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0ced10 0x7f72ec0b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0cf0f0 0x7f72ec0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ec155430 0x7f72ec0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec0b4ad0 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec0b4950 con 0x7f72ec155430 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72eb200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f72ec1406f0 con 0x7f72ec0ced10 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0cf0f0 0x7f72ec0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0cf0f0 0x7f72ec0d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57322/0 (socket says 172.21.15.86:57322) 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 -- 172.21.15.86:0/2902000220 learned_addr learned my addr 172.21.15.86:0/2902000220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9e00640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0ced10 0x7f72ec0b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72ea800640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ec155430 0x7f72ec0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 -- 172.21.15.86:0/2902000220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0ced10 msgr2=0x7f72ec0b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0ced10 0x7f72ec0b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 -- 172.21.15.86:0/2902000220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ec155430 msgr2=0x7f72ec0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ec155430 0x7f72ec0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 -- 172.21.15.86:0/2902000220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec13ac90 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.965+0000 7f72e9400640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0cf0f0 0x7f72ec0d08e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f72d800c520 tx=0x7f72d800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72e2000640 1 -- 172.21.15.86:0/2902000220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8018070 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72e2000640 1 -- 172.21.15.86:0/2902000220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d80074e0 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec13af70 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72e2000640 1 -- 172.21.15.86:0/2902000220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80049f0 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec1393f0 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ec13b490 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72e2000640 1 -- 172.21.15.86:0/2902000220 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f72d801b070 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72e2000640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f72cc067b20 0x7f72cc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72e2000640 1 -- 172.21.15.86:0/2902000220 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f72d8085490 con 0x7f72ec0cf0f0 2024-03-22T04:13:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72e9e00640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f72cc067b20 0x7f72cc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.969+0000 7f72e9e00640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f72cc067b20 0x7f72cc069fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0af000 tx=0x7f72dc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:18.973+0000 7f72e2000640 1 -- 172.21.15.86:0/2902000220 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f72ec13b490 con 0x7f72ec0cf0f0 2024-03-22T04:13:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72ec074470 con 0x7f72ec0cf0f0 2024-03-22T04:13:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72e2000640 1 -- 172.21.15.86:0/2902000220 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16931 (secure 0 0 0) 0x7f72d8055690 con 0x7f72ec0cf0f0 2024-03-22T04:13:19.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:19.136 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":56,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:15.973172+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:12:32.314387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-22T04:12:32.945472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2c","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f72cc067b20 msgr2=0x7f72cc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f72cc067b20 0x7f72cc069fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0af000 tx=0x7f72dc0a7000 comp rx=0 tx=0).stop 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0cf0f0 msgr2=0x7f72ec0d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0cf0f0 0x7f72ec0d08e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f72d800c520 tx=0x7f72d800c9e0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 shutdown_connections 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f72cc067b20 0x7f72cc069fe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ec155430 0x7f72ec0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72ec0cf0f0 0x7f72ec0d08e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 --2- 172.21.15.86:0/2902000220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72ec0ced10 0x7f72ec0b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 >> 172.21.15.86:0/2902000220 conn(0x7f72ec06b830 msgr2=0x7f72ec0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 shutdown_connections 2024-03-22T04:13:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.133+0000 7f72eb200640 1 -- 172.21.15.86:0/2902000220 wait complete. 2024-03-22T04:13:19.157 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:12:32.314387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:12:32.945472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:13:19.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:13:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f4044a00640 1 Processor -- start 2024-03-22T04:13:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f4044a00640 1 -- start start 2024-03-22T04:13:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f4044a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f404006e7e0 0x7f404006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f4044a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4040072650 0x7f4040072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f4044a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 0x7f40400b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f4044a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4040071350 con 0x7f4040072650 2024-03-22T04:13:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f4044a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f404006f100 con 0x7f404006e7e0 2024-03-22T04:13:19.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f4044a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4040073150 con 0x7f40400b1b80 2024-03-22T04:13:19.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4040072650 0x7f4040072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4040072650 0x7f4040072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57332/0 (socket says 172.21.15.86:57332) 2024-03-22T04:13:19.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403d400640 1 -- 172.21.15.86:0/3744213547 learned_addr learned my addr 172.21.15.86:0/3744213547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:19.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403e800640 1 --2- 172.21.15.86:0/3744213547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 0x7f40400b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403de00640 1 --2- 172.21.15.86:0/3744213547 >> v2:172.21.15.196:3300/0 conn(0x7f404006e7e0 0x7f404006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403d400640 1 -- 172.21.15.86:0/3744213547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 msgr2=0x7f40400b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403d400640 1 --2- 172.21.15.86:0/3744213547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 0x7f40400b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403d400640 1 -- 172.21.15.86:0/3744213547 >> v2:172.21.15.196:3300/0 conn(0x7f404006e7e0 msgr2=0x7f404006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403d400640 1 --2- 172.21.15.86:0/3744213547 >> v2:172.21.15.196:3300/0 conn(0x7f404006e7e0 0x7f404006ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.293+0000 7f403d400640 1 -- 172.21.15.86:0/3744213547 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40400715a0 con 0x7f4040072650 2024-03-22T04:13:19.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f403d400640 1 --2- 172.21.15.86:0/3744213547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4040072650 0x7f4040072b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40400ce650 tx=0x7f402c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6967d18498da94 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f403ca00640 1 -- 172.21.15.86:0/3744213547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0b0070 con 0x7f4040072650 2024-03-22T04:13:19.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f403ca00640 1 -- 172.21.15.86:0/3744213547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402c0ab600 con 0x7f4040072650 2024-03-22T04:13:19.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f403ca00640 1 -- 172.21.15.86:0/3744213547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0ab9d0 con 0x7f4040072650 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 -- 172.21.15.86:0/3744213547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4040072650 msgr2=0x7f4040072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 --2- 172.21.15.86:0/3744213547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4040072650 0x7f4040072b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40400ce650 tx=0x7f402c01c460 comp rx=0 tx=0).stop 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 -- 172.21.15.86:0/3744213547 shutdown_connections 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 --2- 172.21.15.86:0/3744213547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 0x7f40400b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 --2- 172.21.15.86:0/3744213547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4040072650 0x7f4040072b00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 --2- 172.21.15.86:0/3744213547 >> v2:172.21.15.196:3300/0 conn(0x7f404006e7e0 0x7f404006ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 -- 172.21.15.86:0/3744213547 >> 172.21.15.86:0/3744213547 conn(0x7f40400cc940 msgr2=0x7f40400ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 -- 172.21.15.86:0/3744213547 shutdown_connections 2024-03-22T04:13:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 -- 172.21.15.86:0/3744213547 wait complete. 2024-03-22T04:13:19.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.297+0000 7f4044a00640 1 Processor -- start 2024-03-22T04:13:19.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 -- start start 2024-03-22T04:13:19.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f404006e7e0 0x7f40401435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4040072650 0x7f40400a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 0x7f404014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4040072080 con 0x7f404006e7e0 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40400ce930 con 0x7f4040072650 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4040143b00 con 0x7f40400b1b80 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f404006e7e0 0x7f40401435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f404006e7e0 0x7f40401435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57346/0 (socket says 172.21.15.86:57346) 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 -- 172.21.15.86:0/1032250980 learned_addr learned my addr 172.21.15.86:0/1032250980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403d400640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4040072650 0x7f40400a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403e800640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 0x7f404014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 -- 172.21.15.86:0/1032250980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 msgr2=0x7f404014fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 0x7f404014fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 -- 172.21.15.86:0/1032250980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4040072650 msgr2=0x7f40400a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4040072650 0x7f40400a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 -- 172.21.15.86:0/1032250980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4040159b10 con 0x7f404006e7e0 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f403de00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f404006e7e0 0x7f40401435c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f403000e870 tx=0x7f403000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4036000640 1 -- 172.21.15.86:0/1032250980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030017070 con 0x7f404006e7e0 2024-03-22T04:13:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4036000640 1 -- 172.21.15.86:0/1032250980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403000ee60 con 0x7f404006e7e0 2024-03-22T04:13:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4036000640 1 -- 172.21.15.86:0/1032250980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403000cb80 con 0x7f404006e7e0 2024-03-22T04:13:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4040159db0 con 0x7f404006e7e0 2024-03-22T04:13:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40401430a0 con 0x7f404006e7e0 2024-03-22T04:13:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.301+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4040074470 con 0x7f404006e7e0 2024-03-22T04:13:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.305+0000 7f4036000640 1 -- 172.21.15.86:0/1032250980 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f403001a070 con 0x7f404006e7e0 2024-03-22T04:13:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.305+0000 7f4036000640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4020067b20 0x7f4020069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.305+0000 7f4036000640 1 -- 172.21.15.86:0/1032250980 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f4030085f10 con 0x7f404006e7e0 2024-03-22T04:13:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.305+0000 7f403d400640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4020067b20 0x7f4020069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.305+0000 7f403d400640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4020067b20 0x7f4020069fe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f402c0ad000 tx=0x7f402c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.309+0000 7f4036000640 1 -- 172.21.15.86:0/1032250980 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4030056110 con 0x7f404006e7e0 2024-03-22T04:13:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.461+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4040072200 con 0x7f404006e7e0 2024-03-22T04:13:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.461+0000 7f4036000640 1 -- 172.21.15.86:0/1032250980 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7f403005a1b0 con 0x7f404006e7e0 2024-03-22T04:13:19.465 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4020067b20 msgr2=0x7f4020069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4020067b20 0x7f4020069fe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f402c0ad000 tx=0x7f402c008040 comp rx=0 tx=0).stop 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f404006e7e0 msgr2=0x7f40401435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f404006e7e0 0x7f40401435c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f403000e870 tx=0x7f403000ed30 comp rx=0 tx=0).stop 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 shutdown_connections 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4020067b20 0x7f4020069fe0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4040072650 0x7f40400a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40400b1b80 0x7f404014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 --2- 172.21.15.86:0/1032250980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f404006e7e0 0x7f40401435c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 >> 172.21.15.86:0/1032250980 conn(0x7f40400cc940 msgr2=0x7f40401517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 shutdown_connections 2024-03-22T04:13:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.465+0000 7f4044a00640 1 -- 172.21.15.86:0/1032250980 wait complete. 2024-03-22T04:13:19.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:13:19.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.621+0000 7f2e7e600640 1 Processor -- start 2024-03-22T04:13:19.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.621+0000 7f2e7e600640 1 -- start start 2024-03-22T04:13:19.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.621+0000 7f2e7e600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2e7815e700 0x7f2e780b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.621+0000 7f2e7e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e78073a50 0x7f2e78072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.621+0000 7f2e7e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e780d0e50 0x7f2e78073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.621+0000 7f2e7e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2e780cdf00 con 0x7f2e78073a50 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.621+0000 7f2e7e600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2e78068f60 con 0x7f2e7815e700 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.621+0000 7f2e7e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2e780cfb50 con 0x7f2e780d0e50 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e78073a50 0x7f2e78072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e78073a50 0x7f2e78072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57354/0 (socket says 172.21.15.86:57354) 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3052915773 learned_addr learned my addr 172.21.15.86:0/3052915773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e77400640 1 --2- 172.21.15.86:0/3052915773 >> v2:172.21.15.196:3300/0 conn(0x7f2e7815e700 0x7f2e780b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e77e00640 1 --2- 172.21.15.86:0/3052915773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e780d0e50 0x7f2e78073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3052915773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e780d0e50 msgr2=0x7f2e78073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 --2- 172.21.15.86:0/3052915773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e780d0e50 0x7f2e78073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3052915773 >> v2:172.21.15.196:3300/0 conn(0x7f2e7815e700 msgr2=0x7f2e780b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 --2- 172.21.15.86:0/3052915773 >> v2:172.21.15.196:3300/0 conn(0x7f2e7815e700 0x7f2e780b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3052915773 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e780ce150 con 0x7f2e78073a50 2024-03-22T04:13:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e6ea00640 1 --2- 172.21.15.86:0/3052915773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e78073a50 0x7f2e78072d10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e78070a70 tx=0x7f2e6001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34da1f696520ca82 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e76a00640 1 -- 172.21.15.86:0/3052915773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600aa430 con 0x7f2e78073a50 2024-03-22T04:13:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e76a00640 1 -- 172.21.15.86:0/3052915773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e600af070 con 0x7f2e78073a50 2024-03-22T04:13:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.625+0000 7f2e76a00640 1 -- 172.21.15.86:0/3052915773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600aa7f0 con 0x7f2e78073a50 2024-03-22T04:13:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- 172.21.15.86:0/3052915773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e78073a50 msgr2=0x7f2e78072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 --2- 172.21.15.86:0/3052915773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e78073a50 0x7f2e78072d10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e78070a70 tx=0x7f2e6001c4f0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- 172.21.15.86:0/3052915773 shutdown_connections 2024-03-22T04:13:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 --2- 172.21.15.86:0/3052915773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e780d0e50 0x7f2e78073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 --2- 172.21.15.86:0/3052915773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e78073a50 0x7f2e78072d10 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 --2- 172.21.15.86:0/3052915773 >> v2:172.21.15.196:3300/0 conn(0x7f2e7815e700 0x7f2e780b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- 172.21.15.86:0/3052915773 >> 172.21.15.86:0/3052915773 conn(0x7f2e780d0330 msgr2=0x7f2e780d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- 172.21.15.86:0/3052915773 shutdown_connections 2024-03-22T04:13:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- 172.21.15.86:0/3052915773 wait complete. 2024-03-22T04:13:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 Processor -- start 2024-03-22T04:13:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- start start 2024-03-22T04:13:19.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e78073a50 0x7f2e780996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780d0e50 0x7f2e781543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e7815e700 0x7f2e78154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7815ec60 con 0x7f2e780d0e50 2024-03-22T04:13:19.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7815eae0 con 0x7f2e7815e700 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e7e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2e780b1110 con 0x7f2e78073a50 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780d0e50 0x7f2e781543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780d0e50 0x7f2e781543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57356/0 (socket says 172.21.15.86:57356) 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 -- 172.21.15.86:0/4204524078 learned_addr learned my addr 172.21.15.86:0/4204524078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e77e00640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e7815e700 0x7f2e78154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e77400640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e78073a50 0x7f2e780996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 -- 172.21.15.86:0/4204524078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e78073a50 msgr2=0x7f2e780996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e78073a50 0x7f2e780996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 -- 172.21.15.86:0/4204524078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e7815e700 msgr2=0x7f2e78154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e7815e700 0x7f2e78154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 -- 172.21.15.86:0/4204524078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7815bc30 con 0x7f2e780d0e50 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.629+0000 7f2e6ea00640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780d0e50 0x7f2e781543d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2e600ac040 tx=0x7f2e6001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e6fe00640 1 -- 172.21.15.86:0/4204524078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600bd3f0 con 0x7f2e780d0e50 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e6fe00640 1 -- 172.21.15.86:0/4204524078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e600bd5b0 con 0x7f2e780d0e50 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7815bf10 con 0x7f2e780d0e50 2024-03-22T04:13:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e6fe00640 1 -- 172.21.15.86:0/4204524078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600bd9a0 con 0x7f2e780d0e50 2024-03-22T04:13:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7815c4d0 con 0x7f2e780d0e50 2024-03-22T04:13:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e78074470 con 0x7f2e780d0e50 2024-03-22T04:13:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e6fe00640 1 -- 172.21.15.86:0/4204524078 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2e600b33f0 con 0x7f2e780d0e50 2024-03-22T04:13:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e6fe00640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2e54067b20 0x7f2e54069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e6fe00640 1 -- 172.21.15.86:0/4204524078 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f2e601204c0 con 0x7f2e780d0e50 2024-03-22T04:13:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e77400640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2e54067b20 0x7f2e54069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.633+0000 7f2e77400640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2e54067b20 0x7f2e54069fe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2e781faca0 tx=0x7f2e68009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.637+0000 7f2e6fe00640 1 -- 172.21.15.86:0/4204524078 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2e600c3060 con 0x7f2e780d0e50 2024-03-22T04:13:19.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e78073590 con 0x7f2e780d0e50 2024-03-22T04:13:19.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e6fe00640 1 -- 172.21.15.86:0/4204524078 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) ==== 104+0+11 (secure 0 0 0) 0x7f2e60016110 con 0x7f2e780d0e50 2024-03-22T04:13:19.795 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:13:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2e54067b20 msgr2=0x7f2e54069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2e54067b20 0x7f2e54069fe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2e781faca0 tx=0x7f2e68009190 comp rx=0 tx=0).stop 2024-03-22T04:13:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780d0e50 msgr2=0x7f2e781543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780d0e50 0x7f2e781543d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2e600ac040 tx=0x7f2e6001ad30 comp rx=0 tx=0).stop 2024-03-22T04:13:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 shutdown_connections 2024-03-22T04:13:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2e54067b20 0x7f2e54069fe0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e7815e700 0x7f2e78154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780d0e50 0x7f2e781543d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 --2- 172.21.15.86:0/4204524078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e78073a50 0x7f2e780996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 >> 172.21.15.86:0/4204524078 conn(0x7f2e780d0330 msgr2=0x7f2e780cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 shutdown_connections 2024-03-22T04:13:19.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.793+0000 7f2e7e600640 1 -- 172.21.15.86:0/4204524078 wait complete. 2024-03-22T04:13:19.816 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:13:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4799200640 1 Processor -- start 2024-03-22T04:13:19.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4799200640 1 -- start start 2024-03-22T04:13:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4799200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47940b3600 0x7f479415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4799200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f479415d710 con 0x7f47940b2360 2024-03-22T04:13:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4799200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f479415fe40 con 0x7f479415cee0 2024-03-22T04:13:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4799200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f47940b2740 con 0x7f47940b3600 2024-03-22T04:13:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4789400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47940b3600 0x7f479415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4789400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47940b3600 0x7f479415c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57372/0 (socket says 172.21.15.86:57372) 2024-03-22T04:13:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4789400640 1 -- 172.21.15.86:0/3768421289 learned_addr learned my addr 172.21.15.86:0/3768421289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4791e00640 1 --1- 172.21.15.86:0/3768421289 >> v1:172.21.15.86:6789/0 conn(0x7f47940b2360 0x7f47940b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47632/0 (socket says 172.21.15.86:47632) 2024-03-22T04:13:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4789400640 1 -- 172.21.15.86:0/3768421289 >> v1:172.21.15.196:6789/0 conn(0x7f479415cee0 legacy=0x7f47940b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4789400640 1 -- 172.21.15.86:0/3768421289 >> v1:172.21.15.86:6789/0 conn(0x7f47940b2360 legacy=0x7f47940b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.953+0000 7f4789400640 1 -- 172.21.15.86:0/3768421289 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4794160120 con 0x7f47940b3600 2024-03-22T04:13:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4789400640 1 --2- 172.21.15.86:0/3768421289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940b3600 0x7f479415c9a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f477c009870 tx=0x7f477c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740847c0471dfe69 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4791400640 1 -- 172.21.15.86:0/3768421289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c0a95f0 con 0x7f47940b3600 2024-03-22T04:13:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4791400640 1 -- 172.21.15.86:0/3768421289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c0a9790 con 0x7f47940b3600 2024-03-22T04:13:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4791400640 1 -- 172.21.15.86:0/3768421289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c0a9b60 con 0x7f47940b3600 2024-03-22T04:13:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4799200640 1 -- 172.21.15.86:0/3768421289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940b3600 msgr2=0x7f479415c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4799200640 1 --2- 172.21.15.86:0/3768421289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940b3600 0x7f479415c9a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f477c009870 tx=0x7f477c01c530 comp rx=0 tx=0).stop 2024-03-22T04:13:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4799200640 1 -- 172.21.15.86:0/3768421289 shutdown_connections 2024-03-22T04:13:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4799200640 1 --2- 172.21.15.86:0/3768421289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940b3600 0x7f479415c9a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4799200640 1 -- 172.21.15.86:0/3768421289 >> 172.21.15.86:0/3768421289 conn(0x7f47940cc940 msgr2=0x7f47940ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4799200640 1 -- 172.21.15.86:0/3768421289 shutdown_connections 2024-03-22T04:13:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.957+0000 7f4799200640 1 -- 172.21.15.86:0/3768421289 wait complete. 2024-03-22T04:13:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 Processor -- start 2024-03-22T04:13:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 -- start start 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47940b2360 0x7f47941487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940b3600 0x7f479415efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f479415cee0 0x7f479415f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f479415d470 con 0x7f479415cee0 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f479415d2f0 con 0x7f47940b3600 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f479416a1f0 con 0x7f47940b2360 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f479415cee0 0x7f479415f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f479415cee0 0x7f479415f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57388/0 (socket says 172.21.15.86:57388) 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 -- 172.21.15.86:0/3667548790 learned_addr learned my addr 172.21.15.86:0/3667548790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4789400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940b3600 0x7f479415efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4791e00640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47940b2360 0x7f47941487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 -- 172.21.15.86:0/3667548790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47940b2360 msgr2=0x7f47941487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47940b2360 0x7f47941487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 -- 172.21.15.86:0/3667548790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940b3600 msgr2=0x7f479415efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940b3600 0x7f479415efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 -- 172.21.15.86:0/3667548790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47941f7650 con 0x7f479415cee0 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4792800640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f479415cee0 0x7f479415f520 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f478c00c340 tx=0x7f478c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f478aa00640 1 -- 172.21.15.86:0/3667548790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c005220 con 0x7f479415cee0 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f478aa00640 1 -- 172.21.15.86:0/3667548790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f478c0053c0 con 0x7f479415cee0 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f478aa00640 1 -- 172.21.15.86:0/3667548790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c007270 con 0x7f479415cee0 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47941f78e0 con 0x7f479415cee0 2024-03-22T04:13:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941f7e60 con 0x7f479415cee0 2024-03-22T04:13:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.961+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4794074470 con 0x7f479415cee0 2024-03-22T04:13:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.965+0000 7f478aa00640 1 -- 172.21.15.86:0/3667548790 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f478c019020 con 0x7f479415cee0 2024-03-22T04:13:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.965+0000 7f478aa00640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4770067b20 0x7f4770069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.965+0000 7f478aa00640 1 -- 172.21.15.86:0/3667548790 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f478c0852f0 con 0x7f479415cee0 2024-03-22T04:13:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.965+0000 7f4791e00640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4770067b20 0x7f4770069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.965+0000 7f4791e00640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4770067b20 0x7f4770069fe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f47840012f0 tx=0x7f4784001700 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:19.969+0000 7f478aa00640 1 -- 172.21.15.86:0/3667548790 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f478c055460 con 0x7f479415cee0 2024-03-22T04:13:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f479415fc20 con 0x7f479415cee0 2024-03-22T04:13:20.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f478aa00640 1 -- 172.21.15.86:0/3667548790 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) ==== 100+0+11 (secure 0 0 0) 0x7f478c059500 con 0x7f479415cee0 2024-03-22T04:13:20.128 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:13:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4770067b20 msgr2=0x7f4770069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4770067b20 0x7f4770069fe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f47840012f0 tx=0x7f4784001700 comp rx=0 tx=0).stop 2024-03-22T04:13:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f479415cee0 msgr2=0x7f479415f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f479415cee0 0x7f479415f520 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f478c00c340 tx=0x7f478c00c800 comp rx=0 tx=0).stop 2024-03-22T04:13:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 shutdown_connections 2024-03-22T04:13:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4770067b20 0x7f4770069fe0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47940b3600 0x7f479415efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f479415cee0 0x7f479415f520 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 --2- 172.21.15.86:0/3667548790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47940b2360 0x7f47941487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 >> 172.21.15.86:0/3667548790 conn(0x7f47940cc940 msgr2=0x7f47940cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 shutdown_connections 2024-03-22T04:13:20.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.125+0000 7f4799200640 1 -- 172.21.15.86:0/3667548790 wait complete. 2024-03-22T04:13:20.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:13:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.281+0000 7face5600640 1 Processor -- start 2024-03-22T04:13:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.281+0000 7face5600640 1 -- start start 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7face5600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7face0161880 0x7face00b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7face5600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7face006de10 0x7face00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7face5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face006e350 con 0x7face0161c60 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7face5600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7face015ee80 con 0x7face006de10 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7face5600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7face006e630 con 0x7face0161880 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facdf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7face006de10 0x7face00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facd6000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7face0161c60 0x7face006d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47646/0 (socket says 172.21.15.86:47646) 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facd6000640 1 -- 172.21.15.86:0/697404880 learned_addr learned my addr 172.21.15.86:0/697404880 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facdea00640 1 --2- 172.21.15.86:0/697404880 >> v2:172.21.15.196:3300/0 conn(0x7face0161880 0x7face00b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facdf400640 1 -- 172.21.15.86:0/697404880 >> v2:172.21.15.196:3300/0 conn(0x7face0161880 msgr2=0x7face00b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:20.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facdf400640 1 --2- 172.21.15.86:0/697404880 >> v2:172.21.15.196:3300/0 conn(0x7face0161880 0x7face00b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facdf400640 1 -- 172.21.15.86:0/697404880 >> v1:172.21.15.86:6789/0 conn(0x7face0161c60 legacy=0x7face006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facdf400640 1 -- 172.21.15.86:0/697404880 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face006e910 con 0x7face006de10 2024-03-22T04:13:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facde000640 1 -- 172.21.15.86:0/697404880 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1163121176 0 0) 0x7face006e350 con 0x7face0161c60 2024-03-22T04:13:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.285+0000 7facdf400640 1 --2- 172.21.15.86:0/697404880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face00b3630 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7face006c510 tx=0x7facd801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d04510a18c22c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7facde000640 1 -- 172.21.15.86:0/697404880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80b2070 con 0x7face006de10 2024-03-22T04:13:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7facde000640 1 -- 172.21.15.86:0/697404880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd80a5930 con 0x7face006de10 2024-03-22T04:13:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7facde000640 1 -- 172.21.15.86:0/697404880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80a5d00 con 0x7face006de10 2024-03-22T04:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- 172.21.15.86:0/697404880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 msgr2=0x7face00b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 --2- 172.21.15.86:0/697404880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face00b3630 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7face006c510 tx=0x7facd801c870 comp rx=0 tx=0).stop 2024-03-22T04:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- 172.21.15.86:0/697404880 shutdown_connections 2024-03-22T04:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 --2- 172.21.15.86:0/697404880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face00b3630 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 --2- 172.21.15.86:0/697404880 >> v2:172.21.15.196:3300/0 conn(0x7face0161880 0x7face00b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- 172.21.15.86:0/697404880 >> 172.21.15.86:0/697404880 conn(0x7face00b1df0 msgr2=0x7face00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- 172.21.15.86:0/697404880 shutdown_connections 2024-03-22T04:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- 172.21.15.86:0/697404880 wait complete. 2024-03-22T04:13:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 Processor -- start 2024-03-22T04:13:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- start start 2024-03-22T04:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face0099560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face0161880 0x7face015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7face0161c60 0x7face015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7face01608d0 con 0x7face006de10 2024-03-22T04:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7face0160750 con 0x7face0161880 2024-03-22T04:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7face5600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7face0160a50 con 0x7face0161c60 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7facdea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face0099560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7facdea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face0099560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57400/0 (socket says 172.21.15.86:57400) 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7facdea00640 1 -- 172.21.15.86:0/541897293 learned_addr learned my addr 172.21.15.86:0/541897293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7facdf400640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7face0161c60 0x7face015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.289+0000 7facd6000640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face0161880 0x7face015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facdea00640 1 -- 172.21.15.86:0/541897293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7face0161c60 msgr2=0x7face015c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facdea00640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7face0161c60 0x7face015c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facdea00640 1 -- 172.21.15.86:0/541897293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face0161880 msgr2=0x7face015c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facdea00640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face0161880 0x7face015c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facdea00640 1 -- 172.21.15.86:0/541897293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face00b18c0 con 0x7face006de10 2024-03-22T04:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facdea00640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face0099560 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7facd000d800 tx=0x7facd000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facd7400640 1 -- 172.21.15.86:0/541897293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd000b6c0 con 0x7face006de10 2024-03-22T04:13:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facd7400640 1 -- 172.21.15.86:0/541897293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd0019070 con 0x7face006de10 2024-03-22T04:13:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7face5600640 1 -- 172.21.15.86:0/541897293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face0159500 con 0x7face006de10 2024-03-22T04:13:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facd7400640 1 -- 172.21.15.86:0/541897293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd000ba60 con 0x7face006de10 2024-03-22T04:13:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7face5600640 1 -- 172.21.15.86:0/541897293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face0159ac0 con 0x7face006de10 2024-03-22T04:13:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7face5600640 1 -- 172.21.15.86:0/541897293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face0068650 con 0x7face006de10 2024-03-22T04:13:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facd7400640 1 -- 172.21.15.86:0/541897293 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7facd000bc00 con 0x7face006de10 2024-03-22T04:13:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facd7400640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facbc067b50 0x7facbc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facd7400640 1 -- 172.21.15.86:0/541897293 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7facd0086460 con 0x7face006de10 2024-03-22T04:13:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.293+0000 7facd6000640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facbc067b50 0x7facbc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.297+0000 7facd6000640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facbc067b50 0x7facbc06a010 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7face01fab90 tx=0x7facc8007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.297+0000 7facd7400640 1 -- 172.21.15.86:0/541897293 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7facd00565d0 con 0x7face006de10 2024-03-22T04:13:20.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 -- 172.21.15.86:0/541897293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7face00b1b60 con 0x7face006de10 2024-03-22T04:13:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7facd7400640 1 -- 172.21.15.86:0/541897293 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) ==== 80+0+0 (secure 0 0 0) 0x7facd005a670 con 0x7face006de10 2024-03-22T04:13:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 -- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facbc067b50 msgr2=0x7facbc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facbc067b50 0x7facbc06a010 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7face01fab90 tx=0x7facc8007d60 comp rx=0 tx=0).stop 2024-03-22T04:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 -- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 msgr2=0x7face0099560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face0099560 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7facd000d800 tx=0x7facd000dcc0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 -- 172.21.15.86:0/541897293 shutdown_connections 2024-03-22T04:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facbc067b50 0x7facbc06a010 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7face0161880 0x7face015c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7face0161c60 0x7face015c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 --2- 172.21.15.86:0/541897293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face006de10 0x7face0099560 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 -- 172.21.15.86:0/541897293 >> 172.21.15.86:0/541897293 conn(0x7face00b1df0 msgr2=0x7face015de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:20.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 -- 172.21.15.86:0/541897293 shutdown_connections 2024-03-22T04:13:20.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.465+0000 7face5600640 1 -- 172.21.15.86:0/541897293 wait complete. 2024-03-22T04:13:20.487 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:13:20.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:20.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:13:20.543 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:13:20.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:20.543 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:13:20.556 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:13:20.556 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:13:20.556 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:20.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:13:20.570 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:13:20.570 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:13:20.570 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:20.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:13:20.606 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:13:20.606 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:13:20.606 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:20.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:13:20.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:20.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f381f200640 1 Processor -- start 2024-03-22T04:13:20.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f381f200640 1 -- start start 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f381f200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38180ad330 0x7f381815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f381f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f381f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f381815c7c0 con 0x7f38180ad710 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f381f200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f381806dc40 con 0x7f38180ad330 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f381f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f381806dea0 con 0x7f381806ef40 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f3817e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38180ad330 0x7f381815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f380f400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f380f400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381806d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:45792/0 (socket says 172.21.15.86:45792) 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f380f400640 1 -- 172.21.15.86:0/3688621900 learned_addr learned my addr 172.21.15.86:0/3688621900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f3817e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38180ad330 0x7f381815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:46650/0 (socket says 172.21.15.86:46650) 2024-03-22T04:13:20.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f380f400640 1 -- 172.21.15.86:0/3688621900 >> v2:172.21.15.196:3300/0 conn(0x7f38180ad330 msgr2=0x7f381815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:20.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f380f400640 1 --2- 172.21.15.86:0/3688621900 >> v2:172.21.15.196:3300/0 conn(0x7f38180ad330 0x7f381815d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f380f400640 1 -- 172.21.15.86:0/3688621900 >> v1:172.21.15.196:6789/0 conn(0x7f38180ad710 legacy=0x7f381813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f380f400640 1 -- 172.21.15.86:0/3688621900 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381806e180 con 0x7f381806ef40 2024-03-22T04:13:20.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.781+0000 7f3817e00640 1 --2- 172.21.15.86:0/3688621900 >> v2:172.21.15.196:3300/0 conn(0x7f38180ad330 0x7f381815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:20.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f380f400640 1 --2- 172.21.15.86:0/3688621900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381806d700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3800009870 tx=0x7f380001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7275b1abcff21d5f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f3817400640 1 -- 172.21.15.86:0/3688621900 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000ab640 con 0x7f381806ef40 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f3817400640 1 -- 172.21.15.86:0/3688621900 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38000b0070 con 0x7f381806ef40 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f3817400640 1 -- 172.21.15.86:0/3688621900 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000aba00 con 0x7f381806ef40 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f381f200640 1 -- 172.21.15.86:0/3688621900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 msgr2=0x7f381806d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f381f200640 1 --2- 172.21.15.86:0/3688621900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381806d700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3800009870 tx=0x7f380001c530 comp rx=0 tx=0).stop 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f381f200640 1 -- 172.21.15.86:0/3688621900 shutdown_connections 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f381f200640 1 --2- 172.21.15.86:0/3688621900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381806d700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f381f200640 1 --2- 172.21.15.86:0/3688621900 >> v2:172.21.15.196:3300/0 conn(0x7f38180ad330 0x7f381815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f381f200640 1 -- 172.21.15.86:0/3688621900 >> 172.21.15.86:0/3688621900 conn(0x7f38180b1df0 msgr2=0x7f38180b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f381f200640 1 -- 172.21.15.86:0/3688621900 shutdown_connections 2024-03-22T04:13:20.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.785+0000 7f381f200640 1 -- 172.21.15.86:0/3688621900 wait complete. 2024-03-22T04:13:20.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 Processor -- start 2024-03-22T04:13:20.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 -- start start 2024-03-22T04:13:20.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381809dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38180ad330 0x7f38181f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38180ad710 0x7f38181662e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38181690e0 con 0x7f38180ad330 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3818168f60 con 0x7f38180ad710 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3818169260 con 0x7f381806ef40 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38180ad330 0x7f38181f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38180ad330 0x7f38181f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57412/0 (socket says 172.21.15.86:57412) 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 -- 172.21.15.86:0/392419026 learned_addr learned my addr 172.21.15.86:0/392419026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f3817e00640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381809dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381d200640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38180ad710 0x7f38181662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 -- 172.21.15.86:0/392419026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 msgr2=0x7f381809dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381809dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 -- 172.21.15.86:0/392419026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38180ad710 msgr2=0x7f38181662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38180ad710 0x7f38181662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 -- 172.21.15.86:0/392419026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38180b4940 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f380f400640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38180ad330 0x7f38181f8450 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f380001a160 tx=0x7f380001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f3814c00640 1 -- 172.21.15.86:0/392419026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000be3f0 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f3814c00640 1 -- 172.21.15.86:0/392419026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38000be590 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f3814c00640 1 -- 172.21.15.86:0/392419026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000be980 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381815a310 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381815a8d0 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.789+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3818068650 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.793+0000 7f3814c00640 1 -- 172.21.15.86:0/392419026 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f38000b43f0 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.793+0000 7f3814c00640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f37f4067b20 0x7f37f4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.793+0000 7f3814c00640 1 -- 172.21.15.86:0/392419026 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f38001200f0 con 0x7f38180ad330 2024-03-22T04:13:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.793+0000 7f3817e00640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f37f4067b20 0x7f37f4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:20.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.793+0000 7f3817e00640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f37f4067b20 0x7f37f4069fe0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3808009580 tx=0x7f3808009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:20.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.797+0000 7f3814c00640 1 -- 172.21.15.86:0/392419026 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3800016150 con 0x7f38180ad330 2024-03-22T04:13:20.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38180b4720 con 0x7f38180ad330 2024-03-22T04:13:20.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f3814c00640 1 -- 172.21.15.86:0/392419026 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5189 (secure 0 0 0) 0x7f38000f0260 con 0x7f38180ad330 2024-03-22T04:13:20.991 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:20.992 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:12:33.981773+0000","modified":"2024-03-22T04:12:37.002106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75245":{"gid":75245,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6834/2728972599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2728972599},{"type":"v1","addr":"172.21.15.196:6834","nonce":2728972599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75245,"qdb_cluster":[75245]},"id":3}]} 2024-03-22T04:13:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T04:13:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f37f4067b20 msgr2=0x7f37f4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f37f4067b20 0x7f37f4069fe0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3808009580 tx=0x7f3808009190 comp rx=0 tx=0).stop 2024-03-22T04:13:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38180ad330 msgr2=0x7f38181f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38180ad330 0x7f38181f8450 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f380001a160 tx=0x7f380001ad70 comp rx=0 tx=0).stop 2024-03-22T04:13:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 shutdown_connections 2024-03-22T04:13:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f37f4067b20 0x7f37f4069fe0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38180ad710 0x7f38181662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38180ad330 0x7f38181f8450 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 --2- 172.21.15.86:0/392419026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381806ef40 0x7f381809dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 >> 172.21.15.86:0/392419026 conn(0x7f38180b1df0 msgr2=0x7f381815e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 shutdown_connections 2024-03-22T04:13:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:20.989+0000 7f381f200640 1 -- 172.21.15.86:0/392419026 wait complete. 2024-03-22T04:13:21.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae68a00640 1 Processor -- start 2024-03-22T04:13:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae68a00640 1 -- start start 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae68a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae640d0de0 0x7fae640ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae68a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d19a0 0x7fae640cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae68a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae640d11c0 con 0x7fae640d1dc0 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae68a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fae640cece0 con 0x7fae640d0de0 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae68a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae640cefc0 con 0x7fae640d19a0 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae61e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae640d0de0 0x7fae640ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae62800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fae640d1dc0 0x7fae6415fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47650/0 (socket says 172.21.15.86:47650) 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae62800640 1 -- 172.21.15.86:0/3151914399 learned_addr learned my addr 172.21.15.86:0/3151914399 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae61400640 1 --2- 172.21.15.86:0/3151914399 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d19a0 0x7fae640cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae61e00640 1 -- 172.21.15.86:0/3151914399 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d19a0 msgr2=0x7fae640cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae61e00640 1 --2- 172.21.15.86:0/3151914399 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d19a0 0x7fae640cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae61e00640 1 -- 172.21.15.86:0/3151914399 >> v1:172.21.15.86:6789/0 conn(0x7fae640d1dc0 legacy=0x7fae6415fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae61e00640 1 -- 172.21.15.86:0/3151914399 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae640cf2a0 con 0x7fae640d0de0 2024-03-22T04:13:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.145+0000 7fae60a00640 1 -- 172.21.15.86:0/3151914399 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407557448 0 0) 0x7fae640d11c0 con 0x7fae640d1dc0 2024-03-22T04:13:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae61e00640 1 --2- 172.21.15.86:0/3151914399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d0de0 0x7fae640ce6b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0097c0 tx=0x7fae4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2e7577eb8abb3f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae60a00640 1 -- 172.21.15.86:0/3151914399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0b2070 con 0x7fae640d0de0 2024-03-22T04:13:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae60a00640 1 -- 172.21.15.86:0/3151914399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4c0ad640 con 0x7fae640d0de0 2024-03-22T04:13:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae60a00640 1 -- 172.21.15.86:0/3151914399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0ada10 con 0x7fae640d0de0 2024-03-22T04:13:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae68a00640 1 -- 172.21.15.86:0/3151914399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d0de0 msgr2=0x7fae640ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae68a00640 1 --2- 172.21.15.86:0/3151914399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d0de0 0x7fae640ce6b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0097c0 tx=0x7fae4c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae68a00640 1 -- 172.21.15.86:0/3151914399 shutdown_connections 2024-03-22T04:13:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae68a00640 1 --2- 172.21.15.86:0/3151914399 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d19a0 0x7fae640cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae68a00640 1 --2- 172.21.15.86:0/3151914399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d0de0 0x7fae640ce6b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae68a00640 1 -- 172.21.15.86:0/3151914399 >> 172.21.15.86:0/3151914399 conn(0x7fae6406f910 msgr2=0x7fae6406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae68a00640 1 -- 172.21.15.86:0/3151914399 shutdown_connections 2024-03-22T04:13:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.149+0000 7fae68a00640 1 -- 172.21.15.86:0/3151914399 wait complete. 2024-03-22T04:13:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 Processor -- start 2024-03-22T04:13:21.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 -- start start 2024-03-22T04:13:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d0de0 0x7fae64153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae640d19a0 0x7fae6415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d1dc0 0x7fae6415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae64160760 con 0x7fae640d1dc0 2024-03-22T04:13:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae641605e0 con 0x7fae640d19a0 2024-03-22T04:13:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae64071e30 con 0x7fae640d0de0 2024-03-22T04:13:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d1dc0 0x7fae6415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae61e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d0de0 0x7fae64153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d1dc0 0x7fae6415a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57426/0 (socket says 172.21.15.86:57426) 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 -- 172.21.15.86:0/3170435060 learned_addr learned my addr 172.21.15.86:0/3170435060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae61400640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae640d19a0 0x7fae6415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 -- 172.21.15.86:0/3170435060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d0de0 msgr2=0x7fae64153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d0de0 0x7fae64153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 -- 172.21.15.86:0/3170435060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae640d19a0 msgr2=0x7fae6415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae640d19a0 0x7fae6415a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 -- 172.21.15.86:0/3170435060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae640b6f30 con 0x7fae640d1dc0 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae62800640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d1dc0 0x7fae6415a9b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c7b0 tx=0x7fae5c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae5a000640 1 -- 172.21.15.86:0/3170435060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c018070 con 0x7fae640d1dc0 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae5a000640 1 -- 172.21.15.86:0/3170435060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c00a640 con 0x7fae640d1dc0 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae5a000640 1 -- 172.21.15.86:0/3170435060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00aa70 con 0x7fae640d1dc0 2024-03-22T04:13:21.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae640b7210 con 0x7fae640d1dc0 2024-03-22T04:13:21.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae640b77d0 con 0x7fae640d1dc0 2024-03-22T04:13:21.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.153+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae64068650 con 0x7fae640d1dc0 2024-03-22T04:13:21.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.157+0000 7fae5a000640 1 -- 172.21.15.86:0/3170435060 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fae5c01b070 con 0x7fae640d1dc0 2024-03-22T04:13:21.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.157+0000 7fae5a000640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae40067b20 0x7fae40069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.157+0000 7fae5a000640 1 -- 172.21.15.86:0/3170435060 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fae5c085560 con 0x7fae640d1dc0 2024-03-22T04:13:21.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.157+0000 7fae61e00640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae40067b20 0x7fae40069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.157+0000 7fae61e00640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae40067b20 0x7fae40069fe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0af000 tx=0x7fae4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.161+0000 7fae5a000640 1 -- 172.21.15.86:0/3170435060 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fae5c055800 con 0x7fae640d1dc0 2024-03-22T04:13:21.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.317+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae6406ffd0 con 0x7fae640d1dc0 2024-03-22T04:13:21.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.317+0000 7fae5a000640 1 -- 172.21.15.86:0/3170435060 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16931 (secure 0 0 0) 0x7fae5c0598a0 con 0x7fae640d1dc0 2024-03-22T04:13:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:21.321 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":56,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:15.973172+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:12:32.314387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-22T04:12:32.945472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2c","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:21.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae40067b20 msgr2=0x7fae40069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae40067b20 0x7fae40069fe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0af000 tx=0x7fae4c008040 comp rx=0 tx=0).stop 2024-03-22T04:13:21.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d1dc0 msgr2=0x7fae6415a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d1dc0 0x7fae6415a9b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c7b0 tx=0x7fae5c00cc70 comp rx=0 tx=0).stop 2024-03-22T04:13:21.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 shutdown_connections 2024-03-22T04:13:21.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae40067b20 0x7fae40069fe0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae640d19a0 0x7fae6415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae640d1dc0 0x7fae6415a9b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 --2- 172.21.15.86:0/3170435060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae640d0de0 0x7fae64153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 >> 172.21.15.86:0/3170435060 conn(0x7fae6406f910 msgr2=0x7fae6415daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:21.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 shutdown_connections 2024-03-22T04:13:21.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.321+0000 7fae68a00640 1 -- 172.21.15.86:0/3170435060 wait complete. 2024-03-22T04:13:21.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:12:32.314387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:12:32.945472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:13:21.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:13:21.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.477+0000 7fa8e8600640 1 Processor -- start 2024-03-22T04:13:21.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.477+0000 7fa8e8600640 1 -- start start 2024-03-22T04:13:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e8600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa8e00b3600 0x7fa8e015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e8600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8e015cee0 0x7fa8e00b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e8600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e015d710 con 0x7fa8e00b2360 2024-03-22T04:13:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e8600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e00b2740 con 0x7fa8e015cee0 2024-03-22T04:13:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e8600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e015fe40 con 0x7fa8e00b3600 2024-03-22T04:13:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e6600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8e015cee0 0x7fa8e00b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e6600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8e015cee0 0x7fa8e00b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57434/0 (socket says 172.21.15.86:57434) 2024-03-22T04:13:21.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e6600640 1 -- 172.21.15.86:0/3691934519 learned_addr learned my addr 172.21.15.86:0/3691934519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e5200640 1 --2- 172.21.15.86:0/3691934519 >> v2:172.21.15.196:3300/0 conn(0x7fa8e00b3600 0x7fa8e015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e6600640 1 -- 172.21.15.86:0/3691934519 >> v2:172.21.15.196:3300/0 conn(0x7fa8e00b3600 msgr2=0x7fa8e015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e6600640 1 --2- 172.21.15.86:0/3691934519 >> v2:172.21.15.196:3300/0 conn(0x7fa8e00b3600 0x7fa8e015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e6600640 1 -- 172.21.15.86:0/3691934519 >> v1:172.21.15.196:6789/0 conn(0x7fa8e00b2360 legacy=0x7fa8e00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.481+0000 7fa8e6600640 1 -- 172.21.15.86:0/3691934519 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e0160120 con 0x7fa8e015cee0 2024-03-22T04:13:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e6600640 1 --2- 172.21.15.86:0/3691934519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e015cee0 0x7fa8e00b4d80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00b2c00 tx=0x7fa8d801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f33006a9df54196 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8dfe00640 1 -- 172.21.15.86:0/3691934519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80aa650 con 0x7fa8e015cee0 2024-03-22T04:13:21.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8dfe00640 1 -- 172.21.15.86:0/3691934519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d80af070 con 0x7fa8e015cee0 2024-03-22T04:13:21.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8dfe00640 1 -- 172.21.15.86:0/3691934519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80aaa10 con 0x7fa8e015cee0 2024-03-22T04:13:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 -- 172.21.15.86:0/3691934519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e015cee0 msgr2=0x7fa8e00b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 --2- 172.21.15.86:0/3691934519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e015cee0 0x7fa8e00b4d80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00b2c00 tx=0x7fa8d801c4f0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 -- 172.21.15.86:0/3691934519 shutdown_connections 2024-03-22T04:13:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 --2- 172.21.15.86:0/3691934519 >> v2:172.21.15.196:3300/0 conn(0x7fa8e00b3600 0x7fa8e015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 --2- 172.21.15.86:0/3691934519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e015cee0 0x7fa8e00b4d80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 -- 172.21.15.86:0/3691934519 >> 172.21.15.86:0/3691934519 conn(0x7fa8e00cc940 msgr2=0x7fa8e00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:21.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 -- 172.21.15.86:0/3691934519 shutdown_connections 2024-03-22T04:13:21.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 -- 172.21.15.86:0/3691934519 wait complete. 2024-03-22T04:13:21.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 Processor -- start 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.485+0000 7fa8e8600640 1 -- start start 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8e00b2360 0x7fa8e01fa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e00b3600 0x7fa8e014c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e015cee0 0x7fa8e0149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e015d4d0 con 0x7fa8e00b3600 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e015d350 con 0x7fa8e015cee0 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e00b5ad0 con 0x7fa8e00b2360 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e00b3600 0x7fa8e014c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e00b3600 0x7fa8e014c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57446/0 (socket says 172.21.15.86:57446) 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 -- 172.21.15.86:0/993689587 learned_addr learned my addr 172.21.15.86:0/993689587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5c00640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8e00b2360 0x7fa8e01fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e6600640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e015cee0 0x7fa8e0149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 -- 172.21.15.86:0/993689587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8e00b2360 msgr2=0x7fa8e01fa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8e00b2360 0x7fa8e01fa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 -- 172.21.15.86:0/993689587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e015cee0 msgr2=0x7fa8e0149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e015cee0 0x7fa8e0149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 -- 172.21.15.86:0/993689587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e0149480 con 0x7fa8e00b3600 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e5200640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e00b3600 0x7fa8e014c4a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e960 tx=0x7fa8cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8dd600640 1 -- 172.21.15.86:0/993689587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc017070 con 0x7fa8e00b3600 2024-03-22T04:13:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8dd600640 1 -- 172.21.15.86:0/993689587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8cc00ca00 con 0x7fa8e00b3600 2024-03-22T04:13:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e01faba0 con 0x7fa8e00b3600 2024-03-22T04:13:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8dd600640 1 -- 172.21.15.86:0/993689587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00cdb0 con 0x7fa8e00b3600 2024-03-22T04:13:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e014d000 con 0x7fa8e00b3600 2024-03-22T04:13:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e0074470 con 0x7fa8e00b3600 2024-03-22T04:13:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.489+0000 7fa8dd600640 1 -- 172.21.15.86:0/993689587 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa8cc01a070 con 0x7fa8e00b3600 2024-03-22T04:13:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.493+0000 7fa8dd600640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8c0067b50 0x7fa8c006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.493+0000 7fa8dd600640 1 -- 172.21.15.86:0/993689587 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fa8cc0860e0 con 0x7fa8e00b3600 2024-03-22T04:13:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.493+0000 7fa8e5c00640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8c0067b50 0x7fa8c006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.493+0000 7fa8e5c00640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8c0067b50 0x7fa8c006a010 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4001ea0 tx=0x7fa8d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.497+0000 7fa8dd600640 1 -- 172.21.15.86:0/993689587 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa8cc056250 con 0x7fa8e00b3600 2024-03-22T04:13:21.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa8e006e9f0 con 0x7fa8e00b3600 2024-03-22T04:13:21.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8dd600640 1 -- 172.21.15.86:0/993689587 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7fa8cc05a2f0 con 0x7fa8e00b3600 2024-03-22T04:13:21.656 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8c0067b50 msgr2=0x7fa8c006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8c0067b50 0x7fa8c006a010 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4001ea0 tx=0x7fa8d4009040 comp rx=0 tx=0).stop 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e00b3600 msgr2=0x7fa8e014c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e00b3600 0x7fa8e014c4a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e960 tx=0x7fa8cc00ee20 comp rx=0 tx=0).stop 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 shutdown_connections 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8c0067b50 0x7fa8c006a010 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8e015cee0 0x7fa8e0149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e00b3600 0x7fa8e014c4a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 --2- 172.21.15.86:0/993689587 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8e00b2360 0x7fa8e01fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 >> 172.21.15.86:0/993689587 conn(0x7fa8e00cc940 msgr2=0x7fa8e00cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 shutdown_connections 2024-03-22T04:13:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.653+0000 7fa8e8600640 1 -- 172.21.15.86:0/993689587 wait complete. 2024-03-22T04:13:21.676 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:13:21.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd488e00640 1 Processor -- start 2024-03-22T04:13:21.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd488e00640 1 -- start start 2024-03-22T04:13:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd488e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd48415c730 0x7fd48409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd488e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b3610 0x7fd48409afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd488e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4840cda00 con 0x7fd4840b11c0 2024-03-22T04:13:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd488e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd48409b910 con 0x7fd48415c730 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd488e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd4840b1710 con 0x7fd4840b3610 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd482800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd4840b11c0 0x7fd4840d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47656/0 (socket says 172.21.15.86:47656) 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd482800640 1 -- 172.21.15.86:0/590353718 learned_addr learned my addr 172.21.15.86:0/590353718 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd481e00640 1 --2- 172.21.15.86:0/590353718 >> v2:172.21.15.196:3300/0 conn(0x7fd48415c730 0x7fd48409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd481400640 1 --2- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b3610 0x7fd48409afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd480a00640 1 -- 172.21.15.86:0/590353718 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3964046815 0 0) 0x7fd4840cda00 con 0x7fd4840b11c0 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd480a00640 1 -- 172.21.15.86:0/590353718 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd468003530 con 0x7fd4840b11c0 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd481e00640 1 -- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b3610 msgr2=0x7fd48409afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd481e00640 1 --2- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b3610 0x7fd48409afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd481e00640 1 -- 172.21.15.86:0/590353718 >> v1:172.21.15.86:6789/0 conn(0x7fd4840b11c0 legacy=0x7fd4840d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd481e00640 1 -- 172.21.15.86:0/590353718 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4840cdce0 con 0x7fd48415c730 2024-03-22T04:13:21.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.813+0000 7fd481400640 1 --2- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b3610 0x7fd48409afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:21.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd481e00640 1 --2- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd48415c730 0x7fd48409aa70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd4740097c0 tx=0x7fd47401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfba2c27706c6fe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd480a00640 1 -- 172.21.15.86:0/590353718 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740b0070 con 0x7fd48415c730 2024-03-22T04:13:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd480a00640 1 -- 172.21.15.86:0/590353718 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4740ab600 con 0x7fd48415c730 2024-03-22T04:13:21.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd480a00640 1 -- 172.21.15.86:0/590353718 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740ab9d0 con 0x7fd48415c730 2024-03-22T04:13:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd488e00640 1 -- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd48415c730 msgr2=0x7fd48409aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd488e00640 1 --2- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd48415c730 0x7fd48409aa70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd4740097c0 tx=0x7fd47401c3e0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd488e00640 1 -- 172.21.15.86:0/590353718 shutdown_connections 2024-03-22T04:13:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd488e00640 1 --2- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b3610 0x7fd48409afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd488e00640 1 --2- 172.21.15.86:0/590353718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd48415c730 0x7fd48409aa70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd488e00640 1 -- 172.21.15.86:0/590353718 >> 172.21.15.86:0/590353718 conn(0x7fd4840b3bf0 msgr2=0x7fd4840b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd488e00640 1 -- 172.21.15.86:0/590353718 shutdown_connections 2024-03-22T04:13:21.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.817+0000 7fd488e00640 1 -- 172.21.15.86:0/590353718 wait complete. 2024-03-22T04:13:21.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 Processor -- start 2024-03-22T04:13:21.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 -- start start 2024-03-22T04:13:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b11c0 0x7fd4840cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4840b3610 0x7fd48415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48415c730 0x7fd4840c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd484169510 con 0x7fd48415c730 2024-03-22T04:13:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd484169390 con 0x7fd4840b3610 2024-03-22T04:13:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd484169690 con 0x7fd4840b11c0 2024-03-22T04:13:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48415c730 0x7fd4840c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48415c730 0x7fd4840c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57460/0 (socket says 172.21.15.86:57460) 2024-03-22T04:13:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 -- 172.21.15.86:0/1069399371 learned_addr learned my addr 172.21.15.86:0/1069399371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd481e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b11c0 0x7fd4840cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd481400640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4840b3610 0x7fd48415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 -- 172.21.15.86:0/1069399371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b11c0 msgr2=0x7fd4840cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b11c0 0x7fd4840cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 -- 172.21.15.86:0/1069399371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4840b3610 msgr2=0x7fd48415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4840b3610 0x7fd48415ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 -- 172.21.15.86:0/1069399371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48415b6b0 con 0x7fd48415c730 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd482800640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48415c730 0x7fd4840c6e10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00cad0 tx=0x7fd47c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd47a000640 1 -- 172.21.15.86:0/1069399371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c017070 con 0x7fd48415c730 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd47a000640 1 -- 172.21.15.86:0/1069399371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c00abd0 con 0x7fd48415c730 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4840c6ae0 con 0x7fd48415c730 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd47a000640 1 -- 172.21.15.86:0/1069399371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c012400 con 0x7fd48415c730 2024-03-22T04:13:21.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.821+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4840ca500 con 0x7fd48415c730 2024-03-22T04:13:21.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.825+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd484074470 con 0x7fd48415c730 2024-03-22T04:13:21.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.825+0000 7fd47a000640 1 -- 172.21.15.86:0/1069399371 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd47c01a070 con 0x7fd48415c730 2024-03-22T04:13:21.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.825+0000 7fd47a000640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd464067b20 0x7fd464069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:21.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.825+0000 7fd47a000640 1 -- 172.21.15.86:0/1069399371 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fd47c085e10 con 0x7fd48415c730 2024-03-22T04:13:21.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.825+0000 7fd481e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd464067b20 0x7fd464069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:21.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.825+0000 7fd481e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd464067b20 0x7fd464069fe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd4740ad000 tx=0x7fd4740a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:21.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.829+0000 7fd47a000640 1 -- 172.21.15.86:0/1069399371 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd47c089030 con 0x7fd48415c730 2024-03-22T04:13:21.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.985+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd48406f810 con 0x7fd48415c730 2024-03-22T04:13:21.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.985+0000 7fd47a000640 1 -- 172.21.15.86:0/1069399371 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) ==== 104+0+11 (secure 0 0 0) 0x7fd47c05c070 con 0x7fd48415c730 2024-03-22T04:13:21.989 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd464067b20 msgr2=0x7fd464069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd464067b20 0x7fd464069fe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd4740ad000 tx=0x7fd4740a6000 comp rx=0 tx=0).stop 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48415c730 msgr2=0x7fd4840c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48415c730 0x7fd4840c6e10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00cad0 tx=0x7fd47c00cf90 comp rx=0 tx=0).stop 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 shutdown_connections 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd464067b20 0x7fd464069fe0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4840b3610 0x7fd48415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd48415c730 0x7fd4840c6e10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 --2- 172.21.15.86:0/1069399371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4840b11c0 0x7fd4840cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 >> 172.21.15.86:0/1069399371 conn(0x7fd4840b3bf0 msgr2=0x7fd48409aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 shutdown_connections 2024-03-22T04:13:21.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:21.989+0000 7fd488e00640 1 -- 172.21.15.86:0/1069399371 wait complete. 2024-03-22T04:13:22.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:13:22.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.145+0000 7f6183c00640 1 Processor -- start 2024-03-22T04:13:22.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.145+0000 7f6183c00640 1 -- start start 2024-03-22T04:13:22.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.145+0000 7f6183c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f617c0c6f20 0x7f617c072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6183c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f617c162740 0x7f617c0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6183c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f617c0cc7c0 con 0x7f617c0cb1d0 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6183c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f617c0cb720 con 0x7f617c0c6f20 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6183c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f617c0cba00 con 0x7f617c162740 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f617c0c6f20 0x7f617c072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f617c0c6f20 0x7f617c072840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57470/0 (socket says 172.21.15.86:57470) 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181200640 1 -- 172.21.15.86:0/4087418344 learned_addr learned my addr 172.21.15.86:0/4087418344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181c00640 1 --2- 172.21.15.86:0/4087418344 >> v2:172.21.15.196:3300/0 conn(0x7f617c162740 0x7f617c0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181200640 1 -- 172.21.15.86:0/4087418344 >> v2:172.21.15.196:3300/0 conn(0x7f617c162740 msgr2=0x7f617c0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181200640 1 --2- 172.21.15.86:0/4087418344 >> v2:172.21.15.196:3300/0 conn(0x7f617c162740 0x7f617c0c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181200640 1 -- 172.21.15.86:0/4087418344 >> v1:172.21.15.196:6789/0 conn(0x7f617c0cb1d0 legacy=0x7f617c162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181200640 1 -- 172.21.15.86:0/4087418344 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c0cbce0 con 0x7f617c0c6f20 2024-03-22T04:13:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f6181200640 1 --2- 172.21.15.86:0/4087418344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c0c6f20 0x7f617c072840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f617c0cad30 tx=0x7f616c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b70049940bc3eab server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f617be00640 1 -- 172.21.15.86:0/4087418344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0aa650 con 0x7f617c0c6f20 2024-03-22T04:13:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f617be00640 1 -- 172.21.15.86:0/4087418344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c0af070 con 0x7f617c0c6f20 2024-03-22T04:13:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.149+0000 7f617be00640 1 -- 172.21.15.86:0/4087418344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0aaa10 con 0x7f617c0c6f20 2024-03-22T04:13:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- 172.21.15.86:0/4087418344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c0c6f20 msgr2=0x7f617c072840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 --2- 172.21.15.86:0/4087418344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c0c6f20 0x7f617c072840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f617c0cad30 tx=0x7f616c01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- 172.21.15.86:0/4087418344 shutdown_connections 2024-03-22T04:13:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 --2- 172.21.15.86:0/4087418344 >> v2:172.21.15.196:3300/0 conn(0x7f617c162740 0x7f617c0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 --2- 172.21.15.86:0/4087418344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c0c6f20 0x7f617c072840 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- 172.21.15.86:0/4087418344 >> 172.21.15.86:0/4087418344 conn(0x7f617c0d0730 msgr2=0x7f617c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- 172.21.15.86:0/4087418344 shutdown_connections 2024-03-22T04:13:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- 172.21.15.86:0/4087418344 wait complete. 2024-03-22T04:13:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 Processor -- start 2024-03-22T04:13:22.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- start start 2024-03-22T04:13:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f617c0c6f20 0x7f617c1535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c0cb1d0 0x7f617c1fb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c162740 0x7f617c0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f617c0c64c0 con 0x7f617c162740 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f617c0c6340 con 0x7f617c0cb1d0 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6183c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f617c0c6640 con 0x7f617c0c6f20 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c162740 0x7f617c0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c162740 0x7f617c0cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57478/0 (socket says 172.21.15.86:57478) 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181c00640 1 -- 172.21.15.86:0/1592547836 learned_addr learned my addr 172.21.15.86:0/1592547836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181200640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f617c0c6f20 0x7f617c1535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6173e00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c0cb1d0 0x7f617c1fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181c00640 1 -- 172.21.15.86:0/1592547836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f617c0c6f20 msgr2=0x7f617c1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f617c0c6f20 0x7f617c1535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181c00640 1 -- 172.21.15.86:0/1592547836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c0cb1d0 msgr2=0x7f617c1fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c0cb1d0 0x7f617c1fb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.153+0000 7f6181c00640 1 -- 172.21.15.86:0/1592547836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c06f040 con 0x7f617c162740 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6181c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c162740 0x7f617c0cd440 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6174009dd0 tx=0x7f617400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6179600640 1 -- 172.21.15.86:0/1592547836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174018020 con 0x7f617c162740 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6179600640 1 -- 172.21.15.86:0/1592547836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6174014480 con 0x7f617c162740 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c06f320 con 0x7f617c162740 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6179600640 1 -- 172.21.15.86:0/1592547836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174014870 con 0x7f617c162740 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c06f8e0 con 0x7f617c162740 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617c074470 con 0x7f617c162740 2024-03-22T04:13:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6179600640 1 -- 172.21.15.86:0/1592547836 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f617401b020 con 0x7f617c162740 2024-03-22T04:13:22.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6179600640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6158067b50 0x7f615806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6179600640 1 -- 172.21.15.86:0/1592547836 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7f6174086d40 con 0x7f617c162740 2024-03-22T04:13:22.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.157+0000 7f6181200640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6158067b50 0x7f615806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.161+0000 7f6181200640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6158067b50 0x7f615806a010 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f616c0ae000 tx=0x7f616c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.161+0000 7f6179600640 1 -- 172.21.15.86:0/1592547836 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6174022030 con 0x7f617c162740 2024-03-22T04:13:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.317+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f617c0cec70 con 0x7f617c162740 2024-03-22T04:13:22.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.317+0000 7f6179600640 1 -- 172.21.15.86:0/1592547836 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) ==== 100+0+11 (secure 0 0 0) 0x7f6174010040 con 0x7f617c162740 2024-03-22T04:13:22.321 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.317+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6158067b50 msgr2=0x7f615806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.317+0000 7f6183c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6158067b50 0x7f615806a010 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f616c0ae000 tx=0x7f616c0a7000 comp rx=0 tx=0).stop 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c162740 msgr2=0x7f617c0cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c162740 0x7f617c0cd440 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6174009dd0 tx=0x7f617400c490 comp rx=0 tx=0).stop 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 shutdown_connections 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6158067b50 0x7f615806a010 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c0cb1d0 0x7f617c1fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f617c162740 0x7f617c0cd440 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 --2- 172.21.15.86:0/1592547836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f617c0c6f20 0x7f617c1535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 >> 172.21.15.86:0/1592547836 conn(0x7f617c0d0730 msgr2=0x7f617c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:22.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 shutdown_connections 2024-03-22T04:13:22.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.321+0000 7f6183c00640 1 -- 172.21.15.86:0/1592547836 wait complete. 2024-03-22T04:13:22.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff876200640 1 Processor -- start 2024-03-22T04:13:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff876200640 1 -- start start 2024-03-22T04:13:22.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff876200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff87006ef40 0x7ff870142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff876200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87015c7c0 con 0x7ff8700ad330 2024-03-22T04:13:22.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff876200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87006dea0 con 0x7ff8700ad710 2024-03-22T04:13:22.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff876200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff87006e180 con 0x7ff87006ef40 2024-03-22T04:13:22.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff86ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff87006ef40 0x7ff870142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff86f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff8700ad330 0x7ff87013fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47666/0 (socket says 172.21.15.86:47666) 2024-03-22T04:13:22.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff86ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff87006ef40 0x7ff870142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57492/0 (socket says 172.21.15.86:57492) 2024-03-22T04:13:22.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff86f400640 1 -- 172.21.15.86:0/4061984702 learned_addr learned my addr 172.21.15.86:0/4061984702 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:22.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff86ea00640 1 -- 172.21.15.86:0/4061984702 >> v1:172.21.15.196:6789/0 conn(0x7ff8700ad710 legacy=0x7ff8701402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff86ea00640 1 -- 172.21.15.86:0/4061984702 >> v1:172.21.15.86:6789/0 conn(0x7ff8700ad330 legacy=0x7ff87013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.473+0000 7ff86ea00640 1 -- 172.21.15.86:0/4061984702 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87006e460 con 0x7ff87006ef40 2024-03-22T04:13:22.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff86ea00640 1 --2- 172.21.15.86:0/4061984702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87006ef40 0x7ff870142690 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0097c0 tx=0x7ff85c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecda0e308c4eb09b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff86e000640 1 -- 172.21.15.86:0/4061984702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0aa5f0 con 0x7ff87006ef40 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff86e000640 1 -- 172.21.15.86:0/4061984702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff85c0af070 con 0x7ff87006ef40 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff86e000640 1 -- 172.21.15.86:0/4061984702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0aa9b0 con 0x7ff87006ef40 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff876200640 1 -- 172.21.15.86:0/4061984702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87006ef40 msgr2=0x7ff870142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff876200640 1 --2- 172.21.15.86:0/4061984702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87006ef40 0x7ff870142690 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0097c0 tx=0x7ff85c01c470 comp rx=0 tx=0).stop 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff876200640 1 -- 172.21.15.86:0/4061984702 shutdown_connections 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff876200640 1 --2- 172.21.15.86:0/4061984702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87006ef40 0x7ff870142690 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff876200640 1 -- 172.21.15.86:0/4061984702 >> 172.21.15.86:0/4061984702 conn(0x7ff8700b1df0 msgr2=0x7ff8700b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff876200640 1 -- 172.21.15.86:0/4061984702 shutdown_connections 2024-03-22T04:13:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.477+0000 7ff876200640 1 -- 172.21.15.86:0/4061984702 wait complete. 2024-03-22T04:13:22.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 Processor -- start 2024-03-22T04:13:22.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 -- start start 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff87006ef40 0x7ff8701564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8700ad330 0x7ff8700995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8700ad710 0x7ff87015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff87006db10 con 0x7ff8700ad330 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8701409d0 con 0x7ff8700ad710 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff87016a780 con 0x7ff87006ef40 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8700ad330 0x7ff8700995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8700ad330 0x7ff8700995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57500/0 (socket says 172.21.15.86:57500) 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86f400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff87006ef40 0x7ff8701564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 -- 172.21.15.86:0/3089675283 learned_addr learned my addr 172.21.15.86:0/3089675283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86fe00640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8700ad710 0x7ff87015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 -- 172.21.15.86:0/3089675283 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff87006ef40 msgr2=0x7ff8701564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff87006ef40 0x7ff8701564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 -- 172.21.15.86:0/3089675283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8700ad710 msgr2=0x7ff87015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8700ad710 0x7ff87015dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 -- 172.21.15.86:0/3089675283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87015ec50 con 0x7ff8700ad330 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff86ea00640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8700ad330 0x7ff8700995f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff85c019080 tx=0x7ff85c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff867400640 1 -- 172.21.15.86:0/3089675283 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0be610 con 0x7ff8700ad330 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff867400640 1 -- 172.21.15.86:0/3089675283 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff85c0be7b0 con 0x7ff8700ad330 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87015ef30 con 0x7ff8700ad330 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff867400640 1 -- 172.21.15.86:0/3089675283 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0beba0 con 0x7ff8700ad330 2024-03-22T04:13:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.481+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87015fa20 con 0x7ff8700ad330 2024-03-22T04:13:22.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.485+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff870068650 con 0x7ff8700ad330 2024-03-22T04:13:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.485+0000 7ff867400640 1 -- 172.21.15.86:0/3089675283 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff85c0bc020 con 0x7ff8700ad330 2024-03-22T04:13:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.485+0000 7ff867400640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff850067b50 0x7ff85006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.485+0000 7ff867400640 1 -- 172.21.15.86:0/3089675283 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7ff85c120780 con 0x7ff8700ad330 2024-03-22T04:13:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.485+0000 7ff86f400640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff850067b50 0x7ff85006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.485+0000 7ff86f400640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff850067b50 0x7ff85006a010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff860000f70 tx=0x7ff860009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.489+0000 7ff867400640 1 -- 172.21.15.86:0/3089675283 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff85c016090 con 0x7ff8700ad330 2024-03-22T04:13:22.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.681+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8701410f0 con 0x7ff8700ad330 2024-03-22T04:13:22.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.681+0000 7ff867400640 1 -- 172.21.15.86:0/3089675283 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5189 (secure 0 0 0) 0x7ff85c0f0860 con 0x7ff8700ad330 2024-03-22T04:13:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:22.685 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:12:33.981773+0000","modified":"2024-03-22T04:12:37.002106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75245":{"gid":75245,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6834/2728972599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2728972599},{"type":"v1","addr":"172.21.15.196:6834","nonce":2728972599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75245,"qdb_cluster":[75245]},"id":3}]} 2024-03-22T04:13:22.686 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T04:13:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff850067b50 msgr2=0x7ff85006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff850067b50 0x7ff85006a010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff860000f70 tx=0x7ff860009040 comp rx=0 tx=0).stop 2024-03-22T04:13:22.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8700ad330 msgr2=0x7ff8700995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8700ad330 0x7ff8700995f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff85c019080 tx=0x7ff85c01acb0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 shutdown_connections 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff850067b50 0x7ff85006a010 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8700ad710 0x7ff87015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8700ad330 0x7ff8700995f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 --2- 172.21.15.86:0/3089675283 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff87006ef40 0x7ff8701564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 >> 172.21.15.86:0/3089675283 conn(0x7ff8700b1df0 msgr2=0x7ff8700b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 shutdown_connections 2024-03-22T04:13:22.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.685+0000 7ff876200640 1 -- 172.21.15.86:0/3089675283 wait complete. 2024-03-22T04:13:22.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.837+0000 7efc2ec00640 1 Processor -- start 2024-03-22T04:13:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.837+0000 7efc2ec00640 1 -- start start 2024-03-22T04:13:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815ac10 0x7efc280b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2ec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc2815bc80 0x7efc2815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2ec00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc2815c550 con 0x7efc2806df90 2024-03-22T04:13:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2ec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efc2815cb60 con 0x7efc2815bc80 2024-03-22T04:13:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efc280b4750 con 0x7efc2815ac10 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc2815bc80 0x7efc2815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc2815bc80 0x7efc2815cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57510/0 (socket says 172.21.15.86:57510) 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc1f400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815ac10 0x7efc280b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2cc00640 1 -- 172.21.15.86:0/2441620465 learned_addr learned my addr 172.21.15.86:0/2441620465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc27e00640 1 --1- 172.21.15.86:0/2441620465 >> v1:172.21.15.196:6789/0 conn(0x7efc2806df90 0x7efc2806e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:59880/0 (socket says 172.21.15.86:59880) 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2cc00640 1 -- 172.21.15.86:0/2441620465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815ac10 msgr2=0x7efc280b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2cc00640 1 --2- 172.21.15.86:0/2441620465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815ac10 0x7efc280b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2cc00640 1 -- 172.21.15.86:0/2441620465 >> v1:172.21.15.196:6789/0 conn(0x7efc2806df90 legacy=0x7efc2806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc2cc00640 1 -- 172.21.15.86:0/2441620465 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc280b4470 con 0x7efc2815bc80 2024-03-22T04:13:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.841+0000 7efc27400640 1 -- 172.21.15.86:0/2441620465 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2775821028 0 0) 0x7efc2815c550 con 0x7efc2806df90 2024-03-22T04:13:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2cc00640 1 --2- 172.21.15.86:0/2441620465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815bc80 0x7efc2815cff0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efc20006fd0 tx=0x7efc2001e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c041efab513757 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc27400640 1 -- 172.21.15.86:0/2441620465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc200ad070 con 0x7efc2815bc80 2024-03-22T04:13:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc27400640 1 -- 172.21.15.86:0/2441620465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc20004230 con 0x7efc2815bc80 2024-03-22T04:13:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc27400640 1 -- 172.21.15.86:0/2441620465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc200a77b0 con 0x7efc2815bc80 2024-03-22T04:13:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 -- 172.21.15.86:0/2441620465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815bc80 msgr2=0x7efc2815cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 --2- 172.21.15.86:0/2441620465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815bc80 0x7efc2815cff0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efc20006fd0 tx=0x7efc2001e9c0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 -- 172.21.15.86:0/2441620465 shutdown_connections 2024-03-22T04:13:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 --2- 172.21.15.86:0/2441620465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815ac10 0x7efc280b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 --2- 172.21.15.86:0/2441620465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815bc80 0x7efc2815cff0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 -- 172.21.15.86:0/2441620465 >> 172.21.15.86:0/2441620465 conn(0x7efc280b1df0 msgr2=0x7efc280b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 -- 172.21.15.86:0/2441620465 shutdown_connections 2024-03-22T04:13:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 -- 172.21.15.86:0/2441620465 wait complete. 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 Processor -- start 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.845+0000 7efc2ec00640 1 -- start start 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2806df90 0x7efc281561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815ac10 0x7efc280cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815bc80 0x7efc280ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc2815d900 con 0x7efc2815ac10 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc2815d780 con 0x7efc2806df90 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efc2815da80 con 0x7efc2815bc80 2024-03-22T04:13:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815ac10 0x7efc280cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815ac10 0x7efc280cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57514/0 (socket says 172.21.15.86:57514) 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc27e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2806df90 0x7efc281561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 -- 172.21.15.86:0/1241800952 learned_addr learned my addr 172.21.15.86:0/1241800952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2cc00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815bc80 0x7efc280ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 -- 172.21.15.86:0/1241800952 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815bc80 msgr2=0x7efc280ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815bc80 0x7efc280ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 -- 172.21.15.86:0/1241800952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2806df90 msgr2=0x7efc281561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2806df90 0x7efc281561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 -- 172.21.15.86:0/1241800952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc281578f0 con 0x7efc2815ac10 2024-03-22T04:13:22.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc1f400640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815ac10 0x7efc280cf0d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efc1000e930 tx=0x7efc1000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc24c00640 1 -- 172.21.15.86:0/1241800952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10017070 con 0x7efc2815ac10 2024-03-22T04:13:22.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc24c00640 1 -- 172.21.15.86:0/1241800952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1000c940 con 0x7efc2815ac10 2024-03-22T04:13:22.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc28157bd0 con 0x7efc2815ac10 2024-03-22T04:13:22.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc24c00640 1 -- 172.21.15.86:0/1241800952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1000ccf0 con 0x7efc2815ac10 2024-03-22T04:13:22.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc28158190 con 0x7efc2815ac10 2024-03-22T04:13:22.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc28068650 con 0x7efc2815ac10 2024-03-22T04:13:22.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.849+0000 7efc24c00640 1 -- 172.21.15.86:0/1241800952 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efc1001a070 con 0x7efc2815ac10 2024-03-22T04:13:22.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.853+0000 7efc24c00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc04067b20 0x7efc04069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:22.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.853+0000 7efc24c00640 1 -- 172.21.15.86:0/1241800952 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7efc10086350 con 0x7efc2815ac10 2024-03-22T04:13:22.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.853+0000 7efc27e00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc04067b20 0x7efc04069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:22.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.853+0000 7efc27e00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc04067b20 0x7efc04069fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc18001950 tx=0x7efc180018e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:22.857+0000 7efc24c00640 1 -- 172.21.15.86:0/1241800952 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efc10056620 con 0x7efc2815ac10 2024-03-22T04:13:23.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.013+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc2806ed60 con 0x7efc2815ac10 2024-03-22T04:13:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.013+0000 7efc24c00640 1 -- 172.21.15.86:0/1241800952 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16931 (secure 0 0 0) 0x7efc1005a6c0 con 0x7efc2815ac10 2024-03-22T04:13:23.018 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:23.018 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":56,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:15.973172+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:12:32.314387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-22T04:12:32.945472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2c","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc04067b20 msgr2=0x7efc04069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc04067b20 0x7efc04069fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc18001950 tx=0x7efc180018e0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815ac10 msgr2=0x7efc280cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815ac10 0x7efc280cf0d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efc1000e930 tx=0x7efc1000ed30 comp rx=0 tx=0).stop 2024-03-22T04:13:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 shutdown_connections 2024-03-22T04:13:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efc04067b20 0x7efc04069fe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2815bc80 0x7efc280ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2815ac10 0x7efc280cf0d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 --2- 172.21.15.86:0/1241800952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2806df90 0x7efc281561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 >> 172.21.15.86:0/1241800952 conn(0x7efc280b1df0 msgr2=0x7efc2815b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 shutdown_connections 2024-03-22T04:13:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.017+0000 7efc2ec00640 1 -- 172.21.15.86:0/1241800952 wait complete. 2024-03-22T04:13:23.043 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:13:23.043 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:23.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa55d200640 1 Processor -- start 2024-03-22T04:13:23.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa55d200640 1 -- start start 2024-03-22T04:13:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa55d200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa5580d0de0 0x7fa5580ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa55d200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d19a0 0x7fa5580cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa55d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5580d11c0 con 0x7fa5580d1dc0 2024-03-22T04:13:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa55d200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa5580cece0 con 0x7fa5580d0de0 2024-03-22T04:13:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa55d200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5580cefc0 con 0x7fa5580d19a0 2024-03-22T04:13:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa556800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa5580d1dc0 0x7fa55815fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47670/0 (socket says 172.21.15.86:47670) 2024-03-22T04:13:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa556800640 1 -- 172.21.15.86:0/2958474171 learned_addr learned my addr 172.21.15.86:0/2958474171 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa555e00640 1 --2- 172.21.15.86:0/2958474171 >> v2:172.21.15.196:3300/0 conn(0x7fa5580d0de0 0x7fa5580ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa54d400640 1 --2- 172.21.15.86:0/2958474171 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d19a0 0x7fa5580cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa555400640 1 -- 172.21.15.86:0/2958474171 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 431868544 0 0) 0x7fa5580d11c0 con 0x7fa5580d1dc0 2024-03-22T04:13:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa555e00640 1 -- 172.21.15.86:0/2958474171 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d19a0 msgr2=0x7fa5580cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa555e00640 1 --2- 172.21.15.86:0/2958474171 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d19a0 0x7fa5580cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa555e00640 1 -- 172.21.15.86:0/2958474171 >> v1:172.21.15.86:6789/0 conn(0x7fa5580d1dc0 legacy=0x7fa55815fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.177+0000 7fa555e00640 1 -- 172.21.15.86:0/2958474171 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5580cf2a0 con 0x7fa5580d0de0 2024-03-22T04:13:23.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa555e00640 1 --2- 172.21.15.86:0/2958474171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d0de0 0x7fa5580ce6b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa5480097c0 tx=0x7fa54801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a78708c4869d183 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa555400640 1 -- 172.21.15.86:0/2958474171 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5480b2070 con 0x7fa5580d0de0 2024-03-22T04:13:23.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa555400640 1 -- 172.21.15.86:0/2958474171 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5480ad640 con 0x7fa5580d0de0 2024-03-22T04:13:23.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa555400640 1 -- 172.21.15.86:0/2958474171 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5480ada10 con 0x7fa5580d0de0 2024-03-22T04:13:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa55d200640 1 -- 172.21.15.86:0/2958474171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d0de0 msgr2=0x7fa5580ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa55d200640 1 --2- 172.21.15.86:0/2958474171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d0de0 0x7fa5580ce6b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa5480097c0 tx=0x7fa54801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa55d200640 1 -- 172.21.15.86:0/2958474171 shutdown_connections 2024-03-22T04:13:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa55d200640 1 --2- 172.21.15.86:0/2958474171 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d19a0 0x7fa5580cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa55d200640 1 --2- 172.21.15.86:0/2958474171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d0de0 0x7fa5580ce6b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa55d200640 1 -- 172.21.15.86:0/2958474171 >> 172.21.15.86:0/2958474171 conn(0x7fa55806f910 msgr2=0x7fa55806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa55d200640 1 -- 172.21.15.86:0/2958474171 shutdown_connections 2024-03-22T04:13:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.181+0000 7fa55d200640 1 -- 172.21.15.86:0/2958474171 wait complete. 2024-03-22T04:13:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 Processor -- start 2024-03-22T04:13:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 -- start start 2024-03-22T04:13:23.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d0de0 0x7fa558153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d19a0 0x7fa55815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580d1dc0 0x7fa55815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa558160760 con 0x7fa5580d1dc0 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581605e0 con 0x7fa5580d19a0 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa558071e30 con 0x7fa5580d0de0 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580d1dc0 0x7fa55815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580d1dc0 0x7fa55815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57524/0 (socket says 172.21.15.86:57524) 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 -- 172.21.15.86:0/3592585931 learned_addr learned my addr 172.21.15.86:0/3592585931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa555e00640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d0de0 0x7fa558153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa54d400640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d19a0 0x7fa55815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 -- 172.21.15.86:0/3592585931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d0de0 msgr2=0x7fa558153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d0de0 0x7fa558153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 -- 172.21.15.86:0/3592585931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d19a0 msgr2=0x7fa55815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d19a0 0x7fa55815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 -- 172.21.15.86:0/3592585931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5580b6f30 con 0x7fa5580d1dc0 2024-03-22T04:13:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa556800640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580d1dc0 0x7fa55815a9b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c7b0 tx=0x7fa55000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa54ea00640 1 -- 172.21.15.86:0/3592585931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550018070 con 0x7fa5580d1dc0 2024-03-22T04:13:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa54ea00640 1 -- 172.21.15.86:0/3592585931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55000a640 con 0x7fa5580d1dc0 2024-03-22T04:13:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5580b7210 con 0x7fa5580d1dc0 2024-03-22T04:13:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa54ea00640 1 -- 172.21.15.86:0/3592585931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55000aa70 con 0x7fa5580d1dc0 2024-03-22T04:13:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5580b77d0 con 0x7fa5580d1dc0 2024-03-22T04:13:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.185+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa558068650 con 0x7fa5580d1dc0 2024-03-22T04:13:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.189+0000 7fa54ea00640 1 -- 172.21.15.86:0/3592585931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa55001b070 con 0x7fa5580d1dc0 2024-03-22T04:13:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.189+0000 7fa54ea00640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa534067b20 0x7fa534069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.189+0000 7fa54ea00640 1 -- 172.21.15.86:0/3592585931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fa550085600 con 0x7fa5580d1dc0 2024-03-22T04:13:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.189+0000 7fa555e00640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa534067b20 0x7fa534069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.189+0000 7fa555e00640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa534067b20 0x7fa534069fe0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5480af000 tx=0x7fa548008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.193+0000 7fa54ea00640 1 -- 172.21.15.86:0/3592585931 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa550055800 con 0x7fa5580d1dc0 2024-03-22T04:13:23.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.381+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa55806ffd0 con 0x7fa5580d1dc0 2024-03-22T04:13:23.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.381+0000 7fa54ea00640 1 -- 172.21.15.86:0/3592585931 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5189 (secure 0 0 0) 0x7fa5500598a0 con 0x7fa5580d1dc0 2024-03-22T04:13:23.385 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:23.386 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:12:33.981773+0000","modified":"2024-03-22T04:12:37.002106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75245":{"gid":75245,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6834/2728972599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2728972599},{"type":"v1","addr":"172.21.15.196:6834","nonce":2728972599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75245,"qdb_cluster":[75245]},"id":3}]} 2024-03-22T04:13:23.388 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T04:13:23.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.381+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa534067b20 msgr2=0x7fa534069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.381+0000 7fa55d200640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa534067b20 0x7fa534069fe0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5480af000 tx=0x7fa548008040 comp rx=0 tx=0).stop 2024-03-22T04:13:23.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.381+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580d1dc0 msgr2=0x7fa55815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.381+0000 7fa55d200640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580d1dc0 0x7fa55815a9b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c7b0 tx=0x7fa55000cc70 comp rx=0 tx=0).stop 2024-03-22T04:13:23.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.385+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 shutdown_connections 2024-03-22T04:13:23.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.385+0000 7fa55d200640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa534067b20 0x7fa534069fe0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.385+0000 7fa55d200640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5580d19a0 0x7fa55815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.385+0000 7fa55d200640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580d1dc0 0x7fa55815a9b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.385+0000 7fa55d200640 1 --2- 172.21.15.86:0/3592585931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5580d0de0 0x7fa558153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.385+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 >> 172.21.15.86:0/3592585931 conn(0x7fa55806f910 msgr2=0x7fa55815daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:23.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.385+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 shutdown_connections 2024-03-22T04:13:23.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.385+0000 7fa55d200640 1 -- 172.21.15.86:0/3592585931 wait complete. 2024-03-22T04:13:23.406 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.537+0000 7fa6d6600640 1 Processor -- start 2024-03-22T04:13:23.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.537+0000 7fa6d6600640 1 -- start start 2024-03-22T04:13:23.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6d6600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015e000 0x7fa6d0165960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6d6600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d015c550 con 0x7fa6d015ac10 2024-03-22T04:13:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6d6600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d00b4750 con 0x7fa6d006df90 2024-03-22T04:13:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6d6600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d015cb60 con 0x7fa6d015e000 2024-03-22T04:13:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cf400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa6d006df90 0x7fa6d006e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:59896/0 (socket says 172.21.15.86:59896) 2024-03-22T04:13:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015e000 0x7fa6d0165960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cf400640 1 -- 172.21.15.86:0/2200483368 learned_addr learned my addr 172.21.15.86:0/2200483368 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6c6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa6d015ac10 0x7fa6d00b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47684/0 (socket says 172.21.15.86:47684) 2024-03-22T04:13:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cea00640 1 -- 172.21.15.86:0/2200483368 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810301659 0 0) 0x7fa6d00b4750 con 0x7fa6d006df90 2024-03-22T04:13:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cea00640 1 -- 172.21.15.86:0/2200483368 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6b4003450 con 0x7fa6d006df90 2024-03-22T04:13:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cfe00640 1 -- 172.21.15.86:0/2200483368 >> v1:172.21.15.196:6789/0 conn(0x7fa6d006df90 legacy=0x7fa6d006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cea00640 1 -- 172.21.15.86:0/2200483368 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832590035 0 0) 0x7fa6d015c550 con 0x7fa6d015ac10 2024-03-22T04:13:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cfe00640 1 -- 172.21.15.86:0/2200483368 >> v1:172.21.15.86:6789/0 conn(0x7fa6d015ac10 legacy=0x7fa6d00b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cfe00640 1 -- 172.21.15.86:0/2200483368 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d00b4470 con 0x7fa6d015e000 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.541+0000 7fa6cfe00640 1 --2- 172.21.15.86:0/2200483368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015e000 0x7fa6d0165960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80097f0 tx=0x7fa6c801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a2db6bf95f7c3d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6cea00640 1 -- 172.21.15.86:0/2200483368 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80ab650 con 0x7fa6d015e000 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6cea00640 1 -- 172.21.15.86:0/2200483368 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c80b0070 con 0x7fa6d015e000 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6cea00640 1 -- 172.21.15.86:0/2200483368 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80aba10 con 0x7fa6d015e000 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- 172.21.15.86:0/2200483368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015e000 msgr2=0x7fa6d0165960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 --2- 172.21.15.86:0/2200483368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015e000 0x7fa6d0165960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80097f0 tx=0x7fa6c801c4b0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- 172.21.15.86:0/2200483368 shutdown_connections 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 --2- 172.21.15.86:0/2200483368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015e000 0x7fa6d0165960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- 172.21.15.86:0/2200483368 >> 172.21.15.86:0/2200483368 conn(0x7fa6d00b1df0 msgr2=0x7fa6d00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:23.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- 172.21.15.86:0/2200483368 shutdown_connections 2024-03-22T04:13:23.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- 172.21.15.86:0/2200483368 wait complete. 2024-03-22T04:13:23.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 Processor -- start 2024-03-22T04:13:23.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- start start 2024-03-22T04:13:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d006df90 0x7fa6d0137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015ac10 0x7fa6d013ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d015e000 0x7fa6d014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0168240 con 0x7fa6d015e000 2024-03-22T04:13:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01680c0 con 0x7fa6d006df90 2024-03-22T04:13:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6d6600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d01683c0 con 0x7fa6d015ac10 2024-03-22T04:13:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d015e000 0x7fa6d014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6cf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d006df90 0x7fa6d0137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6c6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015ac10 0x7fa6d013ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d015e000 0x7fa6d014ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57530/0 (socket says 172.21.15.86:57530) 2024-03-22T04:13:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.545+0000 7fa6cfe00640 1 -- 172.21.15.86:0/3760761895 learned_addr learned my addr 172.21.15.86:0/3760761895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6cf400640 1 -- 172.21.15.86:0/3760761895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015ac10 msgr2=0x7fa6d013ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6cf400640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015ac10 0x7fa6d013ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6cf400640 1 -- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d015e000 msgr2=0x7fa6d014ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6cf400640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d015e000 0x7fa6d014ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6cf400640 1 -- 172.21.15.86:0/3760761895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d01380f0 con 0x7fa6d006df90 2024-03-22T04:13:23.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6c6a00640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015ac10 0x7fa6d013ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:23.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6cfe00640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d015e000 0x7fa6d014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:23.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6cf400640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d006df90 0x7fa6d0137a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000cae0 tx=0x7fa6c000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6c7e00640 1 -- 172.21.15.86:0/3760761895 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0017070 con 0x7fa6d006df90 2024-03-22T04:13:23.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6c7e00640 1 -- 172.21.15.86:0/3760761895 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c000ac40 con 0x7fa6d006df90 2024-03-22T04:13:23.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d01383d0 con 0x7fa6d006df90 2024-03-22T04:13:23.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6c7e00640 1 -- 172.21.15.86:0/3760761895 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0012400 con 0x7fa6d006df90 2024-03-22T04:13:23.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d0138990 con 0x7fa6d006df90 2024-03-22T04:13:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d0068650 con 0x7fa6d006df90 2024-03-22T04:13:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6c7e00640 1 -- 172.21.15.86:0/3760761895 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa6c001a070 con 0x7fa6d006df90 2024-03-22T04:13:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6c7e00640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa6ac067b20 0x7fa6ac069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6c7e00640 1 -- 172.21.15.86:0/3760761895 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fa6c0085da0 con 0x7fa6d006df90 2024-03-22T04:13:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.549+0000 7fa6c6a00640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa6ac067b20 0x7fa6ac069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.553+0000 7fa6c6a00640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa6ac067b20 0x7fa6ac069fe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00cc750 tx=0x7fa6b8007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.557+0000 7fa6c7e00640 1 -- 172.21.15.86:0/3760761895 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa6c0055fa0 con 0x7fa6d006df90 2024-03-22T04:13:23.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6d0068d60 con 0x7fa6d006df90 2024-03-22T04:13:23.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6c7e00640 1 -- 172.21.15.86:0/3760761895 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5189 (secure 0 0 0) 0x7fa6c005a040 con 0x7fa6d006df90 2024-03-22T04:13:23.747 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:23.747 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:12:33.981773+0000","modified":"2024-03-22T04:12:37.002106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75245":{"gid":75245,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6834/2728972599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2728972599},{"type":"v1","addr":"172.21.15.196:6834","nonce":2728972599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75245,"qdb_cluster":[75245]},"id":3}]} 2024-03-22T04:13:23.748 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-03-22T04:13:23.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa6ac067b20 msgr2=0x7fa6ac069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa6ac067b20 0x7fa6ac069fe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00cc750 tx=0x7fa6b8007d60 comp rx=0 tx=0).stop 2024-03-22T04:13:23.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d006df90 msgr2=0x7fa6d0137a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d006df90 0x7fa6d0137a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000cae0 tx=0x7fa6c000a060 comp rx=0 tx=0).stop 2024-03-22T04:13:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 shutdown_connections 2024-03-22T04:13:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa6ac067b20 0x7fa6ac069fe0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6d015e000 0x7fa6d014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d015ac10 0x7fa6d013ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 --2- 172.21.15.86:0/3760761895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d006df90 0x7fa6d0137a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 >> 172.21.15.86:0/3760761895 conn(0x7fa6d00b1df0 msgr2=0x7fa6d0073300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 shutdown_connections 2024-03-22T04:13:23.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.745+0000 7fa6d6600640 1 -- 172.21.15.86:0/3760761895 wait complete. 2024-03-22T04:13:23.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:23.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4c800640 1 Processor -- start 2024-03-22T04:13:23.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4c800640 1 -- start start 2024-03-22T04:13:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4c800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd440ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4c800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efd440d19a0 0x7efd440cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4c800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd440d1dc0 0x7efd4415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4c800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efd440d11c0 con 0x7efd440d19a0 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4c800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efd440cece0 con 0x7efd440d1dc0 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4c800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efd440cefc0 con 0x7efd440d0de0 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efd440d19a0 0x7efd440cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd440ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd4a800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd440d1dc0 0x7efd4415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd440ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:45922/0 (socket says 172.21.15.86:45922) 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49e00640 1 -- 172.21.15.86:0/4077863895 learned_addr learned my addr 172.21.15.86:0/4077863895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49e00640 1 -- 172.21.15.86:0/4077863895 >> v2:172.21.15.196:3300/0 conn(0x7efd440d1dc0 msgr2=0x7efd4415fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49e00640 1 --2- 172.21.15.86:0/4077863895 >> v2:172.21.15.196:3300/0 conn(0x7efd440d1dc0 0x7efd4415fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49e00640 1 -- 172.21.15.86:0/4077863895 >> v2:172.21.15.86:3300/0 conn(0x7efd440d19a0 msgr2=0x7efd440cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49e00640 1 --2- 172.21.15.86:0/4077863895 >> v2:172.21.15.86:3300/0 conn(0x7efd440d19a0 0x7efd440cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.901+0000 7efd49e00640 1 -- 172.21.15.86:0/4077863895 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd440cf2a0 con 0x7efd440d0de0 2024-03-22T04:13:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd49e00640 1 --2- 172.21.15.86:0/4077863895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd440ce6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd440b3e00 tx=0x7efd3801c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc6b27c1cc81f65 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd48a00640 1 -- 172.21.15.86:0/4077863895 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd380b0070 con 0x7efd440d0de0 2024-03-22T04:13:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd48a00640 1 -- 172.21.15.86:0/4077863895 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd380ab600 con 0x7efd440d0de0 2024-03-22T04:13:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd48a00640 1 -- 172.21.15.86:0/4077863895 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd380ab9d0 con 0x7efd440d0de0 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 -- 172.21.15.86:0/4077863895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 msgr2=0x7efd440ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 --2- 172.21.15.86:0/4077863895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd440ce6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd440b3e00 tx=0x7efd3801c520 comp rx=0 tx=0).stop 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 -- 172.21.15.86:0/4077863895 shutdown_connections 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 --2- 172.21.15.86:0/4077863895 >> v2:172.21.15.196:3300/0 conn(0x7efd440d1dc0 0x7efd4415fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 --2- 172.21.15.86:0/4077863895 >> v2:172.21.15.86:3300/0 conn(0x7efd440d19a0 0x7efd440cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 --2- 172.21.15.86:0/4077863895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd440ce6b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 -- 172.21.15.86:0/4077863895 >> 172.21.15.86:0/4077863895 conn(0x7efd4406f910 msgr2=0x7efd4406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 -- 172.21.15.86:0/4077863895 shutdown_connections 2024-03-22T04:13:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.905+0000 7efd4c800640 1 -- 172.21.15.86:0/4077863895 wait complete. 2024-03-22T04:13:23.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 Processor -- start 2024-03-22T04:13:23.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 -- start start 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd44155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd440d19a0 0x7efd440cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd440d1dc0 0x7efd440c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd44071fb0 con 0x7efd440d1dc0 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd44071e30 con 0x7efd440d19a0 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efd4415c8f0 con 0x7efd440d0de0 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd440d1dc0 0x7efd440c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd44155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd440d19a0 0x7efd440cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd44155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:45934/0 (socket says 172.21.15.86:45934) 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 -- 172.21.15.86:0/2585839973 learned_addr learned my addr 172.21.15.86:0/2585839973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 -- 172.21.15.86:0/2585839973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd440d19a0 msgr2=0x7efd440cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd440d19a0 0x7efd440cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 -- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd440d1dc0 msgr2=0x7efd440c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd440d1dc0 0x7efd440c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 -- 172.21.15.86:0/2585839973 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd440cc480 con 0x7efd440d0de0 2024-03-22T04:13:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd49e00640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd44155030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd3801abd0 tx=0x7efd380192c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd3e000640 1 -- 172.21.15.86:0/2585839973 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd380b0040 con 0x7efd440d0de0 2024-03-22T04:13:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd3e000640 1 -- 172.21.15.86:0/2585839973 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd380b4620 con 0x7efd440d0de0 2024-03-22T04:13:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd440caea0 con 0x7efd440d0de0 2024-03-22T04:13:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd3e000640 1 -- 172.21.15.86:0/2585839973 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd380b4a10 con 0x7efd440d0de0 2024-03-22T04:13:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.909+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd440cb460 con 0x7efd440d0de0 2024-03-22T04:13:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.913+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd44068650 con 0x7efd440d0de0 2024-03-22T04:13:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.913+0000 7efd3e000640 1 -- 172.21.15.86:0/2585839973 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efd380ab4c0 con 0x7efd440d0de0 2024-03-22T04:13:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.913+0000 7efd3e000640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efd28067b20 0x7efd28069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.913+0000 7efd3e000640 1 -- 172.21.15.86:0/2585839973 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7efd3811fe80 con 0x7efd440d0de0 2024-03-22T04:13:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.913+0000 7efd49400640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efd28067b20 0x7efd28069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.913+0000 7efd49400640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efd28067b20 0x7efd28069fe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd441fab00 tx=0x7efd34009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:23.917+0000 7efd3e000640 1 -- 172.21.15.86:0/2585839973 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efd38016110 con 0x7efd440d0de0 2024-03-22T04:13:24.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.077+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd440ce290 con 0x7efd440d0de0 2024-03-22T04:13:24.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd3e000640 1 -- 172.21.15.86:0/2585839973 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16931 (secure 0 0 0) 0x7efd380f0100 con 0x7efd440d0de0 2024-03-22T04:13:24.083 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:24.083 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":56,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:15.973172+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:12:32.314387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-22T04:12:32.945472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2c","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efd28067b20 msgr2=0x7efd28069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efd28067b20 0x7efd28069fe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd441fab00 tx=0x7efd34009190 comp rx=0 tx=0).stop 2024-03-22T04:13:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 msgr2=0x7efd44155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd44155030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd3801abd0 tx=0x7efd380192c0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 shutdown_connections 2024-03-22T04:13:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7efd28067b20 0x7efd28069fe0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd440d19a0 0x7efd440cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd440d1dc0 0x7efd440c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 --2- 172.21.15.86:0/2585839973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd440d0de0 0x7efd44155030 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 >> 172.21.15.86:0/2585839973 conn(0x7efd4406f910 msgr2=0x7efd4415a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 shutdown_connections 2024-03-22T04:13:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.081+0000 7efd4c800640 1 -- 172.21.15.86:0/2585839973 wait complete. 2024-03-22T04:13:24.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:13:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc7a0200640 1 Processor -- start 2024-03-22T04:13:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc7a0200640 1 -- start start 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc7a0200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad330 0x7fc79815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc7a0200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79806ef40 0x7fc79806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc7a0200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79815c7c0 con 0x7fc7980ad710 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc7a0200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc79806dc40 con 0x7fc79806ef40 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc7a0200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc79806dea0 con 0x7fc7980ad330 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc79ce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79806ef40 0x7fc79806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc79d800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad330 0x7fc79815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc79ce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79806ef40 0x7fc79806d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:46830/0 (socket says 172.21.15.86:46830) 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc79d800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad330 0x7fc79815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:45950/0 (socket says 172.21.15.86:45950) 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc79ce00640 1 -- 172.21.15.86:0/2121079949 learned_addr learned my addr 172.21.15.86:0/2121079949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:24.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.237+0000 7fc79e200640 1 --1- 172.21.15.86:0/2121079949 >> v1:172.21.15.196:6789/0 conn(0x7fc7980ad710 0x7fc79813fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:59906/0 (socket says 172.21.15.86:59906) 2024-03-22T04:13:24.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc797e00640 1 -- 172.21.15.86:0/2121079949 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667815499 0 0) 0x7fc79815c7c0 con 0x7fc7980ad710 2024-03-22T04:13:24.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc797e00640 1 -- 172.21.15.86:0/2121079949 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc780003530 con 0x7fc7980ad710 2024-03-22T04:13:24.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc79d800640 1 -- 172.21.15.86:0/2121079949 >> v2:172.21.15.196:3300/0 conn(0x7fc79806ef40 msgr2=0x7fc79806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:24.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc79d800640 1 --2- 172.21.15.86:0/2121079949 >> v2:172.21.15.196:3300/0 conn(0x7fc79806ef40 0x7fc79806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc79d800640 1 -- 172.21.15.86:0/2121079949 >> v1:172.21.15.196:6789/0 conn(0x7fc7980ad710 legacy=0x7fc79813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc79d800640 1 -- 172.21.15.86:0/2121079949 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79806e180 con 0x7fc7980ad330 2024-03-22T04:13:24.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc79ce00640 1 --2- 172.21.15.86:0/2121079949 >> v2:172.21.15.196:3300/0 conn(0x7fc79806ef40 0x7fc79806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:24.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc79d800640 1 --2- 172.21.15.86:0/2121079949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad330 0x7fc79815d4a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0097c0 tx=0x7fc78c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af6353848675b2e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:24.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc797e00640 1 -- 172.21.15.86:0/2121079949 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0b2070 con 0x7fc7980ad330 2024-03-22T04:13:24.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc797e00640 1 -- 172.21.15.86:0/2121079949 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c0ad680 con 0x7fc7980ad330 2024-03-22T04:13:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.241+0000 7fc797e00640 1 -- 172.21.15.86:0/2121079949 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0ada50 con 0x7fc7980ad330 2024-03-22T04:13:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- 172.21.15.86:0/2121079949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad330 msgr2=0x7fc79815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2121079949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad330 0x7fc79815d4a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0097c0 tx=0x7fc78c01c470 comp rx=0 tx=0).stop 2024-03-22T04:13:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- 172.21.15.86:0/2121079949 shutdown_connections 2024-03-22T04:13:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2121079949 >> v2:172.21.15.196:3300/0 conn(0x7fc79806ef40 0x7fc79806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2121079949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad330 0x7fc79815d4a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- 172.21.15.86:0/2121079949 >> 172.21.15.86:0/2121079949 conn(0x7fc7980b1df0 msgr2=0x7fc7980b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- 172.21.15.86:0/2121079949 shutdown_connections 2024-03-22T04:13:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- 172.21.15.86:0/2121079949 wait complete. 2024-03-22T04:13:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 Processor -- start 2024-03-22T04:13:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- start start 2024-03-22T04:13:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79806ef40 0x7fc798098a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980ad330 0x7fc798142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad710 0x7fc7980992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7981690e0 con 0x7fc7980ad330 2024-03-22T04:13:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc798168f60 con 0x7fc79806ef40 2024-03-22T04:13:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc7a0200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc798169260 con 0x7fc7980ad710 2024-03-22T04:13:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980ad330 0x7fc798142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980ad330 0x7fc798142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57560/0 (socket says 172.21.15.86:57560) 2024-03-22T04:13:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79ce00640 1 -- 172.21.15.86:0/2000359154 learned_addr learned my addr 172.21.15.86:0/2000359154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79d800640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79806ef40 0x7fc798098a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79e200640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad710 0x7fc7980992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79d800640 1 -- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad710 msgr2=0x7fc7980992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79d800640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad710 0x7fc7980992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79d800640 1 -- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980ad330 msgr2=0x7fc798142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79d800640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980ad330 0x7fc798142d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79d800640 1 -- 172.21.15.86:0/2000359154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc798153d80 con 0x7fc79806ef40 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79ce00640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980ad330 0x7fc798142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.245+0000 7fc79d800640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79806ef40 0x7fc798098a30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0ac000 tx=0x7fc78c01acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc795600640 1 -- 172.21.15.86:0/2000359154 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0bc020 con 0x7fc79806ef40 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc795600640 1 -- 172.21.15.86:0/2000359154 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c0adc30 con 0x7fc79806ef40 2024-03-22T04:13:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc795600640 1 -- 172.21.15.86:0/2000359154 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0b2040 con 0x7fc79806ef40 2024-03-22T04:13:24.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc798159960 con 0x7fc79806ef40 2024-03-22T04:13:24.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79815bb70 con 0x7fc79806ef40 2024-03-22T04:13:24.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc798068650 con 0x7fc79806ef40 2024-03-22T04:13:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc795600640 1 -- 172.21.15.86:0/2000359154 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc78c0bf020 con 0x7fc79806ef40 2024-03-22T04:13:24.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc795600640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc778067b70 0x7fc77806a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc795600640 1 -- 172.21.15.86:0/2000359154 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6130+0+0 (secure 0 0 0) 0x7fc78c11fff0 con 0x7fc79806ef40 2024-03-22T04:13:24.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.249+0000 7fc79ce00640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc778067b70 0x7fc77806a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.253+0000 7fc79ce00640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc778067b70 0x7fc77806a030 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc798068060 tx=0x7fc784009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:24.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.253+0000 7fc795600640 1 -- 172.21.15.86:0/2000359154 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc78c0c6030 con 0x7fc79806ef40 2024-03-22T04:13:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.441+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc7980b0380 con 0x7fc79806ef40 2024-03-22T04:13:24.444 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:13:24.441+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:13:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.449+0000 7fc795600640 1 -- 172.21.15.86:0/2000359154 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7fc78c016090 con 0x7fc79806ef40 2024-03-22T04:13:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:13:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc778067b70 msgr2=0x7fc77806a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc778067b70 0x7fc77806a030 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc798068060 tx=0x7fc784009240 comp rx=0 tx=0).stop 2024-03-22T04:13:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79806ef40 msgr2=0x7fc798098a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79806ef40 0x7fc798098a30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0ac000 tx=0x7fc78c01acb0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 shutdown_connections 2024-03-22T04:13:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc778067b70 0x7fc77806a030 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7980ad710 0x7fc7980992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980ad330 0x7fc798142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 --2- 172.21.15.86:0/2000359154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79806ef40 0x7fc798098a30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 >> 172.21.15.86:0/2000359154 conn(0x7fc7980b1df0 msgr2=0x7fc79815deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 shutdown_connections 2024-03-22T04:13:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.453+0000 7fc7a0200640 1 -- 172.21.15.86:0/2000359154 wait complete. 2024-03-22T04:13:24.462 INFO:tasks.ceph.mds.d.smithi196.stderr: -6204> 2024-03-22T04:12:36.589+0000 7f5545400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:13:24.467 INFO:tasks.ceph.mds.d.smithi196.stderr: -3144> 2024-03-22T04:12:47.300+0000 7f5540400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5922=0-4194304@1} caps={5922=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5922 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x558247a92c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-22T04:13:24.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T04:13:24.510 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:13:24.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.609+0000 7f133f800640 1 Processor -- start 2024-03-22T04:13:24.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.609+0000 7f133f800640 1 -- start start 2024-03-22T04:13:24.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.609+0000 7f133f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f133815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.609+0000 7f133f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1338073150 con 0x7f133806e7e0 2024-03-22T04:13:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.609+0000 7f133f800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13380715a0 con 0x7f1338072650 2024-03-22T04:13:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.609+0000 7f133f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1338071350 con 0x7f1338072b00 2024-03-22T04:13:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.609+0000 7f133ce00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f133806e7e0 0x7f133806ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47688/0 (socket says 172.21.15.86:47688) 2024-03-22T04:13:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.609+0000 7f133ce00640 1 -- 172.21.15.86:0/1910014037 learned_addr learned my addr 172.21.15.86:0/1910014037 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f133d800640 1 --2- 172.21.15.86:0/1910014037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f133815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f1337400640 1 -- 172.21.15.86:0/1910014037 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508699299 0 0) 0x7f1338073150 con 0x7f133806e7e0 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f1337400640 1 -- 172.21.15.86:0/1910014037 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1320005740 con 0x7f133806e7e0 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f1337400640 1 -- 172.21.15.86:0/1910014037 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3918961548 0 0) 0x7f13380715a0 con 0x7f1338072650 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f1337400640 1 -- 172.21.15.86:0/1910014037 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1338073150 con 0x7f1338072650 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f1337400640 1 -- 172.21.15.86:0/1910014037 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1242663959 0 0) 0x7f1320005740 con 0x7f133806e7e0 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f133d800640 1 -- 172.21.15.86:0/1910014037 >> v1:172.21.15.196:6789/0 conn(0x7f1338072650 legacy=0x7f133815b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f133d800640 1 -- 172.21.15.86:0/1910014037 >> v1:172.21.15.86:6789/0 conn(0x7f133806e7e0 legacy=0x7f133806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f133d800640 1 -- 172.21.15.86:0/1910014037 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1338071880 con 0x7f1338072b00 2024-03-22T04:13:24.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f133d800640 1 --2- 172.21.15.86:0/1910014037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f133815be70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1330009dc0 tx=0x7f133001c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3429cd8c784508eb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:24.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f1337400640 1 -- 172.21.15.86:0/1910014037 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300ac070 con 0x7f1338072b00 2024-03-22T04:13:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f1337400640 1 -- 172.21.15.86:0/1910014037 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13300a6480 con 0x7f1338072b00 2024-03-22T04:13:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.613+0000 7f1337400640 1 -- 172.21.15.86:0/1910014037 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c001680 con 0x7f1338072b00 2024-03-22T04:13:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- 172.21.15.86:0/1910014037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 msgr2=0x7f133815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 --2- 172.21.15.86:0/1910014037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f133815be70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1330009dc0 tx=0x7f133001c820 comp rx=0 tx=0).stop 2024-03-22T04:13:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- 172.21.15.86:0/1910014037 shutdown_connections 2024-03-22T04:13:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 --2- 172.21.15.86:0/1910014037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f133815be70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- 172.21.15.86:0/1910014037 >> 172.21.15.86:0/1910014037 conn(0x7f13380cc940 msgr2=0x7f13380ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- 172.21.15.86:0/1910014037 shutdown_connections 2024-03-22T04:13:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- 172.21.15.86:0/1910014037 wait complete. 2024-03-22T04:13:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 Processor -- start 2024-03-22T04:13:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- start start 2024-03-22T04:13:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f133806e7e0 0x7f1338142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1338072650 0x7f1338149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f1338149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1338155c20 con 0x7f1338072650 2024-03-22T04:13:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1338155aa0 con 0x7f133806e7e0 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f133806dee0 con 0x7f1338072b00 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1338072650 0x7f1338149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1338072650 0x7f1338149730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57562/0 (socket says 172.21.15.86:57562) 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 -- 172.21.15.86:0/3248632895 learned_addr learned my addr 172.21.15.86:0/3248632895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133ce00640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f133806e7e0 0x7f1338142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f133d800640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f1338149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 -- 172.21.15.86:0/3248632895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 msgr2=0x7f1338149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f1338149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 -- 172.21.15.86:0/3248632895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f133806e7e0 msgr2=0x7f1338142830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f133806e7e0 0x7f1338142830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 -- 172.21.15.86:0/3248632895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1338155da0 con 0x7f1338072650 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.617+0000 7f1337e00640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1338072650 0x7f1338149730 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f132400c490 tx=0x7f132400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f1334c00640 1 -- 172.21.15.86:0/3248632895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13240052d0 con 0x7f1338072650 2024-03-22T04:13:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f1334c00640 1 -- 172.21.15.86:0/3248632895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1324005470 con 0x7f1338072650 2024-03-22T04:13:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f1334c00640 1 -- 172.21.15.86:0/3248632895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324012870 con 0x7f1338072650 2024-03-22T04:13:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133814a280 con 0x7f1338072650 2024-03-22T04:13:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1338144cd0 con 0x7f1338072650 2024-03-22T04:13:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1338074470 con 0x7f1338072650 2024-03-22T04:13:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f1334c00640 1 -- 172.21.15.86:0/3248632895 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1324012a10 con 0x7f1338072650 2024-03-22T04:13:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f1334c00640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1318067b50 0x7f131806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f1334c00640 1 -- 172.21.15.86:0/3248632895 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6216+0+0 (secure 0 0 0) 0x7f1324085cd0 con 0x7f1338072650 2024-03-22T04:13:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f133ce00640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1318067b50 0x7f131806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.621+0000 7f133ce00640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1318067b50 0x7f131806a010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f132c001000 tx=0x7f132c001160 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:24.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.625+0000 7f1334c00640 1 -- 172.21.15.86:0/3248632895 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1324055df0 con 0x7f1338072650 2024-03-22T04:13:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:24.813+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f13380b4890 con 0x7f1338072650 2024-03-22T04:13:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.445+0000 7f1334c00640 1 -- 172.21.15.86:0/3248632895 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f1324059e90 con 0x7f1338072650 2024-03-22T04:13:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1318067b50 msgr2=0x7f131806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1318067b50 0x7f131806a010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f132c001000 tx=0x7f132c001160 comp rx=0 tx=0).stop 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1338072650 msgr2=0x7f1338149730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1338072650 0x7f1338149730 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f132400c490 tx=0x7f132400c950 comp rx=0 tx=0).stop 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 shutdown_connections 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1318067b50 0x7f131806a010 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1338072b00 0x7f1338149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1338072650 0x7f1338149730 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 --2- 172.21.15.86:0/3248632895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f133806e7e0 0x7f1338142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 >> 172.21.15.86:0/3248632895 conn(0x7f13380cc940 msgr2=0x7f13380ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 shutdown_connections 2024-03-22T04:13:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.449+0000 7f133f800640 1 -- 172.21.15.86:0/3248632895 wait complete. 2024-03-22T04:13:25.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T04:13:25.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.609+0000 7f43aa000640 1 Processor -- start 2024-03-22T04:13:25.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.609+0000 7f43aa000640 1 -- start start 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43aa000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f43a4072500 0x7f43a406f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43aa000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a40cef30 con 0x7f43a415e6d0 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43aa000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a40cf210 con 0x7f43a40afef0 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43aa000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f43a406f080 con 0x7f43a4072500 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f43a415e6d0 0x7f43a40ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47694/0 (socket says 172.21.15.86:47694) 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a3400640 1 -- 172.21.15.86:0/2437887716 learned_addr learned my addr 172.21.15.86:0/2437887716 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a3e00640 1 --2- 172.21.15.86:0/2437887716 >> v2:172.21.15.196:3300/0 conn(0x7f43a4072500 0x7f43a406f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571481679 0 0) 0x7f43a40cef30 con 0x7f43a415e6d0 2024-03-22T04:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4388003530 con 0x7f43a415e6d0 2024-03-22T04:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022035967 0 0) 0x7f43a40cf210 con 0x7f43a40afef0 2024-03-22T04:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43a40cef30 con 0x7f43a40afef0 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1672677593 0 0) 0x7f4388003530 con 0x7f43a415e6d0 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 >> v2:172.21.15.196:3300/0 conn(0x7f43a4072500 msgr2=0x7f43a406f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 --2- 172.21.15.86:0/2437887716 >> v2:172.21.15.196:3300/0 conn(0x7f43a4072500 0x7f43a406f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 >> v1:172.21.15.196:6789/0 conn(0x7f43a40afef0 legacy=0x7f43a4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a406ecc0 con 0x7f43a415e6d0 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f4394001a00 con 0x7f43a415e6d0 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 >> v1:172.21.15.86:6789/0 conn(0x7f43a415e6d0 legacy=0x7f43a40ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388003b50 0x7f4388003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388004810 0x7f4388004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43880051d0 0x7f438800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43a406f080 con 0x7f4388004810 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f438800c070 con 0x7f4388003b50 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f438800c270 con 0x7f43880051d0 2024-03-22T04:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f439aa00640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388004810 0x7f4388004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a3400640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388003b50 0x7f4388003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a3e00640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43880051d0 0x7f438800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f439aa00640 1 -- 172.21.15.86:0/2437887716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43880051d0 msgr2=0x7f438800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f439aa00640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43880051d0 0x7f438800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f439aa00640 1 -- 172.21.15.86:0/2437887716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388003b50 msgr2=0x7f4388003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f439aa00640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388003b50 0x7f4388003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f439aa00640 1 -- 172.21.15.86:0/2437887716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4388003530 con 0x7f4388004810 2024-03-22T04:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.613+0000 7f43a3e00640 1 -- 172.21.15.86:0/2437887716 reap_dead start 2024-03-22T04:13:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f439aa00640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388004810 0x7f4388004c90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f438c0018d0 tx=0x7f438c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea91e2e87563eb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c001520 con 0x7f4388004810 2024-03-22T04:13:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438c0052f0 con 0x7f4388004810 2024-03-22T04:13:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43a2a00640 1 -- 172.21.15.86:0/2437887716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c003840 con 0x7f4388004810 2024-03-22T04:13:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 -- 172.21.15.86:0/2437887716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388004810 msgr2=0x7f4388004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388004810 0x7f4388004c90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f438c0018d0 tx=0x7f438c01c520 comp rx=0 tx=0).stop 2024-03-22T04:13:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 -- 172.21.15.86:0/2437887716 shutdown_connections 2024-03-22T04:13:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 --2- 172.21.15.86:0/2437887716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388004810 0x7f4388004c90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 -- 172.21.15.86:0/2437887716 >> 172.21.15.86:0/2437887716 conn(0x7f43a40d07e0 msgr2=0x7f43a40d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 -- 172.21.15.86:0/2437887716 shutdown_connections 2024-03-22T04:13:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 -- 172.21.15.86:0/2437887716 wait complete. 2024-03-22T04:13:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 Processor -- start 2024-03-22T04:13:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.617+0000 7f43aa000640 1 -- start start 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4388004810 0x7f43a4145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43a41f8090 0x7f43a41f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43a41628c0 0x7f43a4162d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43a406c200 con 0x7f43a41628c0 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43a406c080 con 0x7f43a41f8090 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f43a4145c50 con 0x7f4388004810 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43a41628c0 0x7f43a4162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43a41628c0 0x7f43a4162d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57590/0 (socket says 172.21.15.86:57590) 2024-03-22T04:13:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 -- 172.21.15.86:0/529806980 learned_addr learned my addr 172.21.15.86:0/529806980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f439aa00640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43a41f8090 0x7f43a41f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3400640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4388004810 0x7f43a4145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 -- 172.21.15.86:0/529806980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4388004810 msgr2=0x7f43a4145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4388004810 0x7f43a4145710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 -- 172.21.15.86:0/529806980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43a41f8090 msgr2=0x7f43a41f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43a41f8090 0x7f43a41f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 -- 172.21.15.86:0/529806980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a4162130 con 0x7f43a41628c0 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43a3e00640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43a41628c0 0x7f43a4162d70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f43a4071d80 tx=0x7f439c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f439be00640 1 -- 172.21.15.86:0/529806980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c004440 con 0x7f43a41628c0 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f439be00640 1 -- 172.21.15.86:0/529806980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c01b070 con 0x7f43a41628c0 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f439be00640 1 -- 172.21.15.86:0/529806980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c003860 con 0x7f43a41628c0 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a4154470 con 0x7f43a41628c0 2024-03-22T04:13:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a41f8550 con 0x7f43a41628c0 2024-03-22T04:13:25.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a4154640 con 0x7f43a41628c0 2024-03-22T04:13:25.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.621+0000 7f439be00640 1 -- 172.21.15.86:0/529806980 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f439c003a00 con 0x7f43a41628c0 2024-03-22T04:13:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.625+0000 7f439be00640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4380067b50 0x7f438006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.625+0000 7f439be00640 1 -- 172.21.15.86:0/529806980 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6216+0+0 (secure 0 0 0) 0x7f439c085ff0 con 0x7f43a41628c0 2024-03-22T04:13:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.625+0000 7f43a3400640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4380067b50 0x7f438006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.625+0000 7f43a3400640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4380067b50 0x7f438006a010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f43a4151be0 tx=0x7f439400a040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:25.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.629+0000 7f439be00640 1 -- 172.21.15.86:0/529806980 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f439c0567a0 con 0x7f43a41628c0 2024-03-22T04:13:25.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:25.781+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f43a406ce20 con 0x7f43a41628c0 2024-03-22T04:13:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.453+0000 7f439be00640 1 -- 172.21.15.86:0/529806980 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7f439c0560a0 con 0x7f43a41628c0 2024-03-22T04:13:26.458 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T04:13:26.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4380067b50 msgr2=0x7f438006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4380067b50 0x7f438006a010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f43a4151be0 tx=0x7f439400a040 comp rx=0 tx=0).stop 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43a41628c0 msgr2=0x7f43a4162d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43a41628c0 0x7f43a4162d70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f43a4071d80 tx=0x7f439c011920 comp rx=0 tx=0).stop 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 shutdown_connections 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4380067b50 0x7f438006a010 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43a41f8090 0x7f43a41f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43a41628c0 0x7f43a4162d70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 --2- 172.21.15.86:0/529806980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4388004810 0x7f43a4145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 >> 172.21.15.86:0/529806980 conn(0x7f43a40d07e0 msgr2=0x7f43a40b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 shutdown_connections 2024-03-22T04:13:26.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.457+0000 7f43aa000640 1 -- 172.21.15.86:0/529806980 wait complete. 2024-03-22T04:13:26.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T04:13:26.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f035fa00640 1 Processor -- start 2024-03-22T04:13:26.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f035fa00640 1 -- start start 2024-03-22T04:13:26.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f035fa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03580afef0 0x7f035806f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:26.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f035fa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03580cef30 con 0x7f035815e6d0 2024-03-22T04:13:26.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f035fa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03580cf210 con 0x7f0358071df0 2024-03-22T04:13:26.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f035fa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f035806f080 con 0x7f03580afef0 2024-03-22T04:13:26.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f035d000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f035815e6d0 0x7f03580ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53214/0 (socket says 172.21.15.86:53214) 2024-03-22T04:13:26.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f035d000640 1 -- 172.21.15.86:0/293422555 learned_addr learned my addr 172.21.15.86:0/293422555 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:26.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f0357e00640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03580afef0 0x7f035806f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:26.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747136530 0 0) 0x7f03580cef30 con 0x7f035815e6d0 2024-03-22T04:13:26.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.621+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f033c003530 con 0x7f035815e6d0 2024-03-22T04:13:26.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175044473 0 0) 0x7f03580cf210 con 0x7f0358071df0 2024-03-22T04:13:26.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03580cef30 con 0x7f0358071df0 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 347518735 0 0) 0x7f033c003530 con 0x7f035815e6d0 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03580afef0 msgr2=0x7f035806f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03580afef0 0x7f035806f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 >> v1:172.21.15.196:6789/0 conn(0x7f0358071df0 legacy=0x7f035806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035806ecc0 con 0x7f035815e6d0 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f0348001a20 con 0x7f035815e6d0 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 >> v1:172.21.15.86:6789/0 conn(0x7f035815e6d0 legacy=0x7f03580ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c003b80 0x7f033c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03580afef0 0x7f033c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f033c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f035806f080 con 0x7f033c004d50 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f033c00bb60 con 0x7f033c003b80 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f033c00bd60 con 0x7f03580afef0 2024-03-22T04:13:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f035da00640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f033c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f035d000640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c003b80 0x7f033c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f0357e00640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03580afef0 0x7f033c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f035da00640 1 -- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03580afef0 msgr2=0x7f033c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f035da00640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03580afef0 0x7f033c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f035da00640 1 -- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c003b80 msgr2=0x7f033c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f035da00640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c003b80 0x7f033c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f035da00640 1 -- 172.21.15.86:0/293422555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c003530 con 0x7f033c004d50 2024-03-22T04:13:26.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.625+0000 7f035da00640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f033c0051d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f03500018d0 tx=0x7f035001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92dffc8f1338c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:26.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350001520 con 0x7f033c004d50 2024-03-22T04:13:26.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03500052f0 con 0x7f033c004d50 2024-03-22T04:13:26.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f0357400640 1 -- 172.21.15.86:0/293422555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03500a8950 con 0x7f033c004d50 2024-03-22T04:13:26.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- 172.21.15.86:0/293422555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 msgr2=0x7f033c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:26.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 --2- 172.21.15.86:0/293422555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f033c0051d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f03500018d0 tx=0x7f035001ecf0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035da00640 1 -- 172.21.15.86:0/293422555 reap_dead start 2024-03-22T04:13:26.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- 172.21.15.86:0/293422555 shutdown_connections 2024-03-22T04:13:26.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- 172.21.15.86:0/293422555 >> 172.21.15.86:0/293422555 conn(0x7f03580d07e0 msgr2=0x7f03580d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:26.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- 172.21.15.86:0/293422555 shutdown_connections 2024-03-22T04:13:26.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- 172.21.15.86:0/293422555 wait complete. 2024-03-22T04:13:26.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 Processor -- start 2024-03-22T04:13:26.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- start start 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f03581515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0358161160 0x7f03581615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0358153f20 0x7f03581543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f035806fc10 con 0x7f033c004d50 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f035806fa90 con 0x7f0358153f20 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035fa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0358072260 con 0x7f0358161160 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f03581515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f03581515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45062/0 (socket says 172.21.15.86:45062) 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035d000640 1 -- 172.21.15.86:0/975944434 learned_addr learned my addr 172.21.15.86:0/975944434 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035da00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0358153f20 0x7f03581543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:26.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f0357e00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0358161160 0x7f03581615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035d000640 1 -- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0358161160 msgr2=0x7f03581615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035d000640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0358161160 0x7f03581615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035d000640 1 -- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0358153f20 msgr2=0x7f03581543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035d000640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0358153f20 0x7f03581543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f035d000640 1 -- 172.21.15.86:0/975944434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0358151af0 con 0x7f033c004d50 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.629+0000 7f0357e00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0358161160 0x7f03581615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f035d000640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f03581515b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0348007770 tx=0x7f034800a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f0354c00640 1 -- 172.21.15.86:0/975944434 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348017020 con 0x7f033c004d50 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f0354c00640 1 -- 172.21.15.86:0/975944434 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034800acb0 con 0x7f033c004d50 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0358154c50 con 0x7f033c004d50 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f0354c00640 1 -- 172.21.15.86:0/975944434 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034800ae60 con 0x7f033c004d50 2024-03-22T04:13:26.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0358155210 con 0x7f033c004d50 2024-03-22T04:13:26.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0358155800 con 0x7f033c004d50 2024-03-22T04:13:26.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f0354c00640 1 -- 172.21.15.86:0/975944434 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f034801a020 con 0x7f033c004d50 2024-03-22T04:13:26.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f0354c00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0338067b50 0x7f033806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:26.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f0354c00640 1 -- 172.21.15.86:0/975944434 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5778+0+0 (secure 0 0 0) 0x7f0348086900 con 0x7f033c004d50 2024-03-22T04:13:26.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.633+0000 7f0357e00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0338067b50 0x7f033806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:26.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.637+0000 7f0357e00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0338067b50 0x7f033806a010 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f035815ad20 tx=0x7f0344008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:26.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.637+0000 7f0354c00640 1 -- 172.21.15.86:0/975944434 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0348021020 con 0x7f033c004d50 2024-03-22T04:13:26.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:26.793+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f03580d16c0 con 0x7f033c004d50 2024-03-22T04:13:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.465+0000 7f0354c00640 1 -- 172.21.15.86:0/975944434 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7f0348057220 con 0x7f033c004d50 2024-03-22T04:13:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0338067b50 msgr2=0x7f033806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0338067b50 0x7f033806a010 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f035815ad20 tx=0x7f0344008040 comp rx=0 tx=0).stop 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 msgr2=0x7f03581515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f03581515b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0348007770 tx=0x7f034800a270 comp rx=0 tx=0).stop 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 shutdown_connections 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0338067b50 0x7f033806a010 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0358153f20 0x7f03581543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0358161160 0x7f03581615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 --2- 172.21.15.86:0/975944434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f033c004d50 0x7f03581515b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 >> 172.21.15.86:0/975944434 conn(0x7f03580d07e0 msgr2=0x7f03580b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 shutdown_connections 2024-03-22T04:13:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.469+0000 7f035fa00640 1 -- 172.21.15.86:0/975944434 wait complete. 2024-03-22T04:13:27.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-22T04:13:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.629+0000 7f5184a00640 1 Processor -- start 2024-03-22T04:13:27.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.629+0000 7f5184a00640 1 -- start start 2024-03-22T04:13:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f5184a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800b3600 0x7f518015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f5184a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f518015cee0 0x7f51800b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f5184a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518015d710 con 0x7f51800b2360 2024-03-22T04:13:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f5184a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f51800b2740 con 0x7f518015cee0 2024-03-22T04:13:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f5184a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f518015fe40 con 0x7f51800b3600 2024-03-22T04:13:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f51800b2360 0x7f51800b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53220/0 (socket says 172.21.15.86:53220) 2024-03-22T04:13:27.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517de00640 1 -- 172.21.15.86:0/2090492712 learned_addr learned my addr 172.21.15.86:0/2090492712 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4194957294 0 0) 0x7f518015d710 con 0x7f51800b2360 2024-03-22T04:13:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5164003530 con 0x7f51800b2360 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f5175400640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800b3600 0x7f518015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517e800640 1 --2- 172.21.15.86:0/2090492712 >> v2:172.21.15.196:3300/0 conn(0x7f518015cee0 0x7f51800b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 367541665 0 0) 0x7f5164003530 con 0x7f51800b2360 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800b3600 msgr2=0x7f518015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800b3600 0x7f518015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 >> v2:172.21.15.196:3300/0 conn(0x7f518015cee0 msgr2=0x7f51800b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 --2- 172.21.15.86:0/2090492712 >> v2:172.21.15.196:3300/0 conn(0x7f518015cee0 0x7f51800b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5180160120 con 0x7f51800b2360 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f5170001680 con 0x7f51800b2360 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 >> v1:172.21.15.86:6789/0 conn(0x7f51800b2360 legacy=0x7f51800b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800b3600 0x7f51640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004040 0x7f5164004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164004d50 0x7f51640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51800b2740 con 0x7f5164004d50 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f518015fe40 con 0x7f5164004040 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f516400bb30 con 0x7f51800b3600 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517e800640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164004d50 0x7f51640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f5175400640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004040 0x7f5164004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517de00640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800b3600 0x7f51640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517e800640 1 -- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800b3600 msgr2=0x7f51640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517e800640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800b3600 0x7f51640039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517e800640 1 -- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004040 msgr2=0x7f5164004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517e800640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004040 0x7f5164004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.633+0000 7f517e800640 1 -- 172.21.15.86:0/2090492712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5164003530 con 0x7f5164004d50 2024-03-22T04:13:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f517e800640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164004d50 0x7f51640051d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5178007360 tx=0x7f517801ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f256ae7ce85c31d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51780b2070 con 0x7f5164004d50 2024-03-22T04:13:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51780a5d80 con 0x7f5164004d50 2024-03-22T04:13:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f517d400640 1 -- 172.21.15.86:0/2090492712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178009be0 con 0x7f5164004d50 2024-03-22T04:13:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f5184a00640 1 -- 172.21.15.86:0/2090492712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164004d50 msgr2=0x7f51640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f5184a00640 1 --2- 172.21.15.86:0/2090492712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5164004d50 0x7f51640051d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5178007360 tx=0x7f517801ce80 comp rx=0 tx=0).stop 2024-03-22T04:13:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f517e800640 1 -- 172.21.15.86:0/2090492712 reap_dead start 2024-03-22T04:13:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f5184a00640 1 -- 172.21.15.86:0/2090492712 shutdown_connections 2024-03-22T04:13:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f5184a00640 1 -- 172.21.15.86:0/2090492712 >> 172.21.15.86:0/2090492712 conn(0x7f51800cc940 msgr2=0x7f51800ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f5184a00640 1 -- 172.21.15.86:0/2090492712 shutdown_connections 2024-03-22T04:13:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.637+0000 7f5184a00640 1 -- 172.21.15.86:0/2090492712 wait complete. 2024-03-22T04:13:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 Processor -- start 2024-03-22T04:13:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 -- start start 2024-03-22T04:13:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004d50 0x7f5180147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800aeeb0 0x7f518015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51800af330 0x7f518015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5180160c00 con 0x7f51800af330 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51801606e0 con 0x7f5164004d50 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f51801477a0 con 0x7f51800aeeb0 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51800af330 0x7f518015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51800af330 0x7f518015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45084/0 (socket says 172.21.15.86:45084) 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 -- 172.21.15.86:0/1472998038 learned_addr learned my addr 172.21.15.86:0/1472998038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517de00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004d50 0x7f5180147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5175400640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800aeeb0 0x7f518015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 -- 172.21.15.86:0/1472998038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800aeeb0 msgr2=0x7f518015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800aeeb0 0x7f518015b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 -- 172.21.15.86:0/1472998038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004d50 msgr2=0x7f5180147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004d50 0x7f5180147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 -- 172.21.15.86:0/1472998038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51800a9650 con 0x7f51800af330 2024-03-22T04:13:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f517e800640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51800af330 0x7f518015b7d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5178002240 tx=0x7f5164005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5176a00640 1 -- 172.21.15.86:0/1472998038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51780b2040 con 0x7f51800af330 2024-03-22T04:13:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5176a00640 1 -- 172.21.15.86:0/1472998038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51780a5b80 con 0x7f51800af330 2024-03-22T04:13:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5176a00640 1 -- 172.21.15.86:0/1472998038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178009be0 con 0x7f51800af330 2024-03-22T04:13:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51800a9930 con 0x7f51800af330 2024-03-22T04:13:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51800a9ef0 con 0x7f51800af330 2024-03-22T04:13:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.641+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5180074470 con 0x7f51800af330 2024-03-22T04:13:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.645+0000 7f5176a00640 1 -- 172.21.15.86:0/1472998038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f51780bf060 con 0x7f51800af330 2024-03-22T04:13:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.645+0000 7f5176a00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f515c067b20 0x7f515c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.645+0000 7f5176a00640 1 -- 172.21.15.86:0/1472998038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f517811fda0 con 0x7f51800af330 2024-03-22T04:13:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.645+0000 7f517de00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f515c067b20 0x7f515c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.645+0000 7f517de00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f515c067b20 0x7f515c069fe0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5170001ec0 tx=0x7f5170001e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.649+0000 7f5176a00640 1 -- 172.21.15.86:0/1472998038 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5178016b90 con 0x7f51800af330 2024-03-22T04:13:27.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:13:27.669 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:13:27.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.809+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5180160d80 con 0x7f51800af330 2024-03-22T04:13:27.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5176a00640 1 -- 172.21.15.86:0/1472998038 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f51780f0230 con 0x7f51800af330 2024-03-22T04:13:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f515c067b20 msgr2=0x7f515c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f515c067b20 0x7f515c069fe0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5170001ec0 tx=0x7f5170001e10 comp rx=0 tx=0).stop 2024-03-22T04:13:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51800af330 msgr2=0x7f518015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51800af330 0x7f518015b7d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5178002240 tx=0x7f5164005840 comp rx=0 tx=0).stop 2024-03-22T04:13:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 shutdown_connections 2024-03-22T04:13:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f515c067b20 0x7f515c069fe0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51800af330 0x7f518015b7d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51800aeeb0 0x7f518015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 --2- 172.21.15.86:0/1472998038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5164004d50 0x7f5180147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 >> 172.21.15.86:0/1472998038 conn(0x7f51800cc940 msgr2=0x7f51800cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 shutdown_connections 2024-03-22T04:13:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.813+0000 7f5184a00640 1 -- 172.21.15.86:0/1472998038 wait complete. 2024-03-22T04:13:27.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-22T04:13:27.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7bee00640 1 Processor -- start 2024-03-22T04:13:27.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7bee00640 1 -- start start 2024-03-22T04:13:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7bee00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7b815b3f0 0x7fa7b80729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7bee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80cc3e0 0x7fa7b8072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7bee00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b8071920 con 0x7fa7b80b30f0 2024-03-22T04:13:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7bee00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b80716d0 con 0x7fa7b815b3f0 2024-03-22T04:13:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7bee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b806d990 con 0x7fa7b80cc3e0 2024-03-22T04:13:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7af400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80cc3e0 0x7fa7b8072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7b7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7b815b3f0 0x7fa7b80729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7af400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80cc3e0 0x7fa7b8072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:33950/0 (socket says 172.21.15.86:33950) 2024-03-22T04:13:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7b7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7b815b3f0 0x7fa7b80729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:48892/0 (socket says 172.21.15.86:48892) 2024-03-22T04:13:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7af400640 1 -- 172.21.15.86:0/4257148345 learned_addr learned my addr 172.21.15.86:0/4257148345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.968+0000 7fa7bce00640 1 --1- 172.21.15.86:0/4257148345 >> v1:172.21.15.196:6789/0 conn(0x7fa7b80b30f0 0x7fa7b80ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:37520/0 (socket says 172.21.15.86:37520) 2024-03-22T04:13:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7af400640 1 -- 172.21.15.86:0/4257148345 >> v2:172.21.15.196:3300/0 conn(0x7fa7b815b3f0 msgr2=0x7fa7b80729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7af400640 1 --2- 172.21.15.86:0/4257148345 >> v2:172.21.15.196:3300/0 conn(0x7fa7b815b3f0 0x7fa7b80729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7af400640 1 -- 172.21.15.86:0/4257148345 >> v1:172.21.15.196:6789/0 conn(0x7fa7b80b30f0 legacy=0x7fa7b80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7af400640 1 -- 172.21.15.86:0/4257148345 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b8071c00 con 0x7fa7b80cc3e0 2024-03-22T04:13:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7af400640 1 --2- 172.21.15.86:0/4257148345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80cc3e0 0x7fa7b8072f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009870 tx=0x7fa7a001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3de12d181d55a56 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7b7400640 1 -- 172.21.15.86:0/4257148345 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00b1070 con 0x7fa7b80cc3e0 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7b7400640 1 -- 172.21.15.86:0/4257148345 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a00ac620 con 0x7fa7b80cc3e0 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7b7400640 1 -- 172.21.15.86:0/4257148345 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00ac9f0 con 0x7fa7b80cc3e0 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7bee00640 1 -- 172.21.15.86:0/4257148345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80cc3e0 msgr2=0x7fa7b8072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7bee00640 1 --2- 172.21.15.86:0/4257148345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80cc3e0 0x7fa7b8072f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009870 tx=0x7fa7a001c4d0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7bee00640 1 -- 172.21.15.86:0/4257148345 shutdown_connections 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7bee00640 1 --2- 172.21.15.86:0/4257148345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80cc3e0 0x7fa7b8072f00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7bee00640 1 --2- 172.21.15.86:0/4257148345 >> v2:172.21.15.196:3300/0 conn(0x7fa7b815b3f0 0x7fa7b80729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.972+0000 7fa7bee00640 1 -- 172.21.15.86:0/4257148345 >> 172.21.15.86:0/4257148345 conn(0x7fa7b80b1df0 msgr2=0x7fa7b80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 -- 172.21.15.86:0/4257148345 shutdown_connections 2024-03-22T04:13:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 -- 172.21.15.86:0/4257148345 wait complete. 2024-03-22T04:13:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 Processor -- start 2024-03-22T04:13:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 -- start start 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80b30f0 0x7fa7b81392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b80cc3e0 0x7fa7b8167bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7b815b3f0 0x7fa7b815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b8166570 con 0x7fa7b815b3f0 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81663f0 con 0x7fa7b80cc3e0 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b81666f0 con 0x7fa7b80b30f0 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7b815b3f0 0x7fa7b815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7b815b3f0 0x7fa7b815d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45088/0 (socket says 172.21.15.86:45088) 2024-03-22T04:13:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 -- 172.21.15.86:0/622915472 learned_addr learned my addr 172.21.15.86:0/622915472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7b7e00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80b30f0 0x7fa7b81392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7af400640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b80cc3e0 0x7fa7b8167bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 -- 172.21.15.86:0/622915472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80b30f0 msgr2=0x7fa7b81392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80b30f0 0x7fa7b81392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 -- 172.21.15.86:0/622915472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b80cc3e0 msgr2=0x7fa7b8167bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b80cc3e0 0x7fa7b8167bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 -- 172.21.15.86:0/622915472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b8150760 con 0x7fa7b815b3f0 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bce00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7b815b3f0 0x7fa7b815d170 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000c7a0 tx=0x7fa7b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7b4c00640 1 -- 172.21.15.86:0/622915472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0017070 con 0x7fa7b815b3f0 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7b4c00640 1 -- 172.21.15.86:0/622915472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b000a640 con 0x7fa7b815b3f0 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7b4c00640 1 -- 172.21.15.86:0/622915472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000a9f0 con 0x7fa7b815b3f0 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.976+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b81684c0 con 0x7fa7b815b3f0 2024-03-22T04:13:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.980+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b81397f0 con 0x7fa7b815b3f0 2024-03-22T04:13:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.980+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b8068650 con 0x7fa7b815b3f0 2024-03-22T04:13:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.980+0000 7fa7b4c00640 1 -- 172.21.15.86:0/622915472 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa7b001a070 con 0x7fa7b815b3f0 2024-03-22T04:13:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.980+0000 7fa7b4c00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa794067b20 0x7fa794069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.980+0000 7fa7b4c00640 1 -- 172.21.15.86:0/622915472 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7fa7b0085340 con 0x7fa7b815b3f0 2024-03-22T04:13:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.980+0000 7fa7b7e00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa794067b20 0x7fa794069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.980+0000 7fa7b7e00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa794067b20 0x7fa794069fe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009a50 tx=0x7fa7a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:27.984+0000 7fa7b4c00640 1 -- 172.21.15.86:0/622915472 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa7b0055860 con 0x7fa7b815b3f0 2024-03-22T04:13:28.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.200+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fa7b8152570 con 0x7fa7b815b3f0 2024-03-22T04:13:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.464+0000 7fa7b4c00640 1 -- 172.21.15.86:0/622915472 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v181) ==== 149+0+0 (secure 0 0 0) 0x7fa7b0059900 con 0x7fa7b815b3f0 2024-03-22T04:13:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa794067b20 msgr2=0x7fa794069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa794067b20 0x7fa794069fe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009a50 tx=0x7fa7a8008040 comp rx=0 tx=0).stop 2024-03-22T04:13:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7b815b3f0 msgr2=0x7fa7b815d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7b815b3f0 0x7fa7b815d170 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000c7a0 tx=0x7fa7b000cc60 comp rx=0 tx=0).stop 2024-03-22T04:13:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 shutdown_connections 2024-03-22T04:13:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa794067b20 0x7fa794069fe0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b80cc3e0 0x7fa7b8167bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7b815b3f0 0x7fa7b815d170 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 --2- 172.21.15.86:0/622915472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b80b30f0 0x7fa7b81392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 >> 172.21.15.86:0/622915472 conn(0x7fa7b80b1df0 msgr2=0x7fa7b815b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 shutdown_connections 2024-03-22T04:13:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.468+0000 7fa7bee00640 1 -- 172.21.15.86:0/622915472 wait complete. 2024-03-22T04:13:28.491 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-03-22T04:13:28.492 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-22T04:13:28.492 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-22T04:13:28.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:28.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415e00640 1 Processor -- start 2024-03-22T04:13:28.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415e00640 1 -- start start 2024-03-22T04:13:28.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4100cfab0 0x7fa41015f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41015b680 con 0x7fa41006ddd0 2024-03-22T04:13:28.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41015b8f0 con 0x7fa41013e060 2024-03-22T04:13:28.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa41015c6b0 con 0x7fa4100cfab0 2024-03-22T04:13:28.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa407e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa41006ddd0 0x7fa41015ef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53222/0 (socket says 172.21.15.86:53222) 2024-03-22T04:13:28.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa407e00640 1 -- 172.21.15.86:0/1074532618 learned_addr learned my addr 172.21.15.86:0/1074532618 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:28.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415400640 1 --2- 172.21.15.86:0/1074532618 >> v2:172.21.15.86:3300/0 conn(0x7fa4100cfab0 0x7fa41015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415400640 1 -- 172.21.15.86:0/1074532618 >> v1:172.21.15.196:6789/0 conn(0x7fa41013e060 legacy=0x7fa41006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415400640 1 -- 172.21.15.86:0/1074532618 >> v1:172.21.15.86:6789/0 conn(0x7fa41006ddd0 legacy=0x7fa41015ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa415400640 1 -- 172.21.15.86:0/1074532618 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41015bbd0 con 0x7fa4100cfab0 2024-03-22T04:13:28.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.632+0000 7fa40fe00640 1 -- 172.21.15.86:0/1074532618 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1597139314 0 0) 0x7fa41015b680 con 0x7fa41006ddd0 2024-03-22T04:13:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa415400640 1 --2- 172.21.15.86:0/1074532618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4100cfab0 0x7fa41015f610 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa408009d90 tx=0x7fa40801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52809d63a9f170d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa40fe00640 1 -- 172.21.15.86:0/1074532618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080b2070 con 0x7fa4100cfab0 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa40fe00640 1 -- 172.21.15.86:0/1074532618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4080a5890 con 0x7fa4100cfab0 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa40fe00640 1 -- 172.21.15.86:0/1074532618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a5c60 con 0x7fa4100cfab0 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa415e00640 1 -- 172.21.15.86:0/1074532618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4100cfab0 msgr2=0x7fa41015f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa415e00640 1 --2- 172.21.15.86:0/1074532618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4100cfab0 0x7fa41015f610 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa408009d90 tx=0x7fa40801c8e0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa415e00640 1 -- 172.21.15.86:0/1074532618 shutdown_connections 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa415e00640 1 --2- 172.21.15.86:0/1074532618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4100cfab0 0x7fa41015f610 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa415e00640 1 -- 172.21.15.86:0/1074532618 >> 172.21.15.86:0/1074532618 conn(0x7fa41006b660 msgr2=0x7fa41006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa415e00640 1 -- 172.21.15.86:0/1074532618 shutdown_connections 2024-03-22T04:13:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.636+0000 7fa415e00640 1 -- 172.21.15.86:0/1074532618 wait complete. 2024-03-22T04:13:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 Processor -- start 2024-03-22T04:13:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 -- start start 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41006ddd0 0x7fa410137290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100cfab0 0x7fa41015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa41013e060 0x7fa41015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100b32e0 con 0x7fa41006ddd0 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100b3160 con 0x7fa41013e060 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4100b3460 con 0x7fa4100cfab0 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41006ddd0 0x7fa410137290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41006ddd0 0x7fa410137290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45094/0 (socket says 172.21.15.86:45094) 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 -- 172.21.15.86:0/976005660 learned_addr learned my addr 172.21.15.86:0/976005660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa407e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100cfab0 0x7fa41015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415400640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa41013e060 0x7fa41015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 -- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100cfab0 msgr2=0x7fa41015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100cfab0 0x7fa41015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 -- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa41013e060 msgr2=0x7fa41015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa41013e060 0x7fa41015a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 -- 172.21.15.86:0/976005660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa410154560 con 0x7fa41006ddd0 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415400640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa41013e060 0x7fa41015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa414a00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41006ddd0 0x7fa410137290 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa40000c3c0 tx=0x7fa40000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa40d600640 1 -- 172.21.15.86:0/976005660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400017070 con 0x7fa41006ddd0 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa40d600640 1 -- 172.21.15.86:0/976005660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4000052d0 con 0x7fa41006ddd0 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa40d600640 1 -- 172.21.15.86:0/976005660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400003840 con 0x7fa41006ddd0 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4101534c0 con 0x7fa41006ddd0 2024-03-22T04:13:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa410153a80 con 0x7fa41006ddd0 2024-03-22T04:13:28.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.640+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410074470 con 0x7fa41006ddd0 2024-03-22T04:13:28.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.644+0000 7fa40d600640 1 -- 172.21.15.86:0/976005660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa40001a070 con 0x7fa41006ddd0 2024-03-22T04:13:28.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.644+0000 7fa40d600640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa3ec067b20 0x7fa3ec069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.644+0000 7fa40d600640 1 -- 172.21.15.86:0/976005660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7fa400084f70 con 0x7fa41006ddd0 2024-03-22T04:13:28.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.644+0000 7fa407e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa3ec067b20 0x7fa3ec069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.644+0000 7fa407e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa3ec067b20 0x7fa3ec069fe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa4100b2d40 tx=0x7fa3f8007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.648+0000 7fa40d600640 1 -- 172.21.15.86:0/976005660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa400005480 con 0x7fa41006ddd0 2024-03-22T04:13:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4101377d0 con 0x7fa41006ddd0 2024-03-22T04:13:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa40d600640 1 -- 172.21.15.86:0/976005660 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+13295 (secure 0 0 0) 0x7fa400055530 con 0x7fa41006ddd0 2024-03-22T04:13:28.807 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:28.807 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:27.458726+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6834/2728972599":"2024-03-23T04:13:24.445072+0000","172.21.15.196:6832/2728972599":"2024-03-23T04:13:24.445072+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa3ec067b20 msgr2=0x7fa3ec069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa3ec067b20 0x7fa3ec069fe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa4100b2d40 tx=0x7fa3f8007240 comp rx=0 tx=0).stop 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41006ddd0 msgr2=0x7fa410137290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41006ddd0 0x7fa410137290 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa40000c3c0 tx=0x7fa40000c880 comp rx=0 tx=0).stop 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 shutdown_connections 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa3ec067b20 0x7fa3ec069fe0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa41013e060 0x7fa41015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100cfab0 0x7fa41015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 --2- 172.21.15.86:0/976005660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41006ddd0 0x7fa410137290 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 >> 172.21.15.86:0/976005660 conn(0x7fa41006b660 msgr2=0x7fa41013c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 shutdown_connections 2024-03-22T04:13:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.804+0000 7fa415e00640 1 -- 172.21.15.86:0/976005660 wait complete. 2024-03-22T04:13:28.828 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:13:28.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:13:28.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f60a00640 1 Processor -- start 2024-03-22T04:13:28.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f60a00640 1 -- start start 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f60a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f5c06e7e0 0x7f6f5c06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f60a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c072650 0x7f6f5c072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f60a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f5c15c770 con 0x7f6f5c0b1b80 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f60a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6f5c071350 con 0x7f6f5c06e7e0 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f60a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5c06f100 con 0x7f6f5c072650 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f5a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6f5c0b1b80 0x7f6f5c0b2010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53232/0 (socket says 172.21.15.86:53232) 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f5a800640 1 -- 172.21.15.86:0/885941788 learned_addr learned my addr 172.21.15.86:0/885941788 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59e00640 1 --2- 172.21.15.86:0/885941788 >> v2:172.21.15.196:3300/0 conn(0x7f6f5c06e7e0 0x7f6f5c06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f53e00640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c072650 0x7f6f5c072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202100095 0 0) 0x7f6f540014a0 con 0x7f6f5c0b1b80 2024-03-22T04:13:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f40003530 con 0x7f6f5c0b1b80 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1086712950 0 0) 0x7f6f40003530 con 0x7f6f5c0b1b80 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c072650 msgr2=0x7f6f5c072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c072650 0x7f6f5c072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 >> v2:172.21.15.196:3300/0 conn(0x7f6f5c06e7e0 msgr2=0x7f6f5c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 --2- 172.21.15.86:0/885941788 >> v2:172.21.15.196:3300/0 conn(0x7f6f5c06e7e0 0x7f6f5c06ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c073150 con 0x7f6f5c0b1b80 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f6f54001bd0 con 0x7f6f5c0b1b80 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 >> v1:172.21.15.86:6789/0 conn(0x7f6f5c0b1b80 legacy=0x7f6f5c0b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c072650 0x7f6f400039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f40003f40 0x7f6f40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c071350 con 0x7f6f40004d50 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c06f100 con 0x7f6f40003f40 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4000ba20 con 0x7f6f5c072650 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f5a800640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f59e00640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c072650 0x7f6f400039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f53e00640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f40003f40 0x7f6f40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f5a800640 1 -- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c072650 msgr2=0x7f6f400039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f5a800640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c072650 0x7f6f400039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f5a800640 1 -- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f40003f40 msgr2=0x7f6f40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f5a800640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f40003f40 0x7f6f40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f5a800640 1 -- 172.21.15.86:0/885941788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f40003530 con 0x7f6f40004d50 2024-03-22T04:13:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.964+0000 7f6f53e00640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f40003f40 0x7f6f40004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f5a800640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f400051d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400aaa0 tx=0x7f6f5401fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94590ae87254846b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400a690 con 0x7f6f40004d50 2024-03-22T04:13:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f540ad070 con 0x7f6f40004d50 2024-03-22T04:13:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f59400640 1 -- 172.21.15.86:0/885941788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f540a99c0 con 0x7f6f40004d50 2024-03-22T04:13:28.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f60a00640 1 -- 172.21.15.86:0/885941788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 msgr2=0x7f6f400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f60a00640 1 --2- 172.21.15.86:0/885941788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f400051d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400aaa0 tx=0x7f6f5401fb00 comp rx=0 tx=0).stop 2024-03-22T04:13:28.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f5a800640 1 -- 172.21.15.86:0/885941788 reap_dead start 2024-03-22T04:13:28.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f60a00640 1 -- 172.21.15.86:0/885941788 shutdown_connections 2024-03-22T04:13:28.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f60a00640 1 -- 172.21.15.86:0/885941788 >> 172.21.15.86:0/885941788 conn(0x7f6f5c0cc940 msgr2=0x7f6f5c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f60a00640 1 -- 172.21.15.86:0/885941788 shutdown_connections 2024-03-22T04:13:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.968+0000 7f6f60a00640 1 -- 172.21.15.86:0/885941788 wait complete. 2024-03-22T04:13:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 Processor -- start 2024-03-22T04:13:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 -- start start 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f5c1e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c0caac0 0x7f6f5c1f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c152710 0x7f6f5c152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c071b60 con 0x7f6f40004d50 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c071880 con 0x7f6f5c152710 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5c0ce930 con 0x7f6f5c0caac0 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f5c1e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f5c1e5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45104/0 (socket says 172.21.15.86:45104) 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 -- 172.21.15.86:0/3985998688 learned_addr learned my addr 172.21.15.86:0/3985998688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f5a800640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c152710 0x7f6f5c152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f53e00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c0caac0 0x7f6f5c1f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 -- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c0caac0 msgr2=0x7f6f5c1f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c0caac0 0x7f6f5c1f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 -- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c152710 msgr2=0x7f6f5c152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c152710 0x7f6f5c152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 -- 172.21.15.86:0/3985998688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c1e39d0 con 0x7f6f40004d50 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f53e00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c0caac0 0x7f6f5c1f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f59e00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f5c1e5e50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c011980 tx=0x7f6f4c011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f52000640 1 -- 172.21.15.86:0/3985998688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c00faf0 con 0x7f6f40004d50 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f52000640 1 -- 172.21.15.86:0/3985998688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f4c00fc90 con 0x7f6f40004d50 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f52000640 1 -- 172.21.15.86:0/3985998688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c014440 con 0x7f6f40004d50 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c1530e0 con 0x7f6f40004d50 2024-03-22T04:13:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.972+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c0cb280 con 0x7f6f40004d50 2024-03-22T04:13:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.976+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5c074470 con 0x7f6f40004d50 2024-03-22T04:13:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.976+0000 7f6f52000640 1 -- 172.21.15.86:0/3985998688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6f4c012040 con 0x7f6f40004d50 2024-03-22T04:13:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.976+0000 7f6f52000640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f38067ac0 0x7f6f38069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.976+0000 7f6f52000640 1 -- 172.21.15.86:0/3985998688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f6f4c0864c0 con 0x7f6f40004d50 2024-03-22T04:13:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.976+0000 7f6f53e00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f38067ac0 0x7f6f38069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.980+0000 7f6f53e00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f38067ac0 0x7f6f38069f80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c06e280 tx=0x7f6f44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:28.980+0000 7f6f52000640 1 -- 172.21.15.86:0/3985998688 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6f4c019030 con 0x7f6f40004d50 2024-03-22T04:13:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.136+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f5c072d10 con 0x7f6f40004d50 2024-03-22T04:13:29.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.136+0000 7f6f52000640 1 -- 172.21.15.86:0/3985998688 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f6f4c057030 con 0x7f6f40004d50 2024-03-22T04:13:29.141 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.136+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f38067ac0 msgr2=0x7f6f38069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f38067ac0 0x7f6f38069f80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c06e280 tx=0x7f6f44008040 comp rx=0 tx=0).stop 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 msgr2=0x7f6f5c1e5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f5c1e5e50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c011980 tx=0x7f6f4c011e40 comp rx=0 tx=0).stop 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 shutdown_connections 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f38067ac0 0x7f6f38069f80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f5c152710 0x7f6f5c152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f5c0caac0 0x7f6f5c1f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 --2- 172.21.15.86:0/3985998688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f40004d50 0x7f6f5c1e5e50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 >> 172.21.15.86:0/3985998688 conn(0x7f6f5c0cc940 msgr2=0x7f6f5c0b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:29.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 shutdown_connections 2024-03-22T04:13:29.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.140+0000 7f6f60a00640 1 -- 172.21.15.86:0/3985998688 wait complete. 2024-03-22T04:13:29.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-22T04:13:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f11a5800640 1 Processor -- start 2024-03-22T04:13:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f11a5800640 1 -- start start 2024-03-22T04:13:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f11a5800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a00d0580 0x7f11a0071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f11a5800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f11a006e2f0 0x7f11a0071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f11a5800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a01623d0 con 0x7f11a015e630 2024-03-22T04:13:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f11a5800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f11a01620f0 con 0x7f11a006e2f0 2024-03-22T04:13:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f11a5800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f11a015ec10 con 0x7f11a00d0580 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f1196000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a00d0580 0x7f11a0071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.296+0000 7f119f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f11a006e2f0 0x7f11a0071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f1196000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a00d0580 0x7f11a0071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:34020/0 (socket says 172.21.15.86:34020) 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f1196000640 1 -- 172.21.15.86:0/2524659299 learned_addr learned my addr 172.21.15.86:0/2524659299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f119ea00640 1 --1- 172.21.15.86:0/2524659299 >> v1:172.21.15.196:6789/0 conn(0x7f11a015e630 0x7f11a00cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:37538/0 (socket says 172.21.15.86:37538) 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f1196000640 1 -- 172.21.15.86:0/2524659299 >> v2:172.21.15.196:3300/0 conn(0x7f11a006e2f0 msgr2=0x7f11a0071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f1196000640 1 --2- 172.21.15.86:0/2524659299 >> v2:172.21.15.196:3300/0 conn(0x7f11a006e2f0 0x7f11a0071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f1196000640 1 -- 172.21.15.86:0/2524659299 >> v1:172.21.15.196:6789/0 conn(0x7f11a015e630 legacy=0x7f11a00cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f1196000640 1 -- 172.21.15.86:0/2524659299 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a015eef0 con 0x7f11a00d0580 2024-03-22T04:13:29.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f1196000640 1 --2- 172.21.15.86:0/2524659299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a00d0580 0x7f11a0071100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1188009870 tx=0x7f118801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbc3a2b902d515fa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f119e000640 1 -- 172.21.15.86:0/2524659299 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11880b2070 con 0x7f11a00d0580 2024-03-22T04:13:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f119e000640 1 -- 172.21.15.86:0/2524659299 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11880a9070 con 0x7f11a00d0580 2024-03-22T04:13:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.300+0000 7f119e000640 1 -- 172.21.15.86:0/2524659299 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11880ad880 con 0x7f11a00d0580 2024-03-22T04:13:29.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- 172.21.15.86:0/2524659299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a00d0580 msgr2=0x7f11a0071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 --2- 172.21.15.86:0/2524659299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a00d0580 0x7f11a0071100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1188009870 tx=0x7f118801c560 comp rx=0 tx=0).stop 2024-03-22T04:13:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- 172.21.15.86:0/2524659299 shutdown_connections 2024-03-22T04:13:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 --2- 172.21.15.86:0/2524659299 >> v2:172.21.15.196:3300/0 conn(0x7f11a006e2f0 0x7f11a0071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 --2- 172.21.15.86:0/2524659299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a00d0580 0x7f11a0071100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- 172.21.15.86:0/2524659299 >> 172.21.15.86:0/2524659299 conn(0x7f11a006f910 msgr2=0x7f11a006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- 172.21.15.86:0/2524659299 shutdown_connections 2024-03-22T04:13:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- 172.21.15.86:0/2524659299 wait complete. 2024-03-22T04:13:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 Processor -- start 2024-03-22T04:13:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- start start 2024-03-22T04:13:29.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a006e2f0 0x7f11a0153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:29.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a00d0580 0x7f11a00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:29.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a015e630 0x7f11a00b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11a0071f90 con 0x7f11a015e630 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11a015f9d0 con 0x7f11a00d0580 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f11a5800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f11a00cfaf0 con 0x7f11a006e2f0 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a015e630 0x7f11a00b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a015e630 0x7f11a00b74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45108/0 (socket says 172.21.15.86:45108) 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119f400640 1 -- 172.21.15.86:0/1452627512 learned_addr learned my addr 172.21.15.86:0/1452627512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119ea00640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a006e2f0 0x7f11a0153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f1196000640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a00d0580 0x7f11a00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119f400640 1 -- 172.21.15.86:0/1452627512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a006e2f0 msgr2=0x7f11a0153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119f400640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a006e2f0 0x7f11a0153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119f400640 1 -- 172.21.15.86:0/1452627512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a00d0580 msgr2=0x7f11a00b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119f400640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a00d0580 0x7f11a00b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.304+0000 7f119f400640 1 -- 172.21.15.86:0/1452627512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a01542d0 con 0x7f11a015e630 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f119f400640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a015e630 0x7f11a00b74d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f119800efc0 tx=0x7f119800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:29.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f1197400640 1 -- 172.21.15.86:0/1452627512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198018020 con 0x7f11a015e630 2024-03-22T04:13:29.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f1197400640 1 -- 172.21.15.86:0/1452627512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1198014480 con 0x7f11a015e630 2024-03-22T04:13:29.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f1197400640 1 -- 172.21.15.86:0/1452627512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198014870 con 0x7f11a015e630 2024-03-22T04:13:29.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a0155060 con 0x7f11a015e630 2024-03-22T04:13:29.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a0155620 con 0x7f11a015e630 2024-03-22T04:13:29.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a0155860 con 0x7f11a015e630 2024-03-22T04:13:29.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f1197400640 1 -- 172.21.15.86:0/1452627512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f119801b020 con 0x7f11a015e630 2024-03-22T04:13:29.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f1197400640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f117c067b20 0x7f117c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:29.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f1197400640 1 -- 172.21.15.86:0/1452627512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f11980866f0 con 0x7f11a015e630 2024-03-22T04:13:29.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f119ea00640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f117c067b20 0x7f117c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:29.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.308+0000 7f119ea00640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f117c067b20 0x7f117c069fe0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f11a00cf380 tx=0x7f1190007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:29.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.312+0000 7f1197400640 1 -- 172.21.15.86:0/1452627512 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1198022070 con 0x7f11a015e630 2024-03-22T04:13:29.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:29.524+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f11a00713e0 con 0x7f11a015e630 2024-03-22T04:13:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.472+0000 7f1197400640 1 -- 172.21.15.86:0/1452627512 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v183) ==== 172+0+0 (secure 0 0 0) 0x7f11a01542d0 con 0x7f11a015e630 2024-03-22T04:13:30.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f117c067b20 msgr2=0x7f117c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f117c067b20 0x7f117c069fe0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f11a00cf380 tx=0x7f1190007e90 comp rx=0 tx=0).stop 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a015e630 msgr2=0x7f11a00b74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a015e630 0x7f11a00b74d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f119800efc0 tx=0x7f119800c490 comp rx=0 tx=0).stop 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 shutdown_connections 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f117c067b20 0x7f117c069fe0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11a00d0580 0x7f11a00b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a015e630 0x7f11a00b74d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 --2- 172.21.15.86:0/1452627512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11a006e2f0 0x7f11a0153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 >> 172.21.15.86:0/1452627512 conn(0x7f11a006f910 msgr2=0x7f11a0168970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 shutdown_connections 2024-03-22T04:13:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.476+0000 7f11a5800640 1 -- 172.21.15.86:0/1452627512 wait complete. 2024-03-22T04:13:30.504 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-22T04:13:30.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.640+0000 7f7fc3200640 1 Processor -- start 2024-03-22T04:13:30.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.640+0000 7f7fc3200640 1 -- start start 2024-03-22T04:13:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc3200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7fc406bd70 0x7f7fc40d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7fc40d05f0 0x7f7fc406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fc4162e80 con 0x7f7fc415e4b0 2024-03-22T04:13:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc415e890 con 0x7f7fc40d05f0 2024-03-22T04:13:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc3200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc4162950 con 0x7f7fc406bd70 2024-03-22T04:13:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7fc40d05f0 0x7f7fc406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7fc415e4b0 0x7f7fc406b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53238/0 (socket says 172.21.15.86:53238) 2024-03-22T04:13:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc1e00640 1 -- 172.21.15.86:0/1765057166 learned_addr learned my addr 172.21.15.86:0/1765057166 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fb9400640 1 --2- 172.21.15.86:0/1765057166 >> v2:172.21.15.196:3300/0 conn(0x7f7fc406bd70 0x7f7fc40d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc2800640 1 -- 172.21.15.86:0/1765057166 >> v2:172.21.15.196:3300/0 conn(0x7f7fc406bd70 msgr2=0x7f7fc40d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc2800640 1 --2- 172.21.15.86:0/1765057166 >> v2:172.21.15.196:3300/0 conn(0x7f7fc406bd70 0x7f7fc40d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc2800640 1 -- 172.21.15.86:0/1765057166 >> v1:172.21.15.86:6789/0 conn(0x7f7fc415e4b0 legacy=0x7f7fc406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc2800640 1 -- 172.21.15.86:0/1765057166 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4162ba0 con 0x7f7fc40d05f0 2024-03-22T04:13:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fc1400640 1 -- 172.21.15.86:0/1765057166 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 29367350 0 0) 0x7f7fc4162e80 con 0x7f7fc415e4b0 2024-03-22T04:13:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.644+0000 7f7fb9400640 1 --2- 172.21.15.86:0/1765057166 >> v2:172.21.15.196:3300/0 conn(0x7f7fc406bd70 0x7f7fc40d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:13:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc2800640 1 --2- 172.21.15.86:0/1765057166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc40d05f0 0x7f7fc406e4d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7fc406e220 tx=0x7f7fbc01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e72c3c3675af814 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc1400640 1 -- 172.21.15.86:0/1765057166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0b2070 con 0x7f7fc40d05f0 2024-03-22T04:13:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc1400640 1 -- 172.21.15.86:0/1765057166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc0a57c0 con 0x7f7fc40d05f0 2024-03-22T04:13:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc1400640 1 -- 172.21.15.86:0/1765057166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0a5b90 con 0x7f7fc40d05f0 2024-03-22T04:13:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- 172.21.15.86:0/1765057166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc40d05f0 msgr2=0x7f7fc406e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 --2- 172.21.15.86:0/1765057166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc40d05f0 0x7f7fc406e4d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7fc406e220 tx=0x7f7fbc01c810 comp rx=0 tx=0).stop 2024-03-22T04:13:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- 172.21.15.86:0/1765057166 shutdown_connections 2024-03-22T04:13:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 --2- 172.21.15.86:0/1765057166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc40d05f0 0x7f7fc406e4d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 --2- 172.21.15.86:0/1765057166 >> v2:172.21.15.196:3300/0 conn(0x7f7fc406bd70 0x7f7fc40d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- 172.21.15.86:0/1765057166 >> 172.21.15.86:0/1765057166 conn(0x7f7fc40d0ec0 msgr2=0x7f7fc40d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- 172.21.15.86:0/1765057166 shutdown_connections 2024-03-22T04:13:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- 172.21.15.86:0/1765057166 wait complete. 2024-03-22T04:13:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 Processor -- start 2024-03-22T04:13:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- start start 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fc406bd70 0x7f7fc4155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7fc40d05f0 0x7f7fc4161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc415e4b0 0x7f7fc40ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc406f3a0 con 0x7f7fc415e4b0 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc406f220 con 0x7f7fc406bd70 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc3200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc406f520 con 0x7f7fc40d05f0 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc415e4b0 0x7f7fc40ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc415e4b0 0x7f7fc40ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45124/0 (socket says 172.21.15.86:45124) 2024-03-22T04:13:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.648+0000 7f7fc2800640 1 -- 172.21.15.86:0/3121598886 learned_addr learned my addr 172.21.15.86:0/3121598886 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fb9400640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7fc40d05f0 0x7f7fc4161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc1e00640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fc406bd70 0x7f7fc4155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc2800640 1 -- 172.21.15.86:0/3121598886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7fc40d05f0 msgr2=0x7f7fc4161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc2800640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7fc40d05f0 0x7f7fc4161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc2800640 1 -- 172.21.15.86:0/3121598886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fc406bd70 msgr2=0x7f7fc4155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc2800640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fc406bd70 0x7f7fc4155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc2800640 1 -- 172.21.15.86:0/3121598886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4156390 con 0x7f7fc415e4b0 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc2800640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc415e4b0 0x7f7fc40ae020 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0b0040 tx=0x7f7fbc0195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3121598886 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0bd020 con 0x7f7fc415e4b0 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3121598886 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc0b8690 con 0x7f7fc415e4b0 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3121598886 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0b2040 con 0x7f7fc415e4b0 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc4156670 con 0x7f7fc415e4b0 2024-03-22T04:13:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc4156c30 con 0x7f7fc415e4b0 2024-03-22T04:13:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc404f900 con 0x7f7fc415e4b0 2024-03-22T04:13:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3121598886 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7fbc0c0020 con 0x7f7fc415e4b0 2024-03-22T04:13:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fa0067b20 0x7f7fa0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.652+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3121598886 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f7fbc120ec0 con 0x7f7fc415e4b0 2024-03-22T04:13:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.656+0000 7f7fc1e00640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fa0067b20 0x7f7fa0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.656+0000 7f7fc1e00640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fa0067b20 0x7f7fa0069fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4000f70 tx=0x7f7fb4007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:30.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.656+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3121598886 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7fbc0c7030 con 0x7f7fc415e4b0 2024-03-22T04:13:30.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.812+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7fc40cd430 con 0x7f7fc415e4b0 2024-03-22T04:13:30.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3121598886 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f7fbc016520 con 0x7f7fc415e4b0 2024-03-22T04:13:30.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fa0067b20 msgr2=0x7f7fa0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:30.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fa0067b20 0x7f7fa0069fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4000f70 tx=0x7f7fb4007c10 comp rx=0 tx=0).stop 2024-03-22T04:13:30.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc415e4b0 msgr2=0x7f7fc40ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:30.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc415e4b0 0x7f7fc40ae020 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0b0040 tx=0x7f7fbc0195b0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 shutdown_connections 2024-03-22T04:13:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fa0067b20 0x7f7fa0069fe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fc415e4b0 0x7f7fc40ae020 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7fc40d05f0 0x7f7fc4161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 --2- 172.21.15.86:0/3121598886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fc406bd70 0x7f7fc4155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.816+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 >> 172.21.15.86:0/3121598886 conn(0x7f7fc40d0ec0 msgr2=0x7f7fc40b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.820+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 shutdown_connections 2024-03-22T04:13:30.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:30.820+0000 7f7fc3200640 1 -- 172.21.15.86:0/3121598886 wait complete. 2024-03-22T04:13:30.842 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:13:30.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:13:30.936 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:13:30.985 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:13:30.985 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:13:30.985 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:30.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:13:30.994 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:13:30.994 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:31.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:13:31.004 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:13:31.004 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:31.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:13:31.046 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:13:31.046 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:31.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:13:31.057 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-22T04:13:31.057 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe402000640 1 Processor -- start 2024-03-22T04:13:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe402000640 1 -- start start 2024-03-22T04:13:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe402000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3fc15b3f0 0x7fe3fc0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe402000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc0ae540 0x7fe3fc072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe402000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3fc071920 con 0x7fe3fc0cc3e0 2024-03-22T04:13:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe402000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe3fc0716d0 con 0x7fe3fc15b3f0 2024-03-22T04:13:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe402000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc06d990 con 0x7fe3fc0ae540 2024-03-22T04:13:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe3fb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3fc15b3f0 0x7fe3fc0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe3fb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3fc15b3f0 0x7fe3fc0729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45132/0 (socket says 172.21.15.86:45132) 2024-03-22T04:13:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe3fb400640 1 -- 172.21.15.86:0/2649559195 learned_addr learned my addr 172.21.15.86:0/2649559195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe3fbe00640 1 --2- 172.21.15.86:0/2649559195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc0ae540 0x7fe3fc072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe3fb400640 1 -- 172.21.15.86:0/2649559195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc0ae540 msgr2=0x7fe3fc072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe3fb400640 1 --2- 172.21.15.86:0/2649559195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc0ae540 0x7fe3fc072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe3fb400640 1 -- 172.21.15.86:0/2649559195 >> v1:172.21.15.196:6789/0 conn(0x7fe3fc0cc3e0 legacy=0x7fe3fc0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.220+0000 7fe3fb400640 1 -- 172.21.15.86:0/2649559195 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc071c00 con 0x7fe3fc15b3f0 2024-03-22T04:13:31.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe3fb400640 1 --2- 172.21.15.86:0/2649559195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc15b3f0 0x7fe3fc0729c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa30affe1c2ed72 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe3faa00640 1 -- 172.21.15.86:0/2649559195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0b0070 con 0x7fe3fc15b3f0 2024-03-22T04:13:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe3faa00640 1 -- 172.21.15.86:0/2649559195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ec0ab600 con 0x7fe3fc15b3f0 2024-03-22T04:13:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe3faa00640 1 -- 172.21.15.86:0/2649559195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0ab9d0 con 0x7fe3fc15b3f0 2024-03-22T04:13:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe402000640 1 -- 172.21.15.86:0/2649559195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc15b3f0 msgr2=0x7fe3fc0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe402000640 1 --2- 172.21.15.86:0/2649559195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc15b3f0 0x7fe3fc0729c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec01c490 comp rx=0 tx=0).stop 2024-03-22T04:13:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe402000640 1 -- 172.21.15.86:0/2649559195 shutdown_connections 2024-03-22T04:13:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe402000640 1 --2- 172.21.15.86:0/2649559195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc0ae540 0x7fe3fc072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe402000640 1 --2- 172.21.15.86:0/2649559195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc15b3f0 0x7fe3fc0729c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe402000640 1 -- 172.21.15.86:0/2649559195 >> 172.21.15.86:0/2649559195 conn(0x7fe3fc0b1df0 msgr2=0x7fe3fc0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe402000640 1 -- 172.21.15.86:0/2649559195 shutdown_connections 2024-03-22T04:13:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.224+0000 7fe402000640 1 -- 172.21.15.86:0/2649559195 wait complete. 2024-03-22T04:13:31.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 Processor -- start 2024-03-22T04:13:31.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 -- start start 2024-03-22T04:13:31.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc0ae540 0x7fe3fc14f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3fc0cc3e0 0x7fe3fc0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc15b3f0 0x7fe3fc0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc166570 con 0x7fe3fc0ae540 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc1663f0 con 0x7fe3fc0cc3e0 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc1666f0 con 0x7fe3fc15b3f0 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc0ae540 0x7fe3fc14f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc0ae540 0x7fe3fc14f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45142/0 (socket says 172.21.15.86:45142) 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 -- 172.21.15.86:0/4178942857 learned_addr learned my addr 172.21.15.86:0/4178942857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fbe00640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc15b3f0 0x7fe3fc0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3f3e00640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3fc0cc3e0 0x7fe3fc0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 -- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc15b3f0 msgr2=0x7fe3fc0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc15b3f0 0x7fe3fc0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 -- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3fc0cc3e0 msgr2=0x7fe3fc0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3fc0cc3e0 0x7fe3fc0b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 -- 172.21.15.86:0/4178942857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc15e7d0 con 0x7fe3fc0ae540 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fbe00640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc15b3f0 0x7fe3fc0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3fb400640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc0ae540 0x7fe3fc14f320 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec01cd30 tx=0x7fe3ec019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3f3400640 1 -- 172.21.15.86:0/4178942857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0b0040 con 0x7fe3fc0ae540 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3f3400640 1 -- 172.21.15.86:0/4178942857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ec0b6660 con 0x7fe3fc0ae540 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe3f3400640 1 -- 172.21.15.86:0/4178942857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0b6a50 con 0x7fe3fc0ae540 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3fc15e210 con 0x7fe3fc0ae540 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.228+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3fc0ca050 con 0x7fe3fc0ae540 2024-03-22T04:13:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.232+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3fc068650 con 0x7fe3fc0ae540 2024-03-22T04:13:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.232+0000 7fe3f3400640 1 -- 172.21.15.86:0/4178942857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe3ec0bd060 con 0x7fe3fc0ae540 2024-03-22T04:13:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.232+0000 7fe3f3400640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe3d8067b50 0x7fe3d806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.232+0000 7fe3f3400640 1 -- 172.21.15.86:0/4178942857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7fe3ec11fc10 con 0x7fe3fc0ae540 2024-03-22T04:13:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.232+0000 7fe3f3e00640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe3d8067b50 0x7fe3d806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.232+0000 7fe3f3e00640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe3d8067b50 0x7fe3d806a010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc14fbf0 tx=0x7fe3e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.236+0000 7fe3f3400640 1 -- 172.21.15.86:0/4178942857 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe3ec016110 con 0x7fe3fc0ae540 2024-03-22T04:13:31.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.420+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3fc1527d0 con 0x7fe3fc0ae540 2024-03-22T04:13:31.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.420+0000 7fe3f3400640 1 -- 172.21.15.86:0/4178942857 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3869 (secure 0 0 0) 0x7fe3ec0f0780 con 0x7fe3fc0ae540 2024-03-22T04:13:31.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:31.425 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3991655539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3991655539},{"type":"v1","addr":"172.21.15.196:6835","nonce":3991655539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1296657518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1296657518},{"type":"v1","addr":"172.21.15.86:6835","nonce":1296657518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/312513749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":312513749},{"type":"v1","addr":"172.21.15.86:6837","nonce":312513749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":6091,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/285386945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":285386945},{"type":"v1","addr":"172.21.15.196:6834","nonce":285386945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-03-22T04:13:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-03-22T04:13:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe3d8067b50 msgr2=0x7fe3d806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe3d8067b50 0x7fe3d806a010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc14fbf0 tx=0x7fe3e4009040 comp rx=0 tx=0).stop 2024-03-22T04:13:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc0ae540 msgr2=0x7fe3fc14f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc0ae540 0x7fe3fc14f320 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec01cd30 tx=0x7fe3ec019230 comp rx=0 tx=0).stop 2024-03-22T04:13:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 shutdown_connections 2024-03-22T04:13:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe3d8067b50 0x7fe3d806a010 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3fc0cc3e0 0x7fe3fc0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3fc15b3f0 0x7fe3fc0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 --2- 172.21.15.86:0/4178942857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3fc0ae540 0x7fe3fc14f320 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 >> 172.21.15.86:0/4178942857 conn(0x7fe3fc0b1df0 msgr2=0x7fe3fc152cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 shutdown_connections 2024-03-22T04:13:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.424+0000 7fe402000640 1 -- 172.21.15.86:0/4178942857 wait complete. 2024-03-22T04:13:31.447 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-22T04:13:31.447 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-22T04:13:31.447 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-22T04:13:31.447 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:13:31.447 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-22T04:13:31.448 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-22T04:13:31.448 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-22T04:13:31.448 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:13:31.448 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-22T04:13:31.448 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-22T04:13:31.448 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-22T04:13:31.448 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:13:31.449 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-22T04:13:31.449 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-22T04:13:31.449 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-22T04:13:31.449 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:13:31.461 INFO:tasks.ceph.mds.a:Stopped 2024-03-22T04:13:31.462 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-22T04:13:31.463 INFO:tasks.ceph.mds.c:Stopped 2024-03-22T04:13:31.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-22T04:13:31.464 INFO:tasks.ceph.mds.b:Stopped 2024-03-22T04:13:31.464 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-22T04:13:31.465 INFO:tasks.ceph.mds.d:Stopped 2024-03-22T04:13:31.465 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-22T04:13:31.509 INFO:tasks.ceph.mds.a:Started 2024-03-22T04:13:31.509 DEBUG:teuthology.parallel:result is None 2024-03-22T04:13:31.511 INFO:tasks.ceph.mds.c:Started 2024-03-22T04:13:31.511 DEBUG:teuthology.parallel:result is None 2024-03-22T04:13:31.512 INFO:tasks.ceph.mds.b:Started 2024-03-22T04:13:31.512 DEBUG:teuthology.parallel:result is None 2024-03-22T04:13:31.514 INFO:tasks.ceph.mds.d:Started 2024-03-22T04:13:31.514 DEBUG:teuthology.parallel:result is None 2024-03-22T04:13:31.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-22T04:13:31.596 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-03-22T04:13:31.596 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:13:31.599 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:13:31.599 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:13:31.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.652+0000 7f40a2e00640 1 Processor -- start 2024-03-22T04:13:31.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.652+0000 7f40a2e00640 1 -- start start 2024-03-22T04:13:31.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f40a2e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f409c06e2f0 0x7f409c071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f40a2e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409c1623d0 con 0x7f409c15e630 2024-03-22T04:13:31.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f40a2e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409c1620f0 con 0x7f409c0d0580 2024-03-22T04:13:31.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f40a2e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f409c15ec10 con 0x7f409c06e2f0 2024-03-22T04:13:31.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f409be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f409c15e630 0x7f409c0cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53260/0 (socket says 172.21.15.86:53260) 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f409be00640 1 -- 172.21.15.86:0/2148186359 learned_addr learned my addr 172.21.15.86:0/2148186359 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f40a0e00640 1 --2- 172.21.15.86:0/2148186359 >> v2:172.21.15.196:3300/0 conn(0x7f409c06e2f0 0x7f409c071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f409b400640 1 -- 172.21.15.86:0/2148186359 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349748882 0 0) 0x7f409c1623d0 con 0x7f409c15e630 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f409b400640 1 -- 172.21.15.86:0/2148186359 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4080003530 con 0x7f409c15e630 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f409b400640 1 -- 172.21.15.86:0/2148186359 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651063412 0 0) 0x7f409c1620f0 con 0x7f409c0d0580 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f409b400640 1 -- 172.21.15.86:0/2148186359 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f409c1623d0 con 0x7f409c0d0580 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f40a0e00640 1 -- 172.21.15.86:0/2148186359 >> v1:172.21.15.196:6789/0 conn(0x7f409c0d0580 legacy=0x7f409c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f40a0e00640 1 -- 172.21.15.86:0/2148186359 >> v1:172.21.15.86:6789/0 conn(0x7f409c15e630 legacy=0x7f409c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f409b400640 1 -- 172.21.15.86:0/2148186359 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2956026244 0 0) 0x7f4080003530 con 0x7f409c15e630 2024-03-22T04:13:31.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.656+0000 7f40a0e00640 1 -- 172.21.15.86:0/2148186359 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c15eef0 con 0x7f409c06e2f0 2024-03-22T04:13:31.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a0e00640 1 --2- 172.21.15.86:0/2148186359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c06e2f0 0x7f409c071810 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4094009d90 tx=0x7f409401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8763a85ca1f856a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:31.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f409b400640 1 -- 172.21.15.86:0/2148186359 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c001680 con 0x7f409c06e2f0 2024-03-22T04:13:31.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f409b400640 1 -- 172.21.15.86:0/2148186359 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40940a64e0 con 0x7f409c06e2f0 2024-03-22T04:13:31.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f409b400640 1 -- 172.21.15.86:0/2148186359 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40940a68b0 con 0x7f409c06e2f0 2024-03-22T04:13:31.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- 172.21.15.86:0/2148186359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c06e2f0 msgr2=0x7f409c071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 --2- 172.21.15.86:0/2148186359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c06e2f0 0x7f409c071810 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4094009d90 tx=0x7f409401c890 comp rx=0 tx=0).stop 2024-03-22T04:13:31.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- 172.21.15.86:0/2148186359 shutdown_connections 2024-03-22T04:13:31.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 --2- 172.21.15.86:0/2148186359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c06e2f0 0x7f409c071810 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- 172.21.15.86:0/2148186359 >> 172.21.15.86:0/2148186359 conn(0x7f409c06f910 msgr2=0x7f409c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:31.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- 172.21.15.86:0/2148186359 shutdown_connections 2024-03-22T04:13:31.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- 172.21.15.86:0/2148186359 wait complete. 2024-03-22T04:13:31.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 Processor -- start 2024-03-22T04:13:31.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- start start 2024-03-22T04:13:31.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f409c06e2f0 0x7f409c0cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c0d0580 0x7f409c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f409c15e630 0x7f409c156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f409c0cfaf0 con 0x7f409c15e630 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f409c15f9d0 con 0x7f409c0d0580 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a2e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f409c0cfc70 con 0x7f409c06e2f0 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f409c15e630 0x7f409c156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.660+0000 7f40a0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f409c15e630 0x7f409c156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45182/0 (socket says 172.21.15.86:45182) 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a0e00640 1 -- 172.21.15.86:0/3469285119 learned_addr learned my addr 172.21.15.86:0/3469285119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f4093400640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c0d0580 0x7f409c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f409be00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f409c06e2f0 0x7f409c0cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a0e00640 1 -- 172.21.15.86:0/3469285119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f409c06e2f0 msgr2=0x7f409c0cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a0e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f409c06e2f0 0x7f409c0cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a0e00640 1 -- 172.21.15.86:0/3469285119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c0d0580 msgr2=0x7f409c0cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a0e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c0d0580 0x7f409c0cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a0e00640 1 -- 172.21.15.86:0/3469285119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c0c96e0 con 0x7f409c15e630 2024-03-22T04:13:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a0e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f409c15e630 0x7f409c156060 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f40940b0040 tx=0x7f4094019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f4098c00640 1 -- 172.21.15.86:0/3469285119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094009450 con 0x7f409c15e630 2024-03-22T04:13:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f4098c00640 1 -- 172.21.15.86:0/3469285119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4094003bc0 con 0x7f409c15e630 2024-03-22T04:13:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c0c99c0 con 0x7f409c15e630 2024-03-22T04:13:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f4098c00640 1 -- 172.21.15.86:0/3469285119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094004020 con 0x7f409c15e630 2024-03-22T04:13:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c0c9f80 con 0x7f409c15e630 2024-03-22T04:13:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c0ca1c0 con 0x7f409c15e630 2024-03-22T04:13:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f4098c00640 1 -- 172.21.15.86:0/3469285119 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f40940041e0 con 0x7f409c15e630 2024-03-22T04:13:31.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f4098c00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4078067b20 0x7f4078069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:31.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.664+0000 7f4098c00640 1 -- 172.21.15.86:0/3469285119 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f4094120640 con 0x7f409c15e630 2024-03-22T04:13:31.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.668+0000 7f409be00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4078067b20 0x7f4078069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:31.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.668+0000 7f409be00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4078067b20 0x7f4078069fe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f408c000fe0 tx=0x7f408c001160 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.672+0000 7f4098c00640 1 -- 172.21.15.86:0/3469285119 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4094016480 con 0x7f409c15e630 2024-03-22T04:13:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:31.836+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f409c068650 con 0x7f409c15e630 2024-03-22T04:13:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f4098c00640 1 -- 172.21.15.86:0/3469285119 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) ==== 97+0+0 (secure 0 0 0) 0x7f40940f0b60 con 0x7f409c15e630 2024-03-22T04:13:32.495 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4078067b20 msgr2=0x7f4078069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4078067b20 0x7f4078069fe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f408c000fe0 tx=0x7f408c001160 comp rx=0 tx=0).stop 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f409c15e630 msgr2=0x7f409c156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f409c15e630 0x7f409c156060 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f40940b0040 tx=0x7f4094019630 comp rx=0 tx=0).stop 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 shutdown_connections 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4078067b20 0x7f4078069fe0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f409c0d0580 0x7f409c0cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f409c15e630 0x7f409c156060 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 --2- 172.21.15.86:0/3469285119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f409c06e2f0 0x7f409c0cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 >> 172.21.15.86:0/3469285119 conn(0x7f409c06f910 msgr2=0x7f409c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:32.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 shutdown_connections 2024-03-22T04:13:32.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.492+0000 7f40a2e00640 1 -- 172.21.15.86:0/3469285119 wait complete. 2024-03-22T04:13:32.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-22T04:13:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a68800640 1 Processor -- start 2024-03-22T04:13:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a68800640 1 -- start start 2024-03-22T04:13:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a68800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4a6015b3f0 0x7f4a600729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a68800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a60071920 con 0x7f4a600cc3e0 2024-03-22T04:13:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a68800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a60071c00 con 0x7f4a600ae540 2024-03-22T04:13:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a68800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4a600716d0 con 0x7f4a6015b3f0 2024-03-22T04:13:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a65400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4a600cc3e0 0x7f4a600ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53262/0 (socket says 172.21.15.86:53262) 2024-03-22T04:13:32.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a65400640 1 -- 172.21.15.86:0/2282534224 learned_addr learned my addr 172.21.15.86:0/2282534224 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:32.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a65e00640 1 --2- 172.21.15.86:0/2282534224 >> v2:172.21.15.196:3300/0 conn(0x7f4a6015b3f0 0x7f4a600729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a64a00640 1 -- 172.21.15.86:0/2282534224 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818612418 0 0) 0x7f4a60071920 con 0x7f4a600cc3e0 2024-03-22T04:13:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a64a00640 1 -- 172.21.15.86:0/2282534224 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a48003530 con 0x7f4a600cc3e0 2024-03-22T04:13:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a64a00640 1 -- 172.21.15.86:0/2282534224 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259398758 0 0) 0x7f4a60071c00 con 0x7f4a600ae540 2024-03-22T04:13:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a64a00640 1 -- 172.21.15.86:0/2282534224 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a60071920 con 0x7f4a600ae540 2024-03-22T04:13:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a64a00640 1 -- 172.21.15.86:0/2282534224 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1999133091 0 0) 0x7f4a48003530 con 0x7f4a600cc3e0 2024-03-22T04:13:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a65e00640 1 -- 172.21.15.86:0/2282534224 >> v1:172.21.15.196:6789/0 conn(0x7f4a600ae540 legacy=0x7f4a600b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a65e00640 1 -- 172.21.15.86:0/2282534224 >> v1:172.21.15.86:6789/0 conn(0x7f4a600cc3e0 legacy=0x7f4a600ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.656+0000 7f4a65e00640 1 -- 172.21.15.86:0/2282534224 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a60071ee0 con 0x7f4a6015b3f0 2024-03-22T04:13:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a65e00640 1 --2- 172.21.15.86:0/2282534224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 0x7f4a600729c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4a540097c0 tx=0x7f4a5401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18a47d9b4bd549a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a64a00640 1 -- 172.21.15.86:0/2282534224 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540b0070 con 0x7f4a6015b3f0 2024-03-22T04:13:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a64a00640 1 -- 172.21.15.86:0/2282534224 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a540ab400 con 0x7f4a6015b3f0 2024-03-22T04:13:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a64a00640 1 -- 172.21.15.86:0/2282534224 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50001680 con 0x7f4a6015b3f0 2024-03-22T04:13:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a68800640 1 -- 172.21.15.86:0/2282534224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 msgr2=0x7f4a600729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a68800640 1 --2- 172.21.15.86:0/2282534224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 0x7f4a600729c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4a540097c0 tx=0x7f4a5401c3c0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a68800640 1 -- 172.21.15.86:0/2282534224 shutdown_connections 2024-03-22T04:13:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a68800640 1 --2- 172.21.15.86:0/2282534224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 0x7f4a600729c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a68800640 1 -- 172.21.15.86:0/2282534224 >> 172.21.15.86:0/2282534224 conn(0x7f4a600b1df0 msgr2=0x7f4a600b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a68800640 1 -- 172.21.15.86:0/2282534224 shutdown_connections 2024-03-22T04:13:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.660+0000 7f4a68800640 1 -- 172.21.15.86:0/2282534224 wait complete. 2024-03-22T04:13:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 Processor -- start 2024-03-22T04:13:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 -- start start 2024-03-22T04:13:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a600ae540 0x7f4a60149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a600cc3e0 0x7f4a601583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 0x7f4a60147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a600b30f0 con 0x7f4a600ae540 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a600b4c60 con 0x7f4a6015b3f0 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4a600b3270 con 0x7f4a600cc3e0 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a600ae540 0x7f4a60149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a600ae540 0x7f4a60149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45184/0 (socket says 172.21.15.86:45184) 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 -- 172.21.15.86:0/934266201 learned_addr learned my addr 172.21.15.86:0/934266201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a66800640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 0x7f4a60147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 -- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a600cc3e0 msgr2=0x7f4a601583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65400640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a600cc3e0 0x7f4a601583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a600cc3e0 0x7f4a601583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 -- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 msgr2=0x7f4a60147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 0x7f4a60147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 -- 172.21.15.86:0/934266201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6015a9b0 con 0x7f4a600ae540 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a66800640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 0x7f4a60147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65400640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a600cc3e0 0x7f4a601583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a65e00640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a600ae540 0x7f4a60149be0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4a540ae040 tx=0x7f4a5401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a5a000640 1 -- 172.21.15.86:0/934266201 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540bb020 con 0x7f4a600ae540 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a5a000640 1 -- 172.21.15.86:0/934266201 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a540b1040 con 0x7f4a600ae540 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a5a000640 1 -- 172.21.15.86:0/934266201 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540b0050 con 0x7f4a600ae540 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6014d7b0 con 0x7f4a600ae540 2024-03-22T04:13:32.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6014dd70 con 0x7f4a600ae540 2024-03-22T04:13:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.664+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a60068650 con 0x7f4a600ae540 2024-03-22T04:13:32.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.668+0000 7f4a5a000640 1 -- 172.21.15.86:0/934266201 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4a540be070 con 0x7f4a600ae540 2024-03-22T04:13:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.668+0000 7f4a5a000640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.668+0000 7f4a5a000640 1 -- 172.21.15.86:0/934266201 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f4a5411f4d0 con 0x7f4a600ae540 2024-03-22T04:13:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.668+0000 7f4a65400640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.668+0000 7f4a65400640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4a60167160 tx=0x7f4a50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:32.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.672+0000 7f4a5a000640 1 -- 172.21.15.86:0/934266201 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4a54015fb0 con 0x7f4a600ae540 2024-03-22T04:13:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4a600b0ea0 con 0x7f4a600ae540 2024-03-22T04:13:32.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a5a000640 1 -- 172.21.15.86:0/934266201 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f4a540f00a0 con 0x7f4a600ae540 2024-03-22T04:13:32.883 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:32.883 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:13:32.883 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-03-22T04:13:32.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.883 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-03-22T04:13:32.883 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA==", 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw==", 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:13:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q==", 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.885 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg==", 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-03-22T04:13:32.886 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA==", 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA==", 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg==", 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA==", 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.888 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ==", 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.889 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow==", 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg==", 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.890 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg==", 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA==", 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg==", 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:13:32.892 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w==", 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w==", 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB4BP1lvLoQORAADwVEr78V9aUc1YblO47Gaw==", 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g==", 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:32.895 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ==", 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T04:13:32.896 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:13:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 msgr2=0x7f4a44069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4a60167160 tx=0x7f4a50009040 comp rx=0 tx=0).stop 2024-03-22T04:13:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a600ae540 msgr2=0x7f4a60149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a600ae540 0x7f4a60149be0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4a540ae040 tx=0x7f4a5401ac00 comp rx=0 tx=0).stop 2024-03-22T04:13:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 shutdown_connections 2024-03-22T04:13:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a6015b3f0 0x7f4a60147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a600cc3e0 0x7f4a601583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 --2- 172.21.15.86:0/934266201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a600ae540 0x7f4a60149be0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 >> 172.21.15.86:0/934266201 conn(0x7f4a600b1df0 msgr2=0x7f4a600b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 shutdown_connections 2024-03-22T04:13:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:32.880+0000 7f4a68800640 1 -- 172.21.15.86:0/934266201 wait complete. 2024-03-22T04:13:32.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:33.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe65e800640 1 Processor -- start 2024-03-22T04:13:33.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe65e800640 1 -- start start 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe65e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe65815e630 0x7fe6580cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe65e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6580d03b0 0x7fe65806d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe65e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6580d07d0 con 0x7fe65806e2f0 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe65e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe6581623d0 con 0x7fe65815e630 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe65e800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe6581620f0 con 0x7fe6580d03b0 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe657400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe65815e630 0x7fe6580cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe64ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe65806e2f0 0x7fe65806cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53266/0 (socket says 172.21.15.86:53266) 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe64ea00640 1 -- 172.21.15.86:0/3914095300 learned_addr learned my addr 172.21.15.86:0/3914095300 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:33.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe657e00640 1 --2- 172.21.15.86:0/3914095300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6580d03b0 0x7fe65806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe657400640 1 -- 172.21.15.86:0/3914095300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6580d03b0 msgr2=0x7fe65806d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:33.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe657400640 1 --2- 172.21.15.86:0/3914095300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6580d03b0 0x7fe65806d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe657400640 1 -- 172.21.15.86:0/3914095300 >> v1:172.21.15.86:6789/0 conn(0x7fe65806e2f0 legacy=0x7fe65806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe657400640 1 -- 172.21.15.86:0/3914095300 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65815ec10 con 0x7fe65815e630 2024-03-22T04:13:33.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe656a00640 1 -- 172.21.15.86:0/3914095300 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3929900656 0 0) 0x7fe6580d07d0 con 0x7fe65806e2f0 2024-03-22T04:13:33.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.036+0000 7fe657e00640 1 --2- 172.21.15.86:0/3914095300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6580d03b0 0x7fe65806d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:33.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe657400640 1 --2- 172.21.15.86:0/3914095300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65815e630 0x7fe6580cfe70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe648009d90 tx=0x7fe64801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5befb92a41a200 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe656a00640 1 -- 172.21.15.86:0/3914095300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480b2070 con 0x7fe65815e630 2024-03-22T04:13:33.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe656a00640 1 -- 172.21.15.86:0/3914095300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6480a5830 con 0x7fe65815e630 2024-03-22T04:13:33.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe656a00640 1 -- 172.21.15.86:0/3914095300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480a5c00 con 0x7fe65815e630 2024-03-22T04:13:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe65e800640 1 -- 172.21.15.86:0/3914095300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65815e630 msgr2=0x7fe6580cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe65e800640 1 --2- 172.21.15.86:0/3914095300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65815e630 0x7fe6580cfe70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe648009d90 tx=0x7fe64801c970 comp rx=0 tx=0).stop 2024-03-22T04:13:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe65e800640 1 -- 172.21.15.86:0/3914095300 shutdown_connections 2024-03-22T04:13:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe65e800640 1 --2- 172.21.15.86:0/3914095300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6580d03b0 0x7fe65806d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe65e800640 1 --2- 172.21.15.86:0/3914095300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe65815e630 0x7fe6580cfe70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe65e800640 1 -- 172.21.15.86:0/3914095300 >> 172.21.15.86:0/3914095300 conn(0x7fe65806f910 msgr2=0x7fe65806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:33.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe65e800640 1 -- 172.21.15.86:0/3914095300 shutdown_connections 2024-03-22T04:13:33.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.040+0000 7fe65e800640 1 -- 172.21.15.86:0/3914095300 wait complete. 2024-03-22T04:13:33.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 Processor -- start 2024-03-22T04:13:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 -- start start 2024-03-22T04:13:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65806e2f0 0x7fe6580b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6580d03b0 0x7fe6580b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65815e630 0x7fe658158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe65815f9d0 con 0x7fe6580d03b0 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6580d1f60 con 0x7fe65806e2f0 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe6580cfaf0 con 0x7fe65815e630 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6580d03b0 0x7fe6580b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6580d03b0 0x7fe6580b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45204/0 (socket says 172.21.15.86:45204) 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 -- 172.21.15.86:0/507115282 learned_addr learned my addr 172.21.15.86:0/507115282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe657400640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65806e2f0 0x7fe6580b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe657e00640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65815e630 0x7fe658158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 -- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65815e630 msgr2=0x7fe658158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65815e630 0x7fe658158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 -- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65806e2f0 msgr2=0x7fe6580b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65806e2f0 0x7fe6580b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 -- 172.21.15.86:0/507115282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6580c82b0 con 0x7fe6580d03b0 2024-03-22T04:13:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe657e00640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65815e630 0x7fe658158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:13:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64ea00640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6580d03b0 0x7fe6580b5970 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c480 tx=0x7fe64000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64fe00640 1 -- 172.21.15.86:0/507115282 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640018070 con 0x7fe6580d03b0 2024-03-22T04:13:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64fe00640 1 -- 172.21.15.86:0/507115282 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe640007500 con 0x7fe6580d03b0 2024-03-22T04:13:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe64fe00640 1 -- 172.21.15.86:0/507115282 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640004a30 con 0x7fe6580d03b0 2024-03-22T04:13:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6580b63b0 con 0x7fe6580d03b0 2024-03-22T04:13:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6580b5eb0 con 0x7fe6580d03b0 2024-03-22T04:13:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.044+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe658068650 con 0x7fe6580d03b0 2024-03-22T04:13:33.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.048+0000 7fe64fe00640 1 -- 172.21.15.86:0/507115282 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe640004bd0 con 0x7fe6580d03b0 2024-03-22T04:13:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.048+0000 7fe64fe00640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe634067b20 0x7fe634069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.048+0000 7fe64fe00640 1 -- 172.21.15.86:0/507115282 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7fe640084ff0 con 0x7fe6580d03b0 2024-03-22T04:13:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.048+0000 7fe657400640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe634067b20 0x7fe634069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.048+0000 7fe657400640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe634067b20 0x7fe634069fe0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe6480af000 tx=0x7fe6480a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.052+0000 7fe64fe00640 1 -- 172.21.15.86:0/507115282 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe640055c10 con 0x7fe6580d03b0 2024-03-22T04:13:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.208+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6580b3bc0 con 0x7fe6580d03b0 2024-03-22T04:13:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.208+0000 7fe64fe00640 1 -- 172.21.15.86:0/507115282 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+13164 (secure 0 0 0) 0x7fe640055510 con 0x7fe6580d03b0 2024-03-22T04:13:33.214 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:33.215 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":63,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:32.508141+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:33.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe634067b20 msgr2=0x7fe634069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe634067b20 0x7fe634069fe0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe6480af000 tx=0x7fe6480a7070 comp rx=0 tx=0).stop 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6580d03b0 msgr2=0x7fe6580b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6580d03b0 0x7fe6580b5970 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c480 tx=0x7fe64000c940 comp rx=0 tx=0).stop 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 shutdown_connections 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe634067b20 0x7fe634069fe0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65815e630 0x7fe658158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6580d03b0 0x7fe6580b5970 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 --2- 172.21.15.86:0/507115282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe65806e2f0 0x7fe6580b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 >> 172.21.15.86:0/507115282 conn(0x7fe65806f910 msgr2=0x7fe6580b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 shutdown_connections 2024-03-22T04:13:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.212+0000 7fe65e800640 1 -- 172.21.15.86:0/507115282 wait complete. 2024-03-22T04:13:33.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:13:33.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:13:33.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb892a00640 1 Processor -- start 2024-03-22T04:13:33.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb892a00640 1 -- start start 2024-03-22T04:13:33.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb892a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb88c070090 0x7fb88c162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb892a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c162860 0x7fb88c06cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb892a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb88c071180 con 0x7fb88c147ed0 2024-03-22T04:13:33.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb892a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb88c070f30 con 0x7fb88c070090 2024-03-22T04:13:33.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb892a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb88c071460 con 0x7fb88c162860 2024-03-22T04:13:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb890a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c162860 0x7fb88c06cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb890a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c162860 0x7fb88c06cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:34124/0 (socket says 172.21.15.86:34124) 2024-03-22T04:13:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb890a00640 1 -- 172.21.15.86:0/3315342068 learned_addr learned my addr 172.21.15.86:0/3315342068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb88b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb88c070090 0x7fb88c162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb890a00640 1 -- 172.21.15.86:0/3315342068 >> v2:172.21.15.196:3300/0 conn(0x7fb88c070090 msgr2=0x7fb88c162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb890a00640 1 --2- 172.21.15.86:0/3315342068 >> v2:172.21.15.196:3300/0 conn(0x7fb88c070090 0x7fb88c162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb890a00640 1 -- 172.21.15.86:0/3315342068 >> v1:172.21.15.196:6789/0 conn(0x7fb88c147ed0 legacy=0x7fb88c06f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.372+0000 7fb890a00640 1 -- 172.21.15.86:0/3315342068 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c071740 con 0x7fb88c162860 2024-03-22T04:13:33.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb890a00640 1 --2- 172.21.15.86:0/3315342068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c162860 0x7fb88c06cc80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb884009e50 tx=0x7fb88401c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41a563e51bee5921 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb88aa00640 1 -- 172.21.15.86:0/3315342068 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8840a5770 con 0x7fb88c162860 2024-03-22T04:13:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb88aa00640 1 -- 172.21.15.86:0/3315342068 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8840a5910 con 0x7fb88c162860 2024-03-22T04:13:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb88aa00640 1 -- 172.21.15.86:0/3315342068 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8840a5ce0 con 0x7fb88c162860 2024-03-22T04:13:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb892a00640 1 -- 172.21.15.86:0/3315342068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c162860 msgr2=0x7fb88c06cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb892a00640 1 --2- 172.21.15.86:0/3315342068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c162860 0x7fb88c06cc80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb884009e50 tx=0x7fb88401c9a0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb892a00640 1 -- 172.21.15.86:0/3315342068 shutdown_connections 2024-03-22T04:13:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb892a00640 1 --2- 172.21.15.86:0/3315342068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c162860 0x7fb88c06cc80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb892a00640 1 --2- 172.21.15.86:0/3315342068 >> v2:172.21.15.196:3300/0 conn(0x7fb88c070090 0x7fb88c162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb892a00640 1 -- 172.21.15.86:0/3315342068 >> 172.21.15.86:0/3315342068 conn(0x7fb88c0b3cf0 msgr2=0x7fb88c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb892a00640 1 -- 172.21.15.86:0/3315342068 shutdown_connections 2024-03-22T04:13:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.376+0000 7fb892a00640 1 -- 172.21.15.86:0/3315342068 wait complete. 2024-03-22T04:13:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 Processor -- start 2024-03-22T04:13:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 -- start start 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88c070090 0x7fb88c1f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c147ed0 0x7fb88c140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c162860 0x7fb88c15d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c06dd60 con 0x7fb88c162860 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c06dbe0 con 0x7fb88c070090 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb88c06dee0 con 0x7fb88c147ed0 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c162860 0x7fb88c15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c162860 0x7fb88c15d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45220/0 (socket says 172.21.15.86:45220) 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 -- 172.21.15.86:0/2504458169 learned_addr learned my addr 172.21.15.86:0/2504458169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb88be00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88c070090 0x7fb88c1f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb88b400640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c147ed0 0x7fb88c140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 -- 172.21.15.86:0/2504458169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c147ed0 msgr2=0x7fb88c140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c147ed0 0x7fb88c140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 -- 172.21.15.86:0/2504458169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88c070090 msgr2=0x7fb88c1f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88c070090 0x7fb88c1f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 -- 172.21.15.86:0/2504458169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c0b6b90 con 0x7fb88c162860 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb890a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c162860 0x7fb88c15d510 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb88401a5d0 tx=0x7fb884019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb883e00640 1 -- 172.21.15.86:0/2504458169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8840bc070 con 0x7fb88c162860 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb883e00640 1 -- 172.21.15.86:0/2504458169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8840b8660 con 0x7fb88c162860 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb883e00640 1 -- 172.21.15.86:0/2504458169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8840b8a50 con 0x7fb88c162860 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88c140960 con 0x7fb88c162860 2024-03-22T04:13:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.380+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88c0cc840 con 0x7fb88c162860 2024-03-22T04:13:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.384+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88c074470 con 0x7fb88c162860 2024-03-22T04:13:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.384+0000 7fb883e00640 1 -- 172.21.15.86:0/2504458169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb8840b8bf0 con 0x7fb88c162860 2024-03-22T04:13:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.384+0000 7fb883e00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb86c067b20 0x7fb86c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.384+0000 7fb883e00640 1 -- 172.21.15.86:0/2504458169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7fb884120920 con 0x7fb88c162860 2024-03-22T04:13:33.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.384+0000 7fb88be00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb86c067b20 0x7fb86c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.384+0000 7fb88be00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb86c067b20 0x7fb86c069fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb88c1691e0 tx=0x7fb87c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.388+0000 7fb883e00640 1 -- 172.21.15.86:0/2504458169 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb884016620 con 0x7fb88c162860 2024-03-22T04:13:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.544+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb88c0706d0 con 0x7fb88c162860 2024-03-22T04:13:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb883e00640 1 -- 172.21.15.86:0/2504458169 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7fb8840f0e10 con 0x7fb88c162860 2024-03-22T04:13:33.551 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:13:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb86c067b20 msgr2=0x7fb86c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb86c067b20 0x7fb86c069fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb88c1691e0 tx=0x7fb87c009040 comp rx=0 tx=0).stop 2024-03-22T04:13:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c162860 msgr2=0x7fb88c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c162860 0x7fb88c15d510 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb88401a5d0 tx=0x7fb884019740 comp rx=0 tx=0).stop 2024-03-22T04:13:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 shutdown_connections 2024-03-22T04:13:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb86c067b20 0x7fb86c069fe0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb88c162860 0x7fb88c15d510 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb88c147ed0 0x7fb88c140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 --2- 172.21.15.86:0/2504458169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb88c070090 0x7fb88c1f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 >> 172.21.15.86:0/2504458169 conn(0x7fb88c0b3cf0 msgr2=0x7fb88c06f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 shutdown_connections 2024-03-22T04:13:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.548+0000 7fb892a00640 1 -- 172.21.15.86:0/2504458169 wait complete. 2024-03-22T04:13:33.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T04:13:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.700+0000 7f9b6f000640 1 Processor -- start 2024-03-22T04:13:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.700+0000 7f9b6f000640 1 -- start start 2024-03-22T04:13:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6f000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680730d0 0x7f9b680ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b680cc3e0 0x7f9b680ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b68071920 con 0x7f9b6815b3f0 2024-03-22T04:13:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9b680716d0 con 0x7f9b680cc3e0 2024-03-22T04:13:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6f000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6806d990 con 0x7f9b680730d0 2024-03-22T04:13:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b67e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9b6815b3f0 0x7f9b680729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53276/0 (socket says 172.21.15.86:53276) 2024-03-22T04:13:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b67e00640 1 -- 172.21.15.86:0/2860004904 learned_addr learned my addr 172.21.15.86:0/2860004904 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6d000640 1 --2- 172.21.15.86:0/2860004904 >> v2:172.21.15.86:3300/0 conn(0x7f9b680cc3e0 0x7f9b680ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b66a00640 1 -- 172.21.15.86:0/2860004904 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554863623 0 0) 0x7f9b68071920 con 0x7f9b6815b3f0 2024-03-22T04:13:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b66a00640 1 -- 172.21.15.86:0/2860004904 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b4c003530 con 0x7f9b6815b3f0 2024-03-22T04:13:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b67400640 1 --2- 172.21.15.86:0/2860004904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680730d0 0x7f9b680ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6d000640 1 -- 172.21.15.86:0/2860004904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680730d0 msgr2=0x7f9b680ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6d000640 1 --2- 172.21.15.86:0/2860004904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680730d0 0x7f9b680ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6d000640 1 -- 172.21.15.86:0/2860004904 >> v1:172.21.15.86:6789/0 conn(0x7f9b6815b3f0 legacy=0x7f9b680729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b6d000640 1 -- 172.21.15.86:0/2860004904 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b68071c00 con 0x7f9b680cc3e0 2024-03-22T04:13:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.704+0000 7f9b67400640 1 --2- 172.21.15.86:0/2860004904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680730d0 0x7f9b680ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:13:33.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6d000640 1 --2- 172.21.15.86:0/2860004904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b680cc3e0 0x7f9b680ae370 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009d90 tx=0x7f9b6001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d963d6c0edc44644 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b66a00640 1 -- 172.21.15.86:0/2860004904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b600b2070 con 0x7f9b680cc3e0 2024-03-22T04:13:33.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b66a00640 1 -- 172.21.15.86:0/2860004904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b600a5920 con 0x7f9b680cc3e0 2024-03-22T04:13:33.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b66a00640 1 -- 172.21.15.86:0/2860004904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b600a5cf0 con 0x7f9b680cc3e0 2024-03-22T04:13:33.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 -- 172.21.15.86:0/2860004904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b680cc3e0 msgr2=0x7f9b680ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 --2- 172.21.15.86:0/2860004904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b680cc3e0 0x7f9b680ae370 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009d90 tx=0x7f9b6001c970 comp rx=0 tx=0).stop 2024-03-22T04:13:33.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 -- 172.21.15.86:0/2860004904 shutdown_connections 2024-03-22T04:13:33.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 --2- 172.21.15.86:0/2860004904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b680cc3e0 0x7f9b680ae370 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 --2- 172.21.15.86:0/2860004904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680730d0 0x7f9b680ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 -- 172.21.15.86:0/2860004904 >> 172.21.15.86:0/2860004904 conn(0x7f9b680b1df0 msgr2=0x7f9b680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:33.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 -- 172.21.15.86:0/2860004904 shutdown_connections 2024-03-22T04:13:33.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 -- 172.21.15.86:0/2860004904 wait complete. 2024-03-22T04:13:33.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 Processor -- start 2024-03-22T04:13:33.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.708+0000 7f9b6f000640 1 -- start start 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b680730d0 0x7f9b68099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680cc3e0 0x7f9b680aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6815b3f0 0x7f9b680aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68166570 con 0x7f9b6815b3f0 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9b681663f0 con 0x7f9b680730d0 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9b681666f0 con 0x7f9b680cc3e0 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6815b3f0 0x7f9b680aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6815b3f0 0x7f9b680aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45246/0 (socket says 172.21.15.86:45246) 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 -- 172.21.15.86:0/1453970608 learned_addr learned my addr 172.21.15.86:0/1453970608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b67e00640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b680730d0 0x7f9b68099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b67400640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680cc3e0 0x7f9b680aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 -- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680cc3e0 msgr2=0x7f9b680aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680cc3e0 0x7f9b680aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 -- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b680730d0 msgr2=0x7f9b68099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b680730d0 0x7f9b68099a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 -- 172.21.15.86:0/1453970608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6814fc70 con 0x7f9b6815b3f0 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b67e00640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b680730d0 0x7f9b68099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6d000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6815b3f0 0x7f9b680aef90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9b600b0040 tx=0x7f9b60019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b5fe00640 1 -- 172.21.15.86:0/1453970608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b600b2050 con 0x7f9b6815b3f0 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b5fe00640 1 -- 172.21.15.86:0/1453970608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b600be070 con 0x7f9b6815b3f0 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6814ff50 con 0x7f9b6815b3f0 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b5fe00640 1 -- 172.21.15.86:0/1453970608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b600ada90 con 0x7f9b6815b3f0 2024-03-22T04:13:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b68150510 con 0x7f9b6815b3f0 2024-03-22T04:13:33.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b68068650 con 0x7f9b6815b3f0 2024-03-22T04:13:33.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.712+0000 7f9b5fe00640 1 -- 172.21.15.86:0/1453970608 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9b600adc30 con 0x7f9b6815b3f0 2024-03-22T04:13:33.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.716+0000 7f9b5fe00640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b48067b20 0x7f9b48069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:33.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.716+0000 7f9b5fe00640 1 -- 172.21.15.86:0/1453970608 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f9b60120f40 con 0x7f9b6815b3f0 2024-03-22T04:13:33.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.716+0000 7f9b67e00640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b48067b20 0x7f9b48069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:33.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.716+0000 7f9b67e00640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b48067b20 0x7f9b48069fe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005890 tx=0x7f9b58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.720+0000 7f9b5fe00640 1 -- 172.21.15.86:0/1453970608 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9b60016560 con 0x7f9b6815b3f0 2024-03-22T04:13:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9b6806f7c0 con 0x7f9b6815b3f0 2024-03-22T04:13:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b5fe00640 1 -- 172.21.15.86:0/1453970608 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7f9b600ab340 con 0x7f9b6815b3f0 2024-03-22T04:13:33.884 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:33.884 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:13:33.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:33.884 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T04:13:33.884 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T04:13:33.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:33.884 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:13:33.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b48067b20 msgr2=0x7f9b48069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b48067b20 0x7f9b48069fe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005890 tx=0x7f9b58009040 comp rx=0 tx=0).stop 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6815b3f0 msgr2=0x7f9b680aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6815b3f0 0x7f9b680aef90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9b600b0040 tx=0x7f9b60019710 comp rx=0 tx=0).stop 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 shutdown_connections 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9b48067b20 0x7f9b48069fe0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6815b3f0 0x7f9b680aef90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b680cc3e0 0x7f9b680aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 --2- 172.21.15.86:0/1453970608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b680730d0 0x7f9b68099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 >> 172.21.15.86:0/1453970608 conn(0x7f9b680b1df0 msgr2=0x7f9b680b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:33.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 shutdown_connections 2024-03-22T04:13:33.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:33.880+0000 7f9b6f000640 1 -- 172.21.15.86:0/1453970608 wait complete. 2024-03-22T04:13:33.904 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T04:13:33.904 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T04:13:34.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0e1600640 1 Processor -- start 2024-03-22T04:13:34.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0e1600640 1 -- start start 2024-03-22T04:13:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0e1600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa0dc0cfa70 0x7fa0dc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0e1600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0dc06dca0 0x7fa0dc13f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0e1600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0dc0cecb0 con 0x7fa0dc0b1cd0 2024-03-22T04:13:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0e1600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa0dc0cea80 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0e1600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa0dc06e4d0 con 0x7fa0dc06dca0 2024-03-22T04:13:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0da000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa0dc0cfa70 0x7fa0dc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0daa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa0dc0b1cd0 0x7fa0dc0b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53280/0 (socket says 172.21.15.86:53280) 2024-03-22T04:13:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.040+0000 7fa0daa00640 1 -- 172.21.15.86:0/2395752455 learned_addr learned my addr 172.21.15.86:0/2395752455 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0db400640 1 --2- 172.21.15.86:0/2395752455 >> v2:172.21.15.196:3300/0 conn(0x7fa0dc06dca0 0x7fa0dc13f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0da000640 1 -- 172.21.15.86:0/2395752455 >> v2:172.21.15.196:3300/0 conn(0x7fa0dc06dca0 msgr2=0x7fa0dc13f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0da000640 1 --2- 172.21.15.86:0/2395752455 >> v2:172.21.15.196:3300/0 conn(0x7fa0dc06dca0 0x7fa0dc13f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0da000640 1 -- 172.21.15.86:0/2395752455 >> v1:172.21.15.86:6789/0 conn(0x7fa0dc0b1cd0 legacy=0x7fa0dc0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0d9600640 1 -- 172.21.15.86:0/2395752455 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4195635221 0 0) 0x7fa0dc0cecb0 con 0x7fa0dc0b1cd0 2024-03-22T04:13:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0da000640 1 -- 172.21.15.86:0/2395752455 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc0cef90 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0da000640 1 --2- 172.21.15.86:0/2395752455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc06d760 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0b2d60 tx=0x7fa0c801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ff5f529f9b1950 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0d9600640 1 -- 172.21.15.86:0/2395752455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80ab650 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0d9600640 1 -- 172.21.15.86:0/2395752455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c80b0070 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.044+0000 7fa0d9600640 1 -- 172.21.15.86:0/2395752455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80aba10 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- 172.21.15.86:0/2395752455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 msgr2=0x7fa0dc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 --2- 172.21.15.86:0/2395752455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc06d760 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0b2d60 tx=0x7fa0c801c450 comp rx=0 tx=0).stop 2024-03-22T04:13:34.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- 172.21.15.86:0/2395752455 shutdown_connections 2024-03-22T04:13:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 --2- 172.21.15.86:0/2395752455 >> v2:172.21.15.196:3300/0 conn(0x7fa0dc06dca0 0x7fa0dc13f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 --2- 172.21.15.86:0/2395752455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc06d760 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- 172.21.15.86:0/2395752455 >> 172.21.15.86:0/2395752455 conn(0x7fa0dc06b660 msgr2=0x7fa0dc06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- 172.21.15.86:0/2395752455 shutdown_connections 2024-03-22T04:13:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- 172.21.15.86:0/2395752455 wait complete. 2024-03-22T04:13:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 Processor -- start 2024-03-22T04:13:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- start start 2024-03-22T04:13:34.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0dc06dca0 0x7fa0dc13a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc0b1cd0 0x7fa0dc155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc0b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc06e230 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc06e0b0 con 0x7fa0dc0b1cd0 2024-03-22T04:13:34.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0e1600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa0dc168d80 con 0x7fa0dc06dca0 2024-03-22T04:13:34.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc0b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc0b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45260/0 (socket says 172.21.15.86:45260) 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0db400640 1 -- 172.21.15.86:0/806316864 learned_addr learned my addr 172.21.15.86:0/806316864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0daa00640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0dc06dca0 0x7fa0dc13a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0da000640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc0b1cd0 0x7fa0dc155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0db400640 1 -- 172.21.15.86:0/806316864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0dc06dca0 msgr2=0x7fa0dc13a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0db400640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0dc06dca0 0x7fa0dc13a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0db400640 1 -- 172.21.15.86:0/806316864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc0b1cd0 msgr2=0x7fa0dc155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0db400640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc0b1cd0 0x7fa0dc155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.048+0000 7fa0db400640 1 -- 172.21.15.86:0/806316864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc13ae00 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0db400640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc0b15e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d800 tx=0x7fa0d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0d2a00640 1 -- 172.21.15.86:0/806316864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400b6c0 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0d2a00640 1 -- 172.21.15.86:0/806316864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d4019070 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0d2a00640 1 -- 172.21.15.86:0/806316864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400baa0 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc13b0e0 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc136a90 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0dc13b590 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0d2a00640 1 -- 172.21.15.86:0/806316864 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa0d400bc40 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0d2a00640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0bc067b20 0x7fa0bc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0d2a00640 1 -- 172.21.15.86:0/806316864 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7fa0d40860c0 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.052+0000 7fa0daa00640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0bc067b20 0x7fa0bc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.056+0000 7fa0daa00640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0bc067b20 0x7fa0bc069fe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0012f0 tx=0x7fa0cc001770 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:34.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.056+0000 7fa0d2a00640 1 -- 172.21.15.86:0/806316864 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa0d4056630 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.212+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa0dc155ff0 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.496+0000 7fa0d2a00640 1 -- 172.21.15.86:0/806316864 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7fa0d405a6d0 con 0x7fa0dc0cfa70 2024-03-22T04:13:34.502 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T04:13:34.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0bc067b20 msgr2=0x7fa0bc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0bc067b20 0x7fa0bc069fe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0012f0 tx=0x7fa0cc001770 comp rx=0 tx=0).stop 2024-03-22T04:13:34.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 msgr2=0x7fa0dc0b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc0b15e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d800 tx=0x7fa0d400dcc0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 shutdown_connections 2024-03-22T04:13:34.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0bc067b20 0x7fa0bc069fe0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0dc0b1cd0 0x7fa0dc155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc0cfa70 0x7fa0dc0b15e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 --2- 172.21.15.86:0/806316864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0dc06dca0 0x7fa0dc13a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 >> 172.21.15.86:0/806316864 conn(0x7fa0dc06b660 msgr2=0x7fa0dc136040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:34.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 shutdown_connections 2024-03-22T04:13:34.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.500+0000 7fa0e1600640 1 -- 172.21.15.86:0/806316864 wait complete. 2024-03-22T04:13:34.527 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T04:13:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.664+0000 7fae1ca00640 1 Processor -- start 2024-03-22T04:13:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.664+0000 7fae1ca00640 1 -- start start 2024-03-22T04:13:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae1ca00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fae1815b3d0 0x7fae180cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae1ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae180d03a0 0x7fae180b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae1ca00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae180b4d60 con 0x7fae180b4920 2024-03-22T04:13:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae1ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fae180cd380 con 0x7fae180d03a0 2024-03-22T04:13:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae1ca00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fae180cca50 con 0x7fae1815b3d0 2024-03-22T04:13:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae16800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae180d03a0 0x7fae180b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae16800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae180d03a0 0x7fae180b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45270/0 (socket says 172.21.15.86:45270) 2024-03-22T04:13:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae16800640 1 -- 172.21.15.86:0/3283570677 learned_addr learned my addr 172.21.15.86:0/3283570677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae15e00640 1 --2- 172.21.15.86:0/3283570677 >> v2:172.21.15.196:3300/0 conn(0x7fae1815b3d0 0x7fae180cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae16800640 1 -- 172.21.15.86:0/3283570677 >> v2:172.21.15.196:3300/0 conn(0x7fae1815b3d0 msgr2=0x7fae180cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae16800640 1 --2- 172.21.15.86:0/3283570677 >> v2:172.21.15.196:3300/0 conn(0x7fae1815b3d0 0x7fae180cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae16800640 1 -- 172.21.15.86:0/3283570677 >> v1:172.21.15.196:6789/0 conn(0x7fae180b4920 legacy=0x7fae180cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.668+0000 7fae16800640 1 -- 172.21.15.86:0/3283570677 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae180ccd30 con 0x7fae180d03a0 2024-03-22T04:13:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae16800640 1 --2- 172.21.15.86:0/3283570677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae180d03a0 0x7fae180b3080 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fae180b56c0 tx=0x7fae1001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aec5e6f7ca82110 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:34.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae15400640 1 -- 172.21.15.86:0/3283570677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae100b5070 con 0x7fae180d03a0 2024-03-22T04:13:34.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae15400640 1 -- 172.21.15.86:0/3283570677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae100a8a10 con 0x7fae180d03a0 2024-03-22T04:13:34.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae15400640 1 -- 172.21.15.86:0/3283570677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae100a8de0 con 0x7fae180d03a0 2024-03-22T04:13:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 -- 172.21.15.86:0/3283570677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae180d03a0 msgr2=0x7fae180b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3283570677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae180d03a0 0x7fae180b3080 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fae180b56c0 tx=0x7fae1001ed10 comp rx=0 tx=0).stop 2024-03-22T04:13:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 -- 172.21.15.86:0/3283570677 shutdown_connections 2024-03-22T04:13:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3283570677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae180d03a0 0x7fae180b3080 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3283570677 >> v2:172.21.15.196:3300/0 conn(0x7fae1815b3d0 0x7fae180cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 -- 172.21.15.86:0/3283570677 >> 172.21.15.86:0/3283570677 conn(0x7fae1806ef50 msgr2=0x7fae1806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 -- 172.21.15.86:0/3283570677 shutdown_connections 2024-03-22T04:13:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 -- 172.21.15.86:0/3283570677 wait complete. 2024-03-22T04:13:34.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 Processor -- start 2024-03-22T04:13:34.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.672+0000 7fae1ca00640 1 -- start start 2024-03-22T04:13:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae180b4920 0x7fae181f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae180d03a0 0x7fae180c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae1815b3d0 0x7fae180ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae180d07f0 con 0x7fae1815b3d0 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae180cd810 con 0x7fae180d03a0 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae180b35c0 con 0x7fae180b4920 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae1815b3d0 0x7fae180ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae1815b3d0 0x7fae180ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45276/0 (socket says 172.21.15.86:45276) 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 -- 172.21.15.86:0/3937492310 learned_addr learned my addr 172.21.15.86:0/3937492310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae15e00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae180b4920 0x7fae181f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae0d400640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae180d03a0 0x7fae180c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 -- 172.21.15.86:0/3937492310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae180b4920 msgr2=0x7fae181f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae180b4920 0x7fae181f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 -- 172.21.15.86:0/3937492310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae180d03a0 msgr2=0x7fae180c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae180d03a0 0x7fae180c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 -- 172.21.15.86:0/3937492310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae18149760 con 0x7fae1815b3d0 2024-03-22T04:13:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae16800640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae1815b3d0 0x7fae180ca2e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae100b3040 tx=0x7fae1001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae0ea00640 1 -- 172.21.15.86:0/3937492310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae100b5070 con 0x7fae1815b3d0 2024-03-22T04:13:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae0ea00640 1 -- 172.21.15.86:0/3937492310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae100b0ac0 con 0x7fae1815b3d0 2024-03-22T04:13:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae180cffd0 con 0x7fae1815b3d0 2024-03-22T04:13:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae0ea00640 1 -- 172.21.15.86:0/3937492310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae100bb410 con 0x7fae1815b3d0 2024-03-22T04:13:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae18148b20 con 0x7fae1815b3d0 2024-03-22T04:13:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae18074470 con 0x7fae1815b3d0 2024-03-22T04:13:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.676+0000 7fae0ea00640 1 -- 172.21.15.86:0/3937492310 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fae100c2060 con 0x7fae1815b3d0 2024-03-22T04:13:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.680+0000 7fae0ea00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fadf4067b20 0x7fadf4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.680+0000 7fae0ea00640 1 -- 172.21.15.86:0/3937492310 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5619+0+0 (secure 0 0 0) 0x7fae10123150 con 0x7fae1815b3d0 2024-03-22T04:13:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.680+0000 7fae15e00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fadf4067b20 0x7fadf4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.680+0000 7fae15e00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fadf4067b20 0x7fadf4069fe0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fae080079c0 tx=0x7fae08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:34.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.684+0000 7fae0ea00640 1 -- 172.21.15.86:0/3937492310 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fae10018870 con 0x7fae1815b3d0 2024-03-22T04:13:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:34.840+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fae181f18e0 con 0x7fae1815b3d0 2024-03-22T04:13:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.504+0000 7fae0ea00640 1 -- 172.21.15.86:0/3937492310 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7fae100f3c20 con 0x7fae1815b3d0 2024-03-22T04:13:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T04:13:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fadf4067b20 msgr2=0x7fadf4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fadf4067b20 0x7fadf4069fe0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fae080079c0 tx=0x7fae08008040 comp rx=0 tx=0).stop 2024-03-22T04:13:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae1815b3d0 msgr2=0x7fae180ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae1815b3d0 0x7fae180ca2e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae100b3040 tx=0x7fae1001bab0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 shutdown_connections 2024-03-22T04:13:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fadf4067b20 0x7fadf4069fe0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae180d03a0 0x7fae180c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae1815b3d0 0x7fae180ca2e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 --2- 172.21.15.86:0/3937492310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae180b4920 0x7fae181f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 >> 172.21.15.86:0/3937492310 conn(0x7fae1806ef50 msgr2=0x7fae1806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:35.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 shutdown_connections 2024-03-22T04:13:35.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.508+0000 7fae1ca00640 1 -- 172.21.15.86:0/3937492310 wait complete. 2024-03-22T04:13:35.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T04:13:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c7400640 1 Processor -- start 2024-03-22T04:13:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c7400640 1 -- start start 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f66c00b2360 0x7f66c00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c7400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c7400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c015d710 con 0x7f66c00b3430 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c7400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f66c00b2740 con 0x7f66c00b2360 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c7400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f66c015d4b0 con 0x7f66c015d0b0 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c4a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f66c00b2360 0x7f66c00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c4a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f66c00b2360 0x7f66c00b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45292/0 (socket says 172.21.15.86:45292) 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c4a00640 1 -- 172.21.15.86:0/1389890976 learned_addr learned my addr 172.21.15.86:0/1389890976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c5400640 1 --2- 172.21.15.86:0/1389890976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c4a00640 1 -- 172.21.15.86:0/1389890976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 msgr2=0x7f66c00b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c4a00640 1 --2- 172.21.15.86:0/1389890976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c4a00640 1 -- 172.21.15.86:0/1389890976 >> v1:172.21.15.196:6789/0 conn(0x7f66c00b3430 legacy=0x7f66c015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c4a00640 1 -- 172.21.15.86:0/1389890976 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c015fe40 con 0x7f66c00b2360 2024-03-22T04:13:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.688+0000 7f66c5400640 1 --2- 172.21.15.86:0/1389890976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:13:35.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c4a00640 1 --2- 172.21.15.86:0/1389890976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b2360 0x7f66c00b2ef0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f66c00683e0 tx=0x7f66b001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967c4e4f9c03b762 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:35.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66bfe00640 1 -- 172.21.15.86:0/1389890976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00b2070 con 0x7f66c00b2360 2024-03-22T04:13:35.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66bfe00640 1 -- 172.21.15.86:0/1389890976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b00ad640 con 0x7f66c00b2360 2024-03-22T04:13:35.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66bfe00640 1 -- 172.21.15.86:0/1389890976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00ada10 con 0x7f66c00b2360 2024-03-22T04:13:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 -- 172.21.15.86:0/1389890976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b2360 msgr2=0x7f66c00b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 --2- 172.21.15.86:0/1389890976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b2360 0x7f66c00b2ef0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f66c00683e0 tx=0x7f66b001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 -- 172.21.15.86:0/1389890976 shutdown_connections 2024-03-22T04:13:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 --2- 172.21.15.86:0/1389890976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 --2- 172.21.15.86:0/1389890976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b2360 0x7f66c00b2ef0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 -- 172.21.15.86:0/1389890976 >> 172.21.15.86:0/1389890976 conn(0x7f66c00cc940 msgr2=0x7f66c00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 -- 172.21.15.86:0/1389890976 shutdown_connections 2024-03-22T04:13:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 -- 172.21.15.86:0/1389890976 wait complete. 2024-03-22T04:13:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.692+0000 7f66c7400640 1 Processor -- start 2024-03-22T04:13:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 -- start start 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66c00b2360 0x7f66c00adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b3430 0x7f66c0152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0160c00 con 0x7f66c00b3430 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66c00b5ad0 con 0x7f66c00b2360 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f66c00ae490 con 0x7f66c015d0b0 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b3430 0x7f66c0152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b3430 0x7f66c0152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45306/0 (socket says 172.21.15.86:45306) 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 -- 172.21.15.86:0/3107863987 learned_addr learned my addr 172.21.15.86:0/3107863987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c4a00640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66c00b2360 0x7f66c00adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c5400640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 -- 172.21.15.86:0/3107863987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 msgr2=0x7f66c00ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 -- 172.21.15.86:0/3107863987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66c00b2360 msgr2=0x7f66c00adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66c00b2360 0x7f66c00adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 -- 172.21.15.86:0/3107863987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c0152870 con 0x7f66c00b3430 2024-03-22T04:13:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66b7e00640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b3430 0x7f66c0152330 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c450 tx=0x7f66a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66bd600640 1 -- 172.21.15.86:0/3107863987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8012830 con 0x7f66c00b3430 2024-03-22T04:13:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66bd600640 1 -- 172.21.15.86:0/3107863987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a8017070 con 0x7f66c00b3430 2024-03-22T04:13:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66bd600640 1 -- 172.21.15.86:0/3107863987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8012c30 con 0x7f66c00b3430 2024-03-22T04:13:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c00aba90 con 0x7f66c00b3430 2024-03-22T04:13:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c00ad750 con 0x7f66c00b3430 2024-03-22T04:13:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.696+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c0074470 con 0x7f66c00b3430 2024-03-22T04:13:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.700+0000 7f66bd600640 1 -- 172.21.15.86:0/3107863987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f66a801c460 con 0x7f66c00b3430 2024-03-22T04:13:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.700+0000 7f66bd600640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f669c067b20 0x7f669c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.700+0000 7f66bd600640 1 -- 172.21.15.86:0/3107863987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6001+0+0 (secure 0 0 0) 0x7f66a8086800 con 0x7f66c00b3430 2024-03-22T04:13:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.700+0000 7f66c4a00640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f669c067b20 0x7f669c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:35.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.700+0000 7f66c4a00640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f669c067b20 0x7f669c069fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f66b00af000 tx=0x7f66b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:35.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.704+0000 7f66bd600640 1 -- 172.21.15.86:0/3107863987 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f66a8056a70 con 0x7f66c00b3430 2024-03-22T04:13:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:35.896+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f66c00cc6d0 con 0x7f66c00b3430 2024-03-22T04:13:36.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.516+0000 7f66bd600640 1 -- 172.21.15.86:0/3107863987 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f66a805ab10 con 0x7f66c00b3430 2024-03-22T04:13:36.523 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 9 and data pool 10 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.520+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f669c067b20 msgr2=0x7f669c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.520+0000 7f66c7400640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f669c067b20 0x7f669c069fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f66b00af000 tx=0x7f66b0009040 comp rx=0 tx=0).stop 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.520+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b3430 msgr2=0x7f66c0152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.520+0000 7f66c7400640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b3430 0x7f66c0152330 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c450 tx=0x7f66a800c910 comp rx=0 tx=0).stop 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.524+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 shutdown_connections 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.524+0000 7f66c7400640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f669c067b20 0x7f669c069fe0 secure :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f66b00af000 tx=0x7f66b0009040 comp rx=0 tx=0).stop 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.524+0000 7f66c7400640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66c015d0b0 0x7f66c00ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.524+0000 7f66c7400640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c00b3430 0x7f66c0152330 secure :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c450 tx=0x7f66a800c910 comp rx=0 tx=0).stop 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.524+0000 7f66c7400640 1 --2- 172.21.15.86:0/3107863987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66c00b2360 0x7f66c00adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.524+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 >> 172.21.15.86:0/3107863987 conn(0x7f66c00cc940 msgr2=0x7f66c0156680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.524+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 shutdown_connections 2024-03-22T04:13:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.524+0000 7f66c7400640 1 -- 172.21.15.86:0/3107863987 wait complete. 2024-03-22T04:13:36.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb09400640 1 Processor -- start 2024-03-22T04:13:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb09400640 1 -- start start 2024-03-22T04:13:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb09400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feb04147bd0 0x7feb0406e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb09400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb04162e60 con 0x7feb040b3fe0 2024-03-22T04:13:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb09400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb041622e0 con 0x7feb040d0e10 2024-03-22T04:13:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb09400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7feb041625c0 con 0x7feb04147bd0 2024-03-22T04:13:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb03e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feb040b3fe0 0x7feb040d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38670/0 (socket says 172.21.15.86:38670) 2024-03-22T04:13:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb03e00640 1 -- 172.21.15.86:0/1570289875 learned_addr learned my addr 172.21.15.86:0/1570289875 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb08a00640 1 --2- 172.21.15.86:0/1570289875 >> v2:172.21.15.196:3300/0 conn(0x7feb04147bd0 0x7feb0406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2977842500 0 0) 0x7feb04162e60 con 0x7feb040b3fe0 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.700+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feae8003530 con 0x7feb040b3fe0 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 493928158 0 0) 0x7feae8003530 con 0x7feb040b3fe0 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 >> v2:172.21.15.196:3300/0 conn(0x7feb04147bd0 msgr2=0x7feb0406e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 --2- 172.21.15.86:0/1570289875 >> v2:172.21.15.196:3300/0 conn(0x7feb04147bd0 0x7feb0406e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 >> v1:172.21.15.196:6789/0 conn(0x7feb040d0e10 legacy=0x7feb0406e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb041628a0 con 0x7feb040b3fe0 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7feaf4001680 con 0x7feb040b3fe0 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 >> v1:172.21.15.86:6789/0 conn(0x7feb040b3fe0 legacy=0x7feb040d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae8003aa0 0x7feae8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feae8004810 0x7feae8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae80052a0 0x7feae800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb041625c0 con 0x7feae80052a0 2024-03-22T04:13:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feae800c1f0 con 0x7feae8003aa0 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feae800c3f0 con 0x7feae8004810 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb08a00640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae80052a0 0x7feae800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb03e00640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae8003aa0 0x7feae8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb03400640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feae8004810 0x7feae8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb08a00640 1 -- 172.21.15.86:0/1570289875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feae8004810 msgr2=0x7feae8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb08a00640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feae8004810 0x7feae8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb08a00640 1 -- 172.21.15.86:0/1570289875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae8003aa0 msgr2=0x7feae8003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb08a00640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae8003aa0 0x7feae8003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb08a00640 1 -- 172.21.15.86:0/1570289875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae8003530 con 0x7feae80052a0 2024-03-22T04:13:36.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb08a00640 1 -- 172.21.15.86:0/1570289875 reap_dead start 2024-03-22T04:13:36.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.704+0000 7feb08a00640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae80052a0 0x7feae800b9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feafc009fc0 tx=0x7feafc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c42ed55eacbfb80 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:36.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc009c10 con 0x7feae80052a0 2024-03-22T04:13:36.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feafc0a5bf0 con 0x7feae80052a0 2024-03-22T04:13:36.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb02a00640 1 -- 172.21.15.86:0/1570289875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc0039a0 con 0x7feae80052a0 2024-03-22T04:13:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- 172.21.15.86:0/1570289875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae80052a0 msgr2=0x7feae800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae80052a0 0x7feae800b9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feafc009fc0 tx=0x7feafc01ccc0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- 172.21.15.86:0/1570289875 shutdown_connections 2024-03-22T04:13:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 --2- 172.21.15.86:0/1570289875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae80052a0 0x7feae800b9d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- 172.21.15.86:0/1570289875 >> 172.21.15.86:0/1570289875 conn(0x7feb0406b660 msgr2=0x7feb0406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- 172.21.15.86:0/1570289875 shutdown_connections 2024-03-22T04:13:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- 172.21.15.86:0/1570289875 wait complete. 2024-03-22T04:13:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 Processor -- start 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- start start 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae80052a0 0x7feb040ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb040af240 0x7feb0413fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb04145f30 0x7feb04141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb040b4540 con 0x7feb04145f30 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb040b43c0 con 0x7feae80052a0 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb09400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feb04161f20 con 0x7feb040af240 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb08a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb04145f30 0x7feb04141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb08a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb04145f30 0x7feb04141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45758/0 (socket says 172.21.15.86:45758) 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb03e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae80052a0 0x7feb040ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb08a00640 1 -- 172.21.15.86:0/3542781519 learned_addr learned my addr 172.21.15.86:0/3542781519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.708+0000 7feb03400640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb040af240 0x7feb0413fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb08a00640 1 -- 172.21.15.86:0/3542781519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb040af240 msgr2=0x7feb0413fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb08a00640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb040af240 0x7feb0413fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb08a00640 1 -- 172.21.15.86:0/3542781519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae80052a0 msgr2=0x7feb040ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb08a00640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae80052a0 0x7feb040ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb08a00640 1 -- 172.21.15.86:0/3542781519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb040af9c0 con 0x7feb04145f30 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb03e00640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae80052a0 0x7feb040ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb08a00640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb04145f30 0x7feb04141a90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feafc01c730 tx=0x7feafc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feafbe00640 1 -- 172.21.15.86:0/3542781519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc009c10 con 0x7feb04145f30 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feafbe00640 1 -- 172.21.15.86:0/3542781519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feafc0bd070 con 0x7feb04145f30 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb04145450 con 0x7feb04145f30 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feafbe00640 1 -- 172.21.15.86:0/3542781519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc003db0 con 0x7feb04145f30 2024-03-22T04:13:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb04145a10 con 0x7feb04145f30 2024-03-22T04:13:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb04145c10 con 0x7feb04145f30 2024-03-22T04:13:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feafbe00640 1 -- 172.21.15.86:0/3542781519 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7feafc0ae950 con 0x7feb04145f30 2024-03-22T04:13:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feafbe00640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feae4067b50 0x7feae406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feafbe00640 1 -- 172.21.15.86:0/3542781519 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7feafc1219d0 con 0x7feb04145f30 2024-03-22T04:13:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.712+0000 7feb03e00640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feae4067b50 0x7feae406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.716+0000 7feb03e00640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feae4067b50 0x7feae406a010 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feaf4001060 tx=0x7feaf4007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.716+0000 7feafbe00640 1 -- 172.21.15.86:0/3542781519 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7feb04145c10 con 0x7feb04145f30 2024-03-22T04:13:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb04074470 con 0x7feb04145f30 2024-03-22T04:13:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feafbe00640 1 -- 172.21.15.86:0/3542781519 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16945 (secure 0 0 0) 0x7feafc0f6e20 con 0x7feb04145f30 2024-03-22T04:13:36.880 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:36.880 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":66,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:35.899541+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:13:34.224743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-22T04:13:34.852940+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feae4067b50 msgr2=0x7feae406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feae4067b50 0x7feae406a010 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feaf4001060 tx=0x7feaf4007900 comp rx=0 tx=0).stop 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb04145f30 msgr2=0x7feb04141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb04145f30 0x7feb04141a90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feafc01c730 tx=0x7feafc009750 comp rx=0 tx=0).stop 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 shutdown_connections 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feae4067b50 0x7feae406a010 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb04145f30 0x7feb04141a90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb040af240 0x7feb0413fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 --2- 172.21.15.86:0/3542781519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feae80052a0 0x7feb040ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 >> 172.21.15.86:0/3542781519 conn(0x7feb0406b660 msgr2=0x7feb040b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 shutdown_connections 2024-03-22T04:13:36.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:36.876+0000 7feb09400640 1 -- 172.21.15.86:0/3542781519 wait complete. 2024-03-22T04:13:36.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.032+0000 7ff070000640 1 Processor -- start 2024-03-22T04:13:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.032+0000 7ff070000640 1 -- start start 2024-03-22T04:13:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff070000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06815b3f0 0x7ff0680729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff070000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff0680cc3e0 0x7ff068072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff070000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff068071920 con 0x7ff0680b30f0 2024-03-22T04:13:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff070000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff0680716d0 con 0x7ff0680cc3e0 2024-03-22T04:13:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff070000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff06806d990 con 0x7ff06815b3f0 2024-03-22T04:13:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff0680cc3e0 0x7ff068072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff0680cc3e0 0x7ff068072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45774/0 (socket says 172.21.15.86:45774) 2024-03-22T04:13:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06cc00640 1 -- 172.21.15.86:0/1940975582 learned_addr learned my addr 172.21.15.86:0/1940975582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06d600640 1 --2- 172.21.15.86:0/1940975582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06815b3f0 0x7ff0680729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06cc00640 1 -- 172.21.15.86:0/1940975582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06815b3f0 msgr2=0x7ff0680729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06cc00640 1 --2- 172.21.15.86:0/1940975582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06815b3f0 0x7ff0680729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06cc00640 1 -- 172.21.15.86:0/1940975582 >> v1:172.21.15.196:6789/0 conn(0x7ff0680b30f0 legacy=0x7ff0680ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06cc00640 1 -- 172.21.15.86:0/1940975582 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff068071c00 con 0x7ff0680cc3e0 2024-03-22T04:13:37.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.036+0000 7ff06cc00640 1 --2- 172.21.15.86:0/1940975582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068072f00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff05400bb10 tx=0x7ff05401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a3b62ef18f6eb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff067e00640 1 -- 172.21.15.86:0/1940975582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540b4070 con 0x7ff0680cc3e0 2024-03-22T04:13:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff067e00640 1 -- 172.21.15.86:0/1940975582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0540a7b60 con 0x7ff0680cc3e0 2024-03-22T04:13:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff067e00640 1 -- 172.21.15.86:0/1940975582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540af440 con 0x7ff0680cc3e0 2024-03-22T04:13:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- 172.21.15.86:0/1940975582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 msgr2=0x7ff068072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 --2- 172.21.15.86:0/1940975582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068072f00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff05400bb10 tx=0x7ff05401eba0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- 172.21.15.86:0/1940975582 shutdown_connections 2024-03-22T04:13:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 --2- 172.21.15.86:0/1940975582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068072f00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 --2- 172.21.15.86:0/1940975582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06815b3f0 0x7ff0680729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- 172.21.15.86:0/1940975582 >> 172.21.15.86:0/1940975582 conn(0x7ff0680b1df0 msgr2=0x7ff0680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- 172.21.15.86:0/1940975582 shutdown_connections 2024-03-22T04:13:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- 172.21.15.86:0/1940975582 wait complete. 2024-03-22T04:13:37.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 Processor -- start 2024-03-22T04:13:37.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- start start 2024-03-22T04:13:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0680b30f0 0x7ff0680995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06815b3f0 0x7ff068154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff06806e2d0 con 0x7ff0680cc3e0 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff06806e150 con 0x7ff06815b3f0 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff070000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff06806e450 con 0x7ff0680b30f0 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff06cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff06cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45790/0 (socket says 172.21.15.86:45790) 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff06cc00640 1 -- 172.21.15.86:0/2597068459 learned_addr learned my addr 172.21.15.86:0/2597068459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff06e000640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06815b3f0 0x7ff068154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.040+0000 7ff06d600640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0680b30f0 0x7ff0680995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff06cc00640 1 -- 172.21.15.86:0/2597068459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0680b30f0 msgr2=0x7ff0680995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff06cc00640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0680b30f0 0x7ff0680995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff06cc00640 1 -- 172.21.15.86:0/2597068459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06815b3f0 msgr2=0x7ff068154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff06cc00640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06815b3f0 0x7ff068154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff06cc00640 1 -- 172.21.15.86:0/2597068459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06814b970 con 0x7ff0680cc3e0 2024-03-22T04:13:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff06cc00640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068154600 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff05400b2c0 tx=0x7ff05401b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff065600640 1 -- 172.21.15.86:0/2597068459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540b4040 con 0x7ff0680cc3e0 2024-03-22T04:13:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff065600640 1 -- 172.21.15.86:0/2597068459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0540a7960 con 0x7ff0680cc3e0 2024-03-22T04:13:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff065600640 1 -- 172.21.15.86:0/2597068459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540ba680 con 0x7ff0680cc3e0 2024-03-22T04:13:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06814bc50 con 0x7ff0680cc3e0 2024-03-22T04:13:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06814c210 con 0x7ff0680cc3e0 2024-03-22T04:13:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff068068650 con 0x7ff0680cc3e0 2024-03-22T04:13:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff065600640 1 -- 172.21.15.86:0/2597068459 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff0540c1020 con 0x7ff0680cc3e0 2024-03-22T04:13:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff065600640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff048067b20 0x7ff048069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff065600640 1 -- 172.21.15.86:0/2597068459 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7ff054122c70 con 0x7ff0680cc3e0 2024-03-22T04:13:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.044+0000 7ff06d600640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff048067b20 0x7ff048069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.048+0000 7ff06d600640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff048067b20 0x7ff048069fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff05c011340 tx=0x7ff05c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:37.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.048+0000 7ff065600640 1 -- 172.21.15.86:0/2597068459 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff0540c8020 con 0x7ff0680cc3e0 2024-03-22T04:13:37.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.204+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff068153f80 con 0x7ff0680cc3e0 2024-03-22T04:13:37.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff065600640 1 -- 172.21.15.86:0/2597068459 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16945 (secure 0 0 0) 0x7ff054018820 con 0x7ff0680cc3e0 2024-03-22T04:13:37.211 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:37.211 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":66,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:35.899541+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:13:34.224743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-22T04:13:34.852940+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff048067b20 msgr2=0x7ff048069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff048067b20 0x7ff048069fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff05c011340 tx=0x7ff05c00b040 comp rx=0 tx=0).stop 2024-03-22T04:13:37.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 msgr2=0x7ff068154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068154600 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff05400b2c0 tx=0x7ff05401b940 comp rx=0 tx=0).stop 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 shutdown_connections 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff048067b20 0x7ff048069fe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06815b3f0 0x7ff068154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cc3e0 0x7ff068154600 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 --2- 172.21.15.86:0/2597068459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0680b30f0 0x7ff0680995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 >> 172.21.15.86:0/2597068459 conn(0x7ff0680b1df0 msgr2=0x7ff0681534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 shutdown_connections 2024-03-22T04:13:37.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.208+0000 7ff070000640 1 -- 172.21.15.86:0/2597068459 wait complete. 2024-03-22T04:13:37.232 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T04:13:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.364+0000 7ff5c8c00640 1 Processor -- start 2024-03-22T04:13:38.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.364+0000 7ff5c8c00640 1 -- start start 2024-03-22T04:13:38.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5c406e7e0 0x7ff5c406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c8c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c4072650 0x7ff5c4072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c415c770 con 0x7ff5c40b1b80 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c8c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c4071350 con 0x7ff5c406e7e0 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c8c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c406f100 con 0x7ff5c4072650 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5c406e7e0 0x7ff5c406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c2800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff5c40b1b80 0x7ff5c40b2010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38674/0 (socket says 172.21.15.86:38674) 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c2800640 1 -- 172.21.15.86:0/1820044180 learned_addr learned my addr 172.21.15.86:0/1820044180 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5b9400640 1 --2- 172.21.15.86:0/1820044180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c4072650 0x7ff5c4072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c1400640 1 -- 172.21.15.86:0/1820044180 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1086104130 0 0) 0x7ff5bc0014a0 con 0x7ff5c40b1b80 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c1400640 1 -- 172.21.15.86:0/1820044180 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a8005740 con 0x7ff5c40b1b80 2024-03-22T04:13:38.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c1e00640 1 -- 172.21.15.86:0/1820044180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c4072650 msgr2=0x7ff5c4072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c1e00640 1 --2- 172.21.15.86:0/1820044180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c4072650 0x7ff5c4072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c1e00640 1 -- 172.21.15.86:0/1820044180 >> v1:172.21.15.86:6789/0 conn(0x7ff5c40b1b80 legacy=0x7ff5c40b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5c1e00640 1 -- 172.21.15.86:0/1820044180 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c4073150 con 0x7ff5c406e7e0 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.368+0000 7ff5b9400640 1 --2- 172.21.15.86:0/1820044180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c4072650 0x7ff5c4072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c1e00640 1 --2- 172.21.15.86:0/1820044180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c406ebc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40097c0 tx=0x7ff5b401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3e631a4abfe7e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c1400640 1 -- 172.21.15.86:0/1820044180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40b0070 con 0x7ff5c406e7e0 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c1400640 1 -- 172.21.15.86:0/1820044180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b40ab600 con 0x7ff5c406e7e0 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c1400640 1 -- 172.21.15.86:0/1820044180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40ab9d0 con 0x7ff5c406e7e0 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1820044180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 msgr2=0x7ff5c406ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1820044180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c406ebc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40097c0 tx=0x7ff5b401c3e0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1820044180 shutdown_connections 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1820044180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c4072650 0x7ff5c4072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1820044180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c406ebc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1820044180 >> 172.21.15.86:0/1820044180 conn(0x7ff5c40cc940 msgr2=0x7ff5c40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1820044180 shutdown_connections 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1820044180 wait complete. 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 Processor -- start 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.372+0000 7ff5c8c00640 1 -- start start 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c41faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c4072650 0x7ff5c4156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c40b1b80 0x7ff5c4156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4072080 con 0x7ff5c406e7e0 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40ce930 con 0x7ff5c4072650 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c41fb880 con 0x7ff5c40b1b80 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c41faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c41faf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45804/0 (socket says 172.21.15.86:45804) 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 -- 172.21.15.86:0/1131475655 learned_addr learned my addr 172.21.15.86:0/1131475655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c2800640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c40b1b80 0x7ff5c4156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5b9400640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c4072650 0x7ff5c4156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 -- 172.21.15.86:0/1131475655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c40b1b80 msgr2=0x7ff5c4156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c40b1b80 0x7ff5c4156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 -- 172.21.15.86:0/1131475655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c4072650 msgr2=0x7ff5c4156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c4072650 0x7ff5c4156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 -- 172.21.15.86:0/1131475655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c41fbb60 con 0x7ff5c406e7e0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c1e00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c41faf10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40ae040 tx=0x7ff5b401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5baa00640 1 -- 172.21.15.86:0/1131475655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40b0050 con 0x7ff5c406e7e0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5baa00640 1 -- 172.21.15.86:0/1131475655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b40b1040 con 0x7ff5c406e7e0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5baa00640 1 -- 172.21.15.86:0/1131475655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4006e20 con 0x7ff5c406e7e0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c4155c40 con 0x7ff5c406e7e0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c41fb450 con 0x7ff5c406e7e0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c4074470 con 0x7ff5c406e7e0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.376+0000 7ff5baa00640 1 -- 172.21.15.86:0/1131475655 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff5b40be030 con 0x7ff5c406e7e0 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.380+0000 7ff5baa00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff5a0067b20 0x7ff5a0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.380+0000 7ff5baa00640 1 -- 172.21.15.86:0/1131475655 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7ff5b411fcb0 con 0x7ff5c406e7e0 2024-03-22T04:13:38.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.380+0000 7ff5b9400640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff5a0067b20 0x7ff5a0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.380+0000 7ff5b9400640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff5a0067b20 0x7ff5a0069fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4069cf0 tx=0x7ff5ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:38.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.384+0000 7ff5baa00640 1 -- 172.21.15.86:0/1131475655 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff5b40c5060 con 0x7ff5c406e7e0 2024-03-22T04:13:38.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:37.572+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff5c41fbd30 con 0x7ff5c406e7e0 2024-03-22T04:13:38.504 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:13:38.499+0000 7f42bf400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:13:38.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.540+0000 7ff5baa00640 1 -- 172.21.15.86:0/1131475655 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7ff5b4016090 con 0x7ff5c406e7e0 2024-03-22T04:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff5a0067b20 msgr2=0x7ff5a0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff5a0067b20 0x7ff5a0069fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4069cf0 tx=0x7ff5ac008040 comp rx=0 tx=0).stop 2024-03-22T04:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 msgr2=0x7ff5c41faf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c41faf10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40ae040 tx=0x7ff5b401ac20 comp rx=0 tx=0).stop 2024-03-22T04:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 shutdown_connections 2024-03-22T04:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff5a0067b20 0x7ff5a0069fe0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5c4072650 0x7ff5c4156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5c40b1b80 0x7ff5c4156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 --2- 172.21.15.86:0/1131475655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c406e7e0 0x7ff5c41faf10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 >> 172.21.15.86:0/1131475655 conn(0x7ff5c40cc940 msgr2=0x7ff5c4154a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 shutdown_connections 2024-03-22T04:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.544+0000 7ff5c8c00640 1 -- 172.21.15.86:0/1131475655 wait complete. 2024-03-22T04:13:38.570 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-22T04:13:38.570 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-22T04:13:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2f200640 1 Processor -- start 2024-03-22T04:13:38.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2f200640 1 -- start start 2024-03-22T04:13:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba2806e7e0 0x7fba2806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2f200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fba28072650 0x7fba28072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba2815c770 con 0x7fba280b1b80 2024-03-22T04:13:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2f200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fba28071350 con 0x7fba28072650 2024-03-22T04:13:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fba2806f100 con 0x7fba2806e7e0 2024-03-22T04:13:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2d200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fba280b1b80 0x7fba280b2010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38680/0 (socket says 172.21.15.86:38680) 2024-03-22T04:13:38.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba2d200640 1 -- 172.21.15.86:0/2897672017 learned_addr learned my addr 172.21.15.86:0/2897672017 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba27400640 1 --2- 172.21.15.86:0/2897672017 >> v2:172.21.15.196:3300/0 conn(0x7fba28072650 0x7fba28072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba27e00640 1 --2- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba2806e7e0 0x7fba2806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba26a00640 1 -- 172.21.15.86:0/2897672017 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2241623103 0 0) 0x7fba200014a0 con 0x7fba280b1b80 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba26a00640 1 -- 172.21.15.86:0/2897672017 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba0c003530 con 0x7fba280b1b80 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba27400640 1 -- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba2806e7e0 msgr2=0x7fba2806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba27400640 1 --2- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba2806e7e0 0x7fba2806ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba27400640 1 -- 172.21.15.86:0/2897672017 >> v1:172.21.15.86:6789/0 conn(0x7fba280b1b80 legacy=0x7fba280b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba27400640 1 -- 172.21.15.86:0/2897672017 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba28073150 con 0x7fba28072650 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.708+0000 7fba27e00640 1 --2- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba2806e7e0 0x7fba2806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba27400640 1 --2- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba28072650 0x7fba28072ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fba140097c0 tx=0x7fba1401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa4e71975ee2d09 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba26a00640 1 -- 172.21.15.86:0/2897672017 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140b1070 con 0x7fba28072650 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba26a00640 1 -- 172.21.15.86:0/2897672017 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba140a8050 con 0x7fba28072650 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba26a00640 1 -- 172.21.15.86:0/2897672017 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140ac860 con 0x7fba28072650 2024-03-22T04:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba2f200640 1 -- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba28072650 msgr2=0x7fba28072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba2f200640 1 --2- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba28072650 0x7fba28072ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fba140097c0 tx=0x7fba1401c3e0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba2f200640 1 -- 172.21.15.86:0/2897672017 shutdown_connections 2024-03-22T04:13:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba2f200640 1 --2- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba28072650 0x7fba28072ad0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba2f200640 1 --2- 172.21.15.86:0/2897672017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba2806e7e0 0x7fba2806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba2f200640 1 -- 172.21.15.86:0/2897672017 >> 172.21.15.86:0/2897672017 conn(0x7fba280cc940 msgr2=0x7fba280ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba2f200640 1 -- 172.21.15.86:0/2897672017 shutdown_connections 2024-03-22T04:13:38.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.712+0000 7fba2f200640 1 -- 172.21.15.86:0/2897672017 wait complete. 2024-03-22T04:13:38.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 Processor -- start 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 -- start start 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba2806e7e0 0x7fba2814dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba28072650 0x7fba281502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba280b1b80 0x7fba281507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba28072080 con 0x7fba280b1b80 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba280ce930 con 0x7fba2806e7e0 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fba281540b0 con 0x7fba28072650 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba280b1b80 0x7fba281507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba280b1b80 0x7fba281507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45820/0 (socket says 172.21.15.86:45820) 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 -- 172.21.15.86:0/3087501966 learned_addr learned my addr 172.21.15.86:0/3087501966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba27e00640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba2806e7e0 0x7fba2814dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba27400640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba28072650 0x7fba281502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 -- 172.21.15.86:0/3087501966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba28072650 msgr2=0x7fba281502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba28072650 0x7fba281502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 -- 172.21.15.86:0/3087501966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba2806e7e0 msgr2=0x7fba2814dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba2806e7e0 0x7fba2814dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 -- 172.21.15.86:0/3087501966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2814f530 con 0x7fba280b1b80 2024-03-22T04:13:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2d200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba280b1b80 0x7fba281507f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fba2000cb40 tx=0x7fba2000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba1fe00640 1 -- 172.21.15.86:0/3087501966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20017070 con 0x7fba280b1b80 2024-03-22T04:13:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba1fe00640 1 -- 172.21.15.86:0/3087501966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2000ac40 con 0x7fba280b1b80 2024-03-22T04:13:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba1fe00640 1 -- 172.21.15.86:0/3087501966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20012400 con 0x7fba280b1b80 2024-03-22T04:13:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba28136440 con 0x7fba280b1b80 2024-03-22T04:13:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.716+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2814f7d0 con 0x7fba280b1b80 2024-03-22T04:13:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.720+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba28074470 con 0x7fba280b1b80 2024-03-22T04:13:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.720+0000 7fba1fe00640 1 -- 172.21.15.86:0/3087501966 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fba2001a070 con 0x7fba280b1b80 2024-03-22T04:13:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.720+0000 7fba1fe00640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba08067b50 0x7fba0806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.720+0000 7fba1fe00640 1 -- 172.21.15.86:0/3087501966 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fba20085e60 con 0x7fba280b1b80 2024-03-22T04:13:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.720+0000 7fba27e00640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba08067b50 0x7fba0806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.720+0000 7fba27e00640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba08067b50 0x7fba0806a010 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba18009580 tx=0x7fba18009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:38.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.724+0000 7fba1fe00640 1 -- 172.21.15.86:0/3087501966 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fba2815c770 con 0x7fba280b1b80 2024-03-22T04:13:38.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:38.916+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fba280cc390 con 0x7fba280b1b80 2024-03-22T04:13:39.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.544+0000 7fba1fe00640 1 -- 172.21.15.86:0/3087501966 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fba20055f60 con 0x7fba280b1b80 2024-03-22T04:13:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba08067b50 msgr2=0x7fba0806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba08067b50 0x7fba0806a010 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba18009580 tx=0x7fba18009190 comp rx=0 tx=0).stop 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba280b1b80 msgr2=0x7fba281507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba280b1b80 0x7fba281507f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fba2000cb40 tx=0x7fba2000a060 comp rx=0 tx=0).stop 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 shutdown_connections 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fba08067b50 0x7fba0806a010 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba280b1b80 0x7fba281507f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba28072650 0x7fba281502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 --2- 172.21.15.86:0/3087501966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba2806e7e0 0x7fba2814dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 >> 172.21.15.86:0/3087501966 conn(0x7fba280cc940 msgr2=0x7fba280b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 shutdown_connections 2024-03-22T04:13:39.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.548+0000 7fba2f200640 1 -- 172.21.15.86:0/3087501966 wait complete. 2024-03-22T04:13:39.573 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-22T04:13:39.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:39.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.704+0000 7fa0d0000640 1 Processor -- start 2024-03-22T04:13:39.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.704+0000 7fa0d0000640 1 -- start start 2024-03-22T04:13:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0d0000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa0c80ad330 0x7fa0c815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0d0000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ef40 0x7fa0c806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0d0000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0c80ad710 0x7fa0c813fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:39.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0d0000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa0c806dc40 con 0x7fa0c80ad330 2024-03-22T04:13:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0d0000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa0c815c7c0 con 0x7fa0c80ad710 2024-03-22T04:13:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0d0000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c806dea0 con 0x7fa0c806ef40 2024-03-22T04:13:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0cd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa0c80ad330 0x7fa0c815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0cd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa0c80ad330 0x7fa0c815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45826/0 (socket says 172.21.15.86:45826) 2024-03-22T04:13:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0cd600640 1 -- 172.21.15.86:0/2838441510 learned_addr learned my addr 172.21.15.86:0/2838441510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0ccc00640 1 --2- 172.21.15.86:0/2838441510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ef40 0x7fa0c806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0ce000640 1 --2- 172.21.15.86:0/2838441510 >> v2:172.21.15.196:3300/0 conn(0x7fa0c80ad710 0x7fa0c813fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:39.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0cd600640 1 -- 172.21.15.86:0/2838441510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ef40 msgr2=0x7fa0c806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0cd600640 1 --2- 172.21.15.86:0/2838441510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ef40 0x7fa0c806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0cd600640 1 -- 172.21.15.86:0/2838441510 >> v2:172.21.15.196:3300/0 conn(0x7fa0c80ad710 msgr2=0x7fa0c813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0cd600640 1 --2- 172.21.15.86:0/2838441510 >> v2:172.21.15.196:3300/0 conn(0x7fa0c80ad710 0x7fa0c813fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.708+0000 7fa0cd600640 1 -- 172.21.15.86:0/2838441510 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c806e180 con 0x7fa0c80ad330 2024-03-22T04:13:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0cd600640 1 --2- 172.21.15.86:0/2838441510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad330 0x7fa0c815d4a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa0c806c5c0 tx=0x7fa0bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445f7b871fd116c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0c7e00640 1 -- 172.21.15.86:0/2838441510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0aa610 con 0x7fa0c80ad330 2024-03-22T04:13:39.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0c7e00640 1 -- 172.21.15.86:0/2838441510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0af070 con 0x7fa0c80ad330 2024-03-22T04:13:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0c7e00640 1 -- 172.21.15.86:0/2838441510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0aa9d0 con 0x7fa0c80ad330 2024-03-22T04:13:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 -- 172.21.15.86:0/2838441510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad330 msgr2=0x7fa0c815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 --2- 172.21.15.86:0/2838441510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad330 0x7fa0c815d4a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa0c806c5c0 tx=0x7fa0bc01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 -- 172.21.15.86:0/2838441510 shutdown_connections 2024-03-22T04:13:39.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 --2- 172.21.15.86:0/2838441510 >> v2:172.21.15.196:3300/0 conn(0x7fa0c80ad710 0x7fa0c813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 --2- 172.21.15.86:0/2838441510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ef40 0x7fa0c806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 --2- 172.21.15.86:0/2838441510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad330 0x7fa0c815d4a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 -- 172.21.15.86:0/2838441510 >> 172.21.15.86:0/2838441510 conn(0x7fa0c80b1df0 msgr2=0x7fa0c80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 -- 172.21.15.86:0/2838441510 shutdown_connections 2024-03-22T04:13:39.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 -- 172.21.15.86:0/2838441510 wait complete. 2024-03-22T04:13:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 Processor -- start 2024-03-22T04:13:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.712+0000 7fa0d0000640 1 -- start start 2024-03-22T04:13:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c806ef40 0x7fa0c8157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c80ad330 0x7fa0c8157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad710 0x7fa0c81582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8141800 con 0x7fa0c80ad710 2024-03-22T04:13:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8141680 con 0x7fa0c806ef40 2024-03-22T04:13:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c8141980 con 0x7fa0c80ad330 2024-03-22T04:13:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad710 0x7fa0c81582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad710 0x7fa0c81582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45838/0 (socket says 172.21.15.86:45838) 2024-03-22T04:13:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 -- 172.21.15.86:0/3448045871 learned_addr learned my addr 172.21.15.86:0/3448045871 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0cd600640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c806ef40 0x7fa0c8157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ccc00640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c80ad330 0x7fa0c8157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 -- 172.21.15.86:0/3448045871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c80ad330 msgr2=0x7fa0c8157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c80ad330 0x7fa0c8157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 -- 172.21.15.86:0/3448045871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c806ef40 msgr2=0x7fa0c8157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c806ef40 0x7fa0c8157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 -- 172.21.15.86:0/3448045871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c8158f00 con 0x7fa0c80ad710 2024-03-22T04:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0ce000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad710 0x7fa0c81582f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009d30 tx=0x7fa0c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0c5600640 1 -- 172.21.15.86:0/3448045871 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0018020 con 0x7fa0c80ad710 2024-03-22T04:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0c5600640 1 -- 172.21.15.86:0/3448045871 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c0014480 con 0x7fa0c80ad710 2024-03-22T04:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0c5600640 1 -- 172.21.15.86:0/3448045871 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0014870 con 0x7fa0c80ad710 2024-03-22T04:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c8158940 con 0x7fa0c80ad710 2024-03-22T04:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c81643e0 con 0x7fa0c80ad710 2024-03-22T04:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c8068650 con 0x7fa0c80ad710 2024-03-22T04:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.716+0000 7fa0c5600640 1 -- 172.21.15.86:0/3448045871 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa0c001b020 con 0x7fa0c80ad710 2024-03-22T04:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.720+0000 7fa0c5600640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0a8067b20 0x7fa0a8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.720+0000 7fa0c5600640 1 -- 172.21.15.86:0/3448045871 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fa0c0086c60 con 0x7fa0c80ad710 2024-03-22T04:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.720+0000 7fa0cd600640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0a8067b20 0x7fa0a8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.720+0000 7fa0cd600640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0a8067b20 0x7fa0a8069fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0ae000 tx=0x7fa0bc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.724+0000 7fa0c5600640 1 -- 172.21.15.86:0/3448045871 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa0c0022030 con 0x7fa0c80ad710 2024-03-22T04:13:39.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.912+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0c81593e0 con 0x7fa0c80ad710 2024-03-22T04:13:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0c5600640 1 -- 172.21.15.86:0/3448045871 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7fa0c8158f00 con 0x7fa0c80ad710 2024-03-22T04:13:39.918 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:39.919 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:13:38.919534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:13:39.919 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 35 2024-03-22T04:13:39.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0a8067b20 msgr2=0x7fa0a8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:39.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0a8067b20 0x7fa0a8069fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0ae000 tx=0x7fa0bc0a7000 comp rx=0 tx=0).stop 2024-03-22T04:13:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad710 msgr2=0x7fa0c81582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad710 0x7fa0c81582f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009d30 tx=0x7fa0c000c490 comp rx=0 tx=0).stop 2024-03-22T04:13:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 shutdown_connections 2024-03-22T04:13:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa0a8067b20 0x7fa0a8069fe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c80ad710 0x7fa0c81582f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c80ad330 0x7fa0c8157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 --2- 172.21.15.86:0/3448045871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c806ef40 0x7fa0c8157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 >> 172.21.15.86:0/3448045871 conn(0x7fa0c80b1df0 msgr2=0x7fa0c8071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:39.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 shutdown_connections 2024-03-22T04:13:39.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:39.916+0000 7fa0d0000640 1 -- 172.21.15.86:0/3448045871 wait complete. 2024-03-22T04:13:39.940 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:13:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8c6400640 1 Processor -- start 2024-03-22T04:13:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8c6400640 1 -- start start 2024-03-22T04:13:40.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8c6400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8c006ed10 0x7ff8c00cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8c6400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c015e970 con 0x7ff8c006b660 2024-03-22T04:13:40.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8c6400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c0072aa0 con 0x7ff8c006e8b0 2024-03-22T04:13:40.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8c6400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c0073b40 con 0x7ff8c006ed10 2024-03-22T04:13:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8bf400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff8c006b660 0x7ff8c006ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38696/0 (socket says 172.21.15.86:38696) 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8bf400640 1 -- 172.21.15.86:0/1722321037 learned_addr learned my addr 172.21.15.86:0/1722321037 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8bfe00640 1 --2- 172.21.15.86:0/1722321037 >> v2:172.21.15.86:3300/0 conn(0x7ff8c006ed10 0x7ff8c00cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8be000640 1 -- 172.21.15.86:0/1722321037 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983026362 0 0) 0x7ff8b0001120 con 0x7ff8c006b660 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8be000640 1 -- 172.21.15.86:0/1722321037 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8a4003530 con 0x7ff8c006b660 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8be000640 1 -- 172.21.15.86:0/1722321037 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670205714 0 0) 0x7ff8c0072aa0 con 0x7ff8c006e8b0 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8be000640 1 -- 172.21.15.86:0/1722321037 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8b0001120 con 0x7ff8c006e8b0 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8bfe00640 1 -- 172.21.15.86:0/1722321037 >> v1:172.21.15.196:6789/0 conn(0x7ff8c006e8b0 legacy=0x7ff8c00cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8bfe00640 1 -- 172.21.15.86:0/1722321037 >> v1:172.21.15.86:6789/0 conn(0x7ff8c006b660 legacy=0x7ff8c006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.072+0000 7ff8bfe00640 1 -- 172.21.15.86:0/1722321037 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c0072d80 con 0x7ff8c006ed10 2024-03-22T04:13:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8bfe00640 1 --2- 172.21.15.86:0/1722321037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006ed10 0x7ff8c00cfea0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009d90 tx=0x7ff8b801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4dd34c35a0b15b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8be000640 1 -- 172.21.15.86:0/1722321037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80b2070 con 0x7ff8c006ed10 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8be000640 1 -- 172.21.15.86:0/1722321037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b80a5830 con 0x7ff8c006ed10 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8be000640 1 -- 172.21.15.86:0/1722321037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80a5c00 con 0x7ff8c006ed10 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8c6400640 1 -- 172.21.15.86:0/1722321037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006ed10 msgr2=0x7ff8c00cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8c6400640 1 --2- 172.21.15.86:0/1722321037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006ed10 0x7ff8c00cfea0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009d90 tx=0x7ff8b801c970 comp rx=0 tx=0).stop 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8c6400640 1 -- 172.21.15.86:0/1722321037 shutdown_connections 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8c6400640 1 --2- 172.21.15.86:0/1722321037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006ed10 0x7ff8c00cfea0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8c6400640 1 -- 172.21.15.86:0/1722321037 >> 172.21.15.86:0/1722321037 conn(0x7ff8c006c770 msgr2=0x7ff8c006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8c6400640 1 -- 172.21.15.86:0/1722321037 shutdown_connections 2024-03-22T04:13:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.076+0000 7ff8c6400640 1 -- 172.21.15.86:0/1722321037 wait complete. 2024-03-22T04:13:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 Processor -- start 2024-03-22T04:13:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 -- start start 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c006b660 0x7ff8c0145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006e8b0 0x7ff8c0149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8c006ed10 0x7ff8c0149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c00d0970 con 0x7ff8c006e8b0 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c00d07f0 con 0x7ff8c006b660 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c016c940 con 0x7ff8c006ed10 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006e8b0 0x7ff8c0149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006e8b0 0x7ff8c0149770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45854/0 (socket says 172.21.15.86:45854) 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 -- 172.21.15.86:0/1662174249 learned_addr learned my addr 172.21.15.86:0/1662174249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:40.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bf400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c006b660 0x7ff8c0145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bfe00640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8c006ed10 0x7ff8c0149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 -- 172.21.15.86:0/1662174249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8c006ed10 msgr2=0x7ff8c0149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8c006ed10 0x7ff8c0149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 -- 172.21.15.86:0/1662174249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c006b660 msgr2=0x7ff8c0145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c006b660 0x7ff8c0145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 -- 172.21.15.86:0/1662174249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c0148740 con 0x7ff8c006e8b0 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8bea00640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006e8b0 0x7ff8c0149770 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00c430 tx=0x7ff8ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8b7400640 1 -- 172.21.15.86:0/1662174249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac017070 con 0x7ff8c006e8b0 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8b7400640 1 -- 172.21.15.86:0/1662174249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0052f0 con 0x7ff8c006e8b0 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c0148a20 con 0x7ff8c006e8b0 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8b7400640 1 -- 172.21.15.86:0/1662174249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac003840 con 0x7ff8c006e8b0 2024-03-22T04:13:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c0148fe0 con 0x7ff8c006e8b0 2024-03-22T04:13:40.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.080+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c01491e0 con 0x7ff8c006e8b0 2024-03-22T04:13:40.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.084+0000 7ff8b7400640 1 -- 172.21.15.86:0/1662174249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff8ac01a070 con 0x7ff8c006e8b0 2024-03-22T04:13:40.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.084+0000 7ff8b7400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff8a0067b50 0x7ff8a006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.084+0000 7ff8b7400640 1 -- 172.21.15.86:0/1662174249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff8ac085410 con 0x7ff8c006e8b0 2024-03-22T04:13:40.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.084+0000 7ff8bf400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff8a0067b50 0x7ff8a006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.084+0000 7ff8bf400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff8a0067b50 0x7ff8a006a010 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00013c0 tx=0x7ff8b0007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.088+0000 7ff8b7400640 1 -- 172.21.15.86:0/1662174249 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff8c01491e0 con 0x7ff8c006e8b0 2024-03-22T04:13:40.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8c0068650 con 0x7ff8c006e8b0 2024-03-22T04:13:40.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8b7400640 1 -- 172.21.15.86:0/1662174249 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16932 (secure 0 0 0) 0x7ff8ac0555a0 con 0x7ff8c006e8b0 2024-03-22T04:13:40.248 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:40.248 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:13:37.521415+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:13:34.224743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-22T04:13:34.852940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:13:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff8a0067b50 msgr2=0x7ff8a006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff8a0067b50 0x7ff8a006a010 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00013c0 tx=0x7ff8b0007d40 comp rx=0 tx=0).stop 2024-03-22T04:13:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006e8b0 msgr2=0x7ff8c0149770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006e8b0 0x7ff8c0149770 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00c430 tx=0x7ff8ac00c8f0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 shutdown_connections 2024-03-22T04:13:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff8a0067b50 0x7ff8a006a010 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8c006ed10 0x7ff8c0149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c006e8b0 0x7ff8c0149770 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 --2- 172.21.15.86:0/1662174249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8c006b660 0x7ff8c0145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 >> 172.21.15.86:0/1662174249 conn(0x7ff8c006c770 msgr2=0x7ff8c00af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 shutdown_connections 2024-03-22T04:13:40.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.244+0000 7ff8c6400640 1 -- 172.21.15.86:0/1662174249 wait complete. 2024-03-22T04:13:40.268 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T04:13:40.269 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:13:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3331600640 1 Processor -- start 2024-03-22T04:13:40.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3331600640 1 -- start start 2024-03-22T04:13:40.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3331600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f332c161880 0x7f332c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3331600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f332c161c60 0x7f332c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3331600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332c06e350 con 0x7f332c06d700 2024-03-22T04:13:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3331600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f332c15ee80 con 0x7f332c161880 2024-03-22T04:13:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3331600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f332c06e630 con 0x7f332c161c60 2024-03-22T04:13:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f332aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f332c161880 0x7f332c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f332b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f332c06d700 0x7f332c06db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38706/0 (socket says 172.21.15.86:38706) 2024-03-22T04:13:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f332b400640 1 -- 172.21.15.86:0/4016567626 learned_addr learned my addr 172.21.15.86:0/4016567626 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:40.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3322000640 1 --2- 172.21.15.86:0/4016567626 >> v2:172.21.15.196:3300/0 conn(0x7f332c161c60 0x7f332c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f332aa00640 1 -- 172.21.15.86:0/4016567626 >> v2:172.21.15.196:3300/0 conn(0x7f332c161c60 msgr2=0x7f332c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f332aa00640 1 --2- 172.21.15.86:0/4016567626 >> v2:172.21.15.196:3300/0 conn(0x7f332c161c60 0x7f332c0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f332aa00640 1 -- 172.21.15.86:0/4016567626 >> v1:172.21.15.86:6789/0 conn(0x7f332c06d700 legacy=0x7f332c06db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f332aa00640 1 -- 172.21.15.86:0/4016567626 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c06e910 con 0x7f332c161880 2024-03-22T04:13:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f3322000640 1 --2- 172.21.15.86:0/4016567626 >> v2:172.21.15.196:3300/0 conn(0x7f332c161c60 0x7f332c0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.400+0000 7f332a000640 1 -- 172.21.15.86:0/4016567626 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85650719 0 0) 0x7f332c06e350 con 0x7f332c06d700 2024-03-22T04:13:40.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f332aa00640 1 --2- 172.21.15.86:0/4016567626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161880 0x7f332c0b30f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f332c0b6280 tx=0x7f331c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22b26de155f9a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f332a000640 1 -- 172.21.15.86:0/4016567626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c0b2070 con 0x7f332c161880 2024-03-22T04:13:40.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f332a000640 1 -- 172.21.15.86:0/4016567626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f331c0a5930 con 0x7f332c161880 2024-03-22T04:13:40.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f332a000640 1 -- 172.21.15.86:0/4016567626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c0a5d00 con 0x7f332c161880 2024-03-22T04:13:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f3331600640 1 -- 172.21.15.86:0/4016567626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161880 msgr2=0x7f332c0b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f3331600640 1 --2- 172.21.15.86:0/4016567626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161880 0x7f332c0b30f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f332c0b6280 tx=0x7f331c01c870 comp rx=0 tx=0).stop 2024-03-22T04:13:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f3331600640 1 -- 172.21.15.86:0/4016567626 shutdown_connections 2024-03-22T04:13:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f3331600640 1 --2- 172.21.15.86:0/4016567626 >> v2:172.21.15.196:3300/0 conn(0x7f332c161c60 0x7f332c0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f3331600640 1 --2- 172.21.15.86:0/4016567626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161880 0x7f332c0b30f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f3331600640 1 -- 172.21.15.86:0/4016567626 >> 172.21.15.86:0/4016567626 conn(0x7f332c0b1df0 msgr2=0x7f332c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f3331600640 1 -- 172.21.15.86:0/4016567626 shutdown_connections 2024-03-22T04:13:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.404+0000 7f3331600640 1 -- 172.21.15.86:0/4016567626 wait complete. 2024-03-22T04:13:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 Processor -- start 2024-03-22T04:13:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 -- start start 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f332c06d700 0x7f332c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f332c161880 0x7f332c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161c60 0x7f332c1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1608d0 con 0x7f332c161c60 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f332c160750 con 0x7f332c161880 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f332c160a50 con 0x7f332c06d700 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161c60 0x7f332c1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3322000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f332c161880 0x7f332c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161c60 0x7f332c1561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45872/0 (socket says 172.21.15.86:45872) 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 -- 172.21.15.86:0/1922113297 learned_addr learned my addr 172.21.15.86:0/1922113297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332aa00640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f332c06d700 0x7f332c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 -- 172.21.15.86:0/1922113297 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f332c06d700 msgr2=0x7f332c14e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f332c06d700 0x7f332c14e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 -- 172.21.15.86:0/1922113297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f332c161880 msgr2=0x7f332c155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f332c161880 0x7f332c155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 -- 172.21.15.86:0/1922113297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c0b0740 con 0x7f332c161c60 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f332b400640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161c60 0x7f332c1561f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f332400c3e0 tx=0x7f332400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3323400640 1 -- 172.21.15.86:0/1922113297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33240074e0 con 0x7f332c161c60 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3323400640 1 -- 172.21.15.86:0/1922113297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33240047f0 con 0x7f332c161c60 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332c158f40 con 0x7f332c161c60 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3323400640 1 -- 172.21.15.86:0/1922113297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324004be0 con 0x7f332c161c60 2024-03-22T04:13:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332c1594e0 con 0x7f332c161c60 2024-03-22T04:13:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.408+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332c158d70 con 0x7f332c161c60 2024-03-22T04:13:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.412+0000 7f3323400640 1 -- 172.21.15.86:0/1922113297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3324011070 con 0x7f332c161c60 2024-03-22T04:13:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.412+0000 7f3323400640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3308067b50 0x7f330806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.412+0000 7f3323400640 1 -- 172.21.15.86:0/1922113297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3324085af0 con 0x7f332c161c60 2024-03-22T04:13:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.412+0000 7f332aa00640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3308067b50 0x7f330806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.412+0000 7f332aa00640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3308067b50 0x7f330806a010 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f331c0af000 tx=0x7f331c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.416+0000 7f3323400640 1 -- 172.21.15.86:0/1922113297 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3324055c80 con 0x7f332c161c60 2024-03-22T04:13:40.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.568+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f332c068650 con 0x7f3308067b50 2024-03-22T04:13:40.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.572+0000 7f3323400640 1 -- 172.21.15.86:0/1922113297 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247373 (secure 0 0 0) 0x7f332c068650 con 0x7f3308067b50 2024-03-22T04:13:40.581 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:40.582 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:13:40.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3308067b50 msgr2=0x7f330806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3308067b50 0x7f330806a010 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f331c0af000 tx=0x7f331c0a7070 comp rx=0 tx=0).stop 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161c60 msgr2=0x7f332c1561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161c60 0x7f332c1561f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f332400c3e0 tx=0x7f332400c8a0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 shutdown_connections 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3308067b50 0x7f330806a010 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f332c161880 0x7f332c155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f332c161c60 0x7f332c1561f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 --2- 172.21.15.86:0/1922113297 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f332c06d700 0x7f332c14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 >> 172.21.15.86:0/1922113297 conn(0x7f332c0b1df0 msgr2=0x7f332c0716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:40.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 shutdown_connections 2024-03-22T04:13:40.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.576+0000 7f3331600640 1 -- 172.21.15.86:0/1922113297 wait complete. 2024-03-22T04:13:40.601 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-22T04:13:39.830360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227568,"kb_used_data":7220,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747152,"statfs":{"total":773094113280,"available":772861083648,"internally_reserved":0,"allocated":7393280,"data_stored":3943645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.326188"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521460+0000","last_change":"2024-03-22T04:13:35.521460+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521460+0000","last_undegraded":"2024-03-22T04:13:35.521460+0000","last_fullsized":"2024-03-22T04:13:35.521460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515543+0000","last_change":"2024-03-22T04:13:35.515543+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515543+0000","last_undegraded":"2024-03-22T04:13:35.515543+0000","last_fullsized":"2024-03-22T04:13:35.515543+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521592+0000","last_change":"2024-03-22T04:13:35.521592+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521592+0000","last_undegraded":"2024-03-22T04:13:35.521592+0000","last_fullsized":"2024-03-22T04:13:35.521592+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.523111+0000","last_change":"2024-03-22T04:13:35.523111+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.523111+0000","last_undegraded":"2024-03-22T04:13:35.523111+0000","last_fullsized":"2024-03-22T04:13:35.523111+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521553+0000","last_change":"2024-03-22T04:13:35.521553+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521553+0000","last_undegraded":"2024-03-22T04:13:35.521553+0000","last_fullsized":"2024-03-22T04:13:35.521553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515499+0000","last_change":"2024-03-22T04:13:35.515499+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515499+0000","last_undegraded":"2024-03-22T04:13:35.515499+0000","last_fullsized":"2024-03-22T04:13:35.515499+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522743+0000","last_change":"2024-03-22T04:13:35.522743+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522743+0000","last_undegraded":"2024-03-22T04:13:35.522743+0000","last_fullsized":"2024-03-22T04:13:35.522743+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522009+0000","last_change":"2024-03-22T04:13:35.522009+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522009+0000","last_undegraded":"2024-03-22T04:13:35.522009+0000","last_fullsized":"2024-03-22T04:13:35.522009+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519953+0000","last_change":"2024-03-22T04:13:35.519953+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519953+0000","last_undegraded":"2024-03-22T04:13:35.519953+0000","last_fullsized":"2024-03-22T04:13:35.519953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517617+0000","last_change":"2024-03-22T04:13:35.517617+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517617+0000","last_undegraded":"2024-03-22T04:13:35.517617+0000","last_fullsized":"2024-03-22T04:13:35.517617+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521802+0000","last_change":"2024-03-22T04:13:35.521802+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521802+0000","last_undegraded":"2024-03-22T04:13:35.521802+0000","last_fullsized":"2024-03-22T04:13:35.521802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522711+0000","last_change":"2024-03-22T04:13:35.522711+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522711+0000","last_undegraded":"2024-03-22T04:13:35.522711+0000","last_fullsized":"2024-03-22T04:13:35.522711+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518599+0000","last_change":"2024-03-22T04:13:35.518599+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518599+0000","last_undegraded":"2024-03-22T04:13:35.518599+0000","last_fullsized":"2024-03-22T04:13:35.518599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519660+0000","last_change":"2024-03-22T04:13:35.519660+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519660+0000","last_undegraded":"2024-03-22T04:13:35.519660+0000","last_fullsized":"2024-03-22T04:13:35.519660+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516498+0000","last_change":"2024-03-22T04:13:35.516498+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516498+0000","last_undegraded":"2024-03-22T04:13:35.516498+0000","last_fullsized":"2024-03-22T04:13:35.516498+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518341+0000","last_change":"2024-03-22T04:13:35.518341+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518341+0000","last_undegraded":"2024-03-22T04:13:35.518341+0000","last_fullsized":"2024-03-22T04:13:35.518341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534709+0000","last_change":"2024-03-22T04:13:36.534928+0000","last_active":"2024-03-22T04:13:37.534709+0000","last_peered":"2024-03-22T04:13:37.534709+0000","last_clean":"2024-03-22T04:13:37.534709+0000","last_became_active":"2024-03-22T04:13:36.534458+0000","last_became_peered":"2024-03-22T04:13:36.534458+0000","last_unstale":"2024-03-22T04:13:37.534709+0000","last_undegraded":"2024-03-22T04:13:37.534709+0000","last_fullsized":"2024-03-22T04:13:37.534709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:10:28.784356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518861+0000","last_change":"2024-03-22T04:13:35.518861+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518861+0000","last_undegraded":"2024-03-22T04:13:35.518861+0000","last_fullsized":"2024-03-22T04:13:35.518861+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521381+0000","last_change":"2024-03-22T04:13:35.521381+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521381+0000","last_undegraded":"2024-03-22T04:13:35.521381+0000","last_fullsized":"2024-03-22T04:13:35.521381+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534466+0000","last_change":"2024-03-22T04:13:36.533555+0000","last_active":"2024-03-22T04:13:37.534466+0000","last_peered":"2024-03-22T04:13:37.534466+0000","last_clean":"2024-03-22T04:13:37.534466+0000","last_became_active":"2024-03-22T04:13:36.533261+0000","last_became_peered":"2024-03-22T04:13:36.533261+0000","last_unstale":"2024-03-22T04:13:37.534466+0000","last_undegraded":"2024-03-22T04:13:37.534466+0000","last_fullsized":"2024-03-22T04:13:37.534466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:22.636235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518186+0000","last_change":"2024-03-22T04:13:35.518186+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518186+0000","last_undegraded":"2024-03-22T04:13:35.518186+0000","last_fullsized":"2024-03-22T04:13:35.518186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.514455+0000","last_change":"2024-03-22T04:13:35.514455+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.514455+0000","last_undegraded":"2024-03-22T04:13:35.514455+0000","last_fullsized":"2024-03-22T04:13:35.514455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517884+0000","last_change":"2024-03-22T04:13:35.517884+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517884+0000","last_undegraded":"2024-03-22T04:13:35.517884+0000","last_fullsized":"2024-03-22T04:13:35.517884+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520978+0000","last_change":"2024-03-22T04:13:35.520978+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520978+0000","last_undegraded":"2024-03-22T04:13:35.520978+0000","last_fullsized":"2024-03-22T04:13:35.520978+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517424+0000","last_change":"2024-03-22T04:13:35.517424+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517424+0000","last_undegraded":"2024-03-22T04:13:35.517424+0000","last_fullsized":"2024-03-22T04:13:35.517424+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.526133+0000","last_change":"2024-03-22T04:13:35.526133+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.526133+0000","last_undegraded":"2024-03-22T04:13:35.526133+0000","last_fullsized":"2024-03-22T04:13:35.526133+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517842+0000","last_change":"2024-03-22T04:13:35.517842+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517842+0000","last_undegraded":"2024-03-22T04:13:35.517842+0000","last_fullsized":"2024-03-22T04:13:35.517842+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521298+0000","last_change":"2024-03-22T04:13:35.521298+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521298+0000","last_undegraded":"2024-03-22T04:13:35.521298+0000","last_fullsized":"2024-03-22T04:13:35.521298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518277+0000","last_change":"2024-03-22T04:13:35.518277+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518277+0000","last_undegraded":"2024-03-22T04:13:35.518277+0000","last_fullsized":"2024-03-22T04:13:35.518277+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517462+0000","last_change":"2024-03-22T04:13:35.517462+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517462+0000","last_undegraded":"2024-03-22T04:13:35.517462+0000","last_fullsized":"2024-03-22T04:13:35.517462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520275+0000","last_change":"2024-03-22T04:13:35.520275+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520275+0000","last_undegraded":"2024-03-22T04:13:35.520275+0000","last_fullsized":"2024-03-22T04:13:35.520275+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520777+0000","last_change":"2024-03-22T04:13:35.520777+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520777+0000","last_undegraded":"2024-03-22T04:13:35.520777+0000","last_fullsized":"2024-03-22T04:13:35.520777+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522255+0000","last_change":"2024-03-22T04:13:35.522255+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522255+0000","last_undegraded":"2024-03-22T04:13:35.522255+0000","last_fullsized":"2024-03-22T04:13:35.522255+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536950+0000","last_change":"2024-03-22T04:13:35.531755+0000","last_active":"2024-03-22T04:13:35.536950+0000","last_peered":"2024-03-22T04:13:35.536950+0000","last_clean":"2024-03-22T04:13:35.536950+0000","last_became_active":"2024-03-22T04:13:35.531433+0000","last_became_peered":"2024-03-22T04:13:35.531433+0000","last_unstale":"2024-03-22T04:13:35.536950+0000","last_undegraded":"2024-03-22T04:13:35.536950+0000","last_fullsized":"2024-03-22T04:13:35.536950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516557+0000","last_change":"2024-03-22T04:13:35.516557+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516557+0000","last_undegraded":"2024-03-22T04:13:35.516557+0000","last_fullsized":"2024-03-22T04:13:35.516557+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.535991+0000","last_change":"2024-03-22T04:13:35.529846+0000","last_active":"2024-03-22T04:13:35.535991+0000","last_peered":"2024-03-22T04:13:35.535991+0000","last_clean":"2024-03-22T04:13:35.535991+0000","last_became_active":"2024-03-22T04:13:35.529546+0000","last_became_peered":"2024-03-22T04:13:35.529546+0000","last_unstale":"2024-03-22T04:13:35.535991+0000","last_undegraded":"2024-03-22T04:13:35.535991+0000","last_fullsized":"2024-03-22T04:13:35.535991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515064+0000","last_change":"2024-03-22T04:13:35.515064+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515064+0000","last_undegraded":"2024-03-22T04:13:35.515064+0000","last_fullsized":"2024-03-22T04:13:35.515064+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536333+0000","last_change":"2024-03-22T04:13:35.530455+0000","last_active":"2024-03-22T04:13:35.536333+0000","last_peered":"2024-03-22T04:13:35.536333+0000","last_clean":"2024-03-22T04:13:35.536333+0000","last_became_active":"2024-03-22T04:13:35.530130+0000","last_became_peered":"2024-03-22T04:13:35.530130+0000","last_unstale":"2024-03-22T04:13:35.536333+0000","last_undegraded":"2024-03-22T04:13:35.536333+0000","last_fullsized":"2024-03-22T04:13:35.536333+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519480+0000","last_change":"2024-03-22T04:13:35.519480+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519480+0000","last_undegraded":"2024-03-22T04:13:35.519480+0000","last_fullsized":"2024-03-22T04:13:35.519480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537702+0000","last_change":"2024-03-22T04:13:35.533152+0000","last_active":"2024-03-22T04:13:35.537702+0000","last_peered":"2024-03-22T04:13:35.537702+0000","last_clean":"2024-03-22T04:13:35.537702+0000","last_became_active":"2024-03-22T04:13:35.532739+0000","last_became_peered":"2024-03-22T04:13:35.532739+0000","last_unstale":"2024-03-22T04:13:35.537702+0000","last_undegraded":"2024-03-22T04:13:35.537702+0000","last_fullsized":"2024-03-22T04:13:35.537702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:11:03.089640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533547+0000","last_change":"2024-03-22T04:13:36.539261+0000","last_active":"2024-03-22T04:13:37.533547+0000","last_peered":"2024-03-22T04:13:37.533547+0000","last_clean":"2024-03-22T04:13:37.533547+0000","last_became_active":"2024-03-22T04:13:36.538948+0000","last_became_peered":"2024-03-22T04:13:36.538948+0000","last_unstale":"2024-03-22T04:13:37.533547+0000","last_undegraded":"2024-03-22T04:13:37.533547+0000","last_fullsized":"2024-03-22T04:13:37.533547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.370928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534342+0000","last_change":"2024-03-22T04:13:35.528073+0000","last_active":"2024-03-22T04:13:37.534342+0000","last_peered":"2024-03-22T04:13:37.534342+0000","last_clean":"2024-03-22T04:13:37.534342+0000","last_became_active":"2024-03-22T04:13:35.527410+0000","last_became_peered":"2024-03-22T04:13:35.527410+0000","last_unstale":"2024-03-22T04:13:37.534342+0000","last_undegraded":"2024-03-22T04:13:37.534342+0000","last_fullsized":"2024-03-22T04:13:37.534342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:00:59.588177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522326+0000","last_change":"2024-03-22T04:13:35.522326+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522326+0000","last_undegraded":"2024-03-22T04:13:35.522326+0000","last_fullsized":"2024-03-22T04:13:35.522326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.534280+0000","last_change":"2024-03-22T04:13:35.526516+0000","last_active":"2024-03-22T04:13:35.534280+0000","last_peered":"2024-03-22T04:13:35.534280+0000","last_clean":"2024-03-22T04:13:35.534280+0000","last_became_active":"2024-03-22T04:13:35.526208+0000","last_became_peered":"2024-03-22T04:13:35.526208+0000","last_unstale":"2024-03-22T04:13:35.534280+0000","last_undegraded":"2024-03-22T04:13:35.534280+0000","last_fullsized":"2024-03-22T04:13:35.534280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516439+0000","last_change":"2024-03-22T04:13:35.516439+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516439+0000","last_undegraded":"2024-03-22T04:13:35.516439+0000","last_fullsized":"2024-03-22T04:13:35.516439+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537342+0000","last_change":"2024-03-22T04:13:35.534172+0000","last_active":"2024-03-22T04:13:35.537342+0000","last_peered":"2024-03-22T04:13:35.537342+0000","last_clean":"2024-03-22T04:13:35.537342+0000","last_became_active":"2024-03-22T04:13:35.533651+0000","last_became_peered":"2024-03-22T04:13:35.533651+0000","last_unstale":"2024-03-22T04:13:35.537342+0000","last_undegraded":"2024-03-22T04:13:35.537342+0000","last_fullsized":"2024-03-22T04:13:35.537342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534186+0000","last_change":"2024-03-22T04:13:36.535993+0000","last_active":"2024-03-22T04:13:37.534186+0000","last_peered":"2024-03-22T04:13:37.534186+0000","last_clean":"2024-03-22T04:13:37.534186+0000","last_became_active":"2024-03-22T04:13:36.534295+0000","last_became_peered":"2024-03-22T04:13:36.534295+0000","last_unstale":"2024-03-22T04:13:37.534186+0000","last_undegraded":"2024-03-22T04:13:37.534186+0000","last_fullsized":"2024-03-22T04:13:37.534186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:58.215479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539156+0000","last_change":"2024-03-22T04:13:35.533417+0000","last_active":"2024-03-22T04:13:35.539156+0000","last_peered":"2024-03-22T04:13:35.539156+0000","last_clean":"2024-03-22T04:13:35.539156+0000","last_became_active":"2024-03-22T04:13:35.533053+0000","last_became_peered":"2024-03-22T04:13:35.533053+0000","last_unstale":"2024-03-22T04:13:35.539156+0000","last_undegraded":"2024-03-22T04:13:35.539156+0000","last_fullsized":"2024-03-22T04:13:35.539156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519164+0000","last_change":"2024-03-22T04:13:35.519164+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519164+0000","last_undegraded":"2024-03-22T04:13:35.519164+0000","last_fullsized":"2024-03-22T04:13:35.519164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539366+0000","last_change":"2024-03-22T04:13:35.535058+0000","last_active":"2024-03-22T04:13:35.539366+0000","last_peered":"2024-03-22T04:13:35.539366+0000","last_clean":"2024-03-22T04:13:35.539366+0000","last_became_active":"2024-03-22T04:13:35.534345+0000","last_became_peered":"2024-03-22T04:13:35.534345+0000","last_unstale":"2024-03-22T04:13:35.539366+0000","last_undegraded":"2024-03-22T04:13:35.539366+0000","last_fullsized":"2024-03-22T04:13:35.539366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:50.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534393+0000","last_change":"2024-03-22T04:13:36.527089+0000","last_active":"2024-03-22T04:13:37.534393+0000","last_peered":"2024-03-22T04:13:37.534393+0000","last_clean":"2024-03-22T04:13:37.534393+0000","last_became_active":"2024-03-22T04:13:36.526807+0000","last_became_peered":"2024-03-22T04:13:36.526807+0000","last_unstale":"2024-03-22T04:13:37.534393+0000","last_undegraded":"2024-03-22T04:13:37.534393+0000","last_fullsized":"2024-03-22T04:13:37.534393+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:57:11.984612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.529461+0000","last_change":"2024-03-22T04:13:35.525257+0000","last_active":"2024-03-22T04:13:35.529461+0000","last_peered":"2024-03-22T04:13:35.529461+0000","last_clean":"2024-03-22T04:13:35.529461+0000","last_became_active":"2024-03-22T04:13:35.524936+0000","last_became_peered":"2024-03-22T04:13:35.524936+0000","last_unstale":"2024-03-22T04:13:35.529461+0000","last_undegraded":"2024-03-22T04:13:35.529461+0000","last_fullsized":"2024-03-22T04:13:35.529461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:18:55.065379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515798+0000","last_change":"2024-03-22T04:13:35.515798+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515798+0000","last_undegraded":"2024-03-22T04:13:35.515798+0000","last_fullsized":"2024-03-22T04:13:35.515798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.538958+0000","last_change":"2024-03-22T04:13:35.536397+0000","last_active":"2024-03-22T04:13:35.538958+0000","last_peered":"2024-03-22T04:13:35.538958+0000","last_clean":"2024-03-22T04:13:35.538958+0000","last_became_active":"2024-03-22T04:13:35.536137+0000","last_became_peered":"2024-03-22T04:13:35.536137+0000","last_unstale":"2024-03-22T04:13:35.538958+0000","last_undegraded":"2024-03-22T04:13:35.538958+0000","last_fullsized":"2024-03-22T04:13:35.538958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:05:15.541959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518642+0000","last_change":"2024-03-22T04:13:35.518642+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518642+0000","last_undegraded":"2024-03-22T04:13:35.518642+0000","last_fullsized":"2024-03-22T04:13:35.518642+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536050+0000","last_change":"2024-03-22T04:13:35.529973+0000","last_active":"2024-03-22T04:13:35.536050+0000","last_peered":"2024-03-22T04:13:35.536050+0000","last_clean":"2024-03-22T04:13:35.536050+0000","last_became_active":"2024-03-22T04:13:35.529696+0000","last_became_peered":"2024-03-22T04:13:35.529696+0000","last_unstale":"2024-03-22T04:13:35.536050+0000","last_undegraded":"2024-03-22T04:13:35.536050+0000","last_fullsized":"2024-03-22T04:13:35.536050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515862+0000","last_change":"2024-03-22T04:13:35.515862+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515862+0000","last_undegraded":"2024-03-22T04:13:35.515862+0000","last_fullsized":"2024-03-22T04:13:35.515862+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.533045+0000","last_change":"2024-03-22T04:13:35.529378+0000","last_active":"2024-03-22T04:13:35.533045+0000","last_peered":"2024-03-22T04:13:35.533045+0000","last_clean":"2024-03-22T04:13:35.533045+0000","last_became_active":"2024-03-22T04:13:35.529096+0000","last_became_peered":"2024-03-22T04:13:35.529096+0000","last_unstale":"2024-03-22T04:13:35.533045+0000","last_undegraded":"2024-03-22T04:13:35.533045+0000","last_fullsized":"2024-03-22T04:13:35.533045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:25:37.720655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516023+0000","last_change":"2024-03-22T04:13:35.516023+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516023+0000","last_undegraded":"2024-03-22T04:13:35.516023+0000","last_fullsized":"2024-03-22T04:13:35.516023+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.538455+0000","last_change":"2024-03-22T04:13:35.536795+0000","last_active":"2024-03-22T04:13:35.538455+0000","last_peered":"2024-03-22T04:13:35.538455+0000","last_clean":"2024-03-22T04:13:35.538455+0000","last_became_active":"2024-03-22T04:13:35.534453+0000","last_became_peered":"2024-03-22T04:13:35.534453+0000","last_unstale":"2024-03-22T04:13:35.538455+0000","last_undegraded":"2024-03-22T04:13:35.538455+0000","last_fullsized":"2024-03-22T04:13:35.538455+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516963+0000","last_change":"2024-03-22T04:13:35.516963+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516963+0000","last_undegraded":"2024-03-22T04:13:35.516963+0000","last_fullsized":"2024-03-22T04:13:35.516963+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537398+0000","last_change":"2024-03-22T04:13:35.529571+0000","last_active":"2024-03-22T04:13:35.537398+0000","last_peered":"2024-03-22T04:13:35.537398+0000","last_clean":"2024-03-22T04:13:35.537398+0000","last_became_active":"2024-03-22T04:13:35.528385+0000","last_became_peered":"2024-03-22T04:13:35.528385+0000","last_unstale":"2024-03-22T04:13:35.537398+0000","last_undegraded":"2024-03-22T04:13:35.537398+0000","last_fullsized":"2024-03-22T04:13:35.537398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:55:19.501433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519326+0000","last_change":"2024-03-22T04:13:35.519326+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519326+0000","last_undegraded":"2024-03-22T04:13:35.519326+0000","last_fullsized":"2024-03-22T04:13:35.519326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537868+0000","last_change":"2024-03-22T04:13:35.536315+0000","last_active":"2024-03-22T04:13:35.537868+0000","last_peered":"2024-03-22T04:13:35.537868+0000","last_clean":"2024-03-22T04:13:35.537868+0000","last_became_active":"2024-03-22T04:13:35.535908+0000","last_became_peered":"2024-03-22T04:13:35.535908+0000","last_unstale":"2024-03-22T04:13:35.537868+0000","last_undegraded":"2024-03-22T04:13:35.537868+0000","last_fullsized":"2024-03-22T04:13:35.537868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:18:55.065379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517925+0000","last_change":"2024-03-22T04:13:35.517925+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517925+0000","last_undegraded":"2024-03-22T04:13:35.517925+0000","last_fullsized":"2024-03-22T04:13:35.517925+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539347+0000","last_change":"2024-03-22T04:13:35.534854+0000","last_active":"2024-03-22T04:13:35.539347+0000","last_peered":"2024-03-22T04:13:35.539347+0000","last_clean":"2024-03-22T04:13:35.539347+0000","last_became_active":"2024-03-22T04:13:35.534331+0000","last_became_peered":"2024-03-22T04:13:35.534331+0000","last_unstale":"2024-03-22T04:13:35.539347+0000","last_undegraded":"2024-03-22T04:13:35.539347+0000","last_fullsized":"2024-03-22T04:13:35.539347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515262+0000","last_change":"2024-03-22T04:13:35.515262+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515262+0000","last_undegraded":"2024-03-22T04:13:35.515262+0000","last_fullsized":"2024-03-22T04:13:35.515262+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539147+0000","last_change":"2024-03-22T04:13:35.533530+0000","last_active":"2024-03-22T04:13:35.539147+0000","last_peered":"2024-03-22T04:13:35.539147+0000","last_clean":"2024-03-22T04:13:35.539147+0000","last_became_active":"2024-03-22T04:13:35.533254+0000","last_became_peered":"2024-03-22T04:13:35.533254+0000","last_unstale":"2024-03-22T04:13:35.539147+0000","last_undegraded":"2024-03-22T04:13:35.539147+0000","last_fullsized":"2024-03-22T04:13:35.539147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515325+0000","last_change":"2024-03-22T04:13:35.515325+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515325+0000","last_undegraded":"2024-03-22T04:13:35.515325+0000","last_fullsized":"2024-03-22T04:13:35.515325+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537474+0000","last_change":"2024-03-22T04:13:35.530606+0000","last_active":"2024-03-22T04:13:35.537474+0000","last_peered":"2024-03-22T04:13:35.537474+0000","last_clean":"2024-03-22T04:13:35.537474+0000","last_became_active":"2024-03-22T04:13:35.530250+0000","last_became_peered":"2024-03-22T04:13:35.530250+0000","last_unstale":"2024-03-22T04:13:35.537474+0000","last_undegraded":"2024-03-22T04:13:35.537474+0000","last_fullsized":"2024-03-22T04:13:35.537474+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:01:43.146759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518273+0000","last_change":"2024-03-22T04:13:35.518273+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518273+0000","last_undegraded":"2024-03-22T04:13:35.518273+0000","last_fullsized":"2024-03-22T04:13:35.518273+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.541072+0000","last_change":"2024-03-22T04:13:35.539839+0000","last_active":"2024-03-22T04:13:35.541072+0000","last_peered":"2024-03-22T04:13:35.541072+0000","last_clean":"2024-03-22T04:13:35.541072+0000","last_became_active":"2024-03-22T04:13:35.539572+0000","last_became_peered":"2024-03-22T04:13:35.539572+0000","last_unstale":"2024-03-22T04:13:35.541072+0000","last_undegraded":"2024-03-22T04:13:35.541072+0000","last_fullsized":"2024-03-22T04:13:35.541072+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:23.221070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517337+0000","last_change":"2024-03-22T04:13:35.517337+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517337+0000","last_undegraded":"2024-03-22T04:13:35.517337+0000","last_fullsized":"2024-03-22T04:13:35.517337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537532+0000","last_change":"2024-03-22T04:13:35.533632+0000","last_active":"2024-03-22T04:13:35.537532+0000","last_peered":"2024-03-22T04:13:35.537532+0000","last_clean":"2024-03-22T04:13:35.537532+0000","last_became_active":"2024-03-22T04:13:35.533349+0000","last_became_peered":"2024-03-22T04:13:35.533349+0000","last_unstale":"2024-03-22T04:13:35.537532+0000","last_undegraded":"2024-03-22T04:13:35.537532+0000","last_fullsized":"2024-03-22T04:13:35.537532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:23.697686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515728+0000","last_change":"2024-03-22T04:13:35.515728+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515728+0000","last_undegraded":"2024-03-22T04:13:35.515728+0000","last_fullsized":"2024-03-22T04:13:35.515728+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.531700+0000","last_change":"2024-03-22T04:13:35.527224+0000","last_active":"2024-03-22T04:13:35.531700+0000","last_peered":"2024-03-22T04:13:35.531700+0000","last_clean":"2024-03-22T04:13:35.531700+0000","last_became_active":"2024-03-22T04:13:35.526938+0000","last_became_peered":"2024-03-22T04:13:35.526938+0000","last_unstale":"2024-03-22T04:13:35.531700+0000","last_undegraded":"2024-03-22T04:13:35.531700+0000","last_fullsized":"2024-03-22T04:13:35.531700+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:08:49.483327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516973+0000","last_change":"2024-03-22T04:13:35.516973+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516973+0000","last_undegraded":"2024-03-22T04:13:35.516973+0000","last_fullsized":"2024-03-22T04:13:35.516973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536203+0000","last_change":"2024-03-22T04:13:35.530361+0000","last_active":"2024-03-22T04:13:35.536203+0000","last_peered":"2024-03-22T04:13:35.536203+0000","last_clean":"2024-03-22T04:13:35.536203+0000","last_became_active":"2024-03-22T04:13:35.529768+0000","last_became_peered":"2024-03-22T04:13:35.529768+0000","last_unstale":"2024-03-22T04:13:35.536203+0000","last_undegraded":"2024-03-22T04:13:35.536203+0000","last_fullsized":"2024-03-22T04:13:35.536203+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534117+0000","last_change":"2024-03-22T04:13:36.523859+0000","last_active":"2024-03-22T04:13:37.534117+0000","last_peered":"2024-03-22T04:13:37.534117+0000","last_clean":"2024-03-22T04:13:37.534117+0000","last_became_active":"2024-03-22T04:13:36.523598+0000","last_became_peered":"2024-03-22T04:13:36.523598+0000","last_unstale":"2024-03-22T04:13:37.534117+0000","last_undegraded":"2024-03-22T04:13:37.534117+0000","last_fullsized":"2024-03-22T04:13:37.534117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:04.300928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539332+0000","last_change":"2024-03-22T04:13:35.534630+0000","last_active":"2024-03-22T04:13:35.539332+0000","last_peered":"2024-03-22T04:13:35.539332+0000","last_clean":"2024-03-22T04:13:35.539332+0000","last_became_active":"2024-03-22T04:13:35.533284+0000","last_became_peered":"2024-03-22T04:13:35.533284+0000","last_unstale":"2024-03-22T04:13:35.539332+0000","last_undegraded":"2024-03-22T04:13:35.539332+0000","last_fullsized":"2024-03-22T04:13:35.539332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534628+0000","last_change":"2024-03-22T04:13:36.539056+0000","last_active":"2024-03-22T04:13:37.534628+0000","last_peered":"2024-03-22T04:13:37.534628+0000","last_clean":"2024-03-22T04:13:37.534628+0000","last_became_active":"2024-03-22T04:13:36.538782+0000","last_became_peered":"2024-03-22T04:13:36.538782+0000","last_unstale":"2024-03-22T04:13:37.534628+0000","last_undegraded":"2024-03-22T04:13:37.534628+0000","last_fullsized":"2024-03-22T04:13:37.534628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:07.459840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537876+0000","last_change":"2024-03-22T04:13:35.531937+0000","last_active":"2024-03-22T04:13:35.537876+0000","last_peered":"2024-03-22T04:13:35.537876+0000","last_clean":"2024-03-22T04:13:35.537876+0000","last_became_active":"2024-03-22T04:13:35.528718+0000","last_became_peered":"2024-03-22T04:13:35.528718+0000","last_unstale":"2024-03-22T04:13:35.537876+0000","last_undegraded":"2024-03-22T04:13:35.537876+0000","last_fullsized":"2024-03-22T04:13:35.537876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517454+0000","last_change":"2024-03-22T04:13:35.517454+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517454+0000","last_undegraded":"2024-03-22T04:13:35.517454+0000","last_fullsized":"2024-03-22T04:13:35.517454+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536284+0000","last_change":"2024-03-22T04:13:35.530829+0000","last_active":"2024-03-22T04:13:35.536284+0000","last_peered":"2024-03-22T04:13:35.536284+0000","last_clean":"2024-03-22T04:13:35.536284+0000","last_became_active":"2024-03-22T04:13:35.530554+0000","last_became_peered":"2024-03-22T04:13:35.530554+0000","last_unstale":"2024-03-22T04:13:35.536284+0000","last_undegraded":"2024-03-22T04:13:35.536284+0000","last_fullsized":"2024-03-22T04:13:35.536284+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:50.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517863+0000","last_change":"2024-03-22T04:13:35.517863+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517863+0000","last_undegraded":"2024-03-22T04:13:35.517863+0000","last_fullsized":"2024-03-22T04:13:35.517863+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.540268+0000","last_change":"2024-03-22T04:13:35.538794+0000","last_active":"2024-03-22T04:13:35.540268+0000","last_peered":"2024-03-22T04:13:35.540268+0000","last_clean":"2024-03-22T04:13:35.540268+0000","last_became_active":"2024-03-22T04:13:35.538511+0000","last_became_peered":"2024-03-22T04:13:35.538511+0000","last_unstale":"2024-03-22T04:13:35.540268+0000","last_undegraded":"2024-03-22T04:13:35.540268+0000","last_fullsized":"2024-03-22T04:13:35.540268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:00:58.326961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516763+0000","last_change":"2024-03-22T04:13:35.516763+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516763+0000","last_undegraded":"2024-03-22T04:13:35.516763+0000","last_fullsized":"2024-03-22T04:13:35.516763+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.533960+0000","last_change":"2024-03-22T04:13:35.528468+0000","last_active":"2024-03-22T04:13:35.533960+0000","last_peered":"2024-03-22T04:13:35.533960+0000","last_clean":"2024-03-22T04:13:35.533960+0000","last_became_active":"2024-03-22T04:13:35.528113+0000","last_became_peered":"2024-03-22T04:13:35.528113+0000","last_unstale":"2024-03-22T04:13:35.533960+0000","last_undegraded":"2024-03-22T04:13:35.533960+0000","last_fullsized":"2024-03-22T04:13:35.533960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.514570+0000","last_change":"2024-03-22T04:13:35.514570+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.514570+0000","last_undegraded":"2024-03-22T04:13:35.514570+0000","last_fullsized":"2024-03-22T04:13:35.514570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.531318+0000","last_change":"2024-03-22T04:13:35.525890+0000","last_active":"2024-03-22T04:13:35.531318+0000","last_peered":"2024-03-22T04:13:35.531318+0000","last_clean":"2024-03-22T04:13:35.531318+0000","last_became_active":"2024-03-22T04:13:35.525618+0000","last_became_peered":"2024-03-22T04:13:35.525618+0000","last_unstale":"2024-03-22T04:13:35.531318+0000","last_undegraded":"2024-03-22T04:13:35.531318+0000","last_fullsized":"2024-03-22T04:13:35.531318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515001+0000","last_change":"2024-03-22T04:13:35.515001+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515001+0000","last_undegraded":"2024-03-22T04:13:35.515001+0000","last_fullsized":"2024-03-22T04:13:35.515001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539898+0000","last_change":"2024-03-22T04:13:35.537483+0000","last_active":"2024-03-22T04:13:35.539898+0000","last_peered":"2024-03-22T04:13:35.539898+0000","last_clean":"2024-03-22T04:13:35.539898+0000","last_became_active":"2024-03-22T04:13:35.537070+0000","last_became_peered":"2024-03-22T04:13:35.537070+0000","last_unstale":"2024-03-22T04:13:35.539898+0000","last_undegraded":"2024-03-22T04:13:35.539898+0000","last_fullsized":"2024-03-22T04:13:35.539898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534302+0000","last_change":"2024-03-22T04:13:36.535142+0000","last_active":"2024-03-22T04:13:37.534302+0000","last_peered":"2024-03-22T04:13:37.534302+0000","last_clean":"2024-03-22T04:13:37.534302+0000","last_became_active":"2024-03-22T04:13:36.533918+0000","last_became_peered":"2024-03-22T04:13:36.533918+0000","last_unstale":"2024-03-22T04:13:37.534302+0000","last_undegraded":"2024-03-22T04:13:37.534302+0000","last_fullsized":"2024-03-22T04:13:37.534302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:15.073523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537406+0000","last_change":"2024-03-22T04:13:35.533317+0000","last_active":"2024-03-22T04:13:35.537406+0000","last_peered":"2024-03-22T04:13:35.537406+0000","last_clean":"2024-03-22T04:13:35.537406+0000","last_became_active":"2024-03-22T04:13:35.532424+0000","last_became_peered":"2024-03-22T04:13:35.532424+0000","last_unstale":"2024-03-22T04:13:35.537406+0000","last_undegraded":"2024-03-22T04:13:35.537406+0000","last_fullsized":"2024-03-22T04:13:35.537406+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:40:06.790699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516052+0000","last_change":"2024-03-22T04:13:35.516052+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516052+0000","last_undegraded":"2024-03-22T04:13:35.516052+0000","last_fullsized":"2024-03-22T04:13:35.516052+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.534324+0000","last_change":"2024-03-22T04:13:35.528988+0000","last_active":"2024-03-22T04:13:35.534324+0000","last_peered":"2024-03-22T04:13:35.534324+0000","last_clean":"2024-03-22T04:13:35.534324+0000","last_became_active":"2024-03-22T04:13:35.528529+0000","last_became_peered":"2024-03-22T04:13:35.528529+0000","last_unstale":"2024-03-22T04:13:35.534324+0000","last_undegraded":"2024-03-22T04:13:35.534324+0000","last_fullsized":"2024-03-22T04:13:35.534324+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":247,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.517988+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:13:35.517988+0000","last_peered":"2024-03-22T04:13:35.517988+0000","last_clean":"2024-03-22T04:13:35.517988+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:13:35.517988+0000","last_undegraded":"2024-03-22T04:13:35.517988+0000","last_fullsized":"2024-03-22T04:13:35.517988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":778240,"data_stored":346892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":937260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":346892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":930593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":361333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:13:40.605 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:13:40.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f46a00640 1 Processor -- start 2024-03-22T04:13:40.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f46a00640 1 -- start start 2024-03-22T04:13:40.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f46a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5f400aff50 0x7f5f4006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f46a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5f401444f0 0x7f5f4015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f46a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f400ae5a0 con 0x7f5f4006cbb0 2024-03-22T04:13:40.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f46a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5f4006d010 con 0x7f5f400aff50 2024-03-22T04:13:40.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f46a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5f40144940 con 0x7f5f401444f0 2024-03-22T04:13:40.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f3fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5f400aff50 0x7f5f4006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f3fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5f400aff50 0x7f5f4006c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45882/0 (socket says 172.21.15.86:45882) 2024-03-22T04:13:40.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f3fe00640 1 -- 172.21.15.86:0/3725712894 learned_addr learned my addr 172.21.15.86:0/3725712894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:40.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f44a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5f401444f0 0x7f5f4015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f3f400640 1 --1- 172.21.15.86:0/3725712894 >> v1:172.21.15.196:6789/0 conn(0x7f5f4006cbb0 0x7f5f40143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:44892/0 (socket says 172.21.15.86:44892) 2024-03-22T04:13:40.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f3fe00640 1 -- 172.21.15.86:0/3725712894 >> v2:172.21.15.196:3300/0 conn(0x7f5f401444f0 msgr2=0x7f5f4015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:40.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f3fe00640 1 --2- 172.21.15.86:0/3725712894 >> v2:172.21.15.196:3300/0 conn(0x7f5f401444f0 0x7f5f4015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f3fe00640 1 -- 172.21.15.86:0/3725712894 >> v1:172.21.15.196:6789/0 conn(0x7f5f4006cbb0 legacy=0x7f5f40143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.772+0000 7f5f3fe00640 1 -- 172.21.15.86:0/3725712894 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f40144b80 con 0x7f5f400aff50 2024-03-22T04:13:40.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f3fe00640 1 --2- 172.21.15.86:0/3725712894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f400aff50 0x7f5f4006c670 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5f400702c0 tx=0x7f5f3001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df494e730e79ced server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f3ea00640 1 -- 172.21.15.86:0/3725712894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300b2070 con 0x7f5f400aff50 2024-03-22T04:13:40.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f3ea00640 1 -- 172.21.15.86:0/3725712894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f300ad680 con 0x7f5f400aff50 2024-03-22T04:13:40.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f3ea00640 1 -- 172.21.15.86:0/3725712894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300ada50 con 0x7f5f400aff50 2024-03-22T04:13:40.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f46a00640 1 -- 172.21.15.86:0/3725712894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f400aff50 msgr2=0x7f5f4006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f46a00640 1 --2- 172.21.15.86:0/3725712894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f400aff50 0x7f5f4006c670 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5f400702c0 tx=0x7f5f3001c490 comp rx=0 tx=0).stop 2024-03-22T04:13:40.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f46a00640 1 -- 172.21.15.86:0/3725712894 shutdown_connections 2024-03-22T04:13:40.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f46a00640 1 --2- 172.21.15.86:0/3725712894 >> v2:172.21.15.196:3300/0 conn(0x7f5f401444f0 0x7f5f4015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f46a00640 1 --2- 172.21.15.86:0/3725712894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f400aff50 0x7f5f4006c670 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f46a00640 1 -- 172.21.15.86:0/3725712894 >> 172.21.15.86:0/3725712894 conn(0x7f5f40071fe0 msgr2=0x7f5f400723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:40.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f46a00640 1 -- 172.21.15.86:0/3725712894 shutdown_connections 2024-03-22T04:13:40.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.776+0000 7f5f46a00640 1 -- 172.21.15.86:0/3725712894 wait complete. 2024-03-22T04:13:40.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 Processor -- start 2024-03-22T04:13:40.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 -- start start 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f4006cbb0 0x7f5f400b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f400aff50 0x7f5f4013bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f401444f0 0x7f5f40140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f40162f30 con 0x7f5f401444f0 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f400b0330 con 0x7f5f400aff50 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5f401f78e0 con 0x7f5f4006cbb0 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f44a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f401444f0 0x7f5f40140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f3fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f4006cbb0 0x7f5f400b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f3fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f4006cbb0 0x7f5f400b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:43834/0 (socket says 172.21.15.86:43834) 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f3f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f400aff50 0x7f5f4013bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f3fe00640 1 -- 172.21.15.86:0/2932176612 learned_addr learned my addr 172.21.15.86:0/2932176612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:40.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f44a00640 1 -- 172.21.15.86:0/2932176612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f4006cbb0 msgr2=0x7f5f400b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f44a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f4006cbb0 0x7f5f400b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f44a00640 1 -- 172.21.15.86:0/2932176612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f400aff50 msgr2=0x7f5f4013bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f44a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f400aff50 0x7f5f4013bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f44a00640 1 -- 172.21.15.86:0/2932176612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f400b5f30 con 0x7f5f401444f0 2024-03-22T04:13:40.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f44a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f401444f0 0x7f5f40140e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800eba0 tx=0x7f5f3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f37e00640 1 -- 172.21.15.86:0/2932176612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38018020 con 0x7f5f401444f0 2024-03-22T04:13:40.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f37e00640 1 -- 172.21.15.86:0/2932176612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f3800cd20 con 0x7f5f401444f0 2024-03-22T04:13:40.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f400b6210 con 0x7f5f401444f0 2024-03-22T04:13:40.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f37e00640 1 -- 172.21.15.86:0/2932176612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380146c0 con 0x7f5f401444f0 2024-03-22T04:13:40.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.780+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4013c0b0 con 0x7f5f401444f0 2024-03-22T04:13:40.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.784+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f40068650 con 0x7f5f401444f0 2024-03-22T04:13:40.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.784+0000 7f5f37e00640 1 -- 172.21.15.86:0/2932176612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5f3801b020 con 0x7f5f401444f0 2024-03-22T04:13:40.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.784+0000 7f5f37e00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5f20067b50 0x7f5f2006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:40.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.784+0000 7f5f37e00640 1 -- 172.21.15.86:0/2932176612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f5f38086bf0 con 0x7f5f401444f0 2024-03-22T04:13:40.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.784+0000 7f5f3fe00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5f20067b50 0x7f5f2006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:40.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.784+0000 7f5f3fe00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5f20067b50 0x7f5f2006a010 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000c10 tx=0x7f5f300a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.788+0000 7f5f37e00640 1 -- 172.21.15.86:0/2932176612 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5f38022030 con 0x7f5f401444f0 2024-03-22T04:13:40.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.940+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f40158ca0 con 0x7f5f20067b50 2024-03-22T04:13:40.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f37e00640 1 -- 172.21.15.86:0/2932176612 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247373 (secure 0 0 0) 0x7f5f40158ca0 con 0x7f5f20067b50 2024-03-22T04:13:40.953 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:40.954 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:13:40.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5f20067b50 msgr2=0x7f5f2006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5f20067b50 0x7f5f2006a010 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000c10 tx=0x7f5f300a7000 comp rx=0 tx=0).stop 2024-03-22T04:13:40.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f401444f0 msgr2=0x7f5f40140e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f401444f0 0x7f5f40140e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800eba0 tx=0x7f5f3800c490 comp rx=0 tx=0).stop 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 shutdown_connections 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5f20067b50 0x7f5f2006a010 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f400aff50 0x7f5f4013bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f401444f0 0x7f5f40140e20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 --2- 172.21.15.86:0/2932176612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f4006cbb0 0x7f5f400b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 >> 172.21.15.86:0/2932176612 conn(0x7f5f40071fe0 msgr2=0x7f5f4006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.948+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 shutdown_connections 2024-03-22T04:13:40.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:40.952+0000 7f5f46a00640 1 -- 172.21.15.86:0/2932176612 wait complete. 2024-03-22T04:13:40.973 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-22T04:13:39.830360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227568,"kb_used_data":7220,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747152,"statfs":{"total":773094113280,"available":772861083648,"internally_reserved":0,"allocated":7393280,"data_stored":3943645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.326188"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521460+0000","last_change":"2024-03-22T04:13:35.521460+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521460+0000","last_undegraded":"2024-03-22T04:13:35.521460+0000","last_fullsized":"2024-03-22T04:13:35.521460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515543+0000","last_change":"2024-03-22T04:13:35.515543+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515543+0000","last_undegraded":"2024-03-22T04:13:35.515543+0000","last_fullsized":"2024-03-22T04:13:35.515543+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521592+0000","last_change":"2024-03-22T04:13:35.521592+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521592+0000","last_undegraded":"2024-03-22T04:13:35.521592+0000","last_fullsized":"2024-03-22T04:13:35.521592+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.523111+0000","last_change":"2024-03-22T04:13:35.523111+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.523111+0000","last_undegraded":"2024-03-22T04:13:35.523111+0000","last_fullsized":"2024-03-22T04:13:35.523111+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521553+0000","last_change":"2024-03-22T04:13:35.521553+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521553+0000","last_undegraded":"2024-03-22T04:13:35.521553+0000","last_fullsized":"2024-03-22T04:13:35.521553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515499+0000","last_change":"2024-03-22T04:13:35.515499+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515499+0000","last_undegraded":"2024-03-22T04:13:35.515499+0000","last_fullsized":"2024-03-22T04:13:35.515499+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522743+0000","last_change":"2024-03-22T04:13:35.522743+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522743+0000","last_undegraded":"2024-03-22T04:13:35.522743+0000","last_fullsized":"2024-03-22T04:13:35.522743+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522009+0000","last_change":"2024-03-22T04:13:35.522009+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522009+0000","last_undegraded":"2024-03-22T04:13:35.522009+0000","last_fullsized":"2024-03-22T04:13:35.522009+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519953+0000","last_change":"2024-03-22T04:13:35.519953+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519953+0000","last_undegraded":"2024-03-22T04:13:35.519953+0000","last_fullsized":"2024-03-22T04:13:35.519953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517617+0000","last_change":"2024-03-22T04:13:35.517617+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517617+0000","last_undegraded":"2024-03-22T04:13:35.517617+0000","last_fullsized":"2024-03-22T04:13:35.517617+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521802+0000","last_change":"2024-03-22T04:13:35.521802+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521802+0000","last_undegraded":"2024-03-22T04:13:35.521802+0000","last_fullsized":"2024-03-22T04:13:35.521802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522711+0000","last_change":"2024-03-22T04:13:35.522711+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522711+0000","last_undegraded":"2024-03-22T04:13:35.522711+0000","last_fullsized":"2024-03-22T04:13:35.522711+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518599+0000","last_change":"2024-03-22T04:13:35.518599+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518599+0000","last_undegraded":"2024-03-22T04:13:35.518599+0000","last_fullsized":"2024-03-22T04:13:35.518599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519660+0000","last_change":"2024-03-22T04:13:35.519660+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519660+0000","last_undegraded":"2024-03-22T04:13:35.519660+0000","last_fullsized":"2024-03-22T04:13:35.519660+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516498+0000","last_change":"2024-03-22T04:13:35.516498+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516498+0000","last_undegraded":"2024-03-22T04:13:35.516498+0000","last_fullsized":"2024-03-22T04:13:35.516498+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518341+0000","last_change":"2024-03-22T04:13:35.518341+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518341+0000","last_undegraded":"2024-03-22T04:13:35.518341+0000","last_fullsized":"2024-03-22T04:13:35.518341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534709+0000","last_change":"2024-03-22T04:13:36.534928+0000","last_active":"2024-03-22T04:13:37.534709+0000","last_peered":"2024-03-22T04:13:37.534709+0000","last_clean":"2024-03-22T04:13:37.534709+0000","last_became_active":"2024-03-22T04:13:36.534458+0000","last_became_peered":"2024-03-22T04:13:36.534458+0000","last_unstale":"2024-03-22T04:13:37.534709+0000","last_undegraded":"2024-03-22T04:13:37.534709+0000","last_fullsized":"2024-03-22T04:13:37.534709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:10:28.784356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518861+0000","last_change":"2024-03-22T04:13:35.518861+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518861+0000","last_undegraded":"2024-03-22T04:13:35.518861+0000","last_fullsized":"2024-03-22T04:13:35.518861+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521381+0000","last_change":"2024-03-22T04:13:35.521381+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521381+0000","last_undegraded":"2024-03-22T04:13:35.521381+0000","last_fullsized":"2024-03-22T04:13:35.521381+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534466+0000","last_change":"2024-03-22T04:13:36.533555+0000","last_active":"2024-03-22T04:13:37.534466+0000","last_peered":"2024-03-22T04:13:37.534466+0000","last_clean":"2024-03-22T04:13:37.534466+0000","last_became_active":"2024-03-22T04:13:36.533261+0000","last_became_peered":"2024-03-22T04:13:36.533261+0000","last_unstale":"2024-03-22T04:13:37.534466+0000","last_undegraded":"2024-03-22T04:13:37.534466+0000","last_fullsized":"2024-03-22T04:13:37.534466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:22.636235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518186+0000","last_change":"2024-03-22T04:13:35.518186+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518186+0000","last_undegraded":"2024-03-22T04:13:35.518186+0000","last_fullsized":"2024-03-22T04:13:35.518186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.514455+0000","last_change":"2024-03-22T04:13:35.514455+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.514455+0000","last_undegraded":"2024-03-22T04:13:35.514455+0000","last_fullsized":"2024-03-22T04:13:35.514455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517884+0000","last_change":"2024-03-22T04:13:35.517884+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517884+0000","last_undegraded":"2024-03-22T04:13:35.517884+0000","last_fullsized":"2024-03-22T04:13:35.517884+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520978+0000","last_change":"2024-03-22T04:13:35.520978+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520978+0000","last_undegraded":"2024-03-22T04:13:35.520978+0000","last_fullsized":"2024-03-22T04:13:35.520978+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517424+0000","last_change":"2024-03-22T04:13:35.517424+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517424+0000","last_undegraded":"2024-03-22T04:13:35.517424+0000","last_fullsized":"2024-03-22T04:13:35.517424+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.526133+0000","last_change":"2024-03-22T04:13:35.526133+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.526133+0000","last_undegraded":"2024-03-22T04:13:35.526133+0000","last_fullsized":"2024-03-22T04:13:35.526133+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517842+0000","last_change":"2024-03-22T04:13:35.517842+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517842+0000","last_undegraded":"2024-03-22T04:13:35.517842+0000","last_fullsized":"2024-03-22T04:13:35.517842+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521298+0000","last_change":"2024-03-22T04:13:35.521298+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521298+0000","last_undegraded":"2024-03-22T04:13:35.521298+0000","last_fullsized":"2024-03-22T04:13:35.521298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518277+0000","last_change":"2024-03-22T04:13:35.518277+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518277+0000","last_undegraded":"2024-03-22T04:13:35.518277+0000","last_fullsized":"2024-03-22T04:13:35.518277+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517462+0000","last_change":"2024-03-22T04:13:35.517462+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517462+0000","last_undegraded":"2024-03-22T04:13:35.517462+0000","last_fullsized":"2024-03-22T04:13:35.517462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520275+0000","last_change":"2024-03-22T04:13:35.520275+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520275+0000","last_undegraded":"2024-03-22T04:13:35.520275+0000","last_fullsized":"2024-03-22T04:13:35.520275+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520777+0000","last_change":"2024-03-22T04:13:35.520777+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520777+0000","last_undegraded":"2024-03-22T04:13:35.520777+0000","last_fullsized":"2024-03-22T04:13:35.520777+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522255+0000","last_change":"2024-03-22T04:13:35.522255+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522255+0000","last_undegraded":"2024-03-22T04:13:35.522255+0000","last_fullsized":"2024-03-22T04:13:35.522255+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536950+0000","last_change":"2024-03-22T04:13:35.531755+0000","last_active":"2024-03-22T04:13:35.536950+0000","last_peered":"2024-03-22T04:13:35.536950+0000","last_clean":"2024-03-22T04:13:35.536950+0000","last_became_active":"2024-03-22T04:13:35.531433+0000","last_became_peered":"2024-03-22T04:13:35.531433+0000","last_unstale":"2024-03-22T04:13:35.536950+0000","last_undegraded":"2024-03-22T04:13:35.536950+0000","last_fullsized":"2024-03-22T04:13:35.536950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516557+0000","last_change":"2024-03-22T04:13:35.516557+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516557+0000","last_undegraded":"2024-03-22T04:13:35.516557+0000","last_fullsized":"2024-03-22T04:13:35.516557+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.535991+0000","last_change":"2024-03-22T04:13:35.529846+0000","last_active":"2024-03-22T04:13:35.535991+0000","last_peered":"2024-03-22T04:13:35.535991+0000","last_clean":"2024-03-22T04:13:35.535991+0000","last_became_active":"2024-03-22T04:13:35.529546+0000","last_became_peered":"2024-03-22T04:13:35.529546+0000","last_unstale":"2024-03-22T04:13:35.535991+0000","last_undegraded":"2024-03-22T04:13:35.535991+0000","last_fullsized":"2024-03-22T04:13:35.535991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515064+0000","last_change":"2024-03-22T04:13:35.515064+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515064+0000","last_undegraded":"2024-03-22T04:13:35.515064+0000","last_fullsized":"2024-03-22T04:13:35.515064+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536333+0000","last_change":"2024-03-22T04:13:35.530455+0000","last_active":"2024-03-22T04:13:35.536333+0000","last_peered":"2024-03-22T04:13:35.536333+0000","last_clean":"2024-03-22T04:13:35.536333+0000","last_became_active":"2024-03-22T04:13:35.530130+0000","last_became_peered":"2024-03-22T04:13:35.530130+0000","last_unstale":"2024-03-22T04:13:35.536333+0000","last_undegraded":"2024-03-22T04:13:35.536333+0000","last_fullsized":"2024-03-22T04:13:35.536333+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519480+0000","last_change":"2024-03-22T04:13:35.519480+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519480+0000","last_undegraded":"2024-03-22T04:13:35.519480+0000","last_fullsized":"2024-03-22T04:13:35.519480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537702+0000","last_change":"2024-03-22T04:13:35.533152+0000","last_active":"2024-03-22T04:13:35.537702+0000","last_peered":"2024-03-22T04:13:35.537702+0000","last_clean":"2024-03-22T04:13:35.537702+0000","last_became_active":"2024-03-22T04:13:35.532739+0000","last_became_peered":"2024-03-22T04:13:35.532739+0000","last_unstale":"2024-03-22T04:13:35.537702+0000","last_undegraded":"2024-03-22T04:13:35.537702+0000","last_fullsized":"2024-03-22T04:13:35.537702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:11:03.089640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533547+0000","last_change":"2024-03-22T04:13:36.539261+0000","last_active":"2024-03-22T04:13:37.533547+0000","last_peered":"2024-03-22T04:13:37.533547+0000","last_clean":"2024-03-22T04:13:37.533547+0000","last_became_active":"2024-03-22T04:13:36.538948+0000","last_became_peered":"2024-03-22T04:13:36.538948+0000","last_unstale":"2024-03-22T04:13:37.533547+0000","last_undegraded":"2024-03-22T04:13:37.533547+0000","last_fullsized":"2024-03-22T04:13:37.533547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.370928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534342+0000","last_change":"2024-03-22T04:13:35.528073+0000","last_active":"2024-03-22T04:13:37.534342+0000","last_peered":"2024-03-22T04:13:37.534342+0000","last_clean":"2024-03-22T04:13:37.534342+0000","last_became_active":"2024-03-22T04:13:35.527410+0000","last_became_peered":"2024-03-22T04:13:35.527410+0000","last_unstale":"2024-03-22T04:13:37.534342+0000","last_undegraded":"2024-03-22T04:13:37.534342+0000","last_fullsized":"2024-03-22T04:13:37.534342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:00:59.588177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522326+0000","last_change":"2024-03-22T04:13:35.522326+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522326+0000","last_undegraded":"2024-03-22T04:13:35.522326+0000","last_fullsized":"2024-03-22T04:13:35.522326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.534280+0000","last_change":"2024-03-22T04:13:35.526516+0000","last_active":"2024-03-22T04:13:35.534280+0000","last_peered":"2024-03-22T04:13:35.534280+0000","last_clean":"2024-03-22T04:13:35.534280+0000","last_became_active":"2024-03-22T04:13:35.526208+0000","last_became_peered":"2024-03-22T04:13:35.526208+0000","last_unstale":"2024-03-22T04:13:35.534280+0000","last_undegraded":"2024-03-22T04:13:35.534280+0000","last_fullsized":"2024-03-22T04:13:35.534280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516439+0000","last_change":"2024-03-22T04:13:35.516439+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516439+0000","last_undegraded":"2024-03-22T04:13:35.516439+0000","last_fullsized":"2024-03-22T04:13:35.516439+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537342+0000","last_change":"2024-03-22T04:13:35.534172+0000","last_active":"2024-03-22T04:13:35.537342+0000","last_peered":"2024-03-22T04:13:35.537342+0000","last_clean":"2024-03-22T04:13:35.537342+0000","last_became_active":"2024-03-22T04:13:35.533651+0000","last_became_peered":"2024-03-22T04:13:35.533651+0000","last_unstale":"2024-03-22T04:13:35.537342+0000","last_undegraded":"2024-03-22T04:13:35.537342+0000","last_fullsized":"2024-03-22T04:13:35.537342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534186+0000","last_change":"2024-03-22T04:13:36.535993+0000","last_active":"2024-03-22T04:13:37.534186+0000","last_peered":"2024-03-22T04:13:37.534186+0000","last_clean":"2024-03-22T04:13:37.534186+0000","last_became_active":"2024-03-22T04:13:36.534295+0000","last_became_peered":"2024-03-22T04:13:36.534295+0000","last_unstale":"2024-03-22T04:13:37.534186+0000","last_undegraded":"2024-03-22T04:13:37.534186+0000","last_fullsized":"2024-03-22T04:13:37.534186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:58.215479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539156+0000","last_change":"2024-03-22T04:13:35.533417+0000","last_active":"2024-03-22T04:13:35.539156+0000","last_peered":"2024-03-22T04:13:35.539156+0000","last_clean":"2024-03-22T04:13:35.539156+0000","last_became_active":"2024-03-22T04:13:35.533053+0000","last_became_peered":"2024-03-22T04:13:35.533053+0000","last_unstale":"2024-03-22T04:13:35.539156+0000","last_undegraded":"2024-03-22T04:13:35.539156+0000","last_fullsized":"2024-03-22T04:13:35.539156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519164+0000","last_change":"2024-03-22T04:13:35.519164+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519164+0000","last_undegraded":"2024-03-22T04:13:35.519164+0000","last_fullsized":"2024-03-22T04:13:35.519164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539366+0000","last_change":"2024-03-22T04:13:35.535058+0000","last_active":"2024-03-22T04:13:35.539366+0000","last_peered":"2024-03-22T04:13:35.539366+0000","last_clean":"2024-03-22T04:13:35.539366+0000","last_became_active":"2024-03-22T04:13:35.534345+0000","last_became_peered":"2024-03-22T04:13:35.534345+0000","last_unstale":"2024-03-22T04:13:35.539366+0000","last_undegraded":"2024-03-22T04:13:35.539366+0000","last_fullsized":"2024-03-22T04:13:35.539366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:50.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534393+0000","last_change":"2024-03-22T04:13:36.527089+0000","last_active":"2024-03-22T04:13:37.534393+0000","last_peered":"2024-03-22T04:13:37.534393+0000","last_clean":"2024-03-22T04:13:37.534393+0000","last_became_active":"2024-03-22T04:13:36.526807+0000","last_became_peered":"2024-03-22T04:13:36.526807+0000","last_unstale":"2024-03-22T04:13:37.534393+0000","last_undegraded":"2024-03-22T04:13:37.534393+0000","last_fullsized":"2024-03-22T04:13:37.534393+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:57:11.984612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.529461+0000","last_change":"2024-03-22T04:13:35.525257+0000","last_active":"2024-03-22T04:13:35.529461+0000","last_peered":"2024-03-22T04:13:35.529461+0000","last_clean":"2024-03-22T04:13:35.529461+0000","last_became_active":"2024-03-22T04:13:35.524936+0000","last_became_peered":"2024-03-22T04:13:35.524936+0000","last_unstale":"2024-03-22T04:13:35.529461+0000","last_undegraded":"2024-03-22T04:13:35.529461+0000","last_fullsized":"2024-03-22T04:13:35.529461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:18:55.065379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515798+0000","last_change":"2024-03-22T04:13:35.515798+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515798+0000","last_undegraded":"2024-03-22T04:13:35.515798+0000","last_fullsized":"2024-03-22T04:13:35.515798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.538958+0000","last_change":"2024-03-22T04:13:35.536397+0000","last_active":"2024-03-22T04:13:35.538958+0000","last_peered":"2024-03-22T04:13:35.538958+0000","last_clean":"2024-03-22T04:13:35.538958+0000","last_became_active":"2024-03-22T04:13:35.536137+0000","last_became_peered":"2024-03-22T04:13:35.536137+0000","last_unstale":"2024-03-22T04:13:35.538958+0000","last_undegraded":"2024-03-22T04:13:35.538958+0000","last_fullsized":"2024-03-22T04:13:35.538958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:05:15.541959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518642+0000","last_change":"2024-03-22T04:13:35.518642+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518642+0000","last_undegraded":"2024-03-22T04:13:35.518642+0000","last_fullsized":"2024-03-22T04:13:35.518642+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536050+0000","last_change":"2024-03-22T04:13:35.529973+0000","last_active":"2024-03-22T04:13:35.536050+0000","last_peered":"2024-03-22T04:13:35.536050+0000","last_clean":"2024-03-22T04:13:35.536050+0000","last_became_active":"2024-03-22T04:13:35.529696+0000","last_became_peered":"2024-03-22T04:13:35.529696+0000","last_unstale":"2024-03-22T04:13:35.536050+0000","last_undegraded":"2024-03-22T04:13:35.536050+0000","last_fullsized":"2024-03-22T04:13:35.536050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515862+0000","last_change":"2024-03-22T04:13:35.515862+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515862+0000","last_undegraded":"2024-03-22T04:13:35.515862+0000","last_fullsized":"2024-03-22T04:13:35.515862+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.533045+0000","last_change":"2024-03-22T04:13:35.529378+0000","last_active":"2024-03-22T04:13:35.533045+0000","last_peered":"2024-03-22T04:13:35.533045+0000","last_clean":"2024-03-22T04:13:35.533045+0000","last_became_active":"2024-03-22T04:13:35.529096+0000","last_became_peered":"2024-03-22T04:13:35.529096+0000","last_unstale":"2024-03-22T04:13:35.533045+0000","last_undegraded":"2024-03-22T04:13:35.533045+0000","last_fullsized":"2024-03-22T04:13:35.533045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:25:37.720655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516023+0000","last_change":"2024-03-22T04:13:35.516023+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516023+0000","last_undegraded":"2024-03-22T04:13:35.516023+0000","last_fullsized":"2024-03-22T04:13:35.516023+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.538455+0000","last_change":"2024-03-22T04:13:35.536795+0000","last_active":"2024-03-22T04:13:35.538455+0000","last_peered":"2024-03-22T04:13:35.538455+0000","last_clean":"2024-03-22T04:13:35.538455+0000","last_became_active":"2024-03-22T04:13:35.534453+0000","last_became_peered":"2024-03-22T04:13:35.534453+0000","last_unstale":"2024-03-22T04:13:35.538455+0000","last_undegraded":"2024-03-22T04:13:35.538455+0000","last_fullsized":"2024-03-22T04:13:35.538455+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516963+0000","last_change":"2024-03-22T04:13:35.516963+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516963+0000","last_undegraded":"2024-03-22T04:13:35.516963+0000","last_fullsized":"2024-03-22T04:13:35.516963+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537398+0000","last_change":"2024-03-22T04:13:35.529571+0000","last_active":"2024-03-22T04:13:35.537398+0000","last_peered":"2024-03-22T04:13:35.537398+0000","last_clean":"2024-03-22T04:13:35.537398+0000","last_became_active":"2024-03-22T04:13:35.528385+0000","last_became_peered":"2024-03-22T04:13:35.528385+0000","last_unstale":"2024-03-22T04:13:35.537398+0000","last_undegraded":"2024-03-22T04:13:35.537398+0000","last_fullsized":"2024-03-22T04:13:35.537398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:55:19.501433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519326+0000","last_change":"2024-03-22T04:13:35.519326+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519326+0000","last_undegraded":"2024-03-22T04:13:35.519326+0000","last_fullsized":"2024-03-22T04:13:35.519326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537868+0000","last_change":"2024-03-22T04:13:35.536315+0000","last_active":"2024-03-22T04:13:35.537868+0000","last_peered":"2024-03-22T04:13:35.537868+0000","last_clean":"2024-03-22T04:13:35.537868+0000","last_became_active":"2024-03-22T04:13:35.535908+0000","last_became_peered":"2024-03-22T04:13:35.535908+0000","last_unstale":"2024-03-22T04:13:35.537868+0000","last_undegraded":"2024-03-22T04:13:35.537868+0000","last_fullsized":"2024-03-22T04:13:35.537868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:18:55.065379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517925+0000","last_change":"2024-03-22T04:13:35.517925+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517925+0000","last_undegraded":"2024-03-22T04:13:35.517925+0000","last_fullsized":"2024-03-22T04:13:35.517925+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539347+0000","last_change":"2024-03-22T04:13:35.534854+0000","last_active":"2024-03-22T04:13:35.539347+0000","last_peered":"2024-03-22T04:13:35.539347+0000","last_clean":"2024-03-22T04:13:35.539347+0000","last_became_active":"2024-03-22T04:13:35.534331+0000","last_became_peered":"2024-03-22T04:13:35.534331+0000","last_unstale":"2024-03-22T04:13:35.539347+0000","last_undegraded":"2024-03-22T04:13:35.539347+0000","last_fullsized":"2024-03-22T04:13:35.539347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515262+0000","last_change":"2024-03-22T04:13:35.515262+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515262+0000","last_undegraded":"2024-03-22T04:13:35.515262+0000","last_fullsized":"2024-03-22T04:13:35.515262+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539147+0000","last_change":"2024-03-22T04:13:35.533530+0000","last_active":"2024-03-22T04:13:35.539147+0000","last_peered":"2024-03-22T04:13:35.539147+0000","last_clean":"2024-03-22T04:13:35.539147+0000","last_became_active":"2024-03-22T04:13:35.533254+0000","last_became_peered":"2024-03-22T04:13:35.533254+0000","last_unstale":"2024-03-22T04:13:35.539147+0000","last_undegraded":"2024-03-22T04:13:35.539147+0000","last_fullsized":"2024-03-22T04:13:35.539147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515325+0000","last_change":"2024-03-22T04:13:35.515325+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515325+0000","last_undegraded":"2024-03-22T04:13:35.515325+0000","last_fullsized":"2024-03-22T04:13:35.515325+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537474+0000","last_change":"2024-03-22T04:13:35.530606+0000","last_active":"2024-03-22T04:13:35.537474+0000","last_peered":"2024-03-22T04:13:35.537474+0000","last_clean":"2024-03-22T04:13:35.537474+0000","last_became_active":"2024-03-22T04:13:35.530250+0000","last_became_peered":"2024-03-22T04:13:35.530250+0000","last_unstale":"2024-03-22T04:13:35.537474+0000","last_undegraded":"2024-03-22T04:13:35.537474+0000","last_fullsized":"2024-03-22T04:13:35.537474+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:01:43.146759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518273+0000","last_change":"2024-03-22T04:13:35.518273+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518273+0000","last_undegraded":"2024-03-22T04:13:35.518273+0000","last_fullsized":"2024-03-22T04:13:35.518273+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.541072+0000","last_change":"2024-03-22T04:13:35.539839+0000","last_active":"2024-03-22T04:13:35.541072+0000","last_peered":"2024-03-22T04:13:35.541072+0000","last_clean":"2024-03-22T04:13:35.541072+0000","last_became_active":"2024-03-22T04:13:35.539572+0000","last_became_peered":"2024-03-22T04:13:35.539572+0000","last_unstale":"2024-03-22T04:13:35.541072+0000","last_undegraded":"2024-03-22T04:13:35.541072+0000","last_fullsized":"2024-03-22T04:13:35.541072+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:23.221070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517337+0000","last_change":"2024-03-22T04:13:35.517337+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517337+0000","last_undegraded":"2024-03-22T04:13:35.517337+0000","last_fullsized":"2024-03-22T04:13:35.517337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537532+0000","last_change":"2024-03-22T04:13:35.533632+0000","last_active":"2024-03-22T04:13:35.537532+0000","last_peered":"2024-03-22T04:13:35.537532+0000","last_clean":"2024-03-22T04:13:35.537532+0000","last_became_active":"2024-03-22T04:13:35.533349+0000","last_became_peered":"2024-03-22T04:13:35.533349+0000","last_unstale":"2024-03-22T04:13:35.537532+0000","last_undegraded":"2024-03-22T04:13:35.537532+0000","last_fullsized":"2024-03-22T04:13:35.537532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:23.697686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515728+0000","last_change":"2024-03-22T04:13:35.515728+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515728+0000","last_undegraded":"2024-03-22T04:13:35.515728+0000","last_fullsized":"2024-03-22T04:13:35.515728+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.531700+0000","last_change":"2024-03-22T04:13:35.527224+0000","last_active":"2024-03-22T04:13:35.531700+0000","last_peered":"2024-03-22T04:13:35.531700+0000","last_clean":"2024-03-22T04:13:35.531700+0000","last_became_active":"2024-03-22T04:13:35.526938+0000","last_became_peered":"2024-03-22T04:13:35.526938+0000","last_unstale":"2024-03-22T04:13:35.531700+0000","last_undegraded":"2024-03-22T04:13:35.531700+0000","last_fullsized":"2024-03-22T04:13:35.531700+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:08:49.483327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516973+0000","last_change":"2024-03-22T04:13:35.516973+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516973+0000","last_undegraded":"2024-03-22T04:13:35.516973+0000","last_fullsized":"2024-03-22T04:13:35.516973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536203+0000","last_change":"2024-03-22T04:13:35.530361+0000","last_active":"2024-03-22T04:13:35.536203+0000","last_peered":"2024-03-22T04:13:35.536203+0000","last_clean":"2024-03-22T04:13:35.536203+0000","last_became_active":"2024-03-22T04:13:35.529768+0000","last_became_peered":"2024-03-22T04:13:35.529768+0000","last_unstale":"2024-03-22T04:13:35.536203+0000","last_undegraded":"2024-03-22T04:13:35.536203+0000","last_fullsized":"2024-03-22T04:13:35.536203+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534117+0000","last_change":"2024-03-22T04:13:36.523859+0000","last_active":"2024-03-22T04:13:37.534117+0000","last_peered":"2024-03-22T04:13:37.534117+0000","last_clean":"2024-03-22T04:13:37.534117+0000","last_became_active":"2024-03-22T04:13:36.523598+0000","last_became_peered":"2024-03-22T04:13:36.523598+0000","last_unstale":"2024-03-22T04:13:37.534117+0000","last_undegraded":"2024-03-22T04:13:37.534117+0000","last_fullsized":"2024-03-22T04:13:37.534117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:04.300928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539332+0000","last_change":"2024-03-22T04:13:35.534630+0000","last_active":"2024-03-22T04:13:35.539332+0000","last_peered":"2024-03-22T04:13:35.539332+0000","last_clean":"2024-03-22T04:13:35.539332+0000","last_became_active":"2024-03-22T04:13:35.533284+0000","last_became_peered":"2024-03-22T04:13:35.533284+0000","last_unstale":"2024-03-22T04:13:35.539332+0000","last_undegraded":"2024-03-22T04:13:35.539332+0000","last_fullsized":"2024-03-22T04:13:35.539332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534628+0000","last_change":"2024-03-22T04:13:36.539056+0000","last_active":"2024-03-22T04:13:37.534628+0000","last_peered":"2024-03-22T04:13:37.534628+0000","last_clean":"2024-03-22T04:13:37.534628+0000","last_became_active":"2024-03-22T04:13:36.538782+0000","last_became_peered":"2024-03-22T04:13:36.538782+0000","last_unstale":"2024-03-22T04:13:37.534628+0000","last_undegraded":"2024-03-22T04:13:37.534628+0000","last_fullsized":"2024-03-22T04:13:37.534628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:07.459840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537876+0000","last_change":"2024-03-22T04:13:35.531937+0000","last_active":"2024-03-22T04:13:35.537876+0000","last_peered":"2024-03-22T04:13:35.537876+0000","last_clean":"2024-03-22T04:13:35.537876+0000","last_became_active":"2024-03-22T04:13:35.528718+0000","last_became_peered":"2024-03-22T04:13:35.528718+0000","last_unstale":"2024-03-22T04:13:35.537876+0000","last_undegraded":"2024-03-22T04:13:35.537876+0000","last_fullsized":"2024-03-22T04:13:35.537876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517454+0000","last_change":"2024-03-22T04:13:35.517454+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517454+0000","last_undegraded":"2024-03-22T04:13:35.517454+0000","last_fullsized":"2024-03-22T04:13:35.517454+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536284+0000","last_change":"2024-03-22T04:13:35.530829+0000","last_active":"2024-03-22T04:13:35.536284+0000","last_peered":"2024-03-22T04:13:35.536284+0000","last_clean":"2024-03-22T04:13:35.536284+0000","last_became_active":"2024-03-22T04:13:35.530554+0000","last_became_peered":"2024-03-22T04:13:35.530554+0000","last_unstale":"2024-03-22T04:13:35.536284+0000","last_undegraded":"2024-03-22T04:13:35.536284+0000","last_fullsized":"2024-03-22T04:13:35.536284+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:50.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517863+0000","last_change":"2024-03-22T04:13:35.517863+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517863+0000","last_undegraded":"2024-03-22T04:13:35.517863+0000","last_fullsized":"2024-03-22T04:13:35.517863+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.540268+0000","last_change":"2024-03-22T04:13:35.538794+0000","last_active":"2024-03-22T04:13:35.540268+0000","last_peered":"2024-03-22T04:13:35.540268+0000","last_clean":"2024-03-22T04:13:35.540268+0000","last_became_active":"2024-03-22T04:13:35.538511+0000","last_became_peered":"2024-03-22T04:13:35.538511+0000","last_unstale":"2024-03-22T04:13:35.540268+0000","last_undegraded":"2024-03-22T04:13:35.540268+0000","last_fullsized":"2024-03-22T04:13:35.540268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:00:58.326961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516763+0000","last_change":"2024-03-22T04:13:35.516763+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516763+0000","last_undegraded":"2024-03-22T04:13:35.516763+0000","last_fullsized":"2024-03-22T04:13:35.516763+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.533960+0000","last_change":"2024-03-22T04:13:35.528468+0000","last_active":"2024-03-22T04:13:35.533960+0000","last_peered":"2024-03-22T04:13:35.533960+0000","last_clean":"2024-03-22T04:13:35.533960+0000","last_became_active":"2024-03-22T04:13:35.528113+0000","last_became_peered":"2024-03-22T04:13:35.528113+0000","last_unstale":"2024-03-22T04:13:35.533960+0000","last_undegraded":"2024-03-22T04:13:35.533960+0000","last_fullsized":"2024-03-22T04:13:35.533960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.514570+0000","last_change":"2024-03-22T04:13:35.514570+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.514570+0000","last_undegraded":"2024-03-22T04:13:35.514570+0000","last_fullsized":"2024-03-22T04:13:35.514570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.531318+0000","last_change":"2024-03-22T04:13:35.525890+0000","last_active":"2024-03-22T04:13:35.531318+0000","last_peered":"2024-03-22T04:13:35.531318+0000","last_clean":"2024-03-22T04:13:35.531318+0000","last_became_active":"2024-03-22T04:13:35.525618+0000","last_became_peered":"2024-03-22T04:13:35.525618+0000","last_unstale":"2024-03-22T04:13:35.531318+0000","last_undegraded":"2024-03-22T04:13:35.531318+0000","last_fullsized":"2024-03-22T04:13:35.531318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515001+0000","last_change":"2024-03-22T04:13:35.515001+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515001+0000","last_undegraded":"2024-03-22T04:13:35.515001+0000","last_fullsized":"2024-03-22T04:13:35.515001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539898+0000","last_change":"2024-03-22T04:13:35.537483+0000","last_active":"2024-03-22T04:13:35.539898+0000","last_peered":"2024-03-22T04:13:35.539898+0000","last_clean":"2024-03-22T04:13:35.539898+0000","last_became_active":"2024-03-22T04:13:35.537070+0000","last_became_peered":"2024-03-22T04:13:35.537070+0000","last_unstale":"2024-03-22T04:13:35.539898+0000","last_undegraded":"2024-03-22T04:13:35.539898+0000","last_fullsized":"2024-03-22T04:13:35.539898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534302+0000","last_change":"2024-03-22T04:13:36.535142+0000","last_active":"2024-03-22T04:13:37.534302+0000","last_peered":"2024-03-22T04:13:37.534302+0000","last_clean":"2024-03-22T04:13:37.534302+0000","last_became_active":"2024-03-22T04:13:36.533918+0000","last_became_peered":"2024-03-22T04:13:36.533918+0000","last_unstale":"2024-03-22T04:13:37.534302+0000","last_undegraded":"2024-03-22T04:13:37.534302+0000","last_fullsized":"2024-03-22T04:13:37.534302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:15.073523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537406+0000","last_change":"2024-03-22T04:13:35.533317+0000","last_active":"2024-03-22T04:13:35.537406+0000","last_peered":"2024-03-22T04:13:35.537406+0000","last_clean":"2024-03-22T04:13:35.537406+0000","last_became_active":"2024-03-22T04:13:35.532424+0000","last_became_peered":"2024-03-22T04:13:35.532424+0000","last_unstale":"2024-03-22T04:13:35.537406+0000","last_undegraded":"2024-03-22T04:13:35.537406+0000","last_fullsized":"2024-03-22T04:13:35.537406+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:40:06.790699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516052+0000","last_change":"2024-03-22T04:13:35.516052+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516052+0000","last_undegraded":"2024-03-22T04:13:35.516052+0000","last_fullsized":"2024-03-22T04:13:35.516052+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.534324+0000","last_change":"2024-03-22T04:13:35.528988+0000","last_active":"2024-03-22T04:13:35.534324+0000","last_peered":"2024-03-22T04:13:35.534324+0000","last_clean":"2024-03-22T04:13:35.534324+0000","last_became_active":"2024-03-22T04:13:35.528529+0000","last_became_peered":"2024-03-22T04:13:35.528529+0000","last_unstale":"2024-03-22T04:13:35.534324+0000","last_undegraded":"2024-03-22T04:13:35.534324+0000","last_fullsized":"2024-03-22T04:13:35.534324+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":247,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.517988+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:13:35.517988+0000","last_peered":"2024-03-22T04:13:35.517988+0000","last_clean":"2024-03-22T04:13:35.517988+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:13:35.517988+0000","last_undegraded":"2024-03-22T04:13:35.517988+0000","last_fullsized":"2024-03-22T04:13:35.517988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":778240,"data_stored":346892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":937260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":346892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":930593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":361333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:13:40.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:13:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da6800640 1 Processor -- start 2024-03-22T04:13:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da6800640 1 -- start start 2024-03-22T04:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da6800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da0070190 0x7f5da015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da6800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5da006f0d0 0x7f5da00b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da6800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da00cfc20 con 0x7f5da00b5c20 2024-03-22T04:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da6800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5da00cff00 con 0x7f5da006f0d0 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da6800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5da00d01e0 con 0x7f5da0070190 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5da00b5c20 0x7f5da006fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38710/0 (socket says 172.21.15.86:38710) 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.144+0000 7f5da5400640 1 -- 172.21.15.86:0/4109070847 learned_addr learned my addr 172.21.15.86:0/4109070847 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5da5e00640 1 --2- 172.21.15.86:0/4109070847 >> v2:172.21.15.196:3300/0 conn(0x7f5da006f0d0 0x7f5da00b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5da4a00640 1 --2- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da0070190 0x7f5da015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5d9fe00640 1 -- 172.21.15.86:0/4109070847 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2395180181 0 0) 0x7f5da00cfc20 con 0x7f5da00b5c20 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5d9fe00640 1 -- 172.21.15.86:0/4109070847 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d88003530 con 0x7f5da00b5c20 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5da5e00640 1 -- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da0070190 msgr2=0x7f5da015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5da5e00640 1 --2- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da0070190 0x7f5da015bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5da5e00640 1 -- 172.21.15.86:0/4109070847 >> v1:172.21.15.86:6789/0 conn(0x7f5da00b5c20 legacy=0x7f5da006fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5da5e00640 1 -- 172.21.15.86:0/4109070847 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00d04c0 con 0x7f5da006f0d0 2024-03-22T04:13:41.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5da4a00640 1 --2- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da0070190 0x7f5da015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:41.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5da5e00640 1 --2- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da006f0d0 0x7f5da00b4050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009d90 tx=0x7f5d9801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c0dde0514d7f87 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5d9fe00640 1 -- 172.21.15.86:0/4109070847 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d980b2070 con 0x7f5da006f0d0 2024-03-22T04:13:41.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5d9fe00640 1 -- 172.21.15.86:0/4109070847 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d980a5830 con 0x7f5da006f0d0 2024-03-22T04:13:41.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.148+0000 7f5d9fe00640 1 -- 172.21.15.86:0/4109070847 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d980a5c00 con 0x7f5da006f0d0 2024-03-22T04:13:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da006f0d0 msgr2=0x7f5da00b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 --2- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da006f0d0 0x7f5da00b4050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009d90 tx=0x7f5d9801c970 comp rx=0 tx=0).stop 2024-03-22T04:13:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- 172.21.15.86:0/4109070847 shutdown_connections 2024-03-22T04:13:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 --2- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da006f0d0 0x7f5da00b4050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 --2- 172.21.15.86:0/4109070847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da0070190 0x7f5da015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- 172.21.15.86:0/4109070847 >> 172.21.15.86:0/4109070847 conn(0x7f5da006e540 msgr2=0x7f5da006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:41.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- 172.21.15.86:0/4109070847 shutdown_connections 2024-03-22T04:13:41.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- 172.21.15.86:0/4109070847 wait complete. 2024-03-22T04:13:41.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 Processor -- start 2024-03-22T04:13:41.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- start start 2024-03-22T04:13:41.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da006f0d0 0x7f5da014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da0070190 0x7f5da0154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da00b5c20 0x7f5da0154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00b4f20 con 0x7f5da006f0d0 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00b4da0 con 0x7f5da0070190 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da6800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5da00b50a0 con 0x7f5da00b5c20 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da006f0d0 0x7f5da014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da006f0d0 0x7f5da014efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45894/0 (socket says 172.21.15.86:45894) 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5400640 1 -- 172.21.15.86:0/4156611904 learned_addr learned my addr 172.21.15.86:0/4156611904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da4a00640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da0070190 0x7f5da0154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5e00640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da00b5c20 0x7f5da0154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5400640 1 -- 172.21.15.86:0/4156611904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da00b5c20 msgr2=0x7f5da0154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5400640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da00b5c20 0x7f5da0154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5400640 1 -- 172.21.15.86:0/4156611904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da0070190 msgr2=0x7f5da0154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5400640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da0070190 0x7f5da0154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.152+0000 7f5da5400640 1 -- 172.21.15.86:0/4156611904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da0155080 con 0x7f5da006f0d0 2024-03-22T04:13:41.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5da5400640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da006f0d0 0x7f5da014efd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c3c0 tx=0x7f5d9400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5d9d600640 1 -- 172.21.15.86:0/4156611904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94017070 con 0x7f5da006f0d0 2024-03-22T04:13:41.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5d9d600640 1 -- 172.21.15.86:0/4156611904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d94005cf0 con 0x7f5da006f0d0 2024-03-22T04:13:41.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da0155330 con 0x7f5da006f0d0 2024-03-22T04:13:41.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5d9d600640 1 -- 172.21.15.86:0/4156611904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940072f0 con 0x7f5da006f0d0 2024-03-22T04:13:41.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01558f0 con 0x7f5da006f0d0 2024-03-22T04:13:41.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da0074470 con 0x7f5da006f0d0 2024-03-22T04:13:41.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5d9d600640 1 -- 172.21.15.86:0/4156611904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5d9401a070 con 0x7f5da006f0d0 2024-03-22T04:13:41.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5d9d600640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d80067b20 0x7f5d80069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5d9d600640 1 -- 172.21.15.86:0/4156611904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f5d94085420 con 0x7f5da006f0d0 2024-03-22T04:13:41.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.156+0000 7f5da4a00640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d80067b20 0x7f5d80069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.160+0000 7f5da4a00640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d80067b20 0x7f5d80069fe0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009580 tx=0x7f5d8c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.160+0000 7f5d9d600640 1 -- 172.21.15.86:0/4156611904 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5d94055640 con 0x7f5da006f0d0 2024-03-22T04:13:41.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.312+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5da01505b0 con 0x7f5d80067b20 2024-03-22T04:13:41.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.316+0000 7f5d9d600640 1 -- 172.21.15.86:0/4156611904 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247373 (secure 0 0 0) 0x7f5da01505b0 con 0x7f5d80067b20 2024-03-22T04:13:41.324 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:41.325 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:13:41.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d80067b20 msgr2=0x7f5d80069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d80067b20 0x7f5d80069fe0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009580 tx=0x7f5d8c009190 comp rx=0 tx=0).stop 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da006f0d0 msgr2=0x7f5da014efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da006f0d0 0x7f5da014efd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c3c0 tx=0x7f5d9400c880 comp rx=0 tx=0).stop 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 shutdown_connections 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d80067b20 0x7f5d80069fe0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5da0070190 0x7f5da0154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5da00b5c20 0x7f5da0154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 --2- 172.21.15.86:0/4156611904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5da006f0d0 0x7f5da014efd0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 >> 172.21.15.86:0/4156611904 conn(0x7f5da006e540 msgr2=0x7f5da014b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 shutdown_connections 2024-03-22T04:13:41.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.320+0000 7f5da6800640 1 -- 172.21.15.86:0/4156611904 wait complete. 2024-03-22T04:13:41.344 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-22T04:13:39.830360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227568,"kb_used_data":7220,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747152,"statfs":{"total":773094113280,"available":772861083648,"internally_reserved":0,"allocated":7393280,"data_stored":3943645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.326188"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521460+0000","last_change":"2024-03-22T04:13:35.521460+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521460+0000","last_undegraded":"2024-03-22T04:13:35.521460+0000","last_fullsized":"2024-03-22T04:13:35.521460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515543+0000","last_change":"2024-03-22T04:13:35.515543+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515543+0000","last_undegraded":"2024-03-22T04:13:35.515543+0000","last_fullsized":"2024-03-22T04:13:35.515543+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521592+0000","last_change":"2024-03-22T04:13:35.521592+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521592+0000","last_undegraded":"2024-03-22T04:13:35.521592+0000","last_fullsized":"2024-03-22T04:13:35.521592+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.523111+0000","last_change":"2024-03-22T04:13:35.523111+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.523111+0000","last_undegraded":"2024-03-22T04:13:35.523111+0000","last_fullsized":"2024-03-22T04:13:35.523111+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521553+0000","last_change":"2024-03-22T04:13:35.521553+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521553+0000","last_undegraded":"2024-03-22T04:13:35.521553+0000","last_fullsized":"2024-03-22T04:13:35.521553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515499+0000","last_change":"2024-03-22T04:13:35.515499+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515499+0000","last_undegraded":"2024-03-22T04:13:35.515499+0000","last_fullsized":"2024-03-22T04:13:35.515499+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522743+0000","last_change":"2024-03-22T04:13:35.522743+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522743+0000","last_undegraded":"2024-03-22T04:13:35.522743+0000","last_fullsized":"2024-03-22T04:13:35.522743+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522009+0000","last_change":"2024-03-22T04:13:35.522009+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522009+0000","last_undegraded":"2024-03-22T04:13:35.522009+0000","last_fullsized":"2024-03-22T04:13:35.522009+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519953+0000","last_change":"2024-03-22T04:13:35.519953+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519953+0000","last_undegraded":"2024-03-22T04:13:35.519953+0000","last_fullsized":"2024-03-22T04:13:35.519953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517617+0000","last_change":"2024-03-22T04:13:35.517617+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517617+0000","last_undegraded":"2024-03-22T04:13:35.517617+0000","last_fullsized":"2024-03-22T04:13:35.517617+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521802+0000","last_change":"2024-03-22T04:13:35.521802+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521802+0000","last_undegraded":"2024-03-22T04:13:35.521802+0000","last_fullsized":"2024-03-22T04:13:35.521802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522711+0000","last_change":"2024-03-22T04:13:35.522711+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522711+0000","last_undegraded":"2024-03-22T04:13:35.522711+0000","last_fullsized":"2024-03-22T04:13:35.522711+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518599+0000","last_change":"2024-03-22T04:13:35.518599+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518599+0000","last_undegraded":"2024-03-22T04:13:35.518599+0000","last_fullsized":"2024-03-22T04:13:35.518599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519660+0000","last_change":"2024-03-22T04:13:35.519660+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519660+0000","last_undegraded":"2024-03-22T04:13:35.519660+0000","last_fullsized":"2024-03-22T04:13:35.519660+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516498+0000","last_change":"2024-03-22T04:13:35.516498+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516498+0000","last_undegraded":"2024-03-22T04:13:35.516498+0000","last_fullsized":"2024-03-22T04:13:35.516498+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518341+0000","last_change":"2024-03-22T04:13:35.518341+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518341+0000","last_undegraded":"2024-03-22T04:13:35.518341+0000","last_fullsized":"2024-03-22T04:13:35.518341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534709+0000","last_change":"2024-03-22T04:13:36.534928+0000","last_active":"2024-03-22T04:13:37.534709+0000","last_peered":"2024-03-22T04:13:37.534709+0000","last_clean":"2024-03-22T04:13:37.534709+0000","last_became_active":"2024-03-22T04:13:36.534458+0000","last_became_peered":"2024-03-22T04:13:36.534458+0000","last_unstale":"2024-03-22T04:13:37.534709+0000","last_undegraded":"2024-03-22T04:13:37.534709+0000","last_fullsized":"2024-03-22T04:13:37.534709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:10:28.784356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518861+0000","last_change":"2024-03-22T04:13:35.518861+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518861+0000","last_undegraded":"2024-03-22T04:13:35.518861+0000","last_fullsized":"2024-03-22T04:13:35.518861+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521381+0000","last_change":"2024-03-22T04:13:35.521381+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521381+0000","last_undegraded":"2024-03-22T04:13:35.521381+0000","last_fullsized":"2024-03-22T04:13:35.521381+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534466+0000","last_change":"2024-03-22T04:13:36.533555+0000","last_active":"2024-03-22T04:13:37.534466+0000","last_peered":"2024-03-22T04:13:37.534466+0000","last_clean":"2024-03-22T04:13:37.534466+0000","last_became_active":"2024-03-22T04:13:36.533261+0000","last_became_peered":"2024-03-22T04:13:36.533261+0000","last_unstale":"2024-03-22T04:13:37.534466+0000","last_undegraded":"2024-03-22T04:13:37.534466+0000","last_fullsized":"2024-03-22T04:13:37.534466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:22.636235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518186+0000","last_change":"2024-03-22T04:13:35.518186+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518186+0000","last_undegraded":"2024-03-22T04:13:35.518186+0000","last_fullsized":"2024-03-22T04:13:35.518186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.514455+0000","last_change":"2024-03-22T04:13:35.514455+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.514455+0000","last_undegraded":"2024-03-22T04:13:35.514455+0000","last_fullsized":"2024-03-22T04:13:35.514455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517884+0000","last_change":"2024-03-22T04:13:35.517884+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517884+0000","last_undegraded":"2024-03-22T04:13:35.517884+0000","last_fullsized":"2024-03-22T04:13:35.517884+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520978+0000","last_change":"2024-03-22T04:13:35.520978+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520978+0000","last_undegraded":"2024-03-22T04:13:35.520978+0000","last_fullsized":"2024-03-22T04:13:35.520978+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517424+0000","last_change":"2024-03-22T04:13:35.517424+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517424+0000","last_undegraded":"2024-03-22T04:13:35.517424+0000","last_fullsized":"2024-03-22T04:13:35.517424+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.526133+0000","last_change":"2024-03-22T04:13:35.526133+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.526133+0000","last_undegraded":"2024-03-22T04:13:35.526133+0000","last_fullsized":"2024-03-22T04:13:35.526133+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517842+0000","last_change":"2024-03-22T04:13:35.517842+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517842+0000","last_undegraded":"2024-03-22T04:13:35.517842+0000","last_fullsized":"2024-03-22T04:13:35.517842+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.521298+0000","last_change":"2024-03-22T04:13:35.521298+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.521298+0000","last_undegraded":"2024-03-22T04:13:35.521298+0000","last_fullsized":"2024-03-22T04:13:35.521298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518277+0000","last_change":"2024-03-22T04:13:35.518277+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518277+0000","last_undegraded":"2024-03-22T04:13:35.518277+0000","last_fullsized":"2024-03-22T04:13:35.518277+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517462+0000","last_change":"2024-03-22T04:13:35.517462+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517462+0000","last_undegraded":"2024-03-22T04:13:35.517462+0000","last_fullsized":"2024-03-22T04:13:35.517462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520275+0000","last_change":"2024-03-22T04:13:35.520275+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520275+0000","last_undegraded":"2024-03-22T04:13:35.520275+0000","last_fullsized":"2024-03-22T04:13:35.520275+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.520777+0000","last_change":"2024-03-22T04:13:35.520777+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.520777+0000","last_undegraded":"2024-03-22T04:13:35.520777+0000","last_fullsized":"2024-03-22T04:13:35.520777+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522255+0000","last_change":"2024-03-22T04:13:35.522255+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522255+0000","last_undegraded":"2024-03-22T04:13:35.522255+0000","last_fullsized":"2024-03-22T04:13:35.522255+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536950+0000","last_change":"2024-03-22T04:13:35.531755+0000","last_active":"2024-03-22T04:13:35.536950+0000","last_peered":"2024-03-22T04:13:35.536950+0000","last_clean":"2024-03-22T04:13:35.536950+0000","last_became_active":"2024-03-22T04:13:35.531433+0000","last_became_peered":"2024-03-22T04:13:35.531433+0000","last_unstale":"2024-03-22T04:13:35.536950+0000","last_undegraded":"2024-03-22T04:13:35.536950+0000","last_fullsized":"2024-03-22T04:13:35.536950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516557+0000","last_change":"2024-03-22T04:13:35.516557+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516557+0000","last_undegraded":"2024-03-22T04:13:35.516557+0000","last_fullsized":"2024-03-22T04:13:35.516557+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.535991+0000","last_change":"2024-03-22T04:13:35.529846+0000","last_active":"2024-03-22T04:13:35.535991+0000","last_peered":"2024-03-22T04:13:35.535991+0000","last_clean":"2024-03-22T04:13:35.535991+0000","last_became_active":"2024-03-22T04:13:35.529546+0000","last_became_peered":"2024-03-22T04:13:35.529546+0000","last_unstale":"2024-03-22T04:13:35.535991+0000","last_undegraded":"2024-03-22T04:13:35.535991+0000","last_fullsized":"2024-03-22T04:13:35.535991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515064+0000","last_change":"2024-03-22T04:13:35.515064+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515064+0000","last_undegraded":"2024-03-22T04:13:35.515064+0000","last_fullsized":"2024-03-22T04:13:35.515064+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536333+0000","last_change":"2024-03-22T04:13:35.530455+0000","last_active":"2024-03-22T04:13:35.536333+0000","last_peered":"2024-03-22T04:13:35.536333+0000","last_clean":"2024-03-22T04:13:35.536333+0000","last_became_active":"2024-03-22T04:13:35.530130+0000","last_became_peered":"2024-03-22T04:13:35.530130+0000","last_unstale":"2024-03-22T04:13:35.536333+0000","last_undegraded":"2024-03-22T04:13:35.536333+0000","last_fullsized":"2024-03-22T04:13:35.536333+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519480+0000","last_change":"2024-03-22T04:13:35.519480+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519480+0000","last_undegraded":"2024-03-22T04:13:35.519480+0000","last_fullsized":"2024-03-22T04:13:35.519480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537702+0000","last_change":"2024-03-22T04:13:35.533152+0000","last_active":"2024-03-22T04:13:35.537702+0000","last_peered":"2024-03-22T04:13:35.537702+0000","last_clean":"2024-03-22T04:13:35.537702+0000","last_became_active":"2024-03-22T04:13:35.532739+0000","last_became_peered":"2024-03-22T04:13:35.532739+0000","last_unstale":"2024-03-22T04:13:35.537702+0000","last_undegraded":"2024-03-22T04:13:35.537702+0000","last_fullsized":"2024-03-22T04:13:35.537702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:11:03.089640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533547+0000","last_change":"2024-03-22T04:13:36.539261+0000","last_active":"2024-03-22T04:13:37.533547+0000","last_peered":"2024-03-22T04:13:37.533547+0000","last_clean":"2024-03-22T04:13:37.533547+0000","last_became_active":"2024-03-22T04:13:36.538948+0000","last_became_peered":"2024-03-22T04:13:36.538948+0000","last_unstale":"2024-03-22T04:13:37.533547+0000","last_undegraded":"2024-03-22T04:13:37.533547+0000","last_fullsized":"2024-03-22T04:13:37.533547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.370928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534342+0000","last_change":"2024-03-22T04:13:35.528073+0000","last_active":"2024-03-22T04:13:37.534342+0000","last_peered":"2024-03-22T04:13:37.534342+0000","last_clean":"2024-03-22T04:13:37.534342+0000","last_became_active":"2024-03-22T04:13:35.527410+0000","last_became_peered":"2024-03-22T04:13:35.527410+0000","last_unstale":"2024-03-22T04:13:37.534342+0000","last_undegraded":"2024-03-22T04:13:37.534342+0000","last_fullsized":"2024-03-22T04:13:37.534342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:00:59.588177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.522326+0000","last_change":"2024-03-22T04:13:35.522326+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.522326+0000","last_undegraded":"2024-03-22T04:13:35.522326+0000","last_fullsized":"2024-03-22T04:13:35.522326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.534280+0000","last_change":"2024-03-22T04:13:35.526516+0000","last_active":"2024-03-22T04:13:35.534280+0000","last_peered":"2024-03-22T04:13:35.534280+0000","last_clean":"2024-03-22T04:13:35.534280+0000","last_became_active":"2024-03-22T04:13:35.526208+0000","last_became_peered":"2024-03-22T04:13:35.526208+0000","last_unstale":"2024-03-22T04:13:35.534280+0000","last_undegraded":"2024-03-22T04:13:35.534280+0000","last_fullsized":"2024-03-22T04:13:35.534280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516439+0000","last_change":"2024-03-22T04:13:35.516439+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516439+0000","last_undegraded":"2024-03-22T04:13:35.516439+0000","last_fullsized":"2024-03-22T04:13:35.516439+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537342+0000","last_change":"2024-03-22T04:13:35.534172+0000","last_active":"2024-03-22T04:13:35.537342+0000","last_peered":"2024-03-22T04:13:35.537342+0000","last_clean":"2024-03-22T04:13:35.537342+0000","last_became_active":"2024-03-22T04:13:35.533651+0000","last_became_peered":"2024-03-22T04:13:35.533651+0000","last_unstale":"2024-03-22T04:13:35.537342+0000","last_undegraded":"2024-03-22T04:13:35.537342+0000","last_fullsized":"2024-03-22T04:13:35.537342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534186+0000","last_change":"2024-03-22T04:13:36.535993+0000","last_active":"2024-03-22T04:13:37.534186+0000","last_peered":"2024-03-22T04:13:37.534186+0000","last_clean":"2024-03-22T04:13:37.534186+0000","last_became_active":"2024-03-22T04:13:36.534295+0000","last_became_peered":"2024-03-22T04:13:36.534295+0000","last_unstale":"2024-03-22T04:13:37.534186+0000","last_undegraded":"2024-03-22T04:13:37.534186+0000","last_fullsized":"2024-03-22T04:13:37.534186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:58.215479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539156+0000","last_change":"2024-03-22T04:13:35.533417+0000","last_active":"2024-03-22T04:13:35.539156+0000","last_peered":"2024-03-22T04:13:35.539156+0000","last_clean":"2024-03-22T04:13:35.539156+0000","last_became_active":"2024-03-22T04:13:35.533053+0000","last_became_peered":"2024-03-22T04:13:35.533053+0000","last_unstale":"2024-03-22T04:13:35.539156+0000","last_undegraded":"2024-03-22T04:13:35.539156+0000","last_fullsized":"2024-03-22T04:13:35.539156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519164+0000","last_change":"2024-03-22T04:13:35.519164+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519164+0000","last_undegraded":"2024-03-22T04:13:35.519164+0000","last_fullsized":"2024-03-22T04:13:35.519164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539366+0000","last_change":"2024-03-22T04:13:35.535058+0000","last_active":"2024-03-22T04:13:35.539366+0000","last_peered":"2024-03-22T04:13:35.539366+0000","last_clean":"2024-03-22T04:13:35.539366+0000","last_became_active":"2024-03-22T04:13:35.534345+0000","last_became_peered":"2024-03-22T04:13:35.534345+0000","last_unstale":"2024-03-22T04:13:35.539366+0000","last_undegraded":"2024-03-22T04:13:35.539366+0000","last_fullsized":"2024-03-22T04:13:35.539366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:50.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534393+0000","last_change":"2024-03-22T04:13:36.527089+0000","last_active":"2024-03-22T04:13:37.534393+0000","last_peered":"2024-03-22T04:13:37.534393+0000","last_clean":"2024-03-22T04:13:37.534393+0000","last_became_active":"2024-03-22T04:13:36.526807+0000","last_became_peered":"2024-03-22T04:13:36.526807+0000","last_unstale":"2024-03-22T04:13:37.534393+0000","last_undegraded":"2024-03-22T04:13:37.534393+0000","last_fullsized":"2024-03-22T04:13:37.534393+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:57:11.984612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.529461+0000","last_change":"2024-03-22T04:13:35.525257+0000","last_active":"2024-03-22T04:13:35.529461+0000","last_peered":"2024-03-22T04:13:35.529461+0000","last_clean":"2024-03-22T04:13:35.529461+0000","last_became_active":"2024-03-22T04:13:35.524936+0000","last_became_peered":"2024-03-22T04:13:35.524936+0000","last_unstale":"2024-03-22T04:13:35.529461+0000","last_undegraded":"2024-03-22T04:13:35.529461+0000","last_fullsized":"2024-03-22T04:13:35.529461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:18:55.065379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515798+0000","last_change":"2024-03-22T04:13:35.515798+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515798+0000","last_undegraded":"2024-03-22T04:13:35.515798+0000","last_fullsized":"2024-03-22T04:13:35.515798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.538958+0000","last_change":"2024-03-22T04:13:35.536397+0000","last_active":"2024-03-22T04:13:35.538958+0000","last_peered":"2024-03-22T04:13:35.538958+0000","last_clean":"2024-03-22T04:13:35.538958+0000","last_became_active":"2024-03-22T04:13:35.536137+0000","last_became_peered":"2024-03-22T04:13:35.536137+0000","last_unstale":"2024-03-22T04:13:35.538958+0000","last_undegraded":"2024-03-22T04:13:35.538958+0000","last_fullsized":"2024-03-22T04:13:35.538958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:05:15.541959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518642+0000","last_change":"2024-03-22T04:13:35.518642+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518642+0000","last_undegraded":"2024-03-22T04:13:35.518642+0000","last_fullsized":"2024-03-22T04:13:35.518642+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536050+0000","last_change":"2024-03-22T04:13:35.529973+0000","last_active":"2024-03-22T04:13:35.536050+0000","last_peered":"2024-03-22T04:13:35.536050+0000","last_clean":"2024-03-22T04:13:35.536050+0000","last_became_active":"2024-03-22T04:13:35.529696+0000","last_became_peered":"2024-03-22T04:13:35.529696+0000","last_unstale":"2024-03-22T04:13:35.536050+0000","last_undegraded":"2024-03-22T04:13:35.536050+0000","last_fullsized":"2024-03-22T04:13:35.536050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515862+0000","last_change":"2024-03-22T04:13:35.515862+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515862+0000","last_undegraded":"2024-03-22T04:13:35.515862+0000","last_fullsized":"2024-03-22T04:13:35.515862+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.533045+0000","last_change":"2024-03-22T04:13:35.529378+0000","last_active":"2024-03-22T04:13:35.533045+0000","last_peered":"2024-03-22T04:13:35.533045+0000","last_clean":"2024-03-22T04:13:35.533045+0000","last_became_active":"2024-03-22T04:13:35.529096+0000","last_became_peered":"2024-03-22T04:13:35.529096+0000","last_unstale":"2024-03-22T04:13:35.533045+0000","last_undegraded":"2024-03-22T04:13:35.533045+0000","last_fullsized":"2024-03-22T04:13:35.533045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:25:37.720655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516023+0000","last_change":"2024-03-22T04:13:35.516023+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516023+0000","last_undegraded":"2024-03-22T04:13:35.516023+0000","last_fullsized":"2024-03-22T04:13:35.516023+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.538455+0000","last_change":"2024-03-22T04:13:35.536795+0000","last_active":"2024-03-22T04:13:35.538455+0000","last_peered":"2024-03-22T04:13:35.538455+0000","last_clean":"2024-03-22T04:13:35.538455+0000","last_became_active":"2024-03-22T04:13:35.534453+0000","last_became_peered":"2024-03-22T04:13:35.534453+0000","last_unstale":"2024-03-22T04:13:35.538455+0000","last_undegraded":"2024-03-22T04:13:35.538455+0000","last_fullsized":"2024-03-22T04:13:35.538455+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516963+0000","last_change":"2024-03-22T04:13:35.516963+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516963+0000","last_undegraded":"2024-03-22T04:13:35.516963+0000","last_fullsized":"2024-03-22T04:13:35.516963+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537398+0000","last_change":"2024-03-22T04:13:35.529571+0000","last_active":"2024-03-22T04:13:35.537398+0000","last_peered":"2024-03-22T04:13:35.537398+0000","last_clean":"2024-03-22T04:13:35.537398+0000","last_became_active":"2024-03-22T04:13:35.528385+0000","last_became_peered":"2024-03-22T04:13:35.528385+0000","last_unstale":"2024-03-22T04:13:35.537398+0000","last_undegraded":"2024-03-22T04:13:35.537398+0000","last_fullsized":"2024-03-22T04:13:35.537398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:55:19.501433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.519326+0000","last_change":"2024-03-22T04:13:35.519326+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.519326+0000","last_undegraded":"2024-03-22T04:13:35.519326+0000","last_fullsized":"2024-03-22T04:13:35.519326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537868+0000","last_change":"2024-03-22T04:13:35.536315+0000","last_active":"2024-03-22T04:13:35.537868+0000","last_peered":"2024-03-22T04:13:35.537868+0000","last_clean":"2024-03-22T04:13:35.537868+0000","last_became_active":"2024-03-22T04:13:35.535908+0000","last_became_peered":"2024-03-22T04:13:35.535908+0000","last_unstale":"2024-03-22T04:13:35.537868+0000","last_undegraded":"2024-03-22T04:13:35.537868+0000","last_fullsized":"2024-03-22T04:13:35.537868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:18:55.065379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517925+0000","last_change":"2024-03-22T04:13:35.517925+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517925+0000","last_undegraded":"2024-03-22T04:13:35.517925+0000","last_fullsized":"2024-03-22T04:13:35.517925+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539347+0000","last_change":"2024-03-22T04:13:35.534854+0000","last_active":"2024-03-22T04:13:35.539347+0000","last_peered":"2024-03-22T04:13:35.539347+0000","last_clean":"2024-03-22T04:13:35.539347+0000","last_became_active":"2024-03-22T04:13:35.534331+0000","last_became_peered":"2024-03-22T04:13:35.534331+0000","last_unstale":"2024-03-22T04:13:35.539347+0000","last_undegraded":"2024-03-22T04:13:35.539347+0000","last_fullsized":"2024-03-22T04:13:35.539347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515262+0000","last_change":"2024-03-22T04:13:35.515262+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515262+0000","last_undegraded":"2024-03-22T04:13:35.515262+0000","last_fullsized":"2024-03-22T04:13:35.515262+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539147+0000","last_change":"2024-03-22T04:13:35.533530+0000","last_active":"2024-03-22T04:13:35.539147+0000","last_peered":"2024-03-22T04:13:35.539147+0000","last_clean":"2024-03-22T04:13:35.539147+0000","last_became_active":"2024-03-22T04:13:35.533254+0000","last_became_peered":"2024-03-22T04:13:35.533254+0000","last_unstale":"2024-03-22T04:13:35.539147+0000","last_undegraded":"2024-03-22T04:13:35.539147+0000","last_fullsized":"2024-03-22T04:13:35.539147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515325+0000","last_change":"2024-03-22T04:13:35.515325+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515325+0000","last_undegraded":"2024-03-22T04:13:35.515325+0000","last_fullsized":"2024-03-22T04:13:35.515325+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537474+0000","last_change":"2024-03-22T04:13:35.530606+0000","last_active":"2024-03-22T04:13:35.537474+0000","last_peered":"2024-03-22T04:13:35.537474+0000","last_clean":"2024-03-22T04:13:35.537474+0000","last_became_active":"2024-03-22T04:13:35.530250+0000","last_became_peered":"2024-03-22T04:13:35.530250+0000","last_unstale":"2024-03-22T04:13:35.537474+0000","last_undegraded":"2024-03-22T04:13:35.537474+0000","last_fullsized":"2024-03-22T04:13:35.537474+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:01:43.146759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.518273+0000","last_change":"2024-03-22T04:13:35.518273+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.518273+0000","last_undegraded":"2024-03-22T04:13:35.518273+0000","last_fullsized":"2024-03-22T04:13:35.518273+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.541072+0000","last_change":"2024-03-22T04:13:35.539839+0000","last_active":"2024-03-22T04:13:35.541072+0000","last_peered":"2024-03-22T04:13:35.541072+0000","last_clean":"2024-03-22T04:13:35.541072+0000","last_became_active":"2024-03-22T04:13:35.539572+0000","last_became_peered":"2024-03-22T04:13:35.539572+0000","last_unstale":"2024-03-22T04:13:35.541072+0000","last_undegraded":"2024-03-22T04:13:35.541072+0000","last_fullsized":"2024-03-22T04:13:35.541072+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:23.221070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517337+0000","last_change":"2024-03-22T04:13:35.517337+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517337+0000","last_undegraded":"2024-03-22T04:13:35.517337+0000","last_fullsized":"2024-03-22T04:13:35.517337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537532+0000","last_change":"2024-03-22T04:13:35.533632+0000","last_active":"2024-03-22T04:13:35.537532+0000","last_peered":"2024-03-22T04:13:35.537532+0000","last_clean":"2024-03-22T04:13:35.537532+0000","last_became_active":"2024-03-22T04:13:35.533349+0000","last_became_peered":"2024-03-22T04:13:35.533349+0000","last_unstale":"2024-03-22T04:13:35.537532+0000","last_undegraded":"2024-03-22T04:13:35.537532+0000","last_fullsized":"2024-03-22T04:13:35.537532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:23.697686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515728+0000","last_change":"2024-03-22T04:13:35.515728+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515728+0000","last_undegraded":"2024-03-22T04:13:35.515728+0000","last_fullsized":"2024-03-22T04:13:35.515728+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.531700+0000","last_change":"2024-03-22T04:13:35.527224+0000","last_active":"2024-03-22T04:13:35.531700+0000","last_peered":"2024-03-22T04:13:35.531700+0000","last_clean":"2024-03-22T04:13:35.531700+0000","last_became_active":"2024-03-22T04:13:35.526938+0000","last_became_peered":"2024-03-22T04:13:35.526938+0000","last_unstale":"2024-03-22T04:13:35.531700+0000","last_undegraded":"2024-03-22T04:13:35.531700+0000","last_fullsized":"2024-03-22T04:13:35.531700+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:08:49.483327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516973+0000","last_change":"2024-03-22T04:13:35.516973+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516973+0000","last_undegraded":"2024-03-22T04:13:35.516973+0000","last_fullsized":"2024-03-22T04:13:35.516973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536203+0000","last_change":"2024-03-22T04:13:35.530361+0000","last_active":"2024-03-22T04:13:35.536203+0000","last_peered":"2024-03-22T04:13:35.536203+0000","last_clean":"2024-03-22T04:13:35.536203+0000","last_became_active":"2024-03-22T04:13:35.529768+0000","last_became_peered":"2024-03-22T04:13:35.529768+0000","last_unstale":"2024-03-22T04:13:35.536203+0000","last_undegraded":"2024-03-22T04:13:35.536203+0000","last_fullsized":"2024-03-22T04:13:35.536203+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:12.579260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534117+0000","last_change":"2024-03-22T04:13:36.523859+0000","last_active":"2024-03-22T04:13:37.534117+0000","last_peered":"2024-03-22T04:13:37.534117+0000","last_clean":"2024-03-22T04:13:37.534117+0000","last_became_active":"2024-03-22T04:13:36.523598+0000","last_became_peered":"2024-03-22T04:13:36.523598+0000","last_unstale":"2024-03-22T04:13:37.534117+0000","last_undegraded":"2024-03-22T04:13:37.534117+0000","last_fullsized":"2024-03-22T04:13:37.534117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:04.300928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539332+0000","last_change":"2024-03-22T04:13:35.534630+0000","last_active":"2024-03-22T04:13:35.539332+0000","last_peered":"2024-03-22T04:13:35.539332+0000","last_clean":"2024-03-22T04:13:35.539332+0000","last_became_active":"2024-03-22T04:13:35.533284+0000","last_became_peered":"2024-03-22T04:13:35.533284+0000","last_unstale":"2024-03-22T04:13:35.539332+0000","last_undegraded":"2024-03-22T04:13:35.539332+0000","last_fullsized":"2024-03-22T04:13:35.539332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:03:36.175027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534628+0000","last_change":"2024-03-22T04:13:36.539056+0000","last_active":"2024-03-22T04:13:37.534628+0000","last_peered":"2024-03-22T04:13:37.534628+0000","last_clean":"2024-03-22T04:13:37.534628+0000","last_became_active":"2024-03-22T04:13:36.538782+0000","last_became_peered":"2024-03-22T04:13:36.538782+0000","last_unstale":"2024-03-22T04:13:37.534628+0000","last_undegraded":"2024-03-22T04:13:37.534628+0000","last_fullsized":"2024-03-22T04:13:37.534628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:07.459840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537876+0000","last_change":"2024-03-22T04:13:35.531937+0000","last_active":"2024-03-22T04:13:35.537876+0000","last_peered":"2024-03-22T04:13:35.537876+0000","last_clean":"2024-03-22T04:13:35.537876+0000","last_became_active":"2024-03-22T04:13:35.528718+0000","last_became_peered":"2024-03-22T04:13:35.528718+0000","last_unstale":"2024-03-22T04:13:35.537876+0000","last_undegraded":"2024-03-22T04:13:35.537876+0000","last_fullsized":"2024-03-22T04:13:35.537876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517454+0000","last_change":"2024-03-22T04:13:35.517454+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517454+0000","last_undegraded":"2024-03-22T04:13:35.517454+0000","last_fullsized":"2024-03-22T04:13:35.517454+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.536284+0000","last_change":"2024-03-22T04:13:35.530829+0000","last_active":"2024-03-22T04:13:35.536284+0000","last_peered":"2024-03-22T04:13:35.536284+0000","last_clean":"2024-03-22T04:13:35.536284+0000","last_became_active":"2024-03-22T04:13:35.530554+0000","last_became_peered":"2024-03-22T04:13:35.530554+0000","last_unstale":"2024-03-22T04:13:35.536284+0000","last_undegraded":"2024-03-22T04:13:35.536284+0000","last_fullsized":"2024-03-22T04:13:35.536284+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:50.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.517863+0000","last_change":"2024-03-22T04:13:35.517863+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.517863+0000","last_undegraded":"2024-03-22T04:13:35.517863+0000","last_fullsized":"2024-03-22T04:13:35.517863+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.540268+0000","last_change":"2024-03-22T04:13:35.538794+0000","last_active":"2024-03-22T04:13:35.540268+0000","last_peered":"2024-03-22T04:13:35.540268+0000","last_clean":"2024-03-22T04:13:35.540268+0000","last_became_active":"2024-03-22T04:13:35.538511+0000","last_became_peered":"2024-03-22T04:13:35.538511+0000","last_unstale":"2024-03-22T04:13:35.540268+0000","last_undegraded":"2024-03-22T04:13:35.540268+0000","last_fullsized":"2024-03-22T04:13:35.540268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:00:58.326961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516763+0000","last_change":"2024-03-22T04:13:35.516763+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516763+0000","last_undegraded":"2024-03-22T04:13:35.516763+0000","last_fullsized":"2024-03-22T04:13:35.516763+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.533960+0000","last_change":"2024-03-22T04:13:35.528468+0000","last_active":"2024-03-22T04:13:35.533960+0000","last_peered":"2024-03-22T04:13:35.533960+0000","last_clean":"2024-03-22T04:13:35.533960+0000","last_became_active":"2024-03-22T04:13:35.528113+0000","last_became_peered":"2024-03-22T04:13:35.528113+0000","last_unstale":"2024-03-22T04:13:35.533960+0000","last_undegraded":"2024-03-22T04:13:35.533960+0000","last_fullsized":"2024-03-22T04:13:35.533960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.514570+0000","last_change":"2024-03-22T04:13:35.514570+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.514570+0000","last_undegraded":"2024-03-22T04:13:35.514570+0000","last_fullsized":"2024-03-22T04:13:35.514570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.531318+0000","last_change":"2024-03-22T04:13:35.525890+0000","last_active":"2024-03-22T04:13:35.531318+0000","last_peered":"2024-03-22T04:13:35.531318+0000","last_clean":"2024-03-22T04:13:35.531318+0000","last_became_active":"2024-03-22T04:13:35.525618+0000","last_became_peered":"2024-03-22T04:13:35.525618+0000","last_unstale":"2024-03-22T04:13:35.531318+0000","last_undegraded":"2024-03-22T04:13:35.531318+0000","last_fullsized":"2024-03-22T04:13:35.531318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.515001+0000","last_change":"2024-03-22T04:13:35.515001+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.515001+0000","last_undegraded":"2024-03-22T04:13:35.515001+0000","last_fullsized":"2024-03-22T04:13:35.515001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.539898+0000","last_change":"2024-03-22T04:13:35.537483+0000","last_active":"2024-03-22T04:13:35.539898+0000","last_peered":"2024-03-22T04:13:35.539898+0000","last_clean":"2024-03-22T04:13:35.539898+0000","last_became_active":"2024-03-22T04:13:35.537070+0000","last_became_peered":"2024-03-22T04:13:35.537070+0000","last_unstale":"2024-03-22T04:13:35.539898+0000","last_undegraded":"2024-03-22T04:13:35.539898+0000","last_fullsized":"2024-03-22T04:13:35.539898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:31:10.681722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534302+0000","last_change":"2024-03-22T04:13:36.535142+0000","last_active":"2024-03-22T04:13:37.534302+0000","last_peered":"2024-03-22T04:13:37.534302+0000","last_clean":"2024-03-22T04:13:37.534302+0000","last_became_active":"2024-03-22T04:13:36.533918+0000","last_became_peered":"2024-03-22T04:13:36.533918+0000","last_unstale":"2024-03-22T04:13:37.534302+0000","last_undegraded":"2024-03-22T04:13:37.534302+0000","last_fullsized":"2024-03-22T04:13:37.534302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:15.073523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.537406+0000","last_change":"2024-03-22T04:13:35.533317+0000","last_active":"2024-03-22T04:13:35.537406+0000","last_peered":"2024-03-22T04:13:35.537406+0000","last_clean":"2024-03-22T04:13:35.537406+0000","last_became_active":"2024-03-22T04:13:35.532424+0000","last_became_peered":"2024-03-22T04:13:35.532424+0000","last_unstale":"2024-03-22T04:13:35.537406+0000","last_undegraded":"2024-03-22T04:13:35.537406+0000","last_fullsized":"2024-03-22T04:13:35.537406+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:40:06.790699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-03-22T04:13:35.516052+0000","last_change":"2024-03-22T04:13:35.516052+0000","last_active":"2024-03-22T04:13:35.502450+0000","last_peered":"2024-03-22T04:13:35.502450+0000","last_clean":"2024-03-22T04:13:35.502450+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-22T04:13:35.516052+0000","last_undegraded":"2024-03-22T04:13:35.516052+0000","last_fullsized":"2024-03-22T04:13:35.516052+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.534324+0000","last_change":"2024-03-22T04:13:35.528988+0000","last_active":"2024-03-22T04:13:35.534324+0000","last_peered":"2024-03-22T04:13:35.534324+0000","last_clean":"2024-03-22T04:13:35.534324+0000","last_became_active":"2024-03-22T04:13:35.528529+0000","last_became_peered":"2024-03-22T04:13:35.528529+0000","last_unstale":"2024-03-22T04:13:35.534324+0000","last_undegraded":"2024-03-22T04:13:35.534324+0000","last_fullsized":"2024-03-22T04:13:35.534324+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:49:50.872054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":247,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-03-22T04:13:35.517988+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:13:35.517988+0000","last_peered":"2024-03-22T04:13:35.517988+0000","last_clean":"2024-03-22T04:13:35.517988+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:13:35.517988+0000","last_undegraded":"2024-03-22T04:13:35.517988+0000","last_fullsized":"2024-03-22T04:13:35.517988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":778240,"data_stored":346892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1372160,"data_stored":937260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":346892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":930593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":340225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":361333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:13:41.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2024-03-22T04:13:41.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.521460+0000', 'last_change': '2024-03-22T04:13:35.521460+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.521460+0000', 'last_undegraded': '2024-03-22T04:13:35.521460+0000', 'last_fullsized': '2024-03-22T04:13:35.521460+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:13:41.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2024-03-22T04:13:41.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515543+0000', 'last_change': '2024-03-22T04:13:35.515543+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515543+0000', 'last_undegraded': '2024-03-22T04:13:35.515543+0000', 'last_fullsized': '2024-03-22T04:13:35.515543+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.521592+0000', 'last_change': '2024-03-22T04:13:35.521592+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.521592+0000', 'last_undegraded': '2024-03-22T04:13:35.521592+0000', 'last_fullsized': '2024-03-22T04:13:35.521592+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.523111+0000', 'last_change': '2024-03-22T04:13:35.523111+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.523111+0000', 'last_undegraded': '2024-03-22T04:13:35.523111+0000', 'last_fullsized': '2024-03-22T04:13:35.523111+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.521553+0000', 'last_change': '2024-03-22T04:13:35.521553+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.521553+0000', 'last_undegraded': '2024-03-22T04:13:35.521553+0000', 'last_fullsized': '2024-03-22T04:13:35.521553+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515499+0000', 'last_change': '2024-03-22T04:13:35.515499+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515499+0000', 'last_undegraded': '2024-03-22T04:13:35.515499+0000', 'last_fullsized': '2024-03-22T04:13:35.515499+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.522743+0000', 'last_change': '2024-03-22T04:13:35.522743+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.522743+0000', 'last_undegraded': '2024-03-22T04:13:35.522743+0000', 'last_fullsized': '2024-03-22T04:13:35.522743+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-03-22T04:13:41.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.522009+0000', 'last_change': '2024-03-22T04:13:35.522009+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.522009+0000', 'last_undegraded': '2024-03-22T04:13:35.522009+0000', 'last_fullsized': '2024-03-22T04:13:35.522009+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.519953+0000', 'last_change': '2024-03-22T04:13:35.519953+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.519953+0000', 'last_undegraded': '2024-03-22T04:13:35.519953+0000', 'last_fullsized': '2024-03-22T04:13:35.519953+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517617+0000', 'last_change': '2024-03-22T04:13:35.517617+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517617+0000', 'last_undegraded': '2024-03-22T04:13:35.517617+0000', 'last_fullsized': '2024-03-22T04:13:35.517617+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.521802+0000', 'last_change': '2024-03-22T04:13:35.521802+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.521802+0000', 'last_undegraded': '2024-03-22T04:13:35.521802+0000', 'last_fullsized': '2024-03-22T04:13:35.521802+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.522711+0000', 'last_change': '2024-03-22T04:13:35.522711+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.522711+0000', 'last_undegraded': '2024-03-22T04:13:35.522711+0000', 'last_fullsized': '2024-03-22T04:13:35.522711+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.518599+0000', 'last_change': '2024-03-22T04:13:35.518599+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.518599+0000', 'last_undegraded': '2024-03-22T04:13:35.518599+0000', 'last_fullsized': '2024-03-22T04:13:35.518599+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-03-22T04:13:41.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.519660+0000', 'last_change': '2024-03-22T04:13:35.519660+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.519660+0000', 'last_undegraded': '2024-03-22T04:13:35.519660+0000', 'last_fullsized': '2024-03-22T04:13:35.519660+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.516498+0000', 'last_change': '2024-03-22T04:13:35.516498+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.516498+0000', 'last_undegraded': '2024-03-22T04:13:35.516498+0000', 'last_fullsized': '2024-03-22T04:13:35.516498+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.518341+0000', 'last_change': '2024-03-22T04:13:35.518341+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.518341+0000', 'last_undegraded': '2024-03-22T04:13:35.518341+0000', 'last_fullsized': '2024-03-22T04:13:35.518341+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.518861+0000', 'last_change': '2024-03-22T04:13:35.518861+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.518861+0000', 'last_undegraded': '2024-03-22T04:13:35.518861+0000', 'last_fullsized': '2024-03-22T04:13:35.518861+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.521381+0000', 'last_change': '2024-03-22T04:13:35.521381+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.521381+0000', 'last_undegraded': '2024-03-22T04:13:35.521381+0000', 'last_fullsized': '2024-03-22T04:13:35.521381+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-03-22T04:13:41.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.518186+0000', 'last_change': '2024-03-22T04:13:35.518186+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.518186+0000', 'last_undegraded': '2024-03-22T04:13:35.518186+0000', 'last_fullsized': '2024-03-22T04:13:35.518186+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.514455+0000', 'last_change': '2024-03-22T04:13:35.514455+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.514455+0000', 'last_undegraded': '2024-03-22T04:13:35.514455+0000', 'last_fullsized': '2024-03-22T04:13:35.514455+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517884+0000', 'last_change': '2024-03-22T04:13:35.517884+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517884+0000', 'last_undegraded': '2024-03-22T04:13:35.517884+0000', 'last_fullsized': '2024-03-22T04:13:35.517884+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.520978+0000', 'last_change': '2024-03-22T04:13:35.520978+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.520978+0000', 'last_undegraded': '2024-03-22T04:13:35.520978+0000', 'last_fullsized': '2024-03-22T04:13:35.520978+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517424+0000', 'last_change': '2024-03-22T04:13:35.517424+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517424+0000', 'last_undegraded': '2024-03-22T04:13:35.517424+0000', 'last_fullsized': '2024-03-22T04:13:35.517424+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.526133+0000', 'last_change': '2024-03-22T04:13:35.526133+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.526133+0000', 'last_undegraded': '2024-03-22T04:13:35.526133+0000', 'last_fullsized': '2024-03-22T04:13:35.526133+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-03-22T04:13:41.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517842+0000', 'last_change': '2024-03-22T04:13:35.517842+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517842+0000', 'last_undegraded': '2024-03-22T04:13:35.517842+0000', 'last_fullsized': '2024-03-22T04:13:35.517842+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.521298+0000', 'last_change': '2024-03-22T04:13:35.521298+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.521298+0000', 'last_undegraded': '2024-03-22T04:13:35.521298+0000', 'last_fullsized': '2024-03-22T04:13:35.521298+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.518277+0000', 'last_change': '2024-03-22T04:13:35.518277+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.518277+0000', 'last_undegraded': '2024-03-22T04:13:35.518277+0000', 'last_fullsized': '2024-03-22T04:13:35.518277+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517462+0000', 'last_change': '2024-03-22T04:13:35.517462+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517462+0000', 'last_undegraded': '2024-03-22T04:13:35.517462+0000', 'last_fullsized': '2024-03-22T04:13:35.517462+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.520275+0000', 'last_change': '2024-03-22T04:13:35.520275+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.520275+0000', 'last_undegraded': '2024-03-22T04:13:35.520275+0000', 'last_fullsized': '2024-03-22T04:13:35.520275+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.520777+0000', 'last_change': '2024-03-22T04:13:35.520777+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.520777+0000', 'last_undegraded': '2024-03-22T04:13:35.520777+0000', 'last_fullsized': '2024-03-22T04:13:35.520777+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-03-22T04:13:41.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.522255+0000', 'last_change': '2024-03-22T04:13:35.522255+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.522255+0000', 'last_undegraded': '2024-03-22T04:13:35.522255+0000', 'last_fullsized': '2024-03-22T04:13:35.522255+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.516557+0000', 'last_change': '2024-03-22T04:13:35.516557+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.516557+0000', 'last_undegraded': '2024-03-22T04:13:35.516557+0000', 'last_fullsized': '2024-03-22T04:13:35.516557+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515064+0000', 'last_change': '2024-03-22T04:13:35.515064+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515064+0000', 'last_undegraded': '2024-03-22T04:13:35.515064+0000', 'last_fullsized': '2024-03-22T04:13:35.515064+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.519480+0000', 'last_change': '2024-03-22T04:13:35.519480+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.519480+0000', 'last_undegraded': '2024-03-22T04:13:35.519480+0000', 'last_fullsized': '2024-03-22T04:13:35.519480+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.522326+0000', 'last_change': '2024-03-22T04:13:35.522326+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.522326+0000', 'last_undegraded': '2024-03-22T04:13:35.522326+0000', 'last_fullsized': '2024-03-22T04:13:35.522326+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-03-22T04:13:41.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.516439+0000', 'last_change': '2024-03-22T04:13:35.516439+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.516439+0000', 'last_undegraded': '2024-03-22T04:13:35.516439+0000', 'last_fullsized': '2024-03-22T04:13:35.516439+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:13:41.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-03-22T04:13:41.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.519164+0000', 'last_change': '2024-03-22T04:13:35.519164+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.519164+0000', 'last_undegraded': '2024-03-22T04:13:35.519164+0000', 'last_fullsized': '2024-03-22T04:13:35.519164+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:13:41.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-03-22T04:13:41.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515798+0000', 'last_change': '2024-03-22T04:13:35.515798+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515798+0000', 'last_undegraded': '2024-03-22T04:13:35.515798+0000', 'last_fullsized': '2024-03-22T04:13:35.515798+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-03-22T04:13:41.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.518642+0000', 'last_change': '2024-03-22T04:13:35.518642+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.518642+0000', 'last_undegraded': '2024-03-22T04:13:35.518642+0000', 'last_fullsized': '2024-03-22T04:13:35.518642+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-03-22T04:13:41.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515862+0000', 'last_change': '2024-03-22T04:13:35.515862+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515862+0000', 'last_undegraded': '2024-03-22T04:13:35.515862+0000', 'last_fullsized': '2024-03-22T04:13:35.515862+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.516023+0000', 'last_change': '2024-03-22T04:13:35.516023+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.516023+0000', 'last_undegraded': '2024-03-22T04:13:35.516023+0000', 'last_fullsized': '2024-03-22T04:13:35.516023+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.516963+0000', 'last_change': '2024-03-22T04:13:35.516963+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.516963+0000', 'last_undegraded': '2024-03-22T04:13:35.516963+0000', 'last_fullsized': '2024-03-22T04:13:35.516963+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.519326+0000', 'last_change': '2024-03-22T04:13:35.519326+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.519326+0000', 'last_undegraded': '2024-03-22T04:13:35.519326+0000', 'last_fullsized': '2024-03-22T04:13:35.519326+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517925+0000', 'last_change': '2024-03-22T04:13:35.517925+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517925+0000', 'last_undegraded': '2024-03-22T04:13:35.517925+0000', 'last_fullsized': '2024-03-22T04:13:35.517925+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515262+0000', 'last_change': '2024-03-22T04:13:35.515262+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515262+0000', 'last_undegraded': '2024-03-22T04:13:35.515262+0000', 'last_fullsized': '2024-03-22T04:13:35.515262+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-03-22T04:13:41.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515325+0000', 'last_change': '2024-03-22T04:13:35.515325+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515325+0000', 'last_undegraded': '2024-03-22T04:13:35.515325+0000', 'last_fullsized': '2024-03-22T04:13:35.515325+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.518273+0000', 'last_change': '2024-03-22T04:13:35.518273+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.518273+0000', 'last_undegraded': '2024-03-22T04:13:35.518273+0000', 'last_fullsized': '2024-03-22T04:13:35.518273+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517337+0000', 'last_change': '2024-03-22T04:13:35.517337+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517337+0000', 'last_undegraded': '2024-03-22T04:13:35.517337+0000', 'last_fullsized': '2024-03-22T04:13:35.517337+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515728+0000', 'last_change': '2024-03-22T04:13:35.515728+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515728+0000', 'last_undegraded': '2024-03-22T04:13:35.515728+0000', 'last_fullsized': '2024-03-22T04:13:35.515728+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.516973+0000', 'last_change': '2024-03-22T04:13:35.516973+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.516973+0000', 'last_undegraded': '2024-03-22T04:13:35.516973+0000', 'last_fullsized': '2024-03-22T04:13:35.516973+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517454+0000', 'last_change': '2024-03-22T04:13:35.517454+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517454+0000', 'last_undegraded': '2024-03-22T04:13:35.517454+0000', 'last_fullsized': '2024-03-22T04:13:35.517454+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.517863+0000', 'last_change': '2024-03-22T04:13:35.517863+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.517863+0000', 'last_undegraded': '2024-03-22T04:13:35.517863+0000', 'last_fullsized': '2024-03-22T04:13:35.517863+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.516763+0000', 'last_change': '2024-03-22T04:13:35.516763+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.516763+0000', 'last_undegraded': '2024-03-22T04:13:35.516763+0000', 'last_fullsized': '2024-03-22T04:13:35.516763+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.514570+0000', 'last_change': '2024-03-22T04:13:35.514570+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.514570+0000', 'last_undegraded': '2024-03-22T04:13:35.514570+0000', 'last_fullsized': '2024-03-22T04:13:35.514570+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.515001+0000', 'last_change': '2024-03-22T04:13:35.515001+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.515001+0000', 'last_undegraded': '2024-03-22T04:13:35.515001+0000', 'last_fullsized': '2024-03-22T04:13:35.515001+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-03-22T04:13:41.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-03-22T04:13:35.516052+0000', 'last_change': '2024-03-22T04:13:35.516052+0000', 'last_active': '2024-03-22T04:13:35.502450+0000', 'last_peered': '2024-03-22T04:13:35.502450+0000', 'last_clean': '2024-03-22T04:13:35.502450+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-22T04:13:35.516052+0000', 'last_undegraded': '2024-03-22T04:13:35.516052+0000', 'last_fullsized': '2024-03-22T04:13:35.516052+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'last_clean_scrub_stamp': '2024-03-22T04:13:35.502450+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-22T04:13:41.359 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-22T04:13:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1632a00640 1 Processor -- start 2024-03-22T04:13:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1632a00640 1 -- start start 2024-03-22T04:13:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1632a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f162c15ecb0 0x7f162c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1632a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15b380 0x7f162c15f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1632a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f162c0b2f80 0x7f162c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1632a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f162c072f90 con 0x7f162c0b2f80 2024-03-22T04:13:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1632a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f162c0721d0 con 0x7f162c15ecb0 2024-03-22T04:13:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1632a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f162c0724b0 con 0x7f162c15b380 2024-03-22T04:13:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1630a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f162c0b2f80 0x7f162c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f162c15ecb0 0x7f162c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162b400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15b380 0x7f162c15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f162c15ecb0 0x7f162c06d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:37422/0 (socket says 172.21.15.86:37422) 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162be00640 1 -- 172.21.15.86:0/3180125163 learned_addr learned my addr 172.21.15.86:0/3180125163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1630a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f162c0b2f80 0x7f162c0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45898/0 (socket says 172.21.15.86:45898) 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162be00640 1 -- 172.21.15.86:0/3180125163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15b380 msgr2=0x7f162c15f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162be00640 1 --2- 172.21.15.86:0/3180125163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15b380 0x7f162c15f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162be00640 1 -- 172.21.15.86:0/3180125163 >> v2:172.21.15.86:3300/0 conn(0x7f162c0b2f80 msgr2=0x7f162c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162be00640 1 --2- 172.21.15.86:0/3180125163 >> v2:172.21.15.86:3300/0 conn(0x7f162c0b2f80 0x7f162c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f162be00640 1 -- 172.21.15.86:0/3180125163 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c072790 con 0x7f162c15ecb0 2024-03-22T04:13:41.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.520+0000 7f1630a00640 1 --2- 172.21.15.86:0/3180125163 >> v2:172.21.15.86:3300/0 conn(0x7f162c0b2f80 0x7f162c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f162be00640 1 --2- 172.21.15.86:0/3180125163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15ecb0 0x7f162c06d940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f162c06fb10 tx=0x7f161401ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4223d7a375b6088e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f162aa00640 1 -- 172.21.15.86:0/3180125163 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140b5070 con 0x7f162c15ecb0 2024-03-22T04:13:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f162aa00640 1 -- 172.21.15.86:0/3180125163 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16140a89d0 con 0x7f162c15ecb0 2024-03-22T04:13:41.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f162aa00640 1 -- 172.21.15.86:0/3180125163 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140a8da0 con 0x7f162c15ecb0 2024-03-22T04:13:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 -- 172.21.15.86:0/3180125163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15ecb0 msgr2=0x7f162c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 --2- 172.21.15.86:0/3180125163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15ecb0 0x7f162c06d940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f162c06fb10 tx=0x7f161401ed10 comp rx=0 tx=0).stop 2024-03-22T04:13:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 -- 172.21.15.86:0/3180125163 shutdown_connections 2024-03-22T04:13:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 --2- 172.21.15.86:0/3180125163 >> v2:172.21.15.86:3300/0 conn(0x7f162c0b2f80 0x7f162c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 --2- 172.21.15.86:0/3180125163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15b380 0x7f162c15f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 --2- 172.21.15.86:0/3180125163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15ecb0 0x7f162c06d940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 -- 172.21.15.86:0/3180125163 >> 172.21.15.86:0/3180125163 conn(0x7f162c0b1c80 msgr2=0x7f162c0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 -- 172.21.15.86:0/3180125163 shutdown_connections 2024-03-22T04:13:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.524+0000 7f1632a00640 1 -- 172.21.15.86:0/3180125163 wait complete. 2024-03-22T04:13:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 Processor -- start 2024-03-22T04:13:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 -- start start 2024-03-22T04:13:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f162c0b2f80 0x7f162c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15b380 0x7f162c13ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15ecb0 0x7f162c14f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0b3950 con 0x7f162c0b2f80 2024-03-22T04:13:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f162c15f5d0 con 0x7f162c15b380 2024-03-22T04:13:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f162c0b3ad0 con 0x7f162c15ecb0 2024-03-22T04:13:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f162c0b2f80 0x7f162c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f162c0b2f80 0x7f162c0b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45914/0 (socket says 172.21.15.86:45914) 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 -- 172.21.15.86:0/2308690377 learned_addr learned my addr 172.21.15.86:0/2308690377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162b400640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15b380 0x7f162c13ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1630a00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15ecb0 0x7f162c14f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 -- 172.21.15.86:0/2308690377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15ecb0 msgr2=0x7f162c14f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15ecb0 0x7f162c14f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 -- 172.21.15.86:0/2308690377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15b380 msgr2=0x7f162c13ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15b380 0x7f162c13ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 -- 172.21.15.86:0/2308690377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c15e250 con 0x7f162c0b2f80 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f162be00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f162c0b2f80 0x7f162c0b08b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f16140b3040 tx=0x7f161401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1623e00640 1 -- 172.21.15.86:0/2308690377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140c0070 con 0x7f162c0b2f80 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1623e00640 1 -- 172.21.15.86:0/2308690377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16140b0660 con 0x7f162c0b2f80 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.528+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c09b6b0 con 0x7f162c0b2f80 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.532+0000 7f1623e00640 1 -- 172.21.15.86:0/2308690377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140b5040 con 0x7f162c0b2f80 2024-03-22T04:13:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.532+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c09bc70 con 0x7f162c0b2f80 2024-03-22T04:13:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.532+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c074550 con 0x7f162c0b2f80 2024-03-22T04:13:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.532+0000 7f1623e00640 1 -- 172.21.15.86:0/2308690377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f16140c3070 con 0x7f162c0b2f80 2024-03-22T04:13:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.532+0000 7f1623e00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1608067b50 0x7f160806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.532+0000 7f1623e00640 1 -- 172.21.15.86:0/2308690377 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f1614122d50 con 0x7f162c0b2f80 2024-03-22T04:13:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.532+0000 7f162b400640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1608067b50 0x7f160806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.532+0000 7f162b400640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1608067b50 0x7f160806a010 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f162c0ad610 tx=0x7f161c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.536+0000 7f1623e00640 1 -- 172.21.15.86:0/2308690377 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1614018870 con 0x7f162c0b2f80 2024-03-22T04:13:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.748+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f162c06f300 con 0x7f162c0b2f80 2024-03-22T04:13:41.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.748+0000 7f1623e00640 1 -- 172.21.15.86:0/2308690377 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1196 (secure 0 0 0) 0x7f16140f2e50 con 0x7f162c0b2f80 2024-03-22T04:13:41.754 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:41.754 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":211,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":8,"num_up_osds":8,"osd_up_since":1711080620,"num_in_osds":8,"osd_in_since":1711080611,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"active+clean","count":41}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":233029632,"bytes_avail":772861083648,"bytes_total":773094113280,"inactive_pgs_ratio":0.57731956243515015},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":75617}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-03-22T04:13:31.829115+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1608067b50 msgr2=0x7f160806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1608067b50 0x7f160806a010 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f162c0ad610 tx=0x7f161c009190 comp rx=0 tx=0).stop 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f162c0b2f80 msgr2=0x7f162c0b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f162c0b2f80 0x7f162c0b08b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f16140b3040 tx=0x7f161401bab0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 shutdown_connections 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1608067b50 0x7f160806a010 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c15b380 0x7f162c13ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f162c15ecb0 0x7f162c14f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 --2- 172.21.15.86:0/2308690377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f162c0b2f80 0x7f162c0b08b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 >> 172.21.15.86:0/2308690377 conn(0x7f162c0b1c80 msgr2=0x7f162c151260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 shutdown_connections 2024-03-22T04:13:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.752+0000 7f1632a00640 1 -- 172.21.15.86:0/2308690377 wait complete. 2024-03-22T04:13:41.775 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-22T04:13:41.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:13:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32f0600640 1 Processor -- start 2024-03-22T04:13:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32f0600640 1 -- start start 2024-03-22T04:13:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32f0600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e806ef40 0x7f32e8142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32f0600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e815c7c0 con 0x7f32e80ad710 2024-03-22T04:13:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32f0600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e806dea0 con 0x7f32e80ad330 2024-03-22T04:13:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32f0600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f32e806e180 con 0x7f32e806ef40 2024-03-22T04:13:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32ee600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f32e80ad710 0x7f32e81402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38720/0 (socket says 172.21.15.86:38720) 2024-03-22T04:13:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32ee600640 1 -- 172.21.15.86:0/4168729445 learned_addr learned my addr 172.21.15.86:0/4168729445 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32ed200640 1 --2- 172.21.15.86:0/4168729445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e806ef40 0x7f32e8142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32e7e00640 1 -- 172.21.15.86:0/4168729445 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1319534201 0 0) 0x7f32e815c7c0 con 0x7f32e80ad710 2024-03-22T04:13:41.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32e7e00640 1 -- 172.21.15.86:0/4168729445 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32d0003530 con 0x7f32e80ad710 2024-03-22T04:13:41.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32ed200640 1 -- 172.21.15.86:0/4168729445 >> v1:172.21.15.196:6789/0 conn(0x7f32e80ad330 legacy=0x7f32e813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32e7e00640 1 -- 172.21.15.86:0/4168729445 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303875055 0 0) 0x7f32e806dea0 con 0x7f32e80ad330 2024-03-22T04:13:41.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32ed200640 1 -- 172.21.15.86:0/4168729445 >> v1:172.21.15.86:6789/0 conn(0x7f32e80ad710 legacy=0x7f32e81402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.908+0000 7f32ed200640 1 -- 172.21.15.86:0/4168729445 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e806e460 con 0x7f32e806ef40 2024-03-22T04:13:41.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32ed200640 1 --2- 172.21.15.86:0/4168729445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e806ef40 0x7f32e8142690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097f0 tx=0x7f32d401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c02856242fe0f7c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32e7e00640 1 -- 172.21.15.86:0/4168729445 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40aa430 con 0x7f32e806ef40 2024-03-22T04:13:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32e7e00640 1 -- 172.21.15.86:0/4168729445 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d40af070 con 0x7f32e806ef40 2024-03-22T04:13:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32e7e00640 1 -- 172.21.15.86:0/4168729445 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40aa7f0 con 0x7f32e806ef40 2024-03-22T04:13:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32f0600640 1 -- 172.21.15.86:0/4168729445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e806ef40 msgr2=0x7f32e8142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32f0600640 1 --2- 172.21.15.86:0/4168729445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e806ef40 0x7f32e8142690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097f0 tx=0x7f32d401c4b0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32f0600640 1 -- 172.21.15.86:0/4168729445 shutdown_connections 2024-03-22T04:13:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32f0600640 1 --2- 172.21.15.86:0/4168729445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e806ef40 0x7f32e8142690 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32f0600640 1 -- 172.21.15.86:0/4168729445 >> 172.21.15.86:0/4168729445 conn(0x7f32e80b1df0 msgr2=0x7f32e80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32f0600640 1 -- 172.21.15.86:0/4168729445 shutdown_connections 2024-03-22T04:13:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.912+0000 7f32f0600640 1 -- 172.21.15.86:0/4168729445 wait complete. 2024-03-22T04:13:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 Processor -- start 2024-03-22T04:13:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 -- start start 2024-03-22T04:13:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e806ef40 0x7f32e809bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32e80ad330 0x7f32e8156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e80ad710 0x7f32e8138890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32e806db10 con 0x7f32e806ef40 2024-03-22T04:13:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81409d0 con 0x7f32e80ad330 2024-03-22T04:13:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f32e816a780 con 0x7f32e80ad710 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e806ef40 0x7f32e809bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e806ef40 0x7f32e809bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45918/0 (socket says 172.21.15.86:45918) 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 -- 172.21.15.86:0/2933211809 learned_addr learned my addr 172.21.15.86:0/2933211809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32ee600640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e80ad710 0x7f32e8138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32ed200640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32e80ad330 0x7f32e8156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 -- 172.21.15.86:0/2933211809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e80ad710 msgr2=0x7f32e8138890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e80ad710 0x7f32e8138890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 -- 172.21.15.86:0/2933211809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32e80ad330 msgr2=0x7f32e8156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32e80ad330 0x7f32e8156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 -- 172.21.15.86:0/2933211809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e809ac90 con 0x7f32e806ef40 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32edc00640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e806ef40 0x7f32e809bd30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00c4d0 tx=0x7f32dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32e5600640 1 -- 172.21.15.86:0/2933211809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc017070 con 0x7f32e806ef40 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32e5600640 1 -- 172.21.15.86:0/2933211809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc005480 con 0x7f32e806ef40 2024-03-22T04:13:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e809af70 con 0x7f32e806ef40 2024-03-22T04:13:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32e5600640 1 -- 172.21.15.86:0/2933211809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc003840 con 0x7f32e806ef40 2024-03-22T04:13:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.916+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e809b530 con 0x7f32e806ef40 2024-03-22T04:13:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.920+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e8068650 con 0x7f32e806ef40 2024-03-22T04:13:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.920+0000 7f32e5600640 1 -- 172.21.15.86:0/2933211809 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f32dc01a070 con 0x7f32e806ef40 2024-03-22T04:13:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.920+0000 7f32e5600640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f32c8067b20 0x7f32c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.920+0000 7f32e5600640 1 -- 172.21.15.86:0/2933211809 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f32dc0853b0 con 0x7f32e806ef40 2024-03-22T04:13:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.920+0000 7f32ed200640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f32c8067b20 0x7f32c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:41.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.920+0000 7f32ed200640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f32c8067b20 0x7f32c8069fe0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32d40ae000 tx=0x7f32d40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:41.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:41.924+0000 7f32e5600640 1 -- 172.21.15.86:0/2933211809 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f32dc0556a0 con 0x7f32e806ef40 2024-03-22T04:13:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.076+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32e8158470 con 0x7f32c8067b20 2024-03-22T04:13:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.080+0000 7f32e5600640 1 -- 172.21.15.86:0/2933211809 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252024 (secure 0 0 0) 0x7f32e8158470 con 0x7f32c8067b20 2024-03-22T04:13:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:42.090 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f32c8067b20 msgr2=0x7f32c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f32c8067b20 0x7f32c8069fe0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32d40ae000 tx=0x7f32d40a7000 comp rx=0 tx=0).stop 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e806ef40 msgr2=0x7f32e809bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e806ef40 0x7f32e809bd30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00c4d0 tx=0x7f32dc00c990 comp rx=0 tx=0).stop 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 shutdown_connections 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f32c8067b20 0x7f32c8069fe0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32e80ad330 0x7f32e8156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32e80ad710 0x7f32e8138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 --2- 172.21.15.86:0/2933211809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e806ef40 0x7f32e809bd30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:42.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 >> 172.21.15.86:0/2933211809 conn(0x7f32e80b1df0 msgr2=0x7f32e80b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:42.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 shutdown_connections 2024-03-22T04:13:42.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:42.084+0000 7f32f0600640 1 -- 172.21.15.86:0/2933211809 wait complete. 2024-03-22T04:13:42.110 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-03-22T04:13:41.830703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231988,"kb_used_data":7496,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754742732,"statfs":{"total":773094113280,"available":772856557568,"internally_reserved":0,"allocated":7675904,"data_stored":4071564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.320542"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935185+0000","last_change":"2024-03-22T04:13:36.536643+0000","last_active":"2024-03-22T04:13:37.935185+0000","last_peered":"2024-03-22T04:13:37.935185+0000","last_clean":"2024-03-22T04:13:37.935185+0000","last_became_active":"2024-03-22T04:13:36.536338+0000","last_became_peered":"2024-03-22T04:13:36.536338+0000","last_unstale":"2024-03-22T04:13:37.935185+0000","last_undegraded":"2024-03-22T04:13:37.935185+0000","last_fullsized":"2024-03-22T04:13:37.935185+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:00:47.125992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533027+0000","last_change":"2024-03-22T04:13:36.532727+0000","last_active":"2024-03-22T04:13:37.533027+0000","last_peered":"2024-03-22T04:13:37.533027+0000","last_clean":"2024-03-22T04:13:37.533027+0000","last_became_active":"2024-03-22T04:13:36.532232+0000","last_became_peered":"2024-03-22T04:13:36.532232+0000","last_unstale":"2024-03-22T04:13:37.533027+0000","last_undegraded":"2024-03-22T04:13:37.533027+0000","last_fullsized":"2024-03-22T04:13:37.533027+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:56:07.942982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934770+0000","last_change":"2024-03-22T04:13:36.550839+0000","last_active":"2024-03-22T04:13:37.934770+0000","last_peered":"2024-03-22T04:13:37.934770+0000","last_clean":"2024-03-22T04:13:37.934770+0000","last_became_active":"2024-03-22T04:13:36.550536+0000","last_became_peered":"2024-03-22T04:13:36.550536+0000","last_unstale":"2024-03-22T04:13:37.934770+0000","last_undegraded":"2024-03-22T04:13:37.934770+0000","last_fullsized":"2024-03-22T04:13:37.934770+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:33:43.016697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537165+0000","last_change":"2024-03-22T04:13:36.535093+0000","last_active":"2024-03-22T04:13:37.537165+0000","last_peered":"2024-03-22T04:13:37.537165+0000","last_clean":"2024-03-22T04:13:37.537165+0000","last_became_active":"2024-03-22T04:13:36.533587+0000","last_became_peered":"2024-03-22T04:13:36.533587+0000","last_unstale":"2024-03-22T04:13:37.537165+0000","last_undegraded":"2024-03-22T04:13:37.537165+0000","last_fullsized":"2024-03-22T04:13:37.537165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:56:07.942982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934467+0000","last_change":"2024-03-22T04:13:36.541445+0000","last_active":"2024-03-22T04:13:37.934467+0000","last_peered":"2024-03-22T04:13:37.934467+0000","last_clean":"2024-03-22T04:13:37.934467+0000","last_became_active":"2024-03-22T04:13:36.540929+0000","last_became_peered":"2024-03-22T04:13:36.540929+0000","last_unstale":"2024-03-22T04:13:37.934467+0000","last_undegraded":"2024-03-22T04:13:37.934467+0000","last_fullsized":"2024-03-22T04:13:37.934467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:18:51.428914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532301+0000","last_change":"2024-03-22T04:13:36.528925+0000","last_active":"2024-03-22T04:13:37.532301+0000","last_peered":"2024-03-22T04:13:37.532301+0000","last_clean":"2024-03-22T04:13:37.532301+0000","last_became_active":"2024-03-22T04:13:36.528306+0000","last_became_peered":"2024-03-22T04:13:36.528306+0000","last_unstale":"2024-03-22T04:13:37.532301+0000","last_undegraded":"2024-03-22T04:13:37.532301+0000","last_fullsized":"2024-03-22T04:13:37.532301+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:04:53.585258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934798+0000","last_change":"2024-03-22T04:13:36.537022+0000","last_active":"2024-03-22T04:13:37.934798+0000","last_peered":"2024-03-22T04:13:37.934798+0000","last_clean":"2024-03-22T04:13:37.934798+0000","last_became_active":"2024-03-22T04:13:36.536647+0000","last_became_peered":"2024-03-22T04:13:36.536647+0000","last_unstale":"2024-03-22T04:13:37.934798+0000","last_undegraded":"2024-03-22T04:13:37.934798+0000","last_fullsized":"2024-03-22T04:13:37.934798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:52:03.782357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934493+0000","last_change":"2024-03-22T04:13:36.546972+0000","last_active":"2024-03-22T04:13:37.934493+0000","last_peered":"2024-03-22T04:13:37.934493+0000","last_clean":"2024-03-22T04:13:37.934493+0000","last_became_active":"2024-03-22T04:13:36.546730+0000","last_became_peered":"2024-03-22T04:13:36.546730+0000","last_unstale":"2024-03-22T04:13:37.934493+0000","last_undegraded":"2024-03-22T04:13:37.934493+0000","last_fullsized":"2024-03-22T04:13:37.934493+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:01:01.082434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537330+0000","last_change":"2024-03-22T04:13:36.536370+0000","last_active":"2024-03-22T04:13:37.537330+0000","last_peered":"2024-03-22T04:13:37.537330+0000","last_clean":"2024-03-22T04:13:37.537330+0000","last_became_active":"2024-03-22T04:13:36.536070+0000","last_became_peered":"2024-03-22T04:13:36.536070+0000","last_unstale":"2024-03-22T04:13:37.537330+0000","last_undegraded":"2024-03-22T04:13:37.537330+0000","last_fullsized":"2024-03-22T04:13:37.537330+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:32.466308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533436+0000","last_change":"2024-03-22T04:13:36.534670+0000","last_active":"2024-03-22T04:13:37.533436+0000","last_peered":"2024-03-22T04:13:37.533436+0000","last_clean":"2024-03-22T04:13:37.533436+0000","last_became_active":"2024-03-22T04:13:36.534356+0000","last_became_peered":"2024-03-22T04:13:36.534356+0000","last_unstale":"2024-03-22T04:13:37.533436+0000","last_undegraded":"2024-03-22T04:13:37.533436+0000","last_fullsized":"2024-03-22T04:13:37.533436+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:22.612979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537416+0000","last_change":"2024-03-22T04:13:36.539182+0000","last_active":"2024-03-22T04:13:37.537416+0000","last_peered":"2024-03-22T04:13:37.537416+0000","last_clean":"2024-03-22T04:13:37.537416+0000","last_became_active":"2024-03-22T04:13:36.538916+0000","last_became_peered":"2024-03-22T04:13:36.538916+0000","last_unstale":"2024-03-22T04:13:37.537416+0000","last_undegraded":"2024-03-22T04:13:37.537416+0000","last_fullsized":"2024-03-22T04:13:37.537416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:43:46.967166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537312+0000","last_change":"2024-03-22T04:13:36.543786+0000","last_active":"2024-03-22T04:13:37.537312+0000","last_peered":"2024-03-22T04:13:37.537312+0000","last_clean":"2024-03-22T04:13:37.537312+0000","last_became_active":"2024-03-22T04:13:36.543194+0000","last_became_peered":"2024-03-22T04:13:36.543194+0000","last_unstale":"2024-03-22T04:13:37.537312+0000","last_undegraded":"2024-03-22T04:13:37.537312+0000","last_fullsized":"2024-03-22T04:13:37.537312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:01.059946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935143+0000","last_change":"2024-03-22T04:13:36.533115+0000","last_active":"2024-03-22T04:13:37.935143+0000","last_peered":"2024-03-22T04:13:37.935143+0000","last_clean":"2024-03-22T04:13:37.935143+0000","last_became_active":"2024-03-22T04:13:36.532940+0000","last_became_peered":"2024-03-22T04:13:36.532940+0000","last_unstale":"2024-03-22T04:13:37.935143+0000","last_undegraded":"2024-03-22T04:13:37.935143+0000","last_fullsized":"2024-03-22T04:13:37.935143+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:44.092644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934973+0000","last_change":"2024-03-22T04:13:36.531402+0000","last_active":"2024-03-22T04:13:37.934973+0000","last_peered":"2024-03-22T04:13:37.934973+0000","last_clean":"2024-03-22T04:13:37.934973+0000","last_became_active":"2024-03-22T04:13:36.531066+0000","last_became_peered":"2024-03-22T04:13:36.531066+0000","last_unstale":"2024-03-22T04:13:37.934973+0000","last_undegraded":"2024-03-22T04:13:37.934973+0000","last_fullsized":"2024-03-22T04:13:37.934973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:57.953068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532968+0000","last_change":"2024-03-22T04:13:36.528859+0000","last_active":"2024-03-22T04:13:37.532968+0000","last_peered":"2024-03-22T04:13:37.532968+0000","last_clean":"2024-03-22T04:13:37.532968+0000","last_became_active":"2024-03-22T04:13:36.528573+0000","last_became_peered":"2024-03-22T04:13:36.528573+0000","last_unstale":"2024-03-22T04:13:37.532968+0000","last_undegraded":"2024-03-22T04:13:37.532968+0000","last_fullsized":"2024-03-22T04:13:37.532968+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:40:07.797167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.541124+0000","last_change":"2024-03-22T04:13:36.544779+0000","last_active":"2024-03-22T04:13:37.541124+0000","last_peered":"2024-03-22T04:13:37.541124+0000","last_clean":"2024-03-22T04:13:37.541124+0000","last_became_active":"2024-03-22T04:13:36.544484+0000","last_became_peered":"2024-03-22T04:13:36.544484+0000","last_unstale":"2024-03-22T04:13:37.541124+0000","last_undegraded":"2024-03-22T04:13:37.541124+0000","last_fullsized":"2024-03-22T04:13:37.541124+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:51.911920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534709+0000","last_change":"2024-03-22T04:13:36.534928+0000","last_active":"2024-03-22T04:13:37.534709+0000","last_peered":"2024-03-22T04:13:37.534709+0000","last_clean":"2024-03-22T04:13:37.534709+0000","last_became_active":"2024-03-22T04:13:36.534458+0000","last_became_peered":"2024-03-22T04:13:36.534458+0000","last_unstale":"2024-03-22T04:13:37.534709+0000","last_undegraded":"2024-03-22T04:13:37.534709+0000","last_fullsized":"2024-03-22T04:13:37.534709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:10:28.784356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537311+0000","last_change":"2024-03-22T04:13:36.543008+0000","last_active":"2024-03-22T04:13:37.537311+0000","last_peered":"2024-03-22T04:13:37.537311+0000","last_clean":"2024-03-22T04:13:37.537311+0000","last_became_active":"2024-03-22T04:13:36.542504+0000","last_became_peered":"2024-03-22T04:13:36.542504+0000","last_unstale":"2024-03-22T04:13:37.537311+0000","last_undegraded":"2024-03-22T04:13:37.537311+0000","last_fullsized":"2024-03-22T04:13:37.537311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:17:30.787284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537715+0000","last_change":"2024-03-22T04:13:36.539156+0000","last_active":"2024-03-22T04:13:37.537715+0000","last_peered":"2024-03-22T04:13:37.537715+0000","last_clean":"2024-03-22T04:13:37.537715+0000","last_became_active":"2024-03-22T04:13:36.538851+0000","last_became_peered":"2024-03-22T04:13:36.538851+0000","last_unstale":"2024-03-22T04:13:37.537715+0000","last_undegraded":"2024-03-22T04:13:37.537715+0000","last_fullsized":"2024-03-22T04:13:37.537715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:39:58.057283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534466+0000","last_change":"2024-03-22T04:13:36.533555+0000","last_active":"2024-03-22T04:13:37.534466+0000","last_peered":"2024-03-22T04:13:37.534466+0000","last_clean":"2024-03-22T04:13:37.534466+0000","last_became_active":"2024-03-22T04:13:36.533261+0000","last_became_peered":"2024-03-22T04:13:36.533261+0000","last_unstale":"2024-03-22T04:13:37.534466+0000","last_undegraded":"2024-03-22T04:13:37.534466+0000","last_fullsized":"2024-03-22T04:13:37.534466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:22.636235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935254+0000","last_change":"2024-03-22T04:13:36.529562+0000","last_active":"2024-03-22T04:13:37.935254+0000","last_peered":"2024-03-22T04:13:37.935254+0000","last_clean":"2024-03-22T04:13:37.935254+0000","last_became_active":"2024-03-22T04:13:36.529315+0000","last_became_peered":"2024-03-22T04:13:36.529315+0000","last_unstale":"2024-03-22T04:13:37.935254+0000","last_undegraded":"2024-03-22T04:13:37.935254+0000","last_fullsized":"2024-03-22T04:13:37.935254+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:15:12.469198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532970+0000","last_change":"2024-03-22T04:13:36.529136+0000","last_active":"2024-03-22T04:13:37.532970+0000","last_peered":"2024-03-22T04:13:37.532970+0000","last_clean":"2024-03-22T04:13:37.532970+0000","last_became_active":"2024-03-22T04:13:36.528642+0000","last_became_peered":"2024-03-22T04:13:36.528642+0000","last_unstale":"2024-03-22T04:13:37.532970+0000","last_undegraded":"2024-03-22T04:13:37.532970+0000","last_fullsized":"2024-03-22T04:13:37.532970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:13:42.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.541070+0000","last_change":"2024-03-22T04:13:36.544284+0000","last_active":"2024-03-22T04:13:37.541070+0000","last_peered":"2024-03-22T04:13:37.541070+0000","last_clean":"2024-03-22T04:13:37.541070+0000","last_became_active":"2024-03-22T04:13:36.543996+0000","last_became_peered":"2024-03-22T04:13:36.543996+0000","last_unstale":"2024-03-22T04:13:37.541070+0000","last_undegraded":"2024-03-22T04:13:37.541070+0000","last_fullsized":"2024-03-22T04:13:37.541070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:13.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934473+0000","last_change":"2024-03-22T04:13:36.540542+0000","last_active":"2024-03-22T04:13:37.934473+0000","last_peered":"2024-03-22T04:13:37.934473+0000","last_clean":"2024-03-22T04:13:37.934473+0000","last_became_active":"2024-03-22T04:13:36.540215+0000","last_became_peered":"2024-03-22T04:13:36.540215+0000","last_unstale":"2024-03-22T04:13:37.934473+0000","last_undegraded":"2024-03-22T04:13:37.934473+0000","last_fullsized":"2024-03-22T04:13:37.934473+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:36.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935514+0000","last_change":"2024-03-22T04:13:36.532645+0000","last_active":"2024-03-22T04:13:37.935514+0000","last_peered":"2024-03-22T04:13:37.935514+0000","last_clean":"2024-03-22T04:13:37.935514+0000","last_became_active":"2024-03-22T04:13:36.531626+0000","last_became_peered":"2024-03-22T04:13:36.531626+0000","last_unstale":"2024-03-22T04:13:37.935514+0000","last_undegraded":"2024-03-22T04:13:37.935514+0000","last_fullsized":"2024-03-22T04:13:37.935514+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:22:16.574264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934597+0000","last_change":"2024-03-22T04:13:36.546688+0000","last_active":"2024-03-22T04:13:37.934597+0000","last_peered":"2024-03-22T04:13:37.934597+0000","last_clean":"2024-03-22T04:13:37.934597+0000","last_became_active":"2024-03-22T04:13:36.546381+0000","last_became_peered":"2024-03-22T04:13:36.546381+0000","last_unstale":"2024-03-22T04:13:37.934597+0000","last_undegraded":"2024-03-22T04:13:37.934597+0000","last_fullsized":"2024-03-22T04:13:37.934597+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:27:03.605752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935247+0000","last_change":"2024-03-22T04:13:36.541919+0000","last_active":"2024-03-22T04:13:37.935247+0000","last_peered":"2024-03-22T04:13:37.935247+0000","last_clean":"2024-03-22T04:13:37.935247+0000","last_became_active":"2024-03-22T04:13:36.541564+0000","last_became_peered":"2024-03-22T04:13:36.541564+0000","last_unstale":"2024-03-22T04:13:37.935247+0000","last_undegraded":"2024-03-22T04:13:37.935247+0000","last_fullsized":"2024-03-22T04:13:37.935247+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:22.612979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935030+0000","last_change":"2024-03-22T04:13:36.533681+0000","last_active":"2024-03-22T04:13:37.935030+0000","last_peered":"2024-03-22T04:13:37.935030+0000","last_clean":"2024-03-22T04:13:37.935030+0000","last_became_active":"2024-03-22T04:13:36.533286+0000","last_became_peered":"2024-03-22T04:13:36.533286+0000","last_unstale":"2024-03-22T04:13:37.935030+0000","last_undegraded":"2024-03-22T04:13:37.935030+0000","last_fullsized":"2024-03-22T04:13:37.935030+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:48:38.248410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537228+0000","last_change":"2024-03-22T04:13:36.541069+0000","last_active":"2024-03-22T04:13:37.537228+0000","last_peered":"2024-03-22T04:13:37.537228+0000","last_clean":"2024-03-22T04:13:37.537228+0000","last_became_active":"2024-03-22T04:13:36.540608+0000","last_became_peered":"2024-03-22T04:13:36.540608+0000","last_unstale":"2024-03-22T04:13:37.537228+0000","last_undegraded":"2024-03-22T04:13:37.537228+0000","last_fullsized":"2024-03-22T04:13:37.537228+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:00:14.181624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.536943+0000","last_change":"2024-03-22T04:13:36.529205+0000","last_active":"2024-03-22T04:13:37.536943+0000","last_peered":"2024-03-22T04:13:37.536943+0000","last_clean":"2024-03-22T04:13:37.536943+0000","last_became_active":"2024-03-22T04:13:36.528946+0000","last_became_peered":"2024-03-22T04:13:36.528946+0000","last_unstale":"2024-03-22T04:13:37.536943+0000","last_undegraded":"2024-03-22T04:13:37.536943+0000","last_fullsized":"2024-03-22T04:13:37.536943+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:48:10.956970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934360+0000","last_change":"2024-03-22T04:13:36.546717+0000","last_active":"2024-03-22T04:13:37.934360+0000","last_peered":"2024-03-22T04:13:37.934360+0000","last_clean":"2024-03-22T04:13:37.934360+0000","last_became_active":"2024-03-22T04:13:36.546358+0000","last_became_peered":"2024-03-22T04:13:36.546358+0000","last_unstale":"2024-03-22T04:13:37.934360+0000","last_undegraded":"2024-03-22T04:13:37.934360+0000","last_fullsized":"2024-03-22T04:13:37.934360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:57:05.797600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934769+0000","last_change":"2024-03-22T04:13:36.532329+0000","last_active":"2024-03-22T04:13:37.934769+0000","last_peered":"2024-03-22T04:13:37.934769+0000","last_clean":"2024-03-22T04:13:37.934769+0000","last_became_active":"2024-03-22T04:13:36.532005+0000","last_became_peered":"2024-03-22T04:13:36.532005+0000","last_unstale":"2024-03-22T04:13:37.934769+0000","last_undegraded":"2024-03-22T04:13:37.934769+0000","last_fullsized":"2024-03-22T04:13:37.934769+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:22:16.574264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537416+0000","last_change":"2024-03-22T04:13:36.542842+0000","last_active":"2024-03-22T04:13:37.537416+0000","last_peered":"2024-03-22T04:13:37.537416+0000","last_clean":"2024-03-22T04:13:37.537416+0000","last_became_active":"2024-03-22T04:13:36.538891+0000","last_became_peered":"2024-03-22T04:13:36.538891+0000","last_unstale":"2024-03-22T04:13:37.537416+0000","last_undegraded":"2024-03-22T04:13:37.537416+0000","last_fullsized":"2024-03-22T04:13:37.537416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:36:30.557676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935187+0000","last_change":"2024-03-22T04:13:35.531755+0000","last_active":"2024-03-22T04:13:37.935187+0000","last_peered":"2024-03-22T04:13:37.935187+0000","last_clean":"2024-03-22T04:13:37.935187+0000","last_became_active":"2024-03-22T04:13:35.531433+0000","last_became_peered":"2024-03-22T04:13:35.531433+0000","last_unstale":"2024-03-22T04:13:37.935187+0000","last_undegraded":"2024-03-22T04:13:37.935187+0000","last_fullsized":"2024-03-22T04:13:37.935187+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:31:05.689243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934820+0000","last_change":"2024-03-22T04:13:36.550235+0000","last_active":"2024-03-22T04:13:37.934820+0000","last_peered":"2024-03-22T04:13:37.934820+0000","last_clean":"2024-03-22T04:13:37.934820+0000","last_became_active":"2024-03-22T04:13:36.545989+0000","last_became_peered":"2024-03-22T04:13:36.545989+0000","last_unstale":"2024-03-22T04:13:37.934820+0000","last_undegraded":"2024-03-22T04:13:37.934820+0000","last_fullsized":"2024-03-22T04:13:37.934820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:23.603767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935121+0000","last_change":"2024-03-22T04:13:35.529846+0000","last_active":"2024-03-22T04:13:37.935121+0000","last_peered":"2024-03-22T04:13:37.935121+0000","last_clean":"2024-03-22T04:13:37.935121+0000","last_became_active":"2024-03-22T04:13:35.529546+0000","last_became_peered":"2024-03-22T04:13:35.529546+0000","last_unstale":"2024-03-22T04:13:37.935121+0000","last_undegraded":"2024-03-22T04:13:37.935121+0000","last_fullsized":"2024-03-22T04:13:37.935121+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:15:11.462730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533181+0000","last_change":"2024-03-22T04:13:36.532489+0000","last_active":"2024-03-22T04:13:37.533181+0000","last_peered":"2024-03-22T04:13:37.533181+0000","last_clean":"2024-03-22T04:13:37.533181+0000","last_became_active":"2024-03-22T04:13:36.531886+0000","last_became_peered":"2024-03-22T04:13:36.531886+0000","last_unstale":"2024-03-22T04:13:37.533181+0000","last_undegraded":"2024-03-22T04:13:37.533181+0000","last_fullsized":"2024-03-22T04:13:37.533181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:00:14.181624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935293+0000","last_change":"2024-03-22T04:13:35.530455+0000","last_active":"2024-03-22T04:13:37.935293+0000","last_peered":"2024-03-22T04:13:37.935293+0000","last_clean":"2024-03-22T04:13:37.935293+0000","last_became_active":"2024-03-22T04:13:35.530130+0000","last_became_peered":"2024-03-22T04:13:35.530130+0000","last_unstale":"2024-03-22T04:13:37.935293+0000","last_undegraded":"2024-03-22T04:13:37.935293+0000","last_fullsized":"2024-03-22T04:13:37.935293+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:21.606511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537378+0000","last_change":"2024-03-22T04:13:36.542913+0000","last_active":"2024-03-22T04:13:37.537378+0000","last_peered":"2024-03-22T04:13:37.537378+0000","last_clean":"2024-03-22T04:13:37.537378+0000","last_became_active":"2024-03-22T04:13:36.542587+0000","last_became_peered":"2024-03-22T04:13:36.542587+0000","last_unstale":"2024-03-22T04:13:37.537378+0000","last_undegraded":"2024-03-22T04:13:37.537378+0000","last_fullsized":"2024-03-22T04:13:37.537378+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:21:47.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532775+0000","last_change":"2024-03-22T04:13:35.533152+0000","last_active":"2024-03-22T04:13:37.532775+0000","last_peered":"2024-03-22T04:13:37.532775+0000","last_clean":"2024-03-22T04:13:37.532775+0000","last_became_active":"2024-03-22T04:13:35.532739+0000","last_became_peered":"2024-03-22T04:13:35.532739+0000","last_unstale":"2024-03-22T04:13:37.532775+0000","last_undegraded":"2024-03-22T04:13:37.532775+0000","last_fullsized":"2024-03-22T04:13:37.532775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:31:05.689243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533547+0000","last_change":"2024-03-22T04:13:36.539261+0000","last_active":"2024-03-22T04:13:37.533547+0000","last_peered":"2024-03-22T04:13:37.533547+0000","last_clean":"2024-03-22T04:13:37.533547+0000","last_became_active":"2024-03-22T04:13:36.538948+0000","last_became_peered":"2024-03-22T04:13:36.538948+0000","last_unstale":"2024-03-22T04:13:37.533547+0000","last_undegraded":"2024-03-22T04:13:37.533547+0000","last_fullsized":"2024-03-22T04:13:37.533547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.370928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534342+0000","last_change":"2024-03-22T04:13:35.528073+0000","last_active":"2024-03-22T04:13:37.534342+0000","last_peered":"2024-03-22T04:13:37.534342+0000","last_clean":"2024-03-22T04:13:37.534342+0000","last_became_active":"2024-03-22T04:13:35.527410+0000","last_became_peered":"2024-03-22T04:13:35.527410+0000","last_unstale":"2024-03-22T04:13:37.534342+0000","last_undegraded":"2024-03-22T04:13:37.534342+0000","last_fullsized":"2024-03-22T04:13:37.534342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:00:59.588177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934798+0000","last_change":"2024-03-22T04:13:36.532397+0000","last_active":"2024-03-22T04:13:37.934798+0000","last_peered":"2024-03-22T04:13:37.934798+0000","last_clean":"2024-03-22T04:13:37.934798+0000","last_became_active":"2024-03-22T04:13:36.532117+0000","last_became_peered":"2024-03-22T04:13:36.532117+0000","last_unstale":"2024-03-22T04:13:37.934798+0000","last_undegraded":"2024-03-22T04:13:37.934798+0000","last_fullsized":"2024-03-22T04:13:37.934798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:40.084225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533086+0000","last_change":"2024-03-22T04:13:35.526516+0000","last_active":"2024-03-22T04:13:37.533086+0000","last_peered":"2024-03-22T04:13:37.533086+0000","last_clean":"2024-03-22T04:13:37.533086+0000","last_became_active":"2024-03-22T04:13:35.526208+0000","last_became_peered":"2024-03-22T04:13:35.526208+0000","last_unstale":"2024-03-22T04:13:37.533086+0000","last_undegraded":"2024-03-22T04:13:37.533086+0000","last_fullsized":"2024-03-22T04:13:37.533086+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:56.946600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533142+0000","last_change":"2024-03-22T04:13:36.530615+0000","last_active":"2024-03-22T04:13:37.533142+0000","last_peered":"2024-03-22T04:13:37.533142+0000","last_clean":"2024-03-22T04:13:37.533142+0000","last_became_active":"2024-03-22T04:13:36.530013+0000","last_became_peered":"2024-03-22T04:13:36.530013+0000","last_unstale":"2024-03-22T04:13:37.533142+0000","last_undegraded":"2024-03-22T04:13:37.533142+0000","last_fullsized":"2024-03-22T04:13:37.533142+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:57.953068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934890+0000","last_change":"2024-03-22T04:13:35.534172+0000","last_active":"2024-03-22T04:13:37.934890+0000","last_peered":"2024-03-22T04:13:37.934890+0000","last_clean":"2024-03-22T04:13:37.934890+0000","last_became_active":"2024-03-22T04:13:35.533651+0000","last_became_peered":"2024-03-22T04:13:35.533651+0000","last_unstale":"2024-03-22T04:13:37.934890+0000","last_undegraded":"2024-03-22T04:13:37.934890+0000","last_fullsized":"2024-03-22T04:13:37.934890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:11:03.089640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534186+0000","last_change":"2024-03-22T04:13:36.535993+0000","last_active":"2024-03-22T04:13:37.534186+0000","last_peered":"2024-03-22T04:13:37.534186+0000","last_clean":"2024-03-22T04:13:37.534186+0000","last_became_active":"2024-03-22T04:13:36.534295+0000","last_became_peered":"2024-03-22T04:13:36.534295+0000","last_unstale":"2024-03-22T04:13:37.534186+0000","last_undegraded":"2024-03-22T04:13:37.534186+0000","last_fullsized":"2024-03-22T04:13:37.534186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:58.215479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.536794+0000","last_change":"2024-03-22T04:13:35.533417+0000","last_active":"2024-03-22T04:13:37.536794+0000","last_peered":"2024-03-22T04:13:37.536794+0000","last_clean":"2024-03-22T04:13:37.536794+0000","last_became_active":"2024-03-22T04:13:35.533053+0000","last_became_peered":"2024-03-22T04:13:35.533053+0000","last_unstale":"2024-03-22T04:13:37.536794+0000","last_undegraded":"2024-03-22T04:13:37.536794+0000","last_fullsized":"2024-03-22T04:13:37.536794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:11:03.089640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935033+0000","last_change":"2024-03-22T04:13:36.539032+0000","last_active":"2024-03-22T04:13:37.935033+0000","last_peered":"2024-03-22T04:13:37.935033+0000","last_clean":"2024-03-22T04:13:37.935033+0000","last_became_active":"2024-03-22T04:13:36.538667+0000","last_became_peered":"2024-03-22T04:13:36.538667+0000","last_unstale":"2024-03-22T04:13:37.935033+0000","last_undegraded":"2024-03-22T04:13:37.935033+0000","last_fullsized":"2024-03-22T04:13:37.935033+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:59:33.015881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537482+0000","last_change":"2024-03-22T04:13:35.535058+0000","last_active":"2024-03-22T04:13:37.537482+0000","last_peered":"2024-03-22T04:13:37.537482+0000","last_clean":"2024-03-22T04:13:37.537482+0000","last_became_active":"2024-03-22T04:13:35.534345+0000","last_became_peered":"2024-03-22T04:13:35.534345+0000","last_unstale":"2024-03-22T04:13:37.537482+0000","last_undegraded":"2024-03-22T04:13:37.537482+0000","last_fullsized":"2024-03-22T04:13:37.537482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:40:06.790699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534393+0000","last_change":"2024-03-22T04:13:36.527089+0000","last_active":"2024-03-22T04:13:37.534393+0000","last_peered":"2024-03-22T04:13:37.534393+0000","last_clean":"2024-03-22T04:13:37.534393+0000","last_became_active":"2024-03-22T04:13:36.526807+0000","last_became_peered":"2024-03-22T04:13:36.526807+0000","last_unstale":"2024-03-22T04:13:37.534393+0000","last_undegraded":"2024-03-22T04:13:37.534393+0000","last_fullsized":"2024-03-22T04:13:37.534393+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:57:11.984612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935382+0000","last_change":"2024-03-22T04:13:35.525257+0000","last_active":"2024-03-22T04:13:37.935382+0000","last_peered":"2024-03-22T04:13:37.935382+0000","last_clean":"2024-03-22T04:13:37.935382+0000","last_became_active":"2024-03-22T04:13:35.524936+0000","last_became_peered":"2024-03-22T04:13:35.524936+0000","last_unstale":"2024-03-22T04:13:37.935382+0000","last_undegraded":"2024-03-22T04:13:37.935382+0000","last_fullsized":"2024-03-22T04:13:37.935382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:03:28.004834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935278+0000","last_change":"2024-03-22T04:13:36.532346+0000","last_active":"2024-03-22T04:13:37.935278+0000","last_peered":"2024-03-22T04:13:37.935278+0000","last_clean":"2024-03-22T04:13:37.935278+0000","last_became_active":"2024-03-22T04:13:36.532109+0000","last_became_peered":"2024-03-22T04:13:36.532109+0000","last_unstale":"2024-03-22T04:13:37.935278+0000","last_undegraded":"2024-03-22T04:13:37.935278+0000","last_fullsized":"2024-03-22T04:13:37.935278+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:31:06.695711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934826+0000","last_change":"2024-03-22T04:13:35.536397+0000","last_active":"2024-03-22T04:13:37.934826+0000","last_peered":"2024-03-22T04:13:37.934826+0000","last_clean":"2024-03-22T04:13:37.934826+0000","last_became_active":"2024-03-22T04:13:35.536137+0000","last_became_peered":"2024-03-22T04:13:35.536137+0000","last_unstale":"2024-03-22T04:13:37.934826+0000","last_undegraded":"2024-03-22T04:13:37.934826+0000","last_fullsized":"2024-03-22T04:13:37.934826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:31:32.025082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934682+0000","last_change":"2024-03-22T04:13:36.550088+0000","last_active":"2024-03-22T04:13:37.934682+0000","last_peered":"2024-03-22T04:13:37.934682+0000","last_clean":"2024-03-22T04:13:37.934682+0000","last_became_active":"2024-03-22T04:13:36.549779+0000","last_became_peered":"2024-03-22T04:13:36.549779+0000","last_unstale":"2024-03-22T04:13:37.934682+0000","last_undegraded":"2024-03-22T04:13:37.934682+0000","last_fullsized":"2024-03-22T04:13:37.934682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:26:53.865888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532845+0000","last_change":"2024-03-22T04:13:35.529973+0000","last_active":"2024-03-22T04:13:37.532845+0000","last_peered":"2024-03-22T04:13:37.532845+0000","last_clean":"2024-03-22T04:13:37.532845+0000","last_became_active":"2024-03-22T04:13:35.529696+0000","last_became_peered":"2024-03-22T04:13:35.529696+0000","last_unstale":"2024-03-22T04:13:37.532845+0000","last_undegraded":"2024-03-22T04:13:37.532845+0000","last_fullsized":"2024-03-22T04:13:37.532845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:40:22.406882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533202+0000","last_change":"2024-03-22T04:13:36.528978+0000","last_active":"2024-03-22T04:13:37.533202+0000","last_peered":"2024-03-22T04:13:37.533202+0000","last_clean":"2024-03-22T04:13:37.533202+0000","last_became_active":"2024-03-22T04:13:36.528529+0000","last_became_peered":"2024-03-22T04:13:36.528529+0000","last_unstale":"2024-03-22T04:13:37.533202+0000","last_undegraded":"2024-03-22T04:13:37.533202+0000","last_fullsized":"2024-03-22T04:13:37.533202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:51:43.486787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537182+0000","last_change":"2024-03-22T04:13:35.529378+0000","last_active":"2024-03-22T04:13:37.537182+0000","last_peered":"2024-03-22T04:13:37.537182+0000","last_clean":"2024-03-22T04:13:37.537182+0000","last_became_active":"2024-03-22T04:13:35.529096+0000","last_became_peered":"2024-03-22T04:13:35.529096+0000","last_unstale":"2024-03-22T04:13:37.537182+0000","last_undegraded":"2024-03-22T04:13:37.537182+0000","last_fullsized":"2024-03-22T04:13:37.537182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:56.946600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533081+0000","last_change":"2024-03-22T04:13:36.525479+0000","last_active":"2024-03-22T04:13:37.533081+0000","last_peered":"2024-03-22T04:13:37.533081+0000","last_clean":"2024-03-22T04:13:37.533081+0000","last_became_active":"2024-03-22T04:13:36.525209+0000","last_became_peered":"2024-03-22T04:13:36.525209+0000","last_unstale":"2024-03-22T04:13:37.533081+0000","last_undegraded":"2024-03-22T04:13:37.533081+0000","last_fullsized":"2024-03-22T04:13:37.533081+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:21:42.400738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533082+0000","last_change":"2024-03-22T04:13:35.536795+0000","last_active":"2024-03-22T04:13:37.533082+0000","last_peered":"2024-03-22T04:13:37.533082+0000","last_clean":"2024-03-22T04:13:37.533082+0000","last_became_active":"2024-03-22T04:13:35.534453+0000","last_became_peered":"2024-03-22T04:13:35.534453+0000","last_unstale":"2024-03-22T04:13:37.533082+0000","last_undegraded":"2024-03-22T04:13:37.533082+0000","last_fullsized":"2024-03-22T04:13:37.533082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:23.221070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532993+0000","last_change":"2024-03-22T04:13:36.530862+0000","last_active":"2024-03-22T04:13:37.532993+0000","last_peered":"2024-03-22T04:13:37.532993+0000","last_clean":"2024-03-22T04:13:37.532993+0000","last_became_active":"2024-03-22T04:13:36.530365+0000","last_became_peered":"2024-03-22T04:13:36.530365+0000","last_unstale":"2024-03-22T04:13:37.532993+0000","last_undegraded":"2024-03-22T04:13:37.532993+0000","last_fullsized":"2024-03-22T04:13:37.532993+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:24.704154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934337+0000","last_change":"2024-03-22T04:13:35.529571+0000","last_active":"2024-03-22T04:13:37.934337+0000","last_peered":"2024-03-22T04:13:37.934337+0000","last_clean":"2024-03-22T04:13:37.934337+0000","last_became_active":"2024-03-22T04:13:35.528385+0000","last_became_peered":"2024-03-22T04:13:35.528385+0000","last_unstale":"2024-03-22T04:13:37.934337+0000","last_undegraded":"2024-03-22T04:13:37.934337+0000","last_fullsized":"2024-03-22T04:13:37.934337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:03:20.113486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934905+0000","last_change":"2024-03-22T04:13:36.543079+0000","last_active":"2024-03-22T04:13:37.934905+0000","last_peered":"2024-03-22T04:13:37.934905+0000","last_clean":"2024-03-22T04:13:37.934905+0000","last_became_active":"2024-03-22T04:13:36.542677+0000","last_became_peered":"2024-03-22T04:13:36.542677+0000","last_unstale":"2024-03-22T04:13:37.934905+0000","last_undegraded":"2024-03-22T04:13:37.934905+0000","last_fullsized":"2024-03-22T04:13:37.934905+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:05:32.138647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532855+0000","last_change":"2024-03-22T04:13:35.536315+0000","last_active":"2024-03-22T04:13:37.532855+0000","last_peered":"2024-03-22T04:13:37.532855+0000","last_clean":"2024-03-22T04:13:37.532855+0000","last_became_active":"2024-03-22T04:13:35.535908+0000","last_became_peered":"2024-03-22T04:13:35.535908+0000","last_unstale":"2024-03-22T04:13:37.532855+0000","last_undegraded":"2024-03-22T04:13:37.532855+0000","last_fullsized":"2024-03-22T04:13:37.532855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:00:58.326961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537561+0000","last_change":"2024-03-22T04:13:36.544623+0000","last_active":"2024-03-22T04:13:37.537561+0000","last_peered":"2024-03-22T04:13:37.537561+0000","last_clean":"2024-03-22T04:13:37.537561+0000","last_became_active":"2024-03-22T04:13:36.543545+0000","last_became_peered":"2024-03-22T04:13:36.543545+0000","last_unstale":"2024-03-22T04:13:37.537561+0000","last_undegraded":"2024-03-22T04:13:37.537561+0000","last_fullsized":"2024-03-22T04:13:37.537561+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:13:25.762587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537539+0000","last_change":"2024-03-22T04:13:35.534854+0000","last_active":"2024-03-22T04:13:37.537539+0000","last_peered":"2024-03-22T04:13:37.537539+0000","last_clean":"2024-03-22T04:13:37.537539+0000","last_became_active":"2024-03-22T04:13:35.534331+0000","last_became_peered":"2024-03-22T04:13:35.534331+0000","last_unstale":"2024-03-22T04:13:37.537539+0000","last_undegraded":"2024-03-22T04:13:37.537539+0000","last_fullsized":"2024-03-22T04:13:37.537539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:23.697686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935261+0000","last_change":"2024-03-22T04:13:36.553382+0000","last_active":"2024-03-22T04:13:37.935261+0000","last_peered":"2024-03-22T04:13:37.935261+0000","last_clean":"2024-03-22T04:13:37.935261+0000","last_became_active":"2024-03-22T04:13:36.553128+0000","last_became_peered":"2024-03-22T04:13:36.553128+0000","last_unstale":"2024-03-22T04:13:37.935261+0000","last_undegraded":"2024-03-22T04:13:37.935261+0000","last_fullsized":"2024-03-22T04:13:37.935261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:39:51.985574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537312+0000","last_change":"2024-03-22T04:13:35.533530+0000","last_active":"2024-03-22T04:13:37.537312+0000","last_peered":"2024-03-22T04:13:37.537312+0000","last_clean":"2024-03-22T04:13:37.537312+0000","last_became_active":"2024-03-22T04:13:35.533254+0000","last_became_peered":"2024-03-22T04:13:35.533254+0000","last_unstale":"2024-03-22T04:13:37.537312+0000","last_undegraded":"2024-03-22T04:13:37.537312+0000","last_fullsized":"2024-03-22T04:13:37.537312+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:21.606511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935434+0000","last_change":"2024-03-22T04:13:36.528389+0000","last_active":"2024-03-22T04:13:37.935434+0000","last_peered":"2024-03-22T04:13:37.935434+0000","last_clean":"2024-03-22T04:13:37.935434+0000","last_became_active":"2024-03-22T04:13:36.528139+0000","last_became_peered":"2024-03-22T04:13:36.528139+0000","last_unstale":"2024-03-22T04:13:37.935434+0000","last_undegraded":"2024-03-22T04:13:37.935434+0000","last_fullsized":"2024-03-22T04:13:37.935434+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:40:23.413350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934883+0000","last_change":"2024-03-22T04:13:35.530606+0000","last_active":"2024-03-22T04:13:37.934883+0000","last_peered":"2024-03-22T04:13:37.934883+0000","last_clean":"2024-03-22T04:13:37.934883+0000","last_became_active":"2024-03-22T04:13:35.530250+0000","last_became_peered":"2024-03-22T04:13:35.530250+0000","last_unstale":"2024-03-22T04:13:37.934883+0000","last_undegraded":"2024-03-22T04:13:37.934883+0000","last_fullsized":"2024-03-22T04:13:37.934883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:12:37.805223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934653+0000","last_change":"2024-03-22T04:13:36.545102+0000","last_active":"2024-03-22T04:13:37.934653+0000","last_peered":"2024-03-22T04:13:37.934653+0000","last_clean":"2024-03-22T04:13:37.934653+0000","last_became_active":"2024-03-22T04:13:36.544856+0000","last_became_peered":"2024-03-22T04:13:36.544856+0000","last_unstale":"2024-03-22T04:13:37.934653+0000","last_undegraded":"2024-03-22T04:13:37.934653+0000","last_fullsized":"2024-03-22T04:13:37.934653+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:03:31.551981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.541127+0000","last_change":"2024-03-22T04:13:35.539839+0000","last_active":"2024-03-22T04:13:37.541127+0000","last_peered":"2024-03-22T04:13:37.541127+0000","last_clean":"2024-03-22T04:13:37.541127+0000","last_became_active":"2024-03-22T04:13:35.539572+0000","last_became_peered":"2024-03-22T04:13:35.539572+0000","last_unstale":"2024-03-22T04:13:37.541127+0000","last_undegraded":"2024-03-22T04:13:37.541127+0000","last_fullsized":"2024-03-22T04:13:37.541127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532319+0000","last_change":"2024-03-22T04:13:36.528075+0000","last_active":"2024-03-22T04:13:37.532319+0000","last_peered":"2024-03-22T04:13:37.532319+0000","last_clean":"2024-03-22T04:13:37.532319+0000","last_became_active":"2024-03-22T04:13:36.527766+0000","last_became_peered":"2024-03-22T04:13:36.527766+0000","last_unstale":"2024-03-22T04:13:37.532319+0000","last_undegraded":"2024-03-22T04:13:37.532319+0000","last_fullsized":"2024-03-22T04:13:37.532319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:06:14.434151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532527+0000","last_change":"2024-03-22T04:13:35.533632+0000","last_active":"2024-03-22T04:13:37.532527+0000","last_peered":"2024-03-22T04:13:37.532527+0000","last_clean":"2024-03-22T04:13:37.532527+0000","last_became_active":"2024-03-22T04:13:35.533349+0000","last_became_peered":"2024-03-22T04:13:35.533349+0000","last_unstale":"2024-03-22T04:13:37.532527+0000","last_undegraded":"2024-03-22T04:13:37.532527+0000","last_fullsized":"2024-03-22T04:13:37.532527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:21.606511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934835+0000","last_change":"2024-03-22T04:13:36.521144+0000","last_active":"2024-03-22T04:13:37.934835+0000","last_peered":"2024-03-22T04:13:37.934835+0000","last_clean":"2024-03-22T04:13:37.934835+0000","last_became_active":"2024-03-22T04:13:36.520882+0000","last_became_peered":"2024-03-22T04:13:36.520882+0000","last_unstale":"2024-03-22T04:13:37.934835+0000","last_undegraded":"2024-03-22T04:13:37.934835+0000","last_fullsized":"2024-03-22T04:13:37.934835+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:21:42.400738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533180+0000","last_change":"2024-03-22T04:13:35.527224+0000","last_active":"2024-03-22T04:13:37.533180+0000","last_peered":"2024-03-22T04:13:37.533180+0000","last_clean":"2024-03-22T04:13:37.533180+0000","last_became_active":"2024-03-22T04:13:35.526938+0000","last_became_peered":"2024-03-22T04:13:35.526938+0000","last_unstale":"2024-03-22T04:13:37.533180+0000","last_undegraded":"2024-03-22T04:13:37.533180+0000","last_fullsized":"2024-03-22T04:13:37.533180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:03:28.004834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537011+0000","last_change":"2024-03-22T04:13:36.535391+0000","last_active":"2024-03-22T04:13:37.537011+0000","last_peered":"2024-03-22T04:13:37.537011+0000","last_clean":"2024-03-22T04:13:37.537011+0000","last_became_active":"2024-03-22T04:13:36.535086+0000","last_became_peered":"2024-03-22T04:13:36.535086+0000","last_unstale":"2024-03-22T04:13:37.537011+0000","last_undegraded":"2024-03-22T04:13:37.537011+0000","last_fullsized":"2024-03-22T04:13:37.537011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:13:42.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532179+0000","last_change":"2024-03-22T04:13:35.530361+0000","last_active":"2024-03-22T04:13:37.532179+0000","last_peered":"2024-03-22T04:13:37.532179+0000","last_clean":"2024-03-22T04:13:37.532179+0000","last_became_active":"2024-03-22T04:13:35.529768+0000","last_became_peered":"2024-03-22T04:13:35.529768+0000","last_unstale":"2024-03-22T04:13:37.532179+0000","last_undegraded":"2024-03-22T04:13:37.532179+0000","last_fullsized":"2024-03-22T04:13:37.532179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:39:50.979105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534117+0000","last_change":"2024-03-22T04:13:36.523859+0000","last_active":"2024-03-22T04:13:37.534117+0000","last_peered":"2024-03-22T04:13:37.534117+0000","last_clean":"2024-03-22T04:13:37.534117+0000","last_became_active":"2024-03-22T04:13:36.523598+0000","last_became_peered":"2024-03-22T04:13:36.523598+0000","last_unstale":"2024-03-22T04:13:37.534117+0000","last_undegraded":"2024-03-22T04:13:37.534117+0000","last_fullsized":"2024-03-22T04:13:37.534117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:04.300928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537577+0000","last_change":"2024-03-22T04:13:35.534630+0000","last_active":"2024-03-22T04:13:37.537577+0000","last_peered":"2024-03-22T04:13:37.537577+0000","last_clean":"2024-03-22T04:13:37.537577+0000","last_became_active":"2024-03-22T04:13:35.533284+0000","last_became_peered":"2024-03-22T04:13:35.533284+0000","last_unstale":"2024-03-22T04:13:37.537577+0000","last_undegraded":"2024-03-22T04:13:37.537577+0000","last_fullsized":"2024-03-22T04:13:37.537577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:21:41.394270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534628+0000","last_change":"2024-03-22T04:13:36.539056+0000","last_active":"2024-03-22T04:13:37.534628+0000","last_peered":"2024-03-22T04:13:37.534628+0000","last_clean":"2024-03-22T04:13:37.534628+0000","last_became_active":"2024-03-22T04:13:36.538782+0000","last_became_peered":"2024-03-22T04:13:36.538782+0000","last_unstale":"2024-03-22T04:13:37.534628+0000","last_undegraded":"2024-03-22T04:13:37.534628+0000","last_fullsized":"2024-03-22T04:13:37.534628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:07.459840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537503+0000","last_change":"2024-03-22T04:13:35.531937+0000","last_active":"2024-03-22T04:13:37.537503+0000","last_peered":"2024-03-22T04:13:37.537503+0000","last_clean":"2024-03-22T04:13:37.537503+0000","last_became_active":"2024-03-22T04:13:35.528718+0000","last_became_peered":"2024-03-22T04:13:35.528718+0000","last_unstale":"2024-03-22T04:13:37.537503+0000","last_undegraded":"2024-03-22T04:13:37.537503+0000","last_fullsized":"2024-03-22T04:13:37.537503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:15:11.462730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934662+0000","last_change":"2024-03-22T04:13:36.545245+0000","last_active":"2024-03-22T04:13:37.934662+0000","last_peered":"2024-03-22T04:13:37.934662+0000","last_clean":"2024-03-22T04:13:37.934662+0000","last_became_active":"2024-03-22T04:13:36.544198+0000","last_became_peered":"2024-03-22T04:13:36.544198+0000","last_unstale":"2024-03-22T04:13:37.934662+0000","last_undegraded":"2024-03-22T04:13:37.934662+0000","last_fullsized":"2024-03-22T04:13:37.934662+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:54:58.046272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532826+0000","last_change":"2024-03-22T04:13:35.530829+0000","last_active":"2024-03-22T04:13:37.532826+0000","last_peered":"2024-03-22T04:13:37.532826+0000","last_clean":"2024-03-22T04:13:37.532826+0000","last_became_active":"2024-03-22T04:13:35.530554+0000","last_became_peered":"2024-03-22T04:13:35.530554+0000","last_unstale":"2024-03-22T04:13:37.532826+0000","last_undegraded":"2024-03-22T04:13:37.532826+0000","last_fullsized":"2024-03-22T04:13:37.532826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:15:11.462730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934824+0000","last_change":"2024-03-22T04:13:36.540921+0000","last_active":"2024-03-22T04:13:37.934824+0000","last_peered":"2024-03-22T04:13:37.934824+0000","last_clean":"2024-03-22T04:13:37.934824+0000","last_became_active":"2024-03-22T04:13:36.540455+0000","last_became_peered":"2024-03-22T04:13:36.540455+0000","last_unstale":"2024-03-22T04:13:37.934824+0000","last_undegraded":"2024-03-22T04:13:37.934824+0000","last_fullsized":"2024-03-22T04:13:37.934824+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:53:16.420721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.541159+0000","last_change":"2024-03-22T04:13:35.538794+0000","last_active":"2024-03-22T04:13:37.541159+0000","last_peered":"2024-03-22T04:13:37.541159+0000","last_clean":"2024-03-22T04:13:37.541159+0000","last_became_active":"2024-03-22T04:13:35.538511+0000","last_became_peered":"2024-03-22T04:13:35.538511+0000","last_unstale":"2024-03-22T04:13:37.541159+0000","last_undegraded":"2024-03-22T04:13:37.541159+0000","last_fullsized":"2024-03-22T04:13:37.541159+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.536799+0000","last_change":"2024-03-22T04:13:36.528390+0000","last_active":"2024-03-22T04:13:37.536799+0000","last_peered":"2024-03-22T04:13:37.536799+0000","last_clean":"2024-03-22T04:13:37.536799+0000","last_became_active":"2024-03-22T04:13:36.528094+0000","last_became_peered":"2024-03-22T04:13:36.528094+0000","last_unstale":"2024-03-22T04:13:37.536799+0000","last_undegraded":"2024-03-22T04:13:37.536799+0000","last_fullsized":"2024-03-22T04:13:37.536799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:06:14.434151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935241+0000","last_change":"2024-03-22T04:13:35.528468+0000","last_active":"2024-03-22T04:13:37.935241+0000","last_peered":"2024-03-22T04:13:37.935241+0000","last_clean":"2024-03-22T04:13:37.935241+0000","last_became_active":"2024-03-22T04:13:35.528113+0000","last_became_peered":"2024-03-22T04:13:35.528113+0000","last_unstale":"2024-03-22T04:13:37.935241+0000","last_undegraded":"2024-03-22T04:13:37.935241+0000","last_fullsized":"2024-03-22T04:13:37.935241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:23.221070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934996+0000","last_change":"2024-03-22T04:13:36.531435+0000","last_active":"2024-03-22T04:13:37.934996+0000","last_peered":"2024-03-22T04:13:37.934996+0000","last_clean":"2024-03-22T04:13:37.934996+0000","last_became_active":"2024-03-22T04:13:36.531103+0000","last_became_peered":"2024-03-22T04:13:36.531103+0000","last_unstale":"2024-03-22T04:13:37.934996+0000","last_undegraded":"2024-03-22T04:13:37.934996+0000","last_fullsized":"2024-03-22T04:13:37.934996+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:51:43.486787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934796+0000","last_change":"2024-03-22T04:13:35.525890+0000","last_active":"2024-03-22T04:13:37.934796+0000","last_peered":"2024-03-22T04:13:37.934796+0000","last_clean":"2024-03-22T04:13:37.934796+0000","last_became_active":"2024-03-22T04:13:35.525618+0000","last_became_peered":"2024-03-22T04:13:35.525618+0000","last_unstale":"2024-03-22T04:13:37.934796+0000","last_undegraded":"2024-03-22T04:13:37.934796+0000","last_fullsized":"2024-03-22T04:13:37.934796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:56.946600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532657+0000","last_change":"2024-03-22T04:13:36.527937+0000","last_active":"2024-03-22T04:13:37.532657+0000","last_peered":"2024-03-22T04:13:37.532657+0000","last_clean":"2024-03-22T04:13:37.532657+0000","last_became_active":"2024-03-22T04:13:36.527142+0000","last_became_peered":"2024-03-22T04:13:36.527142+0000","last_unstale":"2024-03-22T04:13:37.532657+0000","last_undegraded":"2024-03-22T04:13:37.532657+0000","last_fullsized":"2024-03-22T04:13:37.532657+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:48:10.956970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.540994+0000","last_change":"2024-03-22T04:13:35.537483+0000","last_active":"2024-03-22T04:13:37.540994+0000","last_peered":"2024-03-22T04:13:37.540994+0000","last_clean":"2024-03-22T04:13:37.540994+0000","last_became_active":"2024-03-22T04:13:35.537070+0000","last_became_peered":"2024-03-22T04:13:35.537070+0000","last_unstale":"2024-03-22T04:13:37.540994+0000","last_undegraded":"2024-03-22T04:13:37.540994+0000","last_fullsized":"2024-03-22T04:13:37.540994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:25:37.720655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534302+0000","last_change":"2024-03-22T04:13:36.535142+0000","last_active":"2024-03-22T04:13:37.534302+0000","last_peered":"2024-03-22T04:13:37.534302+0000","last_clean":"2024-03-22T04:13:37.534302+0000","last_became_active":"2024-03-22T04:13:36.533918+0000","last_became_peered":"2024-03-22T04:13:36.533918+0000","last_unstale":"2024-03-22T04:13:37.534302+0000","last_undegraded":"2024-03-22T04:13:37.534302+0000","last_fullsized":"2024-03-22T04:13:37.534302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:15.073523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532432+0000","last_change":"2024-03-22T04:13:35.533317+0000","last_active":"2024-03-22T04:13:37.532432+0000","last_peered":"2024-03-22T04:13:37.532432+0000","last_clean":"2024-03-22T04:13:37.532432+0000","last_became_active":"2024-03-22T04:13:35.532424+0000","last_became_peered":"2024-03-22T04:13:35.532424+0000","last_unstale":"2024-03-22T04:13:37.532432+0000","last_undegraded":"2024-03-22T04:13:37.532432+0000","last_fullsized":"2024-03-22T04:13:37.532432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:51:42.480319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533309+0000","last_change":"2024-03-22T04:13:36.533360+0000","last_active":"2024-03-22T04:13:37.533309+0000","last_peered":"2024-03-22T04:13:37.533309+0000","last_clean":"2024-03-22T04:13:37.533309+0000","last_became_active":"2024-03-22T04:13:36.527843+0000","last_became_peered":"2024-03-22T04:13:36.527843+0000","last_unstale":"2024-03-22T04:13:37.533309+0000","last_undegraded":"2024-03-22T04:13:37.533309+0000","last_fullsized":"2024-03-22T04:13:37.533309+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:44.092644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532539+0000","last_change":"2024-03-22T04:13:35.528988+0000","last_active":"2024-03-22T04:13:37.532539+0000","last_peered":"2024-03-22T04:13:37.532539+0000","last_clean":"2024-03-22T04:13:37.532539+0000","last_became_active":"2024-03-22T04:13:35.528529+0000","last_became_peered":"2024-03-22T04:13:35.528529+0000","last_unstale":"2024-03-22T04:13:37.532539+0000","last_undegraded":"2024-03-22T04:13:37.532539+0000","last_fullsized":"2024-03-22T04:13:37.532539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:43.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":251,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533053+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:13:37.533053+0000","last_peered":"2024-03-22T04:13:37.533053+0000","last_clean":"2024-03-22T04:13:37.533053+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:13:37.533053+0000","last_undegraded":"2024-03-22T04:13:37.533053+0000","last_fullsized":"2024-03-22T04:13:37.533053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":360751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":363015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28404,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343436,"statfs":{"total":96636764160,"available":96607678464,"internally_reserved":0,"allocated":1417216,"data_stored":952843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":360911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":360807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":951153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":360751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":361333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:13:45.114 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:13:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f68400640 1 Processor -- start 2024-03-22T04:13:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f68400640 1 -- start start 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f68400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015bc80 0x7f2f6006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f68400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2f6006dc40 0x7f2f6006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f68400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6015c550 con 0x7f2f6015ac10 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f68400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6015cb60 con 0x7f2f6006dc40 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f68400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2f600b4750 con 0x7f2f6015bc80 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f65a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2f6015ac10 0x7f2f600b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38728/0 (socket says 172.21.15.86:38728) 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f65a00640 1 -- 172.21.15.86:0/916133915 learned_addr learned my addr 172.21.15.86:0/916133915 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f65000640 1 --2- 172.21.15.86:0/916133915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015bc80 0x7f2f6006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f66400640 1 --2- 172.21.15.86:0/916133915 >> v2:172.21.15.196:3300/0 conn(0x7f2f6006dc40 0x7f2f6006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f5fe00640 1 -- 172.21.15.86:0/916133915 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006288890 0 0) 0x7f2f6015c550 con 0x7f2f6015ac10 2024-03-22T04:13:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f5fe00640 1 -- 172.21.15.86:0/916133915 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f48003530 con 0x7f2f6015ac10 2024-03-22T04:13:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f65000640 1 -- 172.21.15.86:0/916133915 >> v2:172.21.15.196:3300/0 conn(0x7f2f6006dc40 msgr2=0x7f2f6006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f65000640 1 --2- 172.21.15.86:0/916133915 >> v2:172.21.15.196:3300/0 conn(0x7f2f6006dc40 0x7f2f6006e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f65000640 1 -- 172.21.15.86:0/916133915 >> v1:172.21.15.86:6789/0 conn(0x7f2f6015ac10 legacy=0x7f2f600b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f65000640 1 -- 172.21.15.86:0/916133915 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f600b4470 con 0x7f2f6015bc80 2024-03-22T04:13:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.260+0000 7f2f66400640 1 --2- 172.21.15.86:0/916133915 >> v2:172.21.15.196:3300/0 conn(0x7f2f6006dc40 0x7f2f6006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f65000640 1 --2- 172.21.15.86:0/916133915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015bc80 0x7f2f6006d700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0097c0 tx=0x7f2f4c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e64e9f56d1c89b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f5fe00640 1 -- 172.21.15.86:0/916133915 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0aa5f0 con 0x7f2f6015bc80 2024-03-22T04:13:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f5fe00640 1 -- 172.21.15.86:0/916133915 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c0af070 con 0x7f2f6015bc80 2024-03-22T04:13:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f5fe00640 1 -- 172.21.15.86:0/916133915 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0aa9b0 con 0x7f2f6015bc80 2024-03-22T04:13:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f68400640 1 -- 172.21.15.86:0/916133915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015bc80 msgr2=0x7f2f6006d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f68400640 1 --2- 172.21.15.86:0/916133915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015bc80 0x7f2f6006d700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0097c0 tx=0x7f2f4c01c470 comp rx=0 tx=0).stop 2024-03-22T04:13:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f68400640 1 -- 172.21.15.86:0/916133915 shutdown_connections 2024-03-22T04:13:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f68400640 1 --2- 172.21.15.86:0/916133915 >> v2:172.21.15.196:3300/0 conn(0x7f2f6006dc40 0x7f2f6006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f68400640 1 --2- 172.21.15.86:0/916133915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015bc80 0x7f2f6006d700 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f68400640 1 -- 172.21.15.86:0/916133915 >> 172.21.15.86:0/916133915 conn(0x7f2f600b1df0 msgr2=0x7f2f600b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f68400640 1 -- 172.21.15.86:0/916133915 shutdown_connections 2024-03-22T04:13:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.264+0000 7f2f68400640 1 -- 172.21.15.86:0/916133915 wait complete. 2024-03-22T04:13:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 Processor -- start 2024-03-22T04:13:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 -- start start 2024-03-22T04:13:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6006dc40 0x7f2f60155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015ac10 0x7f2f601591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6015bc80 0x7f2f60159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6015d3a0 con 0x7f2f6015bc80 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6015d220 con 0x7f2f6006dc40 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6015d520 con 0x7f2f6015ac10 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f66400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6015bc80 0x7f2f60159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015ac10 0x7f2f601591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015ac10 0x7f2f601591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:43902/0 (socket says 172.21.15.86:43902) 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 -- 172.21.15.86:0/2565175656 learned_addr learned my addr 172.21.15.86:0/2565175656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65a00640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6006dc40 0x7f2f60155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 -- 172.21.15.86:0/2565175656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6006dc40 msgr2=0x7f2f60155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6006dc40 0x7f2f60155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 -- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6015bc80 msgr2=0x7f2f60159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6015bc80 0x7f2f60159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 -- 172.21.15.86:0/2565175656 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f600b0ca0 con 0x7f2f6015ac10 2024-03-22T04:13:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f66400640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6015bc80 0x7f2f60159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f65000640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015ac10 0x7f2f601591c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0ac040 tx=0x7f2f4c01acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f5d600640 1 -- 172.21.15.86:0/2565175656 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0be610 con 0x7f2f6015ac10 2024-03-22T04:13:45.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f5d600640 1 -- 172.21.15.86:0/2565175656 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6015e310 con 0x7f2f6015ac10 2024-03-22T04:13:45.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f60159c40 con 0x7f2f6015ac10 2024-03-22T04:13:45.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f5d600640 1 -- 172.21.15.86:0/2565175656 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0bea10 con 0x7f2f6015ac10 2024-03-22T04:13:45.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.268+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f600b01c0 con 0x7f2f6015ac10 2024-03-22T04:13:45.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.272+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f60068650 con 0x7f2f6015ac10 2024-03-22T04:13:45.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.272+0000 7f2f5d600640 1 -- 172.21.15.86:0/2565175656 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2f4c0bc020 con 0x7f2f6015ac10 2024-03-22T04:13:45.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.272+0000 7f2f5d600640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2f40067b20 0x7f2f40069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.272+0000 7f2f5d600640 1 -- 172.21.15.86:0/2565175656 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f2f4c1204f0 con 0x7f2f6015ac10 2024-03-22T04:13:45.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.272+0000 7f2f65a00640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2f40067b20 0x7f2f40069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.272+0000 7f2f65a00640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2f40067b20 0x7f2f40069fe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2f54007fd0 tx=0x7f2f54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:45.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.284+0000 7f2f5d600640 1 -- 172.21.15.86:0/2565175656 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2f4c016090 con 0x7f2f6015ac10 2024-03-22T04:13:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.464+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f600b39e0 con 0x7f2f40067b20 2024-03-22T04:13:45.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.468+0000 7f2f5d600640 1 -- 172.21.15.86:0/2565175656 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252024 (secure 0 0 0) 0x7f2f600b39e0 con 0x7f2f40067b20 2024-03-22T04:13:45.476 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:45.477 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:13:45.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2f40067b20 msgr2=0x7f2f40069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2f40067b20 0x7f2f40069fe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2f54007fd0 tx=0x7f2f54009040 comp rx=0 tx=0).stop 2024-03-22T04:13:45.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015ac10 msgr2=0x7f2f601591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015ac10 0x7f2f601591c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0ac040 tx=0x7f2f4c01acb0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 shutdown_connections 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2f40067b20 0x7f2f40069fe0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f6015bc80 0x7f2f60159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f6015ac10 0x7f2f601591c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 --2- 172.21.15.86:0/2565175656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6006dc40 0x7f2f60155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 >> 172.21.15.86:0/2565175656 conn(0x7f2f600b1df0 msgr2=0x7f2f60156500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 shutdown_connections 2024-03-22T04:13:45.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.472+0000 7f2f68400640 1 -- 172.21.15.86:0/2565175656 wait complete. 2024-03-22T04:13:45.496 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-03-22T04:13:43.831214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231988,"kb_used_data":7496,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754742732,"statfs":{"total":773094113280,"available":772856557568,"internally_reserved":0,"allocated":7675904,"data_stored":4071564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001516"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935185+0000","last_change":"2024-03-22T04:13:36.536643+0000","last_active":"2024-03-22T04:13:37.935185+0000","last_peered":"2024-03-22T04:13:37.935185+0000","last_clean":"2024-03-22T04:13:37.935185+0000","last_became_active":"2024-03-22T04:13:36.536338+0000","last_became_peered":"2024-03-22T04:13:36.536338+0000","last_unstale":"2024-03-22T04:13:37.935185+0000","last_undegraded":"2024-03-22T04:13:37.935185+0000","last_fullsized":"2024-03-22T04:13:37.935185+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:00:47.125992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533027+0000","last_change":"2024-03-22T04:13:36.532727+0000","last_active":"2024-03-22T04:13:37.533027+0000","last_peered":"2024-03-22T04:13:37.533027+0000","last_clean":"2024-03-22T04:13:37.533027+0000","last_became_active":"2024-03-22T04:13:36.532232+0000","last_became_peered":"2024-03-22T04:13:36.532232+0000","last_unstale":"2024-03-22T04:13:37.533027+0000","last_undegraded":"2024-03-22T04:13:37.533027+0000","last_fullsized":"2024-03-22T04:13:37.533027+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:56:07.942982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934770+0000","last_change":"2024-03-22T04:13:36.550839+0000","last_active":"2024-03-22T04:13:37.934770+0000","last_peered":"2024-03-22T04:13:37.934770+0000","last_clean":"2024-03-22T04:13:37.934770+0000","last_became_active":"2024-03-22T04:13:36.550536+0000","last_became_peered":"2024-03-22T04:13:36.550536+0000","last_unstale":"2024-03-22T04:13:37.934770+0000","last_undegraded":"2024-03-22T04:13:37.934770+0000","last_fullsized":"2024-03-22T04:13:37.934770+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:33:43.016697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537165+0000","last_change":"2024-03-22T04:13:36.535093+0000","last_active":"2024-03-22T04:13:37.537165+0000","last_peered":"2024-03-22T04:13:37.537165+0000","last_clean":"2024-03-22T04:13:37.537165+0000","last_became_active":"2024-03-22T04:13:36.533587+0000","last_became_peered":"2024-03-22T04:13:36.533587+0000","last_unstale":"2024-03-22T04:13:37.537165+0000","last_undegraded":"2024-03-22T04:13:37.537165+0000","last_fullsized":"2024-03-22T04:13:37.537165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:56:07.942982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934467+0000","last_change":"2024-03-22T04:13:36.541445+0000","last_active":"2024-03-22T04:13:37.934467+0000","last_peered":"2024-03-22T04:13:37.934467+0000","last_clean":"2024-03-22T04:13:37.934467+0000","last_became_active":"2024-03-22T04:13:36.540929+0000","last_became_peered":"2024-03-22T04:13:36.540929+0000","last_unstale":"2024-03-22T04:13:37.934467+0000","last_undegraded":"2024-03-22T04:13:37.934467+0000","last_fullsized":"2024-03-22T04:13:37.934467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:18:51.428914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532301+0000","last_change":"2024-03-22T04:13:36.528925+0000","last_active":"2024-03-22T04:13:37.532301+0000","last_peered":"2024-03-22T04:13:37.532301+0000","last_clean":"2024-03-22T04:13:37.532301+0000","last_became_active":"2024-03-22T04:13:36.528306+0000","last_became_peered":"2024-03-22T04:13:36.528306+0000","last_unstale":"2024-03-22T04:13:37.532301+0000","last_undegraded":"2024-03-22T04:13:37.532301+0000","last_fullsized":"2024-03-22T04:13:37.532301+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:04:53.585258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934798+0000","last_change":"2024-03-22T04:13:36.537022+0000","last_active":"2024-03-22T04:13:37.934798+0000","last_peered":"2024-03-22T04:13:37.934798+0000","last_clean":"2024-03-22T04:13:37.934798+0000","last_became_active":"2024-03-22T04:13:36.536647+0000","last_became_peered":"2024-03-22T04:13:36.536647+0000","last_unstale":"2024-03-22T04:13:37.934798+0000","last_undegraded":"2024-03-22T04:13:37.934798+0000","last_fullsized":"2024-03-22T04:13:37.934798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:52:03.782357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934493+0000","last_change":"2024-03-22T04:13:36.546972+0000","last_active":"2024-03-22T04:13:37.934493+0000","last_peered":"2024-03-22T04:13:37.934493+0000","last_clean":"2024-03-22T04:13:37.934493+0000","last_became_active":"2024-03-22T04:13:36.546730+0000","last_became_peered":"2024-03-22T04:13:36.546730+0000","last_unstale":"2024-03-22T04:13:37.934493+0000","last_undegraded":"2024-03-22T04:13:37.934493+0000","last_fullsized":"2024-03-22T04:13:37.934493+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:01:01.082434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537330+0000","last_change":"2024-03-22T04:13:36.536370+0000","last_active":"2024-03-22T04:13:37.537330+0000","last_peered":"2024-03-22T04:13:37.537330+0000","last_clean":"2024-03-22T04:13:37.537330+0000","last_became_active":"2024-03-22T04:13:36.536070+0000","last_became_peered":"2024-03-22T04:13:36.536070+0000","last_unstale":"2024-03-22T04:13:37.537330+0000","last_undegraded":"2024-03-22T04:13:37.537330+0000","last_fullsized":"2024-03-22T04:13:37.537330+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:32.466308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533436+0000","last_change":"2024-03-22T04:13:36.534670+0000","last_active":"2024-03-22T04:13:37.533436+0000","last_peered":"2024-03-22T04:13:37.533436+0000","last_clean":"2024-03-22T04:13:37.533436+0000","last_became_active":"2024-03-22T04:13:36.534356+0000","last_became_peered":"2024-03-22T04:13:36.534356+0000","last_unstale":"2024-03-22T04:13:37.533436+0000","last_undegraded":"2024-03-22T04:13:37.533436+0000","last_fullsized":"2024-03-22T04:13:37.533436+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:22.612979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537416+0000","last_change":"2024-03-22T04:13:36.539182+0000","last_active":"2024-03-22T04:13:37.537416+0000","last_peered":"2024-03-22T04:13:37.537416+0000","last_clean":"2024-03-22T04:13:37.537416+0000","last_became_active":"2024-03-22T04:13:36.538916+0000","last_became_peered":"2024-03-22T04:13:36.538916+0000","last_unstale":"2024-03-22T04:13:37.537416+0000","last_undegraded":"2024-03-22T04:13:37.537416+0000","last_fullsized":"2024-03-22T04:13:37.537416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:43:46.967166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537312+0000","last_change":"2024-03-22T04:13:36.543786+0000","last_active":"2024-03-22T04:13:37.537312+0000","last_peered":"2024-03-22T04:13:37.537312+0000","last_clean":"2024-03-22T04:13:37.537312+0000","last_became_active":"2024-03-22T04:13:36.543194+0000","last_became_peered":"2024-03-22T04:13:36.543194+0000","last_unstale":"2024-03-22T04:13:37.537312+0000","last_undegraded":"2024-03-22T04:13:37.537312+0000","last_fullsized":"2024-03-22T04:13:37.537312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:01.059946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935143+0000","last_change":"2024-03-22T04:13:36.533115+0000","last_active":"2024-03-22T04:13:37.935143+0000","last_peered":"2024-03-22T04:13:37.935143+0000","last_clean":"2024-03-22T04:13:37.935143+0000","last_became_active":"2024-03-22T04:13:36.532940+0000","last_became_peered":"2024-03-22T04:13:36.532940+0000","last_unstale":"2024-03-22T04:13:37.935143+0000","last_undegraded":"2024-03-22T04:13:37.935143+0000","last_fullsized":"2024-03-22T04:13:37.935143+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:44.092644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934973+0000","last_change":"2024-03-22T04:13:36.531402+0000","last_active":"2024-03-22T04:13:37.934973+0000","last_peered":"2024-03-22T04:13:37.934973+0000","last_clean":"2024-03-22T04:13:37.934973+0000","last_became_active":"2024-03-22T04:13:36.531066+0000","last_became_peered":"2024-03-22T04:13:36.531066+0000","last_unstale":"2024-03-22T04:13:37.934973+0000","last_undegraded":"2024-03-22T04:13:37.934973+0000","last_fullsized":"2024-03-22T04:13:37.934973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:57.953068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532968+0000","last_change":"2024-03-22T04:13:36.528859+0000","last_active":"2024-03-22T04:13:37.532968+0000","last_peered":"2024-03-22T04:13:37.532968+0000","last_clean":"2024-03-22T04:13:37.532968+0000","last_became_active":"2024-03-22T04:13:36.528573+0000","last_became_peered":"2024-03-22T04:13:36.528573+0000","last_unstale":"2024-03-22T04:13:37.532968+0000","last_undegraded":"2024-03-22T04:13:37.532968+0000","last_fullsized":"2024-03-22T04:13:37.532968+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:40:07.797167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.541124+0000","last_change":"2024-03-22T04:13:36.544779+0000","last_active":"2024-03-22T04:13:37.541124+0000","last_peered":"2024-03-22T04:13:37.541124+0000","last_clean":"2024-03-22T04:13:37.541124+0000","last_became_active":"2024-03-22T04:13:36.544484+0000","last_became_peered":"2024-03-22T04:13:36.544484+0000","last_unstale":"2024-03-22T04:13:37.541124+0000","last_undegraded":"2024-03-22T04:13:37.541124+0000","last_fullsized":"2024-03-22T04:13:37.541124+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:51.911920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534709+0000","last_change":"2024-03-22T04:13:36.534928+0000","last_active":"2024-03-22T04:13:37.534709+0000","last_peered":"2024-03-22T04:13:37.534709+0000","last_clean":"2024-03-22T04:13:37.534709+0000","last_became_active":"2024-03-22T04:13:36.534458+0000","last_became_peered":"2024-03-22T04:13:36.534458+0000","last_unstale":"2024-03-22T04:13:37.534709+0000","last_undegraded":"2024-03-22T04:13:37.534709+0000","last_fullsized":"2024-03-22T04:13:37.534709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:10:28.784356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537311+0000","last_change":"2024-03-22T04:13:36.543008+0000","last_active":"2024-03-22T04:13:37.537311+0000","last_peered":"2024-03-22T04:13:37.537311+0000","last_clean":"2024-03-22T04:13:37.537311+0000","last_became_active":"2024-03-22T04:13:36.542504+0000","last_became_peered":"2024-03-22T04:13:36.542504+0000","last_unstale":"2024-03-22T04:13:37.537311+0000","last_undegraded":"2024-03-22T04:13:37.537311+0000","last_fullsized":"2024-03-22T04:13:37.537311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:17:30.787284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537715+0000","last_change":"2024-03-22T04:13:36.539156+0000","last_active":"2024-03-22T04:13:37.537715+0000","last_peered":"2024-03-22T04:13:37.537715+0000","last_clean":"2024-03-22T04:13:37.537715+0000","last_became_active":"2024-03-22T04:13:36.538851+0000","last_became_peered":"2024-03-22T04:13:36.538851+0000","last_unstale":"2024-03-22T04:13:37.537715+0000","last_undegraded":"2024-03-22T04:13:37.537715+0000","last_fullsized":"2024-03-22T04:13:37.537715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:39:58.057283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534466+0000","last_change":"2024-03-22T04:13:36.533555+0000","last_active":"2024-03-22T04:13:37.534466+0000","last_peered":"2024-03-22T04:13:37.534466+0000","last_clean":"2024-03-22T04:13:37.534466+0000","last_became_active":"2024-03-22T04:13:36.533261+0000","last_became_peered":"2024-03-22T04:13:36.533261+0000","last_unstale":"2024-03-22T04:13:37.534466+0000","last_undegraded":"2024-03-22T04:13:37.534466+0000","last_fullsized":"2024-03-22T04:13:37.534466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:22.636235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935254+0000","last_change":"2024-03-22T04:13:36.529562+0000","last_active":"2024-03-22T04:13:37.935254+0000","last_peered":"2024-03-22T04:13:37.935254+0000","last_clean":"2024-03-22T04:13:37.935254+0000","last_became_active":"2024-03-22T04:13:36.529315+0000","last_became_peered":"2024-03-22T04:13:36.529315+0000","last_unstale":"2024-03-22T04:13:37.935254+0000","last_undegraded":"2024-03-22T04:13:37.935254+0000","last_fullsized":"2024-03-22T04:13:37.935254+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:15:12.469198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532970+0000","last_change":"2024-03-22T04:13:36.529136+0000","last_active":"2024-03-22T04:13:37.532970+0000","last_peered":"2024-03-22T04:13:37.532970+0000","last_clean":"2024-03-22T04:13:37.532970+0000","last_became_active":"2024-03-22T04:13:36.528642+0000","last_became_peered":"2024-03-22T04:13:36.528642+0000","last_unstale":"2024-03-22T04:13:37.532970+0000","last_undegraded":"2024-03-22T04:13:37.532970+0000","last_fullsized":"2024-03-22T04:13:37.532970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:13:42.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.541070+0000","last_change":"2024-03-22T04:13:36.544284+0000","last_active":"2024-03-22T04:13:37.541070+0000","last_peered":"2024-03-22T04:13:37.541070+0000","last_clean":"2024-03-22T04:13:37.541070+0000","last_became_active":"2024-03-22T04:13:36.543996+0000","last_became_peered":"2024-03-22T04:13:36.543996+0000","last_unstale":"2024-03-22T04:13:37.541070+0000","last_undegraded":"2024-03-22T04:13:37.541070+0000","last_fullsized":"2024-03-22T04:13:37.541070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:13.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934473+0000","last_change":"2024-03-22T04:13:36.540542+0000","last_active":"2024-03-22T04:13:37.934473+0000","last_peered":"2024-03-22T04:13:37.934473+0000","last_clean":"2024-03-22T04:13:37.934473+0000","last_became_active":"2024-03-22T04:13:36.540215+0000","last_became_peered":"2024-03-22T04:13:36.540215+0000","last_unstale":"2024-03-22T04:13:37.934473+0000","last_undegraded":"2024-03-22T04:13:37.934473+0000","last_fullsized":"2024-03-22T04:13:37.934473+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:40:36.496755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935514+0000","last_change":"2024-03-22T04:13:36.532645+0000","last_active":"2024-03-22T04:13:37.935514+0000","last_peered":"2024-03-22T04:13:37.935514+0000","last_clean":"2024-03-22T04:13:37.935514+0000","last_became_active":"2024-03-22T04:13:36.531626+0000","last_became_peered":"2024-03-22T04:13:36.531626+0000","last_unstale":"2024-03-22T04:13:37.935514+0000","last_undegraded":"2024-03-22T04:13:37.935514+0000","last_fullsized":"2024-03-22T04:13:37.935514+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:22:16.574264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934597+0000","last_change":"2024-03-22T04:13:36.546688+0000","last_active":"2024-03-22T04:13:37.934597+0000","last_peered":"2024-03-22T04:13:37.934597+0000","last_clean":"2024-03-22T04:13:37.934597+0000","last_became_active":"2024-03-22T04:13:36.546381+0000","last_became_peered":"2024-03-22T04:13:36.546381+0000","last_unstale":"2024-03-22T04:13:37.934597+0000","last_undegraded":"2024-03-22T04:13:37.934597+0000","last_fullsized":"2024-03-22T04:13:37.934597+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:27:03.605752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935247+0000","last_change":"2024-03-22T04:13:36.541919+0000","last_active":"2024-03-22T04:13:37.935247+0000","last_peered":"2024-03-22T04:13:37.935247+0000","last_clean":"2024-03-22T04:13:37.935247+0000","last_became_active":"2024-03-22T04:13:36.541564+0000","last_became_peered":"2024-03-22T04:13:36.541564+0000","last_unstale":"2024-03-22T04:13:37.935247+0000","last_undegraded":"2024-03-22T04:13:37.935247+0000","last_fullsized":"2024-03-22T04:13:37.935247+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:22.612979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935030+0000","last_change":"2024-03-22T04:13:36.533681+0000","last_active":"2024-03-22T04:13:37.935030+0000","last_peered":"2024-03-22T04:13:37.935030+0000","last_clean":"2024-03-22T04:13:37.935030+0000","last_became_active":"2024-03-22T04:13:36.533286+0000","last_became_peered":"2024-03-22T04:13:36.533286+0000","last_unstale":"2024-03-22T04:13:37.935030+0000","last_undegraded":"2024-03-22T04:13:37.935030+0000","last_fullsized":"2024-03-22T04:13:37.935030+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:48:38.248410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537228+0000","last_change":"2024-03-22T04:13:36.541069+0000","last_active":"2024-03-22T04:13:37.537228+0000","last_peered":"2024-03-22T04:13:37.537228+0000","last_clean":"2024-03-22T04:13:37.537228+0000","last_became_active":"2024-03-22T04:13:36.540608+0000","last_became_peered":"2024-03-22T04:13:36.540608+0000","last_unstale":"2024-03-22T04:13:37.537228+0000","last_undegraded":"2024-03-22T04:13:37.537228+0000","last_fullsized":"2024-03-22T04:13:37.537228+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:00:14.181624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.536943+0000","last_change":"2024-03-22T04:13:36.529205+0000","last_active":"2024-03-22T04:13:37.536943+0000","last_peered":"2024-03-22T04:13:37.536943+0000","last_clean":"2024-03-22T04:13:37.536943+0000","last_became_active":"2024-03-22T04:13:36.528946+0000","last_became_peered":"2024-03-22T04:13:36.528946+0000","last_unstale":"2024-03-22T04:13:37.536943+0000","last_undegraded":"2024-03-22T04:13:37.536943+0000","last_fullsized":"2024-03-22T04:13:37.536943+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:48:10.956970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934360+0000","last_change":"2024-03-22T04:13:36.546717+0000","last_active":"2024-03-22T04:13:37.934360+0000","last_peered":"2024-03-22T04:13:37.934360+0000","last_clean":"2024-03-22T04:13:37.934360+0000","last_became_active":"2024-03-22T04:13:36.546358+0000","last_became_peered":"2024-03-22T04:13:36.546358+0000","last_unstale":"2024-03-22T04:13:37.934360+0000","last_undegraded":"2024-03-22T04:13:37.934360+0000","last_fullsized":"2024-03-22T04:13:37.934360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:57:05.797600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934769+0000","last_change":"2024-03-22T04:13:36.532329+0000","last_active":"2024-03-22T04:13:37.934769+0000","last_peered":"2024-03-22T04:13:37.934769+0000","last_clean":"2024-03-22T04:13:37.934769+0000","last_became_active":"2024-03-22T04:13:36.532005+0000","last_became_peered":"2024-03-22T04:13:36.532005+0000","last_unstale":"2024-03-22T04:13:37.934769+0000","last_undegraded":"2024-03-22T04:13:37.934769+0000","last_fullsized":"2024-03-22T04:13:37.934769+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:22:16.574264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537416+0000","last_change":"2024-03-22T04:13:36.542842+0000","last_active":"2024-03-22T04:13:37.537416+0000","last_peered":"2024-03-22T04:13:37.537416+0000","last_clean":"2024-03-22T04:13:37.537416+0000","last_became_active":"2024-03-22T04:13:36.538891+0000","last_became_peered":"2024-03-22T04:13:36.538891+0000","last_unstale":"2024-03-22T04:13:37.537416+0000","last_undegraded":"2024-03-22T04:13:37.537416+0000","last_fullsized":"2024-03-22T04:13:37.537416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:36:30.557676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935187+0000","last_change":"2024-03-22T04:13:35.531755+0000","last_active":"2024-03-22T04:13:37.935187+0000","last_peered":"2024-03-22T04:13:37.935187+0000","last_clean":"2024-03-22T04:13:37.935187+0000","last_became_active":"2024-03-22T04:13:35.531433+0000","last_became_peered":"2024-03-22T04:13:35.531433+0000","last_unstale":"2024-03-22T04:13:37.935187+0000","last_undegraded":"2024-03-22T04:13:37.935187+0000","last_fullsized":"2024-03-22T04:13:37.935187+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:31:05.689243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934820+0000","last_change":"2024-03-22T04:13:36.550235+0000","last_active":"2024-03-22T04:13:37.934820+0000","last_peered":"2024-03-22T04:13:37.934820+0000","last_clean":"2024-03-22T04:13:37.934820+0000","last_became_active":"2024-03-22T04:13:36.545989+0000","last_became_peered":"2024-03-22T04:13:36.545989+0000","last_unstale":"2024-03-22T04:13:37.934820+0000","last_undegraded":"2024-03-22T04:13:37.934820+0000","last_fullsized":"2024-03-22T04:13:37.934820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:23.603767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935121+0000","last_change":"2024-03-22T04:13:35.529846+0000","last_active":"2024-03-22T04:13:37.935121+0000","last_peered":"2024-03-22T04:13:37.935121+0000","last_clean":"2024-03-22T04:13:37.935121+0000","last_became_active":"2024-03-22T04:13:35.529546+0000","last_became_peered":"2024-03-22T04:13:35.529546+0000","last_unstale":"2024-03-22T04:13:37.935121+0000","last_undegraded":"2024-03-22T04:13:37.935121+0000","last_fullsized":"2024-03-22T04:13:37.935121+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:15:11.462730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533181+0000","last_change":"2024-03-22T04:13:36.532489+0000","last_active":"2024-03-22T04:13:37.533181+0000","last_peered":"2024-03-22T04:13:37.533181+0000","last_clean":"2024-03-22T04:13:37.533181+0000","last_became_active":"2024-03-22T04:13:36.531886+0000","last_became_peered":"2024-03-22T04:13:36.531886+0000","last_unstale":"2024-03-22T04:13:37.533181+0000","last_undegraded":"2024-03-22T04:13:37.533181+0000","last_fullsized":"2024-03-22T04:13:37.533181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:00:14.181624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935293+0000","last_change":"2024-03-22T04:13:35.530455+0000","last_active":"2024-03-22T04:13:37.935293+0000","last_peered":"2024-03-22T04:13:37.935293+0000","last_clean":"2024-03-22T04:13:37.935293+0000","last_became_active":"2024-03-22T04:13:35.530130+0000","last_became_peered":"2024-03-22T04:13:35.530130+0000","last_unstale":"2024-03-22T04:13:37.935293+0000","last_undegraded":"2024-03-22T04:13:37.935293+0000","last_fullsized":"2024-03-22T04:13:37.935293+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:21.606511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537378+0000","last_change":"2024-03-22T04:13:36.542913+0000","last_active":"2024-03-22T04:13:37.537378+0000","last_peered":"2024-03-22T04:13:37.537378+0000","last_clean":"2024-03-22T04:13:37.537378+0000","last_became_active":"2024-03-22T04:13:36.542587+0000","last_became_peered":"2024-03-22T04:13:36.542587+0000","last_unstale":"2024-03-22T04:13:37.537378+0000","last_undegraded":"2024-03-22T04:13:37.537378+0000","last_fullsized":"2024-03-22T04:13:37.537378+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:21:47.679309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532775+0000","last_change":"2024-03-22T04:13:35.533152+0000","last_active":"2024-03-22T04:13:37.532775+0000","last_peered":"2024-03-22T04:13:37.532775+0000","last_clean":"2024-03-22T04:13:37.532775+0000","last_became_active":"2024-03-22T04:13:35.532739+0000","last_became_peered":"2024-03-22T04:13:35.532739+0000","last_unstale":"2024-03-22T04:13:37.532775+0000","last_undegraded":"2024-03-22T04:13:37.532775+0000","last_fullsized":"2024-03-22T04:13:37.532775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:31:05.689243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533547+0000","last_change":"2024-03-22T04:13:36.539261+0000","last_active":"2024-03-22T04:13:37.533547+0000","last_peered":"2024-03-22T04:13:37.533547+0000","last_clean":"2024-03-22T04:13:37.533547+0000","last_became_active":"2024-03-22T04:13:36.538948+0000","last_became_peered":"2024-03-22T04:13:36.538948+0000","last_unstale":"2024-03-22T04:13:37.533547+0000","last_undegraded":"2024-03-22T04:13:37.533547+0000","last_fullsized":"2024-03-22T04:13:37.533547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.370928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534342+0000","last_change":"2024-03-22T04:13:35.528073+0000","last_active":"2024-03-22T04:13:37.534342+0000","last_peered":"2024-03-22T04:13:37.534342+0000","last_clean":"2024-03-22T04:13:37.534342+0000","last_became_active":"2024-03-22T04:13:35.527410+0000","last_became_peered":"2024-03-22T04:13:35.527410+0000","last_unstale":"2024-03-22T04:13:37.534342+0000","last_undegraded":"2024-03-22T04:13:37.534342+0000","last_fullsized":"2024-03-22T04:13:37.534342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:00:59.588177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934798+0000","last_change":"2024-03-22T04:13:36.532397+0000","last_active":"2024-03-22T04:13:37.934798+0000","last_peered":"2024-03-22T04:13:37.934798+0000","last_clean":"2024-03-22T04:13:37.934798+0000","last_became_active":"2024-03-22T04:13:36.532117+0000","last_became_peered":"2024-03-22T04:13:36.532117+0000","last_unstale":"2024-03-22T04:13:37.934798+0000","last_undegraded":"2024-03-22T04:13:37.934798+0000","last_fullsized":"2024-03-22T04:13:37.934798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:40.084225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533086+0000","last_change":"2024-03-22T04:13:35.526516+0000","last_active":"2024-03-22T04:13:37.533086+0000","last_peered":"2024-03-22T04:13:37.533086+0000","last_clean":"2024-03-22T04:13:37.533086+0000","last_became_active":"2024-03-22T04:13:35.526208+0000","last_became_peered":"2024-03-22T04:13:35.526208+0000","last_unstale":"2024-03-22T04:13:37.533086+0000","last_undegraded":"2024-03-22T04:13:37.533086+0000","last_fullsized":"2024-03-22T04:13:37.533086+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:56.946600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533142+0000","last_change":"2024-03-22T04:13:36.530615+0000","last_active":"2024-03-22T04:13:37.533142+0000","last_peered":"2024-03-22T04:13:37.533142+0000","last_clean":"2024-03-22T04:13:37.533142+0000","last_became_active":"2024-03-22T04:13:36.530013+0000","last_became_peered":"2024-03-22T04:13:36.530013+0000","last_unstale":"2024-03-22T04:13:37.533142+0000","last_undegraded":"2024-03-22T04:13:37.533142+0000","last_fullsized":"2024-03-22T04:13:37.533142+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:57.953068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934890+0000","last_change":"2024-03-22T04:13:35.534172+0000","last_active":"2024-03-22T04:13:37.934890+0000","last_peered":"2024-03-22T04:13:37.934890+0000","last_clean":"2024-03-22T04:13:37.934890+0000","last_became_active":"2024-03-22T04:13:35.533651+0000","last_became_peered":"2024-03-22T04:13:35.533651+0000","last_unstale":"2024-03-22T04:13:37.934890+0000","last_undegraded":"2024-03-22T04:13:37.934890+0000","last_fullsized":"2024-03-22T04:13:37.934890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:11:03.089640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534186+0000","last_change":"2024-03-22T04:13:36.535993+0000","last_active":"2024-03-22T04:13:37.534186+0000","last_peered":"2024-03-22T04:13:37.534186+0000","last_clean":"2024-03-22T04:13:37.534186+0000","last_became_active":"2024-03-22T04:13:36.534295+0000","last_became_peered":"2024-03-22T04:13:36.534295+0000","last_unstale":"2024-03-22T04:13:37.534186+0000","last_undegraded":"2024-03-22T04:13:37.534186+0000","last_fullsized":"2024-03-22T04:13:37.534186+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:58.215479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.536794+0000","last_change":"2024-03-22T04:13:35.533417+0000","last_active":"2024-03-22T04:13:37.536794+0000","last_peered":"2024-03-22T04:13:37.536794+0000","last_clean":"2024-03-22T04:13:37.536794+0000","last_became_active":"2024-03-22T04:13:35.533053+0000","last_became_peered":"2024-03-22T04:13:35.533053+0000","last_unstale":"2024-03-22T04:13:37.536794+0000","last_undegraded":"2024-03-22T04:13:37.536794+0000","last_fullsized":"2024-03-22T04:13:37.536794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:11:03.089640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935033+0000","last_change":"2024-03-22T04:13:36.539032+0000","last_active":"2024-03-22T04:13:37.935033+0000","last_peered":"2024-03-22T04:13:37.935033+0000","last_clean":"2024-03-22T04:13:37.935033+0000","last_became_active":"2024-03-22T04:13:36.538667+0000","last_became_peered":"2024-03-22T04:13:36.538667+0000","last_unstale":"2024-03-22T04:13:37.935033+0000","last_undegraded":"2024-03-22T04:13:37.935033+0000","last_fullsized":"2024-03-22T04:13:37.935033+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:59:33.015881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537482+0000","last_change":"2024-03-22T04:13:35.535058+0000","last_active":"2024-03-22T04:13:37.537482+0000","last_peered":"2024-03-22T04:13:37.537482+0000","last_clean":"2024-03-22T04:13:37.537482+0000","last_became_active":"2024-03-22T04:13:35.534345+0000","last_became_peered":"2024-03-22T04:13:35.534345+0000","last_unstale":"2024-03-22T04:13:37.537482+0000","last_undegraded":"2024-03-22T04:13:37.537482+0000","last_fullsized":"2024-03-22T04:13:37.537482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:40:06.790699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534393+0000","last_change":"2024-03-22T04:13:36.527089+0000","last_active":"2024-03-22T04:13:37.534393+0000","last_peered":"2024-03-22T04:13:37.534393+0000","last_clean":"2024-03-22T04:13:37.534393+0000","last_became_active":"2024-03-22T04:13:36.526807+0000","last_became_peered":"2024-03-22T04:13:36.526807+0000","last_unstale":"2024-03-22T04:13:37.534393+0000","last_undegraded":"2024-03-22T04:13:37.534393+0000","last_fullsized":"2024-03-22T04:13:37.534393+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:57:11.984612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935382+0000","last_change":"2024-03-22T04:13:35.525257+0000","last_active":"2024-03-22T04:13:37.935382+0000","last_peered":"2024-03-22T04:13:37.935382+0000","last_clean":"2024-03-22T04:13:37.935382+0000","last_became_active":"2024-03-22T04:13:35.524936+0000","last_became_peered":"2024-03-22T04:13:35.524936+0000","last_unstale":"2024-03-22T04:13:37.935382+0000","last_undegraded":"2024-03-22T04:13:37.935382+0000","last_fullsized":"2024-03-22T04:13:37.935382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:03:28.004834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935278+0000","last_change":"2024-03-22T04:13:36.532346+0000","last_active":"2024-03-22T04:13:37.935278+0000","last_peered":"2024-03-22T04:13:37.935278+0000","last_clean":"2024-03-22T04:13:37.935278+0000","last_became_active":"2024-03-22T04:13:36.532109+0000","last_became_peered":"2024-03-22T04:13:36.532109+0000","last_unstale":"2024-03-22T04:13:37.935278+0000","last_undegraded":"2024-03-22T04:13:37.935278+0000","last_fullsized":"2024-03-22T04:13:37.935278+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:31:06.695711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934826+0000","last_change":"2024-03-22T04:13:35.536397+0000","last_active":"2024-03-22T04:13:37.934826+0000","last_peered":"2024-03-22T04:13:37.934826+0000","last_clean":"2024-03-22T04:13:37.934826+0000","last_became_active":"2024-03-22T04:13:35.536137+0000","last_became_peered":"2024-03-22T04:13:35.536137+0000","last_unstale":"2024-03-22T04:13:37.934826+0000","last_undegraded":"2024-03-22T04:13:37.934826+0000","last_fullsized":"2024-03-22T04:13:37.934826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:31:32.025082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934682+0000","last_change":"2024-03-22T04:13:36.550088+0000","last_active":"2024-03-22T04:13:37.934682+0000","last_peered":"2024-03-22T04:13:37.934682+0000","last_clean":"2024-03-22T04:13:37.934682+0000","last_became_active":"2024-03-22T04:13:36.549779+0000","last_became_peered":"2024-03-22T04:13:36.549779+0000","last_unstale":"2024-03-22T04:13:37.934682+0000","last_undegraded":"2024-03-22T04:13:37.934682+0000","last_fullsized":"2024-03-22T04:13:37.934682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:26:53.865888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532845+0000","last_change":"2024-03-22T04:13:35.529973+0000","last_active":"2024-03-22T04:13:37.532845+0000","last_peered":"2024-03-22T04:13:37.532845+0000","last_clean":"2024-03-22T04:13:37.532845+0000","last_became_active":"2024-03-22T04:13:35.529696+0000","last_became_peered":"2024-03-22T04:13:35.529696+0000","last_unstale":"2024-03-22T04:13:37.532845+0000","last_undegraded":"2024-03-22T04:13:37.532845+0000","last_fullsized":"2024-03-22T04:13:37.532845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:40:22.406882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533202+0000","last_change":"2024-03-22T04:13:36.528978+0000","last_active":"2024-03-22T04:13:37.533202+0000","last_peered":"2024-03-22T04:13:37.533202+0000","last_clean":"2024-03-22T04:13:37.533202+0000","last_became_active":"2024-03-22T04:13:36.528529+0000","last_became_peered":"2024-03-22T04:13:36.528529+0000","last_unstale":"2024-03-22T04:13:37.533202+0000","last_undegraded":"2024-03-22T04:13:37.533202+0000","last_fullsized":"2024-03-22T04:13:37.533202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:51:43.486787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537182+0000","last_change":"2024-03-22T04:13:35.529378+0000","last_active":"2024-03-22T04:13:37.537182+0000","last_peered":"2024-03-22T04:13:37.537182+0000","last_clean":"2024-03-22T04:13:37.537182+0000","last_became_active":"2024-03-22T04:13:35.529096+0000","last_became_peered":"2024-03-22T04:13:35.529096+0000","last_unstale":"2024-03-22T04:13:37.537182+0000","last_undegraded":"2024-03-22T04:13:37.537182+0000","last_fullsized":"2024-03-22T04:13:37.537182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:56.946600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533081+0000","last_change":"2024-03-22T04:13:36.525479+0000","last_active":"2024-03-22T04:13:37.533081+0000","last_peered":"2024-03-22T04:13:37.533081+0000","last_clean":"2024-03-22T04:13:37.533081+0000","last_became_active":"2024-03-22T04:13:36.525209+0000","last_became_peered":"2024-03-22T04:13:36.525209+0000","last_unstale":"2024-03-22T04:13:37.533081+0000","last_undegraded":"2024-03-22T04:13:37.533081+0000","last_fullsized":"2024-03-22T04:13:37.533081+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:21:42.400738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533082+0000","last_change":"2024-03-22T04:13:35.536795+0000","last_active":"2024-03-22T04:13:37.533082+0000","last_peered":"2024-03-22T04:13:37.533082+0000","last_clean":"2024-03-22T04:13:37.533082+0000","last_became_active":"2024-03-22T04:13:35.534453+0000","last_became_peered":"2024-03-22T04:13:35.534453+0000","last_unstale":"2024-03-22T04:13:37.533082+0000","last_undegraded":"2024-03-22T04:13:37.533082+0000","last_fullsized":"2024-03-22T04:13:37.533082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:23.221070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532993+0000","last_change":"2024-03-22T04:13:36.530862+0000","last_active":"2024-03-22T04:13:37.532993+0000","last_peered":"2024-03-22T04:13:37.532993+0000","last_clean":"2024-03-22T04:13:37.532993+0000","last_became_active":"2024-03-22T04:13:36.530365+0000","last_became_peered":"2024-03-22T04:13:36.530365+0000","last_unstale":"2024-03-22T04:13:37.532993+0000","last_undegraded":"2024-03-22T04:13:37.532993+0000","last_fullsized":"2024-03-22T04:13:37.532993+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:24.704154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934337+0000","last_change":"2024-03-22T04:13:35.529571+0000","last_active":"2024-03-22T04:13:37.934337+0000","last_peered":"2024-03-22T04:13:37.934337+0000","last_clean":"2024-03-22T04:13:37.934337+0000","last_became_active":"2024-03-22T04:13:35.528385+0000","last_became_peered":"2024-03-22T04:13:35.528385+0000","last_unstale":"2024-03-22T04:13:37.934337+0000","last_undegraded":"2024-03-22T04:13:37.934337+0000","last_fullsized":"2024-03-22T04:13:37.934337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:03:20.113486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934905+0000","last_change":"2024-03-22T04:13:36.543079+0000","last_active":"2024-03-22T04:13:37.934905+0000","last_peered":"2024-03-22T04:13:37.934905+0000","last_clean":"2024-03-22T04:13:37.934905+0000","last_became_active":"2024-03-22T04:13:36.542677+0000","last_became_peered":"2024-03-22T04:13:36.542677+0000","last_unstale":"2024-03-22T04:13:37.934905+0000","last_undegraded":"2024-03-22T04:13:37.934905+0000","last_fullsized":"2024-03-22T04:13:37.934905+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:05:32.138647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532855+0000","last_change":"2024-03-22T04:13:35.536315+0000","last_active":"2024-03-22T04:13:37.532855+0000","last_peered":"2024-03-22T04:13:37.532855+0000","last_clean":"2024-03-22T04:13:37.532855+0000","last_became_active":"2024-03-22T04:13:35.535908+0000","last_became_peered":"2024-03-22T04:13:35.535908+0000","last_unstale":"2024-03-22T04:13:37.532855+0000","last_undegraded":"2024-03-22T04:13:37.532855+0000","last_fullsized":"2024-03-22T04:13:37.532855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:00:58.326961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537561+0000","last_change":"2024-03-22T04:13:36.544623+0000","last_active":"2024-03-22T04:13:37.537561+0000","last_peered":"2024-03-22T04:13:37.537561+0000","last_clean":"2024-03-22T04:13:37.537561+0000","last_became_active":"2024-03-22T04:13:36.543545+0000","last_became_peered":"2024-03-22T04:13:36.543545+0000","last_unstale":"2024-03-22T04:13:37.537561+0000","last_undegraded":"2024-03-22T04:13:37.537561+0000","last_fullsized":"2024-03-22T04:13:37.537561+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:13:25.762587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537539+0000","last_change":"2024-03-22T04:13:35.534854+0000","last_active":"2024-03-22T04:13:37.537539+0000","last_peered":"2024-03-22T04:13:37.537539+0000","last_clean":"2024-03-22T04:13:37.537539+0000","last_became_active":"2024-03-22T04:13:35.534331+0000","last_became_peered":"2024-03-22T04:13:35.534331+0000","last_unstale":"2024-03-22T04:13:37.537539+0000","last_undegraded":"2024-03-22T04:13:37.537539+0000","last_fullsized":"2024-03-22T04:13:37.537539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:53:23.697686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935261+0000","last_change":"2024-03-22T04:13:36.553382+0000","last_active":"2024-03-22T04:13:37.935261+0000","last_peered":"2024-03-22T04:13:37.935261+0000","last_clean":"2024-03-22T04:13:37.935261+0000","last_became_active":"2024-03-22T04:13:36.553128+0000","last_became_peered":"2024-03-22T04:13:36.553128+0000","last_unstale":"2024-03-22T04:13:37.935261+0000","last_undegraded":"2024-03-22T04:13:37.935261+0000","last_fullsized":"2024-03-22T04:13:37.935261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:39:51.985574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537312+0000","last_change":"2024-03-22T04:13:35.533530+0000","last_active":"2024-03-22T04:13:37.537312+0000","last_peered":"2024-03-22T04:13:37.537312+0000","last_clean":"2024-03-22T04:13:37.537312+0000","last_became_active":"2024-03-22T04:13:35.533254+0000","last_became_peered":"2024-03-22T04:13:35.533254+0000","last_unstale":"2024-03-22T04:13:37.537312+0000","last_undegraded":"2024-03-22T04:13:37.537312+0000","last_fullsized":"2024-03-22T04:13:37.537312+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:21.606511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935434+0000","last_change":"2024-03-22T04:13:36.528389+0000","last_active":"2024-03-22T04:13:37.935434+0000","last_peered":"2024-03-22T04:13:37.935434+0000","last_clean":"2024-03-22T04:13:37.935434+0000","last_became_active":"2024-03-22T04:13:36.528139+0000","last_became_peered":"2024-03-22T04:13:36.528139+0000","last_unstale":"2024-03-22T04:13:37.935434+0000","last_undegraded":"2024-03-22T04:13:37.935434+0000","last_fullsized":"2024-03-22T04:13:37.935434+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:40:23.413350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934883+0000","last_change":"2024-03-22T04:13:35.530606+0000","last_active":"2024-03-22T04:13:37.934883+0000","last_peered":"2024-03-22T04:13:37.934883+0000","last_clean":"2024-03-22T04:13:37.934883+0000","last_became_active":"2024-03-22T04:13:35.530250+0000","last_became_peered":"2024-03-22T04:13:35.530250+0000","last_unstale":"2024-03-22T04:13:37.934883+0000","last_undegraded":"2024-03-22T04:13:37.934883+0000","last_fullsized":"2024-03-22T04:13:37.934883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:12:37.805223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934653+0000","last_change":"2024-03-22T04:13:36.545102+0000","last_active":"2024-03-22T04:13:37.934653+0000","last_peered":"2024-03-22T04:13:37.934653+0000","last_clean":"2024-03-22T04:13:37.934653+0000","last_became_active":"2024-03-22T04:13:36.544856+0000","last_became_peered":"2024-03-22T04:13:36.544856+0000","last_unstale":"2024-03-22T04:13:37.934653+0000","last_undegraded":"2024-03-22T04:13:37.934653+0000","last_fullsized":"2024-03-22T04:13:37.934653+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:03:31.551981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.541127+0000","last_change":"2024-03-22T04:13:35.539839+0000","last_active":"2024-03-22T04:13:37.541127+0000","last_peered":"2024-03-22T04:13:37.541127+0000","last_clean":"2024-03-22T04:13:37.541127+0000","last_became_active":"2024-03-22T04:13:35.539572+0000","last_became_peered":"2024-03-22T04:13:35.539572+0000","last_unstale":"2024-03-22T04:13:37.541127+0000","last_undegraded":"2024-03-22T04:13:37.541127+0000","last_fullsized":"2024-03-22T04:13:37.541127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:17:22.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532319+0000","last_change":"2024-03-22T04:13:36.528075+0000","last_active":"2024-03-22T04:13:37.532319+0000","last_peered":"2024-03-22T04:13:37.532319+0000","last_clean":"2024-03-22T04:13:37.532319+0000","last_became_active":"2024-03-22T04:13:36.527766+0000","last_became_peered":"2024-03-22T04:13:36.527766+0000","last_unstale":"2024-03-22T04:13:37.532319+0000","last_undegraded":"2024-03-22T04:13:37.532319+0000","last_fullsized":"2024-03-22T04:13:37.532319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:06:14.434151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532527+0000","last_change":"2024-03-22T04:13:35.533632+0000","last_active":"2024-03-22T04:13:37.532527+0000","last_peered":"2024-03-22T04:13:37.532527+0000","last_clean":"2024-03-22T04:13:37.532527+0000","last_became_active":"2024-03-22T04:13:35.533349+0000","last_became_peered":"2024-03-22T04:13:35.533349+0000","last_unstale":"2024-03-22T04:13:37.532527+0000","last_undegraded":"2024-03-22T04:13:37.532527+0000","last_fullsized":"2024-03-22T04:13:37.532527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:13:21.606511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934835+0000","last_change":"2024-03-22T04:13:36.521144+0000","last_active":"2024-03-22T04:13:37.934835+0000","last_peered":"2024-03-22T04:13:37.934835+0000","last_clean":"2024-03-22T04:13:37.934835+0000","last_became_active":"2024-03-22T04:13:36.520882+0000","last_became_peered":"2024-03-22T04:13:36.520882+0000","last_unstale":"2024-03-22T04:13:37.934835+0000","last_undegraded":"2024-03-22T04:13:37.934835+0000","last_fullsized":"2024-03-22T04:13:37.934835+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:21:42.400738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533180+0000","last_change":"2024-03-22T04:13:35.527224+0000","last_active":"2024-03-22T04:13:37.533180+0000","last_peered":"2024-03-22T04:13:37.533180+0000","last_clean":"2024-03-22T04:13:37.533180+0000","last_became_active":"2024-03-22T04:13:35.526938+0000","last_became_peered":"2024-03-22T04:13:35.526938+0000","last_unstale":"2024-03-22T04:13:37.533180+0000","last_undegraded":"2024-03-22T04:13:37.533180+0000","last_fullsized":"2024-03-22T04:13:37.533180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:03:28.004834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537011+0000","last_change":"2024-03-22T04:13:36.535391+0000","last_active":"2024-03-22T04:13:37.537011+0000","last_peered":"2024-03-22T04:13:37.537011+0000","last_clean":"2024-03-22T04:13:37.537011+0000","last_became_active":"2024-03-22T04:13:36.535086+0000","last_became_peered":"2024-03-22T04:13:36.535086+0000","last_unstale":"2024-03-22T04:13:37.537011+0000","last_undegraded":"2024-03-22T04:13:37.537011+0000","last_fullsized":"2024-03-22T04:13:37.537011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:13:42.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532179+0000","last_change":"2024-03-22T04:13:35.530361+0000","last_active":"2024-03-22T04:13:37.532179+0000","last_peered":"2024-03-22T04:13:37.532179+0000","last_clean":"2024-03-22T04:13:37.532179+0000","last_became_active":"2024-03-22T04:13:35.529768+0000","last_became_peered":"2024-03-22T04:13:35.529768+0000","last_unstale":"2024-03-22T04:13:37.532179+0000","last_undegraded":"2024-03-22T04:13:37.532179+0000","last_fullsized":"2024-03-22T04:13:37.532179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:39:50.979105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534117+0000","last_change":"2024-03-22T04:13:36.523859+0000","last_active":"2024-03-22T04:13:37.534117+0000","last_peered":"2024-03-22T04:13:37.534117+0000","last_clean":"2024-03-22T04:13:37.534117+0000","last_became_active":"2024-03-22T04:13:36.523598+0000","last_became_peered":"2024-03-22T04:13:36.523598+0000","last_unstale":"2024-03-22T04:13:37.534117+0000","last_undegraded":"2024-03-22T04:13:37.534117+0000","last_fullsized":"2024-03-22T04:13:37.534117+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:04.300928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537577+0000","last_change":"2024-03-22T04:13:35.534630+0000","last_active":"2024-03-22T04:13:37.537577+0000","last_peered":"2024-03-22T04:13:37.537577+0000","last_clean":"2024-03-22T04:13:37.537577+0000","last_became_active":"2024-03-22T04:13:35.533284+0000","last_became_peered":"2024-03-22T04:13:35.533284+0000","last_unstale":"2024-03-22T04:13:37.537577+0000","last_undegraded":"2024-03-22T04:13:37.537577+0000","last_fullsized":"2024-03-22T04:13:37.537577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:21:41.394270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534628+0000","last_change":"2024-03-22T04:13:36.539056+0000","last_active":"2024-03-22T04:13:37.534628+0000","last_peered":"2024-03-22T04:13:37.534628+0000","last_clean":"2024-03-22T04:13:37.534628+0000","last_became_active":"2024-03-22T04:13:36.538782+0000","last_became_peered":"2024-03-22T04:13:36.538782+0000","last_unstale":"2024-03-22T04:13:37.534628+0000","last_undegraded":"2024-03-22T04:13:37.534628+0000","last_fullsized":"2024-03-22T04:13:37.534628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:07.459840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.537503+0000","last_change":"2024-03-22T04:13:35.531937+0000","last_active":"2024-03-22T04:13:37.537503+0000","last_peered":"2024-03-22T04:13:37.537503+0000","last_clean":"2024-03-22T04:13:37.537503+0000","last_became_active":"2024-03-22T04:13:35.528718+0000","last_became_peered":"2024-03-22T04:13:35.528718+0000","last_unstale":"2024-03-22T04:13:37.537503+0000","last_undegraded":"2024-03-22T04:13:37.537503+0000","last_fullsized":"2024-03-22T04:13:37.537503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:15:11.462730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934662+0000","last_change":"2024-03-22T04:13:36.545245+0000","last_active":"2024-03-22T04:13:37.934662+0000","last_peered":"2024-03-22T04:13:37.934662+0000","last_clean":"2024-03-22T04:13:37.934662+0000","last_became_active":"2024-03-22T04:13:36.544198+0000","last_became_peered":"2024-03-22T04:13:36.544198+0000","last_unstale":"2024-03-22T04:13:37.934662+0000","last_undegraded":"2024-03-22T04:13:37.934662+0000","last_fullsized":"2024-03-22T04:13:37.934662+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:54:58.046272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532826+0000","last_change":"2024-03-22T04:13:35.530829+0000","last_active":"2024-03-22T04:13:37.532826+0000","last_peered":"2024-03-22T04:13:37.532826+0000","last_clean":"2024-03-22T04:13:37.532826+0000","last_became_active":"2024-03-22T04:13:35.530554+0000","last_became_peered":"2024-03-22T04:13:35.530554+0000","last_unstale":"2024-03-22T04:13:37.532826+0000","last_undegraded":"2024-03-22T04:13:37.532826+0000","last_fullsized":"2024-03-22T04:13:37.532826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:15:11.462730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934824+0000","last_change":"2024-03-22T04:13:36.540921+0000","last_active":"2024-03-22T04:13:37.934824+0000","last_peered":"2024-03-22T04:13:37.934824+0000","last_clean":"2024-03-22T04:13:37.934824+0000","last_became_active":"2024-03-22T04:13:36.540455+0000","last_became_peered":"2024-03-22T04:13:36.540455+0000","last_unstale":"2024-03-22T04:13:37.934824+0000","last_undegraded":"2024-03-22T04:13:37.934824+0000","last_fullsized":"2024-03-22T04:13:37.934824+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:53:16.420721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.541159+0000","last_change":"2024-03-22T04:13:35.538794+0000","last_active":"2024-03-22T04:13:37.541159+0000","last_peered":"2024-03-22T04:13:37.541159+0000","last_clean":"2024-03-22T04:13:37.541159+0000","last_became_active":"2024-03-22T04:13:35.538511+0000","last_became_peered":"2024-03-22T04:13:35.538511+0000","last_unstale":"2024-03-22T04:13:37.541159+0000","last_undegraded":"2024-03-22T04:13:37.541159+0000","last_fullsized":"2024-03-22T04:13:37.541159+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:14.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.536799+0000","last_change":"2024-03-22T04:13:36.528390+0000","last_active":"2024-03-22T04:13:37.536799+0000","last_peered":"2024-03-22T04:13:37.536799+0000","last_clean":"2024-03-22T04:13:37.536799+0000","last_became_active":"2024-03-22T04:13:36.528094+0000","last_became_peered":"2024-03-22T04:13:36.528094+0000","last_unstale":"2024-03-22T04:13:37.536799+0000","last_undegraded":"2024-03-22T04:13:37.536799+0000","last_fullsized":"2024-03-22T04:13:37.536799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:06:14.434151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.935241+0000","last_change":"2024-03-22T04:13:35.528468+0000","last_active":"2024-03-22T04:13:37.935241+0000","last_peered":"2024-03-22T04:13:37.935241+0000","last_clean":"2024-03-22T04:13:37.935241+0000","last_became_active":"2024-03-22T04:13:35.528113+0000","last_became_peered":"2024-03-22T04:13:35.528113+0000","last_unstale":"2024-03-22T04:13:37.935241+0000","last_undegraded":"2024-03-22T04:13:37.935241+0000","last_fullsized":"2024-03-22T04:13:37.935241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:23.221070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934996+0000","last_change":"2024-03-22T04:13:36.531435+0000","last_active":"2024-03-22T04:13:37.934996+0000","last_peered":"2024-03-22T04:13:37.934996+0000","last_clean":"2024-03-22T04:13:37.934996+0000","last_became_active":"2024-03-22T04:13:36.531103+0000","last_became_peered":"2024-03-22T04:13:36.531103+0000","last_unstale":"2024-03-22T04:13:37.934996+0000","last_undegraded":"2024-03-22T04:13:37.934996+0000","last_fullsized":"2024-03-22T04:13:37.934996+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:51:43.486787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.934796+0000","last_change":"2024-03-22T04:13:35.525890+0000","last_active":"2024-03-22T04:13:37.934796+0000","last_peered":"2024-03-22T04:13:37.934796+0000","last_clean":"2024-03-22T04:13:37.934796+0000","last_became_active":"2024-03-22T04:13:35.525618+0000","last_became_peered":"2024-03-22T04:13:35.525618+0000","last_unstale":"2024-03-22T04:13:37.934796+0000","last_undegraded":"2024-03-22T04:13:37.934796+0000","last_fullsized":"2024-03-22T04:13:37.934796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:56.946600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532657+0000","last_change":"2024-03-22T04:13:36.527937+0000","last_active":"2024-03-22T04:13:37.532657+0000","last_peered":"2024-03-22T04:13:37.532657+0000","last_clean":"2024-03-22T04:13:37.532657+0000","last_became_active":"2024-03-22T04:13:36.527142+0000","last_became_peered":"2024-03-22T04:13:36.527142+0000","last_unstale":"2024-03-22T04:13:37.532657+0000","last_undegraded":"2024-03-22T04:13:37.532657+0000","last_fullsized":"2024-03-22T04:13:37.532657+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:48:10.956970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.540994+0000","last_change":"2024-03-22T04:13:35.537483+0000","last_active":"2024-03-22T04:13:37.540994+0000","last_peered":"2024-03-22T04:13:37.540994+0000","last_clean":"2024-03-22T04:13:37.540994+0000","last_became_active":"2024-03-22T04:13:35.537070+0000","last_became_peered":"2024-03-22T04:13:35.537070+0000","last_unstale":"2024-03-22T04:13:37.540994+0000","last_undegraded":"2024-03-22T04:13:37.540994+0000","last_fullsized":"2024-03-22T04:13:37.540994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:25:37.720655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.534302+0000","last_change":"2024-03-22T04:13:36.535142+0000","last_active":"2024-03-22T04:13:37.534302+0000","last_peered":"2024-03-22T04:13:37.534302+0000","last_clean":"2024-03-22T04:13:37.534302+0000","last_became_active":"2024-03-22T04:13:36.533918+0000","last_became_peered":"2024-03-22T04:13:36.533918+0000","last_unstale":"2024-03-22T04:13:37.534302+0000","last_undegraded":"2024-03-22T04:13:37.534302+0000","last_fullsized":"2024-03-22T04:13:37.534302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:15.073523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532432+0000","last_change":"2024-03-22T04:13:35.533317+0000","last_active":"2024-03-22T04:13:37.532432+0000","last_peered":"2024-03-22T04:13:37.532432+0000","last_clean":"2024-03-22T04:13:37.532432+0000","last_became_active":"2024-03-22T04:13:35.532424+0000","last_became_peered":"2024-03-22T04:13:35.532424+0000","last_unstale":"2024-03-22T04:13:37.532432+0000","last_undegraded":"2024-03-22T04:13:37.532432+0000","last_fullsized":"2024-03-22T04:13:37.532432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:51:42.480319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533309+0000","last_change":"2024-03-22T04:13:36.533360+0000","last_active":"2024-03-22T04:13:37.533309+0000","last_peered":"2024-03-22T04:13:37.533309+0000","last_clean":"2024-03-22T04:13:37.533309+0000","last_became_active":"2024-03-22T04:13:36.527843+0000","last_became_peered":"2024-03-22T04:13:36.527843+0000","last_unstale":"2024-03-22T04:13:37.533309+0000","last_undegraded":"2024-03-22T04:13:37.533309+0000","last_fullsized":"2024-03-22T04:13:37.533309+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:35.502450+0000","last_clean_scrub_stamp":"2024-03-22T04:13:35.502450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:44.092644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.532539+0000","last_change":"2024-03-22T04:13:35.528988+0000","last_active":"2024-03-22T04:13:37.532539+0000","last_peered":"2024-03-22T04:13:37.532539+0000","last_clean":"2024-03-22T04:13:37.532539+0000","last_became_active":"2024-03-22T04:13:35.528529+0000","last_became_peered":"2024-03-22T04:13:35.528529+0000","last_unstale":"2024-03-22T04:13:37.532539+0000","last_undegraded":"2024-03-22T04:13:37.532539+0000","last_fullsized":"2024-03-22T04:13:37.532539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:13:34.495982+0000","last_clean_scrub_stamp":"2024-03-22T04:13:34.495982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:43.086175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":251,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-03-22T04:13:37.533053+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:13:37.533053+0000","last_peered":"2024-03-22T04:13:37.533053+0000","last_clean":"2024-03-22T04:13:37.533053+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:13:37.533053+0000","last_undegraded":"2024-03-22T04:13:37.533053+0000","last_fullsized":"2024-03-22T04:13:37.533053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":360751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":363015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28404,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343436,"statfs":{"total":96636764160,"available":96607678464,"internally_reserved":0,"allocated":1417216,"data_stored":952843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":360911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":360807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":951153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":360751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":361333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:13:45.500 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T04:13:45.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-22T04:13:45.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.676+0000 7ffb77200640 1 Processor -- start 2024-03-22T04:13:45.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.676+0000 7ffb77200640 1 -- start start 2024-03-22T04:13:45.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb77200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7013f330 0x7ffb700cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb77200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb700cf740 0x7ffb700d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb77200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb7015ff50 con 0x7ffb7015c920 2024-03-22T04:13:45.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb77200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffb700cfb60 con 0x7ffb700cf740 2024-03-22T04:13:45.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb77200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ffb70160230 con 0x7ffb7013f330 2024-03-22T04:13:45.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb75200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb700cf740 0x7ffb700d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7013f330 0x7ffb700cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6f400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ffb7015c920 0x7ffb700cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:44906/0 (socket says 172.21.15.86:44906) 2024-03-22T04:13:45.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6f400640 1 -- 172.21.15.86:0/2965672986 learned_addr learned my addr 172.21.15.86:0/2965672986 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:45.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6fe00640 1 -- 172.21.15.86:0/2965672986 >> v2:172.21.15.86:3300/0 conn(0x7ffb700cf740 msgr2=0x7ffb700d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:45.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6fe00640 1 --2- 172.21.15.86:0/2965672986 >> v2:172.21.15.86:3300/0 conn(0x7ffb700cf740 0x7ffb700d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6fe00640 1 -- 172.21.15.86:0/2965672986 >> v1:172.21.15.196:6789/0 conn(0x7ffb7015c920 legacy=0x7ffb700cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6fe00640 1 -- 172.21.15.86:0/2965672986 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb70160510 con 0x7ffb7013f330 2024-03-22T04:13:45.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6ea00640 1 -- 172.21.15.86:0/2965672986 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190658230 0 0) 0x7ffb7015ff50 con 0x7ffb7015c920 2024-03-22T04:13:45.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.680+0000 7ffb6fe00640 1 --2- 172.21.15.86:0/2965672986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7013f330 0x7ffb700cfda0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb600097c0 tx=0x7ffb6001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e685ee2b2bc880 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:45.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb6ea00640 1 -- 172.21.15.86:0/2965672986 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb600b0070 con 0x7ffb7013f330 2024-03-22T04:13:45.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb6ea00640 1 -- 172.21.15.86:0/2965672986 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb600ab600 con 0x7ffb7013f330 2024-03-22T04:13:45.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb6ea00640 1 -- 172.21.15.86:0/2965672986 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb600ab9d0 con 0x7ffb7013f330 2024-03-22T04:13:45.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- 172.21.15.86:0/2965672986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7013f330 msgr2=0x7ffb700cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 --2- 172.21.15.86:0/2965672986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7013f330 0x7ffb700cfda0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb600097c0 tx=0x7ffb6001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- 172.21.15.86:0/2965672986 shutdown_connections 2024-03-22T04:13:45.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 --2- 172.21.15.86:0/2965672986 >> v2:172.21.15.86:3300/0 conn(0x7ffb700cf740 0x7ffb700d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 --2- 172.21.15.86:0/2965672986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7013f330 0x7ffb700cfda0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- 172.21.15.86:0/2965672986 >> 172.21.15.86:0/2965672986 conn(0x7ffb7006ef50 msgr2=0x7ffb7006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:45.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- 172.21.15.86:0/2965672986 shutdown_connections 2024-03-22T04:13:45.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- 172.21.15.86:0/2965672986 wait complete. 2024-03-22T04:13:45.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 Processor -- start 2024-03-22T04:13:45.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- start start 2024-03-22T04:13:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb700cf740 0x7ffb7013a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7013f330 0x7ffb70155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015c920 0x7ffb70156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb70142460 con 0x7ffb7013f330 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffb701422e0 con 0x7ffb700cf740 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb77200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ffb701425e0 con 0x7ffb7015c920 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb6f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7013f330 0x7ffb70155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb6fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb700cf740 0x7ffb7013a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb6f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7013f330 0x7ffb70155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45956/0 (socket says 172.21.15.86:45956) 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb6f400640 1 -- 172.21.15.86:0/3436887856 learned_addr learned my addr 172.21.15.86:0/3436887856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.684+0000 7ffb75200640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015c920 0x7ffb70156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb6f400640 1 -- 172.21.15.86:0/3436887856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015c920 msgr2=0x7ffb70156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb6f400640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015c920 0x7ffb70156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb6f400640 1 -- 172.21.15.86:0/3436887856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb700cf740 msgr2=0x7ffb7013a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb6f400640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb700cf740 0x7ffb7013a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb6f400640 1 -- 172.21.15.86:0/3436887856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb70155b40 con 0x7ffb7013f330 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb75200640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015c920 0x7ffb70156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:45.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb6f400640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7013f330 0x7ffb70155ed0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00c4b0 tx=0x7ffb5c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:45.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb67e00640 1 -- 172.21.15.86:0/3436887856 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c018070 con 0x7ffb7013f330 2024-03-22T04:13:45.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb67e00640 1 -- 172.21.15.86:0/3436887856 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c007500 con 0x7ffb7013f330 2024-03-22T04:13:45.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb67e00640 1 -- 172.21.15.86:0/3436887856 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c004a30 con 0x7ffb7013f330 2024-03-22T04:13:45.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb700b08a0 con 0x7ffb7013f330 2024-03-22T04:13:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb700b0e60 con 0x7ffb7013f330 2024-03-22T04:13:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb70074470 con 0x7ffb7013f330 2024-03-22T04:13:45.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.688+0000 7ffb67e00640 1 -- 172.21.15.86:0/3436887856 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ffb5c004bd0 con 0x7ffb7013f330 2024-03-22T04:13:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.692+0000 7ffb67e00640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffb50067b20 0x7ffb50069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:45.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.692+0000 7ffb67e00640 1 -- 172.21.15.86:0/3436887856 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ffb5c085110 con 0x7ffb7013f330 2024-03-22T04:13:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.692+0000 7ffb6fe00640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffb50067b20 0x7ffb50069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:45.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.692+0000 7ffb6fe00640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffb50067b20 0x7ffb50069fe0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffb600ad000 tx=0x7ffb600a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.696+0000 7ffb67e00640 1 -- 172.21.15.86:0/3436887856 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ffb5c0553d0 con 0x7ffb7013f330 2024-03-22T04:13:45.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.908+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ffb701670c0 con 0x7ffb7013f330 2024-03-22T04:13:45.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.908+0000 7ffb67e00640 1 -- 172.21.15.86:0/3436887856 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7ffb5c059470 con 0x7ffb7013f330 2024-03-22T04:13:45.914 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T04:13:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:13:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-03-22T04:13:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-03-22T04:13:45.914 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-22T04:13:45.914 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffb50067b20 msgr2=0x7ffb50069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffb50067b20 0x7ffb50069fe0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffb600ad000 tx=0x7ffb600a6000 comp rx=0 tx=0).stop 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7013f330 msgr2=0x7ffb70155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7013f330 0x7ffb70155ed0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00c4b0 tx=0x7ffb5c00c970 comp rx=0 tx=0).stop 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 shutdown_connections 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffb50067b20 0x7ffb50069fe0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015c920 0x7ffb70156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7013f330 0x7ffb70155ed0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 --2- 172.21.15.86:0/3436887856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb700cf740 0x7ffb7013a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:45.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 >> 172.21.15.86:0/3436887856 conn(0x7ffb7006ef50 msgr2=0x7ffb700d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:45.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 shutdown_connections 2024-03-22T04:13:45.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:45.912+0000 7ffb77200640 1 -- 172.21.15.86:0/3436887856 wait complete. 2024-03-22T04:13:45.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.068+0000 7f4a9f600640 1 Processor -- start 2024-03-22T04:13:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.068+0000 7f4a9f600640 1 -- start start 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4a980ad330 0x7f4a9815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9f600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 0x7f4a9806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9f600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4a980ad710 0x7f4a9813fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9806dc40 con 0x7f4a980ad330 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9f600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9815c7c0 con 0x7f4a980ad710 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9f600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9806dea0 con 0x7f4a9806ef40 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4a980ad330 0x7f4a9815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4a980ad330 0x7f4a9815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51282/0 (socket says 172.21.15.86:51282) 2024-03-22T04:13:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9cc00640 1 -- 172.21.15.86:0/2901048908 learned_addr learned my addr 172.21.15.86:0/2901048908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:46.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9d600640 1 --2- 172.21.15.86:0/2901048908 >> v2:172.21.15.196:3300/0 conn(0x7f4a980ad710 0x7f4a9813fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a97e00640 1 --2- 172.21.15.86:0/2901048908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 0x7f4a9806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9cc00640 1 -- 172.21.15.86:0/2901048908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 msgr2=0x7f4a9806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9cc00640 1 --2- 172.21.15.86:0/2901048908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 0x7f4a9806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9cc00640 1 -- 172.21.15.86:0/2901048908 >> v2:172.21.15.196:3300/0 conn(0x7f4a980ad710 msgr2=0x7f4a9813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9cc00640 1 --2- 172.21.15.86:0/2901048908 >> v2:172.21.15.196:3300/0 conn(0x7f4a980ad710 0x7f4a9813fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.072+0000 7f4a9cc00640 1 -- 172.21.15.86:0/2901048908 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9806e180 con 0x7f4a980ad330 2024-03-22T04:13:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9cc00640 1 --2- 172.21.15.86:0/2901048908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815d4a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806c5c0 tx=0x7f4a8c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac2035743cb7d5d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a97400640 1 -- 172.21.15.86:0/2901048908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0aa610 con 0x7f4a980ad330 2024-03-22T04:13:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a97400640 1 -- 172.21.15.86:0/2901048908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0af070 con 0x7f4a980ad330 2024-03-22T04:13:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a97400640 1 -- 172.21.15.86:0/2901048908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0aa9d0 con 0x7f4a980ad330 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- 172.21.15.86:0/2901048908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 msgr2=0x7f4a9815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2901048908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815d4a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806c5c0 tx=0x7f4a8c01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- 172.21.15.86:0/2901048908 shutdown_connections 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2901048908 >> v2:172.21.15.196:3300/0 conn(0x7f4a980ad710 0x7f4a9813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2901048908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 0x7f4a9806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2901048908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815d4a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- 172.21.15.86:0/2901048908 >> 172.21.15.86:0/2901048908 conn(0x7f4a980b1df0 msgr2=0x7f4a980b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- 172.21.15.86:0/2901048908 shutdown_connections 2024-03-22T04:13:46.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- 172.21.15.86:0/2901048908 wait complete. 2024-03-22T04:13:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 Processor -- start 2024-03-22T04:13:46.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- start start 2024-03-22T04:13:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 0x7f4a9814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980ad710 0x7f4a9815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98141800 con 0x7f4a980ad330 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98141680 con 0x7f4a980ad710 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.076+0000 7f4a9f600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4a98141980 con 0x7f4a9806ef40 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a9cc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 0x7f4a9814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51294/0 (socket says 172.21.15.86:51294) 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 -- 172.21.15.86:0/2693835275 learned_addr learned my addr 172.21.15.86:0/2693835275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a9d600640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980ad710 0x7f4a9815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 -- 172.21.15.86:0/2693835275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 msgr2=0x7f4a9814ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 0x7f4a9814ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 -- 172.21.15.86:0/2693835275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980ad710 msgr2=0x7f4a9815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980ad710 0x7f4a9815d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 -- 172.21.15.86:0/2693835275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a98159e40 con 0x7f4a980ad330 2024-03-22T04:13:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a97e00640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815b450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400e8b0 tx=0x7f4a8400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a94c00640 1 -- 172.21.15.86:0/2693835275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84017070 con 0x7f4a980ad330 2024-03-22T04:13:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a94c00640 1 -- 172.21.15.86:0/2693835275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8400c940 con 0x7f4a980ad330 2024-03-22T04:13:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a98158760 con 0x7f4a980ad330 2024-03-22T04:13:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a94c00640 1 -- 172.21.15.86:0/2693835275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8400cd10 con 0x7f4a980ad330 2024-03-22T04:13:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9815bec0 con 0x7f4a980ad330 2024-03-22T04:13:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a98068650 con 0x7f4a980ad330 2024-03-22T04:13:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a94c00640 1 -- 172.21.15.86:0/2693835275 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4a8401a070 con 0x7f4a980ad330 2024-03-22T04:13:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a94c00640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a78067b20 0x7f4a78069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a94c00640 1 -- 172.21.15.86:0/2693835275 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f4a84086330 con 0x7f4a980ad330 2024-03-22T04:13:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.080+0000 7f4a9cc00640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a78067b20 0x7f4a78069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.084+0000 7f4a9cc00640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a78067b20 0x7f4a78069fe0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0ae000 tx=0x7f4a8c0b7040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.084+0000 7f4a94c00640 1 -- 172.21.15.86:0/2693835275 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4a84056560 con 0x7f4a980ad330 2024-03-22T04:13:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.276+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a980b37b0 con 0x7f4a980ad330 2024-03-22T04:13:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a94c00640 1 -- 172.21.15.86:0/2693835275 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7f4a8405a600 con 0x7f4a980ad330 2024-03-22T04:13:46.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:46.283 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:13:38.919534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:13:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 35 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a78067b20 msgr2=0x7f4a78069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a78067b20 0x7f4a78069fe0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0ae000 tx=0x7f4a8c0b7040 comp rx=0 tx=0).stop 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 msgr2=0x7f4a9815b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815b450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400e8b0 tx=0x7f4a8400ed70 comp rx=0 tx=0).stop 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 shutdown_connections 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a78067b20 0x7f4a78069fe0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980ad710 0x7f4a9815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a980ad330 0x7f4a9815b450 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 --2- 172.21.15.86:0/2693835275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9806ef40 0x7f4a9814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 >> 172.21.15.86:0/2693835275 conn(0x7f4a980b1df0 msgr2=0x7f4a98158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 shutdown_connections 2024-03-22T04:13:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.280+0000 7f4a9f600640 1 -- 172.21.15.86:0/2693835275 wait complete. 2024-03-22T04:13:46.706 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:13:35.899981+0000', 'modified': '2024-03-22T04:13:38.919534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 75617}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_75617': {'gid': 75617, 'name': 'd', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/3774275697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3774275697}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 3774275697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 75617, 'qdb_cluster': [75617]} 2024-03-22T04:13:46.706 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:13:46.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-22T04:13:46.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9abc00640 1 Processor -- start 2024-03-22T04:13:46.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9abc00640 1 -- start start 2024-03-22T04:13:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9abc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9a40d0de0 0x7fb9a40ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9abc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a415fed0 0x7fb9a4160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9abc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a40d11c0 con 0x7fb9a40d19a0 2024-03-22T04:13:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9abc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a40d0580 con 0x7fb9a40d0de0 2024-03-22T04:13:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9abc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a4160860 con 0x7fb9a415fed0 2024-03-22T04:13:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9a40d0de0 0x7fb9a40ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9a40d0de0 0x7fb9a40ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51296/0 (socket says 172.21.15.86:51296) 2024-03-22T04:13:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a9200640 1 -- 172.21.15.86:0/1446818294 learned_addr learned my addr 172.21.15.86:0/1446818294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a3e00640 1 --1- 172.21.15.86:0/1446818294 >> v1:172.21.15.196:6789/0 conn(0x7fb9a40d19a0 0x7fb9a40cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:47070/0 (socket says 172.21.15.86:47070) 2024-03-22T04:13:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/1446818294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a415fed0 0x7fb9a4160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a9200640 1 -- 172.21.15.86:0/1446818294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a415fed0 msgr2=0x7fb9a4160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a9200640 1 --2- 172.21.15.86:0/1446818294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a415fed0 0x7fb9a4160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a9200640 1 -- 172.21.15.86:0/1446818294 >> v1:172.21.15.196:6789/0 conn(0x7fb9a40d19a0 legacy=0x7fb9a40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a9200640 1 -- 172.21.15.86:0/1446818294 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a40cece0 con 0x7fb9a40d0de0 2024-03-22T04:13:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.840+0000 7fb9a3400640 1 -- 172.21.15.86:0/1446818294 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4175233832 0 0) 0x7fb9a40d11c0 con 0x7fb9a40d19a0 2024-03-22T04:13:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9a9200640 1 --2- 172.21.15.86:0/1446818294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d0de0 0x7fb9a40ce6b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb994009e50 tx=0x7fb99401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd91ab4e56d1802 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9a3400640 1 -- 172.21.15.86:0/1446818294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9940a57f0 con 0x7fb9a40d0de0 2024-03-22T04:13:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9a3400640 1 -- 172.21.15.86:0/1446818294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9940a5990 con 0x7fb9a40d0de0 2024-03-22T04:13:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9a3400640 1 -- 172.21.15.86:0/1446818294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9940a5d60 con 0x7fb9a40d0de0 2024-03-22T04:13:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 -- 172.21.15.86:0/1446818294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d0de0 msgr2=0x7fb9a40ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1446818294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d0de0 0x7fb9a40ce6b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb994009e50 tx=0x7fb99401ca30 comp rx=0 tx=0).stop 2024-03-22T04:13:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 -- 172.21.15.86:0/1446818294 shutdown_connections 2024-03-22T04:13:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1446818294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a415fed0 0x7fb9a4160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1446818294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d0de0 0x7fb9a40ce6b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 -- 172.21.15.86:0/1446818294 >> 172.21.15.86:0/1446818294 conn(0x7fb9a406f910 msgr2=0x7fb9a406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 -- 172.21.15.86:0/1446818294 shutdown_connections 2024-03-22T04:13:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 -- 172.21.15.86:0/1446818294 wait complete. 2024-03-22T04:13:46.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.844+0000 7fb9abc00640 1 Processor -- start 2024-03-22T04:13:46.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 -- start start 2024-03-22T04:13:46.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a40d0de0 0x7fb9a4150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d19a0 0x7fb9a41511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9a415fed0 0x7fb9a41516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a40d1db0 con 0x7fb9a40d19a0 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4071100 con 0x7fb9a415fed0 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a40d1f30 con 0x7fb9a40d0de0 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d19a0 0x7fb9a41511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d19a0 0x7fb9a41511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51300/0 (socket says 172.21.15.86:51300) 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 -- 172.21.15.86:0/1469582475 learned_addr learned my addr 172.21.15.86:0/1469582475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a9200640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a40d0de0 0x7fb9a4150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9a415fed0 0x7fb9a41516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 -- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a40d0de0 msgr2=0x7fb9a4150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a40d0de0 0x7fb9a4150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 -- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9a415fed0 msgr2=0x7fb9a41516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9a415fed0 0x7fb9a41516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 -- 172.21.15.86:0/1469582475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a406e770 con 0x7fb9a40d19a0 2024-03-22T04:13:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a9200640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a40d0de0 0x7fb9a4150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:46.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d19a0 0x7fb9a41511b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb9900011f0 tx=0x7fb99000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a0c00640 1 -- 172.21.15.86:0/1469582475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990018070 con 0x7fb9a40d19a0 2024-03-22T04:13:46.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a0c00640 1 -- 172.21.15.86:0/1469582475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9900074e0 con 0x7fb9a40d19a0 2024-03-22T04:13:46.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9a0c00640 1 -- 172.21.15.86:0/1469582475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990004a10 con 0x7fb9a40d19a0 2024-03-22T04:13:46.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 -- 172.21.15.86:0/1469582475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a406e490 con 0x7fb9a40d19a0 2024-03-22T04:13:46.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.848+0000 7fb9abc00640 1 -- 172.21.15.86:0/1469582475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a4159350 con 0x7fb9a40d19a0 2024-03-22T04:13:46.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a0c00640 1 -- 172.21.15.86:0/1469582475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb990004bb0 con 0x7fb9a40d19a0 2024-03-22T04:13:46.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a0c00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb984067b20 0x7fb984069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a0c00640 1 -- 172.21.15.86:0/1469582475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fb9900853f0 con 0x7fb9a40d19a0 2024-03-22T04:13:46.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9200640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb984067b20 0x7fb984069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9af087000 1 Processor -- start 2024-03-22T04:13:46.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9af087000 1 -- start start 2024-03-22T04:13:46.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9200640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb984067b20 0x7fb984069fe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb9940ae000 tx=0x7fb9940a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9af087000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 0x556e284eb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9af087000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eb790 0x556e2849e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9af087000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e2849f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9af087000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e2829c140 con 0x556e2849ec20 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9af087000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e282efb10 con 0x556e284eb790 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9af087000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556e28276c50 con 0x556e284eae70 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e2849f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e2849f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51310/0 (socket says 172.21.15.86:51310) 2024-03-22T04:13:46.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9c00640 1 -- 172.21.15.86:0/2166904404 learned_addr learned my addr 172.21.15.86:0/2166904404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:46.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eb790 0x556e2849e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9200640 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 0x556e284eb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9c00640 1 -- 172.21.15.86:0/2166904404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 msgr2=0x556e284eb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 0x556e284eb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9c00640 1 -- 172.21.15.86:0/2166904404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eb790 msgr2=0x556e2849e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eb790 0x556e2849e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.852+0000 7fb9a9c00640 1 -- 172.21.15.86:0/2166904404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e28334f20 con 0x556e2849ec20 2024-03-22T04:13:46.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e2849f000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009c20 tx=0x7fb99c011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0e3ebc843cb165 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb999600640 1 -- 172.21.15.86:0/2166904404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c014440 con 0x556e2849ec20 2024-03-22T04:13:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb999600640 1 -- 172.21.15.86:0/2166904404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c0145e0 con 0x556e2849ec20 2024-03-22T04:13:46.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb999600640 1 -- 172.21.15.86:0/2166904404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c014990 con 0x556e2849ec20 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- 172.21.15.86:0/2166904404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 msgr2=0x556e2849f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e2849f000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009c20 tx=0x7fb99c011690 comp rx=0 tx=0).stop 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- 172.21.15.86:0/2166904404 shutdown_connections 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eb790 0x556e2849e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e2849f000 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 --2- 172.21.15.86:0/2166904404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 0x556e284eb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- 172.21.15.86:0/2166904404 >> 172.21.15.86:0/2166904404 conn(0x556e284cbc10 msgr2=0x556e284ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- 172.21.15.86:0/2166904404 shutdown_connections 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- 172.21.15.86:0/2166904404 wait complete. 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- start start 2024-03-22T04:13:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 Processor -- start 2024-03-22T04:13:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e2849ec20 0x556e284a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284eb790 0x556e284a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e282a9ab0 con 0x556e284eb790 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e282ebac0 con 0x556e284eae70 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556e2832fa60 con 0x556e2849ec20 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9af087000 10 client.0 _ll_register_callbacks cb 0x556e2848c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284eb790 0x556e284a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284eb790 0x556e284a3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51326/0 (socket says 172.21.15.86:51326) 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e2849ec20 0x556e284a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 -- 172.21.15.86:0/2326670721 learned_addr learned my addr 172.21.15.86:0/2326670721 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 -- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e2849ec20 msgr2=0x556e284a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e2849ec20 0x556e284a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 -- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 msgr2=0x556e284a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a9c00640 1 -- 172.21.15.86:0/2326670721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e282c5570 con 0x556e284eb790 2024-03-22T04:13:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.856+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284eb790 0x556e284a3d20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb99c000c10 tx=0x7fb99c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafa6f45f3586d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 1 -- 172.21.15.86:0/2326670721 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c014440 con 0x556e284eb790 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 -- 172.21.15.86:0/2326670721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb99c0145e0 con 0x556e284eb790 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 1 -- 172.21.15.86:0/2326670721 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fb99c014ba0 con 0x556e284eb790 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 1 -- 172.21.15.86:0/2326670721 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb99c030050 con 0x556e284eb790 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 10 client.6375 fetch_fsmap learned FSMap version 35 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 -- 172.21.15.86:0/2326670721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556e282bb880 con 0x556e284eb790 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 1 -- 172.21.15.86:0/2326670721 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c0145e0 con 0x556e284eb790 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 1 -- 172.21.15.86:0/2326670721 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fb99c033020 con 0x556e284eb790 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 10 client.6375 fetch_fsmap finished waiting for FSMap version 35 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 10 client.6375 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:13:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 20 client.6375 populate_metadata read hostname 'smithi086' 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x556e284a4ac0 0x556e284fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 4 client.6375 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 -- 172.21.15.86:0/2326670721 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e2828abc0 con 0x556e284a4ac0 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9a9200640 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x556e284a4ac0 0x556e284fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 10 client.6375 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9a9200640 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x556e284a4ac0 0x556e284fc120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5cf57df87f2347 server_cookie=646d0a0dd3ab7fa8 in_seq=0 out_seq=0 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 1 -- 172.21.15.86:0/2326670721 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556e2828abc0 con 0x556e284a4ac0 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 10 client.6375 handle_command_reply: tid=0 2024-03-22T04:13:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 client.6375 shutdown 2024-03-22T04:13:46.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 -- 172.21.15.86:0/2326670721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284eb790 msgr2=0x556e284a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284eb790 0x556e284a3d20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb99c000c10 tx=0x7fb99c004740 comp rx=0 tx=0).stop 2024-03-22T04:13:46.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 -- 172.21.15.86:0/2326670721 shutdown_connections 2024-03-22T04:13:46.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x556e284a4ac0 0x556e284fc120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284eb790 0x556e284a3d20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb98e000640 0 client.6375 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 --2- 172.21.15.86:0/2326670721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e2849ec20 0x556e284a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 -- 172.21.15.86:0/2326670721 >> 172.21.15.86:0/2326670721 conn(0x556e284cbc10 msgr2=0x556e284ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 -- 172.21.15.86:0/2326670721 shutdown_connections 2024-03-22T04:13:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 1 -- 172.21.15.86:0/2326670721 wait complete. 2024-03-22T04:13:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.860+0000 7fb9af087000 20 client.6375 trim_cache size 0 max 16384 2024-03-22T04:13:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 Processor -- start 2024-03-22T04:13:46.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- start start 2024-03-22T04:13:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284cbc10 0x556e284ea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e2849ec20 0x556e284ea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 0x556e284a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4203c70 con 0x556e284cbc10 2024-03-22T04:13:46.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4202d10 con 0x556e2849ec20 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a41549d0 con 0x556e284eae70 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284cbc10 0x556e284ea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284cbc10 0x556e284ea350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51338/0 (socket says 172.21.15.86:51338) 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 -- 172.21.15.86:0/3207734994 learned_addr learned my addr 172.21.15.86:0/3207734994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e2849ec20 0x556e284ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 0x556e284a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 -- 172.21.15.86:0/3207734994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 msgr2=0x556e284a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 0x556e284a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 -- 172.21.15.86:0/3207734994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e2849ec20 msgr2=0x556e284ea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e2849ec20 0x556e284ea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 -- 172.21.15.86:0/3207734994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e28277e80 con 0x556e284cbc10 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9a9200640 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284cbc10 0x556e284ea350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb9940af040 tx=0x7fb994004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cec283a94eddff server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb998c00640 1 -- 172.21.15.86:0/3207734994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9940ba040 con 0x556e284cbc10 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb998c00640 1 -- 172.21.15.86:0/3207734994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9940c4a10 con 0x556e284cbc10 2024-03-22T04:13:46.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb998c00640 1 -- 172.21.15.86:0/3207734994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9940c4de0 con 0x556e284cbc10 2024-03-22T04:13:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- 172.21.15.86:0/3207734994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284cbc10 msgr2=0x556e284ea350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284cbc10 0x556e284ea350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb9940af040 tx=0x7fb994004ac0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- 172.21.15.86:0/3207734994 shutdown_connections 2024-03-22T04:13:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e2849ec20 0x556e284ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284eae70 0x556e284a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 --2- 172.21.15.86:0/3207734994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e284cbc10 0x556e284ea350 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- 172.21.15.86:0/3207734994 >> 172.21.15.86:0/3207734994 conn(0x556e284a5f40 msgr2=0x556e2849dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- 172.21.15.86:0/3207734994 shutdown_connections 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- 172.21.15.86:0/3207734994 wait complete. 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 -- start start 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.880+0000 7fb9af087000 1 Processor -- start 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e285632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284cbc10 0x556e28563830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e282eee20 con 0x556e2849ec20 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4202710 con 0x556e284eae70 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556e282f6720 con 0x556e284cbc10 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 10 client.0 _ll_register_callbacks cb 0x7fb99c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e285632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e285632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51340/0 (socket says 172.21.15.86:51340) 2024-03-22T04:13:46.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 -- 172.21.15.86:0/839316101 learned_addr learned my addr 172.21.15.86:0/839316101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284cbc10 0x556e28563830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9c00640 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 -- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284cbc10 msgr2=0x556e28563830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284cbc10 0x556e28563830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 -- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 msgr2=0x556e284fcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284fcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 -- 172.21.15.86:0/839316101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e28283750 con 0x556e2849ec20 2024-03-22T04:13:46.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a9200640 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e285632f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb994009c90 tx=0x7fb99401b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0035a16784eb73 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 1 -- 172.21.15.86:0/839316101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9940ba040 con 0x556e2849ec20 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 1 -- 172.21.15.86:0/839316101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fb9940c4dc0 con 0x556e2849ec20 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- 172.21.15.86:0/839316101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9940c4a10 con 0x556e2849ec20 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 1 -- 172.21.15.86:0/839316101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9940cf030 con 0x556e2849ec20 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 10 client.6381 fetch_fsmap learned FSMap version 35 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- 172.21.15.86:0/839316101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556e28296450 con 0x556e2849ec20 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 1 -- 172.21.15.86:0/839316101 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9940e0070 con 0x556e2849ec20 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 1 -- 172.21.15.86:0/839316101 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fb994008040 con 0x556e2849ec20 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 10 client.6381 fetch_fsmap finished waiting for FSMap version 35 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 10 client.6381 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 20 client.6381 populate_metadata read hostname 'smithi086' 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x556e284eb790 0x556e284fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 4 client.6381 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:13:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- 172.21.15.86:0/839316101 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "status"}) -- 0x556e282fd050 con 0x556e284eb790 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x556e284eb790 0x556e284fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 10 client.6381 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9a3e00640 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x556e284eb790 0x556e284fd8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d510d3d1334bc6a server_cookie=14217b2ff3b7822d in_seq=0 out_seq=0 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 1 -- 172.21.15.86:0/839316101 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x556e282fd050 con 0x556e284eb790 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 10 client.6381 handle_command_reply: tid=0 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 client.6381 shutdown 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- 172.21.15.86:0/839316101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 msgr2=0x556e285632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e285632f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb994009c90 tx=0x7fb99401b270 comp rx=0 tx=0).stop 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- 172.21.15.86:0/839316101 shutdown_connections 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x556e284eb790 0x556e284fd8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e284eae70 0x556e284fcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb98e000640 0 client.6381 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556e284cbc10 0x556e28563830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 --2- 172.21.15.86:0/839316101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e2849ec20 0x556e285632f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.884+0000 7fb9af087000 1 -- 172.21.15.86:0/839316101 >> 172.21.15.86:0/839316101 conn(0x556e284a5f40 msgr2=0x556e284e8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9af087000 1 -- 172.21.15.86:0/839316101 shutdown_connections 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9af087000 1 -- 172.21.15.86:0/839316101 wait complete. 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9af087000 20 client.6381 trim_cache size 0 max 16384 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 75617, 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:13:46.894 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.365712259, 2024-03-22T04:13:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 35, 2024-03-22T04:13:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 67, 2024-03-22T04:13:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 67, 2024-03-22T04:13:46.895 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.286227422 2024-03-22T04:13:46.895 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 -- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb984067b20 msgr2=0x7fb984069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb984067b20 0x7fb984069fe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb9940ae000 tx=0x7fb9940a7070 comp rx=0 tx=0).stop 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 -- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d19a0 msgr2=0x7fb9a41511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d19a0 0x7fb9a41511b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb9900011f0 tx=0x7fb99000c9b0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 -- 172.21.15.86:0/1469582475 shutdown_connections 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb984067b20 0x7fb984069fe0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9a415fed0 0x7fb9a41516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a40d19a0 0x7fb9a41511b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 --2- 172.21.15.86:0/1469582475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9a40d0de0 0x7fb9a4150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 -- 172.21.15.86:0/1469582475 >> 172.21.15.86:0/1469582475 conn(0x7fb9a406f910 msgr2=0x7fb9a415eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 -- 172.21.15.86:0/1469582475 shutdown_connections 2024-03-22T04:13:46.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:46.888+0000 7fb9abc00640 1 -- 172.21.15.86:0/1469582475 wait complete. 2024-03-22T04:13:46.909 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:13:47.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.040+0000 7fc3a9c00640 1 Processor -- start 2024-03-22T04:13:47.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.040+0000 7fc3a9c00640 1 -- start start 2024-03-22T04:13:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a9c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc3a406ddd0 0x7fc3a415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a9c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a415b680 con 0x7fc3a415f440 2024-03-22T04:13:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a9c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a415b8f0 con 0x7fc3a413e060 2024-03-22T04:13:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a9c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a415c6b0 con 0x7fc3a406ddd0 2024-03-22T04:13:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a9200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc3a415f440 0x7fc3a40ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42434/0 (socket says 172.21.15.86:42434) 2024-03-22T04:13:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a9200640 1 -- 172.21.15.86:0/3371671311 learned_addr learned my addr 172.21.15.86:0/3371671311 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a3400640 1 --2- 172.21.15.86:0/3371671311 >> v2:172.21.15.196:3300/0 conn(0x7fc3a406ddd0 0x7fc3a415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a2a00640 1 -- 172.21.15.86:0/3371671311 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029039592 0 0) 0x7fc3a415b680 con 0x7fc3a415f440 2024-03-22T04:13:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a2a00640 1 -- 172.21.15.86:0/3371671311 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc388003530 con 0x7fc3a415f440 2024-03-22T04:13:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a2a00640 1 -- 172.21.15.86:0/3371671311 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4118917320 0 0) 0x7fc3a415b8f0 con 0x7fc3a413e060 2024-03-22T04:13:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a3400640 1 -- 172.21.15.86:0/3371671311 >> v1:172.21.15.196:6789/0 conn(0x7fc3a413e060 legacy=0x7fc3a406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a3400640 1 -- 172.21.15.86:0/3371671311 >> v1:172.21.15.86:6789/0 conn(0x7fc3a415f440 legacy=0x7fc3a40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.044+0000 7fc3a3400640 1 -- 172.21.15.86:0/3371671311 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a415bbd0 con 0x7fc3a406ddd0 2024-03-22T04:13:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a3400640 1 --2- 172.21.15.86:0/3371671311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a406ddd0 0x7fc3a415ef00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3900097c0 tx=0x7fc39001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5810b35e809e95bf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:47.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a2a00640 1 -- 172.21.15.86:0/3371671311 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900b0070 con 0x7fc3a406ddd0 2024-03-22T04:13:47.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a2a00640 1 -- 172.21.15.86:0/3371671311 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3900ab600 con 0x7fc3a406ddd0 2024-03-22T04:13:47.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a2a00640 1 -- 172.21.15.86:0/3371671311 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900ab9d0 con 0x7fc3a406ddd0 2024-03-22T04:13:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 -- 172.21.15.86:0/3371671311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a406ddd0 msgr2=0x7fc3a415ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 --2- 172.21.15.86:0/3371671311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a406ddd0 0x7fc3a415ef00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3900097c0 tx=0x7fc39001c410 comp rx=0 tx=0).stop 2024-03-22T04:13:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 -- 172.21.15.86:0/3371671311 shutdown_connections 2024-03-22T04:13:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 --2- 172.21.15.86:0/3371671311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a406ddd0 0x7fc3a415ef00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 -- 172.21.15.86:0/3371671311 >> 172.21.15.86:0/3371671311 conn(0x7fc3a406b660 msgr2=0x7fc3a406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 -- 172.21.15.86:0/3371671311 shutdown_connections 2024-03-22T04:13:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 -- 172.21.15.86:0/3371671311 wait complete. 2024-03-22T04:13:47.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 Processor -- start 2024-03-22T04:13:47.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.048+0000 7fc3a9c00640 1 -- start start 2024-03-22T04:13:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3a406ddd0 0x7fc3a41faa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3a413e060 0x7fc3a41f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a415f440 0x7fc3a41394c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a415f9d0 con 0x7fc3a413e060 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a415f850 con 0x7fc3a415f440 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a40cf9c0 con 0x7fc3a406ddd0 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3a413e060 0x7fc3a41f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3a413e060 0x7fc3a41f6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51342/0 (socket says 172.21.15.86:51342) 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 -- 172.21.15.86:0/36698598 learned_addr learned my addr 172.21.15.86:0/36698598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9200640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a415f440 0x7fc3a41394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3e00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3a406ddd0 0x7fc3a41faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 -- 172.21.15.86:0/36698598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3a406ddd0 msgr2=0x7fc3a41faa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3a406ddd0 0x7fc3a41faa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 -- 172.21.15.86:0/36698598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a415f440 msgr2=0x7fc3a41394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a415f440 0x7fc3a41394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 -- 172.21.15.86:0/36698598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a41faf90 con 0x7fc3a413e060 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a3400640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3a413e060 0x7fc3a41f6330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc3900ae040 tx=0x7fc3900191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc39be00640 1 -- 172.21.15.86:0/36698598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900b8070 con 0x7fc3a413e060 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc39be00640 1 -- 172.21.15.86:0/36698598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc390004020 con 0x7fc3a413e060 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc39be00640 1 -- 172.21.15.86:0/36698598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900b0040 con 0x7fc3a413e060 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a4153870 con 0x7fc3a413e060 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a4156170 con 0x7fc3a413e060 2024-03-22T04:13:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a4074470 con 0x7fc3a413e060 2024-03-22T04:13:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.052+0000 7fc39be00640 1 -- 172.21.15.86:0/36698598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc390003620 con 0x7fc3a413e060 2024-03-22T04:13:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.056+0000 7fc39be00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc384067b20 0x7fc384069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.056+0000 7fc39be00640 1 -- 172.21.15.86:0/36698598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fc39011fba0 con 0x7fc3a413e060 2024-03-22T04:13:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.056+0000 7fc3a3e00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc384067b20 0x7fc384069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:47.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.056+0000 7fc3a3e00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc384067b20 0x7fc384069fe0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc3940012f0 tx=0x7fc394009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:47.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.060+0000 7fc39be00640 1 -- 172.21.15.86:0/36698598 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc3a41faf90 con 0x7fc3a413e060 2024-03-22T04:13:47.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.248+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc3a413b0d0 con 0x7fc3a413e060 2024-03-22T04:13:47.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.248+0000 7fc39be00640 1 -- 172.21.15.86:0/36698598 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7fc390016090 con 0x7fc3a413e060 2024-03-22T04:13:47.252 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:13:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.248+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc384067b20 msgr2=0x7fc384069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc384067b20 0x7fc384069fe0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc3940012f0 tx=0x7fc394009040 comp rx=0 tx=0).stop 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3a413e060 msgr2=0x7fc3a41f6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3a413e060 0x7fc3a41f6330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc3900ae040 tx=0x7fc3900191b0 comp rx=0 tx=0).stop 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 shutdown_connections 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc384067b20 0x7fc384069fe0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a415f440 0x7fc3a41394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3a413e060 0x7fc3a41f6330 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 --2- 172.21.15.86:0/36698598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3a406ddd0 0x7fc3a41faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 >> 172.21.15.86:0/36698598 conn(0x7fc3a406b660 msgr2=0x7fc3a413a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 shutdown_connections 2024-03-22T04:13:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47.252+0000 7fc3a9c00640 1 -- 172.21.15.86:0/36698598 wait complete. 2024-03-22T04:13:47.273 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:13:47.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:13:47.273 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:13:47.273 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:13:47.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:13:47.273 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:13:47.274 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:13:47.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:13:47.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:13:47.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:47.274 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:13:47.284 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42305sec preferred_lft 42305sec 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:13:47.285 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:13:47.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:47.287 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:13:47.343 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:13:47.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:47.346 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:13:47.398 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:13:47.399 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:13:47.399 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:13:47.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:13:47.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:13:47.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:13:47.399 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:13:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:13:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:13:47.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:47.534 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:13:47.543 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:13:47.543 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:13:47.544 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:13:47.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:47.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:13:47.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:13:47.609 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:13:47.657 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:13:47.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:13:47.658 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:13:47.705 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:13:47.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:47.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:13:47.793 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:13:47.793 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:13:47.793 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:13:47.808 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:13:47.808 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:13:47.808 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:13:47.808 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:13:47.808 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:13:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:13:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:13:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:13:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:13:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:13:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:13:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:13:47.817 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:13:47.820 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:13:47.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:47.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:13:47.842 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:13:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:13:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:13:47.953 INFO:teuthology.orchestra.run.smithi086.stdout:client6390 2024-03-22T04:13:47.953 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:13:47.953 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:13:47.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:47.962 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-03-22T04:13:48.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:48.021 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-03-22T04:13:48.072 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:13:48.072 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:13:48.072 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:13:48.072 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:13:48.072 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-03-22T04:13:48.072 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:13:48.176 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-03-22T04:13:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:13:48.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:13:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.356+0000 7effaca00640 1 Processor -- start 2024-03-22T04:13:48.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.356+0000 7effaca00640 1 -- start start 2024-03-22T04:13:48.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effaca00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7effa81444f0 0x7effa81692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effaca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa80ae5a0 con 0x7effa80aff50 2024-03-22T04:13:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effaca00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa8144b80 con 0x7effa806cd80 2024-03-22T04:13:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effaca00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7effa8162aa0 con 0x7effa81444f0 2024-03-22T04:13:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7effa80aff50 0x7effa806c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42440/0 (socket says 172.21.15.86:42440) 2024-03-22T04:13:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa5e00640 1 -- 172.21.15.86:0/3027274615 learned_addr learned my addr 172.21.15.86:0/3027274615 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:48.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa6800640 1 --2- 172.21.15.86:0/3027274615 >> v2:172.21.15.196:3300/0 conn(0x7effa81444f0 0x7effa81692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa5400640 1 -- 172.21.15.86:0/3027274615 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900350926 0 0) 0x7eff980014a0 con 0x7effa80aff50 2024-03-22T04:13:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa5400640 1 -- 172.21.15.86:0/3027274615 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff8c003530 con 0x7effa80aff50 2024-03-22T04:13:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa5400640 1 -- 172.21.15.86:0/3027274615 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2586240634 0 0) 0x7effa8144b80 con 0x7effa806cd80 2024-03-22T04:13:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa5400640 1 -- 172.21.15.86:0/3027274615 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff980014a0 con 0x7effa806cd80 2024-03-22T04:13:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa6800640 1 -- 172.21.15.86:0/3027274615 >> v1:172.21.15.196:6789/0 conn(0x7effa806cd80 legacy=0x7effa8143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa5400640 1 -- 172.21.15.86:0/3027274615 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1571356572 0 0) 0x7eff8c003530 con 0x7effa80aff50 2024-03-22T04:13:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa6800640 1 -- 172.21.15.86:0/3027274615 >> v1:172.21.15.86:6789/0 conn(0x7effa80aff50 legacy=0x7effa806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.360+0000 7effa6800640 1 -- 172.21.15.86:0/3027274615 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa8162170 con 0x7effa81444f0 2024-03-22T04:13:48.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effa6800640 1 --2- 172.21.15.86:0/3027274615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa81444f0 0x7effa81692f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7effa00097c0 tx=0x7effa001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94234a2a61299348 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:48.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effa5400640 1 -- 172.21.15.86:0/3027274615 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa00b0070 con 0x7effa81444f0 2024-03-22T04:13:48.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effa5400640 1 -- 172.21.15.86:0/3027274615 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa00ab400 con 0x7effa81444f0 2024-03-22T04:13:48.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effa5400640 1 -- 172.21.15.86:0/3027274615 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98001bd0 con 0x7effa81444f0 2024-03-22T04:13:48.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 -- 172.21.15.86:0/3027274615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa81444f0 msgr2=0x7effa81692f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 --2- 172.21.15.86:0/3027274615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa81444f0 0x7effa81692f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7effa00097c0 tx=0x7effa001c330 comp rx=0 tx=0).stop 2024-03-22T04:13:48.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 -- 172.21.15.86:0/3027274615 shutdown_connections 2024-03-22T04:13:48.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 --2- 172.21.15.86:0/3027274615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa81444f0 0x7effa81692f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 -- 172.21.15.86:0/3027274615 >> 172.21.15.86:0/3027274615 conn(0x7effa8071fe0 msgr2=0x7effa80723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:48.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 -- 172.21.15.86:0/3027274615 shutdown_connections 2024-03-22T04:13:48.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 -- 172.21.15.86:0/3027274615 wait complete. 2024-03-22T04:13:48.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 Processor -- start 2024-03-22T04:13:48.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.364+0000 7effaca00640 1 -- start start 2024-03-22T04:13:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa806cd80 0x7effa8139250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effa80aff50 0x7effa8160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa81444f0 0x7effa8161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7effa80b04b0 con 0x7effa81444f0 2024-03-22T04:13:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effa80b0330 con 0x7effa806cd80 2024-03-22T04:13:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7effa816c460 con 0x7effa80aff50 2024-03-22T04:13:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa81444f0 0x7effa8161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa81444f0 0x7effa8161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51346/0 (socket says 172.21.15.86:51346) 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 -- 172.21.15.86:0/657748976 learned_addr learned my addr 172.21.15.86:0/657748976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa5e00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa806cd80 0x7effa8139250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7eff9d400640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effa80aff50 0x7effa8160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 -- 172.21.15.86:0/657748976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effa80aff50 msgr2=0x7effa8160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effa80aff50 0x7effa8160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 -- 172.21.15.86:0/657748976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa806cd80 msgr2=0x7effa8139250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa806cd80 0x7effa8139250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 -- 172.21.15.86:0/657748976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa8139790 con 0x7effa81444f0 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effa6800640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa81444f0 0x7effa8161390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7effa00097c0 tx=0x7effa00190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7eff9ea00640 1 -- 172.21.15.86:0/657748976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa00b9020 con 0x7effa81444f0 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7eff9ea00640 1 -- 172.21.15.86:0/657748976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa0003820 con 0x7effa81444f0 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7eff9ea00640 1 -- 172.21.15.86:0/657748976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa00b0040 con 0x7effa81444f0 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa80b44b0 con 0x7effa81444f0 2024-03-22T04:13:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa80ce920 con 0x7effa81444f0 2024-03-22T04:13:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.368+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa8068650 con 0x7effa81444f0 2024-03-22T04:13:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.372+0000 7eff9ea00640 1 -- 172.21.15.86:0/657748976 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7effa00bc020 con 0x7effa81444f0 2024-03-22T04:13:48.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.372+0000 7eff9ea00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff84067b20 0x7eff84069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.372+0000 7eff9ea00640 1 -- 172.21.15.86:0/657748976 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7effa011fe80 con 0x7effa81444f0 2024-03-22T04:13:48.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.372+0000 7effa5e00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff84067b20 0x7eff84069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.372+0000 7effa5e00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff84067b20 0x7eff84069fe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7eff98001360 tx=0x7eff98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:48.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.376+0000 7eff9ea00640 1 -- 172.21.15.86:0/657748976 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7effa00c3030 con 0x7effa81444f0 2024-03-22T04:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.564+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effa8169610 con 0x7effa81444f0 2024-03-22T04:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.564+0000 7eff9ea00640 1 -- 172.21.15.86:0/657748976 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5185 (secure 0 0 0) 0x7effa0015fb0 con 0x7effa81444f0 2024-03-22T04:13:48.569 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:48.569 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:13:38.919534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 35 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff84067b20 msgr2=0x7eff84069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff84067b20 0x7eff84069fe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7eff98001360 tx=0x7eff98009040 comp rx=0 tx=0).stop 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa81444f0 msgr2=0x7effa8161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa81444f0 0x7effa8161390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7effa00097c0 tx=0x7effa00190d0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 shutdown_connections 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff84067b20 0x7eff84069fe0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa81444f0 0x7effa8161390 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effa80aff50 0x7effa8160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 --2- 172.21.15.86:0/657748976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effa806cd80 0x7effa8139250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 >> 172.21.15.86:0/657748976 conn(0x7effa8071fe0 msgr2=0x7effa813a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 shutdown_connections 2024-03-22T04:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.568+0000 7effaca00640 1 -- 172.21.15.86:0/657748976 wait complete. 2024-03-22T04:13:48.591 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-03-22T04:13:48.761 INFO:teuthology.orchestra.run.smithi196.stdout:{"message":"","return_code":0} 2024-03-22T04:13:48.762 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:13:48.762 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-22T04:13:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 Processor -- start 2024-03-22T04:13:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- start start 2024-03-22T04:13:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a806d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a7fcb8d0 0x5585a806e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a8070610 0x5585a806e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fe49e0 con 0x5585a7fc8410 2024-03-22T04:13:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fe4740 con 0x5585a8070610 2024-03-22T04:13:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5585a7fe4c80 con 0x5585a7fcb8d0 2024-03-22T04:13:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a806d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51372/0 (socket says 172.21.15.86:51372) 2024-03-22T04:13:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 -- 172.21.15.86:0/3135680150 learned_addr learned my addr 172.21.15.86:0/3135680150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe804c00640 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a7fcb8d0 0x5585a806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe806000640 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a8070610 0x5585a806e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 -- 172.21.15.86:0/3135680150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a7fcb8d0 msgr2=0x5585a806e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a7fcb8d0 0x5585a806e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 -- 172.21.15.86:0/3135680150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a8070610 msgr2=0x5585a806e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a8070610 0x5585a806e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 -- 172.21.15.86:0/3135680150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a7ec5950 con 0x5585a7fc8410 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe805600640 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a806d470 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800a7e0 tx=0x7fe7f800aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611d1b9c778c7af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe7ff400640 1 -- 172.21.15.86:0/3135680150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8018070 con 0x5585a7fc8410 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe7ff400640 1 -- 172.21.15.86:0/3135680150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f8008c00 con 0x5585a7fc8410 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe7ff400640 1 -- 172.21.15.86:0/3135680150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8013430 con 0x5585a7fc8410 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- 172.21.15.86:0/3135680150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 msgr2=0x5585a806d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a806d470 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800a7e0 tx=0x7fe7f800aca0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- 172.21.15.86:0/3135680150 shutdown_connections 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a8070610 0x5585a806e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a7fcb8d0 0x5585a806e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 --2- 172.21.15.86:0/3135680150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a806d470 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- 172.21.15.86:0/3135680150 >> 172.21.15.86:0/3135680150 conn(0x5585a7fc6ff0 msgr2=0x5585a806ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- 172.21.15.86:0/3135680150 shutdown_connections 2024-03-22T04:13:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.804+0000 7fe8083d5080 1 -- 172.21.15.86:0/3135680150 wait complete. 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 Processor -- start 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 -- start start 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a7fdf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a7fcb8d0 0x5585a806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a8070610 0x5585a806b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585a7f99da0 con 0x5585a7fc8410 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5585a806d9b0 con 0x5585a7fcb8d0 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5585a7f99b00 con 0x5585a8070610 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a7fdf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a7fdf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51384/0 (socket says 172.21.15.86:51384) 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 -- 172.21.15.86:0/4191104186 learned_addr learned my addr 172.21.15.86:0/4191104186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe804c00640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a7fcb8d0 0x5585a806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe806000640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a8070610 0x5585a806b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 -- 172.21.15.86:0/4191104186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a8070610 msgr2=0x5585a806b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5585a8070610 0x5585a806b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 -- 172.21.15.86:0/4191104186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a7fcb8d0 msgr2=0x5585a806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585a7fcb8d0 0x5585a806ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 -- 172.21.15.86:0/4191104186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a810aa10 con 0x5585a7fc8410 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe805600640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a7fdf630 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8013d20 tx=0x7fe7f8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe7fe000640 1 -- 172.21.15.86:0/4191104186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8023020 con 0x5585a7fc8410 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe7fe000640 1 -- 172.21.15.86:0/4191104186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f801e470 con 0x5585a7fc8410 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe7fe000640 1 -- 172.21.15.86:0/4191104186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8018040 con 0x5585a7fc8410 2024-03-22T04:13:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5585a810acb0 con 0x5585a7fc8410 2024-03-22T04:13:48.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.808+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a810b1f0 con 0x5585a7fc8410 2024-03-22T04:13:48.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe7fe000640 1 -- 172.21.15.86:0/4191104186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe7f8026020 con 0x5585a7fc8410 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe7fe000640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe7e8066530 0x7fe7e80689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe7fe000640 1 -- 172.21.15.86:0/4191104186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fe7f8010020 con 0x5585a7fc8410 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe804c00640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe7e8066530 0x7fe7e80689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5585a7fdfb70 0x5585a810b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) -- 0x5585a810be70 con 0x5585a7fdfb70 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe806000640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5585a7fdfb70 0x5585a810b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe804c00640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe7e8066530 0x7fe7e80689f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40ac000 tx=0x7fe7f40a5000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe806000640 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5585a7fdfb70 0x5585a810b930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe806000640 1 -- 172.21.15.86:0/4191104186 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7fe800010940 con 0x5585a7fdfb70 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5585a7fdfb70 msgr2=0x5585a810b930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5585a7fdfb70 0x5585a810b930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe7e8066530 msgr2=0x7fe7e80689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe7e8066530 0x7fe7e80689f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40ac000 tx=0x7fe7f40a5000 comp rx=0 tx=0).stop 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 msgr2=0x5585a7fdf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 --2- 172.21.15.86:0/4191104186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585a7fc8410 0x5585a7fdf630 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8013d20 tx=0x7fe7f8003030 comp rx=0 tx=0).stop 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe806000640 1 -- 172.21.15.86:0/4191104186 reap_dead start 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 shutdown_connections 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 >> 172.21.15.86:0/4191104186 conn(0x5585a7fc6ff0 msgr2=0x5585a8068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 shutdown_connections 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.812+0000 7fe8083d5080 1 -- 172.21.15.86:0/4191104186 wait complete. 2024-03-22T04:13:48.820 INFO:teuthology.orchestra.run.smithi086.stdout:Xfiledir_test 2024-03-22T04:13:48.821 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:48.822 DEBUG:teuthology.orchestra.run.smithi086:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-22T04:13:48.906 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:13:48.906 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-03-22T04:13:48.906 INFO:teuthology.orchestra.run.smithi086.stdout: "ancestors": [ 2024-03-22T04:13:48.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1099511627776, 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "file", 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 2 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1, 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "dir_test", 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 10, 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: "old_pools": [] 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:13:48.907 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:13:48.911 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-03-22T04:13:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 Processor -- start 2024-03-22T04:13:49.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 -- start start 2024-03-22T04:13:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4de20cc0 0x561f4de31b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 0x561f4de37850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4dec5450 0x561f4df57fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f4dd96620 con 0x561f4dec5450 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f4dd9a670 con 0x561f4de20cc0 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561f4dec71b0 con 0x561f4de399b0 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4dec5450 0x561f4df57fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9373a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4de20cc0 0x561f4de31b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4dec5450 0x561f4df57fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51400/0 (socket says 172.21.15.86:51400) 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 -- 172.21.15.86:0/4150437009 learned_addr learned my addr 172.21.15.86:0/4150437009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9373000640 1 --2- 172.21.15.86:0/4150437009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 0x561f4de37850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 -- 172.21.15.86:0/4150437009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 msgr2=0x561f4de37850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 --2- 172.21.15.86:0/4150437009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 0x561f4de37850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 -- 172.21.15.86:0/4150437009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4de20cc0 msgr2=0x561f4de31b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 --2- 172.21.15.86:0/4150437009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4de20cc0 0x561f4de31b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 -- 172.21.15.86:0/4150437009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f4dec04f0 con 0x561f4dec5450 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9374400640 1 --2- 172.21.15.86:0/4150437009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4dec5450 0x561f4df57fd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f936c001710 tx=0x7f936c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fb2b5ab5cde9cf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9371c00640 1 -- 172.21.15.86:0/4150437009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c017070 con 0x561f4dec5450 2024-03-22T04:13:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9371c00640 1 -- 172.21.15.86:0/4150437009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c00f710 con 0x561f4dec5450 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f9371c00640 1 -- 172.21.15.86:0/4150437009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c00fb00 con 0x561f4dec5450 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:48.996+0000 7f93766e2080 1 -- 172.21.15.86:0/4150437009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4dec5450 msgr2=0x561f4df57fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 --2- 172.21.15.86:0/4150437009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4dec5450 0x561f4df57fd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f936c001710 tx=0x7f936c006f90 comp rx=0 tx=0).stop 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- 172.21.15.86:0/4150437009 shutdown_connections 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 --2- 172.21.15.86:0/4150437009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4dec5450 0x561f4df57fd0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 --2- 172.21.15.86:0/4150437009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 0x561f4de37850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 --2- 172.21.15.86:0/4150437009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4de20cc0 0x561f4de31b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- 172.21.15.86:0/4150437009 >> 172.21.15.86:0/4150437009 conn(0x561f4de1ceb0 msgr2=0x561f4de1d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- 172.21.15.86:0/4150437009 shutdown_connections 2024-03-22T04:13:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- 172.21.15.86:0/4150437009 wait complete. 2024-03-22T04:13:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 Processor -- start 2024-03-22T04:13:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- start start 2024-03-22T04:13:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4de20cc0 0x561f4df633a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 0x561f4deba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4dec5450 0x561f4debaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f4dec0a30 con 0x561f4de20cc0 2024-03-22T04:13:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561f4dec0790 con 0x561f4dec5450 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561f4de373b0 con 0x561f4de399b0 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4de20cc0 0x561f4df633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4de20cc0 0x561f4df633a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51416/0 (socket says 172.21.15.86:51416) 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 -- 172.21.15.86:0/2461870467 learned_addr learned my addr 172.21.15.86:0/2461870467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9374400640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4dec5450 0x561f4debaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373000640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 0x561f4deba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 -- 172.21.15.86:0/2461870467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 msgr2=0x561f4deba9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561f4de399b0 0x561f4deba9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 -- 172.21.15.86:0/2461870467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4dec5450 msgr2=0x561f4debaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561f4dec5450 0x561f4debaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 -- 172.21.15.86:0/2461870467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f4debdf30 con 0x561f4de20cc0 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f9373a00640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4de20cc0 0x561f4df633a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f93640ac040 tx=0x7f9364019c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f936be00640 1 -- 172.21.15.86:0/2461870467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93640c3070 con 0x561f4de20cc0 2024-03-22T04:13:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f936be00640 1 -- 172.21.15.86:0/2461870467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93640be3f0 con 0x561f4de20cc0 2024-03-22T04:13:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f936be00640 1 -- 172.21.15.86:0/2461870467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93640be7a0 con 0x561f4de20cc0 2024-03-22T04:13:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f4debe1d0 con 0x561f4de20cc0 2024-03-22T04:13:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.000+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f4debe710 con 0x561f4de20cc0 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f936be00640 1 -- 172.21.15.86:0/2461870467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f93640a9070 con 0x561f4de20cc0 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f936be00640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9354066500 0x7f93540689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f936be00640 1 -- 172.21.15.86:0/2461870467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f936411fb00 con 0x561f4de20cc0 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f9373000640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9354066500 0x7f93540689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f93766e2080 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561f4df61a50 0x561f4df6cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) -- 0x561f4df6d190 con 0x561f4df61a50 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f9374400640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561f4df61a50 0x561f4df6cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f9373000640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9354066500 0x7f93540689c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9360001ee0 tx=0x7f9360001bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.004+0000 7f9374400640 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561f4df61a50 0x561f4df6cc50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f9374400640 1 -- 172.21.15.86:0/2461870467 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v67'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f936c00f070 con 0x561f4df61a50 2024-03-22T04:13:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561f4df61a50 msgr2=0x561f4df6cc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561f4df61a50 0x561f4df6cc50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9354066500 msgr2=0x7f93540689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9354066500 0x7f93540689c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9360001ee0 tx=0x7f9360001bd0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4de20cc0 msgr2=0x561f4df633a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 --2- 172.21.15.86:0/2461870467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f4de20cc0 0x561f4df633a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f93640ac040 tx=0x7f9364019c20 comp rx=0 tx=0).stop 2024-03-22T04:13:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f9374400640 1 -- 172.21.15.86:0/2461870467 reap_dead start 2024-03-22T04:13:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 shutdown_connections 2024-03-22T04:13:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 >> 172.21.15.86:0/2461870467 conn(0x561f4de1ceb0 msgr2=0x561f4de31950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 shutdown_connections 2024-03-22T04:13:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.008+0000 7f93766e2080 1 -- 172.21.15.86:0/2461870467 wait complete. 2024-03-22T04:13:49.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:49.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-03-22T04:13:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a86000640 1 Processor -- start 2024-03-22T04:13:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a86000640 1 -- start start 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a86000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a80144a90 0x7f2a800727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a86000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 0x7f2a800ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a86000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a80144e70 con 0x7f2a80072cf0 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a86000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2a800b3e00 con 0x7f2a80144a90 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a86000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a800b40e0 con 0x7f2a800b37c0 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2a80072cf0 0x7f2a800b30b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42470/0 (socket says 172.21.15.86:42470) 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7ea00640 1 -- 172.21.15.86:0/2823446526 learned_addr learned my addr 172.21.15.86:0/2823446526 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7f400640 1 --2- 172.21.15.86:0/2823446526 >> v2:172.21.15.196:3300/0 conn(0x7f2a80144a90 0x7f2a800727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/2823446526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 0x7f2a800ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7f400640 1 -- 172.21.15.86:0/2823446526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 msgr2=0x7f2a800ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7e000640 1 -- 172.21.15.86:0/2823446526 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2364679823 0 0) 0x7f2a80144e70 con 0x7f2a80072cf0 2024-03-22T04:13:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7f400640 1 --2- 172.21.15.86:0/2823446526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 0x7f2a800ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7f400640 1 -- 172.21.15.86:0/2823446526 >> v1:172.21.15.86:6789/0 conn(0x7f2a80072cf0 legacy=0x7f2a800b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.152+0000 7f2a7f400640 1 -- 172.21.15.86:0/2823446526 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a800cdbd0 con 0x7f2a80144a90 2024-03-22T04:13:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a7f400640 1 --2- 172.21.15.86:0/2823446526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 0x7f2a800727b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a700097c0 tx=0x7f2a7001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c386b06771370bd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a7e000640 1 -- 172.21.15.86:0/2823446526 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700b0070 con 0x7f2a80144a90 2024-03-22T04:13:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a7e000640 1 -- 172.21.15.86:0/2823446526 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a700ab600 con 0x7f2a80144a90 2024-03-22T04:13:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a7e000640 1 -- 172.21.15.86:0/2823446526 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700ab9d0 con 0x7f2a80144a90 2024-03-22T04:13:49.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a86000640 1 -- 172.21.15.86:0/2823446526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 msgr2=0x7f2a800727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a86000640 1 --2- 172.21.15.86:0/2823446526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 0x7f2a800727b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a700097c0 tx=0x7f2a7001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a86000640 1 -- 172.21.15.86:0/2823446526 shutdown_connections 2024-03-22T04:13:49.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a86000640 1 --2- 172.21.15.86:0/2823446526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 0x7f2a800ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a86000640 1 --2- 172.21.15.86:0/2823446526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 0x7f2a800727b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a86000640 1 -- 172.21.15.86:0/2823446526 >> 172.21.15.86:0/2823446526 conn(0x7f2a8006b660 msgr2=0x7f2a8006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a86000640 1 -- 172.21.15.86:0/2823446526 shutdown_connections 2024-03-22T04:13:49.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.156+0000 7f2a86000640 1 -- 172.21.15.86:0/2823446526 wait complete. 2024-03-22T04:13:49.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 Processor -- start 2024-03-22T04:13:49.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 -- start start 2024-03-22T04:13:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a80072cf0 0x7f2a8013acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 0x7f2a80136bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 0x7f2a800b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800cf4f0 con 0x7f2a80072cf0 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800cf370 con 0x7f2a80144a90 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a800cf670 con 0x7f2a800b37c0 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a80072cf0 0x7f2a8013acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a80072cf0 0x7f2a8013acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51422/0 (socket says 172.21.15.86:51422) 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 -- 172.21.15.86:0/1669812002 learned_addr learned my addr 172.21.15.86:0/1669812002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 0x7f2a800b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 0x7f2a80136bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 -- 172.21.15.86:0/1669812002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 msgr2=0x7f2a80136bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 0x7f2a80136bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 -- 172.21.15.86:0/1669812002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 msgr2=0x7f2a800b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 0x7f2a800b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 -- 172.21.15.86:0/1669812002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a80137130 con 0x7f2a80072cf0 2024-03-22T04:13:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a7f400640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a80072cf0 0x7f2a8013acf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001a0d0 tx=0x7f2a70019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a77400640 1 -- 172.21.15.86:0/1669812002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700ba070 con 0x7f2a80072cf0 2024-03-22T04:13:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a77400640 1 -- 172.21.15.86:0/1669812002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a700b43f0 con 0x7f2a80072cf0 2024-03-22T04:13:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a77400640 1 -- 172.21.15.86:0/1669812002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700b0040 con 0x7f2a80072cf0 2024-03-22T04:13:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 -- 172.21.15.86:0/1669812002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a800b0c00 con 0x7f2a80072cf0 2024-03-22T04:13:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.160+0000 7f2a86000640 1 -- 172.21.15.86:0/1669812002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8013b5b0 con 0x7f2a80072cf0 2024-03-22T04:13:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a77400640 1 -- 172.21.15.86:0/1669812002 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2a700b47c0 con 0x7f2a80072cf0 2024-03-22T04:13:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a77400640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a60067b20 0x7f2a60069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a77400640 1 -- 172.21.15.86:0/1669812002 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f2a7011f180 con 0x7f2a80072cf0 2024-03-22T04:13:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a60067b20 0x7f2a60069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a89489000 1 Processor -- start 2024-03-22T04:13:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a89489000 1 -- start start 2024-03-22T04:13:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a60067b20 0x7f2a60069fe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2a80140500 tx=0x7f2a6c007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a89489000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 0x559ba0e28370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a89489000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0dde310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a89489000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0ddec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a89489000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559ba0c2ce70 con 0x559ba0e288b0 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a89489000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ba0bca110 con 0x559ba0e27f90 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a89489000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559ba0c28e20 con 0x559ba0dde850 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0dde310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0dde310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51434/0 (socket says 172.21.15.86:51434) 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a7ea00640 1 -- 172.21.15.86:0/276955445 learned_addr learned my addr 172.21.15.86:0/276955445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0ddec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.164+0000 7f2a7f400640 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 0x559ba0e28370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7ea00640 1 -- 172.21.15.86:0/276955445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 msgr2=0x559ba0ddec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0ddec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7ea00640 1 -- 172.21.15.86:0/276955445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 msgr2=0x559ba0e28370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 0x559ba0e28370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7ea00640 1 -- 172.21.15.86:0/276955445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba0cd3f20 con 0x559ba0e288b0 2024-03-22T04:13:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0dde310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c004fd0 tx=0x7f2a6c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7aec36f868a5ced server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a6be00640 1 -- 172.21.15.86:0/276955445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c026070 con 0x559ba0e288b0 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a6be00640 1 -- 172.21.15.86:0/276955445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c013970 con 0x559ba0e288b0 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a6be00640 1 -- 172.21.15.86:0/276955445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c013d60 con 0x559ba0e288b0 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- 172.21.15.86:0/276955445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 msgr2=0x559ba0dde310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0dde310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c004fd0 tx=0x7f2a6c012000 comp rx=0 tx=0).stop 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- 172.21.15.86:0/276955445 shutdown_connections 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0ddec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0dde310 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 --2- 172.21.15.86:0/276955445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 0x559ba0e28370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- 172.21.15.86:0/276955445 >> 172.21.15.86:0/276955445 conn(0x559ba0bd9bd0 msgr2=0x559ba0ddcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- 172.21.15.86:0/276955445 shutdown_connections 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- 172.21.15.86:0/276955445 wait complete. 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- start start 2024-03-22T04:13:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 Processor -- start 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0e2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 0x559ba0e30240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e28ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559ba0bb12e0 con 0x559ba0e288b0 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ba0bea700 con 0x559ba0e27f90 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559ba0c29df0 con 0x559ba0dde850 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a89489000 10 client.0 _ll_register_callbacks cb 0x559ba0bd5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e28ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e28ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51438/0 (socket says 172.21.15.86:51438) 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 0x559ba0e30240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 -- 172.21.15.86:0/1067918888 learned_addr learned my addr 172.21.15.86:0/1067918888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7f400640 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0e2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 -- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 msgr2=0x559ba0e2fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0e2fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 -- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 msgr2=0x559ba0e30240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 0x559ba0e30240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 -- 172.21.15.86:0/1067918888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba0c09050 con 0x559ba0e288b0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e28ea0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2a78014d90 tx=0x7f2a78012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebec738431bdaa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.168+0000 7f2a68c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 1 -- 172.21.15.86:0/1067918888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7801e620 con 0x559ba0e288b0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 -- 172.21.15.86:0/1067918888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6c013970 con 0x559ba0e288b0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 1 -- 172.21.15.86:0/1067918888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f2a7801e7c0 con 0x559ba0e288b0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 1 -- 172.21.15.86:0/1067918888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2a7801e9c0 con 0x559ba0e288b0 2024-03-22T04:13:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 10 client.6429 fetch_fsmap learned FSMap version 35 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 -- 172.21.15.86:0/1067918888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x559ba0c71b90 con 0x559ba0e288b0 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 1 -- 172.21.15.86:0/1067918888 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c013970 con 0x559ba0e288b0 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 1 -- 172.21.15.86:0/1067918888 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f2a7802b080 con 0x559ba0e288b0 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 10 client.6429 fetch_fsmap finished waiting for FSMap version 35 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 10 client.6429 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 20 client.6429 populate_metadata read hostname 'smithi086' 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559ba0e298b0 0x559ba0e29cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 4 client.6429 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 -- 172.21.15.86:0/1067918888 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ba0bc7f70 con 0x559ba0e298b0 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a7f400640 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559ba0e298b0 0x559ba0e29cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a7f400640 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559ba0e298b0 0x559ba0e29cc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef11d70f3c71ce5 server_cookie=f30b2feafac4e65e in_seq=0 out_seq=0 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 10 client.6429 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 1 -- 172.21.15.86:0/1067918888 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559ba0bc7f70 con 0x559ba0e298b0 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 10 client.6429 handle_command_reply: tid=0 2024-03-22T04:13:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 client.6429 shutdown 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 -- 172.21.15.86:0/1067918888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 msgr2=0x559ba0e28ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e28ea0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2a78014d90 tx=0x7f2a78012670 comp rx=0 tx=0).stop 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 -- 172.21.15.86:0/1067918888 shutdown_connections 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559ba0e298b0 0x559ba0e29cc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e27f90 0x559ba0e30240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e28ea0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 --2- 172.21.15.86:0/1067918888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0e2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 -- 172.21.15.86:0/1067918888 >> 172.21.15.86:0/1067918888 conn(0x559ba0bd9bd0 msgr2=0x559ba0ddcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a68c00640 0 client.6429 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 -- 172.21.15.86:0/1067918888 shutdown_connections 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 1 -- 172.21.15.86:0/1067918888 wait complete. 2024-03-22T04:13:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.172+0000 7f2a89489000 20 client.6429 trim_cache size 0 max 16384 2024-03-22T04:13:49.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.188+0000 7f2a89489000 1 Processor -- start 2024-03-22T04:13:49.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.188+0000 7f2a89489000 1 -- start start 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0e3ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e27f90 0x559ba0e3f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e288b0 0x559ba0e3f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a80203c20 con 0x559ba0e27f90 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a80202c20 con 0x559ba0e288b0 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a80139470 con 0x559ba0dde850 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e27f90 0x559ba0e3f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e27f90 0x559ba0e3f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51450/0 (socket says 172.21.15.86:51450) 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 -- 172.21.15.86:0/163637321 learned_addr learned my addr 172.21.15.86:0/163637321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7f400640 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0e3ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e288b0 0x559ba0e3f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 -- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 msgr2=0x559ba0e3ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0e3ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 -- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e288b0 msgr2=0x559ba0e3f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e288b0 0x559ba0e3f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 -- 172.21.15.86:0/163637321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba0bb24e0 con 0x559ba0e27f90 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e288b0 0x559ba0e3f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e27f90 0x559ba0e3f440 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c000f70 tx=0x7f2a6c009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea2f0b67c47bbcd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a6b400640 1 -- 172.21.15.86:0/163637321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c026050 con 0x559ba0e27f90 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a6b400640 1 -- 172.21.15.86:0/163637321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c027040 con 0x559ba0e27f90 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a6b400640 1 -- 172.21.15.86:0/163637321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c013d00 con 0x559ba0e27f90 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- 172.21.15.86:0/163637321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e27f90 msgr2=0x559ba0e3f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e27f90 0x559ba0e3f440 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c000f70 tx=0x7f2a6c009070 comp rx=0 tx=0).stop 2024-03-22T04:13:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- 172.21.15.86:0/163637321 shutdown_connections 2024-03-22T04:13:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0e288b0 0x559ba0e3f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e27f90 0x559ba0e3f440 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 --2- 172.21.15.86:0/163637321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0dde850 0x559ba0e3ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- 172.21.15.86:0/163637321 >> 172.21.15.86:0/163637321 conn(0x559ba0bd9bd0 msgr2=0x559ba0e0ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- 172.21.15.86:0/163637321 shutdown_connections 2024-03-22T04:13:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- 172.21.15.86:0/163637321 wait complete. 2024-03-22T04:13:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 -- start start 2024-03-22T04:13:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.192+0000 7f2a89489000 1 Processor -- start 2024-03-22T04:13:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0dde850 0x559ba0e94e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0e27f90 0x559ba0e2bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e2c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559ba0bf00b0 con 0x559ba0e288b0 2024-03-22T04:13:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a802025d0 con 0x559ba0dde850 2024-03-22T04:13:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559ba0c2c180 con 0x559ba0e27f90 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 10 client.0 _ll_register_callbacks cb 0x7f2a78009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e2c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0dde850 0x559ba0e94e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e2c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51466/0 (socket says 172.21.15.86:51466) 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 -- 172.21.15.86:0/2461287773 learned_addr learned my addr 172.21.15.86:0/2461287773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7ea00640 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0e27f90 0x559ba0e2bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 -- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0e27f90 msgr2=0x559ba0e2bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0e27f90 0x559ba0e2bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 -- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0dde850 msgr2=0x559ba0e94e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0dde850 0x559ba0e94e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 -- 172.21.15.86:0/2461287773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba0cacf80 con 0x559ba0e288b0 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7f400640 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0dde850 0x559ba0e94e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7fe00640 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e2c1e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2a78014d90 tx=0x7f2a78004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141f8af1f538563f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:13:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 1 -- 172.21.15.86:0/2461287773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7802f070 con 0x559ba0e288b0 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 1 -- 172.21.15.86:0/2461287773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f2a78025ab0 con 0x559ba0e288b0 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- 172.21.15.86:0/2461287773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6c027040 con 0x559ba0e288b0 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 1 -- 172.21.15.86:0/2461287773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2a78025cb0 con 0x559ba0e288b0 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 10 client.6435 fetch_fsmap learned FSMap version 35 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- 172.21.15.86:0/2461287773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x559ba0bd3670 con 0x559ba0e288b0 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 1 -- 172.21.15.86:0/2461287773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c027040 con 0x559ba0e288b0 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 1 -- 172.21.15.86:0/2461287773 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f2a780252c0 con 0x559ba0e288b0 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 10 client.6435 fetch_fsmap finished waiting for FSMap version 35 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 10 client.6435 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 20 client.6435 populate_metadata read hostname 'smithi086' 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559ba0e2c720 0x559ba0e2cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 4 client.6435 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-22T04:13:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- 172.21.15.86:0/2461287773 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x559ba0e29230 con 0x559ba0e2c720 2024-03-22T04:13:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7f400640 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559ba0e2c720 0x559ba0e2cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a7f400640 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559ba0e2c720 0x559ba0e2cb30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11496f49932d1b21 server_cookie=7541df009a29782a in_seq=0 out_seq=0 2024-03-22T04:13:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 10 client.6435 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 1 -- 172.21.15.86:0/2461287773 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559ba0e29230 con 0x559ba0e2c720 2024-03-22T04:13:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 10 client.6435 handle_command_reply: tid=0 2024-03-22T04:13:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 client.6435 shutdown 2024-03-22T04:13:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- 172.21.15.86:0/2461287773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 msgr2=0x559ba0e2c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e2c1e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2a78014d90 tx=0x7f2a78004a50 comp rx=0 tx=0).stop 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- 172.21.15.86:0/2461287773 shutdown_connections 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559ba0e2c720 0x559ba0e2cb30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ba0e288b0 0x559ba0e2c1e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559ba0e27f90 0x559ba0e2bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 --2- 172.21.15.86:0/2461287773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ba0dde850 0x559ba0e94e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a89489000 1 -- 172.21.15.86:0/2461287773 >> 172.21.15.86:0/2461287773 conn(0x559ba0bd9bd0 msgr2=0x559ba0e0b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.196+0000 7f2a68c00640 0 client.6435 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a89489000 1 -- 172.21.15.86:0/2461287773 shutdown_connections 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a89489000 1 -- 172.21.15.86:0/2461287773 wait complete. 2024-03-22T04:13:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a89489000 20 client.6435 trim_cache size 0 max 16384 2024-03-22T04:13:49.211 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:13:49.211 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:13:49.211 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "842888e6-e588-455c-a795-7bf2b70daa21", 2024-03-22T04:13:49.211 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:13:49.211 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 -- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a60067b20 msgr2=0x7f2a60069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a60067b20 0x7f2a60069fe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2a80140500 tx=0x7f2a6c007df0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 -- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a80072cf0 msgr2=0x7f2a8013acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a80072cf0 0x7f2a8013acf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001a0d0 tx=0x7f2a70019240 comp rx=0 tx=0).stop 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 -- 172.21.15.86:0/1669812002 shutdown_connections 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a60067b20 0x7f2a60069fe0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144a90 0x7f2a800b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800b37c0 0x7f2a80136bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 --2- 172.21.15.86:0/1669812002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a80072cf0 0x7f2a8013acf0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 -- 172.21.15.86:0/1669812002 >> 172.21.15.86:0/1669812002 conn(0x7f2a8006b660 msgr2=0x7f2a800b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 -- 172.21.15.86:0/1669812002 shutdown_connections 2024-03-22T04:13:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.200+0000 7f2a86000640 1 -- 172.21.15.86:0/1669812002 wait complete. 2024-03-22T04:13:49.214 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:13:49.199+0000 7f42ba400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6390=0-4194304@1} caps={6390=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6390 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562f7ea04580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-22T04:13:49.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:13:49.221 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-22T04:13:49.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f314ca00640 1 Processor -- start 2024-03-22T04:13:49.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f314ca00640 1 -- start start 2024-03-22T04:13:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f314ca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480b1b80 0x7f31480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f314ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f314806d6d0 con 0x7f3148144a90 2024-03-22T04:13:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f314ca00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f314806d970 con 0x7f31480cdf10 2024-03-22T04:13:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f314ca00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3148144e70 con 0x7f31480b1b80 2024-03-22T04:13:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3145e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3148144a90 0x7f314815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42474/0 (socket says 172.21.15.86:42474) 2024-03-22T04:13:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3145e00640 1 -- 172.21.15.86:0/1560573139 learned_addr learned my addr 172.21.15.86:0/1560573139 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:13:49.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3145400640 1 --2- 172.21.15.86:0/1560573139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480b1b80 0x7f31480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3144a00640 1 -- 172.21.15.86:0/1560573139 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147860415 0 0) 0x7f314806d6d0 con 0x7f3148144a90 2024-03-22T04:13:49.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3144a00640 1 -- 172.21.15.86:0/1560573139 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f312c003530 con 0x7f3148144a90 2024-03-22T04:13:49.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3144a00640 1 -- 172.21.15.86:0/1560573139 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082257861 0 0) 0x7f314806d970 con 0x7f31480cdf10 2024-03-22T04:13:49.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3144a00640 1 -- 172.21.15.86:0/1560573139 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f314806d6d0 con 0x7f31480cdf10 2024-03-22T04:13:49.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3145400640 1 -- 172.21.15.86:0/1560573139 >> v1:172.21.15.196:6789/0 conn(0x7f31480cdf10 legacy=0x7f31480ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3145400640 1 -- 172.21.15.86:0/1560573139 >> v1:172.21.15.86:6789/0 conn(0x7f3148144a90 legacy=0x7f314815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.356+0000 7f3145400640 1 -- 172.21.15.86:0/1560573139 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314806dc50 con 0x7f31480b1b80 2024-03-22T04:13:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f3145400640 1 --2- 172.21.15.86:0/1560573139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480b1b80 0x7f31480b2000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31340097f0 tx=0x7f313401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24de8f3c8ba2061b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f3144a00640 1 -- 172.21.15.86:0/1560573139 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340b1070 con 0x7f31480b1b80 2024-03-22T04:13:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f3144a00640 1 -- 172.21.15.86:0/1560573139 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31340ac630 con 0x7f31480b1b80 2024-03-22T04:13:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f3144a00640 1 -- 172.21.15.86:0/1560573139 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340a8050 con 0x7f31480b1b80 2024-03-22T04:13:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f314ca00640 1 -- 172.21.15.86:0/1560573139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480b1b80 msgr2=0x7f31480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f314ca00640 1 --2- 172.21.15.86:0/1560573139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480b1b80 0x7f31480b2000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31340097f0 tx=0x7f313401c4e0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f314ca00640 1 -- 172.21.15.86:0/1560573139 shutdown_connections 2024-03-22T04:13:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f314ca00640 1 --2- 172.21.15.86:0/1560573139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480b1b80 0x7f31480b2000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f314ca00640 1 -- 172.21.15.86:0/1560573139 >> 172.21.15.86:0/1560573139 conn(0x7f31480cc940 msgr2=0x7f31480ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f314ca00640 1 -- 172.21.15.86:0/1560573139 shutdown_connections 2024-03-22T04:13:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.360+0000 7f314ca00640 1 -- 172.21.15.86:0/1560573139 wait complete. 2024-03-22T04:13:49.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 Processor -- start 2024-03-22T04:13:49.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 -- start start 2024-03-22T04:13:49.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31480b1b80 0x7f3148141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480cdf10 0x7f31480ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3148144a90 0x7f31480c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31480add70 con 0x7f31480b1b80 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31480adbf0 con 0x7f3148144a90 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f314816c490 con 0x7f31480cdf10 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31480b1b80 0x7f3148141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31480b1b80 0x7f3148141940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51476/0 (socket says 172.21.15.86:51476) 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 -- 172.21.15.86:0/4132907161 learned_addr learned my addr 172.21.15.86:0/4132907161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145400640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480cdf10 0x7f31480ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3146800640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3148144a90 0x7f31480c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 -- 172.21.15.86:0/4132907161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480cdf10 msgr2=0x7f31480ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480cdf10 0x7f31480ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 -- 172.21.15.86:0/4132907161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3148144a90 msgr2=0x7f31480c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3148144a90 0x7f31480c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 -- 172.21.15.86:0/4132907161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3148141e80 con 0x7f31480b1b80 2024-03-22T04:13:49.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f3145e00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31480b1b80 0x7f3148141940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f313800c4d0 tx=0x7f313800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f313e000640 1 -- 172.21.15.86:0/4132907161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138017070 con 0x7f31480b1b80 2024-03-22T04:13:49.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f313e000640 1 -- 172.21.15.86:0/4132907161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3138005cf0 con 0x7f31480b1b80 2024-03-22T04:13:49.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f313e000640 1 -- 172.21.15.86:0/4132907161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31380072b0 con 0x7f31480b1b80 2024-03-22T04:13:49.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 -- 172.21.15.86:0/4132907161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31480cad10 con 0x7f31480b1b80 2024-03-22T04:13:49.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.364+0000 7f314ca00640 1 -- 172.21.15.86:0/4132907161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31480cb2d0 con 0x7f31480b1b80 2024-03-22T04:13:49.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f313e000640 1 -- 172.21.15.86:0/4132907161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f313801a070 con 0x7f31480b1b80 2024-03-22T04:13:49.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f313e000640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3128067b20 0x7f3128069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f313e000640 1 -- 172.21.15.86:0/4132907161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3138085430 con 0x7f31480b1b80 2024-03-22T04:13:49.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f3145400640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3128067b20 0x7f3128069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f314ff73000 1 Processor -- start 2024-03-22T04:13:49.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f314ff73000 1 -- start start 2024-03-22T04:13:49.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f3145400640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3128067b20 0x7f3128069fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f31340ae000 tx=0x7f3134008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f314ff73000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d213f80 0x55ef3d214360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f314ff73000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d1c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f314ff73000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 0x55ef3d1c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f314ff73000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef3d018c40 con 0x55ef3d2148a0 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f314ff73000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef3cfb5d90 con 0x55ef3d213f80 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f314ff73000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ef3d014bf0 con 0x55ef3d1c7f50 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.368+0000 7f3145400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d1c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3146800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 0x55ef3d1c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d1c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51482/0 (socket says 172.21.15.86:51482) 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 -- 172.21.15.86:0/1823794237 learned_addr learned my addr 172.21.15.86:0/1823794237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145e00640 1 --2- 172.21.15.86:0/1823794237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d213f80 0x55ef3d214360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 -- 172.21.15.86:0/1823794237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 msgr2=0x55ef3d1c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 --2- 172.21.15.86:0/1823794237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 0x55ef3d1c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 -- 172.21.15.86:0/1823794237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d213f80 msgr2=0x55ef3d214360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 --2- 172.21.15.86:0/1823794237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d213f80 0x55ef3d214360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 -- 172.21.15.86:0/1823794237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3d0c0130 con 0x55ef3d2148a0 2024-03-22T04:13:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 --2- 172.21.15.86:0/1823794237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d1c7a10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f313401c520 tx=0x7f3134004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1e458b3e57f253 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3132a00640 1 -- 172.21.15.86:0/1823794237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340c8070 con 0x55ef3d2148a0 2024-03-22T04:13:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3132a00640 1 -- 172.21.15.86:0/1823794237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31340b87d0 con 0x55ef3d2148a0 2024-03-22T04:13:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3132a00640 1 -- 172.21.15.86:0/1823794237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340b1040 con 0x55ef3d2148a0 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- 172.21.15.86:0/1823794237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 msgr2=0x55ef3d1c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 --2- 172.21.15.86:0/1823794237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d1c7a10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f313401c520 tx=0x7f3134004740 comp rx=0 tx=0).stop 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- 172.21.15.86:0/1823794237 shutdown_connections 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 --2- 172.21.15.86:0/1823794237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 0x55ef3d1c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 --2- 172.21.15.86:0/1823794237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d1c7a10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 --2- 172.21.15.86:0/1823794237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d213f80 0x55ef3d214360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- 172.21.15.86:0/1823794237 >> 172.21.15.86:0/1823794237 conn(0x55ef3d0b1990 msgr2=0x55ef3d1f7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- 172.21.15.86:0/1823794237 shutdown_connections 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- 172.21.15.86:0/1823794237 wait complete. 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- start start 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 Processor -- start 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 0x55ef3d1cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d213f80 0x55ef3d1f8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 0x55ef3d1f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef3cfd4710 con 0x55ef3d213f80 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef3cfd6580 con 0x55ef3d2148a0 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ef3d01d270 con 0x55ef3d1c7f50 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f314ff73000 10 client.0 _ll_register_callbacks cb 0x55ef3d07a200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d213f80 0x55ef3d1f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d213f80 0x55ef3d1f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51498/0 (socket says 172.21.15.86:51498) 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 -- 172.21.15.86:0/1046587506 learned_addr learned my addr 172.21.15.86:0/1046587506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 0x55ef3d1cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3146800640 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 0x55ef3d1f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 -- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 msgr2=0x55ef3d1cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 0x55ef3d1cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 -- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 msgr2=0x55ef3d1f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 0x55ef3d1f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.372+0000 7f3145400640 1 -- 172.21.15.86:0/1046587506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3cff1d30 con 0x55ef3d213f80 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3145400640 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d213f80 0x55ef3d1f8380 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f313401c520 tx=0x7f313401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1de3ef3a478971e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 1 -- 172.21.15.86:0/1046587506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340d1060 con 0x55ef3d213f80 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 1 -- 172.21.15.86:0/1046587506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f31340cc660 con 0x55ef3d213f80 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 -- 172.21.15.86:0/1046587506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31340b87d0 con 0x55ef3d213f80 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 1 -- 172.21.15.86:0/1046587506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f31340d7050 con 0x55ef3d213f80 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 10 client.6447 fetch_fsmap learned FSMap version 35 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 -- 172.21.15.86:0/1046587506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ef3d05dfc0 con 0x55ef3d213f80 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 1 -- 172.21.15.86:0/1046587506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31340b87d0 con 0x55ef3d213f80 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 1 -- 172.21.15.86:0/1046587506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f31340da020 con 0x55ef3d213f80 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 10 client.6447 fetch_fsmap finished waiting for FSMap version 35 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 10 client.6447 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 20 client.6447 populate_metadata read hostname 'smithi086' 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55ef3d21a5c0 0x55ef3d21a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 4 client.6447 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:13:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 -- 172.21.15.86:0/1046587506 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef3cfb3dd0 con 0x55ef3d21a5c0 2024-03-22T04:13:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3145e00640 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55ef3d21a5c0 0x55ef3d21a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 10 client.6447 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3145e00640 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55ef3d21a5c0 0x55ef3d21a9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7b6682ff216f48 server_cookie=9be9ee4dc6933805 in_seq=0 out_seq=0 2024-03-22T04:13:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 1 -- 172.21.15.86:0/1046587506 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ef3cfb3dd0 con 0x55ef3d21a5c0 2024-03-22T04:13:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 10 client.6447 handle_command_reply: tid=0 2024-03-22T04:13:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 client.6447 shutdown 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 -- 172.21.15.86:0/1046587506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d213f80 msgr2=0x55ef3d1f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d213f80 0x55ef3d1f8380 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f313401c520 tx=0x7f313401ad20 comp rx=0 tx=0).stop 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 -- 172.21.15.86:0/1046587506 shutdown_connections 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55ef3d21a5c0 0x55ef3d21a9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 0x55ef3d1f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d213f80 0x55ef3d1f8380 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f3127400640 0 client.6447 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 --2- 172.21.15.86:0/1046587506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d1c7f50 0x55ef3d1cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 -- 172.21.15.86:0/1046587506 >> 172.21.15.86:0/1046587506 conn(0x55ef3d0b1990 msgr2=0x55ef3d1c7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 -- 172.21.15.86:0/1046587506 shutdown_connections 2024-03-22T04:13:49.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 1 -- 172.21.15.86:0/1046587506 wait complete. 2024-03-22T04:13:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.376+0000 7f314ff73000 20 client.6447 trim_cache size 0 max 16384 2024-03-22T04:13:49.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 Processor -- start 2024-03-22T04:13:49.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- start start 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d1c7f50 0x55ef3d22a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d2209f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 0x55ef3d2193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3148203b90 con 0x55ef3d1c7f50 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3148202c30 con 0x55ef3d2148a0 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f314815cd00 con 0x55ef3d213f80 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d1c7f50 0x55ef3d22a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d1c7f50 0x55ef3d22a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51506/0 (socket says 172.21.15.86:51506) 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 -- 172.21.15.86:0/210349813 learned_addr learned my addr 172.21.15.86:0/210349813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3146800640 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 0x55ef3d2193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145400640 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d2209f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 -- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 msgr2=0x55ef3d2209f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d2209f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 -- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 msgr2=0x55ef3d2193b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 0x55ef3d2193b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 -- 172.21.15.86:0/210349813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3cfee3a0 con 0x55ef3d1c7f50 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145400640 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d2209f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3145e00640 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d1c7f50 0x55ef3d22a6f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f313800ce90 tx=0x7f313801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b2e3d6724cf456 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3132000640 1 -- 172.21.15.86:0/210349813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31380850b0 con 0x55ef3d1c7f50 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3132000640 1 -- 172.21.15.86:0/210349813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3138080070 con 0x55ef3d1c7f50 2024-03-22T04:13:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f3132000640 1 -- 172.21.15.86:0/210349813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313805da00 con 0x55ef3d1c7f50 2024-03-22T04:13:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- 172.21.15.86:0/210349813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d1c7f50 msgr2=0x55ef3d22a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d1c7f50 0x55ef3d22a6f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f313800ce90 tx=0x7f313801e040 comp rx=0 tx=0).stop 2024-03-22T04:13:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- 172.21.15.86:0/210349813 shutdown_connections 2024-03-22T04:13:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d2148a0 0x55ef3d2193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d2209f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 --2- 172.21.15.86:0/210349813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d1c7f50 0x55ef3d22a6f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- 172.21.15.86:0/210349813 >> 172.21.15.86:0/210349813 conn(0x55ef3d0b1990 msgr2=0x55ef3d218ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- 172.21.15.86:0/210349813 shutdown_connections 2024-03-22T04:13:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- 172.21.15.86:0/210349813 wait complete. 2024-03-22T04:13:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 -- start start 2024-03-22T04:13:49.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.396+0000 7f314ff73000 1 Processor -- start 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d1c7f50 0x55ef3d283820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d283d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d21c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef3d0d4810 con 0x55ef3d2148a0 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f314815d350 con 0x55ef3d1c7f50 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ef3cfbf350 con 0x55ef3d213f80 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 10 client.0 _ll_register_callbacks cb 0x7f31340c6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d21c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d21c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51514/0 (socket says 172.21.15.86:51514) 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 -- 172.21.15.86:0/3585174060 learned_addr learned my addr 172.21.15.86:0/3585174060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3145e00640 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d1c7f50 0x55ef3d283820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3145400640 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d283d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 -- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 msgr2=0x55ef3d283d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d283d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 -- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d1c7f50 msgr2=0x55ef3d283820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d1c7f50 0x55ef3d283820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 -- 172.21.15.86:0/3585174060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3cf9e380 con 0x55ef3d2148a0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3146800640 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d21c9b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f31400169f0 tx=0x7f3140016eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f8768434796d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 1 -- 172.21.15.86:0/3585174060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140003b90 con 0x55ef3d2148a0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 1 -- 172.21.15.86:0/3585174060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3140010040 con 0x55ef3d2148a0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- 172.21.15.86:0/3585174060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3138080070 con 0x55ef3d2148a0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 1 -- 172.21.15.86:0/3585174060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3140028050 con 0x55ef3d2148a0 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 10 client.6453 fetch_fsmap learned FSMap version 35 2024-03-22T04:13:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- 172.21.15.86:0/3585174060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ef3d09a460 con 0x55ef3d2148a0 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 1 -- 172.21.15.86:0/3585174060 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3138080070 con 0x55ef3d2148a0 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 1 -- 172.21.15.86:0/3585174060 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f314002b020 con 0x55ef3d2148a0 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 10 client.6453 fetch_fsmap finished waiting for FSMap version 35 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 10 client.6453 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 20 client.6453 populate_metadata read hostname 'smithi086' 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55ef3d21cef0 0x55ef3d21d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 4 client.6453 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- 172.21.15.86:0/3585174060 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef3cfb8160 con 0x55ef3d21cef0 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3145e00640 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55ef3d21cef0 0x55ef3d21d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3145e00640 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55ef3d21cef0 0x55ef3d21d300 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e938233b26834d server_cookie=f77b72c935701abd in_seq=0 out_seq=0 2024-03-22T04:13:49.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 10 client.6453 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 1 -- 172.21.15.86:0/3585174060 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x55ef3cfb8160 con 0x55ef3d21cef0 2024-03-22T04:13:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 10 client.6453 handle_command_reply: tid=0 2024-03-22T04:13:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 client.6453 shutdown 2024-03-22T04:13:49.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- 172.21.15.86:0/3585174060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 msgr2=0x55ef3d21c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d21c9b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f31400169f0 tx=0x7f3140016eb0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- 172.21.15.86:0/3585174060 shutdown_connections 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55ef3d21cef0 0x55ef3d21d300 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3d2148a0 0x55ef3d21c9b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ef3d213f80 0x55ef3d283d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 --2- 172.21.15.86:0/3585174060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef3d1c7f50 0x55ef3d283820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f3127400640 0 client.6453 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- 172.21.15.86:0/3585174060 >> 172.21.15.86:0/3585174060 conn(0x55ef3d0b1990 msgr2=0x55ef3d1f7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- 172.21.15.86:0/3585174060 shutdown_connections 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.400+0000 7f314ff73000 1 -- 172.21.15.86:0/3585174060 wait complete. 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ff73000 20 client.6453 trim_cache size 0 max 16384 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:13:49.409 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:13:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:13:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: "842888e6-e588-455c-a795-7bf2b70daa21": { 2024-03-22T04:13:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T04:13:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "842888e6-e588-455c-a795-7bf2b70daa21", 2024-03-22T04:13:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive" 2024-03-22T04:13:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:13:49.410 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 -- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3128067b20 msgr2=0x7f3128069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3128067b20 0x7f3128069fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f31340ae000 tx=0x7f3134008040 comp rx=0 tx=0).stop 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 -- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31480b1b80 msgr2=0x7f3148141940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31480b1b80 0x7f3148141940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f313800c4d0 tx=0x7f313800c990 comp rx=0 tx=0).stop 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 -- 172.21.15.86:0/4132907161 shutdown_connections 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3128067b20 0x7f3128069fe0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3148144a90 0x7f31480c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31480cdf10 0x7f31480ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 --2- 172.21.15.86:0/4132907161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31480b1b80 0x7f3148141940 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 -- 172.21.15.86:0/4132907161 >> 172.21.15.86:0/4132907161 conn(0x7f31480cc940 msgr2=0x7f31480afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 -- 172.21.15.86:0/4132907161 shutdown_connections 2024-03-22T04:13:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:13:49.404+0000 7f314ca00640 1 -- 172.21.15.86:0/4132907161 wait complete. 2024-03-22T04:13:49.424 INFO:tasks.cephfs.filesystem:scrub status for tag:842888e6-e588-455c-a795-7bf2b70daa21 - {'path': '/', 'tag': '842888e6-e588-455c-a795-7bf2b70daa21', 'options': 'recursive'} 2024-03-22T04:13:52.915 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:13:52.912+0000 7f3dd0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-22T04:13:57.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:13:57.732 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:14:19.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:14:19.426 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-22T04:14:19.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040ec00640 1 Processor -- start 2024-03-22T04:14:19.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040ec00640 1 -- start start 2024-03-22T04:14:19.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f040815c730 0x7f040809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040ec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04080b11c0 0x7f04080b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040ec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04080cda00 con 0x7f04080b3610 2024-03-22T04:14:19.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040ec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f040809b910 con 0x7f04080b11c0 2024-03-22T04:14:19.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f04080b1710 con 0x7f040815c730 2024-03-22T04:14:19.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04080b11c0 0x7f04080b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04080b11c0 0x7f04080b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42914/0 (socket says 172.21.15.86:42914) 2024-03-22T04:14:19.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f0407400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f04080b3610 0x7f040809afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55956/0 (socket says 172.21.15.86:55956) 2024-03-22T04:14:19.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040cc00640 1 -- 172.21.15.86:0/2566217153 learned_addr learned my addr 172.21.15.86:0/2566217153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:19.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f0407e00640 1 --2- 172.21.15.86:0/2566217153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f040815c730 0x7f040809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040cc00640 1 -- 172.21.15.86:0/2566217153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f040815c730 msgr2=0x7f040809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040cc00640 1 --2- 172.21.15.86:0/2566217153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f040815c730 0x7f040809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040cc00640 1 -- 172.21.15.86:0/2566217153 >> v1:172.21.15.86:6789/0 conn(0x7f04080b3610 legacy=0x7f040809afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f040cc00640 1 -- 172.21.15.86:0/2566217153 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04080cdce0 con 0x7f04080b11c0 2024-03-22T04:14:19.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.572+0000 7f0406a00640 1 -- 172.21.15.86:0/2566217153 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3823937232 0 0) 0x7f04080cda00 con 0x7f04080b3610 2024-03-22T04:14:19.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040cc00640 1 --2- 172.21.15.86:0/2566217153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f04080b1ba0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0400009870 tx=0x7f040001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc807c0afca0eed9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f0406a00640 1 -- 172.21.15.86:0/2566217153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000b1070 con 0x7f04080b11c0 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f0406a00640 1 -- 172.21.15.86:0/2566217153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04000a8050 con 0x7f04080b11c0 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f0406a00640 1 -- 172.21.15.86:0/2566217153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000ac860 con 0x7f04080b11c0 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040ec00640 1 -- 172.21.15.86:0/2566217153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 msgr2=0x7f04080b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040ec00640 1 --2- 172.21.15.86:0/2566217153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f04080b1ba0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0400009870 tx=0x7f040001c4d0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040ec00640 1 -- 172.21.15.86:0/2566217153 shutdown_connections 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040ec00640 1 --2- 172.21.15.86:0/2566217153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f04080b1ba0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040ec00640 1 --2- 172.21.15.86:0/2566217153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f040815c730 0x7f040809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040ec00640 1 -- 172.21.15.86:0/2566217153 >> 172.21.15.86:0/2566217153 conn(0x7f04080b3bf0 msgr2=0x7f04080b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040ec00640 1 -- 172.21.15.86:0/2566217153 shutdown_connections 2024-03-22T04:14:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.576+0000 7f040ec00640 1 -- 172.21.15.86:0/2566217153 wait complete. 2024-03-22T04:14:19.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 Processor -- start 2024-03-22T04:14:19.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 -- start start 2024-03-22T04:14:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f0408154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04080b3610 0x7f040815a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040815c730 0x7f0408153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04080b2cc0 con 0x7f04080b11c0 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04080b2b40 con 0x7f040815c730 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f04080b2e40 con 0x7f04080b3610 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f0408154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f0408154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42928/0 (socket says 172.21.15.86:42928) 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 -- 172.21.15.86:0/3746909428 learned_addr learned my addr 172.21.15.86:0/3746909428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407400640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04080b3610 0x7f040815a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040cc00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040815c730 0x7f0408153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 -- 172.21.15.86:0/3746909428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04080b3610 msgr2=0x7f040815a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04080b3610 0x7f040815a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 -- 172.21.15.86:0/3746909428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040815c730 msgr2=0x7f0408153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040815c730 0x7f0408153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 -- 172.21.15.86:0/3746909428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04080cbca0 con 0x7f04080b11c0 2024-03-22T04:14:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f0407e00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f0408154760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f03f800efc0 tx=0x7f03f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f03ffe00640 1 -- 172.21.15.86:0/3746909428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8018020 con 0x7f04080b11c0 2024-03-22T04:14:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f03ffe00640 1 -- 172.21.15.86:0/3746909428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f8014480 con 0x7f04080b11c0 2024-03-22T04:14:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 -- 172.21.15.86:0/3746909428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04080cbf80 con 0x7f04080b11c0 2024-03-22T04:14:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f03ffe00640 1 -- 172.21.15.86:0/3746909428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8014870 con 0x7f04080b11c0 2024-03-22T04:14:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.580+0000 7f040ec00640 1 -- 172.21.15.86:0/3746909428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040815f660 con 0x7f04080b11c0 2024-03-22T04:14:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f03ffe00640 1 -- 172.21.15.86:0/3746909428 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f03f801b020 con 0x7f04080b11c0 2024-03-22T04:14:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f03ffe00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f03e8067b20 0x7f03e8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f03ffe00640 1 -- 172.21.15.86:0/3746909428 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f03f8086af0 con 0x7f04080b11c0 2024-03-22T04:14:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f0407400640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f03e8067b20 0x7f03e8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f04120fa000 1 Processor -- start 2024-03-22T04:14:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f04120fa000 1 -- start start 2024-03-22T04:14:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f0407400640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f03e8067b20 0x7f03e8069fe0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0408154dd0 tx=0x7f03f4007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f04120fa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b71f80 0x55e041b72360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f04120fa000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b728a0 0x55e041b25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f04120fa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 0x55e041b26330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f04120fa000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e041976c40 con 0x55e041b728a0 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f04120fa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e041913d90 con 0x55e041b71f80 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f04120fa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e041972bf0 con 0x55e041b25f50 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f0407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b728a0 0x55e041b25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f0407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b728a0 0x55e041b25a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42942/0 (socket says 172.21.15.86:42942) 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f0407400640 1 -- 172.21.15.86:0/3714763625 learned_addr learned my addr 172.21.15.86:0/3714763625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f0407e00640 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b71f80 0x55e041b72360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.584+0000 7f040cc00640 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 0x55e041b26330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 -- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 msgr2=0x55e041b26330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 0x55e041b26330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 -- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b71f80 msgr2=0x55e041b72360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b71f80 0x55e041b72360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 -- 172.21.15.86:0/3714763625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e041a1e130 con 0x55e041b728a0 2024-03-22T04:14:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407e00640 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b71f80 0x55e041b72360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b728a0 0x55e041b25a10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f03f4004fd0 tx=0x7f03f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df3bb5b615561eb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f03fcc00640 1 -- 172.21.15.86:0/3714763625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4026070 con 0x55e041b728a0 2024-03-22T04:14:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f03fcc00640 1 -- 172.21.15.86:0/3714763625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4013970 con 0x55e041b728a0 2024-03-22T04:14:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f03fcc00640 1 -- 172.21.15.86:0/3714763625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4013d60 con 0x55e041b728a0 2024-03-22T04:14:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- 172.21.15.86:0/3714763625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b728a0 msgr2=0x55e041b25a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b728a0 0x55e041b25a10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f03f4004fd0 tx=0x7f03f4012000 comp rx=0 tx=0).stop 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- 172.21.15.86:0/3714763625 shutdown_connections 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 0x55e041b26330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b728a0 0x55e041b25a10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 --2- 172.21.15.86:0/3714763625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b71f80 0x55e041b72360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- 172.21.15.86:0/3714763625 >> 172.21.15.86:0/3714763625 conn(0x55e041a0f990 msgr2=0x55e041b55150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- 172.21.15.86:0/3714763625 shutdown_connections 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- 172.21.15.86:0/3714763625 wait complete. 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- start start 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 Processor -- start 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 0x55e041b2da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b71f80 0x55e041b56380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b728a0 0x55e041b568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e041932710 con 0x55e041b71f80 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e041934580 con 0x55e041b728a0 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e04197b270 con 0x55e041b25f50 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f04120fa000 10 client.0 _ll_register_callbacks cb 0x55e0419d8200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b71f80 0x55e041b56380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b71f80 0x55e041b56380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42952/0 (socket says 172.21.15.86:42952) 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 -- 172.21.15.86:0/1458679826 learned_addr learned my addr 172.21.15.86:0/1458679826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407e00640 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 0x55e041b2da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f040cc00640 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b728a0 0x55e041b568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 -- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 msgr2=0x55e041b2da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 0x55e041b2da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 -- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b728a0 msgr2=0x55e041b568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b728a0 0x55e041b568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.588+0000 7f0407400640 1 -- 172.21.15.86:0/1458679826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e04194fd30 con 0x55e041b71f80 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f0407400640 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b71f80 0x55e041b56380 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55e04197e0c0 tx=0x7f03f4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bb590810b885ef server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 1 -- 172.21.15.86:0/1458679826 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4030060 con 0x55e041b71f80 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 1 -- 172.21.15.86:0/1458679826 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f03f4020a60 con 0x55e041b71f80 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 -- 172.21.15.86:0/1458679826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f4013970 con 0x55e041b71f80 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 1 -- 172.21.15.86:0/1458679826 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f03f4026070 con 0x55e041b71f80 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 10 client.6468 fetch_fsmap learned FSMap version 36 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 -- 172.21.15.86:0/1458679826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55e0419bbfc0 con 0x55e041b71f80 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 1 -- 172.21.15.86:0/1458679826 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4013970 con 0x55e041b71f80 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 1 -- 172.21.15.86:0/1458679826 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f03f4020c60 con 0x55e041b71f80 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 10 client.6468 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 10 client.6468 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 20 client.6468 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e041b78630 0x55e041b78a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 4 client.6468 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:14:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 -- 172.21.15.86:0/1458679826 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e041911dd0 con 0x55e041b78630 2024-03-22T04:14:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f0407e00640 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e041b78630 0x55e041b78a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f0407e00640 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e041b78630 0x55e041b78a40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ba6e28a642bc025 server_cookie=e53afb257194869c in_seq=0 out_seq=0 2024-03-22T04:14:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 10 client.6468 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 1 -- 172.21.15.86:0/1458679826 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e041911dd0 con 0x55e041b78630 2024-03-22T04:14:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 10 client.6468 handle_command_reply: tid=0 2024-03-22T04:14:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 client.6468 shutdown 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 -- 172.21.15.86:0/1458679826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b71f80 msgr2=0x55e041b56380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b71f80 0x55e041b56380 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55e04197e0c0 tx=0x7f03f4000ee0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 -- 172.21.15.86:0/1458679826 shutdown_connections 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e041b78630 0x55e041b78a40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041b728a0 0x55e041b568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b71f80 0x55e041b56380 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 --2- 172.21.15.86:0/1458679826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b25f50 0x55e041b2da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f03f1600640 0 client.6468 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 -- 172.21.15.86:0/1458679826 >> 172.21.15.86:0/1458679826 conn(0x55e041a0f990 msgr2=0x55e041b25110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 -- 172.21.15.86:0/1458679826 shutdown_connections 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 1 -- 172.21.15.86:0/1458679826 wait complete. 2024-03-22T04:14:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.592+0000 7f04120fa000 20 client.6468 trim_cache size 0 max 16384 2024-03-22T04:14:19.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 Processor -- start 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- start start 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b88970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041b2c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041b2ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0408201ff0 con 0x55e041b25f50 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0408200fd0 con 0x55e041a0f990 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0408141b50 con 0x55e041b71f80 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041b2c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041b2c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42964/0 (socket says 172.21.15.86:42964) 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 -- 172.21.15.86:0/2370128540 learned_addr learned my addr 172.21.15.86:0/2370128540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f040cc00640 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041b2ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407e00640 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b88970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 -- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 msgr2=0x55e041b2ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041b2ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 -- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 msgr2=0x55e041b88970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b88970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 -- 172.21.15.86:0/2370128540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e04194c3a0 con 0x55e041b25f50 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407e00640 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b88970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f040cc00640 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041b2ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f0407400640 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041b2c8d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f03f4004fd0 tx=0x7f03f4029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eedfd0939496fd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f03f3e00640 1 -- 172.21.15.86:0/2370128540 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4042070 con 0x55e041b25f50 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f03f3e00640 1 -- 172.21.15.86:0/2370128540 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4027040 con 0x55e041b25f50 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f03f3e00640 1 -- 172.21.15.86:0/2370128540 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4020a60 con 0x55e041b25f50 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- 172.21.15.86:0/2370128540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 msgr2=0x55e041b2c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041b2c8d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f03f4004fd0 tx=0x7f03f4029000 comp rx=0 tx=0).stop 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- 172.21.15.86:0/2370128540 shutdown_connections 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041b2ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041b2c8d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 --2- 172.21.15.86:0/2370128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b88970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- 172.21.15.86:0/2370128540 >> 172.21.15.86:0/2370128540 conn(0x55e041b55400 msgr2=0x55e041b2c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- 172.21.15.86:0/2370128540 shutdown_connections 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- 172.21.15.86:0/2370128540 wait complete. 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.612+0000 7f04120fa000 1 -- start start 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 Processor -- start 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041ba1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041ba20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e041a32810 con 0x55e041b25f50 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04081421a0 con 0x55e041a0f990 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e041975f50 con 0x55e041b71f80 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 10 client.0 _ll_register_callbacks cb 0x7f03f4024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041ba1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041ba1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42976/0 (socket says 172.21.15.86:42976) 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 -- 172.21.15.86:0/4228902761 learned_addr learned my addr 172.21.15.86:0/4228902761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407e00640 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f040cc00640 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041ba20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 -- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 msgr2=0x55e041ba20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041ba20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 -- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 msgr2=0x55e041b885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 -- 172.21.15.86:0/4228902761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0418fc380 con 0x55e041b25f50 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407400640 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041ba1bb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f03f4022010 tx=0x7f03f400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48bd014bbe7ab75 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 1 -- 172.21.15.86:0/4228902761 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4042070 con 0x55e041b25f50 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 1 -- 172.21.15.86:0/4228902761 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f03f4049d10 con 0x55e041b25f50 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 -- 172.21.15.86:0/4228902761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f4027040 con 0x55e041b25f50 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 1 -- 172.21.15.86:0/4228902761 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f03f4050050 con 0x55e041b25f50 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 10 client.6474 fetch_fsmap learned FSMap version 36 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 -- 172.21.15.86:0/4228902761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55e0419f8460 con 0x55e041b25f50 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 1 -- 172.21.15.86:0/4228902761 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4027040 con 0x55e041b25f50 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 1 -- 172.21.15.86:0/4228902761 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f03f4053020 con 0x55e041b25f50 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 10 client.6474 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T04:14:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 10 client.6474 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 20 client.6474 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e041b728a0 0x55e041be55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 4 client.6474 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 -- 172.21.15.86:0/4228902761 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e041916160 con 0x55e041b728a0 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407e00640 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e041b728a0 0x55e041be55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f0407e00640 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e041b728a0 0x55e041be55e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67257d1a0004efb server_cookie=c8b1f7ad15327d5c in_seq=0 out_seq=0 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 10 client.6474 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 1 -- 172.21.15.86:0/4228902761 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e041916160 con 0x55e041b728a0 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f03f1600640 10 client.6474 handle_command_reply: tid=0 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.616+0000 7f04120fa000 1 client.6474 shutdown 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 -- 172.21.15.86:0/4228902761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 msgr2=0x55e041ba1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041ba1bb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f03f4022010 tx=0x7f03f400c000 comp rx=0 tx=0).stop 2024-03-22T04:14:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 -- 172.21.15.86:0/4228902761 shutdown_connections 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e041b728a0 0x55e041be55e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e041b71f80 0x55e041ba20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e041b25f50 0x55e041ba1bb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 --2- 172.21.15.86:0/4228902761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e041a0f990 0x55e041b885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f03f1600640 0 client.6474 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 -- 172.21.15.86:0/4228902761 >> 172.21.15.86:0/4228902761 conn(0x55e041b55400 msgr2=0x55e041b2ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 -- 172.21.15.86:0/4228902761 shutdown_connections 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 1 -- 172.21.15.86:0/4228902761 wait complete. 2024-03-22T04:14:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f04120fa000 20 client.6474 trim_cache size 0 max 16384 2024-03-22T04:14:19.634 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:14:19.634 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:14:19.634 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:14:19.634 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:14:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 -- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f03e8067b20 msgr2=0x7f03e8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f03e8067b20 0x7f03e8069fe0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0408154dd0 tx=0x7f03f4007df0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 -- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 msgr2=0x7f0408154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f0408154760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f03f800efc0 tx=0x7f03f800c490 comp rx=0 tx=0).stop 2024-03-22T04:14:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 -- 172.21.15.86:0/3746909428 shutdown_connections 2024-03-22T04:14:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f03e8067b20 0x7f03e8069fe0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040815c730 0x7f0408153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04080b3610 0x7f040815a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 --2- 172.21.15.86:0/3746909428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04080b11c0 0x7f0408154760 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 -- 172.21.15.86:0/3746909428 >> 172.21.15.86:0/3746909428 conn(0x7f04080b3bf0 msgr2=0x7f04080aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 -- 172.21.15.86:0/3746909428 shutdown_connections 2024-03-22T04:14:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.620+0000 7f040ec00640 1 -- 172.21.15.86:0/3746909428 wait complete. 2024-03-22T04:14:19.642 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:14:19.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:14:19.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87f4e00640 1 Processor -- start 2024-03-22T04:14:19.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87f4e00640 1 -- start start 2024-03-22T04:14:19.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87f4e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f015bc80 0x7f87f006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87f4e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f87f006dc40 0x7f87f006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87f4e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f015c550 con 0x7f87f015ac10 2024-03-22T04:14:19.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87f4e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f87f015cb60 con 0x7f87f006dc40 2024-03-22T04:14:19.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87f4e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f87f00b4750 con 0x7f87f015bc80 2024-03-22T04:14:19.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87ede00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f87f015ac10 0x7f87f00b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55972/0 (socket says 172.21.15.86:55972) 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87ede00640 1 -- 172.21.15.86:0/3157802272 learned_addr learned my addr 172.21.15.86:0/3157802272 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87e5400640 1 --2- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f015bc80 0x7f87f006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.776+0000 7f87ee800640 1 --2- 172.21.15.86:0/3157802272 >> v2:172.21.15.196:3300/0 conn(0x7f87f006dc40 0x7f87f006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ed400640 1 -- 172.21.15.86:0/3157802272 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060620189 0 0) 0x7f87f015c550 con 0x7f87f015ac10 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ed400640 1 -- 172.21.15.86:0/3157802272 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87d4003530 con 0x7f87f015ac10 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ee800640 1 -- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f015bc80 msgr2=0x7f87f006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ee800640 1 --2- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f015bc80 0x7f87f006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ee800640 1 -- 172.21.15.86:0/3157802272 >> v1:172.21.15.86:6789/0 conn(0x7f87f015ac10 legacy=0x7f87f00b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ee800640 1 -- 172.21.15.86:0/3157802272 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f00b4470 con 0x7f87f006dc40 2024-03-22T04:14:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87e5400640 1 --2- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f015bc80 0x7f87f006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ee800640 1 --2- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f006dc40 0x7f87f006e0a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009d90 tx=0x7f87e801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d94dd7c3a7c991 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ed400640 1 -- 172.21.15.86:0/3157802272 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80b2070 con 0x7f87f006dc40 2024-03-22T04:14:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ed400640 1 -- 172.21.15.86:0/3157802272 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e80a5a20 con 0x7f87f006dc40 2024-03-22T04:14:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.780+0000 7f87ed400640 1 -- 172.21.15.86:0/3157802272 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80a5dd0 con 0x7f87f006dc40 2024-03-22T04:14:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f006dc40 msgr2=0x7f87f006e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 --2- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f006dc40 0x7f87f006e0a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009d90 tx=0x7f87e801c940 comp rx=0 tx=0).stop 2024-03-22T04:14:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- 172.21.15.86:0/3157802272 shutdown_connections 2024-03-22T04:14:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 --2- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f006dc40 0x7f87f006e0a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 --2- 172.21.15.86:0/3157802272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f015bc80 0x7f87f006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- 172.21.15.86:0/3157802272 >> 172.21.15.86:0/3157802272 conn(0x7f87f00b1df0 msgr2=0x7f87f00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- 172.21.15.86:0/3157802272 shutdown_connections 2024-03-22T04:14:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- 172.21.15.86:0/3157802272 wait complete. 2024-03-22T04:14:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 Processor -- start 2024-03-22T04:14:19.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- start start 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f006dc40 0x7f87f0156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f015ac10 0x7f87f014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f015bc80 0x7f87f01548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0168390 con 0x7f87f015ac10 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87f015e310 con 0x7f87f015bc80 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87f4e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f87f0168510 con 0x7f87f006dc40 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f015ac10 0x7f87f014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f015ac10 0x7f87f014f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42980/0 (socket says 172.21.15.86:42980) 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 -- 172.21.15.86:0/2626448886 learned_addr learned my addr 172.21.15.86:0/2626448886 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87ede00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f006dc40 0x7f87f0156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87ee800640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f015bc80 0x7f87f01548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 -- 172.21.15.86:0/2626448886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f006dc40 msgr2=0x7f87f0156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f006dc40 0x7f87f0156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 -- 172.21.15.86:0/2626448886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f015bc80 msgr2=0x7f87f01548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f015bc80 0x7f87f01548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 -- 172.21.15.86:0/2626448886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f01551c0 con 0x7f87f015ac10 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.784+0000 7f87e5400640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f015ac10 0x7f87f014f6a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f87d800e980 tx=0x7f87d800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87e6a00640 1 -- 172.21.15.86:0/2626448886 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8017070 con 0x7f87f015ac10 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87e6a00640 1 -- 172.21.15.86:0/2626448886 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d800ca00 con 0x7f87f015ac10 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87e6a00640 1 -- 172.21.15.86:0/2626448886 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d800cdd0 con 0x7f87f015ac10 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f01554a0 con 0x7f87f015ac10 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f0155a60 con 0x7f87f015ac10 2024-03-22T04:14:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f0068650 con 0x7f87f015ac10 2024-03-22T04:14:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87e6a00640 1 -- 172.21.15.86:0/2626448886 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f87d801a070 con 0x7f87f015ac10 2024-03-22T04:14:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87e6a00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f87cc067b20 0x7f87cc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87e6a00640 1 -- 172.21.15.86:0/2626448886 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f87d8086370 con 0x7f87f015ac10 2024-03-22T04:14:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87ede00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f87cc067b20 0x7f87cc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.788+0000 7f87ede00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f87cc067b20 0x7f87cc069fe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87e0007fd0 tx=0x7f87e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:19.792+0000 7f87e6a00640 1 -- 172.21.15.86:0/2626448886 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f87d8056460 con 0x7f87f015ac10 2024-03-22T04:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.000+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87f00b39e0 con 0x7f87f015ac10 2024-03-22T04:14:20.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.000+0000 7f87e6a00640 1 -- 172.21.15.86:0/2626448886 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5185 (secure 0 0 0) 0x7f87d805a500 con 0x7f87f015ac10 2024-03-22T04:14:20.005 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:20.005 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:13:52.561205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-03-22T04:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f87cc067b20 msgr2=0x7f87cc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f87cc067b20 0x7f87cc069fe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87e0007fd0 tx=0x7f87e0009040 comp rx=0 tx=0).stop 2024-03-22T04:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f015ac10 msgr2=0x7f87f014f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f015ac10 0x7f87f014f6a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f87d800e980 tx=0x7f87d800ee40 comp rx=0 tx=0).stop 2024-03-22T04:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 shutdown_connections 2024-03-22T04:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f87cc067b20 0x7f87cc069fe0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87f015bc80 0x7f87f01548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f015ac10 0x7f87f014f6a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 --2- 172.21.15.86:0/2626448886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87f006dc40 0x7f87f0156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 >> 172.21.15.86:0/2626448886 conn(0x7f87f00b1df0 msgr2=0x7f87f015cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 shutdown_connections 2024-03-22T04:14:20.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.004+0000 7f87f4e00640 1 -- 172.21.15.86:0/2626448886 wait complete. 2024-03-22T04:14:20.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-03-22T04:14:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bd1400640 1 Processor -- start 2024-03-22T04:14:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bd1400640 1 -- start start 2024-03-22T04:14:20.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bd1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2bcc0c6930 0x7f2bcc0c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bd1400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc0c7280 con 0x7f2bcc0af910 2024-03-22T04:14:20.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bd1400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc15c770 con 0x7f2bcc15c390 2024-03-22T04:14:20.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bd1400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2bcc0cb7a0 con 0x7f2bcc0c6930 2024-03-22T04:14:20.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bcb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2bcc0c6930 0x7f2bcc0c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bc9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2bcc0af910 0x7f2bcc06df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55978/0 (socket says 172.21.15.86:55978) 2024-03-22T04:14:20.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bc9e00640 1 -- 172.21.15.86:0/466532157 learned_addr learned my addr 172.21.15.86:0/466532157 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bcb200640 1 -- 172.21.15.86:0/466532157 >> v1:172.21.15.196:6789/0 conn(0x7f2bcc15c390 legacy=0x7f2bcc160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bcb200640 1 -- 172.21.15.86:0/466532157 >> v1:172.21.15.86:6789/0 conn(0x7f2bcc0af910 legacy=0x7f2bcc06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.160+0000 7f2bcb200640 1 -- 172.21.15.86:0/466532157 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc0cba80 con 0x7f2bcc0c6930 2024-03-22T04:14:20.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bcb200640 1 --2- 172.21.15.86:0/466532157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc0c6d40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009d90 tx=0x7f2bc401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6f592dbed3c512 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bc9400640 1 -- 172.21.15.86:0/466532157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40b2070 con 0x7f2bcc0c6930 2024-03-22T04:14:20.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bc9400640 1 -- 172.21.15.86:0/466532157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc40a5a20 con 0x7f2bcc0c6930 2024-03-22T04:14:20.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bc9400640 1 -- 172.21.15.86:0/466532157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40a5df0 con 0x7f2bcc0c6930 2024-03-22T04:14:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 -- 172.21.15.86:0/466532157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 msgr2=0x7f2bcc0c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 --2- 172.21.15.86:0/466532157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc0c6d40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009d90 tx=0x7f2bc401c940 comp rx=0 tx=0).stop 2024-03-22T04:14:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 -- 172.21.15.86:0/466532157 shutdown_connections 2024-03-22T04:14:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 --2- 172.21.15.86:0/466532157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc0c6d40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 -- 172.21.15.86:0/466532157 >> 172.21.15.86:0/466532157 conn(0x7f2bcc0d0730 msgr2=0x7f2bcc0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 -- 172.21.15.86:0/466532157 shutdown_connections 2024-03-22T04:14:20.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 -- 172.21.15.86:0/466532157 wait complete. 2024-03-22T04:14:20.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 Processor -- start 2024-03-22T04:14:20.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.164+0000 7f2bd1400640 1 -- start start 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bd1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bcc0af910 0x7f2bcc15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bd1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bd1400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bcc15c390 0x7f2bcc153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bd1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc072d90 con 0x7f2bcc0c6930 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bd1400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc06e630 con 0x7f2bcc0af910 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bd1400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2bcc072f10 con 0x7f2bcc15c390 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43004/0 (socket says 172.21.15.86:43004) 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 -- 172.21.15.86:0/1404293829 learned_addr learned my addr 172.21.15.86:0/1404293829 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bca800640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bcc0af910 0x7f2bcc15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bcb200640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bcc15c390 0x7f2bcc153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 -- 172.21.15.86:0/1404293829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bcc15c390 msgr2=0x7f2bcc153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bcc15c390 0x7f2bcc153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 -- 172.21.15.86:0/1404293829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bcc0af910 msgr2=0x7f2bcc15dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bcc0af910 0x7f2bcc15dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 -- 172.21.15.86:0/1404293829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc15d980 con 0x7f2bcc0c6930 2024-03-22T04:14:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc152d20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c400 tx=0x7f2bb800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc2a00640 1 -- 172.21.15.86:0/1404293829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8017070 con 0x7f2bcc0c6930 2024-03-22T04:14:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc2a00640 1 -- 172.21.15.86:0/1404293829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb8005cf0 con 0x7f2bcc0c6930 2024-03-22T04:14:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc2a00640 1 -- 172.21.15.86:0/1404293829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80072d0 con 0x7f2bcc0c6930 2024-03-22T04:14:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bd1400640 1 -- 172.21.15.86:0/1404293829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc15cbc0 con 0x7f2bcc0c6930 2024-03-22T04:14:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bd1400640 1 -- 172.21.15.86:0/1404293829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc15d180 con 0x7f2bcc0c6930 2024-03-22T04:14:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.168+0000 7f2bc2a00640 1 -- 172.21.15.86:0/1404293829 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2bb801a070 con 0x7f2bcc0c6930 2024-03-22T04:14:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bc2a00640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2bac067b20 0x7f2bac069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bc2a00640 1 -- 172.21.15.86:0/1404293829 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f2bb8085490 con 0x7f2bcc0c6930 2024-03-22T04:14:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2bac067b20 0x7f2bac069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bd47c1000 1 Processor -- start 2024-03-22T04:14:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bd47c1000 1 -- start start 2024-03-22T04:14:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2bac067b20 0x7f2bac069fe0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc001810 tx=0x7f2bbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 0x559322691c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x559322692550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593224dcd80 con 0x5593226db750 2024-03-22T04:14:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559322479ce0 con 0x5593226dc070 2024-03-22T04:14:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5593224d8d30 con 0x559322692170 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226dbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43010/0 (socket says 172.21.15.86:43010) 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 -- 172.21.15.86:0/2547701743 learned_addr learned my addr 172.21.15.86:0/2547701743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bcb200640 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x559322692550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 0x559322691c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 -- 172.21.15.86:0/2547701743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 msgr2=0x559322692550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x559322692550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 -- 172.21.15.86:0/2547701743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 msgr2=0x559322691c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 0x559322691c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 -- 172.21.15.86:0/2547701743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593224b7340 con 0x5593226db750 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bca800640 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226dbb30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00fa90 tx=0x7f2bbc000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b9b41862d6f292 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bb7400640 1 -- 172.21.15.86:0/2547701743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc025070 con 0x5593226db750 2024-03-22T04:14:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bb7400640 1 -- 172.21.15.86:0/2547701743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bbc015bf0 con 0x5593226db750 2024-03-22T04:14:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.172+0000 7f2bb7400640 1 -- 172.21.15.86:0/2547701743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc020400 con 0x5593226db750 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- 172.21.15.86:0/2547701743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 msgr2=0x5593226dbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226dbb30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00fa90 tx=0x7f2bbc000ee0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- 172.21.15.86:0/2547701743 shutdown_connections 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 0x559322691c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x559322692550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/2547701743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226dbb30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- 172.21.15.86:0/2547701743 >> 172.21.15.86:0/2547701743 conn(0x559322689d10 msgr2=0x55932268c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- 172.21.15.86:0/2547701743 shutdown_connections 2024-03-22T04:14:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- 172.21.15.86:0/2547701743 wait complete. 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- start start 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 Processor -- start 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x5593226e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 0x5593226c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55932249a4e0 con 0x5593226db750 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559322689b90 con 0x5593226dc070 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559322483230 con 0x559322692170 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 10 client.0 _ll_register_callbacks cb 0x559322678c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226e2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43012/0 (socket says 172.21.15.86:43012) 2024-03-22T04:14:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bc9e00640 1 -- 172.21.15.86:0/187100552 learned_addr learned my addr 172.21.15.86:0/187100552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bca800640 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x5593226e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bcb200640 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 0x5593226c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bca800640 1 -- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 msgr2=0x5593226c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bca800640 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 0x5593226c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bca800640 1 -- 172.21.15.86:0/187100552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 msgr2=0x5593226e2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bca800640 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226e2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bca800640 1 -- 172.21.15.86:0/187100552 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5593225215e0 con 0x559322692170 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226e2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bca800640 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x5593226e2ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc022000 tx=0x7f2bbc000bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20dcae066cff82a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2babe00640 10 client.0 ms_handle_connect on v2:172.21.15.196:3301/0 2024-03-22T04:14:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2babe00640 1 -- 172.21.15.86:0/187100552 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc02e070 con 0x559322692170 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2babe00640 1 -- 172.21.15.86:0/187100552 <== mon.2 v2:172.21.15.196:3301/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f2bbc020de0 con 0x559322692170 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- 172.21.15.86:0/187100552 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bbc015bf0 con 0x559322692170 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2babe00640 1 -- 172.21.15.86:0/187100552 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2bbc039050 con 0x559322692170 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 10 client.75821 fetch_fsmap learned FSMap version 36 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 -- 172.21.15.86:0/187100552 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5593224985a0 con 0x559322692170 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2babe00640 1 -- 172.21.15.86:0/187100552 <== mon.2 v2:172.21.15.196:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bbc015bf0 con 0x559322692170 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2babe00640 1 -- 172.21.15.86:0/187100552 <== mon.2 v2:172.21.15.196:3301/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f2bbc03c020 con 0x559322692170 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 10 client.75821 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 10 client.75821 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 20 client.75821 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.176+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5593226ca0c0 0x5593226ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 4 client.75821 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:14:20.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 -- 172.21.15.86:0/187100552 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559322477e40 con 0x5593226ca0c0 2024-03-22T04:14:20.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5593226ca0c0 0x5593226ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5593226ca0c0 0x5593226ca4a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28dd4fb40fc2d33f server_cookie=192a051b46c2e7ea in_seq=0 out_seq=0 2024-03-22T04:14:20.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2babe00640 10 client.75821 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2babe00640 1 -- 172.21.15.86:0/187100552 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559322477e40 con 0x5593226ca0c0 2024-03-22T04:14:20.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2babe00640 10 client.75821 handle_command_reply: tid=0 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 client.75821 shutdown 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 -- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 msgr2=0x5593226e2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x5593226e2ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc022000 tx=0x7f2bbc000bd0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 -- 172.21.15.86:0/187100552 shutdown_connections 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5593226ca0c0 0x5593226ca4a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226dc070 0x5593226c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2babe00640 0 client.75821 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593226db750 0x5593226e2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/187100552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559322692170 0x5593226e2ab0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 -- 172.21.15.86:0/187100552 >> 172.21.15.86:0/187100552 conn(0x559322689d10 msgr2=0x559322692de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 -- 172.21.15.86:0/187100552 shutdown_connections 2024-03-22T04:14:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 1 -- 172.21.15.86:0/187100552 wait complete. 2024-03-22T04:14:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.180+0000 7f2bd47c1000 20 client.75821 trim_cache size 0 max 16384 2024-03-22T04:14:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.192+0000 7f2bd47c1000 1 Processor -- start 2024-03-22T04:14:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.192+0000 7f2bd47c1000 1 -- start start 2024-03-22T04:14:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 0x55932268c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x55932268c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1ffdf0 con 0x559322692170 2024-03-22T04:14:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc0cff60 con 0x5593226db750 2024-03-22T04:14:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2bcc0c91c0 con 0x5593226dc070 2024-03-22T04:14:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43028/0 (socket says 172.21.15.86:43028) 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 -- 172.21.15.86:0/260324949 learned_addr learned my addr 172.21.15.86:0/260324949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 0x55932268c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bcb200640 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x55932268c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 -- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 msgr2=0x55932268c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x55932268c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 -- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 msgr2=0x55932268c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 0x55932268c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 -- 172.21.15.86:0/260324949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593226dbea0 con 0x559322692170 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bcb200640 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x55932268c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226f3070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc001640 tx=0x7f2bbc028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae605c1f37013e04 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bb6a00640 1 -- 172.21.15.86:0/260324949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc02e040 con 0x559322692170 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bb6a00640 1 -- 172.21.15.86:0/260324949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bbc020710 con 0x559322692170 2024-03-22T04:14:20.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bb6a00640 1 -- 172.21.15.86:0/260324949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc025040 con 0x559322692170 2024-03-22T04:14:20.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- 172.21.15.86:0/260324949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 msgr2=0x5593226f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226f3070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc001640 tx=0x7f2bbc028000 comp rx=0 tx=0).stop 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- 172.21.15.86:0/260324949 shutdown_connections 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 0x55932268c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x55932268c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/260324949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226f3070 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- 172.21.15.86:0/260324949 >> 172.21.15.86:0/260324949 conn(0x559322689d10 msgr2=0x5593226dadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- 172.21.15.86:0/260324949 shutdown_connections 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- 172.21.15.86:0/260324949 wait complete. 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- start start 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 Processor -- start 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 0x5593226df9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x5593226dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593224e9490 con 0x559322692170 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc0cf890 con 0x5593226db750 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559322483230 con 0x5593226dc070 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bd47c1000 10 client.0 _ll_register_callbacks cb 0x7f2bbc023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226df4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43038/0 (socket says 172.21.15.86:43038) 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bca800640 1 -- 172.21.15.86:0/877241437 learned_addr learned my addr 172.21.15.86:0/877241437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bcb200640 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x5593226dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.196+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 0x5593226df9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bca800640 1 -- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 msgr2=0x5593226dff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bca800640 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x5593226dff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bca800640 1 -- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 msgr2=0x5593226df9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bca800640 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 0x5593226df9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bca800640 1 -- 172.21.15.86:0/877241437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559322498a50 con 0x559322692170 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bca800640 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226df4b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00f5a0 tx=0x7f2bbc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d54981b6452af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 1 -- 172.21.15.86:0/877241437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc025070 con 0x559322692170 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 1 -- 172.21.15.86:0/877241437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f2bbc007220 con 0x559322692170 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 -- 172.21.15.86:0/877241437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bbc020710 con 0x559322692170 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 1 -- 172.21.15.86:0/877241437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2bbc02e040 con 0x559322692170 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 10 client.6501 fetch_fsmap learned FSMap version 36 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 -- 172.21.15.86:0/877241437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55932255d840 con 0x559322692170 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 1 -- 172.21.15.86:0/877241437 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bbc020710 con 0x559322692170 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 1 -- 172.21.15.86:0/877241437 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f2bbc03c040 con 0x559322692170 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 10 client.6501 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 10 client.6501 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:14:20.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 20 client.6501 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559322747c50 0x5593226e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 4 client.6501 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 -- 172.21.15.86:0/877241437 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5593224ea2c0 con 0x559322747c50 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559322747c50 0x5593226e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bc9e00640 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559322747c50 0x5593226e0470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261f6c27996e2452 server_cookie=48c1b51627a9eb96 in_seq=0 out_seq=0 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 10 client.6501 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 1 -- 172.21.15.86:0/877241437 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+140 (crc 0 0 0) 0x5593224ea2c0 con 0x559322747c50 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 10 client.6501 handle_command_reply: tid=0 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 client.6501 shutdown 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 -- 172.21.15.86:0/877241437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 msgr2=0x5593226df4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226df4b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00f5a0 tx=0x7f2bbc01d000 comp rx=0 tx=0).stop 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 -- 172.21.15.86:0/877241437 shutdown_connections 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x559322747c50 0x5593226e0470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593226db750 0x5593226df9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593226dc070 0x5593226dff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 --2- 172.21.15.86:0/877241437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559322692170 0x5593226df4b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 -- 172.21.15.86:0/877241437 >> 172.21.15.86:0/877241437 conn(0x559322689d10 msgr2=0x559322692550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2babe00640 0 client.6501 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 -- 172.21.15.86:0/877241437 shutdown_connections 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 1 -- 172.21.15.86:0/877241437 wait complete. 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd47c1000 20 client.6501 trim_cache size 0 max 16384 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:14:20.210 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-22T04:14:20.211 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 315260494, 2024-03-22T04:14:20.211 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-03-22T04:14:20.211 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir_test/file" 2024-03-22T04:14:20.211 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:14:20.211 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:14:20.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 -- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2bac067b20 msgr2=0x7f2bac069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2bac067b20 0x7f2bac069fe0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc001810 tx=0x7f2bbc009040 comp rx=0 tx=0).stop 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 -- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 msgr2=0x7f2bcc152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc152d20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c400 tx=0x7f2bb800c8c0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 -- 172.21.15.86:0/1404293829 shutdown_connections 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2bac067b20 0x7f2bac069fe0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bcc15c390 0x7f2bcc153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc0c6930 0x7f2bcc152d20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 --2- 172.21.15.86:0/1404293829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bcc0af910 0x7f2bcc15dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 -- 172.21.15.86:0/1404293829 >> 172.21.15.86:0/1404293829 conn(0x7f2bcc0d0730 msgr2=0x7f2bcc06df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 -- 172.21.15.86:0/1404293829 shutdown_connections 2024-03-22T04:14:20.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.200+0000 7f2bd1400640 1 -- 172.21.15.86:0/1404293829 wait complete. 2024-03-22T04:14:20.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-22T04:14:20.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.352+0000 7f30c0e00640 1 Processor -- start 2024-03-22T04:14:20.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30c0e00640 1 -- start start 2024-03-22T04:14:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30c0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30bc15e2e0 0x7f30bc162b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30c0e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30bc0d0cb0 0x7f30bc0d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30c0e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 0x7f30bc0d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30c0e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc163040 con 0x7f30bc15e2e0 2024-03-22T04:14:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30c0e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc0d0070 con 0x7f30bc0d0cb0 2024-03-22T04:14:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30c0e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc0cf460 con 0x7f30bc0d2000 2024-03-22T04:14:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30bc15e2e0 0x7f30bc162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30bc15e2e0 0x7f30bc162b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43046/0 (socket says 172.21.15.86:43046) 2024-03-22T04:14:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9e00640 1 -- 172.21.15.86:0/2892071016 learned_addr learned my addr 172.21.15.86:0/2892071016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9400640 1 --2- 172.21.15.86:0/2892071016 >> v2:172.21.15.196:3300/0 conn(0x7f30bc0d0cb0 0x7f30bc0d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30ba800640 1 --2- 172.21.15.86:0/2892071016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 0x7f30bc0d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9e00640 1 -- 172.21.15.86:0/2892071016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 msgr2=0x7f30bc0d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9e00640 1 --2- 172.21.15.86:0/2892071016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 0x7f30bc0d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9e00640 1 -- 172.21.15.86:0/2892071016 >> v2:172.21.15.196:3300/0 conn(0x7f30bc0d0cb0 msgr2=0x7f30bc0d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9e00640 1 --2- 172.21.15.86:0/2892071016 >> v2:172.21.15.196:3300/0 conn(0x7f30bc0d0cb0 0x7f30bc0d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.356+0000 7f30b9e00640 1 -- 172.21.15.86:0/2892071016 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0cf740 con 0x7f30bc15e2e0 2024-03-22T04:14:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30b9e00640 1 --2- 172.21.15.86:0/2892071016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc162b00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0b3e40 tx=0x7f30ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8262c9ca3e0bb00 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30b8a00640 1 -- 172.21.15.86:0/2892071016 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0b0070 con 0x7f30bc15e2e0 2024-03-22T04:14:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30b8a00640 1 -- 172.21.15.86:0/2892071016 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ac0ab400 con 0x7f30bc15e2e0 2024-03-22T04:14:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30b8a00640 1 -- 172.21.15.86:0/2892071016 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0ab7d0 con 0x7f30bc15e2e0 2024-03-22T04:14:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 -- 172.21.15.86:0/2892071016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 msgr2=0x7f30bc162b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 --2- 172.21.15.86:0/2892071016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc162b00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0b3e40 tx=0x7f30ac01c520 comp rx=0 tx=0).stop 2024-03-22T04:14:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 -- 172.21.15.86:0/2892071016 shutdown_connections 2024-03-22T04:14:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 --2- 172.21.15.86:0/2892071016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 0x7f30bc0d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 --2- 172.21.15.86:0/2892071016 >> v2:172.21.15.196:3300/0 conn(0x7f30bc0d0cb0 0x7f30bc0d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 --2- 172.21.15.86:0/2892071016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc162b00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 -- 172.21.15.86:0/2892071016 >> 172.21.15.86:0/2892071016 conn(0x7f30bc09d740 msgr2=0x7f30bc0d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 -- 172.21.15.86:0/2892071016 shutdown_connections 2024-03-22T04:14:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 -- 172.21.15.86:0/2892071016 wait complete. 2024-03-22T04:14:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 Processor -- start 2024-03-22T04:14:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.360+0000 7f30c0e00640 1 -- start start 2024-03-22T04:14:20.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30bc0d0cb0 0x7f30bc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 0x7f30bc0b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0d1d60 con 0x7f30bc15e2e0 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0d1be0 con 0x7f30bc0d0cb0 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc15e6c0 con 0x7f30bc0d2000 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc14de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43060/0 (socket says 172.21.15.86:43060) 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 -- 172.21.15.86:0/1119949799 learned_addr learned my addr 172.21.15.86:0/1119949799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30b9e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30bc0d0cb0 0x7f30bc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30b9400640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 0x7f30bc0b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 -- 172.21.15.86:0/1119949799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 msgr2=0x7f30bc0b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 0x7f30bc0b9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 -- 172.21.15.86:0/1119949799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30bc0d0cb0 msgr2=0x7f30bc0b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30bc0d0cb0 0x7f30bc0b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 -- 172.21.15.86:0/1119949799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc14fd50 con 0x7f30bc15e2e0 2024-03-22T04:14:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30ba800640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc14de10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f30b4007dc0 tx=0x7f30b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30b2000640 1 -- 172.21.15.86:0/1119949799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4014420 con 0x7f30bc15e2e0 2024-03-22T04:14:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30b2000640 1 -- 172.21.15.86:0/1119949799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b4019070 con 0x7f30bc15e2e0 2024-03-22T04:14:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc0b94d0 con 0x7f30bc15e2e0 2024-03-22T04:14:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30b2000640 1 -- 172.21.15.86:0/1119949799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4014820 con 0x7f30bc15e2e0 2024-03-22T04:14:20.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc14e600 con 0x7f30bc15e2e0 2024-03-22T04:14:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc074550 con 0x7f30bc15e2e0 2024-03-22T04:14:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.364+0000 7f30b2000640 1 -- 172.21.15.86:0/1119949799 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f30b40149c0 con 0x7f30bc15e2e0 2024-03-22T04:14:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.368+0000 7f30b2000640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f309c067b50 0x7f309c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.368+0000 7f30b2000640 1 -- 172.21.15.86:0/1119949799 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f30b40867f0 con 0x7f30bc15e2e0 2024-03-22T04:14:20.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.368+0000 7f30b9e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f309c067b50 0x7f309c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.368+0000 7f30b9e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f309c067b50 0x7f309c06a010 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0ad000 tx=0x7f30ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.372+0000 7f30b2000640 1 -- 172.21.15.86:0/1119949799 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f30b4056970 con 0x7f30bc15e2e0 2024-03-22T04:14:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.556+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f30bc0c7bc0 con 0x7f30bc15e2e0 2024-03-22T04:14:20.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.556+0000 7f30b2000640 1 -- 172.21.15.86:0/1119949799 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f30b405aa10 con 0x7f30bc15e2e0 2024-03-22T04:14:20.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:20.562 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-22T04:14:20.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f309c067b50 msgr2=0x7f309c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f309c067b50 0x7f309c06a010 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0ad000 tx=0x7f30ac008040 comp rx=0 tx=0).stop 2024-03-22T04:14:20.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 msgr2=0x7f30bc14de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc14de10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f30b4007dc0 tx=0x7f30b400e4b0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 shutdown_connections 2024-03-22T04:14:20.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f309c067b50 0x7f309c06a010 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15e2e0 0x7f30bc14de10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30bc0d2000 0x7f30bc0b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 --2- 172.21.15.86:0/1119949799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30bc0d0cb0 0x7f30bc0b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 >> 172.21.15.86:0/1119949799 conn(0x7f30bc09d740 msgr2=0x7f30bc157070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 shutdown_connections 2024-03-22T04:14:20.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.560+0000 7f30c0e00640 1 -- 172.21.15.86:0/1119949799 wait complete. 2024-03-22T04:14:20.582 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-22T04:14:20.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:14:20.583 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-03-22T04:14:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.716+0000 7f302d200640 1 Processor -- start 2024-03-22T04:14:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.716+0000 7f302d200640 1 -- start start 2024-03-22T04:14:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f302d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f302806de70 0x7f3028166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f302d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30280cecb0 con 0x7f30280b1cd0 2024-03-22T04:14:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f302d200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30280cef90 con 0x7f30280cfa70 2024-03-22T04:14:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f302d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f30280cea80 con 0x7f302806de70 2024-03-22T04:14:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f3026800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f302806de70 0x7f3028166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f3025e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f30280b1cd0 0x7f30280b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55982/0 (socket says 172.21.15.86:55982) 2024-03-22T04:14:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f3025e00640 1 -- 172.21.15.86:0/517019098 learned_addr learned my addr 172.21.15.86:0/517019098 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f3026800640 1 -- 172.21.15.86:0/517019098 >> v1:172.21.15.196:6789/0 conn(0x7f30280cfa70 legacy=0x7f302806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f3026800640 1 -- 172.21.15.86:0/517019098 >> v1:172.21.15.86:6789/0 conn(0x7f30280b1cd0 legacy=0x7f30280b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f3026800640 1 -- 172.21.15.86:0/517019098 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30280cf270 con 0x7f302806de70 2024-03-22T04:14:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.720+0000 7f3025400640 1 -- 172.21.15.86:0/517019098 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575279604 0 0) 0x7f30280cecb0 con 0x7f30280b1cd0 2024-03-22T04:14:20.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f3026800640 1 --2- 172.21.15.86:0/517019098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302806de70 0x7f3028166d70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3020009d90 tx=0x7f302001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e443748a803bd65 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f3025400640 1 -- 172.21.15.86:0/517019098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30200b2070 con 0x7f302806de70 2024-03-22T04:14:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f3025400640 1 -- 172.21.15.86:0/517019098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30200a5830 con 0x7f302806de70 2024-03-22T04:14:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f3025400640 1 -- 172.21.15.86:0/517019098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30200a5c00 con 0x7f302806de70 2024-03-22T04:14:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 -- 172.21.15.86:0/517019098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302806de70 msgr2=0x7f3028166d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 --2- 172.21.15.86:0/517019098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302806de70 0x7f3028166d70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3020009d90 tx=0x7f302001c970 comp rx=0 tx=0).stop 2024-03-22T04:14:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 -- 172.21.15.86:0/517019098 shutdown_connections 2024-03-22T04:14:20.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 --2- 172.21.15.86:0/517019098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302806de70 0x7f3028166d70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 -- 172.21.15.86:0/517019098 >> 172.21.15.86:0/517019098 conn(0x7f302806b660 msgr2=0x7f302806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 -- 172.21.15.86:0/517019098 shutdown_connections 2024-03-22T04:14:20.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 -- 172.21.15.86:0/517019098 wait complete. 2024-03-22T04:14:20.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 Processor -- start 2024-03-22T04:14:20.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.724+0000 7f302d200640 1 -- start start 2024-03-22T04:14:20.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f302d200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f302806de70 0x7f30280b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f302d200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280b1cd0 0x7f30280727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f302d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280cfa70 0x7f302813a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f302d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3028169e60 con 0x7f30280cfa70 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f302d200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3028169ce0 con 0x7f30280b1cd0 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f302d200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3028169fe0 con 0x7f302806de70 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280cfa70 0x7f302813a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280cfa70 0x7f302813a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43086/0 (socket says 172.21.15.86:43086) 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 -- 172.21.15.86:0/1174246096 learned_addr learned my addr 172.21.15.86:0/1174246096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3025e00640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f302806de70 0x7f30280b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f301d400640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280b1cd0 0x7f30280727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 -- 172.21.15.86:0/1174246096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f302806de70 msgr2=0x7f30280b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f302806de70 0x7f30280b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 -- 172.21.15.86:0/1174246096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280b1cd0 msgr2=0x7f30280727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280b1cd0 0x7f30280727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 -- 172.21.15.86:0/1174246096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3028071870 con 0x7f30280cfa70 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f3026800640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280cfa70 0x7f302813a4f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3020009d90 tx=0x7f3020019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f301ea00640 1 -- 172.21.15.86:0/1174246096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30200b2050 con 0x7f30280cfa70 2024-03-22T04:14:20.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f301ea00640 1 -- 172.21.15.86:0/1174246096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30200b3040 con 0x7f30280cfa70 2024-03-22T04:14:20.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f301ea00640 1 -- 172.21.15.86:0/1174246096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020003dc0 con 0x7f30280cfa70 2024-03-22T04:14:20.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f302d200640 1 -- 172.21.15.86:0/1174246096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3028071b50 con 0x7f30280cfa70 2024-03-22T04:14:20.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f302d200640 1 -- 172.21.15.86:0/1174246096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3028072110 con 0x7f30280cfa70 2024-03-22T04:14:20.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.728+0000 7f301ea00640 1 -- 172.21.15.86:0/1174246096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f30200ad650 con 0x7f30280cfa70 2024-03-22T04:14:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f301ea00640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067b20 0x7f3004069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f301ea00640 1 -- 172.21.15.86:0/1174246096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f3020120880 con 0x7f30280cfa70 2024-03-22T04:14:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3025e00640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067b20 0x7f3004069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 Processor -- start 2024-03-22T04:14:20.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- start start 2024-03-22T04:14:20.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3025e00640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067b20 0x7f3004069fe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3018000f70 tx=0x7f3018007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa66e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa6219c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622fa473b20 con 0x5622fa621f00 2024-03-22T04:14:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622fa3fa9b0 con 0x5622fa66e5b0 2024-03-22T04:14:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5622fa46fad0 con 0x5622fa66eed0 2024-03-22T04:14:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6222e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43102/0 (socket says 172.21.15.86:43102) 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 -- 172.21.15.86:0/1654655960 learned_addr learned my addr 172.21.15.86:0/1654655960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3025e00640 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa66e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f301d400640 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa6219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 -- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 msgr2=0x5622fa6219c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa6219c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 -- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 msgr2=0x5622fa66e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa66e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 -- 172.21.15.86:0/1654655960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622fa47eb70 con 0x5622fa621f00 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f301d400640 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa6219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3026800640 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6222e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f30200b0040 tx=0x7f30200b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603b2b1418275abc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3016a00640 1 -- 172.21.15.86:0/1654655960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30200f4a10 con 0x5622fa621f00 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3016a00640 1 -- 172.21.15.86:0/1654655960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30200f9240 con 0x5622fa621f00 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f3016a00640 1 -- 172.21.15.86:0/1654655960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30200ecfe0 con 0x5622fa621f00 2024-03-22T04:14:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- 172.21.15.86:0/1654655960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 msgr2=0x5622fa6222e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6222e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f30200b0040 tx=0x7f30200b9000 comp rx=0 tx=0).stop 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- 172.21.15.86:0/1654655960 shutdown_connections 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6222e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa6219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 --2- 172.21.15.86:0/1654655960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa66e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- 172.21.15.86:0/1654655960 >> 172.21.15.86:0/1654655960 conn(0x5622fa64f350 msgr2=0x5622fa651740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- 172.21.15.86:0/1654655960 shutdown_connections 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.732+0000 7f30305c2000 1 -- 172.21.15.86:0/1654655960 wait complete. 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 -- start start 2024-03-22T04:14:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 Processor -- start 2024-03-22T04:14:20.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa6285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa628b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622fa52e920 con 0x5622fa621f00 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622fa42dbc0 con 0x5622fa66e5b0 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5622fa42f010 con 0x5622fa66eed0 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 10 client.0 _ll_register_callbacks cb 0x5622fa60f860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43104/0 (socket says 172.21.15.86:43104) 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f301d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa6285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 -- 172.21.15.86:0/3148368811 learned_addr learned my addr 172.21.15.86:0/3148368811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3026800640 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa628b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 -- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 msgr2=0x5622fa628b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa628b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 -- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 msgr2=0x5622fa6285f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa6285f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 -- 172.21.15.86:0/3148368811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622fa437600 con 0x5622fa621f00 2024-03-22T04:14:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f301d400640 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa6285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f3025e00640 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6280b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3018013670 tx=0x7f3018007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2f37db6f3f3c5a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f300b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f300b400640 1 -- 172.21.15.86:0/3148368811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018020580 con 0x5622fa621f00 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f300b400640 1 -- 172.21.15.86:0/3148368811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f3018020720 con 0x5622fa621f00 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 -- 172.21.15.86:0/3148368811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30200f9240 con 0x5622fa621f00 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f300b400640 1 -- 172.21.15.86:0/3148368811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f301802e050 con 0x5622fa621f00 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 10 client.6522 fetch_fsmap learned FSMap version 36 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 -- 172.21.15.86:0/3148368811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5622fa40e610 con 0x5622fa621f00 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f300b400640 1 -- 172.21.15.86:0/3148368811 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30200f9240 con 0x5622fa621f00 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f300b400640 1 -- 172.21.15.86:0/3148368811 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f3018031020 con 0x5622fa621f00 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 10 client.6522 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 10 client.6522 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 20 client.6522 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5622fa629540 0x5622fa680660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 4 client.6522 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f30305c2000 1 -- 172.21.15.86:0/3148368811 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622fa40eb40 con 0x5622fa629540 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.736+0000 7f301d400640 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5622fa629540 0x5622fa680660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f301d400640 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5622fa629540 0x5622fa680660 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496f07b68b0bf02d server_cookie=7d0050574b35c842 in_seq=0 out_seq=0 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f300b400640 10 client.6522 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f300b400640 1 -- 172.21.15.86:0/3148368811 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5622fa40eb40 con 0x5622fa629540 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f300b400640 10 client.6522 handle_command_reply: tid=0 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 client.6522 shutdown 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 -- 172.21.15.86:0/3148368811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 msgr2=0x5622fa6280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6280b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3018013670 tx=0x7f3018007d20 comp rx=0 tx=0).stop 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 -- 172.21.15.86:0/3148368811 shutdown_connections 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5622fa629540 0x5622fa680660 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66e5b0 0x5622fa6285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66eed0 0x5622fa628b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 --2- 172.21.15.86:0/3148368811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa621f00 0x5622fa6280b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f300b400640 0 client.6522 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 -- 172.21.15.86:0/3148368811 >> 172.21.15.86:0/3148368811 conn(0x5622fa64f350 msgr2=0x5622fa6210c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 -- 172.21.15.86:0/3148368811 shutdown_connections 2024-03-22T04:14:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 1 -- 172.21.15.86:0/3148368811 wait complete. 2024-03-22T04:14:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.740+0000 7f30305c2000 20 client.6522 trim_cache size 0 max 16384 2024-03-22T04:14:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f30305c2000 1 Processor -- start 2024-03-22T04:14:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f30305c2000 1 -- start start 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f30305c2000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa621f00 0x5622fa620830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f30305c2000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66e5b0 0x5622fa620d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f30305c2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66eed0 0x5622fa680660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f30305c2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3028204070 con 0x5622fa66eed0 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f30305c2000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3028203050 con 0x5622fa621f00 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f30305c2000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30281f7cb0 con 0x5622fa66e5b0 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66eed0 0x5622fa680660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66eed0 0x5622fa680660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43118/0 (socket says 172.21.15.86:43118) 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 -- 172.21.15.86:0/2675709149 learned_addr learned my addr 172.21.15.86:0/2675709149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f301d400640 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66e5b0 0x5622fa620d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3025e00640 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa621f00 0x5622fa620830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 -- 172.21.15.86:0/2675709149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66e5b0 msgr2=0x5622fa620d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66e5b0 0x5622fa620d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 -- 172.21.15.86:0/2675709149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa621f00 msgr2=0x5622fa620830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa621f00 0x5622fa620830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 -- 172.21.15.86:0/2675709149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622fa66ed00 con 0x5622fa66eed0 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.756+0000 7f3026800640 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66eed0 0x5622fa680660 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f30200f0670 tx=0x7f30200c4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0d94bdaf0aa7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f3016000640 1 -- 172.21.15.86:0/2675709149 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30200f9240 con 0x5622fa66eed0 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f3016000640 1 -- 172.21.15.86:0/2675709149 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30200f4a10 con 0x5622fa66eed0 2024-03-22T04:14:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f3016000640 1 -- 172.21.15.86:0/2675709149 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30200ecfe0 con 0x5622fa66eed0 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- 172.21.15.86:0/2675709149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66eed0 msgr2=0x5622fa680660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66eed0 0x5622fa680660 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f30200f0670 tx=0x7f30200c4000 comp rx=0 tx=0).stop 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- 172.21.15.86:0/2675709149 shutdown_connections 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66eed0 0x5622fa680660 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa66e5b0 0x5622fa620d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 --2- 172.21.15.86:0/2675709149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa621f00 0x5622fa620830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- 172.21.15.86:0/2675709149 >> 172.21.15.86:0/2675709149 conn(0x5622fa64f350 msgr2=0x5622fa620120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- 172.21.15.86:0/2675709149 shutdown_connections 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- 172.21.15.86:0/2675709149 wait complete. 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- start start 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 Processor -- start 2024-03-22T04:14:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa621f00 0x5622fa6832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66e5b0 0x5622fa683830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66eed0 0x5622fa6db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622fa3feba0 con 0x5622fa66e5b0 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30282029a0 con 0x5622fa66eed0 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5622fa40dad0 con 0x5622fa621f00 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 10 client.0 _ll_register_callbacks cb 0x7f3018001510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66e5b0 0x5622fa683830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66e5b0 0x5622fa683830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43120/0 (socket says 172.21.15.86:43120) 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 -- 172.21.15.86:0/2258870756 learned_addr learned my addr 172.21.15.86:0/2258870756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f3025e00640 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa621f00 0x5622fa6832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f3026800640 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66eed0 0x5622fa6db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 -- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa621f00 msgr2=0x5622fa6832f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa621f00 0x5622fa6832f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 -- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66eed0 msgr2=0x5622fa6db0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66eed0 0x5622fa6db0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 -- 172.21.15.86:0/2258870756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622fa4f4680 con 0x5622fa66e5b0 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f3025e00640 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa621f00 0x5622fa6832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f301d400640 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66e5b0 0x5622fa683830 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3028139630 tx=0x7f3010016ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7d81dcabbd0554 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f300b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f300b400640 1 -- 172.21.15.86:0/2258870756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010001700 con 0x5622fa66e5b0 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f300b400640 1 -- 172.21.15.86:0/2258870756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f30100018a0 con 0x5622fa66e5b0 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- 172.21.15.86:0/2258870756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30200f4a10 con 0x5622fa66e5b0 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f300b400640 1 -- 172.21.15.86:0/2258870756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f301002c050 con 0x5622fa66e5b0 2024-03-22T04:14:20.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 10 client.6528 fetch_fsmap learned FSMap version 36 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f30305c2000 1 -- 172.21.15.86:0/2258870756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5622fa4cb110 con 0x5622fa66e5b0 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.760+0000 7f300b400640 1 -- 172.21.15.86:0/2258870756 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30200f4a10 con 0x5622fa66e5b0 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f300b400640 1 -- 172.21.15.86:0/2258870756 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f301002f020 con 0x5622fa66e5b0 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 10 client.6528 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 10 client.6528 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 20 client.6528 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5622fa6dbf90 0x5622fa6e7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 4 client.6528 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 -- 172.21.15.86:0/2258870756 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x5622fa481060 con 0x5622fa6dbf90 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f3025e00640 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5622fa6dbf90 0x5622fa6e7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f3025e00640 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5622fa6dbf90 0x5622fa6e7ba0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99763be4dcd5e827 server_cookie=ba0a2fc7fd67f3d5 in_seq=0 out_seq=0 2024-03-22T04:14:20.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f300b400640 10 client.6528 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.772 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:14:20.762+0000 7f42ba400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6390=0-4194304@1} caps={6390=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6390 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x562f7ea04580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-22T04:14:20.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f300b400640 1 -- 172.21.15.86:0/2258870756 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5622fa481060 con 0x5622fa6dbf90 2024-03-22T04:14:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f300b400640 10 client.6528 handle_command_reply: tid=0 2024-03-22T04:14:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 client.6528 shutdown 2024-03-22T04:14:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 -- 172.21.15.86:0/2258870756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66e5b0 msgr2=0x5622fa683830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66e5b0 0x5622fa683830 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3028139630 tx=0x7f3010016ae0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 -- 172.21.15.86:0/2258870756 shutdown_connections 2024-03-22T04:14:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x5622fa6dbf90 0x5622fa6e7ba0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622fa66eed0 0x5622fa6db0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622fa66e5b0 0x5622fa683830 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 --2- 172.21.15.86:0/2258870756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622fa621f00 0x5622fa6832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f300b400640 0 client.6528 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 -- 172.21.15.86:0/2258870756 >> 172.21.15.86:0/2258870756 conn(0x5622fa64f350 msgr2=0x5622fa6209b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 -- 172.21.15.86:0/2258870756 shutdown_connections 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 1 -- 172.21.15.86:0/2258870756 wait complete. 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f30305c2000 20 client.6528 trim_cache size 0 max 16384 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "49832eb9-0496-4b0f-b9a7-7c45b2a9903b", 2024-03-22T04:14:20.775 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:14:20.776 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:14:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 -- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067b20 msgr2=0x7f3004069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067b20 0x7f3004069fe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3018000f70 tx=0x7f3018007240 comp rx=0 tx=0).stop 2024-03-22T04:14:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 -- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280cfa70 msgr2=0x7f302813a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280cfa70 0x7f302813a4f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3020009d90 tx=0x7f3020019710 comp rx=0 tx=0).stop 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 -- 172.21.15.86:0/1174246096 shutdown_connections 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3004067b20 0x7f3004069fe0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280b1cd0 0x7f30280727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280cfa70 0x7f302813a4f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 --2- 172.21.15.86:0/1174246096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f302806de70 0x7f30280b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 -- 172.21.15.86:0/1174246096 >> 172.21.15.86:0/1174246096 conn(0x7f302806b660 msgr2=0x7f30281455b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 -- 172.21.15.86:0/1174246096 shutdown_connections 2024-03-22T04:14:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.764+0000 7f302d200640 1 -- 172.21.15.86:0/1174246096 wait complete. 2024-03-22T04:14:20.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:14:20.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-22T04:14:20.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.920+0000 7f262c000640 1 Processor -- start 2024-03-22T04:14:20.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f262c000640 1 -- start start 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f262c000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26240b2360 0x7f26240b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f262c000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 0x7f26240b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f262c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f262415d710 con 0x7f26240b3430 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f262c000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f26240b2740 con 0x7f26240b2360 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f262c000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f262415d4b0 con 0x7f262415d0b0 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2629600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26240b2360 0x7f26240b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2629600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26240b2360 0x7f26240b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43132/0 (socket says 172.21.15.86:43132) 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2628c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f26240b3430 0x7f262415c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55992/0 (socket says 172.21.15.86:55992) 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2629600640 1 -- 172.21.15.86:0/1585517994 learned_addr learned my addr 172.21.15.86:0/1585517994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f262a000640 1 --2- 172.21.15.86:0/1585517994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 0x7f26240b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2629600640 1 -- 172.21.15.86:0/1585517994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 msgr2=0x7f26240b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2629600640 1 --2- 172.21.15.86:0/1585517994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 0x7f26240b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2629600640 1 -- 172.21.15.86:0/1585517994 >> v1:172.21.15.86:6789/0 conn(0x7f26240b3430 legacy=0x7f262415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2623e00640 1 -- 172.21.15.86:0/1585517994 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 286767102 0 0) 0x7f262415d710 con 0x7f26240b3430 2024-03-22T04:14:20.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.924+0000 7f2629600640 1 -- 172.21.15.86:0/1585517994 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262415fe40 con 0x7f26240b2360 2024-03-22T04:14:20.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f2629600640 1 --2- 172.21.15.86:0/1585517994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b2360 0x7f26240b2ef0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f26180098a0 tx=0x7f261801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2038b55458b639 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f2623e00640 1 -- 172.21.15.86:0/1585517994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180ab430 con 0x7f26240b2360 2024-03-22T04:14:20.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f2623e00640 1 -- 172.21.15.86:0/1585517994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26180b0070 con 0x7f26240b2360 2024-03-22T04:14:20.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f2623e00640 1 -- 172.21.15.86:0/1585517994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180ab7f0 con 0x7f26240b2360 2024-03-22T04:14:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 -- 172.21.15.86:0/1585517994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b2360 msgr2=0x7f26240b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 --2- 172.21.15.86:0/1585517994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b2360 0x7f26240b2ef0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f26180098a0 tx=0x7f261801c540 comp rx=0 tx=0).stop 2024-03-22T04:14:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 -- 172.21.15.86:0/1585517994 shutdown_connections 2024-03-22T04:14:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 --2- 172.21.15.86:0/1585517994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 0x7f26240b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 --2- 172.21.15.86:0/1585517994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b2360 0x7f26240b2ef0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 -- 172.21.15.86:0/1585517994 >> 172.21.15.86:0/1585517994 conn(0x7f26240cc940 msgr2=0x7f26240ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 -- 172.21.15.86:0/1585517994 shutdown_connections 2024-03-22T04:14:20.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 -- 172.21.15.86:0/1585517994 wait complete. 2024-03-22T04:14:20.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 Processor -- start 2024-03-22T04:14:20.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.928+0000 7f262c000640 1 -- start start 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262c000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26240b2360 0x7f262415b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b3430 0x7f262415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262c000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 0x7f26241550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2624160c00 con 0x7f26240b3430 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262c000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26240b5ad0 con 0x7f26240b2360 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262c000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f262414c7a0 con 0x7f262415d0b0 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b3430 0x7f262415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b3430 0x7f262415b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43146/0 (socket says 172.21.15.86:43146) 2024-03-22T04:14:20.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 -- 172.21.15.86:0/2986578987 learned_addr learned my addr 172.21.15.86:0/2986578987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2629600640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26240b2360 0x7f262415b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262a000640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 0x7f26241550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 -- 172.21.15.86:0/2986578987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 msgr2=0x7f26241550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 0x7f26241550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 -- 172.21.15.86:0/2986578987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26240b2360 msgr2=0x7f262415b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26240b2360 0x7f262415b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 -- 172.21.15.86:0/2986578987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262414ca80 con 0x7f26240b3430 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2628c00640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b3430 0x7f262415b860 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f261000c4b0 tx=0x7f261000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2621600640 1 -- 172.21.15.86:0/2986578987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610018070 con 0x7f26240b3430 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2621600640 1 -- 172.21.15.86:0/2986578987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610007480 con 0x7f26240b3430 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2621600640 1 -- 172.21.15.86:0/2986578987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610004a30 con 0x7f26240b3430 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262c000640 1 -- 172.21.15.86:0/2986578987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262414cd60 con 0x7f26240b3430 2024-03-22T04:14:20.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f262c000640 1 -- 172.21.15.86:0/2986578987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262414d320 con 0x7f26240b3430 2024-03-22T04:14:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.932+0000 7f2621600640 1 -- 172.21.15.86:0/2986578987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2610004bd0 con 0x7f26240b3430 2024-03-22T04:14:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2621600640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2604067b20 0x7f2604069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2621600640 1 -- 172.21.15.86:0/2986578987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f26100854c0 con 0x7f26240b3430 2024-03-22T04:14:20.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2629600640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2604067b20 0x7f2604069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 Processor -- start 2024-03-22T04:14:20.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- start start 2024-03-22T04:14:20.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2629600640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2604067b20 0x7f2604069fe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f26180af000 tx=0x7f26180a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d475360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d4758a0 0x55b19d428a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 0x55b19d429330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b19d279c40 con 0x55b19d4758a0 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b19d216d90 con 0x55b19d474f80 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b19d275bf0 con 0x55b19d428f50 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d4758a0 0x55b19d428a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d4758a0 0x55b19d428a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43158/0 (socket says 172.21.15.86:43158) 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 -- 172.21.15.86:0/2707526131 learned_addr learned my addr 172.21.15.86:0/2707526131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262a000640 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 0x55b19d429330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2629600640 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d475360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 -- 172.21.15.86:0/2707526131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 msgr2=0x55b19d429330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 0x55b19d429330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 -- 172.21.15.86:0/2707526131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 msgr2=0x55b19d475360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d475360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 -- 172.21.15.86:0/2707526131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19d321130 con 0x55b19d4758a0 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2628c00640 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d4758a0 0x55b19d428a10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2610083660 tx=0x7f2610055720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae07573f0fc734e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2616000640 1 -- 172.21.15.86:0/2707526131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610080030 con 0x55b19d4758a0 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2616000640 1 -- 172.21.15.86:0/2707526131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26100538e0 con 0x55b19d4758a0 2024-03-22T04:14:20.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f2616000640 1 -- 172.21.15.86:0/2707526131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610051c50 con 0x55b19d4758a0 2024-03-22T04:14:20.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- 172.21.15.86:0/2707526131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d4758a0 msgr2=0x55b19d428a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d4758a0 0x55b19d428a10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2610083660 tx=0x7f2610055720 comp rx=0 tx=0).stop 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- 172.21.15.86:0/2707526131 shutdown_connections 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 0x55b19d429330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d4758a0 0x55b19d428a10 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 --2- 172.21.15.86:0/2707526131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d475360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- 172.21.15.86:0/2707526131 >> 172.21.15.86:0/2707526131 conn(0x55b19d312990 msgr2=0x55b19d458150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- 172.21.15.86:0/2707526131 shutdown_connections 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.936+0000 7f262f575000 1 -- 172.21.15.86:0/2707526131 wait complete. 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 -- start start 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 Processor -- start 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d430990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d474f80 0x55b19d459380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d4758a0 0x55b19d4598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b19d235710 con 0x55b19d428f50 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b19d237580 con 0x55b19d4758a0 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b19d27e270 con 0x55b19d474f80 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 10 client.0 _ll_register_callbacks cb 0x55b19d2db200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d430990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d430990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43164/0 (socket says 172.21.15.86:43164) 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 -- 172.21.15.86:0/641906006 learned_addr learned my addr 172.21.15.86:0/641906006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262a000640 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d4758a0 0x55b19d4598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2628c00640 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d474f80 0x55b19d459380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 -- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d474f80 msgr2=0x55b19d459380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d474f80 0x55b19d459380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 -- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d4758a0 msgr2=0x55b19d4598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d4758a0 0x55b19d4598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 -- 172.21.15.86:0/641906006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b19d252d30 con 0x55b19d428f50 2024-03-22T04:14:20.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262a000640 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d4758a0 0x55b19d4598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2629600640 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d430990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f26180098a0 tx=0x7f261801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9025d7a8884def9c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f260aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f260aa00640 1 -- 172.21.15.86:0/641906006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180ce060 con 0x55b19d428f50 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f260aa00640 1 -- 172.21.15.86:0/641906006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f26180a8040 con 0x55b19d428f50 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 -- 172.21.15.86:0/641906006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26100538e0 con 0x55b19d428f50 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f260aa00640 1 -- 172.21.15.86:0/641906006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f26180d4050 con 0x55b19d428f50 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 10 client.6543 fetch_fsmap learned FSMap version 36 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 -- 172.21.15.86:0/641906006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55b19d2befc0 con 0x55b19d428f50 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f260aa00640 1 -- 172.21.15.86:0/641906006 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26100538e0 con 0x55b19d428f50 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f260aa00640 1 -- 172.21.15.86:0/641906006 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f26180d7020 con 0x55b19d428f50 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 10 client.6543 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 10 client.6543 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 20 client.6543 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55b19d47b5c0 0x55b19d47b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 4 client.6543 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:14:20.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f262f575000 1 -- 172.21.15.86:0/641906006 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b19d214dd0 con 0x55b19d47b5c0 2024-03-22T04:14:20.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.940+0000 7f2628c00640 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55b19d47b5c0 0x55b19d47b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f260aa00640 10 client.6543 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f2628c00640 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55b19d47b5c0 0x55b19d47b9a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4281d81fab5c9bf server_cookie=7a80c5974e576a32 in_seq=0 out_seq=0 2024-03-22T04:14:20.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f260aa00640 1 -- 172.21.15.86:0/641906006 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b19d214dd0 con 0x55b19d47b5c0 2024-03-22T04:14:20.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f260aa00640 10 client.6543 handle_command_reply: tid=0 2024-03-22T04:14:20.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 client.6543 shutdown 2024-03-22T04:14:20.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 -- 172.21.15.86:0/641906006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 msgr2=0x55b19d430990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d430990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f26180098a0 tx=0x7f261801ad80 comp rx=0 tx=0).stop 2024-03-22T04:14:20.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 -- 172.21.15.86:0/641906006 shutdown_connections 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55b19d47b5c0 0x55b19d47b9a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d4758a0 0x55b19d4598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f260aa00640 0 client.6543 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d474f80 0x55b19d459380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 --2- 172.21.15.86:0/641906006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d430990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 -- 172.21.15.86:0/641906006 >> 172.21.15.86:0/641906006 conn(0x55b19d312990 msgr2=0x55b19d428110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 -- 172.21.15.86:0/641906006 shutdown_connections 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 1 -- 172.21.15.86:0/641906006 wait complete. 2024-03-22T04:14:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.944+0000 7f262f575000 20 client.6543 trim_cache size 0 max 16384 2024-03-22T04:14:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262f575000 1 Processor -- start 2024-03-22T04:14:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262f575000 1 -- start start 2024-03-22T04:14:20.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262f575000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d312990 0x55b19d48b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262f575000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d42f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262f575000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d42fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262f575000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2624204060 con 0x55b19d428f50 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262f575000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2624203040 con 0x55b19d474f80 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262f575000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2624072300 con 0x55b19d312990 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f2628c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d42f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f2628c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d42f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43174/0 (socket says 172.21.15.86:43174) 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f2628c00640 1 -- 172.21.15.86:0/2788069211 learned_addr learned my addr 172.21.15.86:0/2788069211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f2629600640 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d312990 0x55b19d48b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.960+0000 7f262a000640 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d42fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2628c00640 1 -- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d312990 msgr2=0x55b19d48b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2628c00640 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d312990 0x55b19d48b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2628c00640 1 -- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 msgr2=0x55b19d42fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2628c00640 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d42fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2628c00640 1 -- 172.21.15.86:0/2788069211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19d24f3a0 con 0x55b19d428f50 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262a000640 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d42fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2628c00640 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d42f870 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f26100011f0 tx=0x7f2610085000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9e732524bced28 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2615600640 1 -- 172.21.15.86:0/2788069211 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610080040 con 0x55b19d428f50 2024-03-22T04:14:20.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2615600640 1 -- 172.21.15.86:0/2788069211 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610051c50 con 0x55b19d428f50 2024-03-22T04:14:20.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2615600640 1 -- 172.21.15.86:0/2788069211 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610085290 con 0x55b19d428f50 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- 172.21.15.86:0/2788069211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 msgr2=0x55b19d42f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d42f870 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f26100011f0 tx=0x7f2610085000 comp rx=0 tx=0).stop 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- 172.21.15.86:0/2788069211 shutdown_connections 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d42fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d428f50 0x55b19d42f870 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 --2- 172.21.15.86:0/2788069211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d312990 0x55b19d48b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- 172.21.15.86:0/2788069211 >> 172.21.15.86:0/2788069211 conn(0x55b19d458400 msgr2=0x55b19d42f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- 172.21.15.86:0/2788069211 shutdown_connections 2024-03-22T04:14:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- 172.21.15.86:0/2788069211 wait complete. 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- start start 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 Processor -- start 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d312990 0x55b19d48b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 0x55b19d4e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d4e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b19d335810 con 0x55b19d312990 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2624202ac0 con 0x55b19d474f80 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b19d278f50 con 0x55b19d428f50 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262f575000 10 client.0 _ll_register_callbacks cb 0x7f2618006ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d312990 0x55b19d48b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d312990 0x55b19d48b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43176/0 (socket says 172.21.15.86:43176) 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 -- 172.21.15.86:0/132699236 learned_addr learned my addr 172.21.15.86:0/132699236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2628c00640 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 0x55b19d4e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f262a000640 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d4e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 -- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 msgr2=0x55b19d4e02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 0x55b19d4e02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 -- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 msgr2=0x55b19d4e07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d4e07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 -- 172.21.15.86:0/132699236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b19d1ff380 con 0x55b19d312990 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f2629600640 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d312990 0x55b19d48b5b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f261801cab0 tx=0x7f26180b7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaf4ff6749ba540 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.964+0000 7f260aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 1 -- 172.21.15.86:0/132699236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180bb070 con 0x55b19d312990 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 1 -- 172.21.15.86:0/132699236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f26180c6040 con 0x55b19d312990 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 -- 172.21.15.86:0/132699236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2610051c50 con 0x55b19d312990 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 1 -- 172.21.15.86:0/132699236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f26180ce020 con 0x55b19d312990 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 10 client.6549 fetch_fsmap learned FSMap version 36 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 -- 172.21.15.86:0/132699236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55b19d2fb460 con 0x55b19d312990 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 1 -- 172.21.15.86:0/132699236 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610051c50 con 0x55b19d312990 2024-03-22T04:14:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 1 -- 172.21.15.86:0/132699236 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f26180d7040 con 0x55b19d312990 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 10 client.6549 fetch_fsmap finished waiting for FSMap version 36 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 10 client.6549 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 20 client.6549 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55b19d4758a0 0x55b19d4e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 4 client.6549 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 -- 172.21.15.86:0/132699236 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b19d219160 con 0x55b19d4758a0 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f2628c00640 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55b19d4758a0 0x55b19d4e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f2628c00640 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55b19d4758a0 0x55b19d4e0d20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b61b19a6821d116 server_cookie=b991554002a53009 in_seq=0 out_seq=0 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 10 client.6549 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 1 -- 172.21.15.86:0/132699236 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55b19d219160 con 0x55b19d4758a0 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 10 client.6549 handle_command_reply: tid=0 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 client.6549 shutdown 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 -- 172.21.15.86:0/132699236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d312990 msgr2=0x55b19d48b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d312990 0x55b19d48b5b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f261801cab0 tx=0x7f26180b7070 comp rx=0 tx=0).stop 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 -- 172.21.15.86:0/132699236 shutdown_connections 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55b19d4758a0 0x55b19d4e0d20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19d474f80 0x55b19d4e07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b19d428f50 0x55b19d4e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f260aa00640 0 client.6549 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 --2- 172.21.15.86:0/132699236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b19d312990 0x55b19d48b5b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 -- 172.21.15.86:0/132699236 >> 172.21.15.86:0/132699236 conn(0x55b19d458400 msgr2=0x55b19d42f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 -- 172.21.15.86:0/132699236 shutdown_connections 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 1 -- 172.21.15.86:0/132699236 wait complete. 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262f575000 20 client.6549 trim_cache size 0 max 16384 2024-03-22T04:14:20.976 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: "49832eb9-0496-4b0f-b9a7-7c45b2a9903b": { 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "49832eb9-0496-4b0f-b9a7-7c45b2a9903b", 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair,force" 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:14:20.977 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:14:20.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 -- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2604067b20 msgr2=0x7f2604069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2604067b20 0x7f2604069fe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f26180af000 tx=0x7f26180a7000 comp rx=0 tx=0).stop 2024-03-22T04:14:20.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 -- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b3430 msgr2=0x7f262415b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b3430 0x7f262415b860 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f261000c4b0 tx=0x7f261000c970 comp rx=0 tx=0).stop 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 -- 172.21.15.86:0/2986578987 shutdown_connections 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2604067b20 0x7f2604069fe0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f262415d0b0 0x7f26241550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26240b3430 0x7f262415b860 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 --2- 172.21.15.86:0/2986578987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26240b2360 0x7f262415b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 -- 172.21.15.86:0/2986578987 >> 172.21.15.86:0/2986578987 conn(0x7f26240cc940 msgr2=0x7f26240cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 -- 172.21.15.86:0/2986578987 shutdown_connections 2024-03-22T04:14:20.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:20.968+0000 7f262c000640 1 -- 172.21.15.86:0/2986578987 wait complete. 2024-03-22T04:14:20.991 INFO:tasks.cephfs.filesystem:scrub status for tag:49832eb9-0496-4b0f-b9a7-7c45b2a9903b - {'path': '/', 'tag': '49832eb9-0496-4b0f-b9a7-7c45b2a9903b', 'options': 'recursive,repair,force'} 2024-03-22T04:14:27.808 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:14:27.812 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:14:50.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:14:50.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-22T04:14:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f539ac00640 1 Processor -- start 2024-03-22T04:14:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f539ac00640 1 -- start start 2024-03-22T04:14:51.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f539ac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f539415d0b0 0x7f53941671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f539ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539415d710 con 0x7f53940b2360 2024-03-22T04:14:51.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f539ac00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539415fe40 con 0x7f53940b3600 2024-03-22T04:14:51.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f539ac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f53940b2740 con 0x7f539415d0b0 2024-03-22T04:14:51.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f5393e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f53940b2360 0x7f53940b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52728/0 (socket says 172.21.15.86:52728) 2024-03-22T04:14:51.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f5393e00640 1 -- 172.21.15.86:0/1798728222 learned_addr learned my addr 172.21.15.86:0/1798728222 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:51.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f5398c00640 1 --2- 172.21.15.86:0/1798728222 >> v2:172.21.15.86:3300/0 conn(0x7f539415d0b0 0x7f53941671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f5392a00640 1 -- 172.21.15.86:0/1798728222 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797483657 0 0) 0x7f539415d710 con 0x7f53940b2360 2024-03-22T04:14:51.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f5392a00640 1 -- 172.21.15.86:0/1798728222 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5378003530 con 0x7f53940b2360 2024-03-22T04:14:51.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f5398c00640 1 -- 172.21.15.86:0/1798728222 >> v1:172.21.15.196:6789/0 conn(0x7f53940b3600 legacy=0x7f539415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f5398c00640 1 -- 172.21.15.86:0/1798728222 >> v1:172.21.15.86:6789/0 conn(0x7f53940b2360 legacy=0x7f53940b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.131+0000 7f5398c00640 1 -- 172.21.15.86:0/1798728222 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5394160120 con 0x7f539415d0b0 2024-03-22T04:14:51.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f5398c00640 1 --2- 172.21.15.86:0/1798728222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f539415d0b0 0x7f53941671d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f538c009d90 tx=0x7f538c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0010fa1eb8df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f5392a00640 1 -- 172.21.15.86:0/1798728222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0b2070 con 0x7f539415d0b0 2024-03-22T04:14:51.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f5392a00640 1 -- 172.21.15.86:0/1798728222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c0a5920 con 0x7f539415d0b0 2024-03-22T04:14:51.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f5392a00640 1 -- 172.21.15.86:0/1798728222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0a5cf0 con 0x7f539415d0b0 2024-03-22T04:14:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 -- 172.21.15.86:0/1798728222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f539415d0b0 msgr2=0x7f53941671d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 --2- 172.21.15.86:0/1798728222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f539415d0b0 0x7f53941671d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f538c009d90 tx=0x7f538c01c970 comp rx=0 tx=0).stop 2024-03-22T04:14:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 -- 172.21.15.86:0/1798728222 shutdown_connections 2024-03-22T04:14:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 --2- 172.21.15.86:0/1798728222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f539415d0b0 0x7f53941671d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 -- 172.21.15.86:0/1798728222 >> 172.21.15.86:0/1798728222 conn(0x7f53940cc940 msgr2=0x7f53940ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 -- 172.21.15.86:0/1798728222 shutdown_connections 2024-03-22T04:14:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 -- 172.21.15.86:0/1798728222 wait complete. 2024-03-22T04:14:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 Processor -- start 2024-03-22T04:14:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.135+0000 7f539ac00640 1 -- start start 2024-03-22T04:14:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f539ac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53940b2360 0x7f5394072510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f539ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53940b3600 0x7f539414ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f539ac00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539415d0b0 0x7f539414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f539ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5394160c00 con 0x7f53940b3600 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f539ac00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5394169f30 con 0x7f539415d0b0 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f539ac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5394148280 con 0x7f53940b2360 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53940b3600 0x7f539414ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53940b3600 0x7f539414ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48252/0 (socket says 172.21.15.86:48252) 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 -- 172.21.15.86:0/2988286683 learned_addr learned my addr 172.21.15.86:0/2988286683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5398c00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539415d0b0 0x7f539414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393e00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53940b2360 0x7f5394072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 -- 172.21.15.86:0/2988286683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53940b2360 msgr2=0x7f5394072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53940b2360 0x7f5394072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 -- 172.21.15.86:0/2988286683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539415d0b0 msgr2=0x7f539414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539415d0b0 0x7f539414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 -- 172.21.15.86:0/2988286683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539414d3b0 con 0x7f53940b3600 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f5393400640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53940b3600 0x7f539414ce70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f538000c420 tx=0x7f538000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f538be00640 1 -- 172.21.15.86:0/2988286683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53800052d0 con 0x7f53940b3600 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f538be00640 1 -- 172.21.15.86:0/2988286683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380005470 con 0x7f53940b3600 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f539ac00640 1 -- 172.21.15.86:0/2988286683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539414d650 con 0x7f53940b3600 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f538be00640 1 -- 172.21.15.86:0/2988286683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380007260 con 0x7f53940b3600 2024-03-22T04:14:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.139+0000 7f539ac00640 1 -- 172.21.15.86:0/2988286683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539414dc10 con 0x7f53940b3600 2024-03-22T04:14:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f538be00640 1 -- 172.21.15.86:0/2988286683 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5380007400 con 0x7f53940b3600 2024-03-22T04:14:51.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f538be00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5374067b20 0x7f5374069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f538be00640 1 -- 172.21.15.86:0/2988286683 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f53800856c0 con 0x7f53940b3600 2024-03-22T04:14:51.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393e00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5374067b20 0x7f5374069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f539dfc5000 1 Processor -- start 2024-03-22T04:14:51.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f539dfc5000 1 -- start start 2024-03-22T04:14:51.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393e00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5374067b20 0x7f5374069fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f53840011f0 tx=0x7f5384007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead5df80 0x55e2ead5e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead11f50 0x55e2ead12330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f539dfc5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2eab62c40 con 0x55e2ead5e8a0 2024-03-22T04:14:51.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f539dfc5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e2eaaffd90 con 0x55e2ead11f50 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f539dfc5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e2eab5ebf0 con 0x55e2ead5df80 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead11a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48260/0 (socket says 172.21.15.86:48260) 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead5df80 0x55e2ead5e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 -- 172.21.15.86:0/3799353070 learned_addr learned my addr 172.21.15.86:0/3799353070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5398c00640 1 --2- 172.21.15.86:0/3799353070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead11f50 0x55e2ead12330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 -- 172.21.15.86:0/3799353070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead5df80 msgr2=0x55e2ead5e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 --2- 172.21.15.86:0/3799353070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead5df80 0x55e2ead5e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 -- 172.21.15.86:0/3799353070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead11f50 msgr2=0x55e2ead12330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 --2- 172.21.15.86:0/3799353070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead11f50 0x55e2ead12330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 -- 172.21.15.86:0/3799353070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2eac0a130 con 0x55e2ead5e8a0 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5393400640 1 --2- 172.21.15.86:0/3799353070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead11a10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f538005db50 tx=0x7f538000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5533c5ce1f6859a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5388c00640 1 -- 172.21.15.86:0/3799353070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538006d020 con 0x55e2ead5e8a0 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5388c00640 1 -- 172.21.15.86:0/3799353070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380053a20 con 0x55e2ead5e8a0 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.143+0000 7f5388c00640 1 -- 172.21.15.86:0/3799353070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380054b70 con 0x55e2ead5e8a0 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- 172.21.15.86:0/3799353070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 msgr2=0x55e2ead11a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 --2- 172.21.15.86:0/3799353070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead11a10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f538005db50 tx=0x7f538000cf90 comp rx=0 tx=0).stop 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- 172.21.15.86:0/3799353070 shutdown_connections 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 --2- 172.21.15.86:0/3799353070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead11f50 0x55e2ead12330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 --2- 172.21.15.86:0/3799353070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead11a10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 --2- 172.21.15.86:0/3799353070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead5df80 0x55e2ead5e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- 172.21.15.86:0/3799353070 >> 172.21.15.86:0/3799353070 conn(0x55e2eabfb990 msgr2=0x55e2ead41150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- 172.21.15.86:0/3799353070 shutdown_connections 2024-03-22T04:14:51.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- 172.21.15.86:0/3799353070 wait complete. 2024-03-22T04:14:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- start start 2024-03-22T04:14:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 Processor -- start 2024-03-22T04:14:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead11f50 0x55e2ead199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead42380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2eab1e710 con 0x55e2ead5e8a0 2024-03-22T04:14:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e2eab20580 con 0x55e2ead5df80 2024-03-22T04:14:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e2eab67270 con 0x55e2ead11f50 2024-03-22T04:14:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 10 client.0 _ll_register_callbacks cb 0x55e2eabc4200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48274/0 (socket says 172.21.15.86:48274) 2024-03-22T04:14:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 -- 172.21.15.86:0/1989160495 learned_addr learned my addr 172.21.15.86:0/1989160495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5393e00640 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead11f50 0x55e2ead199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5393400640 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead42380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 -- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead11f50 msgr2=0x55e2ead199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead11f50 0x55e2ead199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 -- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 msgr2=0x55e2ead42380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead42380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 -- 172.21.15.86:0/1989160495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2eab3bd30 con 0x55e2ead5e8a0 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5393e00640 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead11f50 0x55e2ead199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f5398c00640 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead428c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f538c009bd0 tx=0x7f538c01b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21725a66af513abc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f537d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f537d600640 1 -- 172.21.15.86:0/1989160495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0b2040 con 0x55e2ead5e8a0 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- 172.21.15.86:0/1989160495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5380053a20 con 0x55e2ead5e8a0 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f537d600640 1 -- 172.21.15.86:0/1989160495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f538c0a5c00 con 0x55e2ead5e8a0 2024-03-22T04:14:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f537d600640 1 -- 172.21.15.86:0/1989160495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f538c0cd050 con 0x55e2ead5e8a0 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 10 client.6564 fetch_fsmap learned FSMap version 37 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- 172.21.15.86:0/1989160495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55e2eaba7fc0 con 0x55e2ead5e8a0 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f537d600640 1 -- 172.21.15.86:0/1989160495 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380053a20 con 0x55e2ead5e8a0 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f537d600640 1 -- 172.21.15.86:0/1989160495 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7f538c0d0020 con 0x55e2ead5e8a0 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 10 client.6564 fetch_fsmap finished waiting for FSMap version 37 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 10 client.6564 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 20 client.6564 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e2ead645c0 0x55e2ead649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 4 client.6564 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.147+0000 7f539dfc5000 1 -- 172.21.15.86:0/1989160495 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2eaafddd0 con 0x55e2ead645c0 2024-03-22T04:14:51.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f5393e00640 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e2ead645c0 0x55e2ead649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f5393e00640 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e2ead645c0 0x55e2ead649a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e6f583b268fb57 server_cookie=24a231809637b985 in_seq=0 out_seq=0 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f537d600640 10 client.6564 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f537d600640 1 -- 172.21.15.86:0/1989160495 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e2eaafddd0 con 0x55e2ead645c0 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f537d600640 10 client.6564 handle_command_reply: tid=0 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 client.6564 shutdown 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 -- 172.21.15.86:0/1989160495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 msgr2=0x55e2ead428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead428c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f538c009bd0 tx=0x7f538c01b1b0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 -- 172.21.15.86:0/1989160495 shutdown_connections 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e2ead645c0 0x55e2ead649a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead42380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead5e8a0 0x55e2ead428c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f537d600640 0 client.6564 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:51.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 --2- 172.21.15.86:0/1989160495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2ead11f50 0x55e2ead199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 -- 172.21.15.86:0/1989160495 >> 172.21.15.86:0/1989160495 conn(0x55e2eabfb990 msgr2=0x55e2ead11110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 -- 172.21.15.86:0/1989160495 shutdown_connections 2024-03-22T04:14:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 1 -- 172.21.15.86:0/1989160495 wait complete. 2024-03-22T04:14:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.151+0000 7f539dfc5000 20 client.6564 trim_cache size 0 max 16384 2024-03-22T04:14:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f539dfc5000 1 Processor -- start 2024-03-22T04:14:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f539dfc5000 1 -- start start 2024-03-22T04:14:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 0x55e2ead74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead18e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f539dfc5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5394203fb0 con 0x55e2ead11f50 2024-03-22T04:14:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f539dfc5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5394202fc0 con 0x55e2ead5df80 2024-03-22T04:14:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f539dfc5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53941546a0 con 0x55e2eabfb990 2024-03-22T04:14:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f5393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f5393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead188d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48276/0 (socket says 172.21.15.86:48276) 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f5393400640 1 -- 172.21.15.86:0/3077063881 learned_addr learned my addr 172.21.15.86:0/3077063881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.167+0000 7f5393e00640 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 0x55e2ead74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5398c00640 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead18e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 -- 172.21.15.86:0/3077063881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 msgr2=0x55e2ead74680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 0x55e2ead74680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 -- 172.21.15.86:0/3077063881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 msgr2=0x55e2ead18e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead18e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 -- 172.21.15.86:0/3077063881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2eab383a0 con 0x55e2ead11f50 2024-03-22T04:14:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead188d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5380007fd0 tx=0x7f5380058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa462fac42d7cda server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f537fe00640 1 -- 172.21.15.86:0/3077063881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538006d070 con 0x55e2ead11f50 2024-03-22T04:14:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f537fe00640 1 -- 172.21.15.86:0/3077063881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380051d90 con 0x55e2ead11f50 2024-03-22T04:14:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f537fe00640 1 -- 172.21.15.86:0/3077063881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53800554f0 con 0x55e2ead11f50 2024-03-22T04:14:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- 172.21.15.86:0/3077063881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 msgr2=0x55e2ead188d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead188d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5380007fd0 tx=0x7f5380058000 comp rx=0 tx=0).stop 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- 172.21.15.86:0/3077063881 shutdown_connections 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead18e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead188d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 --2- 172.21.15.86:0/3077063881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 0x55e2ead74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- 172.21.15.86:0/3077063881 >> 172.21.15.86:0/3077063881 conn(0x55e2ead41400 msgr2=0x55e2ead181c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- 172.21.15.86:0/3077063881 shutdown_connections 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- 172.21.15.86:0/3077063881 wait complete. 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- start start 2024-03-22T04:14:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 Processor -- start 2024-03-22T04:14:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 0x55e2ead67a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead67f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead68480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2eac1e810 con 0x55e2ead11f50 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5394202a10 con 0x55e2ead5df80 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e2eab1e710 con 0x55e2eabfb990 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f539dfc5000 10 client.0 _ll_register_callbacks cb 0x7f538c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead67f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead67f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48282/0 (socket says 172.21.15.86:48282) 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 -- 172.21.15.86:0/4141351628 learned_addr learned my addr 172.21.15.86:0/4141351628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5398c00640 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead68480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393e00640 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 0x55e2ead67a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 -- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 msgr2=0x55e2ead67a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 0x55e2ead67a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 -- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 msgr2=0x55e2ead68480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead68480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 -- 172.21.15.86:0/4141351628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2eaae8380 con 0x55e2ead11f50 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f5393400640 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead67f40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f53800539d0 tx=0x7f5380000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659dc6131bf8aaf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.171+0000 7f537d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 1 -- 172.21.15.86:0/4141351628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538006d070 con 0x55e2ead11f50 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 1 -- 172.21.15.86:0/4141351628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f5380051470 con 0x55e2ead11f50 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 -- 172.21.15.86:0/4141351628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5380051d90 con 0x55e2ead11f50 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 1 -- 172.21.15.86:0/4141351628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f53800554f0 con 0x55e2ead11f50 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 10 client.6570 fetch_fsmap learned FSMap version 37 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 -- 172.21.15.86:0/4141351628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55e2eabe4460 con 0x55e2ead11f50 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 1 -- 172.21.15.86:0/4141351628 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380051d90 con 0x55e2ead11f50 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 1 -- 172.21.15.86:0/4141351628 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7f5380051670 con 0x55e2ead11f50 2024-03-22T04:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 10 client.6570 fetch_fsmap finished waiting for FSMap version 37 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 10 client.6570 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 20 client.6570 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e2ead5e8a0 0x55e2ead689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 4 client.6570 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 -- 172.21.15.86:0/4141351628 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2eab02160 con 0x55e2ead5e8a0 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f5393e00640 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e2ead5e8a0 0x55e2ead689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 10 client.6570 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f5393e00640 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e2ead5e8a0 0x55e2ead689c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f13f23b70d091c server_cookie=3f0e9b2a33a05dc4 in_seq=0 out_seq=0 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 1 -- 172.21.15.86:0/4141351628 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e2eab02160 con 0x55e2ead5e8a0 2024-03-22T04:14:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 10 client.6570 handle_command_reply: tid=0 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 client.6570 shutdown 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 -- 172.21.15.86:0/4141351628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 msgr2=0x55e2ead67f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead67f40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f53800539d0 tx=0x7f5380000bd0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 -- 172.21.15.86:0/4141351628 shutdown_connections 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55e2ead5e8a0 0x55e2ead689c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2ead5df80 0x55e2ead68480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ead11f50 0x55e2ead67f40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 --2- 172.21.15.86:0/4141351628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e2eabfb990 0x55e2ead67a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f537d600640 0 client.6570 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 -- 172.21.15.86:0/4141351628 >> 172.21.15.86:0/4141351628 conn(0x55e2ead41400 msgr2=0x55e2ead18a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 -- 172.21.15.86:0/4141351628 shutdown_connections 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 1 -- 172.21.15.86:0/4141351628 wait complete. 2024-03-22T04:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539dfc5000 20 client.6570 trim_cache size 0 max 16384 2024-03-22T04:14:51.185 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:14:51.185 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:14:51.185 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:14:51.185 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:14:51.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 -- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5374067b20 msgr2=0x7f5374069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5374067b20 0x7f5374069fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f53840011f0 tx=0x7f5384007d40 comp rx=0 tx=0).stop 2024-03-22T04:14:51.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 -- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53940b3600 msgr2=0x7f539414ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53940b3600 0x7f539414ce70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f538000c420 tx=0x7f538000c8e0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 -- 172.21.15.86:0/2988286683 shutdown_connections 2024-03-22T04:14:51.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5374067b20 0x7f5374069fe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539415d0b0 0x7f539414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53940b3600 0x7f539414ce70 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 --2- 172.21.15.86:0/2988286683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53940b2360 0x7f5394072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 -- 172.21.15.86:0/2988286683 >> 172.21.15.86:0/2988286683 conn(0x7f53940cc940 msgr2=0x7f53940b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 -- 172.21.15.86:0/2988286683 shutdown_connections 2024-03-22T04:14:51.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.175+0000 7f539ac00640 1 -- 172.21.15.86:0/2988286683 wait complete. 2024-03-22T04:14:51.200 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:14:51.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:14:51.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6703200640 1 Processor -- start 2024-03-22T04:14:51.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6703200640 1 -- start start 2024-03-22T04:14:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6703200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f670415e3a0 0x7f67040affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6703200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f670406bd70 0x7f670406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6703200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670415ebb0 con 0x7f67040b0680 2024-03-22T04:14:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6703200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f67040b04e0 con 0x7f670415e3a0 2024-03-22T04:14:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6703200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f670415ee90 con 0x7f670406bd70 2024-03-22T04:14:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6701e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f670415e3a0 0x7f67040affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6701e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f670415e3a0 0x7f67040affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48298/0 (socket says 172.21.15.86:48298) 2024-03-22T04:14:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6701e00640 1 -- 172.21.15.86:0/2360379748 learned_addr learned my addr 172.21.15.86:0/2360379748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6702800640 1 --2- 172.21.15.86:0/2360379748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f670406bd70 0x7f670406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6701e00640 1 -- 172.21.15.86:0/2360379748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f670406bd70 msgr2=0x7f670406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6701e00640 1 --2- 172.21.15.86:0/2360379748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f670406bd70 0x7f670406fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6701e00640 1 -- 172.21.15.86:0/2360379748 >> v1:172.21.15.196:6789/0 conn(0x7f67040b0680 legacy=0x7f670406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.331+0000 7f6701e00640 1 -- 172.21.15.86:0/2360379748 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670415f170 con 0x7f670415e3a0 2024-03-22T04:14:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6701e00640 1 --2- 172.21.15.86:0/2360379748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670415e3a0 0x7f67040affa0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009870 tx=0x7f66f401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de78b72994a2e4d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6700a00640 1 -- 172.21.15.86:0/2360379748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40b0070 con 0x7f670415e3a0 2024-03-22T04:14:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6700a00640 1 -- 172.21.15.86:0/2360379748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f40ab600 con 0x7f670415e3a0 2024-03-22T04:14:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6700a00640 1 -- 172.21.15.86:0/2360379748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40ab9d0 con 0x7f670415e3a0 2024-03-22T04:14:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 -- 172.21.15.86:0/2360379748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670415e3a0 msgr2=0x7f67040affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 --2- 172.21.15.86:0/2360379748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670415e3a0 0x7f67040affa0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009870 tx=0x7f66f401c4d0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 -- 172.21.15.86:0/2360379748 shutdown_connections 2024-03-22T04:14:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 --2- 172.21.15.86:0/2360379748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f670406bd70 0x7f670406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 --2- 172.21.15.86:0/2360379748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670415e3a0 0x7f67040affa0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 -- 172.21.15.86:0/2360379748 >> 172.21.15.86:0/2360379748 conn(0x7f67040d1000 msgr2=0x7f67040d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 -- 172.21.15.86:0/2360379748 shutdown_connections 2024-03-22T04:14:51.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 -- 172.21.15.86:0/2360379748 wait complete. 2024-03-22T04:14:51.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 Processor -- start 2024-03-22T04:14:51.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.335+0000 7f6703200640 1 -- start start 2024-03-22T04:14:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670406bd70 0x7f67041532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67040b0680 0x7f6704156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670415e3a0 0x7f670415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6704070aa0 con 0x7f670406bd70 2024-03-22T04:14:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f670415e990 con 0x7f670415e3a0 2024-03-22T04:14:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6704070c20 con 0x7f67040b0680 2024-03-22T04:14:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670406bd70 0x7f67041532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670406bd70 0x7f67041532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48306/0 (socket says 172.21.15.86:48306) 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 -- 172.21.15.86:0/884112209 learned_addr learned my addr 172.21.15.86:0/884112209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6702800640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670415e3a0 0x7f670415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701400640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67040b0680 0x7f6704156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 -- 172.21.15.86:0/884112209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67040b0680 msgr2=0x7f6704156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67040b0680 0x7f6704156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 -- 172.21.15.86:0/884112209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670415e3a0 msgr2=0x7f670415caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670415e3a0 0x7f670415caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 -- 172.21.15.86:0/884112209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67041577b0 con 0x7f670406bd70 2024-03-22T04:14:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6701e00640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670406bd70 0x7f67041532d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f66f40ae040 tx=0x7f66f4019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f66fa000640 1 -- 172.21.15.86:0/884112209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40bb070 con 0x7f670406bd70 2024-03-22T04:14:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f66fa000640 1 -- 172.21.15.86:0/884112209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f40b6430 con 0x7f670406bd70 2024-03-22T04:14:51.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67041571f0 con 0x7f670406bd70 2024-03-22T04:14:51.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f66fa000640 1 -- 172.21.15.86:0/884112209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f40b0040 con 0x7f670406bd70 2024-03-22T04:14:51.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670415c2a0 con 0x7f670406bd70 2024-03-22T04:14:51.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6704074470 con 0x7f670406bd70 2024-03-22T04:14:51.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.339+0000 7f66fa000640 1 -- 172.21.15.86:0/884112209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f66f40be070 con 0x7f670406bd70 2024-03-22T04:14:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.343+0000 7f66fa000640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f66e4067b50 0x7f66e406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.343+0000 7f66fa000640 1 -- 172.21.15.86:0/884112209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f66f411f570 con 0x7f670406bd70 2024-03-22T04:14:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.343+0000 7f6701400640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f66e4067b50 0x7f66e406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.343+0000 7f6701400640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f66e4067b50 0x7f66e406a010 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f67041544b0 tx=0x7f66f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.347+0000 7f66fa000640 1 -- 172.21.15.86:0/884112209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f67041577b0 con 0x7f670406bd70 2024-03-22T04:14:51.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.535+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67040d1b20 con 0x7f670406bd70 2024-03-22T04:14:51.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.535+0000 7f66fa000640 1 -- 172.21.15.86:0/884112209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5186 (secure 0 0 0) 0x7f66f4016150 con 0x7f670406bd70 2024-03-22T04:14:51.541 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:51.541 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:14:24.562106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:14:51.541 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 37 2024-03-22T04:14:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f66e4067b50 msgr2=0x7f66e406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f66e4067b50 0x7f66e406a010 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f67041544b0 tx=0x7f66f0009040 comp rx=0 tx=0).stop 2024-03-22T04:14:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670406bd70 msgr2=0x7f67041532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670406bd70 0x7f67041532d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f66f40ae040 tx=0x7f66f4019270 comp rx=0 tx=0).stop 2024-03-22T04:14:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 shutdown_connections 2024-03-22T04:14:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f66e4067b50 0x7f66e406a010 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670415e3a0 0x7f670415caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67040b0680 0x7f6704156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 --2- 172.21.15.86:0/884112209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670406bd70 0x7f67041532d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 >> 172.21.15.86:0/884112209 conn(0x7f67040d1000 msgr2=0x7f6704160a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 shutdown_connections 2024-03-22T04:14:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.539+0000 7f6703200640 1 -- 172.21.15.86:0/884112209 wait complete. 2024-03-22T04:14:51.562 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-03-22T04:14:51.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.695+0000 7fafde400640 1 Processor -- start 2024-03-22T04:14:51.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.695+0000 7fafde400640 1 -- start start 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafde400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fafd815c730 0x7fafd809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafde400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafd80b3610 0x7fafd809afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafde400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd80cda00 con 0x7fafd80b11c0 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafde400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fafd809b910 con 0x7fafd815c730 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafde400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fafd80b1710 con 0x7fafd80b3610 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fafd815c730 0x7fafd809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fafd80b11c0 0x7fafd80d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52736/0 (socket says 172.21.15.86:52736) 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7e00640 1 -- 172.21.15.86:0/205938040 learned_addr learned my addr 172.21.15.86:0/205938040 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fafd815c730 0x7fafd809aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48314/0 (socket says 172.21.15.86:48314) 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd6a00640 1 --2- 172.21.15.86:0/205938040 >> v2:172.21.15.196:3300/0 conn(0x7fafd80b3610 0x7fafd809afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7400640 1 -- 172.21.15.86:0/205938040 >> v2:172.21.15.196:3300/0 conn(0x7fafd80b3610 msgr2=0x7fafd809afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7400640 1 --2- 172.21.15.86:0/205938040 >> v2:172.21.15.196:3300/0 conn(0x7fafd80b3610 0x7fafd809afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7400640 1 -- 172.21.15.86:0/205938040 >> v1:172.21.15.86:6789/0 conn(0x7fafd80b11c0 legacy=0x7fafd80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7400640 1 -- 172.21.15.86:0/205938040 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd80cdce0 con 0x7fafd815c730 2024-03-22T04:14:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd6000640 1 -- 172.21.15.86:0/205938040 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 984464448 0 0) 0x7fafd80cda00 con 0x7fafd80b11c0 2024-03-22T04:14:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.699+0000 7fafd7400640 1 --2- 172.21.15.86:0/205938040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd809aa70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fafc80097c0 tx=0x7fafc801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427dc78715770cd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafd6000640 1 -- 172.21.15.86:0/205938040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80b1070 con 0x7fafd815c730 2024-03-22T04:14:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafd6000640 1 -- 172.21.15.86:0/205938040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc80ac620 con 0x7fafd815c730 2024-03-22T04:14:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafd6000640 1 -- 172.21.15.86:0/205938040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80ac9f0 con 0x7fafd815c730 2024-03-22T04:14:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- 172.21.15.86:0/205938040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 msgr2=0x7fafd809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 --2- 172.21.15.86:0/205938040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd809aa70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fafc80097c0 tx=0x7fafc801c3a0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- 172.21.15.86:0/205938040 shutdown_connections 2024-03-22T04:14:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 --2- 172.21.15.86:0/205938040 >> v2:172.21.15.196:3300/0 conn(0x7fafd80b3610 0x7fafd809afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 --2- 172.21.15.86:0/205938040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd809aa70 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- 172.21.15.86:0/205938040 >> 172.21.15.86:0/205938040 conn(0x7fafd80b3bf0 msgr2=0x7fafd80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- 172.21.15.86:0/205938040 shutdown_connections 2024-03-22T04:14:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- 172.21.15.86:0/205938040 wait complete. 2024-03-22T04:14:51.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 Processor -- start 2024-03-22T04:14:51.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- start start 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd80b11c0 0x7fafd8154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd80b3610 0x7fafd80b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8169510 con 0x7fafd815c730 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8169390 con 0x7fafd80b11c0 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafde400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fafd8169690 con 0x7fafd80b3610 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd80b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48326/0 (socket says 172.21.15.86:48326) 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafd7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd80b11c0 0x7fafd8154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafd7e00640 1 -- 172.21.15.86:0/2718640738 learned_addr learned my addr 172.21.15.86:0/2718640738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.703+0000 7fafd6a00640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd80b3610 0x7fafd80b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafd7e00640 1 -- 172.21.15.86:0/2718640738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd80b3610 msgr2=0x7fafd80b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafd7e00640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd80b3610 0x7fafd80b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafd7e00640 1 -- 172.21.15.86:0/2718640738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd80b11c0 msgr2=0x7fafd8154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafd7e00640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd80b11c0 0x7fafd8154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafd7e00640 1 -- 172.21.15.86:0/2718640738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd8159350 con 0x7fafd815c730 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafd7400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd80b11c0 0x7fafd8154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafd7e00640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd80b4f80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fafd000c700 tx=0x7fafd000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafcf400640 1 -- 172.21.15.86:0/2718640738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0018070 con 0x7fafd815c730 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafcf400640 1 -- 172.21.15.86:0/2718640738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd000ce10 con 0x7fafd815c730 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafde400640 1 -- 172.21.15.86:0/2718640738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd81564b0 con 0x7fafd815c730 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafcf400640 1 -- 172.21.15.86:0/2718640738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd000a860 con 0x7fafd815c730 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafde400640 1 -- 172.21.15.86:0/2718640738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd8158b50 con 0x7fafd815c730 2024-03-22T04:14:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafcf400640 1 -- 172.21.15.86:0/2718640738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fafd001b070 con 0x7fafd815c730 2024-03-22T04:14:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafcf400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafb8067b20 0x7fafb8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafcf400640 1 -- 172.21.15.86:0/2718640738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fafd00855d0 con 0x7fafd815c730 2024-03-22T04:14:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.707+0000 7fafd7400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafb8067b20 0x7fafb8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 Processor -- start 2024-03-22T04:14:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafb8067b20 0x7fafb8069fe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ae000 tx=0x7fafc80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- start start 2024-03-22T04:14:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219980b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219934130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219934a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557219781d80 con 0x557219981070 2024-03-22T04:14:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55721971ece0 con 0x557219934670 2024-03-22T04:14:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55721977dd30 con 0x557219980750 2024-03-22T04:14:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219934130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219980b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219934130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48334/0 (socket says 172.21.15.86:48334) 2024-03-22T04:14:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd6a00640 1 -- 172.21.15.86:0/1364524040 learned_addr learned my addr 172.21.15.86:0/1364524040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7e00640 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219934a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7400640 1 -- 172.21.15.86:0/1364524040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 msgr2=0x557219934a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7400640 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219934a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7400640 1 -- 172.21.15.86:0/1364524040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 msgr2=0x557219934130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7400640 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219934130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7400640 1 -- 172.21.15.86:0/1364524040 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721975c340 con 0x557219980750 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd6a00640 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219934130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafd7400640 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219980b30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ac000 tx=0x7fafc8004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2d6432836509c60 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafc3e00640 1 -- 172.21.15.86:0/1364524040 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80b1040 con 0x557219980750 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafc3e00640 1 -- 172.21.15.86:0/1364524040 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc80b2040 con 0x557219980750 2024-03-22T04:14:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafc3e00640 1 -- 172.21.15.86:0/1364524040 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80b7ae0 con 0x557219980750 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- 172.21.15.86:0/1364524040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 msgr2=0x557219980b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219980b30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ac000 tx=0x7fafc8004740 comp rx=0 tx=0).stop 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- 172.21.15.86:0/1364524040 shutdown_connections 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219934a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219934130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 --2- 172.21.15.86:0/1364524040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219980b30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- 172.21.15.86:0/1364524040 >> 172.21.15.86:0/1364524040 conn(0x5572199614f0 msgr2=0x5572199638e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- 172.21.15.86:0/1364524040 shutdown_connections 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.711+0000 7fafe1842000 1 -- 172.21.15.86:0/1364524040 wait complete. 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 -- start start 2024-03-22T04:14:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 Processor -- start 2024-03-22T04:14:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x557219938540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219965780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 0x557219965cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572197863b0 con 0x557219934670 2024-03-22T04:14:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55721973f4e0 con 0x557219981070 2024-03-22T04:14:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5572198023b0 con 0x557219980750 2024-03-22T04:14:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 10 client.0 _ll_register_callbacks cb 0x55721991dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x557219938540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x557219938540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48342/0 (socket says 172.21.15.86:48342) 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219965780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 -- 172.21.15.86:0/3963438599 learned_addr learned my addr 172.21.15.86:0/3963438599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7e00640 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 0x557219965cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 -- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 msgr2=0x557219965780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219965780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 -- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 msgr2=0x557219965cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 0x557219965cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 -- 172.21.15.86:0/3963438599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572197c65e0 con 0x557219934670 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd7400640 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x557219938540 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fafc801c3e0 tx=0x7fafc801abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972ac7461658d2d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafc0c00640 1 -- 172.21.15.86:0/3963438599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80b1040 con 0x557219934670 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafc0c00640 1 -- 172.21.15.86:0/3963438599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fafc80c7040 con 0x557219934670 2024-03-22T04:14:51.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 -- 172.21.15.86:0/3963438599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc80b2040 con 0x557219934670 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafc0c00640 1 -- 172.21.15.86:0/3963438599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fafc80d6050 con 0x557219934670 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 10 client.6591 fetch_fsmap learned FSMap version 37 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 -- 172.21.15.86:0/3963438599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55721973d5a0 con 0x557219934670 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafc0c00640 1 -- 172.21.15.86:0/3963438599 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc80b2040 con 0x557219934670 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafc0c00640 1 -- 172.21.15.86:0/3963438599 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7fafc80d9020 con 0x557219934670 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 10 client.6591 fetch_fsmap finished waiting for FSMap version 37 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 10 client.6591 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 20 client.6591 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55721998fc00 0x557219991ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 4 client.6591 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafe1842000 1 -- 172.21.15.86:0/3963438599 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55721971ce40 con 0x55721998fc00 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.715+0000 7fafd6a00640 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55721998fc00 0x557219991ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafd6a00640 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55721998fc00 0x557219991ff0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=544c53636cbd6254 server_cookie=af60b243ad2ed318 in_seq=0 out_seq=0 2024-03-22T04:14:51.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafc0c00640 10 client.6591 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafc0c00640 1 -- 172.21.15.86:0/3963438599 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55721971ce40 con 0x55721998fc00 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafc0c00640 10 client.6591 handle_command_reply: tid=0 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 client.6591 shutdown 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 -- 172.21.15.86:0/3963438599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 msgr2=0x557219938540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x557219938540 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fafc801c3e0 tx=0x7fafc801abe0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 -- 172.21.15.86:0/3963438599 shutdown_connections 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x55721998fc00 0x557219991ff0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 0x557219965cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219965780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 --2- 172.21.15.86:0/3963438599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x557219938540 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafc0c00640 0 client.6591 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 -- 172.21.15.86:0/3963438599 >> 172.21.15.86:0/3963438599 conn(0x5572199614f0 msgr2=0x5572199638e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 -- 172.21.15.86:0/3963438599 shutdown_connections 2024-03-22T04:14:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 1 -- 172.21.15.86:0/3963438599 wait complete. 2024-03-22T04:14:51.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.719+0000 7fafe1842000 20 client.6591 trim_cache size 0 max 16384 2024-03-22T04:14:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafe1842000 1 Processor -- start 2024-03-22T04:14:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafe1842000 1 -- start start 2024-03-22T04:14:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafe1842000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219960a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafe1842000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x5572199656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafe1842000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219965c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafe1842000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8200000 con 0x557219981070 2024-03-22T04:14:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafe1842000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81fefe0 con 0x557219934670 2024-03-22T04:14:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafe1842000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fafd806e360 con 0x557219980750 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219965c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafd7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219960a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219965c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48348/0 (socket says 172.21.15.86:48348) 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafd7e00640 1 -- 172.21.15.86:0/2761594304 learned_addr learned my addr 172.21.15.86:0/2761594304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.731+0000 7fafd6a00640 1 --2- 172.21.15.86:0/2761594304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x5572199656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 -- 172.21.15.86:0/2761594304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 msgr2=0x5572199656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 --2- 172.21.15.86:0/2761594304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x5572199656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 -- 172.21.15.86:0/2761594304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 msgr2=0x557219965c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 --2- 172.21.15.86:0/2761594304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219965c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 -- 172.21.15.86:0/2761594304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557219980ea0 con 0x557219934670 2024-03-22T04:14:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 --2- 172.21.15.86:0/2761594304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219960a00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fafc801c880 tx=0x7fafc800b050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df2bc63de9307df server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafc3400640 1 -- 172.21.15.86:0/2761594304 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80de070 con 0x557219934670 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafc3400640 1 -- 172.21.15.86:0/2761594304 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc80cb890 con 0x557219934670 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafc3400640 1 -- 172.21.15.86:0/2761594304 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80b1040 con 0x557219934670 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- 172.21.15.86:0/2761594304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 msgr2=0x557219960a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 --2- 172.21.15.86:0/2761594304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219960a00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fafc801c880 tx=0x7fafc800b050 comp rx=0 tx=0).stop 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- 172.21.15.86:0/2761594304 shutdown_connections 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 --2- 172.21.15.86:0/2761594304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219981070 0x557219965c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 --2- 172.21.15.86:0/2761594304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x5572199656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 --2- 172.21.15.86:0/2761594304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219934670 0x557219960a00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- 172.21.15.86:0/2761594304 >> 172.21.15.86:0/2761594304 conn(0x5572199614f0 msgr2=0x557219964fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- 172.21.15.86:0/2761594304 shutdown_connections 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- 172.21.15.86:0/2761594304 wait complete. 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- start start 2024-03-22T04:14:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 Processor -- start 2024-03-22T04:14:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x5572199ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219993040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 0x557219993580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55721978e490 con 0x557219934670 2024-03-22T04:14:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81fe930 con 0x557219981070 2024-03-22T04:14:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x557219780c60 con 0x557219980750 2024-03-22T04:14:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafe1842000 10 client.0 _ll_register_callbacks cb 0x7fafc4009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:14:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x5572199ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x5572199ef650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48354/0 (socket says 172.21.15.86:48354) 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7400640 1 -- 172.21.15.86:0/4010618656 learned_addr learned my addr 172.21.15.86:0/4010618656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd6a00640 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219993040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.735+0000 7fafd7e00640 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 0x557219993580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafd7400640 1 -- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 msgr2=0x557219993040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafd7400640 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219993040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafd7400640 1 -- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 msgr2=0x557219993580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafd7400640 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 0x557219993580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafd7400640 1 -- 172.21.15.86:0/4010618656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721973da50 con 0x557219934670 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafd7400640 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x5572199ef650 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fafc80092d0 tx=0x7fafc8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41082cd97b514b07 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 1 -- 172.21.15.86:0/4010618656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80de070 con 0x557219934670 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 1 -- 172.21.15.86:0/4010618656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fafc80e3d40 con 0x557219934670 2024-03-22T04:14:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 -- 172.21.15.86:0/4010618656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc80cb890 con 0x557219934670 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 1 -- 172.21.15.86:0/4010618656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fafc80b1050 con 0x557219934670 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 10 client.6597 fetch_fsmap learned FSMap version 37 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 -- 172.21.15.86:0/4010618656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x557219802840 con 0x557219934670 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 1 -- 172.21.15.86:0/4010618656 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc80cb890 con 0x557219934670 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 1 -- 172.21.15.86:0/4010618656 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 3494+0+0 (secure 0 0 0) 0x7fafc80d9030 con 0x557219934670 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 10 client.6597 fetch_fsmap finished waiting for FSMap version 37 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 10 client.6597 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 20 client.6597 populate_metadata read hostname 'smithi086' 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x557219993ac0 0x557219993ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 4 client.6597 mds_command: new command op to 75617 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 -- 172.21.15.86:0/4010618656 --> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55721978f2c0 con 0x557219993ac0 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafd6a00640 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x557219993ac0 0x557219993ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafd6a00640 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x557219993ac0 0x557219993ea0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9086bf39bad012e9 server_cookie=665fd76122ec964b in_seq=0 out_seq=0 2024-03-22T04:14:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 10 client.6597 ms_handle_connect on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 1 -- 172.21.15.86:0/4010618656 <== mds.0 v2:172.21.15.196:6832/3774275697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55721978f2c0 con 0x557219993ac0 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 10 client.6597 handle_command_reply: tid=0 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 client.6597 shutdown 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 -- 172.21.15.86:0/4010618656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 msgr2=0x5572199ef650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x5572199ef650 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fafc80092d0 tx=0x7fafc8009190 comp rx=0 tx=0).stop 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 -- 172.21.15.86:0/4010618656 shutdown_connections 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:6832/3774275697,v1:172.21.15.196:6835/3774275697] conn(0x557219993ac0 0x557219993ea0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557219981070 0x557219993580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557219980750 0x557219993040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafc0c00640 0 client.6597 ms_handle_reset on v2:172.21.15.196:6832/3774275697 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 --2- 172.21.15.86:0/4010618656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557219934670 0x5572199ef650 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 -- 172.21.15.86:0/4010618656 >> 172.21.15.86:0/4010618656 conn(0x5572199614f0 msgr2=0x5572199656d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 -- 172.21.15.86:0/4010618656 shutdown_connections 2024-03-22T04:14:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 1 -- 172.21.15.86:0/4010618656 wait complete. 2024-03-22T04:14:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafe1842000 20 client.6597 trim_cache size 0 max 16384 2024-03-22T04:14:51.749 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 -- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafb8067b20 msgr2=0x7fafb8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafb8067b20 0x7fafb8069fe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ae000 tx=0x7fafc80a7000 comp rx=0 tx=0).stop 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 -- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 msgr2=0x7fafd80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd80b4f80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fafd000c700 tx=0x7fafd000cbc0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 -- 172.21.15.86:0/2718640738 shutdown_connections 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fafb8067b20 0x7fafb8069fe0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafd815c730 0x7fafd80b4f80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd80b3610 0x7fafd80b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 --2- 172.21.15.86:0/2718640738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd80b11c0 0x7fafd8154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 -- 172.21.15.86:0/2718640738 >> 172.21.15.86:0/2718640738 conn(0x7fafd80b3bf0 msgr2=0x7fafd815b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 -- 172.21.15.86:0/2718640738 shutdown_connections 2024-03-22T04:14:51.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.739+0000 7fafde400640 1 -- 172.21.15.86:0/2718640738 wait complete. 2024-03-22T04:14:51.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-22T04:14:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc8ec00640 1 Processor -- start 2024-03-22T04:14:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc8ec00640 1 -- start start 2024-03-22T04:14:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc8ec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcc8815c310 0x7fcc880b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc8ec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc880d1610 con 0x7fcc88071ae0 2024-03-22T04:14:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc8ec00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc880d18f0 con 0x7fcc880cf350 2024-03-22T04:14:51.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc8ec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcc88071f40 con 0x7fcc8815c310 2024-03-22T04:14:51.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc87400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcc88071ae0 0x7fcc880cec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52746/0 (socket says 172.21.15.86:52746) 2024-03-22T04:14:51.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc87400640 1 -- 172.21.15.86:0/3009192215 learned_addr learned my addr 172.21.15.86:0/3009192215 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.895+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3009192215 >> v2:172.21.15.196:3300/0 conn(0x7fcc8815c310 0x7fcc880b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc86a00640 1 -- 172.21.15.86:0/3009192215 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367405549 0 0) 0x7fcc880d1610 con 0x7fcc88071ae0 2024-03-22T04:14:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc87e00640 1 -- 172.21.15.86:0/3009192215 >> v1:172.21.15.196:6789/0 conn(0x7fcc880cf350 legacy=0x7fcc881600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc87e00640 1 -- 172.21.15.86:0/3009192215 >> v1:172.21.15.86:6789/0 conn(0x7fcc88071ae0 legacy=0x7fcc880cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc87e00640 1 -- 172.21.15.86:0/3009192215 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc880cc920 con 0x7fcc8815c310 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3009192215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc8815c310 0x7fcc880b5100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcc780097c0 tx=0x7fcc7801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd54f560f423048f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc86a00640 1 -- 172.21.15.86:0/3009192215 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc780b2070 con 0x7fcc8815c310 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc86a00640 1 -- 172.21.15.86:0/3009192215 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc780ad680 con 0x7fcc8815c310 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc86a00640 1 -- 172.21.15.86:0/3009192215 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc780ada50 con 0x7fcc8815c310 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3009192215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc8815c310 msgr2=0x7fcc880b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.899+0000 7fcc8ec00640 1 --2- 172.21.15.86:0/3009192215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc8815c310 0x7fcc880b5100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcc780097c0 tx=0x7fcc7801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3009192215 shutdown_connections 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 --2- 172.21.15.86:0/3009192215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc8815c310 0x7fcc880b5100 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3009192215 >> 172.21.15.86:0/3009192215 conn(0x7fcc8806f930 msgr2=0x7fcc8806fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3009192215 shutdown_connections 2024-03-22T04:14:51.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3009192215 wait complete. 2024-03-22T04:14:51.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 Processor -- start 2024-03-22T04:14:51.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- start start 2024-03-22T04:14:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc88071ae0 0x7fcc880b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc880cf350 0x7fcc881434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc8815c310 0x7fcc88159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8816a240 con 0x7fcc88071ae0 2024-03-22T04:14:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8816a0c0 con 0x7fcc880cf350 2024-03-22T04:14:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8816a3c0 con 0x7fcc8815c310 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc88071ae0 0x7fcc880b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc88071ae0 0x7fcc880b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48366/0 (socket says 172.21.15.86:48366) 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 -- 172.21.15.86:0/3925129535 learned_addr learned my addr 172.21.15.86:0/3925129535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87400640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc880cf350 0x7fcc881434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8cc00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc8815c310 0x7fcc88159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 -- 172.21.15.86:0/3925129535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc8815c310 msgr2=0x7fcc88159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc8815c310 0x7fcc88159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 -- 172.21.15.86:0/3925129535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc880cf350 msgr2=0x7fcc881434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc880cf350 0x7fcc881434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 -- 172.21.15.86:0/3925129535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc880b6910 con 0x7fcc88071ae0 2024-03-22T04:14:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc88071ae0 0x7fcc880b63d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcc780190b0 tx=0x7fcc7801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc7fe00640 1 -- 172.21.15.86:0/3925129535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc780b2040 con 0x7fcc88071ae0 2024-03-22T04:14:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc7fe00640 1 -- 172.21.15.86:0/3925129535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc780bd070 con 0x7fcc88071ae0 2024-03-22T04:14:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.903+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc881439f0 con 0x7fcc88071ae0 2024-03-22T04:14:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.907+0000 7fcc7fe00640 1 -- 172.21.15.86:0/3925129535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc780b8450 con 0x7fcc88071ae0 2024-03-22T04:14:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.907+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc880b5610 con 0x7fcc88071ae0 2024-03-22T04:14:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.907+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc88074550 con 0x7fcc88071ae0 2024-03-22T04:14:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.907+0000 7fcc7fe00640 1 -- 172.21.15.86:0/3925129535 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcc780b85f0 con 0x7fcc88071ae0 2024-03-22T04:14:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.907+0000 7fcc7fe00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcc68067b20 0x7fcc68069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.907+0000 7fcc7fe00640 1 -- 172.21.15.86:0/3925129535 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fcc781202a0 con 0x7fcc88071ae0 2024-03-22T04:14:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.907+0000 7fcc87400640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcc68067b20 0x7fcc68069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.907+0000 7fcc87400640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcc68067b20 0x7fcc68069fe0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcc880d0e80 tx=0x7fcc74007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:51.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:51.911+0000 7fcc7fe00640 1 -- 172.21.15.86:0/3925129535 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcc78016090 con 0x7fcc88071ae0 2024-03-22T04:14:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.103+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcc8815fc90 con 0x7fcc88071ae0 2024-03-22T04:14:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.103+0000 7fcc7fe00640 1 -- 172.21.15.86:0/3925129535 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fcc780f0420 con 0x7fcc88071ae0 2024-03-22T04:14:52.109 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:52.109 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T04:14:52.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcc68067b20 msgr2=0x7fcc68069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcc68067b20 0x7fcc68069fe0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcc880d0e80 tx=0x7fcc74007240 comp rx=0 tx=0).stop 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc88071ae0 msgr2=0x7fcc880b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc88071ae0 0x7fcc880b63d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcc780190b0 tx=0x7fcc7801ace0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 shutdown_connections 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcc68067b20 0x7fcc68069fe0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc880cf350 0x7fcc881434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc8815c310 0x7fcc88159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 --2- 172.21.15.86:0/3925129535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc88071ae0 0x7fcc880b63d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 >> 172.21.15.86:0/3925129535 conn(0x7fcc8806f930 msgr2=0x7fcc880cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 shutdown_connections 2024-03-22T04:14:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.107+0000 7fcc8ec00640 1 -- 172.21.15.86:0/3925129535 wait complete. 2024-03-22T04:14:52.129 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-22T04:14:52.130 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:14:52.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:14:52.173 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:14:52.272 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:14:52.272 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:14:52.273 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:14:52.273 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:14:52.282 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:14:52.282 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:14:52.334 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:14:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.507+0000 7ff6c7c00640 1 Processor -- start 2024-03-22T04:14:52.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c7c00640 1 -- start start 2024-03-22T04:14:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c7c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c015b3f0 0x7ff6c00729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c7c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6c00ae540 0x7ff6c0072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c7c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c0071920 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c7c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c00716d0 con 0x7ff6c00ae540 2024-03-22T04:14:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c7c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c006d990 con 0x7ff6c015b3f0 2024-03-22T04:14:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6c00ae540 0x7ff6c0072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6c00ae540 0x7ff6c0072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48378/0 (socket says 172.21.15.86:48378) 2024-03-22T04:14:52.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5c00640 1 -- 172.21.15.86:0/830743914 learned_addr learned my addr 172.21.15.86:0/830743914 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6bfe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff6c00cc3e0 0x7ff6c00ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52734/0 (socket says 172.21.15.86:52734) 2024-03-22T04:14:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5200640 1 --2- 172.21.15.86:0/830743914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c015b3f0 0x7ff6c00729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5c00640 1 -- 172.21.15.86:0/830743914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c015b3f0 msgr2=0x7ff6c00729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5c00640 1 --2- 172.21.15.86:0/830743914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c015b3f0 0x7ff6c00729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5c00640 1 -- 172.21.15.86:0/830743914 >> v1:172.21.15.196:6789/0 conn(0x7ff6c00cc3e0 legacy=0x7ff6c00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5c00640 1 -- 172.21.15.86:0/830743914 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c0071c00 con 0x7ff6c00ae540 2024-03-22T04:14:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.511+0000 7ff6c5200640 1 --2- 172.21.15.86:0/830743914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c015b3f0 0x7ff6c00729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c5c00640 1 --2- 172.21.15.86:0/830743914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00ae540 0x7ff6c0072f00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009e50 tx=0x7ff6b801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfcc821057b1cf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6bf400640 1 -- 172.21.15.86:0/830743914 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80a5750 con 0x7ff6c00ae540 2024-03-22T04:14:52.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6bf400640 1 -- 172.21.15.86:0/830743914 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b80a58f0 con 0x7ff6c00ae540 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6bf400640 1 -- 172.21.15.86:0/830743914 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80a5cc0 con 0x7ff6c00ae540 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 -- 172.21.15.86:0/830743914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00ae540 msgr2=0x7ff6c0072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/830743914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00ae540 0x7ff6c0072f00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009e50 tx=0x7ff6b801c9a0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 -- 172.21.15.86:0/830743914 shutdown_connections 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/830743914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00ae540 0x7ff6c0072f00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/830743914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c015b3f0 0x7ff6c00729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 -- 172.21.15.86:0/830743914 >> 172.21.15.86:0/830743914 conn(0x7ff6c00b1df0 msgr2=0x7ff6c00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 -- 172.21.15.86:0/830743914 shutdown_connections 2024-03-22T04:14:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 -- 172.21.15.86:0/830743914 wait complete. 2024-03-22T04:14:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 Processor -- start 2024-03-22T04:14:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.515+0000 7ff6c7c00640 1 -- start start 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c00ae540 0x7ff6c0099540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00cc3e0 0x7ff6c0151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6c015b3f0 0x7ff6c0155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0168580 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0168400 con 0x7ff6c015b3f0 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c0168700 con 0x7ff6c00ae540 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00cc3e0 0x7ff6c0151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00cc3e0 0x7ff6c0151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48390/0 (socket says 172.21.15.86:48390) 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 -- 172.21.15.86:0/3717466679 learned_addr learned my addr 172.21.15.86:0/3717466679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c5200640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c00ae540 0x7ff6c0099540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c5c00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6c015b3f0 0x7ff6c0155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 -- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c00ae540 msgr2=0x7ff6c0099540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c00ae540 0x7ff6c0099540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 -- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6c015b3f0 msgr2=0x7ff6c0155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6c015b3f0 0x7ff6c0155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 -- 172.21.15.86:0/3717466679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c0153720 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c5c00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6c015b3f0 0x7ff6c0155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bfe00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00cc3e0 0x7ff6c0151810 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c370 tx=0x7ff6ac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bcc00640 1 -- 172.21.15.86:0/3717466679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0074e0 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bcc00640 1 -- 172.21.15.86:0/3717466679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ac007680 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c0153a00 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bcc00640 1 -- 172.21.15.86:0/3717466679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004a30 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c0153fc0 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c0068650 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.519+0000 7ff6bcc00640 1 -- 172.21.15.86:0/3717466679 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff6ac01a020 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.523+0000 7ff6bcc00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6a0067b20 0x7ff6a0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.523+0000 7ff6bcc00640 1 -- 172.21.15.86:0/3717466679 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7ff6ac0858e0 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.523+0000 7ff6c5200640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6a0067b20 0x7ff6a0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.523+0000 7ff6c5200640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6a0067b20 0x7ff6a0069fe0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40045f0 tx=0x7ff6b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:52.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.527+0000 7ff6bcc00640 1 -- 172.21.15.86:0/3717466679 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff6ac056020 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.687+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6c00b0d90 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6bcc00640 1 -- 172.21.15.86:0/3717466679 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17268 (secure 0 0 0) 0x7ff6ac055920 con 0x7ff6c00cc3e0 2024-03-22T04:14:52.695 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:52.695 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":69,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:14:16.029299+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:13:34.224743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-22T04:13:34.852940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.2","mappings":[{"from":2,"to":7}]},{"pgid":"10.a","mappings":[{"from":2,"to":1}]},{"pgid":"10.10","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":5}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.26","mappings":[{"from":4,"to":1}]},{"pgid":"10.2d","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:14:52.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6a0067b20 msgr2=0x7ff6a0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6a0067b20 0x7ff6a0069fe0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40045f0 tx=0x7ff6b4008040 comp rx=0 tx=0).stop 2024-03-22T04:14:52.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00cc3e0 msgr2=0x7ff6c0151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00cc3e0 0x7ff6c0151810 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c370 tx=0x7ff6ac00c830 comp rx=0 tx=0).stop 2024-03-22T04:14:52.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 shutdown_connections 2024-03-22T04:14:52.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6a0067b20 0x7ff6a0069fe0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6c015b3f0 0x7ff6c0155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c00cc3e0 0x7ff6c0151810 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 --2- 172.21.15.86:0/3717466679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6c00ae540 0x7ff6c0099540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 >> 172.21.15.86:0/3717466679 conn(0x7ff6c00b1df0 msgr2=0x7ff6c00b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:52.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 shutdown_connections 2024-03-22T04:14:52.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.691+0000 7ff6c7c00640 1 -- 172.21.15.86:0/3717466679 wait complete. 2024-03-22T04:14:52.715 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:13:34.224743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:13:34.852940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:14:52.716 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:14:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.847+0000 7f451f000640 1 Processor -- start 2024-03-22T04:14:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.847+0000 7f451f000640 1 -- start start 2024-03-22T04:14:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4518162740 0x7f4518169380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45180cc7c0 con 0x7f45180c6f20 2024-03-22T04:14:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451f000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45180cb720 con 0x7f45180cb1d0 2024-03-22T04:14:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f45180cba00 con 0x7f4518162740 2024-03-22T04:14:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4518162740 0x7f4518169380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f4517e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f45180c6f20 0x7f4518072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52762/0 (socket says 172.21.15.86:52762) 2024-03-22T04:14:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4518162740 0x7f4518169380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48404/0 (socket says 172.21.15.86:48404) 2024-03-22T04:14:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f4517e00640 1 -- 172.21.15.86:0/1429682767 learned_addr learned my addr 172.21.15.86:0/1429682767 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451d000640 1 -- 172.21.15.86:0/1429682767 >> v1:172.21.15.196:6789/0 conn(0x7f45180cb1d0 legacy=0x7f4518162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451d000640 1 -- 172.21.15.86:0/1429682767 >> v1:172.21.15.86:6789/0 conn(0x7f45180c6f20 legacy=0x7f4518072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.851+0000 7f451d000640 1 -- 172.21.15.86:0/1429682767 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45180cbce0 con 0x7f4518162740 2024-03-22T04:14:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451d000640 1 --2- 172.21.15.86:0/1429682767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4518162740 0x7f4518169380 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4510009d90 tx=0x7f451001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784af0801ebcc8a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f4516a00640 1 -- 172.21.15.86:0/1429682767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100b2070 con 0x7f4518162740 2024-03-22T04:14:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f4516a00640 1 -- 172.21.15.86:0/1429682767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45100a5920 con 0x7f4518162740 2024-03-22T04:14:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f4516a00640 1 -- 172.21.15.86:0/1429682767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100a5cf0 con 0x7f4518162740 2024-03-22T04:14:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 -- 172.21.15.86:0/1429682767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4518162740 msgr2=0x7f4518169380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 --2- 172.21.15.86:0/1429682767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4518162740 0x7f4518169380 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4510009d90 tx=0x7f451001c970 comp rx=0 tx=0).stop 2024-03-22T04:14:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 -- 172.21.15.86:0/1429682767 shutdown_connections 2024-03-22T04:14:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 --2- 172.21.15.86:0/1429682767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4518162740 0x7f4518169380 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 -- 172.21.15.86:0/1429682767 >> 172.21.15.86:0/1429682767 conn(0x7f45180d0730 msgr2=0x7f45180d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 -- 172.21.15.86:0/1429682767 shutdown_connections 2024-03-22T04:14:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 -- 172.21.15.86:0/1429682767 wait complete. 2024-03-22T04:14:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 Processor -- start 2024-03-22T04:14:52.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 -- start start 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.855+0000 7f451f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45180c6f20 0x7f4518153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451f000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180cb1d0 0x7f45180c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451f000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518162740 0x7f4518158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f451816c670 con 0x7f45180c6f20 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451f000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f451816c4f0 con 0x7f4518162740 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451f000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f451816c7f0 con 0x7f45180cb1d0 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45180c6f20 0x7f4518153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45180c6f20 0x7f4518153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48414/0 (socket says 172.21.15.86:48414) 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 -- 172.21.15.86:0/2662167612 learned_addr learned my addr 172.21.15.86:0/2662167612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:52.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517400640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180cb1d0 0x7f45180c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451d000640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518162740 0x7f4518158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 -- 172.21.15.86:0/2662167612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180cb1d0 msgr2=0x7f45180c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180cb1d0 0x7f45180c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 -- 172.21.15.86:0/2662167612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518162740 msgr2=0x7f4518158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518162740 0x7f4518158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 -- 172.21.15.86:0/2662167612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45180c8170 con 0x7f45180c6f20 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f4517e00640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45180c6f20 0x7f4518153c00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f450800c420 tx=0x7f450800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f450fe00640 1 -- 172.21.15.86:0/2662167612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508017070 con 0x7f45180c6f20 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f450fe00640 1 -- 172.21.15.86:0/2662167612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4508005cf0 con 0x7f45180c6f20 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f450fe00640 1 -- 172.21.15.86:0/2662167612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45080072f0 con 0x7f45180c6f20 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451806e060 con 0x7f45180c6f20 2024-03-22T04:14:52.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4518154140 con 0x7f45180c6f20 2024-03-22T04:14:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4518074470 con 0x7f45180c6f20 2024-03-22T04:14:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.859+0000 7f450fe00640 1 -- 172.21.15.86:0/2662167612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f450801a070 con 0x7f45180c6f20 2024-03-22T04:14:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.863+0000 7f450fe00640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f8067b20 0x7f44f8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.863+0000 7f450fe00640 1 -- 172.21.15.86:0/2662167612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f45080854f0 con 0x7f45180c6f20 2024-03-22T04:14:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.863+0000 7f4517400640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f8067b20 0x7f44f8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.863+0000 7f4517400640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f8067b20 0x7f44f8069fe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f45180b5be0 tx=0x7f4504009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:52.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:52.867+0000 7f450fe00640 1 -- 172.21.15.86:0/2662167612 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f45080555d0 con 0x7f45180c6f20 2024-03-22T04:14:53.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.019+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45180afb10 con 0x7f45180c6f20 2024-03-22T04:14:53.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.019+0000 7f450fe00640 1 -- 172.21.15.86:0/2662167612 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f4508059670 con 0x7f45180c6f20 2024-03-22T04:14:53.025 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:14:53.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f8067b20 msgr2=0x7f44f8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f8067b20 0x7f44f8069fe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f45180b5be0 tx=0x7f4504009040 comp rx=0 tx=0).stop 2024-03-22T04:14:53.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45180c6f20 msgr2=0x7f4518153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45180c6f20 0x7f4518153c00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f450800c420 tx=0x7f450800c8e0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 shutdown_connections 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f8067b20 0x7f44f8069fe0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4518162740 0x7f4518158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180cb1d0 0x7f45180c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 --2- 172.21.15.86:0/2662167612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45180c6f20 0x7f4518153c00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 >> 172.21.15.86:0/2662167612 conn(0x7f45180d0730 msgr2=0x7f4518156b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 shutdown_connections 2024-03-22T04:14:53.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.023+0000 7f451f000640 1 -- 172.21.15.86:0/2662167612 wait complete. 2024-03-22T04:14:53.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:14:53.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.179+0000 7fe9c1400640 1 Processor -- start 2024-03-22T04:14:53.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.179+0000 7fe9c1400640 1 -- start start 2024-03-22T04:14:53.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.179+0000 7fe9c1400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc06e7e0 0x7fe9bc06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.179+0000 7fe9c1400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe9bc072ad0 0x7fe9bc0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.179+0000 7fe9c1400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9bc15c770 con 0x7fe9bc072650 2024-03-22T04:14:53.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.179+0000 7fe9c1400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe9bc071350 con 0x7fe9bc072ad0 2024-03-22T04:14:53.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.179+0000 7fe9c1400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe9bc06f100 con 0x7fe9bc06e7e0 2024-03-22T04:14:53.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9ba800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc06e7e0 0x7fe9bc06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9ba800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc06e7e0 0x7fe9bc06ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:55288/0 (socket says 172.21.15.86:55288) 2024-03-22T04:14:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9b1e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe9bc072650 0x7fe9bc0b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52752/0 (socket says 172.21.15.86:52752) 2024-03-22T04:14:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9ba800640 1 -- 172.21.15.86:0/1398161839 learned_addr learned my addr 172.21.15.86:0/1398161839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9bb200640 1 --2- 172.21.15.86:0/1398161839 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc072ad0 0x7fe9bc0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9ba800640 1 -- 172.21.15.86:0/1398161839 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc072ad0 msgr2=0x7fe9bc0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9ba800640 1 --2- 172.21.15.86:0/1398161839 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc072ad0 0x7fe9bc0b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9b9e00640 1 -- 172.21.15.86:0/1398161839 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658634967 0 0) 0x7fe9a4001120 con 0x7fe9bc072650 2024-03-22T04:14:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9ba800640 1 -- 172.21.15.86:0/1398161839 >> v1:172.21.15.196:6789/0 conn(0x7fe9bc072650 legacy=0x7fe9bc0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9ba800640 1 -- 172.21.15.86:0/1398161839 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc073150 con 0x7fe9bc06e7e0 2024-03-22T04:14:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9ba800640 1 --2- 172.21.15.86:0/1398161839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc06e7e0 0x7fe9bc06ebc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00bb10 tx=0x7fe9ac01eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4383b8c56ba183cb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9b9e00640 1 -- 172.21.15.86:0/1398161839 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0b4070 con 0x7fe9bc06e7e0 2024-03-22T04:14:53.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9b9e00640 1 -- 172.21.15.86:0/1398161839 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0a7a60 con 0x7fe9bc06e7e0 2024-03-22T04:14:53.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.183+0000 7fe9b9e00640 1 -- 172.21.15.86:0/1398161839 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0a7e30 con 0x7fe9bc06e7e0 2024-03-22T04:14:53.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- 172.21.15.86:0/1398161839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc06e7e0 msgr2=0x7fe9bc06ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 --2- 172.21.15.86:0/1398161839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc06e7e0 0x7fe9bc06ebc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00bb10 tx=0x7fe9ac01eba0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- 172.21.15.86:0/1398161839 shutdown_connections 2024-03-22T04:14:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 --2- 172.21.15.86:0/1398161839 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc072ad0 0x7fe9bc0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 --2- 172.21.15.86:0/1398161839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc06e7e0 0x7fe9bc06ebc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- 172.21.15.86:0/1398161839 >> 172.21.15.86:0/1398161839 conn(0x7fe9bc0cc940 msgr2=0x7fe9bc0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- 172.21.15.86:0/1398161839 shutdown_connections 2024-03-22T04:14:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- 172.21.15.86:0/1398161839 wait complete. 2024-03-22T04:14:53.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 Processor -- start 2024-03-22T04:14:53.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- start start 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc06e7e0 0x7fe9bc1f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc072650 0x7fe9bc143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9bc072ad0 0x7fe9bc156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc072080 con 0x7fe9bc072ad0 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc0ce930 con 0x7fe9bc06e7e0 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9c1400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe9bc1f7890 con 0x7fe9bc072650 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9ba800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc06e7e0 0x7fe9bc1f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9bc072ad0 0x7fe9bc156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9b1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc072650 0x7fe9bc143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9ba800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc06e7e0 0x7fe9bc1f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:59732/0 (socket says 172.21.15.86:59732) 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9ba800640 1 -- 172.21.15.86:0/2753808374 learned_addr learned my addr 172.21.15.86:0/2753808374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9bb200640 1 -- 172.21.15.86:0/2753808374 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc072650 msgr2=0x7fe9bc143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9bb200640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc072650 0x7fe9bc143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9bb200640 1 -- 172.21.15.86:0/2753808374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc06e7e0 msgr2=0x7fe9bc1f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9bb200640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc06e7e0 0x7fe9bc1f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9bb200640 1 -- 172.21.15.86:0/2753808374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc155c40 con 0x7fe9bc072ad0 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.187+0000 7fe9ba800640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc06e7e0 0x7fe9bc1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9bb200640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9bc072ad0 0x7fe9bc156ce0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400eba0 tx=0x7fe9b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9b3400640 1 -- 172.21.15.86:0/2753808374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4018020 con 0x7fe9bc072ad0 2024-03-22T04:14:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9b3400640 1 -- 172.21.15.86:0/2753808374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b400ccc0 con 0x7fe9bc072ad0 2024-03-22T04:14:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc155f20 con 0x7fe9bc072ad0 2024-03-22T04:14:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9b3400640 1 -- 172.21.15.86:0/2753808374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40146c0 con 0x7fe9bc072ad0 2024-03-22T04:14:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc1564e0 con 0x7fe9bc072ad0 2024-03-22T04:14:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc074470 con 0x7fe9bc072ad0 2024-03-22T04:14:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9b3400640 1 -- 172.21.15.86:0/2753808374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe9b401b020 con 0x7fe9bc072ad0 2024-03-22T04:14:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9b3400640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe998067b20 0x7fe998069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9b3400640 1 -- 172.21.15.86:0/2753808374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fe9b4086c60 con 0x7fe9bc072ad0 2024-03-22T04:14:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.191+0000 7fe9ba800640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe998067b20 0x7fe998069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.195+0000 7fe9ba800640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe998067b20 0x7fe998069fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00bb10 tx=0x7fe9ac0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.195+0000 7fe9b3400640 1 -- 172.21.15.86:0/2753808374 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe9b4022030 con 0x7fe9bc072ad0 2024-03-22T04:14:53.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe9bc072200 con 0x7fe9bc072ad0 2024-03-22T04:14:53.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9b3400640 1 -- 172.21.15.86:0/2753808374 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7fe9b4010040 con 0x7fe9bc072ad0 2024-03-22T04:14:53.356 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe998067b20 msgr2=0x7fe998069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe998067b20 0x7fe998069fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00bb10 tx=0x7fe9ac0a9070 comp rx=0 tx=0).stop 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9bc072ad0 msgr2=0x7fe9bc156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9bc072ad0 0x7fe9bc156ce0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400eba0 tx=0x7fe9b400c490 comp rx=0 tx=0).stop 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 shutdown_connections 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe998067b20 0x7fe998069fe0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9bc072ad0 0x7fe9bc156ce0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc072650 0x7fe9bc143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 --2- 172.21.15.86:0/2753808374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc06e7e0 0x7fe9bc1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.351+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 >> 172.21.15.86:0/2753808374 conn(0x7fe9bc0cc940 msgr2=0x7fe9bc0cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.355+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 shutdown_connections 2024-03-22T04:14:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.355+0000 7fe9c1400640 1 -- 172.21.15.86:0/2753808374 wait complete. 2024-03-22T04:14:53.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:14:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.507+0000 7fb618400640 1 Processor -- start 2024-03-22T04:14:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb618400640 1 -- start start 2024-03-22T04:14:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb618400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b2360 0x7fb6100b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb618400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6100b3430 0x7fb61015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb618400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb61015d710 con 0x7fb61015cee0 2024-03-22T04:14:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb618400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb6100b3890 con 0x7fb6100b3430 2024-03-22T04:14:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb618400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb6100b2740 con 0x7fb6100b2360 2024-03-22T04:14:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb615000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6100b3430 0x7fb61015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb615000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6100b3430 0x7fb61015c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48432/0 (socket says 172.21.15.86:48432) 2024-03-22T04:14:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb615000640 1 -- 172.21.15.86:0/3007738159 learned_addr learned my addr 172.21.15.86:0/3007738159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb615a00640 1 --2- 172.21.15.86:0/3007738159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b2360 0x7fb6100b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb615000640 1 -- 172.21.15.86:0/3007738159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b2360 msgr2=0x7fb6100b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:53.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb615000640 1 --2- 172.21.15.86:0/3007738159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b2360 0x7fb6100b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb615000640 1 -- 172.21.15.86:0/3007738159 >> v1:172.21.15.86:6789/0 conn(0x7fb61015cee0 legacy=0x7fb61006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb615000640 1 -- 172.21.15.86:0/3007738159 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61015fe40 con 0x7fb6100b3430 2024-03-22T04:14:53.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.511+0000 7fb60fe00640 1 -- 172.21.15.86:0/3007738159 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2420438378 0 0) 0x7fb61015d710 con 0x7fb61015cee0 2024-03-22T04:14:53.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb615000640 1 --2- 172.21.15.86:0/3007738159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6100b3430 0x7fb61015c9a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009870 tx=0x7fb5fc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da449801b1a066f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb60fe00640 1 -- 172.21.15.86:0/3007738159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0ab650 con 0x7fb6100b3430 2024-03-22T04:14:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb60fe00640 1 -- 172.21.15.86:0/3007738159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0b0070 con 0x7fb6100b3430 2024-03-22T04:14:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb60fe00640 1 -- 172.21.15.86:0/3007738159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0aba10 con 0x7fb6100b3430 2024-03-22T04:14:53.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 -- 172.21.15.86:0/3007738159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6100b3430 msgr2=0x7fb61015c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 --2- 172.21.15.86:0/3007738159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6100b3430 0x7fb61015c9a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009870 tx=0x7fb5fc01c530 comp rx=0 tx=0).stop 2024-03-22T04:14:53.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 -- 172.21.15.86:0/3007738159 shutdown_connections 2024-03-22T04:14:53.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 --2- 172.21.15.86:0/3007738159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6100b3430 0x7fb61015c9a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 --2- 172.21.15.86:0/3007738159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b2360 0x7fb6100b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 -- 172.21.15.86:0/3007738159 >> 172.21.15.86:0/3007738159 conn(0x7fb6100cc940 msgr2=0x7fb6100ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 -- 172.21.15.86:0/3007738159 shutdown_connections 2024-03-22T04:14:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 -- 172.21.15.86:0/3007738159 wait complete. 2024-03-22T04:14:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 Processor -- start 2024-03-22T04:14:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.515+0000 7fb618400640 1 -- start start 2024-03-22T04:14:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6100b2360 0x7fb610072720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b3430 0x7fb61014bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb61015cee0 0x7fb61014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb610160c00 con 0x7fb61015cee0 2024-03-22T04:14:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6100b5d90 con 0x7fb6100b2360 2024-03-22T04:14:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb61014e580 con 0x7fb6100b3430 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb61015cee0 0x7fb61014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb615a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6100b2360 0x7fb610072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb61015cee0 0x7fb61014eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48436/0 (socket says 172.21.15.86:48436) 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 -- 172.21.15.86:0/1475471150 learned_addr learned my addr 172.21.15.86:0/1475471150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb615000640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b3430 0x7fb61014bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 -- 172.21.15.86:0/1475471150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b3430 msgr2=0x7fb61014bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b3430 0x7fb61014bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 -- 172.21.15.86:0/1475471150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6100b2360 msgr2=0x7fb610072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6100b2360 0x7fb610072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 -- 172.21.15.86:0/1475471150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb610071680 con 0x7fb61015cee0 2024-03-22T04:14:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb616400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb61015cee0 0x7fb61014eac0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb60800c7b0 tx=0x7fb60800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb60d600640 1 -- 172.21.15.86:0/1475471150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60800ce60 con 0x7fb61015cee0 2024-03-22T04:14:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb60d600640 1 -- 172.21.15.86:0/1475471150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb608003db0 con 0x7fb61015cee0 2024-03-22T04:14:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb60d600640 1 -- 172.21.15.86:0/1475471150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6080052b0 con 0x7fb61015cee0 2024-03-22T04:14:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb610071960 con 0x7fb61015cee0 2024-03-22T04:14:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb610071f20 con 0x7fb61015cee0 2024-03-22T04:14:53.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.519+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb610074470 con 0x7fb61015cee0 2024-03-22T04:14:53.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.523+0000 7fb60d600640 1 -- 172.21.15.86:0/1475471150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb608005450 con 0x7fb61015cee0 2024-03-22T04:14:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.523+0000 7fb60d600640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb5f0067b50 0x7fb5f006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.523+0000 7fb60d600640 1 -- 172.21.15.86:0/1475471150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fb608085e40 con 0x7fb61015cee0 2024-03-22T04:14:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.523+0000 7fb615a00640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb5f0067b50 0x7fb5f006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.523+0000 7fb615a00640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb5f0067b50 0x7fb5f006a010 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb61014d2f0 tx=0x7fb604009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.527+0000 7fb60d600640 1 -- 172.21.15.86:0/1475471150 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb608055fb0 con 0x7fb61015cee0 2024-03-22T04:14:53.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb61006e9f0 con 0x7fb61015cee0 2024-03-22T04:14:53.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb60d600640 1 -- 172.21.15.86:0/1475471150 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7fb60805a050 con 0x7fb61015cee0 2024-03-22T04:14:53.686 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb5f0067b50 msgr2=0x7fb5f006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb5f0067b50 0x7fb5f006a010 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb61014d2f0 tx=0x7fb604009190 comp rx=0 tx=0).stop 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb61015cee0 msgr2=0x7fb61014eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb61015cee0 0x7fb61014eac0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb60800c7b0 tx=0x7fb60800cc70 comp rx=0 tx=0).stop 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 shutdown_connections 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb5f0067b50 0x7fb5f006a010 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb61015cee0 0x7fb61014eac0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6100b3430 0x7fb61014bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 --2- 172.21.15.86:0/1475471150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6100b2360 0x7fb610072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 >> 172.21.15.86:0/1475471150 conn(0x7fb6100cc940 msgr2=0x7fb61014f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 shutdown_connections 2024-03-22T04:14:53.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.683+0000 7fb618400640 1 -- 172.21.15.86:0/1475471150 wait complete. 2024-03-22T04:14:53.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:14:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f095dc00640 1 Processor -- start 2024-03-22T04:14:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f095dc00640 1 -- start start 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f095dc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f095809b180 0x7f09580d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f095dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f095806f930 con 0x7f095815e630 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f095dc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f095806fc10 con 0x7f09580cde40 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f095dc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f09580cc9a0 con 0x7f095809b180 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0957e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f095815e630 0x7f095809aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52782/0 (socket says 172.21.15.86:52782) 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0957e00640 1 -- 172.21.15.86:0/1862706748 learned_addr learned my addr 172.21.15.86:0/1862706748 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0957400640 1 --2- 172.21.15.86:0/1862706748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f095809b180 0x7f09580d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0956a00640 1 -- 172.21.15.86:0/1862706748 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594844123 0 0) 0x7f095806f930 con 0x7f095815e630 2024-03-22T04:14:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0956a00640 1 -- 172.21.15.86:0/1862706748 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f093c003530 con 0x7f095815e630 2024-03-22T04:14:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0956a00640 1 -- 172.21.15.86:0/1862706748 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 587420599 0 0) 0x7f095806fc10 con 0x7f09580cde40 2024-03-22T04:14:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0956a00640 1 -- 172.21.15.86:0/1862706748 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f095806f930 con 0x7f09580cde40 2024-03-22T04:14:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0957400640 1 -- 172.21.15.86:0/1862706748 >> v1:172.21.15.196:6789/0 conn(0x7f09580cde40 legacy=0x7f09580d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0956a00640 1 -- 172.21.15.86:0/1862706748 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 706150533 0 0) 0x7f093c003530 con 0x7f095815e630 2024-03-22T04:14:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0957400640 1 -- 172.21.15.86:0/1862706748 >> v1:172.21.15.86:6789/0 conn(0x7f095815e630 legacy=0x7f095809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.839+0000 7f0957400640 1 -- 172.21.15.86:0/1862706748 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09580cc630 con 0x7f095809b180 2024-03-22T04:14:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f0957400640 1 --2- 172.21.15.86:0/1862706748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f095809b180 0x7f09580d0170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09440097f0 tx=0x7f094401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c1aeb688664655 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f0956a00640 1 -- 172.21.15.86:0/1862706748 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440b0070 con 0x7f095809b180 2024-03-22T04:14:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f0956a00640 1 -- 172.21.15.86:0/1862706748 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09440ab400 con 0x7f095809b180 2024-03-22T04:14:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f0956a00640 1 -- 172.21.15.86:0/1862706748 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948001680 con 0x7f095809b180 2024-03-22T04:14:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f095dc00640 1 -- 172.21.15.86:0/1862706748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f095809b180 msgr2=0x7f09580d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f095dc00640 1 --2- 172.21.15.86:0/1862706748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f095809b180 0x7f09580d0170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09440097f0 tx=0x7f094401c370 comp rx=0 tx=0).stop 2024-03-22T04:14:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f095dc00640 1 -- 172.21.15.86:0/1862706748 shutdown_connections 2024-03-22T04:14:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f095dc00640 1 --2- 172.21.15.86:0/1862706748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f095809b180 0x7f09580d0170 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f095dc00640 1 -- 172.21.15.86:0/1862706748 >> 172.21.15.86:0/1862706748 conn(0x7f09580b3ad0 msgr2=0x7f09580b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f095dc00640 1 -- 172.21.15.86:0/1862706748 shutdown_connections 2024-03-22T04:14:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.843+0000 7f095dc00640 1 -- 172.21.15.86:0/1862706748 wait complete. 2024-03-22T04:14:53.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 Processor -- start 2024-03-22T04:14:53.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 -- start start 2024-03-22T04:14:53.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095809b180 0x7f0958141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09580cde40 0x7f09580c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095815e630 0x7f0958154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f095816c380 con 0x7f095809b180 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f095816c200 con 0x7f095815e630 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f095816c500 con 0x7f09580cde40 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095809b180 0x7f0958141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095809b180 0x7f0958141690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48452/0 (socket says 172.21.15.86:48452) 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957e00640 1 -- 172.21.15.86:0/4077916954 learned_addr learned my addr 172.21.15.86:0/4077916954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957400640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09580cde40 0x7f09580c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095d200640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095815e630 0x7f0958154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957400640 1 -- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095815e630 msgr2=0x7f0958154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957400640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095815e630 0x7f0958154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957400640 1 -- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095809b180 msgr2=0x7f0958141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:53.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957400640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095809b180 0x7f0958141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957400640 1 -- 172.21.15.86:0/4077916954 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09580c7680 con 0x7f09580cde40 2024-03-22T04:14:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f0957400640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09580cde40 0x7f09580c7e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09440ab000 tx=0x7f094401abb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f094fe00640 1 -- 172.21.15.86:0/4077916954 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440b0050 con 0x7f09580cde40 2024-03-22T04:14:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f094fe00640 1 -- 172.21.15.86:0/4077916954 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09440b1040 con 0x7f09580cde40 2024-03-22T04:14:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095815a6e0 con 0x7f09580cde40 2024-03-22T04:14:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f094fe00640 1 -- 172.21.15.86:0/4077916954 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440b7690 con 0x7f09580cde40 2024-03-22T04:14:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.847+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f09580c7920 con 0x7f09580cde40 2024-03-22T04:14:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.851+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0958074470 con 0x7f09580cde40 2024-03-22T04:14:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.851+0000 7f094fe00640 1 -- 172.21.15.86:0/4077916954 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f09440be030 con 0x7f09580cde40 2024-03-22T04:14:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.851+0000 7f094fe00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0938067b20 0x7f0938069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.851+0000 7f094fe00640 1 -- 172.21.15.86:0/4077916954 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f094411fb50 con 0x7f09580cde40 2024-03-22T04:14:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.851+0000 7f0957e00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0938067b20 0x7f0938069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.851+0000 7f0957e00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0938067b20 0x7f0938069fe0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0948000fe0 tx=0x7f0948001b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:53.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:53.859+0000 7f094fe00640 1 -- 172.21.15.86:0/4077916954 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f09440c5020 con 0x7f09580cde40 2024-03-22T04:14:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.023+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f09580c7b60 con 0x7f09580cde40 2024-03-22T04:14:54.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f094fe00640 1 -- 172.21.15.86:0/4077916954 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f0944015ff0 con 0x7f09580cde40 2024-03-22T04:14:54.030 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:14:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0938067b20 msgr2=0x7f0938069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0938067b20 0x7f0938069fe0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0948000fe0 tx=0x7f0948001b10 comp rx=0 tx=0).stop 2024-03-22T04:14:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09580cde40 msgr2=0x7f09580c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09580cde40 0x7f09580c7e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09440ab000 tx=0x7f094401abb0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 shutdown_connections 2024-03-22T04:14:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0938067b20 0x7f0938069fe0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f095815e630 0x7f0958154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09580cde40 0x7f09580c7e40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 --2- 172.21.15.86:0/4077916954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f095809b180 0x7f0958141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 >> 172.21.15.86:0/4077916954 conn(0x7f09580b3ad0 msgr2=0x7f09580b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 shutdown_connections 2024-03-22T04:14:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.027+0000 7f095dc00640 1 -- 172.21.15.86:0/4077916954 wait complete. 2024-03-22T04:14:54.050 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:14:54.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:14:54.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:14:54.101 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:14:54.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:14:54.101 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:14:54.118 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:14:54.118 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:14:54.119 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:14:54.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:14:54.129 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:14:54.129 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:14:54.129 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:14:54.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:14:54.170 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:14:54.171 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:14:54.171 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:14:54.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:14:54.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:14:54.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7e5400640 1 Processor -- start 2024-03-22T04:14:54.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7e5400640 1 -- start start 2024-03-22T04:14:54.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7e5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb7e006e2f0 0x7fb7e0071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7e5400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e01623d0 con 0x7fb7e00d0580 2024-03-22T04:14:54.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7e5400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e01620f0 con 0x7fb7e015e630 2024-03-22T04:14:54.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7e5400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e015ec10 con 0x7fb7e006e2f0 2024-03-22T04:14:54.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7dde00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb7e00d0580 0x7fb7e0071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52798/0 (socket says 172.21.15.86:52798) 2024-03-22T04:14:54.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7dde00640 1 -- 172.21.15.86:0/3576855003 learned_addr learned my addr 172.21.15.86:0/3576855003 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:54.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7df200640 1 --2- 172.21.15.86:0/3576855003 >> v2:172.21.15.86:3300/0 conn(0x7fb7e006e2f0 0x7fb7e0071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7dd400640 1 -- 172.21.15.86:0/3576855003 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173051416 0 0) 0x7fb7e01623d0 con 0x7fb7e00d0580 2024-03-22T04:14:54.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7df200640 1 -- 172.21.15.86:0/3576855003 >> v1:172.21.15.196:6789/0 conn(0x7fb7e015e630 legacy=0x7fb7e00cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7df200640 1 -- 172.21.15.86:0/3576855003 >> v1:172.21.15.86:6789/0 conn(0x7fb7e00d0580 legacy=0x7fb7e0071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.343+0000 7fb7df200640 1 -- 172.21.15.86:0/3576855003 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e015eef0 con 0x7fb7e006e2f0 2024-03-22T04:14:54.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7df200640 1 --2- 172.21.15.86:0/3576855003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e006e2f0 0x7fb7e0071810 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009740 tx=0x7fb7d801c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae7fdf41e060107 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:54.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7dd400640 1 -- 172.21.15.86:0/3576855003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80b0070 con 0x7fb7e006e2f0 2024-03-22T04:14:54.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7dd400640 1 -- 172.21.15.86:0/3576855003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d80ab600 con 0x7fb7e006e2f0 2024-03-22T04:14:54.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7dd400640 1 -- 172.21.15.86:0/3576855003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80ab9d0 con 0x7fb7e006e2f0 2024-03-22T04:14:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7e5400640 1 -- 172.21.15.86:0/3576855003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e006e2f0 msgr2=0x7fb7e0071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7e5400640 1 --2- 172.21.15.86:0/3576855003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e006e2f0 0x7fb7e0071810 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009740 tx=0x7fb7d801c2d0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7e5400640 1 -- 172.21.15.86:0/3576855003 shutdown_connections 2024-03-22T04:14:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7e5400640 1 --2- 172.21.15.86:0/3576855003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e006e2f0 0x7fb7e0071810 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7e5400640 1 -- 172.21.15.86:0/3576855003 >> 172.21.15.86:0/3576855003 conn(0x7fb7e006f910 msgr2=0x7fb7e006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7e5400640 1 -- 172.21.15.86:0/3576855003 shutdown_connections 2024-03-22T04:14:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.347+0000 7fb7e5400640 1 -- 172.21.15.86:0/3576855003 wait complete. 2024-03-22T04:14:54.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 Processor -- start 2024-03-22T04:14:54.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 -- start start 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e006e2f0 0x7fb7e00ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e00d0580 0x7fb7e006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e015e630 0x7fb7e01594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e00cfaf0 con 0x7fb7e015e630 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e015f9d0 con 0x7fb7e00d0580 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e00cfc70 con 0x7fb7e006e2f0 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e015e630 0x7fb7e01594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e015e630 0x7fb7e01594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48472/0 (socket says 172.21.15.86:48472) 2024-03-22T04:14:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 -- 172.21.15.86:0/2250413510 learned_addr learned my addr 172.21.15.86:0/2250413510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7de800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e006e2f0 0x7fb7e00ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7dde00640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e00d0580 0x7fb7e006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 -- 172.21.15.86:0/2250413510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e006e2f0 msgr2=0x7fb7e00ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e006e2f0 0x7fb7e00ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 -- 172.21.15.86:0/2250413510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e00d0580 msgr2=0x7fb7e006d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e00d0580 0x7fb7e006d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 -- 172.21.15.86:0/2250413510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e00b3a80 con 0x7fb7e015e630 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7df200640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e015e630 0x7fb7e01594e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8018ee0 tx=0x7fb7d801ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7d6a00640 1 -- 172.21.15.86:0/2250413510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80b0050 con 0x7fb7e015e630 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7d6a00640 1 -- 172.21.15.86:0/2250413510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d80bc070 con 0x7fb7e015e630 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e00ca700 con 0x7fb7e015e630 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7d6a00640 1 -- 172.21.15.86:0/2250413510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80b5830 con 0x7fb7e015e630 2024-03-22T04:14:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.351+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e015c610 con 0x7fb7e015e630 2024-03-22T04:14:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.355+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e0068650 con 0x7fb7e015e630 2024-03-22T04:14:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.355+0000 7fb7d6a00640 1 -- 172.21.15.86:0/2250413510 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb7d80b59d0 con 0x7fb7e015e630 2024-03-22T04:14:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.355+0000 7fb7d6a00640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb7bc067b20 0x7fb7bc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.355+0000 7fb7d6a00640 1 -- 172.21.15.86:0/2250413510 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fb7d8120880 con 0x7fb7e015e630 2024-03-22T04:14:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.355+0000 7fb7de800640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb7bc067b20 0x7fb7bc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.355+0000 7fb7de800640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb7bc067b20 0x7fb7bc069fe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00012f0 tx=0x7fb7d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:54.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.359+0000 7fb7d6a00640 1 -- 172.21.15.86:0/2250413510 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb7d8015f50 con 0x7fb7e015e630 2024-03-22T04:14:54.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.543+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7e00ca9e0 con 0x7fb7e015e630 2024-03-22T04:14:54.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.543+0000 7fb7d6a00640 1 -- 172.21.15.86:0/2250413510 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5186 (secure 0 0 0) 0x7fb7d80f0a00 con 0x7fb7e015e630 2024-03-22T04:14:54.550 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:54.550 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:14:24.562106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:14:54.550 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 37 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb7bc067b20 msgr2=0x7fb7bc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb7bc067b20 0x7fb7bc069fe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00012f0 tx=0x7fb7d0009040 comp rx=0 tx=0).stop 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e015e630 msgr2=0x7fb7e01594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e015e630 0x7fb7e01594e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8018ee0 tx=0x7fb7d801ab10 comp rx=0 tx=0).stop 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 shutdown_connections 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb7bc067b20 0x7fb7bc069fe0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e00d0580 0x7fb7e006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e015e630 0x7fb7e01594e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 --2- 172.21.15.86:0/2250413510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e006e2f0 0x7fb7e00ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 >> 172.21.15.86:0/2250413510 conn(0x7fb7e006f910 msgr2=0x7fb7e0162600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:54.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 shutdown_connections 2024-03-22T04:14:54.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.547+0000 7fb7e5400640 1 -- 172.21.15.86:0/2250413510 wait complete. 2024-03-22T04:14:54.570 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:14:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8988400640 1 Processor -- start 2024-03-22T04:14:54.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8988400640 1 -- start start 2024-03-22T04:14:54.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8988400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad330 0x7f898015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8988400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f898006ef40 0x7f898006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8988400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f898015c7c0 con 0x7f89800ad710 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8988400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f898006dc40 con 0x7f898006ef40 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8988400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f898006dea0 con 0x7f89800ad330 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8985a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad330 0x7f898015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8985000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f898006ef40 0x7f898006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8986400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f89800ad710 0x7f898013fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52778/0 (socket says 172.21.15.86:52778) 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.703+0000 7f8986400640 1 -- 172.21.15.86:0/2389645746 learned_addr learned my addr 172.21.15.86:0/2389645746 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f897fe00640 1 -- 172.21.15.86:0/2389645746 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72925008 0 0) 0x7f898015c7c0 con 0x7f89800ad710 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f897fe00640 1 -- 172.21.15.86:0/2389645746 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8968003530 con 0x7f89800ad710 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f8985000640 1 -- 172.21.15.86:0/2389645746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad330 msgr2=0x7f898015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f8985000640 1 --2- 172.21.15.86:0/2389645746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad330 0x7f898015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f8985000640 1 -- 172.21.15.86:0/2389645746 >> v1:172.21.15.196:6789/0 conn(0x7f89800ad710 legacy=0x7f898013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f8985000640 1 -- 172.21.15.86:0/2389645746 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898006e180 con 0x7f898006ef40 2024-03-22T04:14:54.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f8985a00640 1 --2- 172.21.15.86:0/2389645746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad330 0x7f898015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f8985000640 1 --2- 172.21.15.86:0/2389645746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898006ef40 0x7f898006d700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f896c0097c0 tx=0x7f896c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c0784ddf32c6dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:54.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f897fe00640 1 -- 172.21.15.86:0/2389645746 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c0b2070 con 0x7f898006ef40 2024-03-22T04:14:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f897fe00640 1 -- 172.21.15.86:0/2389645746 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f896c0ad680 con 0x7f898006ef40 2024-03-22T04:14:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.707+0000 7f897fe00640 1 -- 172.21.15.86:0/2389645746 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c0ada50 con 0x7f898006ef40 2024-03-22T04:14:54.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- 172.21.15.86:0/2389645746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898006ef40 msgr2=0x7f898006d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 --2- 172.21.15.86:0/2389645746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898006ef40 0x7f898006d700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f896c0097c0 tx=0x7f896c01c470 comp rx=0 tx=0).stop 2024-03-22T04:14:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- 172.21.15.86:0/2389645746 shutdown_connections 2024-03-22T04:14:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 --2- 172.21.15.86:0/2389645746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f898006ef40 0x7f898006d700 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 --2- 172.21.15.86:0/2389645746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad330 0x7f898015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- 172.21.15.86:0/2389645746 >> 172.21.15.86:0/2389645746 conn(0x7f89800b1df0 msgr2=0x7f89800b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- 172.21.15.86:0/2389645746 shutdown_connections 2024-03-22T04:14:54.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- 172.21.15.86:0/2389645746 wait complete. 2024-03-22T04:14:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 Processor -- start 2024-03-22T04:14:54.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- start start 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898006ef40 0x7f89801fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89800ad330 0x7f898015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad710 0x7f898015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89801690e0 con 0x7f898006ef40 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8980168f60 con 0x7f89800ad330 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8988400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8980169260 con 0x7f89800ad710 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898006ef40 0x7f89801fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898006ef40 0x7f89801fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48482/0 (socket says 172.21.15.86:48482) 2024-03-22T04:14:54.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 -- 172.21.15.86:0/3374780370 learned_addr learned my addr 172.21.15.86:0/3374780370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985000640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89800ad330 0x7f898015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8986400640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad710 0x7f898015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 -- 172.21.15.86:0/3374780370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad710 msgr2=0x7f898015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad710 0x7f898015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 -- 172.21.15.86:0/3374780370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89800ad330 msgr2=0x7f898015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89800ad330 0x7f898015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 -- 172.21.15.86:0/3374780370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89801faa10 con 0x7f898006ef40 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.711+0000 7f8985a00640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898006ef40 0x7f89801fb4b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f897400e980 tx=0x7f897400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f897d600640 1 -- 172.21.15.86:0/3374780370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974017070 con 0x7f898006ef40 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f897d600640 1 -- 172.21.15.86:0/3374780370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897400ca00 con 0x7f898006ef40 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898009e210 con 0x7f898006ef40 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f897d600640 1 -- 172.21.15.86:0/3374780370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400cdd0 con 0x7f898006ef40 2024-03-22T04:14:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89801facb0 con 0x7f898006ef40 2024-03-22T04:14:54.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8980068650 con 0x7f898006ef40 2024-03-22T04:14:54.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f897d600640 1 -- 172.21.15.86:0/3374780370 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f897401a070 con 0x7f898006ef40 2024-03-22T04:14:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f897d600640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8960067b20 0x7f8960069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f897d600640 1 -- 172.21.15.86:0/3374780370 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f8974085fe0 con 0x7f898006ef40 2024-03-22T04:14:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f8985000640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8960067b20 0x7f8960069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.715+0000 7f8985000640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8960067b20 0x7f8960069fe0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f896c0af000 tx=0x7f896c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:54.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.719+0000 7f897d600640 1 -- 172.21.15.86:0/3374780370 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8974056160 con 0x7f898006ef40 2024-03-22T04:14:54.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.875+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89800cfcc0 con 0x7f898006ef40 2024-03-22T04:14:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f897d600640 1 -- 172.21.15.86:0/3374780370 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17268 (secure 0 0 0) 0x7f897405a200 con 0x7f898006ef40 2024-03-22T04:14:54.882 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:54.883 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":69,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:14:16.029299+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:13:34.224743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-22T04:13:34.852940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.2","mappings":[{"from":2,"to":7}]},{"pgid":"10.a","mappings":[{"from":2,"to":1}]},{"pgid":"10.10","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":5}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.26","mappings":[{"from":4,"to":1}]},{"pgid":"10.2d","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:14:54.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8960067b20 msgr2=0x7f8960069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8960067b20 0x7f8960069fe0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f896c0af000 tx=0x7f896c0a7000 comp rx=0 tx=0).stop 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898006ef40 msgr2=0x7f89801fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898006ef40 0x7f89801fb4b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f897400e980 tx=0x7f897400ee40 comp rx=0 tx=0).stop 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 shutdown_connections 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8960067b20 0x7f8960069fe0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89800ad330 0x7f898015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89800ad710 0x7f898015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 --2- 172.21.15.86:0/3374780370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f898006ef40 0x7f89801fb4b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 >> 172.21.15.86:0/3374780370 conn(0x7f89800b1df0 msgr2=0x7f8980072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 shutdown_connections 2024-03-22T04:14:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:54.879+0000 7f8988400640 1 -- 172.21.15.86:0/3374780370 wait complete. 2024-03-22T04:14:54.903 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:13:34.224743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:13:34.852940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:14:54.904 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:14:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.035+0000 7fc828e00640 1 Processor -- start 2024-03-22T04:14:55.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc828e00640 1 -- start start 2024-03-22T04:14:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc828e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8240b2360 0x7fc8240b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc828e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8240b3430 0x7fc82415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc828e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc82415d710 con 0x7fc82415cee0 2024-03-22T04:14:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc828e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc8240b2740 con 0x7fc8240b2360 2024-03-22T04:14:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc828e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc82415fe40 con 0x7fc8240b3430 2024-03-22T04:14:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc821e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8240b2360 0x7fc8240b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc821e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8240b2360 0x7fc8240b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48490/0 (socket says 172.21.15.86:48490) 2024-03-22T04:14:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc821e00640 1 -- 172.21.15.86:0/3494530241 learned_addr learned my addr 172.21.15.86:0/3494530241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc819400640 1 --2- 172.21.15.86:0/3494530241 >> v2:172.21.15.196:3300/0 conn(0x7fc8240b3430 0x7fc82415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc821e00640 1 -- 172.21.15.86:0/3494530241 >> v2:172.21.15.196:3300/0 conn(0x7fc8240b3430 msgr2=0x7fc82415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc821e00640 1 --2- 172.21.15.86:0/3494530241 >> v2:172.21.15.196:3300/0 conn(0x7fc8240b3430 0x7fc82415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc821e00640 1 -- 172.21.15.86:0/3494530241 >> v1:172.21.15.196:6789/0 conn(0x7fc82415cee0 legacy=0x7fc82406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.039+0000 7fc821e00640 1 -- 172.21.15.86:0/3494530241 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc824160120 con 0x7fc8240b2360 2024-03-22T04:14:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc821e00640 1 --2- 172.21.15.86:0/3494530241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b2360 0x7fc8240b2ef0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc8240afb10 tx=0x7fc81401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67f4f446660c648 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc821400640 1 -- 172.21.15.86:0/3494530241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8140b0070 con 0x7fc8240b2360 2024-03-22T04:14:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc821400640 1 -- 172.21.15.86:0/3494530241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8140ab600 con 0x7fc8240b2360 2024-03-22T04:14:55.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc821400640 1 -- 172.21.15.86:0/3494530241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8140ab9d0 con 0x7fc8240b2360 2024-03-22T04:14:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 -- 172.21.15.86:0/3494530241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b2360 msgr2=0x7fc8240b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 --2- 172.21.15.86:0/3494530241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b2360 0x7fc8240b2ef0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc8240afb10 tx=0x7fc81401c520 comp rx=0 tx=0).stop 2024-03-22T04:14:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 -- 172.21.15.86:0/3494530241 shutdown_connections 2024-03-22T04:14:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 --2- 172.21.15.86:0/3494530241 >> v2:172.21.15.196:3300/0 conn(0x7fc8240b3430 0x7fc82415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 --2- 172.21.15.86:0/3494530241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b2360 0x7fc8240b2ef0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 -- 172.21.15.86:0/3494530241 >> 172.21.15.86:0/3494530241 conn(0x7fc8240cc940 msgr2=0x7fc8240ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 -- 172.21.15.86:0/3494530241 shutdown_connections 2024-03-22T04:14:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 -- 172.21.15.86:0/3494530241 wait complete. 2024-03-22T04:14:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 Processor -- start 2024-03-22T04:14:55.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.043+0000 7fc828e00640 1 -- start start 2024-03-22T04:14:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8240b2360 0x7fc8241f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b3430 0x7fc8241492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc82415cee0 0x7fc824071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc824160c00 con 0x7fc8240b3430 2024-03-22T04:14:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc82415d2c0 con 0x7fc82415cee0 2024-03-22T04:14:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8241f79a0 con 0x7fc8240b2360 2024-03-22T04:14:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b3430 0x7fc8241492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b3430 0x7fc8241492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48500/0 (socket says 172.21.15.86:48500) 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 -- 172.21.15.86:0/1248561947 learned_addr learned my addr 172.21.15.86:0/1248561947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc821e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8240b2360 0x7fc8241f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc822800640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc82415cee0 0x7fc824071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 -- 172.21.15.86:0/1248561947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8240b2360 msgr2=0x7fc8241f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8240b2360 0x7fc8241f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 -- 172.21.15.86:0/1248561947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc82415cee0 msgr2=0x7fc824071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc82415cee0 0x7fc824071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 -- 172.21.15.86:0/1248561947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82415ebf0 con 0x7fc8240b3430 2024-03-22T04:14:55.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc819400640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b3430 0x7fc8241492d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00e870 tx=0x7fc80c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc81aa00640 1 -- 172.21.15.86:0/1248561947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c017070 con 0x7fc8240b3430 2024-03-22T04:14:55.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc81aa00640 1 -- 172.21.15.86:0/1248561947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc80c00c940 con 0x7fc8240b3430 2024-03-22T04:14:55.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc81aa00640 1 -- 172.21.15.86:0/1248561947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c00ccf0 con 0x7fc8240b3430 2024-03-22T04:14:55.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82415eed0 con 0x7fc8240b3430 2024-03-22T04:14:55.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82415f490 con 0x7fc8240b3430 2024-03-22T04:14:55.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc824074470 con 0x7fc8240b3430 2024-03-22T04:14:55.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.047+0000 7fc81aa00640 1 -- 172.21.15.86:0/1248561947 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc80c01a070 con 0x7fc8240b3430 2024-03-22T04:14:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.051+0000 7fc81aa00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc800067b20 0x7fc800069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.051+0000 7fc81aa00640 1 -- 172.21.15.86:0/1248561947 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fc80c0863a0 con 0x7fc8240b3430 2024-03-22T04:14:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.051+0000 7fc821e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc800067b20 0x7fc800069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.051+0000 7fc821e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc800067b20 0x7fc800069fe0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc8140ad000 tx=0x7fc814008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.055+0000 7fc81aa00640 1 -- 172.21.15.86:0/1248561947 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc80c056490 con 0x7fc8240b3430 2024-03-22T04:14:55.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc82406e9f0 con 0x7fc8240b3430 2024-03-22T04:14:55.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc81aa00640 1 -- 172.21.15.86:0/1248561947 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7fc80c05a530 con 0x7fc8240b3430 2024-03-22T04:14:55.215 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:14:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc800067b20 msgr2=0x7fc800069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc800067b20 0x7fc800069fe0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc8140ad000 tx=0x7fc814008040 comp rx=0 tx=0).stop 2024-03-22T04:14:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b3430 msgr2=0x7fc8241492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b3430 0x7fc8241492d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00e870 tx=0x7fc80c00ec70 comp rx=0 tx=0).stop 2024-03-22T04:14:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 shutdown_connections 2024-03-22T04:14:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc800067b20 0x7fc800069fe0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc82415cee0 0x7fc824071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8240b3430 0x7fc8241492d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 --2- 172.21.15.86:0/1248561947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8240b2360 0x7fc8241f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 >> 172.21.15.86:0/1248561947 conn(0x7fc8240cc940 msgr2=0x7fc8240cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 shutdown_connections 2024-03-22T04:14:55.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.211+0000 7fc828e00640 1 -- 172.21.15.86:0/1248561947 wait complete. 2024-03-22T04:14:55.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:14:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4ac400640 1 Processor -- start 2024-03-22T04:14:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4ac400640 1 -- start start 2024-03-22T04:14:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4ac400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a4162740 0x7fb4a4169380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4ac400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a40cc7c0 con 0x7fb4a40c6f20 2024-03-22T04:14:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4ac400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a40cb720 con 0x7fb4a40cb1d0 2024-03-22T04:14:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4ac400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a40cba00 con 0x7fb4a4162740 2024-03-22T04:14:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4a9a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb4a40c6f20 0x7fb4a4072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52810/0 (socket says 172.21.15.86:52810) 2024-03-22T04:14:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4a9a00640 1 -- 172.21.15.86:0/3673332534 learned_addr learned my addr 172.21.15.86:0/3673332534 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4aa400640 1 --2- 172.21.15.86:0/3673332534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a4162740 0x7fb4a4169380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4a3e00640 1 -- 172.21.15.86:0/3673332534 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1067012025 0 0) 0x7fb4a40cc7c0 con 0x7fb4a40c6f20 2024-03-22T04:14:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4a3e00640 1 -- 172.21.15.86:0/3673332534 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb48c003530 con 0x7fb4a40c6f20 2024-03-22T04:14:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4a3e00640 1 -- 172.21.15.86:0/3673332534 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2354789772 0 0) 0x7fb4a40cb720 con 0x7fb4a40cb1d0 2024-03-22T04:14:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4a3e00640 1 -- 172.21.15.86:0/3673332534 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a40cc7c0 con 0x7fb4a40cb1d0 2024-03-22T04:14:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4aa400640 1 -- 172.21.15.86:0/3673332534 >> v1:172.21.15.196:6789/0 conn(0x7fb4a40cb1d0 legacy=0x7fb4a4162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4aa400640 1 -- 172.21.15.86:0/3673332534 >> v1:172.21.15.86:6789/0 conn(0x7fb4a40c6f20 legacy=0x7fb4a4072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.371+0000 7fb4aa400640 1 -- 172.21.15.86:0/3673332534 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a40cbce0 con 0x7fb4a4162740 2024-03-22T04:14:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4aa400640 1 --2- 172.21.15.86:0/3673332534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a4162740 0x7fb4a4169380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009dc0 tx=0x7fb49c01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5f5a5832cbd84d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4a3e00640 1 -- 172.21.15.86:0/3673332534 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0b2070 con 0x7fb4a4162740 2024-03-22T04:14:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4a3e00640 1 -- 172.21.15.86:0/3673332534 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49c0a5940 con 0x7fb4a4162740 2024-03-22T04:14:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4a3e00640 1 -- 172.21.15.86:0/3673332534 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0a5d10 con 0x7fb4a4162740 2024-03-22T04:14:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4ac400640 1 -- 172.21.15.86:0/3673332534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a4162740 msgr2=0x7fb4a4169380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4ac400640 1 --2- 172.21.15.86:0/3673332534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a4162740 0x7fb4a4169380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009dc0 tx=0x7fb49c01c990 comp rx=0 tx=0).stop 2024-03-22T04:14:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4ac400640 1 -- 172.21.15.86:0/3673332534 shutdown_connections 2024-03-22T04:14:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4ac400640 1 --2- 172.21.15.86:0/3673332534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a4162740 0x7fb4a4169380 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4ac400640 1 -- 172.21.15.86:0/3673332534 >> 172.21.15.86:0/3673332534 conn(0x7fb4a40d0730 msgr2=0x7fb4a40d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4ac400640 1 -- 172.21.15.86:0/3673332534 shutdown_connections 2024-03-22T04:14:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.375+0000 7fb4ac400640 1 -- 172.21.15.86:0/3673332534 wait complete. 2024-03-22T04:14:55.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 Processor -- start 2024-03-22T04:14:55.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 -- start start 2024-03-22T04:14:55.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a40c6f20 0x7fb4a4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a40cb1d0 0x7fb4a40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a4162740 0x7fb4a40aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a416c670 con 0x7fb4a40c6f20 2024-03-22T04:14:55.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a416c4f0 con 0x7fb4a4162740 2024-03-22T04:14:55.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a416c7f0 con 0x7fb4a40cb1d0 2024-03-22T04:14:55.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a40c6f20 0x7fb4a4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a40cb1d0 0x7fb4a40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a40c6f20 0x7fb4a4156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48510/0 (socket says 172.21.15.86:48510) 2024-03-22T04:14:55.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4aa400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a4162740 0x7fb4a40aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 -- 172.21.15.86:0/1799001322 learned_addr learned my addr 172.21.15.86:0/1799001322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:55.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 -- 172.21.15.86:0/1799001322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a40cb1d0 msgr2=0x7fb4a40b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a40cb1d0 0x7fb4a40b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 -- 172.21.15.86:0/1799001322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a4162740 msgr2=0x7fb4a40aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a4162740 0x7fb4a40aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 -- 172.21.15.86:0/1799001322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a40ca3d0 con 0x7fb4a40c6f20 2024-03-22T04:14:55.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4aa400640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a4162740 0x7fb4a40aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:55.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a9a00640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a40c6f20 0x7fb4a4156b40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb49800c420 tx=0x7fb49800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a1600640 1 -- 172.21.15.86:0/1799001322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498017070 con 0x7fb4a40c6f20 2024-03-22T04:14:55.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a1600640 1 -- 172.21.15.86:0/1799001322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb498005cf0 con 0x7fb4a40c6f20 2024-03-22T04:14:55.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4a1600640 1 -- 172.21.15.86:0/1799001322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980072d0 con 0x7fb4a40c6f20 2024-03-22T04:14:55.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a40c89b0 con 0x7fb4a40c6f20 2024-03-22T04:14:55.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a4157490 con 0x7fb4a40c6f20 2024-03-22T04:14:55.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.379+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a4074470 con 0x7fb4a40c6f20 2024-03-22T04:14:55.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.383+0000 7fb4a1600640 1 -- 172.21.15.86:0/1799001322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb49801a070 con 0x7fb4a40c6f20 2024-03-22T04:14:55.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.383+0000 7fb4a1600640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb484067b20 0x7fb484069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.383+0000 7fb4a1600640 1 -- 172.21.15.86:0/1799001322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fb4980854a0 con 0x7fb4a40c6f20 2024-03-22T04:14:55.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.383+0000 7fb4a9000640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb484067b20 0x7fb484069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.383+0000 7fb4a9000640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb484067b20 0x7fb484069fe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40c88c0 tx=0x7fb490009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.387+0000 7fb4a1600640 1 -- 172.21.15.86:0/1799001322 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb498055590 con 0x7fb4a40c6f20 2024-03-22T04:14:55.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4a4068af0 con 0x7fb4a40c6f20 2024-03-22T04:14:55.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4a1600640 1 -- 172.21.15.86:0/1799001322 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7fb498059630 con 0x7fb4a40c6f20 2024-03-22T04:14:55.547 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb484067b20 msgr2=0x7fb484069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb484067b20 0x7fb484069fe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40c88c0 tx=0x7fb490009040 comp rx=0 tx=0).stop 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a40c6f20 msgr2=0x7fb4a4156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a40c6f20 0x7fb4a4156b40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb49800c420 tx=0x7fb49800c8e0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 shutdown_connections 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb484067b20 0x7fb484069fe0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4a4162740 0x7fb4a40aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4a40cb1d0 0x7fb4a40b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 --2- 172.21.15.86:0/1799001322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a40c6f20 0x7fb4a4156b40 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 >> 172.21.15.86:0/1799001322 conn(0x7fb4a40d0730 msgr2=0x7fb4a40b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:55.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 shutdown_connections 2024-03-22T04:14:55.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.543+0000 7fb4ac400640 1 -- 172.21.15.86:0/1799001322 wait complete. 2024-03-22T04:14:55.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:14:55.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd099e00640 1 Processor -- start 2024-03-22T04:14:55.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd099e00640 1 -- start start 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd099e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 0x7fd09415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd099e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd09415cee0 0x7fd0940b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd099e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd09415d710 con 0x7fd0940b2360 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd099e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd0940b2740 con 0x7fd09415cee0 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd099e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd09415fe40 con 0x7fd0940b3600 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd092a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd0940b2360 0x7fd0940b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52816/0 (socket says 172.21.15.86:52816) 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd092a00640 1 -- 172.21.15.86:0/1271146280 learned_addr learned my addr 172.21.15.86:0/1271146280 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd092000640 1 --2- 172.21.15.86:0/1271146280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 0x7fd09415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd093400640 1 --2- 172.21.15.86:0/1271146280 >> v2:172.21.15.86:3300/0 conn(0x7fd09415cee0 0x7fd0940b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd091600640 1 -- 172.21.15.86:0/1271146280 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1163741204 0 0) 0x7fd09415d710 con 0x7fd0940b2360 2024-03-22T04:14:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd091600640 1 -- 172.21.15.86:0/1271146280 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd078003530 con 0x7fd0940b2360 2024-03-22T04:14:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd093400640 1 -- 172.21.15.86:0/1271146280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 msgr2=0x7fd09415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd093400640 1 --2- 172.21.15.86:0/1271146280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 0x7fd09415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd093400640 1 -- 172.21.15.86:0/1271146280 >> v1:172.21.15.86:6789/0 conn(0x7fd0940b2360 legacy=0x7fd0940b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.703+0000 7fd093400640 1 -- 172.21.15.86:0/1271146280 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd094160120 con 0x7fd09415cee0 2024-03-22T04:14:55.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd093400640 1 --2- 172.21.15.86:0/1271146280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09415cee0 0x7fd0940b4d80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009d90 tx=0x7fd08c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c523f16460f1eb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd091600640 1 -- 172.21.15.86:0/1271146280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0b2070 con 0x7fd09415cee0 2024-03-22T04:14:55.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd091600640 1 -- 172.21.15.86:0/1271146280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08c0a5a00 con 0x7fd09415cee0 2024-03-22T04:14:55.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd091600640 1 -- 172.21.15.86:0/1271146280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0a5db0 con 0x7fd09415cee0 2024-03-22T04:14:55.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd099e00640 1 -- 172.21.15.86:0/1271146280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09415cee0 msgr2=0x7fd0940b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd099e00640 1 --2- 172.21.15.86:0/1271146280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09415cee0 0x7fd0940b4d80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009d90 tx=0x7fd08c01c940 comp rx=0 tx=0).stop 2024-03-22T04:14:55.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd099e00640 1 -- 172.21.15.86:0/1271146280 shutdown_connections 2024-03-22T04:14:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd099e00640 1 --2- 172.21.15.86:0/1271146280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09415cee0 0x7fd0940b4d80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd099e00640 1 --2- 172.21.15.86:0/1271146280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 0x7fd09415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd099e00640 1 -- 172.21.15.86:0/1271146280 >> 172.21.15.86:0/1271146280 conn(0x7fd0940cc940 msgr2=0x7fd0940ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd099e00640 1 -- 172.21.15.86:0/1271146280 shutdown_connections 2024-03-22T04:14:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.707+0000 7fd099e00640 1 -- 172.21.15.86:0/1271146280 wait complete. 2024-03-22T04:14:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 Processor -- start 2024-03-22T04:14:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 -- start start 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0940b2360 0x7fd09414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 0x7fd09414fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd09415cee0 0x7fd0940cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd09415d490 con 0x7fd0940b2360 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd09415d310 con 0x7fd09415cee0 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd0940b5ad0 con 0x7fd0940b3600 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0940b2360 0x7fd09414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0940b2360 0x7fd09414bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48530/0 (socket says 172.21.15.86:48530) 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 -- 172.21.15.86:0/2410202694 learned_addr learned my addr 172.21.15.86:0/2410202694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092000640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 0x7fd09414fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd093400640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd09415cee0 0x7fd0940cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 -- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 msgr2=0x7fd09414fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 0x7fd09414fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 -- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd09415cee0 msgr2=0x7fd0940cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd09415cee0 0x7fd0940cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 -- 172.21.15.86:0/2410202694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09414c1d0 con 0x7fd0940b2360 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd093400640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd09415cee0 0x7fd0940cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd092a00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0940b2360 0x7fd09414bc90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c450 tx=0x7fd08400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd08b400640 1 -- 172.21.15.86:0/2410202694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084017070 con 0x7fd0940b2360 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd08b400640 1 -- 172.21.15.86:0/2410202694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd084005cf0 con 0x7fd0940b2360 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd094150480 con 0x7fd0940b2360 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd08b400640 1 -- 172.21.15.86:0/2410202694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084007290 con 0x7fd0940b2360 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd094150a40 con 0x7fd0940b2360 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.711+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd094074470 con 0x7fd0940b2360 2024-03-22T04:14:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.715+0000 7fd08b400640 1 -- 172.21.15.86:0/2410202694 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd08401a070 con 0x7fd0940b2360 2024-03-22T04:14:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.715+0000 7fd08b400640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd074067ba0 0x7fd07406a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.715+0000 7fd08b400640 1 -- 172.21.15.86:0/2410202694 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fd084085480 con 0x7fd0940b2360 2024-03-22T04:14:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.715+0000 7fd092000640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd074067ba0 0x7fd07406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.715+0000 7fd092000640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd074067ba0 0x7fd07406a060 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd0940ced80 tx=0x7fd080009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.719+0000 7fd08b400640 1 -- 172.21.15.86:0/2410202694 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd0840555f0 con 0x7fd0940b2360 2024-03-22T04:14:55.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.875+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd09406e9f0 con 0x7fd0940b2360 2024-03-22T04:14:55.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.875+0000 7fd08b400640 1 -- 172.21.15.86:0/2410202694 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7fd084059690 con 0x7fd0940b2360 2024-03-22T04:14:55.882 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:14:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd074067ba0 msgr2=0x7fd07406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd074067ba0 0x7fd07406a060 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd0940ced80 tx=0x7fd080009190 comp rx=0 tx=0).stop 2024-03-22T04:14:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0940b2360 msgr2=0x7fd09414bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0940b2360 0x7fd09414bc90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c450 tx=0x7fd08400c910 comp rx=0 tx=0).stop 2024-03-22T04:14:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 shutdown_connections 2024-03-22T04:14:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd074067ba0 0x7fd07406a060 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd09415cee0 0x7fd0940cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0940b3600 0x7fd09414fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 --2- 172.21.15.86:0/2410202694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0940b2360 0x7fd09414bc90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:55.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 >> 172.21.15.86:0/2410202694 conn(0x7fd0940cc940 msgr2=0x7fd0940cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:55.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 shutdown_connections 2024-03-22T04:14:55.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:55.879+0000 7fd099e00640 1 -- 172.21.15.86:0/2410202694 wait complete. 2024-03-22T04:14:55.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:14:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ec000640 1 Processor -- start 2024-03-22T04:14:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ec000640 1 -- start start 2024-03-22T04:14:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ec000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f57e415fed0 0x7f57e41602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ec000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e40d11c0 con 0x7f57e40d0de0 2024-03-22T04:14:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ec000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e4160820 con 0x7f57e40ce6b0 2024-03-22T04:14:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ec000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f57e40d0580 con 0x7f57e415fed0 2024-03-22T04:14:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57e9600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f57e40d0de0 0x7f57e40d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37850/0 (socket says 172.21.15.86:37850) 2024-03-22T04:14:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57e9600640 1 -- 172.21.15.86:0/1688862746 learned_addr learned my addr 172.21.15.86:0/1688862746 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ea000640 1 --2- 172.21.15.86:0/1688862746 >> v2:172.21.15.86:3300/0 conn(0x7f57e415fed0 0x7f57e41602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57e3e00640 1 -- 172.21.15.86:0/1688862746 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1077466133 0 0) 0x7f57e40d11c0 con 0x7f57e40d0de0 2024-03-22T04:14:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57e3e00640 1 -- 172.21.15.86:0/1688862746 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57cc003530 con 0x7f57e40d0de0 2024-03-22T04:14:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ea000640 1 -- 172.21.15.86:0/1688862746 >> v1:172.21.15.196:6789/0 conn(0x7f57e40ce6b0 legacy=0x7f57e40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57e3e00640 1 -- 172.21.15.86:0/1688862746 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201186870 0 0) 0x7f57d0001120 con 0x7f57e40ce6b0 2024-03-22T04:14:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ea000640 1 -- 172.21.15.86:0/1688862746 >> v1:172.21.15.86:6789/0 conn(0x7f57e40d0de0 legacy=0x7f57e40d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.035+0000 7f57ea000640 1 -- 172.21.15.86:0/1688862746 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e40cece0 con 0x7f57e415fed0 2024-03-22T04:14:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57ea000640 1 --2- 172.21.15.86:0/1688862746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e415fed0 0x7f57e41602e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009d90 tx=0x7f57dc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d902df8f3356d9b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57e3e00640 1 -- 172.21.15.86:0/1688862746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0b2070 con 0x7f57e415fed0 2024-03-22T04:14:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57e3e00640 1 -- 172.21.15.86:0/1688862746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc0a5830 con 0x7f57e415fed0 2024-03-22T04:14:56.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57e3e00640 1 -- 172.21.15.86:0/1688862746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0a5c00 con 0x7f57e415fed0 2024-03-22T04:14:56.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57ec000640 1 -- 172.21.15.86:0/1688862746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e415fed0 msgr2=0x7f57e41602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57ec000640 1 --2- 172.21.15.86:0/1688862746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e415fed0 0x7f57e41602e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009d90 tx=0x7f57dc01c970 comp rx=0 tx=0).stop 2024-03-22T04:14:56.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57ec000640 1 -- 172.21.15.86:0/1688862746 shutdown_connections 2024-03-22T04:14:56.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57ec000640 1 --2- 172.21.15.86:0/1688862746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e415fed0 0x7f57e41602e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57ec000640 1 -- 172.21.15.86:0/1688862746 >> 172.21.15.86:0/1688862746 conn(0x7f57e406f910 msgr2=0x7f57e406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:56.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57ec000640 1 -- 172.21.15.86:0/1688862746 shutdown_connections 2024-03-22T04:14:56.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.039+0000 7f57ec000640 1 -- 172.21.15.86:0/1688862746 wait complete. 2024-03-22T04:14:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 Processor -- start 2024-03-22T04:14:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 -- start start 2024-03-22T04:14:56.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e40ce6b0 0x7f57e4153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57e40d0de0 0x7f57e4152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57e415fed0 0x7f57e41586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4071e30 con 0x7f57e40ce6b0 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4071100 con 0x7f57e415fed0 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f57e4071fb0 con 0x7f57e40d0de0 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e40ce6b0 0x7f57e4153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e40ce6b0 0x7f57e4153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52278/0 (socket says 172.21.15.86:52278) 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 -- 172.21.15.86:0/145390225 learned_addr learned my addr 172.21.15.86:0/145390225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ea000640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57e415fed0 0x7f57e41586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e8c00640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57e40d0de0 0x7f57e4152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 -- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57e40d0de0 msgr2=0x7f57e4152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57e40d0de0 0x7f57e4152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 -- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57e415fed0 msgr2=0x7f57e41586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57e415fed0 0x7f57e41586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 -- 172.21.15.86:0/145390225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e40b6c30 con 0x7f57e40ce6b0 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e8c00640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57e40d0de0 0x7f57e4152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e9600640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e40ce6b0 0x7f57e4153300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f57d800c3f0 tx=0x7f57d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e1600640 1 -- 172.21.15.86:0/145390225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8017070 con 0x7f57e40ce6b0 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e1600640 1 -- 172.21.15.86:0/145390225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d8005cf0 con 0x7f57e40ce6b0 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e40b6f10 con 0x7f57e40ce6b0 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57e1600640 1 -- 172.21.15.86:0/145390225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d80072f0 con 0x7f57e40ce6b0 2024-03-22T04:14:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e40b74d0 con 0x7f57e40ce6b0 2024-03-22T04:14:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.043+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e4068650 con 0x7f57e40ce6b0 2024-03-22T04:14:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.047+0000 7f57e1600640 1 -- 172.21.15.86:0/145390225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f57d801a070 con 0x7f57e40ce6b0 2024-03-22T04:14:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.047+0000 7f57e1600640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57c4067b20 0x7f57c4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.047+0000 7f57e1600640 1 -- 172.21.15.86:0/145390225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f57d80854f0 con 0x7f57e40ce6b0 2024-03-22T04:14:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.047+0000 7f57e8c00640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57c4067b20 0x7f57c4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.047+0000 7f57e8c00640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57c4067b20 0x7f57c4069fe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57e4167880 tx=0x7f57d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.051+0000 7f57e1600640 1 -- 172.21.15.86:0/145390225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f57d8055670 con 0x7f57e40ce6b0 2024-03-22T04:14:56.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.235+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57e4160a00 con 0x7f57e40ce6b0 2024-03-22T04:14:56.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57e1600640 1 -- 172.21.15.86:0/145390225 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5186 (secure 0 0 0) 0x7f57d8059710 con 0x7f57e40ce6b0 2024-03-22T04:14:56.243 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:56.243 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:14:24.562106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:14:56.243 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 37 2024-03-22T04:14:56.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57c4067b20 msgr2=0x7f57c4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57c4067b20 0x7f57c4069fe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57e4167880 tx=0x7f57d0009040 comp rx=0 tx=0).stop 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e40ce6b0 msgr2=0x7f57e4153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e40ce6b0 0x7f57e4153300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f57d800c3f0 tx=0x7f57d800c8b0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 shutdown_connections 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57c4067b20 0x7f57c4069fe0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57e415fed0 0x7f57e41586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57e40d0de0 0x7f57e4152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 --2- 172.21.15.86:0/145390225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e40ce6b0 0x7f57e4153300 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 >> 172.21.15.86:0/145390225 conn(0x7f57e406f910 msgr2=0x7f57e40cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 shutdown_connections 2024-03-22T04:14:56.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.239+0000 7f57ec000640 1 -- 172.21.15.86:0/145390225 wait complete. 2024-03-22T04:14:56.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:14:56.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a6b400640 1 Processor -- start 2024-03-22T04:14:56.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a6b400640 1 -- start start 2024-03-22T04:14:56.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a6b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4a640ce6b0 0x7f4a640cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a6b400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a640d11c0 con 0x7f4a640d0de0 2024-03-22T04:14:56.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a6b400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a640cece0 con 0x7f4a6415fed0 2024-03-22T04:14:56.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a6b400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4a640cefc0 con 0x7f4a640ce6b0 2024-03-22T04:14:56.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a68a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4a640d0de0 0x7f4a640d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37864/0 (socket says 172.21.15.86:37864) 2024-03-22T04:14:56.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a68a00640 1 -- 172.21.15.86:0/3184291247 learned_addr learned my addr 172.21.15.86:0/3184291247 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:56.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a63e00640 1 --2- 172.21.15.86:0/3184291247 >> v2:172.21.15.196:3300/0 conn(0x7f4a640ce6b0 0x7f4a640cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a63400640 1 -- 172.21.15.86:0/3184291247 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 219387604 0 0) 0x7f4a640d11c0 con 0x7f4a640d0de0 2024-03-22T04:14:56.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a63400640 1 -- 172.21.15.86:0/3184291247 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a48003530 con 0x7f4a640d0de0 2024-03-22T04:14:56.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a63400640 1 -- 172.21.15.86:0/3184291247 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930428943 0 0) 0x7f4a640cece0 con 0x7f4a6415fed0 2024-03-22T04:14:56.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a63400640 1 -- 172.21.15.86:0/3184291247 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a640d11c0 con 0x7f4a6415fed0 2024-03-22T04:14:56.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a63e00640 1 -- 172.21.15.86:0/3184291247 >> v1:172.21.15.196:6789/0 conn(0x7f4a6415fed0 legacy=0x7f4a641602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a63e00640 1 -- 172.21.15.86:0/3184291247 >> v1:172.21.15.86:6789/0 conn(0x7f4a640d0de0 legacy=0x7f4a640d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.395+0000 7f4a63e00640 1 -- 172.21.15.86:0/3184291247 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640cf2a0 con 0x7f4a640ce6b0 2024-03-22T04:14:56.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a63e00640 1 --2- 172.21.15.86:0/3184291247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 0x7f4a640cfe70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a500097c0 tx=0x7f4a5001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ef77ef359c0a30 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a63400640 1 -- 172.21.15.86:0/3184291247 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500b0070 con 0x7f4a640ce6b0 2024-03-22T04:14:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a63400640 1 -- 172.21.15.86:0/3184291247 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a500ab600 con 0x7f4a640ce6b0 2024-03-22T04:14:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a63400640 1 -- 172.21.15.86:0/3184291247 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500ab9d0 con 0x7f4a640ce6b0 2024-03-22T04:14:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a6b400640 1 -- 172.21.15.86:0/3184291247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 msgr2=0x7f4a640cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a6b400640 1 --2- 172.21.15.86:0/3184291247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 0x7f4a640cfe70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a500097c0 tx=0x7f4a5001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a6b400640 1 -- 172.21.15.86:0/3184291247 shutdown_connections 2024-03-22T04:14:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a6b400640 1 --2- 172.21.15.86:0/3184291247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 0x7f4a640cfe70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a6b400640 1 -- 172.21.15.86:0/3184291247 >> 172.21.15.86:0/3184291247 conn(0x7f4a6406f910 msgr2=0x7f4a6406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a6b400640 1 -- 172.21.15.86:0/3184291247 shutdown_connections 2024-03-22T04:14:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.399+0000 7f4a6b400640 1 -- 172.21.15.86:0/3184291247 wait complete. 2024-03-22T04:14:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 Processor -- start 2024-03-22T04:14:56.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 -- start start 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 0x7f4a64155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a640d0de0 0x7f4a640ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a6415fed0 0x7f4a640ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64071fb0 con 0x7f4a6415fed0 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64071e30 con 0x7f4a640ce6b0 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6416a050 con 0x7f4a640d0de0 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a6415fed0 0x7f4a640ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a68a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 0x7f4a64155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a63e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a640d0de0 0x7f4a640ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a6415fed0 0x7f4a640ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52290/0 (socket says 172.21.15.86:52290) 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 -- 172.21.15.86:0/399849959 learned_addr learned my addr 172.21.15.86:0/399849959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 -- 172.21.15.86:0/399849959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a640d0de0 msgr2=0x7f4a640ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a640d0de0 0x7f4a640ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 -- 172.21.15.86:0/399849959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 msgr2=0x7f4a64155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 0x7f4a64155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 -- 172.21.15.86:0/399849959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a641564a0 con 0x7f4a6415fed0 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a69400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a6415fed0 0x7f4a640ca930 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c410 tx=0x7f4a5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a60c00640 1 -- 172.21.15.86:0/399849959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c017070 con 0x7f4a6415fed0 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a60c00640 1 -- 172.21.15.86:0/399849959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5c0052d0 con 0x7f4a6415fed0 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a60c00640 1 -- 172.21.15.86:0/399849959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c003800 con 0x7f4a6415fed0 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a640b33e0 con 0x7f4a6415fed0 2024-03-22T04:14:56.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.403+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a640b2ab0 con 0x7f4a6415fed0 2024-03-22T04:14:56.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.407+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a64068650 con 0x7f4a6415fed0 2024-03-22T04:14:56.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.407+0000 7f4a60c00640 1 -- 172.21.15.86:0/399849959 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4a5c01a070 con 0x7f4a6415fed0 2024-03-22T04:14:56.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.407+0000 7f4a60c00640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.407+0000 7f4a60c00640 1 -- 172.21.15.86:0/399849959 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f4a5c085480 con 0x7f4a6415fed0 2024-03-22T04:14:56.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.407+0000 7f4a68a00640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.407+0000 7f4a68a00640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a540012f0 tx=0x7f4a54001ae0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.411+0000 7f4a60c00640 1 -- 172.21.15.86:0/399849959 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4a641564a0 con 0x7f4a6415fed0 2024-03-22T04:14:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.571+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a6415c760 con 0x7f4a6415fed0 2024-03-22T04:14:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a60c00640 1 -- 172.21.15.86:0/399849959 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17268 (secure 0 0 0) 0x7f4a5c005480 con 0x7f4a6415fed0 2024-03-22T04:14:56.579 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:56.579 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":69,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:14:16.029299+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:13:34.224743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-22T04:13:34.852940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.2","mappings":[{"from":2,"to":7}]},{"pgid":"10.a","mappings":[{"from":2,"to":1}]},{"pgid":"10.10","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":5}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.26","mappings":[{"from":4,"to":1}]},{"pgid":"10.2d","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:14:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 msgr2=0x7f4a44069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a540012f0 tx=0x7f4a54001ae0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a6415fed0 msgr2=0x7f4a640ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a6415fed0 0x7f4a640ca930 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c410 tx=0x7f4a5c00c8d0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 shutdown_connections 2024-03-22T04:14:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4a44067b20 0x7f4a44069fe0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a6415fed0 0x7f4a640ca930 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a640d0de0 0x7f4a640ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 --2- 172.21.15.86:0/399849959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a640ce6b0 0x7f4a64155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 >> 172.21.15.86:0/399849959 conn(0x7f4a6406f910 msgr2=0x7f4a641597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 shutdown_connections 2024-03-22T04:14:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.575+0000 7f4a6b400640 1 -- 172.21.15.86:0/399849959 wait complete. 2024-03-22T04:14:56.600 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:14:56.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:14:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.731+0000 7f4788c00640 1 Processor -- start 2024-03-22T04:14:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.731+0000 7f4788c00640 1 -- start start 2024-03-22T04:14:56.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.731+0000 7f4788c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47840b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.731+0000 7f4788c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4784070110 0x7f478406cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.731+0000 7f4788c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47841621c0 con 0x7f47840b3e00 2024-03-22T04:14:56.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.731+0000 7f4788c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f47841624a0 con 0x7f4784070110 2024-03-22T04:14:56.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.731+0000 7f4788c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4784162780 con 0x7f4784147d10 2024-03-22T04:14:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4779400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f47840b3e00 0x7f478406fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37868/0 (socket says 172.21.15.86:37868) 2024-03-22T04:14:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4779400640 1 -- 172.21.15.86:0/464661520 learned_addr learned my addr 172.21.15.86:0/464661520 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781e00640 1 --2- 172.21.15.86:0/464661520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47840b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4782800640 1 --2- 172.21.15.86:0/464661520 >> v2:172.21.15.196:3300/0 conn(0x7f4784070110 0x7f478406cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781400640 1 -- 172.21.15.86:0/464661520 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 268700124 0 0) 0x7f47841621c0 con 0x7f47840b3e00 2024-03-22T04:14:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781400640 1 -- 172.21.15.86:0/464661520 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4768003530 con 0x7f47840b3e00 2024-03-22T04:14:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781e00640 1 -- 172.21.15.86:0/464661520 >> v2:172.21.15.196:3300/0 conn(0x7f4784070110 msgr2=0x7f478406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781e00640 1 --2- 172.21.15.86:0/464661520 >> v2:172.21.15.196:3300/0 conn(0x7f4784070110 0x7f478406cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781e00640 1 -- 172.21.15.86:0/464661520 >> v1:172.21.15.86:6789/0 conn(0x7f47840b3e00 legacy=0x7f478406fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781400640 1 -- 172.21.15.86:0/464661520 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 158783589 0 0) 0x7f4768003530 con 0x7f47840b3e00 2024-03-22T04:14:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781e00640 1 -- 172.21.15.86:0/464661520 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4784162a60 con 0x7f4784147d10 2024-03-22T04:14:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4782800640 1 --2- 172.21.15.86:0/464661520 >> v2:172.21.15.196:3300/0 conn(0x7f4784070110 0x7f478406cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.735+0000 7f4781e00640 1 --2- 172.21.15.86:0/464661520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47840b4400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4774009d90 tx=0x7f477401c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc0787f6d4ed1bf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4781400640 1 -- 172.21.15.86:0/464661520 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47740b2070 con 0x7f4784147d10 2024-03-22T04:14:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4781400640 1 -- 172.21.15.86:0/464661520 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47740a5680 con 0x7f4784147d10 2024-03-22T04:14:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4781400640 1 -- 172.21.15.86:0/464661520 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c001680 con 0x7f4784147d10 2024-03-22T04:14:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- 172.21.15.86:0/464661520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 msgr2=0x7f47840b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 --2- 172.21.15.86:0/464661520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47840b4400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4774009d90 tx=0x7f477401c860 comp rx=0 tx=0).stop 2024-03-22T04:14:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- 172.21.15.86:0/464661520 shutdown_connections 2024-03-22T04:14:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 --2- 172.21.15.86:0/464661520 >> v2:172.21.15.196:3300/0 conn(0x7f4784070110 0x7f478406cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 --2- 172.21.15.86:0/464661520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47840b4400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- 172.21.15.86:0/464661520 >> 172.21.15.86:0/464661520 conn(0x7f47840b75b0 msgr2=0x7f47840b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:56.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- 172.21.15.86:0/464661520 shutdown_connections 2024-03-22T04:14:56.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- 172.21.15.86:0/464661520 wait complete. 2024-03-22T04:14:56.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 Processor -- start 2024-03-22T04:14:56.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- start start 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784070110 0x7f47840b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840b3e00 0x7f47841633a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47841638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f478406d9d0 con 0x7f47840b3e00 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47840b4b50 con 0x7f4784070110 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4788c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f478406db50 con 0x7f4784147d10 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4779400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840b3e00 0x7f47841633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4779400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840b3e00 0x7f47841633a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52298/0 (socket says 172.21.15.86:52298) 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4779400640 1 -- 172.21.15.86:0/4193518807 learned_addr learned my addr 172.21.15.86:0/4193518807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4782800640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47841638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4781e00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784070110 0x7f47840b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4779400640 1 -- 172.21.15.86:0/4193518807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 msgr2=0x7f47841638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4779400640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47841638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4779400640 1 -- 172.21.15.86:0/4193518807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784070110 msgr2=0x7f47840b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.739+0000 7f4779400640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784070110 0x7f47840b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f4779400640 1 -- 172.21.15.86:0/4193518807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478415e8a0 con 0x7f47840b3e00 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f4779400640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840b3e00 0x7f47841633a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c780 tx=0x7f476c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f477aa00640 1 -- 172.21.15.86:0/4193518807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c005ae0 con 0x7f47840b3e00 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f477aa00640 1 -- 172.21.15.86:0/4193518807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476c005ca0 con 0x7f47840b3e00 2024-03-22T04:14:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f477aa00640 1 -- 172.21.15.86:0/4193518807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c0072b0 con 0x7f47840b3e00 2024-03-22T04:14:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4784163ef0 con 0x7f47840b3e00 2024-03-22T04:14:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47841644b0 con 0x7f47840b3e00 2024-03-22T04:14:56.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4784068650 con 0x7f47840b3e00 2024-03-22T04:14:56.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f477aa00640 1 -- 172.21.15.86:0/4193518807 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f476c007450 con 0x7f47840b3e00 2024-03-22T04:14:56.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f477aa00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4760067b20 0x7f4760069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:56.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f477aa00640 1 -- 172.21.15.86:0/4193518807 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f476c085e20 con 0x7f47840b3e00 2024-03-22T04:14:56.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.743+0000 7f4781e00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4760067b20 0x7f4760069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:56.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.747+0000 7f4781e00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4760067b20 0x7f4760069fe0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f47740af000 tx=0x7f47740a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.747+0000 7f477aa00640 1 -- 172.21.15.86:0/4193518807 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f476c056670 con 0x7f47840b3e00 2024-03-22T04:14:56.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f478415bbb0 con 0x7f47840b3e00 2024-03-22T04:14:56.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f477aa00640 1 -- 172.21.15.86:0/4193518807 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5186 (secure 0 0 0) 0x7f476c055f70 con 0x7f47840b3e00 2024-03-22T04:14:56.943 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:56.943 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:14:24.562106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:14:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 37 2024-03-22T04:14:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4760067b20 msgr2=0x7f4760069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4760067b20 0x7f4760069fe0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f47740af000 tx=0x7f47740a7070 comp rx=0 tx=0).stop 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840b3e00 msgr2=0x7f47841633a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840b3e00 0x7f47841633a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c780 tx=0x7f476c00cb80 comp rx=0 tx=0).stop 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 shutdown_connections 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4760067b20 0x7f4760069fe0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4784147d10 0x7f47841638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47840b3e00 0x7f47841633a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 --2- 172.21.15.86:0/4193518807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784070110 0x7f47840b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 >> 172.21.15.86:0/4193518807 conn(0x7f47840b75b0 msgr2=0x7f478415f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 shutdown_connections 2024-03-22T04:14:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:56.939+0000 7f4788c00640 1 -- 172.21.15.86:0/4193518807 wait complete. 2024-03-22T04:14:56.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:14:57.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155f200640 1 Processor -- start 2024-03-22T04:14:57.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155f200640 1 -- start start 2024-03-22T04:14:57.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f155806e2f0 0x7f1558071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15581623d0 con 0x7f155815e630 2024-03-22T04:14:57.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15581620f0 con 0x7f15580d0580 2024-03-22T04:14:57.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f155815ec10 con 0x7f155806e2f0 2024-03-22T04:14:57.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f1557e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f155815e630 0x7f15580cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37882/0 (socket says 172.21.15.86:37882) 2024-03-22T04:14:57.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f1557e00640 1 -- 172.21.15.86:0/4124774061 learned_addr learned my addr 172.21.15.86:0/4124774061 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:57.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155d200640 1 --2- 172.21.15.86:0/4124774061 >> v2:172.21.15.86:3300/0 conn(0x7f155806e2f0 0x7f1558071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f1556a00640 1 -- 172.21.15.86:0/4124774061 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3811942531 0 0) 0x7f15581623d0 con 0x7f155815e630 2024-03-22T04:14:57.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f1556a00640 1 -- 172.21.15.86:0/4124774061 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f153c003530 con 0x7f155815e630 2024-03-22T04:14:57.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f1556a00640 1 -- 172.21.15.86:0/4124774061 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512468867 0 0) 0x7f15581620f0 con 0x7f15580d0580 2024-03-22T04:14:57.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f1556a00640 1 -- 172.21.15.86:0/4124774061 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15581623d0 con 0x7f15580d0580 2024-03-22T04:14:57.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155d200640 1 -- 172.21.15.86:0/4124774061 >> v1:172.21.15.196:6789/0 conn(0x7f15580d0580 legacy=0x7f1558071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155d200640 1 -- 172.21.15.86:0/4124774061 >> v1:172.21.15.86:6789/0 conn(0x7f155815e630 legacy=0x7f15580cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.095+0000 7f155d200640 1 -- 172.21.15.86:0/4124774061 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155815eef0 con 0x7f155806e2f0 2024-03-22T04:14:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155d200640 1 --2- 172.21.15.86:0/4124774061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155806e2f0 0x7f1558071810 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1550009d90 tx=0x7f155001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a81ffdbaf81e670 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f1556a00640 1 -- 172.21.15.86:0/4124774061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15500b2070 con 0x7f155806e2f0 2024-03-22T04:14:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f1556a00640 1 -- 172.21.15.86:0/4124774061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15500a5830 con 0x7f155806e2f0 2024-03-22T04:14:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f1556a00640 1 -- 172.21.15.86:0/4124774061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15500a5c00 con 0x7f155806e2f0 2024-03-22T04:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155f200640 1 -- 172.21.15.86:0/4124774061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155806e2f0 msgr2=0x7f1558071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155f200640 1 --2- 172.21.15.86:0/4124774061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155806e2f0 0x7f1558071810 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1550009d90 tx=0x7f155001c970 comp rx=0 tx=0).stop 2024-03-22T04:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155f200640 1 -- 172.21.15.86:0/4124774061 shutdown_connections 2024-03-22T04:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155f200640 1 --2- 172.21.15.86:0/4124774061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155806e2f0 0x7f1558071810 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155f200640 1 -- 172.21.15.86:0/4124774061 >> 172.21.15.86:0/4124774061 conn(0x7f155806f910 msgr2=0x7f155806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155f200640 1 -- 172.21.15.86:0/4124774061 shutdown_connections 2024-03-22T04:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155f200640 1 -- 172.21.15.86:0/4124774061 wait complete. 2024-03-22T04:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.099+0000 7f155f200640 1 Processor -- start 2024-03-22T04:14:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 -- start start 2024-03-22T04:14:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f155806e2f0 0x7f15580c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15580d0580 0x7f15580c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155815e630 0x7f15580b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15580cfaf0 con 0x7f155815e630 2024-03-22T04:14:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f155815f9d0 con 0x7f155806e2f0 2024-03-22T04:14:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15580cfc70 con 0x7f15580d0580 2024-03-22T04:14:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155815e630 0x7f15580b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155815e630 0x7f15580b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52322/0 (socket says 172.21.15.86:52322) 2024-03-22T04:14:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 -- 172.21.15.86:0/1219566177 learned_addr learned my addr 172.21.15.86:0/1219566177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f1557e00640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f155806e2f0 0x7f15580c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f1557400640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15580d0580 0x7f15580c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 -- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15580d0580 msgr2=0x7f15580c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15580d0580 0x7f15580c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 -- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f155806e2f0 msgr2=0x7f15580c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f155806e2f0 0x7f15580c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 -- 172.21.15.86:0/1219566177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15580ca080 con 0x7f155815e630 2024-03-22T04:14:57.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f1557e00640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f155806e2f0 0x7f15580c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155d200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155815e630 0x7f15580b4f60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f155001a5a0 tx=0x7f1550019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f154fe00640 1 -- 172.21.15.86:0/1219566177 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15500b2040 con 0x7f155815e630 2024-03-22T04:14:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f154fe00640 1 -- 172.21.15.86:0/1219566177 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15500bf070 con 0x7f155815e630 2024-03-22T04:14:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f154fe00640 1 -- 172.21.15.86:0/1219566177 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550009450 con 0x7f155815e630 2024-03-22T04:14:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15580ca360 con 0x7f155815e630 2024-03-22T04:14:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155815c500 con 0x7f155815e630 2024-03-22T04:14:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.103+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1558068650 con 0x7f155815e630 2024-03-22T04:14:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.107+0000 7f154fe00640 1 -- 172.21.15.86:0/1219566177 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f15500b8670 con 0x7f155815e630 2024-03-22T04:14:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.107+0000 7f154fe00640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1534067b20 0x7f1534069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.107+0000 7f154fe00640 1 -- 172.21.15.86:0/1219566177 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7f1550120ed0 con 0x7f155815e630 2024-03-22T04:14:57.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.107+0000 7f1557e00640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1534067b20 0x7f1534069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.107+0000 7f1557e00640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1534067b20 0x7f1534069fe0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1540007fd0 tx=0x7f1540009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.111+0000 7f154fe00640 1 -- 172.21.15.86:0/1219566177 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1550016560 con 0x7f155815e630 2024-03-22T04:14:57.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.295+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f155815c740 con 0x7f155815e630 2024-03-22T04:14:57.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.295+0000 7f154fe00640 1 -- 172.21.15.86:0/1219566177 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5186 (secure 0 0 0) 0x7f15500f10e0 con 0x7f155815e630 2024-03-22T04:14:57.301 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:57.302 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:13:35.899981+0000","modified":"2024-03-22T04:14:24.562106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":75617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_75617":{"gid":75617,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.196:6835/3774275697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3774275697},{"type":"v1","addr":"172.21.15.196:6835","nonce":3774275697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":75617,"qdb_cluster":[75617]},"id":4}]} 2024-03-22T04:14:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 37 2024-03-22T04:14:57.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1534067b20 msgr2=0x7f1534069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1534067b20 0x7f1534069fe0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1540007fd0 tx=0x7f1540009040 comp rx=0 tx=0).stop 2024-03-22T04:14:57.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155815e630 msgr2=0x7f15580b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155815e630 0x7f15580b4f60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f155001a5a0 tx=0x7f1550019710 comp rx=0 tx=0).stop 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 shutdown_connections 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1534067b20 0x7f1534069fe0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f155815e630 0x7f15580b4f60 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15580d0580 0x7f15580c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 --2- 172.21.15.86:0/1219566177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f155806e2f0 0x7f15580c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 >> 172.21.15.86:0/1219566177 conn(0x7f155806f910 msgr2=0x7f15580b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 shutdown_connections 2024-03-22T04:14:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.299+0000 7f155f200640 1 -- 172.21.15.86:0/1219566177 wait complete. 2024-03-22T04:14:57.322 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:14:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.451+0000 7fccf6e00640 1 Processor -- start 2024-03-22T04:14:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.451+0000 7fccf6e00640 1 -- start start 2024-03-22T04:14:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf6e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fccf015be40 0x7fccf015f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf6e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf006be00 con 0x7fccf00b0750 2024-03-22T04:14:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf6e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf006c0e0 con 0x7fccf006f060 2024-03-22T04:14:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf6e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fccf015ec60 con 0x7fccf015be40 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf6400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fccf00b0750 0x7fccf00b4050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37894/0 (socket says 172.21.15.86:37894) 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf6400640 1 -- 172.21.15.86:0/3162829507 learned_addr learned my addr 172.21.15.86:0/3162829507 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf5a00640 1 --2- 172.21.15.86:0/3162829507 >> v2:172.21.15.196:3300/0 conn(0x7fccf015be40 0x7fccf015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccefe00640 1 -- 172.21.15.86:0/3162829507 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245824081 0 0) 0x7fccf006be00 con 0x7fccf00b0750 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccefe00640 1 -- 172.21.15.86:0/3162829507 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccd8003530 con 0x7fccf00b0750 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccefe00640 1 -- 172.21.15.86:0/3162829507 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1146914971 0 0) 0x7fccf006c0e0 con 0x7fccf006f060 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccefe00640 1 -- 172.21.15.86:0/3162829507 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccf006be00 con 0x7fccf006f060 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf5a00640 1 -- 172.21.15.86:0/3162829507 >> v1:172.21.15.196:6789/0 conn(0x7fccf006f060 legacy=0x7fccf00b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf5a00640 1 -- 172.21.15.86:0/3162829507 >> v1:172.21.15.86:6789/0 conn(0x7fccf00b0750 legacy=0x7fccf00b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.455+0000 7fccf5a00640 1 -- 172.21.15.86:0/3162829507 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf015ef40 con 0x7fccf015be40 2024-03-22T04:14:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf5a00640 1 --2- 172.21.15.86:0/3162829507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf015be40 0x7fccf015f460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcce40097c0 tx=0x7fcce401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a710a5e1cfbc801 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccefe00640 1 -- 172.21.15.86:0/3162829507 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce40b1070 con 0x7fccf015be40 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccefe00640 1 -- 172.21.15.86:0/3162829507 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce40a8050 con 0x7fccf015be40 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccefe00640 1 -- 172.21.15.86:0/3162829507 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce40ac860 con 0x7fccf015be40 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 -- 172.21.15.86:0/3162829507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf015be40 msgr2=0x7fccf015f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 --2- 172.21.15.86:0/3162829507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf015be40 0x7fccf015f460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcce40097c0 tx=0x7fcce401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 -- 172.21.15.86:0/3162829507 shutdown_connections 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 --2- 172.21.15.86:0/3162829507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf015be40 0x7fccf015f460 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 -- 172.21.15.86:0/3162829507 >> 172.21.15.86:0/3162829507 conn(0x7fccf006e540 msgr2=0x7fccf006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 -- 172.21.15.86:0/3162829507 shutdown_connections 2024-03-22T04:14:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 -- 172.21.15.86:0/3162829507 wait complete. 2024-03-22T04:14:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 Processor -- start 2024-03-22T04:14:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.459+0000 7fccf6e00640 1 -- start start 2024-03-22T04:14:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf006f060 0x7fccf0146a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf00b0750 0x7fccf006fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccf015be40 0x7fccf0070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fccf00b4cf0 con 0x7fccf00b0750 2024-03-22T04:14:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccf00b4b70 con 0x7fccf006f060 2024-03-22T04:14:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fccf016cac0 con 0x7fccf015be40 2024-03-22T04:14:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf00b0750 0x7fccf006fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf006f060 0x7fccf0146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf00b0750 0x7fccf006fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52338/0 (socket says 172.21.15.86:52338) 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 -- 172.21.15.86:0/4272726935 learned_addr learned my addr 172.21.15.86:0/4272726935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6400640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccf015be40 0x7fccf0070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 -- 172.21.15.86:0/4272726935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccf015be40 msgr2=0x7fccf0070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccf015be40 0x7fccf0070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 -- 172.21.15.86:0/4272726935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf006f060 msgr2=0x7fccf0146a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf006f060 0x7fccf0146a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 -- 172.21.15.86:0/4272726935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf0152bc0 con 0x7fccf00b0750 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5a00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf006f060 0x7fccf0146a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf5000640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf00b0750 0x7fccf006fdf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00c500 tx=0x7fccdc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fcced600640 1 -- 172.21.15.86:0/4272726935 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0052d0 con 0x7fccf00b0750 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fcced600640 1 -- 172.21.15.86:0/4272726935 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccdc005490 con 0x7fccf00b0750 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fcced600640 1 -- 172.21.15.86:0/4272726935 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc012850 con 0x7fccf00b0750 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf00cf410 con 0x7fccf00b0750 2024-03-22T04:14:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf00cf9d0 con 0x7fccf00b0750 2024-03-22T04:14:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf0074470 con 0x7fccf00b0750 2024-03-22T04:14:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.463+0000 7fcced600640 1 -- 172.21.15.86:0/4272726935 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fccdc0129f0 con 0x7fccf00b0750 2024-03-22T04:14:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.467+0000 7fcced600640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fccd0067b20 0x7fccd0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.467+0000 7fcced600640 1 -- 172.21.15.86:0/4272726935 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fccdc085cd0 con 0x7fccf00b0750 2024-03-22T04:14:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.467+0000 7fccf5a00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fccd0067b20 0x7fccd0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.467+0000 7fccf5a00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fccd0067b20 0x7fccd0069fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcce40ae000 tx=0x7fcce40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.471+0000 7fcced600640 1 -- 172.21.15.86:0/4272726935 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fccdc055e50 con 0x7fccf00b0750 2024-03-22T04:14:57.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.627+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccf01508a0 con 0x7fccf00b0750 2024-03-22T04:14:57.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.627+0000 7fcced600640 1 -- 172.21.15.86:0/4272726935 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17268 (secure 0 0 0) 0x7fccdc059ef0 con 0x7fccf00b0750 2024-03-22T04:14:57.633 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:14:57.633 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":69,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:14:16.029299+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:13:34.224743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-22T04:13:34.852940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.2","mappings":[{"from":2,"to":7}]},{"pgid":"10.a","mappings":[{"from":2,"to":1}]},{"pgid":"10.10","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":5}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.26","mappings":[{"from":4,"to":1}]},{"pgid":"10.2d","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:14:57.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fccd0067b20 msgr2=0x7fccd0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fccd0067b20 0x7fccd0069fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcce40ae000 tx=0x7fcce40a7000 comp rx=0 tx=0).stop 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf00b0750 msgr2=0x7fccf006fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf00b0750 0x7fccf006fdf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00c500 tx=0x7fccdc00c9c0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 shutdown_connections 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fccd0067b20 0x7fccd0069fe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fccf015be40 0x7fccf0070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccf00b0750 0x7fccf006fdf0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 --2- 172.21.15.86:0/4272726935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf006f060 0x7fccf0146a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 >> 172.21.15.86:0/4272726935 conn(0x7fccf006e540 msgr2=0x7fccf00ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 shutdown_connections 2024-03-22T04:14:57.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.631+0000 7fccf6e00640 1 -- 172.21.15.86:0/4272726935 wait complete. 2024-03-22T04:14:57.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:14:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb92e600640 1 Processor -- start 2024-03-22T04:14:57.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb92e600640 1 -- start start 2024-03-22T04:14:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb92e600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9280ae540 0x7fb9280b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb92e600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb928071920 con 0x7fb9280730d0 2024-03-22T04:14:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb92e600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb928071c00 con 0x7fb92815b3f0 2024-03-22T04:14:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb92e600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb9280716d0 con 0x7fb9280ae540 2024-03-22T04:14:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb926a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb9280730d0 0x7fb9280ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37900/0 (socket says 172.21.15.86:37900) 2024-03-22T04:14:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb926a00640 1 -- 172.21.15.86:0/716089776 learned_addr learned my addr 172.21.15.86:0/716089776 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb927e00640 1 --2- 172.21.15.86:0/716089776 >> v2:172.21.15.196:3300/0 conn(0x7fb9280ae540 0x7fb9280b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb926000640 1 -- 172.21.15.86:0/716089776 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 165413329 0 0) 0x7fb928071920 con 0x7fb9280730d0 2024-03-22T04:14:57.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb926000640 1 -- 172.21.15.86:0/716089776 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb90c003530 con 0x7fb9280730d0 2024-03-22T04:14:57.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb926000640 1 -- 172.21.15.86:0/716089776 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2544555613 0 0) 0x7fb928071c00 con 0x7fb92815b3f0 2024-03-22T04:14:57.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb926000640 1 -- 172.21.15.86:0/716089776 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb928071920 con 0x7fb92815b3f0 2024-03-22T04:14:57.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb927e00640 1 -- 172.21.15.86:0/716089776 >> v1:172.21.15.196:6789/0 conn(0x7fb92815b3f0 legacy=0x7fb9280729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb927e00640 1 -- 172.21.15.86:0/716089776 >> v1:172.21.15.86:6789/0 conn(0x7fb9280730d0 legacy=0x7fb9280ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.787+0000 7fb927e00640 1 -- 172.21.15.86:0/716089776 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928071ee0 con 0x7fb9280ae540 2024-03-22T04:14:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb927e00640 1 --2- 172.21.15.86:0/716089776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280ae540 0x7fb9280b4300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb920009d90 tx=0x7fb92001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7296946aad7b2411 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb926000640 1 -- 172.21.15.86:0/716089776 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9200a6640 con 0x7fb9280ae540 2024-03-22T04:14:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb926000640 1 -- 172.21.15.86:0/716089776 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9200a67e0 con 0x7fb9280ae540 2024-03-22T04:14:57.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb926000640 1 -- 172.21.15.86:0/716089776 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9200a6bb0 con 0x7fb9280ae540 2024-03-22T04:14:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb92e600640 1 -- 172.21.15.86:0/716089776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280ae540 msgr2=0x7fb9280b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb92e600640 1 --2- 172.21.15.86:0/716089776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280ae540 0x7fb9280b4300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb920009d90 tx=0x7fb92001c970 comp rx=0 tx=0).stop 2024-03-22T04:14:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb92e600640 1 -- 172.21.15.86:0/716089776 shutdown_connections 2024-03-22T04:14:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb92e600640 1 --2- 172.21.15.86:0/716089776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280ae540 0x7fb9280b4300 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb92e600640 1 -- 172.21.15.86:0/716089776 >> 172.21.15.86:0/716089776 conn(0x7fb9280b1df0 msgr2=0x7fb9280b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb92e600640 1 -- 172.21.15.86:0/716089776 shutdown_connections 2024-03-22T04:14:57.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.791+0000 7fb92e600640 1 -- 172.21.15.86:0/716089776 wait complete. 2024-03-22T04:14:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 Processor -- start 2024-03-22T04:14:57.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 -- start start 2024-03-22T04:14:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280730d0 0x7fb928099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9280ae540 0x7fb9281538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb92815b3f0 0x7fb92815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9280b4c50 con 0x7fb9280ae540 2024-03-22T04:14:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9280b4ad0 con 0x7fb9280730d0 2024-03-22T04:14:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb92806e7a0 con 0x7fb92815b3f0 2024-03-22T04:14:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9280ae540 0x7fb9281538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9280ae540 0x7fb9281538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52344/0 (socket says 172.21.15.86:52344) 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 -- 172.21.15.86:0/770937945 learned_addr learned my addr 172.21.15.86:0/770937945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb927e00640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb92815b3f0 0x7fb92815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb927400640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280730d0 0x7fb928099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 -- 172.21.15.86:0/770937945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb92815b3f0 msgr2=0x7fb92815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb92815b3f0 0x7fb92815cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 -- 172.21.15.86:0/770937945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280730d0 msgr2=0x7fb928099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280730d0 0x7fb928099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 -- 172.21.15.86:0/770937945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928067ea0 con 0x7fb9280ae540 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb926a00640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9280ae540 0x7fb9281538c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb91000c4d0 tx=0x7fb91000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb91f400640 1 -- 172.21.15.86:0/770937945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb910017070 con 0x7fb9280ae540 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb91f400640 1 -- 172.21.15.86:0/770937945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb910005cf0 con 0x7fb9280ae540 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb928141c20 con 0x7fb9280ae540 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb91f400640 1 -- 172.21.15.86:0/770937945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9100072b0 con 0x7fb9280ae540 2024-03-22T04:14:57.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb928158ef0 con 0x7fb9280ae540 2024-03-22T04:14:57.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.795+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb928068650 con 0x7fb9280ae540 2024-03-22T04:14:57.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.799+0000 7fb91f400640 1 -- 172.21.15.86:0/770937945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb91001a070 con 0x7fb9280ae540 2024-03-22T04:14:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.799+0000 7fb91f400640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb904067b20 0x7fb904069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.799+0000 7fb91f400640 1 -- 172.21.15.86:0/770937945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6304+0+0 (secure 0 0 0) 0x7fb910085550 con 0x7fb9280ae540 2024-03-22T04:14:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.799+0000 7fb927400640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb904067b20 0x7fb904069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:57.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.799+0000 7fb927400640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb904067b20 0x7fb904069fe0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb918001720 tx=0x7fb918009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:57.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.803+0000 7fb91f400640 1 -- 172.21.15.86:0/770937945 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb910055640 con 0x7fb9280ae540 2024-03-22T04:14:57.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:14:57.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.995+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb928099e60 con 0x7fb9280ae540 2024-03-22T04:14:58.001 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:14:58.001 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:14:57.995+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:14:58.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.995+0000 7fb91f400640 1 -- 172.21.15.86:0/770937945 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7fb9100596e0 con 0x7fb9280ae540 2024-03-22T04:14:58.002 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:14:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb904067b20 msgr2=0x7fb904069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb904067b20 0x7fb904069fe0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb918001720 tx=0x7fb918009040 comp rx=0 tx=0).stop 2024-03-22T04:14:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9280ae540 msgr2=0x7fb9281538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9280ae540 0x7fb9281538c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb91000c4d0 tx=0x7fb91000c990 comp rx=0 tx=0).stop 2024-03-22T04:14:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 shutdown_connections 2024-03-22T04:14:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb904067b20 0x7fb904069fe0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb92815b3f0 0x7fb92815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9280ae540 0x7fb9281538c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 --2- 172.21.15.86:0/770937945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9280730d0 0x7fb928099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 >> 172.21.15.86:0/770937945 conn(0x7fb9280b1df0 msgr2=0x7fb928150420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 shutdown_connections 2024-03-22T04:14:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:57.999+0000 7fb92e600640 1 -- 172.21.15.86:0/770937945 wait complete. 2024-03-22T04:14:58.017 INFO:tasks.ceph.mds.d.smithi196.stderr: -7937> 2024-03-22T04:13:38.499+0000 7f42bf400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:14:58.020 INFO:tasks.ceph.mds.d.smithi196.stderr: -4822> 2024-03-22T04:13:49.199+0000 7f42ba400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6390=0-4194304@1} caps={6390=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6390 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562f7ea04580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-22T04:14:58.022 INFO:tasks.ceph.mds.d.smithi196.stderr: -3080> 2024-03-22T04:14:20.762+0000 7f42ba400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6390=0-4194304@1} caps={6390=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6390 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x562f7ea04580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-22T04:14:58.025 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T04:14:58.066 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:14:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e9200640 1 Processor -- start 2024-03-22T04:14:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e9200640 1 -- start start 2024-03-22T04:14:58.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7e415e5b0 0x7fa7e40b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e9200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e40b5470 con 0x7fa7e40b4eb0 2024-03-22T04:14:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e9200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e40b5750 con 0x7fa7e406b660 2024-03-22T04:14:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e9200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e4162e50 con 0x7fa7e415e5b0 2024-03-22T04:14:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7e415e5b0 0x7fa7e40b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7e40b4eb0 0x7fa7e4162740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37902/0 (socket says 172.21.15.86:37902) 2024-03-22T04:14:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e1400640 1 -- 172.21.15.86:0/3717206737 learned_addr learned my addr 172.21.15.86:0/3717206737 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:14:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e1e00640 1 -- 172.21.15.86:0/3717206737 >> v1:172.21.15.196:6789/0 conn(0x7fa7e406b660 legacy=0x7fa7e4072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e1e00640 1 -- 172.21.15.86:0/3717206737 >> v1:172.21.15.86:6789/0 conn(0x7fa7e40b4eb0 legacy=0x7fa7e4162740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:58.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.227+0000 7fa7e1e00640 1 -- 172.21.15.86:0/3717206737 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e40b5a30 con 0x7fa7e415e5b0 2024-03-22T04:14:58.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e1e00640 1 --2- 172.21.15.86:0/3717206737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b4170 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097c0 tx=0x7fa7d401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276b0705a98883d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e0a00640 1 -- 172.21.15.86:0/3717206737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40b0070 con 0x7fa7e415e5b0 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e0a00640 1 -- 172.21.15.86:0/3717206737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d40ab600 con 0x7fa7e415e5b0 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e0a00640 1 -- 172.21.15.86:0/3717206737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40a4e60 con 0x7fa7e415e5b0 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e9200640 1 -- 172.21.15.86:0/3717206737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 msgr2=0x7fa7e40b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e9200640 1 --2- 172.21.15.86:0/3717206737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b4170 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097c0 tx=0x7fa7d401c3d0 comp rx=0 tx=0).stop 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e9200640 1 -- 172.21.15.86:0/3717206737 shutdown_connections 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e9200640 1 --2- 172.21.15.86:0/3717206737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b4170 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e9200640 1 -- 172.21.15.86:0/3717206737 >> 172.21.15.86:0/3717206737 conn(0x7fa7e40d0fa0 msgr2=0x7fa7e40d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e9200640 1 -- 172.21.15.86:0/3717206737 shutdown_connections 2024-03-22T04:14:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.231+0000 7fa7e9200640 1 -- 172.21.15.86:0/3717206737 wait complete. 2024-03-22T04:14:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 Processor -- start 2024-03-22T04:14:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 -- start start 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e406b660 0x7fa7e4138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e40b4eb0 0x7fa7e40afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e416cc80 con 0x7fa7e415e5b0 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e416cb00 con 0x7fa7e40b4eb0 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e416ce00 con 0x7fa7e406b660 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52362/0 (socket says 172.21.15.86:52362) 2024-03-22T04:14:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 -- 172.21.15.86:0/2381922880 learned_addr learned my addr 172.21.15.86:0/2381922880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e1400640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e40b4eb0 0x7fa7e40afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e1e00640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e406b660 0x7fa7e4138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 -- 172.21.15.86:0/2381922880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e406b660 msgr2=0x7fa7e4138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e406b660 0x7fa7e4138d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 -- 172.21.15.86:0/2381922880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e40b4eb0 msgr2=0x7fa7e40afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e40b4eb0 0x7fa7e40afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 -- 172.21.15.86:0/2381922880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e413c5a0 con 0x7fa7e415e5b0 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e2800640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b0050 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00b9a0 tx=0x7fa7dc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7da000640 1 -- 172.21.15.86:0/2381922880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc019070 con 0x7fa7e415e5b0 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7da000640 1 -- 172.21.15.86:0/2381922880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc00cba0 con 0x7fa7e415e5b0 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7da000640 1 -- 172.21.15.86:0/2381922880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc014440 con 0x7fa7e415e5b0 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e4139290 con 0x7fa7e415e5b0 2024-03-22T04:14:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.235+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e41399a0 con 0x7fa7e415e5b0 2024-03-22T04:14:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.239+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e4074470 con 0x7fa7e415e5b0 2024-03-22T04:14:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.239+0000 7fa7da000640 1 -- 172.21.15.86:0/2381922880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa7dc01c070 con 0x7fa7e415e5b0 2024-03-22T04:14:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.239+0000 7fa7da000640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b20 0x7fa7c4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.239+0000 7fa7da000640 1 -- 172.21.15.86:0/2381922880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6390+0+0 (secure 0 0 0) 0x7fa7dc085e50 con 0x7fa7e415e5b0 2024-03-22T04:14:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.239+0000 7fa7e1e00640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b20 0x7fa7c4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.239+0000 7fa7e1e00640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b20 0x7fa7c4069fe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40ad000 tx=0x7fa7d40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.243+0000 7fa7da000640 1 -- 172.21.15.86:0/2381922880 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa7dc055e30 con 0x7fa7e415e5b0 2024-03-22T04:14:58.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.427+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa7e416a0c0 con 0x7fa7e415e5b0 2024-03-22T04:14:59.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:58.999+0000 7fa7da000640 1 -- 172.21.15.86:0/2381922880 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7fa7dc059ed0 con 0x7fa7e415e5b0 2024-03-22T04:14:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b20 msgr2=0x7fa7c4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b20 0x7fa7c4069fe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40ad000 tx=0x7fa7d40a6000 comp rx=0 tx=0).stop 2024-03-22T04:14:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 msgr2=0x7fa7e40b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b0050 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00b9a0 tx=0x7fa7dc00be60 comp rx=0 tx=0).stop 2024-03-22T04:14:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 shutdown_connections 2024-03-22T04:14:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7c4067b20 0x7fa7c4069fe0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e40b4eb0 0x7fa7e40afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e415e5b0 0x7fa7e40b0050 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 --2- 172.21.15.86:0/2381922880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e406b660 0x7fa7e4138d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 >> 172.21.15.86:0/2381922880 conn(0x7fa7e40d0fa0 msgr2=0x7fa7e4162740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 shutdown_connections 2024-03-22T04:14:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.003+0000 7fa7e9200640 1 -- 172.21.15.86:0/2381922880 wait complete. 2024-03-22T04:14:59.030 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T04:14:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6d0000640 1 Processor -- start 2024-03-22T04:14:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6d0000640 1 -- start start 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6d0000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc6c8072ec0 0x7fc6c80b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6d0000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc6c80b35f0 0x7fc6c80ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6d0000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c8144e70 con 0x7fc6c8144a90 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6d0000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc6c80b3e00 con 0x7fc6c80b35f0 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6d0000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc6c80b40e0 con 0x7fc6c8072ec0 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6ce000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc6c80b35f0 0x7fc6c80ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6ce000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc6c80b35f0 0x7fc6c80ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52368/0 (socket says 172.21.15.86:52368) 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6ce000640 1 -- 172.21.15.86:0/1125967127 learned_addr learned my addr 172.21.15.86:0/1125967127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6ccc00640 1 --2- 172.21.15.86:0/1125967127 >> v2:172.21.15.196:3300/0 conn(0x7fc6c8072ec0 0x7fc6c80b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6ce000640 1 -- 172.21.15.86:0/1125967127 >> v2:172.21.15.196:3300/0 conn(0x7fc6c8072ec0 msgr2=0x7fc6c80b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6ce000640 1 --2- 172.21.15.86:0/1125967127 >> v2:172.21.15.196:3300/0 conn(0x7fc6c8072ec0 0x7fc6c80b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6ce000640 1 -- 172.21.15.86:0/1125967127 >> v1:172.21.15.196:6789/0 conn(0x7fc6c8144a90 legacy=0x7fc6c80727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.167+0000 7fc6ce000640 1 -- 172.21.15.86:0/1125967127 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c80cdbd0 con 0x7fc6c80b35f0 2024-03-22T04:14:59.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6ce000640 1 --2- 172.21.15.86:0/1125967127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c80ce3d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc6c815b740 tx=0x7fc6c001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effd4abe7c7adedf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6c7e00640 1 -- 172.21.15.86:0/1125967127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00a58a0 con 0x7fc6c80b35f0 2024-03-22T04:14:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6c7e00640 1 -- 172.21.15.86:0/1125967127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6c00a5a40 con 0x7fc6c80b35f0 2024-03-22T04:14:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6c7e00640 1 -- 172.21.15.86:0/1125967127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00a5e10 con 0x7fc6c80b35f0 2024-03-22T04:14:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6d0000640 1 -- 172.21.15.86:0/1125967127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 msgr2=0x7fc6c80ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6d0000640 1 --2- 172.21.15.86:0/1125967127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c80ce3d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc6c815b740 tx=0x7fc6c001c9c0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6d0000640 1 -- 172.21.15.86:0/1125967127 shutdown_connections 2024-03-22T04:14:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6d0000640 1 --2- 172.21.15.86:0/1125967127 >> v2:172.21.15.196:3300/0 conn(0x7fc6c8072ec0 0x7fc6c80b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6d0000640 1 --2- 172.21.15.86:0/1125967127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c80ce3d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6d0000640 1 -- 172.21.15.86:0/1125967127 >> 172.21.15.86:0/1125967127 conn(0x7fc6c806b660 msgr2=0x7fc6c806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:14:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6d0000640 1 -- 172.21.15.86:0/1125967127 shutdown_connections 2024-03-22T04:14:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.171+0000 7fc6d0000640 1 -- 172.21.15.86:0/1125967127 wait complete. 2024-03-22T04:14:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 Processor -- start 2024-03-22T04:14:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 -- start start 2024-03-22T04:14:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6c8072ec0 0x7fc6c81f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c813d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6c8144a90 0x7fc6c81f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c80cf2a0 con 0x7fc6c80b35f0 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c80cf120 con 0x7fc6c8072ec0 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c80cf420 con 0x7fc6c8144a90 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c813d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c813d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52372/0 (socket says 172.21.15.86:52372) 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 -- 172.21.15.86:0/524456233 learned_addr learned my addr 172.21.15.86:0/524456233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6cd600640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6c8072ec0 0x7fc6c81f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ce000640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6c8144a90 0x7fc6c81f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 -- 172.21.15.86:0/524456233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6c8144a90 msgr2=0x7fc6c81f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6c8144a90 0x7fc6c81f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 -- 172.21.15.86:0/524456233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6c8072ec0 msgr2=0x7fc6c81f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:14:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6c8072ec0 0x7fc6c81f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:14:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 -- 172.21.15.86:0/524456233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c81f53e0 con 0x7fc6c80b35f0 2024-03-22T04:14:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6ccc00640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c813d490 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400e960 tx=0x7fc6b400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6c5600640 1 -- 172.21.15.86:0/524456233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4017070 con 0x7fc6c80b35f0 2024-03-22T04:14:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6c5600640 1 -- 172.21.15.86:0/524456233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b400ca00 con 0x7fc6c80b35f0 2024-03-22T04:14:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6c5600640 1 -- 172.21.15.86:0/524456233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400cdb0 con 0x7fc6c80b35f0 2024-03-22T04:14:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c8133620 con 0x7fc6c80b35f0 2024-03-22T04:14:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.175+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c8133be0 con 0x7fc6c80b35f0 2024-03-22T04:14:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.179+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c8074580 con 0x7fc6c80b35f0 2024-03-22T04:14:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.179+0000 7fc6c5600640 1 -- 172.21.15.86:0/524456233 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc6b401a070 con 0x7fc6c80b35f0 2024-03-22T04:14:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.179+0000 7fc6c5600640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6a8067b20 0x7fc6a8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:14:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.179+0000 7fc6c5600640 1 -- 172.21.15.86:0/524456233 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6390+0+0 (secure 0 0 0) 0x7fc6b4086170 con 0x7fc6c80b35f0 2024-03-22T04:14:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.179+0000 7fc6cd600640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6a8067b20 0x7fc6a8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:14:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.179+0000 7fc6cd600640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6a8067b20 0x7fc6a8069fe0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc007fd0 tx=0x7fc6bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:14:59.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.183+0000 7fc6c5600640 1 -- 172.21.15.86:0/524456233 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc6b4056280 con 0x7fc6c80b35f0 2024-03-22T04:14:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:14:59.339+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc6c813b910 con 0x7fc6c80b35f0 2024-03-22T04:15:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.003+0000 7fc6c5600640 1 -- 172.21.15.86:0/524456233 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v71) ==== 181+0+0 (secure 0 0 0) 0x7fc6b405a320 con 0x7fc6c80b35f0 2024-03-22T04:15:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6a8067b20 msgr2=0x7fc6a8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6a8067b20 0x7fc6a8069fe0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc007fd0 tx=0x7fc6bc009040 comp rx=0 tx=0).stop 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 msgr2=0x7fc6c813d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c813d490 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400e960 tx=0x7fc6b400ee20 comp rx=0 tx=0).stop 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 shutdown_connections 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6a8067b20 0x7fc6a8069fe0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6c8144a90 0x7fc6c81f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c80b35f0 0x7fc6c813d490 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 --2- 172.21.15.86:0/524456233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6c8072ec0 0x7fc6c81f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 >> 172.21.15.86:0/524456233 conn(0x7fc6c806b660 msgr2=0x7fc6c813eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 shutdown_connections 2024-03-22T04:15:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.007+0000 7fc6d0000640 1 -- 172.21.15.86:0/524456233 wait complete. 2024-03-22T04:15:00.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T04:15:00.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.171+0000 7fa1a9200640 1 Processor -- start 2024-03-22T04:15:00.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.171+0000 7fa1a9200640 1 -- start start 2024-03-22T04:15:00.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.171+0000 7fa1a9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1a4144a90 0x7fa1a40727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:00.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.171+0000 7fa1a9200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa1a40b37c0 0x7fa1a40ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:00.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.171+0000 7fa1a9200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a4144e70 con 0x7fa1a4072cf0 2024-03-22T04:15:00.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.171+0000 7fa1a9200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a40b3e00 con 0x7fa1a4144a90 2024-03-22T04:15:00.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.171+0000 7fa1a9200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a40b40e0 con 0x7fa1a40b37c0 2024-03-22T04:15:00.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1a4144a90 0x7fa1a40727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:00.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1a4144a90 0x7fa1a40727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52384/0 (socket says 172.21.15.86:52384) 2024-03-22T04:15:00.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a1e00640 1 -- 172.21.15.86:0/3618374037 learned_addr learned my addr 172.21.15.86:0/3618374037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:00.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a2800640 1 --2- 172.21.15.86:0/3618374037 >> v2:172.21.15.196:3300/0 conn(0x7fa1a40b37c0 0x7fa1a40ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:00.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a1e00640 1 -- 172.21.15.86:0/3618374037 >> v2:172.21.15.196:3300/0 conn(0x7fa1a40b37c0 msgr2=0x7fa1a40ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:00.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a1e00640 1 --2- 172.21.15.86:0/3618374037 >> v2:172.21.15.196:3300/0 conn(0x7fa1a40b37c0 0x7fa1a40ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a1e00640 1 -- 172.21.15.86:0/3618374037 >> v1:172.21.15.196:6789/0 conn(0x7fa1a4072cf0 legacy=0x7fa1a40b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:00.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a1e00640 1 -- 172.21.15.86:0/3618374037 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40cdbd0 con 0x7fa1a4144a90 2024-03-22T04:15:00.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a1e00640 1 --2- 172.21.15.86:0/3618374037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a40727b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa194009870 tx=0x7fa19401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91b8bb9fa7a531 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:00.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a0a00640 1 -- 172.21.15.86:0/3618374037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940b0070 con 0x7fa1a4144a90 2024-03-22T04:15:00.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a0a00640 1 -- 172.21.15.86:0/3618374037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1940ab600 con 0x7fa1a4144a90 2024-03-22T04:15:00.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.175+0000 7fa1a0a00640 1 -- 172.21.15.86:0/3618374037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940ab9d0 con 0x7fa1a4144a90 2024-03-22T04:15:00.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- 172.21.15.86:0/3618374037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 msgr2=0x7fa1a40727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:00.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 --2- 172.21.15.86:0/3618374037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a40727b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa194009870 tx=0x7fa19401c520 comp rx=0 tx=0).stop 2024-03-22T04:15:00.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- 172.21.15.86:0/3618374037 shutdown_connections 2024-03-22T04:15:00.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 --2- 172.21.15.86:0/3618374037 >> v2:172.21.15.196:3300/0 conn(0x7fa1a40b37c0 0x7fa1a40ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 --2- 172.21.15.86:0/3618374037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a40727b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- 172.21.15.86:0/3618374037 >> 172.21.15.86:0/3618374037 conn(0x7fa1a406b660 msgr2=0x7fa1a406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:00.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- 172.21.15.86:0/3618374037 shutdown_connections 2024-03-22T04:15:00.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- 172.21.15.86:0/3618374037 wait complete. 2024-03-22T04:15:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 Processor -- start 2024-03-22T04:15:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- start start 2024-03-22T04:15:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a4072cf0 0x7fa1a415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a40b37c0 0x7fa1a40cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a413dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a40cf4b0 con 0x7fa1a4144a90 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a40cf330 con 0x7fa1a40b37c0 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a9200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a40cf630 con 0x7fa1a4072cf0 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a413dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a413dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52396/0 (socket says 172.21.15.86:52396) 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a2800640 1 -- 172.21.15.86:0/1680861470 learned_addr learned my addr 172.21.15.86:0/1680861470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a1400640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a40b37c0 0x7fa1a40cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a1e00640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a4072cf0 0x7fa1a415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a2800640 1 -- 172.21.15.86:0/1680861470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a4072cf0 msgr2=0x7fa1a415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a2800640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a4072cf0 0x7fa1a415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a2800640 1 -- 172.21.15.86:0/1680861470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a40b37c0 msgr2=0x7fa1a40cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a2800640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a40b37c0 0x7fa1a40cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.179+0000 7fa1a2800640 1 -- 172.21.15.86:0/1680861470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40b11e0 con 0x7fa1a4144a90 2024-03-22T04:15:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa1a2800640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a413dc90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00efc0 tx=0x7fa19c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa19a000640 1 -- 172.21.15.86:0/1680861470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c018020 con 0x7fa1a4144a90 2024-03-22T04:15:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa19a000640 1 -- 172.21.15.86:0/1680861470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c014480 con 0x7fa1a4144a90 2024-03-22T04:15:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a40b14c0 con 0x7fa1a4144a90 2024-03-22T04:15:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa19a000640 1 -- 172.21.15.86:0/1680861470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c014870 con 0x7fa1a4144a90 2024-03-22T04:15:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a40b1a80 con 0x7fa1a4144a90 2024-03-22T04:15:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a4074580 con 0x7fa1a4144a90 2024-03-22T04:15:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa19a000640 1 -- 172.21.15.86:0/1680861470 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa19c01b020 con 0x7fa1a4144a90 2024-03-22T04:15:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa19a000640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa184067b50 0x7fa18406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa19a000640 1 -- 172.21.15.86:0/1680861470 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5952+0+0 (secure 0 0 0) 0x7fa19c086960 con 0x7fa1a4144a90 2024-03-22T04:15:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.183+0000 7fa1a1e00640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa184067b50 0x7fa18406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.187+0000 7fa1a1e00640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa184067b50 0x7fa18406a010 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa1940ad000 tx=0x7fa194008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.187+0000 7fa19a000640 1 -- 172.21.15.86:0/1680861470 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa19c022070 con 0x7fa1a4144a90 2024-03-22T04:15:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:00.343+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa1a4142850 con 0x7fa1a4144a90 2024-03-22T04:15:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.007+0000 7fa19a000640 1 -- 172.21.15.86:0/1680861470 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v72) ==== 169+0+0 (secure 0 0 0) 0x7fa19c010040 con 0x7fa1a4144a90 2024-03-22T04:15:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T04:15:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.007+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa184067b50 msgr2=0x7fa18406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.007+0000 7fa1a9200640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa184067b50 0x7fa18406a010 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa1940ad000 tx=0x7fa194008040 comp rx=0 tx=0).stop 2024-03-22T04:15:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.007+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 msgr2=0x7fa1a413dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.007+0000 7fa1a9200640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a413dc90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00efc0 tx=0x7fa19c00c490 comp rx=0 tx=0).stop 2024-03-22T04:15:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.011+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 shutdown_connections 2024-03-22T04:15:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.011+0000 7fa1a9200640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa184067b50 0x7fa18406a010 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.011+0000 7fa1a9200640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a40b37c0 0x7fa1a40cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.011+0000 7fa1a9200640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a4144a90 0x7fa1a413dc90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.011+0000 7fa1a9200640 1 --2- 172.21.15.86:0/1680861470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a4072cf0 0x7fa1a415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.011+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 >> 172.21.15.86:0/1680861470 conn(0x7fa1a406b660 msgr2=0x7fa1a4141980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.011+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 shutdown_connections 2024-03-22T04:15:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.011+0000 7fa1a9200640 1 -- 172.21.15.86:0/1680861470 wait complete. 2024-03-22T04:15:01.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-22T04:15:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.171+0000 7fc92b600640 1 Processor -- start 2024-03-22T04:15:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.171+0000 7fc92b600640 1 -- start start 2024-03-22T04:15:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc92b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc92415e7e0 0x7fc9240d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc92b600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9240b4710 0x7fc92406fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc92b600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc924070390 0x7fc92409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc92b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc9240b4b70 con 0x7fc92415e7e0 2024-03-22T04:15:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc92b600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc92406f970 con 0x7fc924070390 2024-03-22T04:15:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc92b600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9240d15f0 con 0x7fc9240b4710 2024-03-22T04:15:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc928c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc92415e7e0 0x7fc9240d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc928c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc92415e7e0 0x7fc9240d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52412/0 (socket says 172.21.15.86:52412) 2024-03-22T04:15:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc928c00640 1 -- 172.21.15.86:0/1919117569 learned_addr learned my addr 172.21.15.86:0/1919117569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc929600640 1 --2- 172.21.15.86:0/1919117569 >> v2:172.21.15.196:3300/0 conn(0x7fc924070390 0x7fc92409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc928c00640 1 -- 172.21.15.86:0/1919117569 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9240b4710 msgr2=0x7fc92406fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc928c00640 1 --2- 172.21.15.86:0/1919117569 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9240b4710 0x7fc92406fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc928c00640 1 -- 172.21.15.86:0/1919117569 >> v2:172.21.15.196:3300/0 conn(0x7fc924070390 msgr2=0x7fc92409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc928c00640 1 --2- 172.21.15.86:0/1919117569 >> v2:172.21.15.196:3300/0 conn(0x7fc924070390 0x7fc92409aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc928c00640 1 -- 172.21.15.86:0/1919117569 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc924070820 con 0x7fc92415e7e0 2024-03-22T04:15:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.175+0000 7fc929600640 1 --2- 172.21.15.86:0/1919117569 >> v2:172.21.15.196:3300/0 conn(0x7fc924070390 0x7fc92409aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc928c00640 1 --2- 172.21.15.86:0/1919117569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92415e7e0 0x7fc9240d0d30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cc650 tx=0x7fc91801c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf61c0b3b812d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc923400640 1 -- 172.21.15.86:0/1919117569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9180b0070 con 0x7fc92415e7e0 2024-03-22T04:15:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc923400640 1 -- 172.21.15.86:0/1919117569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9180ab600 con 0x7fc92415e7e0 2024-03-22T04:15:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc923400640 1 -- 172.21.15.86:0/1919117569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9180ab9d0 con 0x7fc92415e7e0 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 -- 172.21.15.86:0/1919117569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92415e7e0 msgr2=0x7fc9240d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 --2- 172.21.15.86:0/1919117569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92415e7e0 0x7fc9240d0d30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cc650 tx=0x7fc91801c110 comp rx=0 tx=0).stop 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 -- 172.21.15.86:0/1919117569 shutdown_connections 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 --2- 172.21.15.86:0/1919117569 >> v2:172.21.15.196:3300/0 conn(0x7fc924070390 0x7fc92409aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 --2- 172.21.15.86:0/1919117569 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9240b4710 0x7fc92406fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 --2- 172.21.15.86:0/1919117569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92415e7e0 0x7fc9240d0d30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 -- 172.21.15.86:0/1919117569 >> 172.21.15.86:0/1919117569 conn(0x7fc9240b3bf0 msgr2=0x7fc9240b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 -- 172.21.15.86:0/1919117569 shutdown_connections 2024-03-22T04:15:01.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 -- 172.21.15.86:0/1919117569 wait complete. 2024-03-22T04:15:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 Processor -- start 2024-03-22T04:15:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.179+0000 7fc92b600640 1 -- start start 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc924070390 0x7fc92414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9240b4710 0x7fc92415cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc92415e7e0 0x7fc92415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9240cd360 con 0x7fc9240b4710 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9240cce40 con 0x7fc924070390 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9240cd4e0 con 0x7fc92415e7e0 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9240b4710 0x7fc92415cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9240b4710 0x7fc92415cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52428/0 (socket says 172.21.15.86:52428) 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 -- 172.21.15.86:0/2528910413 learned_addr learned my addr 172.21.15.86:0/2528910413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc928c00640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc924070390 0x7fc92414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc929600640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc92415e7e0 0x7fc92415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 -- 172.21.15.86:0/2528910413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc92415e7e0 msgr2=0x7fc92415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc92415e7e0 0x7fc92415d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 -- 172.21.15.86:0/2528910413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc924070390 msgr2=0x7fc92414ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc924070390 0x7fc92414ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 -- 172.21.15.86:0/2528910413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9240af640 con 0x7fc9240b4710 2024-03-22T04:15:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc923e00640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9240b4710 0x7fc92415cf70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc91000b550 tx=0x7fc91000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc920c00640 1 -- 172.21.15.86:0/2528910413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910014070 con 0x7fc9240b4710 2024-03-22T04:15:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc920c00640 1 -- 172.21.15.86:0/2528910413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910010890 con 0x7fc9240b4710 2024-03-22T04:15:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9240af920 con 0x7fc9240b4710 2024-03-22T04:15:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc920c00640 1 -- 172.21.15.86:0/2528910413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910010c80 con 0x7fc9240b4710 2024-03-22T04:15:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92414e410 con 0x7fc9240b4710 2024-03-22T04:15:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc924074470 con 0x7fc9240b4710 2024-03-22T04:15:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.183+0000 7fc920c00640 1 -- 172.21.15.86:0/2528910413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc910017070 con 0x7fc9240b4710 2024-03-22T04:15:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.187+0000 7fc920c00640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc904067b20 0x7fc904069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.187+0000 7fc920c00640 1 -- 172.21.15.86:0/2528910413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5335+0+0 (secure 0 0 0) 0x7fc910085b80 con 0x7fc9240b4710 2024-03-22T04:15:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.187+0000 7fc928c00640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc904067b20 0x7fc904069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.187+0000 7fc928c00640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc904067b20 0x7fc904069fe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc9180ad000 tx=0x7fc9180a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:01.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.191+0000 7fc920c00640 1 -- 172.21.15.86:0/2528910413 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc910056010 con 0x7fc9240b4710 2024-03-22T04:15:01.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.347+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc92406fc30 con 0x7fc9240b4710 2024-03-22T04:15:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc920c00640 1 -- 172.21.15.86:0/2528910413 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fc910010e20 con 0x7fc9240b4710 2024-03-22T04:15:01.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc904067b20 msgr2=0x7fc904069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc904067b20 0x7fc904069fe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc9180ad000 tx=0x7fc9180a6000 comp rx=0 tx=0).stop 2024-03-22T04:15:01.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9240b4710 msgr2=0x7fc92415cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9240b4710 0x7fc92415cf70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc91000b550 tx=0x7fc91000ba10 comp rx=0 tx=0).stop 2024-03-22T04:15:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 shutdown_connections 2024-03-22T04:15:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc904067b20 0x7fc904069fe0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc92415e7e0 0x7fc92415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9240b4710 0x7fc92415cf70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 --2- 172.21.15.86:0/2528910413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc924070390 0x7fc92414ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 >> 172.21.15.86:0/2528910413 conn(0x7fc9240b3bf0 msgr2=0x7fc92406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 shutdown_connections 2024-03-22T04:15:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.351+0000 7fc92b600640 1 -- 172.21.15.86:0/2528910413 wait complete. 2024-03-22T04:15:01.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-22T04:15:01.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18f1c00640 1 Processor -- start 2024-03-22T04:15:01.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18f1c00640 1 -- start start 2024-03-22T04:15:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18f1c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f18ec0ae540 0x7f18ec0b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18f1c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18ec071920 con 0x7f18ec0730d0 2024-03-22T04:15:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18f1c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18ec071c00 con 0x7f18ec15b3f0 2024-03-22T04:15:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18f1c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f18ec0716d0 con 0x7f18ec0ae540 2024-03-22T04:15:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18ea000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f18ec0730d0 0x7f18ec0ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37912/0 (socket says 172.21.15.86:37912) 2024-03-22T04:15:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18ea000640 1 -- 172.21.15.86:0/4278189215 learned_addr learned my addr 172.21.15.86:0/4278189215 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18eb400640 1 --2- 172.21.15.86:0/4278189215 >> v2:172.21.15.196:3300/0 conn(0x7f18ec0ae540 0x7f18ec0b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18e9600640 1 -- 172.21.15.86:0/4278189215 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1120182961 0 0) 0x7f18ec071920 con 0x7f18ec0730d0 2024-03-22T04:15:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18e9600640 1 -- 172.21.15.86:0/4278189215 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18d0005740 con 0x7f18ec0730d0 2024-03-22T04:15:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18e9600640 1 -- 172.21.15.86:0/4278189215 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2463809073 0 0) 0x7f18ec071c00 con 0x7f18ec15b3f0 2024-03-22T04:15:01.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18e9600640 1 -- 172.21.15.86:0/4278189215 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18ec071920 con 0x7f18ec15b3f0 2024-03-22T04:15:01.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18eb400640 1 -- 172.21.15.86:0/4278189215 >> v1:172.21.15.196:6789/0 conn(0x7f18ec15b3f0 legacy=0x7f18ec0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18eb400640 1 -- 172.21.15.86:0/4278189215 >> v1:172.21.15.86:6789/0 conn(0x7f18ec0730d0 legacy=0x7f18ec0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.507+0000 7f18eb400640 1 -- 172.21.15.86:0/4278189215 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec071ee0 con 0x7f18ec0ae540 2024-03-22T04:15:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18eb400640 1 --2- 172.21.15.86:0/4278189215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0ae540 0x7f18ec0b4300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009d90 tx=0x7f18e401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d0c1f6a1998cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18e9600640 1 -- 172.21.15.86:0/4278189215 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e40a6640 con 0x7f18ec0ae540 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18e9600640 1 -- 172.21.15.86:0/4278189215 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18e40a67e0 con 0x7f18ec0ae540 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18e9600640 1 -- 172.21.15.86:0/4278189215 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e40a6bb0 con 0x7f18ec0ae540 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18f1c00640 1 -- 172.21.15.86:0/4278189215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0ae540 msgr2=0x7f18ec0b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18f1c00640 1 --2- 172.21.15.86:0/4278189215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0ae540 0x7f18ec0b4300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009d90 tx=0x7f18e401c970 comp rx=0 tx=0).stop 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18f1c00640 1 -- 172.21.15.86:0/4278189215 shutdown_connections 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18f1c00640 1 --2- 172.21.15.86:0/4278189215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0ae540 0x7f18ec0b4300 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18f1c00640 1 -- 172.21.15.86:0/4278189215 >> 172.21.15.86:0/4278189215 conn(0x7f18ec0b1df0 msgr2=0x7f18ec0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18f1c00640 1 -- 172.21.15.86:0/4278189215 shutdown_connections 2024-03-22T04:15:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.511+0000 7f18f1c00640 1 -- 172.21.15.86:0/4278189215 wait complete. 2024-03-22T04:15:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 Processor -- start 2024-03-22T04:15:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 -- start start 2024-03-22T04:15:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0730d0 0x7f18ec14a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ec0ae540 0x7f18ec156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18ec15b3f0 0x7f18ec156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec0b4c50 con 0x7f18ec0ae540 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec0b4ad0 con 0x7f18ec0730d0 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f18ec06e7a0 con 0x7f18ec15b3f0 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ec0ae540 0x7f18ec156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ec0ae540 0x7f18ec156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52434/0 (socket says 172.21.15.86:52434) 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 -- 172.21.15.86:0/309511271 learned_addr learned my addr 172.21.15.86:0/309511271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18eaa00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0730d0 0x7f18ec14a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18eb400640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18ec15b3f0 0x7f18ec156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 -- 172.21.15.86:0/309511271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18ec15b3f0 msgr2=0x7f18ec156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18ec15b3f0 0x7f18ec156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 -- 172.21.15.86:0/309511271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0730d0 msgr2=0x7f18ec14a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0730d0 0x7f18ec14a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 -- 172.21.15.86:0/309511271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec14ff30 con 0x7f18ec0ae540 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18ea000640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ec0ae540 0x7f18ec156a70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c500 tx=0x7f18dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18e2a00640 1 -- 172.21.15.86:0/309511271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc017070 con 0x7f18ec0ae540 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18e2a00640 1 -- 172.21.15.86:0/309511271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18dc005cf0 con 0x7f18ec0ae540 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18e2a00640 1 -- 172.21.15.86:0/309511271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc007330 con 0x7f18ec0ae540 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec150210 con 0x7f18ec0ae540 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.515+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1507d0 con 0x7f18ec0ae540 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.519+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec068650 con 0x7f18ec0ae540 2024-03-22T04:15:01.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.519+0000 7f18e2a00640 1 -- 172.21.15.86:0/309511271 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f18dc01a070 con 0x7f18ec0ae540 2024-03-22T04:15:01.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.519+0000 7f18e2a00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18c8067b50 0x7f18c806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:01.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.519+0000 7f18e2a00640 1 -- 172.21.15.86:0/309511271 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5335+0+0 (secure 0 0 0) 0x7f18dc085170 con 0x7f18ec0ae540 2024-03-22T04:15:01.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.519+0000 7f18eaa00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18c8067b50 0x7f18c806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:01.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.519+0000 7f18eaa00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18c8067b50 0x7f18c806a010 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f18d40011f0 tx=0x7f18d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:01.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.523+0000 7f18e2a00640 1 -- 172.21.15.86:0/309511271 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f18dc055690 con 0x7f18ec0ae540 2024-03-22T04:15:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:01.735+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f18ec14aad0 con 0x7f18ec0ae540 2024-03-22T04:15:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.007+0000 7f18e2a00640 1 -- 172.21.15.86:0/309511271 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v247) ==== 169+0+0 (secure 0 0 0) 0x7f18dc059730 con 0x7f18ec0ae540 2024-03-22T04:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18c8067b50 msgr2=0x7f18c806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18c8067b50 0x7f18c806a010 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f18d40011f0 tx=0x7f18d4009040 comp rx=0 tx=0).stop 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ec0ae540 msgr2=0x7f18ec156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ec0ae540 0x7f18ec156a70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c500 tx=0x7f18dc00c9c0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 shutdown_connections 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18c8067b50 0x7f18c806a010 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18ec15b3f0 0x7f18ec156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ec0ae540 0x7f18ec156a70 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 --2- 172.21.15.86:0/309511271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18ec0730d0 0x7f18ec14a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 >> 172.21.15.86:0/309511271 conn(0x7f18ec0b1df0 msgr2=0x7f18ec0729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 shutdown_connections 2024-03-22T04:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.011+0000 7f18f1c00640 1 -- 172.21.15.86:0/309511271 wait complete. 2024-03-22T04:15:02.034 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-22T04:15:02.034 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-22T04:15:02.034 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-22T04:15:02.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:15:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792c00640 1 Processor -- start 2024-03-22T04:15:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792c00640 1 -- start start 2024-03-22T04:15:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff78c06ddd0 0x7ff78c15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff78c15f440 0x7ff78c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff78c15b680 con 0x7ff78c13e060 2024-03-22T04:15:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff78c15c6b0 con 0x7ff78c06ddd0 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff78c15b8f0 con 0x7ff78c15f440 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff791800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff78c13e060 0x7ff78c06d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37928/0 (socket says 172.21.15.86:37928) 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff791800640 1 -- 172.21.15.86:0/2378742038 learned_addr learned my addr 172.21.15.86:0/2378742038 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff790e00640 1 --2- 172.21.15.86:0/2378742038 >> v2:172.21.15.86:3300/0 conn(0x7ff78c06ddd0 0x7ff78c15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792200640 1 --2- 172.21.15.86:0/2378742038 >> v2:172.21.15.196:3300/0 conn(0x7ff78c15f440 0x7ff78c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff78be00640 1 -- 172.21.15.86:0/2378742038 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287037153 0 0) 0x7ff78c15b680 con 0x7ff78c13e060 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff78be00640 1 -- 172.21.15.86:0/2378742038 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff774003530 con 0x7ff78c13e060 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff790e00640 1 -- 172.21.15.86:0/2378742038 >> v2:172.21.15.196:3300/0 conn(0x7ff78c15f440 msgr2=0x7ff78c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff790e00640 1 --2- 172.21.15.86:0/2378742038 >> v2:172.21.15.196:3300/0 conn(0x7ff78c15f440 0x7ff78c0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff790e00640 1 -- 172.21.15.86:0/2378742038 >> v1:172.21.15.86:6789/0 conn(0x7ff78c13e060 legacy=0x7ff78c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff790e00640 1 -- 172.21.15.86:0/2378742038 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c15bbd0 con 0x7ff78c06ddd0 2024-03-22T04:15:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.167+0000 7ff792200640 1 --2- 172.21.15.86:0/2378742038 >> v2:172.21.15.196:3300/0 conn(0x7ff78c15f440 0x7ff78c0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:02.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff790e00640 1 --2- 172.21.15.86:0/2378742038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c06ddd0 0x7ff78c15ef00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0cdd40 tx=0x7ff77801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd159a0cd616919d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff78be00640 1 -- 172.21.15.86:0/2378742038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780b0070 con 0x7ff78c06ddd0 2024-03-22T04:15:02.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff78be00640 1 -- 172.21.15.86:0/2378742038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7780ab600 con 0x7ff78c06ddd0 2024-03-22T04:15:02.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff78be00640 1 -- 172.21.15.86:0/2378742038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780ab9b0 con 0x7ff78c06ddd0 2024-03-22T04:15:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 -- 172.21.15.86:0/2378742038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c06ddd0 msgr2=0x7ff78c15ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 --2- 172.21.15.86:0/2378742038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c06ddd0 0x7ff78c15ef00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0cdd40 tx=0x7ff77801c3f0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 -- 172.21.15.86:0/2378742038 shutdown_connections 2024-03-22T04:15:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 --2- 172.21.15.86:0/2378742038 >> v2:172.21.15.196:3300/0 conn(0x7ff78c15f440 0x7ff78c0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 --2- 172.21.15.86:0/2378742038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c06ddd0 0x7ff78c15ef00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 -- 172.21.15.86:0/2378742038 >> 172.21.15.86:0/2378742038 conn(0x7ff78c06b660 msgr2=0x7ff78c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 -- 172.21.15.86:0/2378742038 shutdown_connections 2024-03-22T04:15:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 -- 172.21.15.86:0/2378742038 wait complete. 2024-03-22T04:15:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 Processor -- start 2024-03-22T04:15:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.171+0000 7ff792c00640 1 -- start start 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c06ddd0 0x7ff78c0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c13e060 0x7ff78c14ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f440 0x7ff78c13c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c15f9d0 con 0x7ff78c13e060 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c15f850 con 0x7ff78c06ddd0 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff78c0cf7f0 con 0x7ff78c15f440 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c13e060 0x7ff78c14ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c13e060 0x7ff78c14ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52452/0 (socket says 172.21.15.86:52452) 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 -- 172.21.15.86:0/1043262066 learned_addr learned my addr 172.21.15.86:0/1043262066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 -- 172.21.15.86:0/1043262066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f440 msgr2=0x7ff78c13c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f440 0x7ff78c13c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 -- 172.21.15.86:0/1043262066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c06ddd0 msgr2=0x7ff78c0b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c06ddd0 0x7ff78c0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 -- 172.21.15.86:0/1043262066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c13d770 con 0x7ff78c13e060 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff790e00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c13e060 0x7ff78c14ab10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff7780ae040 tx=0x7ff778019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff789600640 1 -- 172.21.15.86:0/1043262066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780b0040 con 0x7ff78c13e060 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff789600640 1 -- 172.21.15.86:0/1043262066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7780b9070 con 0x7ff78c13e060 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff789600640 1 -- 172.21.15.86:0/1043262066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780b4690 con 0x7ff78c13e060 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c0b0910 con 0x7ff78c13e060 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c0b0ed0 con 0x7ff78c13e060 2024-03-22T04:15:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c074470 con 0x7ff78c13e060 2024-03-22T04:15:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.175+0000 7ff789600640 1 -- 172.21.15.86:0/1043262066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff7780b4830 con 0x7ff78c13e060 2024-03-22T04:15:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.179+0000 7ff789600640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff76c067b50 0x7ff76c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.179+0000 7ff789600640 1 -- 172.21.15.86:0/1043262066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5335+0+0 (secure 0 0 0) 0x7ff77811fad0 con 0x7ff78c13e060 2024-03-22T04:15:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.179+0000 7ff791800640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff76c067b50 0x7ff76c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.179+0000 7ff791800640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff76c067b50 0x7ff76c06a010 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff780007fd0 tx=0x7ff780007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.183+0000 7ff789600640 1 -- 172.21.15.86:0/1043262066 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff778016070 con 0x7ff78c13e060 2024-03-22T04:15:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.347+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff78c06e330 con 0x7ff78c13e060 2024-03-22T04:15:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.347+0000 7ff789600640 1 -- 172.21.15.86:0/1043262066 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+13296 (secure 0 0 0) 0x7ff7780f0640 con 0x7ff78c13e060 2024-03-22T04:15:02.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:02.353 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":72,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:01.008955+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/3774275697":"2024-03-23T04:14:58.000077+0000","172.21.15.196:6832/3774275697":"2024-03-23T04:14:58.000077+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:15:02.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff76c067b50 msgr2=0x7ff76c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff76c067b50 0x7ff76c06a010 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff780007fd0 tx=0x7ff780007900 comp rx=0 tx=0).stop 2024-03-22T04:15:02.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c13e060 msgr2=0x7ff78c14ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c13e060 0x7ff78c14ab10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff7780ae040 tx=0x7ff778019190 comp rx=0 tx=0).stop 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 shutdown_connections 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff76c067b50 0x7ff76c06a010 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f440 0x7ff78c13c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c13e060 0x7ff78c14ab10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 --2- 172.21.15.86:0/1043262066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c06ddd0 0x7ff78c0b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 >> 172.21.15.86:0/1043262066 conn(0x7ff78c06b660 msgr2=0x7ff78c071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 shutdown_connections 2024-03-22T04:15:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.351+0000 7ff792c00640 1 -- 172.21.15.86:0/1043262066 wait complete. 2024-03-22T04:15:02.375 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:15:02.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:15:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea82600640 1 Processor -- start 2024-03-22T04:15:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea82600640 1 -- start start 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea82600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea7c0c6f20 0x7fea7c072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea82600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0cb1d0 0x7fea7c162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea82600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fea7c0c7300 0x7fea7c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea82600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c0cc7c0 con 0x7fea7c0c6f20 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea82600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c0cb720 con 0x7fea7c0c7300 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea82600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c0cba00 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea7b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea7c0c6f20 0x7fea7c072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea7b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea7c0c6f20 0x7fea7c072840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52460/0 (socket says 172.21.15.86:52460) 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea7b400640 1 -- 172.21.15.86:0/1979572451 learned_addr learned my addr 172.21.15.86:0/1979572451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea72a00640 1 --2- 172.21.15.86:0/1979572451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0cb1d0 0x7fea7c162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.507+0000 7fea7be00640 1 --2- 172.21.15.86:0/1979572451 >> v2:172.21.15.196:3300/0 conn(0x7fea7c0c7300 0x7fea7c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7b400640 1 -- 172.21.15.86:0/1979572451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0cb1d0 msgr2=0x7fea7c162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7b400640 1 --2- 172.21.15.86:0/1979572451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0cb1d0 0x7fea7c162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7b400640 1 -- 172.21.15.86:0/1979572451 >> v2:172.21.15.196:3300/0 conn(0x7fea7c0c7300 msgr2=0x7fea7c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7b400640 1 --2- 172.21.15.86:0/1979572451 >> v2:172.21.15.196:3300/0 conn(0x7fea7c0c7300 0x7fea7c162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7b400640 1 -- 172.21.15.86:0/1979572451 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c0cbce0 con 0x7fea7c0c6f20 2024-03-22T04:15:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7b400640 1 --2- 172.21.15.86:0/1979572451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0c6f20 0x7fea7c072840 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0d1400 tx=0x7fea6c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15763507b18b076 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7aa00640 1 -- 172.21.15.86:0/1979572451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0aa610 con 0x7fea7c0c6f20 2024-03-22T04:15:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7aa00640 1 -- 172.21.15.86:0/1979572451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6c0af070 con 0x7fea7c0c6f20 2024-03-22T04:15:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea7aa00640 1 -- 172.21.15.86:0/1979572451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0aa9d0 con 0x7fea7c0c6f20 2024-03-22T04:15:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 -- 172.21.15.86:0/1979572451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0c6f20 msgr2=0x7fea7c072840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 --2- 172.21.15.86:0/1979572451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0c6f20 0x7fea7c072840 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0d1400 tx=0x7fea6c01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 -- 172.21.15.86:0/1979572451 shutdown_connections 2024-03-22T04:15:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 --2- 172.21.15.86:0/1979572451 >> v2:172.21.15.196:3300/0 conn(0x7fea7c0c7300 0x7fea7c162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 --2- 172.21.15.86:0/1979572451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0cb1d0 0x7fea7c162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 --2- 172.21.15.86:0/1979572451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0c6f20 0x7fea7c072840 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 -- 172.21.15.86:0/1979572451 >> 172.21.15.86:0/1979572451 conn(0x7fea7c0d0730 msgr2=0x7fea7c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 -- 172.21.15.86:0/1979572451 shutdown_connections 2024-03-22T04:15:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.511+0000 7fea82600640 1 -- 172.21.15.86:0/1979572451 wait complete. 2024-03-22T04:15:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 Processor -- start 2024-03-22T04:15:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 -- start start 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0c6f20 0x7fea7c156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea7c0c7300 0x7fea7c148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0cb1d0 0x7fea7c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0ceea0 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0ced20 con 0x7fea7c0c7300 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c0cf020 con 0x7fea7c0c6f20 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0cb1d0 0x7fea7c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0cb1d0 0x7fea7c14d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52466/0 (socket says 172.21.15.86:52466) 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 -- 172.21.15.86:0/718565389 learned_addr learned my addr 172.21.15.86:0/718565389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7b400640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0c6f20 0x7fea7c156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea72a00640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea7c0c7300 0x7fea7c148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 -- 172.21.15.86:0/718565389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0c6f20 msgr2=0x7fea7c156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0c6f20 0x7fea7c156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 -- 172.21.15.86:0/718565389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea7c0c7300 msgr2=0x7fea7c148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea7c0c7300 0x7fea7c148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 -- 172.21.15.86:0/718565389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c155690 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea7be00640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0cb1d0 0x7fea7c14d8e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fea74007e70 tx=0x7fea7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea73e00640 1 -- 172.21.15.86:0/718565389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74014620 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea73e00640 1 -- 172.21.15.86:0/718565389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea74019070 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea73e00640 1 -- 172.21.15.86:0/718565389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74014a20 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c155970 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.515+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c155f30 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.519+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c074470 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.519+0000 7fea73e00640 1 -- 172.21.15.86:0/718565389 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fea74014bc0 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.519+0000 7fea73e00640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea58067b20 0x7fea58069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.519+0000 7fea73e00640 1 -- 172.21.15.86:0/718565389 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5335+0+0 (secure 0 0 0) 0x7fea74086670 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.519+0000 7fea7b400640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea58067b20 0x7fea58069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.519+0000 7fea7b400640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea58067b20 0x7fea58069fe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0ae000 tx=0x7fea6c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.523+0000 7fea73e00640 1 -- 172.21.15.86:0/718565389 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fea74056a70 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea7c06f8f0 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea73e00640 1 -- 172.21.15.86:0/718565389 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fea7405ab10 con 0x7fea7c0cb1d0 2024-03-22T04:15:02.683 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:15:02.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea58067b20 msgr2=0x7fea58069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea58067b20 0x7fea58069fe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0ae000 tx=0x7fea6c0a7000 comp rx=0 tx=0).stop 2024-03-22T04:15:02.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0cb1d0 msgr2=0x7fea7c14d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0cb1d0 0x7fea7c14d8e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fea74007e70 tx=0x7fea7400e4b0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 shutdown_connections 2024-03-22T04:15:02.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea58067b20 0x7fea58069fe0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea7c0c7300 0x7fea7c148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea7c0cb1d0 0x7fea7c14d8e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 --2- 172.21.15.86:0/718565389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea7c0c6f20 0x7fea7c156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 >> 172.21.15.86:0/718565389 conn(0x7fea7c0d0730 msgr2=0x7fea7c06fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:02.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 shutdown_connections 2024-03-22T04:15:02.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.679+0000 7fea82600640 1 -- 172.21.15.86:0/718565389 wait complete. 2024-03-22T04:15:02.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-22T04:15:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.839+0000 7f5586800640 1 Processor -- start 2024-03-22T04:15:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f5586800640 1 -- start start 2024-03-22T04:15:02.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f5586800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f55800ad330 0x7f558015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f5586800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55800ad710 0x7f5580142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f5586800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f558015c7c0 con 0x7f558006ef40 2024-03-22T04:15:02.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f5586800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f558006dea0 con 0x7f55800ad710 2024-03-22T04:15:02.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f5586800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f558006e180 con 0x7f55800ad330 2024-03-22T04:15:02.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f557fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55800ad710 0x7f5580142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f557fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55800ad710 0x7f5580142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52476/0 (socket says 172.21.15.86:52476) 2024-03-22T04:15:02.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f557fe00640 1 -- 172.21.15.86:0/2060218307 learned_addr learned my addr 172.21.15.86:0/2060218307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:02.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f557f400640 1 --2- 172.21.15.86:0/2060218307 >> v2:172.21.15.196:3300/0 conn(0x7f55800ad330 0x7f558015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f557fe00640 1 -- 172.21.15.86:0/2060218307 >> v2:172.21.15.196:3300/0 conn(0x7f55800ad330 msgr2=0x7f558015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:02.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f557fe00640 1 --2- 172.21.15.86:0/2060218307 >> v2:172.21.15.196:3300/0 conn(0x7f55800ad330 0x7f558015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f557fe00640 1 -- 172.21.15.86:0/2060218307 >> v1:172.21.15.196:6789/0 conn(0x7f558006ef40 legacy=0x7f558013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.843+0000 7f557fe00640 1 -- 172.21.15.86:0/2060218307 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558006e460 con 0x7f55800ad710 2024-03-22T04:15:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f557fe00640 1 --2- 172.21.15.86:0/2060218307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f5580142690 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f558006d080 tx=0x7f557801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7bf6671eef5310 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f557e000640 1 -- 172.21.15.86:0/2060218307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780a5880 con 0x7f55800ad710 2024-03-22T04:15:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f557e000640 1 -- 172.21.15.86:0/2060218307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55780a5a20 con 0x7f55800ad710 2024-03-22T04:15:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f557e000640 1 -- 172.21.15.86:0/2060218307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780a5df0 con 0x7f55800ad710 2024-03-22T04:15:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 -- 172.21.15.86:0/2060218307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 msgr2=0x7f5580142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 --2- 172.21.15.86:0/2060218307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f5580142690 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f558006d080 tx=0x7f557801c9c0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 -- 172.21.15.86:0/2060218307 shutdown_connections 2024-03-22T04:15:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 --2- 172.21.15.86:0/2060218307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f5580142690 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 --2- 172.21.15.86:0/2060218307 >> v2:172.21.15.196:3300/0 conn(0x7f55800ad330 0x7f558015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 -- 172.21.15.86:0/2060218307 >> 172.21.15.86:0/2060218307 conn(0x7f55800b1df0 msgr2=0x7f55800b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 -- 172.21.15.86:0/2060218307 shutdown_connections 2024-03-22T04:15:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 -- 172.21.15.86:0/2060218307 wait complete. 2024-03-22T04:15:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 Processor -- start 2024-03-22T04:15:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.847+0000 7f5586800640 1 -- start start 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f558006ef40 0x7f558009dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55800ad330 0x7f5580158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f558009aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55801690e0 con 0x7f55800ad710 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5580168f60 con 0x7f55800ad330 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5580169260 con 0x7f558006ef40 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f558009aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f558009aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52488/0 (socket says 172.21.15.86:52488) 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 -- 172.21.15.86:0/2203160867 learned_addr learned my addr 172.21.15.86:0/2203160867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557f400640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f558006ef40 0x7f558009dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557ea00640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55800ad330 0x7f5580158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 -- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f558006ef40 msgr2=0x7f558009dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f558006ef40 0x7f558009dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 -- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55800ad330 msgr2=0x7f5580158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55800ad330 0x7f5580158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 -- 172.21.15.86:0/2203160867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558009e240 con 0x7f55800ad710 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557ea00640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55800ad330 0x7f5580158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557f400640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f558006ef40 0x7f558009dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f557fe00640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f558009aad0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f55780195d0 tx=0x7f5578019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5577400640 1 -- 172.21.15.86:0/2203160867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780b8670 con 0x7f55800ad710 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5577400640 1 -- 172.21.15.86:0/2203160867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55780be070 con 0x7f55800ad710 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5580164540 con 0x7f55800ad710 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5577400640 1 -- 172.21.15.86:0/2203160867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780b8a90 con 0x7f55800ad710 2024-03-22T04:15:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558009d7e0 con 0x7f55800ad710 2024-03-22T04:15:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5580068650 con 0x7f55800ad710 2024-03-22T04:15:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.851+0000 7f5577400640 1 -- 172.21.15.86:0/2203160867 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f55780b8c30 con 0x7f55800ad710 2024-03-22T04:15:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.855+0000 7f5577400640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5560067b20 0x7f5560069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.855+0000 7f5577400640 1 -- 172.21.15.86:0/2203160867 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5335+0+0 (secure 0 0 0) 0x7f5578120fe0 con 0x7f55800ad710 2024-03-22T04:15:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.855+0000 7f557f400640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5560067b20 0x7f5560069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.855+0000 7f557f400640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5560067b20 0x7f5560069fe0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f558015f010 tx=0x7f5570008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:02.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:02.859+0000 7f5577400640 1 -- 172.21.15.86:0/2203160867 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f55780165e0 con 0x7f55800ad710 2024-03-22T04:15:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:03.067+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f5580164a60 con 0x7f55800ad710 2024-03-22T04:15:04.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5577400640 1 -- 172.21.15.86:0/2203160867 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v249) ==== 169+0+0 (secure 0 0 0) 0x7f55780f1c70 con 0x7f55800ad710 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5560067b20 msgr2=0x7f5560069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5560067b20 0x7f5560069fe0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f558015f010 tx=0x7f5570008040 comp rx=0 tx=0).stop 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 msgr2=0x7f558009aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f558009aad0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f55780195d0 tx=0x7f5578019760 comp rx=0 tx=0).stop 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 shutdown_connections 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5560067b20 0x7f5560069fe0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55800ad330 0x7f5580158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55800ad710 0x7f558009aad0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 --2- 172.21.15.86:0/2203160867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f558006ef40 0x7f558009dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 >> 172.21.15.86:0/2203160867 conn(0x7f55800b1df0 msgr2=0x7f5580071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:04.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.015+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 shutdown_connections 2024-03-22T04:15:04.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.019+0000 7f5586800640 1 -- 172.21.15.86:0/2203160867 wait complete. 2024-03-22T04:15:04.045 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-22T04:15:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.179+0000 7f1bd0800640 1 Processor -- start 2024-03-22T04:15:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.179+0000 7f1bd0800640 1 -- start start 2024-03-22T04:15:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bd0800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b2360 0x7f1bc80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bd0800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1bc815d0b0 0x7f1bc80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bd0800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc815d710 con 0x7f1bc80b3430 2024-03-22T04:15:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bd0800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc80b3890 con 0x7f1bc815d0b0 2024-03-22T04:15:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bd0800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc80b2740 con 0x7f1bc80b2360 2024-03-22T04:15:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcd400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1bc80b3430 0x7f1bc815c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37934/0 (socket says 172.21.15.86:37934) 2024-03-22T04:15:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcd400640 1 -- 172.21.15.86:0/293126416 learned_addr learned my addr 172.21.15.86:0/293126416 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:04.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bce800640 1 --2- 172.21.15.86:0/293126416 >> v2:172.21.15.196:3300/0 conn(0x7f1bc815d0b0 0x7f1bc80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcde00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b2360 0x7f1bc80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467701643 0 0) 0x7f1bc815d710 con 0x7f1bc80b3430 2024-03-22T04:15:04.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bb0003530 con 0x7f1bc80b3430 2024-03-22T04:15:04.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2395985110 0 0) 0x7f1bb0003530 con 0x7f1bc80b3430 2024-03-22T04:15:04.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b2360 msgr2=0x7f1bc80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:04.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b2360 0x7f1bc80b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 >> v2:172.21.15.196:3300/0 conn(0x7f1bc815d0b0 msgr2=0x7f1bc80b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 --2- 172.21.15.86:0/293126416 >> v2:172.21.15.196:3300/0 conn(0x7f1bc815d0b0 0x7f1bc80b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc815fe40 con 0x7f1bc80b3430 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f1bb8001680 con 0x7f1bc80b3430 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 >> v1:172.21.15.86:6789/0 conn(0x7f1bc80b3430 legacy=0x7f1bc815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb00039c0 0x7f1bb0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b2360 0x7f1bb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb0004d50 0x7f1bb00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc80b2740 con 0x7f1bb00039c0 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb000b9d0 con 0x7f1bb0004d50 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb000bb50 con 0x7f1bc80b2360 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcde00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb00039c0 0x7f1bb0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcd400640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b2360 0x7f1bb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bce800640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb0004d50 0x7f1bb00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcde00640 1 -- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b2360 msgr2=0x7f1bb0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:04.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcde00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc80b2360 0x7f1bb0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcde00640 1 -- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb0004d50 msgr2=0x7f1bb00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:04.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcde00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb0004d50 0x7f1bb00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.183+0000 7f1bcde00640 1 -- 172.21.15.86:0/293126416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb0003530 con 0x7f1bb00039c0 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bcde00640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb00039c0 0x7f1bb0003de0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009ae0 tx=0x7f1bbc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25669ea6d2d9b94a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0096b0 con 0x7f1bb00039c0 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc0a6450 con 0x7f1bb00039c0 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bcca00640 1 -- 172.21.15.86:0/293126416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0a6840 con 0x7f1bb00039c0 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bd0800640 1 -- 172.21.15.86:0/293126416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb00039c0 msgr2=0x7f1bb0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bd0800640 1 --2- 172.21.15.86:0/293126416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb00039c0 0x7f1bb0003de0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009ae0 tx=0x7f1bbc01ca70 comp rx=0 tx=0).stop 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bce800640 1 -- 172.21.15.86:0/293126416 reap_dead start 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bd0800640 1 -- 172.21.15.86:0/293126416 shutdown_connections 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bd0800640 1 -- 172.21.15.86:0/293126416 >> 172.21.15.86:0/293126416 conn(0x7f1bc80cc940 msgr2=0x7f1bc80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bd0800640 1 -- 172.21.15.86:0/293126416 shutdown_connections 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.187+0000 7f1bd0800640 1 -- 172.21.15.86:0/293126416 wait complete. 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 Processor -- start 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 -- start start 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb00039c0 0x7f1bc8150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc80715e0 0x7f1bc8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc815b320 0x7f1bc815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc8160c00 con 0x7f1bc80715e0 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc80b5ad0 con 0x7f1bb00039c0 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc8150fe0 con 0x7f1bc815b320 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc80715e0 0x7f1bc8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc80715e0 0x7f1bc8071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52504/0 (socket says 172.21.15.86:52504) 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 -- 172.21.15.86:0/392315532 learned_addr learned my addr 172.21.15.86:0/392315532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcde00640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb00039c0 0x7f1bc8150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bce800640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc815b320 0x7f1bc815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 -- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc815b320 msgr2=0x7f1bc815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc815b320 0x7f1bc815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 -- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb00039c0 msgr2=0x7f1bc8150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb00039c0 0x7f1bc8150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 -- 172.21.15.86:0/392315532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc80726b0 con 0x7f1bc80715e0 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcde00640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb00039c0 0x7f1bc8150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bcd400640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc80715e0 0x7f1bc8071d40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800e7b0 tx=0x7f1bb800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bc2000640 1 -- 172.21.15.86:0/392315532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb800c660 con 0x7f1bc80715e0 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bc2000640 1 -- 172.21.15.86:0/392315532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb8018070 con 0x7f1bc80715e0 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bc2000640 1 -- 172.21.15.86:0/392315532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb800ca40 con 0x7f1bc80715e0 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc8072990 con 0x7f1bc80715e0 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc8072f50 con 0x7f1bc80715e0 2024-03-22T04:15:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.191+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc8074470 con 0x7f1bc80715e0 2024-03-22T04:15:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.195+0000 7f1bc2000640 1 -- 172.21.15.86:0/392315532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1bb800cbe0 con 0x7f1bc80715e0 2024-03-22T04:15:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.195+0000 7f1bc2000640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bac067b20 0x7f1bac069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.195+0000 7f1bc2000640 1 -- 172.21.15.86:0/392315532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5335+0+0 (secure 0 0 0) 0x7f1bb8085c00 con 0x7f1bc80715e0 2024-03-22T04:15:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.195+0000 7f1bcde00640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bac067b20 0x7f1bac069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.195+0000 7f1bcde00640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bac067b20 0x7f1bac069fe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0ad000 tx=0x7f1bbc0b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.199+0000 7f1bc2000640 1 -- 172.21.15.86:0/392315532 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1bb8056770 con 0x7f1bc80715e0 2024-03-22T04:15:04.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.355+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1bc8160d80 con 0x7f1bc80715e0 2024-03-22T04:15:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.355+0000 7f1bc2000640 1 -- 172.21.15.86:0/392315532 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f1bb8056070 con 0x7f1bc80715e0 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bac067b20 msgr2=0x7f1bac069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bac067b20 0x7f1bac069fe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0ad000 tx=0x7f1bbc0b2040 comp rx=0 tx=0).stop 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc80715e0 msgr2=0x7f1bc8071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc80715e0 0x7f1bc8071d40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800e7b0 tx=0x7f1bb800ec70 comp rx=0 tx=0).stop 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 shutdown_connections 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bac067b20 0x7f1bac069fe0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bc815b320 0x7f1bc815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bc80715e0 0x7f1bc8071d40 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 --2- 172.21.15.86:0/392315532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bb00039c0 0x7f1bc8150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 >> 172.21.15.86:0/392315532 conn(0x7f1bc80cc940 msgr2=0x7f1bc80cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 shutdown_connections 2024-03-22T04:15:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.359+0000 7f1bd0800640 1 -- 172.21.15.86:0/392315532 wait complete. 2024-03-22T04:15:04.384 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:15:04.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:15:04.484 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:15:04.569 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:15:04.655 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:15:04.656 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:04.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:15:04.668 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:15:04.668 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:04.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:15:04.679 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:15:04.680 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:04.719 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:15:04.720 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:15:04.720 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:04.732 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:15:04.733 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-22T04:15:04.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:15:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.895+0000 7fb909600640 1 Processor -- start 2024-03-22T04:15:04.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.895+0000 7fb909600640 1 -- start start 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.895+0000 7fb909600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9040aff50 0x7fb90406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.895+0000 7fb909600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9041444f0 0x7fb90415eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.895+0000 7fb909600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9040ae5a0 con 0x7fb90406cbb0 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.895+0000 7fb909600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb904144960 con 0x7fb9041444f0 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.895+0000 7fb909600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb90406d010 con 0x7fb9040aff50 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb8fa000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb90406cbb0 0x7fb904143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37936/0 (socket says 172.21.15.86:37936) 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb8fa000640 1 -- 172.21.15.86:0/3035190274 learned_addr learned my addr 172.21.15.86:0/3035190274 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb903400640 1 --2- 172.21.15.86:0/3035190274 >> v2:172.21.15.196:3300/0 conn(0x7fb9041444f0 0x7fb90415eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb902a00640 1 --2- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9040aff50 0x7fb90406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb902000640 1 -- 172.21.15.86:0/3035190274 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930349407 0 0) 0x7fb8ec001120 con 0x7fb90406cbb0 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb902000640 1 -- 172.21.15.86:0/3035190274 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8e8003530 con 0x7fb90406cbb0 2024-03-22T04:15:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb903400640 1 -- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9040aff50 msgr2=0x7fb90406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb903400640 1 --2- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9040aff50 0x7fb90406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb903400640 1 -- 172.21.15.86:0/3035190274 >> v1:172.21.15.86:6789/0 conn(0x7fb90406cbb0 legacy=0x7fb904143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb903400640 1 -- 172.21.15.86:0/3035190274 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb904144b80 con 0x7fb9041444f0 2024-03-22T04:15:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb902a00640 1 --2- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9040aff50 0x7fb90406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.899+0000 7fb903400640 1 --2- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9041444f0 0x7fb90415eb80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009d90 tx=0x7fb8fc01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f6631a55ef26c33 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb902000640 1 -- 172.21.15.86:0/3035190274 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0b2070 con 0x7fb9041444f0 2024-03-22T04:15:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb902000640 1 -- 172.21.15.86:0/3035190274 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8fc0a5890 con 0x7fb9041444f0 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb902000640 1 -- 172.21.15.86:0/3035190274 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0a5c40 con 0x7fb9041444f0 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9041444f0 msgr2=0x7fb90415eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 --2- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9041444f0 0x7fb90415eb80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009d90 tx=0x7fb8fc01c8e0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- 172.21.15.86:0/3035190274 shutdown_connections 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 --2- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9041444f0 0x7fb90415eb80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 --2- 172.21.15.86:0/3035190274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9040aff50 0x7fb90406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- 172.21.15.86:0/3035190274 >> 172.21.15.86:0/3035190274 conn(0x7fb904071fe0 msgr2=0x7fb9040723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- 172.21.15.86:0/3035190274 shutdown_connections 2024-03-22T04:15:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- 172.21.15.86:0/3035190274 wait complete. 2024-03-22T04:15:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 Processor -- start 2024-03-22T04:15:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- start start 2024-03-22T04:15:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90406cbb0 0x7fb90413d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9040aff50 0x7fb904156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9041444f0 0x7fb904139040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb904163090 con 0x7fb90406cbb0 2024-03-22T04:15:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb904162f10 con 0x7fb9040aff50 2024-03-22T04:15:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb909600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb9040b0330 con 0x7fb9041444f0 2024-03-22T04:15:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb902a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90406cbb0 0x7fb90413d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb902a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90406cbb0 0x7fb90413d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52506/0 (socket says 172.21.15.86:52506) 2024-03-22T04:15:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb902a00640 1 -- 172.21.15.86:0/1379082262 learned_addr learned my addr 172.21.15.86:0/1379082262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb8fa000640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9040aff50 0x7fb904156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.903+0000 7fb903400640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9041444f0 0x7fb904139040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb902a00640 1 -- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9041444f0 msgr2=0x7fb904139040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb902a00640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9041444f0 0x7fb904139040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb902a00640 1 -- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9040aff50 msgr2=0x7fb904156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb902a00640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9040aff50 0x7fb904156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb902a00640 1 -- 172.21.15.86:0/1379082262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9040ce880 con 0x7fb90406cbb0 2024-03-22T04:15:04.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb8fa000640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9040aff50 0x7fb904156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:04.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb903400640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9041444f0 0x7fb904139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:04.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb902a00640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90406cbb0 0x7fb90413d030 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400ec10 tx=0x7fb8f400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:04.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb8fb400640 1 -- 172.21.15.86:0/1379082262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4017070 con 0x7fb90406cbb0 2024-03-22T04:15:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb8fb400640 1 -- 172.21.15.86:0/1379082262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f400cc30 con 0x7fb90406cbb0 2024-03-22T04:15:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb904139580 con 0x7fb90406cbb0 2024-03-22T04:15:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb8fb400640 1 -- 172.21.15.86:0/1379082262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4013470 con 0x7fb90406cbb0 2024-03-22T04:15:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb904139ae0 con 0x7fb90406cbb0 2024-03-22T04:15:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb904068650 con 0x7fb90406cbb0 2024-03-22T04:15:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb8fb400640 1 -- 172.21.15.86:0/1379082262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb8f401a070 con 0x7fb90406cbb0 2024-03-22T04:15:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb8fb400640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8e0067b20 0x7fb8e0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb8fb400640 1 -- 172.21.15.86:0/1379082262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5335+0+0 (secure 0 0 0) 0x7fb8f4085dd0 con 0x7fb90406cbb0 2024-03-22T04:15:04.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.907+0000 7fb8fa000640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8e0067b20 0x7fb8e0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:04.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.911+0000 7fb8fa000640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8e0067b20 0x7fb8e0069fe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0012c0 tx=0x7fb8ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:04.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:04.911+0000 7fb8fb400640 1 -- 172.21.15.86:0/1379082262 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb904139580 con 0x7fb90406cbb0 2024-03-22T04:15:05.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.099+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb90413b240 con 0x7fb90406cbb0 2024-03-22T04:15:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb8fb400640 1 -- 172.21.15.86:0/1379082262 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3868 (secure 0 0 0) 0x7fb9040ce880 con 0x7fb90406cbb0 2024-03-22T04:15:05.108 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:05.108 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/570504939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":570504939},{"type":"v1","addr":"172.21.15.196:6834","nonce":570504939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6207,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/851917768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":851917768},{"type":"v1","addr":"172.21.15.86:6836","nonce":851917768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":75611,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/746435327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":746435327},{"type":"v1","addr":"172.21.15.86:6837","nonce":746435327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":75983,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2796614575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2796614575},{"type":"v1","addr":"172.21.15.196:6835","nonce":2796614575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-03-22T04:15:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 39 2024-03-22T04:15:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8e0067b20 msgr2=0x7fb8e0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8e0067b20 0x7fb8e0069fe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0012c0 tx=0x7fb8ec009040 comp rx=0 tx=0).stop 2024-03-22T04:15:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90406cbb0 msgr2=0x7fb90413d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90406cbb0 0x7fb90413d030 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400ec10 tx=0x7fb8f400c520 comp rx=0 tx=0).stop 2024-03-22T04:15:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 shutdown_connections 2024-03-22T04:15:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8e0067b20 0x7fb8e0069fe0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9040aff50 0x7fb904156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9041444f0 0x7fb904139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 --2- 172.21.15.86:0/1379082262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90406cbb0 0x7fb90413d030 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 >> 172.21.15.86:0/1379082262 conn(0x7fb904071fe0 msgr2=0x7fb9041614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 shutdown_connections 2024-03-22T04:15:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.103+0000 7fb909600640 1 -- 172.21.15.86:0/1379082262 wait complete. 2024-03-22T04:15:05.129 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-22T04:15:05.129 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-22T04:15:05.129 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-22T04:15:05.129 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:15:05.130 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-22T04:15:05.130 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-22T04:15:05.130 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-22T04:15:05.130 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:15:05.130 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-22T04:15:05.131 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-22T04:15:05.131 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-22T04:15:05.131 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:15:05.131 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-22T04:15:05.131 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-22T04:15:05.132 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-22T04:15:05.132 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:15:05.149 INFO:tasks.ceph.mds.c:Stopped 2024-03-22T04:15:05.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-22T04:15:05.150 INFO:tasks.ceph.mds.a:Stopped 2024-03-22T04:15:05.150 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-22T04:15:05.152 INFO:tasks.ceph.mds.b:Stopped 2024-03-22T04:15:05.152 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-22T04:15:05.152 INFO:tasks.ceph.mds.d:Stopped 2024-03-22T04:15:05.153 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-22T04:15:05.203 INFO:tasks.ceph.mds.b:Started 2024-03-22T04:15:05.203 DEBUG:teuthology.parallel:result is None 2024-03-22T04:15:05.204 INFO:tasks.ceph.mds.d:Started 2024-03-22T04:15:05.205 DEBUG:teuthology.parallel:result is None 2024-03-22T04:15:05.210 INFO:tasks.ceph.mds.c:Started 2024-03-22T04:15:05.210 DEBUG:teuthology.parallel:result is None 2024-03-22T04:15:05.216 INFO:tasks.ceph.mds.a:Started 2024-03-22T04:15:05.217 DEBUG:teuthology.parallel:result is None 2024-03-22T04:15:05.217 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-22T04:15:05.297 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-03-22T04:15:05.298 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:15:05.299 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:15:05.305 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:15:05.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a9200640 1 Processor -- start 2024-03-22T04:15:05.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a9200640 1 -- start start 2024-03-22T04:15:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a9200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415fed0 0x7fb5a4160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a9200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a40d11c0 con 0x7fb5a40ce6b0 2024-03-22T04:15:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a9200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a40cece0 con 0x7fb5a40d0de0 2024-03-22T04:15:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a9200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a40d0580 con 0x7fb5a415fed0 2024-03-22T04:15:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb599400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb5a40ce6b0 0x7fb5a40cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37970/0 (socket says 172.21.15.86:37970) 2024-03-22T04:15:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb599400640 1 -- 172.21.15.86:0/2261020544 learned_addr learned my addr 172.21.15.86:0/2261020544 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006071836 0 0) 0x7fb5a40d11c0 con 0x7fb5a40ce6b0 2024-03-22T04:15:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb588003530 con 0x7fb5a40ce6b0 2024-03-22T04:15:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a2800640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415fed0 0x7fb5a4160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1112069586 0 0) 0x7fb588003530 con 0x7fb5a40ce6b0 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415fed0 msgr2=0x7fb5a4160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415fed0 0x7fb5a4160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 >> v1:172.21.15.196:6789/0 conn(0x7fb5a40d0de0 legacy=0x7fb5a40d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a40cefc0 con 0x7fb5a40ce6b0 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fb58c001680 con 0x7fb5a40ce6b0 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 >> v1:172.21.15.86:6789/0 conn(0x7fb5a40ce6b0 legacy=0x7fb5a40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb588003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415fed0 0x7fb588004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588004d50 0x7fb5880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a40d0580 con 0x7fb5880039f0 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb58800ba80 con 0x7fb588004d50 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb58800bc80 con 0x7fb5a415fed0 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1e00640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb588003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb599400640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415fed0 0x7fb588004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a2800640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588004d50 0x7fb5880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1e00640 1 -- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415fed0 msgr2=0x7fb588004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1e00640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415fed0 0x7fb588004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1e00640 1 -- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588004d50 msgr2=0x7fb5880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1e00640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588004d50 0x7fb5880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a1e00640 1 -- 172.21.15.86:0/2261020544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb588003530 con 0x7fb5880039f0 2024-03-22T04:15:05.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.367+0000 7fb5a2800640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588004d50 0x7fb5880051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a1e00640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb588003e10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb594001750 tx=0x7fb59401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c3dc67a66807e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5940013a0 con 0x7fb5880039f0 2024-03-22T04:15:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5940052f0 con 0x7fb5880039f0 2024-03-22T04:15:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a1400640 1 -- 172.21.15.86:0/2261020544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594003840 con 0x7fb5880039f0 2024-03-22T04:15:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a9200640 1 -- 172.21.15.86:0/2261020544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 msgr2=0x7fb588003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a9200640 1 --2- 172.21.15.86:0/2261020544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb588003e10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb594001750 tx=0x7fb59401c3e0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a2800640 1 -- 172.21.15.86:0/2261020544 reap_dead start 2024-03-22T04:15:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a9200640 1 -- 172.21.15.86:0/2261020544 shutdown_connections 2024-03-22T04:15:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a9200640 1 -- 172.21.15.86:0/2261020544 >> 172.21.15.86:0/2261020544 conn(0x7fb5a406f910 msgr2=0x7fb5a406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a9200640 1 -- 172.21.15.86:0/2261020544 shutdown_connections 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.371+0000 7fb5a9200640 1 -- 172.21.15.86:0/2261020544 wait complete. 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 Processor -- start 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 -- start start 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb5a4155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4158de0 0x7fb5a4157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415dc70 0x7fb5a40c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41606f0 con 0x7fb5880039f0 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4071e30 con 0x7fb5a4158de0 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a4157d40 con 0x7fb5a415dc70 2024-03-22T04:15:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb5a4155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb5a4155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52554/0 (socket says 172.21.15.86:52554) 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 -- 172.21.15.86:0/177502585 learned_addr learned my addr 172.21.15.86:0/177502585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a2800640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415dc70 0x7fb5a40c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb599400640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4158de0 0x7fb5a4157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 -- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415dc70 msgr2=0x7fb5a40c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415dc70 0x7fb5a40c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 -- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4158de0 msgr2=0x7fb5a4157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4158de0 0x7fb5a4157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 -- 172.21.15.86:0/177502585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a4158020 con 0x7fb5880039f0 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb599400640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4158de0 0x7fb5a4157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a2800640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415dc70 0x7fb5a40c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a1e00640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb5a4155850 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb594005040 tx=0x7fb588005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb59aa00640 1 -- 172.21.15.86:0/177502585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594007a40 con 0x7fb5880039f0 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb59aa00640 1 -- 172.21.15.86:0/177502585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5940bb070 con 0x7fb5880039f0 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a4158300 con 0x7fb5880039f0 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb59aa00640 1 -- 172.21.15.86:0/177502585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5940013a0 con 0x7fb5880039f0 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.375+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41588c0 con 0x7fb5880039f0 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.379+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a4068650 con 0x7fb5880039f0 2024-03-22T04:15:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.379+0000 7fb59aa00640 1 -- 172.21.15.86:0/177502585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb5940b4040 con 0x7fb5880039f0 2024-03-22T04:15:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.379+0000 7fb59aa00640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb580067b20 0x7fb580069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.379+0000 7fb59aa00640 1 -- 172.21.15.86:0/177502585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5335+0+0 (secure 0 0 0) 0x7fb59411fe00 con 0x7fb5880039f0 2024-03-22T04:15:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.379+0000 7fb599400640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb580067b20 0x7fb580069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.379+0000 7fb599400640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb580067b20 0x7fb580069fe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4156230 tx=0x7fb58c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:05.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.383+0000 7fb59aa00640 1 -- 172.21.15.86:0/177502585 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb5940160f0 con 0x7fb5880039f0 2024-03-22T04:15:05.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:05.551+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb5a4156480 con 0x7fb5880039f0 2024-03-22T04:15:06.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.027+0000 7fb59aa00640 1 -- 172.21.15.86:0/177502585 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) ==== 97+0+0 (secure 0 0 0) 0x7fb5940f08e0 con 0x7fb5880039f0 2024-03-22T04:15:06.032 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb580067b20 msgr2=0x7fb580069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb580067b20 0x7fb580069fe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4156230 tx=0x7fb58c009040 comp rx=0 tx=0).stop 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 msgr2=0x7fb5a4155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb5a4155850 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb594005040 tx=0x7fb588005840 comp rx=0 tx=0).stop 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 shutdown_connections 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb580067b20 0x7fb580069fe0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4158de0 0x7fb5a4157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5a415dc70 0x7fb5a40c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 --2- 172.21.15.86:0/177502585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5880039f0 0x7fb5a4155850 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 >> 172.21.15.86:0/177502585 conn(0x7fb5a406f910 msgr2=0x7fb5a415eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 shutdown_connections 2024-03-22T04:15:06.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.031+0000 7fb5a9200640 1 -- 172.21.15.86:0/177502585 wait complete. 2024-03-22T04:15:06.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-22T04:15:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f3976600640 1 Processor -- start 2024-03-22T04:15:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f3976600640 1 -- start start 2024-03-22T04:15:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f3976600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39700d0580 0x7f3970071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f3976600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39701623d0 con 0x7f397015e630 2024-03-22T04:15:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f3976600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39701620f0 con 0x7f397006e2f0 2024-03-22T04:15:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f3976600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f397015ec10 con 0x7f39700d0580 2024-03-22T04:15:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f397015e630 0x7f39700cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44074/0 (socket says 172.21.15.86:44074) 2024-03-22T04:15:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396f400640 1 -- 172.21.15.86:0/1781729276 learned_addr learned my addr 172.21.15.86:0/1781729276 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064404279 0 0) 0x7f39701623d0 con 0x7f397015e630 2024-03-22T04:15:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396ea00640 1 --2- 172.21.15.86:0/1781729276 >> v2:172.21.15.196:3300/0 conn(0x7f39700d0580 0x7f3970071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3954003530 con 0x7f397015e630 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3194096350 0 0) 0x7f3954003530 con 0x7f397015e630 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 >> v2:172.21.15.196:3300/0 conn(0x7f39700d0580 msgr2=0x7f3970071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 --2- 172.21.15.86:0/1781729276 >> v2:172.21.15.196:3300/0 conn(0x7f39700d0580 0x7f3970071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 >> v1:172.21.15.196:6789/0 conn(0x7f397006e2f0 legacy=0x7f3970071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397015eef0 con 0x7f397015e630 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f3960001680 con 0x7f397015e630 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 >> v1:172.21.15.86:6789/0 conn(0x7f397015e630 legacy=0x7f39700cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954003a10 0x7f3954003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39540051d0 0x7f395400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f397015ec10 con 0x7f3954004810 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f395400c0d0 con 0x7f3954003a10 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f395400c2d0 con 0x7f39540051d0 2024-03-22T04:15:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396ea00640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396f400640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954003a10 0x7f3954003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396fe00640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39540051d0 0x7f395400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396ea00640 1 -- 172.21.15.86:0/1781729276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39540051d0 msgr2=0x7f395400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396ea00640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39540051d0 0x7f395400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396ea00640 1 -- 172.21.15.86:0/1781729276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954003a10 msgr2=0x7f3954003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396ea00640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954003a10 0x7f3954003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396ea00640 1 -- 172.21.15.86:0/1781729276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3954003530 con 0x7f3954004810 2024-03-22T04:15:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.199+0000 7f396fe00640 1 -- 172.21.15.86:0/1781729276 reap_dead start 2024-03-22T04:15:06.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f396ea00640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f395c009ae0 tx=0x7f395c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e66b3ce57c9972 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0b2070 con 0x7f3954004810 2024-03-22T04:15:06.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c0a5550 con 0x7f3954004810 2024-03-22T04:15:06.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f396e000640 1 -- 172.21.15.86:0/1781729276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0096b0 con 0x7f3954004810 2024-03-22T04:15:06.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f3976600640 1 -- 172.21.15.86:0/1781729276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 msgr2=0x7f3954004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f3976600640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f395c009ae0 tx=0x7f395c01ca70 comp rx=0 tx=0).stop 2024-03-22T04:15:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f3976600640 1 -- 172.21.15.86:0/1781729276 shutdown_connections 2024-03-22T04:15:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f3976600640 1 --2- 172.21.15.86:0/1781729276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f3976600640 1 -- 172.21.15.86:0/1781729276 >> 172.21.15.86:0/1781729276 conn(0x7f397006f910 msgr2=0x7f397006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f3976600640 1 -- 172.21.15.86:0/1781729276 shutdown_connections 2024-03-22T04:15:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.203+0000 7f3976600640 1 -- 172.21.15.86:0/1781729276 wait complete. 2024-03-22T04:15:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 Processor -- start 2024-03-22T04:15:06.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 -- start start 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954004810 0x7f3970148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970159f60 0x7f3970149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397015b570 0x7f39700c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39700d1f60 con 0x7f3970159f60 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f397015f4b0 con 0x7f3954004810 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f397015f9d0 con 0x7f397015b570 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970159f60 0x7f3970149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970159f60 0x7f3970149300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43566/0 (socket says 172.21.15.86:43566) 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 -- 172.21.15.86:0/311584078 learned_addr learned my addr 172.21.15.86:0/311584078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396fe00640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397015b570 0x7f39700c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396f400640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954004810 0x7f3970148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 -- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397015b570 msgr2=0x7f39700c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397015b570 0x7f39700c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 -- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954004810 msgr2=0x7f3970148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954004810 0x7f3970148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 -- 172.21.15.86:0/311584078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397014a6f0 con 0x7f3970159f60 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396fe00640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397015b570 0x7f39700c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f396ea00640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970159f60 0x7f3970149300 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f395c0b0040 tx=0x7f395c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3967400640 1 -- 172.21.15.86:0/311584078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0b2040 con 0x7f3970159f60 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3967400640 1 -- 172.21.15.86:0/311584078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c0a5da0 con 0x7f3970159f60 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397015a4d0 con 0x7f3970159f60 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3967400640 1 -- 172.21.15.86:0/311584078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c004260 con 0x7f3970159f60 2024-03-22T04:15:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397015aa90 con 0x7f3970159f60 2024-03-22T04:15:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.207+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3970068650 con 0x7f3970159f60 2024-03-22T04:15:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.211+0000 7f3967400640 1 -- 172.21.15.86:0/311584078 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f395c003620 con 0x7f3970159f60 2024-03-22T04:15:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.211+0000 7f3967400640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3950067b50 0x7f395006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.211+0000 7f3967400640 1 -- 172.21.15.86:0/311584078 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f395c11fb00 con 0x7f3970159f60 2024-03-22T04:15:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.211+0000 7f396f400640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3950067b50 0x7f395006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.211+0000 7f396f400640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3950067b50 0x7f395006a010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3960001060 tx=0x7f3960009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.215+0000 7f3967400640 1 -- 172.21.15.86:0/311584078 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f395c016780 con 0x7f3970159f60 2024-03-22T04:15:06.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.419+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f397015c970 con 0x7f3970159f60 2024-03-22T04:15:06.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.419+0000 7f3967400640 1 -- 172.21.15.86:0/311584078 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f395c11fe20 con 0x7f3970159f60 2024-03-22T04:15:06.426 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:06.426 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:15:06.426 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-03-22T04:15:06.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.426 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-03-22T04:15:06.426 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA==", 2024-03-22T04:15:06.426 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-03-22T04:15:06.427 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw==", 2024-03-22T04:15:06.428 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:15:06.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:15:06.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:15:06.428 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q==", 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.429 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg==", 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.431 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-03-22T04:15:06.431 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA==", 2024-03-22T04:15:06.431 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.431 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:15:06.431 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:15:06.431 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA==", 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:15:06.432 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg==", 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.433 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.434 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-03-22T04:15:06.434 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA==", 2024-03-22T04:15:06.434 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.434 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:15:06.434 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:15:06.435 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.435 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.435 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-03-22T04:15:06.435 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ==", 2024-03-22T04:15:06.435 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.435 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:15:06.436 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:15:06.436 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.436 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-03-22T04:15:06.436 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow==", 2024-03-22T04:15:06.437 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:15:06.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:15:06.437 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.438 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-03-22T04:15:06.438 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg==", 2024-03-22T04:15:06.438 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:15:06.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:15:06.439 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.439 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-03-22T04:15:06.439 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg==", 2024-03-22T04:15:06.439 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.439 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA==", 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg==", 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:15:06.441 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:15:06.442 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:15:06.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:15:06.442 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.442 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-03-22T04:15:06.442 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w==", 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-03-22T04:15:06.443 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w==", 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-03-22T04:15:06.444 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB4BP1lvLoQORAADwVEr78V9aUc1YblO47Gaw==", 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-03-22T04:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g==", 2024-03-22T04:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-03-22T04:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ==", 2024-03-22T04:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T04:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:15:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3950067b50 msgr2=0x7f395006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3950067b50 0x7f395006a010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3960001060 tx=0x7f3960009040 comp rx=0 tx=0).stop 2024-03-22T04:15:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970159f60 msgr2=0x7f3970149300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970159f60 0x7f3970149300 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f395c0b0040 tx=0x7f395c019810 comp rx=0 tx=0).stop 2024-03-22T04:15:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 shutdown_connections 2024-03-22T04:15:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3950067b50 0x7f395006a010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397015b570 0x7f39700c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970159f60 0x7f3970149300 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 --2- 172.21.15.86:0/311584078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3954004810 0x7f3970148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 >> 172.21.15.86:0/311584078 conn(0x7f397006f910 msgr2=0x7f3970162600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 shutdown_connections 2024-03-22T04:15:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.423+0000 7f3976600640 1 -- 172.21.15.86:0/311584078 wait complete. 2024-03-22T04:15:06.452 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:15:06.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.587+0000 7f7554400640 1 Processor -- start 2024-03-22T04:15:06.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.587+0000 7f7554400640 1 -- start start 2024-03-22T04:15:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7554400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f754c162710 0x7f754c169140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7554400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f754c0b2bd0 con 0x7f754c147e10 2024-03-22T04:15:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7554400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f754c0b2610 con 0x7f754c06c270 2024-03-22T04:15:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7554400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f754c0b28f0 con 0x7f754c162710 2024-03-22T04:15:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7551a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f754c147e10 0x7f754c0b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44088/0 (socket says 172.21.15.86:44088) 2024-03-22T04:15:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7551a00640 1 -- 172.21.15.86:0/896438918 learned_addr learned my addr 172.21.15.86:0/896438918 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7552400640 1 --2- 172.21.15.86:0/896438918 >> v2:172.21.15.196:3300/0 conn(0x7f754c162710 0x7f754c169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1476824850 0 0) 0x7f754c0b2bd0 con 0x7f754c147e10 2024-03-22T04:15:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7534003530 con 0x7f754c147e10 2024-03-22T04:15:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3653192058 0 0) 0x7f7534003530 con 0x7f754c147e10 2024-03-22T04:15:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 >> v2:172.21.15.196:3300/0 conn(0x7f754c162710 msgr2=0x7f754c169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 --2- 172.21.15.86:0/896438918 >> v2:172.21.15.196:3300/0 conn(0x7f754c162710 0x7f754c169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 >> v1:172.21.15.196:6789/0 conn(0x7f754c06c270 legacy=0x7f754c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c0b5ce0 con 0x7f754c147e10 2024-03-22T04:15:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f7540001680 con 0x7f754c147e10 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 >> v1:172.21.15.86:6789/0 conn(0x7f754c147e10 legacy=0x7f754c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340039c0 0x7f7534003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75340043c0 0x7f7534004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f753400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0b28f0 con 0x7f7534005090 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f753400bff0 con 0x7f75340039c0 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f753400c1f0 con 0x7f75340043c0 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7552400640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f753400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7551a00640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340039c0 0x7f7534003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7551000640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75340043c0 0x7f7534004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7552400640 1 -- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75340043c0 msgr2=0x7f7534004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7552400640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75340043c0 0x7f7534004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7552400640 1 -- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340039c0 msgr2=0x7f7534003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:06.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7552400640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340039c0 0x7f7534003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7552400640 1 -- 172.21.15.86:0/896438918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7534003530 con 0x7f7534005090 2024-03-22T04:15:06.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.591+0000 7f7551000640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75340043c0 0x7f7534004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:06.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7552400640 1 -- 172.21.15.86:0/896438918 reap_dead start 2024-03-22T04:15:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7552400640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f753400b9d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7544009f90 tx=0x7f754401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab1bdda2b8a1ae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544009c10 con 0x7f7534005090 2024-03-22T04:15:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7544003ba0 con 0x7f7534005090 2024-03-22T04:15:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f754be00640 1 -- 172.21.15.86:0/896438918 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75440a5990 con 0x7f7534005090 2024-03-22T04:15:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7554400640 1 -- 172.21.15.86:0/896438918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 msgr2=0x7f753400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7554400640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f753400b9d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7544009f90 tx=0x7f754401cc80 comp rx=0 tx=0).stop 2024-03-22T04:15:06.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7554400640 1 -- 172.21.15.86:0/896438918 shutdown_connections 2024-03-22T04:15:06.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7554400640 1 --2- 172.21.15.86:0/896438918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f753400b9d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7554400640 1 -- 172.21.15.86:0/896438918 >> 172.21.15.86:0/896438918 conn(0x7f754c0b6620 msgr2=0x7f754c0b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:06.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7554400640 1 -- 172.21.15.86:0/896438918 shutdown_connections 2024-03-22T04:15:06.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.595+0000 7f7554400640 1 -- 172.21.15.86:0/896438918 wait complete. 2024-03-22T04:15:06.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 Processor -- start 2024-03-22T04:15:06.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 -- start start 2024-03-22T04:15:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f754c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f754c15f1c0 0x7f754c0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f754c0b1970 0x7f754c0ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f754c162000 con 0x7f7534005090 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0b2470 con 0x7f754c0b1970 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f754c162180 con 0x7f754c15f1c0 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f754c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f754c15db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43592/0 (socket says 172.21.15.86:43592) 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 -- 172.21.15.86:0/326199397 learned_addr learned my addr 172.21.15.86:0/326199397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 -- 172.21.15.86:0/326199397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f754c15f1c0 msgr2=0x7f754c0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f754c15f1c0 0x7f754c0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 -- 172.21.15.86:0/326199397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f754c0b1970 msgr2=0x7f754c0ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f754c0b1970 0x7f754c0ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 -- 172.21.15.86:0/326199397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c15e120 con 0x7f7534005090 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7551a00640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f754c15db40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f75400124c0 tx=0x7f7540012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7549600640 1 -- 172.21.15.86:0/326199397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754001c070 con 0x7f7534005090 2024-03-22T04:15:06.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7549600640 1 -- 172.21.15.86:0/326199397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7540014750 con 0x7f7534005090 2024-03-22T04:15:06.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c15e400 con 0x7f7534005090 2024-03-22T04:15:06.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7549600640 1 -- 172.21.15.86:0/326199397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540014b40 con 0x7f7534005090 2024-03-22T04:15:06.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c15e9c0 con 0x7f7534005090 2024-03-22T04:15:06.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.599+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c068650 con 0x7f7534005090 2024-03-22T04:15:06.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.603+0000 7f7549600640 1 -- 172.21.15.86:0/326199397 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f754001f070 con 0x7f7534005090 2024-03-22T04:15:06.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.603+0000 7f7549600640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f752c067b20 0x7f752c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.603+0000 7f7549600640 1 -- 172.21.15.86:0/326199397 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f75400886a0 con 0x7f7534005090 2024-03-22T04:15:06.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.603+0000 7f7551000640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f752c067b20 0x7f752c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.603+0000 7f7551000640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f752c067b20 0x7f752c069fe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f754c15c400 tx=0x7f7538009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.607+0000 7f7549600640 1 -- 172.21.15.86:0/326199397 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7540059260 con 0x7f7534005090 2024-03-22T04:15:06.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.763+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f754c0b1d90 con 0x7f7534005090 2024-03-22T04:15:06.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.763+0000 7f7549600640 1 -- 172.21.15.86:0/326199397 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+13165 (secure 0 0 0) 0x7f7540058b60 con 0x7f7534005090 2024-03-22T04:15:06.770 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:06.770 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:06.039177+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:15:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f752c067b20 msgr2=0x7f752c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f752c067b20 0x7f752c069fe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f754c15c400 tx=0x7f7538009040 comp rx=0 tx=0).stop 2024-03-22T04:15:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 msgr2=0x7f754c15db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f754c15db40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f75400124c0 tx=0x7f7540012980 comp rx=0 tx=0).stop 2024-03-22T04:15:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 shutdown_connections 2024-03-22T04:15:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f752c067b20 0x7f752c069fe0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f754c0b1970 0x7f754c0ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f754c15f1c0 0x7f754c0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 --2- 172.21.15.86:0/326199397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7534005090 0x7f754c15db40 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 >> 172.21.15.86:0/326199397 conn(0x7f754c0b6620 msgr2=0x7f754c1455b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:06.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 shutdown_connections 2024-03-22T04:15:06.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.767+0000 7f7554400640 1 -- 172.21.15.86:0/326199397 wait complete. 2024-03-22T04:15:06.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:15:06.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:15:06.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.927+0000 7f0e21400640 1 Processor -- start 2024-03-22T04:15:06.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.927+0000 7f0e21400640 1 -- start start 2024-03-22T04:15:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e21400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 0x7f0e1c0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e21400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e1c071d20 0x7f0e1c072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e21400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c072680 con 0x7f0e1c071860 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e21400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0cfbb0 con 0x7f0e1c071d20 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e21400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c0cf1e0 con 0x7f0e1c15a830 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e11e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0e1c071860 0x7f0e1c15ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44102/0 (socket says 172.21.15.86:44102) 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e11e00640 1 -- 172.21.15.86:0/3463591014 learned_addr learned my addr 172.21.15.86:0/3463591014 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e1b200640 1 --2- 172.21.15.86:0/3463591014 >> v2:172.21.15.86:3300/0 conn(0x7f0e1c071d20 0x7f0e1c072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e19e00640 1 -- 172.21.15.86:0/3463591014 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612683210 0 0) 0x7f0e1c072680 con 0x7f0e1c071860 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e1a800640 1 --2- 172.21.15.86:0/3463591014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 0x7f0e1c0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e19e00640 1 -- 172.21.15.86:0/3463591014 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e00003530 con 0x7f0e1c071860 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e1b200640 1 -- 172.21.15.86:0/3463591014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 msgr2=0x7f0e1c0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e1b200640 1 --2- 172.21.15.86:0/3463591014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 0x7f0e1c0cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e1b200640 1 -- 172.21.15.86:0/3463591014 >> v1:172.21.15.86:6789/0 conn(0x7f0e1c071860 legacy=0x7f0e1c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e1b200640 1 -- 172.21.15.86:0/3463591014 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0af900 con 0x7f0e1c071d20 2024-03-22T04:15:06.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.931+0000 7f0e1b200640 1 --2- 172.21.15.86:0/3463591014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c072140 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400aea0 tx=0x7f0e1401da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c098ce05e047a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e19e00640 1 -- 172.21.15.86:0/3463591014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140a67a0 con 0x7f0e1c071d20 2024-03-22T04:15:06.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e19e00640 1 -- 172.21.15.86:0/3463591014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e140a6940 con 0x7f0e1c071d20 2024-03-22T04:15:06.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e19e00640 1 -- 172.21.15.86:0/3463591014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140a6d10 con 0x7f0e1c071d20 2024-03-22T04:15:06.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- 172.21.15.86:0/3463591014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 msgr2=0x7f0e1c072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 --2- 172.21.15.86:0/3463591014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c072140 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400aea0 tx=0x7f0e1401da70 comp rx=0 tx=0).stop 2024-03-22T04:15:06.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- 172.21.15.86:0/3463591014 shutdown_connections 2024-03-22T04:15:06.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 --2- 172.21.15.86:0/3463591014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c072140 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 --2- 172.21.15.86:0/3463591014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 0x7f0e1c0cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- 172.21.15.86:0/3463591014 >> 172.21.15.86:0/3463591014 conn(0x7f0e1c0cc940 msgr2=0x7f0e1c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:06.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- 172.21.15.86:0/3463591014 shutdown_connections 2024-03-22T04:15:06.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- 172.21.15.86:0/3463591014 wait complete. 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 Processor -- start 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- start start 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c071860 0x7f0e1c145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 0x7f0e1c153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c06d6d0 con 0x7f0e1c071d20 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0cee70 con 0x7f0e1c071860 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e21400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c06d850 con 0x7f0e1c15a830 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e11e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e11e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43616/0 (socket says 172.21.15.86:43616) 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e11e00640 1 -- 172.21.15.86:0/1842091090 learned_addr learned my addr 172.21.15.86:0/1842091090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e1b200640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 0x7f0e1c153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e1a800640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c071860 0x7f0e1c145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e11e00640 1 -- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 msgr2=0x7f0e1c153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e11e00640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 0x7f0e1c153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e11e00640 1 -- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c071860 msgr2=0x7f0e1c145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e11e00640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c071860 0x7f0e1c145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e11e00640 1 -- 172.21.15.86:0/1842091090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c153190 con 0x7f0e1c071d20 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.935+0000 7f0e1a800640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c071860 0x7f0e1c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e11e00640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c153a40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c450 tx=0x7f0e0400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e13400640 1 -- 172.21.15.86:0/1842091090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04017070 con 0x7f0e1c071d20 2024-03-22T04:15:06.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e13400640 1 -- 172.21.15.86:0/1842091090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e04005cf0 con 0x7f0e1c071d20 2024-03-22T04:15:06.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c152eb0 con 0x7f0e1c071d20 2024-03-22T04:15:06.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e13400640 1 -- 172.21.15.86:0/1842091090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04007290 con 0x7f0e1c071d20 2024-03-22T04:15:06.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c1549a0 con 0x7f0e1c071d20 2024-03-22T04:15:06.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c074470 con 0x7f0e1c071d20 2024-03-22T04:15:06.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e13400640 1 -- 172.21.15.86:0/1842091090 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0e0401a070 con 0x7f0e1c071d20 2024-03-22T04:15:06.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e13400640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0df8067b20 0x7f0df8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:06.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e13400640 1 -- 172.21.15.86:0/1842091090 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f0e04085070 con 0x7f0e1c071d20 2024-03-22T04:15:06.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.939+0000 7f0e1a800640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0df8067b20 0x7f0df8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:06.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.943+0000 7f0e1a800640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0df8067b20 0x7f0df8069fe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c069cf0 tx=0x7f0e0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:06.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:06.943+0000 7f0e13400640 1 -- 172.21.15.86:0/1842091090 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0e04055cd0 con 0x7f0e1c071d20 2024-03-22T04:15:07.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.103+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e1c0ce0e0 con 0x7f0e1c071d20 2024-03-22T04:15:07.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.103+0000 7f0e13400640 1 -- 172.21.15.86:0/1842091090 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f0e040555d0 con 0x7f0e1c071d20 2024-03-22T04:15:07.109 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:15:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0df8067b20 msgr2=0x7f0df8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0df8067b20 0x7f0df8069fe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c069cf0 tx=0x7f0e0c008040 comp rx=0 tx=0).stop 2024-03-22T04:15:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 msgr2=0x7f0e1c153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c153a40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c450 tx=0x7f0e0400c910 comp rx=0 tx=0).stop 2024-03-22T04:15:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 shutdown_connections 2024-03-22T04:15:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0df8067b20 0x7f0df8069fe0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1c15a830 0x7f0e1c153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c071d20 0x7f0e1c153a40 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 --2- 172.21.15.86:0/1842091090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c071860 0x7f0e1c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 >> 172.21.15.86:0/1842091090 conn(0x7f0e1c0cc940 msgr2=0x7f0e1c0ae080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 shutdown_connections 2024-03-22T04:15:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.107+0000 7f0e21400640 1 -- 172.21.15.86:0/1842091090 wait complete. 2024-03-22T04:15:07.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T04:15:07.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.259+0000 7faabee00640 1 Processor -- start 2024-03-22T04:15:07.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabee00640 1 -- start start 2024-03-22T04:15:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab815ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faab815f440 0x7faab80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab815b680 con 0x7faab813e060 2024-03-22T04:15:07.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faab815c6b0 con 0x7faab815f440 2024-03-22T04:15:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faab815b8f0 con 0x7faab806ddd0 2024-03-22T04:15:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabda00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faab813e060 0x7faab806d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44118/0 (socket says 172.21.15.86:44118) 2024-03-22T04:15:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabda00640 1 -- 172.21.15.86:0/3977174685 learned_addr learned my addr 172.21.15.86:0/3977174685 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:07.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabe400640 1 --2- 172.21.15.86:0/3977174685 >> v2:172.21.15.86:3300/0 conn(0x7faab815f440 0x7faab80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabe400640 1 -- 172.21.15.86:0/3977174685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 msgr2=0x7faab815ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabd000640 1 --2- 172.21.15.86:0/3977174685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab815ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabe400640 1 --2- 172.21.15.86:0/3977174685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab815ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabe400640 1 -- 172.21.15.86:0/3977174685 >> v1:172.21.15.86:6789/0 conn(0x7faab813e060 legacy=0x7faab806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faab7e00640 1 -- 172.21.15.86:0/3977174685 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098409985 0 0) 0x7faab815b680 con 0x7faab813e060 2024-03-22T04:15:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabe400640 1 -- 172.21.15.86:0/3977174685 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab815bbd0 con 0x7faab815f440 2024-03-22T04:15:07.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.263+0000 7faabd000640 1 --2- 172.21.15.86:0/3977174685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab815ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:07.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabe400640 1 --2- 172.21.15.86:0/3977174685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab80ceaa0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faab0009d90 tx=0x7faab001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671e7adee7b1537b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:07.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faab7e00640 1 -- 172.21.15.86:0/3977174685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00b2070 con 0x7faab815f440 2024-03-22T04:15:07.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faab7e00640 1 -- 172.21.15.86:0/3977174685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab00a5920 con 0x7faab815f440 2024-03-22T04:15:07.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faab7e00640 1 -- 172.21.15.86:0/3977174685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00a5cf0 con 0x7faab815f440 2024-03-22T04:15:07.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 -- 172.21.15.86:0/3977174685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 msgr2=0x7faab80ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 --2- 172.21.15.86:0/3977174685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab80ceaa0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faab0009d90 tx=0x7faab001c970 comp rx=0 tx=0).stop 2024-03-22T04:15:07.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 -- 172.21.15.86:0/3977174685 shutdown_connections 2024-03-22T04:15:07.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 --2- 172.21.15.86:0/3977174685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab80ceaa0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 --2- 172.21.15.86:0/3977174685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab815ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 -- 172.21.15.86:0/3977174685 >> 172.21.15.86:0/3977174685 conn(0x7faab806b660 msgr2=0x7faab806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:07.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 -- 172.21.15.86:0/3977174685 shutdown_connections 2024-03-22T04:15:07.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 -- 172.21.15.86:0/3977174685 wait complete. 2024-03-22T04:15:07.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 Processor -- start 2024-03-22T04:15:07.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.267+0000 7faabee00640 1 -- start start 2024-03-22T04:15:07.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab8137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab813e060 0x7faab80afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab8071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faab815fa30 con 0x7faab815f440 2024-03-22T04:15:07.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faab815f8b0 con 0x7faab813e060 2024-03-22T04:15:07.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faab80cf7f0 con 0x7faab806ddd0 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab8071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab8071fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43640/0 (socket says 172.21.15.86:43640) 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 -- 172.21.15.86:0/2663979691 learned_addr learned my addr 172.21.15.86:0/2663979691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabd000640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab813e060 0x7faab80afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 -- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 msgr2=0x7faab8137890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabda00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab8137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab8137890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 -- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab813e060 msgr2=0x7faab80afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab813e060 0x7faab80afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 -- 172.21.15.86:0/2663979691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab8137dd0 con 0x7faab815f440 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabd000640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab813e060 0x7faab80afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabda00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab8137890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:07.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabe400640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab8071fa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faab00b0040 tx=0x7faab0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:07.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faab5600640 1 -- 172.21.15.86:0/2663979691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00b2040 con 0x7faab815f440 2024-03-22T04:15:07.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faab5600640 1 -- 172.21.15.86:0/2663979691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab00bf070 con 0x7faab815f440 2024-03-22T04:15:07.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab814d100 con 0x7faab815f440 2024-03-22T04:15:07.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faab5600640 1 -- 172.21.15.86:0/2663979691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00ad8c0 con 0x7faab815f440 2024-03-22T04:15:07.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab814d6c0 con 0x7faab815f440 2024-03-22T04:15:07.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab8074470 con 0x7faab815f440 2024-03-22T04:15:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.271+0000 7faab5600640 1 -- 172.21.15.86:0/2663979691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7faab00ada60 con 0x7faab815f440 2024-03-22T04:15:07.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.275+0000 7faab5600640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa98067b20 0x7faa98069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.275+0000 7faab5600640 1 -- 172.21.15.86:0/2663979691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7faab01210a0 con 0x7faab815f440 2024-03-22T04:15:07.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.275+0000 7faabda00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa98067b20 0x7faa98069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.275+0000 7faabda00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa98067b20 0x7faa98069fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faaac000f70 tx=0x7faaac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.279+0000 7faab5600640 1 -- 172.21.15.86:0/2663979691 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7faab0016560 con 0x7faab815f440 2024-03-22T04:15:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faab806e330 con 0x7faab815f440 2024-03-22T04:15:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faab5600640 1 -- 172.21.15.86:0/2663979691 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7faab00f1be0 con 0x7faab815f440 2024-03-22T04:15:07.439 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:07.439 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:15:07.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:15:07.439 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T04:15:07.439 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T04:15:07.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:07.439 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa98067b20 msgr2=0x7faa98069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa98067b20 0x7faa98069fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faaac000f70 tx=0x7faaac009040 comp rx=0 tx=0).stop 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 msgr2=0x7faab8071fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab8071fa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faab00b0040 tx=0x7faab0019710 comp rx=0 tx=0).stop 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 shutdown_connections 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa98067b20 0x7faa98069fe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab813e060 0x7faab80afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab815f440 0x7faab8071fa0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 --2- 172.21.15.86:0/2663979691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab806ddd0 0x7faab8137890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 >> 172.21.15.86:0/2663979691 conn(0x7faab806b660 msgr2=0x7faab8154be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 shutdown_connections 2024-03-22T04:15:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.435+0000 7faabee00640 1 -- 172.21.15.86:0/2663979691 wait complete. 2024-03-22T04:15:07.462 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T04:15:07.462 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T04:15:07.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.595+0000 7f3217a00640 1 Processor -- start 2024-03-22T04:15:07.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.595+0000 7f3217a00640 1 -- start start 2024-03-22T04:15:07.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3217a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f321015c000 0x7f321015f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3217a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32100aff30 0x7f32100b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3217a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3210072720 con 0x7f3210162050 2024-03-22T04:15:07.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3217a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f32100735c0 con 0x7f321015c000 2024-03-22T04:15:07.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3217a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f32100b08c0 con 0x7f32100aff30 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3215a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32100aff30 0x7f32100b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3207e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3210162050 0x7f3210072eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:44736/0 (socket says 172.21.15.86:44736) 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3207e00640 1 -- 172.21.15.86:0/3217202112 learned_addr learned my addr 172.21.15.86:0/3217202112 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3215000640 1 --2- 172.21.15.86:0/3217202112 >> v2:172.21.15.196:3300/0 conn(0x7f321015c000 0x7f321015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f320fe00640 1 -- 172.21.15.86:0/3217202112 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1922898074 0 0) 0x7f3210072720 con 0x7f3210162050 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3215a00640 1 -- 172.21.15.86:0/3217202112 >> v2:172.21.15.196:3300/0 conn(0x7f321015c000 msgr2=0x7f321015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3215a00640 1 --2- 172.21.15.86:0/3217202112 >> v2:172.21.15.196:3300/0 conn(0x7f321015c000 0x7f321015f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3215a00640 1 -- 172.21.15.86:0/3217202112 >> v1:172.21.15.196:6789/0 conn(0x7f3210162050 legacy=0x7f3210072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3215a00640 1 -- 172.21.15.86:0/3217202112 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321015c3e0 con 0x7f32100aff30 2024-03-22T04:15:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.599+0000 7f3215000640 1 --2- 172.21.15.86:0/3217202112 >> v2:172.21.15.196:3300/0 conn(0x7f321015c000 0x7f321015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:07.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3215a00640 1 --2- 172.21.15.86:0/3217202112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32100aff30 0x7f32100b0380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3208009d90 tx=0x7f320801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd7b87723c3fd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f320fe00640 1 -- 172.21.15.86:0/3217202112 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080b2070 con 0x7f32100aff30 2024-03-22T04:15:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f320fe00640 1 -- 172.21.15.86:0/3217202112 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32080a5920 con 0x7f32100aff30 2024-03-22T04:15:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f320fe00640 1 -- 172.21.15.86:0/3217202112 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080a5cf0 con 0x7f32100aff30 2024-03-22T04:15:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 -- 172.21.15.86:0/3217202112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32100aff30 msgr2=0x7f32100b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 --2- 172.21.15.86:0/3217202112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32100aff30 0x7f32100b0380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3208009d90 tx=0x7f320801c970 comp rx=0 tx=0).stop 2024-03-22T04:15:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 -- 172.21.15.86:0/3217202112 shutdown_connections 2024-03-22T04:15:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 --2- 172.21.15.86:0/3217202112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32100aff30 0x7f32100b0380 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 --2- 172.21.15.86:0/3217202112 >> v2:172.21.15.196:3300/0 conn(0x7f321015c000 0x7f321015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 -- 172.21.15.86:0/3217202112 >> 172.21.15.86:0/3217202112 conn(0x7f32100d1120 msgr2=0x7f32100d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 -- 172.21.15.86:0/3217202112 shutdown_connections 2024-03-22T04:15:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 -- 172.21.15.86:0/3217202112 wait complete. 2024-03-22T04:15:07.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 Processor -- start 2024-03-22T04:15:07.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.603+0000 7f3217a00640 1 -- start start 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32100aff30 0x7f321015d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321015c000 0x7f3210161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210162050 0x7f3210152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32101624a0 con 0x7f32100aff30 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3210162e60 con 0x7f3210162050 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3210162620 con 0x7f321015c000 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32100aff30 0x7f321015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32100aff30 0x7f321015d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43646/0 (socket says 172.21.15.86:43646) 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 -- 172.21.15.86:0/150799617 learned_addr learned my addr 172.21.15.86:0/150799617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3207e00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321015c000 0x7f3210161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215a00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210162050 0x7f3210152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 -- 172.21.15.86:0/150799617 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321015c000 msgr2=0x7f3210161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321015c000 0x7f3210161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 -- 172.21.15.86:0/150799617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210162050 msgr2=0x7f3210152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210162050 0x7f3210152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 -- 172.21.15.86:0/150799617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3210161880 con 0x7f32100aff30 2024-03-22T04:15:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3215000640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32100aff30 0x7f321015d610 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f320000ebe0 tx=0x7f320000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:07.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f320d600640 1 -- 172.21.15.86:0/150799617 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200017070 con 0x7f32100aff30 2024-03-22T04:15:07.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f320d600640 1 -- 172.21.15.86:0/150799617 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320000cc30 con 0x7f32100aff30 2024-03-22T04:15:07.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f320d600640 1 -- 172.21.15.86:0/150799617 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200013470 con 0x7f32100aff30 2024-03-22T04:15:07.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3210161b00 con 0x7f32100aff30 2024-03-22T04:15:07.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32101534b0 con 0x7f32100aff30 2024-03-22T04:15:07.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.607+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3210074580 con 0x7f32100aff30 2024-03-22T04:15:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.611+0000 7f320d600640 1 -- 172.21.15.86:0/150799617 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f320001a070 con 0x7f32100aff30 2024-03-22T04:15:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.611+0000 7f320d600640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31ec067b20 0x7f31ec069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.611+0000 7f320d600640 1 -- 172.21.15.86:0/150799617 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f3200085b80 con 0x7f32100aff30 2024-03-22T04:15:07.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.611+0000 7f3207e00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31ec067b20 0x7f31ec069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:07.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.611+0000 7f3207e00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31ec067b20 0x7f31ec069fe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f321015db50 tx=0x7f31f8007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:07.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.615+0000 7f320d600640 1 -- 172.21.15.86:0/150799617 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3210161880 con 0x7f32100aff30 2024-03-22T04:15:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:07.767+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f321006fee0 con 0x7f32100aff30 2024-03-22T04:15:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f320d600640 1 -- 172.21.15.86:0/150799617 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f3200056190 con 0x7f32100aff30 2024-03-22T04:15:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31ec067b20 msgr2=0x7f31ec069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31ec067b20 0x7f31ec069fe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f321015db50 tx=0x7f31f8007df0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32100aff30 msgr2=0x7f321015d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32100aff30 0x7f321015d610 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f320000ebe0 tx=0x7f320000c520 comp rx=0 tx=0).stop 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 shutdown_connections 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31ec067b20 0x7f31ec069fe0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3210162050 0x7f3210152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321015c000 0x7f3210161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 --2- 172.21.15.86:0/150799617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32100aff30 0x7f321015d610 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 >> 172.21.15.86:0/150799617 conn(0x7f32100d1120 msgr2=0x7f32100d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 shutdown_connections 2024-03-22T04:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.039+0000 7f3217a00640 1 -- 172.21.15.86:0/150799617 wait complete. 2024-03-22T04:15:08.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T04:15:08.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f407b400640 1 Processor -- start 2024-03-22T04:15:08.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f407b400640 1 -- start start 2024-03-22T04:15:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f407b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f407406dc80 0x7f40740cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f407b400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f407415e8f0 con 0x7f407406d8a0 2024-03-22T04:15:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f407b400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f407415ebd0 con 0x7f40740cf410 2024-03-22T04:15:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f407b400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f40740cd890 con 0x7f407406dc80 2024-03-22T04:15:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4078a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f407406d8a0 0x7f40740715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44120/0 (socket says 172.21.15.86:44120) 2024-03-22T04:15:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4078a00640 1 -- 172.21.15.86:0/2718951135 learned_addr learned my addr 172.21.15.86:0/2718951135 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:08.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073e00640 1 --2- 172.21.15.86:0/2718951135 >> v2:172.21.15.196:3300/0 conn(0x7f407406dc80 0x7f40740cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:08.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073400640 1 -- 172.21.15.86:0/2718951135 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507046466 0 0) 0x7f407415e8f0 con 0x7f407406d8a0 2024-03-22T04:15:08.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073400640 1 -- 172.21.15.86:0/2718951135 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4058003530 con 0x7f407406d8a0 2024-03-22T04:15:08.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073400640 1 -- 172.21.15.86:0/2718951135 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1047142546 0 0) 0x7f407415ebd0 con 0x7f40740cf410 2024-03-22T04:15:08.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073400640 1 -- 172.21.15.86:0/2718951135 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f407415e8f0 con 0x7f40740cf410 2024-03-22T04:15:08.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073e00640 1 -- 172.21.15.86:0/2718951135 >> v1:172.21.15.196:6789/0 conn(0x7f40740cf410 legacy=0x7f407415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:08.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073400640 1 -- 172.21.15.86:0/2718951135 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3203977890 0 0) 0x7f4058003530 con 0x7f407406d8a0 2024-03-22T04:15:08.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073e00640 1 -- 172.21.15.86:0/2718951135 >> v1:172.21.15.86:6789/0 conn(0x7f407406d8a0 legacy=0x7f40740715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:08.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.207+0000 7f4073e00640 1 -- 172.21.15.86:0/2718951135 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407415eeb0 con 0x7f407406dc80 2024-03-22T04:15:08.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f4073e00640 1 --2- 172.21.15.86:0/2718951135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407406dc80 0x7f40740cc8b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f40600097c0 tx=0x7f406001cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d63904a919f8d7c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:08.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f4073400640 1 -- 172.21.15.86:0/2718951135 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40600b0070 con 0x7f407406dc80 2024-03-22T04:15:08.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f4073400640 1 -- 172.21.15.86:0/2718951135 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40600ab400 con 0x7f407406dc80 2024-03-22T04:15:08.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f4073400640 1 -- 172.21.15.86:0/2718951135 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064001a00 con 0x7f407406dc80 2024-03-22T04:15:08.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f407b400640 1 -- 172.21.15.86:0/2718951135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407406dc80 msgr2=0x7f40740cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:08.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f407b400640 1 --2- 172.21.15.86:0/2718951135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407406dc80 0x7f40740cc8b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f40600097c0 tx=0x7f406001cdf0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f407b400640 1 -- 172.21.15.86:0/2718951135 shutdown_connections 2024-03-22T04:15:08.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f407b400640 1 --2- 172.21.15.86:0/2718951135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407406dc80 0x7f40740cc8b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f407b400640 1 -- 172.21.15.86:0/2718951135 >> 172.21.15.86:0/2718951135 conn(0x7f407406ef50 msgr2=0x7f407406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:08.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f407b400640 1 -- 172.21.15.86:0/2718951135 shutdown_connections 2024-03-22T04:15:08.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.211+0000 7f407b400640 1 -- 172.21.15.86:0/2718951135 wait complete. 2024-03-22T04:15:08.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 Processor -- start 2024-03-22T04:15:08.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 -- start start 2024-03-22T04:15:08.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f407406d8a0 0x7f40741584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:08.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f407406dc80 0x7f407414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:08.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40740cf410 0x7f40741590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:08.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f407415be00 con 0x7f407406d8a0 2024-03-22T04:15:08.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40740cfc50 con 0x7f40740cf410 2024-03-22T04:15:08.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f40741589f0 con 0x7f407406dc80 2024-03-22T04:15:08.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f407406d8a0 0x7f40741584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:08.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f407406d8a0 0x7f40741584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43658/0 (socket says 172.21.15.86:43658) 2024-03-22T04:15:08.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 -- 172.21.15.86:0/273611580 learned_addr learned my addr 172.21.15.86:0/273611580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:08.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 -- 172.21.15.86:0/273611580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f407406dc80 msgr2=0x7f407414c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:08.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f407406dc80 0x7f407414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 -- 172.21.15.86:0/273611580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40740cf410 msgr2=0x7f40741590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:08.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4079400640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40740cf410 0x7f40741590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40740cf410 0x7f40741590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 -- 172.21.15.86:0/273611580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407414ea60 con 0x7f407406d8a0 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4079400640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40740cf410 0x7f40741590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4078a00640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f407406d8a0 0x7f40741584b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f406400cc50 tx=0x7f406400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4070c00640 1 -- 172.21.15.86:0/273611580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064017070 con 0x7f407406d8a0 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4070c00640 1 -- 172.21.15.86:0/273611580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4064012400 con 0x7f407406d8a0 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407414ed40 con 0x7f407406d8a0 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f4070c00640 1 -- 172.21.15.86:0/273611580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406400ce60 con 0x7f407406d8a0 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.215+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407414f300 con 0x7f407406d8a0 2024-03-22T04:15:08.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.219+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4074074470 con 0x7f407406d8a0 2024-03-22T04:15:08.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.219+0000 7f4070c00640 1 -- 172.21.15.86:0/273611580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f406401a070 con 0x7f407406d8a0 2024-03-22T04:15:08.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.219+0000 7f4070c00640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4054067b20 0x7f4054069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:08.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.219+0000 7f4070c00640 1 -- 172.21.15.86:0/273611580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5619+0+0 (secure 0 0 0) 0x7f4064085ed0 con 0x7f407406d8a0 2024-03-22T04:15:08.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.219+0000 7f4073e00640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4054067b20 0x7f4054069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:08.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.219+0000 7f4073e00640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4054067b20 0x7f4054069fe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f40600ad000 tx=0x7f4060008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:08.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.223+0000 7f4070c00640 1 -- 172.21.15.86:0/273611580 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4064056890 con 0x7f407406d8a0 2024-03-22T04:15:08.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:08.379+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4074155450 con 0x7f407406d8a0 2024-03-22T04:15:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.039+0000 7f4070c00640 1 -- 172.21.15.86:0/273611580 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7f4064056190 con 0x7f407406d8a0 2024-03-22T04:15:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4054067b20 msgr2=0x7f4054069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4054067b20 0x7f4054069fe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f40600ad000 tx=0x7f4060008040 comp rx=0 tx=0).stop 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f407406d8a0 msgr2=0x7f40741584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f407406d8a0 0x7f40741584b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f406400cc50 tx=0x7f406400a430 comp rx=0 tx=0).stop 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 shutdown_connections 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4054067b20 0x7f4054069fe0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40740cf410 0x7f40741590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f407406dc80 0x7f407414c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 --2- 172.21.15.86:0/273611580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f407406d8a0 0x7f40741584b0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 >> 172.21.15.86:0/273611580 conn(0x7f407406ef50 msgr2=0x7f40740b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 shutdown_connections 2024-03-22T04:15:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.043+0000 7f407b400640 1 -- 172.21.15.86:0/273611580 wait complete. 2024-03-22T04:15:09.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T04:15:09.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.219+0000 7f385e600640 1 Processor -- start 2024-03-22T04:15:09.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.219+0000 7f385e600640 1 -- start start 2024-03-22T04:15:09.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f385e600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3858156620 0x7f38580acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:09.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f385e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f385806f1a0 0x7f38580d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:09.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f385e600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385815fbe0 con 0x7f38580d0640 2024-03-22T04:15:09.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f385e600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f385815d560 con 0x7f3858156620 2024-03-22T04:15:09.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f385e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3858156a00 con 0x7f385806f1a0 2024-03-22T04:15:09.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f384fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f385806f1a0 0x7f38580d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f384fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f385806f1a0 0x7f38580d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:41190/0 (socket says 172.21.15.86:41190) 2024-03-22T04:15:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f384fe00640 1 -- 172.21.15.86:0/3706414058 learned_addr learned my addr 172.21.15.86:0/3706414058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f385dc00640 1 --1- 172.21.15.86:0/3706414058 >> v1:172.21.15.196:6789/0 conn(0x7f38580d0640 0x7f3858071300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:44752/0 (socket says 172.21.15.86:44752) 2024-03-22T04:15:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f385d200640 1 --2- 172.21.15.86:0/3706414058 >> v2:172.21.15.196:3300/0 conn(0x7f3858156620 0x7f38580acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f384fe00640 1 -- 172.21.15.86:0/3706414058 >> v2:172.21.15.196:3300/0 conn(0x7f3858156620 msgr2=0x7f38580acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f384fe00640 1 --2- 172.21.15.86:0/3706414058 >> v2:172.21.15.196:3300/0 conn(0x7f3858156620 0x7f38580acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f384fe00640 1 -- 172.21.15.86:0/3706414058 >> v1:172.21.15.196:6789/0 conn(0x7f38580d0640 legacy=0x7f3858071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.223+0000 7f384fe00640 1 -- 172.21.15.86:0/3706414058 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385815fe50 con 0x7f385806f1a0 2024-03-22T04:15:09.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f384fe00640 1 --2- 172.21.15.86:0/3706414058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f385806f1a0 0x7f38580d0100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3840009870 tx=0x7f384001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0f646cc6da41c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:09.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f3857e00640 1 -- 172.21.15.86:0/3706414058 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400b0070 con 0x7f385806f1a0 2024-03-22T04:15:09.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f3857e00640 1 -- 172.21.15.86:0/3706414058 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38400ab600 con 0x7f385806f1a0 2024-03-22T04:15:09.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f3857e00640 1 -- 172.21.15.86:0/3706414058 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400ab9d0 con 0x7f385806f1a0 2024-03-22T04:15:09.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 -- 172.21.15.86:0/3706414058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f385806f1a0 msgr2=0x7f38580d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:09.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 --2- 172.21.15.86:0/3706414058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f385806f1a0 0x7f38580d0100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3840009870 tx=0x7f384001c560 comp rx=0 tx=0).stop 2024-03-22T04:15:09.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 -- 172.21.15.86:0/3706414058 shutdown_connections 2024-03-22T04:15:09.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 --2- 172.21.15.86:0/3706414058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f385806f1a0 0x7f38580d0100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 --2- 172.21.15.86:0/3706414058 >> v2:172.21.15.196:3300/0 conn(0x7f3858156620 0x7f38580acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 -- 172.21.15.86:0/3706414058 >> 172.21.15.86:0/3706414058 conn(0x7f385806ba60 msgr2=0x7f385806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:09.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 -- 172.21.15.86:0/3706414058 shutdown_connections 2024-03-22T04:15:09.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 -- 172.21.15.86:0/3706414058 wait complete. 2024-03-22T04:15:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 Processor -- start 2024-03-22T04:15:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.227+0000 7f385e600640 1 -- start start 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385806f1a0 0x7f38581428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38580d0640 0x7f385814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3858156620 0x7f385814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38580ad460 con 0x7f385806f1a0 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38580ad2e0 con 0x7f38580d0640 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f385806d300 con 0x7f3858156620 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385806f1a0 0x7f38581428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385806f1a0 0x7f38581428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43670/0 (socket says 172.21.15.86:43670) 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 -- 172.21.15.86:0/416726329 learned_addr learned my addr 172.21.15.86:0/416726329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f384fe00640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38580d0640 0x7f385814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385dc00640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3858156620 0x7f385814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:09.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 -- 172.21.15.86:0/416726329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3858156620 msgr2=0x7f385814c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:09.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3858156620 0x7f385814c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 -- 172.21.15.86:0/416726329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38580d0640 msgr2=0x7f385814bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:09.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38580d0640 0x7f385814bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:09.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 -- 172.21.15.86:0/416726329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385806e280 con 0x7f385806f1a0 2024-03-22T04:15:09.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385d200640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385806f1a0 0x7f38581428c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f384800e930 tx=0x7f384800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:09.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f3855600640 1 -- 172.21.15.86:0/416726329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848017070 con 0x7f385806f1a0 2024-03-22T04:15:09.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f3855600640 1 -- 172.21.15.86:0/416726329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384800ca00 con 0x7f385806f1a0 2024-03-22T04:15:09.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3858154860 con 0x7f385806f1a0 2024-03-22T04:15:09.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f3855600640 1 -- 172.21.15.86:0/416726329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384800cdb0 con 0x7f385806f1a0 2024-03-22T04:15:09.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3858154e20 con 0x7f385806f1a0 2024-03-22T04:15:09.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.231+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385804f900 con 0x7f385806f1a0 2024-03-22T04:15:09.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.235+0000 7f3855600640 1 -- 172.21.15.86:0/416726329 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f384801a070 con 0x7f385806f1a0 2024-03-22T04:15:09.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.235+0000 7f3855600640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3834067b20 0x7f3834069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:09.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.235+0000 7f3855600640 1 -- 172.21.15.86:0/416726329 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6001+0+0 (secure 0 0 0) 0x7f38480867c0 con 0x7f385806f1a0 2024-03-22T04:15:09.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.235+0000 7f384fe00640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3834067b20 0x7f3834069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:09.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.235+0000 7f384fe00640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3834067b20 0x7f3834069fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f38400ad000 tx=0x7f3840008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:09.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.239+0000 7f3855600640 1 -- 172.21.15.86:0/416726329 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3848056b80 con 0x7f385806f1a0 2024-03-22T04:15:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:09.427+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f385814ca10 con 0x7f385806f1a0 2024-03-22T04:15:10.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.067+0000 7f3855600640 1 -- 172.21.15.86:0/416726329 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v42) ==== 174+0+0 (secure 0 0 0) 0x7f384805ac20 con 0x7f385806f1a0 2024-03-22T04:15:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 11 and data pool 12 2024-03-22T04:15:10.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3834067b20 msgr2=0x7f3834069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3834067b20 0x7f3834069fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f38400ad000 tx=0x7f3840008040 comp rx=0 tx=0).stop 2024-03-22T04:15:10.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385806f1a0 msgr2=0x7f38581428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385806f1a0 0x7f38581428c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f384800e930 tx=0x7f384800edf0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 shutdown_connections 2024-03-22T04:15:10.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3834067b20 0x7f3834069fe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38580d0640 0x7f385814bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3858156620 0x7f385814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 --2- 172.21.15.86:0/416726329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385806f1a0 0x7f38581428c0 secure :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f384800e930 tx=0x7f384800edf0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.075+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 >> 172.21.15.86:0/416726329 conn(0x7f385806ba60 msgr2=0x7f385806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:10.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.079+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 shutdown_connections 2024-03-22T04:15:10.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.079+0000 7f385e600640 1 -- 172.21.15.86:0/416726329 wait complete. 2024-03-22T04:15:10.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:15:10.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcee00640 1 Processor -- start 2024-03-22T04:15:10.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcee00640 1 -- start start 2024-03-22T04:15:10.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 0x7f4bc8071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc81623d0 con 0x7f4bc815e630 2024-03-22T04:15:10.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcee00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc81620f0 con 0x7f4bc80d0580 2024-03-22T04:15:10.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc815ec10 con 0x7f4bc806e2f0 2024-03-22T04:15:10.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4bc815e630 0x7f4bc80cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44124/0 (socket says 172.21.15.86:44124) 2024-03-22T04:15:10.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7e00640 1 -- 172.21.15.86:0/2201640612 learned_addr learned my addr 172.21.15.86:0/2201640612 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:10.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcce00640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 0x7f4bc8071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 223088579 0 0) 0x7f4bc81623d0 con 0x7f4bc815e630 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bac003530 con 0x7f4bc815e630 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903990627 0 0) 0x7f4bc81620f0 con 0x7f4bc80d0580 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bc81623d0 con 0x7f4bc80d0580 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3406950237 0 0) 0x7f4bac003530 con 0x7f4bc815e630 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 msgr2=0x7f4bc8071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 0x7f4bc8071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 >> v1:172.21.15.196:6789/0 conn(0x7f4bc80d0580 legacy=0x7f4bc8071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc815eef0 con 0x7f4bc815e630 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f4bb8001680 con 0x7f4bc815e630 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcce00640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 0x7f4bc8071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 >> v1:172.21.15.86:6789/0 conn(0x7f4bc815e630 legacy=0x7f4bc80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bac003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac004810 0x7f4bac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 0x7f4bac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc815ec10 con 0x7f4bac003b80 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac00ba00 con 0x7f4bac004810 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac00bc00 con 0x7f4bc806e2f0 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bc7e00640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bac003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bbf400640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac004810 0x7f4bac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.251+0000 7f4bcce00640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 0x7f4bac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7e00640 1 -- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 msgr2=0x7f4bac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7e00640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc806e2f0 0x7f4bac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7e00640 1 -- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac004810 msgr2=0x7f4bac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7e00640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac004810 0x7f4bac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7e00640 1 -- 172.21.15.86:0/2201640612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac003530 con 0x7f4bac003b80 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7e00640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bac003f80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800c190 tx=0x7f4bb801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2e6c6cfae98c4e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8007cf0 con 0x7f4bac003b80 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb80a9020 con 0x7f4bac003b80 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bc7400640 1 -- 172.21.15.86:0/2201640612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8005b00 con 0x7f4bac003b80 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bcee00640 1 -- 172.21.15.86:0/2201640612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 msgr2=0x7f4bac003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bcee00640 1 --2- 172.21.15.86:0/2201640612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bac003f80 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800c190 tx=0x7f4bb801cbe0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bcce00640 1 -- 172.21.15.86:0/2201640612 reap_dead start 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bcee00640 1 -- 172.21.15.86:0/2201640612 shutdown_connections 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bcee00640 1 -- 172.21.15.86:0/2201640612 >> 172.21.15.86:0/2201640612 conn(0x7f4bc806f910 msgr2=0x7f4bc806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bcee00640 1 -- 172.21.15.86:0/2201640612 shutdown_connections 2024-03-22T04:15:10.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.255+0000 7f4bcee00640 1 -- 172.21.15.86:0/2201640612 wait complete. 2024-03-22T04:15:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 Processor -- start 2024-03-22T04:15:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 -- start start 2024-03-22T04:15:10.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bc80cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc80b2040 0x7f4bc814e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc814e580 0x7f4bc8157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc80cfaf0 con 0x7f4bac003b80 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc815f9d0 con 0x7f4bc80b2040 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc80cfc70 con 0x7f4bc814e580 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bc80cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bc80cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43678/0 (socket says 172.21.15.86:43678) 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 -- 172.21.15.86:0/3123081615 learned_addr learned my addr 172.21.15.86:0/3123081615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:10.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bbf400640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc80b2040 0x7f4bc814e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcce00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc814e580 0x7f4bc8157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 -- 172.21.15.86:0/3123081615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc814e580 msgr2=0x7f4bc8157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc814e580 0x7f4bc8157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 -- 172.21.15.86:0/3123081615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc80b2040 msgr2=0x7f4bc814e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc80b2040 0x7f4bc814e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 -- 172.21.15.86:0/3123081615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc80cdf10 con 0x7f4bac003b80 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc7e00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bc80cd6c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4bb801ca10 tx=0x7f4bac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc4c00640 1 -- 172.21.15.86:0/3123081615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8007cf0 con 0x7f4bac003b80 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc4c00640 1 -- 172.21.15.86:0/3123081615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb8007090 con 0x7f4bac003b80 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc80ce1f0 con 0x7f4bac003b80 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bc4c00640 1 -- 172.21.15.86:0/3123081615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8001c20 con 0x7f4bac003b80 2024-03-22T04:15:10.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.259+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc80ce7b0 con 0x7f4bac003b80 2024-03-22T04:15:10.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.263+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc8068650 con 0x7f4bac003b80 2024-03-22T04:15:10.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.263+0000 7f4bc4c00640 1 -- 172.21.15.86:0/3123081615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4bb8007440 con 0x7f4bac003b80 2024-03-22T04:15:10.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.263+0000 7f4bc4c00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ba4067b20 0x7f4ba4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.263+0000 7f4bc4c00640 1 -- 172.21.15.86:0/3123081615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f4bb811f5a0 con 0x7f4bac003b80 2024-03-22T04:15:10.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.263+0000 7f4bbf400640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ba4067b20 0x7f4ba4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.263+0000 7f4bbf400640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ba4067b20 0x7f4ba4069fe0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ccd30 tx=0x7f4bb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.267+0000 7f4bc4c00640 1 -- 172.21.15.86:0/3123081615 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4bb80168f0 con 0x7f4bac003b80 2024-03-22T04:15:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.423+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bc8161af0 con 0x7f4bac003b80 2024-03-22T04:15:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.423+0000 7f4bc4c00640 1 -- 172.21.15.86:0/3123081615 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16898 (secure 0 0 0) 0x7f4bb80efd80 con 0x7f4bac003b80 2024-03-22T04:15:10.429 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:10.429 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":79,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:09.433544+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:15:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ba4067b20 msgr2=0x7f4ba4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ba4067b20 0x7f4ba4069fe0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ccd30 tx=0x7f4bb0009040 comp rx=0 tx=0).stop 2024-03-22T04:15:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 msgr2=0x7f4bc80cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bc80cd6c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4bb801ca10 tx=0x7f4bac005840 comp rx=0 tx=0).stop 2024-03-22T04:15:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 shutdown_connections 2024-03-22T04:15:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ba4067b20 0x7f4ba4069fe0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bc80b2040 0x7f4bc814e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bc814e580 0x7f4bc8157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 --2- 172.21.15.86:0/3123081615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bac003b80 0x7f4bc80cd6c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 >> 172.21.15.86:0/3123081615 conn(0x7f4bc806f910 msgr2=0x7f4bc8158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 shutdown_connections 2024-03-22T04:15:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.427+0000 7f4bcee00640 1 -- 172.21.15.86:0/3123081615 wait complete. 2024-03-22T04:15:10.450 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:15:10.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce11800640 1 Processor -- start 2024-03-22T04:15:10.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce11800640 1 -- start start 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce11800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce11800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0c071920 con 0x7fce0c15b3f0 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce11800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0c071c00 con 0x7fce0c0cc3e0 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce11800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c0716d0 con 0x7fce0c0730d0 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce0aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fce0c15b3f0 0x7fce0c0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44126/0 (socket says 172.21.15.86:44126) 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce0aa00640 1 -- 172.21.15.86:0/4246235015 learned_addr learned my addr 172.21.15.86:0/4246235015 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce0a000640 1 --2- 172.21.15.86:0/4246235015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce09600640 1 -- 172.21.15.86:0/4246235015 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965307625 0 0) 0x7fce0c071920 con 0x7fce0c15b3f0 2024-03-22T04:15:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce09600640 1 -- 172.21.15.86:0/4246235015 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdf0003530 con 0x7fce0c15b3f0 2024-03-22T04:15:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce09600640 1 -- 172.21.15.86:0/4246235015 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 284420157 0 0) 0x7fce0c071c00 con 0x7fce0c0cc3e0 2024-03-22T04:15:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce0a000640 1 -- 172.21.15.86:0/4246235015 >> v1:172.21.15.196:6789/0 conn(0x7fce0c0cc3e0 legacy=0x7fce0c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce0a000640 1 -- 172.21.15.86:0/4246235015 >> v1:172.21.15.86:6789/0 conn(0x7fce0c15b3f0 legacy=0x7fce0c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.583+0000 7fce0a000640 1 -- 172.21.15.86:0/4246235015 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c071ee0 con 0x7fce0c0730d0 2024-03-22T04:15:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce0a000640 1 --2- 172.21.15.86:0/4246235015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c0ade30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80097f0 tx=0x7fcdf801c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af4f848f5dd480ff server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce09600640 1 -- 172.21.15.86:0/4246235015 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80b0070 con 0x7fce0c0730d0 2024-03-22T04:15:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce09600640 1 -- 172.21.15.86:0/4246235015 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf80ab400 con 0x7fce0c0730d0 2024-03-22T04:15:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce09600640 1 -- 172.21.15.86:0/4246235015 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc001680 con 0x7fce0c0730d0 2024-03-22T04:15:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce11800640 1 -- 172.21.15.86:0/4246235015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 msgr2=0x7fce0c0ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce11800640 1 --2- 172.21.15.86:0/4246235015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c0ade30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80097f0 tx=0x7fcdf801c320 comp rx=0 tx=0).stop 2024-03-22T04:15:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce11800640 1 -- 172.21.15.86:0/4246235015 shutdown_connections 2024-03-22T04:15:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce11800640 1 --2- 172.21.15.86:0/4246235015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c0ade30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce11800640 1 -- 172.21.15.86:0/4246235015 >> 172.21.15.86:0/4246235015 conn(0x7fce0c0b1df0 msgr2=0x7fce0c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce11800640 1 -- 172.21.15.86:0/4246235015 shutdown_connections 2024-03-22T04:15:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.587+0000 7fce11800640 1 -- 172.21.15.86:0/4246235015 wait complete. 2024-03-22T04:15:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 Processor -- start 2024-03-22T04:15:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 -- start start 2024-03-22T04:15:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cc3e0 0x7fce0c099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c15b3f0 0x7fce0c1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c0b30f0 con 0x7fce0c15b3f0 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c0b4c20 con 0x7fce0c0cc3e0 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c0b3270 con 0x7fce0c0730d0 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c15b3f0 0x7fce0c1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c15b3f0 0x7fce0c1584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43690/0 (socket says 172.21.15.86:43690) 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0aa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 -- 172.21.15.86:0/624546055 learned_addr learned my addr 172.21.15.86:0/624546055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0a000640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cc3e0 0x7fce0c099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 -- 172.21.15.86:0/624546055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 msgr2=0x7fce0c157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 -- 172.21.15.86:0/624546055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cc3e0 msgr2=0x7fce0c099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cc3e0 0x7fce0c099ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 -- 172.21.15.86:0/624546055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c1589f0 con 0x7fce0c15b3f0 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce0b400640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c15b3f0 0x7fce0c1584b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c780 tx=0x7fce0400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce02a00640 1 -- 172.21.15.86:0/624546055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04003db0 con 0x7fce0c15b3f0 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce02a00640 1 -- 172.21.15.86:0/624546055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce04005070 con 0x7fce0c15b3f0 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce02a00640 1 -- 172.21.15.86:0/624546055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04005460 con 0x7fce0c15b3f0 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c158c70 con 0x7fce0c15b3f0 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c15bb50 con 0x7fce0c15b3f0 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.591+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c068650 con 0x7fce0c15b3f0 2024-03-22T04:15:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.595+0000 7fce02a00640 1 -- 172.21.15.86:0/624546055 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fce04005600 con 0x7fce0c15b3f0 2024-03-22T04:15:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.595+0000 7fce02a00640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.595+0000 7fce02a00640 1 -- 172.21.15.86:0/624546055 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7fce04085e50 con 0x7fce0c15b3f0 2024-03-22T04:15:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.595+0000 7fce0aa00640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.595+0000 7fce0aa00640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000f70 tx=0x7fcdfc0047f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.599+0000 7fce02a00640 1 -- 172.21.15.86:0/624546055 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fce0c1589f0 con 0x7fce0c15b3f0 2024-03-22T04:15:10.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.751+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce0c0b42f0 con 0x7fce0c15b3f0 2024-03-22T04:15:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce02a00640 1 -- 172.21.15.86:0/624546055 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16898 (secure 0 0 0) 0x7fce04056080 con 0x7fce0c15b3f0 2024-03-22T04:15:10.759 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:10.759 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":79,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:09.433544+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:15:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 msgr2=0x7fcdec069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000f70 tx=0x7fcdfc0047f0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c15b3f0 msgr2=0x7fce0c1584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c15b3f0 0x7fce0c1584b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c780 tx=0x7fce0400cc40 comp rx=0 tx=0).stop 2024-03-22T04:15:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 shutdown_connections 2024-03-22T04:15:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcdec067b20 0x7fcdec069fe0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce0c0cc3e0 0x7fce0c099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c15b3f0 0x7fce0c1584b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 --2- 172.21.15.86:0/624546055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce0c0730d0 0x7fce0c157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 >> 172.21.15.86:0/624546055 conn(0x7fce0c0b1df0 msgr2=0x7fce0c0b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 shutdown_connections 2024-03-22T04:15:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.755+0000 7fce11800640 1 -- 172.21.15.86:0/624546055 wait complete. 2024-03-22T04:15:10.780 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T04:15:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.915+0000 7f7a01200640 1 Processor -- start 2024-03-22T04:15:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.915+0000 7f7a01200640 1 -- start start 2024-03-22T04:15:10.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.915+0000 7f7a01200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f79fc0b2360 0x7f79fc0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.915+0000 7f7a01200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc0b3430 0x7f79fc15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.915+0000 7f7a01200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc15d710 con 0x7f79fc15cee0 2024-03-22T04:15:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.915+0000 7f7a01200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f79fc0b2740 con 0x7f79fc0b2360 2024-03-22T04:15:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.915+0000 7f7a01200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc15fe40 con 0x7f79fc0b3430 2024-03-22T04:15:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79fa800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f79fc15cee0 0x7f79fc06d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44136/0 (socket says 172.21.15.86:44136) 2024-03-22T04:15:10.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79fa800640 1 -- 172.21.15.86:0/401246394 learned_addr learned my addr 172.21.15.86:0/401246394 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f9400640 1 --2- 172.21.15.86:0/401246394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc0b3430 0x7f79fc15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f9e00640 1 --2- 172.21.15.86:0/401246394 >> v2:172.21.15.196:3300/0 conn(0x7f79fc0b2360 0x7f79fc0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f8a00640 1 -- 172.21.15.86:0/401246394 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 997635447 0 0) 0x7f79fc15d710 con 0x7f79fc15cee0 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f8a00640 1 -- 172.21.15.86:0/401246394 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79e0003530 con 0x7f79fc15cee0 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f9400640 1 -- 172.21.15.86:0/401246394 >> v2:172.21.15.196:3300/0 conn(0x7f79fc0b2360 msgr2=0x7f79fc0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f9400640 1 --2- 172.21.15.86:0/401246394 >> v2:172.21.15.196:3300/0 conn(0x7f79fc0b2360 0x7f79fc0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f8a00640 1 -- 172.21.15.86:0/401246394 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2676799677 0 0) 0x7f79e0003530 con 0x7f79fc15cee0 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f9400640 1 -- 172.21.15.86:0/401246394 >> v1:172.21.15.86:6789/0 conn(0x7f79fc15cee0 legacy=0x7f79fc06d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f9400640 1 -- 172.21.15.86:0/401246394 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc160120 con 0x7f79fc0b3430 2024-03-22T04:15:10.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.919+0000 7f79f9400640 1 --2- 172.21.15.86:0/401246394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc0b3430 0x7f79fc15c9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79e80097c0 tx=0x7f79e801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d084f23a4e7b26f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f79f8a00640 1 -- 172.21.15.86:0/401246394 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e80b0070 con 0x7f79fc0b3430 2024-03-22T04:15:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f79f8a00640 1 -- 172.21.15.86:0/401246394 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e80ab400 con 0x7f79fc0b3430 2024-03-22T04:15:10.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f79f8a00640 1 -- 172.21.15.86:0/401246394 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4001680 con 0x7f79fc0b3430 2024-03-22T04:15:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- 172.21.15.86:0/401246394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc0b3430 msgr2=0x7f79fc15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 --2- 172.21.15.86:0/401246394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc0b3430 0x7f79fc15c9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79e80097c0 tx=0x7f79e801c3c0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- 172.21.15.86:0/401246394 shutdown_connections 2024-03-22T04:15:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 --2- 172.21.15.86:0/401246394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc0b3430 0x7f79fc15c9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 --2- 172.21.15.86:0/401246394 >> v2:172.21.15.196:3300/0 conn(0x7f79fc0b2360 0x7f79fc0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- 172.21.15.86:0/401246394 >> 172.21.15.86:0/401246394 conn(0x7f79fc0cc940 msgr2=0x7f79fc0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- 172.21.15.86:0/401246394 shutdown_connections 2024-03-22T04:15:10.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- 172.21.15.86:0/401246394 wait complete. 2024-03-22T04:15:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 Processor -- start 2024-03-22T04:15:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- start start 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc0b2360 0x7f79fc148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79fc0b3430 0x7f79fc137e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc15cee0 0x7f79fc0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc160c00 con 0x7f79fc0b2360 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc15d350 con 0x7f79fc0b3430 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f7a01200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc149340 con 0x7f79fc15cee0 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f79f9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc0b2360 0x7f79fc148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f79f9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc0b2360 0x7f79fc148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43694/0 (socket says 172.21.15.86:43694) 2024-03-22T04:15:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f79f9e00640 1 -- 172.21.15.86:0/1114731404 learned_addr learned my addr 172.21.15.86:0/1114731404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.923+0000 7f79f9400640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79fc0b3430 0x7f79fc137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79fa800640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc15cee0 0x7f79fc0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f9e00640 1 -- 172.21.15.86:0/1114731404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc15cee0 msgr2=0x7f79fc0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f9e00640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc15cee0 0x7f79fc0cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f9e00640 1 -- 172.21.15.86:0/1114731404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79fc0b3430 msgr2=0x7f79fc137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f9e00640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79fc0b3430 0x7f79fc137e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f9e00640 1 -- 172.21.15.86:0/1114731404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc15d710 con 0x7f79fc0b2360 2024-03-22T04:15:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f9e00640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc0b2360 0x7f79fc148e00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00efc0 tx=0x7f79ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f2000640 1 -- 172.21.15.86:0/1114731404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec018020 con 0x7f79fc0b2360 2024-03-22T04:15:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f2000640 1 -- 172.21.15.86:0/1114731404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec014480 con 0x7f79fc0b2360 2024-03-22T04:15:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f2000640 1 -- 172.21.15.86:0/1114731404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec014850 con 0x7f79fc0b2360 2024-03-22T04:15:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc1383d0 con 0x7f79fc0b2360 2024-03-22T04:15:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc15fb40 con 0x7f79fc0b2360 2024-03-22T04:15:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc074470 con 0x7f79fc0b2360 2024-03-22T04:15:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f2000640 1 -- 172.21.15.86:0/1114731404 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f79ec01b020 con 0x7f79fc0b2360 2024-03-22T04:15:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f2000640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f79dc067b20 0x7f79dc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f2000640 1 -- 172.21.15.86:0/1114731404 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f79ec086920 con 0x7f79fc0b2360 2024-03-22T04:15:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.927+0000 7f79f9400640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f79dc067b20 0x7f79dc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.931+0000 7f79f9400640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f79dc067b20 0x7f79dc069fe0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f79e80ad000 tx=0x7f79e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:10.931+0000 7f79f2000640 1 -- 172.21.15.86:0/1114731404 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f79ec022070 con 0x7f79fc0b2360 2024-03-22T04:15:11.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:11.127+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f79fc06e9f0 con 0x7f79fc0b2360 2024-03-22T04:15:12.038 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-03-22T04:15:12.035+0000 7f2bd2600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:15:12.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.075+0000 7f79f2000640 1 -- 172.21.15.86:0/1114731404 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f79ec010040 con 0x7f79fc0b2360 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f79dc067b20 msgr2=0x7f79dc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f79dc067b20 0x7f79dc069fe0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f79e80ad000 tx=0x7f79e8008040 comp rx=0 tx=0).stop 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc0b2360 msgr2=0x7f79fc148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc0b2360 0x7f79fc148e00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00efc0 tx=0x7f79ec00c490 comp rx=0 tx=0).stop 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 shutdown_connections 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f79dc067b20 0x7f79dc069fe0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79fc0b3430 0x7f79fc137e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79fc15cee0 0x7f79fc0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 --2- 172.21.15.86:0/1114731404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79fc0b2360 0x7f79fc148e00 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 >> 172.21.15.86:0/1114731404 conn(0x7f79fc0cc940 msgr2=0x7f79fc0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 shutdown_connections 2024-03-22T04:15:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.079+0000 7f7a01200640 1 -- 172.21.15.86:0/1114731404 wait complete. 2024-03-22T04:15:12.107 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-22T04:15:12.107 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-22T04:15:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3de5800640 1 Processor -- start 2024-03-22T04:15:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3de5800640 1 -- start start 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3de5800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3de015c730 0x7f3de009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3de5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3de00b11c0 0x7f3de00b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3de5800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3de00cda00 con 0x7f3de00b3610 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3de5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3de009b910 con 0x7f3de00b11c0 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3de5800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3de00b1710 con 0x7f3de015c730 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3ddf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3de00b11c0 0x7f3de00b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3ddf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3de00b11c0 0x7f3de00b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43704/0 (socket says 172.21.15.86:43704) 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.243+0000 7f3ddf400640 1 -- 172.21.15.86:0/2842219385 learned_addr learned my addr 172.21.15.86:0/2842219385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3ddea00640 1 --2- 172.21.15.86:0/2842219385 >> v2:172.21.15.196:3300/0 conn(0x7f3de015c730 0x7f3de009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3ddf400640 1 -- 172.21.15.86:0/2842219385 >> v2:172.21.15.196:3300/0 conn(0x7f3de015c730 msgr2=0x7f3de009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3ddf400640 1 --2- 172.21.15.86:0/2842219385 >> v2:172.21.15.196:3300/0 conn(0x7f3de015c730 0x7f3de009aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3ddf400640 1 -- 172.21.15.86:0/2842219385 >> v1:172.21.15.86:6789/0 conn(0x7f3de00b3610 legacy=0x7f3de009afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:12.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3ddf400640 1 -- 172.21.15.86:0/2842219385 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de00cdce0 con 0x7f3de00b11c0 2024-03-22T04:15:12.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3ddf400640 1 --2- 172.21.15.86:0/2842219385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de00b11c0 0x7f3de00b1ba0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009e50 tx=0x7f3dd801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97fde8b0ddb473d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3dde000640 1 -- 172.21.15.86:0/2842219385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80a5730 con 0x7f3de00b11c0 2024-03-22T04:15:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3dde000640 1 -- 172.21.15.86:0/2842219385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd80a58d0 con 0x7f3de00b11c0 2024-03-22T04:15:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3dde000640 1 -- 172.21.15.86:0/2842219385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80a5ca0 con 0x7f3de00b11c0 2024-03-22T04:15:12.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3de5800640 1 -- 172.21.15.86:0/2842219385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de00b11c0 msgr2=0x7f3de00b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:12.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3de5800640 1 --2- 172.21.15.86:0/2842219385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de00b11c0 0x7f3de00b1ba0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009e50 tx=0x7f3dd801c960 comp rx=0 tx=0).stop 2024-03-22T04:15:12.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3de5800640 1 -- 172.21.15.86:0/2842219385 shutdown_connections 2024-03-22T04:15:12.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3de5800640 1 --2- 172.21.15.86:0/2842219385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de00b11c0 0x7f3de00b1ba0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.247+0000 7f3de5800640 1 --2- 172.21.15.86:0/2842219385 >> v2:172.21.15.196:3300/0 conn(0x7f3de015c730 0x7f3de009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 -- 172.21.15.86:0/2842219385 >> 172.21.15.86:0/2842219385 conn(0x7f3de00b3bf0 msgr2=0x7f3de00b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:12.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 -- 172.21.15.86:0/2842219385 shutdown_connections 2024-03-22T04:15:12.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 -- 172.21.15.86:0/2842219385 wait complete. 2024-03-22T04:15:12.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 Processor -- start 2024-03-22T04:15:12.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 -- start start 2024-03-22T04:15:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de00b11c0 0x7f3de014d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3de00b3610 0x7f3de00af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de015c730 0x7f3de00afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3de00b2c80 con 0x7f3de015c730 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3de00b2b00 con 0x7f3de00b11c0 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3de00b2e00 con 0x7f3de00b3610 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de015c730 0x7f3de00afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de015c730 0x7f3de00afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43708/0 (socket says 172.21.15.86:43708) 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 -- 172.21.15.86:0/1804392310 learned_addr learned my addr 172.21.15.86:0/1804392310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3dd7e00640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3de00b3610 0x7f3de00af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddea00640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de00b11c0 0x7f3de014d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 -- 172.21.15.86:0/1804392310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3de00b3610 msgr2=0x7f3de00af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3de00b3610 0x7f3de00af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 -- 172.21.15.86:0/1804392310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de00b11c0 msgr2=0x7f3de014d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de00b11c0 0x7f3de014d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 -- 172.21.15.86:0/1804392310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de00b0700 con 0x7f3de015c730 2024-03-22T04:15:12.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3ddf400640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de015c730 0x7f3de00afc10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8019570 tx=0x7f3dd8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3dd6a00640 1 -- 172.21.15.86:0/1804392310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80bc020 con 0x7f3de015c730 2024-03-22T04:15:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3dd6a00640 1 -- 172.21.15.86:0/1804392310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd80b8670 con 0x7f3de015c730 2024-03-22T04:15:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3dd6a00640 1 -- 172.21.15.86:0/1804392310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80b8a20 con 0x7f3de015c730 2024-03-22T04:15:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.251+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de00b09e0 con 0x7f3de015c730 2024-03-22T04:15:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.255+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de014d8c0 con 0x7f3de015c730 2024-03-22T04:15:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.255+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de0074470 con 0x7f3de015c730 2024-03-22T04:15:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.255+0000 7f3dd6a00640 1 -- 172.21.15.86:0/1804392310 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3dd80bf020 con 0x7f3de015c730 2024-03-22T04:15:12.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.255+0000 7f3dd6a00640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3dbc067b20 0x7f3dbc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:12.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.255+0000 7f3dd6a00640 1 -- 172.21.15.86:0/1804392310 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f3dd8120ff0 con 0x7f3de015c730 2024-03-22T04:15:12.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.255+0000 7f3ddea00640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3dbc067b20 0x7f3dbc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.255+0000 7f3ddea00640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3dbc067b20 0x7f3dbc069fe0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3de01fa670 tx=0x7f3dd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:12.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.263+0000 7f3dd6a00640 1 -- 172.21.15.86:0/1804392310 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3dd80c6030 con 0x7f3de015c730 2024-03-22T04:15:12.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:12.455+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3de015b420 con 0x7f3de015c730 2024-03-22T04:15:13.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.087+0000 7f3dd6a00640 1 -- 172.21.15.86:0/1804392310 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f3dd80165e0 con 0x7f3de015c730 2024-03-22T04:15:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3dbc067b20 msgr2=0x7f3dbc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3dbc067b20 0x7f3dbc069fe0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3de01fa670 tx=0x7f3dd0008040 comp rx=0 tx=0).stop 2024-03-22T04:15:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de015c730 msgr2=0x7f3de00afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de015c730 0x7f3de00afc10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8019570 tx=0x7f3dd8019700 comp rx=0 tx=0).stop 2024-03-22T04:15:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 shutdown_connections 2024-03-22T04:15:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3dbc067b20 0x7f3dbc069fe0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de015c730 0x7f3de00afc10 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3de00b3610 0x7f3de00af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 --2- 172.21.15.86:0/1804392310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3de00b11c0 0x7f3de014d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 >> 172.21.15.86:0/1804392310 conn(0x7f3de00b3bf0 msgr2=0x7f3de00ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:13.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 shutdown_connections 2024-03-22T04:15:13.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.091+0000 7f3de5800640 1 -- 172.21.15.86:0/1804392310 wait complete. 2024-03-22T04:15:13.115 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-22T04:15:13.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:15:13.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fc400640 1 Processor -- start 2024-03-22T04:15:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fc400640 1 -- start start 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fc400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f64f40d0580 0x7f64f4071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fc400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64f406e2f0 0x7f64f4071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fc400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f41623d0 con 0x7f64f415e630 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fc400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f64f41620f0 con 0x7f64f406e2f0 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fc400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f64f415ec10 con 0x7f64f40d0580 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fa400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64f406e2f0 0x7f64f4071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fa400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64f406e2f0 0x7f64f4071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43714/0 (socket says 172.21.15.86:43714) 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fa400640 1 -- 172.21.15.86:0/2634248841 learned_addr learned my addr 172.21.15.86:0/2634248841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:13.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64f9000640 1 --2- 172.21.15.86:0/2634248841 >> v2:172.21.15.196:3300/0 conn(0x7f64f40d0580 0x7f64f4071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fa400640 1 -- 172.21.15.86:0/2634248841 >> v2:172.21.15.196:3300/0 conn(0x7f64f40d0580 msgr2=0x7f64f4071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:13.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fa400640 1 --2- 172.21.15.86:0/2634248841 >> v2:172.21.15.196:3300/0 conn(0x7f64f40d0580 0x7f64f4071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fa400640 1 -- 172.21.15.86:0/2634248841 >> v1:172.21.15.196:6789/0 conn(0x7f64f415e630 legacy=0x7f64f40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.247+0000 7f64fa400640 1 -- 172.21.15.86:0/2634248841 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f415eef0 con 0x7f64f406e2f0 2024-03-22T04:15:13.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fa400640 1 --2- 172.21.15.86:0/2634248841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f406e2f0 0x7f64f4071640 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f64f406c5e0 tx=0x7f64ec01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2445c3aa2f7f02b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:13.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64f3e00640 1 -- 172.21.15.86:0/2634248841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0a56c0 con 0x7f64f406e2f0 2024-03-22T04:15:13.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64f3e00640 1 -- 172.21.15.86:0/2634248841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec0a5860 con 0x7f64f406e2f0 2024-03-22T04:15:13.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64f3e00640 1 -- 172.21.15.86:0/2634248841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0a5c30 con 0x7f64f406e2f0 2024-03-22T04:15:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fc400640 1 -- 172.21.15.86:0/2634248841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f406e2f0 msgr2=0x7f64f4071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fc400640 1 --2- 172.21.15.86:0/2634248841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f406e2f0 0x7f64f4071640 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f64f406c5e0 tx=0x7f64ec01c9f0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fc400640 1 -- 172.21.15.86:0/2634248841 shutdown_connections 2024-03-22T04:15:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fc400640 1 --2- 172.21.15.86:0/2634248841 >> v2:172.21.15.196:3300/0 conn(0x7f64f40d0580 0x7f64f4071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fc400640 1 --2- 172.21.15.86:0/2634248841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f406e2f0 0x7f64f4071640 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fc400640 1 -- 172.21.15.86:0/2634248841 >> 172.21.15.86:0/2634248841 conn(0x7f64f406f910 msgr2=0x7f64f406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fc400640 1 -- 172.21.15.86:0/2634248841 shutdown_connections 2024-03-22T04:15:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.251+0000 7f64fc400640 1 -- 172.21.15.86:0/2634248841 wait complete. 2024-03-22T04:15:13.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.255+0000 7f64fc400640 1 Processor -- start 2024-03-22T04:15:13.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.255+0000 7f64fc400640 1 -- start start 2024-03-22T04:15:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64f406e2f0 0x7f64f40c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f40d0580 0x7f64f40c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f415e630 0x7f64f40cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4071f90 con 0x7f64f415e630 2024-03-22T04:15:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64f415f9d0 con 0x7f64f40d0580 2024-03-22T04:15:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64f40cfaf0 con 0x7f64f406e2f0 2024-03-22T04:15:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f415e630 0x7f64f40cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f415e630 0x7f64f40cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43720/0 (socket says 172.21.15.86:43720) 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 -- 172.21.15.86:0/1529149931 learned_addr learned my addr 172.21.15.86:0/1529149931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64f9000640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f40d0580 0x7f64f40c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64f9a00640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64f406e2f0 0x7f64f40c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 -- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64f406e2f0 msgr2=0x7f64f40c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64f406e2f0 0x7f64f40c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 -- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f40d0580 msgr2=0x7f64f40c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f40d0580 0x7f64f40c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 -- 172.21.15.86:0/1529149931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f40c5f30 con 0x7f64f415e630 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64f9a00640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64f406e2f0 0x7f64f40c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fa400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f415e630 0x7f64f40cc990 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f64ec019600 tx=0x7f64ec019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64f1600640 1 -- 172.21.15.86:0/1529149931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0b8670 con 0x7f64f415e630 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64f1600640 1 -- 172.21.15.86:0/1529149931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec0be070 con 0x7f64f415e630 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f40c6210 con 0x7f64f415e630 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64f1600640 1 -- 172.21.15.86:0/1529149931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0b8a50 con 0x7f64f415e630 2024-03-22T04:15:13.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f40c67d0 con 0x7f64f415e630 2024-03-22T04:15:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f4068650 con 0x7f64f415e630 2024-03-22T04:15:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.259+0000 7f64f1600640 1 -- 172.21.15.86:0/1529149931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f64ec0b8bf0 con 0x7f64f415e630 2024-03-22T04:15:13.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.263+0000 7f64f1600640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d4067b20 0x7f64d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.263+0000 7f64f1600640 1 -- 172.21.15.86:0/1529149931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f64ec121300 con 0x7f64f415e630 2024-03-22T04:15:13.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.263+0000 7f64f9a00640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d4067b20 0x7f64d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.263+0000 7f64f9a00640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d4067b20 0x7f64d4069fe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f64f40cbdb0 tx=0x7f64e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:13.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.267+0000 7f64f1600640 1 -- 172.21.15.86:0/1529149931 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f64ec0165e0 con 0x7f64f415e630 2024-03-22T04:15:13.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64f40c7e90 con 0x7f64f415e630 2024-03-22T04:15:13.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64f1600640 1 -- 172.21.15.86:0/1529149931 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5189 (secure 0 0 0) 0x7f64ec0f1c00 con 0x7f64f415e630 2024-03-22T04:15:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:13.460 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:15:12.461110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1130704586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1130704586},{"type":"v1","addr":"172.21.15.86:6837","nonce":1130704586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6807,"qdb_cluster":[6807]},"id":5}]} 2024-03-22T04:15:13.460 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-03-22T04:15:13.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d4067b20 msgr2=0x7f64d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d4067b20 0x7f64d4069fe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f64f40cbdb0 tx=0x7f64e8009040 comp rx=0 tx=0).stop 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f415e630 msgr2=0x7f64f40cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f415e630 0x7f64f40cc990 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f64ec019600 tx=0x7f64ec019790 comp rx=0 tx=0).stop 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 shutdown_connections 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64d4067b20 0x7f64d4069fe0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f40d0580 0x7f64f40c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f415e630 0x7f64f40cc990 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 --2- 172.21.15.86:0/1529149931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64f406e2f0 0x7f64f40c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 >> 172.21.15.86:0/1529149931 conn(0x7f64f406f910 msgr2=0x7f64f406d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 shutdown_connections 2024-03-22T04:15:13.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.455+0000 7f64fc400640 1 -- 172.21.15.86:0/1529149931 wait complete. 2024-03-22T04:15:13.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:15:13.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99dd000640 1 Processor -- start 2024-03-22T04:15:13.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99dd000640 1 -- start start 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99dd000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d80aff50 0x7f99d806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99dd000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99d806cbb0 0x7f99d8143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99dd000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f99d8144320 0x7f99d806e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99dd000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f99d806d010 con 0x7f99d8144320 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99dd000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f99d8162aa0 con 0x7f99d806cbb0 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99dd000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99d80ae5a0 con 0x7f99d80aff50 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f99d8144320 0x7f99d806e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f99d8144320 0x7f99d806e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43728/0 (socket says 172.21.15.86:43728) 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d6800640 1 -- 172.21.15.86:0/452871322 learned_addr learned my addr 172.21.15.86:0/452871322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99cfe00640 1 --2- 172.21.15.86:0/452871322 >> v2:172.21.15.196:3300/0 conn(0x7f99d806cbb0 0x7f99d8143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d5e00640 1 --2- 172.21.15.86:0/452871322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d80aff50 0x7f99d806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d6800640 1 -- 172.21.15.86:0/452871322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d80aff50 msgr2=0x7f99d806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d6800640 1 --2- 172.21.15.86:0/452871322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d80aff50 0x7f99d806c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d6800640 1 -- 172.21.15.86:0/452871322 >> v2:172.21.15.196:3300/0 conn(0x7f99d806cbb0 msgr2=0x7f99d8143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:13.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d6800640 1 --2- 172.21.15.86:0/452871322 >> v2:172.21.15.196:3300/0 conn(0x7f99d806cbb0 0x7f99d8143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.615+0000 7f99d6800640 1 -- 172.21.15.86:0/452871322 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d8144b80 con 0x7f99d8144320 2024-03-22T04:15:13.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99d6800640 1 --2- 172.21.15.86:0/452871322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d8144320 0x7f99d806e220 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f99d80ae750 tx=0x7f99d001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a3b8f24af8f960 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:13.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99d5400640 1 -- 172.21.15.86:0/452871322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00a6500 con 0x7f99d8144320 2024-03-22T04:15:13.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99d5400640 1 -- 172.21.15.86:0/452871322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d00a66a0 con 0x7f99d8144320 2024-03-22T04:15:13.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99d5400640 1 -- 172.21.15.86:0/452871322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00a6a70 con 0x7f99d8144320 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 -- 172.21.15.86:0/452871322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d8144320 msgr2=0x7f99d806e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 --2- 172.21.15.86:0/452871322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d8144320 0x7f99d806e220 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f99d80ae750 tx=0x7f99d001c670 comp rx=0 tx=0).stop 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 -- 172.21.15.86:0/452871322 shutdown_connections 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 --2- 172.21.15.86:0/452871322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d8144320 0x7f99d806e220 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 --2- 172.21.15.86:0/452871322 >> v2:172.21.15.196:3300/0 conn(0x7f99d806cbb0 0x7f99d8143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 --2- 172.21.15.86:0/452871322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d80aff50 0x7f99d806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 -- 172.21.15.86:0/452871322 >> 172.21.15.86:0/452871322 conn(0x7f99d8071fe0 msgr2=0x7f99d80723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 -- 172.21.15.86:0/452871322 shutdown_connections 2024-03-22T04:15:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.619+0000 7f99dd000640 1 -- 172.21.15.86:0/452871322 wait complete. 2024-03-22T04:15:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99dd000640 1 Processor -- start 2024-03-22T04:15:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99dd000640 1 -- start start 2024-03-22T04:15:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806cbb0 0x7f99d8143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99dd000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d80aff50 0x7f99d8131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99dd000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d8144320 0x7f99d81565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99dd000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8163090 con 0x7f99d806cbb0 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99dd000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8162f10 con 0x7f99d80aff50 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99dd000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99d8144790 con 0x7f99d8144320 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806cbb0 0x7f99d8143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806cbb0 0x7f99d8143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43742/0 (socket says 172.21.15.86:43742) 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99d5e00640 1 -- 172.21.15.86:0/895705384 learned_addr learned my addr 172.21.15.86:0/895705384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99d6800640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d8144320 0x7f99d81565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.623+0000 7f99cfe00640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d80aff50 0x7f99d8131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99d5e00640 1 -- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d8144320 msgr2=0x7f99d81565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99d5e00640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d8144320 0x7f99d81565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99d5e00640 1 -- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d80aff50 msgr2=0x7f99d8131ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99d5e00640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d80aff50 0x7f99d8131ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99d5e00640 1 -- 172.21.15.86:0/895705384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d8141320 con 0x7f99d806cbb0 2024-03-22T04:15:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99cfe00640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d80aff50 0x7f99d8131ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99d5e00640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806cbb0 0x7f99d8143760 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f99c800e960 tx=0x7f99c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99ce000640 1 -- 172.21.15.86:0/895705384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8017070 con 0x7f99d806cbb0 2024-03-22T04:15:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99ce000640 1 -- 172.21.15.86:0/895705384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c800ca00 con 0x7f99d806cbb0 2024-03-22T04:15:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99ce000640 1 -- 172.21.15.86:0/895705384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c800cdb0 con 0x7f99d806cbb0 2024-03-22T04:15:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d8140a80 con 0x7f99d806cbb0 2024-03-22T04:15:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d8141040 con 0x7f99d806cbb0 2024-03-22T04:15:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d8143d60 con 0x7f99d806cbb0 2024-03-22T04:15:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.627+0000 7f99ce000640 1 -- 172.21.15.86:0/895705384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f99c801a070 con 0x7f99d806cbb0 2024-03-22T04:15:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.631+0000 7f99ce000640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99b4067b20 0x7f99b4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.631+0000 7f99ce000640 1 -- 172.21.15.86:0/895705384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f99c8086360 con 0x7f99d806cbb0 2024-03-22T04:15:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.631+0000 7f99cfe00640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99b4067b20 0x7f99b4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.631+0000 7f99cfe00640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99b4067b20 0x7f99b4069fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f99d81419d0 tx=0x7f99c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.635+0000 7f99ce000640 1 -- 172.21.15.86:0/895705384 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f99c8056580 con 0x7f99d806cbb0 2024-03-22T04:15:13.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.931+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99d80cfaa0 con 0x7f99d806cbb0 2024-03-22T04:15:13.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.931+0000 7f99ce000640 1 -- 172.21.15.86:0/895705384 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16885 (secure 0 0 0) 0x7f99c805a620 con 0x7f99d806cbb0 2024-03-22T04:15:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:13.936 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:11.057726+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:15:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99b4067b20 msgr2=0x7f99b4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99b4067b20 0x7f99b4069fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f99d81419d0 tx=0x7f99c0008040 comp rx=0 tx=0).stop 2024-03-22T04:15:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806cbb0 msgr2=0x7f99d8143760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806cbb0 0x7f99d8143760 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f99c800e960 tx=0x7f99c800ee20 comp rx=0 tx=0).stop 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 shutdown_connections 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99b4067b20 0x7f99b4069fe0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99d80aff50 0x7f99d8131ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99d8144320 0x7f99d81565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 --2- 172.21.15.86:0/895705384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d806cbb0 0x7f99d8143760 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 >> 172.21.15.86:0/895705384 conn(0x7f99d8071fe0 msgr2=0x7f99d80b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 shutdown_connections 2024-03-22T04:15:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:13.935+0000 7f99dd000640 1 -- 172.21.15.86:0/895705384 wait complete. 2024-03-22T04:15:13.960 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T04:15:13.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:15:14.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc9400640 1 Processor -- start 2024-03-22T04:15:14.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc9400640 1 -- start start 2024-03-22T04:15:14.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc9400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc40d0580 0x7f5dc4071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc9400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc41623d0 con 0x7f5dc406e2f0 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc9400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc41620f0 con 0x7f5dc415e630 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc9400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc415ec10 con 0x7f5dc40d0580 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc3200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5dc406e2f0 0x7f5dc4071640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44146/0 (socket says 172.21.15.86:44146) 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc3200640 1 -- 172.21.15.86:0/942494878 learned_addr learned my addr 172.21.15.86:0/942494878 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc1e00640 1 --2- 172.21.15.86:0/942494878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc40d0580 0x7f5dc4071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc1e00640 1 -- 172.21.15.86:0/942494878 >> v1:172.21.15.196:6789/0 conn(0x7f5dc415e630 legacy=0x7f5dc40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc1e00640 1 -- 172.21.15.86:0/942494878 >> v1:172.21.15.86:6789/0 conn(0x7f5dc406e2f0 legacy=0x7f5dc4071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.095+0000 7f5dc1e00640 1 -- 172.21.15.86:0/942494878 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc415eef0 con 0x7f5dc40d0580 2024-03-22T04:15:14.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc1e00640 1 --2- 172.21.15.86:0/942494878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc40d0580 0x7f5dc4071100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5db00097f0 tx=0x7f5db001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f1dcb689de86a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc1400640 1 -- 172.21.15.86:0/942494878 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00b0070 con 0x7f5dc40d0580 2024-03-22T04:15:14.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc1400640 1 -- 172.21.15.86:0/942494878 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db00ab600 con 0x7f5dc40d0580 2024-03-22T04:15:14.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc1400640 1 -- 172.21.15.86:0/942494878 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00ab9d0 con 0x7f5dc40d0580 2024-03-22T04:15:14.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc9400640 1 -- 172.21.15.86:0/942494878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc40d0580 msgr2=0x7f5dc4071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc9400640 1 --2- 172.21.15.86:0/942494878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc40d0580 0x7f5dc4071100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5db00097f0 tx=0x7f5db001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc9400640 1 -- 172.21.15.86:0/942494878 shutdown_connections 2024-03-22T04:15:14.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc9400640 1 --2- 172.21.15.86:0/942494878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc40d0580 0x7f5dc4071100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc9400640 1 -- 172.21.15.86:0/942494878 >> 172.21.15.86:0/942494878 conn(0x7f5dc406f910 msgr2=0x7f5dc406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:14.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc9400640 1 -- 172.21.15.86:0/942494878 shutdown_connections 2024-03-22T04:15:14.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.099+0000 7f5dc9400640 1 -- 172.21.15.86:0/942494878 wait complete. 2024-03-22T04:15:14.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 Processor -- start 2024-03-22T04:15:14.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 -- start start 2024-03-22T04:15:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc406e2f0 0x7f5dc4155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dc40d0580 0x7f5dc4153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dc415e630 0x7f5dc40c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40cfaf0 con 0x7f5dc415e630 2024-03-22T04:15:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc415f9d0 con 0x7f5dc40d0580 2024-03-22T04:15:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc40cfc70 con 0x7f5dc406e2f0 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dc415e630 0x7f5dc40c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dc415e630 0x7f5dc40c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43748/0 (socket says 172.21.15.86:43748) 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 -- 172.21.15.86:0/3107726430 learned_addr learned my addr 172.21.15.86:0/3107726430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc2800640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc406e2f0 0x7f5dc4155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc1e00640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dc40d0580 0x7f5dc4153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 -- 172.21.15.86:0/3107726430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc406e2f0 msgr2=0x7f5dc4155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc406e2f0 0x7f5dc4155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 -- 172.21.15.86:0/3107726430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dc40d0580 msgr2=0x7f5dc4153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dc40d0580 0x7f5dc4153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 -- 172.21.15.86:0/3107726430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc4155670 con 0x7f5dc415e630 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc3200640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dc415e630 0x7f5dc40c9500 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00cab0 tx=0x7f5dbc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dbaa00640 1 -- 172.21.15.86:0/3107726430 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc017070 con 0x7f5dc415e630 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dbaa00640 1 -- 172.21.15.86:0/3107726430 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00ac30 con 0x7f5dc415e630 2024-03-22T04:15:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dbaa00640 1 -- 172.21.15.86:0/3107726430 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc012400 con 0x7f5dc415e630 2024-03-22T04:15:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc4155910 con 0x7f5dc415e630 2024-03-22T04:15:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.103+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc4155ed0 con 0x7f5dc415e630 2024-03-22T04:15:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.107+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc4068650 con 0x7f5dc415e630 2024-03-22T04:15:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.107+0000 7f5dbaa00640 1 -- 172.21.15.86:0/3107726430 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5dbc01a070 con 0x7f5dc415e630 2024-03-22T04:15:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.107+0000 7f5dbaa00640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5da4067b20 0x7f5da4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.107+0000 7f5dbaa00640 1 -- 172.21.15.86:0/3107726430 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f5dbc085bd0 con 0x7f5dc415e630 2024-03-22T04:15:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.107+0000 7f5dc2800640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5da4067b20 0x7f5da4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.111+0000 7f5dc2800640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5da4067b20 0x7f5da4069fe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5db4007fd0 tx=0x7f5db4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.111+0000 7f5dbaa00640 1 -- 172.21.15.86:0/3107726430 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5dbc055e00 con 0x7f5dc415e630 2024-03-22T04:15:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.267+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5dc40cf720 con 0x7f5da4067b20 2024-03-22T04:15:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.271+0000 7f5dbaa00640 1 -- 172.21.15.86:0/3107726430 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252008 (secure 0 0 0) 0x7f5dc40cf720 con 0x7f5da4067b20 2024-03-22T04:15:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:15:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5da4067b20 msgr2=0x7f5da4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5da4067b20 0x7f5da4069fe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5db4007fd0 tx=0x7f5db4009040 comp rx=0 tx=0).stop 2024-03-22T04:15:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dc415e630 msgr2=0x7f5dc40c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dc415e630 0x7f5dc40c9500 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00cab0 tx=0x7f5dbc00cf70 comp rx=0 tx=0).stop 2024-03-22T04:15:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 shutdown_connections 2024-03-22T04:15:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5da4067b20 0x7f5da4069fe0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dc40d0580 0x7f5dc4153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dc415e630 0x7f5dc40c9500 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 --2- 172.21.15.86:0/3107726430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dc406e2f0 0x7f5dc4155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 >> 172.21.15.86:0/3107726430 conn(0x7f5dc406f910 msgr2=0x7f5dc4071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:14.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 shutdown_connections 2024-03-22T04:15:14.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.275+0000 7f5dc9400640 1 -- 172.21.15.86:0/3107726430 wait complete. 2024-03-22T04:15:14.304 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-03-22T04:15:13.853550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254008,"kb_used_data":8676,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720712,"statfs":{"total":773094113280,"available":772834009088,"internally_reserved":0,"allocated":8884224,"data_stored":4658308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.810041"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.085434+0000","last_change":"2024-03-22T04:15:10.082611+0000","last_active":"2024-03-22T04:15:10.085434+0000","last_peered":"2024-03-22T04:15:10.085434+0000","last_clean":"2024-03-22T04:15:10.085434+0000","last_became_active":"2024-03-22T04:15:10.082361+0000","last_became_peered":"2024-03-22T04:15:10.082361+0000","last_unstale":"2024-03-22T04:15:10.085434+0000","last_undegraded":"2024-03-22T04:15:10.085434+0000","last_fullsized":"2024-03-22T04:15:10.085434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:13.597157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.095052+0000","last_change":"2024-03-22T04:15:10.092417+0000","last_active":"2024-03-22T04:15:10.095052+0000","last_peered":"2024-03-22T04:15:10.095052+0000","last_clean":"2024-03-22T04:15:10.095052+0000","last_became_active":"2024-03-22T04:15:10.089025+0000","last_became_peered":"2024-03-22T04:15:10.089025+0000","last_unstale":"2024-03-22T04:15:10.095052+0000","last_undegraded":"2024-03-22T04:15:10.095052+0000","last_fullsized":"2024-03-22T04:15:10.095052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:29.599987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092363+0000","last_change":"2024-03-22T04:15:10.088024+0000","last_active":"2024-03-22T04:15:10.092363+0000","last_peered":"2024-03-22T04:15:10.092363+0000","last_clean":"2024-03-22T04:15:10.092363+0000","last_became_active":"2024-03-22T04:15:10.087566+0000","last_became_peered":"2024-03-22T04:15:10.087566+0000","last_unstale":"2024-03-22T04:15:10.092363+0000","last_undegraded":"2024-03-22T04:15:10.092363+0000","last_fullsized":"2024-03-22T04:15:10.092363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:47:57.144223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083640+0000","last_change":"2024-03-22T04:15:10.074035+0000","last_active":"2024-03-22T04:15:10.083640+0000","last_peered":"2024-03-22T04:15:10.083640+0000","last_clean":"2024-03-22T04:15:10.083640+0000","last_became_active":"2024-03-22T04:15:10.073772+0000","last_became_peered":"2024-03-22T04:15:10.073772+0000","last_unstale":"2024-03-22T04:15:10.083640+0000","last_undegraded":"2024-03-22T04:15:10.083640+0000","last_fullsized":"2024-03-22T04:15:10.083640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:58:39.338057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.086619+0000","last_change":"2024-03-22T04:15:10.081975+0000","last_active":"2024-03-22T04:15:10.086619+0000","last_peered":"2024-03-22T04:15:10.086619+0000","last_clean":"2024-03-22T04:15:10.086619+0000","last_became_active":"2024-03-22T04:15:10.081664+0000","last_became_peered":"2024-03-22T04:15:10.081664+0000","last_unstale":"2024-03-22T04:15:10.086619+0000","last_undegraded":"2024-03-22T04:15:10.086619+0000","last_fullsized":"2024-03-22T04:15:10.086619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:50.088884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088149+0000","last_change":"2024-03-22T04:15:10.075370+0000","last_active":"2024-03-22T04:15:10.088149+0000","last_peered":"2024-03-22T04:15:10.088149+0000","last_clean":"2024-03-22T04:15:10.088149+0000","last_became_active":"2024-03-22T04:15:10.074881+0000","last_became_peered":"2024-03-22T04:15:10.074881+0000","last_unstale":"2024-03-22T04:15:10.088149+0000","last_undegraded":"2024-03-22T04:15:10.088149+0000","last_fullsized":"2024-03-22T04:15:10.088149+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:17.223573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.082886+0000","last_change":"2024-03-22T04:15:10.070748+0000","last_active":"2024-03-22T04:15:10.082886+0000","last_peered":"2024-03-22T04:15:10.082886+0000","last_clean":"2024-03-22T04:15:10.082886+0000","last_became_active":"2024-03-22T04:15:10.070438+0000","last_became_peered":"2024-03-22T04:15:10.070438+0000","last_unstale":"2024-03-22T04:15:10.082886+0000","last_undegraded":"2024-03-22T04:15:10.082886+0000","last_fullsized":"2024-03-22T04:15:10.082886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:13.597157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063713+0000","last_change":"2024-03-22T04:15:10.073399+0000","last_active":"2024-03-22T04:15:11.063713+0000","last_peered":"2024-03-22T04:15:11.063713+0000","last_clean":"2024-03-22T04:15:11.063713+0000","last_became_active":"2024-03-22T04:15:10.073131+0000","last_became_peered":"2024-03-22T04:15:10.073131+0000","last_unstale":"2024-03-22T04:15:11.063713+0000","last_undegraded":"2024-03-22T04:15:11.063713+0000","last_fullsized":"2024-03-22T04:15:11.063713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:06:06.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089948+0000","last_change":"2024-03-22T04:15:10.084268+0000","last_active":"2024-03-22T04:15:10.089948+0000","last_peered":"2024-03-22T04:15:10.089948+0000","last_clean":"2024-03-22T04:15:10.089948+0000","last_became_active":"2024-03-22T04:15:10.084071+0000","last_became_peered":"2024-03-22T04:15:10.084071+0000","last_unstale":"2024-03-22T04:15:10.089948+0000","last_undegraded":"2024-03-22T04:15:10.089948+0000","last_fullsized":"2024-03-22T04:15:10.089948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:28:37.146208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.079902+0000","last_change":"2024-03-22T04:15:10.075269+0000","last_active":"2024-03-22T04:15:10.079902+0000","last_peered":"2024-03-22T04:15:10.079902+0000","last_clean":"2024-03-22T04:15:10.079902+0000","last_became_active":"2024-03-22T04:15:10.074981+0000","last_became_peered":"2024-03-22T04:15:10.074981+0000","last_unstale":"2024-03-22T04:15:10.079902+0000","last_undegraded":"2024-03-22T04:15:10.079902+0000","last_fullsized":"2024-03-22T04:15:10.079902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:33:44.279996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.085320+0000","last_change":"2024-03-22T04:15:10.080689+0000","last_active":"2024-03-22T04:15:10.085320+0000","last_peered":"2024-03-22T04:15:10.085320+0000","last_clean":"2024-03-22T04:15:10.085320+0000","last_became_active":"2024-03-22T04:15:10.080395+0000","last_became_peered":"2024-03-22T04:15:10.080395+0000","last_unstale":"2024-03-22T04:15:10.085320+0000","last_undegraded":"2024-03-22T04:15:10.085320+0000","last_fullsized":"2024-03-22T04:15:10.085320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:42:10.037211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083917+0000","last_change":"2024-03-22T04:15:10.078227+0000","last_active":"2024-03-22T04:15:10.083917+0000","last_peered":"2024-03-22T04:15:10.083917+0000","last_clean":"2024-03-22T04:15:10.083917+0000","last_became_active":"2024-03-22T04:15:10.077913+0000","last_became_peered":"2024-03-22T04:15:10.077913+0000","last_unstale":"2024-03-22T04:15:10.083917+0000","last_undegraded":"2024-03-22T04:15:10.083917+0000","last_fullsized":"2024-03-22T04:15:10.083917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:39:10.645964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.087686+0000","last_change":"2024-03-22T04:15:10.085091+0000","last_active":"2024-03-22T04:15:10.087686+0000","last_peered":"2024-03-22T04:15:10.087686+0000","last_clean":"2024-03-22T04:15:10.087686+0000","last_became_active":"2024-03-22T04:15:10.084732+0000","last_became_peered":"2024-03-22T04:15:10.084732+0000","last_unstale":"2024-03-22T04:15:10.087686+0000","last_undegraded":"2024-03-22T04:15:10.087686+0000","last_fullsized":"2024-03-22T04:15:10.087686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:56:31.586728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063300+0000","last_change":"2024-03-22T04:15:10.080842+0000","last_active":"2024-03-22T04:15:11.063300+0000","last_peered":"2024-03-22T04:15:11.063300+0000","last_clean":"2024-03-22T04:15:11.063300+0000","last_became_active":"2024-03-22T04:15:10.080355+0000","last_became_peered":"2024-03-22T04:15:10.080355+0000","last_unstale":"2024-03-22T04:15:11.063300+0000","last_undegraded":"2024-03-22T04:15:11.063300+0000","last_fullsized":"2024-03-22T04:15:11.063300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:06:34.989426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.093749+0000","last_change":"2024-03-22T04:15:10.088854+0000","last_active":"2024-03-22T04:15:10.093749+0000","last_peered":"2024-03-22T04:15:10.093749+0000","last_clean":"2024-03-22T04:15:10.093749+0000","last_became_active":"2024-03-22T04:15:10.088544+0000","last_became_peered":"2024-03-22T04:15:10.088544+0000","last_unstale":"2024-03-22T04:15:10.093749+0000","last_undegraded":"2024-03-22T04:15:10.093749+0000","last_fullsized":"2024-03-22T04:15:10.093749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:41:47.643800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088085+0000","last_change":"2024-03-22T04:15:10.076959+0000","last_active":"2024-03-22T04:15:10.088085+0000","last_peered":"2024-03-22T04:15:10.088085+0000","last_clean":"2024-03-22T04:15:10.088085+0000","last_became_active":"2024-03-22T04:15:10.076680+0000","last_became_peered":"2024-03-22T04:15:10.076680+0000","last_unstale":"2024-03-22T04:15:10.088085+0000","last_undegraded":"2024-03-22T04:15:10.088085+0000","last_fullsized":"2024-03-22T04:15:10.088085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:42:10.037211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063823+0000","last_change":"2024-03-22T04:15:10.080152+0000","last_active":"2024-03-22T04:15:11.063823+0000","last_peered":"2024-03-22T04:15:11.063823+0000","last_clean":"2024-03-22T04:15:11.063823+0000","last_became_active":"2024-03-22T04:15:10.079541+0000","last_became_peered":"2024-03-22T04:15:10.079541+0000","last_unstale":"2024-03-22T04:15:11.063823+0000","last_undegraded":"2024-03-22T04:15:11.063823+0000","last_fullsized":"2024-03-22T04:15:11.063823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:09:39.974023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092601+0000","last_change":"2024-03-22T04:15:10.087272+0000","last_active":"2024-03-22T04:15:10.092601+0000","last_peered":"2024-03-22T04:15:10.092601+0000","last_clean":"2024-03-22T04:15:10.092601+0000","last_became_active":"2024-03-22T04:15:10.084332+0000","last_became_peered":"2024-03-22T04:15:10.084332+0000","last_unstale":"2024-03-22T04:15:10.092601+0000","last_undegraded":"2024-03-22T04:15:10.092601+0000","last_fullsized":"2024-03-22T04:15:10.092601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:28:27.406345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063489+0000","last_change":"2024-03-22T04:15:10.075412+0000","last_active":"2024-03-22T04:15:11.063489+0000","last_peered":"2024-03-22T04:15:11.063489+0000","last_clean":"2024-03-22T04:15:11.063489+0000","last_became_active":"2024-03-22T04:15:10.075146+0000","last_became_peered":"2024-03-22T04:15:10.075146+0000","last_unstale":"2024-03-22T04:15:11.063489+0000","last_undegraded":"2024-03-22T04:15:11.063489+0000","last_fullsized":"2024-03-22T04:15:11.063489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:28:05.582341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090681+0000","last_change":"2024-03-22T04:15:10.078433+0000","last_active":"2024-03-22T04:15:10.090681+0000","last_peered":"2024-03-22T04:15:10.090681+0000","last_clean":"2024-03-22T04:15:10.090681+0000","last_became_active":"2024-03-22T04:15:10.078090+0000","last_became_peered":"2024-03-22T04:15:10.078090+0000","last_unstale":"2024-03-22T04:15:10.090681+0000","last_undegraded":"2024-03-22T04:15:10.090681+0000","last_fullsized":"2024-03-22T04:15:10.090681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:39:10.645964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.087037+0000","last_change":"2024-03-22T04:15:10.082472+0000","last_active":"2024-03-22T04:15:10.087037+0000","last_peered":"2024-03-22T04:15:10.087037+0000","last_clean":"2024-03-22T04:15:10.087037+0000","last_became_active":"2024-03-22T04:15:10.082209+0000","last_became_peered":"2024-03-22T04:15:10.082209+0000","last_unstale":"2024-03-22T04:15:10.087037+0000","last_undegraded":"2024-03-22T04:15:10.087037+0000","last_fullsized":"2024-03-22T04:15:10.087037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:17:10.391829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.094237+0000","last_change":"2024-03-22T04:15:10.091739+0000","last_active":"2024-03-22T04:15:10.094237+0000","last_peered":"2024-03-22T04:15:10.094237+0000","last_clean":"2024-03-22T04:15:10.094237+0000","last_became_active":"2024-03-22T04:15:10.091368+0000","last_became_peered":"2024-03-22T04:15:10.091368+0000","last_unstale":"2024-03-22T04:15:10.094237+0000","last_undegraded":"2024-03-22T04:15:10.094237+0000","last_fullsized":"2024-03-22T04:15:10.094237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:17.223573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.084217+0000","last_change":"2024-03-22T04:15:10.078117+0000","last_active":"2024-03-22T04:15:10.084217+0000","last_peered":"2024-03-22T04:15:10.084217+0000","last_clean":"2024-03-22T04:15:10.084217+0000","last_became_active":"2024-03-22T04:15:10.077855+0000","last_became_peered":"2024-03-22T04:15:10.077855+0000","last_unstale":"2024-03-22T04:15:10.084217+0000","last_undegraded":"2024-03-22T04:15:10.084217+0000","last_fullsized":"2024-03-22T04:15:10.084217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:14.447353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092316+0000","last_change":"2024-03-22T04:15:10.087888+0000","last_active":"2024-03-22T04:15:10.092316+0000","last_peered":"2024-03-22T04:15:10.092316+0000","last_clean":"2024-03-22T04:15:10.092316+0000","last_became_active":"2024-03-22T04:15:10.087600+0000","last_became_peered":"2024-03-22T04:15:10.087600+0000","last_unstale":"2024-03-22T04:15:10.092316+0000","last_undegraded":"2024-03-22T04:15:10.092316+0000","last_fullsized":"2024-03-22T04:15:10.092316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:35:16.557153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.093152+0000","last_change":"2024-03-22T04:15:10.085552+0000","last_active":"2024-03-22T04:15:10.093152+0000","last_peered":"2024-03-22T04:15:10.093152+0000","last_clean":"2024-03-22T04:15:10.093152+0000","last_became_active":"2024-03-22T04:15:10.085271+0000","last_became_peered":"2024-03-22T04:15:10.085271+0000","last_unstale":"2024-03-22T04:15:10.093152+0000","last_undegraded":"2024-03-22T04:15:10.093152+0000","last_fullsized":"2024-03-22T04:15:10.093152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:18:07.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083426+0000","last_change":"2024-03-22T04:15:10.073032+0000","last_active":"2024-03-22T04:15:10.083426+0000","last_peered":"2024-03-22T04:15:10.083426+0000","last_clean":"2024-03-22T04:15:10.083426+0000","last_became_active":"2024-03-22T04:15:10.072736+0000","last_became_peered":"2024-03-22T04:15:10.072736+0000","last_unstale":"2024-03-22T04:15:10.083426+0000","last_undegraded":"2024-03-22T04:15:10.083426+0000","last_fullsized":"2024-03-22T04:15:10.083426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:58:32.530161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089893+0000","last_change":"2024-03-22T04:15:10.075573+0000","last_active":"2024-03-22T04:15:10.089893+0000","last_peered":"2024-03-22T04:15:10.089893+0000","last_clean":"2024-03-22T04:15:10.089893+0000","last_became_active":"2024-03-22T04:15:10.075315+0000","last_became_peered":"2024-03-22T04:15:10.075315+0000","last_unstale":"2024-03-22T04:15:10.089893+0000","last_undegraded":"2024-03-22T04:15:10.089893+0000","last_fullsized":"2024-03-22T04:15:10.089893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:20:24.969370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089617+0000","last_change":"2024-03-22T04:15:10.083576+0000","last_active":"2024-03-22T04:15:10.089617+0000","last_peered":"2024-03-22T04:15:10.089617+0000","last_clean":"2024-03-22T04:15:10.089617+0000","last_became_active":"2024-03-22T04:15:10.082410+0000","last_became_peered":"2024-03-22T04:15:10.082410+0000","last_unstale":"2024-03-22T04:15:10.089617+0000","last_undegraded":"2024-03-22T04:15:10.089617+0000","last_fullsized":"2024-03-22T04:15:10.089617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:04:54.660411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089588+0000","last_change":"2024-03-22T04:15:10.078053+0000","last_active":"2024-03-22T04:15:10.089588+0000","last_peered":"2024-03-22T04:15:10.089588+0000","last_clean":"2024-03-22T04:15:10.089588+0000","last_became_active":"2024-03-22T04:15:10.077750+0000","last_became_peered":"2024-03-22T04:15:10.077750+0000","last_unstale":"2024-03-22T04:15:10.089588+0000","last_undegraded":"2024-03-22T04:15:10.089588+0000","last_fullsized":"2024-03-22T04:15:10.089588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:33:25.874350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.072581+0000","last_change":"2024-03-22T04:15:10.068372+0000","last_active":"2024-03-22T04:15:10.072581+0000","last_peered":"2024-03-22T04:15:10.072581+0000","last_clean":"2024-03-22T04:15:10.072581+0000","last_became_active":"2024-03-22T04:15:10.068119+0000","last_became_peered":"2024-03-22T04:15:10.068119+0000","last_unstale":"2024-03-22T04:15:10.072581+0000","last_undegraded":"2024-03-22T04:15:10.072581+0000","last_fullsized":"2024-03-22T04:15:10.072581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:16.933942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.084936+0000","last_change":"2024-03-22T04:15:10.081030+0000","last_active":"2024-03-22T04:15:10.084936+0000","last_peered":"2024-03-22T04:15:10.084936+0000","last_clean":"2024-03-22T04:15:10.084936+0000","last_became_active":"2024-03-22T04:15:10.080706+0000","last_became_peered":"2024-03-22T04:15:10.080706+0000","last_unstale":"2024-03-22T04:15:10.084936+0000","last_undegraded":"2024-03-22T04:15:10.084936+0000","last_fullsized":"2024-03-22T04:15:10.084936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:02:34.622890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083719+0000","last_change":"2024-03-22T04:15:10.073135+0000","last_active":"2024-03-22T04:15:10.083719+0000","last_peered":"2024-03-22T04:15:10.083719+0000","last_clean":"2024-03-22T04:15:10.083719+0000","last_became_active":"2024-03-22T04:15:10.072394+0000","last_became_peered":"2024-03-22T04:15:10.072394+0000","last_unstale":"2024-03-22T04:15:10.083719+0000","last_undegraded":"2024-03-22T04:15:10.083719+0000","last_fullsized":"2024-03-22T04:15:10.083719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:37:52.528838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.087928+0000","last_change":"2024-03-22T04:15:10.083260+0000","last_active":"2024-03-22T04:15:10.087928+0000","last_peered":"2024-03-22T04:15:10.087928+0000","last_clean":"2024-03-22T04:15:10.087928+0000","last_became_active":"2024-03-22T04:15:10.082565+0000","last_became_peered":"2024-03-22T04:15:10.082565+0000","last_unstale":"2024-03-22T04:15:10.087928+0000","last_undegraded":"2024-03-22T04:15:10.087928+0000","last_fullsized":"2024-03-22T04:15:10.087928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:56:41.158886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.102151+0000","last_change":"2024-03-22T04:15:09.070927+0000","last_active":"2024-03-22T04:15:10.102151+0000","last_peered":"2024-03-22T04:15:10.102151+0000","last_clean":"2024-03-22T04:15:10.102151+0000","last_became_active":"2024-03-22T04:15:09.070528+0000","last_became_peered":"2024-03-22T04:15:09.070528+0000","last_unstale":"2024-03-22T04:15:10.102151+0000","last_undegraded":"2024-03-22T04:15:10.102151+0000","last_fullsized":"2024-03-22T04:15:10.102151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:08.704172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090474+0000","last_change":"2024-03-22T04:15:10.087934+0000","last_active":"2024-03-22T04:15:10.090474+0000","last_peered":"2024-03-22T04:15:10.090474+0000","last_clean":"2024-03-22T04:15:10.090474+0000","last_became_active":"2024-03-22T04:15:10.086642+0000","last_became_peered":"2024-03-22T04:15:10.086642+0000","last_unstale":"2024-03-22T04:15:10.090474+0000","last_undegraded":"2024-03-22T04:15:10.090474+0000","last_fullsized":"2024-03-22T04:15:10.090474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:31:42.571291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.099265+0000","last_change":"2024-03-22T04:15:09.071764+0000","last_active":"2024-03-22T04:15:10.099265+0000","last_peered":"2024-03-22T04:15:10.099265+0000","last_clean":"2024-03-22T04:15:10.099265+0000","last_became_active":"2024-03-22T04:15:09.071518+0000","last_became_peered":"2024-03-22T04:15:09.071518+0000","last_unstale":"2024-03-22T04:15:10.099265+0000","last_undegraded":"2024-03-22T04:15:10.099265+0000","last_fullsized":"2024-03-22T04:15:10.099265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:03:41.138335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.094983+0000","last_change":"2024-03-22T04:15:10.090363+0000","last_active":"2024-03-22T04:15:10.094983+0000","last_peered":"2024-03-22T04:15:10.094983+0000","last_clean":"2024-03-22T04:15:10.094983+0000","last_became_active":"2024-03-22T04:15:10.090052+0000","last_became_peered":"2024-03-22T04:15:10.090052+0000","last_unstale":"2024-03-22T04:15:10.094983+0000","last_undegraded":"2024-03-22T04:15:10.094983+0000","last_fullsized":"2024-03-22T04:15:10.094983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:39:45.020238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.101958+0000","last_change":"2024-03-22T04:15:09.076848+0000","last_active":"2024-03-22T04:15:10.101958+0000","last_peered":"2024-03-22T04:15:10.101958+0000","last_clean":"2024-03-22T04:15:10.101958+0000","last_became_active":"2024-03-22T04:15:09.076533+0000","last_became_peered":"2024-03-22T04:15:09.076533+0000","last_unstale":"2024-03-22T04:15:10.101958+0000","last_undegraded":"2024-03-22T04:15:10.101958+0000","last_fullsized":"2024-03-22T04:15:10.101958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:14:11.346993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088286+0000","last_change":"2024-03-22T04:15:10.083119+0000","last_active":"2024-03-22T04:15:10.088286+0000","last_peered":"2024-03-22T04:15:10.088286+0000","last_clean":"2024-03-22T04:15:10.088286+0000","last_became_active":"2024-03-22T04:15:10.079664+0000","last_became_peered":"2024-03-22T04:15:10.079664+0000","last_unstale":"2024-03-22T04:15:10.088286+0000","last_undegraded":"2024-03-22T04:15:10.088286+0000","last_fullsized":"2024-03-22T04:15:10.088286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:02:34.622890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.100991+0000","last_change":"2024-03-22T04:15:09.064032+0000","last_active":"2024-03-22T04:15:10.100991+0000","last_peered":"2024-03-22T04:15:10.100991+0000","last_clean":"2024-03-22T04:15:10.100991+0000","last_became_active":"2024-03-22T04:15:09.063769+0000","last_became_peered":"2024-03-22T04:15:09.063769+0000","last_unstale":"2024-03-22T04:15:10.100991+0000","last_undegraded":"2024-03-22T04:15:10.100991+0000","last_fullsized":"2024-03-22T04:15:10.100991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:36:33.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090658+0000","last_change":"2024-03-22T04:15:10.077747+0000","last_active":"2024-03-22T04:15:10.090658+0000","last_peered":"2024-03-22T04:15:10.090658+0000","last_clean":"2024-03-22T04:15:10.090658+0000","last_became_active":"2024-03-22T04:15:10.077458+0000","last_became_peered":"2024-03-22T04:15:10.077458+0000","last_unstale":"2024-03-22T04:15:10.090658+0000","last_undegraded":"2024-03-22T04:15:10.090658+0000","last_fullsized":"2024-03-22T04:15:10.090658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:36:34.966608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.102062+0000","last_change":"2024-03-22T04:15:09.064915+0000","last_active":"2024-03-22T04:15:10.102062+0000","last_peered":"2024-03-22T04:15:10.102062+0000","last_clean":"2024-03-22T04:15:10.102062+0000","last_became_active":"2024-03-22T04:15:09.063555+0000","last_became_peered":"2024-03-22T04:15:09.063555+0000","last_unstale":"2024-03-22T04:15:10.102062+0000","last_undegraded":"2024-03-22T04:15:10.102062+0000","last_fullsized":"2024-03-22T04:15:10.102062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:16.218419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.085242+0000","last_change":"2024-03-22T04:15:10.081580+0000","last_active":"2024-03-22T04:15:10.085242+0000","last_peered":"2024-03-22T04:15:10.085242+0000","last_clean":"2024-03-22T04:15:10.085242+0000","last_became_active":"2024-03-22T04:15:10.080117+0000","last_became_peered":"2024-03-22T04:15:10.080117+0000","last_unstale":"2024-03-22T04:15:10.085242+0000","last_undegraded":"2024-03-22T04:15:10.085242+0000","last_fullsized":"2024-03-22T04:15:10.085242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:28:37.146208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.096836+0000","last_change":"2024-03-22T04:15:09.066910+0000","last_active":"2024-03-22T04:15:10.096836+0000","last_peered":"2024-03-22T04:15:10.096836+0000","last_clean":"2024-03-22T04:15:10.096836+0000","last_became_active":"2024-03-22T04:15:09.066609+0000","last_became_peered":"2024-03-22T04:15:09.066609+0000","last_unstale":"2024-03-22T04:15:10.096836+0000","last_undegraded":"2024-03-22T04:15:10.096836+0000","last_fullsized":"2024-03-22T04:15:10.096836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:00:43.098673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090408+0000","last_change":"2024-03-22T04:15:10.079593+0000","last_active":"2024-03-22T04:15:10.090408+0000","last_peered":"2024-03-22T04:15:10.090408+0000","last_clean":"2024-03-22T04:15:10.090408+0000","last_became_active":"2024-03-22T04:15:10.077268+0000","last_became_peered":"2024-03-22T04:15:10.077268+0000","last_unstale":"2024-03-22T04:15:10.090408+0000","last_undegraded":"2024-03-22T04:15:10.090408+0000","last_fullsized":"2024-03-22T04:15:10.090408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:18:07.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.062924+0000","last_change":"2024-03-22T04:15:09.071999+0000","last_active":"2024-03-22T04:15:11.062924+0000","last_peered":"2024-03-22T04:15:11.062924+0000","last_clean":"2024-03-22T04:15:11.062924+0000","last_became_active":"2024-03-22T04:15:09.071682+0000","last_became_peered":"2024-03-22T04:15:09.071682+0000","last_unstale":"2024-03-22T04:15:11.062924+0000","last_undegraded":"2024-03-22T04:15:11.062924+0000","last_fullsized":"2024-03-22T04:15:11.062924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:19:57.127216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.093700+0000","last_change":"2024-03-22T04:15:10.092143+0000","last_active":"2024-03-22T04:15:10.093700+0000","last_peered":"2024-03-22T04:15:10.093700+0000","last_clean":"2024-03-22T04:15:10.093700+0000","last_became_active":"2024-03-22T04:15:10.089295+0000","last_became_peered":"2024-03-22T04:15:10.089295+0000","last_unstale":"2024-03-22T04:15:10.093700+0000","last_undegraded":"2024-03-22T04:15:10.093700+0000","last_fullsized":"2024-03-22T04:15:10.093700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:43.492391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069495+0000","last_change":"2024-03-22T04:15:09.069492+0000","last_active":"2024-03-22T04:15:10.069495+0000","last_peered":"2024-03-22T04:15:10.069495+0000","last_clean":"2024-03-22T04:15:10.069495+0000","last_became_active":"2024-03-22T04:15:09.069042+0000","last_became_peered":"2024-03-22T04:15:09.069042+0000","last_unstale":"2024-03-22T04:15:10.069495+0000","last_undegraded":"2024-03-22T04:15:10.069495+0000","last_fullsized":"2024-03-22T04:15:10.069495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:58:31.525007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.084937+0000","last_change":"2024-03-22T04:15:10.081630+0000","last_active":"2024-03-22T04:15:10.084937+0000","last_peered":"2024-03-22T04:15:10.084937+0000","last_clean":"2024-03-22T04:15:10.084937+0000","last_became_active":"2024-03-22T04:15:10.081219+0000","last_became_peered":"2024-03-22T04:15:10.081219+0000","last_unstale":"2024-03-22T04:15:10.084937+0000","last_undegraded":"2024-03-22T04:15:10.084937+0000","last_fullsized":"2024-03-22T04:15:10.084937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:05:05.092437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069972+0000","last_change":"2024-03-22T04:15:09.071929+0000","last_active":"2024-03-22T04:15:10.069972+0000","last_peered":"2024-03-22T04:15:10.069972+0000","last_clean":"2024-03-22T04:15:10.069972+0000","last_became_active":"2024-03-22T04:15:09.071329+0000","last_became_peered":"2024-03-22T04:15:09.071329+0000","last_unstale":"2024-03-22T04:15:10.069972+0000","last_undegraded":"2024-03-22T04:15:10.069972+0000","last_fullsized":"2024-03-22T04:15:10.069972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:33:43.274842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.078364+0000","last_change":"2024-03-22T04:15:10.074473+0000","last_active":"2024-03-22T04:15:10.078364+0000","last_peered":"2024-03-22T04:15:10.078364+0000","last_clean":"2024-03-22T04:15:10.078364+0000","last_became_active":"2024-03-22T04:15:10.074185+0000","last_became_peered":"2024-03-22T04:15:10.074185+0000","last_unstale":"2024-03-22T04:15:10.078364+0000","last_undegraded":"2024-03-22T04:15:10.078364+0000","last_fullsized":"2024-03-22T04:15:10.078364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:33:25.874350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.060208+0000","last_change":"2024-03-22T04:15:09.074718+0000","last_active":"2024-03-22T04:15:10.060208+0000","last_peered":"2024-03-22T04:15:10.060208+0000","last_clean":"2024-03-22T04:15:10.060208+0000","last_became_active":"2024-03-22T04:15:09.072279+0000","last_became_peered":"2024-03-22T04:15:09.072279+0000","last_unstale":"2024-03-22T04:15:10.060208+0000","last_undegraded":"2024-03-22T04:15:10.060208+0000","last_fullsized":"2024-03-22T04:15:10.060208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:05.566853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063549+0000","last_change":"2024-03-22T04:15:10.080610+0000","last_active":"2024-03-22T04:15:11.063549+0000","last_peered":"2024-03-22T04:15:11.063549+0000","last_clean":"2024-03-22T04:15:11.063549+0000","last_became_active":"2024-03-22T04:15:10.079916+0000","last_became_peered":"2024-03-22T04:15:10.079916+0000","last_unstale":"2024-03-22T04:15:11.063549+0000","last_undegraded":"2024-03-22T04:15:11.063549+0000","last_fullsized":"2024-03-22T04:15:11.063549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:55:45.277621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.104900+0000","last_change":"2024-03-22T04:15:09.072031+0000","last_active":"2024-03-22T04:15:10.104900+0000","last_peered":"2024-03-22T04:15:10.104900+0000","last_clean":"2024-03-22T04:15:10.104900+0000","last_became_active":"2024-03-22T04:15:09.071759+0000","last_became_peered":"2024-03-22T04:15:09.071759+0000","last_unstale":"2024-03-22T04:15:10.104900+0000","last_undegraded":"2024-03-22T04:15:10.104900+0000","last_fullsized":"2024-03-22T04:15:10.104900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:28:26.401190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089531+0000","last_change":"2024-03-22T04:15:10.077371+0000","last_active":"2024-03-22T04:15:10.089531+0000","last_peered":"2024-03-22T04:15:10.089531+0000","last_clean":"2024-03-22T04:15:10.089531+0000","last_became_active":"2024-03-22T04:15:10.077086+0000","last_became_peered":"2024-03-22T04:15:10.077086+0000","last_unstale":"2024-03-22T04:15:10.089531+0000","last_undegraded":"2024-03-22T04:15:10.089531+0000","last_fullsized":"2024-03-22T04:15:10.089531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:29.599987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.108307+0000","last_change":"2024-03-22T04:15:09.074220+0000","last_active":"2024-03-22T04:15:10.108307+0000","last_peered":"2024-03-22T04:15:10.108307+0000","last_clean":"2024-03-22T04:15:10.108307+0000","last_became_active":"2024-03-22T04:15:09.073974+0000","last_became_peered":"2024-03-22T04:15:09.073974+0000","last_unstale":"2024-03-22T04:15:10.108307+0000","last_undegraded":"2024-03-22T04:15:10.108307+0000","last_fullsized":"2024-03-22T04:15:10.108307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:40:14.849742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083391+0000","last_change":"2024-03-22T04:15:10.072948+0000","last_active":"2024-03-22T04:15:10.083391+0000","last_peered":"2024-03-22T04:15:10.083391+0000","last_clean":"2024-03-22T04:15:10.083391+0000","last_became_active":"2024-03-22T04:15:10.072665+0000","last_became_peered":"2024-03-22T04:15:10.072665+0000","last_unstale":"2024-03-22T04:15:10.083391+0000","last_undegraded":"2024-03-22T04:15:10.083391+0000","last_fullsized":"2024-03-22T04:15:10.083391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:55.903538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.105835+0000","last_change":"2024-03-22T04:15:09.071696+0000","last_active":"2024-03-22T04:15:10.105835+0000","last_peered":"2024-03-22T04:15:10.105835+0000","last_clean":"2024-03-22T04:15:10.105835+0000","last_became_active":"2024-03-22T04:15:09.071423+0000","last_became_peered":"2024-03-22T04:15:09.071423+0000","last_unstale":"2024-03-22T04:15:10.105835+0000","last_undegraded":"2024-03-22T04:15:10.105835+0000","last_fullsized":"2024-03-22T04:15:10.105835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:04:53.655257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092601+0000","last_change":"2024-03-22T04:15:10.088144+0000","last_active":"2024-03-22T04:15:10.092601+0000","last_peered":"2024-03-22T04:15:10.092601+0000","last_clean":"2024-03-22T04:15:10.092601+0000","last_became_active":"2024-03-22T04:15:10.087887+0000","last_became_peered":"2024-03-22T04:15:10.087887+0000","last_unstale":"2024-03-22T04:15:10.092601+0000","last_undegraded":"2024-03-22T04:15:10.092601+0000","last_fullsized":"2024-03-22T04:15:10.092601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:58:32.530161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.066108+0000","last_change":"2024-03-22T04:15:09.073275+0000","last_active":"2024-03-22T04:15:10.066108+0000","last_peered":"2024-03-22T04:15:10.066108+0000","last_clean":"2024-03-22T04:15:10.066108+0000","last_became_active":"2024-03-22T04:15:09.073002+0000","last_became_peered":"2024-03-22T04:15:09.073002+0000","last_unstale":"2024-03-22T04:15:10.066108+0000","last_undegraded":"2024-03-22T04:15:10.066108+0000","last_fullsized":"2024-03-22T04:15:10.066108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:28.594833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090432+0000","last_change":"2024-03-22T04:15:10.079907+0000","last_active":"2024-03-22T04:15:10.090432+0000","last_peered":"2024-03-22T04:15:10.090432+0000","last_clean":"2024-03-22T04:15:10.090432+0000","last_became_active":"2024-03-22T04:15:10.077157+0000","last_became_peered":"2024-03-22T04:15:10.077157+0000","last_unstale":"2024-03-22T04:15:10.090432+0000","last_undegraded":"2024-03-22T04:15:10.090432+0000","last_fullsized":"2024-03-22T04:15:10.090432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.491834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.072139+0000","last_change":"2024-03-22T04:15:09.075340+0000","last_active":"2024-03-22T04:15:10.072139+0000","last_peered":"2024-03-22T04:15:10.072139+0000","last_clean":"2024-03-22T04:15:10.072139+0000","last_became_active":"2024-03-22T04:15:09.075038+0000","last_became_peered":"2024-03-22T04:15:09.075038+0000","last_unstale":"2024-03-22T04:15:10.072139+0000","last_undegraded":"2024-03-22T04:15:10.072139+0000","last_fullsized":"2024-03-22T04:15:10.072139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:56:53.043203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.078265+0000","last_change":"2024-03-22T04:15:10.071029+0000","last_active":"2024-03-22T04:15:10.078265+0000","last_peered":"2024-03-22T04:15:10.078265+0000","last_clean":"2024-03-22T04:15:10.078265+0000","last_became_active":"2024-03-22T04:15:10.069833+0000","last_became_peered":"2024-03-22T04:15:10.069833+0000","last_unstale":"2024-03-22T04:15:10.078265+0000","last_undegraded":"2024-03-22T04:15:10.078265+0000","last_fullsized":"2024-03-22T04:15:10.078265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:10:10.851197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.104916+0000","last_change":"2024-03-22T04:15:09.067979+0000","last_active":"2024-03-22T04:15:10.104916+0000","last_peered":"2024-03-22T04:15:10.104916+0000","last_clean":"2024-03-22T04:15:10.104916+0000","last_became_active":"2024-03-22T04:15:09.067662+0000","last_became_peered":"2024-03-22T04:15:09.067662+0000","last_unstale":"2024-03-22T04:15:10.104916+0000","last_undegraded":"2024-03-22T04:15:10.104916+0000","last_fullsized":"2024-03-22T04:15:10.104916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:56:30.581574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090383+0000","last_change":"2024-03-22T04:15:10.087746+0000","last_active":"2024-03-22T04:15:10.090383+0000","last_peered":"2024-03-22T04:15:10.090383+0000","last_clean":"2024-03-22T04:15:10.090383+0000","last_became_active":"2024-03-22T04:15:10.087456+0000","last_became_peered":"2024-03-22T04:15:10.087456+0000","last_unstale":"2024-03-22T04:15:10.090383+0000","last_undegraded":"2024-03-22T04:15:10.090383+0000","last_fullsized":"2024-03-22T04:15:10.090383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:14:12.352147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.097912+0000","last_change":"2024-03-22T04:15:09.075102+0000","last_active":"2024-03-22T04:15:10.097912+0000","last_peered":"2024-03-22T04:15:10.097912+0000","last_clean":"2024-03-22T04:15:10.097912+0000","last_became_active":"2024-03-22T04:15:09.074118+0000","last_became_peered":"2024-03-22T04:15:09.074118+0000","last_unstale":"2024-03-22T04:15:10.097912+0000","last_undegraded":"2024-03-22T04:15:10.097912+0000","last_fullsized":"2024-03-22T04:15:10.097912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:13.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.084841+0000","last_change":"2024-03-22T04:15:10.080375+0000","last_active":"2024-03-22T04:15:10.084841+0000","last_peered":"2024-03-22T04:15:10.084841+0000","last_clean":"2024-03-22T04:15:10.084841+0000","last_became_active":"2024-03-22T04:15:10.080011+0000","last_became_peered":"2024-03-22T04:15:10.080011+0000","last_unstale":"2024-03-22T04:15:10.084841+0000","last_undegraded":"2024-03-22T04:15:10.084841+0000","last_fullsized":"2024-03-22T04:15:10.084841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:20:24.969370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.097684+0000","last_change":"2024-03-22T04:15:09.073727+0000","last_active":"2024-03-22T04:15:10.097684+0000","last_peered":"2024-03-22T04:15:10.097684+0000","last_clean":"2024-03-22T04:15:10.097684+0000","last_became_active":"2024-03-22T04:15:09.073420+0000","last_became_peered":"2024-03-22T04:15:09.073420+0000","last_unstale":"2024-03-22T04:15:10.097684+0000","last_undegraded":"2024-03-22T04:15:10.097684+0000","last_fullsized":"2024-03-22T04:15:10.097684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:41:46.638646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063350+0000","last_change":"2024-03-22T04:15:10.072479+0000","last_active":"2024-03-22T04:15:11.063350+0000","last_peered":"2024-03-22T04:15:11.063350+0000","last_clean":"2024-03-22T04:15:11.063350+0000","last_became_active":"2024-03-22T04:15:10.072192+0000","last_became_peered":"2024-03-22T04:15:10.072192+0000","last_unstale":"2024-03-22T04:15:11.063350+0000","last_undegraded":"2024-03-22T04:15:11.063350+0000","last_fullsized":"2024-03-22T04:15:11.063350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:09.358200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.100656+0000","last_change":"2024-03-22T04:15:09.064410+0000","last_active":"2024-03-22T04:15:10.100656+0000","last_peered":"2024-03-22T04:15:10.100656+0000","last_clean":"2024-03-22T04:15:10.100656+0000","last_became_active":"2024-03-22T04:15:09.064086+0000","last_became_peered":"2024-03-22T04:15:09.064086+0000","last_unstale":"2024-03-22T04:15:10.100656+0000","last_undegraded":"2024-03-22T04:15:10.100656+0000","last_fullsized":"2024-03-22T04:15:10.100656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:41:30.592585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.095176+0000","last_change":"2024-03-22T04:15:10.090296+0000","last_active":"2024-03-22T04:15:10.095176+0000","last_peered":"2024-03-22T04:15:10.095176+0000","last_clean":"2024-03-22T04:15:10.095176+0000","last_became_active":"2024-03-22T04:15:10.089812+0000","last_became_peered":"2024-03-22T04:15:10.089812+0000","last_unstale":"2024-03-22T04:15:10.095176+0000","last_undegraded":"2024-03-22T04:15:10.095176+0000","last_fullsized":"2024-03-22T04:15:10.095176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:16.933942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.071277+0000","last_change":"2024-03-22T04:15:09.071831+0000","last_active":"2024-03-22T04:15:10.071277+0000","last_peered":"2024-03-22T04:15:10.071277+0000","last_clean":"2024-03-22T04:15:10.071277+0000","last_became_active":"2024-03-22T04:15:09.071559+0000","last_became_peered":"2024-03-22T04:15:09.071559+0000","last_unstale":"2024-03-22T04:15:10.071277+0000","last_undegraded":"2024-03-22T04:15:10.071277+0000","last_fullsized":"2024-03-22T04:15:10.071277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:14:40.493536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.095207+0000","last_change":"2024-03-22T04:15:10.092218+0000","last_active":"2024-03-22T04:15:10.095207+0000","last_peered":"2024-03-22T04:15:10.095207+0000","last_clean":"2024-03-22T04:15:10.095207+0000","last_became_active":"2024-03-22T04:15:10.091929+0000","last_became_peered":"2024-03-22T04:15:10.091929+0000","last_unstale":"2024-03-22T04:15:10.095207+0000","last_undegraded":"2024-03-22T04:15:10.095207+0000","last_fullsized":"2024-03-22T04:15:10.095207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:10:10.851197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.100076+0000","last_change":"2024-03-22T04:15:09.068761+0000","last_active":"2024-03-22T04:15:10.100076+0000","last_peered":"2024-03-22T04:15:10.100076+0000","last_clean":"2024-03-22T04:15:10.100076+0000","last_became_active":"2024-03-22T04:15:09.068488+0000","last_became_peered":"2024-03-22T04:15:09.068488+0000","last_unstale":"2024-03-22T04:15:10.100076+0000","last_undegraded":"2024-03-22T04:15:10.100076+0000","last_fullsized":"2024-03-22T04:15:10.100076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:58:38.332903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.085330+0000","last_change":"2024-03-22T04:15:10.077342+0000","last_active":"2024-03-22T04:15:10.085330+0000","last_peered":"2024-03-22T04:15:10.085330+0000","last_clean":"2024-03-22T04:15:10.085330+0000","last_became_active":"2024-03-22T04:15:10.076964+0000","last_became_peered":"2024-03-22T04:15:10.076964+0000","last_unstale":"2024-03-22T04:15:10.085330+0000","last_undegraded":"2024-03-22T04:15:10.085330+0000","last_fullsized":"2024-03-22T04:15:10.085330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:03:17.693683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063046+0000","last_change":"2024-03-22T04:15:09.070075+0000","last_active":"2024-03-22T04:15:11.063046+0000","last_peered":"2024-03-22T04:15:11.063046+0000","last_clean":"2024-03-22T04:15:11.063046+0000","last_became_active":"2024-03-22T04:15:09.069803+0000","last_became_peered":"2024-03-22T04:15:09.069803+0000","last_unstale":"2024-03-22T04:15:11.063046+0000","last_undegraded":"2024-03-22T04:15:11.063046+0000","last_fullsized":"2024-03-22T04:15:11.063046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:19:17.186232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092285+0000","last_change":"2024-03-22T04:15:10.085272+0000","last_active":"2024-03-22T04:15:10.092285+0000","last_peered":"2024-03-22T04:15:10.092285+0000","last_clean":"2024-03-22T04:15:10.092285+0000","last_became_active":"2024-03-22T04:15:10.084757+0000","last_became_peered":"2024-03-22T04:15:10.084757+0000","last_unstale":"2024-03-22T04:15:10.092285+0000","last_undegraded":"2024-03-22T04:15:10.092285+0000","last_fullsized":"2024-03-22T04:15:10.092285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:22:14.987034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069646+0000","last_change":"2024-03-22T04:15:09.073776+0000","last_active":"2024-03-22T04:15:10.069646+0000","last_peered":"2024-03-22T04:15:10.069646+0000","last_clean":"2024-03-22T04:15:10.069646+0000","last_became_active":"2024-03-22T04:15:09.073509+0000","last_became_peered":"2024-03-22T04:15:09.073509+0000","last_unstale":"2024-03-22T04:15:10.069646+0000","last_undegraded":"2024-03-22T04:15:10.069646+0000","last_fullsized":"2024-03-22T04:15:10.069646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:37:51.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083740+0000","last_change":"2024-03-22T04:15:10.073405+0000","last_active":"2024-03-22T04:15:10.083740+0000","last_peered":"2024-03-22T04:15:10.083740+0000","last_clean":"2024-03-22T04:15:10.083740+0000","last_became_active":"2024-03-22T04:15:10.072803+0000","last_became_peered":"2024-03-22T04:15:10.072803+0000","last_unstale":"2024-03-22T04:15:10.083740+0000","last_undegraded":"2024-03-22T04:15:10.083740+0000","last_fullsized":"2024-03-22T04:15:10.083740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:31.358038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.104834+0000","last_change":"2024-03-22T04:15:09.070795+0000","last_active":"2024-03-22T04:15:10.104834+0000","last_peered":"2024-03-22T04:15:10.104834+0000","last_clean":"2024-03-22T04:15:10.104834+0000","last_became_active":"2024-03-22T04:15:09.070537+0000","last_became_peered":"2024-03-22T04:15:09.070537+0000","last_unstale":"2024-03-22T04:15:10.104834+0000","last_undegraded":"2024-03-22T04:15:10.104834+0000","last_fullsized":"2024-03-22T04:15:10.104834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:41:46.638646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090271+0000","last_change":"2024-03-22T04:15:10.085620+0000","last_active":"2024-03-22T04:15:10.090271+0000","last_peered":"2024-03-22T04:15:10.090271+0000","last_clean":"2024-03-22T04:15:10.090271+0000","last_became_active":"2024-03-22T04:15:10.085255+0000","last_became_peered":"2024-03-22T04:15:10.085255+0000","last_unstale":"2024-03-22T04:15:10.090271+0000","last_undegraded":"2024-03-22T04:15:10.090271+0000","last_fullsized":"2024-03-22T04:15:10.090271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:55.903538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.063452+0000","last_change":"2024-03-22T04:15:09.072099+0000","last_active":"2024-03-22T04:15:10.063452+0000","last_peered":"2024-03-22T04:15:10.063452+0000","last_clean":"2024-03-22T04:15:10.063452+0000","last_became_active":"2024-03-22T04:15:09.071839+0000","last_became_peered":"2024-03-22T04:15:09.071839+0000","last_unstale":"2024-03-22T04:15:10.063452+0000","last_undegraded":"2024-03-22T04:15:10.063452+0000","last_fullsized":"2024-03-22T04:15:10.063452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:47:56.139069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.097700+0000","last_change":"2024-03-22T04:15:10.092371+0000","last_active":"2024-03-22T04:15:10.097700+0000","last_peered":"2024-03-22T04:15:10.097700+0000","last_clean":"2024-03-22T04:15:10.097700+0000","last_became_active":"2024-03-22T04:15:10.092086+0000","last_became_peered":"2024-03-22T04:15:10.092086+0000","last_unstale":"2024-03-22T04:15:10.097700+0000","last_undegraded":"2024-03-22T04:15:10.097700+0000","last_fullsized":"2024-03-22T04:15:10.097700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:23:46.775958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.060483+0000","last_change":"2024-03-22T04:15:09.071706+0000","last_active":"2024-03-22T04:15:10.060483+0000","last_peered":"2024-03-22T04:15:10.060483+0000","last_clean":"2024-03-22T04:15:10.060483+0000","last_became_active":"2024-03-22T04:15:09.071433+0000","last_became_peered":"2024-03-22T04:15:09.071433+0000","last_unstale":"2024-03-22T04:15:10.060483+0000","last_undegraded":"2024-03-22T04:15:10.060483+0000","last_fullsized":"2024-03-22T04:15:10.060483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:31:41.566136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063472+0000","last_change":"2024-03-22T04:15:10.081684+0000","last_active":"2024-03-22T04:15:11.063472+0000","last_peered":"2024-03-22T04:15:11.063472+0000","last_clean":"2024-03-22T04:15:11.063472+0000","last_became_active":"2024-03-22T04:15:10.081204+0000","last_became_peered":"2024-03-22T04:15:10.081204+0000","last_unstale":"2024-03-22T04:15:11.063472+0000","last_undegraded":"2024-03-22T04:15:11.063472+0000","last_fullsized":"2024-03-22T04:15:11.063472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:24:25.797823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069580+0000","last_change":"2024-03-22T04:15:09.068727+0000","last_active":"2024-03-22T04:15:10.069580+0000","last_peered":"2024-03-22T04:15:10.069580+0000","last_clean":"2024-03-22T04:15:10.069580+0000","last_became_active":"2024-03-22T04:15:09.068464+0000","last_became_peered":"2024-03-22T04:15:09.068464+0000","last_unstale":"2024-03-22T04:15:10.069580+0000","last_undegraded":"2024-03-22T04:15:10.069580+0000","last_fullsized":"2024-03-22T04:15:10.069580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:10:09.846043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088523+0000","last_change":"2024-03-22T04:15:10.082282+0000","last_active":"2024-03-22T04:15:10.088523+0000","last_peered":"2024-03-22T04:15:10.088523+0000","last_clean":"2024-03-22T04:15:10.088523+0000","last_became_active":"2024-03-22T04:15:10.081458+0000","last_became_peered":"2024-03-22T04:15:10.081458+0000","last_unstale":"2024-03-22T04:15:10.088523+0000","last_undegraded":"2024-03-22T04:15:10.088523+0000","last_fullsized":"2024-03-22T04:15:10.088523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:08.175064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.066145+0000","last_change":"2024-03-22T04:15:09.068628+0000","last_active":"2024-03-22T04:15:10.066145+0000","last_peered":"2024-03-22T04:15:10.066145+0000","last_clean":"2024-03-22T04:15:10.066145+0000","last_became_active":"2024-03-22T04:15:09.068316+0000","last_became_peered":"2024-03-22T04:15:09.068316+0000","last_unstale":"2024-03-22T04:15:10.066145+0000","last_undegraded":"2024-03-22T04:15:10.066145+0000","last_fullsized":"2024-03-22T04:15:10.066145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:53:59.753788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.094698+0000","last_change":"2024-03-22T04:15:10.090200+0000","last_active":"2024-03-22T04:15:10.094698+0000","last_peered":"2024-03-22T04:15:10.094698+0000","last_clean":"2024-03-22T04:15:10.094698+0000","last_became_active":"2024-03-22T04:15:10.086625+0000","last_became_peered":"2024-03-22T04:15:10.086625+0000","last_unstale":"2024-03-22T04:15:10.094698+0000","last_undegraded":"2024-03-22T04:15:10.094698+0000","last_fullsized":"2024-03-22T04:15:10.094698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:05:05.092437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.063555+0000","last_change":"2024-03-22T04:15:09.068790+0000","last_active":"2024-03-22T04:15:10.063555+0000","last_peered":"2024-03-22T04:15:10.063555+0000","last_clean":"2024-03-22T04:15:10.063555+0000","last_became_active":"2024-03-22T04:15:09.068490+0000","last_became_peered":"2024-03-22T04:15:09.068490+0000","last_unstale":"2024-03-22T04:15:10.063555+0000","last_undegraded":"2024-03-22T04:15:10.063555+0000","last_fullsized":"2024-03-22T04:15:10.063555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:54:48.956024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.093441+0000","last_change":"2024-03-22T04:15:10.088838+0000","last_active":"2024-03-22T04:15:10.093441+0000","last_peered":"2024-03-22T04:15:10.093441+0000","last_clean":"2024-03-22T04:15:10.093441+0000","last_became_active":"2024-03-22T04:15:10.086301+0000","last_became_peered":"2024-03-22T04:15:10.086301+0000","last_unstale":"2024-03-22T04:15:10.093441+0000","last_undegraded":"2024-03-22T04:15:10.093441+0000","last_fullsized":"2024-03-22T04:15:10.093441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:54:49.961178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069364+0000","last_change":"2024-03-22T04:15:09.071117+0000","last_active":"2024-03-22T04:15:10.069364+0000","last_peered":"2024-03-22T04:15:10.069364+0000","last_clean":"2024-03-22T04:15:10.069364+0000","last_became_active":"2024-03-22T04:15:09.070800+0000","last_became_peered":"2024-03-22T04:15:09.070800+0000","last_unstale":"2024-03-22T04:15:10.069364+0000","last_undegraded":"2024-03-22T04:15:10.069364+0000","last_fullsized":"2024-03-22T04:15:10.069364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:15.928788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088551+0000","last_change":"2024-03-22T04:15:10.084958+0000","last_active":"2024-03-22T04:15:10.088551+0000","last_peered":"2024-03-22T04:15:10.088551+0000","last_clean":"2024-03-22T04:15:10.088551+0000","last_became_active":"2024-03-22T04:15:10.082658+0000","last_became_peered":"2024-03-22T04:15:10.082658+0000","last_unstale":"2024-03-22T04:15:10.088551+0000","last_undegraded":"2024-03-22T04:15:10.088551+0000","last_fullsized":"2024-03-22T04:15:10.088551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:06.572007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069817+0000","last_change":"2024-03-22T04:15:09.065130+0000","last_active":"2024-03-22T04:15:10.069817+0000","last_peered":"2024-03-22T04:15:10.069817+0000","last_clean":"2024-03-22T04:15:10.069817+0000","last_became_active":"2024-03-22T04:15:09.064861+0000","last_became_peered":"2024-03-22T04:15:09.064861+0000","last_unstale":"2024-03-22T04:15:10.069817+0000","last_undegraded":"2024-03-22T04:15:10.069817+0000","last_fullsized":"2024-03-22T04:15:10.069817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:30.352884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.076756+0000","last_change":"2024-03-22T04:15:10.073842+0000","last_active":"2024-03-22T04:15:10.076756+0000","last_peered":"2024-03-22T04:15:10.076756+0000","last_clean":"2024-03-22T04:15:10.076756+0000","last_became_active":"2024-03-22T04:15:10.073563+0000","last_became_peered":"2024-03-22T04:15:10.073563+0000","last_unstale":"2024-03-22T04:15:10.076756+0000","last_undegraded":"2024-03-22T04:15:10.076756+0000","last_fullsized":"2024-03-22T04:15:10.076756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:02:03.207869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.063381+0000","last_change":"2024-03-22T04:15:09.068066+0000","last_active":"2024-03-22T04:15:10.063381+0000","last_peered":"2024-03-22T04:15:10.063381+0000","last_clean":"2024-03-22T04:15:10.063381+0000","last_became_active":"2024-03-22T04:15:09.067712+0000","last_became_peered":"2024-03-22T04:15:09.067712+0000","last_unstale":"2024-03-22T04:15:10.063381+0000","last_undegraded":"2024-03-22T04:15:10.063381+0000","last_fullsized":"2024-03-22T04:15:10.063381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:35:15.551999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":274,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.059668+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:15:10.059668+0000","last_peered":"2024-03-22T04:15:10.059668+0000","last_clean":"2024-03-22T04:15:10.059668+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:15:10.059668+0000","last_undegraded":"2024-03-22T04:15:10.059668+0000","last_fullsized":"2024-03-22T04:15:10.059668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":435991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":428172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":1025797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509501,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":437055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":958464,"data_stored":435339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":1018540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":435361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":442053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:15:14.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:15:14.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e49800640 1 Processor -- start 2024-03-22T04:15:14.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e49800640 1 -- start start 2024-03-22T04:15:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e49800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e440d0580 0x7f3e44071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e49800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4406e2f0 0x7f3e44071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e49800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e441623d0 con 0x7f3e4415e630 2024-03-22T04:15:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e49800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e441620f0 con 0x7f3e440d0580 2024-03-22T04:15:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e49800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4415ec10 con 0x7f3e4406e2f0 2024-03-22T04:15:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e3be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e440d0580 0x7f3e44071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e3be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e440d0580 0x7f3e44071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43758/0 (socket says 172.21.15.86:43758) 2024-03-22T04:15:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e3be00640 1 -- 172.21.15.86:0/221908732 learned_addr learned my addr 172.21.15.86:0/221908732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.475+0000 7f3e43400640 1 --2- 172.21.15.86:0/221908732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4406e2f0 0x7f3e44071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.479+0000 7f3e3be00640 1 -- 172.21.15.86:0/221908732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4406e2f0 msgr2=0x7f3e44071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.479+0000 7f3e3be00640 1 --2- 172.21.15.86:0/221908732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4406e2f0 0x7f3e44071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.479+0000 7f3e3be00640 1 -- 172.21.15.86:0/221908732 >> v1:172.21.15.196:6789/0 conn(0x7f3e4415e630 legacy=0x7f3e440cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.479+0000 7f3e3be00640 1 -- 172.21.15.86:0/221908732 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4415eef0 con 0x7f3e440d0580 2024-03-22T04:15:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.479+0000 7f3e3be00640 1 --2- 172.21.15.86:0/221908732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e440d0580 0x7f3e44071100 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009870 tx=0x7f3e2c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3263797777e9fa07 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.479+0000 7f3e42000640 1 -- 172.21.15.86:0/221908732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0b2070 con 0x7f3e440d0580 2024-03-22T04:15:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.479+0000 7f3e42000640 1 -- 172.21.15.86:0/221908732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2c0a9070 con 0x7f3e440d0580 2024-03-22T04:15:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.479+0000 7f3e42000640 1 -- 172.21.15.86:0/221908732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0ad880 con 0x7f3e440d0580 2024-03-22T04:15:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- 172.21.15.86:0/221908732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e440d0580 msgr2=0x7f3e44071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 --2- 172.21.15.86:0/221908732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e440d0580 0x7f3e44071100 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009870 tx=0x7f3e2c01c560 comp rx=0 tx=0).stop 2024-03-22T04:15:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- 172.21.15.86:0/221908732 shutdown_connections 2024-03-22T04:15:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 --2- 172.21.15.86:0/221908732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4406e2f0 0x7f3e44071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 --2- 172.21.15.86:0/221908732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e440d0580 0x7f3e44071100 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- 172.21.15.86:0/221908732 >> 172.21.15.86:0/221908732 conn(0x7f3e4406f910 msgr2=0x7f3e4406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- 172.21.15.86:0/221908732 shutdown_connections 2024-03-22T04:15:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- 172.21.15.86:0/221908732 wait complete. 2024-03-22T04:15:14.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 Processor -- start 2024-03-22T04:15:14.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- start start 2024-03-22T04:15:14.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4406e2f0 0x7f3e440c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e440d0580 0x7f3e440c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e4415e630 0x7f3e440ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44071f90 con 0x7f3e4415e630 2024-03-22T04:15:14.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4415f9d0 con 0x7f3e4406e2f0 2024-03-22T04:15:14.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e49800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3e440cfaf0 con 0x7f3e440d0580 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e4415e630 0x7f3e440ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e4415e630 0x7f3e440ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43760/0 (socket says 172.21.15.86:43760) 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e43400640 1 -- 172.21.15.86:0/2545081571 learned_addr learned my addr 172.21.15.86:0/2545081571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e3be00640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e440d0580 0x7f3e440c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e42a00640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4406e2f0 0x7f3e440c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e43400640 1 -- 172.21.15.86:0/2545081571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e440d0580 msgr2=0x7f3e440c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e43400640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e440d0580 0x7f3e440c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e43400640 1 -- 172.21.15.86:0/2545081571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4406e2f0 msgr2=0x7f3e440c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e43400640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4406e2f0 0x7f3e440c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.483+0000 7f3e43400640 1 -- 172.21.15.86:0/2545081571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e440c7870 con 0x7f3e4415e630 2024-03-22T04:15:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e43400640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e4415e630 0x7f3e440ca6e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00efc0 tx=0x7f3e3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e3aa00640 1 -- 172.21.15.86:0/2545081571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c018020 con 0x7f3e4415e630 2024-03-22T04:15:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e3aa00640 1 -- 172.21.15.86:0/2545081571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3c014480 con 0x7f3e4415e630 2024-03-22T04:15:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e3aa00640 1 -- 172.21.15.86:0/2545081571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c014870 con 0x7f3e4415e630 2024-03-22T04:15:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e440c7e50 con 0x7f3e4415e630 2024-03-22T04:15:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e440c8410 con 0x7f3e4415e630 2024-03-22T04:15:14.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e440c8610 con 0x7f3e4415e630 2024-03-22T04:15:14.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e3aa00640 1 -- 172.21.15.86:0/2545081571 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3e3c01b020 con 0x7f3e4415e630 2024-03-22T04:15:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e20067b20 0x7f3e20069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e3aa00640 1 -- 172.21.15.86:0/2545081571 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f3e3c086b50 con 0x7f3e4415e630 2024-03-22T04:15:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.487+0000 7f3e42a00640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e20067b20 0x7f3e20069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.491+0000 7f3e42a00640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e20067b20 0x7f3e20069fe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3e4406d860 tx=0x7f3e34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.491+0000 7f3e3aa00640 1 -- 172.21.15.86:0/2545081571 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3e440c8610 con 0x7f3e4415e630 2024-03-22T04:15:14.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.643+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e44068650 con 0x7f3e20067b20 2024-03-22T04:15:14.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.651+0000 7f3e3aa00640 1 -- 172.21.15.86:0/2545081571 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252008 (secure 0 0 0) 0x7f3e44068650 con 0x7f3e20067b20 2024-03-22T04:15:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:15:14.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e20067b20 msgr2=0x7f3e20069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e20067b20 0x7f3e20069fe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3e4406d860 tx=0x7f3e34009040 comp rx=0 tx=0).stop 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e4415e630 msgr2=0x7f3e440ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e4415e630 0x7f3e440ca6e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00efc0 tx=0x7f3e3c00c490 comp rx=0 tx=0).stop 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 shutdown_connections 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e20067b20 0x7f3e20069fe0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e4415e630 0x7f3e440ca6e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e440d0580 0x7f3e440c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 --2- 172.21.15.86:0/2545081571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4406e2f0 0x7f3e440c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 >> 172.21.15.86:0/2545081571 conn(0x7f3e4406f910 msgr2=0x7f3e44156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 shutdown_connections 2024-03-22T04:15:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.655+0000 7f3e49800640 1 -- 172.21.15.86:0/2545081571 wait complete. 2024-03-22T04:15:14.677 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-03-22T04:15:13.853550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254008,"kb_used_data":8676,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720712,"statfs":{"total":773094113280,"available":772834009088,"internally_reserved":0,"allocated":8884224,"data_stored":4658308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.810041"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.085434+0000","last_change":"2024-03-22T04:15:10.082611+0000","last_active":"2024-03-22T04:15:10.085434+0000","last_peered":"2024-03-22T04:15:10.085434+0000","last_clean":"2024-03-22T04:15:10.085434+0000","last_became_active":"2024-03-22T04:15:10.082361+0000","last_became_peered":"2024-03-22T04:15:10.082361+0000","last_unstale":"2024-03-22T04:15:10.085434+0000","last_undegraded":"2024-03-22T04:15:10.085434+0000","last_fullsized":"2024-03-22T04:15:10.085434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:13.597157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.095052+0000","last_change":"2024-03-22T04:15:10.092417+0000","last_active":"2024-03-22T04:15:10.095052+0000","last_peered":"2024-03-22T04:15:10.095052+0000","last_clean":"2024-03-22T04:15:10.095052+0000","last_became_active":"2024-03-22T04:15:10.089025+0000","last_became_peered":"2024-03-22T04:15:10.089025+0000","last_unstale":"2024-03-22T04:15:10.095052+0000","last_undegraded":"2024-03-22T04:15:10.095052+0000","last_fullsized":"2024-03-22T04:15:10.095052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:29.599987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092363+0000","last_change":"2024-03-22T04:15:10.088024+0000","last_active":"2024-03-22T04:15:10.092363+0000","last_peered":"2024-03-22T04:15:10.092363+0000","last_clean":"2024-03-22T04:15:10.092363+0000","last_became_active":"2024-03-22T04:15:10.087566+0000","last_became_peered":"2024-03-22T04:15:10.087566+0000","last_unstale":"2024-03-22T04:15:10.092363+0000","last_undegraded":"2024-03-22T04:15:10.092363+0000","last_fullsized":"2024-03-22T04:15:10.092363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:47:57.144223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083640+0000","last_change":"2024-03-22T04:15:10.074035+0000","last_active":"2024-03-22T04:15:10.083640+0000","last_peered":"2024-03-22T04:15:10.083640+0000","last_clean":"2024-03-22T04:15:10.083640+0000","last_became_active":"2024-03-22T04:15:10.073772+0000","last_became_peered":"2024-03-22T04:15:10.073772+0000","last_unstale":"2024-03-22T04:15:10.083640+0000","last_undegraded":"2024-03-22T04:15:10.083640+0000","last_fullsized":"2024-03-22T04:15:10.083640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:58:39.338057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.086619+0000","last_change":"2024-03-22T04:15:10.081975+0000","last_active":"2024-03-22T04:15:10.086619+0000","last_peered":"2024-03-22T04:15:10.086619+0000","last_clean":"2024-03-22T04:15:10.086619+0000","last_became_active":"2024-03-22T04:15:10.081664+0000","last_became_peered":"2024-03-22T04:15:10.081664+0000","last_unstale":"2024-03-22T04:15:10.086619+0000","last_undegraded":"2024-03-22T04:15:10.086619+0000","last_fullsized":"2024-03-22T04:15:10.086619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:06:50.088884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088149+0000","last_change":"2024-03-22T04:15:10.075370+0000","last_active":"2024-03-22T04:15:10.088149+0000","last_peered":"2024-03-22T04:15:10.088149+0000","last_clean":"2024-03-22T04:15:10.088149+0000","last_became_active":"2024-03-22T04:15:10.074881+0000","last_became_peered":"2024-03-22T04:15:10.074881+0000","last_unstale":"2024-03-22T04:15:10.088149+0000","last_undegraded":"2024-03-22T04:15:10.088149+0000","last_fullsized":"2024-03-22T04:15:10.088149+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:17.223573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.082886+0000","last_change":"2024-03-22T04:15:10.070748+0000","last_active":"2024-03-22T04:15:10.082886+0000","last_peered":"2024-03-22T04:15:10.082886+0000","last_clean":"2024-03-22T04:15:10.082886+0000","last_became_active":"2024-03-22T04:15:10.070438+0000","last_became_peered":"2024-03-22T04:15:10.070438+0000","last_unstale":"2024-03-22T04:15:10.082886+0000","last_undegraded":"2024-03-22T04:15:10.082886+0000","last_fullsized":"2024-03-22T04:15:10.082886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:45:13.597157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063713+0000","last_change":"2024-03-22T04:15:10.073399+0000","last_active":"2024-03-22T04:15:11.063713+0000","last_peered":"2024-03-22T04:15:11.063713+0000","last_clean":"2024-03-22T04:15:11.063713+0000","last_became_active":"2024-03-22T04:15:10.073131+0000","last_became_peered":"2024-03-22T04:15:10.073131+0000","last_unstale":"2024-03-22T04:15:11.063713+0000","last_undegraded":"2024-03-22T04:15:11.063713+0000","last_fullsized":"2024-03-22T04:15:11.063713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:06:06.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089948+0000","last_change":"2024-03-22T04:15:10.084268+0000","last_active":"2024-03-22T04:15:10.089948+0000","last_peered":"2024-03-22T04:15:10.089948+0000","last_clean":"2024-03-22T04:15:10.089948+0000","last_became_active":"2024-03-22T04:15:10.084071+0000","last_became_peered":"2024-03-22T04:15:10.084071+0000","last_unstale":"2024-03-22T04:15:10.089948+0000","last_undegraded":"2024-03-22T04:15:10.089948+0000","last_fullsized":"2024-03-22T04:15:10.089948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:28:37.146208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.079902+0000","last_change":"2024-03-22T04:15:10.075269+0000","last_active":"2024-03-22T04:15:10.079902+0000","last_peered":"2024-03-22T04:15:10.079902+0000","last_clean":"2024-03-22T04:15:10.079902+0000","last_became_active":"2024-03-22T04:15:10.074981+0000","last_became_peered":"2024-03-22T04:15:10.074981+0000","last_unstale":"2024-03-22T04:15:10.079902+0000","last_undegraded":"2024-03-22T04:15:10.079902+0000","last_fullsized":"2024-03-22T04:15:10.079902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:33:44.279996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.085320+0000","last_change":"2024-03-22T04:15:10.080689+0000","last_active":"2024-03-22T04:15:10.085320+0000","last_peered":"2024-03-22T04:15:10.085320+0000","last_clean":"2024-03-22T04:15:10.085320+0000","last_became_active":"2024-03-22T04:15:10.080395+0000","last_became_peered":"2024-03-22T04:15:10.080395+0000","last_unstale":"2024-03-22T04:15:10.085320+0000","last_undegraded":"2024-03-22T04:15:10.085320+0000","last_fullsized":"2024-03-22T04:15:10.085320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:42:10.037211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083917+0000","last_change":"2024-03-22T04:15:10.078227+0000","last_active":"2024-03-22T04:15:10.083917+0000","last_peered":"2024-03-22T04:15:10.083917+0000","last_clean":"2024-03-22T04:15:10.083917+0000","last_became_active":"2024-03-22T04:15:10.077913+0000","last_became_peered":"2024-03-22T04:15:10.077913+0000","last_unstale":"2024-03-22T04:15:10.083917+0000","last_undegraded":"2024-03-22T04:15:10.083917+0000","last_fullsized":"2024-03-22T04:15:10.083917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:39:10.645964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.087686+0000","last_change":"2024-03-22T04:15:10.085091+0000","last_active":"2024-03-22T04:15:10.087686+0000","last_peered":"2024-03-22T04:15:10.087686+0000","last_clean":"2024-03-22T04:15:10.087686+0000","last_became_active":"2024-03-22T04:15:10.084732+0000","last_became_peered":"2024-03-22T04:15:10.084732+0000","last_unstale":"2024-03-22T04:15:10.087686+0000","last_undegraded":"2024-03-22T04:15:10.087686+0000","last_fullsized":"2024-03-22T04:15:10.087686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:56:31.586728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063300+0000","last_change":"2024-03-22T04:15:10.080842+0000","last_active":"2024-03-22T04:15:11.063300+0000","last_peered":"2024-03-22T04:15:11.063300+0000","last_clean":"2024-03-22T04:15:11.063300+0000","last_became_active":"2024-03-22T04:15:10.080355+0000","last_became_peered":"2024-03-22T04:15:10.080355+0000","last_unstale":"2024-03-22T04:15:11.063300+0000","last_undegraded":"2024-03-22T04:15:11.063300+0000","last_fullsized":"2024-03-22T04:15:11.063300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:06:34.989426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.093749+0000","last_change":"2024-03-22T04:15:10.088854+0000","last_active":"2024-03-22T04:15:10.093749+0000","last_peered":"2024-03-22T04:15:10.093749+0000","last_clean":"2024-03-22T04:15:10.093749+0000","last_became_active":"2024-03-22T04:15:10.088544+0000","last_became_peered":"2024-03-22T04:15:10.088544+0000","last_unstale":"2024-03-22T04:15:10.093749+0000","last_undegraded":"2024-03-22T04:15:10.093749+0000","last_fullsized":"2024-03-22T04:15:10.093749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:41:47.643800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088085+0000","last_change":"2024-03-22T04:15:10.076959+0000","last_active":"2024-03-22T04:15:10.088085+0000","last_peered":"2024-03-22T04:15:10.088085+0000","last_clean":"2024-03-22T04:15:10.088085+0000","last_became_active":"2024-03-22T04:15:10.076680+0000","last_became_peered":"2024-03-22T04:15:10.076680+0000","last_unstale":"2024-03-22T04:15:10.088085+0000","last_undegraded":"2024-03-22T04:15:10.088085+0000","last_fullsized":"2024-03-22T04:15:10.088085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:42:10.037211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063823+0000","last_change":"2024-03-22T04:15:10.080152+0000","last_active":"2024-03-22T04:15:11.063823+0000","last_peered":"2024-03-22T04:15:11.063823+0000","last_clean":"2024-03-22T04:15:11.063823+0000","last_became_active":"2024-03-22T04:15:10.079541+0000","last_became_peered":"2024-03-22T04:15:10.079541+0000","last_unstale":"2024-03-22T04:15:11.063823+0000","last_undegraded":"2024-03-22T04:15:11.063823+0000","last_fullsized":"2024-03-22T04:15:11.063823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:09:39.974023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092601+0000","last_change":"2024-03-22T04:15:10.087272+0000","last_active":"2024-03-22T04:15:10.092601+0000","last_peered":"2024-03-22T04:15:10.092601+0000","last_clean":"2024-03-22T04:15:10.092601+0000","last_became_active":"2024-03-22T04:15:10.084332+0000","last_became_peered":"2024-03-22T04:15:10.084332+0000","last_unstale":"2024-03-22T04:15:10.092601+0000","last_undegraded":"2024-03-22T04:15:10.092601+0000","last_fullsized":"2024-03-22T04:15:10.092601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:28:27.406345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063489+0000","last_change":"2024-03-22T04:15:10.075412+0000","last_active":"2024-03-22T04:15:11.063489+0000","last_peered":"2024-03-22T04:15:11.063489+0000","last_clean":"2024-03-22T04:15:11.063489+0000","last_became_active":"2024-03-22T04:15:10.075146+0000","last_became_peered":"2024-03-22T04:15:10.075146+0000","last_unstale":"2024-03-22T04:15:11.063489+0000","last_undegraded":"2024-03-22T04:15:11.063489+0000","last_fullsized":"2024-03-22T04:15:11.063489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:28:05.582341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090681+0000","last_change":"2024-03-22T04:15:10.078433+0000","last_active":"2024-03-22T04:15:10.090681+0000","last_peered":"2024-03-22T04:15:10.090681+0000","last_clean":"2024-03-22T04:15:10.090681+0000","last_became_active":"2024-03-22T04:15:10.078090+0000","last_became_peered":"2024-03-22T04:15:10.078090+0000","last_unstale":"2024-03-22T04:15:10.090681+0000","last_undegraded":"2024-03-22T04:15:10.090681+0000","last_fullsized":"2024-03-22T04:15:10.090681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:39:10.645964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.087037+0000","last_change":"2024-03-22T04:15:10.082472+0000","last_active":"2024-03-22T04:15:10.087037+0000","last_peered":"2024-03-22T04:15:10.087037+0000","last_clean":"2024-03-22T04:15:10.087037+0000","last_became_active":"2024-03-22T04:15:10.082209+0000","last_became_peered":"2024-03-22T04:15:10.082209+0000","last_unstale":"2024-03-22T04:15:10.087037+0000","last_undegraded":"2024-03-22T04:15:10.087037+0000","last_fullsized":"2024-03-22T04:15:10.087037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:17:10.391829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.094237+0000","last_change":"2024-03-22T04:15:10.091739+0000","last_active":"2024-03-22T04:15:10.094237+0000","last_peered":"2024-03-22T04:15:10.094237+0000","last_clean":"2024-03-22T04:15:10.094237+0000","last_became_active":"2024-03-22T04:15:10.091368+0000","last_became_peered":"2024-03-22T04:15:10.091368+0000","last_unstale":"2024-03-22T04:15:10.094237+0000","last_undegraded":"2024-03-22T04:15:10.094237+0000","last_fullsized":"2024-03-22T04:15:10.094237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:17.223573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.084217+0000","last_change":"2024-03-22T04:15:10.078117+0000","last_active":"2024-03-22T04:15:10.084217+0000","last_peered":"2024-03-22T04:15:10.084217+0000","last_clean":"2024-03-22T04:15:10.084217+0000","last_became_active":"2024-03-22T04:15:10.077855+0000","last_became_peered":"2024-03-22T04:15:10.077855+0000","last_unstale":"2024-03-22T04:15:10.084217+0000","last_undegraded":"2024-03-22T04:15:10.084217+0000","last_fullsized":"2024-03-22T04:15:10.084217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:14.447353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092316+0000","last_change":"2024-03-22T04:15:10.087888+0000","last_active":"2024-03-22T04:15:10.092316+0000","last_peered":"2024-03-22T04:15:10.092316+0000","last_clean":"2024-03-22T04:15:10.092316+0000","last_became_active":"2024-03-22T04:15:10.087600+0000","last_became_peered":"2024-03-22T04:15:10.087600+0000","last_unstale":"2024-03-22T04:15:10.092316+0000","last_undegraded":"2024-03-22T04:15:10.092316+0000","last_fullsized":"2024-03-22T04:15:10.092316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:35:16.557153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.093152+0000","last_change":"2024-03-22T04:15:10.085552+0000","last_active":"2024-03-22T04:15:10.093152+0000","last_peered":"2024-03-22T04:15:10.093152+0000","last_clean":"2024-03-22T04:15:10.093152+0000","last_became_active":"2024-03-22T04:15:10.085271+0000","last_became_peered":"2024-03-22T04:15:10.085271+0000","last_unstale":"2024-03-22T04:15:10.093152+0000","last_undegraded":"2024-03-22T04:15:10.093152+0000","last_fullsized":"2024-03-22T04:15:10.093152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:18:07.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083426+0000","last_change":"2024-03-22T04:15:10.073032+0000","last_active":"2024-03-22T04:15:10.083426+0000","last_peered":"2024-03-22T04:15:10.083426+0000","last_clean":"2024-03-22T04:15:10.083426+0000","last_became_active":"2024-03-22T04:15:10.072736+0000","last_became_peered":"2024-03-22T04:15:10.072736+0000","last_unstale":"2024-03-22T04:15:10.083426+0000","last_undegraded":"2024-03-22T04:15:10.083426+0000","last_fullsized":"2024-03-22T04:15:10.083426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:58:32.530161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089893+0000","last_change":"2024-03-22T04:15:10.075573+0000","last_active":"2024-03-22T04:15:10.089893+0000","last_peered":"2024-03-22T04:15:10.089893+0000","last_clean":"2024-03-22T04:15:10.089893+0000","last_became_active":"2024-03-22T04:15:10.075315+0000","last_became_peered":"2024-03-22T04:15:10.075315+0000","last_unstale":"2024-03-22T04:15:10.089893+0000","last_undegraded":"2024-03-22T04:15:10.089893+0000","last_fullsized":"2024-03-22T04:15:10.089893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:20:24.969370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089617+0000","last_change":"2024-03-22T04:15:10.083576+0000","last_active":"2024-03-22T04:15:10.089617+0000","last_peered":"2024-03-22T04:15:10.089617+0000","last_clean":"2024-03-22T04:15:10.089617+0000","last_became_active":"2024-03-22T04:15:10.082410+0000","last_became_peered":"2024-03-22T04:15:10.082410+0000","last_unstale":"2024-03-22T04:15:10.089617+0000","last_undegraded":"2024-03-22T04:15:10.089617+0000","last_fullsized":"2024-03-22T04:15:10.089617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:04:54.660411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089588+0000","last_change":"2024-03-22T04:15:10.078053+0000","last_active":"2024-03-22T04:15:10.089588+0000","last_peered":"2024-03-22T04:15:10.089588+0000","last_clean":"2024-03-22T04:15:10.089588+0000","last_became_active":"2024-03-22T04:15:10.077750+0000","last_became_peered":"2024-03-22T04:15:10.077750+0000","last_unstale":"2024-03-22T04:15:10.089588+0000","last_undegraded":"2024-03-22T04:15:10.089588+0000","last_fullsized":"2024-03-22T04:15:10.089588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:33:25.874350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.072581+0000","last_change":"2024-03-22T04:15:10.068372+0000","last_active":"2024-03-22T04:15:10.072581+0000","last_peered":"2024-03-22T04:15:10.072581+0000","last_clean":"2024-03-22T04:15:10.072581+0000","last_became_active":"2024-03-22T04:15:10.068119+0000","last_became_peered":"2024-03-22T04:15:10.068119+0000","last_unstale":"2024-03-22T04:15:10.072581+0000","last_undegraded":"2024-03-22T04:15:10.072581+0000","last_fullsized":"2024-03-22T04:15:10.072581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:16.933942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.084936+0000","last_change":"2024-03-22T04:15:10.081030+0000","last_active":"2024-03-22T04:15:10.084936+0000","last_peered":"2024-03-22T04:15:10.084936+0000","last_clean":"2024-03-22T04:15:10.084936+0000","last_became_active":"2024-03-22T04:15:10.080706+0000","last_became_peered":"2024-03-22T04:15:10.080706+0000","last_unstale":"2024-03-22T04:15:10.084936+0000","last_undegraded":"2024-03-22T04:15:10.084936+0000","last_fullsized":"2024-03-22T04:15:10.084936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:02:34.622890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083719+0000","last_change":"2024-03-22T04:15:10.073135+0000","last_active":"2024-03-22T04:15:10.083719+0000","last_peered":"2024-03-22T04:15:10.083719+0000","last_clean":"2024-03-22T04:15:10.083719+0000","last_became_active":"2024-03-22T04:15:10.072394+0000","last_became_peered":"2024-03-22T04:15:10.072394+0000","last_unstale":"2024-03-22T04:15:10.083719+0000","last_undegraded":"2024-03-22T04:15:10.083719+0000","last_fullsized":"2024-03-22T04:15:10.083719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:37:52.528838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.087928+0000","last_change":"2024-03-22T04:15:10.083260+0000","last_active":"2024-03-22T04:15:10.087928+0000","last_peered":"2024-03-22T04:15:10.087928+0000","last_clean":"2024-03-22T04:15:10.087928+0000","last_became_active":"2024-03-22T04:15:10.082565+0000","last_became_peered":"2024-03-22T04:15:10.082565+0000","last_unstale":"2024-03-22T04:15:10.087928+0000","last_undegraded":"2024-03-22T04:15:10.087928+0000","last_fullsized":"2024-03-22T04:15:10.087928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:56:41.158886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.102151+0000","last_change":"2024-03-22T04:15:09.070927+0000","last_active":"2024-03-22T04:15:10.102151+0000","last_peered":"2024-03-22T04:15:10.102151+0000","last_clean":"2024-03-22T04:15:10.102151+0000","last_became_active":"2024-03-22T04:15:09.070528+0000","last_became_peered":"2024-03-22T04:15:09.070528+0000","last_unstale":"2024-03-22T04:15:10.102151+0000","last_undegraded":"2024-03-22T04:15:10.102151+0000","last_fullsized":"2024-03-22T04:15:10.102151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:25:08.704172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090474+0000","last_change":"2024-03-22T04:15:10.087934+0000","last_active":"2024-03-22T04:15:10.090474+0000","last_peered":"2024-03-22T04:15:10.090474+0000","last_clean":"2024-03-22T04:15:10.090474+0000","last_became_active":"2024-03-22T04:15:10.086642+0000","last_became_peered":"2024-03-22T04:15:10.086642+0000","last_unstale":"2024-03-22T04:15:10.090474+0000","last_undegraded":"2024-03-22T04:15:10.090474+0000","last_fullsized":"2024-03-22T04:15:10.090474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:31:42.571291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.099265+0000","last_change":"2024-03-22T04:15:09.071764+0000","last_active":"2024-03-22T04:15:10.099265+0000","last_peered":"2024-03-22T04:15:10.099265+0000","last_clean":"2024-03-22T04:15:10.099265+0000","last_became_active":"2024-03-22T04:15:09.071518+0000","last_became_peered":"2024-03-22T04:15:09.071518+0000","last_unstale":"2024-03-22T04:15:10.099265+0000","last_undegraded":"2024-03-22T04:15:10.099265+0000","last_fullsized":"2024-03-22T04:15:10.099265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:03:41.138335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.094983+0000","last_change":"2024-03-22T04:15:10.090363+0000","last_active":"2024-03-22T04:15:10.094983+0000","last_peered":"2024-03-22T04:15:10.094983+0000","last_clean":"2024-03-22T04:15:10.094983+0000","last_became_active":"2024-03-22T04:15:10.090052+0000","last_became_peered":"2024-03-22T04:15:10.090052+0000","last_unstale":"2024-03-22T04:15:10.094983+0000","last_undegraded":"2024-03-22T04:15:10.094983+0000","last_fullsized":"2024-03-22T04:15:10.094983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:39:45.020238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.101958+0000","last_change":"2024-03-22T04:15:09.076848+0000","last_active":"2024-03-22T04:15:10.101958+0000","last_peered":"2024-03-22T04:15:10.101958+0000","last_clean":"2024-03-22T04:15:10.101958+0000","last_became_active":"2024-03-22T04:15:09.076533+0000","last_became_peered":"2024-03-22T04:15:09.076533+0000","last_unstale":"2024-03-22T04:15:10.101958+0000","last_undegraded":"2024-03-22T04:15:10.101958+0000","last_fullsized":"2024-03-22T04:15:10.101958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:14:11.346993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088286+0000","last_change":"2024-03-22T04:15:10.083119+0000","last_active":"2024-03-22T04:15:10.088286+0000","last_peered":"2024-03-22T04:15:10.088286+0000","last_clean":"2024-03-22T04:15:10.088286+0000","last_became_active":"2024-03-22T04:15:10.079664+0000","last_became_peered":"2024-03-22T04:15:10.079664+0000","last_unstale":"2024-03-22T04:15:10.088286+0000","last_undegraded":"2024-03-22T04:15:10.088286+0000","last_fullsized":"2024-03-22T04:15:10.088286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:02:34.622890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.100991+0000","last_change":"2024-03-22T04:15:09.064032+0000","last_active":"2024-03-22T04:15:10.100991+0000","last_peered":"2024-03-22T04:15:10.100991+0000","last_clean":"2024-03-22T04:15:10.100991+0000","last_became_active":"2024-03-22T04:15:09.063769+0000","last_became_peered":"2024-03-22T04:15:09.063769+0000","last_unstale":"2024-03-22T04:15:10.100991+0000","last_undegraded":"2024-03-22T04:15:10.100991+0000","last_fullsized":"2024-03-22T04:15:10.100991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:36:33.961454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090658+0000","last_change":"2024-03-22T04:15:10.077747+0000","last_active":"2024-03-22T04:15:10.090658+0000","last_peered":"2024-03-22T04:15:10.090658+0000","last_clean":"2024-03-22T04:15:10.090658+0000","last_became_active":"2024-03-22T04:15:10.077458+0000","last_became_peered":"2024-03-22T04:15:10.077458+0000","last_unstale":"2024-03-22T04:15:10.090658+0000","last_undegraded":"2024-03-22T04:15:10.090658+0000","last_fullsized":"2024-03-22T04:15:10.090658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:36:34.966608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.102062+0000","last_change":"2024-03-22T04:15:09.064915+0000","last_active":"2024-03-22T04:15:10.102062+0000","last_peered":"2024-03-22T04:15:10.102062+0000","last_clean":"2024-03-22T04:15:10.102062+0000","last_became_active":"2024-03-22T04:15:09.063555+0000","last_became_peered":"2024-03-22T04:15:09.063555+0000","last_unstale":"2024-03-22T04:15:10.102062+0000","last_undegraded":"2024-03-22T04:15:10.102062+0000","last_fullsized":"2024-03-22T04:15:10.102062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:45:16.218419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.085242+0000","last_change":"2024-03-22T04:15:10.081580+0000","last_active":"2024-03-22T04:15:10.085242+0000","last_peered":"2024-03-22T04:15:10.085242+0000","last_clean":"2024-03-22T04:15:10.085242+0000","last_became_active":"2024-03-22T04:15:10.080117+0000","last_became_peered":"2024-03-22T04:15:10.080117+0000","last_unstale":"2024-03-22T04:15:10.085242+0000","last_undegraded":"2024-03-22T04:15:10.085242+0000","last_fullsized":"2024-03-22T04:15:10.085242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:28:37.146208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.096836+0000","last_change":"2024-03-22T04:15:09.066910+0000","last_active":"2024-03-22T04:15:10.096836+0000","last_peered":"2024-03-22T04:15:10.096836+0000","last_clean":"2024-03-22T04:15:10.096836+0000","last_became_active":"2024-03-22T04:15:09.066609+0000","last_became_peered":"2024-03-22T04:15:09.066609+0000","last_unstale":"2024-03-22T04:15:10.096836+0000","last_undegraded":"2024-03-22T04:15:10.096836+0000","last_fullsized":"2024-03-22T04:15:10.096836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:00:43.098673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090408+0000","last_change":"2024-03-22T04:15:10.079593+0000","last_active":"2024-03-22T04:15:10.090408+0000","last_peered":"2024-03-22T04:15:10.090408+0000","last_clean":"2024-03-22T04:15:10.090408+0000","last_became_active":"2024-03-22T04:15:10.077268+0000","last_became_peered":"2024-03-22T04:15:10.077268+0000","last_unstale":"2024-03-22T04:15:10.090408+0000","last_undegraded":"2024-03-22T04:15:10.090408+0000","last_fullsized":"2024-03-22T04:15:10.090408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:18:07.082588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.062924+0000","last_change":"2024-03-22T04:15:09.071999+0000","last_active":"2024-03-22T04:15:11.062924+0000","last_peered":"2024-03-22T04:15:11.062924+0000","last_clean":"2024-03-22T04:15:11.062924+0000","last_became_active":"2024-03-22T04:15:09.071682+0000","last_became_peered":"2024-03-22T04:15:09.071682+0000","last_unstale":"2024-03-22T04:15:11.062924+0000","last_undegraded":"2024-03-22T04:15:11.062924+0000","last_fullsized":"2024-03-22T04:15:11.062924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:19:57.127216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.093700+0000","last_change":"2024-03-22T04:15:10.092143+0000","last_active":"2024-03-22T04:15:10.093700+0000","last_peered":"2024-03-22T04:15:10.093700+0000","last_clean":"2024-03-22T04:15:10.093700+0000","last_became_active":"2024-03-22T04:15:10.089295+0000","last_became_peered":"2024-03-22T04:15:10.089295+0000","last_unstale":"2024-03-22T04:15:10.093700+0000","last_undegraded":"2024-03-22T04:15:10.093700+0000","last_fullsized":"2024-03-22T04:15:10.093700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:43.492391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069495+0000","last_change":"2024-03-22T04:15:09.069492+0000","last_active":"2024-03-22T04:15:10.069495+0000","last_peered":"2024-03-22T04:15:10.069495+0000","last_clean":"2024-03-22T04:15:10.069495+0000","last_became_active":"2024-03-22T04:15:09.069042+0000","last_became_peered":"2024-03-22T04:15:09.069042+0000","last_unstale":"2024-03-22T04:15:10.069495+0000","last_undegraded":"2024-03-22T04:15:10.069495+0000","last_fullsized":"2024-03-22T04:15:10.069495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:58:31.525007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.084937+0000","last_change":"2024-03-22T04:15:10.081630+0000","last_active":"2024-03-22T04:15:10.084937+0000","last_peered":"2024-03-22T04:15:10.084937+0000","last_clean":"2024-03-22T04:15:10.084937+0000","last_became_active":"2024-03-22T04:15:10.081219+0000","last_became_peered":"2024-03-22T04:15:10.081219+0000","last_unstale":"2024-03-22T04:15:10.084937+0000","last_undegraded":"2024-03-22T04:15:10.084937+0000","last_fullsized":"2024-03-22T04:15:10.084937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:05:05.092437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069972+0000","last_change":"2024-03-22T04:15:09.071929+0000","last_active":"2024-03-22T04:15:10.069972+0000","last_peered":"2024-03-22T04:15:10.069972+0000","last_clean":"2024-03-22T04:15:10.069972+0000","last_became_active":"2024-03-22T04:15:09.071329+0000","last_became_peered":"2024-03-22T04:15:09.071329+0000","last_unstale":"2024-03-22T04:15:10.069972+0000","last_undegraded":"2024-03-22T04:15:10.069972+0000","last_fullsized":"2024-03-22T04:15:10.069972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:33:43.274842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.078364+0000","last_change":"2024-03-22T04:15:10.074473+0000","last_active":"2024-03-22T04:15:10.078364+0000","last_peered":"2024-03-22T04:15:10.078364+0000","last_clean":"2024-03-22T04:15:10.078364+0000","last_became_active":"2024-03-22T04:15:10.074185+0000","last_became_peered":"2024-03-22T04:15:10.074185+0000","last_unstale":"2024-03-22T04:15:10.078364+0000","last_undegraded":"2024-03-22T04:15:10.078364+0000","last_fullsized":"2024-03-22T04:15:10.078364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:33:25.874350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.060208+0000","last_change":"2024-03-22T04:15:09.074718+0000","last_active":"2024-03-22T04:15:10.060208+0000","last_peered":"2024-03-22T04:15:10.060208+0000","last_clean":"2024-03-22T04:15:10.060208+0000","last_became_active":"2024-03-22T04:15:09.072279+0000","last_became_peered":"2024-03-22T04:15:09.072279+0000","last_unstale":"2024-03-22T04:15:10.060208+0000","last_undegraded":"2024-03-22T04:15:10.060208+0000","last_fullsized":"2024-03-22T04:15:10.060208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:05.566853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063549+0000","last_change":"2024-03-22T04:15:10.080610+0000","last_active":"2024-03-22T04:15:11.063549+0000","last_peered":"2024-03-22T04:15:11.063549+0000","last_clean":"2024-03-22T04:15:11.063549+0000","last_became_active":"2024-03-22T04:15:10.079916+0000","last_became_peered":"2024-03-22T04:15:10.079916+0000","last_unstale":"2024-03-22T04:15:11.063549+0000","last_undegraded":"2024-03-22T04:15:11.063549+0000","last_fullsized":"2024-03-22T04:15:11.063549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:55:45.277621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.104900+0000","last_change":"2024-03-22T04:15:09.072031+0000","last_active":"2024-03-22T04:15:10.104900+0000","last_peered":"2024-03-22T04:15:10.104900+0000","last_clean":"2024-03-22T04:15:10.104900+0000","last_became_active":"2024-03-22T04:15:09.071759+0000","last_became_peered":"2024-03-22T04:15:09.071759+0000","last_unstale":"2024-03-22T04:15:10.104900+0000","last_undegraded":"2024-03-22T04:15:10.104900+0000","last_fullsized":"2024-03-22T04:15:10.104900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:28:26.401190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.089531+0000","last_change":"2024-03-22T04:15:10.077371+0000","last_active":"2024-03-22T04:15:10.089531+0000","last_peered":"2024-03-22T04:15:10.089531+0000","last_clean":"2024-03-22T04:15:10.089531+0000","last_became_active":"2024-03-22T04:15:10.077086+0000","last_became_peered":"2024-03-22T04:15:10.077086+0000","last_unstale":"2024-03-22T04:15:10.089531+0000","last_undegraded":"2024-03-22T04:15:10.089531+0000","last_fullsized":"2024-03-22T04:15:10.089531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:29.599987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.108307+0000","last_change":"2024-03-22T04:15:09.074220+0000","last_active":"2024-03-22T04:15:10.108307+0000","last_peered":"2024-03-22T04:15:10.108307+0000","last_clean":"2024-03-22T04:15:10.108307+0000","last_became_active":"2024-03-22T04:15:09.073974+0000","last_became_peered":"2024-03-22T04:15:09.073974+0000","last_unstale":"2024-03-22T04:15:10.108307+0000","last_undegraded":"2024-03-22T04:15:10.108307+0000","last_fullsized":"2024-03-22T04:15:10.108307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:40:14.849742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083391+0000","last_change":"2024-03-22T04:15:10.072948+0000","last_active":"2024-03-22T04:15:10.083391+0000","last_peered":"2024-03-22T04:15:10.083391+0000","last_clean":"2024-03-22T04:15:10.083391+0000","last_became_active":"2024-03-22T04:15:10.072665+0000","last_became_peered":"2024-03-22T04:15:10.072665+0000","last_unstale":"2024-03-22T04:15:10.083391+0000","last_undegraded":"2024-03-22T04:15:10.083391+0000","last_fullsized":"2024-03-22T04:15:10.083391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:55.903538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.105835+0000","last_change":"2024-03-22T04:15:09.071696+0000","last_active":"2024-03-22T04:15:10.105835+0000","last_peered":"2024-03-22T04:15:10.105835+0000","last_clean":"2024-03-22T04:15:10.105835+0000","last_became_active":"2024-03-22T04:15:09.071423+0000","last_became_peered":"2024-03-22T04:15:09.071423+0000","last_unstale":"2024-03-22T04:15:10.105835+0000","last_undegraded":"2024-03-22T04:15:10.105835+0000","last_fullsized":"2024-03-22T04:15:10.105835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:04:53.655257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092601+0000","last_change":"2024-03-22T04:15:10.088144+0000","last_active":"2024-03-22T04:15:10.092601+0000","last_peered":"2024-03-22T04:15:10.092601+0000","last_clean":"2024-03-22T04:15:10.092601+0000","last_became_active":"2024-03-22T04:15:10.087887+0000","last_became_peered":"2024-03-22T04:15:10.087887+0000","last_unstale":"2024-03-22T04:15:10.092601+0000","last_undegraded":"2024-03-22T04:15:10.092601+0000","last_fullsized":"2024-03-22T04:15:10.092601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:58:32.530161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.066108+0000","last_change":"2024-03-22T04:15:09.073275+0000","last_active":"2024-03-22T04:15:10.066108+0000","last_peered":"2024-03-22T04:15:10.066108+0000","last_clean":"2024-03-22T04:15:10.066108+0000","last_became_active":"2024-03-22T04:15:09.073002+0000","last_became_peered":"2024-03-22T04:15:09.073002+0000","last_unstale":"2024-03-22T04:15:10.066108+0000","last_undegraded":"2024-03-22T04:15:10.066108+0000","last_fullsized":"2024-03-22T04:15:10.066108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:28.594833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090432+0000","last_change":"2024-03-22T04:15:10.079907+0000","last_active":"2024-03-22T04:15:10.090432+0000","last_peered":"2024-03-22T04:15:10.090432+0000","last_clean":"2024-03-22T04:15:10.090432+0000","last_became_active":"2024-03-22T04:15:10.077157+0000","last_became_peered":"2024-03-22T04:15:10.077157+0000","last_unstale":"2024-03-22T04:15:10.090432+0000","last_undegraded":"2024-03-22T04:15:10.090432+0000","last_fullsized":"2024-03-22T04:15:10.090432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:23:11.491834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.072139+0000","last_change":"2024-03-22T04:15:09.075340+0000","last_active":"2024-03-22T04:15:10.072139+0000","last_peered":"2024-03-22T04:15:10.072139+0000","last_clean":"2024-03-22T04:15:10.072139+0000","last_became_active":"2024-03-22T04:15:09.075038+0000","last_became_peered":"2024-03-22T04:15:09.075038+0000","last_unstale":"2024-03-22T04:15:10.072139+0000","last_undegraded":"2024-03-22T04:15:10.072139+0000","last_fullsized":"2024-03-22T04:15:10.072139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:56:53.043203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.078265+0000","last_change":"2024-03-22T04:15:10.071029+0000","last_active":"2024-03-22T04:15:10.078265+0000","last_peered":"2024-03-22T04:15:10.078265+0000","last_clean":"2024-03-22T04:15:10.078265+0000","last_became_active":"2024-03-22T04:15:10.069833+0000","last_became_peered":"2024-03-22T04:15:10.069833+0000","last_unstale":"2024-03-22T04:15:10.078265+0000","last_undegraded":"2024-03-22T04:15:10.078265+0000","last_fullsized":"2024-03-22T04:15:10.078265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:10:10.851197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.104916+0000","last_change":"2024-03-22T04:15:09.067979+0000","last_active":"2024-03-22T04:15:10.104916+0000","last_peered":"2024-03-22T04:15:10.104916+0000","last_clean":"2024-03-22T04:15:10.104916+0000","last_became_active":"2024-03-22T04:15:09.067662+0000","last_became_peered":"2024-03-22T04:15:09.067662+0000","last_unstale":"2024-03-22T04:15:10.104916+0000","last_undegraded":"2024-03-22T04:15:10.104916+0000","last_fullsized":"2024-03-22T04:15:10.104916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:56:30.581574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090383+0000","last_change":"2024-03-22T04:15:10.087746+0000","last_active":"2024-03-22T04:15:10.090383+0000","last_peered":"2024-03-22T04:15:10.090383+0000","last_clean":"2024-03-22T04:15:10.090383+0000","last_became_active":"2024-03-22T04:15:10.087456+0000","last_became_peered":"2024-03-22T04:15:10.087456+0000","last_unstale":"2024-03-22T04:15:10.090383+0000","last_undegraded":"2024-03-22T04:15:10.090383+0000","last_fullsized":"2024-03-22T04:15:10.090383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:14:12.352147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.097912+0000","last_change":"2024-03-22T04:15:09.075102+0000","last_active":"2024-03-22T04:15:10.097912+0000","last_peered":"2024-03-22T04:15:10.097912+0000","last_clean":"2024-03-22T04:15:10.097912+0000","last_became_active":"2024-03-22T04:15:09.074118+0000","last_became_peered":"2024-03-22T04:15:09.074118+0000","last_unstale":"2024-03-22T04:15:10.097912+0000","last_undegraded":"2024-03-22T04:15:10.097912+0000","last_fullsized":"2024-03-22T04:15:10.097912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:13.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.084841+0000","last_change":"2024-03-22T04:15:10.080375+0000","last_active":"2024-03-22T04:15:10.084841+0000","last_peered":"2024-03-22T04:15:10.084841+0000","last_clean":"2024-03-22T04:15:10.084841+0000","last_became_active":"2024-03-22T04:15:10.080011+0000","last_became_peered":"2024-03-22T04:15:10.080011+0000","last_unstale":"2024-03-22T04:15:10.084841+0000","last_undegraded":"2024-03-22T04:15:10.084841+0000","last_fullsized":"2024-03-22T04:15:10.084841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:20:24.969370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.097684+0000","last_change":"2024-03-22T04:15:09.073727+0000","last_active":"2024-03-22T04:15:10.097684+0000","last_peered":"2024-03-22T04:15:10.097684+0000","last_clean":"2024-03-22T04:15:10.097684+0000","last_became_active":"2024-03-22T04:15:09.073420+0000","last_became_peered":"2024-03-22T04:15:09.073420+0000","last_unstale":"2024-03-22T04:15:10.097684+0000","last_undegraded":"2024-03-22T04:15:10.097684+0000","last_fullsized":"2024-03-22T04:15:10.097684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:41:46.638646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063350+0000","last_change":"2024-03-22T04:15:10.072479+0000","last_active":"2024-03-22T04:15:11.063350+0000","last_peered":"2024-03-22T04:15:11.063350+0000","last_clean":"2024-03-22T04:15:11.063350+0000","last_became_active":"2024-03-22T04:15:10.072192+0000","last_became_peered":"2024-03-22T04:15:10.072192+0000","last_unstale":"2024-03-22T04:15:11.063350+0000","last_undegraded":"2024-03-22T04:15:11.063350+0000","last_fullsized":"2024-03-22T04:15:11.063350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:09.358200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.100656+0000","last_change":"2024-03-22T04:15:09.064410+0000","last_active":"2024-03-22T04:15:10.100656+0000","last_peered":"2024-03-22T04:15:10.100656+0000","last_clean":"2024-03-22T04:15:10.100656+0000","last_became_active":"2024-03-22T04:15:09.064086+0000","last_became_peered":"2024-03-22T04:15:09.064086+0000","last_unstale":"2024-03-22T04:15:10.100656+0000","last_undegraded":"2024-03-22T04:15:10.100656+0000","last_fullsized":"2024-03-22T04:15:10.100656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:41:30.592585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.095176+0000","last_change":"2024-03-22T04:15:10.090296+0000","last_active":"2024-03-22T04:15:10.095176+0000","last_peered":"2024-03-22T04:15:10.095176+0000","last_clean":"2024-03-22T04:15:10.095176+0000","last_became_active":"2024-03-22T04:15:10.089812+0000","last_became_peered":"2024-03-22T04:15:10.089812+0000","last_unstale":"2024-03-22T04:15:10.095176+0000","last_undegraded":"2024-03-22T04:15:10.095176+0000","last_fullsized":"2024-03-22T04:15:10.095176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:16.933942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.071277+0000","last_change":"2024-03-22T04:15:09.071831+0000","last_active":"2024-03-22T04:15:10.071277+0000","last_peered":"2024-03-22T04:15:10.071277+0000","last_clean":"2024-03-22T04:15:10.071277+0000","last_became_active":"2024-03-22T04:15:09.071559+0000","last_became_peered":"2024-03-22T04:15:09.071559+0000","last_unstale":"2024-03-22T04:15:10.071277+0000","last_undegraded":"2024-03-22T04:15:10.071277+0000","last_fullsized":"2024-03-22T04:15:10.071277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:14:40.493536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.095207+0000","last_change":"2024-03-22T04:15:10.092218+0000","last_active":"2024-03-22T04:15:10.095207+0000","last_peered":"2024-03-22T04:15:10.095207+0000","last_clean":"2024-03-22T04:15:10.095207+0000","last_became_active":"2024-03-22T04:15:10.091929+0000","last_became_peered":"2024-03-22T04:15:10.091929+0000","last_unstale":"2024-03-22T04:15:10.095207+0000","last_undegraded":"2024-03-22T04:15:10.095207+0000","last_fullsized":"2024-03-22T04:15:10.095207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:10:10.851197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.100076+0000","last_change":"2024-03-22T04:15:09.068761+0000","last_active":"2024-03-22T04:15:10.100076+0000","last_peered":"2024-03-22T04:15:10.100076+0000","last_clean":"2024-03-22T04:15:10.100076+0000","last_became_active":"2024-03-22T04:15:09.068488+0000","last_became_peered":"2024-03-22T04:15:09.068488+0000","last_unstale":"2024-03-22T04:15:10.100076+0000","last_undegraded":"2024-03-22T04:15:10.100076+0000","last_fullsized":"2024-03-22T04:15:10.100076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:58:38.332903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.085330+0000","last_change":"2024-03-22T04:15:10.077342+0000","last_active":"2024-03-22T04:15:10.085330+0000","last_peered":"2024-03-22T04:15:10.085330+0000","last_clean":"2024-03-22T04:15:10.085330+0000","last_became_active":"2024-03-22T04:15:10.076964+0000","last_became_peered":"2024-03-22T04:15:10.076964+0000","last_unstale":"2024-03-22T04:15:10.085330+0000","last_undegraded":"2024-03-22T04:15:10.085330+0000","last_fullsized":"2024-03-22T04:15:10.085330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:03:17.693683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063046+0000","last_change":"2024-03-22T04:15:09.070075+0000","last_active":"2024-03-22T04:15:11.063046+0000","last_peered":"2024-03-22T04:15:11.063046+0000","last_clean":"2024-03-22T04:15:11.063046+0000","last_became_active":"2024-03-22T04:15:09.069803+0000","last_became_peered":"2024-03-22T04:15:09.069803+0000","last_unstale":"2024-03-22T04:15:11.063046+0000","last_undegraded":"2024-03-22T04:15:11.063046+0000","last_fullsized":"2024-03-22T04:15:11.063046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:19:17.186232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.092285+0000","last_change":"2024-03-22T04:15:10.085272+0000","last_active":"2024-03-22T04:15:10.092285+0000","last_peered":"2024-03-22T04:15:10.092285+0000","last_clean":"2024-03-22T04:15:10.092285+0000","last_became_active":"2024-03-22T04:15:10.084757+0000","last_became_peered":"2024-03-22T04:15:10.084757+0000","last_unstale":"2024-03-22T04:15:10.092285+0000","last_undegraded":"2024-03-22T04:15:10.092285+0000","last_fullsized":"2024-03-22T04:15:10.092285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:22:14.987034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069646+0000","last_change":"2024-03-22T04:15:09.073776+0000","last_active":"2024-03-22T04:15:10.069646+0000","last_peered":"2024-03-22T04:15:10.069646+0000","last_clean":"2024-03-22T04:15:10.069646+0000","last_became_active":"2024-03-22T04:15:09.073509+0000","last_became_peered":"2024-03-22T04:15:09.073509+0000","last_unstale":"2024-03-22T04:15:10.069646+0000","last_undegraded":"2024-03-22T04:15:10.069646+0000","last_fullsized":"2024-03-22T04:15:10.069646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:37:51.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.083740+0000","last_change":"2024-03-22T04:15:10.073405+0000","last_active":"2024-03-22T04:15:10.083740+0000","last_peered":"2024-03-22T04:15:10.083740+0000","last_clean":"2024-03-22T04:15:10.083740+0000","last_became_active":"2024-03-22T04:15:10.072803+0000","last_became_peered":"2024-03-22T04:15:10.072803+0000","last_unstale":"2024-03-22T04:15:10.083740+0000","last_undegraded":"2024-03-22T04:15:10.083740+0000","last_fullsized":"2024-03-22T04:15:10.083740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:31.358038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.104834+0000","last_change":"2024-03-22T04:15:09.070795+0000","last_active":"2024-03-22T04:15:10.104834+0000","last_peered":"2024-03-22T04:15:10.104834+0000","last_clean":"2024-03-22T04:15:10.104834+0000","last_became_active":"2024-03-22T04:15:09.070537+0000","last_became_peered":"2024-03-22T04:15:09.070537+0000","last_unstale":"2024-03-22T04:15:10.104834+0000","last_undegraded":"2024-03-22T04:15:10.104834+0000","last_fullsized":"2024-03-22T04:15:10.104834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:41:46.638646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.090271+0000","last_change":"2024-03-22T04:15:10.085620+0000","last_active":"2024-03-22T04:15:10.090271+0000","last_peered":"2024-03-22T04:15:10.090271+0000","last_clean":"2024-03-22T04:15:10.090271+0000","last_became_active":"2024-03-22T04:15:10.085255+0000","last_became_peered":"2024-03-22T04:15:10.085255+0000","last_unstale":"2024-03-22T04:15:10.090271+0000","last_undegraded":"2024-03-22T04:15:10.090271+0000","last_fullsized":"2024-03-22T04:15:10.090271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:55.903538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.063452+0000","last_change":"2024-03-22T04:15:09.072099+0000","last_active":"2024-03-22T04:15:10.063452+0000","last_peered":"2024-03-22T04:15:10.063452+0000","last_clean":"2024-03-22T04:15:10.063452+0000","last_became_active":"2024-03-22T04:15:09.071839+0000","last_became_peered":"2024-03-22T04:15:09.071839+0000","last_unstale":"2024-03-22T04:15:10.063452+0000","last_undegraded":"2024-03-22T04:15:10.063452+0000","last_fullsized":"2024-03-22T04:15:10.063452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:47:56.139069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.097700+0000","last_change":"2024-03-22T04:15:10.092371+0000","last_active":"2024-03-22T04:15:10.097700+0000","last_peered":"2024-03-22T04:15:10.097700+0000","last_clean":"2024-03-22T04:15:10.097700+0000","last_became_active":"2024-03-22T04:15:10.092086+0000","last_became_peered":"2024-03-22T04:15:10.092086+0000","last_unstale":"2024-03-22T04:15:10.097700+0000","last_undegraded":"2024-03-22T04:15:10.097700+0000","last_fullsized":"2024-03-22T04:15:10.097700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:23:46.775958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.060483+0000","last_change":"2024-03-22T04:15:09.071706+0000","last_active":"2024-03-22T04:15:10.060483+0000","last_peered":"2024-03-22T04:15:10.060483+0000","last_clean":"2024-03-22T04:15:10.060483+0000","last_became_active":"2024-03-22T04:15:09.071433+0000","last_became_peered":"2024-03-22T04:15:09.071433+0000","last_unstale":"2024-03-22T04:15:10.060483+0000","last_undegraded":"2024-03-22T04:15:10.060483+0000","last_fullsized":"2024-03-22T04:15:10.060483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:31:41.566136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-22T04:15:11.063472+0000","last_change":"2024-03-22T04:15:10.081684+0000","last_active":"2024-03-22T04:15:11.063472+0000","last_peered":"2024-03-22T04:15:11.063472+0000","last_clean":"2024-03-22T04:15:11.063472+0000","last_became_active":"2024-03-22T04:15:10.081204+0000","last_became_peered":"2024-03-22T04:15:10.081204+0000","last_unstale":"2024-03-22T04:15:11.063472+0000","last_undegraded":"2024-03-22T04:15:11.063472+0000","last_fullsized":"2024-03-22T04:15:11.063472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:24:25.797823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069580+0000","last_change":"2024-03-22T04:15:09.068727+0000","last_active":"2024-03-22T04:15:10.069580+0000","last_peered":"2024-03-22T04:15:10.069580+0000","last_clean":"2024-03-22T04:15:10.069580+0000","last_became_active":"2024-03-22T04:15:09.068464+0000","last_became_peered":"2024-03-22T04:15:09.068464+0000","last_unstale":"2024-03-22T04:15:10.069580+0000","last_undegraded":"2024-03-22T04:15:10.069580+0000","last_fullsized":"2024-03-22T04:15:10.069580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:10:09.846043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088523+0000","last_change":"2024-03-22T04:15:10.082282+0000","last_active":"2024-03-22T04:15:10.088523+0000","last_peered":"2024-03-22T04:15:10.088523+0000","last_clean":"2024-03-22T04:15:10.088523+0000","last_became_active":"2024-03-22T04:15:10.081458+0000","last_became_peered":"2024-03-22T04:15:10.081458+0000","last_unstale":"2024-03-22T04:15:10.088523+0000","last_undegraded":"2024-03-22T04:15:10.088523+0000","last_fullsized":"2024-03-22T04:15:10.088523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:57:08.175064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.066145+0000","last_change":"2024-03-22T04:15:09.068628+0000","last_active":"2024-03-22T04:15:10.066145+0000","last_peered":"2024-03-22T04:15:10.066145+0000","last_clean":"2024-03-22T04:15:10.066145+0000","last_became_active":"2024-03-22T04:15:09.068316+0000","last_became_peered":"2024-03-22T04:15:09.068316+0000","last_unstale":"2024-03-22T04:15:10.066145+0000","last_undegraded":"2024-03-22T04:15:10.066145+0000","last_fullsized":"2024-03-22T04:15:10.066145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:53:59.753788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.094698+0000","last_change":"2024-03-22T04:15:10.090200+0000","last_active":"2024-03-22T04:15:10.094698+0000","last_peered":"2024-03-22T04:15:10.094698+0000","last_clean":"2024-03-22T04:15:10.094698+0000","last_became_active":"2024-03-22T04:15:10.086625+0000","last_became_peered":"2024-03-22T04:15:10.086625+0000","last_unstale":"2024-03-22T04:15:10.094698+0000","last_undegraded":"2024-03-22T04:15:10.094698+0000","last_fullsized":"2024-03-22T04:15:10.094698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:05:05.092437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.063555+0000","last_change":"2024-03-22T04:15:09.068790+0000","last_active":"2024-03-22T04:15:10.063555+0000","last_peered":"2024-03-22T04:15:10.063555+0000","last_clean":"2024-03-22T04:15:10.063555+0000","last_became_active":"2024-03-22T04:15:09.068490+0000","last_became_peered":"2024-03-22T04:15:09.068490+0000","last_unstale":"2024-03-22T04:15:10.063555+0000","last_undegraded":"2024-03-22T04:15:10.063555+0000","last_fullsized":"2024-03-22T04:15:10.063555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:54:48.956024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.093441+0000","last_change":"2024-03-22T04:15:10.088838+0000","last_active":"2024-03-22T04:15:10.093441+0000","last_peered":"2024-03-22T04:15:10.093441+0000","last_clean":"2024-03-22T04:15:10.093441+0000","last_became_active":"2024-03-22T04:15:10.086301+0000","last_became_peered":"2024-03-22T04:15:10.086301+0000","last_unstale":"2024-03-22T04:15:10.093441+0000","last_undegraded":"2024-03-22T04:15:10.093441+0000","last_fullsized":"2024-03-22T04:15:10.093441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:54:49.961178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069364+0000","last_change":"2024-03-22T04:15:09.071117+0000","last_active":"2024-03-22T04:15:10.069364+0000","last_peered":"2024-03-22T04:15:10.069364+0000","last_clean":"2024-03-22T04:15:10.069364+0000","last_became_active":"2024-03-22T04:15:09.070800+0000","last_became_peered":"2024-03-22T04:15:09.070800+0000","last_unstale":"2024-03-22T04:15:10.069364+0000","last_undegraded":"2024-03-22T04:15:10.069364+0000","last_fullsized":"2024-03-22T04:15:10.069364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:15.928788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.088551+0000","last_change":"2024-03-22T04:15:10.084958+0000","last_active":"2024-03-22T04:15:10.088551+0000","last_peered":"2024-03-22T04:15:10.088551+0000","last_clean":"2024-03-22T04:15:10.088551+0000","last_became_active":"2024-03-22T04:15:10.082658+0000","last_became_peered":"2024-03-22T04:15:10.082658+0000","last_unstale":"2024-03-22T04:15:10.088551+0000","last_undegraded":"2024-03-22T04:15:10.088551+0000","last_fullsized":"2024-03-22T04:15:10.088551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:06.572007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.069817+0000","last_change":"2024-03-22T04:15:09.065130+0000","last_active":"2024-03-22T04:15:10.069817+0000","last_peered":"2024-03-22T04:15:10.069817+0000","last_clean":"2024-03-22T04:15:10.069817+0000","last_became_active":"2024-03-22T04:15:09.064861+0000","last_became_peered":"2024-03-22T04:15:09.064861+0000","last_unstale":"2024-03-22T04:15:10.069817+0000","last_undegraded":"2024-03-22T04:15:10.069817+0000","last_fullsized":"2024-03-22T04:15:10.069817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:55:30.352884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.076756+0000","last_change":"2024-03-22T04:15:10.073842+0000","last_active":"2024-03-22T04:15:10.076756+0000","last_peered":"2024-03-22T04:15:10.076756+0000","last_clean":"2024-03-22T04:15:10.076756+0000","last_became_active":"2024-03-22T04:15:10.073563+0000","last_became_peered":"2024-03-22T04:15:10.073563+0000","last_unstale":"2024-03-22T04:15:10.076756+0000","last_undegraded":"2024-03-22T04:15:10.076756+0000","last_fullsized":"2024-03-22T04:15:10.076756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:09.042906+0000","last_clean_scrub_stamp":"2024-03-22T04:15:09.042906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:02:03.207869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.063381+0000","last_change":"2024-03-22T04:15:09.068066+0000","last_active":"2024-03-22T04:15:10.063381+0000","last_peered":"2024-03-22T04:15:10.063381+0000","last_clean":"2024-03-22T04:15:10.063381+0000","last_became_active":"2024-03-22T04:15:09.067712+0000","last_became_peered":"2024-03-22T04:15:09.067712+0000","last_unstale":"2024-03-22T04:15:10.063381+0000","last_undegraded":"2024-03-22T04:15:10.063381+0000","last_fullsized":"2024-03-22T04:15:10.063381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:15:08.037752+0000","last_clean_scrub_stamp":"2024-03-22T04:15:08.037752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:35:15.551999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":274,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-22T04:15:10.059668+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:15:10.059668+0000","last_peered":"2024-03-22T04:15:10.059668+0000","last_clean":"2024-03-22T04:15:10.059668+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:15:10.059668+0000","last_undegraded":"2024-03-22T04:15:10.059668+0000","last_fullsized":"2024-03-22T04:15:10.059668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":435991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":428172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":1025797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509501,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":437055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":958464,"data_stored":435339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":1018540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":435361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":442053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:15:14.681 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T04:15:14.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-22T04:15:14.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9765c00640 1 Processor -- start 2024-03-22T04:15:14.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9765c00640 1 -- start start 2024-03-22T04:15:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9765c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f976006dde0 0x7f976006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9765c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97600d2ac0 con 0x7f976015e980 2024-03-22T04:15:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9765c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f976015ee60 con 0x7f976015c010 2024-03-22T04:15:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9765c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9760161f50 con 0x7f976006dde0 2024-03-22T04:15:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9757e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f976015e980 0x7f976015f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44156/0 (socket says 172.21.15.86:44156) 2024-03-22T04:15:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9757e00640 1 -- 172.21.15.86:0/240024747 learned_addr learned my addr 172.21.15.86:0/240024747 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975f400640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f976006dde0 0x7f976006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648022620 0 0) 0x7f97600d2ac0 con 0x7f976015e980 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9744003530 con 0x7f976015e980 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1712735437 0 0) 0x7f9744003530 con 0x7f976015e980 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f976006dde0 msgr2=0x7f976006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f976006dde0 0x7f976006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 >> v1:172.21.15.196:6789/0 conn(0x7f976015c010 legacy=0x7f97600d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976015f140 con 0x7f976015e980 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f9748001680 con 0x7f976015e980 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 >> v1:172.21.15.86:6789/0 conn(0x7f976015e980 legacy=0x7f976015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9744003a60 0x7f9744003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9744004810 0x7f9744004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f976006dde0 0x7f97440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9760161f50 con 0x7f9744004810 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f974400ba80 con 0x7f9744003a60 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f974400bc50 con 0x7f976006dde0 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9757e00640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9744004810 0x7f9744004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975ea00640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9744003a60 0x7f9744003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f975f400640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f976006dde0 0x7f97440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9757e00640 1 -- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f976006dde0 msgr2=0x7f97440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9757e00640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f976006dde0 0x7f97440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9757e00640 1 -- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9744003a60 msgr2=0x7f9744003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9757e00640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9744003a60 0x7f9744003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.863+0000 7f9757e00640 1 -- 172.21.15.86:0/240024747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9744003530 con 0x7f9744004810 2024-03-22T04:15:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f9757e00640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9744004810 0x7f9744004c90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f974800c240 tx=0x7f974801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a23fc7a58c2fec server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748007d40 con 0x7f9744004810 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97480a9070 con 0x7f9744004810 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f975e000640 1 -- 172.21.15.86:0/240024747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97480a5d80 con 0x7f9744004810 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f9765c00640 1 -- 172.21.15.86:0/240024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9744004810 msgr2=0x7f9744004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f9765c00640 1 --2- 172.21.15.86:0/240024747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9744004810 0x7f9744004c90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f974800c240 tx=0x7f974801cc20 comp rx=0 tx=0).stop 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f975f400640 1 -- 172.21.15.86:0/240024747 reap_dead start 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f9765c00640 1 -- 172.21.15.86:0/240024747 shutdown_connections 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f9765c00640 1 -- 172.21.15.86:0/240024747 >> 172.21.15.86:0/240024747 conn(0x7f976006b660 msgr2=0x7f976006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f9765c00640 1 -- 172.21.15.86:0/240024747 shutdown_connections 2024-03-22T04:15:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.867+0000 7f9765c00640 1 -- 172.21.15.86:0/240024747 wait complete. 2024-03-22T04:15:14.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 Processor -- start 2024-03-22T04:15:14.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 -- start start 2024-03-22T04:15:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9744004810 0x7f97601574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760155220 0x7f97601556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97601458f0 0x7f9760145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97601600f0 con 0x7f97601458f0 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f976015f700 con 0x7f9760155220 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f976006e290 con 0x7f9744004810 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97601458f0 0x7f9760145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97601458f0 0x7f9760145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43778/0 (socket says 172.21.15.86:43778) 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 -- 172.21.15.86:0/2519302034 learned_addr learned my addr 172.21.15.86:0/2519302034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9757e00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760155220 0x7f97601556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 -- 172.21.15.86:0/2519302034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9744004810 msgr2=0x7f97601574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9744004810 0x7f97601574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 -- 172.21.15.86:0/2519302034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760155220 msgr2=0x7f97601556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760155220 0x7f97601556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 -- 172.21.15.86:0/2519302034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9760157b20 con 0x7f97601458f0 2024-03-22T04:15:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9757e00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760155220 0x7f97601556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f975f400640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97601458f0 0x7f9760145d70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f97600b82d0 tx=0x7f975800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9756a00640 1 -- 172.21.15.86:0/2519302034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758015430 con 0x7f97601458f0 2024-03-22T04:15:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9756a00640 1 -- 172.21.15.86:0/2519302034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97580155f0 con 0x7f97601458f0 2024-03-22T04:15:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9756a00640 1 -- 172.21.15.86:0/2519302034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580159e0 con 0x7f97601458f0 2024-03-22T04:15:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9760157e00 con 0x7f97601458f0 2024-03-22T04:15:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.871+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97601583c0 con 0x7f97601458f0 2024-03-22T04:15:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.875+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9760074470 con 0x7f97601458f0 2024-03-22T04:15:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.875+0000 7f9756a00640 1 -- 172.21.15.86:0/2519302034 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9758015b80 con 0x7f97601458f0 2024-03-22T04:15:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.875+0000 7f9756a00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f973c067b50 0x7f973c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.875+0000 7f9756a00640 1 -- 172.21.15.86:0/2519302034 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f9758086eb0 con 0x7f97601458f0 2024-03-22T04:15:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.875+0000 7f975ea00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f973c067b50 0x7f973c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:14.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.875+0000 7f975ea00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f973c067b50 0x7f973c06a010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9750007fd0 tx=0x7f975000a880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:14.879+0000 7f9756a00640 1 -- 172.21.15.86:0/2519302034 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9758019030 con 0x7f97601458f0 2024-03-22T04:15:15.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.091+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f97600d04e0 con 0x7f97601458f0 2024-03-22T04:15:15.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.095+0000 7f9756a00640 1 -- 172.21.15.86:0/2519302034 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f9758010040 con 0x7f97601458f0 2024-03-22T04:15:15.101 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T04:15:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:15:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-03-22T04:15:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-03-22T04:15:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-22T04:15:15.101 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-03-22T04:15:15.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f973c067b50 msgr2=0x7f973c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:15.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f973c067b50 0x7f973c06a010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9750007fd0 tx=0x7f975000a880 comp rx=0 tx=0).stop 2024-03-22T04:15:15.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97601458f0 msgr2=0x7f9760145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:15.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97601458f0 0x7f9760145d70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f97600b82d0 tx=0x7f975800f580 comp rx=0 tx=0).stop 2024-03-22T04:15:15.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 shutdown_connections 2024-03-22T04:15:15.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f973c067b50 0x7f973c06a010 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760155220 0x7f97601556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97601458f0 0x7f9760145d70 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 --2- 172.21.15.86:0/2519302034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9744004810 0x7f97601574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 >> 172.21.15.86:0/2519302034 conn(0x7f976006b660 msgr2=0x7f976015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 shutdown_connections 2024-03-22T04:15:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.099+0000 7f9765c00640 1 -- 172.21.15.86:0/2519302034 wait complete. 2024-03-22T04:15:15.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:15:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.255+0000 7f76b3600640 1 Processor -- start 2024-03-22T04:15:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.255+0000 7f76b3600640 1 -- start start 2024-03-22T04:15:15.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.255+0000 7f76b3600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f76ac15e630 0x7f76ac0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:15.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.255+0000 7f76b3600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac0d03b0 0x7f76ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:15.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.255+0000 7f76b3600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac0d07d0 con 0x7f76ac06e2f0 2024-03-22T04:15:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.255+0000 7f76b3600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f76ac1623d0 con 0x7f76ac15e630 2024-03-22T04:15:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.255+0000 7f76b3600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac1620f0 con 0x7f76ac0d03b0 2024-03-22T04:15:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.255+0000 7f76b1600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac0d03b0 0x7f76ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76b0c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f76ac15e630 0x7f76ac0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76abe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f76ac06e2f0 0x7f76ac06cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:44806/0 (socket says 172.21.15.86:44806) 2024-03-22T04:15:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76abe00640 1 -- 172.21.15.86:0/2766350659 learned_addr learned my addr 172.21.15.86:0/2766350659 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:15.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76ab400640 1 -- 172.21.15.86:0/2766350659 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3495548825 0 0) 0x7f76ac0d07d0 con 0x7f76ac06e2f0 2024-03-22T04:15:15.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76ab400640 1 -- 172.21.15.86:0/2766350659 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7694003530 con 0x7f76ac06e2f0 2024-03-22T04:15:15.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76b0c00640 1 -- 172.21.15.86:0/2766350659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac0d03b0 msgr2=0x7f76ac06d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:15.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76b0c00640 1 --2- 172.21.15.86:0/2766350659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac0d03b0 0x7f76ac06d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76b0c00640 1 -- 172.21.15.86:0/2766350659 >> v1:172.21.15.196:6789/0 conn(0x7f76ac06e2f0 legacy=0x7f76ac06cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:15.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76b0c00640 1 -- 172.21.15.86:0/2766350659 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac15ec10 con 0x7f76ac15e630 2024-03-22T04:15:15.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76b1600640 1 --2- 172.21.15.86:0/2766350659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac0d03b0 0x7f76ac06d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:15.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76b0c00640 1 --2- 172.21.15.86:0/2766350659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac15e630 0x7f76ac0cfe70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097c0 tx=0x7f76a001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6460ee3c9def8232 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76ab400640 1 -- 172.21.15.86:0/2766350659 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00b0070 con 0x7f76ac15e630 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76ab400640 1 -- 172.21.15.86:0/2766350659 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a00ab600 con 0x7f76ac15e630 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.259+0000 7f76ab400640 1 -- 172.21.15.86:0/2766350659 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00ab9d0 con 0x7f76ac15e630 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- 172.21.15.86:0/2766350659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac15e630 msgr2=0x7f76ac0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 --2- 172.21.15.86:0/2766350659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac15e630 0x7f76ac0cfe70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097c0 tx=0x7f76a001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- 172.21.15.86:0/2766350659 shutdown_connections 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 --2- 172.21.15.86:0/2766350659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac0d03b0 0x7f76ac06d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 --2- 172.21.15.86:0/2766350659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac15e630 0x7f76ac0cfe70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- 172.21.15.86:0/2766350659 >> 172.21.15.86:0/2766350659 conn(0x7f76ac06f910 msgr2=0x7f76ac06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- 172.21.15.86:0/2766350659 shutdown_connections 2024-03-22T04:15:15.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- 172.21.15.86:0/2766350659 wait complete. 2024-03-22T04:15:15.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 Processor -- start 2024-03-22T04:15:15.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- start start 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac06e2f0 0x7f76ac154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac0d03b0 0x7f76ac0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76ac15e630 0x7f76ac0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac15f9d0 con 0x7f76ac15e630 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0d1f60 con 0x7f76ac0d03b0 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b3600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0cfaf0 con 0x7f76ac06e2f0 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76ac15e630 0x7f76ac0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76ac15e630 0x7f76ac0cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43794/0 (socket says 172.21.15.86:43794) 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 -- 172.21.15.86:0/3096852407 learned_addr learned my addr 172.21.15.86:0/3096852407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76abe00640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac0d03b0 0x7f76ac0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b0c00640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac06e2f0 0x7f76ac154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 -- 172.21.15.86:0/3096852407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac06e2f0 msgr2=0x7f76ac154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac06e2f0 0x7f76ac154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 -- 172.21.15.86:0/3096852407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac0d03b0 msgr2=0x7f76ac0cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac0d03b0 0x7f76ac0cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 -- 172.21.15.86:0/3096852407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac152fe0 con 0x7f76ac15e630 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.263+0000 7f76b1600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76ac15e630 0x7f76ac0cd210 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f76a400efc0 tx=0x7f76a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76a8c00640 1 -- 172.21.15.86:0/3096852407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4018030 con 0x7f76ac15e630 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76a8c00640 1 -- 172.21.15.86:0/3096852407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a400cdf0 con 0x7f76ac15e630 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76a8c00640 1 -- 172.21.15.86:0/3096852407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4003bc0 con 0x7f76ac15e630 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac1532c0 con 0x7f76ac15e630 2024-03-22T04:15:15.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac153880 con 0x7f76ac15e630 2024-03-22T04:15:15.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac068650 con 0x7f76ac15e630 2024-03-22T04:15:15.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76a8c00640 1 -- 172.21.15.86:0/3096852407 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f76a401b070 con 0x7f76ac15e630 2024-03-22T04:15:15.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76a8c00640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f768c067b20 0x7f768c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:15.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76a8c00640 1 -- 172.21.15.86:0/3096852407 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f76a4086a80 con 0x7f76ac15e630 2024-03-22T04:15:15.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.267+0000 7f76b0c00640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f768c067b20 0x7f768c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:15.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.271+0000 7f76b0c00640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f768c067b20 0x7f768c069fe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ad000 tx=0x7f76a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:15.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.271+0000 7f76a8c00640 1 -- 172.21.15.86:0/3096852407 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f76a4088e20 con 0x7f76ac15e630 2024-03-22T04:15:15.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.459+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76ac161a00 con 0x7f76ac15e630 2024-03-22T04:15:15.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.459+0000 7f76a8c00640 1 -- 172.21.15.86:0/3096852407 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5189 (secure 0 0 0) 0x7f76a4010040 con 0x7f76ac15e630 2024-03-22T04:15:15.465 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:15.465 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:15:12.461110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1130704586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1130704586},{"type":"v1","addr":"172.21.15.86:6837","nonce":1130704586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6807,"qdb_cluster":[6807]},"id":5}]} 2024-03-22T04:15:15.465 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-03-22T04:15:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f768c067b20 msgr2=0x7f768c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f768c067b20 0x7f768c069fe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f76a00ad000 tx=0x7f76a0008040 comp rx=0 tx=0).stop 2024-03-22T04:15:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76ac15e630 msgr2=0x7f76ac0cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76ac15e630 0x7f76ac0cd210 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f76a400efc0 tx=0x7f76a400c490 comp rx=0 tx=0).stop 2024-03-22T04:15:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 shutdown_connections 2024-03-22T04:15:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f768c067b20 0x7f768c069fe0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76ac0d03b0 0x7f76ac0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76ac15e630 0x7f76ac0cd210 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 --2- 172.21.15.86:0/3096852407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76ac06e2f0 0x7f76ac154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 >> 172.21.15.86:0/3096852407 conn(0x7f76ac06f910 msgr2=0x7f76ac0b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 shutdown_connections 2024-03-22T04:15:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:15.463+0000 7f76b3600640 1 -- 172.21.15.86:0/3096852407 wait complete. 2024-03-22T04:15:15.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:15:09.433883+0000', 'modified': '2024-03-22T04:15:12.461110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6807}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6807': {'gid': 6807, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/1130704586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 1130704586}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1130704586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6807, 'qdb_cluster': [6807]} 2024-03-22T04:15:15.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:15:15.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-22T04:15:16.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.023+0000 7f7eb9200640 1 Processor -- start 2024-03-22T04:15:16.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb9200640 1 -- start start 2024-03-22T04:15:16.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb9200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7eb406e2f0 0x7f7eb4071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb9200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb41623d0 con 0x7f7eb415e630 2024-03-22T04:15:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb9200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb41620f0 con 0x7f7eb40d0580 2024-03-22T04:15:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb9200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb415ec10 con 0x7f7eb406e2f0 2024-03-22T04:15:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7eb415e630 0x7f7eb40cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36518/0 (socket says 172.21.15.86:36518) 2024-03-22T04:15:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1e00640 1 -- 172.21.15.86:0/701242355 learned_addr learned my addr 172.21.15.86:0/701242355 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb2800640 1 --2- 172.21.15.86:0/701242355 >> v2:172.21.15.196:3300/0 conn(0x7f7eb406e2f0 0x7f7eb4071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068503443 0 0) 0x7f7eb41623d0 con 0x7f7eb415e630 2024-03-22T04:15:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e98003530 con 0x7f7eb415e630 2024-03-22T04:15:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 188043332 0 0) 0x7f7e98003530 con 0x7f7eb415e630 2024-03-22T04:15:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 >> v2:172.21.15.196:3300/0 conn(0x7f7eb406e2f0 msgr2=0x7f7eb4071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 --2- 172.21.15.86:0/701242355 >> v2:172.21.15.196:3300/0 conn(0x7f7eb406e2f0 0x7f7eb4071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 >> v1:172.21.15.196:6789/0 conn(0x7f7eb40d0580 legacy=0x7f7eb4071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb415eef0 con 0x7f7eb415e630 2024-03-22T04:15:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f7ea4001680 con 0x7f7eb415e630 2024-03-22T04:15:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 >> v1:172.21.15.86:6789/0 conn(0x7f7eb415e630 legacy=0x7f7eb40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e980039c0 0x7f7e98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e98004810 0x7f7e98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e980051d0 0x7f7e9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb415ec10 con 0x7f7e98004810 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9800c070 con 0x7f7e980039c0 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9800c270 con 0x7f7e980051d0 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e980039c0 0x7f7e98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb2800640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e980051d0 0x7f7e9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eabe00640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e98004810 0x7f7e98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eabe00640 1 -- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e980051d0 msgr2=0x7f7e9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eabe00640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e980051d0 0x7f7e9800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eabe00640 1 -- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e980039c0 msgr2=0x7f7e98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eabe00640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e980039c0 0x7f7e98003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eabe00640 1 -- 172.21.15.86:0/701242355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e98003530 con 0x7f7e98004810 2024-03-22T04:15:16.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb2800640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e980051d0 0x7f7e9800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:16.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.027+0000 7f7eb2800640 1 -- 172.21.15.86:0/701242355 reap_dead start 2024-03-22T04:15:16.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eabe00640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e98004810 0x7f7e98004c90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c001780 tx=0x7f7e9c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe72179bef2e6b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0013a0 con 0x7f7e98004810 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0052f0 con 0x7f7e98004810 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eb1400640 1 -- 172.21.15.86:0/701242355 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c003840 con 0x7f7e98004810 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eb9200640 1 -- 172.21.15.86:0/701242355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e98004810 msgr2=0x7f7e98004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eb9200640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e98004810 0x7f7e98004c90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c001780 tx=0x7f7e9c01c400 comp rx=0 tx=0).stop 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eb9200640 1 -- 172.21.15.86:0/701242355 shutdown_connections 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eb9200640 1 --2- 172.21.15.86:0/701242355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e98004810 0x7f7e98004c90 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.031+0000 7f7eb9200640 1 -- 172.21.15.86:0/701242355 >> 172.21.15.86:0/701242355 conn(0x7f7eb406f910 msgr2=0x7f7eb406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 -- 172.21.15.86:0/701242355 shutdown_connections 2024-03-22T04:15:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 -- 172.21.15.86:0/701242355 wait complete. 2024-03-22T04:15:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 Processor -- start 2024-03-22T04:15:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 -- start start 2024-03-22T04:15:16.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e98004810 0x7f7eb41596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb40cb680 0x7f7eb41f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eb40cd900 0x7f7eb40cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40d1f60 con 0x7f7eb40cb680 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb415f4b0 con 0x7f7e98004810 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb9200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb415f9d0 con 0x7f7eb40cd900 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb40cb680 0x7f7eb41f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb40cb680 0x7f7eb41f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44986/0 (socket says 172.21.15.86:44986) 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eabe00640 1 -- 172.21.15.86:0/2834169264 learned_addr learned my addr 172.21.15.86:0/2834169264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e98004810 0x7f7eb41596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eb2800640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eb40cd900 0x7f7eb40cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eabe00640 1 -- 172.21.15.86:0/2834169264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eb40cd900 msgr2=0x7f7eb40cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eabe00640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eb40cd900 0x7f7eb40cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eabe00640 1 -- 172.21.15.86:0/2834169264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e98004810 msgr2=0x7f7eb41596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eabe00640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e98004810 0x7f7eb41596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.035+0000 7f7eabe00640 1 -- 172.21.15.86:0/2834169264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb40cbb00 con 0x7f7eb40cb680 2024-03-22T04:15:16.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eabe00640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb40cb680 0x7f7eb41f38c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005040 tx=0x7f7e98005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eaa000640 1 -- 172.21.15.86:0/2834169264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0b8020 con 0x7f7eb40cb680 2024-03-22T04:15:16.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eaa000640 1 -- 172.21.15.86:0/2834169264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0054a0 con 0x7f7eb40cb680 2024-03-22T04:15:16.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eb9200640 1 -- 172.21.15.86:0/2834169264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb4159d50 con 0x7f7eb40cb680 2024-03-22T04:15:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eaa000640 1 -- 172.21.15.86:0/2834169264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0013a0 con 0x7f7eb40cb680 2024-03-22T04:15:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eb9200640 1 -- 172.21.15.86:0/2834169264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb415a310 con 0x7f7eb40cb680 2024-03-22T04:15:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eaa000640 1 -- 172.21.15.86:0/2834169264 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7e9c0bb020 con 0x7f7eb40cb680 2024-03-22T04:15:16.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eaa000640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e90067b20 0x7f7e90069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.039+0000 7f7eaa000640 1 -- 172.21.15.86:0/2834169264 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7e9c120210 con 0x7f7eb40cb680 2024-03-22T04:15:16.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e90067b20 0x7f7e90069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 Processor -- start 2024-03-22T04:15:16.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- start start 2024-03-22T04:15:16.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e90067b20 0x7f7e90069fe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4007d10 tx=0x7f7ea4001c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 0x5646c9468250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c941b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 0x5646c941c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646c9219140 con 0x5646c9468790 2024-03-22T04:15:16.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646c926cb10 con 0x5646c9467e70 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5646c91f3c50 con 0x5646c941bc20 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c941b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c941b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44998/0 (socket says 172.21.15.86:44998) 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 -- 172.21.15.86:0/3549060203 learned_addr learned my addr 172.21.15.86:0/3549060203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 -- 172.21.15.86:0/3549060203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 msgr2=0x5646c941c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 --2- 172.21.15.86:0/3549060203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 0x5646c941c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 -- 172.21.15.86:0/3549060203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 msgr2=0x5646c9468250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 --2- 172.21.15.86:0/3549060203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 0x5646c9468250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 -- 172.21.15.86:0/3549060203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c92b1f20 con 0x5646c9468790 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7eabe00640 1 --2- 172.21.15.86:0/3549060203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c941b6e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0f0130 tx=0x7f7e9c0f0430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb3699450e5edae server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ea2a00640 1 -- 172.21.15.86:0/3549060203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c108020 con 0x5646c9468790 2024-03-22T04:15:16.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ea2a00640 1 -- 172.21.15.86:0/3549060203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0f8d00 con 0x5646c9468790 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ea2a00640 1 -- 172.21.15.86:0/3549060203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0f4160 con 0x5646c9468790 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- 172.21.15.86:0/3549060203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 msgr2=0x5646c941b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/3549060203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c941b6e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0f0130 tx=0x7f7e9c0f0430 comp rx=0 tx=0).stop 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- 172.21.15.86:0/3549060203 shutdown_connections 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/3549060203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 0x5646c941c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/3549060203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c941b6e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/3549060203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 0x5646c9468250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- 172.21.15.86:0/3549060203 >> 172.21.15.86:0/3549060203 conn(0x5646c9448c10 msgr2=0x5646c944b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- 172.21.15.86:0/3549060203 shutdown_connections 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- 172.21.15.86:0/3549060203 wait complete. 2024-03-22T04:15:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.043+0000 7f7ebc66c000 1 -- start start 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 Processor -- start 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c941bc20 0x5646c9420810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9467e70 0x5646c9420d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c9421290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646c9226ab0 con 0x5646c9468790 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646c9268ac0 con 0x5646c941bc20 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5646c9219140 con 0x5646c9467e70 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 10 client.0 _ll_register_callbacks cb 0x5646c9409380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c9421290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c9421290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45010/0 (socket says 172.21.15.86:45010) 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9467e70 0x5646c9420d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 -- 172.21.15.86:0/2217087525 learned_addr learned my addr 172.21.15.86:0/2217087525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c941bc20 0x5646c9420810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 -- 172.21.15.86:0/2217087525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9467e70 msgr2=0x5646c9420d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9467e70 0x5646c9420d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 -- 172.21.15.86:0/2217087525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c941bc20 msgr2=0x5646c9420810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c941bc20 0x5646c9420810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 -- 172.21.15.86:0/2217087525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c9242570 con 0x5646c9468790 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eabe00640 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9467e70 0x5646c9420d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb2800640 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c9421290 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7eac014d80 tx=0x7f7eac012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b58cea8d990ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7e97400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7e97400640 1 -- 172.21.15.86:0/2217087525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac009c10 con 0x5646c9468790 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7e97400640 1 -- 172.21.15.86:0/2217087525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7eac003ba0 con 0x5646c9468790 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2217087525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9c0f8d00 con 0x5646c9468790 2024-03-22T04:15:16.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7e97400640 1 -- 172.21.15.86:0/2217087525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7eac027050 con 0x5646c9468790 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 10 client.6924 fetch_fsmap learned FSMap version 46 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2217087525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5646c9238880 con 0x5646c9468790 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7e97400640 1 -- 172.21.15.86:0/2217087525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0f8d00 con 0x5646c9468790 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7e97400640 1 -- 172.21.15.86:0/2217087525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f7eac02a020 con 0x5646c9468790 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 10 client.6924 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 10 client.6924 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 20 client.6924 populate_metadata read hostname 'smithi086' 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] conn(0x5646c9421ca0 0x5646c9479120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 4 client.6924 mds_command: new command op to 6807 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2217087525 --> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c9207bc0 con 0x5646c9421ca0 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.047+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] conn(0x5646c9421ca0 0x5646c9479120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] conn(0x5646c9421ca0 0x5646c9479120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99e2dbe3924d238 server_cookie=7ca5425c98bc8dfc in_seq=0 out_seq=0 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7e97400640 10 client.6924 ms_handle_connect on v2:172.21.15.86:6836/1130704586 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7e97400640 1 -- 172.21.15.86:0/2217087525 <== mds.0 v2:172.21.15.86:6836/1130704586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5646c9207bc0 con 0x5646c9421ca0 2024-03-22T04:15:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7e97400640 10 client.6924 handle_command_reply: tid=0 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 client.6924 shutdown 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2217087525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 msgr2=0x5646c9421290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c9421290 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7eac014d80 tx=0x7f7eac012670 comp rx=0 tx=0).stop 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2217087525 shutdown_connections 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9468790 0x5646c9421290 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9467e70 0x5646c9420d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] conn(0x5646c9421ca0 0x5646c9479120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2217087525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c941bc20 0x5646c9420810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2217087525 >> 172.21.15.86:0/2217087525 conn(0x5646c9448c10 msgr2=0x5646c944b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7e97400640 0 client.6924 ms_handle_reset on v2:172.21.15.86:6836/1130704586 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2217087525 shutdown_connections 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2217087525 wait complete. 2024-03-22T04:15:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.051+0000 7f7ebc66c000 20 client.6924 trim_cache size 0 max 16384 2024-03-22T04:15:16.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.067+0000 7f7ebc66c000 1 Processor -- start 2024-03-22T04:15:16.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.067+0000 7f7ebc66c000 1 -- start start 2024-03-22T04:15:16.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9448c10 0x5646c9466f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c941bc20 0x5646c9467470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 0x5646c9479120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb4200890 con 0x5646c941bc20 2024-03-22T04:15:16.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40b72f0 con 0x5646c9467e70 2024-03-22T04:15:16.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb4200240 con 0x5646c9448c10 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c941bc20 0x5646c9467470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c941bc20 0x5646c9467470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45020/0 (socket says 172.21.15.86:45020) 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 -- 172.21.15.86:0/2375461727 learned_addr learned my addr 172.21.15.86:0/2375461727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eb2800640 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 0x5646c9479120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9448c10 0x5646c9466f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 -- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9448c10 msgr2=0x5646c9466f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9448c10 0x5646c9466f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 -- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 msgr2=0x5646c9479120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 0x5646c9479120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 -- 172.21.15.86:0/2375461727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c91f4e80 con 0x5646c941bc20 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9448c10 0x5646c9466f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7eabe00640 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c941bc20 0x5646c9467470 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c01c8e0 tx=0x7f7e9c0f3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fff1d1fcba62db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ea2000640 1 -- 172.21.15.86:0/2375461727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c108040 con 0x5646c941bc20 2024-03-22T04:15:16.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ea2000640 1 -- 172.21.15.86:0/2375461727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0ee730 con 0x5646c941bc20 2024-03-22T04:15:16.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ea2000640 1 -- 172.21.15.86:0/2375461727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0f0690 con 0x5646c941bc20 2024-03-22T04:15:16.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2375461727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c941bc20 msgr2=0x5646c9467470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c941bc20 0x5646c9467470 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c01c8e0 tx=0x7f7e9c0f3000 comp rx=0 tx=0).stop 2024-03-22T04:15:16.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2375461727 shutdown_connections 2024-03-22T04:15:16.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9467e70 0x5646c9479120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c941bc20 0x5646c9467470 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/2375461727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c9448c10 0x5646c9466f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2375461727 >> 172.21.15.86:0/2375461727 conn(0x5646c9423490 msgr2=0x5646c941ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2375461727 shutdown_connections 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- 172.21.15.86:0/2375461727 wait complete. 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 -- start start 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.071+0000 7f7ebc66c000 1 Processor -- start 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 0x5646c947c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9448c10 0x5646c9492a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9467e70 0x5646c9492fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646c9212e80 con 0x5646c9467e70 2024-03-22T04:15:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646c926cb10 con 0x5646c9448c10 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5646c9328b30 con 0x5646c941bc20 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 10 client.0 _ll_register_callbacks cb 0x7f7eac004ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 0x5646c947c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9467e70 0x5646c9492fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9448c10 0x5646c9492a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 0x5646c947c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:49398/0 (socket says 172.21.15.86:49398) 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb1e00640 1 -- 172.21.15.86:0/649759419 learned_addr learned my addr 172.21.15.86:0/649759419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb2800640 1 -- 172.21.15.86:0/649759419 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 msgr2=0x5646c947c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb2800640 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 0x5646c947c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb2800640 1 -- 172.21.15.86:0/649759419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9448c10 msgr2=0x5646c9492a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb2800640 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9448c10 0x5646c9492a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb2800640 1 -- 172.21.15.86:0/649759419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c9200750 con 0x5646c9467e70 2024-03-22T04:15:16.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eabe00640 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9448c10 0x5646c9492a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:16.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb2800640 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9467e70 0x5646c9492fb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7eac014740 tx=0x7f7eac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82196fbc9fc1771 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:15:16.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 1 -- 172.21.15.86:0/649759419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac024070 con 0x5646c9467e70 2024-03-22T04:15:16.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 1 -- 172.21.15.86:0/649759419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7eac01fbc0 con 0x5646c9467e70 2024-03-22T04:15:16.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- 172.21.15.86:0/649759419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9c0ee730 con 0x5646c9467e70 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 1 -- 172.21.15.86:0/649759419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7eac009bf0 con 0x5646c9467e70 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 10 client.6930 fetch_fsmap learned FSMap version 46 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- 172.21.15.86:0/649759419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5646c9213450 con 0x5646c9467e70 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 1 -- 172.21.15.86:0/649759419 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0ee730 con 0x5646c9467e70 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 1 -- 172.21.15.86:0/649759419 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f7eac033020 con 0x5646c9467e70 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 10 client.6930 fetch_fsmap finished waiting for FSMap version 46 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 10 client.6930 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 20 client.6930 populate_metadata read hostname 'smithi086' 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] conn(0x5646c9468790 0x5646c94d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 4 client.6930 mds_command: new command op to 6807 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- 172.21.15.86:0/649759419 --> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] -- command(tid 0: {"prefix": "status"}) -- 0x5646c927a050 con 0x5646c9468790 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] conn(0x5646c9468790 0x5646c94d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7eb1e00640 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] conn(0x5646c9468790 0x5646c94d7310 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c943345b113b123d server_cookie=5120bb4fb3512d4b in_seq=0 out_seq=0 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 10 client.6930 ms_handle_connect on v2:172.21.15.86:6836/1130704586 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 1 -- 172.21.15.86:0/649759419 <== mds.0 v2:172.21.15.86:6836/1130704586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5646c927a050 con 0x5646c9468790 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 10 client.6930 handle_command_reply: tid=0 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 client.6930 shutdown 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- 172.21.15.86:0/649759419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9467e70 msgr2=0x5646c9492fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9467e70 0x5646c9492fb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7eac014740 tx=0x7f7eac00b040 comp rx=0 tx=0).stop 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- 172.21.15.86:0/649759419 shutdown_connections 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.86:6836/1130704586,v1:172.21.15.86:6837/1130704586] conn(0x5646c9468790 0x5646c94d7310 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c9448c10 0x5646c9492a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646c9467e70 0x5646c9492fb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 --2- 172.21.15.86:0/649759419 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646c941bc20 0x5646c947c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7e97400640 0 client.6930 ms_handle_reset on v2:172.21.15.86:6836/1130704586 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- 172.21.15.86:0/649759419 >> 172.21.15.86:0/649759419 conn(0x5646c9423490 msgr2=0x5646c9465f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- 172.21.15.86:0/649759419 shutdown_connections 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 1 -- 172.21.15.86:0/649759419 wait complete. 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.075+0000 7f7ebc66c000 20 client.6930 trim_cache size 0 max 16384 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:15:16.086 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6807, 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.0074978420000003, 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 46, 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 80, 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 10.774742001 2024-03-22T04:15:16.087 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 -- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e90067b20 msgr2=0x7f7e90069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e90067b20 0x7f7e90069fe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4007d10 tx=0x7f7ea4001c50 comp rx=0 tx=0).stop 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 -- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb40cb680 msgr2=0x7f7eb41f38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb40cb680 0x7f7eb41f38c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005040 tx=0x7f7e98005840 comp rx=0 tx=0).stop 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 -- 172.21.15.86:0/2834169264 shutdown_connections 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e90067b20 0x7f7e90069fe0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eb40cd900 0x7f7eb40cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7eb40cb680 0x7f7eb41f38c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 --2- 172.21.15.86:0/2834169264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e98004810 0x7f7eb41596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 -- 172.21.15.86:0/2834169264 >> 172.21.15.86:0/2834169264 conn(0x7f7eb406f910 msgr2=0x7f7eb415c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 -- 172.21.15.86:0/2834169264 shutdown_connections 2024-03-22T04:15:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.079+0000 7f7eb9200640 1 -- 172.21.15.86:0/2834169264 wait complete. 2024-03-22T04:15:16.110 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:15:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5879000640 1 Processor -- start 2024-03-22T04:15:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5879000640 1 -- start start 2024-03-22T04:15:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5879000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58740d0de0 0x7f58740ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5879000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d19a0 0x7f58740cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5879000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f58740d1dc0 0x7f587415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5879000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f58740d11c0 con 0x7f58740d0de0 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5879000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f58740cece0 con 0x7f58740d1dc0 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5879000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58740cefc0 con 0x7f58740d19a0 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5871e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58740d0de0 0x7f58740ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5871e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58740d0de0 0x7f58740ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45040/0 (socket says 172.21.15.86:45040) 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5871e00640 1 -- 172.21.15.86:0/3019157046 learned_addr learned my addr 172.21.15.86:0/3019157046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5872800640 1 --2- 172.21.15.86:0/3019157046 >> v2:172.21.15.196:3300/0 conn(0x7f58740d1dc0 0x7f587415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5869400640 1 --2- 172.21.15.86:0/3019157046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d19a0 0x7f58740cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5871e00640 1 -- 172.21.15.86:0/3019157046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d19a0 msgr2=0x7f58740cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5871e00640 1 --2- 172.21.15.86:0/3019157046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d19a0 0x7f58740cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5871e00640 1 -- 172.21.15.86:0/3019157046 >> v2:172.21.15.196:3300/0 conn(0x7f58740d1dc0 msgr2=0x7f587415fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5871e00640 1 --2- 172.21.15.86:0/3019157046 >> v2:172.21.15.196:3300/0 conn(0x7f58740d1dc0 0x7f587415fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5871e00640 1 -- 172.21.15.86:0/3019157046 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58740cf2a0 con 0x7f58740d0de0 2024-03-22T04:15:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5872800640 1 --2- 172.21.15.86:0/3019157046 >> v2:172.21.15.196:3300/0 conn(0x7f58740d1dc0 0x7f587415fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.243+0000 7f5869400640 1 --2- 172.21.15.86:0/3019157046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d19a0 0x7f58740cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.247+0000 7f5871e00640 1 --2- 172.21.15.86:0/3019157046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d0de0 0x7f58740ce6b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f58740b3e00 tx=0x7f586401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3f599e1f2dcc09 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5871400640 1 -- 172.21.15.86:0/3019157046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58640b0070 con 0x7f58740d0de0 2024-03-22T04:15:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5871400640 1 -- 172.21.15.86:0/3019157046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58640ab600 con 0x7f58740d0de0 2024-03-22T04:15:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5871400640 1 -- 172.21.15.86:0/3019157046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58640ab9d0 con 0x7f58740d0de0 2024-03-22T04:15:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 -- 172.21.15.86:0/3019157046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d0de0 msgr2=0x7f58740ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 --2- 172.21.15.86:0/3019157046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d0de0 0x7f58740ce6b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f58740b3e00 tx=0x7f586401c520 comp rx=0 tx=0).stop 2024-03-22T04:15:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 -- 172.21.15.86:0/3019157046 shutdown_connections 2024-03-22T04:15:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 --2- 172.21.15.86:0/3019157046 >> v2:172.21.15.196:3300/0 conn(0x7f58740d1dc0 0x7f587415fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 --2- 172.21.15.86:0/3019157046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d19a0 0x7f58740cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 --2- 172.21.15.86:0/3019157046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d0de0 0x7f58740ce6b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 -- 172.21.15.86:0/3019157046 >> 172.21.15.86:0/3019157046 conn(0x7f587406f910 msgr2=0x7f587406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 -- 172.21.15.86:0/3019157046 shutdown_connections 2024-03-22T04:15:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 -- 172.21.15.86:0/3019157046 wait complete. 2024-03-22T04:15:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 Processor -- start 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.251+0000 7f5879000640 1 -- start start 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58740d0de0 0x7f58741fb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d19a0 0x7f58740b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d1dc0 0x7f58740b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5874071fb0 con 0x7f58740d19a0 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5874071e30 con 0x7f58740d0de0 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f587415c8f0 con 0x7f58740d1dc0 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d19a0 0x7f58740b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d19a0 0x7f58740b7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45050/0 (socket says 172.21.15.86:45050) 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 -- 172.21.15.86:0/2075763765 learned_addr learned my addr 172.21.15.86:0/2075763765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5872800640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d1dc0 0x7f58740b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5871e00640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58740d0de0 0x7f58741fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 -- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d1dc0 msgr2=0x7f58740b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d1dc0 0x7f58740b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 -- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58740d0de0 msgr2=0x7f58741fb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58740d0de0 0x7f58741fb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 -- 172.21.15.86:0/2075763765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58740b52f0 con 0x7f58740d19a0 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5872800640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d1dc0 0x7f58740b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5871e00640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58740d0de0 0x7f58741fb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5869400640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d19a0 0x7f58740b7ab0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e810 tx=0x7f585c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f586aa00640 1 -- 172.21.15.86:0/2075763765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c00f040 con 0x7f58740d19a0 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f586aa00640 1 -- 172.21.15.86:0/2075763765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c004030 con 0x7f58740d19a0 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58740b6cf0 con 0x7f58740d19a0 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f586aa00640 1 -- 172.21.15.86:0/2075763765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c004400 con 0x7f58740d19a0 2024-03-22T04:15:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58740b7590 con 0x7f58740d19a0 2024-03-22T04:15:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.255+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58740b70e0 con 0x7f58740d19a0 2024-03-22T04:15:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.259+0000 7f586aa00640 1 -- 172.21.15.86:0/2075763765 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f585c00c6d0 con 0x7f58740d19a0 2024-03-22T04:15:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.259+0000 7f586aa00640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5850067b20 0x7f5850069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.259+0000 7f586aa00640 1 -- 172.21.15.86:0/2075763765 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f585c085cf0 con 0x7f58740d19a0 2024-03-22T04:15:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.259+0000 7f5871e00640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5850067b20 0x7f5850069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.259+0000 7f5871e00640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5850067b20 0x7f5850069fe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f58640ad000 tx=0x7f5864008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.263+0000 7f586aa00640 1 -- 172.21.15.86:0/2075763765 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f585c08a070 con 0x7f58740d19a0 2024-03-22T04:15:16.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5874068650 con 0x7f58740d19a0 2024-03-22T04:15:16.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f586aa00640 1 -- 172.21.15.86:0/2075763765 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f585c017020 con 0x7f58740d19a0 2024-03-22T04:15:16.458 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5850067b20 msgr2=0x7f5850069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5850067b20 0x7f5850069fe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f58640ad000 tx=0x7f5864008040 comp rx=0 tx=0).stop 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d19a0 msgr2=0x7f58740b7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d19a0 0x7f58740b7ab0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e810 tx=0x7f585c00ecd0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 shutdown_connections 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5850067b20 0x7f5850069fe0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58740d1dc0 0x7f58740b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58740d19a0 0x7f58740b7ab0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 --2- 172.21.15.86:0/2075763765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58740d0de0 0x7f58741fb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 >> 172.21.15.86:0/2075763765 conn(0x7f587406f910 msgr2=0x7f58741592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 shutdown_connections 2024-03-22T04:15:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16.455+0000 7f5879000640 1 -- 172.21.15.86:0/2075763765 wait complete. 2024-03-22T04:15:16.479 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:15:16.479 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:15:16.479 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:15:16.479 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:15:16.479 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:16.479 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:15:16.479 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:15:16.479 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:15:16.480 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:15:16.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:16.480 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:15:16.489 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:15:16.490 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42215sec preferred_lft 42215sec 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:16.491 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:15:16.492 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:16.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:16.493 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:15:16.555 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:15:16.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:16.557 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:15:16.613 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:15:16.614 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:15:16.614 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:15:16.614 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:15:16.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:15:16.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:15:16.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:15:16.614 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:15:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:15:16.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:15:16.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:16.751 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:15:16.760 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:15:16.760 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:15:16.760 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:15:16.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:16.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:15:16.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:15:16.821 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:16.870 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:15:16.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:15:16.871 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:16.922 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:15:16.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:16.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:15:17.018 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:15:17.019 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:15:17.019 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:15:17.033 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:15:17.034 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:15:17.034 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:15:17.034 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:15:17.034 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:15:17.043 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:15:17.043 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:15:17.043 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:15:17.043 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:15:17.044 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:15:17.044 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:15:17.044 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:15:17.044 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:15:17.048 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:15:17.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:17.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:17.068 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:15:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:15:17.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:15:17.173 INFO:teuthology.orchestra.run.smithi086.stdout:client6859 2024-03-22T04:15:17.173 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:15:17.173 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:17.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:17.182 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-03-22T04:15:17.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:17.240 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-03-22T04:15:17.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:17.292 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-03-22T04:15:17.342 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:15:17.342 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:15:17.342 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:15:17.342 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:15:17.342 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-03-22T04:15:17.342 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:15:17.448 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-03-22T04:15:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:15:17.457 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:15:17.457 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:17.506 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:15:17.506 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:17.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:15:17.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.731+0000 7f3bf6e00640 1 Processor -- start 2024-03-22T04:15:17.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.731+0000 7f3bf6e00640 1 -- start start 2024-03-22T04:15:17.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.731+0000 7f3bf6e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3bf00ce6b0 0x7f3bf00cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:17.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.731+0000 7f3bf6e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 0x7f3bf0160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.731+0000 7f3bf6e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf00d11c0 con 0x7f3bf00d0de0 2024-03-22T04:15:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.731+0000 7f3bf6e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3bf0160860 con 0x7f3bf00ce6b0 2024-03-22T04:15:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.731+0000 7f3bf6e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf00cece0 con 0x7f3bf015fed0 2024-03-22T04:15:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3be7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3bf00ce6b0 0x7f3bf00cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3befe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3bf00d0de0 0x7f3bf00d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36534/0 (socket says 172.21.15.86:36534) 2024-03-22T04:15:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3befe00640 1 -- 172.21.15.86:0/113561417 learned_addr learned my addr 172.21.15.86:0/113561417 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3bf4e00640 1 --2- 172.21.15.86:0/113561417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 0x7f3bf0160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:17.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3bef400640 1 -- 172.21.15.86:0/113561417 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155881094 0 0) 0x7f3bf00d11c0 con 0x7f3bf00d0de0 2024-03-22T04:15:17.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3bef400640 1 -- 172.21.15.86:0/113561417 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bd4003530 con 0x7f3bf00d0de0 2024-03-22T04:15:17.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3be7e00640 1 -- 172.21.15.86:0/113561417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 msgr2=0x7f3bf0160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:17.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3be7e00640 1 --2- 172.21.15.86:0/113561417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 0x7f3bf0160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3be7e00640 1 -- 172.21.15.86:0/113561417 >> v1:172.21.15.86:6789/0 conn(0x7f3bf00d0de0 legacy=0x7f3bf00d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:17.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3be7e00640 1 -- 172.21.15.86:0/113561417 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf00cefc0 con 0x7f3bf00ce6b0 2024-03-22T04:15:17.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3be7e00640 1 --2- 172.21.15.86:0/113561417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00ce6b0 0x7f3bf00cfe70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80097c0 tx=0x7f3bd801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c96fc99a6e94880 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:17.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3bef400640 1 -- 172.21.15.86:0/113561417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80b0070 con 0x7f3bf00ce6b0 2024-03-22T04:15:17.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3bef400640 1 -- 172.21.15.86:0/113561417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd80ab600 con 0x7f3bf00ce6b0 2024-03-22T04:15:17.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.735+0000 7f3bef400640 1 -- 172.21.15.86:0/113561417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80ab9d0 con 0x7f3bf00ce6b0 2024-03-22T04:15:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- 172.21.15.86:0/113561417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00ce6b0 msgr2=0x7f3bf00cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/113561417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00ce6b0 0x7f3bf00cfe70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80097c0 tx=0x7f3bd801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- 172.21.15.86:0/113561417 shutdown_connections 2024-03-22T04:15:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/113561417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 0x7f3bf0160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/113561417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00ce6b0 0x7f3bf00cfe70 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- 172.21.15.86:0/113561417 >> 172.21.15.86:0/113561417 conn(0x7f3bf006f910 msgr2=0x7f3bf006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- 172.21.15.86:0/113561417 shutdown_connections 2024-03-22T04:15:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- 172.21.15.86:0/113561417 wait complete. 2024-03-22T04:15:17.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 Processor -- start 2024-03-22T04:15:17.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- start start 2024-03-22T04:15:17.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bf00ce6b0 0x7f3bf0150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00d0de0 0x7f3bf006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 0x7f3bf01653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf00d0530 con 0x7f3bf00d0de0 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf00d03b0 con 0x7f3bf00ce6b0 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf6e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf0071e30 con 0x7f3bf015fed0 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00d0de0 0x7f3bf006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00d0de0 0x7f3bf006d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45066/0 (socket says 172.21.15.86:45066) 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3be7e00640 1 -- 172.21.15.86:0/4193949726 learned_addr learned my addr 172.21.15.86:0/4193949726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3befe00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bf00ce6b0 0x7f3bf0150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3bf4e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 0x7f3bf01653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3be7e00640 1 -- 172.21.15.86:0/4193949726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 msgr2=0x7f3bf01653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3be7e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 0x7f3bf01653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3be7e00640 1 -- 172.21.15.86:0/4193949726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bf00ce6b0 msgr2=0x7f3bf0150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3be7e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bf00ce6b0 0x7f3bf0150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.739+0000 7f3be7e00640 1 -- 172.21.15.86:0/4193949726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf006dc40 con 0x7f3bf00d0de0 2024-03-22T04:15:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3be7e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00d0de0 0x7f3bf006d180 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80ae040 tx=0x7f3bd8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3becc00640 1 -- 172.21.15.86:0/4193949726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80ba070 con 0x7f3bf00d0de0 2024-03-22T04:15:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3becc00640 1 -- 172.21.15.86:0/4193949726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd80b43f0 con 0x7f3bf00d0de0 2024-03-22T04:15:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3becc00640 1 -- 172.21.15.86:0/4193949726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80b0040 con 0x7f3bf00d0de0 2024-03-22T04:15:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf006df20 con 0x7f3bf00d0de0 2024-03-22T04:15:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf006e4e0 con 0x7f3bf00d0de0 2024-03-22T04:15:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf0068650 con 0x7f3bf00d0de0 2024-03-22T04:15:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3becc00640 1 -- 172.21.15.86:0/4193949726 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3bd80b4800 con 0x7f3bf00d0de0 2024-03-22T04:15:17.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3becc00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3bcc067b50 0x7f3bcc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:17.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3becc00640 1 -- 172.21.15.86:0/4193949726 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f3bd811efb0 con 0x7f3bf00d0de0 2024-03-22T04:15:17.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.743+0000 7f3befe00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3bcc067b50 0x7f3bcc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:17.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.747+0000 7f3befe00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3bcc067b50 0x7f3bcc06a010 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007fd0 tx=0x7f3be0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.747+0000 7f3becc00640 1 -- 172.21.15.86:0/4193949726 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3bd8016090 con 0x7f3bf00d0de0 2024-03-22T04:15:17.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.907+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bf00ccef0 con 0x7f3bf00d0de0 2024-03-22T04:15:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.907+0000 7f3becc00640 1 -- 172.21.15.86:0/4193949726 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16885 (secure 0 0 0) 0x7f3bd80ef270 con 0x7f3bf00d0de0 2024-03-22T04:15:17.913 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:17.913 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:11.057726+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:15:17.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3bcc067b50 msgr2=0x7f3bcc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:17.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3bcc067b50 0x7f3bcc06a010 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007fd0 tx=0x7f3be0009040 comp rx=0 tx=0).stop 2024-03-22T04:15:17.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00d0de0 msgr2=0x7f3bf006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:17.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00d0de0 0x7f3bf006d180 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80ae040 tx=0x7f3bd8019240 comp rx=0 tx=0).stop 2024-03-22T04:15:17.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 shutdown_connections 2024-03-22T04:15:17.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3bcc067b50 0x7f3bcc06a010 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bf015fed0 0x7f3bf01653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf00d0de0 0x7f3bf006d180 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 --2- 172.21.15.86:0/4193949726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bf00ce6b0 0x7f3bf0150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:17.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 >> 172.21.15.86:0/4193949726 conn(0x7f3bf006f910 msgr2=0x7f3bf00d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:17.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 shutdown_connections 2024-03-22T04:15:17.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:17.911+0000 7f3bf6e00640 1 -- 172.21.15.86:0/4193949726 wait complete. 2024-03-22T04:15:17.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:15:07.783245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:15:08.394997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:15:17.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:15:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.071+0000 7f905a800640 1 Processor -- start 2024-03-22T04:15:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f905a800640 1 -- start start 2024-03-22T04:15:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f905a800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f905415c2f0 0x7f90540ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f905a800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90540d2af0 0x7f90540cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f905a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90540d02d0 con 0x7f90540b20e0 2024-03-22T04:15:18.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f905a800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f905415c6d0 con 0x7f905415c2f0 2024-03-22T04:15:18.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f905a800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f90540cfac0 con 0x7f90540d2af0 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9059e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f90540b20e0 0x7f90540b2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36542/0 (socket says 172.21.15.86:36542) 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9059e00640 1 -- 172.21.15.86:0/3652782795 learned_addr learned my addr 172.21.15.86:0/3652782795 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9058a00640 1 --2- 172.21.15.86:0/3652782795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90540d2af0 0x7f90540cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9059400640 1 --2- 172.21.15.86:0/3652782795 >> v2:172.21.15.196:3300/0 conn(0x7f905415c2f0 0x7f90540ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9053e00640 1 -- 172.21.15.86:0/3652782795 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 578611784 0 0) 0x7f904c0014a0 con 0x7f90540b20e0 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9053e00640 1 -- 172.21.15.86:0/3652782795 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f903c003530 con 0x7f90540b20e0 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9058a00640 1 -- 172.21.15.86:0/3652782795 >> v2:172.21.15.196:3300/0 conn(0x7f905415c2f0 msgr2=0x7f90540ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9058a00640 1 --2- 172.21.15.86:0/3652782795 >> v2:172.21.15.196:3300/0 conn(0x7f905415c2f0 0x7f90540ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9058a00640 1 -- 172.21.15.86:0/3652782795 >> v1:172.21.15.86:6789/0 conn(0x7f90540b20e0 legacy=0x7f90540b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.075+0000 7f9058a00640 1 -- 172.21.15.86:0/3652782795 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90540cfd10 con 0x7f90540d2af0 2024-03-22T04:15:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f9058a00640 1 --2- 172.21.15.86:0/3652782795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90540d2af0 0x7f90540cedb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f90400097c0 tx=0x7f904001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16460d83492a8b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f9053e00640 1 -- 172.21.15.86:0/3652782795 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90400b1070 con 0x7f90540d2af0 2024-03-22T04:15:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f9053e00640 1 -- 172.21.15.86:0/3652782795 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90400a8050 con 0x7f90540d2af0 2024-03-22T04:15:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f9053e00640 1 -- 172.21.15.86:0/3652782795 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90400ac660 con 0x7f90540d2af0 2024-03-22T04:15:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f905a800640 1 -- 172.21.15.86:0/3652782795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90540d2af0 msgr2=0x7f90540cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f905a800640 1 --2- 172.21.15.86:0/3652782795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90540d2af0 0x7f90540cedb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f90400097c0 tx=0x7f904001c3e0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f905a800640 1 -- 172.21.15.86:0/3652782795 shutdown_connections 2024-03-22T04:15:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f905a800640 1 --2- 172.21.15.86:0/3652782795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90540d2af0 0x7f90540cedb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f905a800640 1 --2- 172.21.15.86:0/3652782795 >> v2:172.21.15.196:3300/0 conn(0x7f905415c2f0 0x7f90540ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f905a800640 1 -- 172.21.15.86:0/3652782795 >> 172.21.15.86:0/3652782795 conn(0x7f90540d1ef0 msgr2=0x7f90540d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:18.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f905a800640 1 -- 172.21.15.86:0/3652782795 shutdown_connections 2024-03-22T04:15:18.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.079+0000 7f905a800640 1 -- 172.21.15.86:0/3652782795 wait complete. 2024-03-22T04:15:18.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f905a800640 1 Processor -- start 2024-03-22T04:15:18.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f905a800640 1 -- start start 2024-03-22T04:15:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f905a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90540b20e0 0x7f90540ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f905a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90540d2af0 0x7f9054154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f905a800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f905415c2f0 0x7f905415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f905a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90540ce280 con 0x7f90540d2af0 2024-03-22T04:15:18.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f905a800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90540ce100 con 0x7f90540b20e0 2024-03-22T04:15:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f905a800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9054166da0 con 0x7f905415c2f0 2024-03-22T04:15:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9058a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90540d2af0 0x7f9054154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9058a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90540d2af0 0x7f9054154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45078/0 (socket says 172.21.15.86:45078) 2024-03-22T04:15:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9058a00640 1 -- 172.21.15.86:0/3082093777 learned_addr learned my addr 172.21.15.86:0/3082093777 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9059400640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90540b20e0 0x7f90540ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9059e00640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f905415c2f0 0x7f905415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9058a00640 1 -- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f905415c2f0 msgr2=0x7f905415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9058a00640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f905415c2f0 0x7f905415d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9058a00640 1 -- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90540b20e0 msgr2=0x7f90540ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9058a00640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90540b20e0 0x7f90540ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9058a00640 1 -- 172.21.15.86:0/3082093777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90540c7120 con 0x7f90540d2af0 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.083+0000 7f9059e00640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f905415c2f0 0x7f905415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f9058a00640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90540d2af0 0x7f9054154410 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f90400af040 tx=0x7f9040019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f9051600640 1 -- 172.21.15.86:0/3082093777 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90400b1040 con 0x7f90540d2af0 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f9051600640 1 -- 172.21.15.86:0/3082093777 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90400bb070 con 0x7f90540d2af0 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90540c7400 con 0x7f90540d2af0 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f9051600640 1 -- 172.21.15.86:0/3082093777 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90400b5410 con 0x7f90540d2af0 2024-03-22T04:15:18.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90540c79c0 con 0x7f90540d2af0 2024-03-22T04:15:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905404f900 con 0x7f90540d2af0 2024-03-22T04:15:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f9051600640 1 -- 172.21.15.86:0/3082093777 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f90400b55b0 con 0x7f90540d2af0 2024-03-22T04:15:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.087+0000 7f9051600640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9034067b20 0x7f9034069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.091+0000 7f9051600640 1 -- 172.21.15.86:0/3082093777 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f904011fd00 con 0x7f90540d2af0 2024-03-22T04:15:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.091+0000 7f9059400640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9034067b20 0x7f9034069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.091+0000 7f9059400640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9034067b20 0x7f9034069fe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f904800df60 tx=0x7f904800f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.095+0000 7f9051600640 1 -- 172.21.15.86:0/3082093777 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f90400c5020 con 0x7f90540d2af0 2024-03-22T04:15:18.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f90540c89f0 con 0x7f90540d2af0 2024-03-22T04:15:18.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f9051600640 1 -- 172.21.15.86:0/3082093777 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f9040016090 con 0x7f90540d2af0 2024-03-22T04:15:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:15:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9034067b20 msgr2=0x7f9034069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9034067b20 0x7f9034069fe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f904800df60 tx=0x7f904800f040 comp rx=0 tx=0).stop 2024-03-22T04:15:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90540d2af0 msgr2=0x7f9054154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90540d2af0 0x7f9054154410 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f90400af040 tx=0x7f9040019180 comp rx=0 tx=0).stop 2024-03-22T04:15:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 shutdown_connections 2024-03-22T04:15:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9034067b20 0x7f9034069fe0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f905415c2f0 0x7f905415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90540d2af0 0x7f9054154410 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.251+0000 7f905a800640 1 --2- 172.21.15.86:0/3082093777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90540b20e0 0x7f90540ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.255+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 >> 172.21.15.86:0/3082093777 conn(0x7f90540d1ef0 msgr2=0x7f90540b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.255+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 shutdown_connections 2024-03-22T04:15:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.255+0000 7f905a800640 1 -- 172.21.15.86:0/3082093777 wait complete. 2024-03-22T04:15:18.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:15:18.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae77800640 1 Processor -- start 2024-03-22T04:15:18.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae77800640 1 -- start start 2024-03-22T04:15:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae77800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae7015e970 0x7fae700b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae77800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7006d8f0 0x7fae7009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae77800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7006dd70 con 0x7fae7009afb0 2024-03-22T04:15:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae77800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fae700b49a0 con 0x7fae7015e970 2024-03-22T04:15:18.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae77800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae700ccda0 con 0x7fae7006d8f0 2024-03-22T04:15:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae74e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae7015e970 0x7fae700b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae74e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae7015e970 0x7fae700b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45088/0 (socket says 172.21.15.86:45088) 2024-03-22T04:15:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae74e00640 1 -- 172.21.15.86:0/1224937168 learned_addr learned my addr 172.21.15.86:0/1224937168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae74e00640 1 -- 172.21.15.86:0/1224937168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7006d8f0 msgr2=0x7fae7009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:18.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae6fe00640 1 --2- 172.21.15.86:0/1224937168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7006d8f0 0x7fae7009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae74e00640 1 --2- 172.21.15.86:0/1224937168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7006d8f0 0x7fae7009aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae74e00640 1 -- 172.21.15.86:0/1224937168 >> v1:172.21.15.196:6789/0 conn(0x7fae7009afb0 legacy=0x7fae700d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:18.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae74e00640 1 -- 172.21.15.86:0/1224937168 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7006dff0 con 0x7fae7015e970 2024-03-22T04:15:18.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.415+0000 7fae6fe00640 1 --2- 172.21.15.86:0/1224937168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7006d8f0 0x7fae7009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:18.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae74e00640 1 --2- 172.21.15.86:0/1224937168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7015e970 0x7fae700b12e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fae64009870 tx=0x7fae6401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d658aa6d57f56e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae6f400640 1 -- 172.21.15.86:0/1224937168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640b0070 con 0x7fae7015e970 2024-03-22T04:15:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae6f400640 1 -- 172.21.15.86:0/1224937168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae640ab600 con 0x7fae7015e970 2024-03-22T04:15:18.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae6f400640 1 -- 172.21.15.86:0/1224937168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640ab9d0 con 0x7fae7015e970 2024-03-22T04:15:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae77800640 1 -- 172.21.15.86:0/1224937168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7015e970 msgr2=0x7fae700b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae77800640 1 --2- 172.21.15.86:0/1224937168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7015e970 0x7fae700b12e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fae64009870 tx=0x7fae6401c340 comp rx=0 tx=0).stop 2024-03-22T04:15:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae77800640 1 -- 172.21.15.86:0/1224937168 shutdown_connections 2024-03-22T04:15:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae77800640 1 --2- 172.21.15.86:0/1224937168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7006d8f0 0x7fae7009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae77800640 1 --2- 172.21.15.86:0/1224937168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7015e970 0x7fae700b12e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae77800640 1 -- 172.21.15.86:0/1224937168 >> 172.21.15.86:0/1224937168 conn(0x7fae700b3bf0 msgr2=0x7fae700b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae77800640 1 -- 172.21.15.86:0/1224937168 shutdown_connections 2024-03-22T04:15:18.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.419+0000 7fae77800640 1 -- 172.21.15.86:0/1224937168 wait complete. 2024-03-22T04:15:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 Processor -- start 2024-03-22T04:15:18.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 -- start start 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7006d8f0 0x7fae701530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7009afb0 0x7fae70155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7015e970 0x7fae701557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae7016b550 con 0x7fae7009afb0 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae7015ed50 con 0x7fae7006d8f0 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae7016b6d0 con 0x7fae7015e970 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7009afb0 0x7fae70155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7009afb0 0x7fae70155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45102/0 (socket says 172.21.15.86:45102) 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 -- 172.21.15.86:0/437960138 learned_addr learned my addr 172.21.15.86:0/437960138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:18.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae74e00640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7006d8f0 0x7fae701530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae75800640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7015e970 0x7fae701557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 -- 172.21.15.86:0/437960138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7015e970 msgr2=0x7fae701557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7015e970 0x7fae701557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 -- 172.21.15.86:0/437960138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7006d8f0 msgr2=0x7fae701530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7006d8f0 0x7fae701530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 -- 172.21.15.86:0/437960138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae700c5b60 con 0x7fae7009afb0 2024-03-22T04:15:18.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6fe00640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7009afb0 0x7fae70155270 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c7f0 tx=0x7fae5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6cc00640 1 -- 172.21.15.86:0/437960138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c007a30 con 0x7fae7009afb0 2024-03-22T04:15:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6cc00640 1 -- 172.21.15.86:0/437960138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c007bd0 con 0x7fae7009afb0 2024-03-22T04:15:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae6cc00640 1 -- 172.21.15.86:0/437960138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0113f0 con 0x7fae7009afb0 2024-03-22T04:15:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.423+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae700c5e40 con 0x7fae7009afb0 2024-03-22T04:15:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.427+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae700c6400 con 0x7fae7009afb0 2024-03-22T04:15:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.427+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae70074470 con 0x7fae7009afb0 2024-03-22T04:15:18.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.427+0000 7fae6cc00640 1 -- 172.21.15.86:0/437960138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fae5c011590 con 0x7fae7009afb0 2024-03-22T04:15:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.427+0000 7fae6cc00640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae50067b20 0x7fae50069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.427+0000 7fae6cc00640 1 -- 172.21.15.86:0/437960138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fae5c0861e0 con 0x7fae7009afb0 2024-03-22T04:15:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.427+0000 7fae74e00640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae50067b20 0x7fae50069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.427+0000 7fae74e00640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae50067b20 0x7fae50069fe0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fae640ad000 tx=0x7fae640a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.431+0000 7fae6cc00640 1 -- 172.21.15.86:0/437960138 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fae5c015020 con 0x7fae7009afb0 2024-03-22T04:15:18.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae701592d0 con 0x7fae7009afb0 2024-03-22T04:15:18.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae6cc00640 1 -- 172.21.15.86:0/437960138 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7fae5c056400 con 0x7fae7009afb0 2024-03-22T04:15:18.594 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:15:18.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae50067b20 msgr2=0x7fae50069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae50067b20 0x7fae50069fe0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fae640ad000 tx=0x7fae640a6000 comp rx=0 tx=0).stop 2024-03-22T04:15:18.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7009afb0 msgr2=0x7fae70155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7009afb0 0x7fae70155270 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c7f0 tx=0x7fae5c00ccb0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 shutdown_connections 2024-03-22T04:15:18.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fae50067b20 0x7fae50069fe0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae7015e970 0x7fae701557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7009afb0 0x7fae70155270 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 --2- 172.21.15.86:0/437960138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae7006d8f0 0x7fae701530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 >> 172.21.15.86:0/437960138 conn(0x7fae700b3bf0 msgr2=0x7fae700b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:18.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 shutdown_connections 2024-03-22T04:15:18.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.591+0000 7fae77800640 1 -- 172.21.15.86:0/437960138 wait complete. 2024-03-22T04:15:18.615 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:15:18.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c9000640 1 Processor -- start 2024-03-22T04:15:18.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c9000640 1 -- start start 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c9000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78c415c730 0x7f78c409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78c40b11c0 0x7f78c40b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c9000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c40cda00 con 0x7f78c40b3610 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c9000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f78c409b910 con 0x7f78c40b11c0 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c9000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f78c40b1710 con 0x7f78c415c730 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78c40b11c0 0x7f78c40b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f78c40b3610 0x7f78c409afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36558/0 (socket says 172.21.15.86:36558) 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c1400640 1 -- 172.21.15.86:0/250310000 learned_addr learned my addr 172.21.15.86:0/250310000 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c1e00640 1 --2- 172.21.15.86:0/250310000 >> v2:172.21.15.196:3300/0 conn(0x7f78c415c730 0x7f78c409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c2800640 1 -- 172.21.15.86:0/250310000 >> v2:172.21.15.196:3300/0 conn(0x7f78c415c730 msgr2=0x7f78c409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:18.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c2800640 1 --2- 172.21.15.86:0/250310000 >> v2:172.21.15.196:3300/0 conn(0x7f78c415c730 0x7f78c409aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c2800640 1 -- 172.21.15.86:0/250310000 >> v1:172.21.15.86:6789/0 conn(0x7f78c40b3610 legacy=0x7f78c409afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.751+0000 7f78c2800640 1 -- 172.21.15.86:0/250310000 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c40cdce0 con 0x7f78c40b11c0 2024-03-22T04:15:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c2800640 1 --2- 172.21.15.86:0/250310000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c40b1ba0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009d90 tx=0x7f78bc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1889e777a985c76 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c0a00640 1 -- 172.21.15.86:0/250310000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0b2070 con 0x7f78c40b11c0 2024-03-22T04:15:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c0a00640 1 -- 172.21.15.86:0/250310000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78bc0a5870 con 0x7f78c40b11c0 2024-03-22T04:15:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c0a00640 1 -- 172.21.15.86:0/250310000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0a5c40 con 0x7f78c40b11c0 2024-03-22T04:15:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c9000640 1 -- 172.21.15.86:0/250310000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 msgr2=0x7f78c40b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c9000640 1 --2- 172.21.15.86:0/250310000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c40b1ba0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009d90 tx=0x7f78bc01c8a0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c9000640 1 -- 172.21.15.86:0/250310000 shutdown_connections 2024-03-22T04:15:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c9000640 1 --2- 172.21.15.86:0/250310000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c40b1ba0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c9000640 1 --2- 172.21.15.86:0/250310000 >> v2:172.21.15.196:3300/0 conn(0x7f78c415c730 0x7f78c409aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c9000640 1 -- 172.21.15.86:0/250310000 >> 172.21.15.86:0/250310000 conn(0x7f78c40b3bf0 msgr2=0x7f78c40b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c9000640 1 -- 172.21.15.86:0/250310000 shutdown_connections 2024-03-22T04:15:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.755+0000 7f78c9000640 1 -- 172.21.15.86:0/250310000 wait complete. 2024-03-22T04:15:18.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 Processor -- start 2024-03-22T04:15:18.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 -- start start 2024-03-22T04:15:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78c40b3610 0x7f78c415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c415c730 0x7f78c406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40b2c80 con 0x7f78c40b11c0 2024-03-22T04:15:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40b2b00 con 0x7f78c415c730 2024-03-22T04:15:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f78c40b2e00 con 0x7f78c40b3610 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45126/0 (socket says 172.21.15.86:45126) 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 -- 172.21.15.86:0/2954442679 learned_addr learned my addr 172.21.15.86:0/2954442679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 -- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78c40b3610 msgr2=0x7f78c415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1400640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78c40b3610 0x7f78c415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c2800640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c415c730 0x7f78c406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78c40b3610 0x7f78c415a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 -- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c415c730 msgr2=0x7f78c406d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c415c730 0x7f78c406d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 -- 172.21.15.86:0/2954442679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c4159d90 con 0x7f78c40b11c0 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1400640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78c40b3610 0x7f78c415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:18.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c2800640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c415c730 0x7f78c406d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c1e00640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c415a260 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f78b400e870 tx=0x7f78b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78ba000640 1 -- 172.21.15.86:0/2954442679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4017070 con 0x7f78c40b11c0 2024-03-22T04:15:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c406dd60 con 0x7f78c40b11c0 2024-03-22T04:15:18.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c4153250 con 0x7f78c40b11c0 2024-03-22T04:15:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78ba000640 1 -- 172.21.15.86:0/2954442679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b40059a0 con 0x7f78c40b11c0 2024-03-22T04:15:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78ba000640 1 -- 172.21.15.86:0/2954442679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4021060 con 0x7f78c40b11c0 2024-03-22T04:15:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.759+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f789c0052a0 con 0x7f78c40b11c0 2024-03-22T04:15:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.763+0000 7f78ba000640 1 -- 172.21.15.86:0/2954442679 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f78b401a070 con 0x7f78c40b11c0 2024-03-22T04:15:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.763+0000 7f78ba000640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f78a4067b50 0x7f78a406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:18.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.763+0000 7f78ba000640 1 -- 172.21.15.86:0/2954442679 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f78b4086120 con 0x7f78c40b11c0 2024-03-22T04:15:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.763+0000 7f78c1400640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f78a4067b50 0x7f78a406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:18.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.763+0000 7f78c1400640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f78a4067b50 0x7f78a406a010 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f78b0007fd0 tx=0x7f78b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:18.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.767+0000 7f78ba000640 1 -- 172.21.15.86:0/2954442679 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f78c4159d90 con 0x7f78c40b11c0 2024-03-22T04:15:18.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.939+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f789c005da0 con 0x7f78c40b11c0 2024-03-22T04:15:18.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78ba000640 1 -- 172.21.15.86:0/2954442679 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f78b40567f0 con 0x7f78c40b11c0 2024-03-22T04:15:18.946 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f78a4067b50 msgr2=0x7f78a406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f78a4067b50 0x7f78a406a010 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f78b0007fd0 tx=0x7f78b0009040 comp rx=0 tx=0).stop 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 msgr2=0x7f78c415a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c415a260 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f78b400e870 tx=0x7f78b400ec70 comp rx=0 tx=0).stop 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 shutdown_connections 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f78a4067b50 0x7f78a406a010 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c415c730 0x7f78c406d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78c40b3610 0x7f78c415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 --2- 172.21.15.86:0/2954442679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c40b11c0 0x7f78c415a260 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 >> 172.21.15.86:0/2954442679 conn(0x7f78c40b3bf0 msgr2=0x7f78c40cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 shutdown_connections 2024-03-22T04:15:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:18.943+0000 7f78c9000640 1 -- 172.21.15.86:0/2954442679 wait complete. 2024-03-22T04:15:18.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:15:19.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.099+0000 7f745c800640 1 Processor -- start 2024-03-22T04:15:19.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.099+0000 7f745c800640 1 -- start start 2024-03-22T04:15:19.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f745c800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74540b0810 0x7f745406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f745c800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f74540af700 0x7f74540d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f745c800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74541601d0 con 0x7f745406bba0 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f745c800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7454160490 con 0x7f74540b0810 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f745c800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7454160770 con 0x7f74540af700 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f7459e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74540b0810 0x7f745406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f7459e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74540b0810 0x7f745406b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45132/0 (socket says 172.21.15.86:45132) 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f7459e00640 1 -- 172.21.15.86:0/194685183 learned_addr learned my addr 172.21.15.86:0/194685183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f745a800640 1 --2- 172.21.15.86:0/194685183 >> v2:172.21.15.196:3300/0 conn(0x7f74540af700 0x7f74540d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f7459e00640 1 -- 172.21.15.86:0/194685183 >> v2:172.21.15.196:3300/0 conn(0x7f74540af700 msgr2=0x7f74540d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f7459e00640 1 --2- 172.21.15.86:0/194685183 >> v2:172.21.15.196:3300/0 conn(0x7f74540af700 0x7f74540d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f7459e00640 1 -- 172.21.15.86:0/194685183 >> v1:172.21.15.196:6789/0 conn(0x7f745406bba0 legacy=0x7f74540aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:19.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.103+0000 7f7459e00640 1 -- 172.21.15.86:0/194685183 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7454160a50 con 0x7f74540b0810 2024-03-22T04:15:19.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f7459e00640 1 --2- 172.21.15.86:0/194685183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74540b0810 0x7f745406b660 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f74540cb750 tx=0x7f744801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c28194ae88a7847 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:19.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f7458a00640 1 -- 172.21.15.86:0/194685183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74480ab5f0 con 0x7f74540b0810 2024-03-22T04:15:19.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f7458a00640 1 -- 172.21.15.86:0/194685183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74480b0070 con 0x7f74540b0810 2024-03-22T04:15:19.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f7458a00640 1 -- 172.21.15.86:0/194685183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74480ab9b0 con 0x7f74540b0810 2024-03-22T04:15:19.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 -- 172.21.15.86:0/194685183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74540b0810 msgr2=0x7f745406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 --2- 172.21.15.86:0/194685183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74540b0810 0x7f745406b660 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f74540cb750 tx=0x7f744801c4e0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 -- 172.21.15.86:0/194685183 shutdown_connections 2024-03-22T04:15:19.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 --2- 172.21.15.86:0/194685183 >> v2:172.21.15.196:3300/0 conn(0x7f74540af700 0x7f74540d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 --2- 172.21.15.86:0/194685183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74540b0810 0x7f745406b660 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 -- 172.21.15.86:0/194685183 >> 172.21.15.86:0/194685183 conn(0x7f74540b2d00 msgr2=0x7f74540b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:19.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 -- 172.21.15.86:0/194685183 shutdown_connections 2024-03-22T04:15:19.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 -- 172.21.15.86:0/194685183 wait complete. 2024-03-22T04:15:19.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 Processor -- start 2024-03-22T04:15:19.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.107+0000 7f745c800640 1 -- start start 2024-03-22T04:15:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745406bba0 0x7f7454149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74540af700 0x7f745415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74540b0810 0x7f74540b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f745415cc80 con 0x7f745406bba0 2024-03-22T04:15:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74541539b0 con 0x7f74540af700 2024-03-22T04:15:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f745415ce00 con 0x7f74540b0810 2024-03-22T04:15:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745406bba0 0x7f7454149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745406bba0 0x7f7454149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45144/0 (socket says 172.21.15.86:45144) 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 -- 172.21.15.86:0/1822868944 learned_addr learned my addr 172.21.15.86:0/1822868944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745a800640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74540b0810 0x7f74540b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459400640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74540af700 0x7f745415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 -- 172.21.15.86:0/1822868944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74540b0810 msgr2=0x7f74540b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74540b0810 0x7f74540b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 -- 172.21.15.86:0/1822868944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74540af700 msgr2=0x7f745415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74540af700 0x7f745415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 -- 172.21.15.86:0/1822868944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745415bc40 con 0x7f745406bba0 2024-03-22T04:15:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f7459e00640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745406bba0 0x7f7454149b00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f74480ad040 tx=0x7f744801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f744e000640 1 -- 172.21.15.86:0/1822868944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74480be5f0 con 0x7f745406bba0 2024-03-22T04:15:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f744e000640 1 -- 172.21.15.86:0/1822868944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74480be790 con 0x7f745406bba0 2024-03-22T04:15:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f744e000640 1 -- 172.21.15.86:0/1822868944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74480beb80 con 0x7f745406bba0 2024-03-22T04:15:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745415bf20 con 0x7f745406bba0 2024-03-22T04:15:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745415c4e0 con 0x7f745406bba0 2024-03-22T04:15:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.111+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7454068650 con 0x7f745406bba0 2024-03-22T04:15:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.115+0000 7f744e000640 1 -- 172.21.15.86:0/1822868944 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f74480b43f0 con 0x7f745406bba0 2024-03-22T04:15:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.115+0000 7f744e000640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7438067b50 0x7f743806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.115+0000 7f744e000640 1 -- 172.21.15.86:0/1822868944 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7448120620 con 0x7f745406bba0 2024-03-22T04:15:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.115+0000 7f7459400640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7438067b50 0x7f743806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.115+0000 7f7459400640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7438067b50 0x7f743806a010 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f74540c7a50 tx=0x7f7444007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:19.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.119+0000 7f744e000640 1 -- 172.21.15.86:0/1822868944 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f74480160d0 con 0x7f745406bba0 2024-03-22T04:15:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.283+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f74540d02c0 con 0x7f745406bba0 2024-03-22T04:15:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f744e000640 1 -- 172.21.15.86:0/1822868944 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) ==== 80+0+0 (secure 0 0 0) 0x7f74480f08d0 con 0x7f745406bba0 2024-03-22T04:15:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7438067b50 msgr2=0x7f743806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7438067b50 0x7f743806a010 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f74540c7a50 tx=0x7f7444007e40 comp rx=0 tx=0).stop 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745406bba0 msgr2=0x7f7454149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745406bba0 0x7f7454149b00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f74480ad040 tx=0x7f744801ad20 comp rx=0 tx=0).stop 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 shutdown_connections 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7438067b50 0x7f743806a010 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74540af700 0x7f745415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74540b0810 0x7f74540b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 --2- 172.21.15.86:0/1822868944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745406bba0 0x7f7454149b00 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 >> 172.21.15.86:0/1822868944 conn(0x7f74540b2d00 msgr2=0x7f74540aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:19.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 shutdown_connections 2024-03-22T04:15:19.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.287+0000 7f745c800640 1 -- 172.21.15.86:0/1822868944 wait complete. 2024-03-22T04:15:19.310 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:15:19.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:19.311 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:19.380 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:15:19.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:19.380 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:15:19.395 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:15:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.591+0000 7f0513400640 1 Processor -- start 2024-03-22T04:15:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0513400640 1 -- start start 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0513400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f050c15b0d0 0x7f050c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0513400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 0x7f050c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0513400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f050c0b5570 con 0x7f050c0b5150 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0513400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f050c0b22e0 con 0x7f050c15b0d0 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0513400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f050c15bd30 con 0x7f050c0b2570 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0510a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f050c15b0d0 0x7f050c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0510a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f050c15b0d0 0x7f050c15b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45154/0 (socket says 172.21.15.86:45154) 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0510a00640 1 -- 172.21.15.86:0/395162446 learned_addr learned my addr 172.21.15.86:0/395162446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f050be00640 1 --2- 172.21.15.86:0/395162446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 0x7f050c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0510a00640 1 -- 172.21.15.86:0/395162446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 msgr2=0x7f050c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0510a00640 1 --2- 172.21.15.86:0/395162446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 0x7f050c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0510a00640 1 -- 172.21.15.86:0/395162446 >> v1:172.21.15.196:6789/0 conn(0x7f050c0b5150 legacy=0x7f050c15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.595+0000 7f0510a00640 1 -- 172.21.15.86:0/395162446 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c15c010 con 0x7f050c15b0d0 2024-03-22T04:15:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0510a00640 1 --2- 172.21.15.86:0/395162446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c15b4b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009870 tx=0x7f04fc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d27cc6dcb1fce8b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f050b400640 1 -- 172.21.15.86:0/395162446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0ab650 con 0x7f050c15b0d0 2024-03-22T04:15:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f050b400640 1 -- 172.21.15.86:0/395162446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04fc0b0070 con 0x7f050c15b0d0 2024-03-22T04:15:19.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f050b400640 1 -- 172.21.15.86:0/395162446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0aba10 con 0x7f050c15b0d0 2024-03-22T04:15:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0513400640 1 -- 172.21.15.86:0/395162446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 msgr2=0x7f050c15b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0513400640 1 --2- 172.21.15.86:0/395162446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c15b4b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009870 tx=0x7f04fc01c560 comp rx=0 tx=0).stop 2024-03-22T04:15:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0513400640 1 -- 172.21.15.86:0/395162446 shutdown_connections 2024-03-22T04:15:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0513400640 1 --2- 172.21.15.86:0/395162446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 0x7f050c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0513400640 1 --2- 172.21.15.86:0/395162446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c15b4b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0513400640 1 -- 172.21.15.86:0/395162446 >> 172.21.15.86:0/395162446 conn(0x7f050c0b6620 msgr2=0x7f050c0b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0513400640 1 -- 172.21.15.86:0/395162446 shutdown_connections 2024-03-22T04:15:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.599+0000 7f0513400640 1 -- 172.21.15.86:0/395162446 wait complete. 2024-03-22T04:15:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0513400640 1 Processor -- start 2024-03-22T04:15:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0513400640 1 -- start start 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0513400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 0x7f050c0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0513400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f050c0b5150 0x7f050c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0513400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c14fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0513400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f050c168270 con 0x7f050c15b0d0 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0513400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f050c15b9f0 con 0x7f050c0b5150 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0513400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f050c1683f0 con 0x7f050c0b2570 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0511400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c14fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0511400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c14fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45170/0 (socket says 172.21.15.86:45170) 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0511400640 1 -- 172.21.15.86:0/3904118346 learned_addr learned my addr 172.21.15.86:0/3904118346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f050be00640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f050c0b5150 0x7f050c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0510a00640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 0x7f050c0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0511400640 1 -- 172.21.15.86:0/3904118346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 msgr2=0x7f050c0b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0511400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 0x7f050c0b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0511400640 1 -- 172.21.15.86:0/3904118346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f050c0b5150 msgr2=0x7f050c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0511400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f050c0b5150 0x7f050c158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.603+0000 7f0511400640 1 -- 172.21.15.86:0/3904118346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c0b1870 con 0x7f050c15b0d0 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0511400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c14fda0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f050400c770 tx=0x7f050400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0508c00640 1 -- 172.21.15.86:0/3904118346 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504003b80 con 0x7f050c15b0d0 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0508c00640 1 -- 172.21.15.86:0/3904118346 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0504003d20 con 0x7f050c15b0d0 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050c165250 con 0x7f050c15b0d0 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0508c00640 1 -- 172.21.15.86:0/3904118346 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400aa30 con 0x7f050c15b0d0 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050c165810 con 0x7f050c15b0d0 2024-03-22T04:15:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050c068650 con 0x7f050c15b0d0 2024-03-22T04:15:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0508c00640 1 -- 172.21.15.86:0/3904118346 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f050400abd0 con 0x7f050c15b0d0 2024-03-22T04:15:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0508c00640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f04e8067b50 0x7f04e806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0508c00640 1 -- 172.21.15.86:0/3904118346 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f0504085d60 con 0x7f050c15b0d0 2024-03-22T04:15:19.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.607+0000 7f0510a00640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f04e8067b50 0x7f04e806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.611+0000 7f0510a00640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f04e8067b50 0x7f04e806a010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0af000 tx=0x7f04fc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:19.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.611+0000 7f0508c00640 1 -- 172.21.15.86:0/3904118346 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f050c0b1870 con 0x7f050c15b0d0 2024-03-22T04:15:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.835+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f050c15ec40 con 0x7f050c15b0d0 2024-03-22T04:15:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.835+0000 7f0508c00640 1 -- 172.21.15.86:0/3904118346 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5189 (secure 0 0 0) 0x7f0504056020 con 0x7f050c15b0d0 2024-03-22T04:15:19.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:19.841 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:15:12.461110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1130704586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1130704586},{"type":"v1","addr":"172.21.15.86:6837","nonce":1130704586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6807,"qdb_cluster":[6807]},"id":5}]} 2024-03-22T04:15:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 46 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f04e8067b50 msgr2=0x7f04e806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f04e8067b50 0x7f04e806a010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0af000 tx=0x7f04fc0a7000 comp rx=0 tx=0).stop 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 msgr2=0x7f050c14fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c14fda0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f050400c770 tx=0x7f050400cc30 comp rx=0 tx=0).stop 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 shutdown_connections 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f04e8067b50 0x7f04e806a010 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f050c0b5150 0x7f050c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050c15b0d0 0x7f050c14fda0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 --2- 172.21.15.86:0/3904118346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f050c0b2570 0x7f050c0b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 >> 172.21.15.86:0/3904118346 conn(0x7f050c0b6620 msgr2=0x7f050c06f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 shutdown_connections 2024-03-22T04:15:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:19.839+0000 7f0513400640 1 -- 172.21.15.86:0/3904118346 wait complete. 2024-03-22T04:15:19.863 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-22T04:15:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:{"message":"","return_code":0} 2024-03-22T04:15:20.036 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:15:20.036 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:15:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb775e00640 1 Processor -- start 2024-03-22T04:15:20.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb775e00640 1 -- start start 2024-03-22T04:15:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb775e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb7700724b0 0x7fb77006de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb775e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77006e500 con 0x7fb77015aee0 2024-03-22T04:15:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb775e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77006e7e0 con 0x7fb7700b4780 2024-03-22T04:15:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb775e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb7700b43d0 con 0x7fb7700724b0 2024-03-22T04:15:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb774a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb77015aee0 0x7fb77006f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36570/0 (socket says 172.21.15.86:36570) 2024-03-22T04:15:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb774a00640 1 -- 172.21.15.86:0/2607594258 learned_addr learned my addr 172.21.15.86:0/2607594258 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.171+0000 7fb775400640 1 --2- 172.21.15.86:0/2607594258 >> v2:172.21.15.86:3300/0 conn(0x7fb7700724b0 0x7fb77006de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb76f400640 1 -- 172.21.15.86:0/2607594258 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508359977 0 0) 0x7fb77006e500 con 0x7fb77015aee0 2024-03-22T04:15:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb76f400640 1 -- 172.21.15.86:0/2607594258 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb758003530 con 0x7fb77015aee0 2024-03-22T04:15:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb775400640 1 -- 172.21.15.86:0/2607594258 >> v1:172.21.15.196:6789/0 conn(0x7fb7700b4780 legacy=0x7fb770071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb775400640 1 -- 172.21.15.86:0/2607594258 >> v1:172.21.15.86:6789/0 conn(0x7fb77015aee0 legacy=0x7fb77006f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb775400640 1 -- 172.21.15.86:0/2607594258 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77006eac0 con 0x7fb7700724b0 2024-03-22T04:15:20.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb775400640 1 --2- 172.21.15.86:0/2607594258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7700724b0 0x7fb77006de00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb768009d90 tx=0x7fb76801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea65aa7947a2f48b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb76f400640 1 -- 172.21.15.86:0/2607594258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680b2070 con 0x7fb7700724b0 2024-03-22T04:15:20.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb76f400640 1 -- 172.21.15.86:0/2607594258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7680a5a50 con 0x7fb7700724b0 2024-03-22T04:15:20.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.175+0000 7fb76f400640 1 -- 172.21.15.86:0/2607594258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680a5e00 con 0x7fb7700724b0 2024-03-22T04:15:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 -- 172.21.15.86:0/2607594258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7700724b0 msgr2=0x7fb77006de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 --2- 172.21.15.86:0/2607594258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7700724b0 0x7fb77006de00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb768009d90 tx=0x7fb76801c970 comp rx=0 tx=0).stop 2024-03-22T04:15:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 -- 172.21.15.86:0/2607594258 shutdown_connections 2024-03-22T04:15:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 --2- 172.21.15.86:0/2607594258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7700724b0 0x7fb77006de00 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 -- 172.21.15.86:0/2607594258 >> 172.21.15.86:0/2607594258 conn(0x7fb77006b660 msgr2=0x7fb77006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 -- 172.21.15.86:0/2607594258 shutdown_connections 2024-03-22T04:15:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 -- 172.21.15.86:0/2607594258 wait complete. 2024-03-22T04:15:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 Processor -- start 2024-03-22T04:15:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.179+0000 7fb775e00640 1 -- start start 2024-03-22T04:15:20.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7700724b0 0x7fb770152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7700b4780 0x7fb7701547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77015aee0 0x7fb7701f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7700ce7e0 con 0x7fb77015aee0 2024-03-22T04:15:20.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb77006fcb0 con 0x7fb7700b4780 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb7700ce960 con 0x7fb7700724b0 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77015aee0 0x7fb7701f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77015aee0 0x7fb7701f9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45178/0 (socket says 172.21.15.86:45178) 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 -- 172.21.15.86:0/2439388465 learned_addr learned my addr 172.21.15.86:0/2439388465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb76fe00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7700b4780 0x7fb7701547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb774a00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7700724b0 0x7fb770152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 -- 172.21.15.86:0/2439388465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7700724b0 msgr2=0x7fb770152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7700724b0 0x7fb770152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 -- 172.21.15.86:0/2439388465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7700b4780 msgr2=0x7fb7701547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7700b4780 0x7fb7701547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 -- 172.21.15.86:0/2439388465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb770154cf0 con 0x7fb77015aee0 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775400640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77015aee0 0x7fb7701f9d80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb768009d90 tx=0x7fb768019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb76cc00640 1 -- 172.21.15.86:0/2439388465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680b2050 con 0x7fb77015aee0 2024-03-22T04:15:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb76cc00640 1 -- 172.21.15.86:0/2439388465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7680b3040 con 0x7fb77015aee0 2024-03-22T04:15:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb76cc00640 1 -- 172.21.15.86:0/2439388465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768009450 con 0x7fb77015aee0 2024-03-22T04:15:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb770154f90 con 0x7fb77015aee0 2024-03-22T04:15:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb770155550 con 0x7fb77015aee0 2024-03-22T04:15:20.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.183+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb770074470 con 0x7fb77015aee0 2024-03-22T04:15:20.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.187+0000 7fb76cc00640 1 -- 172.21.15.86:0/2439388465 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb7680c0020 con 0x7fb77015aee0 2024-03-22T04:15:20.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.187+0000 7fb76cc00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb750067b20 0x7fb750069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.187+0000 7fb76cc00640 1 -- 172.21.15.86:0/2439388465 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fb7680ade00 con 0x7fb77015aee0 2024-03-22T04:15:20.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.187+0000 7fb774a00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb750067b20 0x7fb750069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.187+0000 7fb774a00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb750067b20 0x7fb750069fe0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb7640011f0 tx=0x7fb764007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.191+0000 7fb76cc00640 1 -- 172.21.15.86:0/2439388465 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb768016560 con 0x7fb77015aee0 2024-03-22T04:15:20.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.387+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb77006cd80 con 0x7fb77015aee0 2024-03-22T04:15:20.392 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:15:20.387+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:15:20.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.391+0000 7fb76cc00640 1 -- 172.21.15.86:0/2439388465 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7fb7680f10b0 con 0x7fb77015aee0 2024-03-22T04:15:20.395 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:15:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb750067b20 msgr2=0x7fb750069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb750067b20 0x7fb750069fe0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb7640011f0 tx=0x7fb764007d40 comp rx=0 tx=0).stop 2024-03-22T04:15:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77015aee0 msgr2=0x7fb7701f9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77015aee0 0x7fb7701f9d80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb768009d90 tx=0x7fb768019710 comp rx=0 tx=0).stop 2024-03-22T04:15:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 shutdown_connections 2024-03-22T04:15:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb750067b20 0x7fb750069fe0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7700b4780 0x7fb7701547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77015aee0 0x7fb7701f9d80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 --2- 172.21.15.86:0/2439388465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7700724b0 0x7fb770152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 >> 172.21.15.86:0/2439388465 conn(0x7fb77006b660 msgr2=0x7fb77015a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 shutdown_connections 2024-03-22T04:15:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.395+0000 7fb775e00640 1 -- 172.21.15.86:0/2439388465 wait complete. 2024-03-22T04:15:20.413 INFO:tasks.ceph.mds.a.smithi086.stderr: -4709> 2024-03-22T04:15:12.035+0000 7f2bd2600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:15:20.420 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-22T04:15:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f90103a0080 1 Processor -- start 2024-03-22T04:15:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f90103a0080 1 -- start start 2024-03-22T04:15:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f90103a0080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 0x561d1dad1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f90103a0080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 0x561d1dad20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f90103a0080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f90103a0080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561d1d9a3620 con 0x561d1dad2630 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f90103a0080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d1d9a7670 con 0x561d1d9fdd30 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f90103a0080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561d1da48730 con 0x7f8ff40022e0 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f900e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f900e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45196/0 (socket says 172.21.15.86:45196) 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f900e000640 1 -- 172.21.15.86:0/21573570 learned_addr learned my addr 172.21.15.86:0/21573570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f900cc00640 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 0x561d1dad20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.467+0000 7f900d600640 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 0x561d1dad1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900e000640 1 -- 172.21.15.86:0/21573570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 msgr2=0x561d1dad1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900e000640 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 0x561d1dad1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900e000640 1 -- 172.21.15.86:0/21573570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 msgr2=0x561d1dad20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900e000640 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 0x561d1dad20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900e000640 1 -- 172.21.15.86:0/21573570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d1dacd7a0 con 0x561d1dad2630 2024-03-22T04:15:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900e000640 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad09c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x561d1da053c0 tx=0x7f9008005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763cafeabb91dc75 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f9007400640 1 -- 172.21.15.86:0/21573570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008015690 con 0x561d1dad2630 2024-03-22T04:15:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f9007400640 1 -- 172.21.15.86:0/21573570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9008015830 con 0x561d1dad2630 2024-03-22T04:15:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f9007400640 1 -- 172.21.15.86:0/21573570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008015a20 con 0x561d1dad2630 2024-03-22T04:15:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- 172.21.15.86:0/21573570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 msgr2=0x561d1dad09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad09c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x561d1da053c0 tx=0x7f9008005d90 comp rx=0 tx=0).stop 2024-03-22T04:15:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- 172.21.15.86:0/21573570 shutdown_connections 2024-03-22T04:15:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 0x561d1dad20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad09c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 --2- 172.21.15.86:0/21573570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 0x561d1dad1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- 172.21.15.86:0/21573570 >> 172.21.15.86:0/21573570 conn(0x561d1da2afe0 msgr2=0x561d1dad67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- 172.21.15.86:0/21573570 shutdown_connections 2024-03-22T04:15:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- 172.21.15.86:0/21573570 wait complete. 2024-03-22T04:15:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 Processor -- start 2024-03-22T04:15:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- start start 2024-03-22T04:15:20.477 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 0x561d1dabf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 0x561d1dad1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561d1da302f0 con 0x561d1dad2630 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d1dad2a40 con 0x561d1d9fdd30 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f90103a0080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561d1da2ee60 con 0x7f8ff40022e0 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45200/0 (socket says 172.21.15.86:45200) 2024-03-22T04:15:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900cc00640 1 -- 172.21.15.86:0/3243342844 learned_addr learned my addr 172.21.15.86:0/3243342844 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900e000640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 0x561d1dad1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.471+0000 7f900d600640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 0x561d1dabf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f900cc00640 1 -- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 msgr2=0x561d1dad1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f900cc00640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ff40022e0 0x561d1dad1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f900cc00640 1 -- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 msgr2=0x561d1dabf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f900cc00640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d1d9fdd30 0x561d1dabf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f900cc00640 1 -- 172.21.15.86:0/3243342844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d1dabb2e0 con 0x561d1dad2630 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f900cc00640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad18f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00ef50 tx=0x7f8ffc00d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f9006000640 1 -- 172.21.15.86:0/3243342844 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00b580 con 0x561d1dad2630 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f9006000640 1 -- 172.21.15.86:0/3243342844 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00b720 con 0x561d1dad2630 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d1dabca20 con 0x561d1dad2630 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f9006000640 1 -- 172.21.15.86:0/3243342844 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00bad0 con 0x561d1dad2630 2024-03-22T04:15:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d1dabcf60 con 0x561d1dad2630 2024-03-22T04:15:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f9006000640 1 -- 172.21.15.86:0/3243342844 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8ffc00bc70 con 0x561d1dad2630 2024-03-22T04:15:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f9006000640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ff0066500 0x7f8ff00689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.475+0000 7f9006000640 1 -- 172.21.15.86:0/3243342844 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f8ffc085410 con 0x561d1dad2630 2024-03-22T04:15:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f900d600640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ff0066500 0x7f8ff00689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561d1dabd6a0 0x561d1dabda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81) -- 0x561d1dabdfc0 con 0x561d1dabd6a0 2024-03-22T04:15:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f900e000640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561d1dabd6a0 0x561d1dabda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f900d600640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ff0066500 0x7f8ff00689c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f90000ab000 tx=0x7f900000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f900e000640 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561d1dabd6a0 0x561d1dabda80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f900e000640 1 -- 172.21.15.86:0/3243342844 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v81'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9008013430 con 0x561d1dabd6a0 2024-03-22T04:15:20.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561d1dabd6a0 msgr2=0x561d1dabda80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561d1dabd6a0 0x561d1dabda80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ff0066500 msgr2=0x7f8ff00689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ff0066500 0x7f8ff00689c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f90000ab000 tx=0x7f900000b040 comp rx=0 tx=0).stop 2024-03-22T04:15:20.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 msgr2=0x561d1dad18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 --2- 172.21.15.86:0/3243342844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d1dad2630 0x561d1dad18f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00ef50 tx=0x7f8ffc00d810 comp rx=0 tx=0).stop 2024-03-22T04:15:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f900e000640 1 -- 172.21.15.86:0/3243342844 reap_dead start 2024-03-22T04:15:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 shutdown_connections 2024-03-22T04:15:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 >> 172.21.15.86:0/3243342844 conn(0x561d1da2afe0 msgr2=0x561d1da304d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 shutdown_connections 2024-03-22T04:15:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.479+0000 7f90103a0080 1 -- 172.21.15.86:0/3243342844 wait complete. 2024-03-22T04:15:20.489 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-22T04:15:20.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.623+0000 7f1530400640 1 Processor -- start 2024-03-22T04:15:20.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.623+0000 7f1530400640 1 -- start start 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.623+0000 7f1530400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f15280b2360 0x7f15280b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.623+0000 7f1530400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15280b3430 0x7f152815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.623+0000 7f1530400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f152815d710 con 0x7f152815cee0 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.623+0000 7f1530400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f15280b2740 con 0x7f15280b3430 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.623+0000 7f1530400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f152815fe40 con 0x7f15280b2360 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15280b3430 0x7f152815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152e400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f152815cee0 0x7f152806d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36580/0 (socket says 172.21.15.86:36580) 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152e400640 1 -- 172.21.15.86:0/3754323418 learned_addr learned my addr 172.21.15.86:0/3754323418 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:20.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15280b3430 0x7f152815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45224/0 (socket says 172.21.15.86:45224) 2024-03-22T04:15:20.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152da00640 1 --2- 172.21.15.86:0/3754323418 >> v2:172.21.15.196:3300/0 conn(0x7f15280b2360 0x7f15280b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152d000640 1 -- 172.21.15.86:0/3754323418 >> v2:172.21.15.196:3300/0 conn(0x7f15280b2360 msgr2=0x7f15280b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:20.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152d000640 1 --2- 172.21.15.86:0/3754323418 >> v2:172.21.15.196:3300/0 conn(0x7f15280b2360 0x7f15280b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152d000640 1 -- 172.21.15.86:0/3754323418 >> v1:172.21.15.86:6789/0 conn(0x7f152815cee0 legacy=0x7f152806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f1527e00640 1 -- 172.21.15.86:0/3754323418 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506424823 0 0) 0x7f152815d710 con 0x7f152815cee0 2024-03-22T04:15:20.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152d000640 1 -- 172.21.15.86:0/3754323418 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1528160120 con 0x7f15280b3430 2024-03-22T04:15:20.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152da00640 1 --2- 172.21.15.86:0/3754323418 >> v2:172.21.15.196:3300/0 conn(0x7f15280b2360 0x7f15280b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f152d000640 1 --2- 172.21.15.86:0/3754323418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15280b3430 0x7f152815c9a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f15280683e0 tx=0x7f151401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42985193b1c75a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f1527e00640 1 -- 172.21.15.86:0/3754323418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140b0070 con 0x7f15280b3430 2024-03-22T04:15:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f1527e00640 1 -- 172.21.15.86:0/3754323418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15140ab600 con 0x7f15280b3430 2024-03-22T04:15:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.627+0000 7f1527e00640 1 -- 172.21.15.86:0/3754323418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140ab9d0 con 0x7f15280b3430 2024-03-22T04:15:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- 172.21.15.86:0/3754323418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15280b3430 msgr2=0x7f152815c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 --2- 172.21.15.86:0/3754323418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15280b3430 0x7f152815c9a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f15280683e0 tx=0x7f151401c480 comp rx=0 tx=0).stop 2024-03-22T04:15:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- 172.21.15.86:0/3754323418 shutdown_connections 2024-03-22T04:15:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 --2- 172.21.15.86:0/3754323418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15280b3430 0x7f152815c9a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 --2- 172.21.15.86:0/3754323418 >> v2:172.21.15.196:3300/0 conn(0x7f15280b2360 0x7f15280b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- 172.21.15.86:0/3754323418 >> 172.21.15.86:0/3754323418 conn(0x7f15280cc940 msgr2=0x7f15280ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- 172.21.15.86:0/3754323418 shutdown_connections 2024-03-22T04:15:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- 172.21.15.86:0/3754323418 wait complete. 2024-03-22T04:15:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 Processor -- start 2024-03-22T04:15:20.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- start start 2024-03-22T04:15:20.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15280b2360 0x7f152815b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15280b3430 0x7f15280724c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f152815cee0 0x7f152813d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1528160c00 con 0x7f152815cee0 2024-03-22T04:15:20.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f152815d2c0 con 0x7f15280b3430 2024-03-22T04:15:20.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f1530400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1528155ae0 con 0x7f15280b2360 2024-03-22T04:15:20.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f152e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f152815cee0 0x7f152813d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f152e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f152815cee0 0x7f152813d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45230/0 (socket says 172.21.15.86:45230) 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f152e400640 1 -- 172.21.15.86:0/1273481755 learned_addr learned my addr 172.21.15.86:0/1273481755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f152d000640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15280b3430 0x7f15280724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f152da00640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15280b2360 0x7f152815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f152e400640 1 -- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15280b2360 msgr2=0x7f152815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.631+0000 7f152e400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15280b2360 0x7f152815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f152e400640 1 -- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15280b3430 msgr2=0x7f15280724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f152e400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15280b3430 0x7f15280724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f152e400640 1 -- 172.21.15.86:0/1273481755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1528155dc0 con 0x7f152815cee0 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f152da00640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15280b2360 0x7f152815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f152e400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f152815cee0 0x7f152813d2e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f152000c7a0 tx=0x7f152000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1525600640 1 -- 172.21.15.86:0/1273481755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520018070 con 0x7f152815cee0 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1525600640 1 -- 172.21.15.86:0/1273481755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f152000a640 con 0x7f152815cee0 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15281560a0 con 0x7f152815cee0 2024-03-22T04:15:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1525600640 1 -- 172.21.15.86:0/1273481755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152000aab0 con 0x7f152815cee0 2024-03-22T04:15:20.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152815bac0 con 0x7f152815cee0 2024-03-22T04:15:20.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1528074470 con 0x7f152815cee0 2024-03-22T04:15:20.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1525600640 1 -- 172.21.15.86:0/1273481755 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f152001b070 con 0x7f152815cee0 2024-03-22T04:15:20.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1525600640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1508067b20 0x7f1508069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:20.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f1525600640 1 -- 172.21.15.86:0/1273481755 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f15200856c0 con 0x7f152815cee0 2024-03-22T04:15:20.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.635+0000 7f152da00640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1508067b20 0x7f1508069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:20.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.639+0000 7f152da00640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1508067b20 0x7f1508069fe0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f151c0079d0 tx=0x7f151c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.639+0000 7f1525600640 1 -- 172.21.15.86:0/1273481755 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1520055810 con 0x7f152815cee0 2024-03-22T04:15:20.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:20.831+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f152806e9f0 con 0x7f152815cee0 2024-03-22T04:15:21.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.395+0000 7f1525600640 1 -- 172.21.15.86:0/1273481755 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v48) ==== 162+0+0 (secure 0 0 0) 0x7f15200598b0 con 0x7f152815cee0 2024-03-22T04:15:21.399 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-22T04:15:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1508067b20 msgr2=0x7f1508069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1508067b20 0x7f1508069fe0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f151c0079d0 tx=0x7f151c008040 comp rx=0 tx=0).stop 2024-03-22T04:15:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f152815cee0 msgr2=0x7f152813d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f152815cee0 0x7f152813d2e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f152000c7a0 tx=0x7f152000cc60 comp rx=0 tx=0).stop 2024-03-22T04:15:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 shutdown_connections 2024-03-22T04:15:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1508067b20 0x7f1508069fe0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15280b3430 0x7f15280724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f152815cee0 0x7f152813d2e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 --2- 172.21.15.86:0/1273481755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15280b2360 0x7f152815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 >> 172.21.15.86:0/1273481755 conn(0x7f15280cc940 msgr2=0x7f15280cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 shutdown_connections 2024-03-22T04:15:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.399+0000 7f1530400640 1 -- 172.21.15.86:0/1273481755 wait complete. 2024-03-22T04:15:21.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:15:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d98000640 1 Processor -- start 2024-03-22T04:15:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d98000640 1 -- start start 2024-03-22T04:15:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d98000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d9015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d98000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d9006ef40 0x7f3d9006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d98000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9015c7c0 con 0x7f3d900ad710 2024-03-22T04:15:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d98000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3d9006dc40 con 0x7f3d9006ef40 2024-03-22T04:15:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d98000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9006dea0 con 0x7f3d900ad330 2024-03-22T04:15:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d95600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d9015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d96000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3d900ad710 0x7f3d9013fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:36832/0 (socket says 172.21.15.86:36832) 2024-03-22T04:15:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d96000640 1 -- 172.21.15.86:0/3743219858 learned_addr learned my addr 172.21.15.86:0/3743219858 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.563+0000 7f3d94c00640 1 --2- 172.21.15.86:0/3743219858 >> v2:172.21.15.196:3300/0 conn(0x7f3d9006ef40 0x7f3d9006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d8fe00640 1 -- 172.21.15.86:0/3743219858 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3421728584 0 0) 0x7f3d9015c7c0 con 0x7f3d900ad710 2024-03-22T04:15:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d8fe00640 1 -- 172.21.15.86:0/3743219858 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d78003530 con 0x7f3d900ad710 2024-03-22T04:15:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d94c00640 1 -- 172.21.15.86:0/3743219858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 msgr2=0x7f3d9015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d94c00640 1 --2- 172.21.15.86:0/3743219858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d9015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d94c00640 1 -- 172.21.15.86:0/3743219858 >> v1:172.21.15.196:6789/0 conn(0x7f3d900ad710 legacy=0x7f3d9013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d94c00640 1 -- 172.21.15.86:0/3743219858 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9006e180 con 0x7f3d9006ef40 2024-03-22T04:15:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d95600640 1 --2- 172.21.15.86:0/3743219858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d9015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d94c00640 1 --2- 172.21.15.86:0/3743219858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d9006ef40 0x7f3d9006d700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0097c0 tx=0x7f3d7c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec36372406072ef server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d8fe00640 1 -- 172.21.15.86:0/3743219858 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0b2070 con 0x7f3d9006ef40 2024-03-22T04:15:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d8fe00640 1 -- 172.21.15.86:0/3743219858 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c0ad680 con 0x7f3d9006ef40 2024-03-22T04:15:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d8fe00640 1 -- 172.21.15.86:0/3743219858 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0ada50 con 0x7f3d9006ef40 2024-03-22T04:15:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d98000640 1 -- 172.21.15.86:0/3743219858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d9006ef40 msgr2=0x7f3d9006d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.567+0000 7f3d98000640 1 --2- 172.21.15.86:0/3743219858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d9006ef40 0x7f3d9006d700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0097c0 tx=0x7f3d7c01c470 comp rx=0 tx=0).stop 2024-03-22T04:15:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- 172.21.15.86:0/3743219858 shutdown_connections 2024-03-22T04:15:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 --2- 172.21.15.86:0/3743219858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d9006ef40 0x7f3d9006d700 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 --2- 172.21.15.86:0/3743219858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d9015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- 172.21.15.86:0/3743219858 >> 172.21.15.86:0/3743219858 conn(0x7f3d900b1df0 msgr2=0x7f3d900b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- 172.21.15.86:0/3743219858 shutdown_connections 2024-03-22T04:15:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- 172.21.15.86:0/3743219858 wait complete. 2024-03-22T04:15:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 Processor -- start 2024-03-22T04:15:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- start start 2024-03-22T04:15:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9006ef40 0x7f3d9015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d900b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d900ad710 0x7f3d90159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901690e0 con 0x7f3d9006ef40 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90168f60 con 0x7f3d900ad710 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3d90169260 con 0x7f3d900ad330 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9006ef40 0x7f3d9015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9006ef40 0x7f3d9015fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45232/0 (socket says 172.21.15.86:45232) 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 -- 172.21.15.86:0/1109941304 learned_addr learned my addr 172.21.15.86:0/1109941304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d96000640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d900ad710 0x7f3d90159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d900b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 -- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 msgr2=0x7f3d900b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d900b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 -- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d900ad710 msgr2=0x7f3d90159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d900ad710 0x7f3d90159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 -- 172.21.15.86:0/1109941304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d90143ec0 con 0x7f3d9006ef40 2024-03-22T04:15:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d900b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d95600640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9006ef40 0x7f3d9015fea0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400e980 tx=0x7f3d8400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d8d600640 1 -- 172.21.15.86:0/1109941304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84017070 con 0x7f3d9006ef40 2024-03-22T04:15:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d8d600640 1 -- 172.21.15.86:0/1109941304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8400ca00 con 0x7f3d9006ef40 2024-03-22T04:15:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901441a0 con 0x7f3d9006ef40 2024-03-22T04:15:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.571+0000 7f3d8d600640 1 -- 172.21.15.86:0/1109941304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400cdd0 con 0x7f3d9006ef40 2024-03-22T04:15:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.575+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d90144a40 con 0x7f3d9006ef40 2024-03-22T04:15:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.575+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d90144370 con 0x7f3d9006ef40 2024-03-22T04:15:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.575+0000 7f3d8d600640 1 -- 172.21.15.86:0/1109941304 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3d8401a070 con 0x7f3d9006ef40 2024-03-22T04:15:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.575+0000 7f3d8d600640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d70067b20 0x7f3d70069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.575+0000 7f3d8d600640 1 -- 172.21.15.86:0/1109941304 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f3d840866f0 con 0x7f3d9006ef40 2024-03-22T04:15:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.575+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d70067b20 0x7f3d70069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.575+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d70067b20 0x7f3d70069fe0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0af000 tx=0x7f3d7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.579+0000 7f3d8d600640 1 -- 172.21.15.86:0/1109941304 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3d84056940 con 0x7f3d9006ef40 2024-03-22T04:15:21.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d90068650 con 0x7f3d9006ef40 2024-03-22T04:15:21.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d8d600640 1 -- 172.21.15.86:0/1109941304 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7f3d8405a9e0 con 0x7f3d9006ef40 2024-03-22T04:15:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:21.771 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:15:21.402615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-03-22T04:15:21.771 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 49 2024-03-22T04:15:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d70067b20 msgr2=0x7f3d70069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d70067b20 0x7f3d70069fe0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0af000 tx=0x7f3d7c008040 comp rx=0 tx=0).stop 2024-03-22T04:15:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9006ef40 msgr2=0x7f3d9015fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9006ef40 0x7f3d9015fea0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400e980 tx=0x7f3d8400ee40 comp rx=0 tx=0).stop 2024-03-22T04:15:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 shutdown_connections 2024-03-22T04:15:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d70067b20 0x7f3d70069fe0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d900ad710 0x7f3d90159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d900ad330 0x7f3d900b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 --2- 172.21.15.86:0/1109941304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9006ef40 0x7f3d9015fea0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:21.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 >> 172.21.15.86:0/1109941304 conn(0x7f3d900b1df0 msgr2=0x7f3d9015da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:21.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 shutdown_connections 2024-03-22T04:15:21.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:21.767+0000 7f3d98000640 1 -- 172.21.15.86:0/1109941304 wait complete. 2024-03-22T04:15:22.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:15:09.433883+0000', 'modified': '2024-03-22T04:15:21.402615+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 81, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7002}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7002': {'gid': 7002, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/736870197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 736870197}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 736870197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:15:22.195 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7002:up:replay 2024-03-22T04:15:23.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:15:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.335+0000 7f2819a00640 1 Processor -- start 2024-03-22T04:15:23.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819a00640 1 -- start start 2024-03-22T04:15:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f28140724b0 0x7f281415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28140affc0 0x7f28140cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28140cd140 con 0x7f28140afbe0 2024-03-22T04:15:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f281406de00 con 0x7f28140affc0 2024-03-22T04:15:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f28140cd420 con 0x7f28140724b0 2024-03-22T04:15:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28140affc0 0x7f28140cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2813e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f28140afbe0 0x7f2814071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36590/0 (socket says 172.21.15.86:36590) 2024-03-22T04:15:23.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2813e00640 1 -- 172.21.15.86:0/652399770 learned_addr learned my addr 172.21.15.86:0/652399770 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f280b400640 1 --2- 172.21.15.86:0/652399770 >> v2:172.21.15.196:3300/0 conn(0x7f28140724b0 0x7f281415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819000640 1 -- 172.21.15.86:0/652399770 >> v2:172.21.15.196:3300/0 conn(0x7f28140724b0 msgr2=0x7f281415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819000640 1 --2- 172.21.15.86:0/652399770 >> v2:172.21.15.196:3300/0 conn(0x7f28140724b0 0x7f281415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819000640 1 -- 172.21.15.86:0/652399770 >> v1:172.21.15.86:6789/0 conn(0x7f28140afbe0 legacy=0x7f2814071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f2819000640 1 -- 172.21.15.86:0/652399770 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28140cd700 con 0x7f28140affc0 2024-03-22T04:15:23.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.339+0000 7f280b400640 1 --2- 172.21.15.86:0/652399770 >> v2:172.21.15.196:3300/0 conn(0x7f28140724b0 0x7f281415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:23.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819000640 1 --2- 172.21.15.86:0/652399770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140affc0 0x7f28140cca10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2814156df0 tx=0x7f280c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b6df351315ee53 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:23.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2813400640 1 -- 172.21.15.86:0/652399770 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0b0070 con 0x7f28140affc0 2024-03-22T04:15:23.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2813400640 1 -- 172.21.15.86:0/652399770 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280c0ab600 con 0x7f28140affc0 2024-03-22T04:15:23.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2813400640 1 -- 172.21.15.86:0/652399770 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0ab9d0 con 0x7f28140affc0 2024-03-22T04:15:23.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 -- 172.21.15.86:0/652399770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140affc0 msgr2=0x7f28140cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:23.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 --2- 172.21.15.86:0/652399770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140affc0 0x7f28140cca10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2814156df0 tx=0x7f280c01c480 comp rx=0 tx=0).stop 2024-03-22T04:15:23.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 -- 172.21.15.86:0/652399770 shutdown_connections 2024-03-22T04:15:23.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 --2- 172.21.15.86:0/652399770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140affc0 0x7f28140cca10 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 --2- 172.21.15.86:0/652399770 >> v2:172.21.15.196:3300/0 conn(0x7f28140724b0 0x7f281415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 -- 172.21.15.86:0/652399770 >> 172.21.15.86:0/652399770 conn(0x7f281406b660 msgr2=0x7f281406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:23.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 -- 172.21.15.86:0/652399770 shutdown_connections 2024-03-22T04:15:23.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 -- 172.21.15.86:0/652399770 wait complete. 2024-03-22T04:15:23.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 Processor -- start 2024-03-22T04:15:23.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.343+0000 7f2819a00640 1 -- start start 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140724b0 0x7f28141652d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28140afbe0 0x7f2814152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28140affc0 0x7f281414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f281406ee30 con 0x7f28140724b0 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f281406ecb0 con 0x7f28140affc0 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f281406efb0 con 0x7f28140afbe0 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140724b0 0x7f28141652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140724b0 0x7f28141652d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45254/0 (socket says 172.21.15.86:45254) 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 -- 172.21.15.86:0/4079370527 learned_addr learned my addr 172.21.15.86:0/4079370527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f280b400640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28140afbe0 0x7f2814152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819000640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28140affc0 0x7f281414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:23.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 -- 172.21.15.86:0/4079370527 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28140afbe0 msgr2=0x7f2814152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28140afbe0 0x7f2814152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 -- 172.21.15.86:0/4079370527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28140affc0 msgr2=0x7f281414d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28140affc0 0x7f281414d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 -- 172.21.15.86:0/4079370527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2814068f70 con 0x7f28140724b0 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2813e00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140724b0 0x7f28141652d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f280400cb10 tx=0x7f280400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2810c00640 1 -- 172.21.15.86:0/4079370527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804017070 con 0x7f28140724b0 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2810c00640 1 -- 172.21.15.86:0/4079370527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280400ac70 con 0x7f28140724b0 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2810c00640 1 -- 172.21.15.86:0/4079370527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804012620 con 0x7f28140724b0 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2814165920 con 0x7f28140724b0 2024-03-22T04:15:23.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28140b3ac0 con 0x7f28140724b0 2024-03-22T04:15:23.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281406c6d0 con 0x7f28140724b0 2024-03-22T04:15:23.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.347+0000 7f2810c00640 1 -- 172.21.15.86:0/4079370527 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f280401a070 con 0x7f28140724b0 2024-03-22T04:15:23.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.351+0000 7f2810c00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27f0067b20 0x7f27f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:23.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.351+0000 7f2810c00640 1 -- 172.21.15.86:0/4079370527 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f2804085eb0 con 0x7f28140724b0 2024-03-22T04:15:23.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.351+0000 7f280b400640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27f0067b20 0x7f27f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:23.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.351+0000 7f280b400640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27f0067b20 0x7f27f0069fe0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2814152690 tx=0x7f27fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:23.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.355+0000 7f2810c00640 1 -- 172.21.15.86:0/4079370527 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f28040566e0 con 0x7f28140724b0 2024-03-22T04:15:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.543+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2814074470 con 0x7f28140724b0 2024-03-22T04:15:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.543+0000 7f2810c00640 1 -- 172.21.15.86:0/4079370527 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5180 (secure 0 0 0) 0x7f2804055fe0 con 0x7f28140724b0 2024-03-22T04:15:23.548 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:23.548 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:15:22.412062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-03-22T04:15:23.548 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 51 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.543+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27f0067b20 msgr2=0x7f27f0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.543+0000 7f2819a00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27f0067b20 0x7f27f0069fe0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2814152690 tx=0x7f27fc008040 comp rx=0 tx=0).stop 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140724b0 msgr2=0x7f28141652d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140724b0 0x7f28141652d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f280400cb10 tx=0x7f280400a060 comp rx=0 tx=0).stop 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 shutdown_connections 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27f0067b20 0x7f27f0069fe0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28140affc0 0x7f281414d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28140afbe0 0x7f2814152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 --2- 172.21.15.86:0/4079370527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28140724b0 0x7f28141652d0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:23.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 >> 172.21.15.86:0/4079370527 conn(0x7f281406b660 msgr2=0x7f28140d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 shutdown_connections 2024-03-22T04:15:23.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:23.547+0000 7f2819a00640 1 -- 172.21.15.86:0/4079370527 wait complete. 2024-03-22T04:15:23.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:15:09.433883+0000', 'modified': '2024-03-22T04:15:22.412062+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 81, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7002}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7002': {'gid': 7002, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.86:6837/736870197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 736870197}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 736870197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:15:23.971 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7002:up:rejoin 2024-03-22T04:15:24.972 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:15:25.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f72200640 1 Processor -- start 2024-03-22T04:15:25.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f72200640 1 -- start start 2024-03-22T04:15:25.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f72200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f6c15b460 0x7f1f6c06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f72200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c0cf2f0 0x7f1f6c072850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f72200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6c0cfb40 con 0x7f1f6c072d90 2024-03-22T04:15:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f72200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6c0d0750 con 0x7f1f6c15b460 2024-03-22T04:15:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f72200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c0cfe20 con 0x7f1f6c0cf2f0 2024-03-22T04:15:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f6c15b460 0x7f1f6c06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1f6c072d90 0x7f1f6c15efe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36606/0 (socket says 172.21.15.86:36606) 2024-03-22T04:15:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6be00640 1 -- 172.21.15.86:0/79712314 learned_addr learned my addr 172.21.15.86:0/79712314 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6aa00640 1 --2- 172.21.15.86:0/79712314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c0cf2f0 0x7f1f6c072850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6b400640 1 -- 172.21.15.86:0/79712314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c0cf2f0 msgr2=0x7f1f6c072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6b400640 1 --2- 172.21.15.86:0/79712314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c0cf2f0 0x7f1f6c072850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6b400640 1 -- 172.21.15.86:0/79712314 >> v1:172.21.15.86:6789/0 conn(0x7f1f6c072d90 legacy=0x7f1f6c15efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6b400640 1 -- 172.21.15.86:0/79712314 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c0d0100 con 0x7f1f6c15b460 2024-03-22T04:15:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6a000640 1 -- 172.21.15.86:0/79712314 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800008417 0 0) 0x7f1f6c0cfb40 con 0x7f1f6c072d90 2024-03-22T04:15:25.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.107+0000 7f1f6aa00640 1 --2- 172.21.15.86:0/79712314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c0cf2f0 0x7f1f6c072850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f6b400640 1 --2- 172.21.15.86:0/79712314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c06f8d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097c0 tx=0x7f1f5c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1362f9d341f056 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f6a000640 1 -- 172.21.15.86:0/79712314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0b0070 con 0x7f1f6c15b460 2024-03-22T04:15:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f6a000640 1 -- 172.21.15.86:0/79712314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c0ab600 con 0x7f1f6c15b460 2024-03-22T04:15:25.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f6a000640 1 -- 172.21.15.86:0/79712314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0ab9d0 con 0x7f1f6c15b460 2024-03-22T04:15:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 -- 172.21.15.86:0/79712314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 msgr2=0x7f1f6c06f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 --2- 172.21.15.86:0/79712314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c06f8d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097c0 tx=0x7f1f5c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 -- 172.21.15.86:0/79712314 shutdown_connections 2024-03-22T04:15:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 --2- 172.21.15.86:0/79712314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c0cf2f0 0x7f1f6c072850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 --2- 172.21.15.86:0/79712314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c06f8d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 -- 172.21.15.86:0/79712314 >> 172.21.15.86:0/79712314 conn(0x7f1f6c0b2d00 msgr2=0x7f1f6c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 -- 172.21.15.86:0/79712314 shutdown_connections 2024-03-22T04:15:25.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 -- 172.21.15.86:0/79712314 wait complete. 2024-03-22T04:15:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.115+0000 7f1f72200640 1 Processor -- start 2024-03-22T04:15:25.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 -- start start 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c072d90 0x7f1f6c155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f6c0cf2f0 0x7f1f6c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c168320 con 0x7f1f6c15b460 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c0cf8e0 con 0x7f1f6c0cf2f0 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c1684a0 con 0x7f1f6c072d90 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c0c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45274/0 (socket says 172.21.15.86:45274) 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 -- 172.21.15.86:0/267162317 learned_addr learned my addr 172.21.15.86:0/267162317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6b400640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c072d90 0x7f1f6c155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 -- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c072d90 msgr2=0x7f1f6c155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6aa00640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f6c0cf2f0 0x7f1f6c1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c072d90 0x7f1f6c155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 -- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f6c0cf2f0 msgr2=0x7f1f6c1582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f6c0cf2f0 0x7f1f6c1582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 -- 172.21.15.86:0/267162317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c0b3ec0 con 0x7f1f6c15b460 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6b400640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c072d90 0x7f1f6c155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6aa00640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f6c0cf2f0 0x7f1f6c1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f6be00640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c0c8240 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400c7a0 tx=0x7f1f6400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f63400640 1 -- 172.21.15.86:0/267162317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64018070 con 0x7f1f6c15b460 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f63400640 1 -- 172.21.15.86:0/267162317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f6400a640 con 0x7f1f6c15b460 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f63400640 1 -- 172.21.15.86:0/267162317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6400aab0 con 0x7f1f6c15b460 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6c0b41a0 con 0x7f1f6c15b460 2024-03-22T04:15:25.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6c0b4760 con 0x7f1f6c15b460 2024-03-22T04:15:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.119+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6c068650 con 0x7f1f6c15b460 2024-03-22T04:15:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.123+0000 7f1f63400640 1 -- 172.21.15.86:0/267162317 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1f6401b070 con 0x7f1f6c15b460 2024-03-22T04:15:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.123+0000 7f1f63400640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f4c067b20 0x7f1f4c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.123+0000 7f1f63400640 1 -- 172.21.15.86:0/267162317 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f1f640857a0 con 0x7f1f6c15b460 2024-03-22T04:15:25.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.123+0000 7f1f6b400640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f4c067b20 0x7f1f4c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.123+0000 7f1f6b400640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f4c067b20 0x7f1f4c069fe0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0ad000 tx=0x7f1f5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.127+0000 7f1f63400640 1 -- 172.21.15.86:0/267162317 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1f64055f30 con 0x7f1f6c15b460 2024-03-22T04:15:25.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.315+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f6c0b4930 con 0x7f1f6c15b460 2024-03-22T04:15:25.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f63400640 1 -- 172.21.15.86:0/267162317 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5187 (secure 0 0 0) 0x7f1f64055830 con 0x7f1f6c15b460 2024-03-22T04:15:25.322 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:25.322 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:15:24.416742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7002,"qdb_cluster":[7002]},"id":5}]} 2024-03-22T04:15:25.322 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 52 2024-03-22T04:15:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f4c067b20 msgr2=0x7f1f4c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f4c067b20 0x7f1f4c069fe0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0ad000 tx=0x7f1f5c008040 comp rx=0 tx=0).stop 2024-03-22T04:15:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 msgr2=0x7f1f6c0c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c0c8240 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400c7a0 tx=0x7f1f6400cc60 comp rx=0 tx=0).stop 2024-03-22T04:15:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 shutdown_connections 2024-03-22T04:15:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f4c067b20 0x7f1f4c069fe0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f6c0cf2f0 0x7f1f6c1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6c15b460 0x7f1f6c0c8240 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 --2- 172.21.15.86:0/267162317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f6c072d90 0x7f1f6c155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 >> 172.21.15.86:0/267162317 conn(0x7f1f6c0b2d00 msgr2=0x7f1f6c0b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 shutdown_connections 2024-03-22T04:15:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.319+0000 7f1f72200640 1 -- 172.21.15.86:0/267162317 wait complete. 2024-03-22T04:15:25.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:15:09.433883+0000', 'modified': '2024-03-22T04:15:24.416742+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 81, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7002}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7002': {'gid': 7002, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6837/736870197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 736870197}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 736870197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7002, 'qdb_cluster': [7002]} 2024-03-22T04:15:25.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:15:25.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-22T04:15:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.879+0000 7f144f800640 1 Processor -- start 2024-03-22T04:15:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.879+0000 7f144f800640 1 -- start start 2024-03-22T04:15:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1448070390 0x7f14480b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f144815fa10 con 0x7f144815f560 2024-03-22T04:15:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144f800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14480b4810 con 0x7f14480ae5e0 2024-03-22T04:15:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14480b4af0 con 0x7f1448070390 2024-03-22T04:15:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1448070390 0x7f14480b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1448070390 0x7f14480b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40618/0 (socket says 172.21.15.86:40618) 2024-03-22T04:15:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144ce00640 1 -- 172.21.15.86:0/3305980304 learned_addr learned my addr 172.21.15.86:0/3305980304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f143fe00640 1 --1- 172.21.15.86:0/3305980304 >> v1:172.21.15.86:6789/0 conn(0x7f144815f560 0x7f14480aded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59926/0 (socket says 172.21.15.86:59926) 2024-03-22T04:15:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144ce00640 1 -- 172.21.15.86:0/3305980304 >> v1:172.21.15.196:6789/0 conn(0x7f14480ae5e0 legacy=0x7f144806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144ce00640 1 -- 172.21.15.86:0/3305980304 >> v1:172.21.15.86:6789/0 conn(0x7f144815f560 legacy=0x7f14480aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f1447e00640 1 -- 172.21.15.86:0/3305980304 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 865549935 0 0) 0x7f144815fa10 con 0x7f144815f560 2024-03-22T04:15:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.883+0000 7f144ce00640 1 -- 172.21.15.86:0/3305980304 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14480b4dd0 con 0x7f1448070390 2024-03-22T04:15:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144ce00640 1 --2- 172.21.15.86:0/3305980304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1448070390 0x7f14480b4150 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1438009870 tx=0x7f143801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b907467bb52e9ed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f1447e00640 1 -- 172.21.15.86:0/3305980304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14380a95f0 con 0x7f1448070390 2024-03-22T04:15:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f1447e00640 1 -- 172.21.15.86:0/3305980304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14380a9790 con 0x7f1448070390 2024-03-22T04:15:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f1447e00640 1 -- 172.21.15.86:0/3305980304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14380a9b60 con 0x7f1448070390 2024-03-22T04:15:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 -- 172.21.15.86:0/3305980304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1448070390 msgr2=0x7f14480b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 --2- 172.21.15.86:0/3305980304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1448070390 0x7f14480b4150 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1438009870 tx=0x7f143801c530 comp rx=0 tx=0).stop 2024-03-22T04:15:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 -- 172.21.15.86:0/3305980304 shutdown_connections 2024-03-22T04:15:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 --2- 172.21.15.86:0/3305980304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1448070390 0x7f14480b4150 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 -- 172.21.15.86:0/3305980304 >> 172.21.15.86:0/3305980304 conn(0x7f14480d0860 msgr2=0x7f14480d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 -- 172.21.15.86:0/3305980304 shutdown_connections 2024-03-22T04:15:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 -- 172.21.15.86:0/3305980304 wait complete. 2024-03-22T04:15:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 Processor -- start 2024-03-22T04:15:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.887+0000 7f144f800640 1 -- start start 2024-03-22T04:15:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144f800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1448070390 0x7f144814fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14480ae5e0 0x7f14481483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f144815f560 0x7f1448153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14480708f0 con 0x7f14480ae5e0 2024-03-22T04:15:25.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144f800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1448070770 con 0x7f1448070390 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f144816c200 con 0x7f144815f560 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14480ae5e0 0x7f14481483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14480ae5e0 0x7f14481483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40632/0 (socket says 172.21.15.86:40632) 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 -- 172.21.15.86:0/1089929413 learned_addr learned my addr 172.21.15.86:0/1089929413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144ce00640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1448070390 0x7f144814fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144d800640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f144815f560 0x7f1448153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 -- 172.21.15.86:0/1089929413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f144815f560 msgr2=0x7f1448153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f144815f560 0x7f1448153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 -- 172.21.15.86:0/1089929413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1448070390 msgr2=0x7f144814fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1448070390 0x7f144814fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 -- 172.21.15.86:0/1089929413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1448157e00 con 0x7f14480ae5e0 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f143fe00640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14480ae5e0 0x7f14481483f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f143000c4a0 tx=0x7f143000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f1445600640 1 -- 172.21.15.86:0/1089929413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430018070 con 0x7f14480ae5e0 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f1445600640 1 -- 172.21.15.86:0/1089929413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14300074e0 con 0x7f14480ae5e0 2024-03-22T04:15:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f1445600640 1 -- 172.21.15.86:0/1089929413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430004a30 con 0x7f14480ae5e0 2024-03-22T04:15:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144f800640 1 -- 172.21.15.86:0/1089929413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14481580e0 con 0x7f14480ae5e0 2024-03-22T04:15:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f144f800640 1 -- 172.21.15.86:0/1089929413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14481586a0 con 0x7f14480ae5e0 2024-03-22T04:15:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.891+0000 7f1445600640 1 -- 172.21.15.86:0/1089929413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f143001b070 con 0x7f14480ae5e0 2024-03-22T04:15:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.895+0000 7f1445600640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1424067b20 0x7f1424069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.895+0000 7f1445600640 1 -- 172.21.15.86:0/1089929413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f1430085370 con 0x7f14480ae5e0 2024-03-22T04:15:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.895+0000 7f144ce00640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1424067b20 0x7f1424069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.895+0000 7f144ce00640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1424067b20 0x7f1424069fe0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f14380ae000 tx=0x7f14380b4040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.895+0000 7f1452d16000 1 Processor -- start 2024-03-22T04:15:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.895+0000 7f1452d16000 1 -- start start 2024-03-22T04:15:25.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f42c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3df6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f3e0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56286f1dd140 con 0x56286f42c790 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56286f230b10 con 0x56286f42be70 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56286f1b7c50 con 0x56286f3dfc20 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3df6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3df6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40648/0 (socket says 172.21.15.86:40648) 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 -- 172.21.15.86:0/1215063860 learned_addr learned my addr 172.21.15.86:0/1215063860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f144ce00640 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f42c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f144d800640 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f3e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 -- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 msgr2=0x56286f3e0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f3e0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 -- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 msgr2=0x56286f42c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f42c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 -- 172.21.15.86:0/1215063860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56286f275f20 con 0x56286f42c790 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f144d800640 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f3e0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143fe00640 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3df6e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f143005d8e0 tx=0x7f1430055540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0aa7756cc6f08d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143d600640 1 -- 172.21.15.86:0/1215063860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14300850c0 con 0x56286f42c790 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143d600640 1 -- 172.21.15.86:0/1215063860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1430053840 con 0x56286f42c790 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f143d600640 1 -- 172.21.15.86:0/1215063860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430059150 con 0x56286f42c790 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- 172.21.15.86:0/1215063860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 msgr2=0x56286f3df6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3df6e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f143005d8e0 tx=0x7f1430055540 comp rx=0 tx=0).stop 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- 172.21.15.86:0/1215063860 shutdown_connections 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f3e0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3df6e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 --2- 172.21.15.86:0/1215063860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f42c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- 172.21.15.86:0/1215063860 >> 172.21.15.86:0/1215063860 conn(0x56286f40cc10 msgr2=0x56286f40f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- 172.21.15.86:0/1215063860 shutdown_connections 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- 172.21.15.86:0/1215063860 wait complete. 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 -- start start 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.899+0000 7f1452d16000 1 Processor -- start 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f3dfc20 0x56286f3e4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f42be70 0x56286f3e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3e5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56286f1eaab0 con 0x56286f42c790 2024-03-22T04:15:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56286f22cac0 con 0x56286f3dfc20 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56286f2ecb30 con 0x56286f42be70 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 10 client.0 _ll_register_callbacks cb 0x56286f3cd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3e5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40662/0 (socket says 172.21.15.86:40662) 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 -- 172.21.15.86:0/3580360814 learned_addr learned my addr 172.21.15.86:0/3580360814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144ce00640 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f3dfc20 0x56286f3e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f143fe00640 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f42be70 0x56286f3e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 -- 172.21.15.86:0/3580360814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f42be70 msgr2=0x56286f3e4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f42be70 0x56286f3e4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 -- 172.21.15.86:0/3580360814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f3dfc20 msgr2=0x56286f3e4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f3dfc20 0x56286f3e4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 -- 172.21.15.86:0/3580360814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56286f206570 con 0x56286f42c790 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144d800640 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3e5290 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1440001e00 tx=0x7f1440005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93f9432753a6f7d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1436000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1436000640 1 -- 172.21.15.86:0/3580360814 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144001a060 con 0x56286f42c790 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 -- 172.21.15.86:0/3580360814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1430053840 con 0x56286f42c790 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1436000640 1 -- 172.21.15.86:0/3580360814 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f14400124d0 con 0x56286f42c790 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1436000640 1 -- 172.21.15.86:0/3580360814 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1440020050 con 0x56286f42c790 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 10 client.7047 fetch_fsmap learned FSMap version 52 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 -- 172.21.15.86:0/3580360814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x56286f1fc880 con 0x56286f42c790 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1436000640 1 -- 172.21.15.86:0/3580360814 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1430053840 con 0x56286f42c790 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1436000640 1 -- 172.21.15.86:0/3580360814 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f1440023020 con 0x56286f42c790 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 10 client.7047 fetch_fsmap finished waiting for FSMap version 52 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 10 client.7047 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:15:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 20 client.7047 populate_metadata read hostname 'smithi086' 2024-03-22T04:15:25.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x56286f3e5ca0 0x56286f43d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 4 client.7047 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:15:25.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1452d16000 1 -- 172.21.15.86:0/3580360814 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56286f1cbbc0 con 0x56286f3e5ca0 2024-03-22T04:15:25.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144ce00640 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x56286f3e5ca0 0x56286f43d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f144ce00640 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x56286f3e5ca0 0x56286f43d120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac095efb0652a336 server_cookie=2905320b041cc58e in_seq=0 out_seq=0 2024-03-22T04:15:25.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.903+0000 7f1436000640 10 client.7047 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:25.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1436000640 1 -- 172.21.15.86:0/3580360814 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56286f1cbbc0 con 0x56286f3e5ca0 2024-03-22T04:15:25.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1436000640 10 client.7047 handle_command_reply: tid=0 2024-03-22T04:15:25.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 client.7047 shutdown 2024-03-22T04:15:25.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 -- 172.21.15.86:0/3580360814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 msgr2=0x56286f3e5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3e5290 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1440001e00 tx=0x7f1440005420 comp rx=0 tx=0).stop 2024-03-22T04:15:25.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 -- 172.21.15.86:0/3580360814 shutdown_connections 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x56286f3e5ca0 0x56286f43d120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f3e5290 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f42be70 0x56286f3e4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1436000640 0 client.7047 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 --2- 172.21.15.86:0/3580360814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f3dfc20 0x56286f3e4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 -- 172.21.15.86:0/3580360814 >> 172.21.15.86:0/3580360814 conn(0x56286f40cc10 msgr2=0x56286f40f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 -- 172.21.15.86:0/3580360814 shutdown_connections 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 1 -- 172.21.15.86:0/3580360814 wait complete. 2024-03-22T04:15:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.907+0000 7f1452d16000 20 client.7047 trim_cache size 0 max 16384 2024-03-22T04:15:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f1452d16000 1 Processor -- start 2024-03-22T04:15:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f1452d16000 1 -- start start 2024-03-22T04:15:25.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f1452d16000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f42af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f1452d16000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f42b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f1452d16000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f43d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f1452d16000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1448203fa0 con 0x56286f42c790 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f1452d16000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1448202fc0 con 0x56286f42be70 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f1452d16000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14481571a0 con 0x56286f3dfc20 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f43d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f43d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40676/0 (socket says 172.21.15.86:40676) 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144d800640 1 -- 172.21.15.86:0/4087326040 learned_addr learned my addr 172.21.15.86:0/4087326040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f143fe00640 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f42b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144ce00640 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f42af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144d800640 1 -- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 msgr2=0x56286f42af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144d800640 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f42af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144d800640 1 -- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 msgr2=0x56286f42b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144d800640 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f42b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144d800640 1 -- 172.21.15.86:0/4087326040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56286f1b8e80 con 0x56286f42c790 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.923+0000 7f144ce00640 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f42af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f43d120 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1440001720 tx=0x7f144000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa45e1cefac2eeaf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f143cc00640 1 -- 172.21.15.86:0/4087326040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14400124d0 con 0x56286f42c790 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f143cc00640 1 -- 172.21.15.86:0/4087326040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1440001940 con 0x56286f42c790 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f143cc00640 1 -- 172.21.15.86:0/4087326040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440006d80 con 0x56286f42c790 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- 172.21.15.86:0/4087326040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 msgr2=0x56286f43d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f43d120 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1440001720 tx=0x7f144000e000 comp rx=0 tx=0).stop 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- 172.21.15.86:0/4087326040 shutdown_connections 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f42b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f43d120 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 --2- 172.21.15.86:0/4087326040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f42af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- 172.21.15.86:0/4087326040 >> 172.21.15.86:0/4087326040 conn(0x56286f40cc10 msgr2=0x56286f3ded40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- 172.21.15.86:0/4087326040 shutdown_connections 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- 172.21.15.86:0/4087326040 wait complete. 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- start start 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 Processor -- start 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f4402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f456a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f456fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56286f270a60 con 0x56286f42c790 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1448202ac0 con 0x56286f42be70 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56286f22fe20 con 0x56286f3dfc20 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1452d16000 10 client.0 _ll_register_callbacks cb 0x7f1440003980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f456fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144ce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f4402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f456fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40688/0 (socket says 172.21.15.86:40688) 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 -- 172.21.15.86:0/2750205244 learned_addr learned my addr 172.21.15.86:0/2750205244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f143fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f456a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 -- 172.21.15.86:0/2750205244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 msgr2=0x56286f4402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f4402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 -- 172.21.15.86:0/2750205244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 msgr2=0x56286f456a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f456a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 -- 172.21.15.86:0/2750205244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56286f1c4750 con 0x56286f42c790 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f144d800640 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f456fb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f14400013e0 tx=0x7f14400014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20828d6e347485 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.927+0000 7f1436000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:15:25.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 1 -- 172.21.15.86:0/2750205244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14400124d0 con 0x56286f42c790 2024-03-22T04:15:25.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 1 -- 172.21.15.86:0/2750205244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f1440031a70 con 0x56286f42c790 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 -- 172.21.15.86:0/2750205244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1440001940 con 0x56286f42c790 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 1 -- 172.21.15.86:0/2750205244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f144003b050 con 0x56286f42c790 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 10 client.7053 fetch_fsmap learned FSMap version 52 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 -- 172.21.15.86:0/2750205244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x56286f1d7450 con 0x56286f42c790 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 1 -- 172.21.15.86:0/2750205244 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1440001940 con 0x56286f42c790 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 1 -- 172.21.15.86:0/2750205244 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f1440023040 con 0x56286f42c790 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 10 client.7053 fetch_fsmap finished waiting for FSMap version 52 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 10 client.7053 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 20 client.7053 populate_metadata read hostname 'smithi086' 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x56286f49b130 0x56286f49b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 4 client.7053 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 -- 172.21.15.86:0/2750205244 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "status"}) -- 0x56286f23e050 con 0x56286f49b130 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144ce00640 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x56286f49b130 0x56286f49b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 10 client.7053 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144ce00640 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x56286f49b130 0x56286f49b510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65dca07a57e681fe server_cookie=e89a3e802a9551ba in_seq=0 out_seq=0 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 1 -- 172.21.15.86:0/2750205244 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56286f23e050 con 0x56286f49b130 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 10 client.7053 handle_command_reply: tid=0 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 client.7053 shutdown 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 -- 172.21.15.86:0/2750205244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 msgr2=0x56286f456fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f456fb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f14400013e0 tx=0x7f14400014b0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 -- 172.21.15.86:0/2750205244 shutdown_connections 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x56286f49b130 0x56286f49b510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56286f42be70 0x56286f456a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56286f42c790 0x56286f456fb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 --2- 172.21.15.86:0/2750205244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56286f3dfc20 0x56286f4402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 -- 172.21.15.86:0/2750205244 >> 172.21.15.86:0/2750205244 conn(0x56286f40cc10 msgr2=0x56286f429f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1436000640 0 client.7053 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 -- 172.21.15.86:0/2750205244 shutdown_connections 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 1 -- 172.21.15.86:0/2750205244 wait complete. 2024-03-22T04:15:25.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f1452d16000 20 client.7053 trim_cache size 0 max 16384 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7002, 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.5276183059999999, 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 52, 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 81, 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 81, 2024-03-22T04:15:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 5.4551239479999998 2024-03-22T04:15:25.941 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:15:25.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 -- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1424067b20 msgr2=0x7f1424069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1424067b20 0x7f1424069fe0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f14380ae000 tx=0x7f14380b4040 comp rx=0 tx=0).stop 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 -- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14480ae5e0 msgr2=0x7f14481483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14480ae5e0 0x7f14481483f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f143000c4a0 tx=0x7f143000c960 comp rx=0 tx=0).stop 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 -- 172.21.15.86:0/1089929413 shutdown_connections 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1424067b20 0x7f1424069fe0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f144815f560 0x7f1448153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14480ae5e0 0x7f14481483f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 --2- 172.21.15.86:0/1089929413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1448070390 0x7f144814fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 -- 172.21.15.86:0/1089929413 >> 172.21.15.86:0/1089929413 conn(0x7f14480d0860 msgr2=0x7f14480b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 -- 172.21.15.86:0/1089929413 shutdown_connections 2024-03-22T04:15:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:25.931+0000 7f144f800640 1 -- 172.21.15.86:0/1089929413 wait complete. 2024-03-22T04:15:25.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:15:26.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1cabe00640 1 Processor -- start 2024-03-22T04:15:26.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1cabe00640 1 -- start start 2024-03-22T04:15:26.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1cabe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ca40ad330 0x7f1ca415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1cabe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca40ad710 0x7f1ca4142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1cabe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca415c7c0 con 0x7f1ca406ef40 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1cabe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca406dea0 con 0x7f1ca40ad330 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1cabe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca406e180 con 0x7f1ca40ad710 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1ca9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ca40ad330 0x7f1ca415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1ca8a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1ca406ef40 0x7f1ca413fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59932/0 (socket says 172.21.15.86:59932) 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1ca8a00640 1 -- 172.21.15.86:0/746394837 learned_addr learned my addr 172.21.15.86:0/746394837 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.091+0000 7f1ca9e00640 1 --2- 172.21.15.86:0/746394837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca40ad710 0x7f1ca4142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.095+0000 7f1ca9400640 1 -- 172.21.15.86:0/746394837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca40ad710 msgr2=0x7f1ca4142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.095+0000 7f1ca9400640 1 --2- 172.21.15.86:0/746394837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca40ad710 0x7f1ca4142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.095+0000 7f1ca9400640 1 -- 172.21.15.86:0/746394837 >> v1:172.21.15.86:6789/0 conn(0x7f1ca406ef40 legacy=0x7f1ca413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.095+0000 7f1ca9400640 1 -- 172.21.15.86:0/746394837 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca406e460 con 0x7f1ca40ad330 2024-03-22T04:15:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.095+0000 7f1ca3e00640 1 -- 172.21.15.86:0/746394837 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491190058 0 0) 0x7f1ca415c7c0 con 0x7f1ca406ef40 2024-03-22T04:15:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1ca9400640 1 --2- 172.21.15.86:0/746394837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca415d4a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1c980097c0 tx=0x7f1c9801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46860e40fc0cb0ba server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1ca3e00640 1 -- 172.21.15.86:0/746394837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c980a7070 con 0x7f1ca40ad330 2024-03-22T04:15:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1ca3e00640 1 -- 172.21.15.86:0/746394837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c980b0070 con 0x7f1ca40ad330 2024-03-22T04:15:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1ca3e00640 1 -- 172.21.15.86:0/746394837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c980ab890 con 0x7f1ca40ad330 2024-03-22T04:15:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 -- 172.21.15.86:0/746394837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 msgr2=0x7f1ca415d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:26.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 --2- 172.21.15.86:0/746394837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca415d4a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1c980097c0 tx=0x7f1c9801c470 comp rx=0 tx=0).stop 2024-03-22T04:15:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 -- 172.21.15.86:0/746394837 shutdown_connections 2024-03-22T04:15:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 --2- 172.21.15.86:0/746394837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca40ad710 0x7f1ca4142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 --2- 172.21.15.86:0/746394837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca415d4a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 -- 172.21.15.86:0/746394837 >> 172.21.15.86:0/746394837 conn(0x7f1ca40b1df0 msgr2=0x7f1ca40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 -- 172.21.15.86:0/746394837 shutdown_connections 2024-03-22T04:15:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 -- 172.21.15.86:0/746394837 wait complete. 2024-03-22T04:15:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 Processor -- start 2024-03-22T04:15:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.099+0000 7f1cabe00640 1 -- start start 2024-03-22T04:15:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca406ef40 0x7f1ca415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca4165330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca40ad710 0x7f1ca41411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca41690e0 con 0x7f1ca40ad330 2024-03-22T04:15:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4168f60 con 0x7f1ca40ad710 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca4169260 con 0x7f1ca406ef40 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca4165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca4165330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40704/0 (socket says 172.21.15.86:40704) 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 -- 172.21.15.86:0/1541076793 learned_addr learned my addr 172.21.15.86:0/1541076793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca9400640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca406ef40 0x7f1ca415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca9e00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca40ad710 0x7f1ca41411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 -- 172.21.15.86:0/1541076793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca406ef40 msgr2=0x7f1ca415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca406ef40 0x7f1ca415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 -- 172.21.15.86:0/1541076793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca40ad710 msgr2=0x7f1ca41411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca40ad710 0x7f1ca41411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 -- 172.21.15.86:0/1541076793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca415b140 con 0x7f1ca40ad330 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca8a00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca4165330 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1c900011f0 tx=0x7f1c9000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca1600640 1 -- 172.21.15.86:0/1541076793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90018070 con 0x7f1ca40ad330 2024-03-22T04:15:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca1600640 1 -- 172.21.15.86:0/1541076793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c900074e0 con 0x7f1ca40ad330 2024-03-22T04:15:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca1600640 1 -- 172.21.15.86:0/1541076793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90004a10 con 0x7f1ca40ad330 2024-03-22T04:15:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca415b420 con 0x7f1ca40ad330 2024-03-22T04:15:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca415bcc0 con 0x7f1ca40ad330 2024-03-22T04:15:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca415b5f0 con 0x7f1ca40ad330 2024-03-22T04:15:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.103+0000 7f1ca1600640 1 -- 172.21.15.86:0/1541076793 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1c9001b070 con 0x7f1ca40ad330 2024-03-22T04:15:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.107+0000 7f1ca1600640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c84067b20 0x7f1c84069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.107+0000 7f1ca1600640 1 -- 172.21.15.86:0/1541076793 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f1c90085510 con 0x7f1ca40ad330 2024-03-22T04:15:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.107+0000 7f1ca9400640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c84067b20 0x7f1c84069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:26.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.107+0000 7f1ca9400640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c84067b20 0x7f1c84069fe0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c980af000 tx=0x7f1c980a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:26.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.111+0000 7f1ca1600640 1 -- 172.21.15.86:0/1541076793 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1c90055ca0 con 0x7f1ca40ad330 2024-03-22T04:15:26.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.303+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ca4068650 con 0x7f1ca40ad330 2024-03-22T04:15:26.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.303+0000 7f1ca1600640 1 -- 172.21.15.86:0/1541076793 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7f1c900555a0 con 0x7f1ca40ad330 2024-03-22T04:15:26.309 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c84067b20 msgr2=0x7f1c84069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c84067b20 0x7f1c84069fe0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c980af000 tx=0x7f1c980a7000 comp rx=0 tx=0).stop 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 msgr2=0x7f1ca4165330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca4165330 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1c900011f0 tx=0x7f1c9000c9b0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 shutdown_connections 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1c84067b20 0x7f1c84069fe0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca40ad710 0x7f1ca41411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca40ad330 0x7f1ca4165330 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 --2- 172.21.15.86:0/1541076793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ca406ef40 0x7f1ca415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:26.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 >> 172.21.15.86:0/1541076793 conn(0x7f1ca40b1df0 msgr2=0x7f1ca40730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:26.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 shutdown_connections 2024-03-22T04:15:26.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26.307+0000 7f1cabe00640 1 -- 172.21.15.86:0/1541076793 wait complete. 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:15:26.330 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:15:26.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:26.331 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:15:26.340 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:15:26.340 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:15:26.340 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:15:26.340 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:26.340 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:15:26.340 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:26.341 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:26.342 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:15:26.343 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:15:26.343 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:15:26.343 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:15:26.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:26.343 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:15:26.398 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:15:26.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:26.400 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:15:26.410 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-03-22T04:15:26.407+0000 7febf8c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:15:26.420 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:15:26.421 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:15:26.421 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:15:26.422 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:15:26.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:15:26.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:15:26.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:15:26.422 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:15:26.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:15:26.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:15:26.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:26.557 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:15:26.605 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:15:26.605 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:15:26.606 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:15:26.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:26.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:15:26.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:15:26.670 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:26.722 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:15:26.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:15:26.722 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:26.772 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:15:26.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:26.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:15:26.861 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:15:26.861 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:15:26.861 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:15:26.877 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:15:26.877 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:15:26.877 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:15:26.877 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:15:26.877 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:15:26.885 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:15:26.886 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:15:26.886 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:15:26.886 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:15:26.886 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:15:26.886 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:15:26.886 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:15:26.886 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:15:26.889 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:15:26.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:26.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:26.910 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:15:27.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:15:27.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:15:27.010 INFO:teuthology.orchestra.run.smithi086.stdout:client7065 2024-03-22T04:15:27.010 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:15:27.010 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:27.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:27.062 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-03-22T04:15:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:file_undamaged 2024-03-22T04:15:27.121 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> import json 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> import sys 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> try: 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> except OSError as e: 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> sys.exit(e.errno) 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> print(json.dumps( 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> dict([(a, getattr(s, a)) for a in attrs]), 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> indent=2)) 2024-03-22T04:15:27.122 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:15:27.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-03-22T04:15:27.248 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-22T04:15:27.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:27.248 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-22T04:15:27.299 INFO:teuthology.orchestra.run.smithi086.stdout:2 2024-03-22T04:15:27.300 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:15:27.300 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:27.354 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:15:27.354 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:15:27.406 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:15:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b6d800640 1 Processor -- start 2024-03-22T04:15:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b6d800640 1 -- start start 2024-03-22T04:15:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b6d800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0b680ad330 0x7f0b6815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b6d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b6806ef40 0x7f0b6806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b6d800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b6813fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b6d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0b6806dc40 con 0x7f0b6806ef40 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b6d800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0b6815c7c0 con 0x7f0b680ad330 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b6d800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6806dea0 con 0x7f0b680ad710 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b6806ef40 0x7f0b6806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b6806ef40 0x7f0b6806d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40716/0 (socket says 172.21.15.86:40716) 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66000640 1 -- 172.21.15.86:0/2660794769 learned_addr learned my addr 172.21.15.86:0/2660794769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66a00640 1 --2- 172.21.15.86:0/2660794769 >> v2:172.21.15.196:3300/0 conn(0x7f0b680ad330 0x7f0b6815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b67400640 1 --2- 172.21.15.86:0/2660794769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b6813fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66000640 1 -- 172.21.15.86:0/2660794769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 msgr2=0x7f0b6813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66000640 1 --2- 172.21.15.86:0/2660794769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b6813fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66000640 1 -- 172.21.15.86:0/2660794769 >> v2:172.21.15.196:3300/0 conn(0x7f0b680ad330 msgr2=0x7f0b6815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66000640 1 --2- 172.21.15.86:0/2660794769 >> v2:172.21.15.196:3300/0 conn(0x7f0b680ad330 0x7f0b6815d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b66000640 1 -- 172.21.15.86:0/2660794769 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6806e180 con 0x7f0b6806ef40 2024-03-22T04:15:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.583+0000 7f0b67400640 1 --2- 172.21.15.86:0/2660794769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b6813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.587+0000 7f0b66000640 1 --2- 172.21.15.86:0/2660794769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6806d700 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0b6806d080 tx=0x7f0b5401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becd90ee5898c159 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b65600640 1 -- 172.21.15.86:0/2660794769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b540aa610 con 0x7f0b6806ef40 2024-03-22T04:15:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b65600640 1 -- 172.21.15.86:0/2660794769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b540af070 con 0x7f0b6806ef40 2024-03-22T04:15:27.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b65600640 1 -- 172.21.15.86:0/2660794769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b540aa9d0 con 0x7f0b6806ef40 2024-03-22T04:15:27.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 -- 172.21.15.86:0/2660794769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 msgr2=0x7f0b6806d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 --2- 172.21.15.86:0/2660794769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6806d700 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0b6806d080 tx=0x7f0b5401c4f0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 -- 172.21.15.86:0/2660794769 shutdown_connections 2024-03-22T04:15:27.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 --2- 172.21.15.86:0/2660794769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b6813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 --2- 172.21.15.86:0/2660794769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6806d700 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 --2- 172.21.15.86:0/2660794769 >> v2:172.21.15.196:3300/0 conn(0x7f0b680ad330 0x7f0b6815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 -- 172.21.15.86:0/2660794769 >> 172.21.15.86:0/2660794769 conn(0x7f0b680b1df0 msgr2=0x7f0b680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:27.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 -- 172.21.15.86:0/2660794769 shutdown_connections 2024-03-22T04:15:27.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 -- 172.21.15.86:0/2660794769 wait complete. 2024-03-22T04:15:27.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 Processor -- start 2024-03-22T04:15:27.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.591+0000 7f0b6d800640 1 -- start start 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6814a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b680ad330 0x7f0b681642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b68164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68141800 con 0x7f0b6806ef40 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68141680 con 0x7f0b680ad330 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0b68141980 con 0x7f0b680ad710 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6814a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6814a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40724/0 (socket says 172.21.15.86:40724) 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 -- 172.21.15.86:0/887867388 learned_addr learned my addr 172.21.15.86:0/887867388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66000640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b680ad330 0x7f0b681642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b67400640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b68164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 -- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 msgr2=0x7f0b68164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b68164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 -- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b680ad330 msgr2=0x7f0b681642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b680ad330 0x7f0b681642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 -- 172.21.15.86:0/887867388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6815b380 con 0x7f0b6806ef40 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66000640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b680ad330 0x7f0b681642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b66a00640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6814a4d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800e8b0 tx=0x7f0b5800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b5ea00640 1 -- 172.21.15.86:0/887867388 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58017070 con 0x7f0b6806ef40 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b5ea00640 1 -- 172.21.15.86:0/887867388 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5800c940 con 0x7f0b6806ef40 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6815b660 con 0x7f0b6806ef40 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b5ea00640 1 -- 172.21.15.86:0/887867388 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800cd10 con 0x7f0b6806ef40 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6815bc20 con 0x7f0b6806ef40 2024-03-22T04:15:27.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.595+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b68068650 con 0x7f0b6806ef40 2024-03-22T04:15:27.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.599+0000 7f0b5ea00640 1 -- 172.21.15.86:0/887867388 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0b5801a070 con 0x7f0b6806ef40 2024-03-22T04:15:27.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.599+0000 7f0b5ea00640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b48067b20 0x7f0b48069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.599+0000 7f0b5ea00640 1 -- 172.21.15.86:0/887867388 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f0b580865a0 con 0x7f0b6806ef40 2024-03-22T04:15:27.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.599+0000 7f0b66000640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b48067b20 0x7f0b48069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.599+0000 7f0b66000640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b48067b20 0x7f0b48069fe0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0b540ae000 tx=0x7f0b540a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:27.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.603+0000 7f0b5ea00640 1 -- 172.21.15.86:0/887867388 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0b58056ca0 con 0x7f0b6806ef40 2024-03-22T04:15:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.759+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b6815c410 con 0x7f0b6806ef40 2024-03-22T04:15:27.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b5ea00640 1 -- 172.21.15.86:0/887867388 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+17014 (secure 0 0 0) 0x7f0b58083290 con 0x7f0b6806ef40 2024-03-22T04:15:27.767 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:15:27.767 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:20.393125+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1130704586":"2024-03-23T04:15:20.393105+0000","172.21.15.86:6836/1130704586":"2024-03-23T04:15:20.393105+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b48067b20 msgr2=0x7f0b48069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b48067b20 0x7f0b48069fe0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0b540ae000 tx=0x7f0b540a7000 comp rx=0 tx=0).stop 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 msgr2=0x7f0b6814a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6814a4d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800e8b0 tx=0x7f0b5800ed70 comp rx=0 tx=0).stop 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 shutdown_connections 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b48067b20 0x7f0b48069fe0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b680ad330 0x7f0b681642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b680ad710 0x7f0b68164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 --2- 172.21.15.86:0/887867388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6806ef40 0x7f0b6814a4d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 >> 172.21.15.86:0/887867388 conn(0x7f0b680b1df0 msgr2=0x7f0b68073070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 shutdown_connections 2024-03-22T04:15:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.763+0000 7f0b6d800640 1 -- 172.21.15.86:0/887867388 wait complete. 2024-03-22T04:15:27.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:15:07.783245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:15:08.394997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:15:27.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:15:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.923+0000 7f3bc9000640 1 Processor -- start 2024-03-22T04:15:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.923+0000 7f3bc9000640 1 -- start start 2024-03-22T04:15:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3bc406bd70 0x7f3bc4169580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc9000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc40ceb30 con 0x7f3bc4161db0 2024-03-22T04:15:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc9000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc40cee10 con 0x7f3bc41619d0 2024-03-22T04:15:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc9000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc40cf0f0 con 0x7f3bc406bd70 2024-03-22T04:15:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bbbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3bc4161db0 0x7f3bc406b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59942/0 (socket says 172.21.15.86:59942) 2024-03-22T04:15:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bbbe00640 1 -- 172.21.15.86:0/2826783960 learned_addr learned my addr 172.21.15.86:0/2826783960 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc3e00640 1 --2- 172.21.15.86:0/2826783960 >> v2:172.21.15.86:3300/0 conn(0x7f3bc406bd70 0x7f3bc4169580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc2a00640 1 -- 172.21.15.86:0/2826783960 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 494502683 0 0) 0x7f3bc40ceb30 con 0x7f3bc4161db0 2024-03-22T04:15:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc2a00640 1 -- 172.21.15.86:0/2826783960 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ba8003530 con 0x7f3bc4161db0 2024-03-22T04:15:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc3e00640 1 -- 172.21.15.86:0/2826783960 >> v1:172.21.15.196:6789/0 conn(0x7f3bc41619d0 legacy=0x7f3bc40affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc3e00640 1 -- 172.21.15.86:0/2826783960 >> v1:172.21.15.86:6789/0 conn(0x7f3bc4161db0 legacy=0x7f3bc406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.927+0000 7f3bc3e00640 1 -- 172.21.15.86:0/2826783960 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc40cf3d0 con 0x7f3bc406bd70 2024-03-22T04:15:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc3e00640 1 --2- 172.21.15.86:0/2826783960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc406bd70 0x7f3bc4169580 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009d90 tx=0x7f3bbc01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a143b3e242ada71 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc2a00640 1 -- 172.21.15.86:0/2826783960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0b2070 con 0x7f3bc406bd70 2024-03-22T04:15:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc2a00640 1 -- 172.21.15.86:0/2826783960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bbc0a5a20 con 0x7f3bc406bd70 2024-03-22T04:15:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc2a00640 1 -- 172.21.15.86:0/2826783960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0a5df0 con 0x7f3bc406bd70 2024-03-22T04:15:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 -- 172.21.15.86:0/2826783960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc406bd70 msgr2=0x7f3bc4169580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 --2- 172.21.15.86:0/2826783960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc406bd70 0x7f3bc4169580 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009d90 tx=0x7f3bbc01d3a0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 -- 172.21.15.86:0/2826783960 shutdown_connections 2024-03-22T04:15:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 --2- 172.21.15.86:0/2826783960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc406bd70 0x7f3bc4169580 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 -- 172.21.15.86:0/2826783960 >> 172.21.15.86:0/2826783960 conn(0x7f3bc40d0e30 msgr2=0x7f3bc40d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 -- 172.21.15.86:0/2826783960 shutdown_connections 2024-03-22T04:15:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 -- 172.21.15.86:0/2826783960 wait complete. 2024-03-22T04:15:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 Processor -- start 2024-03-22T04:15:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.931+0000 7f3bc9000640 1 -- start start 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc406bd70 0x7f3bc406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc41619d0 0x7f3bc40addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc4161db0 0x7f3bc40ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40cc820 con 0x7f3bc4161db0 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4163010 con 0x7f3bc41619d0 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc40cc9a0 con 0x7f3bc406bd70 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc4161db0 0x7f3bc40ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc4161db0 0x7f3bc40ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40748/0 (socket says 172.21.15.86:40748) 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 -- 172.21.15.86:0/1202361872 learned_addr learned my addr 172.21.15.86:0/1202361872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bbbe00640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc41619d0 0x7f3bc40addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3400640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc406bd70 0x7f3bc406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 -- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc406bd70 msgr2=0x7f3bc406e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc406bd70 0x7f3bc406e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 -- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc41619d0 msgr2=0x7f3bc40addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc41619d0 0x7f3bc40addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 -- 172.21.15.86:0/1202361872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc415bc60 con 0x7f3bc4161db0 2024-03-22T04:15:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bbbe00640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc41619d0 0x7f3bc40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc3e00640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc4161db0 0x7f3bc40ae2f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0b0040 tx=0x7f3bbc0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bbb400640 1 -- 172.21.15.86:0/1202361872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0bd020 con 0x7f3bc4161db0 2024-03-22T04:15:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bbb400640 1 -- 172.21.15.86:0/1202361872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bbc0ad640 con 0x7f3bc4161db0 2024-03-22T04:15:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bbb400640 1 -- 172.21.15.86:0/1202361872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0b2040 con 0x7f3bc4161db0 2024-03-22T04:15:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc406ea30 con 0x7f3bc4161db0 2024-03-22T04:15:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc406ecb0 con 0x7f3bc4161db0 2024-03-22T04:15:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc404f900 con 0x7f3bc4161db0 2024-03-22T04:15:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.935+0000 7f3bbb400640 1 -- 172.21.15.86:0/1202361872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3bbc0c0070 con 0x7f3bc4161db0 2024-03-22T04:15:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.939+0000 7f3bbb400640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ba0067b20 0x7f3ba0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.939+0000 7f3bbb400640 1 -- 172.21.15.86:0/1202361872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f3bbc1201c0 con 0x7f3bc4161db0 2024-03-22T04:15:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.939+0000 7f3bc3400640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ba0067b20 0x7f3ba0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.939+0000 7f3bc3400640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ba0067b20 0x7f3ba0069fe0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4001810 tx=0x7f3bb4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:27.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:27.947+0000 7f3bbb400640 1 -- 172.21.15.86:0/1202361872 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3bc406ea30 con 0x7f3bc4161db0 2024-03-22T04:15:28.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:15:28.063 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:15:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.107+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3bc40c5da0 con 0x7f3bc4161db0 2024-03-22T04:15:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.107+0000 7f3bbb400640 1 -- 172.21.15.86:0/1202361872 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f3bbc016560 con 0x7f3bc4161db0 2024-03-22T04:15:28.113 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ba0067b20 msgr2=0x7f3ba0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ba0067b20 0x7f3ba0069fe0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4001810 tx=0x7f3bb4009040 comp rx=0 tx=0).stop 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc4161db0 msgr2=0x7f3bc40ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc4161db0 0x7f3bc40ae2f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0b0040 tx=0x7f3bbc0196e0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 shutdown_connections 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ba0067b20 0x7f3ba0069fe0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc41619d0 0x7f3bc40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bc4161db0 0x7f3bc40ae2f0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 --2- 172.21.15.86:0/1202361872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc406bd70 0x7f3bc406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 >> 172.21.15.86:0/1202361872 conn(0x7f3bc40d0e30 msgr2=0x7f3bc41621e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 shutdown_connections 2024-03-22T04:15:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.111+0000 7f3bc9000640 1 -- 172.21.15.86:0/1202361872 wait complete. 2024-03-22T04:15:28.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.323+0000 7f702e200640 1 Processor -- start 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.323+0000 7f702e200640 1 -- start start 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.323+0000 7f702e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 0x7f702809b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.323+0000 7f702e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70280cda00 con 0x7f702815c730 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.323+0000 7f702e200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70280cdce0 con 0x7f70280b3610 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.323+0000 7f702e200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f702809b910 con 0x7f70280b11c0 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7027400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f702815c730 0x7f702809aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59958/0 (socket says 172.21.15.86:59958) 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7027400640 1 -- 172.21.15.86:0/1389776694 learned_addr learned my addr 172.21.15.86:0/1389776694 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7027e00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 0x7f702809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3879566924 0 0) 0x7f70280cda00 con 0x7f702815c730 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f700c003530 con 0x7f702815c730 2024-03-22T04:15:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 125165873 0 0) 0x7f700c003530 con 0x7f702815c730 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 msgr2=0x7f702809b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 0x7f702809b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 >> v1:172.21.15.196:6789/0 conn(0x7f70280b3610 legacy=0x7f70280b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70280cdfc0 con 0x7f702815c730 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f7018001680 con 0x7f702815c730 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7027e00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 0x7f702809b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 >> v1:172.21.15.86:6789/0 conn(0x7f702815c730 legacy=0x7f702809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 0x7f700c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f700c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f700c004d50 0x7f700c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f702809b910 con 0x7f700c003f30 2024-03-22T04:15:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f700c00bb10 con 0x7f700c004d50 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f700c00bd10 con 0x7f70280b11c0 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f701fe00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f700c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7027400640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 0x7f700c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f7027e00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f700c004d50 0x7f700c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f701fe00640 1 -- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 msgr2=0x7f700c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f701fe00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280b11c0 0x7f700c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f701fe00640 1 -- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f700c004d50 msgr2=0x7f700c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f701fe00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f700c004d50 0x7f700c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.327+0000 7f701fe00640 1 -- 172.21.15.86:0/1389776694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c003530 con 0x7f700c003f30 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f701fe00640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f700c004810 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7010001860 tx=0x7f701001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa59ee0662d3082 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70100b0070 con 0x7f700c003f30 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70100ab600 con 0x7f700c003f30 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f7026a00640 1 -- 172.21.15.86:0/1389776694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010001460 con 0x7f700c003f30 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f702e200640 1 -- 172.21.15.86:0/1389776694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 msgr2=0x7f700c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f702e200640 1 --2- 172.21.15.86:0/1389776694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f700c004810 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7010001860 tx=0x7f701001c500 comp rx=0 tx=0).stop 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f7027e00640 1 -- 172.21.15.86:0/1389776694 reap_dead start 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f702e200640 1 -- 172.21.15.86:0/1389776694 shutdown_connections 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f702e200640 1 -- 172.21.15.86:0/1389776694 >> 172.21.15.86:0/1389776694 conn(0x7f70280b3bf0 msgr2=0x7f70280b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f702e200640 1 -- 172.21.15.86:0/1389776694 shutdown_connections 2024-03-22T04:15:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f702e200640 1 -- 172.21.15.86:0/1389776694 wait complete. 2024-03-22T04:15:28.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f702e200640 1 Processor -- start 2024-03-22T04:15:28.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.331+0000 7f702e200640 1 -- start start 2024-03-22T04:15:28.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f70280c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7028155490 0x7f7028157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cb4d0 0x7f70280cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70280b2e50 con 0x7f700c003f30 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70280b2cd0 con 0x7f70280cb4d0 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7028168390 con 0x7f7028155490 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f70280c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f70280c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40778/0 (socket says 172.21.15.86:40778) 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 -- 172.21.15.86:0/247789924 learned_addr learned my addr 172.21.15.86:0/247789924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f701fe00640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7028155490 0x7f7028157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027e00640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cb4d0 0x7f70280cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 -- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7028155490 msgr2=0x7f7028157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7028155490 0x7f7028157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 -- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cb4d0 msgr2=0x7f70280cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cb4d0 0x7f70280cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 -- 172.21.15.86:0/247789924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7028158430 con 0x7f700c003f30 2024-03-22T04:15:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027e00640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cb4d0 0x7f70280cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f7027400640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f70280c9d00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7018004e50 tx=0x7f7018007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f701f400640 1 -- 172.21.15.86:0/247789924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701800f620 con 0x7f700c003f30 2024-03-22T04:15:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f701f400640 1 -- 172.21.15.86:0/247789924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7018018070 con 0x7f700c003f30 2024-03-22T04:15:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f701f400640 1 -- 172.21.15.86:0/247789924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701800fa20 con 0x7f700c003f30 2024-03-22T04:15:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702806c9c0 con 0x7f700c003f30 2024-03-22T04:15:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702806cf80 con 0x7f700c003f30 2024-03-22T04:15:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702806d180 con 0x7f700c003f30 2024-03-22T04:15:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.335+0000 7f701f400640 1 -- 172.21.15.86:0/247789924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f701800fbc0 con 0x7f700c003f30 2024-03-22T04:15:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.339+0000 7f701f400640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7004067b20 0x7f7004069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.339+0000 7f701f400640 1 -- 172.21.15.86:0/247789924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f7018085fc0 con 0x7f700c003f30 2024-03-22T04:15:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.339+0000 7f701fe00640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7004067b20 0x7f7004069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.339+0000 7f701fe00640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7004067b20 0x7f7004069fe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f70100ad000 tx=0x7f70100a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:28.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.343+0000 7f701f400640 1 -- 172.21.15.86:0/247789924 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f702806d180 con 0x7f700c003f30 2024-03-22T04:15:28.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7028074470 con 0x7f700c003f30 2024-03-22T04:15:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f701f400640 1 -- 172.21.15.86:0/247789924 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f7018056750 con 0x7f700c003f30 2024-03-22T04:15:28.503 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:15:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7004067b20 msgr2=0x7f7004069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7004067b20 0x7f7004069fe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f70100ad000 tx=0x7f70100a6000 comp rx=0 tx=0).stop 2024-03-22T04:15:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 msgr2=0x7f70280c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f70280c9d00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7018004e50 tx=0x7f7018007430 comp rx=0 tx=0).stop 2024-03-22T04:15:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 shutdown_connections 2024-03-22T04:15:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7004067b20 0x7f7004069fe0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cb4d0 0x7f70280cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7028155490 0x7f7028157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 --2- 172.21.15.86:0/247789924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c003f30 0x7f70280c9d00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 >> 172.21.15.86:0/247789924 conn(0x7f70280b3bf0 msgr2=0x7f70280aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 shutdown_connections 2024-03-22T04:15:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.499+0000 7f702e200640 1 -- 172.21.15.86:0/247789924 wait complete. 2024-03-22T04:15:28.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:15:28.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.659+0000 7f7740c00640 1 Processor -- start 2024-03-22T04:15:28.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.659+0000 7f7740c00640 1 -- start start 2024-03-22T04:15:28.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f7740c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f773c06b660 0x7f773c06bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f7740c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c15ea10 0x7f773c15eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f7740c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f773c15f970 con 0x7f773c158a40 2024-03-22T04:15:28.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f7740c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f773c15f3e0 con 0x7f773c06b660 2024-03-22T04:15:28.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f7740c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f773c158e20 con 0x7f773c15ea10 2024-03-22T04:15:28.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f773be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c15ea10 0x7f773c15eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f773b400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f773c158a40 0x7f773c0ae9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:55388/0 (socket says 172.21.15.86:55388) 2024-03-22T04:15:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f773b400640 1 -- 172.21.15.86:0/2804246908 learned_addr learned my addr 172.21.15.86:0/2804246908 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f7733e00640 1 --2- 172.21.15.86:0/2804246908 >> v2:172.21.15.196:3300/0 conn(0x7f773c06b660 0x7f773c06bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f773aa00640 1 -- 172.21.15.86:0/2804246908 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1233502787 0 0) 0x7f773c15f970 con 0x7f773c158a40 2024-03-22T04:15:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f773be00640 1 -- 172.21.15.86:0/2804246908 >> v2:172.21.15.196:3300/0 conn(0x7f773c06b660 msgr2=0x7f773c06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f773be00640 1 --2- 172.21.15.86:0/2804246908 >> v2:172.21.15.196:3300/0 conn(0x7f773c06b660 0x7f773c06bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f773be00640 1 -- 172.21.15.86:0/2804246908 >> v1:172.21.15.196:6789/0 conn(0x7f773c158a40 legacy=0x7f773c0ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.663+0000 7f773be00640 1 -- 172.21.15.86:0/2804246908 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c15f690 con 0x7f773c15ea10 2024-03-22T04:15:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f773be00640 1 --2- 172.21.15.86:0/2804246908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c15ea10 0x7f773c15eea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7734009d90 tx=0x7f773401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=444405988a87f434 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f773aa00640 1 -- 172.21.15.86:0/2804246908 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340b2070 con 0x7f773c15ea10 2024-03-22T04:15:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f773aa00640 1 -- 172.21.15.86:0/2804246908 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77340a5710 con 0x7f773c15ea10 2024-03-22T04:15:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f773aa00640 1 -- 172.21.15.86:0/2804246908 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340a5ae0 con 0x7f773c15ea10 2024-03-22T04:15:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f7740c00640 1 -- 172.21.15.86:0/2804246908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c15ea10 msgr2=0x7f773c15eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f7740c00640 1 --2- 172.21.15.86:0/2804246908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c15ea10 0x7f773c15eea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7734009d90 tx=0x7f773401c850 comp rx=0 tx=0).stop 2024-03-22T04:15:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f7740c00640 1 -- 172.21.15.86:0/2804246908 shutdown_connections 2024-03-22T04:15:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f7740c00640 1 --2- 172.21.15.86:0/2804246908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c15ea10 0x7f773c15eea0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f7740c00640 1 --2- 172.21.15.86:0/2804246908 >> v2:172.21.15.196:3300/0 conn(0x7f773c06b660 0x7f773c06bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f7740c00640 1 -- 172.21.15.86:0/2804246908 >> 172.21.15.86:0/2804246908 conn(0x7f773c0d0e30 msgr2=0x7f773c0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f7740c00640 1 -- 172.21.15.86:0/2804246908 shutdown_connections 2024-03-22T04:15:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.667+0000 7f7740c00640 1 -- 172.21.15.86:0/2804246908 wait complete. 2024-03-22T04:15:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 Processor -- start 2024-03-22T04:15:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 -- start start 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c06b660 0x7f773c157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c158a40 0x7f773c14a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f773c15ea10 0x7f773c152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f773c06eb00 con 0x7f773c15ea10 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f773c06e980 con 0x7f773c158a40 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f773c06ec80 con 0x7f773c06b660 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f773c15ea10 0x7f773c152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773b400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c06b660 0x7f773c157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f773c15ea10 0x7f773c152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40790/0 (socket says 172.21.15.86:40790) 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 -- 172.21.15.86:0/2429325811 learned_addr learned my addr 172.21.15.86:0/2429325811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7733e00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c158a40 0x7f773c14a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 -- 172.21.15.86:0/2429325811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c06b660 msgr2=0x7f773c157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c06b660 0x7f773c157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 -- 172.21.15.86:0/2429325811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c158a40 msgr2=0x7f773c14a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c158a40 0x7f773c14a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 -- 172.21.15.86:0/2429325811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c14a700 con 0x7f773c15ea10 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f773be00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f773c15ea10 0x7f773c152480 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7734009d90 tx=0x7f77340195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7733400640 1 -- 172.21.15.86:0/2429325811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340bd030 con 0x7f773c15ea10 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7733400640 1 -- 172.21.15.86:0/2429325811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77340a5510 con 0x7f773c15ea10 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c158230 con 0x7f773c15ea10 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7733400640 1 -- 172.21.15.86:0/2429325811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340b2040 con 0x7f773c15ea10 2024-03-22T04:15:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.671+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c156460 con 0x7f773c15ea10 2024-03-22T04:15:28.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.675+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773c04f900 con 0x7f773c15ea10 2024-03-22T04:15:28.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.675+0000 7f7733400640 1 -- 172.21.15.86:0/2429325811 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f77340c0070 con 0x7f773c15ea10 2024-03-22T04:15:28.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.675+0000 7f7733400640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7718067b20 0x7f7718069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:28.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.675+0000 7f7733400640 1 -- 172.21.15.86:0/2429325811 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f77341209a0 con 0x7f773c15ea10 2024-03-22T04:15:28.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.675+0000 7f773b400640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7718067b20 0x7f7718069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:28.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.679+0000 7f773b400640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7718067b20 0x7f7718069fe0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f772c000f70 tx=0x7f772c007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:28.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.679+0000 7f7733400640 1 -- 172.21.15.86:0/2429325811 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f77340c7070 con 0x7f773c15ea10 2024-03-22T04:15:28.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f773c0d1950 con 0x7f773c15ea10 2024-03-22T04:15:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7733400640 1 -- 172.21.15.86:0/2429325811 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f7734016560 con 0x7f773c15ea10 2024-03-22T04:15:29.114 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:15:29.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7718067b20 msgr2=0x7f7718069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7718067b20 0x7f7718069fe0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f772c000f70 tx=0x7f772c007df0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f773c15ea10 msgr2=0x7f773c152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f773c15ea10 0x7f773c152480 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7734009d90 tx=0x7f77340195f0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 shutdown_connections 2024-03-22T04:15:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7718067b20 0x7f7718069fe0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c158a40 0x7f773c14a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f773c15ea10 0x7f773c152480 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 --2- 172.21.15.86:0/2429325811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f773c06b660 0x7f773c157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 >> 172.21.15.86:0/2429325811 conn(0x7f773c0d0e30 msgr2=0x7f773c15c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 shutdown_connections 2024-03-22T04:15:29.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:28.839+0000 7f7740c00640 1 -- 172.21.15.86:0/2429325811 wait complete. 2024-03-22T04:15:29.123 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:15:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5de5e00640 1 Processor -- start 2024-03-22T04:15:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5de5e00640 1 -- start start 2024-03-22T04:15:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5de5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de00d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5de5e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5de0071b30 0x7f5de00b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5de5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5de00cfe70 0x7f5de00d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5de5e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5de015c880 con 0x7f5de00cfe70 2024-03-22T04:15:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5de5e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5de00ce900 con 0x7f5de0071b30 2024-03-22T04:15:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5de5e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5de00d0ca0 con 0x7f5de015c4a0 2024-03-22T04:15:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5de00cfe70 0x7f5de00d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5de00cfe70 0x7f5de00d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40796/0 (socket says 172.21.15.86:40796) 2024-03-22T04:15:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddf400640 1 -- 172.21.15.86:0/3499481250 learned_addr learned my addr 172.21.15.86:0/3499481250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddea00640 1 --2- 172.21.15.86:0/3499481250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de00d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5dd7e00640 1 --2- 172.21.15.86:0/3499481250 >> v2:172.21.15.196:3300/0 conn(0x7f5de0071b30 0x7f5de00b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddf400640 1 -- 172.21.15.86:0/3499481250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 msgr2=0x7f5de00d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddf400640 1 --2- 172.21.15.86:0/3499481250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de00d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddf400640 1 -- 172.21.15.86:0/3499481250 >> v2:172.21.15.196:3300/0 conn(0x7f5de0071b30 msgr2=0x7f5de00b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddf400640 1 --2- 172.21.15.86:0/3499481250 >> v2:172.21.15.196:3300/0 conn(0x7f5de0071b30 0x7f5de00b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.259+0000 7f5ddf400640 1 -- 172.21.15.86:0/3499481250 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de00ce5d0 con 0x7f5de00cfe70 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5ddf400640 1 --2- 172.21.15.86:0/3499481250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de00cfe70 0x7f5de00d0300 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5de006f480 tx=0x7f5dd801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0af0593a275e027 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5dde000640 1 -- 172.21.15.86:0/3499481250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80a5780 con 0x7f5de00cfe70 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5dde000640 1 -- 172.21.15.86:0/3499481250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd80a5920 con 0x7f5de00cfe70 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5dde000640 1 -- 172.21.15.86:0/3499481250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80a5cf0 con 0x7f5de00cfe70 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 -- 172.21.15.86:0/3499481250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de00cfe70 msgr2=0x7f5de00d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 --2- 172.21.15.86:0/3499481250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de00cfe70 0x7f5de00d0300 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5de006f480 tx=0x7f5dd801c9f0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 -- 172.21.15.86:0/3499481250 shutdown_connections 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 --2- 172.21.15.86:0/3499481250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de00cfe70 0x7f5de00d0300 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 --2- 172.21.15.86:0/3499481250 >> v2:172.21.15.196:3300/0 conn(0x7f5de0071b30 0x7f5de00b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 --2- 172.21.15.86:0/3499481250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de00d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 -- 172.21.15.86:0/3499481250 >> 172.21.15.86:0/3499481250 conn(0x7f5de006f930 msgr2=0x7f5de006fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 -- 172.21.15.86:0/3499481250 shutdown_connections 2024-03-22T04:15:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.263+0000 7f5de5e00640 1 -- 172.21.15.86:0/3499481250 wait complete. 2024-03-22T04:15:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5de5e00640 1 Processor -- start 2024-03-22T04:15:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5de5e00640 1 -- start start 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5de5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de0071b30 0x7f5de01f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5de5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de00cfe70 0x7f5de00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5de5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5de5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0072090 con 0x7f5de0071b30 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5de5e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0071f10 con 0x7f5de00cfe70 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5de5e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5de00b56b0 con 0x7f5de015c4a0 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5ddea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de0071b30 0x7f5de01f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5ddea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de0071b30 0x7f5de01f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40812/0 (socket says 172.21.15.86:40812) 2024-03-22T04:15:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.267+0000 7f5ddea00640 1 -- 172.21.15.86:0/2032971549 learned_addr learned my addr 172.21.15.86:0/2032971549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5ddea00640 1 -- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 msgr2=0x7f5de015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5ddf400640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5ddea00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5dd7e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de00cfe70 0x7f5de00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5ddea00640 1 -- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de00cfe70 msgr2=0x7f5de00c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5ddea00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de00cfe70 0x7f5de00c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5ddea00640 1 -- 172.21.15.86:0/2032971549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de00cb760 con 0x7f5de0071b30 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5ddf400640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5dd7e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de00cfe70 0x7f5de00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5ddea00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de0071b30 0x7f5de01f6420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000d5f0 tx=0x7f5dd000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5dd7400640 1 -- 172.21.15.86:0/2032971549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00125f0 con 0x7f5de0071b30 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5dd7400640 1 -- 172.21.15.86:0/2032971549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd0019070 con 0x7f5de0071b30 2024-03-22T04:15:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de00cba40 con 0x7f5de0071b30 2024-03-22T04:15:29.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5dd7400640 1 -- 172.21.15.86:0/2032971549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00129b0 con 0x7f5de0071b30 2024-03-22T04:15:29.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de01f6960 con 0x7f5de0071b30 2024-03-22T04:15:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de0068650 con 0x7f5de0071b30 2024-03-22T04:15:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.271+0000 7f5dd7400640 1 -- 172.21.15.86:0/2032971549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5dd0012b50 con 0x7f5de0071b30 2024-03-22T04:15:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.275+0000 7f5dd7400640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dbc067b20 0x7f5dbc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.275+0000 7f5dd7400640 1 -- 172.21.15.86:0/2032971549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f5dd00865f0 con 0x7f5de0071b30 2024-03-22T04:15:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.275+0000 7f5dd7e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dbc067b20 0x7f5dbc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.275+0000 7f5dd7e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dbc067b20 0x7f5dbc069fe0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5de0070490 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.279+0000 7f5dd7400640 1 -- 172.21.15.86:0/2032971549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5dd0056d80 con 0x7f5de0071b30 2024-03-22T04:15:29.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5de00cbdd0 con 0x7f5de0071b30 2024-03-22T04:15:29.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5dd7400640 1 -- 172.21.15.86:0/2032971549 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v81) ==== 80+0+122 (secure 0 0 0) 0x7f5dd0056680 con 0x7f5de0071b30 2024-03-22T04:15:29.476 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/1130704586 2024-03-23T04:15:20.393105+0000 2024-03-22T04:15:29.476 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6836/1130704586 2024-03-23T04:15:20.393105+0000 2024-03-22T04:15:29.476 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-03-22T04:15:29.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dbc067b20 msgr2=0x7f5dbc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dbc067b20 0x7f5dbc069fe0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5de0070490 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de0071b30 msgr2=0x7f5de01f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de0071b30 0x7f5de01f6420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000d5f0 tx=0x7f5dd000dab0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 shutdown_connections 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5dbc067b20 0x7f5dbc069fe0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de00cfe70 0x7f5de00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5de015c4a0 0x7f5de015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 --2- 172.21.15.86:0/2032971549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de0071b30 0x7f5de01f6420 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.471+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 >> 172.21.15.86:0/2032971549 conn(0x7f5de006f930 msgr2=0x7f5de00b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.475+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 shutdown_connections 2024-03-22T04:15:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.475+0000 7f5de5e00640 1 -- 172.21.15.86:0/2032971549 wait complete. 2024-03-22T04:15:29.497 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:15:29.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:29.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:15:29.522 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:15:29.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:29.523 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:15:29.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:29.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-03-22T04:15:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb37b800640 1 Processor -- start 2024-03-22T04:15:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb37b800640 1 -- start start 2024-03-22T04:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb37b800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb37406c980 0x7fb37406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb37b800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3740d05c0 con 0x7fb37415e670 2024-03-22T04:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb37b800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3740d08a0 con 0x7fb37409aa70 2024-03-22T04:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb37b800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb37415ea50 con 0x7fb37406c980 2024-03-22T04:15:29.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb378e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb37415e670 0x7fb3740d1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59972/0 (socket says 172.21.15.86:59972) 2024-03-22T04:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb378e00640 1 -- 172.21.15.86:0/3425075317 learned_addr learned my addr 172.21.15.86:0/3425075317 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb373e00640 1 --2- 172.21.15.86:0/3425075317 >> v2:172.21.15.86:3300/0 conn(0x7fb37406c980 0x7fb37406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb373400640 1 -- 172.21.15.86:0/3425075317 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464398780 0 0) 0x7fb3740d05c0 con 0x7fb37415e670 2024-03-22T04:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb373400640 1 -- 172.21.15.86:0/3425075317 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb35c003530 con 0x7fb37415e670 2024-03-22T04:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb373e00640 1 -- 172.21.15.86:0/3425075317 >> v1:172.21.15.196:6789/0 conn(0x7fb37409aa70 legacy=0x7fb3740cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb373e00640 1 -- 172.21.15.86:0/3425075317 >> v1:172.21.15.86:6789/0 conn(0x7fb37415e670 legacy=0x7fb3740d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.759+0000 7fb373e00640 1 -- 172.21.15.86:0/3425075317 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3740d0b80 con 0x7fb37406c980 2024-03-22T04:15:29.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.763+0000 7fb373e00640 1 --2- 172.21.15.86:0/3425075317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37406c980 0x7fb37406d870 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb3600097c0 tx=0x7fb36001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382964196a30b8c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.763+0000 7fb373400640 1 -- 172.21.15.86:0/3425075317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600b0070 con 0x7fb37406c980 2024-03-22T04:15:29.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.763+0000 7fb373400640 1 -- 172.21.15.86:0/3425075317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3600ab600 con 0x7fb37406c980 2024-03-22T04:15:29.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.763+0000 7fb373400640 1 -- 172.21.15.86:0/3425075317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600ab9d0 con 0x7fb37406c980 2024-03-22T04:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- 172.21.15.86:0/3425075317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37406c980 msgr2=0x7fb37406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 --2- 172.21.15.86:0/3425075317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37406c980 0x7fb37406d870 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb3600097c0 tx=0x7fb36001c460 comp rx=0 tx=0).stop 2024-03-22T04:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- 172.21.15.86:0/3425075317 shutdown_connections 2024-03-22T04:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 --2- 172.21.15.86:0/3425075317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37406c980 0x7fb37406d870 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- 172.21.15.86:0/3425075317 >> 172.21.15.86:0/3425075317 conn(0x7fb3740b3ba0 msgr2=0x7fb3740b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- 172.21.15.86:0/3425075317 shutdown_connections 2024-03-22T04:15:29.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- 172.21.15.86:0/3425075317 wait complete. 2024-03-22T04:15:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 Processor -- start 2024-03-22T04:15:29.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- start start 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37406c980 0x7fb374158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb37409aa70 0x7fb3741604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37415e670 0x7fb374152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb37416cc60 con 0x7fb37415e670 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb37409b330 con 0x7fb37406c980 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb37b800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb37416cde0 con 0x7fb37409aa70 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37415e670 0x7fb374152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37415e670 0x7fb374152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40826/0 (socket says 172.21.15.86:40826) 2024-03-22T04:15:29.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 -- 172.21.15.86:0/231838971 learned_addr learned my addr 172.21.15.86:0/231838971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb378e00640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37406c980 0x7fb374158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb373e00640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb37409aa70 0x7fb3741604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 -- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb37409aa70 msgr2=0x7fb3741604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb37409aa70 0x7fb3741604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 -- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37406c980 msgr2=0x7fb374158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37406c980 0x7fb374158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 -- 172.21.15.86:0/231838971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb374157cc0 con 0x7fb37415e670 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb373e00640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb37409aa70 0x7fb3741604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.767+0000 7fb379800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37415e670 0x7fb374152f70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00c7b0 tx=0x7fb36c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.771+0000 7fb370c00640 1 -- 172.21.15.86:0/231838971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c004b70 con 0x7fb37415e670 2024-03-22T04:15:29.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.771+0000 7fb370c00640 1 -- 172.21.15.86:0/231838971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c004d10 con 0x7fb37415e670 2024-03-22T04:15:29.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.771+0000 7fb370c00640 1 -- 172.21.15.86:0/231838971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00aa30 con 0x7fb37415e670 2024-03-22T04:15:29.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.771+0000 7fb37b800640 1 -- 172.21.15.86:0/231838971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb374157fa0 con 0x7fb37415e670 2024-03-22T04:15:29.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.771+0000 7fb37b800640 1 -- 172.21.15.86:0/231838971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb374158560 con 0x7fb37415e670 2024-03-22T04:15:29.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.771+0000 7fb370c00640 1 -- 172.21.15.86:0/231838971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb36c00abd0 con 0x7fb37415e670 2024-03-22T04:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.771+0000 7fb370c00640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb354067b70 0x7fb35406a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.771+0000 7fb370c00640 1 -- 172.21.15.86:0/231838971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7fb36c0862e0 con 0x7fb37415e670 2024-03-22T04:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb378e00640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb354067b70 0x7fb35406a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 Processor -- start 2024-03-22T04:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- start start 2024-03-22T04:15:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb378e00640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb354067b70 0x7fb35406a030 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb368005890 tx=0x7fb368009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef37fe70 0x5592ef380250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef336570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef336ab0 0x5592ef336e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5592ef184b20 con 0x5592ef380790 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592ef121f90 con 0x5592ef37fe70 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5592ef180ad0 con 0x5592ef336ab0 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef336570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef336570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40838/0 (socket says 172.21.15.86:40838) 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 -- 172.21.15.86:0/628384711 learned_addr learned my addr 172.21.15.86:0/628384711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb378e00640 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef37fe70 0x5592ef380250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb379800640 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef336ab0 0x5592ef336e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 -- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef336ab0 msgr2=0x5592ef336e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef336ab0 0x5592ef336e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 -- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef37fe70 msgr2=0x5592ef380250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef37fe70 0x5592ef380250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 -- 172.21.15.86:0/628384711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ef1c9e50 con 0x5592ef380790 2024-03-22T04:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb379800640 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef336ab0 0x5592ef336e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb373e00640 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef336570 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb3600ae040 tx=0x7fb3600a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7fd581b338be9b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb365600640 1 -- 172.21.15.86:0/628384711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600bc070 con 0x5592ef380790 2024-03-22T04:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb365600640 1 -- 172.21.15.86:0/628384711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3600b1040 con 0x5592ef380790 2024-03-22T04:15:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb365600640 1 -- 172.21.15.86:0/628384711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600b0050 con 0x5592ef380790 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- 172.21.15.86:0/628384711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 msgr2=0x5592ef336570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef336570 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb3600ae040 tx=0x7fb3600a6000 comp rx=0 tx=0).stop 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- 172.21.15.86:0/628384711 shutdown_connections 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef336ab0 0x5592ef336e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef336570 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/628384711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef37fe70 0x5592ef380250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- 172.21.15.86:0/628384711 >> 172.21.15.86:0/628384711 conn(0x5592ef332a60 msgr2=0x5592ef334e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- 172.21.15.86:0/628384711 shutdown_connections 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- 172.21.15.86:0/628384711 wait complete. 2024-03-22T04:15:29.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.775+0000 7fb37ecb6000 1 -- start start 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 Processor -- start 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef36afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef36b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 0x5592ef36ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5592ef189150 con 0x5592ef37fe70 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592ef1424f0 con 0x5592ef336ab0 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5592ef240360 con 0x5592ef380790 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 10 client.0 _ll_register_callbacks cb 0x5592ef321970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef36b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef36b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40844/0 (socket says 172.21.15.86:40844) 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 -- 172.21.15.86:0/942988700 learned_addr learned my addr 172.21.15.86:0/942988700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb379800640 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 0x5592ef36ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb378e00640 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef36afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 -- 172.21.15.86:0/942988700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 msgr2=0x5592ef36ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 0x5592ef36ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 -- 172.21.15.86:0/942988700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 msgr2=0x5592ef36afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef36afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 -- 172.21.15.86:0/942988700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ef1bfa60 con 0x5592ef37fe70 2024-03-22T04:15:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb373e00640 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef36b520 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb360008010 tx=0x7fb3600046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8364cf5a9173d52 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb35a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:15:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb35a000640 1 -- 172.21.15.86:0/942988700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600b0050 con 0x5592ef37fe70 2024-03-22T04:15:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 -- 172.21.15.86:0/942988700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3600b1040 con 0x5592ef37fe70 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb35a000640 1 -- 172.21.15.86:0/942988700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7fb3600ab800 con 0x5592ef37fe70 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb35a000640 1 -- 172.21.15.86:0/942988700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb3600cd050 con 0x5592ef37fe70 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 10 client.7110 fetch_fsmap learned FSMap version 52 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 -- 172.21.15.86:0/942988700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5592ef13f5a0 con 0x5592ef37fe70 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb35a000640 1 -- 172.21.15.86:0/942988700 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3600b1040 con 0x5592ef37fe70 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb35a000640 1 -- 172.21.15.86:0/942988700 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fb3600d0020 con 0x5592ef37fe70 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 10 client.7110 fetch_fsmap finished waiting for FSMap version 52 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 10 client.7110 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 20 client.7110 populate_metadata read hostname 'smithi086' 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x5592ef38ff10 0x5592ef392300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 4 client.7110 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb37ecb6000 1 -- 172.21.15.86:0/942988700 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592ef11fdf0 con 0x5592ef38ff10 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb378e00640 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x5592ef38ff10 0x5592ef392300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb378e00640 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x5592ef38ff10 0x5592ef392300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d75e78d39bfc90 server_cookie=bdf0a6a80cdb363b in_seq=0 out_seq=0 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.779+0000 7fb35a000640 10 client.7110 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb35a000640 1 -- 172.21.15.86:0/942988700 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5592ef11fdf0 con 0x5592ef38ff10 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb35a000640 10 client.7110 handle_command_reply: tid=0 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 client.7110 shutdown 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 -- 172.21.15.86:0/942988700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 msgr2=0x5592ef36b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef36b520 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb360008010 tx=0x7fb3600046d0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 -- 172.21.15.86:0/942988700 shutdown_connections 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x5592ef38ff10 0x5592ef392300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 0x5592ef36ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef36b520 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/942988700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef36afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 -- 172.21.15.86:0/942988700 >> 172.21.15.86:0/942988700 conn(0x5592ef332a60 msgr2=0x5592ef3356e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb35a000640 0 client.7110 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 -- 172.21.15.86:0/942988700 shutdown_connections 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 1 -- 172.21.15.86:0/942988700 wait complete. 2024-03-22T04:15:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.783+0000 7fb37ecb6000 20 client.7110 trim_cache size 0 max 16384 2024-03-22T04:15:29.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb37ecb6000 1 Processor -- start 2024-03-22T04:15:29.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb37ecb6000 1 -- start start 2024-03-22T04:15:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef335e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef37fe70 0x5592ef3363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef368d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb374204720 con 0x5592ef380790 2024-03-22T04:15:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb374203960 con 0x5592ef336ab0 2024-03-22T04:15:29.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb3740b6610 con 0x5592ef37fe70 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef368d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef368d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40858/0 (socket says 172.21.15.86:40858) 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 -- 172.21.15.86:0/2053302747 learned_addr learned my addr 172.21.15.86:0/2053302747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb373e00640 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef37fe70 0x5592ef3363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb378e00640 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef335e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 -- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef37fe70 msgr2=0x5592ef3363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef37fe70 0x5592ef3363a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 -- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 msgr2=0x5592ef335e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef335e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 -- 172.21.15.86:0/2053302747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ef205ed0 con 0x5592ef380790 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb373e00640 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef37fe70 0x5592ef3363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb378e00640 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef335e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.799+0000 7fb379800640 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef368d60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0846a0 tx=0x7fb36c01d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332fde284a85a903 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb364c00640 1 -- 172.21.15.86:0/2053302747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c05a5d0 con 0x5592ef380790 2024-03-22T04:15:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb364c00640 1 -- 172.21.15.86:0/2053302747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c05ee00 con 0x5592ef380790 2024-03-22T04:15:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb364c00640 1 -- 172.21.15.86:0/2053302747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c010d90 con 0x5592ef380790 2024-03-22T04:15:29.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- 172.21.15.86:0/2053302747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 msgr2=0x5592ef368d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef368d60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0846a0 tx=0x7fb36c01d040 comp rx=0 tx=0).stop 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- 172.21.15.86:0/2053302747 shutdown_connections 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef380790 0x5592ef368d60 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef37fe70 0x5592ef3363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/2053302747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef335e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- 172.21.15.86:0/2053302747 >> 172.21.15.86:0/2053302747 conn(0x5592ef332a60 msgr2=0x5592ef33b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- 172.21.15.86:0/2053302747 shutdown_connections 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- 172.21.15.86:0/2053302747 wait complete. 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- start start 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 Processor -- start 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef3f0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef3f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 0x5592ef392300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5592ef183e30 con 0x5592ef37fe70 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb3740b6c60 con 0x5592ef336ab0 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5592ef205a40 con 0x5592ef380790 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 10 client.0 _ll_register_callbacks cb 0x7fb3600ba070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef3f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef3f0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40864/0 (socket says 172.21.15.86:40864) 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 -- 172.21.15.86:0/332491836 learned_addr learned my addr 172.21.15.86:0/332491836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb379800640 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 0x5592ef392300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb378e00640 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef3f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 -- 172.21.15.86:0/332491836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 msgr2=0x5592ef392300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 0x5592ef392300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 -- 172.21.15.86:0/332491836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 msgr2=0x5592ef3f0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef3f0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 -- 172.21.15.86:0/332491836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ef1dcdc0 con 0x5592ef37fe70 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb373e00640 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef3f0d80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb36001a090 tx=0x7fb36001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7af0e45cceb0de7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb35a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb35a000640 1 -- 172.21.15.86:0/332491836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600b0020 con 0x5592ef37fe70 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb35a000640 1 -- 172.21.15.86:0/332491836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7fb360003830 con 0x5592ef37fe70 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- 172.21.15.86:0/332491836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36c05ee00 con 0x5592ef37fe70 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb35a000640 1 -- 172.21.15.86:0/332491836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb3600bc040 con 0x5592ef37fe70 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 10 client.7116 fetch_fsmap learned FSMap version 52 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.803+0000 7fb37ecb6000 1 -- 172.21.15.86:0/332491836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5592ef12b380 con 0x5592ef37fe70 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb35a000640 1 -- 172.21.15.86:0/332491836 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c05ee00 con 0x5592ef37fe70 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb35a000640 1 -- 172.21.15.86:0/332491836 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fb3600d0070 con 0x5592ef37fe70 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 10 client.7116 fetch_fsmap finished waiting for FSMap version 52 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 10 client.7116 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 20 client.7116 populate_metadata read hostname 'smithi086' 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x5592ef392840 0x5592ef392c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 4 client.7116 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-03-22T04:15:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 -- 172.21.15.86:0/332491836 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x5592ef192060 con 0x5592ef392840 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb378e00640 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x5592ef392840 0x5592ef392c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb378e00640 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x5592ef392840 0x5592ef392c50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb055d836e6ab340 server_cookie=15cdb9c613885d40 in_seq=0 out_seq=0 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb35a000640 10 client.7116 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb35a000640 1 -- 172.21.15.86:0/332491836 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5592ef192060 con 0x5592ef392840 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb35a000640 10 client.7116 handle_command_reply: tid=0 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 client.7116 shutdown 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 -- 172.21.15.86:0/332491836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 msgr2=0x5592ef3f0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef3f0d80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb36001a090 tx=0x7fb36001aca0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 -- 172.21.15.86:0/332491836 shutdown_connections 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x5592ef392840 0x5592ef392c50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5592ef380790 0x5592ef392300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592ef37fe70 0x5592ef3f0d80 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb35a000640 0 client.7116 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 --2- 172.21.15.86:0/332491836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592ef336ab0 0x5592ef3f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 -- 172.21.15.86:0/332491836 >> 172.21.15.86:0/332491836 conn(0x5592ef332a60 msgr2=0x5592ef37e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 -- 172.21.15.86:0/332491836 shutdown_connections 2024-03-22T04:15:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 1 -- 172.21.15.86:0/332491836 wait complete. 2024-03-22T04:15:29.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37ecb6000 20 client.7116 trim_cache size 0 max 16384 2024-03-22T04:15:29.815 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:15:29.815 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:15:29.816 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "44e4ffa5-5e8e-46a8-93c1-066a148f02fa", 2024-03-22T04:15:29.816 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:15:29.816 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:15:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 -- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb354067b70 msgr2=0x7fb35406a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb354067b70 0x7fb35406a030 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb368005890 tx=0x7fb368009040 comp rx=0 tx=0).stop 2024-03-22T04:15:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 -- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37415e670 msgr2=0x7fb374152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37415e670 0x7fb374152f70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00c7b0 tx=0x7fb36c00cc70 comp rx=0 tx=0).stop 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 -- 172.21.15.86:0/231838971 shutdown_connections 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb354067b70 0x7fb35406a030 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb37415e670 0x7fb374152f70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb37409aa70 0x7fb3741604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 --2- 172.21.15.86:0/231838971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37406c980 0x7fb374158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 -- 172.21.15.86:0/231838971 >> 172.21.15.86:0/231838971 conn(0x7fb3740b3ba0 msgr2=0x7fb3740b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 -- 172.21.15.86:0/231838971 shutdown_connections 2024-03-22T04:15:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.807+0000 7fb37b800640 1 -- 172.21.15.86:0/231838971 wait complete. 2024-03-22T04:15:29.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:15:29.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-22T04:15:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f085da00640 1 Processor -- start 2024-03-22T04:15:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f085da00640 1 -- start start 2024-03-22T04:15:29.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f085da00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f08580b11c0 0x7f085809b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f085da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08580cda00 con 0x7f085815c730 2024-03-22T04:15:29.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f085da00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08580cdce0 con 0x7f08580b3610 2024-03-22T04:15:29.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f085da00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f085809b910 con 0x7f08580b11c0 2024-03-22T04:15:29.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f0856a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f085815c730 0x7f085809aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59988/0 (socket says 172.21.15.86:59988) 2024-03-22T04:15:29.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f0856a00640 1 -- 172.21.15.86:0/421039641 learned_addr learned my addr 172.21.15.86:0/421039641 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:15:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f0857400640 1 --2- 172.21.15.86:0/421039641 >> v2:172.21.15.196:3300/0 conn(0x7f08580b11c0 0x7f085809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2673117960 0 0) 0x7f08580cda00 con 0x7f085815c730 2024-03-22T04:15:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.967+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f083c003530 con 0x7f085815c730 2024-03-22T04:15:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1743837649 0 0) 0x7f083c003530 con 0x7f085815c730 2024-03-22T04:15:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 >> v2:172.21.15.196:3300/0 conn(0x7f08580b11c0 msgr2=0x7f085809b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 --2- 172.21.15.86:0/421039641 >> v2:172.21.15.196:3300/0 conn(0x7f08580b11c0 0x7f085809b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 >> v1:172.21.15.196:6789/0 conn(0x7f08580b3610 legacy=0x7f08580b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08580cdfc0 con 0x7f085815c730 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f0848001680 con 0x7f085815c730 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 >> v1:172.21.15.86:6789/0 conn(0x7f085815c730 legacy=0x7f085809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f083c0039f0 0x7f083c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f083c004810 0x7f083c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c0051d0 0x7f083c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f085809b910 con 0x7f083c004810 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f083c00c0d0 con 0x7f083c0051d0 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f083c00c2d0 con 0x7f083c0039f0 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0857400640 1 --2- 172.21.15.86:0/421039641 >> v2:172.21.15.196:3300/0 conn(0x7f08580b11c0 0x7f085809b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0856000640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f083c004810 0x7f083c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0856a00640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f083c0039f0 0x7f083c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0857400640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c0051d0 0x7f083c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0856000640 1 -- 172.21.15.86:0/421039641 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f083c0039f0 msgr2=0x7f083c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0856000640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f083c0039f0 0x7f083c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0856000640 1 -- 172.21.15.86:0/421039641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c0051d0 msgr2=0x7f083c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0856000640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c0051d0 0x7f083c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0857400640 1 -- 172.21.15.86:0/421039641 reap_dead start 2024-03-22T04:15:29.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0856000640 1 -- 172.21.15.86:0/421039641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c003530 con 0x7f083c004810 2024-03-22T04:15:29.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.971+0000 7f0856000640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f083c004810 0x7f083c004c90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0844001860 tx=0x7f084401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa42b01277642fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440b1070 con 0x7f083c004810 2024-03-22T04:15:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08440ac640 con 0x7f083c004810 2024-03-22T04:15:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f0855600640 1 -- 172.21.15.86:0/421039641 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844001460 con 0x7f083c004810 2024-03-22T04:15:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- 172.21.15.86:0/421039641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f083c004810 msgr2=0x7f083c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f083c004810 0x7f083c004c90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0844001860 tx=0x7f084401c4e0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- 172.21.15.86:0/421039641 shutdown_connections 2024-03-22T04:15:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 --2- 172.21.15.86:0/421039641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f083c004810 0x7f083c004c90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- 172.21.15.86:0/421039641 >> 172.21.15.86:0/421039641 conn(0x7f08580b3bf0 msgr2=0x7f08580b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- 172.21.15.86:0/421039641 shutdown_connections 2024-03-22T04:15:29.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- 172.21.15.86:0/421039641 wait complete. 2024-03-22T04:15:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 Processor -- start 2024-03-22T04:15:29.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- start start 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c004810 0x7f085815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08580b6ae0 0x7f08581e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08581e9600 0x7f08580b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f085809b610 con 0x7f08580b6ae0 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f085809b490 con 0x7f083c004810 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f085da00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f08580d0000 con 0x7f08581e9600 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f0856000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08580b6ae0 0x7f08581e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f0856000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08580b6ae0 0x7f08581e90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40892/0 (socket says 172.21.15.86:40892) 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f0856000640 1 -- 172.21.15.86:0/4118530415 learned_addr learned my addr 172.21.15.86:0/4118530415 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f0856a00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c004810 0x7f085815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.975+0000 7f0857400640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08581e9600 0x7f08580b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f0856000640 1 -- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08581e9600 msgr2=0x7f08580b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f0856000640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08581e9600 0x7f08580b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f0856000640 1 -- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c004810 msgr2=0x7f085815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f0856000640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c004810 0x7f085815b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f0856000640 1 -- 172.21.15.86:0/4118530415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085815b930 con 0x7f08580b6ae0 2024-03-22T04:15:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f0857400640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08581e9600 0x7f08580b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:29.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f0856000640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08580b6ae0 0x7f08581e90c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f08440af040 tx=0x7f083c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f084ea00640 1 -- 172.21.15.86:0/4118530415 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440ba020 con 0x7f08580b6ae0 2024-03-22T04:15:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f084ea00640 1 -- 172.21.15.86:0/4118530415 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0844001440 con 0x7f08580b6ae0 2024-03-22T04:15:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f085da00640 1 -- 172.21.15.86:0/4118530415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085815e2a0 con 0x7f08580b6ae0 2024-03-22T04:15:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f084ea00640 1 -- 172.21.15.86:0/4118530415 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440b1040 con 0x7f08580b6ae0 2024-03-22T04:15:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f085da00640 1 -- 172.21.15.86:0/4118530415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085815d200 con 0x7f08580b6ae0 2024-03-22T04:15:29.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.979+0000 7f084ea00640 1 -- 172.21.15.86:0/4118530415 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f08440bd020 con 0x7f08580b6ae0 2024-03-22T04:15:29.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f084ea00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0838067b20 0x7f0838069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f084ea00640 1 -- 172.21.15.86:0/4118530415 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f084411fe90 con 0x7f08580b6ae0 2024-03-22T04:15:29.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0856a00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0838067b20 0x7f0838069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0860eb8000 1 Processor -- start 2024-03-22T04:15:29.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0860eb8000 1 -- start start 2024-03-22T04:15:29.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0856a00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0838067b20 0x7f0838069fe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0858068980 tx=0x7f0848000fb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b0f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85ac2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85ac3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0860eb8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f85913c40 con 0x559f85ac2f50 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0860eb8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f858b0d90 con 0x559f85b0f8a0 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0860eb8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559f8590fbf0 con 0x559f85b0ef80 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85ac3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85ac3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40900/0 (socket says 172.21.15.86:40900) 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 -- 172.21.15.86:0/127430466 learned_addr learned my addr 172.21.15.86:0/127430466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0856000640 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85ac2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0856a00640 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b0f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 -- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 msgr2=0x559f85b0f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b0f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 -- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 msgr2=0x559f85ac2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85ac2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 -- 172.21.15.86:0/127430466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f859bb130 con 0x559f85ac2f50 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0856000640 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85ac2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0856a00640 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b0f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0857400640 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85ac3330 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0858069080 tx=0x7f0850009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2650b11e5df6baa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0843400640 1 -- 172.21.15.86:0/127430466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085001b020 con 0x559f85ac2f50 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0843400640 1 -- 172.21.15.86:0/127430466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0850007e30 con 0x559f85ac2f50 2024-03-22T04:15:29.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.983+0000 7f0843400640 1 -- 172.21.15.86:0/127430466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500178c0 con 0x559f85ac2f50 2024-03-22T04:15:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- 172.21.15.86:0/127430466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 msgr2=0x559f85ac3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85ac3330 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0858069080 tx=0x7f0850009750 comp rx=0 tx=0).stop 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- 172.21.15.86:0/127430466 shutdown_connections 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85ac2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85ac3330 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 --2- 172.21.15.86:0/127430466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b0f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- 172.21.15.86:0/127430466 >> 172.21.15.86:0/127430466 conn(0x559f859ac990 msgr2=0x559f85af2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- 172.21.15.86:0/127430466 shutdown_connections 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- 172.21.15.86:0/127430466 wait complete. 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- start start 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 Processor -- start 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85ac2f50 0x559f85acaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85b0ef80 0x559f85af3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85af38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f858cf710 con 0x559f85b0ef80 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f858d1580 con 0x559f85b0f8a0 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559f85918270 con 0x559f85ac2f50 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 10 client.0 _ll_register_callbacks cb 0x559f85975200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85b0ef80 0x559f85af3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85b0ef80 0x559f85af3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40910/0 (socket says 172.21.15.86:40910) 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 -- 172.21.15.86:0/1210725311 learned_addr learned my addr 172.21.15.86:0/1210725311 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856a00640 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85ac2f50 0x559f85acaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0857400640 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85af38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 -- 172.21.15.86:0/1210725311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85ac2f50 msgr2=0x559f85acaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85ac2f50 0x559f85acaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 -- 172.21.15.86:0/1210725311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 msgr2=0x559f85af38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85af38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 -- 172.21.15.86:0/1210725311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f858ecd30 con 0x559f85b0ef80 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0856000640 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85b0ef80 0x559f85af3380 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f08440f84a0 tx=0x7f08440f0240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e90c39669f0fb32 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0837e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0837e00640 1 -- 172.21.15.86:0/1210725311 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440ee4c0 con 0x559f85b0ef80 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0837e00640 1 -- 172.21.15.86:0/1210725311 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f08440ef550 con 0x559f85b0ef80 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- 172.21.15.86:0/1210725311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0850007e30 con 0x559f85b0ef80 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0837e00640 1 -- 172.21.15.86:0/1210725311 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f084411c050 con 0x559f85b0ef80 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 10 client.7128 fetch_fsmap learned FSMap version 52 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- 172.21.15.86:0/1210725311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x559f85958fc0 con 0x559f85b0ef80 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0837e00640 1 -- 172.21.15.86:0/1210725311 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0850007e30 con 0x559f85b0ef80 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0837e00640 1 -- 172.21.15.86:0/1210725311 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f0844117bd0 con 0x559f85b0ef80 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 10 client.7128 fetch_fsmap finished waiting for FSMap version 52 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 10 client.7128 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 20 client.7128 populate_metadata read hostname 'smithi086' 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x559f85b155c0 0x559f85b159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 4 client.7128 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:15:29.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.987+0000 7f0860eb8000 1 -- 172.21.15.86:0/1210725311 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f858aedd0 con 0x559f85b155c0 2024-03-22T04:15:29.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0856a00640 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x559f85b155c0 0x559f85b159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0837e00640 10 client.7128 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0856a00640 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x559f85b155c0 0x559f85b159a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c921dc7a7a5368 server_cookie=4192ec262350fed8 in_seq=0 out_seq=0 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0837e00640 1 -- 172.21.15.86:0/1210725311 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559f858aedd0 con 0x559f85b155c0 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0837e00640 10 client.7128 handle_command_reply: tid=0 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 client.7128 shutdown 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 -- 172.21.15.86:0/1210725311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85b0ef80 msgr2=0x559f85af3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85b0ef80 0x559f85af3380 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f08440f84a0 tx=0x7f08440f0240 comp rx=0 tx=0).stop 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 -- 172.21.15.86:0/1210725311 shutdown_connections 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x559f85b155c0 0x559f85b159a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85b0f8a0 0x559f85af38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85b0ef80 0x559f85af3380 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1210725311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85ac2f50 0x559f85acaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0837e00640 0 client.7128 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 -- 172.21.15.86:0/1210725311 >> 172.21.15.86:0/1210725311 conn(0x559f859ac990 msgr2=0x559f85ac2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 -- 172.21.15.86:0/1210725311 shutdown_connections 2024-03-22T04:15:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 1 -- 172.21.15.86:0/1210725311 wait complete. 2024-03-22T04:15:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:29.991+0000 7f0860eb8000 20 client.7128 trim_cache size 0 max 16384 2024-03-22T04:15:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.007+0000 7f0860eb8000 1 Processor -- start 2024-03-22T04:15:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.007+0000 7f0860eb8000 1 -- start start 2024-03-22T04:15:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.007+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f859ac990 0x559f85b25680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.007+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85ac2f50 0x559f85b143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.007+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.007+0000 7f0860eb8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0858204280 con 0x559f859ac990 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.007+0000 7f0860eb8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0858203280 con 0x559f85ac2f50 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.007+0000 7f0860eb8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0858202730 con 0x559f85b0ef80 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f859ac990 0x559f85b25680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f859ac990 0x559f85b25680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40924/0 (socket says 172.21.15.86:40924) 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 -- 172.21.15.86:0/4144235840 learned_addr learned my addr 172.21.15.86:0/4144235840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856000640 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85ac2f50 0x559f85b143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0857400640 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 -- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 msgr2=0x559f85b148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b148f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 -- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85ac2f50 msgr2=0x559f85b143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85ac2f50 0x559f85b143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 -- 172.21.15.86:0/4144235840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f858e93a0 con 0x559f859ac990 2024-03-22T04:15:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0857400640 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:15:30.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f859ac990 0x559f85b25680 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x559f859d4780 tx=0x7f084800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e35b727986c93b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:30.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0842a00640 1 -- 172.21.15.86:0/4144235840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848027030 con 0x559f859ac990 2024-03-22T04:15:30.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0842a00640 1 -- 172.21.15.86:0/4144235840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084802d020 con 0x559f859ac990 2024-03-22T04:15:30.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0842a00640 1 -- 172.21.15.86:0/4144235840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848015440 con 0x559f859ac990 2024-03-22T04:15:30.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- 172.21.15.86:0/4144235840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f859ac990 msgr2=0x559f85b25680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:30.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f859ac990 0x559f85b25680 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x559f859d4780 tx=0x7f084800c000 comp rx=0 tx=0).stop 2024-03-22T04:15:30.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- 172.21.15.86:0/4144235840 shutdown_connections 2024-03-22T04:15:30.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f85ac2f50 0x559f85b143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 --2- 172.21.15.86:0/4144235840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f859ac990 0x559f85b25680 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- 172.21.15.86:0/4144235840 >> 172.21.15.86:0/4144235840 conn(0x559f85af2400 msgr2=0x559f85b13ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- 172.21.15.86:0/4144235840 shutdown_connections 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- 172.21.15.86:0/4144235840 wait complete. 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- start start 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 Processor -- start 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f859ac990 0x559f85b18a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85b18f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b19480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f859cf810 con 0x559f85ac2f50 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0858202c30 con 0x559f859ac990 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559f858ba350 con 0x559f85b0ef80 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0860eb8000 10 client.0 _ll_register_callbacks cb 0x7f08440efa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:15:30.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85b18f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85b18f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40934/0 (socket says 172.21.15.86:40934) 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856000640 1 -- 172.21.15.86:0/1865278978 learned_addr learned my addr 172.21.15.86:0/1865278978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.011+0000 7f0856a00640 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f859ac990 0x559f85b18a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0857400640 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b19480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856000640 1 -- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 msgr2=0x559f85b19480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856000640 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b19480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856000640 1 -- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f859ac990 msgr2=0x559f85b18a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856000640 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f859ac990 0x559f85b18a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856000640 1 -- 172.21.15.86:0/1865278978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f85899380 con 0x559f85ac2f50 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856a00640 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f859ac990 0x559f85b18a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856000640 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85b18f40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f08440effd0 tx=0x7f08440b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fe1ce1b83fcfd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 1 -- 172.21.15.86:0/1865278978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440ec3a0 con 0x559f85ac2f50 2024-03-22T04:15:30.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 1 -- 172.21.15.86:0/1865278978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f08440f63a0 con 0x559f85ac2f50 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 -- 172.21.15.86:0/1865278978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084802d020 con 0x559f85ac2f50 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 1 -- 172.21.15.86:0/1865278978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f08440ee4c0 con 0x559f85ac2f50 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 10 client.7134 fetch_fsmap learned FSMap version 52 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 -- 172.21.15.86:0/1865278978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x559f85995460 con 0x559f85ac2f50 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 1 -- 172.21.15.86:0/1865278978 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084802d020 con 0x559f85ac2f50 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 1 -- 172.21.15.86:0/1865278978 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f08440ebe50 con 0x559f85ac2f50 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 10 client.7134 fetch_fsmap finished waiting for FSMap version 52 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 10 client.7134 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 20 client.7134 populate_metadata read hostname 'smithi086' 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x559f85b0f8a0 0x559f85b199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 4 client.7134 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 -- 172.21.15.86:0/1865278978 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f858b3160 con 0x559f85b0f8a0 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856a00640 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x559f85b0f8a0 0x559f85b199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:15:30.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0856a00640 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x559f85b0f8a0 0x559f85b199c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c224e4cf009196 server_cookie=a040a05f64c43a9 in_seq=0 out_seq=0 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 10 client.7134 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 1 -- 172.21.15.86:0/1865278978 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559f858b3160 con 0x559f85b0f8a0 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 10 client.7134 handle_command_reply: tid=0 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 client.7134 shutdown 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 -- 172.21.15.86:0/1865278978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 msgr2=0x559f85b18f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85b18f40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f08440effd0 tx=0x7f08440b8040 comp rx=0 tx=0).stop 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 -- 172.21.15.86:0/1865278978 shutdown_connections 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x559f85b0f8a0 0x559f85b199c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f85b0ef80 0x559f85b19480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f85ac2f50 0x559f85b18f40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 --2- 172.21.15.86:0/1865278978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f859ac990 0x559f85b18a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0837e00640 0 client.7134 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 -- 172.21.15.86:0/1865278978 >> 172.21.15.86:0/1865278978 conn(0x559f85af2400 msgr2=0x559f85b14530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:30.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 -- 172.21.15.86:0/1865278978 shutdown_connections 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 1 -- 172.21.15.86:0/1865278978 wait complete. 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f0860eb8000 20 client.7134 trim_cache size 0 max 16384 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "44e4ffa5-5e8e-46a8-93c1-066a148f02fa": { 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/subdir", 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "44e4ffa5-5e8e-46a8-93c1-066a148f02fa", 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive" 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:30.025 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:15:30.026 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:15:30.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 -- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0838067b20 msgr2=0x7f0838069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:30.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0838067b20 0x7f0838069fe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0858068980 tx=0x7f0848000fb0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 -- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08580b6ae0 msgr2=0x7f08581e90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08580b6ae0 0x7f08581e90c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f08440af040 tx=0x7f083c005840 comp rx=0 tx=0).stop 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 -- 172.21.15.86:0/4118530415 shutdown_connections 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0838067b20 0x7f0838069fe0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08581e9600 0x7f08580b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08580b6ae0 0x7f08581e90c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 --2- 172.21.15.86:0/4118530415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f083c004810 0x7f085815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 -- 172.21.15.86:0/4118530415 >> 172.21.15.86:0/4118530415 conn(0x7f08580b3bf0 msgr2=0x7f0858160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 -- 172.21.15.86:0/4118530415 shutdown_connections 2024-03-22T04:15:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:15:30.015+0000 7f085da00640 1 -- 172.21.15.86:0/4118530415 wait complete. 2024-03-22T04:15:30.039 INFO:tasks.cephfs.filesystem:scrub status for tag:44e4ffa5-5e8e-46a8-93c1-066a148f02fa - {'path': '/subdir', 'tag': '44e4ffa5-5e8e-46a8-93c1-066a148f02fa', 'options': 'recursive'} 2024-03-22T04:15:31.547 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:15:31.543+0000 7f3dd0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-22T04:15:58.114 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:15:58.117 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:16:00.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:00.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-22T04:16:00.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad15200640 1 Processor -- start 2024-03-22T04:16:00.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad15200640 1 -- start start 2024-03-22T04:16:00.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad15200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad1015c700 0x7fad100d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad15200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 0x7fad1006fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad15200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad100d1f00 con 0x7fad1015fe00 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad15200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fad10070320 con 0x7fad1015c700 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad15200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fad1015cae0 con 0x7fad1006f980 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad0de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad1015c700 0x7fad100d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.174+0000 7fad0e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fad1015fe00 0x7fad10160290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55572/0 (socket says 172.21.15.86:55572) 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0e800640 1 -- 172.21.15.86:0/2960007290 learned_addr learned my addr 172.21.15.86:0/2960007290 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0de00640 1 -- 172.21.15.86:0/2960007290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 msgr2=0x7fad1006fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0de00640 1 --2- 172.21.15.86:0/2960007290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 0x7fad1006fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0de00640 1 -- 172.21.15.86:0/2960007290 >> v1:172.21.15.86:6789/0 conn(0x7fad1015fe00 legacy=0x7fad10160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0de00640 1 -- 172.21.15.86:0/2960007290 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad10160b70 con 0x7fad1015c700 2024-03-22T04:16:00.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0de00640 1 --2- 172.21.15.86:0/2960007290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad100d19c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fad000097c0 tx=0x7fad0001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af0ef12b3fc4231 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0ca00640 1 -- 172.21.15.86:0/2960007290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000b0070 con 0x7fad1015c700 2024-03-22T04:16:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0ca00640 1 -- 172.21.15.86:0/2960007290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad000ab600 con 0x7fad1015c700 2024-03-22T04:16:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.178+0000 7fad0ca00640 1 -- 172.21.15.86:0/2960007290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000ab9d0 con 0x7fad1015c700 2024-03-22T04:16:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- 172.21.15.86:0/2960007290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 msgr2=0x7fad100d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 --2- 172.21.15.86:0/2960007290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad100d19c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fad000097c0 tx=0x7fad0001c410 comp rx=0 tx=0).stop 2024-03-22T04:16:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- 172.21.15.86:0/2960007290 shutdown_connections 2024-03-22T04:16:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 --2- 172.21.15.86:0/2960007290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 0x7fad1006fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 --2- 172.21.15.86:0/2960007290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad100d19c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- 172.21.15.86:0/2960007290 >> 172.21.15.86:0/2960007290 conn(0x7fad100b3cf0 msgr2=0x7fad100b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- 172.21.15.86:0/2960007290 shutdown_connections 2024-03-22T04:16:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- 172.21.15.86:0/2960007290 wait complete. 2024-03-22T04:16:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 Processor -- start 2024-03-22T04:16:00.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- start start 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 0x7fad10152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad1006db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1015fe00 0x7fad1014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad100b5c10 con 0x7fad1015c700 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad100d0b90 con 0x7fad1015fe00 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad15200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fad100b5d90 con 0x7fad1006f980 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad1006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad1006db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56082/0 (socket says 172.21.15.86:56082) 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0d400640 1 -- 172.21.15.86:0/3181760400 learned_addr learned my addr 172.21.15.86:0/3181760400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0de00640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 0x7fad10152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0d400640 1 -- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 msgr2=0x7fad10152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0e800640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1015fe00 0x7fad1014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0d400640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 0x7fad10152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0d400640 1 -- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1015fe00 msgr2=0x7fad1014f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0d400640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1015fe00 0x7fad1014f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0d400640 1 -- 172.21.15.86:0/3181760400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1006cae0 con 0x7fad1015c700 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0de00640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 0x7fad10152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.182+0000 7fad0e800640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1015fe00 0x7fad1014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad0d400640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad1006db80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7facfc00b410 tx=0x7facfc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad06000640 1 -- 172.21.15.86:0/3181760400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc015020 con 0x7fad1015c700 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad06000640 1 -- 172.21.15.86:0/3181760400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc011830 con 0x7fad1015c700 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad06000640 1 -- 172.21.15.86:0/3181760400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc011c20 con 0x7fad1015c700 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad15200640 1 -- 172.21.15.86:0/3181760400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1006cdc0 con 0x7fad1015c700 2024-03-22T04:16:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad15200640 1 -- 172.21.15.86:0/3181760400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1006d380 con 0x7fad1015c700 2024-03-22T04:16:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad06000640 1 -- 172.21.15.86:0/3181760400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7facfc018020 con 0x7fad1015c700 2024-03-22T04:16:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad06000640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facf0067b20 0x7facf0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad06000640 1 -- 172.21.15.86:0/3181760400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7facfc0860b0 con 0x7fad1015c700 2024-03-22T04:16:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.186+0000 7fad0de00640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facf0067b20 0x7facf0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 Processor -- start 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- start start 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facf0067b20 0x7facf0069fe0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fad000ad000 tx=0x7fad00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede84360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 0x55cdede38330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdedc88c40 con 0x55cdede83f80 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cdedc25d90 con 0x55cdede37f50 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cdedc84bf0 con 0x55cdede848a0 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede84360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede84360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56084/0 (socket says 172.21.15.86:56084) 2024-03-22T04:16:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 -- 172.21.15.86:0/218983265 learned_addr learned my addr 172.21.15.86:0/218983265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 -- 172.21.15.86:0/218983265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 msgr2=0x55cdede37a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 --2- 172.21.15.86:0/218983265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 -- 172.21.15.86:0/218983265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 msgr2=0x55cdede38330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 --2- 172.21.15.86:0/218983265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 0x55cdede38330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 -- 172.21.15.86:0/218983265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdedd30130 con 0x55cdede83f80 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad0de00640 1 --2- 172.21.15.86:0/218983265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede84360 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fad00008010 tx=0x7fad000a4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dad63350de4d720 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7facfaa00640 1 -- 172.21.15.86:0/218983265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000c7070 con 0x55cdede83f80 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7facfaa00640 1 -- 172.21.15.86:0/218983265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad000b6670 con 0x55cdede83f80 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7facfaa00640 1 -- 172.21.15.86:0/218983265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000b0040 con 0x55cdede83f80 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- 172.21.15.86:0/218983265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 msgr2=0x55cdede84360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 --2- 172.21.15.86:0/218983265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede84360 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fad00008010 tx=0x7fad000a4f90 comp rx=0 tx=0).stop 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- 172.21.15.86:0/218983265 shutdown_connections 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 --2- 172.21.15.86:0/218983265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 0x55cdede38330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 --2- 172.21.15.86:0/218983265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 --2- 172.21.15.86:0/218983265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede84360 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- 172.21.15.86:0/218983265 >> 172.21.15.86:0/218983265 conn(0x55cdedd21990 msgr2=0x55cdede67150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- 172.21.15.86:0/218983265 shutdown_connections 2024-03-22T04:16:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- 172.21.15.86:0/218983265 wait complete. 2024-03-22T04:16:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.190+0000 7fad1873f000 1 -- start start 2024-03-22T04:16:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 Processor -- start 2024-03-22T04:16:00.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 0x55cdede3fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede68380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede688c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdedc44710 con 0x55cdede83f80 2024-03-22T04:16:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cdedc46580 con 0x55cdede37f50 2024-03-22T04:16:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cdedc8d270 con 0x55cdede848a0 2024-03-22T04:16:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 10 client.0 _ll_register_callbacks cb 0x55cdedcea200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 0x55cdede3fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede68380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 0x55cdede3fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:36130/0 (socket says 172.21.15.86:36130) 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0de00640 1 -- 172.21.15.86:0/2135532263 learned_addr learned my addr 172.21.15.86:0/2135532263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0e800640 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0d400640 1 -- 172.21.15.86:0/2135532263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 msgr2=0x55cdede688c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0d400640 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede688c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0d400640 1 -- 172.21.15.86:0/2135532263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 msgr2=0x55cdede3fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0d400640 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 0x55cdede3fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0d400640 1 -- 172.21.15.86:0/2135532263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdedc61d30 con 0x55cdede83f80 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad0d400640 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede68380 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7facfc00bdd0 tx=0x7facfc056980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03fc02d91c797a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7facef400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7facef400640 1 -- 172.21.15.86:0/2135532263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0527b0 con 0x55cdede83f80 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7facef400640 1 -- 172.21.15.86:0/2135532263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7facfc06fd60 con 0x55cdede83f80 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 -- 172.21.15.86:0/2135532263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad000b6670 con 0x55cdede83f80 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7facef400640 1 -- 172.21.15.86:0/2135532263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7facfc052950 con 0x55cdede83f80 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 10 client.7149 fetch_fsmap learned FSMap version 53 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 -- 172.21.15.86:0/2135532263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55cdedccdfc0 con 0x55cdede83f80 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7facef400640 1 -- 172.21.15.86:0/2135532263 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad000b6670 con 0x55cdede83f80 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7facef400640 1 -- 172.21.15.86:0/2135532263 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7facfc06e120 con 0x55cdede83f80 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 10 client.7149 fetch_fsmap finished waiting for FSMap version 53 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 10 client.7149 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 20 client.7149 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55cdede8a630 0x55cdede8aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 4 client.7149 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:16:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.194+0000 7fad1873f000 1 -- 172.21.15.86:0/2135532263 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdedc23dd0 con 0x55cdede8a630 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad0de00640 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55cdede8a630 0x55cdede8aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7facef400640 10 client.7149 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad0de00640 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55cdede8a630 0x55cdede8aa10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c7dc5fe78d4851 server_cookie=965fc44f7925d345 in_seq=0 out_seq=0 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7facef400640 1 -- 172.21.15.86:0/2135532263 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cdedc23dd0 con 0x55cdede8a630 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7facef400640 10 client.7149 handle_command_reply: tid=0 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 client.7149 shutdown 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 -- 172.21.15.86:0/2135532263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 msgr2=0x55cdede68380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede68380 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7facfc00bdd0 tx=0x7facfc056980 comp rx=0 tx=0).stop 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 -- 172.21.15.86:0/2135532263 shutdown_connections 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55cdede8a630 0x55cdede8aa10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede68380 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7facef400640 0 client.7149 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 --2- 172.21.15.86:0/2135532263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede37f50 0x55cdede3fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 -- 172.21.15.86:0/2135532263 >> 172.21.15.86:0/2135532263 conn(0x55cdedd21990 msgr2=0x55cdede37110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 -- 172.21.15.86:0/2135532263 shutdown_connections 2024-03-22T04:16:00.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 1 -- 172.21.15.86:0/2135532263 wait complete. 2024-03-22T04:16:00.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.198+0000 7fad1873f000 20 client.7149 trim_cache size 0 max 16384 2024-03-22T04:16:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.214+0000 7fad1873f000 1 Processor -- start 2024-03-22T04:16:00.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.214+0000 7fad1873f000 1 -- start start 2024-03-22T04:16:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.214+0000 7fad1873f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede37f50 0x55cdede899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.214+0000 7fad1873f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.214+0000 7fad1873f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede848a0 0x55cdede88ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.214+0000 7fad1873f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad10203ff0 con 0x55cdede83f80 2024-03-22T04:16:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.214+0000 7fad1873f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad10203070 con 0x55cdede848a0 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.214+0000 7fad1873f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fad10202400 con 0x55cdede37f50 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede90a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56098/0 (socket says 172.21.15.86:56098) 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 -- 172.21.15.86:0/184576683 learned_addr learned my addr 172.21.15.86:0/184576683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0de00640 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede37f50 0x55cdede899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0e800640 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede848a0 0x55cdede88ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 -- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede37f50 msgr2=0x55cdede899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede37f50 0x55cdede899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 -- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede848a0 msgr2=0x55cdede88ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede848a0 0x55cdede88ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 -- 172.21.15.86:0/184576683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdedc5e3a0 con 0x55cdede83f80 2024-03-22T04:16:00.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0e800640 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede848a0 0x55cdede88ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:00.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede90a60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7facfc00b3e0 tx=0x7facfc059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f0238e319c9728 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7facfa000640 1 -- 172.21.15.86:0/184576683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc06e120 con 0x55cdede83f80 2024-03-22T04:16:00.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7facfa000640 1 -- 172.21.15.86:0/184576683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc06e2c0 con 0x55cdede83f80 2024-03-22T04:16:00.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7facfa000640 1 -- 172.21.15.86:0/184576683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0558c0 con 0x55cdede83f80 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- 172.21.15.86:0/184576683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 msgr2=0x55cdede90a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede90a60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7facfc00b3e0 tx=0x7facfc059000 comp rx=0 tx=0).stop 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- 172.21.15.86:0/184576683 shutdown_connections 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede848a0 0x55cdede88ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede83f80 0x55cdede90a60 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 --2- 172.21.15.86:0/184576683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede37f50 0x55cdede899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- 172.21.15.86:0/184576683 >> 172.21.15.86:0/184576683 conn(0x55cdedd21990 msgr2=0x55cdede400f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- 172.21.15.86:0/184576683 shutdown_connections 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- 172.21.15.86:0/184576683 wait complete. 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- start start 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 Processor -- start 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede37f50 0x55cdedef3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede83f80 0x55cdedef3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede8ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdedd44810 con 0x55cdede37f50 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad10202a50 con 0x55cdede83f80 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cdedc2f350 con 0x55cdede848a0 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad1873f000 10 client.0 _ll_register_callbacks cb 0x7facfc052460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede37f50 0x55cdedef3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede37f50 0x55cdedef3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56114/0 (socket says 172.21.15.86:56114) 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0de00640 1 -- 172.21.15.86:0/3208928079 learned_addr learned my addr 172.21.15.86:0/3208928079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0e800640 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede8ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.218+0000 7fad0d400640 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede83f80 0x55cdedef3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad0de00640 1 -- 172.21.15.86:0/3208928079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 msgr2=0x55cdede8ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad0de00640 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede8ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad0de00640 1 -- 172.21.15.86:0/3208928079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede83f80 msgr2=0x55cdedef3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad0de00640 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede83f80 0x55cdedef3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad0de00640 1 -- 172.21.15.86:0/3208928079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdedc0e380 con 0x55cdede37f50 2024-03-22T04:16:00.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad0de00640 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede37f50 0x55cdedef3530 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fad0001aac0 tx=0x7fad0001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2126cae16f086d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:00.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 1 -- 172.21.15.86:0/3208928079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000d9060 con 0x55cdede37f50 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 1 -- 172.21.15.86:0/3208928079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7fad00004240 con 0x55cdede37f50 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 -- 172.21.15.86:0/3208928079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc06e2c0 con 0x55cdede37f50 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 1 -- 172.21.15.86:0/3208928079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fad000df050 con 0x55cdede37f50 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 10 client.7155 fetch_fsmap learned FSMap version 53 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 -- 172.21.15.86:0/3208928079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55cdedd0a460 con 0x55cdede37f50 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 1 -- 172.21.15.86:0/3208928079 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc06e2c0 con 0x55cdede37f50 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 1 -- 172.21.15.86:0/3208928079 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7fad000e2020 con 0x55cdede37f50 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 10 client.7155 fetch_fsmap finished waiting for FSMap version 53 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 10 client.7155 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 20 client.7155 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55cdede8d7a0 0x55cdede8dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 4 client.7155 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 -- 172.21.15.86:0/3208928079 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cdedc28160 con 0x55cdede8d7a0 2024-03-22T04:16:00.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad0d400640 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55cdede8d7a0 0x55cdede8dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 10 client.7155 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad0d400640 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55cdede8d7a0 0x55cdede8dbb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2abe6c2bcc6801d server_cookie=b809c98eb4fb3a4e in_seq=0 out_seq=0 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 1 -- 172.21.15.86:0/3208928079 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cdedc28160 con 0x55cdede8d7a0 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 10 client.7155 handle_command_reply: tid=0 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 client.7155 shutdown 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 -- 172.21.15.86:0/3208928079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede37f50 msgr2=0x55cdedef3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede37f50 0x55cdedef3530 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fad0001aac0 tx=0x7fad0001ac50 comp rx=0 tx=0).stop 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 -- 172.21.15.86:0/3208928079 shutdown_connections 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55cdede8d7a0 0x55cdede8dbb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cdede83f80 0x55cdedef3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cdede848a0 0x55cdede8ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7facef400640 0 client.7155 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 --2- 172.21.15.86:0/3208928079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdede37f50 0x55cdedef3530 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 -- 172.21.15.86:0/3208928079 >> 172.21.15.86:0/3208928079 conn(0x55cdedd21990 msgr2=0x55cdede3e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 -- 172.21.15.86:0/3208928079 shutdown_connections 2024-03-22T04:16:00.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 1 -- 172.21.15.86:0/3208928079 wait complete. 2024-03-22T04:16:00.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad1873f000 20 client.7155 trim_cache size 0 max 16384 2024-03-22T04:16:00.236 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:16:00.236 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:16:00.236 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:16:00.236 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 -- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facf0067b20 msgr2=0x7facf0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facf0067b20 0x7facf0069fe0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fad000ad000 tx=0x7fad00008040 comp rx=0 tx=0).stop 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 -- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 msgr2=0x7fad1006db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad1006db80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7facfc00b410 tx=0x7facfc00b8d0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 -- 172.21.15.86:0/3181760400 shutdown_connections 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7facf0067b20 0x7facf0069fe0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1015fe00 0x7fad1014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1015c700 0x7fad1006db80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 --2- 172.21.15.86:0/3181760400 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad1006f980 0x7fad10152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 -- 172.21.15.86:0/3181760400 >> 172.21.15.86:0/3181760400 conn(0x7fad100b3cf0 msgr2=0x7fad10153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 -- 172.21.15.86:0/3181760400 shutdown_connections 2024-03-22T04:16:00.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.222+0000 7fad15200640 1 -- 172.21.15.86:0/3181760400 wait complete. 2024-03-22T04:16:00.250 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:16:00.250 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:16:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbebc400640 1 Processor -- start 2024-03-22T04:16:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbebc400640 1 -- start start 2024-03-22T04:16:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbebc400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbeb40d0de0 0x7fbeb40ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbebc400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb415fed0 0x7fbeb4160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbebc400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb40d11c0 con 0x7fbeb40d19a0 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbebc400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbeb40d0580 con 0x7fbeb40d0de0 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbebc400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb4160860 con 0x7fbeb415fed0 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbeb40d0de0 0x7fbeb40ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbeb40d0de0 0x7fbeb40ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56126/0 (socket says 172.21.15.86:56126) 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb9a00640 1 -- 172.21.15.86:0/3417638757 learned_addr learned my addr 172.21.15.86:0/3417638757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb9000640 1 --1- 172.21.15.86:0/3417638757 >> v1:172.21.15.86:6789/0 conn(0x7fbeb40d19a0 0x7fbeb40cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55574/0 (socket says 172.21.15.86:55574) 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeba400640 1 --2- 172.21.15.86:0/3417638757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb415fed0 0x7fbeb4160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb9a00640 1 -- 172.21.15.86:0/3417638757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb415fed0 msgr2=0x7fbeb4160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb9a00640 1 --2- 172.21.15.86:0/3417638757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb415fed0 0x7fbeb4160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb9a00640 1 -- 172.21.15.86:0/3417638757 >> v1:172.21.15.86:6789/0 conn(0x7fbeb40d19a0 legacy=0x7fbeb40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb9a00640 1 -- 172.21.15.86:0/3417638757 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb40cece0 con 0x7fbeb40d0de0 2024-03-22T04:16:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.382+0000 7fbeb3e00640 1 -- 172.21.15.86:0/3417638757 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143029242 0 0) 0x7fbeb40d11c0 con 0x7fbeb40d19a0 2024-03-22T04:16:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbeb9a00640 1 --2- 172.21.15.86:0/3417638757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb40d0de0 0x7fbeb40ce6b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009870 tx=0x7fbea801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e2c4def44e5fe8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbeb3e00640 1 -- 172.21.15.86:0/3417638757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea80ab650 con 0x7fbeb40d0de0 2024-03-22T04:16:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbeb3e00640 1 -- 172.21.15.86:0/3417638757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea80b0070 con 0x7fbeb40d0de0 2024-03-22T04:16:00.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbeb3e00640 1 -- 172.21.15.86:0/3417638757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea80aba10 con 0x7fbeb40d0de0 2024-03-22T04:16:00.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbebc400640 1 -- 172.21.15.86:0/3417638757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb40d0de0 msgr2=0x7fbeb40ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbebc400640 1 --2- 172.21.15.86:0/3417638757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb40d0de0 0x7fbeb40ce6b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009870 tx=0x7fbea801c560 comp rx=0 tx=0).stop 2024-03-22T04:16:00.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbebc400640 1 -- 172.21.15.86:0/3417638757 shutdown_connections 2024-03-22T04:16:00.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbebc400640 1 --2- 172.21.15.86:0/3417638757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb415fed0 0x7fbeb4160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbebc400640 1 --2- 172.21.15.86:0/3417638757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb40d0de0 0x7fbeb40ce6b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbebc400640 1 -- 172.21.15.86:0/3417638757 >> 172.21.15.86:0/3417638757 conn(0x7fbeb406f910 msgr2=0x7fbeb406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbebc400640 1 -- 172.21.15.86:0/3417638757 shutdown_connections 2024-03-22T04:16:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.386+0000 7fbebc400640 1 -- 172.21.15.86:0/3417638757 wait complete. 2024-03-22T04:16:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 Processor -- start 2024-03-22T04:16:00.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 -- start start 2024-03-22T04:16:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb40d0de0 0x7fbeb40c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb40d19a0 0x7fbeb41653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb415fed0 0x7fbeb4165930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb40d1db0 con 0x7fbeb415fed0 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4071100 con 0x7fbeb40d19a0 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb40d1f30 con 0x7fbeb40d0de0 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb415fed0 0x7fbeb4165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeb9a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb40d0de0 0x7fbeb40c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb415fed0 0x7fbeb4165930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56136/0 (socket says 172.21.15.86:56136) 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 -- 172.21.15.86:0/1528960121 learned_addr learned my addr 172.21.15.86:0/1528960121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeb9000640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb40d19a0 0x7fbeb41653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 -- 172.21.15.86:0/1528960121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb40d0de0 msgr2=0x7fbeb40c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb40d0de0 0x7fbeb40c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 -- 172.21.15.86:0/1528960121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb40d19a0 msgr2=0x7fbeb41653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb40d19a0 0x7fbeb41653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 -- 172.21.15.86:0/1528960121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb41582f0 con 0x7fbeb415fed0 2024-03-22T04:16:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeba400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb415fed0 0x7fbeb4165930 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbeac007e40 tx=0x7fbeac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeb1600640 1 -- 172.21.15.86:0/1528960121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac014620 con 0x7fbeb415fed0 2024-03-22T04:16:00.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeb1600640 1 -- 172.21.15.86:0/1528960121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeac019070 con 0x7fbeb415fed0 2024-03-22T04:16:00.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb41585d0 con 0x7fbeb415fed0 2024-03-22T04:16:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbeb1600640 1 -- 172.21.15.86:0/1528960121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac014a20 con 0x7fbeb415fed0 2024-03-22T04:16:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb4158b90 con 0x7fbeb415fed0 2024-03-22T04:16:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.390+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb4068650 con 0x7fbeb415fed0 2024-03-22T04:16:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.394+0000 7fbeb1600640 1 -- 172.21.15.86:0/1528960121 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbeac014bc0 con 0x7fbeb415fed0 2024-03-22T04:16:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.394+0000 7fbeb1600640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe94067b20 0x7fbe94069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.394+0000 7fbeb1600640 1 -- 172.21.15.86:0/1528960121 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7fbeac086950 con 0x7fbeb415fed0 2024-03-22T04:16:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.394+0000 7fbeb9a00640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe94067b20 0x7fbe94069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.394+0000 7fbeb9a00640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe94067b20 0x7fbe94069fe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbea80af000 tx=0x7fbea80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.398+0000 7fbeb1600640 1 -- 172.21.15.86:0/1528960121 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbeac056a90 con 0x7fbeb415fed0 2024-03-22T04:16:00.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.598+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeb4070560 con 0x7fbeb415fed0 2024-03-22T04:16:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.598+0000 7fbeb1600640 1 -- 172.21.15.86:0/1528960121 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5187 (secure 0 0 0) 0x7fbeac05ab30 con 0x7fbeb415fed0 2024-03-22T04:16:00.610 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:00.610 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:15:31.417832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":6,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7002,"qdb_cluster":[7002]},"id":5}]} 2024-03-22T04:16:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 53 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe94067b20 msgr2=0x7fbe94069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe94067b20 0x7fbe94069fe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbea80af000 tx=0x7fbea80a7000 comp rx=0 tx=0).stop 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb415fed0 msgr2=0x7fbeb4165930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb415fed0 0x7fbeb4165930 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbeac007e40 tx=0x7fbeac00e4b0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 shutdown_connections 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe94067b20 0x7fbe94069fe0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeb40d19a0 0x7fbeb41653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb415fed0 0x7fbeb4165930 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 --2- 172.21.15.86:0/1528960121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeb40d0de0 0x7fbeb40c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 >> 172.21.15.86:0/1528960121 conn(0x7fbeb406f910 msgr2=0x7fbeb415ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 shutdown_connections 2024-03-22T04:16:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.602+0000 7fbebc400640 1 -- 172.21.15.86:0/1528960121 wait complete. 2024-03-22T04:16:00.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-03-22T04:16:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f21600640 1 Processor -- start 2024-03-22T04:16:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f21600640 1 -- start start 2024-03-22T04:16:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f21600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c15e970 0x7f8f1c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f21600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f1c06dd70 con 0x7f8f1c06d8f0 2024-03-22T04:16:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f21600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f1c06dff0 con 0x7f8f1c09b180 2024-03-22T04:16:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f21600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c0b49a0 con 0x7f8f1c15e970 2024-03-22T04:16:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f13e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8f1c06d8f0 0x7f8f1c09aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55580/0 (socket says 172.21.15.86:55580) 2024-03-22T04:16:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f13e00640 1 -- 172.21.15.86:0/1862515563 learned_addr learned my addr 172.21.15.86:0/1862515563 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/1862515563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c15e970 0x7f8f1c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1a000640 1 -- 172.21.15.86:0/1862515563 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1470823876 0 0) 0x7f8f04001120 con 0x7f8f1c06d8f0 2024-03-22T04:16:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1a000640 1 -- 172.21.15.86:0/1862515563 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f00003530 con 0x7f8f1c06d8f0 2024-03-22T04:16:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1a000640 1 -- 172.21.15.86:0/1862515563 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911604826 0 0) 0x7f8f1c06dff0 con 0x7f8f1c09b180 2024-03-22T04:16:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1a000640 1 -- 172.21.15.86:0/1862515563 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f04001120 con 0x7f8f1c09b180 2024-03-22T04:16:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1aa00640 1 -- 172.21.15.86:0/1862515563 >> v1:172.21.15.196:6789/0 conn(0x7f8f1c09b180 legacy=0x7f8f1c0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1aa00640 1 -- 172.21.15.86:0/1862515563 >> v1:172.21.15.86:6789/0 conn(0x7f8f1c06d8f0 legacy=0x7f8f1c09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1a000640 1 -- 172.21.15.86:0/1862515563 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 684026125 0 0) 0x7f8f00003530 con 0x7f8f1c06d8f0 2024-03-22T04:16:00.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.766+0000 7f8f1aa00640 1 -- 172.21.15.86:0/1862515563 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c06e2d0 con 0x7f8f1c15e970 2024-03-22T04:16:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/1862515563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c15e970 0x7f8f1c0b12e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0097f0 tx=0x7f8f0c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc7ecb449ee2d3f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f1a000640 1 -- 172.21.15.86:0/1862515563 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0b0070 con 0x7f8f1c15e970 2024-03-22T04:16:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f1a000640 1 -- 172.21.15.86:0/1862515563 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0ab400 con 0x7f8f1c15e970 2024-03-22T04:16:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f1a000640 1 -- 172.21.15.86:0/1862515563 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04001850 con 0x7f8f1c15e970 2024-03-22T04:16:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f21600640 1 -- 172.21.15.86:0/1862515563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c15e970 msgr2=0x7f8f1c0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f21600640 1 --2- 172.21.15.86:0/1862515563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c15e970 0x7f8f1c0b12e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0097f0 tx=0x7f8f0c01c400 comp rx=0 tx=0).stop 2024-03-22T04:16:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f21600640 1 -- 172.21.15.86:0/1862515563 shutdown_connections 2024-03-22T04:16:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f21600640 1 --2- 172.21.15.86:0/1862515563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c15e970 0x7f8f1c0b12e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f21600640 1 -- 172.21.15.86:0/1862515563 >> 172.21.15.86:0/1862515563 conn(0x7f8f1c0b3bf0 msgr2=0x7f8f1c0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f21600640 1 -- 172.21.15.86:0/1862515563 shutdown_connections 2024-03-22T04:16:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.770+0000 7f8f21600640 1 -- 172.21.15.86:0/1862515563 wait complete. 2024-03-22T04:16:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 Processor -- start 2024-03-22T04:16:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 -- start start 2024-03-22T04:16:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c06d8f0 0x7f8f1c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c09b180 0x7f8f1c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c15e970 0x7f8f1c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c0ce2d0 con 0x7f8f1c09b180 2024-03-22T04:16:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c0ce150 con 0x7f8f1c15e970 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c0ce450 con 0x7f8f1c06d8f0 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c09b180 0x7f8f1c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c09b180 0x7f8f1c157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56146/0 (socket says 172.21.15.86:56146) 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 -- 172.21.15.86:0/1636855337 learned_addr learned my addr 172.21.15.86:0/1636855337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c06d8f0 0x7f8f1c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f1b400640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c15e970 0x7f8f1c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 -- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c06d8f0 msgr2=0x7f8f1c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c06d8f0 0x7f8f1c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 -- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c15e970 msgr2=0x7f8f1c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c15e970 0x7f8f1c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 -- 172.21.15.86:0/1636855337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c06d5c0 con 0x7f8f1c09b180 2024-03-22T04:16:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f1b400640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c15e970 0x7f8f1c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f13e00640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c09b180 0x7f8f1c157890 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400c780 tx=0x7f8f0400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f12a00640 1 -- 172.21.15.86:0/1636855337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04017070 con 0x7f8f1c09b180 2024-03-22T04:16:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f12a00640 1 -- 172.21.15.86:0/1636855337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f04005d10 con 0x7f8f1c09b180 2024-03-22T04:16:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f12a00640 1 -- 172.21.15.86:0/1636855337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f040072b0 con 0x7f8f1c09b180 2024-03-22T04:16:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 -- 172.21.15.86:0/1636855337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c0b52e0 con 0x7f8f1c09b180 2024-03-22T04:16:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.774+0000 7f8f21600640 1 -- 172.21.15.86:0/1636855337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c0b58a0 con 0x7f8f1c09b180 2024-03-22T04:16:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f12a00640 1 -- 172.21.15.86:0/1636855337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8f0401a070 con 0x7f8f1c09b180 2024-03-22T04:16:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f24a56000 1 Processor -- start 2024-03-22T04:16:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f24a56000 1 -- start start 2024-03-22T04:16:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f12a00640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ef8067b20 0x7f8ef8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f12a00640 1 -- 172.21.15.86:0/1636855337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f8f04085750 con 0x7f8f1c09b180 2024-03-22T04:16:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f1b400640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ef8067b20 0x7f8ef8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 0x55f5c8ecb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 0x55f5c8ecc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f24a56000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5c8d1ad80 con 0x55f5c8f152f0 2024-03-22T04:16:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f24a56000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5c8cb7ce0 con 0x55f5c8f15c10 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f24a56000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f5c8d16d30 con 0x55f5c8ecbd60 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f1b400640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ef8067b20 0x7f8ef8069fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8f14007fd0 tx=0x7f8f14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.778+0000 7f8f1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f156d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56160/0 (socket says 172.21.15.86:56160) 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1aa00640 1 -- 172.21.15.86:0/493428489 learned_addr learned my addr 172.21.15.86:0/493428489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f13e00640 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 0x55f5c8ecb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1b400640 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 0x55f5c8ecc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1aa00640 1 -- 172.21.15.86:0/493428489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 msgr2=0x55f5c8ecc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 0x55f5c8ecc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1aa00640 1 -- 172.21.15.86:0/493428489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 msgr2=0x55f5c8ecb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 0x55f5c8ecb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1aa00640 1 -- 172.21.15.86:0/493428489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c8cf5340 con 0x55f5c8f152f0 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f156d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c01a030 tx=0x7f8f0c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a6cb9dca6a1f69 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f0b400640 1 -- 172.21.15.86:0/493428489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0b0040 con 0x55f5c8f152f0 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f0b400640 1 -- 172.21.15.86:0/493428489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0b1040 con 0x55f5c8f152f0 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f0b400640 1 -- 172.21.15.86:0/493428489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c004460 con 0x55f5c8f152f0 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- 172.21.15.86:0/493428489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 msgr2=0x55f5c8f156d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f156d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c01a030 tx=0x7f8f0c004740 comp rx=0 tx=0).stop 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- 172.21.15.86:0/493428489 shutdown_connections 2024-03-22T04:16:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 0x55f5c8ecb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 0x55f5c8ecc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 --2- 172.21.15.86:0/493428489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f156d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- 172.21.15.86:0/493428489 >> 172.21.15.86:0/493428489 conn(0x55f5c8ec7d10 msgr2=0x55f5c8eca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- 172.21.15.86:0/493428489 shutdown_connections 2024-03-22T04:16:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- 172.21.15.86:0/493428489 wait complete. 2024-03-22T04:16:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- start start 2024-03-22T04:16:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 Processor -- start 2024-03-22T04:16:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 0x55f5c8ed0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f152f0 0x55f5c8ed0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f15c10 0x55f5c8f15ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5c8cd84e0 con 0x55f5c8f15c10 2024-03-22T04:16:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5c8d27490 con 0x55f5c8f152f0 2024-03-22T04:16:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f5c8ec7b90 con 0x55f5c8ecbd60 2024-03-22T04:16:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f24a56000 10 client.0 _ll_register_callbacks cb 0x55f5c8eb6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f15c10 0x55f5c8f15ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f15c10 0x55f5c8f15ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56176/0 (socket says 172.21.15.86:56176) 2024-03-22T04:16:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f1b400640 1 -- 172.21.15.86:0/3935433218 learned_addr learned my addr 172.21.15.86:0/3935433218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.782+0000 7f8f13e00640 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f152f0 0x55f5c8ed0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 0x55f5c8ed0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1b400640 1 -- 172.21.15.86:0/3935433218 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 msgr2=0x55f5c8ed0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1b400640 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 0x55f5c8ed0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1b400640 1 -- 172.21.15.86:0/3935433218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f152f0 msgr2=0x55f5c8ed0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1b400640 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f152f0 0x55f5c8ed0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1b400640 1 -- 172.21.15.86:0/3935433218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c8d5f5e0 con 0x55f5c8f15c10 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1b400640 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f15c10 0x55f5c8f15ff0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400ebb0 tx=0x7f8f14001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9fc0f3cfec092a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 1 -- 172.21.15.86:0/3935433218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14027060 con 0x55f5c8f15c10 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 1 -- 172.21.15.86:0/3935433218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f8f14010040 con 0x55f5c8f15c10 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 -- 172.21.15.86:0/3935433218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f0c0b1040 con 0x55f5c8f15c10 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 1 -- 172.21.15.86:0/3935433218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8f14013420 con 0x55f5c8f15c10 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 10 client.7176 fetch_fsmap learned FSMap version 53 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 -- 172.21.15.86:0/3935433218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55f5c8cd65a0 con 0x55f5c8f15c10 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 1 -- 172.21.15.86:0/3935433218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0b1040 con 0x55f5c8f15c10 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 1 -- 172.21.15.86:0/3935433218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f8f140135c0 con 0x55f5c8f15c10 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 10 client.7176 fetch_fsmap finished waiting for FSMap version 53 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 10 client.7176 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 20 client.7176 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55f5c8f24670 0x55f5c8f26a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 4 client.7176 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 -- 172.21.15.86:0/3935433218 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c8cb5e40 con 0x55f5c8f24670 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55f5c8f24670 0x55f5c8f26a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55f5c8f24670 0x55f5c8f26a60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a63ab80c1940a04 server_cookie=16df0f4cc14f2c5f in_seq=0 out_seq=0 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 10 client.7176 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 1 -- 172.21.15.86:0/3935433218 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f5c8cb5e40 con 0x55f5c8f24670 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 10 client.7176 handle_command_reply: tid=0 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 client.7176 shutdown 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 -- 172.21.15.86:0/3935433218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f15c10 msgr2=0x55f5c8f15ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f15c10 0x55f5c8f15ff0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400ebb0 tx=0x7f8f14001260 comp rx=0 tx=0).stop 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 -- 172.21.15.86:0/3935433218 shutdown_connections 2024-03-22T04:16:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55f5c8f24670 0x55f5c8f26a60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f152f0 0x55f5c8ed0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f15c10 0x55f5c8f15ff0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 --2- 172.21.15.86:0/3935433218 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8ecbd60 0x55f5c8ed0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 -- 172.21.15.86:0/3935433218 >> 172.21.15.86:0/3935433218 conn(0x55f5c8ec7d10 msgr2=0x55f5c8eca990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8effe00640 0 client.7176 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 -- 172.21.15.86:0/3935433218 shutdown_connections 2024-03-22T04:16:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 1 -- 172.21.15.86:0/3935433218 wait complete. 2024-03-22T04:16:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.786+0000 7f8f24a56000 20 client.7176 trim_cache size 0 max 16384 2024-03-22T04:16:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 Processor -- start 2024-03-22T04:16:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- start start 2024-03-22T04:16:00.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f7f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f152f0 0x55f5c8f7fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 0x55f5c8f80430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c202de0 con 0x55f5c8ecbd60 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c155d30 con 0x55f5c8f15c10 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c14b860 con 0x55f5c8f152f0 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f7f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f7f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56180/0 (socket says 172.21.15.86:56180) 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 -- 172.21.15.86:0/852788244 learned_addr learned my addr 172.21.15.86:0/852788244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f13e00640 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f152f0 0x55f5c8f7fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1b400640 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 0x55f5c8f80430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 -- 172.21.15.86:0/852788244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f152f0 msgr2=0x55f5c8f7fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f152f0 0x55f5c8f7fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 -- 172.21.15.86:0/852788244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 msgr2=0x55f5c8f80430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 0x55f5c8f80430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 -- 172.21.15.86:0/852788244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c8cd6a50 con 0x55f5c8ecbd60 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f7f9b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c008010 tx=0x7f8f0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae490c5bc27324e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f0aa00640 1 -- 172.21.15.86:0/852788244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0b0070 con 0x55f5c8ecbd60 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f0aa00640 1 -- 172.21.15.86:0/852788244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0cb5f0 con 0x55f5c8ecbd60 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f0aa00640 1 -- 172.21.15.86:0/852788244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c004020 con 0x55f5c8ecbd60 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- 172.21.15.86:0/852788244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8ecbd60 msgr2=0x55f5c8f7f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f7f9b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c008010 tx=0x7f8f0c008040 comp rx=0 tx=0).stop 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- 172.21.15.86:0/852788244 shutdown_connections 2024-03-22T04:16:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8f15c10 0x55f5c8f80430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f152f0 0x55f5c8f7fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 --2- 172.21.15.86:0/852788244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f7f9b0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- 172.21.15.86:0/852788244 >> 172.21.15.86:0/852788244 conn(0x55f5c8ec7d10 msgr2=0x55f5c8eca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- 172.21.15.86:0/852788244 shutdown_connections 2024-03-22T04:16:00.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.802+0000 7f8f24a56000 1 -- 172.21.15.86:0/852788244 wait complete. 2024-03-22T04:16:00.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 -- start start 2024-03-22T04:16:00.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 Processor -- start 2024-03-22T04:16:00.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f28d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f15c10 0x55f5c8f84220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5c8ce8c20 con 0x55f5c8f152f0 2024-03-22T04:16:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5c8d1f3b0 con 0x55f5c8ecbd60 2024-03-22T04:16:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f5c8ca2550 con 0x55f5c8f15c10 2024-03-22T04:16:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 10 client.0 _ll_register_callbacks cb 0x7f8f14001680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f83ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56182/0 (socket says 172.21.15.86:56182) 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 -- 172.21.15.86:0/1299100148 learned_addr learned my addr 172.21.15.86:0/1299100148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f28d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f1b400640 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f15c10 0x55f5c8f84220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 -- 172.21.15.86:0/1299100148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f15c10 msgr2=0x55f5c8f84220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f15c10 0x55f5c8f84220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 -- 172.21.15.86:0/1299100148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8ecbd60 msgr2=0x55f5c8f28d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f28d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 -- 172.21.15.86:0/1299100148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c8d9b840 con 0x55f5c8f152f0 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f13e00640 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f83ce0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8f04055590 tx=0x7f8f04051b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d782fc1bd2e2a1a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 1 -- 172.21.15.86:0/1299100148 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04083e60 con 0x55f5c8f152f0 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 1 -- 172.21.15.86:0/1299100148 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f8f0405ded0 con 0x55f5c8f152f0 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 -- 172.21.15.86:0/1299100148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f0c0cb5f0 con 0x55f5c8f152f0 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 1 -- 172.21.15.86:0/1299100148 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8f0406db70 con 0x55f5c8f152f0 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 10 client.7182 fetch_fsmap learned FSMap version 53 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 -- 172.21.15.86:0/1299100148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55f5c8cc13e0 con 0x55f5c8f152f0 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 1 -- 172.21.15.86:0/1299100148 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0cb5f0 con 0x55f5c8f152f0 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 1 -- 172.21.15.86:0/1299100148 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f8f04007450 con 0x55f5c8f152f0 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 10 client.7182 fetch_fsmap finished waiting for FSMap version 53 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 10 client.7182 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 20 client.7182 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55f5c8f403b0 0x55f5c8f40790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 4 client.7182 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 -- 172.21.15.86:0/1299100148 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55f5c8ecf0d0 con 0x55f5c8f403b0 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55f5c8f403b0 0x55f5c8f40790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f1aa00640 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55f5c8f403b0 0x55f5c8f40790 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40014da02b1a9880 server_cookie=a13b0e255351b844 in_seq=0 out_seq=0 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 10 client.7182 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 1 -- 172.21.15.86:0/1299100148 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+236 (crc 0 0 0) 0x55f5c8ecf0d0 con 0x55f5c8f403b0 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8effe00640 10 client.7182 handle_command_reply: tid=0 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.806+0000 7f8f24a56000 1 client.7182 shutdown 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 -- 172.21.15.86:0/1299100148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 msgr2=0x55f5c8f83ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f83ce0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8f04055590 tx=0x7f8f04051b30 comp rx=0 tx=0).stop 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 -- 172.21.15.86:0/1299100148 shutdown_connections 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55f5c8f403b0 0x55f5c8f40790 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f5c8f15c10 0x55f5c8f84220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c8f152f0 0x55f5c8f83ce0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 --2- 172.21.15.86:0/1299100148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5c8ecbd60 0x55f5c8f28d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 -- 172.21.15.86:0/1299100148 >> 172.21.15.86:0/1299100148 conn(0x55f5c8ec7d10 msgr2=0x55f5c8eca990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8effe00640 0 client.7182 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 -- 172.21.15.86:0/1299100148 shutdown_connections 2024-03-22T04:16:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 1 -- 172.21.15.86:0/1299100148 wait complete. 2024-03-22T04:16:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f24a56000 20 client.7182 trim_cache size 0 max 16384 2024-03-22T04:16:00.822 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "dentry", 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 620949119, 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627776, 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: "frag": "*", 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "file_to_be_damaged", 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: "snap_id": "head", 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/subdir/file_to_be_damaged" 2024-03-22T04:16:00.823 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:00.824 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:16:00.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 -- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ef8067b20 msgr2=0x7f8ef8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ef8067b20 0x7f8ef8069fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8f14007fd0 tx=0x7f8f14009040 comp rx=0 tx=0).stop 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 -- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c09b180 msgr2=0x7f8f1c157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c09b180 0x7f8f1c157890 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400c780 tx=0x7f8f0400cc40 comp rx=0 tx=0).stop 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 -- 172.21.15.86:0/1636855337 shutdown_connections 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ef8067b20 0x7f8ef8069fe0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f1c15e970 0x7f8f1c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f1c09b180 0x7f8f1c157890 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 --2- 172.21.15.86:0/1636855337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f1c06d8f0 0x7f8f1c0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 -- 172.21.15.86:0/1636855337 >> 172.21.15.86:0/1636855337 conn(0x7f8f1c0b3bf0 msgr2=0x7f8f1c09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 -- 172.21.15.86:0/1636855337 shutdown_connections 2024-03-22T04:16:00.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.810+0000 7f8f21600640 1 -- 172.21.15.86:0/1636855337 wait complete. 2024-03-22T04:16:00.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-22T04:16:00.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.966+0000 7f1bef000640 1 Processor -- start 2024-03-22T04:16:00.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.966+0000 7f1bef000640 1 -- start start 2024-03-22T04:16:00.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.966+0000 7f1bef000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1be80b3d60 0x7f1be80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.966+0000 7f1bef000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be80d0690 0x7f1be815ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.966+0000 7f1bef000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1be80b4680 con 0x7f1be8147940 2024-03-22T04:16:00.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.966+0000 7f1bef000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1be815f280 con 0x7f1be80b3d60 2024-03-22T04:16:00.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.966+0000 7f1bef000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1be815f530 con 0x7f1be80d0690 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1be80b3d60 0x7f1be80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1be8147940 0x7f1be815eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55594/0 (socket says 172.21.15.86:55594) 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7400640 1 -- 172.21.15.86:0/2105600941 learned_addr learned my addr 172.21.15.86:0/2105600941 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1be80b3d60 0x7f1be80b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56186/0 (socket says 172.21.15.86:56186) 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1bed000640 1 --2- 172.21.15.86:0/2105600941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be80d0690 0x7f1be815ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7e00640 1 -- 172.21.15.86:0/2105600941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be80d0690 msgr2=0x7f1be815ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7e00640 1 --2- 172.21.15.86:0/2105600941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be80d0690 0x7f1be815ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7e00640 1 -- 172.21.15.86:0/2105600941 >> v1:172.21.15.86:6789/0 conn(0x7f1be8147940 legacy=0x7f1be815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7e00640 1 -- 172.21.15.86:0/2105600941 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be815f810 con 0x7f1be80b3d60 2024-03-22T04:16:00.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be7e00640 1 --2- 172.21.15.86:0/2105600941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be80b4140 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80097c0 tx=0x7f1bd801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e805b830028d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be6a00640 1 -- 172.21.15.86:0/2105600941 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80b1070 con 0x7f1be80b3d60 2024-03-22T04:16:00.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be6a00640 1 -- 172.21.15.86:0/2105600941 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd80a8050 con 0x7f1be80b3d60 2024-03-22T04:16:00.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.970+0000 7f1be6a00640 1 -- 172.21.15.86:0/2105600941 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80ac860 con 0x7f1be80b3d60 2024-03-22T04:16:00.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 -- 172.21.15.86:0/2105600941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 msgr2=0x7f1be80b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 --2- 172.21.15.86:0/2105600941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be80b4140 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80097c0 tx=0x7f1bd801c3e0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 -- 172.21.15.86:0/2105600941 shutdown_connections 2024-03-22T04:16:00.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 --2- 172.21.15.86:0/2105600941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be80d0690 0x7f1be815ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 --2- 172.21.15.86:0/2105600941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be80b4140 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 -- 172.21.15.86:0/2105600941 >> 172.21.15.86:0/2105600941 conn(0x7f1be806fa20 msgr2=0x7f1be806fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:00.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 -- 172.21.15.86:0/2105600941 shutdown_connections 2024-03-22T04:16:00.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 -- 172.21.15.86:0/2105600941 wait complete. 2024-03-22T04:16:00.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 Processor -- start 2024-03-22T04:16:00.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.974+0000 7f1bef000640 1 -- start start 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be815e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be80d0690 0x7f1be80c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be8147940 0x7f1be80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1be816b680 con 0x7f1be80b3d60 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1be816b500 con 0x7f1be80d0690 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1be816b800 con 0x7f1be8147940 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be815e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56192/0 (socket says 172.21.15.86:56192) 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 -- 172.21.15.86:0/1894975935 learned_addr learned my addr 172.21.15.86:0/1894975935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 -- 172.21.15.86:0/1894975935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be8147940 msgr2=0x7f1be80b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be8147940 0x7f1be80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 -- 172.21.15.86:0/1894975935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be80d0690 msgr2=0x7f1be80c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be80d0690 0x7f1be80c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:00.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 -- 172.21.15.86:0/1894975935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be80c62c0 con 0x7f1be80b3d60 2024-03-22T04:16:00.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1be7e00640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be815e130 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1bd801aa90 tx=0x7f1bd801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bdfe00640 1 -- 172.21.15.86:0/1894975935 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80b1040 con 0x7f1be80b3d60 2024-03-22T04:16:00.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bdfe00640 1 -- 172.21.15.86:0/1894975935 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd80bd070 con 0x7f1be80b3d60 2024-03-22T04:16:00.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bdfe00640 1 -- 172.21.15.86:0/1894975935 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80b7660 con 0x7f1be80b3d60 2024-03-22T04:16:00.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be80c5d00 con 0x7f1be80b3d60 2024-03-22T04:16:00.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be8148f10 con 0x7f1be80b3d60 2024-03-22T04:16:00.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.978+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be8074550 con 0x7f1be80b3d60 2024-03-22T04:16:00.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.982+0000 7f1bdfe00640 1 -- 172.21.15.86:0/1894975935 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1bd80b7800 con 0x7f1be80b3d60 2024-03-22T04:16:00.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.982+0000 7f1bdfe00640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bc8067b20 0x7f1bc8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:00.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.982+0000 7f1bdfe00640 1 -- 172.21.15.86:0/1894975935 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f1bd81205d0 con 0x7f1be80b3d60 2024-03-22T04:16:00.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.982+0000 7f1be7400640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bc8067b20 0x7f1bc8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:00.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.982+0000 7f1be7400640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bc8067b20 0x7f1bc8069fe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4000f70 tx=0x7f1bd4007a90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:00.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:00.986+0000 7f1bdfe00640 1 -- 172.21.15.86:0/1894975935 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1bd80c7020 con 0x7f1be80b3d60 2024-03-22T04:16:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1be81692d0 con 0x7f1be80b3d60 2024-03-22T04:16:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bdfe00640 1 -- 172.21.15.86:0/1894975935 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f1bd8016090 con 0x7f1be80b3d60 2024-03-22T04:16:01.178 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:01.179 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-22T04:16:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bc8067b20 msgr2=0x7f1bc8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bc8067b20 0x7f1bc8069fe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4000f70 tx=0x7f1bd4007a90 comp rx=0 tx=0).stop 2024-03-22T04:16:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 msgr2=0x7f1be815e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be815e130 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1bd801aa90 tx=0x7f1bd801ac20 comp rx=0 tx=0).stop 2024-03-22T04:16:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 shutdown_connections 2024-03-22T04:16:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1bc8067b20 0x7f1bc8069fe0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be80d0690 0x7f1be80c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1be8147940 0x7f1be80b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 --2- 172.21.15.86:0/1894975935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be80b3d60 0x7f1be815e130 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 >> 172.21.15.86:0/1894975935 conn(0x7f1be806fa20 msgr2=0x7f1be806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 shutdown_connections 2024-03-22T04:16:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.174+0000 7f1bef000640 1 -- 172.21.15.86:0/1894975935 wait complete. 2024-03-22T04:16:01.200 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-22T04:16:01.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:01.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-03-22T04:16:01.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbfe00640 1 Processor -- start 2024-03-22T04:16:01.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbfe00640 1 -- start start 2024-03-22T04:16:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbfe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0fc00715b0 0x7f0fc0161db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc006cf60 0x7f0fc00d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbfe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc0162c30 con 0x7f0fc0147cb0 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbfe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc0162390 con 0x7f0fc00715b0 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbfe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc0162670 con 0x7f0fc006cf60 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0fc0147cb0 0x7f0fc0070ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55600/0 (socket says 172.21.15.86:55600) 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbea00640 1 -- 172.21.15.86:0/2645369073 learned_addr learned my addr 172.21.15.86:0/2645369073 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbf400640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc006cf60 0x7f0fc00d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbe000640 1 --2- 172.21.15.86:0/2645369073 >> v2:172.21.15.196:3300/0 conn(0x7f0fc00715b0 0x7f0fc0161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708906441 0 0) 0x7f0fc0162c30 con 0x7f0fc0147cb0 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fa4003530 con 0x7f0fc0147cb0 2024-03-22T04:16:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 899489137 0 0) 0x7f0fa4003530 con 0x7f0fc0147cb0 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc006cf60 msgr2=0x7f0fc00d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc006cf60 0x7f0fc00d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 >> v2:172.21.15.196:3300/0 conn(0x7f0fc00715b0 msgr2=0x7f0fc0161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 --2- 172.21.15.86:0/2645369073 >> v2:172.21.15.196:3300/0 conn(0x7f0fc00715b0 0x7f0fc0161db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc0162950 con 0x7f0fc0147cb0 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f0fb0001680 con 0x7f0fc0147cb0 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 >> v1:172.21.15.86:6789/0 conn(0x7f0fc0147cb0 legacy=0x7f0fc0070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa40039c0 0x7f0fa4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc006cf60 0x7f0fa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4004d50 0x7f0fa40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0162390 con 0x7f0fa40039c0 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0162670 con 0x7f0fa4004d50 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa400b9d0 con 0x7f0fc006cf60 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbea00640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa40039c0 0x7f0fa4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbe000640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc006cf60 0x7f0fa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbf400640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4004d50 0x7f0fa40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbea00640 1 -- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc006cf60 msgr2=0x7f0fa4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbea00640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc006cf60 0x7f0fa4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbea00640 1 -- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4004d50 msgr2=0x7f0fa40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbea00640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa4004d50 0x7f0fa40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.338+0000 7f0fbea00640 1 -- 172.21.15.86:0/2645369073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa4003530 con 0x7f0fa40039c0 2024-03-22T04:16:01.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbea00640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa40039c0 0x7f0fa4003de0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c2f0 tx=0x7f0fb001cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8842ff65ad7fd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0007d60 con 0x7f0fa40039c0 2024-03-22T04:16:01.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb00a9020 con 0x7f0fa40039c0 2024-03-22T04:16:01.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbd600640 1 -- 172.21.15.86:0/2645369073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00a5ba0 con 0x7f0fa40039c0 2024-03-22T04:16:01.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2645369073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa40039c0 msgr2=0x7f0fa4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2645369073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa40039c0 0x7f0fa4003de0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c2f0 tx=0x7f0fb001cd10 comp rx=0 tx=0).stop 2024-03-22T04:16:01.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbf400640 1 -- 172.21.15.86:0/2645369073 reap_dead start 2024-03-22T04:16:01.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2645369073 shutdown_connections 2024-03-22T04:16:01.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2645369073 >> 172.21.15.86:0/2645369073 conn(0x7f0fc00d1fb0 msgr2=0x7f0fc00d23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2645369073 shutdown_connections 2024-03-22T04:16:01.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.342+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2645369073 wait complete. 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 Processor -- start 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 -- start start 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40039c0 0x7f0fc015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0142c10 0x7f0fc0164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc0166820 0x7f0fc01436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc00d1640 con 0x7f0fc0142c10 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc00d14c0 con 0x7f0fc0166820 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc00d17c0 con 0x7f0fa40039c0 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0142c10 0x7f0fc0164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0142c10 0x7f0fc0164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56220/0 (socket says 172.21.15.86:56220) 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 -- 172.21.15.86:0/2566907490 learned_addr learned my addr 172.21.15.86:0/2566907490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbf400640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc0166820 0x7f0fc01436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbea00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40039c0 0x7f0fc015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 -- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40039c0 msgr2=0x7f0fc015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40039c0 0x7f0fc015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 -- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc0166820 msgr2=0x7f0fc01436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc0166820 0x7f0fc01436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 -- 172.21.15.86:0/2566907490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc0165780 con 0x7f0fc0142c10 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbf400640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc0166820 0x7f0fc01436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbea00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40039c0 0x7f0fc015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbe000640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0142c10 0x7f0fc0164e80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0fac011a00 tx=0x7f0fac011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fb6a00640 1 -- 172.21.15.86:0/2566907490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00fb50 con 0x7f0fc0142c10 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fb6a00640 1 -- 172.21.15.86:0/2566907490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fac00fcf0 con 0x7f0fc0142c10 2024-03-22T04:16:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fb6a00640 1 -- 172.21.15.86:0/2566907490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00fea0 con 0x7f0fc0142c10 2024-03-22T04:16:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2566907490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc0165a60 con 0x7f0fc0142c10 2024-03-22T04:16:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.346+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2566907490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc0166020 con 0x7f0fc0142c10 2024-03-22T04:16:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.350+0000 7f0fb6a00640 1 -- 172.21.15.86:0/2566907490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0fac012040 con 0x7f0fc0142c10 2024-03-22T04:16:01.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.350+0000 7f0fb6a00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0fa0067b20 0x7f0fa0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.350+0000 7f0fb6a00640 1 -- 172.21.15.86:0/2566907490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f0fac086790 con 0x7f0fc0142c10 2024-03-22T04:16:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.350+0000 7f0fbea00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0fa0067b20 0x7f0fa0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.350+0000 7f0fc8b09000 1 Processor -- start 2024-03-22T04:16:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.350+0000 7f0fc8b09000 1 -- start start 2024-03-22T04:16:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.350+0000 7f0fbea00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0fa0067b20 0x7f0fa0069fe0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00aa000 tx=0x7f0fb0001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93717960 0x55db93717d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93718280 0x55db936ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db936ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db9351ca60 con 0x55db93717960 2024-03-22T04:16:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db934a3d40 con 0x55db936ce7a0 2024-03-22T04:16:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55db93518a10 con 0x55db93718280 2024-03-22T04:16:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93717960 0x55db93717d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93717960 0x55db93717d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56224/0 (socket says 172.21.15.86:56224) 2024-03-22T04:16:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 -- 172.21.15.86:0/40439244 learned_addr learned my addr 172.21.15.86:0/40439244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbf400640 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db936ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbe000640 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93718280 0x55db936ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 -- 172.21.15.86:0/40439244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93718280 msgr2=0x55db936ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93718280 0x55db936ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 -- 172.21.15.86:0/40439244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 msgr2=0x55db936ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db936ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 -- 172.21.15.86:0/40439244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db935c8db0 con 0x55db93717960 2024-03-22T04:16:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fbea00640 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93717960 0x55db93717d40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00986a0 tx=0x7f0fc0162b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135f47f45b5d3fb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fab400640 1 -- 172.21.15.86:0/40439244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00047f0 con 0x55db93717960 2024-03-22T04:16:01.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fab400640 1 -- 172.21.15.86:0/40439244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb00a9050 con 0x55db93717960 2024-03-22T04:16:01.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fab400640 1 -- 172.21.15.86:0/40439244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0004b90 con 0x55db93717960 2024-03-22T04:16:01.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- 172.21.15.86:0/40439244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93717960 msgr2=0x55db93717d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93717960 0x55db93717d40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00986a0 tx=0x7f0fc0162b20 comp rx=0 tx=0).stop 2024-03-22T04:16:01.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- 172.21.15.86:0/40439244 shutdown_connections 2024-03-22T04:16:01.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db936ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93718280 0x55db936ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/40439244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93717960 0x55db93717d40 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- 172.21.15.86:0/40439244 >> 172.21.15.86:0/40439244 conn(0x55db936ca750 msgr2=0x55db936ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- 172.21.15.86:0/40439244 shutdown_connections 2024-03-22T04:16:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- 172.21.15.86:0/40439244 wait complete. 2024-03-22T04:16:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 -- start start 2024-03-22T04:16:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.354+0000 7f0fc8b09000 1 Processor -- start 2024-03-22T04:16:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db9371ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db9371f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 0x55db936d8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db934b48b0 con 0x55db936ce7a0 2024-03-22T04:16:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db934d6c30 con 0x55db93718280 2024-03-22T04:16:01.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55db93521090 con 0x55db93717960 2024-03-22T04:16:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 10 client.0 _ll_register_callbacks cb 0x55db934dcd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db9371ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db9371ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56232/0 (socket says 172.21.15.86:56232) 2024-03-22T04:16:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 -- 172.21.15.86:0/1032366086 learned_addr learned my addr 172.21.15.86:0/1032366086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbf400640 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 0x55db936d8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbe000640 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db9371f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 -- 172.21.15.86:0/1032366086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 msgr2=0x55db9371f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db9371f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 -- 172.21.15.86:0/1032366086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 msgr2=0x55db936d8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 0x55db936d8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 -- 172.21.15.86:0/1032366086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db935c4d60 con 0x55db936ce7a0 2024-03-22T04:16:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db9371ecc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00986a0 tx=0x7f0fb0019ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b2bab71b36ebfd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1032366086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0007070 con 0x55db936ce7a0 2024-03-22T04:16:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1032366086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f0fb00c79e0 con 0x55db936ce7a0 2024-03-22T04:16:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1032366086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb00a9050 con 0x55db936ce7a0 2024-03-22T04:16:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1032366086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f0fb00c7be0 con 0x55db936ce7a0 2024-03-22T04:16:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 10 client.7203 fetch_fsmap learned FSMap version 53 2024-03-22T04:16:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1032366086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55db93562a00 con 0x55db936ce7a0 2024-03-22T04:16:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1032366086 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb00a9050 con 0x55db936ce7a0 2024-03-22T04:16:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1032366086 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f0fb0007210 con 0x55db936ce7a0 2024-03-22T04:16:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 10 client.7203 fetch_fsmap finished waiting for FSMap version 53 2024-03-22T04:16:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 10 client.7203 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:16:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 20 client.7203 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55db936d98d0 0x55db936d9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 4 client.7203 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:16:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1032366086 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db934b7cc0 con 0x55db936d98d0 2024-03-22T04:16:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbe000640 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55db936d98d0 0x55db936d9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 10 client.7203 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fbe000640 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55db936d98d0 0x55db936d9ce0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ffba137423cc579 server_cookie=91b53382d7bf4ddf in_seq=0 out_seq=0 2024-03-22T04:16:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1032366086 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55db934b7cc0 con 0x55db936d98d0 2024-03-22T04:16:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 10 client.7203 handle_command_reply: tid=0 2024-03-22T04:16:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 client.7203 shutdown 2024-03-22T04:16:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1032366086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 msgr2=0x55db9371ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db9371ecc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00986a0 tx=0x7f0fb0019ab0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1032366086 shutdown_connections 2024-03-22T04:16:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55db936d98d0 0x55db936d9ce0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 0x55db936d8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db9371f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1032366086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db9371ecc0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0f9fe00640 0 client.7203 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1032366086 >> 172.21.15.86:0/1032366086 conn(0x55db936ca750 msgr2=0x55db936cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.358+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1032366086 shutdown_connections 2024-03-22T04:16:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.362+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1032366086 wait complete. 2024-03-22T04:16:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.362+0000 7f0fc8b09000 20 client.7203 trim_cache size 0 max 16384 2024-03-22T04:16:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 Processor -- start 2024-03-22T04:16:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- start start 2024-03-22T04:16:01.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db9372f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db936d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93718280 0x55db936d3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0204400 con 0x55db93718280 2024-03-22T04:16:01.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0203460 con 0x55db936ce7a0 2024-03-22T04:16:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc0161020 con 0x55db93717960 2024-03-22T04:16:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93718280 0x55db936d3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93718280 0x55db936d3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56238/0 (socket says 172.21.15.86:56238) 2024-03-22T04:16:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 -- 172.21.15.86:0/1125377513 learned_addr learned my addr 172.21.15.86:0/1125377513 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbe000640 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db936d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db9372f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 -- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 msgr2=0x55db936d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db936d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 -- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 msgr2=0x55db9372f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db9372f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 -- 172.21.15.86:0/1125377513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db934dff50 con 0x55db93718280 2024-03-22T04:16:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db9372f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbe000640 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db936d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fbf400640 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93718280 0x55db936d3ac0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0070af0 tx=0x7f0fb8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d6b4f4aeab0c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0faaa00640 1 -- 172.21.15.86:0/1125377513 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8004700 con 0x55db93718280 2024-03-22T04:16:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0faaa00640 1 -- 172.21.15.86:0/1125377513 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb8021420 con 0x55db93718280 2024-03-22T04:16:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0faaa00640 1 -- 172.21.15.86:0/1125377513 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80217f0 con 0x55db93718280 2024-03-22T04:16:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1125377513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93718280 msgr2=0x55db936d3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93718280 0x55db936d3ac0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0070af0 tx=0x7f0fb8009750 comp rx=0 tx=0).stop 2024-03-22T04:16:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1125377513 shutdown_connections 2024-03-22T04:16:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db93718280 0x55db936d3ac0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db936d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1125377513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db936ce7a0 0x55db9372f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1125377513 >> 172.21.15.86:0/1125377513 conn(0x55db936ca750 msgr2=0x55db936d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1125377513 shutdown_connections 2024-03-22T04:16:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.378+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1125377513 wait complete. 2024-03-22T04:16:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 -- start start 2024-03-22T04:16:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 Processor -- start 2024-03-22T04:16:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db93783a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db9371b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 0x55db9371bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db9351bd70 con 0x55db936ce7a0 2024-03-22T04:16:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0202e40 con 0x55db93718280 2024-03-22T04:16:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55db935270c0 con 0x55db93717960 2024-03-22T04:16:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 10 client.0 _ll_register_callbacks cb 0x7f0fb8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db93783a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db93783a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56242/0 (socket says 172.21.15.86:56242) 2024-03-22T04:16:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 -- 172.21.15.86:0/1600641924 learned_addr learned my addr 172.21.15.86:0/1600641924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbe000640 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db9371b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbf400640 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 0x55db9371bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 -- 172.21.15.86:0/1600641924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 msgr2=0x55db9371b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db9371b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 -- 172.21.15.86:0/1600641924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 msgr2=0x55db9371bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 0x55db9371bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 -- 172.21.15.86:0/1600641924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9359f0f0 con 0x55db936ce7a0 2024-03-22T04:16:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db93783a20 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c210 tx=0x7f0fb00a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23c3edc199827bb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0f9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1600641924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0007760 con 0x55db936ce7a0 2024-03-22T04:16:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1600641924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f0fb0001660 con 0x55db936ce7a0 2024-03-22T04:16:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1600641924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb8021420 con 0x55db936ce7a0 2024-03-22T04:16:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1600641924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f0fb00d4070 con 0x55db936ce7a0 2024-03-22T04:16:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 10 client.7209 fetch_fsmap learned FSMap version 53 2024-03-22T04:16:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1600641924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55db93575760 con 0x55db936ce7a0 2024-03-22T04:16:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1600641924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb8021420 con 0x55db936ce7a0 2024-03-22T04:16:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1600641924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f0fb0004b00 con 0x55db936ce7a0 2024-03-22T04:16:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 10 client.7209 fetch_fsmap finished waiting for FSMap version 53 2024-03-22T04:16:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 10 client.7209 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:16:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 20 client.7209 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55db9371c140 0x55db9371c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 4 client.7209 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-03-22T04:16:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1600641924 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55db934f24e0 con 0x55db9371c140 2024-03-22T04:16:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.382+0000 7f0fbe000640 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55db9371c140 0x55db9371c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0f9fe00640 10 client.7209 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbe000640 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55db9371c140 0x55db9371c550 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8735898bfadebcb4 server_cookie=34758778ce4ccd16 in_seq=0 out_seq=0 2024-03-22T04:16:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0f9fe00640 1 -- 172.21.15.86:0/1600641924 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55db934f24e0 con 0x55db9371c140 2024-03-22T04:16:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0f9fe00640 10 client.7209 handle_command_reply: tid=0 2024-03-22T04:16:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 client.7209 shutdown 2024-03-22T04:16:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1600641924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 msgr2=0x55db93783a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db93783a20 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c210 tx=0x7f0fb00a7070 comp rx=0 tx=0).stop 2024-03-22T04:16:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1600641924 shutdown_connections 2024-03-22T04:16:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55db9371c140 0x55db9371c550 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db93718280 0x55db9371bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55db93717960 0x55db9371b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 --2- 172.21.15.86:0/1600641924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db936ce7a0 0x55db93783a20 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1600641924 >> 172.21.15.86:0/1600641924 conn(0x55db936ca750 msgr2=0x55db937165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0f9fe00640 0 client.7209 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1600641924 shutdown_connections 2024-03-22T04:16:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 1 -- 172.21.15.86:0/1600641924 wait complete. 2024-03-22T04:16:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fc8b09000 20 client.7209 trim_cache size 0 max 16384 2024-03-22T04:16:01.405 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:16:01.405 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:16:01.405 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "b77e3722-ac4c-4e41-9546-806b4e6e4ec3", 2024-03-22T04:16:01.405 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:16:01.405 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:16:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0fa0067b20 msgr2=0x7f0fa0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0fa0067b20 0x7f0fa0069fe0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00aa000 tx=0x7f0fb0001a50 comp rx=0 tx=0).stop 2024-03-22T04:16:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0142c10 msgr2=0x7f0fc0164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0142c10 0x7f0fc0164e80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0fac011a00 tx=0x7f0fac011ec0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2566907490 shutdown_connections 2024-03-22T04:16:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0fa0067b20 0x7f0fa0069fe0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc0166820 0x7f0fc01436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0142c10 0x7f0fc0164e80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2566907490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40039c0 0x7f0fc015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2566907490 >> 172.21.15.86:0/2566907490 conn(0x7f0fc00d1fb0 msgr2=0x7f0fc00cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2566907490 shutdown_connections 2024-03-22T04:16:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.386+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2566907490 wait complete. 2024-03-22T04:16:01.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:01.410 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-22T04:16:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b0d000640 1 Processor -- start 2024-03-22T04:16:01.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b0d000640 1 -- start start 2024-03-22T04:16:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b0d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b080b11c0 0x7f3b0809b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b0d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b080cda00 con 0x7f3b080b3610 2024-03-22T04:16:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b0d000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b080cdce0 con 0x7f3b0815c730 2024-03-22T04:16:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b0d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0809b910 con 0x7f3b080b11c0 2024-03-22T04:16:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b05400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3b080b3610 0x7f3b080b1ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55606/0 (socket says 172.21.15.86:55606) 2024-03-22T04:16:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b05400640 1 -- 172.21.15.86:0/2465068804 learned_addr learned my addr 172.21.15.86:0/2465068804 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:01.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b06800640 1 --2- 172.21.15.86:0/2465068804 >> v2:172.21.15.86:3300/0 conn(0x7f3b080b11c0 0x7f3b0809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b06800640 1 -- 172.21.15.86:0/2465068804 >> v1:172.21.15.196:6789/0 conn(0x7f3b0815c730 legacy=0x7f3b0809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b06800640 1 -- 172.21.15.86:0/2465068804 >> v1:172.21.15.86:6789/0 conn(0x7f3b080b3610 legacy=0x7f3b080b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b06800640 1 -- 172.21.15.86:0/2465068804 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b080cdfc0 con 0x7f3b080b11c0 2024-03-22T04:16:01.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.546+0000 7f3b04a00640 1 -- 172.21.15.86:0/2465068804 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384941536 0 0) 0x7f3b080cda00 con 0x7f3b080b3610 2024-03-22T04:16:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b06800640 1 --2- 172.21.15.86:0/2465068804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b11c0 0x7f3b0809b180 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009dc0 tx=0x7f3b0001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638a54280f0e79e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b04a00640 1 -- 172.21.15.86:0/2465068804 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b000b2070 con 0x7f3b080b11c0 2024-03-22T04:16:01.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b04a00640 1 -- 172.21.15.86:0/2465068804 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b000a5890 con 0x7f3b080b11c0 2024-03-22T04:16:01.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b04a00640 1 -- 172.21.15.86:0/2465068804 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b000a5c60 con 0x7f3b080b11c0 2024-03-22T04:16:01.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b0d000640 1 -- 172.21.15.86:0/2465068804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b11c0 msgr2=0x7f3b0809b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b0d000640 1 --2- 172.21.15.86:0/2465068804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b11c0 0x7f3b0809b180 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009dc0 tx=0x7f3b0001c8c0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b0d000640 1 -- 172.21.15.86:0/2465068804 shutdown_connections 2024-03-22T04:16:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b0d000640 1 --2- 172.21.15.86:0/2465068804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b11c0 0x7f3b0809b180 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b0d000640 1 -- 172.21.15.86:0/2465068804 >> 172.21.15.86:0/2465068804 conn(0x7f3b080b3bf0 msgr2=0x7f3b080b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b0d000640 1 -- 172.21.15.86:0/2465068804 shutdown_connections 2024-03-22T04:16:01.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.550+0000 7f3b0d000640 1 -- 172.21.15.86:0/2465068804 wait complete. 2024-03-22T04:16:01.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 Processor -- start 2024-03-22T04:16:01.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 -- start start 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b080b11c0 0x7f3b080c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b3610 0x7f3b08153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b0815c730 0x7f3b080b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080b2e50 con 0x7f3b080b3610 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080b2cd0 con 0x7f3b0815c730 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0809b6c0 con 0x7f3b080b11c0 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b3610 0x7f3b08153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b3610 0x7f3b08153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56250/0 (socket says 172.21.15.86:56250) 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 -- 172.21.15.86:0/1219097849 learned_addr learned my addr 172.21.15.86:0/1219097849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05e00640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b080b11c0 0x7f3b080c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b06800640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b0815c730 0x7f3b080b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 -- 172.21.15.86:0/1219097849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b080b11c0 msgr2=0x7f3b080c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b080b11c0 0x7f3b080c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 -- 172.21.15.86:0/1219097849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b0815c730 msgr2=0x7f3b080b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b0815c730 0x7f3b080b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 -- 172.21.15.86:0/1219097849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b080c6c20 con 0x7f3b080b3610 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b05400640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b3610 0x7f3b08153680 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c440 tx=0x7f3af400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3afe000640 1 -- 172.21.15.86:0/1219097849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4018070 con 0x7f3b080b3610 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3afe000640 1 -- 172.21.15.86:0/1219097849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af40074e0 con 0x7f3b080b3610 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3afe000640 1 -- 172.21.15.86:0/1219097849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4004a10 con 0x7f3b080b3610 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 -- 172.21.15.86:0/1219097849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b080c6f00 con 0x7f3b080b3610 2024-03-22T04:16:01.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.554+0000 7f3b0d000640 1 -- 172.21.15.86:0/1219097849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b080c74c0 con 0x7f3b080b3610 2024-03-22T04:16:01.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3afe000640 1 -- 172.21.15.86:0/1219097849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3af4004bb0 con 0x7f3b080b3610 2024-03-22T04:16:01.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3afe000640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ae8067b20 0x7f3ae8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3afe000640 1 -- 172.21.15.86:0/1219097849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f3af40853e0 con 0x7f3b080b3610 2024-03-22T04:16:01.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05e00640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ae8067b20 0x7f3ae8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b105b1000 1 Processor -- start 2024-03-22T04:16:01.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b105b1000 1 -- start start 2024-03-22T04:16:01.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05e00640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ae8067b20 0x7f3ae8069fe0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3af8007fd0 tx=0x7f3af8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed5345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed4e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b105b1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fed334c40 con 0x558fed534af0 2024-03-22T04:16:01.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b105b1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558fed2d1d90 con 0x558fed4e8360 2024-03-22T04:16:01.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b105b1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x558fed330bf0 con 0x558fed5341d0 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed4e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed4e7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56254/0 (socket says 172.21.15.86:56254) 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05400640 1 -- 172.21.15.86:0/613366247 learned_addr learned my addr 172.21.15.86:0/613366247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b06800640 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05e00640 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed5345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05400640 1 -- 172.21.15.86:0/613366247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 msgr2=0x558fed5345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05400640 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed5345b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05400640 1 -- 172.21.15.86:0/613366247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 msgr2=0x558fed4e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05400640 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.558+0000 7f3b05400640 1 -- 172.21.15.86:0/613366247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fed3dc130 con 0x558fed534af0 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b05400640 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed4e7e20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3af405d950 tx=0x7f3af4054fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b845aed8ecb2536 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3af2a00640 1 -- 172.21.15.86:0/613366247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40538b0 con 0x558fed534af0 2024-03-22T04:16:01.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3af2a00640 1 -- 172.21.15.86:0/613366247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af40850c0 con 0x558fed534af0 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3af2a00640 1 -- 172.21.15.86:0/613366247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4059150 con 0x558fed534af0 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- 172.21.15.86:0/613366247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 msgr2=0x558fed4e7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed4e7e20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3af405d950 tx=0x7f3af4054fd0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- 172.21.15.86:0/613366247 shutdown_connections 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed4e7e20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 --2- 172.21.15.86:0/613366247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed5345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- 172.21.15.86:0/613366247 >> 172.21.15.86:0/613366247 conn(0x558fed3cd990 msgr2=0x558fed513150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- 172.21.15.86:0/613366247 shutdown_connections 2024-03-22T04:16:01.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- 172.21.15.86:0/613366247 wait complete. 2024-03-22T04:16:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- start start 2024-03-22T04:16:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 Processor -- start 2024-03-22T04:16:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed4e8360 0x558fed517bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed5341d0 0x558fed518110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed518650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fed333f50 con 0x558fed534af0 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558fed2f2580 con 0x558fed5341d0 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x558fed339270 con 0x558fed4e8360 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b105b1000 10 client.0 _ll_register_callbacks cb 0x558fed396200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b06800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed518650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b06800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed518650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56270/0 (socket says 172.21.15.86:56270) 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b06800640 1 -- 172.21.15.86:0/235189880 learned_addr learned my addr 172.21.15.86:0/235189880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b05e00640 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed4e8360 0x558fed517bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b05400640 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed5341d0 0x558fed518110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b06800640 1 -- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed4e8360 msgr2=0x558fed517bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b06800640 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed4e8360 0x558fed517bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b06800640 1 -- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed5341d0 msgr2=0x558fed518110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b06800640 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed5341d0 0x558fed518110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b06800640 1 -- 172.21.15.86:0/235189880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fed30dd30 con 0x558fed534af0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.562+0000 7f3b05400640 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed5341d0 0x558fed518110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b06800640 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed518650 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3b0001c900 tx=0x7f3b0001b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57da354a0e93176 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 1 -- 172.21.15.86:0/235189880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b000b2040 con 0x558fed534af0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 1 -- 172.21.15.86:0/235189880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f3b000c4d10 con 0x558fed534af0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 -- 172.21.15.86:0/235189880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af40850c0 con 0x558fed534af0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 1 -- 172.21.15.86:0/235189880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3b000a5890 con 0x558fed534af0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 10 client.7224 fetch_fsmap learned FSMap version 53 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 -- 172.21.15.86:0/235189880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x558fed379fc0 con 0x558fed534af0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 1 -- 172.21.15.86:0/235189880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af40850c0 con 0x558fed534af0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 1 -- 172.21.15.86:0/235189880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f3b000a5a30 con 0x558fed534af0 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 10 client.7224 fetch_fsmap finished waiting for FSMap version 53 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 10 client.7224 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:16:01.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 20 client.7224 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x558fed543260 0x558fed545650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 4 client.7224 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 -- 172.21.15.86:0/235189880 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fed2cfdd0 con 0x558fed543260 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b05e00640 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x558fed543260 0x558fed545650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 10 client.7224 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b05e00640 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x558fed543260 0x558fed545650 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21da2dae6ea8c52 server_cookie=ee7f3add0be9e1a8 in_seq=0 out_seq=0 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 1 -- 172.21.15.86:0/235189880 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558fed2cfdd0 con 0x558fed543260 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 10 client.7224 handle_command_reply: tid=0 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 client.7224 shutdown 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 -- 172.21.15.86:0/235189880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 msgr2=0x558fed518650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed518650 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3b0001c900 tx=0x7f3b0001b100 comp rx=0 tx=0).stop 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 -- 172.21.15.86:0/235189880 shutdown_connections 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x558fed543260 0x558fed545650 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed5341d0 0x558fed518110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed518650 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 --2- 172.21.15.86:0/235189880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed4e8360 0x558fed517bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3ae7400640 0 client.7224 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:01.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 -- 172.21.15.86:0/235189880 >> 172.21.15.86:0/235189880 conn(0x558fed3cd990 msgr2=0x558fed4e3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 -- 172.21.15.86:0/235189880 shutdown_connections 2024-03-22T04:16:01.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 1 -- 172.21.15.86:0/235189880 wait complete. 2024-03-22T04:16:01.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.566+0000 7f3b105b1000 20 client.7224 trim_cache size 0 max 16384 2024-03-22T04:16:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 Processor -- start 2024-03-22T04:16:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- start start 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed4e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed545650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081ffa00 con 0x558fed534af0 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0806ddb0 con 0x558fed4e8360 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0815a650 con 0x558fed5341d0 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed545650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed545650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56282/0 (socket says 172.21.15.86:56282) 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 -- 172.21.15.86:0/2579318293 learned_addr learned my addr 172.21.15.86:0/2579318293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b05400640 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed4e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b05e00640 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 -- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 msgr2=0x558fed4e26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed4e26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 -- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 msgr2=0x558fed4e2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 -- 172.21.15.86:0/2579318293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fed30a3a0 con 0x558fed534af0 2024-03-22T04:16:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b05400640 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed4e26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b05e00640 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b06800640 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed545650 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3b0001cda0 tx=0x7f3b000bc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259e082f3ddf1478 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3af2000640 1 -- 172.21.15.86:0/2579318293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b000b2040 con 0x558fed534af0 2024-03-22T04:16:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3af2000640 1 -- 172.21.15.86:0/2579318293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b000c3480 con 0x558fed534af0 2024-03-22T04:16:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3af2000640 1 -- 172.21.15.86:0/2579318293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b000c3830 con 0x558fed534af0 2024-03-22T04:16:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- 172.21.15.86:0/2579318293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 msgr2=0x558fed545650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed545650 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3b0001cda0 tx=0x7f3b000bc070 comp rx=0 tx=0).stop 2024-03-22T04:16:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- 172.21.15.86:0/2579318293 shutdown_connections 2024-03-22T04:16:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed534af0 0x558fed545650 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed5341d0 0x558fed4e26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2579318293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed4e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- 172.21.15.86:0/2579318293 >> 172.21.15.86:0/2579318293 conn(0x558fed3cd990 msgr2=0x558fed5488a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- 172.21.15.86:0/2579318293 shutdown_connections 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- 172.21.15.86:0/2579318293 wait complete. 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 -- start start 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.586+0000 7f3b105b1000 1 Processor -- start 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed547400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed5341d0 0x558fed55efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed534af0 0x558fed55f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fed3f0810 con 0x558fed5341d0 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0815aca0 con 0x558fed4e8360 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x558fed2f0710 con 0x558fed534af0 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 10 client.0 _ll_register_callbacks cb 0x7f3b00003b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed5341d0 0x558fed55efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed5341d0 0x558fed55efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56296/0 (socket says 172.21.15.86:56296) 2024-03-22T04:16:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 -- 172.21.15.86:0/2956598560 learned_addr learned my addr 172.21.15.86:0/2956598560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05e00640 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed547400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b06800640 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed534af0 0x558fed55f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 -- 172.21.15.86:0/2956598560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed534af0 msgr2=0x558fed55f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed534af0 0x558fed55f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 -- 172.21.15.86:0/2956598560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 msgr2=0x558fed547400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed547400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 -- 172.21.15.86:0/2956598560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fed2ba380 con 0x558fed5341d0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05400640 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed5341d0 0x558fed55efa0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3af405d950 tx=0x7f3af4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fcfd432ebd287c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 1 -- 172.21.15.86:0/2956598560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af405de80 con 0x558fed5341d0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 1 -- 172.21.15.86:0/2956598560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f3af4059150 con 0x558fed5341d0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- 172.21.15.86:0/2956598560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b000c3480 con 0x558fed5341d0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 1 -- 172.21.15.86:0/2956598560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3af40593b0 con 0x558fed5341d0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 10 client.7230 fetch_fsmap learned FSMap version 53 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- 172.21.15.86:0/2956598560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x558fed3b6460 con 0x558fed5341d0 2024-03-22T04:16:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 1 -- 172.21.15.86:0/2956598560 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b000c3480 con 0x558fed5341d0 2024-03-22T04:16:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 1 -- 172.21.15.86:0/2956598560 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f3af4054a00 con 0x558fed5341d0 2024-03-22T04:16:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 10 client.7230 fetch_fsmap finished waiting for FSMap version 53 2024-03-22T04:16:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 10 client.7230 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:16:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 20 client.7230 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x558fed5a2ec0 0x558fed5a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 4 client.7230 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:16:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- 172.21.15.86:0/2956598560 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fed2d4160 con 0x558fed5a2ec0 2024-03-22T04:16:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05e00640 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x558fed5a2ec0 0x558fed5a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 10 client.7230 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b05e00640 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x558fed5a2ec0 0x558fed5a32d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5255d159b72436d server_cookie=a188401a6a328865 in_seq=0 out_seq=0 2024-03-22T04:16:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 1 -- 172.21.15.86:0/2956598560 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x558fed2d4160 con 0x558fed5a2ec0 2024-03-22T04:16:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 10 client.7230 handle_command_reply: tid=0 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 client.7230 shutdown 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- 172.21.15.86:0/2956598560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed5341d0 msgr2=0x558fed55efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed5341d0 0x558fed55efa0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3af405d950 tx=0x7f3af4000ee0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- 172.21.15.86:0/2956598560 shutdown_connections 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x558fed5a2ec0 0x558fed5a32d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558fed534af0 0x558fed55f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fed5341d0 0x558fed55efa0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3ae7400640 0 client.7230 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 --2- 172.21.15.86:0/2956598560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558fed4e8360 0x558fed547400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- 172.21.15.86:0/2956598560 >> 172.21.15.86:0/2956598560 conn(0x558fed3cd990 msgr2=0x558fed4ea750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- 172.21.15.86:0/2956598560 shutdown_connections 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 1 -- 172.21.15.86:0/2956598560 wait complete. 2024-03-22T04:16:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b105b1000 20 client.7230 trim_cache size 0 max 16384 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout: "b77e3722-ac4c-4e41-9546-806b4e6e4ec3": { 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/subdir", 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "b77e3722-ac4c-4e41-9546-806b4e6e4ec3", 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:01.601 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:16:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.590+0000 7f3b0d000640 1 -- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ae8067b20 msgr2=0x7f3ae8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ae8067b20 0x7f3ae8069fe0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3af8007fd0 tx=0x7f3af8009040 comp rx=0 tx=0).stop 2024-03-22T04:16:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 -- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b3610 msgr2=0x7f3b08153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b3610 0x7f3b08153680 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c440 tx=0x7f3af400c900 comp rx=0 tx=0).stop 2024-03-22T04:16:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 -- 172.21.15.86:0/1219097849 shutdown_connections 2024-03-22T04:16:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3ae8067b20 0x7f3ae8069fe0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b0815c730 0x7f3b080b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b080b3610 0x7f3b08153680 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 --2- 172.21.15.86:0/1219097849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b080b11c0 0x7f3b080c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 -- 172.21.15.86:0/1219097849 >> 172.21.15.86:0/1219097849 conn(0x7f3b080b3bf0 msgr2=0x7f3b080b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 -- 172.21.15.86:0/1219097849 shutdown_connections 2024-03-22T04:16:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:01.594+0000 7f3b0d000640 1 -- 172.21.15.86:0/1219097849 wait complete. 2024-03-22T04:16:01.616 INFO:tasks.cephfs.filesystem:scrub status for tag:b77e3722-ac4c-4e41-9546-806b4e6e4ec3 - {'path': '/subdir', 'tag': 'b77e3722-ac4c-4e41-9546-806b4e6e4ec3', 'options': 'recursive,repair'} 2024-03-22T04:16:28.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:16:28.180 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:16:31.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:31.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-22T04:16:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.754+0000 7f05bd800640 1 Processor -- start 2024-03-22T04:16:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.754+0000 7f05bd800640 1 -- start start 2024-03-22T04:16:31.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05bd800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05bd800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05b80737d0 0x7f05b80b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05bd800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b80b5ad0 con 0x7f05b815e710 2024-03-22T04:16:31.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05bd800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05b80b4370 con 0x7f05b80737d0 2024-03-22T04:16:31.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05bd800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f05b8072a10 con 0x7f05b815e330 2024-03-22T04:16:31.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05afe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f05b815e710 0x7f05b80b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51036/0 (socket says 172.21.15.86:51036) 2024-03-22T04:16:31.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05afe00640 1 -- 172.21.15.86:0/4095938040 learned_addr learned my addr 172.21.15.86:0/4095938040 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05b7400640 1 --2- 172.21.15.86:0/4095938040 >> v2:172.21.15.86:3300/0 conn(0x7f05b80737d0 0x7f05b80b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05b6a00640 1 --2- 172.21.15.86:0/4095938040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05b7400640 1 -- 172.21.15.86:0/4095938040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 msgr2=0x7f05b80b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05b6000640 1 -- 172.21.15.86:0/4095938040 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136118960 0 0) 0x7f05b80b5ad0 con 0x7f05b815e710 2024-03-22T04:16:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05b7400640 1 --2- 172.21.15.86:0/4095938040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b80b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05b7400640 1 -- 172.21.15.86:0/4095938040 >> v1:172.21.15.86:6789/0 conn(0x7f05b815e710 legacy=0x7f05b80b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05b7400640 1 -- 172.21.15.86:0/4095938040 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b8072cf0 con 0x7f05b80737d0 2024-03-22T04:16:31.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.758+0000 7f05b6a00640 1 --2- 172.21.15.86:0/4095938040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b80b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:31.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05b7400640 1 --2- 172.21.15.86:0/4095938040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b80b54a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009d90 tx=0x7f05b001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6308a9210f0e745e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05b6000640 1 -- 172.21.15.86:0/4095938040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b00b2070 con 0x7f05b80737d0 2024-03-22T04:16:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05b6000640 1 -- 172.21.15.86:0/4095938040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b00a5a00 con 0x7f05b80737d0 2024-03-22T04:16:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05b6000640 1 -- 172.21.15.86:0/4095938040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b00a5dd0 con 0x7f05b80737d0 2024-03-22T04:16:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 -- 172.21.15.86:0/4095938040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 msgr2=0x7f05b80b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 --2- 172.21.15.86:0/4095938040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b80b54a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009d90 tx=0x7f05b001c940 comp rx=0 tx=0).stop 2024-03-22T04:16:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 -- 172.21.15.86:0/4095938040 shutdown_connections 2024-03-22T04:16:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 --2- 172.21.15.86:0/4095938040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b80b54a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 --2- 172.21.15.86:0/4095938040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b80b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 -- 172.21.15.86:0/4095938040 >> 172.21.15.86:0/4095938040 conn(0x7f05b80d0fe0 msgr2=0x7f05b80d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 -- 172.21.15.86:0/4095938040 shutdown_connections 2024-03-22T04:16:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 -- 172.21.15.86:0/4095938040 wait complete. 2024-03-22T04:16:31.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 Processor -- start 2024-03-22T04:16:31.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.762+0000 7f05bd800640 1 -- start start 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05bd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b8160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05bd800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b8155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05bd800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b815e710 0x7f05b8147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05bd800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05b806efd0 con 0x7f05b80737d0 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05bd800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05b806ee50 con 0x7f05b815e710 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05bd800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f05b806f150 con 0x7f05b815e330 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b8160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b8160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52260/0 (socket says 172.21.15.86:52260) 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 -- 172.21.15.86:0/2824816621 learned_addr learned my addr 172.21.15.86:0/2824816621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b7400640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b815e710 0x7f05b8147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 -- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 msgr2=0x7f05b8155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05afe00640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b8155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b8155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 -- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b815e710 msgr2=0x7f05b8147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b815e710 0x7f05b8147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 -- 172.21.15.86:0/2824816621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b8156120 con 0x7f05b80737d0 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b7400640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b815e710 0x7f05b8147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05afe00640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b8155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05b6a00640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b8160180 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f05a800e840 tx=0x7f05a800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05aea00640 1 -- 172.21.15.86:0/2824816621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8017070 con 0x7f05b80737d0 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05aea00640 1 -- 172.21.15.86:0/2824816621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a800c8d0 con 0x7f05b80737d0 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05bd800640 1 -- 172.21.15.86:0/2824816621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b8160b60 con 0x7f05b80737d0 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05aea00640 1 -- 172.21.15.86:0/2824816621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a800f040 con 0x7f05b80737d0 2024-03-22T04:16:31.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.766+0000 7f05bd800640 1 -- 172.21.15.86:0/2824816621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b8160880 con 0x7f05b80737d0 2024-03-22T04:16:31.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05aea00640 1 -- 172.21.15.86:0/2824816621 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f05a801a070 con 0x7f05b80737d0 2024-03-22T04:16:31.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05aea00640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0594067b20 0x7f0594069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05aea00640 1 -- 172.21.15.86:0/2824816621 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f05a8085d40 con 0x7f05b80737d0 2024-03-22T04:16:31.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0594067b20 0x7f0594069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05c0c7b000 1 Processor -- start 2024-03-22T04:16:31.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05c0c7b000 1 -- start start 2024-03-22T04:16:31.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0594067b20 0x7f0594069fe0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f05b80d05f0 tx=0x7f05a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e6cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e680a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e680f50 0x55ec4e681330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e4d1c40 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e46ed90 con 0x55ec4e680f50 2024-03-22T04:16:31.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ec4e4cdbf0 con 0x55ec4e6ccf80 2024-03-22T04:16:31.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e680a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e680a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52262/0 (socket says 172.21.15.86:52262) 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 -- 172.21.15.86:0/2783625710 learned_addr learned my addr 172.21.15.86:0/2783625710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05b6a00640 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e6cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05b7400640 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e680f50 0x55ec4e681330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 -- 172.21.15.86:0/2783625710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 msgr2=0x55ec4e6cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e6cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 -- 172.21.15.86:0/2783625710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e680f50 msgr2=0x55ec4e681330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e680f50 0x55ec4e681330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 -- 172.21.15.86:0/2783625710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4e579130 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05afe00640 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e680a10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f05a0007360 tx=0x7f05a0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3d4f613d5a54f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05a7400640 1 -- 172.21.15.86:0/2783625710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000db50 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05a7400640 1 -- 172.21.15.86:0/2783625710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a000dcf0 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.770+0000 7f05a7400640 1 -- 172.21.15.86:0/2783625710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000dea0 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2783625710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 msgr2=0x55ec4e680a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e680a10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f05a0007360 tx=0x7f05a0010000 comp rx=0 tx=0).stop 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2783625710 shutdown_connections 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e680f50 0x55ec4e681330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e680a10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2783625710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e6cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2783625710 >> 172.21.15.86:0/2783625710 conn(0x55ec4e56a990 msgr2=0x55ec4e6b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2783625710 shutdown_connections 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2783625710 wait complete. 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- start start 2024-03-22T04:16:31.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 Processor -- start 2024-03-22T04:16:31.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 0x55ec4e6b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e6b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e48d710 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e48f580 con 0x55ec4e6ccf80 2024-03-22T04:16:31.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ec4e4d6270 con 0x55ec4e680f50 2024-03-22T04:16:31.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05c0c7b000 10 client.0 _ll_register_callbacks cb 0x55ec4e533200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:31.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e6b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e6b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52272/0 (socket says 172.21.15.86:52272) 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 -- 172.21.15.86:0/2069567546 learned_addr learned my addr 172.21.15.86:0/2069567546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05afe00640 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 0x55ec4e6b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 -- 172.21.15.86:0/2069567546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 msgr2=0x55ec4e6889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 -- 172.21.15.86:0/2069567546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 msgr2=0x55ec4e6b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 0x55ec4e6b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 -- 172.21.15.86:0/2069567546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4e4aad30 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f05b7400640 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e6b18c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f05b00b0040 tx=0x7f05b001b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac194d5bb1e8ecab server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.774+0000 7f059be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 1 -- 172.21.15.86:0/2069567546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b00c1020 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2069567546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a000dcf0 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 1 -- 172.21.15.86:0/2069567546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f05b00a5c00 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 1 -- 172.21.15.86:0/2069567546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f05b00b2040 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 10 client.7245 fetch_fsmap learned FSMap version 54 2024-03-22T04:16:31.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2069567546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55ec4e516fc0 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 1 -- 172.21.15.86:0/2069567546 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a000dcf0 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 1 -- 172.21.15.86:0/2069567546 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 54) ==== 3494+0+0 (secure 0 0 0) 0x7f05b00b9460 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 10 client.7245 fetch_fsmap finished waiting for FSMap version 54 2024-03-22T04:16:31.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 10 client.7245 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:16:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 20 client.7245 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55ec4e6d35c0 0x55ec4e6d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 4 client.7245 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:16:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2069567546 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec4e46cdd0 con 0x55ec4e6d35c0 2024-03-22T04:16:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05b6a00640 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55ec4e6d35c0 0x55ec4e6d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05b6a00640 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55ec4e6d35c0 0x55ec4e6d39a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b242b6d82adc267a server_cookie=40d32107c6efabc2 in_seq=0 out_seq=0 2024-03-22T04:16:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 10 client.7245 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 1 -- 172.21.15.86:0/2069567546 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ec4e46cdd0 con 0x55ec4e6d35c0 2024-03-22T04:16:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 10 client.7245 handle_command_reply: tid=0 2024-03-22T04:16:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 client.7245 shutdown 2024-03-22T04:16:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2069567546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 msgr2=0x55ec4e6b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e6b18c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f05b00b0040 tx=0x7f05b001b180 comp rx=0 tx=0).stop 2024-03-22T04:16:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2069567546 shutdown_connections 2024-03-22T04:16:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55ec4e6d35c0 0x55ec4e6d39a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 0x55ec4e6b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e6cd8a0 0x55ec4e6b18c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f059be00640 0 client.7245 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/2069567546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2069567546 >> 172.21.15.86:0/2069567546 conn(0x55ec4e56a990 msgr2=0x55ec4e680110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2069567546 shutdown_connections 2024-03-22T04:16:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 1 -- 172.21.15.86:0/2069567546 wait complete. 2024-03-22T04:16:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.778+0000 7f05c0c7b000 20 client.7245 trim_cache size 0 max 16384 2024-03-22T04:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.794+0000 7f05c0c7b000 1 Processor -- start 2024-03-22T04:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.794+0000 7f05c0c7b000 1 -- start start 2024-03-22T04:16:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e56a990 0x55ec4e6e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e680f50 0x55ec4e6878d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e687e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8204020 con 0x55ec4e680f50 2024-03-22T04:16:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8203040 con 0x55ec4e56a990 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f05b815a0f0 con 0x55ec4e6ccf80 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e680f50 0x55ec4e6878d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e680f50 0x55ec4e6878d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52278/0 (socket says 172.21.15.86:52278) 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 -- 172.21.15.86:0/140948838 learned_addr learned my addr 172.21.15.86:0/140948838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05b6a00640 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e56a990 0x55ec4e6e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05b7400640 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e687e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 -- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 msgr2=0x55ec4e687e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e687e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 -- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e56a990 msgr2=0x55ec4e6e3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e56a990 0x55ec4e6e3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 -- 172.21.15.86:0/140948838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4e4a73a0 con 0x55ec4e680f50 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05b7400640 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e687e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05afe00640 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e680f50 0x55ec4e6878d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009010 tx=0x7f05a0007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcda8c7b2d13413b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05a6a00640 1 -- 172.21.15.86:0/140948838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000dc10 con 0x55ec4e680f50 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05a6a00640 1 -- 172.21.15.86:0/140948838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a000ddb0 con 0x55ec4e680f50 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05a6a00640 1 -- 172.21.15.86:0/140948838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0020650 con 0x55ec4e680f50 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- 172.21.15.86:0/140948838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e680f50 msgr2=0x55ec4e6878d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e680f50 0x55ec4e6878d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009010 tx=0x7f05a0007000 comp rx=0 tx=0).stop 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- 172.21.15.86:0/140948838 shutdown_connections 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e6ccf80 0x55ec4e687e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e680f50 0x55ec4e6878d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/140948838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e56a990 0x55ec4e6e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- 172.21.15.86:0/140948838 >> 172.21.15.86:0/140948838 conn(0x55ec4e6b0400 msgr2=0x55ec4e6871c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- 172.21.15.86:0/140948838 shutdown_connections 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- 172.21.15.86:0/140948838 wait complete. 2024-03-22T04:16:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 -- start start 2024-03-22T04:16:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.798+0000 7f05c0c7b000 1 Processor -- start 2024-03-22T04:16:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e56a990 0x55ec4e6d6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6d6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 0x55ec4e6d7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e58d810 con 0x55ec4e56a990 2024-03-22T04:16:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05b815a740 con 0x55ec4e6ccf80 2024-03-22T04:16:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ec4e48d710 con 0x55ec4e680f50 2024-03-22T04:16:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 10 client.0 _ll_register_callbacks cb 0x7f05b0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e56a990 0x55ec4e6d6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e56a990 0x55ec4e6d6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52280/0 (socket says 172.21.15.86:52280) 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 -- 172.21.15.86:0/4235226255 learned_addr learned my addr 172.21.15.86:0/4235226255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b7400640 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 0x55ec4e6d7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05afe00640 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6d6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 -- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 msgr2=0x55ec4e6d6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6d6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 -- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 msgr2=0x55ec4e6d7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 0x55ec4e6d7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 -- 172.21.15.86:0/4235226255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4e457380 con 0x55ec4e56a990 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05afe00640 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6d6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05b6a00640 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e56a990 0x55ec4e6d6a00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f05a800e810 tx=0x7f05a8052180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140657b088b500b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 1 -- 172.21.15.86:0/4235226255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a800cca0 con 0x55ec4e56a990 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 1 -- 172.21.15.86:0/4235226255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f05a800ce40 con 0x55ec4e56a990 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- 172.21.15.86:0/4235226255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a000ddb0 con 0x55ec4e56a990 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 1 -- 172.21.15.86:0/4235226255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f05a80552b0 con 0x55ec4e56a990 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 10 client.7251 fetch_fsmap learned FSMap version 54 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- 172.21.15.86:0/4235226255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55ec4e553460 con 0x55ec4e56a990 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 1 -- 172.21.15.86:0/4235226255 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a000ddb0 con 0x55ec4e56a990 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 1 -- 172.21.15.86:0/4235226255 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 54) ==== 3494+0+0 (secure 0 0 0) 0x7f05a8012050 con 0x55ec4e56a990 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 10 client.7251 fetch_fsmap finished waiting for FSMap version 54 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 10 client.7251 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 20 client.7251 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55ec4e6cd8a0 0x55ec4e6d79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 4 client.7251 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- 172.21.15.86:0/4235226255 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec4e471160 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05afe00640 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55ec4e6cd8a0 0x55ec4e6d79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05afe00640 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55ec4e6cd8a0 0x55ec4e6d79c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1263c2fe5d9eed95 server_cookie=abef4132a1886da6 in_seq=0 out_seq=0 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 10 client.7251 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 1 -- 172.21.15.86:0/4235226255 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ec4e471160 con 0x55ec4e6cd8a0 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 10 client.7251 handle_command_reply: tid=0 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 client.7251 shutdown 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- 172.21.15.86:0/4235226255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e56a990 msgr2=0x55ec4e6d6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e56a990 0x55ec4e6d6a00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f05a800e810 tx=0x7f05a8052180 comp rx=0 tx=0).stop 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- 172.21.15.86:0/4235226255 shutdown_connections 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55ec4e6cd8a0 0x55ec4e6d79c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec4e6ccf80 0x55ec4e6d7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec4e680f50 0x55ec4e6d6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 --2- 172.21.15.86:0/4235226255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4e56a990 0x55ec4e6d6a00 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- 172.21.15.86:0/4235226255 >> 172.21.15.86:0/4235226255 conn(0x55ec4e6b0400 msgr2=0x55ec4e687a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f059be00640 0 client.7251 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- 172.21.15.86:0/4235226255 shutdown_connections 2024-03-22T04:16:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 1 -- 172.21.15.86:0/4235226255 wait complete. 2024-03-22T04:16:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05c0c7b000 20 client.7251 trim_cache size 0 max 16384 2024-03-22T04:16:31.809 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:16:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:16:31.809 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:16:31.809 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:16:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 -- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0594067b20 msgr2=0x7f0594069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0594067b20 0x7f0594069fe0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f05b80d05f0 tx=0x7f05a0009040 comp rx=0 tx=0).stop 2024-03-22T04:16:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 -- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 msgr2=0x7f05b8160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b8160180 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f05a800e840 tx=0x7f05a800ed00 comp rx=0 tx=0).stop 2024-03-22T04:16:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 -- 172.21.15.86:0/2824816621 shutdown_connections 2024-03-22T04:16:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0594067b20 0x7f0594069fe0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b815e710 0x7f05b8147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b815e330 0x7f05b8155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 --2- 172.21.15.86:0/2824816621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b80737d0 0x7f05b8160180 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.802+0000 7f05bd800640 1 -- 172.21.15.86:0/2824816621 >> 172.21.15.86:0/2824816621 conn(0x7f05b80d0fe0 msgr2=0x7f05b815d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.806+0000 7f05bd800640 1 -- 172.21.15.86:0/2824816621 shutdown_connections 2024-03-22T04:16:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.806+0000 7f05bd800640 1 -- 172.21.15.86:0/2824816621 wait complete. 2024-03-22T04:16:31.824 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:16:31.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:16:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.970+0000 7f3abb200640 1 Processor -- start 2024-03-22T04:16:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3abb200640 1 -- start start 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3abb200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc0b4eb0 0x7f3abc143b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3abb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3abc06d6c0 0x7f3abc06db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3abb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3abc1444d0 con 0x7f3abc141220 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3abb200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3abc1447b0 con 0x7f3abc06d6c0 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3abb200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3abc0b3190 con 0x7f3abc0b4eb0 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3aba800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3abc06d6c0 0x7f3abc06db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3aba800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3abc06d6c0 0x7f3abc06db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52290/0 (socket says 172.21.15.86:52290) 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3ab9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3abc141220 0x7f3abc0b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51048/0 (socket says 172.21.15.86:51048) 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3aba800640 1 -- 172.21.15.86:0/3116525808 learned_addr learned my addr 172.21.15.86:0/3116525808 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3ab9400640 1 --2- 172.21.15.86:0/3116525808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc0b4eb0 0x7f3abc143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3aba800640 1 -- 172.21.15.86:0/3116525808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc0b4eb0 msgr2=0x7f3abc143b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3aba800640 1 --2- 172.21.15.86:0/3116525808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc0b4eb0 0x7f3abc143b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3aba800640 1 -- 172.21.15.86:0/3116525808 >> v1:172.21.15.86:6789/0 conn(0x7f3abc141220 legacy=0x7f3abc0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3aba800640 1 -- 172.21.15.86:0/3116525808 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc0b3470 con 0x7f3abc06d6c0 2024-03-22T04:16:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.974+0000 7f3ab8a00640 1 -- 172.21.15.86:0/3116525808 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672100954 0 0) 0x7f3abc1444d0 con 0x7f3abc141220 2024-03-22T04:16:31.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3aba800640 1 --2- 172.21.15.86:0/3116525808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc06db50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009e50 tx=0x7f3ab401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcb7e6b12876618 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3ab8a00640 1 -- 172.21.15.86:0/3116525808 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40a5660 con 0x7f3abc06d6c0 2024-03-22T04:16:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3ab8a00640 1 -- 172.21.15.86:0/3116525808 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab40a5800 con 0x7f3abc06d6c0 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3ab8a00640 1 -- 172.21.15.86:0/3116525808 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40a5bd0 con 0x7f3abc06d6c0 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 -- 172.21.15.86:0/3116525808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 msgr2=0x7f3abc06db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 --2- 172.21.15.86:0/3116525808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc06db50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009e50 tx=0x7f3ab401c9a0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 -- 172.21.15.86:0/3116525808 shutdown_connections 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 --2- 172.21.15.86:0/3116525808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc06db50 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 --2- 172.21.15.86:0/3116525808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc0b4eb0 0x7f3abc143b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 -- 172.21.15.86:0/3116525808 >> 172.21.15.86:0/3116525808 conn(0x7f3abc06b660 msgr2=0x7f3abc06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 -- 172.21.15.86:0/3116525808 shutdown_connections 2024-03-22T04:16:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 -- 172.21.15.86:0/3116525808 wait complete. 2024-03-22T04:16:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 Processor -- start 2024-03-22T04:16:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.978+0000 7f3abb200640 1 -- start start 2024-03-22T04:16:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc141220 0x7f3abc15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3abc13c040 0x7f3abc13c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc073040 con 0x7f3abc06d6c0 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc1442b0 con 0x7f3abc13c040 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3abc0731c0 con 0x7f3abc141220 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc0b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52304/0 (socket says 172.21.15.86:52304) 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 -- 172.21.15.86:0/411133390 learned_addr learned my addr 172.21.15.86:0/411133390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3aba800640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3abc13c040 0x7f3abc13c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 -- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc141220 msgr2=0x7f3abc15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9400640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc141220 0x7f3abc15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc141220 0x7f3abc15d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 -- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3abc13c040 msgr2=0x7f3abc13c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3abc13c040 0x7f3abc13c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:31.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 -- 172.21.15.86:0/411133390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc0b1620 con 0x7f3abc06d6c0 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9400640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc141220 0x7f3abc15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3aba800640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3abc13c040 0x7f3abc13c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab9e00640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc0b23e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00c480 tx=0x7f3aac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab2000640 1 -- 172.21.15.86:0/411133390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac018070 con 0x7f3abc06d6c0 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab2000640 1 -- 172.21.15.86:0/411133390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aac007500 con 0x7f3abc06d6c0 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab2000640 1 -- 172.21.15.86:0/411133390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004a30 con 0x7f3abc06d6c0 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc0b1900 con 0x7f3abc06d6c0 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc0b1ec0 con 0x7f3abc06d6c0 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc0b20c0 con 0x7f3abc06d6c0 2024-03-22T04:16:31.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.982+0000 7f3ab2000640 1 -- 172.21.15.86:0/411133390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3aac004bd0 con 0x7f3abc06d6c0 2024-03-22T04:16:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.986+0000 7f3ab2000640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3a9c067b20 0x7f3a9c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.986+0000 7f3ab2000640 1 -- 172.21.15.86:0/411133390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f3aac0853a0 con 0x7f3abc06d6c0 2024-03-22T04:16:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.986+0000 7f3ab9400640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3a9c067b20 0x7f3a9c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.986+0000 7f3ab9400640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3a9c067b20 0x7f3a9c069fe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3abc142430 tx=0x7f3aa8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:31.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:31.994+0000 7f3ab2000640 1 -- 172.21.15.86:0/411133390 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3abc0b20c0 con 0x7f3abc06d6c0 2024-03-22T04:16:32.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.182+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3abc074470 con 0x7f3abc06d6c0 2024-03-22T04:16:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.182+0000 7f3ab2000640 1 -- 172.21.15.86:0/411133390 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5188 (secure 0 0 0) 0x7f3aac055bc0 con 0x7f3abc06d6c0 2024-03-22T04:16:32.183 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:32.183 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:16:07.415273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7002,"qdb_cluster":[7002]},"id":5}]} 2024-03-22T04:16:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 54 2024-03-22T04:16:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.182+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3a9c067b20 msgr2=0x7f3a9c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.182+0000 7f3abb200640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3a9c067b20 0x7f3a9c069fe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3abc142430 tx=0x7f3aa8008040 comp rx=0 tx=0).stop 2024-03-22T04:16:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.182+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 msgr2=0x7f3abc0b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.182+0000 7f3abb200640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc0b23e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00c480 tx=0x7f3aac00c940 comp rx=0 tx=0).stop 2024-03-22T04:16:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.186+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 shutdown_connections 2024-03-22T04:16:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.186+0000 7f3abb200640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3a9c067b20 0x7f3a9c069fe0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.186+0000 7f3abb200640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3abc13c040 0x7f3abc13c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.186+0000 7f3abb200640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3abc141220 0x7f3abc15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.186+0000 7f3abb200640 1 --2- 172.21.15.86:0/411133390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3abc06d6c0 0x7f3abc0b23e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.186+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 >> 172.21.15.86:0/411133390 conn(0x7f3abc06b660 msgr2=0x7f3abc141810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.186+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 shutdown_connections 2024-03-22T04:16:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.186+0000 7f3abb200640 1 -- 172.21.15.86:0/411133390 wait complete. 2024-03-22T04:16:32.205 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-03-22T04:16:32.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.342+0000 7f14b9a00640 1 Processor -- start 2024-03-22T04:16:32.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b9a00640 1 -- start start 2024-03-22T04:16:32.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b9a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f14b406e7e0 0x7f14b406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b9a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b415c770 con 0x7f14b4072b00 2024-03-22T04:16:32.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b9a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b40b29a0 con 0x7f14b4072650 2024-03-22T04:16:32.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b9a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f14b4071350 con 0x7f14b406e7e0 2024-03-22T04:16:32.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f14b4072b00 0x7f14b40b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51052/0 (socket says 172.21.15.86:51052) 2024-03-22T04:16:32.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b3400640 1 -- 172.21.15.86:0/2959567187 learned_addr learned my addr 172.21.15.86:0/2959567187 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b2a00640 1 --2- 172.21.15.86:0/2959567187 >> v2:172.21.15.196:3300/0 conn(0x7f14b406e7e0 0x7f14b406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1919173177 0 0) 0x7f14ac0014a0 con 0x7f14b4072b00 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1498003530 con 0x7f14b4072b00 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548607116 0 0) 0x7f14b40b29a0 con 0x7f14b4072650 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14ac0014a0 con 0x7f14b4072650 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3773722905 0 0) 0x7f1498003530 con 0x7f14b4072b00 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 >> v2:172.21.15.196:3300/0 conn(0x7f14b406e7e0 msgr2=0x7f14b406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 --2- 172.21.15.86:0/2959567187 >> v2:172.21.15.196:3300/0 conn(0x7f14b406e7e0 0x7f14b406ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 >> v1:172.21.15.196:6789/0 conn(0x7f14b4072650 legacy=0x7f14b40b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b406f100 con 0x7f14b4072b00 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f14ac001bd0 con 0x7f14b4072b00 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 >> v1:172.21.15.86:6789/0 conn(0x7f14b4072b00 legacy=0x7f14b40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1498003b50 0x7f1498003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1498004810 0x7f1498004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14980051d0 0x7f149800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4071350 con 0x7f14980051d0 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f149800c100 con 0x7f1498003b50 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f149800c300 con 0x7f1498004810 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b3400640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14980051d0 0x7f149800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b2000640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1498004810 0x7f1498004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b2a00640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1498003b50 0x7f1498003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b3400640 1 -- 172.21.15.86:0/2959567187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1498004810 msgr2=0x7f1498004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b3400640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1498004810 0x7f1498004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b3400640 1 -- 172.21.15.86:0/2959567187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1498003b50 msgr2=0x7f1498003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b3400640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1498003b50 0x7f1498003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.346+0000 7f14b3400640 1 -- 172.21.15.86:0/2959567187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1498003530 con 0x7f14980051d0 2024-03-22T04:16:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b3400640 1 -- 172.21.15.86:0/2959567187 reap_dead start 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b3400640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14980051d0 0x7f149800b9d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00cc50 tx=0x7f14ac01f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d3076433cc62f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00c8a0 con 0x7f14980051d0 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac0ac030 con 0x7f14980051d0 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b1600640 1 -- 172.21.15.86:0/2959567187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0a8c40 con 0x7f14980051d0 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b9a00640 1 -- 172.21.15.86:0/2959567187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14980051d0 msgr2=0x7f149800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b9a00640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14980051d0 0x7f149800b9d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00cc50 tx=0x7f14ac01f9d0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b9a00640 1 -- 172.21.15.86:0/2959567187 shutdown_connections 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b9a00640 1 --2- 172.21.15.86:0/2959567187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14980051d0 0x7f149800b9d0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b9a00640 1 -- 172.21.15.86:0/2959567187 >> 172.21.15.86:0/2959567187 conn(0x7f14b40cc940 msgr2=0x7f14b40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b9a00640 1 -- 172.21.15.86:0/2959567187 shutdown_connections 2024-03-22T04:16:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.350+0000 7f14b9a00640 1 -- 172.21.15.86:0/2959567187 wait complete. 2024-03-22T04:16:32.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 Processor -- start 2024-03-22T04:16:32.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 -- start start 2024-03-22T04:16:32.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14980051d0 0x7f14b40c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b414db90 0x7f14b40cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14b4155aa0 0x7f14b4155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4071b60 con 0x7f14b414db90 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4071880 con 0x7f14b4155aa0 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14b40ce930 con 0x7f14980051d0 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b414db90 0x7f14b40cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b414db90 0x7f14b40cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52334/0 (socket says 172.21.15.86:52334) 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 -- 172.21.15.86:0/3343424437 learned_addr learned my addr 172.21.15.86:0/3343424437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14980051d0 0x7f14b40c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b3400640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14b4155aa0 0x7f14b4155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 -- 172.21.15.86:0/3343424437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14980051d0 msgr2=0x7f14b40c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14980051d0 0x7f14b40c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 -- 172.21.15.86:0/3343424437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14b4155aa0 msgr2=0x7f14b4155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14b4155aa0 0x7f14b4155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 -- 172.21.15.86:0/3343424437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b40ca260 con 0x7f14b414db90 2024-03-22T04:16:32.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b2000640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b414db90 0x7f14b40cb640 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f14a0012800 tx=0x7f14a0012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14aaa00640 1 -- 172.21.15.86:0/3343424437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0010b50 con 0x7f14b414db90 2024-03-22T04:16:32.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14aaa00640 1 -- 172.21.15.86:0/3343424437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a001b070 con 0x7f14b414db90 2024-03-22T04:16:32.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 -- 172.21.15.86:0/3343424437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b40ca540 con 0x7f14b414db90 2024-03-22T04:16:32.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14aaa00640 1 -- 172.21.15.86:0/3343424437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0016480 con 0x7f14b414db90 2024-03-22T04:16:32.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.354+0000 7f14b9a00640 1 -- 172.21.15.86:0/3343424437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b40cab00 con 0x7f14b414db90 2024-03-22T04:16:32.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.358+0000 7f14aaa00640 1 -- 172.21.15.86:0/3343424437 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f14a0016620 con 0x7f14b414db90 2024-03-22T04:16:32.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.358+0000 7f14aaa00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1494067b50 0x7f149406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.358+0000 7f14aaa00640 1 -- 172.21.15.86:0/3343424437 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f14a0089810 con 0x7f14b414db90 2024-03-22T04:16:32.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.358+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1494067b50 0x7f149406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.358+0000 7f14bcf5f000 1 Processor -- start 2024-03-22T04:16:32.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.358+0000 7f14bcf5f000 1 -- start start 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.358+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1494067b50 0x7f149406a010 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f14b415e6a0 tx=0x7f14a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 0x55d97b5556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b50b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b50bd60 0x55d97b50c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d97b35ad80 con 0x55d97b555c10 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d97b2f7ce0 con 0x55d97b50bd60 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d97b356d30 con 0x55d97b5552f0 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b50b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b50b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52346/0 (socket says 172.21.15.86:52346) 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 -- 172.21.15.86:0/2199268389 learned_addr learned my addr 172.21.15.86:0/2199268389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b3400640 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b50bd60 0x55d97b50c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2a00640 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 0x55d97b5556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 -- 172.21.15.86:0/2199268389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 msgr2=0x55d97b5556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 0x55d97b5556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 -- 172.21.15.86:0/2199268389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b50bd60 msgr2=0x55d97b50c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b50bd60 0x55d97b50c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 -- 172.21.15.86:0/2199268389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97b335340 con 0x55d97b555c10 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14b2000640 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b50b820 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f14a00105b0 tx=0x7f14a0059290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca0aae6fde70162 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f149f400640 1 -- 172.21.15.86:0/2199268389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a005da80 con 0x55d97b555c10 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f149f400640 1 -- 172.21.15.86:0/2199268389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a00622b0 con 0x55d97b555c10 2024-03-22T04:16:32.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f149f400640 1 -- 172.21.15.86:0/2199268389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0055fa0 con 0x55d97b555c10 2024-03-22T04:16:32.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2199268389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 msgr2=0x55d97b50b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b50b820 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f14a00105b0 tx=0x7f14a0059290 comp rx=0 tx=0).stop 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2199268389 shutdown_connections 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b50bd60 0x55d97b50c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b50b820 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2199268389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 0x55d97b5556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2199268389 >> 172.21.15.86:0/2199268389 conn(0x55d97b507d10 msgr2=0x55d97b50a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2199268389 shutdown_connections 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2199268389 wait complete. 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- start start 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 Processor -- start 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b50bd60 0x55d97b516280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b5167c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b516d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d97b3184e0 con 0x55d97b555c10 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d97b35f3b0 con 0x55d97b5552f0 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d97b367490 con 0x55d97b50bd60 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.362+0000 7f14bcf5f000 10 client.0 _ll_register_callbacks cb 0x55d97b4f6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b516d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b516d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52354/0 (socket says 172.21.15.86:52354) 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 -- 172.21.15.86:0/2903479776 learned_addr learned my addr 172.21.15.86:0/2903479776 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b2000640 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b5167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b2a00640 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b50bd60 0x55d97b516280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 -- 172.21.15.86:0/2903479776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b50bd60 msgr2=0x55d97b516280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b50bd60 0x55d97b516280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 -- 172.21.15.86:0/2903479776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 msgr2=0x55d97b5167c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b5167c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 -- 172.21.15.86:0/2903479776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97b39f5e0 con 0x55d97b555c10 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b3400640 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b516d00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f14ac01f800 tx=0x7f14ac01c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5d995298853056 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 1 -- 172.21.15.86:0/2903479776 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0a87e0 con 0x55d97b555c10 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 1 -- 172.21.15.86:0/2903479776 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f14ac0a8980 con 0x55d97b555c10 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2903479776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a00622b0 con 0x55d97b555c10 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 1 -- 172.21.15.86:0/2903479776 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f14ac0bf050 con 0x55d97b555c10 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 10 client.7272 fetch_fsmap learned FSMap version 54 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2903479776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55d97b3165a0 con 0x55d97b555c10 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 1 -- 172.21.15.86:0/2903479776 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a00622b0 con 0x55d97b555c10 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 1 -- 172.21.15.86:0/2903479776 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 54) ==== 3494+0+0 (secure 0 0 0) 0x7f14ac0c2020 con 0x55d97b555c10 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 10 client.7272 fetch_fsmap finished waiting for FSMap version 54 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 10 client.7272 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:16:32.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 20 client.7272 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55d97b564590 0x55d97b566980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 4 client.7272 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2903479776 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d97b2f5e40 con 0x55d97b564590 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b2a00640 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55d97b564590 0x55d97b566980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 10 client.7272 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14b2a00640 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55d97b564590 0x55d97b566980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6de56eba2c371ef server_cookie=4d7e8e56296c7cf2 in_seq=0 out_seq=0 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 1 -- 172.21.15.86:0/2903479776 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d97b2f5e40 con 0x55d97b564590 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 10 client.7272 handle_command_reply: tid=0 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 client.7272 shutdown 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2903479776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 msgr2=0x55d97b516d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b516d00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f14ac01f800 tx=0x7f14ac01c770 comp rx=0 tx=0).stop 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2903479776 shutdown_connections 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55d97b564590 0x55d97b566980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b5167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b555c10 0x55d97b516d00 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/2903479776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b50bd60 0x55d97b516280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f1493e00640 0 client.7272 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:32.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2903479776 >> 172.21.15.86:0/2903479776 conn(0x55d97b507d10 msgr2=0x55d97b50a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2903479776 shutdown_connections 2024-03-22T04:16:32.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 1 -- 172.21.15.86:0/2903479776 wait complete. 2024-03-22T04:16:32.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.366+0000 7f14bcf5f000 20 client.7272 trim_cache size 0 max 16384 2024-03-22T04:16:32.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 Processor -- start 2024-03-22T04:16:32.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- start start 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b50b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b50b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b555c10 0x55d97b566980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4200850 con 0x55d97b50bd60 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41ffc00 con 0x55d97b5552f0 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14b4200200 con 0x55d97b555c10 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b50b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b50b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52364/0 (socket says 172.21.15.86:52364) 2024-03-22T04:16:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 -- 172.21.15.86:0/3754697050 learned_addr learned my addr 172.21.15.86:0/3754697050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b3400640 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b555c10 0x55d97b566980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2000640 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b50b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 -- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b555c10 msgr2=0x55d97b566980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b555c10 0x55d97b566980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 -- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 msgr2=0x55d97b50b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b50b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 -- 172.21.15.86:0/3754697050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97b316a50 con 0x55d97b50bd60 2024-03-22T04:16:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2000640 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b50b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:32.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b50b0a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f14a4014d80 tx=0x7f14a401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d4db9511e56126 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f149ea00640 1 -- 172.21.15.86:0/3754697050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4025e20 con 0x55d97b50bd60 2024-03-22T04:16:32.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f149ea00640 1 -- 172.21.15.86:0/3754697050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a4032070 con 0x55d97b50bd60 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f149ea00640 1 -- 172.21.15.86:0/3754697050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a402dc30 con 0x55d97b50bd60 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- 172.21.15.86:0/3754697050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 msgr2=0x55d97b50b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b50b0a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f14a4014d80 tx=0x7f14a401a000 comp rx=0 tx=0).stop 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- 172.21.15.86:0/3754697050 shutdown_connections 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b5552f0 0x55d97b50b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b555c10 0x55d97b566980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/3754697050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b50b0a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- 172.21.15.86:0/3754697050 >> 172.21.15.86:0/3754697050 conn(0x55d97b507d10 msgr2=0x55d97b50a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- 172.21.15.86:0/3754697050 shutdown_connections 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.382+0000 7f14bcf5f000 1 -- 172.21.15.86:0/3754697050 wait complete. 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 -- start start 2024-03-22T04:16:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 Processor -- start 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b568bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 0x55d97b5690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b555c10 0x55d97b5c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d97b35a090 con 0x55d97b50bd60 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d97b301230 con 0x55d97b555c10 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d97b357d00 con 0x55d97b5552f0 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 10 client.0 _ll_register_callbacks cb 0x7f1498003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b568bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b568bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52368/0 (socket says 172.21.15.86:52368) 2024-03-22T04:16:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 -- 172.21.15.86:0/1977303150 learned_addr learned my addr 172.21.15.86:0/1977303150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b3400640 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b555c10 0x55d97b5c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2000640 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 0x55d97b5690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 -- 172.21.15.86:0/1977303150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 msgr2=0x55d97b5690f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 0x55d97b5690f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 -- 172.21.15.86:0/1977303150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b555c10 msgr2=0x55d97b5c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b555c10 0x55d97b5c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 -- 172.21.15.86:0/1977303150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97b3db840 con 0x55d97b50bd60 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2a00640 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b568bb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f14a4014d80 tx=0x7f14a4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eddd2246ea8d3f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f1493e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f1493e00640 1 -- 172.21.15.86:0/1977303150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a402de60 con 0x55d97b50bd60 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f1493e00640 1 -- 172.21.15.86:0/1977303150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f14a402d750 con 0x55d97b50bd60 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 -- 172.21.15.86:0/1977303150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a4032070 con 0x55d97b50bd60 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f1493e00640 1 -- 172.21.15.86:0/1977303150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f14a4046050 con 0x55d97b50bd60 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 10 client.7278 fetch_fsmap learned FSMap version 54 2024-03-22T04:16:32.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 -- 172.21.15.86:0/1977303150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55d97b3013e0 con 0x55d97b50bd60 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f1493e00640 1 -- 172.21.15.86:0/1977303150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a4032070 con 0x55d97b50bd60 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f1493e00640 1 -- 172.21.15.86:0/1977303150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 54) ==== 3494+0+0 (secure 0 0 0) 0x7f14a4049020 con 0x55d97b50bd60 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 10 client.7278 fetch_fsmap finished waiting for FSMap version 54 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 10 client.7278 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 20 client.7278 populate_metadata read hostname 'smithi086' 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55d97b5c1a40 0x55d97b5c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 4 client.7278 mds_command: new command op to 7002 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14bcf5f000 1 -- 172.21.15.86:0/1977303150 --> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55d97b3682c0 con 0x55d97b5c1a40 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.386+0000 7f14b2000640 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55d97b5c1a40 0x55d97b5c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f1493e00640 10 client.7278 ms_handle_connect on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b2000640 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55d97b5c1a40 0x55d97b5c1e20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ce0b3f2b53d378 server_cookie=f35fe6e7ece542d7 in_seq=0 out_seq=0 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f1493e00640 1 -- 172.21.15.86:0/1977303150 <== mds.0 v2:172.21.15.86:6836/736870197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d97b3682c0 con 0x55d97b5c1a40 2024-03-22T04:16:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f1493e00640 10 client.7278 handle_command_reply: tid=0 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 client.7278 shutdown 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 -- 172.21.15.86:0/1977303150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 msgr2=0x55d97b568bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b568bb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f14a4014d80 tx=0x7f14a4004740 comp rx=0 tx=0).stop 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 -- 172.21.15.86:0/1977303150 shutdown_connections 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.86:6836/736870197,v1:172.21.15.86:6837/736870197] conn(0x55d97b5c1a40 0x55d97b5c1e20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d97b555c10 0x55d97b5c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d97b5552f0 0x55d97b5690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f1493e00640 0 client.7278 ms_handle_reset on v2:172.21.15.86:6836/736870197 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 --2- 172.21.15.86:0/1977303150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d97b50bd60 0x55d97b568bb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 -- 172.21.15.86:0/1977303150 >> 172.21.15.86:0/1977303150 conn(0x55d97b507d10 msgr2=0x55d97b50b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 -- 172.21.15.86:0/1977303150 shutdown_connections 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 1 -- 172.21.15.86:0/1977303150 wait complete. 2024-03-22T04:16:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14bcf5f000 20 client.7278 trim_cache size 0 max 16384 2024-03-22T04:16:32.394 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:32.394 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T04:16:32.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 -- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1494067b50 msgr2=0x7f149406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1494067b50 0x7f149406a010 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f14b415e6a0 tx=0x7f14a4008040 comp rx=0 tx=0).stop 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 -- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b414db90 msgr2=0x7f14b40cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b414db90 0x7f14b40cb640 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f14a0012800 tx=0x7f14a0012cc0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 -- 172.21.15.86:0/3343424437 shutdown_connections 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1494067b50 0x7f149406a010 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14b4155aa0 0x7f14b4155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b414db90 0x7f14b40cb640 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 --2- 172.21.15.86:0/3343424437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14980051d0 0x7f14b40c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 -- 172.21.15.86:0/3343424437 >> 172.21.15.86:0/3343424437 conn(0x7f14b40cc940 msgr2=0x7f14b40cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 -- 172.21.15.86:0/3343424437 shutdown_connections 2024-03-22T04:16:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.390+0000 7f14b9a00640 1 -- 172.21.15.86:0/3343424437 wait complete. 2024-03-22T04:16:32.410 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-22T04:16:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.546+0000 7faabf600640 1 Processor -- start 2024-03-22T04:16:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.546+0000 7faabf600640 1 -- start start 2024-03-22T04:16:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabf600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab80d0690 0x7faab815ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabf600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab815f280 con 0x7faab80b3d60 2024-03-22T04:16:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabf600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab815f530 con 0x7faab8147940 2024-03-22T04:16:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabf600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faab815f810 con 0x7faab80d0690 2024-03-22T04:16:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabcc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faab80b3d60 0x7faab80b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51056/0 (socket says 172.21.15.86:51056) 2024-03-22T04:16:32.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabcc00640 1 -- 172.21.15.86:0/2941867202 learned_addr learned my addr 172.21.15.86:0/2941867202 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabd600640 1 --2- 172.21.15.86:0/2941867202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab80d0690 0x7faab815ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faab7e00640 1 -- 172.21.15.86:0/2941867202 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192613835 0 0) 0x7faab815f280 con 0x7faab80b3d60 2024-03-22T04:16:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faab7e00640 1 -- 172.21.15.86:0/2941867202 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa9c003530 con 0x7faab80b3d60 2024-03-22T04:16:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faab7e00640 1 -- 172.21.15.86:0/2941867202 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 625702870 0 0) 0x7faab815f530 con 0x7faab8147940 2024-03-22T04:16:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faab7e00640 1 -- 172.21.15.86:0/2941867202 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faab815f280 con 0x7faab8147940 2024-03-22T04:16:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabd600640 1 -- 172.21.15.86:0/2941867202 >> v1:172.21.15.196:6789/0 conn(0x7faab8147940 legacy=0x7faab815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faab7e00640 1 -- 172.21.15.86:0/2941867202 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2240461990 0 0) 0x7faa9c003530 con 0x7faab80b3d60 2024-03-22T04:16:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabd600640 1 -- 172.21.15.86:0/2941867202 >> v1:172.21.15.86:6789/0 conn(0x7faab80b3d60 legacy=0x7faab80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.550+0000 7faabd600640 1 -- 172.21.15.86:0/2941867202 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab815faf0 con 0x7faab80d0690 2024-03-22T04:16:32.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.554+0000 7faabd600640 1 --2- 172.21.15.86:0/2941867202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab80d0690 0x7faab815ff40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faab0009dc0 tx=0x7faab001c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606f283441b5e631 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.554+0000 7faab7e00640 1 -- 172.21.15.86:0/2941867202 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00b2070 con 0x7faab80d0690 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.554+0000 7faab7e00640 1 -- 172.21.15.86:0/2941867202 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab00a5630 con 0x7faab80d0690 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.554+0000 7faab7e00640 1 -- 172.21.15.86:0/2941867202 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8001680 con 0x7faab80d0690 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.554+0000 7faabf600640 1 -- 172.21.15.86:0/2941867202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab80d0690 msgr2=0x7faab815ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.554+0000 7faabf600640 1 --2- 172.21.15.86:0/2941867202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab80d0690 0x7faab815ff40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faab0009dc0 tx=0x7faab001c7f0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 -- 172.21.15.86:0/2941867202 shutdown_connections 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 --2- 172.21.15.86:0/2941867202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab80d0690 0x7faab815ff40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 -- 172.21.15.86:0/2941867202 >> 172.21.15.86:0/2941867202 conn(0x7faab806fa20 msgr2=0x7faab806fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 -- 172.21.15.86:0/2941867202 shutdown_connections 2024-03-22T04:16:32.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 -- 172.21.15.86:0/2941867202 wait complete. 2024-03-22T04:16:32.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 Processor -- start 2024-03-22T04:16:32.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 -- start start 2024-03-22T04:16:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab80b3d60 0x7faab81570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80d0690 0x7faab80c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab8147940 0x7faab80c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faab816c8a0 con 0x7faab80d0690 2024-03-22T04:16:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faab816c720 con 0x7faab80b3d60 2024-03-22T04:16:32.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabf600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faab816ca20 con 0x7faab8147940 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80d0690 0x7faab80c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80d0690 0x7faab80c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52384/0 (socket says 172.21.15.86:52384) 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 -- 172.21.15.86:0/1574048923 learned_addr learned my addr 172.21.15.86:0/1574048923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabd600640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab8147940 0x7faab80c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabcc00640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab80b3d60 0x7faab81570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 -- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab8147940 msgr2=0x7faab80c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab8147940 0x7faab80c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 -- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab80b3d60 msgr2=0x7faab81570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab80b3d60 0x7faab81570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 -- 172.21.15.86:0/1574048923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab80b5920 con 0x7faab80d0690 2024-03-22T04:16:32.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faabcc00640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab80b3d60 0x7faab81570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:32.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.558+0000 7faaafe00640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80d0690 0x7faab80c5d00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7faaa000c430 tx=0x7faaa000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faab5600640 1 -- 172.21.15.86:0/1574048923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0017070 con 0x7faab80d0690 2024-03-22T04:16:32.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faab5600640 1 -- 172.21.15.86:0/1574048923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa00052f0 con 0x7faab80d0690 2024-03-22T04:16:32.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faab5600640 1 -- 172.21.15.86:0/1574048923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0003840 con 0x7faab80d0690 2024-03-22T04:16:32.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab80b5c00 con 0x7faab80d0690 2024-03-22T04:16:32.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab80b61c0 con 0x7faab80d0690 2024-03-22T04:16:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab80b63c0 con 0x7faab80d0690 2024-03-22T04:16:32.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faab5600640 1 -- 172.21.15.86:0/1574048923 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7faaa001a070 con 0x7faab80d0690 2024-03-22T04:16:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faab5600640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa94067b20 0x7faa94069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faab5600640 1 -- 172.21.15.86:0/1574048923 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7faaa00854a0 con 0x7faab80d0690 2024-03-22T04:16:32.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.562+0000 7faabcc00640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa94067b20 0x7faa94069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.566+0000 7faabcc00640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa94067b20 0x7faa94069fe0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faaa80011f0 tx=0x7faaa8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.566+0000 7faab5600640 1 -- 172.21.15.86:0/1574048923 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7faab80b63c0 con 0x7faab80d0690 2024-03-22T04:16:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.770+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faab8074550 con 0x7faab80d0690 2024-03-22T04:16:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.770+0000 7faab5600640 1 -- 172.21.15.86:0/1574048923 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7faaa00555e0 con 0x7faab80d0690 2024-03-22T04:16:32.771 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:32.771 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T04:16:32.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa94067b20 msgr2=0x7faa94069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa94067b20 0x7faa94069fe0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faaa80011f0 tx=0x7faaa8009040 comp rx=0 tx=0).stop 2024-03-22T04:16:32.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80d0690 msgr2=0x7faab80c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80d0690 0x7faab80c5d00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7faaa000c430 tx=0x7faaa000c8f0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 shutdown_connections 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7faa94067b20 0x7faa94069fe0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faab8147940 0x7faab80c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80d0690 0x7faab80c5d00 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 --2- 172.21.15.86:0/1574048923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faab80b3d60 0x7faab81570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 >> 172.21.15.86:0/1574048923 conn(0x7faab806fa20 msgr2=0x7faab81611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 shutdown_connections 2024-03-22T04:16:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.774+0000 7faabf600640 1 -- 172.21.15.86:0/1574048923 wait complete. 2024-03-22T04:16:32.794 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-22T04:16:32.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:16:32.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f429da00640 1 Processor -- start 2024-03-22T04:16:32.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f429da00640 1 -- start start 2024-03-22T04:16:32.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f429da00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 0x7f42980cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f429da00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f429815ead0 0x7f429806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f429da00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429815f090 con 0x7f429806b660 2024-03-22T04:16:32.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f429da00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f429815f370 con 0x7f429815ead0 2024-03-22T04:16:32.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f429da00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f429815f650 con 0x7f42980d0630 2024-03-22T04:16:32.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4297400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f429806b660 0x7f42980d1320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:38668/0 (socket says 172.21.15.86:38668) 2024-03-22T04:16:32.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4297400640 1 -- 172.21.15.86:0/2611163092 learned_addr learned my addr 172.21.15.86:0/2611163092 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:32.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4296a00640 1 --2- 172.21.15.86:0/2611163092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 0x7f42980cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4296000640 1 --2- 172.21.15.86:0/2611163092 >> v2:172.21.15.196:3300/0 conn(0x7f429815ead0 0x7f429806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4295600640 1 -- 172.21.15.86:0/2611163092 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042823782 0 0) 0x7f429815f090 con 0x7f429806b660 2024-03-22T04:16:32.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4295600640 1 -- 172.21.15.86:0/2611163092 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f427c003530 con 0x7f429806b660 2024-03-22T04:16:32.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4296000640 1 -- 172.21.15.86:0/2611163092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 msgr2=0x7f42980cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:32.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4296000640 1 --2- 172.21.15.86:0/2611163092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 0x7f42980cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4296000640 1 -- 172.21.15.86:0/2611163092 >> v1:172.21.15.196:6789/0 conn(0x7f429806b660 legacy=0x7f42980d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.930+0000 7f4296000640 1 -- 172.21.15.86:0/2611163092 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429815f930 con 0x7f429815ead0 2024-03-22T04:16:32.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.934+0000 7f4296000640 1 --2- 172.21.15.86:0/2611163092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429815ead0 0x7f429806e8b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f42840097c0 tx=0x7f428401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee8faabaf9e8d1e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f4295600640 1 -- 172.21.15.86:0/2611163092 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840aa620 con 0x7f429815ead0 2024-03-22T04:16:32.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f4295600640 1 -- 172.21.15.86:0/2611163092 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42840af070 con 0x7f429815ead0 2024-03-22T04:16:32.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f4295600640 1 -- 172.21.15.86:0/2611163092 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840aa9e0 con 0x7f429815ead0 2024-03-22T04:16:32.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 -- 172.21.15.86:0/2611163092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429815ead0 msgr2=0x7f429806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 --2- 172.21.15.86:0/2611163092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429815ead0 0x7f429806e8b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f42840097c0 tx=0x7f428401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 -- 172.21.15.86:0/2611163092 shutdown_connections 2024-03-22T04:16:32.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 --2- 172.21.15.86:0/2611163092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429815ead0 0x7f429806e8b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 --2- 172.21.15.86:0/2611163092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 0x7f42980cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 -- 172.21.15.86:0/2611163092 >> 172.21.15.86:0/2611163092 conn(0x7f429806c770 msgr2=0x7f429806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:32.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 -- 172.21.15.86:0/2611163092 shutdown_connections 2024-03-22T04:16:32.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 -- 172.21.15.86:0/2611163092 wait complete. 2024-03-22T04:16:32.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.938+0000 7f429da00640 1 Processor -- start 2024-03-22T04:16:32.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 -- start start 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429806b660 0x7f4298158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 0x7f42980c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f429815ead0 0x7f42980af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f429806be80 con 0x7f429815ead0 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f429806edf0 con 0x7f429806b660 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f42980b4080 con 0x7f42980d0630 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f429815ead0 0x7f42980af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f429815ead0 0x7f42980af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52396/0 (socket says 172.21.15.86:52396) 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 -- 172.21.15.86:0/4033803157 learned_addr learned my addr 172.21.15.86:0/4033803157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4296a00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429806b660 0x7f4298158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4296000640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 0x7f42980c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 -- 172.21.15.86:0/4033803157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 msgr2=0x7f42980c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 0x7f42980c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 -- 172.21.15.86:0/4033803157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429806b660 msgr2=0x7f4298158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429806b660 0x7f4298158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 -- 172.21.15.86:0/4033803157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42980b3790 con 0x7f429815ead0 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f4297400640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f429815ead0 0x7f42980af3c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f429000cad0 tx=0x7f429000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f428ea00640 1 -- 172.21.15.86:0/4033803157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290017070 con 0x7f429815ead0 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f428ea00640 1 -- 172.21.15.86:0/4033803157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429000ac30 con 0x7f429815ead0 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f428ea00640 1 -- 172.21.15.86:0/4033803157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290012400 con 0x7f429815ead0 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42980b3a30 con 0x7f429815ead0 2024-03-22T04:16:32.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4298158d60 con 0x7f429815ead0 2024-03-22T04:16:32.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.942+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4298068650 con 0x7f429815ead0 2024-03-22T04:16:32.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.946+0000 7f428ea00640 1 -- 172.21.15.86:0/4033803157 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f429001a070 con 0x7f429815ead0 2024-03-22T04:16:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.946+0000 7f428ea00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4278067b20 0x7f4278069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.946+0000 7f428ea00640 1 -- 172.21.15.86:0/4033803157 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f4290085eb0 con 0x7f429815ead0 2024-03-22T04:16:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.946+0000 7f4296a00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4278067b20 0x7f4278069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:32.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.946+0000 7f4296a00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4278067b20 0x7f4278069fe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f429815c220 tx=0x7f4288009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:32.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:32.950+0000 7f428ea00640 1 -- 172.21.15.86:0/4033803157 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4290055ff0 con 0x7f429815ead0 2024-03-22T04:16:33.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.138+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f42980ce250 con 0x7f429815ead0 2024-03-22T04:16:33.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f428ea00640 1 -- 172.21.15.86:0/4033803157 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7f429005a090 con 0x7f429815ead0 2024-03-22T04:16:33.141 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:16:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4278067b20 msgr2=0x7f4278069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4278067b20 0x7f4278069fe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f429815c220 tx=0x7f4288009190 comp rx=0 tx=0).stop 2024-03-22T04:16:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f429815ead0 msgr2=0x7f42980af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f429815ead0 0x7f42980af3c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f429000cad0 tx=0x7f429000cf90 comp rx=0 tx=0).stop 2024-03-22T04:16:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 shutdown_connections 2024-03-22T04:16:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4278067b20 0x7f4278069fe0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:33.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f429815ead0 0x7f42980af3c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42980d0630 0x7f42980c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 --2- 172.21.15.86:0/4033803157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f429806b660 0x7f4298158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 >> 172.21.15.86:0/4033803157 conn(0x7f429806c770 msgr2=0x7f429815c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 shutdown_connections 2024-03-22T04:16:33.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33.142+0000 7f429da00640 1 -- 172.21.15.86:0/4033803157 wait complete. 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:16:33.162 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:16:33.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:33.163 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:16:33.172 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42139sec preferred_lft 42139sec 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:16:33.173 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:16:33.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:33.174 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:16:33.233 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:16:33.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:33.235 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:16:33.287 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:16:33.287 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:16:33.287 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:16:33.287 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:16:33.287 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:16:33.287 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:16:33.288 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:16:33.288 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:16:33.288 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:16:33.288 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:16:33.288 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:16:33.288 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:16:33.288 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:16:33.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:16:33.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:16:33.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:16:33.288 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:16:33.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:16:33.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:16:33.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:33.421 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:16:33.430 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:16:33.431 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:16:33.431 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:16:33.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:33.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:16:33.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:16:33.492 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:16:33.544 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:16:33.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:16:33.544 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:16:33.595 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:16:33.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:33.595 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:16:33.687 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:16:33.688 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:16:33.688 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:16:33.703 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:16:33.703 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:16:33.703 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:16:33.703 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:16:33.703 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:16:33.714 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:16:33.714 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:16:33.714 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:16:33.714 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:16:33.714 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:16:33.715 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:16:33.715 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:16:33.715 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:16:33.718 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:16:33.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:33.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:16:33.779 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:16:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:16:33.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:16:33.906 INFO:teuthology.orchestra.run.smithi086.stdout:client7123 2024-03-22T04:16:33.906 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:16:33.907 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:33.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:33.915 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-22T04:16:33.966 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:16:33.966 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-22T04:16:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.054+0000 7f739b42a080 1 Processor -- start 2024-03-22T04:16:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.054+0000 7f739b42a080 1 -- start start 2024-03-22T04:16:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.054+0000 7f739b42a080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae68802cc0 0x55ae68802160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.054+0000 7f739b42a080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.054+0000 7f739b42a080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae688a7830 0x55ae688a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.054+0000 7f739b42a080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae688a8a60 con 0x55ae688a7450 2024-03-22T04:16:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.054+0000 7f739b42a080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae68778620 con 0x55ae68802cc0 2024-03-22T04:16:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.054+0000 7f739b42a080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ae688a8be0 con 0x55ae688a7830 2024-03-22T04:16:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52406/0 (socket says 172.21.15.86:52406) 2024-03-22T04:16:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 -- 172.21.15.86:0/349669527 learned_addr learned my addr 172.21.15.86:0/349669527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7398600640 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae68802cc0 0x55ae68802160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7399000640 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae688a7830 0x55ae688a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 -- 172.21.15.86:0/349669527 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae688a7830 msgr2=0x55ae688a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae688a7830 0x55ae688a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 -- 172.21.15.86:0/349669527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae68802cc0 msgr2=0x55ae68802160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae68802cc0 0x55ae68802160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 -- 172.21.15.86:0/349669527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae68802870 con 0x55ae688a7450 2024-03-22T04:16:34.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7397c00640 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d5bc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7384009920 tx=0x7f7384005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4c5f1720bff8d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7396800640 1 -- 172.21.15.86:0/349669527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384008d40 con 0x55ae688a7450 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7396800640 1 -- 172.21.15.86:0/349669527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7384018070 con 0x55ae688a7450 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f7396800640 1 -- 172.21.15.86:0/349669527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384013640 con 0x55ae688a7450 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- 172.21.15.86:0/349669527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 msgr2=0x55ae687d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d5bc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7384009920 tx=0x7f7384005a10 comp rx=0 tx=0).stop 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- 172.21.15.86:0/349669527 shutdown_connections 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae688a7830 0x55ae688a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d5bc0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 --2- 172.21.15.86:0/349669527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae68802cc0 0x55ae68802160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- 172.21.15.86:0/349669527 >> 172.21.15.86:0/349669527 conn(0x55ae687feeb0 msgr2=0x55ae6881c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- 172.21.15.86:0/349669527 shutdown_connections 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- 172.21.15.86:0/349669527 wait complete. 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 Processor -- start 2024-03-22T04:16:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- start start 2024-03-22T04:16:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae68802cc0 0x55ae688a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae688a7830 0x55ae687d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae688a5810 con 0x55ae688a7450 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ae688a5570 con 0x55ae688a7830 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.058+0000 7f739b42a080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ae688a5ab0 con 0x55ae68802cc0 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52416/0 (socket says 172.21.15.86:52416) 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 -- 172.21.15.86:0/2312813985 learned_addr learned my addr 172.21.15.86:0/2312813985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7398600640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae68802cc0 0x55ae688a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7399000640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae688a7830 0x55ae687d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 -- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae68802cc0 msgr2=0x55ae688a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ae68802cc0 0x55ae688a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 -- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae688a7830 msgr2=0x55ae687d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ae688a7830 0x55ae687d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 -- 172.21.15.86:0/2312813985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae687d2e30 con 0x55ae688a7450 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7397c00640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d2550 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f73840136d0 tx=0x7f7384003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7395400640 1 -- 172.21.15.86:0/2312813985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384013ac0 con 0x55ae688a7450 2024-03-22T04:16:34.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7395400640 1 -- 172.21.15.86:0/2312813985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7384021030 con 0x55ae688a7450 2024-03-22T04:16:34.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ae687d30d0 con 0x55ae688a7450 2024-03-22T04:16:34.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7395400640 1 -- 172.21.15.86:0/2312813985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384026420 con 0x55ae688a7450 2024-03-22T04:16:34.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae687d3610 con 0x55ae688a7450 2024-03-22T04:16:34.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.062+0000 7f7395400640 1 -- 172.21.15.86:0/2312813985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f738401c420 con 0x55ae688a7450 2024-03-22T04:16:34.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f7395400640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7378066530 0x7f73780689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f7395400640 1 -- 172.21.15.86:0/2312813985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f7384086df0 con 0x55ae688a7450 2024-03-22T04:16:34.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f7398600640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7378066530 0x7f73780689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55ae687d4290 0x55ae687d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81) -- 0x55ae688ad4a0 con 0x55ae687d4290 2024-03-22T04:16:34.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f7399000640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55ae687d4290 0x55ae687d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f7398600640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7378066530 0x7f73780689f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7388011570 tx=0x7f73880017a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f7399000640 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55ae687d4290 0x55ae687d4df0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f7399000640 1 -- 172.21.15.86:0/2312813985 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v81'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f73900ab6f0 con 0x55ae687d4290 2024-03-22T04:16:34.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55ae687d4290 msgr2=0x55ae687d4df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55ae687d4290 0x55ae687d4df0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7378066530 msgr2=0x7f73780689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7378066530 0x7f73780689f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7388011570 tx=0x7f73880017a0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 msgr2=0x55ae687d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 --2- 172.21.15.86:0/2312813985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae688a7450 0x55ae687d2550 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f73840136d0 tx=0x7f7384003030 comp rx=0 tx=0).stop 2024-03-22T04:16:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f7399000640 1 -- 172.21.15.86:0/2312813985 reap_dead start 2024-03-22T04:16:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 shutdown_connections 2024-03-22T04:16:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 >> 172.21.15.86:0/2312813985 conn(0x55ae687feeb0 msgr2=0x55ae68813210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 shutdown_connections 2024-03-22T04:16:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.066+0000 7f739b42a080 1 -- 172.21.15.86:0/2312813985 wait complete. 2024-03-22T04:16:34.072 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:16:34.072 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:16:34.183 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:16:34.247 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:16:34.248 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:16:34.248 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:16:34.248 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:34.256 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:16:34.256 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:34.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:16:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.486+0000 7f7090a00640 1 Processor -- start 2024-03-22T04:16:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.486+0000 7f7090a00640 1 -- start start 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7090a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f708c15e630 0x7f708c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7090a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c06e2f0 0x7f708c0d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7090a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f708c06cc80 0x7f708c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7090a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f708c1623d0 con 0x7f708c15e630 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7090a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f708c1620f0 con 0x7f708c06cc80 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7090a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f708c15ec10 con 0x7f708c06e2f0 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f708c15e630 0x7f708c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f708c15e630 0x7f708c0cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52428/0 (socket says 172.21.15.86:52428) 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089e00640 1 -- 172.21.15.86:0/3851890389 learned_addr learned my addr 172.21.15.86:0/3851890389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089400640 1 --2- 172.21.15.86:0/3851890389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c06e2f0 0x7f708c0d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f708a800640 1 --2- 172.21.15.86:0/3851890389 >> v2:172.21.15.196:3300/0 conn(0x7f708c06cc80 0x7f708c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089e00640 1 -- 172.21.15.86:0/3851890389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c06e2f0 msgr2=0x7f708c0d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:34.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089e00640 1 --2- 172.21.15.86:0/3851890389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c06e2f0 0x7f708c0d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089e00640 1 -- 172.21.15.86:0/3851890389 >> v2:172.21.15.196:3300/0 conn(0x7f708c06cc80 msgr2=0x7f708c06d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:34.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089e00640 1 --2- 172.21.15.86:0/3851890389 >> v2:172.21.15.196:3300/0 conn(0x7f708c06cc80 0x7f708c06d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089e00640 1 -- 172.21.15.86:0/3851890389 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c15eef0 con 0x7f708c15e630 2024-03-22T04:16:34.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.490+0000 7f7089400640 1 --2- 172.21.15.86:0/3851890389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c06e2f0 0x7f708c0d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:34.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7089e00640 1 --2- 172.21.15.86:0/3851890389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c15e630 0x7f708c0cfe70 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f708c0d32b0 tx=0x7f707c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a140a3b030751378 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7088a00640 1 -- 172.21.15.86:0/3851890389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0b0070 con 0x7f708c15e630 2024-03-22T04:16:34.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7088a00640 1 -- 172.21.15.86:0/3851890389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707c0ab600 con 0x7f708c15e630 2024-03-22T04:16:34.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7088a00640 1 -- 172.21.15.86:0/3851890389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0ab9d0 con 0x7f708c15e630 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- 172.21.15.86:0/3851890389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c15e630 msgr2=0x7f708c0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 --2- 172.21.15.86:0/3851890389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c15e630 0x7f708c0cfe70 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f708c0d32b0 tx=0x7f707c01c520 comp rx=0 tx=0).stop 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- 172.21.15.86:0/3851890389 shutdown_connections 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 --2- 172.21.15.86:0/3851890389 >> v2:172.21.15.196:3300/0 conn(0x7f708c06cc80 0x7f708c06d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 --2- 172.21.15.86:0/3851890389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c06e2f0 0x7f708c0d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 --2- 172.21.15.86:0/3851890389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c15e630 0x7f708c0cfe70 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- 172.21.15.86:0/3851890389 >> 172.21.15.86:0/3851890389 conn(0x7f708c06f910 msgr2=0x7f708c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- 172.21.15.86:0/3851890389 shutdown_connections 2024-03-22T04:16:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- 172.21.15.86:0/3851890389 wait complete. 2024-03-22T04:16:34.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 Processor -- start 2024-03-22T04:16:34.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- start start 2024-03-22T04:16:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708c06cc80 0x7f708c153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c06e2f0 0x7f708c0cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c15e630 0x7f708c0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f708c15f9d0 con 0x7f708c06e2f0 2024-03-22T04:16:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f708c0d1f60 con 0x7f708c06cc80 2024-03-22T04:16:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.494+0000 7f7090a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f708c0cfaf0 con 0x7f708c15e630 2024-03-22T04:16:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c06e2f0 0x7f708c0cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c06e2f0 0x7f708c0cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52434/0 (socket says 172.21.15.86:52434) 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 -- 172.21.15.86:0/228099366 learned_addr learned my addr 172.21.15.86:0/228099366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089e00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708c06cc80 0x7f708c153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f708a800640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c15e630 0x7f708c0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 -- 172.21.15.86:0/228099366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c15e630 msgr2=0x7f708c0c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c15e630 0x7f708c0c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 -- 172.21.15.86:0/228099366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708c06cc80 msgr2=0x7f708c153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708c06cc80 0x7f708c153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 -- 172.21.15.86:0/228099366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c161360 con 0x7f708c06e2f0 2024-03-22T04:16:34.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7089400640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c06e2f0 0x7f708c0cae40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f707800d800 tx=0x7f707800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7082000640 1 -- 172.21.15.86:0/228099366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078018020 con 0x7f708c06e2f0 2024-03-22T04:16:34.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7082000640 1 -- 172.21.15.86:0/228099366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70780096c0 con 0x7f708c06e2f0 2024-03-22T04:16:34.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c161640 con 0x7f708c06e2f0 2024-03-22T04:16:34.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7082000640 1 -- 172.21.15.86:0/228099366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078009ab0 con 0x7f708c06e2f0 2024-03-22T04:16:34.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c161c00 con 0x7f708c06e2f0 2024-03-22T04:16:34.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c068650 con 0x7f708c06e2f0 2024-03-22T04:16:34.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7082000640 1 -- 172.21.15.86:0/228099366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f707801b020 con 0x7f708c06e2f0 2024-03-22T04:16:34.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.498+0000 7f7082000640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f706c067b20 0x7f706c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.502+0000 7f7082000640 1 -- 172.21.15.86:0/228099366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f7078086560 con 0x7f708c06e2f0 2024-03-22T04:16:34.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.502+0000 7f7089e00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f706c067b20 0x7f706c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.502+0000 7f7089e00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f706c067b20 0x7f706c069fe0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f707c0ad000 tx=0x7f707c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.506+0000 7f7082000640 1 -- 172.21.15.86:0/228099366 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7078022020 con 0x7f708c06e2f0 2024-03-22T04:16:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.662+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f708c1552b0 con 0x7f708c06e2f0 2024-03-22T04:16:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7082000640 1 -- 172.21.15.86:0/228099366 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+17014 (secure 0 0 0) 0x7f7078056730 con 0x7f708c06e2f0 2024-03-22T04:16:34.665 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:34.666 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:20.393125+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1130704586":"2024-03-23T04:15:20.393105+0000","172.21.15.86:6836/1130704586":"2024-03-23T04:15:20.393105+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:16:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f706c067b20 msgr2=0x7f706c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f706c067b20 0x7f706c069fe0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f707c0ad000 tx=0x7f707c008040 comp rx=0 tx=0).stop 2024-03-22T04:16:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c06e2f0 msgr2=0x7f708c0cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c06e2f0 0x7f708c0cae40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f707800d800 tx=0x7f707800dcc0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 shutdown_connections 2024-03-22T04:16:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f706c067b20 0x7f706c069fe0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f708c15e630 0x7f708c0c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f708c06e2f0 0x7f708c0cae40 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 --2- 172.21.15.86:0/228099366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f708c06cc80 0x7f708c153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 >> 172.21.15.86:0/228099366 conn(0x7f708c06f910 msgr2=0x7f708c0cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 shutdown_connections 2024-03-22T04:16:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.666+0000 7f7090a00640 1 -- 172.21.15.86:0/228099366 wait complete. 2024-03-22T04:16:34.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:15:07.783245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:15:08.394997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:16:34.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:16:34.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.834+0000 7f9c9e600640 1 Processor -- start 2024-03-22T04:16:34.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.834+0000 7f9c9e600640 1 -- start start 2024-03-22T04:16:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.834+0000 7f9c9e600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9c9815c730 0x7f9c9809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.834+0000 7f9c9e600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c980cda00 con 0x7f9c980b11c0 2024-03-22T04:16:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.834+0000 7f9c9e600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c980cdce0 con 0x7f9c980b3610 2024-03-22T04:16:34.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.834+0000 7f9c9e600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9c9809b910 con 0x7f9c9815c730 2024-03-22T04:16:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c97e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9c980b11c0 0x7f9c980b1ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51058/0 (socket says 172.21.15.86:51058) 2024-03-22T04:16:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c97e00640 1 -- 172.21.15.86:0/461708781 learned_addr learned my addr 172.21.15.86:0/461708781 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c97400640 1 --2- 172.21.15.86:0/461708781 >> v2:172.21.15.196:3300/0 conn(0x7f9c9815c730 0x7f9c9809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c96a00640 1 -- 172.21.15.86:0/461708781 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092074057 0 0) 0x7f9c980cda00 con 0x7f9c980b11c0 2024-03-22T04:16:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c96a00640 1 -- 172.21.15.86:0/461708781 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c7c003530 con 0x7f9c980b11c0 2024-03-22T04:16:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c97400640 1 -- 172.21.15.86:0/461708781 >> v1:172.21.15.196:6789/0 conn(0x7f9c980b3610 legacy=0x7f9c9809afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c97400640 1 -- 172.21.15.86:0/461708781 >> v1:172.21.15.86:6789/0 conn(0x7f9c980b11c0 legacy=0x7f9c980b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c97400640 1 -- 172.21.15.86:0/461708781 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c980cdfc0 con 0x7f9c9815c730 2024-03-22T04:16:34.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c97400640 1 --2- 172.21.15.86:0/461708781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9815c730 0x7f9c9809aa70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c880097f0 tx=0x7f9c8801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9105980ec87001a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.838+0000 7f9c96a00640 1 -- 172.21.15.86:0/461708781 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c880b0070 con 0x7f9c9815c730 2024-03-22T04:16:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c96a00640 1 -- 172.21.15.86:0/461708781 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c880ab600 con 0x7f9c9815c730 2024-03-22T04:16:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c96a00640 1 -- 172.21.15.86:0/461708781 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c880ab9d0 con 0x7f9c9815c730 2024-03-22T04:16:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 -- 172.21.15.86:0/461708781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9815c730 msgr2=0x7f9c9809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 --2- 172.21.15.86:0/461708781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9815c730 0x7f9c9809aa70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c880097f0 tx=0x7f9c8801c410 comp rx=0 tx=0).stop 2024-03-22T04:16:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 -- 172.21.15.86:0/461708781 shutdown_connections 2024-03-22T04:16:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 --2- 172.21.15.86:0/461708781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9815c730 0x7f9c9809aa70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 -- 172.21.15.86:0/461708781 >> 172.21.15.86:0/461708781 conn(0x7f9c980b3bf0 msgr2=0x7f9c980b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 -- 172.21.15.86:0/461708781 shutdown_connections 2024-03-22T04:16:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 -- 172.21.15.86:0/461708781 wait complete. 2024-03-22T04:16:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 Processor -- start 2024-03-22T04:16:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.842+0000 7f9c9e600640 1 -- start start 2024-03-22T04:16:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c980b11c0 0x7f9c980b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c980b3610 0x7f9c980b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9815c730 0x7f9c980c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c980b2e90 con 0x7f9c980b3610 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c980b2d10 con 0x7f9c980b11c0 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9809b6c0 con 0x7f9c9815c730 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c980b3610 0x7f9c980b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c980b3610 0x7f9c980b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52440/0 (socket says 172.21.15.86:52440) 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 -- 172.21.15.86:0/3638157508 learned_addr learned my addr 172.21.15.86:0/3638157508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c97e00640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9815c730 0x7f9c980c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c97400640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c980b11c0 0x7f9c980b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 -- 172.21.15.86:0/3638157508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9815c730 msgr2=0x7f9c980c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9815c730 0x7f9c980c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 -- 172.21.15.86:0/3638157508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c980b11c0 msgr2=0x7f9c980b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c980b11c0 0x7f9c980b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 -- 172.21.15.86:0/3638157508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c980c5e80 con 0x7f9c980b3610 2024-03-22T04:16:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8fe00640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c980b3610 0x7f9c980b7980 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c3a0 tx=0x7f9c8000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8f400640 1 -- 172.21.15.86:0/3638157508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c800052d0 con 0x7f9c980b3610 2024-03-22T04:16:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8f400640 1 -- 172.21.15.86:0/3638157508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c80005470 con 0x7f9c980b3610 2024-03-22T04:16:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c8f400640 1 -- 172.21.15.86:0/3638157508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80007260 con 0x7f9c980b3610 2024-03-22T04:16:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c980c6160 con 0x7f9c980b3610 2024-03-22T04:16:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c980c6720 con 0x7f9c980b3610 2024-03-22T04:16:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.846+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c980c6920 con 0x7f9c980b3610 2024-03-22T04:16:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.850+0000 7f9c8f400640 1 -- 172.21.15.86:0/3638157508 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9c80007400 con 0x7f9c980b3610 2024-03-22T04:16:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.850+0000 7f9c8f400640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c74067b20 0x7f9c74069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.850+0000 7f9c8f400640 1 -- 172.21.15.86:0/3638157508 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f9c80085580 con 0x7f9c980b3610 2024-03-22T04:16:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.850+0000 7f9c97400640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c74067b20 0x7f9c74069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:34.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.850+0000 7f9c97400640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c74067b20 0x7f9c74069fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9c880ad000 tx=0x7f9c880a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:34.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:34.854+0000 7f9c8f400640 1 -- 172.21.15.86:0/3638157508 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9c800556c0 con 0x7f9c980b3610 2024-03-22T04:16:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c98074470 con 0x7f9c980b3610 2024-03-22T04:16:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c8f400640 1 -- 172.21.15.86:0/3638157508 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f9c80059760 con 0x7f9c980b3610 2024-03-22T04:16:35.018 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c74067b20 msgr2=0x7f9c74069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c74067b20 0x7f9c74069fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9c880ad000 tx=0x7f9c880a6000 comp rx=0 tx=0).stop 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c980b3610 msgr2=0x7f9c980b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c980b3610 0x7f9c980b7980 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c3a0 tx=0x7f9c8000c860 comp rx=0 tx=0).stop 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 shutdown_connections 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c74067b20 0x7f9c74069fe0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9815c730 0x7f9c980c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c980b3610 0x7f9c980b7980 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 --2- 172.21.15.86:0/3638157508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c980b11c0 0x7f9c980b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 >> 172.21.15.86:0/3638157508 conn(0x7f9c980b3bf0 msgr2=0x7f9c9809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 shutdown_connections 2024-03-22T04:16:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.018+0000 7f9c9e600640 1 -- 172.21.15.86:0/3638157508 wait complete. 2024-03-22T04:16:35.039 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:16:35.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d49200640 1 Processor -- start 2024-03-22T04:16:35.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d49200640 1 -- start start 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d49200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5d4415e970 0x7f5d440b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d49200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5d4409b180 0x7f5d440cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d49200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4406dd70 con 0x7f5d4406d8f0 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d49200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5d440b49a0 con 0x7f5d4415e970 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d49200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5d440ccda0 con 0x7f5d4409b180 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d41e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5d4415e970 0x7f5d440b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d41400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5d4406d8f0 0x7f5d4409aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51066/0 (socket says 172.21.15.86:51066) 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d41400640 1 -- 172.21.15.86:0/3646834395 learned_addr learned my addr 172.21.15.86:0/3646834395 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.178+0000 7f5d42800640 1 --2- 172.21.15.86:0/3646834395 >> v2:172.21.15.196:3300/0 conn(0x7f5d4409b180 0x7f5d440cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d40a00640 1 -- 172.21.15.86:0/3646834395 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930911896 0 0) 0x7f5d30001120 con 0x7f5d4406d8f0 2024-03-22T04:16:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d41e00640 1 -- 172.21.15.86:0/3646834395 >> v2:172.21.15.196:3300/0 conn(0x7f5d4409b180 msgr2=0x7f5d440cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d41e00640 1 --2- 172.21.15.86:0/3646834395 >> v2:172.21.15.196:3300/0 conn(0x7f5d4409b180 0x7f5d440cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d41e00640 1 -- 172.21.15.86:0/3646834395 >> v1:172.21.15.86:6789/0 conn(0x7f5d4406d8f0 legacy=0x7f5d4409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d41e00640 1 -- 172.21.15.86:0/3646834395 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4406dff0 con 0x7f5d4415e970 2024-03-22T04:16:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d42800640 1 --2- 172.21.15.86:0/3646834395 >> v2:172.21.15.196:3300/0 conn(0x7f5d4409b180 0x7f5d440cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d41e00640 1 --2- 172.21.15.86:0/3646834395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4415e970 0x7f5d440b12e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5d44073160 tx=0x7f5d3401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939a87bb3e2291a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d40a00640 1 -- 172.21.15.86:0/3646834395 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340ab650 con 0x7f5d4415e970 2024-03-22T04:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d40a00640 1 -- 172.21.15.86:0/3646834395 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d340b0070 con 0x7f5d4415e970 2024-03-22T04:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d40a00640 1 -- 172.21.15.86:0/3646834395 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340aba10 con 0x7f5d4415e970 2024-03-22T04:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d49200640 1 -- 172.21.15.86:0/3646834395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4415e970 msgr2=0x7f5d440b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d49200640 1 --2- 172.21.15.86:0/3646834395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4415e970 0x7f5d440b12e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5d44073160 tx=0x7f5d3401c450 comp rx=0 tx=0).stop 2024-03-22T04:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d49200640 1 -- 172.21.15.86:0/3646834395 shutdown_connections 2024-03-22T04:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d49200640 1 --2- 172.21.15.86:0/3646834395 >> v2:172.21.15.196:3300/0 conn(0x7f5d4409b180 0x7f5d440cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d49200640 1 --2- 172.21.15.86:0/3646834395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4415e970 0x7f5d440b12e0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d49200640 1 -- 172.21.15.86:0/3646834395 >> 172.21.15.86:0/3646834395 conn(0x7f5d440b3bf0 msgr2=0x7f5d440b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.182+0000 7f5d49200640 1 -- 172.21.15.86:0/3646834395 shutdown_connections 2024-03-22T04:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.186+0000 7f5d49200640 1 -- 172.21.15.86:0/3646834395 wait complete. 2024-03-22T04:16:35.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.186+0000 7f5d49200640 1 Processor -- start 2024-03-22T04:16:35.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.186+0000 7f5d49200640 1 -- start start 2024-03-22T04:16:35.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4406d8f0 0x7f5d440af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4409b180 0x7f5d44151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4415e970 0x7f5d440b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440cdf80 con 0x7f5d4409b180 2024-03-22T04:16:35.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4415ed50 con 0x7f5d4406d8f0 2024-03-22T04:16:35.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5d440ce100 con 0x7f5d4415e970 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4409b180 0x7f5d44151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4409b180 0x7f5d44151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52456/0 (socket says 172.21.15.86:52456) 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 -- 172.21.15.86:0/3729463590 learned_addr learned my addr 172.21.15.86:0/3729463590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41e00640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4406d8f0 0x7f5d440af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d42800640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4415e970 0x7f5d440b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 -- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4415e970 msgr2=0x7f5d440b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4415e970 0x7f5d440b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 -- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4406d8f0 msgr2=0x7f5d440af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4406d8f0 0x7f5d440af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 -- 172.21.15.86:0/3729463590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d440aff00 con 0x7f5d4409b180 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d42800640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4415e970 0x7f5d440b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d41400640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4409b180 0x7f5d44151930 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c780 tx=0x7f5d3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d3a000640 1 -- 172.21.15.86:0/3729463590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30018070 con 0x7f5d4409b180 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d3a000640 1 -- 172.21.15.86:0/3729463590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d300072f0 con 0x7f5d4409b180 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d441558b0 con 0x7f5d4409b180 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d3a000640 1 -- 172.21.15.86:0/3729463590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d300047f0 con 0x7f5d4409b180 2024-03-22T04:16:35.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d44155e70 con 0x7f5d4409b180 2024-03-22T04:16:35.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.190+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d44074470 con 0x7f5d4409b180 2024-03-22T04:16:35.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.194+0000 7f5d3a000640 1 -- 172.21.15.86:0/3729463590 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5d3001b070 con 0x7f5d4409b180 2024-03-22T04:16:35.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.194+0000 7f5d3a000640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d24067b20 0x7f5d24069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.194+0000 7f5d3a000640 1 -- 172.21.15.86:0/3729463590 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f5d30085730 con 0x7f5d4409b180 2024-03-22T04:16:35.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.194+0000 7f5d41e00640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d24067b20 0x7f5d24069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.194+0000 7f5d41e00640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d24067b20 0x7f5d24069fe0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5d340af000 tx=0x7f5d340a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.198+0000 7f5d3a000640 1 -- 172.21.15.86:0/3729463590 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5d440aff00 con 0x7f5d4409b180 2024-03-22T04:16:35.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.350+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5d440afb80 con 0x7f5d4409b180 2024-03-22T04:16:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.350+0000 7f5d3a000640 1 -- 172.21.15.86:0/3729463590 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f5d4406dd70 con 0x7f5d4409b180 2024-03-22T04:16:35.352 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d24067b20 msgr2=0x7f5d24069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d24067b20 0x7f5d24069fe0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5d340af000 tx=0x7f5d340a7000 comp rx=0 tx=0).stop 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4409b180 msgr2=0x7f5d44151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4409b180 0x7f5d44151930 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c780 tx=0x7f5d3000cc40 comp rx=0 tx=0).stop 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 shutdown_connections 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d24067b20 0x7f5d24069fe0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4415e970 0x7f5d440b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4409b180 0x7f5d44151930 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 --2- 172.21.15.86:0/3729463590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4406d8f0 0x7f5d440af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 >> 172.21.15.86:0/3729463590 conn(0x7f5d440b3bf0 msgr2=0x7f5d440b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 shutdown_connections 2024-03-22T04:16:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.354+0000 7f5d49200640 1 -- 172.21.15.86:0/3729463590 wait complete. 2024-03-22T04:16:35.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:16:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.510+0000 7ff6b8800640 1 Processor -- start 2024-03-22T04:16:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.510+0000 7ff6b8800640 1 -- start start 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b8800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b009b180 0x7ff6b00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b8800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6b006d8f0 0x7ff6b00cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b8800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6b006dff0 con 0x7ff6b015e970 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b8800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b00b49a0 con 0x7ff6b006d8f0 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b8800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b00ccda0 con 0x7ff6b009b180 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6b006d8f0 0x7ff6b00cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6b006d8f0 0x7ff6b00cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52468/0 (socket says 172.21.15.86:52468) 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b6800640 1 -- 172.21.15.86:0/1320827354 learned_addr learned my addr 172.21.15.86:0/1320827354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:35.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b5400640 1 --2- 172.21.15.86:0/1320827354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b009b180 0x7ff6b00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b6800640 1 -- 172.21.15.86:0/1320827354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b009b180 msgr2=0x7ff6b00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:35.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b6800640 1 --2- 172.21.15.86:0/1320827354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b009b180 0x7ff6b00b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b6800640 1 -- 172.21.15.86:0/1320827354 >> v1:172.21.15.196:6789/0 conn(0x7ff6b015e970 legacy=0x7ff6b009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.514+0000 7ff6b6800640 1 -- 172.21.15.86:0/1320827354 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b006e2d0 con 0x7ff6b006d8f0 2024-03-22T04:16:35.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b6800640 1 --2- 172.21.15.86:0/1320827354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b006d8f0 0x7ff6b00cd230 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009e50 tx=0x7ff6ac01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336a677dcfb81c76 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b4a00640 1 -- 172.21.15.86:0/1320827354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0a57f0 con 0x7ff6b006d8f0 2024-03-22T04:16:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b4a00640 1 -- 172.21.15.86:0/1320827354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ac0a5990 con 0x7ff6b006d8f0 2024-03-22T04:16:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b4a00640 1 -- 172.21.15.86:0/1320827354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0a5d60 con 0x7ff6b006d8f0 2024-03-22T04:16:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 -- 172.21.15.86:0/1320827354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b006d8f0 msgr2=0x7ff6b00cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 --2- 172.21.15.86:0/1320827354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b006d8f0 0x7ff6b00cd230 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009e50 tx=0x7ff6ac01ca30 comp rx=0 tx=0).stop 2024-03-22T04:16:35.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 -- 172.21.15.86:0/1320827354 shutdown_connections 2024-03-22T04:16:35.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 --2- 172.21.15.86:0/1320827354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b009b180 0x7ff6b00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 --2- 172.21.15.86:0/1320827354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b006d8f0 0x7ff6b00cd230 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 -- 172.21.15.86:0/1320827354 >> 172.21.15.86:0/1320827354 conn(0x7ff6b00b3bf0 msgr2=0x7ff6b00b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:35.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 -- 172.21.15.86:0/1320827354 shutdown_connections 2024-03-22T04:16:35.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 -- 172.21.15.86:0/1320827354 wait complete. 2024-03-22T04:16:35.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 Processor -- start 2024-03-22T04:16:35.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.518+0000 7ff6b8800640 1 -- start start 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b006d8f0 0x7ff6b00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b009b180 0x7ff6b0151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b015e970 0x7ff6b0151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b00ce100 con 0x7ff6b015e970 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b00cdf80 con 0x7ff6b009b180 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b00ce280 con 0x7ff6b006d8f0 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b015e970 0x7ff6b0151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b006d8f0 0x7ff6b00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b015e970 0x7ff6b0151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52470/0 (socket says 172.21.15.86:52470) 2024-03-22T04:16:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 -- 172.21.15.86:0/3875582422 learned_addr learned my addr 172.21.15.86:0/3875582422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:35.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b5400640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b009b180 0x7ff6b0151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 -- 172.21.15.86:0/3875582422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b006d8f0 msgr2=0x7ff6b00afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b006d8f0 0x7ff6b00afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 -- 172.21.15.86:0/3875582422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b009b180 msgr2=0x7ff6b0151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b009b180 0x7ff6b0151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 -- 172.21.15.86:0/3875582422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b0151be0 con 0x7ff6b015e970 2024-03-22T04:16:35.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b5400640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b009b180 0x7ff6b0151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:16:35.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b6800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b015e970 0x7ff6b0151540 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0af040 tx=0x7ff6ac0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6aa000640 1 -- 172.21.15.86:0/3875582422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0bc020 con 0x7ff6b015e970 2024-03-22T04:16:35.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6aa000640 1 -- 172.21.15.86:0/3875582422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ac0b8670 con 0x7ff6b015e970 2024-03-22T04:16:35.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b0151ec0 con 0x7ff6b015e970 2024-03-22T04:16:35.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6aa000640 1 -- 172.21.15.86:0/3875582422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0b8a60 con 0x7ff6b015e970 2024-03-22T04:16:35.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b01571a0 con 0x7ff6b015e970 2024-03-22T04:16:35.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b0074470 con 0x7ff6b015e970 2024-03-22T04:16:35.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.522+0000 7ff6aa000640 1 -- 172.21.15.86:0/3875582422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff6ac0bf020 con 0x7ff6b015e970 2024-03-22T04:16:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.526+0000 7ff6aa000640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff694067b20 0x7ff694069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.526+0000 7ff6aa000640 1 -- 172.21.15.86:0/3875582422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7ff6ac120ff0 con 0x7ff6b015e970 2024-03-22T04:16:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.526+0000 7ff6b5e00640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff694067b20 0x7ff694069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.526+0000 7ff6b5e00640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff694067b20 0x7ff694069fe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4001770 tx=0x7ff6a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.530+0000 7ff6aa000640 1 -- 172.21.15.86:0/3875582422 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff6ac0c6030 con 0x7ff6b015e970 2024-03-22T04:16:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff6b00d12f0 con 0x7ff6b015e970 2024-03-22T04:16:35.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6aa000640 1 -- 172.21.15.86:0/3875582422 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7ff6ac016620 con 0x7ff6b015e970 2024-03-22T04:16:35.738 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:16:35.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff694067b20 msgr2=0x7ff694069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff694067b20 0x7ff694069fe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4001770 tx=0x7ff6a4009040 comp rx=0 tx=0).stop 2024-03-22T04:16:35.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b015e970 msgr2=0x7ff6b0151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b015e970 0x7ff6b0151540 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0af040 tx=0x7ff6ac0197d0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 shutdown_connections 2024-03-22T04:16:35.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff694067b20 0x7ff694069fe0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b009b180 0x7ff6b0151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b015e970 0x7ff6b0151540 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 --2- 172.21.15.86:0/3875582422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b006d8f0 0x7ff6b00afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 >> 172.21.15.86:0/3875582422 conn(0x7ff6b00b3bf0 msgr2=0x7ff6b0155b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:35.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 shutdown_connections 2024-03-22T04:16:35.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.738+0000 7ff6b8800640 1 -- 172.21.15.86:0/3875582422 wait complete. 2024-03-22T04:16:35.759 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:16:35.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f800e600640 1 Processor -- start 2024-03-22T04:16:35.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f800e600640 1 -- start start 2024-03-22T04:16:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f800e600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f80080d0de0 0x7f80080ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f800e600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80080d11c0 con 0x7f80080d19a0 2024-03-22T04:16:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f800e600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80080cece0 con 0x7f800815fed0 2024-03-22T04:16:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f800e600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f80080d0580 con 0x7f80080d0de0 2024-03-22T04:16:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f7fffe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f80080d19a0 0x7f80080cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40518/0 (socket says 172.21.15.86:40518) 2024-03-22T04:16:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f7fffe00640 1 -- 172.21.15.86:0/2780001022 learned_addr learned my addr 172.21.15.86:0/2780001022 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.898+0000 7f8007400640 1 --2- 172.21.15.86:0/2780001022 >> v2:172.21.15.196:3300/0 conn(0x7f80080d0de0 0x7f80080ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8006a00640 1 -- 172.21.15.86:0/2780001022 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979184527 0 0) 0x7f80080d11c0 con 0x7f80080d19a0 2024-03-22T04:16:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8006a00640 1 -- 172.21.15.86:0/2780001022 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fec003530 con 0x7f80080d19a0 2024-03-22T04:16:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8007400640 1 -- 172.21.15.86:0/2780001022 >> v1:172.21.15.196:6789/0 conn(0x7f800815fed0 legacy=0x7f80081602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8007400640 1 -- 172.21.15.86:0/2780001022 >> v1:172.21.15.86:6789/0 conn(0x7f80080d19a0 legacy=0x7f80080cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8007400640 1 -- 172.21.15.86:0/2780001022 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080cefc0 con 0x7f80080d0de0 2024-03-22T04:16:35.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8007400640 1 --2- 172.21.15.86:0/2780001022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 0x7f80080ce6b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80097c0 tx=0x7f7ff801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560d316da01cb104 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8006a00640 1 -- 172.21.15.86:0/2780001022 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80b1070 con 0x7f80080d0de0 2024-03-22T04:16:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8006a00640 1 -- 172.21.15.86:0/2780001022 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff80ac5f0 con 0x7f80080d0de0 2024-03-22T04:16:35.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f8006a00640 1 -- 172.21.15.86:0/2780001022 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80ac9c0 con 0x7f80080d0de0 2024-03-22T04:16:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f800e600640 1 -- 172.21.15.86:0/2780001022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 msgr2=0x7f80080ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.902+0000 7f800e600640 1 --2- 172.21.15.86:0/2780001022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 0x7f80080ce6b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80097c0 tx=0x7f7ff801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 -- 172.21.15.86:0/2780001022 shutdown_connections 2024-03-22T04:16:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 --2- 172.21.15.86:0/2780001022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 0x7f80080ce6b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 -- 172.21.15.86:0/2780001022 >> 172.21.15.86:0/2780001022 conn(0x7f800806f910 msgr2=0x7f800806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 -- 172.21.15.86:0/2780001022 shutdown_connections 2024-03-22T04:16:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 -- 172.21.15.86:0/2780001022 wait complete. 2024-03-22T04:16:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 Processor -- start 2024-03-22T04:16:35.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 -- start start 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 0x7f80080b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080d19a0 0x7f80081546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f800815fed0 0x7f80080c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80080d1f30 con 0x7f800815fed0 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80080d1db0 con 0x7f80080d0de0 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f800e600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8008071e30 con 0x7f80080d19a0 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f800815fed0 0x7f80080c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 0x7f80080b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 0x7f80080b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:45080/0 (socket says 172.21.15.86:45080) 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f800815fed0 0x7f80080c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43000/0 (socket says 172.21.15.86:43000) 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007400640 1 -- 172.21.15.86:0/756681215 learned_addr learned my addr 172.21.15.86:0/756681215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f7fffe00640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080d19a0 0x7f80081546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007e00640 1 -- 172.21.15.86:0/756681215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080d19a0 msgr2=0x7f80081546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007e00640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080d19a0 0x7f80081546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007e00640 1 -- 172.21.15.86:0/756681215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 msgr2=0x7f80080b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007e00640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 0x7f80080b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007e00640 1 -- 172.21.15.86:0/756681215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080b3620 con 0x7f800815fed0 2024-03-22T04:16:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.906+0000 7f8007e00640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f800815fed0 0x7f80080c9370 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f800000c700 tx=0x7f800000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f7fff400640 1 -- 172.21.15.86:0/756681215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000003b80 con 0x7f800815fed0 2024-03-22T04:16:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f7fff400640 1 -- 172.21.15.86:0/756681215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000003d20 con 0x7f800815fed0 2024-03-22T04:16:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f7fff400640 1 -- 172.21.15.86:0/756681215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800000aa30 con 0x7f800815fed0 2024-03-22T04:16:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80080b4db0 con 0x7f800815fed0 2024-03-22T04:16:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80080b5650 con 0x7f800815fed0 2024-03-22T04:16:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8008068650 con 0x7f800815fed0 2024-03-22T04:16:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f7fff400640 1 -- 172.21.15.86:0/756681215 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f800000abd0 con 0x7f800815fed0 2024-03-22T04:16:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f7fff400640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe4067b20 0x7f7fe4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f7fff400640 1 -- 172.21.15.86:0/756681215 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f8000085ec0 con 0x7f800815fed0 2024-03-22T04:16:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.910+0000 7f8007400640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe4067b20 0x7f7fe4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.914+0000 7f8007400640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe4067b20 0x7f7fe4069fe0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80097c0 tx=0x7f7ff8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:35.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:35.914+0000 7f7fff400640 1 -- 172.21.15.86:0/756681215 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8000056000 con 0x7f800815fed0 2024-03-22T04:16:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.082+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f800815e3c0 con 0x7f800815fed0 2024-03-22T04:16:36.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.082+0000 7f7fff400640 1 -- 172.21.15.86:0/756681215 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v81) ==== 80+0+122 (secure 0 0 0) 0x7f800005a0a0 con 0x7f800815fed0 2024-03-22T04:16:36.084 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/1130704586 2024-03-23T04:15:20.393105+0000 2024-03-22T04:16:36.084 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6836/1130704586 2024-03-23T04:15:20.393105+0000 2024-03-22T04:16:36.085 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-03-22T04:16:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe4067b20 msgr2=0x7f7fe4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe4067b20 0x7f7fe4069fe0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80097c0 tx=0x7f7ff8008040 comp rx=0 tx=0).stop 2024-03-22T04:16:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f800815fed0 msgr2=0x7f80080c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f800815fed0 0x7f80080c9370 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f800000c700 tx=0x7f800000cbc0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 shutdown_connections 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7fe4067b20 0x7f7fe4069fe0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f800815fed0 0x7f80080c9370 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80080d19a0 0x7f80081546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 --2- 172.21.15.86:0/756681215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80080d0de0 0x7f80080b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 >> 172.21.15.86:0/756681215 conn(0x7f800806f910 msgr2=0x7f800815f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 shutdown_connections 2024-03-22T04:16:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.086+0000 7f800e600640 1 -- 172.21.15.86:0/756681215 wait complete. 2024-03-22T04:16:36.104 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:16:36.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:36.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:16:36.129 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:16:36.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:16:36.130 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:16:36.182 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:16:36.182 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:16:36.182 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:36.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:16:36.192 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:16:36.192 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:16:36.192 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:36.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:16:36.233 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:16:36.233 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:16:36.233 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:36.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:16:36.242 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:16:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.410+0000 7fabac000640 1 Processor -- start 2024-03-22T04:16:36.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabac000640 1 -- start start 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabac000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faba406e480 0x7faba40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabac000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40d03b0 0x7faba40cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabac000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faba40ccfc0 0x7faba4071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabac000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faba40ce3f0 con 0x7faba406e480 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabac000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faba40d07f0 con 0x7faba40ccfc0 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabac000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faba415eb00 con 0x7faba40d03b0 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba9600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faba406e480 0x7faba40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba9600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faba406e480 0x7faba40cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43004/0 (socket says 172.21.15.86:43004) 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba9600640 1 -- 172.21.15.86:0/2373587154 learned_addr learned my addr 172.21.15.86:0/2373587154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabaa000640 1 --2- 172.21.15.86:0/2373587154 >> v2:172.21.15.196:3300/0 conn(0x7faba40ccfc0 0x7faba4071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba9600640 1 -- 172.21.15.86:0/2373587154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40d03b0 msgr2=0x7faba40cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba8c00640 1 --2- 172.21.15.86:0/2373587154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40d03b0 0x7faba40cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba9600640 1 --2- 172.21.15.86:0/2373587154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40d03b0 0x7faba40cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba9600640 1 -- 172.21.15.86:0/2373587154 >> v2:172.21.15.196:3300/0 conn(0x7faba40ccfc0 msgr2=0x7faba4071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:36.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba9600640 1 --2- 172.21.15.86:0/2373587154 >> v2:172.21.15.196:3300/0 conn(0x7faba40ccfc0 0x7faba4071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba9600640 1 -- 172.21.15.86:0/2373587154 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba40cd7e0 con 0x7faba406e480 2024-03-22T04:16:36.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7fabaa000640 1 --2- 172.21.15.86:0/2373587154 >> v2:172.21.15.196:3300/0 conn(0x7faba40ccfc0 0x7faba4071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:36.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.414+0000 7faba8c00640 1 --2- 172.21.15.86:0/2373587154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40d03b0 0x7faba40cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:36.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7faba9600640 1 --2- 172.21.15.86:0/2373587154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba406e480 0x7faba40cfe70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7faba40709a0 tx=0x7fab9801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1e3727b4f94163 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7faba3e00640 1 -- 172.21.15.86:0/2373587154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab980ab5f0 con 0x7faba406e480 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7faba3e00640 1 -- 172.21.15.86:0/2373587154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab980b0070 con 0x7faba406e480 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7faba3e00640 1 -- 172.21.15.86:0/2373587154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab980ab9b0 con 0x7faba406e480 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 -- 172.21.15.86:0/2373587154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba406e480 msgr2=0x7faba40cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 --2- 172.21.15.86:0/2373587154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba406e480 0x7faba40cfe70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7faba40709a0 tx=0x7fab9801c520 comp rx=0 tx=0).stop 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 -- 172.21.15.86:0/2373587154 shutdown_connections 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 --2- 172.21.15.86:0/2373587154 >> v2:172.21.15.196:3300/0 conn(0x7faba40ccfc0 0x7faba4071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 --2- 172.21.15.86:0/2373587154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40d03b0 0x7faba40cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 --2- 172.21.15.86:0/2373587154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba406e480 0x7faba40cfe70 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 -- 172.21.15.86:0/2373587154 >> 172.21.15.86:0/2373587154 conn(0x7faba406f930 msgr2=0x7faba406fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 -- 172.21.15.86:0/2373587154 shutdown_connections 2024-03-22T04:16:36.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 -- 172.21.15.86:0/2373587154 wait complete. 2024-03-22T04:16:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 Processor -- start 2024-03-22T04:16:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.418+0000 7fabac000640 1 -- start start 2024-03-22T04:16:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba406e480 0x7faba4153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40ccfc0 0x7faba41fb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba40d03b0 0x7faba40c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faba40cd520 con 0x7faba40d03b0 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faba40cd3a0 con 0x7faba406e480 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faba406c770 con 0x7faba40ccfc0 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba40d03b0 0x7faba40c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba40d03b0 0x7faba40c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43006/0 (socket says 172.21.15.86:43006) 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 -- 172.21.15.86:0/3699105047 learned_addr learned my addr 172.21.15.86:0/3699105047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7faba9600640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba406e480 0x7faba4153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7faba8c00640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40ccfc0 0x7faba41fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 -- 172.21.15.86:0/3699105047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40ccfc0 msgr2=0x7faba41fb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40ccfc0 0x7faba41fb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 -- 172.21.15.86:0/3699105047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba406e480 msgr2=0x7faba4153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba406e480 0x7faba4153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 -- 172.21.15.86:0/3699105047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba40c34d0 con 0x7faba40d03b0 2024-03-22T04:16:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabaa000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba40d03b0 0x7faba40c44f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00cf20 tx=0x7fab9c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7faba1600640 1 -- 172.21.15.86:0/3699105047 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c01a070 con 0x7faba40d03b0 2024-03-22T04:16:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7faba1600640 1 -- 172.21.15.86:0/3699105047 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9c00d840 con 0x7faba40d03b0 2024-03-22T04:16:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba40c37b0 con 0x7faba40d03b0 2024-03-22T04:16:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7faba1600640 1 -- 172.21.15.86:0/3699105047 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00dc30 con 0x7faba40d03b0 2024-03-22T04:16:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba40c3d70 con 0x7faba40d03b0 2024-03-22T04:16:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.422+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba4068650 con 0x7faba40d03b0 2024-03-22T04:16:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.426+0000 7faba1600640 1 -- 172.21.15.86:0/3699105047 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fab9c01d070 con 0x7faba40d03b0 2024-03-22T04:16:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.426+0000 7faba1600640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab84067b20 0x7fab84069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.426+0000 7faba1600640 1 -- 172.21.15.86:0/3699105047 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7fab9c086900 con 0x7faba40d03b0 2024-03-22T04:16:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.426+0000 7faba9600640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab84067b20 0x7fab84069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.426+0000 7faba9600640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab84067b20 0x7fab84069fe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fab980af000 tx=0x7fab980a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:36.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.430+0000 7faba1600640 1 -- 172.21.15.86:0/3699105047 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fab9c056a40 con 0x7faba40d03b0 2024-03-22T04:16:36.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.614+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faba415fc20 con 0x7faba40d03b0 2024-03-22T04:16:36.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.614+0000 7faba1600640 1 -- 172.21.15.86:0/3699105047 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5188 (secure 0 0 0) 0x7fab9c05aae0 con 0x7faba40d03b0 2024-03-22T04:16:36.616 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:36.616 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:16:07.415273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7002,"qdb_cluster":[7002]},"id":5}]} 2024-03-22T04:16:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 54 2024-03-22T04:16:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab84067b20 msgr2=0x7fab84069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab84067b20 0x7fab84069fe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fab980af000 tx=0x7fab980a7000 comp rx=0 tx=0).stop 2024-03-22T04:16:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba40d03b0 msgr2=0x7faba40c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba40d03b0 0x7faba40c44f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00cf20 tx=0x7fab9c00a710 comp rx=0 tx=0).stop 2024-03-22T04:16:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 shutdown_connections 2024-03-22T04:16:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab84067b20 0x7fab84069fe0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba40d03b0 0x7faba40c44f0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba40ccfc0 0x7faba41fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 --2- 172.21.15.86:0/3699105047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba406e480 0x7faba4153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 >> 172.21.15.86:0/3699105047 conn(0x7faba406f930 msgr2=0x7faba415ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:36.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 shutdown_connections 2024-03-22T04:16:36.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.618+0000 7fabac000640 1 -- 172.21.15.86:0/3699105047 wait complete. 2024-03-22T04:16:36.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:16:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.770+0000 7f2871600640 1 Processor -- start 2024-03-22T04:16:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.770+0000 7f2871600640 1 -- start start 2024-03-22T04:16:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f2871600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f286c0d0580 0x7f286c071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f2871600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f286c06e2f0 0x7f286c071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f2871600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286c1623d0 con 0x7f286c15e630 2024-03-22T04:16:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f2871600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f286c1620f0 con 0x7f286c0d0580 2024-03-22T04:16:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f2871600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f286c15ec10 con 0x7f286c06e2f0 2024-03-22T04:16:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f286c0d0580 0x7f286c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f286c0d0580 0x7f286c071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43018/0 (socket says 172.21.15.86:43018) 2024-03-22T04:16:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286a000640 1 -- 172.21.15.86:0/3837613095 learned_addr learned my addr 172.21.15.86:0/3837613095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286b400640 1 --2- 172.21.15.86:0/3837613095 >> v2:172.21.15.196:3300/0 conn(0x7f286c06e2f0 0x7f286c071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286a000640 1 -- 172.21.15.86:0/3837613095 >> v2:172.21.15.196:3300/0 conn(0x7f286c06e2f0 msgr2=0x7f286c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286a000640 1 --2- 172.21.15.86:0/3837613095 >> v2:172.21.15.196:3300/0 conn(0x7f286c06e2f0 0x7f286c071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286a000640 1 -- 172.21.15.86:0/3837613095 >> v1:172.21.15.196:6789/0 conn(0x7f286c15e630 legacy=0x7f286c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286a000640 1 -- 172.21.15.86:0/3837613095 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c15eef0 con 0x7f286c0d0580 2024-03-22T04:16:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.774+0000 7f286b400640 1 --2- 172.21.15.86:0/3837613095 >> v2:172.21.15.196:3300/0 conn(0x7f286c06e2f0 0x7f286c071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f286a000640 1 --2- 172.21.15.86:0/3837613095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c0d0580 0x7f286c071100 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f286c0d2200 tx=0x7f285801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b88d12a3896fcbf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2869600640 1 -- 172.21.15.86:0/3837613095 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28580b0070 con 0x7f286c0d0580 2024-03-22T04:16:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2869600640 1 -- 172.21.15.86:0/3837613095 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28580ab600 con 0x7f286c0d0580 2024-03-22T04:16:36.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2869600640 1 -- 172.21.15.86:0/3837613095 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28580ab9d0 con 0x7f286c0d0580 2024-03-22T04:16:36.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2871600640 1 -- 172.21.15.86:0/3837613095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c0d0580 msgr2=0x7f286c071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2871600640 1 --2- 172.21.15.86:0/3837613095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c0d0580 0x7f286c071100 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f286c0d2200 tx=0x7f285801c520 comp rx=0 tx=0).stop 2024-03-22T04:16:36.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2871600640 1 -- 172.21.15.86:0/3837613095 shutdown_connections 2024-03-22T04:16:36.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2871600640 1 --2- 172.21.15.86:0/3837613095 >> v2:172.21.15.196:3300/0 conn(0x7f286c06e2f0 0x7f286c071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2871600640 1 --2- 172.21.15.86:0/3837613095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c0d0580 0x7f286c071100 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2871600640 1 -- 172.21.15.86:0/3837613095 >> 172.21.15.86:0/3837613095 conn(0x7f286c06f910 msgr2=0x7f286c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2871600640 1 -- 172.21.15.86:0/3837613095 shutdown_connections 2024-03-22T04:16:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.778+0000 7f2871600640 1 -- 172.21.15.86:0/3837613095 wait complete. 2024-03-22T04:16:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 Processor -- start 2024-03-22T04:16:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 -- start start 2024-03-22T04:16:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c06e2f0 0x7f286c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f286c0d0580 0x7f286c0b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f286c15e630 0x7f286c0ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f286c071f90 con 0x7f286c06e2f0 2024-03-22T04:16:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f286c15f9d0 con 0x7f286c15e630 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f286c0cfaf0 con 0x7f286c0d0580 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c06e2f0 0x7f286c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c06e2f0 0x7f286c155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43024/0 (socket says 172.21.15.86:43024) 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 -- 172.21.15.86:0/1033470992 learned_addr learned my addr 172.21.15.86:0/1033470992 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286a000640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f286c0d0580 0x7f286c0b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286b400640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f286c15e630 0x7f286c0ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 -- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f286c0d0580 msgr2=0x7f286c0b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f286c0d0580 0x7f286c0b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 -- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f286c15e630 msgr2=0x7f286c0ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f286c15e630 0x7f286c0ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 -- 172.21.15.86:0/1033470992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c0b66d0 con 0x7f286c06e2f0 2024-03-22T04:16:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286a000640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f286c0d0580 0x7f286c0b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f286aa00640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c06e2f0 0x7f286c155db0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f285c00c370 tx=0x7f285c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2862a00640 1 -- 172.21.15.86:0/1033470992 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0052d0 con 0x7f286c06e2f0 2024-03-22T04:16:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2862a00640 1 -- 172.21.15.86:0/1033470992 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c005470 con 0x7f286c06e2f0 2024-03-22T04:16:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2862a00640 1 -- 172.21.15.86:0/1033470992 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c007260 con 0x7f286c06e2f0 2024-03-22T04:16:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c0b69b0 con 0x7f286c06e2f0 2024-03-22T04:16:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.782+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c0b6f70 con 0x7f286c06e2f0 2024-03-22T04:16:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.786+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c068650 con 0x7f286c06e2f0 2024-03-22T04:16:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.786+0000 7f2862a00640 1 -- 172.21.15.86:0/1033470992 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f285c007400 con 0x7f286c06e2f0 2024-03-22T04:16:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.786+0000 7f2862a00640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f284c067b20 0x7f284c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.786+0000 7f2862a00640 1 -- 172.21.15.86:0/1033470992 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f285c085560 con 0x7f286c06e2f0 2024-03-22T04:16:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.786+0000 7f286a000640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f284c067b20 0x7f284c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.786+0000 7f286a000640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f284c067b20 0x7f284c069fe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f28580ad000 tx=0x7f2858008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:36.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.790+0000 7f2862a00640 1 -- 172.21.15.86:0/1033470992 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f285c055d80 con 0x7f286c06e2f0 2024-03-22T04:16:36.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.946+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f286c168f40 con 0x7f286c06e2f0 2024-03-22T04:16:36.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2862a00640 1 -- 172.21.15.86:0/1033470992 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+17014 (secure 0 0 0) 0x7f285c055680 con 0x7f286c06e2f0 2024-03-22T04:16:36.949 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:36.949 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:20.393125+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1130704586":"2024-03-23T04:15:20.393105+0000","172.21.15.86:6836/1130704586":"2024-03-23T04:15:20.393105+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:16:36.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f284c067b20 msgr2=0x7f284c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f284c067b20 0x7f284c069fe0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f28580ad000 tx=0x7f2858008040 comp rx=0 tx=0).stop 2024-03-22T04:16:36.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c06e2f0 msgr2=0x7f286c155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c06e2f0 0x7f286c155db0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f285c00c370 tx=0x7f285c00c830 comp rx=0 tx=0).stop 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 shutdown_connections 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f284c067b20 0x7f284c069fe0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f286c15e630 0x7f286c0ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f286c0d0580 0x7f286c0b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 --2- 172.21.15.86:0/1033470992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286c06e2f0 0x7f286c155db0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 >> 172.21.15.86:0/1033470992 conn(0x7f286c06f910 msgr2=0x7f286c156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 shutdown_connections 2024-03-22T04:16:36.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:36.950+0000 7f2871600640 1 -- 172.21.15.86:0/1033470992 wait complete. 2024-03-22T04:16:36.972 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:15:07.783245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:15:08.394997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:16:36.972 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:16:37.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.114+0000 7ff86bc00640 1 Processor -- start 2024-03-22T04:16:37.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.114+0000 7ff86bc00640 1 -- start start 2024-03-22T04:16:37.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.114+0000 7ff86bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8640b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.114+0000 7ff86bc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff86406d8f0 0x7ff86409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.114+0000 7ff86bc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff86406dd70 con 0x7ff86409afb0 2024-03-22T04:16:37.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.114+0000 7ff86bc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff8640b49a0 con 0x7ff86406d8f0 2024-03-22T04:16:37.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.114+0000 7ff86bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff8640ccda0 con 0x7ff86415e970 2024-03-22T04:16:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8640b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8640b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:46888/0 (socket says 172.21.15.86:46888) 2024-03-22T04:16:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff86409afb0 0x7ff8640d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:53852/0 (socket says 172.21.15.86:53852) 2024-03-22T04:16:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869200640 1 -- 172.21.15.86:0/3581648462 learned_addr learned my addr 172.21.15.86:0/3581648462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff863e00640 1 --2- 172.21.15.86:0/3581648462 >> v2:172.21.15.196:3300/0 conn(0x7ff86406d8f0 0x7ff86409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869200640 1 -- 172.21.15.86:0/3581648462 >> v2:172.21.15.196:3300/0 conn(0x7ff86406d8f0 msgr2=0x7ff86409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869200640 1 --2- 172.21.15.86:0/3581648462 >> v2:172.21.15.196:3300/0 conn(0x7ff86406d8f0 0x7ff86409aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869200640 1 -- 172.21.15.86:0/3581648462 >> v1:172.21.15.196:6789/0 conn(0x7ff86409afb0 legacy=0x7ff8640d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869200640 1 -- 172.21.15.86:0/3581648462 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86406dff0 con 0x7ff86415e970 2024-03-22T04:16:37.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff863400640 1 -- 172.21.15.86:0/3581648462 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1595926440 0 0) 0x7ff85c0014a0 con 0x7ff86409afb0 2024-03-22T04:16:37.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff869200640 1 --2- 172.21.15.86:0/3581648462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8640b12e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff858009870 tx=0x7ff85801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e575a78588e2de2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff863400640 1 -- 172.21.15.86:0/3581648462 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580ab650 con 0x7ff86415e970 2024-03-22T04:16:37.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff863400640 1 -- 172.21.15.86:0/3581648462 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8580b0070 con 0x7ff86415e970 2024-03-22T04:16:37.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.118+0000 7ff863400640 1 -- 172.21.15.86:0/3581648462 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580aba10 con 0x7ff86415e970 2024-03-22T04:16:37.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- 172.21.15.86:0/3581648462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 msgr2=0x7ff8640b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3581648462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8640b12e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff858009870 tx=0x7ff85801c560 comp rx=0 tx=0).stop 2024-03-22T04:16:37.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- 172.21.15.86:0/3581648462 shutdown_connections 2024-03-22T04:16:37.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3581648462 >> v2:172.21.15.196:3300/0 conn(0x7ff86406d8f0 0x7ff86409aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3581648462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8640b12e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- 172.21.15.86:0/3581648462 >> 172.21.15.86:0/3581648462 conn(0x7ff8640b3bf0 msgr2=0x7ff8640b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- 172.21.15.86:0/3581648462 shutdown_connections 2024-03-22T04:16:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- 172.21.15.86:0/3581648462 wait complete. 2024-03-22T04:16:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 Processor -- start 2024-03-22T04:16:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- start start 2024-03-22T04:16:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86406d8f0 0x7ff86414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86409afb0 0x7ff864151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8641518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff86416b550 con 0x7ff86406d8f0 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff86415ed50 con 0x7ff86409afb0 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff86bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff86416b6d0 con 0x7ff86415e970 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86406d8f0 0x7ff86414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86406d8f0 0x7ff86414d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43038/0 (socket says 172.21.15.86:43038) 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869200640 1 -- 172.21.15.86:0/3929074671 learned_addr learned my addr 172.21.15.86:0/3929074671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff863e00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86409afb0 0x7ff864151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869c00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8641518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869200640 1 -- 172.21.15.86:0/3929074671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 msgr2=0x7ff8641518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869200640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8641518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869200640 1 -- 172.21.15.86:0/3929074671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86409afb0 msgr2=0x7ff864151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869200640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86409afb0 0x7ff864151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.122+0000 7ff869200640 1 -- 172.21.15.86:0/3929074671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff864154f30 con 0x7ff86406d8f0 2024-03-22T04:16:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff869200640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86406d8f0 0x7ff86414d750 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff85801ac10 tx=0x7ff858019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff860c00640 1 -- 172.21.15.86:0/3929074671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858004220 con 0x7ff86406d8f0 2024-03-22T04:16:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff860c00640 1 -- 172.21.15.86:0/3929074671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8580043c0 con 0x7ff86406d8f0 2024-03-22T04:16:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff860c00640 1 -- 172.21.15.86:0/3929074671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858003820 con 0x7ff86406d8f0 2024-03-22T04:16:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff864155210 con 0x7ff86406d8f0 2024-03-22T04:16:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8641557d0 con 0x7ff86406d8f0 2024-03-22T04:16:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff864074470 con 0x7ff86406d8f0 2024-03-22T04:16:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff860c00640 1 -- 172.21.15.86:0/3929074671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff8580039c0 con 0x7ff86406d8f0 2024-03-22T04:16:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff860c00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff844067b20 0x7ff844069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff860c00640 1 -- 172.21.15.86:0/3929074671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7ff858120970 con 0x7ff86406d8f0 2024-03-22T04:16:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.126+0000 7ff863e00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff844067b20 0x7ff844069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.130+0000 7ff863e00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff844067b20 0x7ff844069fe0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff8641fae90 tx=0x7ff850009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.130+0000 7ff860c00640 1 -- 172.21.15.86:0/3929074671 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff858016150 con 0x7ff86406d8f0 2024-03-22T04:16:37.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.286+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff864168f70 con 0x7ff86406d8f0 2024-03-22T04:16:37.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.286+0000 7ff860c00640 1 -- 172.21.15.86:0/3929074671 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7ff8580f0ab0 con 0x7ff86406d8f0 2024-03-22T04:16:37.287 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.286+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff844067b20 msgr2=0x7ff844069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.286+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff844067b20 0x7ff844069fe0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff8641fae90 tx=0x7ff850009190 comp rx=0 tx=0).stop 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.286+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86406d8f0 msgr2=0x7ff86414d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.286+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86406d8f0 0x7ff86414d750 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff85801ac10 tx=0x7ff858019300 comp rx=0 tx=0).stop 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.290+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 shutdown_connections 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.290+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff844067b20 0x7ff844069fe0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.290+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff86409afb0 0x7ff864151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.290+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff86415e970 0x7ff8641518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.290+0000 7ff86bc00640 1 --2- 172.21.15.86:0/3929074671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff86406d8f0 0x7ff86414d750 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.290+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 >> 172.21.15.86:0/3929074671 conn(0x7ff8640b3bf0 msgr2=0x7ff86406f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:37.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.290+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 shutdown_connections 2024-03-22T04:16:37.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.290+0000 7ff86bc00640 1 -- 172.21.15.86:0/3929074671 wait complete. 2024-03-22T04:16:37.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:16:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.446+0000 7f3e8fc00640 1 Processor -- start 2024-03-22T04:16:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.446+0000 7f3e8fc00640 1 -- start start 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.446+0000 7f3e8fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e880b3610 0x7f3e8809b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.446+0000 7f3e8fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e880cda00 con 0x7f3e880b11c0 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.446+0000 7f3e8fc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e880cdce0 con 0x7f3e8815c730 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.446+0000 7f3e8fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e8809b910 con 0x7f3e880b3610 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e880b3610 0x7f3e8809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e880b3610 0x7f3e8809b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43040/0 (socket says 172.21.15.86:43040) 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87e00640 1 -- 172.21.15.86:0/2753652796 learned_addr learned my addr 172.21.15.86:0/2753652796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e8dc00640 1 --1- 172.21.15.86:0/2753652796 >> v1:172.21.15.86:6789/0 conn(0x7f3e880b11c0 0x7f3e880b1ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40534/0 (socket says 172.21.15.86:40534) 2024-03-22T04:16:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87e00640 1 -- 172.21.15.86:0/2753652796 >> v1:172.21.15.196:6789/0 conn(0x7f3e8815c730 legacy=0x7f3e8809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87e00640 1 -- 172.21.15.86:0/2753652796 >> v1:172.21.15.86:6789/0 conn(0x7f3e880b11c0 legacy=0x7f3e880b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87e00640 1 -- 172.21.15.86:0/2753652796 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e880cdfc0 con 0x7f3e880b3610 2024-03-22T04:16:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87e00640 1 --2- 172.21.15.86:0/2753652796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e8809b180 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098a0 tx=0x7f3e7401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4534d6e18b3ccf45 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87400640 1 -- 172.21.15.86:0/2753652796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740b1070 con 0x7f3e880b3610 2024-03-22T04:16:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87400640 1 -- 172.21.15.86:0/2753652796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e740ac620 con 0x7f3e880b3610 2024-03-22T04:16:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.450+0000 7f3e87400640 1 -- 172.21.15.86:0/2753652796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740ac9f0 con 0x7f3e880b3610 2024-03-22T04:16:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- 172.21.15.86:0/2753652796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 msgr2=0x7f3e8809b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 --2- 172.21.15.86:0/2753652796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e8809b180 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098a0 tx=0x7f3e7401c4b0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- 172.21.15.86:0/2753652796 shutdown_connections 2024-03-22T04:16:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 --2- 172.21.15.86:0/2753652796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e8809b180 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- 172.21.15.86:0/2753652796 >> 172.21.15.86:0/2753652796 conn(0x7f3e880b3bf0 msgr2=0x7f3e880b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- 172.21.15.86:0/2753652796 shutdown_connections 2024-03-22T04:16:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- 172.21.15.86:0/2753652796 wait complete. 2024-03-22T04:16:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 Processor -- start 2024-03-22T04:16:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- start start 2024-03-22T04:16:37.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e880b11c0 0x7f3e88154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e880b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e8815c730 0x7f3e880c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e880b2e50 con 0x7f3e880b3610 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e880b2cd0 con 0x7f3e8815c730 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8fc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3e8809b6c0 con 0x7f3e880b11c0 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e880b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e880b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43054/0 (socket says 172.21.15.86:43054) 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e87e00640 1 -- 172.21.15.86:0/1767756055 learned_addr learned my addr 172.21.15.86:0/1767756055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8d200640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e880b11c0 0x7f3e88154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8dc00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e8815c730 0x7f3e880c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e87e00640 1 -- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e880b11c0 msgr2=0x7f3e88154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e87e00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e880b11c0 0x7f3e88154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e87e00640 1 -- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e8815c730 msgr2=0x7f3e880c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e87e00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e8815c730 0x7f3e880c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e87e00640 1 -- 172.21.15.86:0/1767756055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e880b5840 con 0x7f3e880b3610 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8d200640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e880b11c0 0x7f3e88154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.454+0000 7f3e8dc00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e8815c730 0x7f3e880c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e87e00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e880b7660 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3e740af040 tx=0x7f3e7401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e84c00640 1 -- 172.21.15.86:0/1767756055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740bb020 con 0x7f3e880b3610 2024-03-22T04:16:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e84c00640 1 -- 172.21.15.86:0/1767756055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e740b7640 con 0x7f3e880b3610 2024-03-22T04:16:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e84c00640 1 -- 172.21.15.86:0/1767756055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740b1040 con 0x7f3e880b3610 2024-03-22T04:16:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e880b5b20 con 0x7f3e880b3610 2024-03-22T04:16:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e880b60e0 con 0x7f3e880b3610 2024-03-22T04:16:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e88074470 con 0x7f3e880b3610 2024-03-22T04:16:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e84c00640 1 -- 172.21.15.86:0/1767756055 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3e740be020 con 0x7f3e880b3610 2024-03-22T04:16:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e84c00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e68067b20 0x7f3e68069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e84c00640 1 -- 172.21.15.86:0/1767756055 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f3e74120330 con 0x7f3e880b3610 2024-03-22T04:16:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.458+0000 7f3e8d200640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e68067b20 0x7f3e68069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.462+0000 7f3e8d200640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e68067b20 0x7f3e68069fe0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e880c9b80 tx=0x7f3e7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.462+0000 7f3e84c00640 1 -- 172.21.15.86:0/1767756055 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3e740c5030 con 0x7f3e880b3610 2024-03-22T04:16:37.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.626+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3e880c7910 con 0x7f3e880b3610 2024-03-22T04:16:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.626+0000 7f3e84c00640 1 -- 172.21.15.86:0/1767756055 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f3e74016130 con 0x7f3e880b3610 2024-03-22T04:16:37.628 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:16:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e68067b20 msgr2=0x7f3e68069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e68067b20 0x7f3e68069fe0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e880c9b80 tx=0x7f3e7c009040 comp rx=0 tx=0).stop 2024-03-22T04:16:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 msgr2=0x7f3e880b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e880b7660 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3e740af040 tx=0x7f3e7401acf0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 shutdown_connections 2024-03-22T04:16:37.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3e68067b20 0x7f3e68069fe0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e8815c730 0x7f3e880c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e880b3610 0x7f3e880b7660 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 --2- 172.21.15.86:0/1767756055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e880b11c0 0x7f3e88154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 >> 172.21.15.86:0/1767756055 conn(0x7f3e880b3bf0 msgr2=0x7f3e8806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 shutdown_connections 2024-03-22T04:16:37.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.630+0000 7f3e8fc00640 1 -- 172.21.15.86:0/1767756055 wait complete. 2024-03-22T04:16:37.649 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:16:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.786+0000 7fbda1e00640 1 Processor -- start 2024-03-22T04:16:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.786+0000 7fbda1e00640 1 -- start start 2024-03-22T04:16:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbda1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 0x7fbd9c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbda1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd9c06d8f0 0x7fbd9c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbda1e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c06dff0 con 0x7fbd9c15e970 2024-03-22T04:16:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbda1e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c0b49a0 con 0x7fbd9c06d8f0 2024-03-22T04:16:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbda1e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c0ccda0 con 0x7fbd9c09b180 2024-03-22T04:16:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbd9b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd9c06d8f0 0x7fbd9c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbd9b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd9c06d8f0 0x7fbd9c0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43060/0 (socket says 172.21.15.86:43060) 2024-03-22T04:16:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbd9b400640 1 -- 172.21.15.86:0/1947562464 learned_addr learned my addr 172.21.15.86:0/1947562464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbd9a000640 1 --2- 172.21.15.86:0/1947562464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 0x7fbd9c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbd9b400640 1 -- 172.21.15.86:0/1947562464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 msgr2=0x7fbd9c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbd9b400640 1 --2- 172.21.15.86:0/1947562464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 0x7fbd9c0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbd9b400640 1 -- 172.21.15.86:0/1947562464 >> v1:172.21.15.196:6789/0 conn(0x7fbd9c15e970 legacy=0x7fbd9c09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.790+0000 7fbd9b400640 1 -- 172.21.15.86:0/1947562464 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c06e2d0 con 0x7fbd9c06d8f0 2024-03-22T04:16:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbd9b400640 1 --2- 172.21.15.86:0/1947562464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c06d8f0 0x7fbd9c0cd230 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009e50 tx=0x7fbd9401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc5c9d08573c2aa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbd99600640 1 -- 172.21.15.86:0/1947562464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940a5760 con 0x7fbd9c06d8f0 2024-03-22T04:16:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbd99600640 1 -- 172.21.15.86:0/1947562464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd940a5900 con 0x7fbd9c06d8f0 2024-03-22T04:16:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbd99600640 1 -- 172.21.15.86:0/1947562464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940a5cd0 con 0x7fbd9c06d8f0 2024-03-22T04:16:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbda1e00640 1 -- 172.21.15.86:0/1947562464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c06d8f0 msgr2=0x7fbd9c0cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbda1e00640 1 --2- 172.21.15.86:0/1947562464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c06d8f0 0x7fbd9c0cd230 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009e50 tx=0x7fbd9401c9f0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbda1e00640 1 -- 172.21.15.86:0/1947562464 shutdown_connections 2024-03-22T04:16:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbda1e00640 1 --2- 172.21.15.86:0/1947562464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 0x7fbd9c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbda1e00640 1 --2- 172.21.15.86:0/1947562464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c06d8f0 0x7fbd9c0cd230 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbda1e00640 1 -- 172.21.15.86:0/1947562464 >> 172.21.15.86:0/1947562464 conn(0x7fbd9c0b3bf0 msgr2=0x7fbd9c0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbda1e00640 1 -- 172.21.15.86:0/1947562464 shutdown_connections 2024-03-22T04:16:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.794+0000 7fbda1e00640 1 -- 172.21.15.86:0/1947562464 wait complete. 2024-03-22T04:16:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 Processor -- start 2024-03-22T04:16:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 -- start start 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd9c06d8f0 0x7fbd9c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 0x7fbd9c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c15e970 0x7fbd9c0c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c0ce100 con 0x7fbd9c15e970 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c0cdf80 con 0x7fbd9c06d8f0 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c0ce280 con 0x7fbd9c09b180 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c15e970 0x7fbd9c0c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c15e970 0x7fbd9c0c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43066/0 (socket says 172.21.15.86:43066) 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 -- 172.21.15.86:0/3327782720 learned_addr learned my addr 172.21.15.86:0/3327782720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 -- 172.21.15.86:0/3327782720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 msgr2=0x7fbd9c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 0x7fbd9c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 -- 172.21.15.86:0/3327782720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd9c06d8f0 msgr2=0x7fbd9c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd9c06d8f0 0x7fbd9c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 -- 172.21.15.86:0/3327782720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c160110 con 0x7fbd9c15e970 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd9b400640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c15e970 0x7fbd9c0c5b60 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbd940af040 tx=0x7fbd94019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd93400640 1 -- 172.21.15.86:0/3327782720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940bc020 con 0x7fbd9c15e970 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd93400640 1 -- 172.21.15.86:0/3327782720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd940b8690 con 0x7fbd9c15e970 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbd93400640 1 -- 172.21.15.86:0/3327782720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940b8a80 con 0x7fbd9c15e970 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c1603b0 con 0x7fbd9c15e970 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.798+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c160970 con 0x7fbd9c15e970 2024-03-22T04:16:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.802+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c074470 con 0x7fbd9c15e970 2024-03-22T04:16:37.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.802+0000 7fbd93400640 1 -- 172.21.15.86:0/3327782720 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbd940bf020 con 0x7fbd9c15e970 2024-03-22T04:16:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.802+0000 7fbd93400640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd7c067b50 0x7fbd7c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.802+0000 7fbd93400640 1 -- 172.21.15.86:0/3327782720 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7fbd94120cb0 con 0x7fbd9c15e970 2024-03-22T04:16:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.802+0000 7fbd9aa00640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd7c067b50 0x7fbd7c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.802+0000 7fbd9aa00640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd7c067b50 0x7fbd7c06a010 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009670 tx=0x7fbd8c006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.806+0000 7fbd93400640 1 -- 172.21.15.86:0/3327782720 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbd940165e0 con 0x7fbd9c15e970 2024-03-22T04:16:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd9c0b2370 con 0x7fbd9c15e970 2024-03-22T04:16:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbd93400640 1 -- 172.21.15.86:0/3327782720 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7fbd940f1570 con 0x7fbd9c15e970 2024-03-22T04:16:37.965 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:16:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd7c067b50 msgr2=0x7fbd7c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd7c067b50 0x7fbd7c06a010 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009670 tx=0x7fbd8c006bd0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c15e970 msgr2=0x7fbd9c0c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c15e970 0x7fbd9c0c5b60 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbd940af040 tx=0x7fbd94019790 comp rx=0 tx=0).stop 2024-03-22T04:16:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 shutdown_connections 2024-03-22T04:16:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd7c067b50 0x7fbd7c06a010 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c15e970 0x7fbd9c0c5b60 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd9c09b180 0x7fbd9c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 --2- 172.21.15.86:0/3327782720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd9c06d8f0 0x7fbd9c0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 >> 172.21.15.86:0/3327782720 conn(0x7fbd9c0b3bf0 msgr2=0x7fbd9c0b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 shutdown_connections 2024-03-22T04:16:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:37.966+0000 7fbda1e00640 1 -- 172.21.15.86:0/3327782720 wait complete. 2024-03-22T04:16:37.986 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:16:38.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0db0000640 1 Processor -- start 2024-03-22T04:16:38.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0db0000640 1 -- start start 2024-03-22T04:16:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0db0000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da80cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0db0000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0da80d03b0 0x7f0da806d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0db0000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da81623d0 con 0x7f0da806e2f0 2024-03-22T04:16:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0db0000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0da80d0800 con 0x7f0da80d03b0 2024-03-22T04:16:38.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0db0000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0da81620f0 con 0x7f0da815e630 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dad600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da80cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dae000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0da80d03b0 0x7f0da806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dacc00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f0da806e2f0 0x7f0da806cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:53882/0 (socket says 172.21.15.86:53882) 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dacc00640 1 -- 172.21.15.86:0/979307533 learned_addr learned my addr 172.21.15.86:0/979307533 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dad600640 1 -- 172.21.15.86:0/979307533 >> v2:172.21.15.196:3300/0 conn(0x7f0da80d03b0 msgr2=0x7f0da806d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dad600640 1 --2- 172.21.15.86:0/979307533 >> v2:172.21.15.196:3300/0 conn(0x7f0da80d03b0 0x7f0da806d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dad600640 1 -- 172.21.15.86:0/979307533 >> v1:172.21.15.196:6789/0 conn(0x7f0da806e2f0 legacy=0x7f0da806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dad600640 1 -- 172.21.15.86:0/979307533 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da815ec10 con 0x7f0da815e630 2024-03-22T04:16:38.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.138+0000 7f0dae000640 1 --2- 172.21.15.86:0/979307533 >> v2:172.21.15.196:3300/0 conn(0x7f0da80d03b0 0x7f0da806d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:16:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0dad600640 1 --2- 172.21.15.86:0/979307533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da80cfe70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0097c0 tx=0x7f0d9c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5193faf64e340908 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0da7e00640 1 -- 172.21.15.86:0/979307533 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0b0070 con 0x7f0da815e630 2024-03-22T04:16:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0da7e00640 1 -- 172.21.15.86:0/979307533 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0ab600 con 0x7f0da815e630 2024-03-22T04:16:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0da7e00640 1 -- 172.21.15.86:0/979307533 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0ab9d0 con 0x7f0da815e630 2024-03-22T04:16:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0db0000640 1 -- 172.21.15.86:0/979307533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 msgr2=0x7f0da80cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0db0000640 1 --2- 172.21.15.86:0/979307533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da80cfe70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0097c0 tx=0x7f0d9c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0db0000640 1 -- 172.21.15.86:0/979307533 shutdown_connections 2024-03-22T04:16:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0db0000640 1 --2- 172.21.15.86:0/979307533 >> v2:172.21.15.196:3300/0 conn(0x7f0da80d03b0 0x7f0da806d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0db0000640 1 --2- 172.21.15.86:0/979307533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da80cfe70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0db0000640 1 -- 172.21.15.86:0/979307533 >> 172.21.15.86:0/979307533 conn(0x7f0da806f910 msgr2=0x7f0da806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0db0000640 1 -- 172.21.15.86:0/979307533 shutdown_connections 2024-03-22T04:16:38.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.142+0000 7f0db0000640 1 -- 172.21.15.86:0/979307533 wait complete. 2024-03-22T04:16:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 Processor -- start 2024-03-22T04:16:38.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 -- start start 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da806e2f0 0x7f0da8156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80d03b0 0x7f0da815c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da814b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0da815f9d0 con 0x7f0da806e2f0 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0da80d1f60 con 0x7f0da80d03b0 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0da80cfaf0 con 0x7f0da815e630 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da806e2f0 0x7f0da8156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da806e2f0 0x7f0da8156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43082/0 (socket says 172.21.15.86:43082) 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 -- 172.21.15.86:0/3674883294 learned_addr learned my addr 172.21.15.86:0/3674883294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dae000640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da814b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dacc00640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80d03b0 0x7f0da815c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 -- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 msgr2=0x7f0da814b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da814b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 -- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80d03b0 msgr2=0x7f0da815c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80d03b0 0x7f0da815c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 -- 172.21.15.86:0/3674883294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da815b770 con 0x7f0da806e2f0 2024-03-22T04:16:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dae000640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da814b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dacc00640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80d03b0 0x7f0da815c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:38.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0dad600640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da806e2f0 0x7f0da8156b20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c01a0d0 tx=0x7f0d9c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0da5600640 1 -- 172.21.15.86:0/3674883294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0b0040 con 0x7f0da806e2f0 2024-03-22T04:16:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0da5600640 1 -- 172.21.15.86:0/3674883294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0bd070 con 0x7f0da806e2f0 2024-03-22T04:16:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0da5600640 1 -- 172.21.15.86:0/3674883294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0b56c0 con 0x7f0da806e2f0 2024-03-22T04:16:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da815ba50 con 0x7f0da806e2f0 2024-03-22T04:16:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.146+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da815c010 con 0x7f0da806e2f0 2024-03-22T04:16:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.150+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da8068650 con 0x7f0da806e2f0 2024-03-22T04:16:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.150+0000 7f0da5600640 1 -- 172.21.15.86:0/3674883294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0d9c0b3070 con 0x7f0da806e2f0 2024-03-22T04:16:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.150+0000 7f0da5600640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d88067b20 0x7f0d88069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.150+0000 7f0da5600640 1 -- 172.21.15.86:0/3674883294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f0d9c120280 con 0x7f0da806e2f0 2024-03-22T04:16:38.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.150+0000 7f0dacc00640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d88067b20 0x7f0d88069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.150+0000 7f0dacc00640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d88067b20 0x7f0d88069fe0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0da8159400 tx=0x7f0d94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:38.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.154+0000 7f0da5600640 1 -- 172.21.15.86:0/3674883294 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0d9c016090 con 0x7f0da806e2f0 2024-03-22T04:16:38.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0da8161b60 con 0x7f0da806e2f0 2024-03-22T04:16:38.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0da5600640 1 -- 172.21.15.86:0/3674883294 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5188 (secure 0 0 0) 0x7f0d9c0f0a10 con 0x7f0da806e2f0 2024-03-22T04:16:38.445 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:38.445 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:16:07.415273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7002,"qdb_cluster":[7002]},"id":5}]} 2024-03-22T04:16:38.446 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 54 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d88067b20 msgr2=0x7f0d88069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d88067b20 0x7f0d88069fe0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0da8159400 tx=0x7f0d94009040 comp rx=0 tx=0).stop 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da806e2f0 msgr2=0x7f0da8156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da806e2f0 0x7f0da8156b20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c01a0d0 tx=0x7f0d9c019240 comp rx=0 tx=0).stop 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 shutdown_connections 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d88067b20 0x7f0d88069fe0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0da80d03b0 0x7f0da815c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0da815e630 0x7f0da814b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 --2- 172.21.15.86:0/3674883294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0da806e2f0 0x7f0da8156b20 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 >> 172.21.15.86:0/3674883294 conn(0x7f0da806f910 msgr2=0x7f0da806d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 shutdown_connections 2024-03-22T04:16:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.446+0000 7f0db0000640 1 -- 172.21.15.86:0/3674883294 wait complete. 2024-03-22T04:16:38.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:16:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.602+0000 7f57ffc00640 1 Processor -- start 2024-03-22T04:16:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.602+0000 7f57ffc00640 1 -- start start 2024-03-22T04:16:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57ffc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f57f80d0de0 0x7f57f80ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57ffc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f80d11c0 con 0x7f57f815fed0 2024-03-22T04:16:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57ffc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f80cece0 con 0x7f57f80d19a0 2024-03-22T04:16:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57ffc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f57f80d0580 con 0x7f57f80d0de0 2024-03-22T04:16:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57fd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f57f80d0de0 0x7f57f80ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57fdc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f57f815fed0 0x7f57f81602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40544/0 (socket says 172.21.15.86:40544) 2024-03-22T04:16:38.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57fdc00640 1 -- 172.21.15.86:0/2269022138 learned_addr learned my addr 172.21.15.86:0/2269022138 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57fd200640 1 -- 172.21.15.86:0/2269022138 >> v1:172.21.15.196:6789/0 conn(0x7f57f80d19a0 legacy=0x7f57f80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57fd200640 1 -- 172.21.15.86:0/2269022138 >> v1:172.21.15.86:6789/0 conn(0x7f57f815fed0 legacy=0x7f57f81602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57fd200640 1 -- 172.21.15.86:0/2269022138 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f80cefc0 con 0x7f57f80d0de0 2024-03-22T04:16:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57f7400640 1 -- 172.21.15.86:0/2269022138 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307225295 0 0) 0x7f57f80d11c0 con 0x7f57f815fed0 2024-03-22T04:16:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.606+0000 7f57fd200640 1 --2- 172.21.15.86:0/2269022138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f80d0de0 0x7f57f80ce6b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0097c0 tx=0x7f57ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f9ec12560e7899 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:38.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57f7400640 1 -- 172.21.15.86:0/2269022138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0b1070 con 0x7f57f80d0de0 2024-03-22T04:16:38.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57f7400640 1 -- 172.21.15.86:0/2269022138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec0ac5f0 con 0x7f57f80d0de0 2024-03-22T04:16:38.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57f7400640 1 -- 172.21.15.86:0/2269022138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0ac9c0 con 0x7f57f80d0de0 2024-03-22T04:16:38.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- 172.21.15.86:0/2269022138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f80d0de0 msgr2=0x7f57f80ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 --2- 172.21.15.86:0/2269022138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f80d0de0 0x7f57f80ce6b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0097c0 tx=0x7f57ec01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- 172.21.15.86:0/2269022138 shutdown_connections 2024-03-22T04:16:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 --2- 172.21.15.86:0/2269022138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f80d0de0 0x7f57f80ce6b0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- 172.21.15.86:0/2269022138 >> 172.21.15.86:0/2269022138 conn(0x7f57f806f910 msgr2=0x7f57f806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- 172.21.15.86:0/2269022138 shutdown_connections 2024-03-22T04:16:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- 172.21.15.86:0/2269022138 wait complete. 2024-03-22T04:16:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 Processor -- start 2024-03-22T04:16:38.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- start start 2024-03-22T04:16:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f80d0de0 0x7f57f81506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57f80d19a0 0x7f57f80c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815fed0 0x7f57f8159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f80d1f30 con 0x7f57f815fed0 2024-03-22T04:16:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57f80d1db0 con 0x7f57f80d0de0 2024-03-22T04:16:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57ffc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f57f8071e30 con 0x7f57f80d19a0 2024-03-22T04:16:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815fed0 0x7f57f8159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815fed0 0x7f57f8159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43096/0 (socket says 172.21.15.86:43096) 2024-03-22T04:16:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fdc00640 1 -- 172.21.15.86:0/2828393410 learned_addr learned my addr 172.21.15.86:0/2828393410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fd200640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f80d0de0 0x7f57f81506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57f7e00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57f80d19a0 0x7f57f80c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fdc00640 1 -- 172.21.15.86:0/2828393410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57f80d19a0 msgr2=0x7f57f80c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fdc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57f80d19a0 0x7f57f80c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fdc00640 1 -- 172.21.15.86:0/2828393410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f80d0de0 msgr2=0x7f57f81506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fdc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f80d0de0 0x7f57f81506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.610+0000 7f57fdc00640 1 -- 172.21.15.86:0/2828393410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f81527e0 con 0x7f57f815fed0 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57fdc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815fed0 0x7f57f8159120 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c7a0 tx=0x7f57f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57f4c00640 1 -- 172.21.15.86:0/2828393410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0018070 con 0x7f57f815fed0 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57f4c00640 1 -- 172.21.15.86:0/2828393410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f000a640 con 0x7f57f815fed0 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57f4c00640 1 -- 172.21.15.86:0/2828393410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f000aab0 con 0x7f57f815fed0 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f8159ae0 con 0x7f57f815fed0 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f8150c30 con 0x7f57f815fed0 2024-03-22T04:16:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f8068650 con 0x7f57f815fed0 2024-03-22T04:16:38.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57f4c00640 1 -- 172.21.15.86:0/2828393410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f57f000ac50 con 0x7f57f815fed0 2024-03-22T04:16:38.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.614+0000 7f57f4c00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57d8067b20 0x7f57d8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:38.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.618+0000 7f57f4c00640 1 -- 172.21.15.86:0/2828393410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f57f0085760 con 0x7f57f815fed0 2024-03-22T04:16:38.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.618+0000 7f57fd200640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57d8067b20 0x7f57d8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:38.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.618+0000 7f57fd200640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57d8067b20 0x7f57d8069fe0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ae000 tx=0x7f57ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:38.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.618+0000 7f57f4c00640 1 -- 172.21.15.86:0/2828393410 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f57f00558a0 con 0x7f57f815fed0 2024-03-22T04:16:38.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.830+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57f80cc5b0 con 0x7f57f815fed0 2024-03-22T04:16:38.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.830+0000 7f57f4c00640 1 -- 172.21.15.86:0/2828393410 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+17014 (secure 0 0 0) 0x7f57f0059940 con 0x7f57f815fed0 2024-03-22T04:16:38.830 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:38.830 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:20.393125+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1130704586":"2024-03-23T04:15:20.393105+0000","172.21.15.86:6836/1130704586":"2024-03-23T04:15:20.393105+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57d8067b20 msgr2=0x7f57d8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57d8067b20 0x7f57d8069fe0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ae000 tx=0x7f57ec008040 comp rx=0 tx=0).stop 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815fed0 msgr2=0x7f57f8159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815fed0 0x7f57f8159120 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c7a0 tx=0x7f57f000cc60 comp rx=0 tx=0).stop 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 shutdown_connections 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f57d8067b20 0x7f57d8069fe0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815fed0 0x7f57f8159120 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57f80d19a0 0x7f57f80c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 --2- 172.21.15.86:0/2828393410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57f80d0de0 0x7f57f81506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:38.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 >> 172.21.15.86:0/2828393410 conn(0x7f57f806f910 msgr2=0x7f57f815a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:38.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 shutdown_connections 2024-03-22T04:16:38.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.834+0000 7f57ffc00640 1 -- 172.21.15.86:0/2828393410 wait complete. 2024-03-22T04:16:38.855 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:16:38.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:16:39.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.998+0000 7f5690000640 1 Processor -- start 2024-03-22T04:16:39.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:38.998+0000 7f5690000640 1 -- start start 2024-03-22T04:16:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f5690000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f56880730d0 0x7f56880ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f5690000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5688071920 con 0x7f56880cc3e0 2024-03-22T04:16:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f5690000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5688071c00 con 0x7f568815b3f0 2024-03-22T04:16:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f5690000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f56880716d0 con 0x7f56880730d0 2024-03-22T04:16:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f568e000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f56880cc3e0 0x7f56880b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40546/0 (socket says 172.21.15.86:40546) 2024-03-22T04:16:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f568e000640 1 -- 172.21.15.86:0/1364132000 learned_addr learned my addr 172.21.15.86:0/1364132000 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f568cc00640 1 --2- 172.21.15.86:0/1364132000 >> v2:172.21.15.196:3300/0 conn(0x7f56880730d0 0x7f56880ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f5687e00640 1 -- 172.21.15.86:0/1364132000 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143067814 0 0) 0x7f5688071920 con 0x7f56880cc3e0 2024-03-22T04:16:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f5687e00640 1 -- 172.21.15.86:0/1364132000 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5670005740 con 0x7f56880cc3e0 2024-03-22T04:16:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f5687e00640 1 -- 172.21.15.86:0/1364132000 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2497814233 0 0) 0x7f5688071c00 con 0x7f568815b3f0 2024-03-22T04:16:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f5687e00640 1 -- 172.21.15.86:0/1364132000 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5688071920 con 0x7f568815b3f0 2024-03-22T04:16:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f568cc00640 1 -- 172.21.15.86:0/1364132000 >> v1:172.21.15.196:6789/0 conn(0x7f568815b3f0 legacy=0x7f56880729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f568cc00640 1 -- 172.21.15.86:0/1364132000 >> v1:172.21.15.86:6789/0 conn(0x7f56880cc3e0 legacy=0x7f56880b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.002+0000 7f568cc00640 1 -- 172.21.15.86:0/1364132000 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5688071ee0 con 0x7f56880730d0 2024-03-22T04:16:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f568cc00640 1 --2- 172.21.15.86:0/1364132000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880730d0 0x7f56880ade30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f56740097c0 tx=0x7f567401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b45df4f737863c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5687e00640 1 -- 172.21.15.86:0/1364132000 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740b0070 con 0x7f56880730d0 2024-03-22T04:16:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5687e00640 1 -- 172.21.15.86:0/1364132000 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56740ab600 con 0x7f56880730d0 2024-03-22T04:16:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5687e00640 1 -- 172.21.15.86:0/1364132000 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740ab9b0 con 0x7f56880730d0 2024-03-22T04:16:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5690000640 1 -- 172.21.15.86:0/1364132000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880730d0 msgr2=0x7f56880ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5690000640 1 --2- 172.21.15.86:0/1364132000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880730d0 0x7f56880ade30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f56740097c0 tx=0x7f567401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5690000640 1 -- 172.21.15.86:0/1364132000 shutdown_connections 2024-03-22T04:16:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5690000640 1 --2- 172.21.15.86:0/1364132000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880730d0 0x7f56880ade30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5690000640 1 -- 172.21.15.86:0/1364132000 >> 172.21.15.86:0/1364132000 conn(0x7f56880b1df0 msgr2=0x7f56880b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5690000640 1 -- 172.21.15.86:0/1364132000 shutdown_connections 2024-03-22T04:16:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.006+0000 7f5690000640 1 -- 172.21.15.86:0/1364132000 wait complete. 2024-03-22T04:16:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5690000640 1 Processor -- start 2024-03-22T04:16:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5690000640 1 -- start start 2024-03-22T04:16:39.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5690000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880730d0 0x7f568809a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5690000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880cc3e0 0x7f568809ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5690000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f568815b3f0 0x7f5688153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5690000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56880b30f0 con 0x7f56880730d0 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5690000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56880b3270 con 0x7f56880cc3e0 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5690000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f56880b33f0 con 0x7f568815b3f0 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880730d0 0x7f568809a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880730d0 0x7f568809a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43106/0 (socket says 172.21.15.86:43106) 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 -- 172.21.15.86:0/1333479435 learned_addr learned my addr 172.21.15.86:0/1333479435 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568cc00640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880cc3e0 0x7f568809ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568e000640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f568815b3f0 0x7f5688153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 -- 172.21.15.86:0/1333479435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f568815b3f0 msgr2=0x7f5688153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f568815b3f0 0x7f5688153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 -- 172.21.15.86:0/1333479435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880cc3e0 msgr2=0x7f568809ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880cc3e0 0x7f568809ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 -- 172.21.15.86:0/1333479435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568809d5a0 con 0x7f56880730d0 2024-03-22T04:16:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f568d600640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880730d0 0x7f568809a250 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c490 tx=0x7f567c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.010+0000 7f5685600640 1 -- 172.21.15.86:0/1333479435 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0052d0 con 0x7f56880730d0 2024-03-22T04:16:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568809d880 con 0x7f56880730d0 2024-03-22T04:16:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f5685600640 1 -- 172.21.15.86:0/1333479435 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c005470 con 0x7f56880730d0 2024-03-22T04:16:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f5685600640 1 -- 172.21.15.86:0/1333479435 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c012870 con 0x7f56880730d0 2024-03-22T04:16:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568809de40 con 0x7f56880730d0 2024-03-22T04:16:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568809e220 con 0x7f56880730d0 2024-03-22T04:16:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f5685600640 1 -- 172.21.15.86:0/1333479435 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f567c012a10 con 0x7f56880730d0 2024-03-22T04:16:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f5685600640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5668067b20 0x7f5668069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f5685600640 1 -- 172.21.15.86:0/1333479435 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f567c085e20 con 0x7f56880730d0 2024-03-22T04:16:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f568cc00640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5668067b20 0x7f5668069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.014+0000 7f568cc00640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5668067b20 0x7f5668069fe0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f56740ad000 tx=0x7f5674008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.018+0000 7f5685600640 1 -- 172.21.15.86:0/1333479435 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f568809e220 con 0x7f56880730d0 2024-03-22T04:16:39.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.210+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5688068650 con 0x7f56880730d0 2024-03-22T04:16:39.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.210+0000 7f5685600640 1 -- 172.21.15.86:0/1333479435 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5188 (secure 0 0 0) 0x7f567c056530 con 0x7f56880730d0 2024-03-22T04:16:39.211 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:39.211 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:16:07.415273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7002,"qdb_cluster":[7002]},"id":5}]} 2024-03-22T04:16:39.212 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 54 2024-03-22T04:16:39.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5668067b20 msgr2=0x7f5668069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5668067b20 0x7f5668069fe0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f56740ad000 tx=0x7f5674008040 comp rx=0 tx=0).stop 2024-03-22T04:16:39.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880730d0 msgr2=0x7f568809a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880730d0 0x7f568809a250 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c490 tx=0x7f567c00c950 comp rx=0 tx=0).stop 2024-03-22T04:16:39.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 shutdown_connections 2024-03-22T04:16:39.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5668067b20 0x7f5668069fe0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56880cc3e0 0x7f568809ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f568815b3f0 0x7f5688153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 --2- 172.21.15.86:0/1333479435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880730d0 0x7f568809a250 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 >> 172.21.15.86:0/1333479435 conn(0x7f56880b1df0 msgr2=0x7f56880b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:39.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 shutdown_connections 2024-03-22T04:16:39.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.214+0000 7f5690000640 1 -- 172.21.15.86:0/1333479435 wait complete. 2024-03-22T04:16:39.232 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:16:39.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.366+0000 7f436ba00640 1 Processor -- start 2024-03-22T04:16:39.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.366+0000 7f436ba00640 1 -- start start 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f436ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f436415e630 0x7f43640cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f436ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436406e2f0 0x7f43640d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f436ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43641623d0 con 0x7f436406cc80 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f436ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f43641620f0 con 0x7f436415e630 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f436ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f436415ec10 con 0x7f436406e2f0 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f4369000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f436415e630 0x7f43640cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f4369000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f436415e630 0x7f43640cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43122/0 (socket says 172.21.15.86:43122) 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f4369000640 1 -- 172.21.15.86:0/2410795631 learned_addr learned my addr 172.21.15.86:0/2410795631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f4363e00640 1 --2- 172.21.15.86:0/2410795631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436406e2f0 0x7f43640d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f4369000640 1 -- 172.21.15.86:0/2410795631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436406e2f0 msgr2=0x7f43640d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f4369000640 1 --2- 172.21.15.86:0/2410795631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436406e2f0 0x7f43640d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f4369000640 1 -- 172.21.15.86:0/2410795631 >> v1:172.21.15.86:6789/0 conn(0x7f436406cc80 legacy=0x7f436406d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.370+0000 7f4369000640 1 -- 172.21.15.86:0/2410795631 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436415eef0 con 0x7f436415e630 2024-03-22T04:16:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f4369000640 1 --2- 172.21.15.86:0/2410795631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436415e630 0x7f43640cfe70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4358009870 tx=0x7f435801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4cf44399413d9e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f4363400640 1 -- 172.21.15.86:0/2410795631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43580b2070 con 0x7f436415e630 2024-03-22T04:16:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f4363400640 1 -- 172.21.15.86:0/2410795631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43580a9070 con 0x7f436415e630 2024-03-22T04:16:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f4363400640 1 -- 172.21.15.86:0/2410795631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43580ad880 con 0x7f436415e630 2024-03-22T04:16:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f436ba00640 1 -- 172.21.15.86:0/2410795631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436415e630 msgr2=0x7f43640cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f436ba00640 1 --2- 172.21.15.86:0/2410795631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436415e630 0x7f43640cfe70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4358009870 tx=0x7f435801c560 comp rx=0 tx=0).stop 2024-03-22T04:16:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f436ba00640 1 -- 172.21.15.86:0/2410795631 shutdown_connections 2024-03-22T04:16:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f436ba00640 1 --2- 172.21.15.86:0/2410795631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436406e2f0 0x7f43640d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f436ba00640 1 --2- 172.21.15.86:0/2410795631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436415e630 0x7f43640cfe70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f436ba00640 1 -- 172.21.15.86:0/2410795631 >> 172.21.15.86:0/2410795631 conn(0x7f436406f910 msgr2=0x7f436406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f436ba00640 1 -- 172.21.15.86:0/2410795631 shutdown_connections 2024-03-22T04:16:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.374+0000 7f436ba00640 1 -- 172.21.15.86:0/2410795631 wait complete. 2024-03-22T04:16:39.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 Processor -- start 2024-03-22T04:16:39.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 -- start start 2024-03-22T04:16:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436406cc80 0x7f43640c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436406e2f0 0x7f43640b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436415e630 0x7f436415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43640cfaf0 con 0x7f436406e2f0 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f436415f9d0 con 0x7f436406cc80 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f43640cfc70 con 0x7f436415e630 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436406e2f0 0x7f43640b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436406e2f0 0x7f43640b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43134/0 (socket says 172.21.15.86:43134) 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 -- 172.21.15.86:0/66699567 learned_addr learned my addr 172.21.15.86:0/66699567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 -- 172.21.15.86:0/66699567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436415e630 msgr2=0x7f436415bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4369000640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436406cc80 0x7f43640c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436415e630 0x7f436415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 -- 172.21.15.86:0/66699567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436406cc80 msgr2=0x7f43640c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436406cc80 0x7f43640c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 -- 172.21.15.86:0/66699567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43640b56d0 con 0x7f436406e2f0 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4369000640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436406cc80 0x7f43640c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4363e00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436406e2f0 0x7f43640b4f60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f435000e870 tx=0x7f435000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4360c00640 1 -- 172.21.15.86:0/66699567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350017070 con 0x7f436406e2f0 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4360c00640 1 -- 172.21.15.86:0/66699567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435000c940 con 0x7f436406e2f0 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f4360c00640 1 -- 172.21.15.86:0/66699567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435000ccf0 con 0x7f436406e2f0 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43640b59b0 con 0x7f436406e2f0 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.378+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43640b5f70 con 0x7f436406e2f0 2024-03-22T04:16:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.382+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4364068650 con 0x7f436406e2f0 2024-03-22T04:16:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.382+0000 7f4360c00640 1 -- 172.21.15.86:0/66699567 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f435001a070 con 0x7f436406e2f0 2024-03-22T04:16:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.382+0000 7f4360c00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4344067b50 0x7f434406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.382+0000 7f4360c00640 1 -- 172.21.15.86:0/66699567 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f4350086350 con 0x7f436406e2f0 2024-03-22T04:16:39.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.382+0000 7f4369000640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4344067b50 0x7f434406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.382+0000 7f4369000640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4344067b50 0x7f434406a010 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f43580af000 tx=0x7f4358008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.386+0000 7f4360c00640 1 -- 172.21.15.86:0/66699567 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4350056b70 con 0x7f436406e2f0 2024-03-22T04:16:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.574+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f436406d5b0 con 0x7f436406e2f0 2024-03-22T04:16:39.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.574+0000 7f4360c00640 1 -- 172.21.15.86:0/66699567 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5188 (secure 0 0 0) 0x7f4350056470 con 0x7f436406e2f0 2024-03-22T04:16:39.575 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:39.575 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:15:09.433883+0000","modified":"2024-03-22T04:16:07.415273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.86:6837/736870197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":736870197},{"type":"v1","addr":"172.21.15.86:6837","nonce":736870197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7002,"qdb_cluster":[7002]},"id":5}]} 2024-03-22T04:16:39.575 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 54 2024-03-22T04:16:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4344067b50 msgr2=0x7f434406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4344067b50 0x7f434406a010 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f43580af000 tx=0x7f4358008040 comp rx=0 tx=0).stop 2024-03-22T04:16:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436406e2f0 msgr2=0x7f43640b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436406e2f0 0x7f43640b4f60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f435000e870 tx=0x7f435000ec70 comp rx=0 tx=0).stop 2024-03-22T04:16:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 shutdown_connections 2024-03-22T04:16:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4344067b50 0x7f434406a010 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436415e630 0x7f436415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436406e2f0 0x7f43640b4f60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 --2- 172.21.15.86:0/66699567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436406cc80 0x7f43640c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 >> 172.21.15.86:0/66699567 conn(0x7f436406f910 msgr2=0x7f4364168970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 shutdown_connections 2024-03-22T04:16:39.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.578+0000 7f436ba00640 1 -- 172.21.15.86:0/66699567 wait complete. 2024-03-22T04:16:39.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:16:39.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.730+0000 7fe1c4800640 1 Processor -- start 2024-03-22T04:16:39.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.730+0000 7fe1c4800640 1 -- start start 2024-03-22T04:16:39.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c4800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe1bc06e2f0 0x7fe1bc071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c4800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1bc1623d0 con 0x7fe1bc0d0580 2024-03-22T04:16:39.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c4800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1bc1620f0 con 0x7fe1bc15e630 2024-03-22T04:16:39.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c4800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe1bc15ec10 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe1bc0d0580 0x7fe1bc071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40556/0 (socket says 172.21.15.86:40556) 2024-03-22T04:16:39.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c1400640 1 -- 172.21.15.86:0/872624776 learned_addr learned my addr 172.21.15.86:0/872624776 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:39.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c2800640 1 --2- 172.21.15.86:0/872624776 >> v2:172.21.15.86:3300/0 conn(0x7fe1bc06e2f0 0x7fe1bc071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c2800640 1 -- 172.21.15.86:0/872624776 >> v1:172.21.15.196:6789/0 conn(0x7fe1bc15e630 legacy=0x7fe1bc0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c2800640 1 -- 172.21.15.86:0/872624776 >> v1:172.21.15.86:6789/0 conn(0x7fe1bc0d0580 legacy=0x7fe1bc071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c2800640 1 -- 172.21.15.86:0/872624776 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc15eef0 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.734+0000 7fe1c0a00640 1 -- 172.21.15.86:0/872624776 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713433957 0 0) 0x7fe1bc1623d0 con 0x7fe1bc0d0580 2024-03-22T04:16:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c2800640 1 --2- 172.21.15.86:0/872624776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc071810 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009d90 tx=0x7fe1b801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305979b14323b6b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c0a00640 1 -- 172.21.15.86:0/872624776 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80b2070 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c0a00640 1 -- 172.21.15.86:0/872624776 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b80a5830 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c0a00640 1 -- 172.21.15.86:0/872624776 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80a5c00 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 -- 172.21.15.86:0/872624776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 msgr2=0x7fe1bc071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 --2- 172.21.15.86:0/872624776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc071810 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009d90 tx=0x7fe1b801c970 comp rx=0 tx=0).stop 2024-03-22T04:16:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 -- 172.21.15.86:0/872624776 shutdown_connections 2024-03-22T04:16:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 --2- 172.21.15.86:0/872624776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc071810 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 -- 172.21.15.86:0/872624776 >> 172.21.15.86:0/872624776 conn(0x7fe1bc06f910 msgr2=0x7fe1bc06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 -- 172.21.15.86:0/872624776 shutdown_connections 2024-03-22T04:16:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 -- 172.21.15.86:0/872624776 wait complete. 2024-03-22T04:16:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 Processor -- start 2024-03-22T04:16:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.738+0000 7fe1c4800640 1 -- start start 2024-03-22T04:16:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc0d0580 0x7fe1bc15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc15e630 0x7fe1bc153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc0cfaf0 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc15f9d0 con 0x7fe1bc0d0580 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1bc0cfc70 con 0x7fe1bc15e630 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc0c5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43148/0 (socket says 172.21.15.86:43148) 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 -- 172.21.15.86:0/3202363619 learned_addr learned my addr 172.21.15.86:0/3202363619 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1400640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc0d0580 0x7fe1bc15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c2800640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc15e630 0x7fe1bc153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 -- 172.21.15.86:0/3202363619 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc15e630 msgr2=0x7fe1bc153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc15e630 0x7fe1bc153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 -- 172.21.15.86:0/3202363619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc0d0580 msgr2=0x7fe1bc15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc0d0580 0x7fe1bc15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 -- 172.21.15.86:0/3202363619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc0b3290 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c1e00640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc0c5d90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c500 tx=0x7fe1b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1b6000640 1 -- 172.21.15.86:0/3202363619 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0016020 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1b6000640 1 -- 172.21.15.86:0/3202363619 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b0005220 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc0c9180 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1b6000640 1 -- 172.21.15.86:0/3202363619 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0007030 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc15a600 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc068650 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.742+0000 7fe1b6000640 1 -- 172.21.15.86:0/3202363619 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe1b00071f0 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.746+0000 7fe1b6000640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe1a0067b20 0x7fe1a0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.746+0000 7fe1b6000640 1 -- 172.21.15.86:0/3202363619 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7fe1b0085b40 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.746+0000 7fe1c1400640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe1a0067b20 0x7fe1a0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:39.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.746+0000 7fe1c1400640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe1a0067b20 0x7fe1a0069fe0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007fd0 tx=0x7fe1ac007ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:39.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.750+0000 7fe1b6000640 1 -- 172.21.15.86:0/3202363619 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe1b0020020 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.910+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1bc0b2160 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.910+0000 7fe1b6000640 1 -- 172.21.15.86:0/3202363619 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+17014 (secure 0 0 0) 0x7fe1b00562e0 con 0x7fe1bc06e2f0 2024-03-22T04:16:39.911 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:39.911 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:15:20.393125+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:15:07.783245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-22T04:15:08.394997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1130704586":"2024-03-23T04:15:20.393105+0000","172.21.15.86:6836/1130704586":"2024-03-23T04:15:20.393105+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:16:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe1a0067b20 msgr2=0x7fe1a0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe1a0067b20 0x7fe1a0069fe0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007fd0 tx=0x7fe1ac007ba0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 msgr2=0x7fe1bc0c5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc0c5d90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c500 tx=0x7fe1b000c9c0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 shutdown_connections 2024-03-22T04:16:39.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe1a0067b20 0x7fe1a0069fe0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc0d0580 0x7fe1bc15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc15e630 0x7fe1bc153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 --2- 172.21.15.86:0/3202363619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc06e2f0 0x7fe1bc0c5d90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:39.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 >> 172.21.15.86:0/3202363619 conn(0x7fe1bc06f910 msgr2=0x7fe1bc0b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:39.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 shutdown_connections 2024-03-22T04:16:39.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:39.914+0000 7fe1c4800640 1 -- 172.21.15.86:0/3202363619 wait complete. 2024-03-22T04:16:39.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:16:40.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.070+0000 7f91ba000640 1 Processor -- start 2024-03-22T04:16:40.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.070+0000 7f91ba000640 1 -- start start 2024-03-22T04:16:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91ba000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f91b415b3f0 0x7f91b40729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91ba000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b40cc3e0 0x7f91b4072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91ba000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b4071920 con 0x7f91b40b30f0 2024-03-22T04:16:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91ba000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f91b40716d0 con 0x7f91b415b3f0 2024-03-22T04:16:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91ba000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f91b406d990 con 0x7f91b40cc3e0 2024-03-22T04:16:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f91b40b30f0 0x7f91b40ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40564/0 (socket says 172.21.15.86:40564) 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b3e00640 1 -- 172.21.15.86:0/4081123307 learned_addr learned my addr 172.21.15.86:0/4081123307 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b3400640 1 --2- 172.21.15.86:0/4081123307 >> v2:172.21.15.196:3300/0 conn(0x7f91b415b3f0 0x7f91b40729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b2a00640 1 --2- 172.21.15.86:0/4081123307 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b40cc3e0 0x7f91b4072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b2000640 1 -- 172.21.15.86:0/4081123307 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1694177271 0 0) 0x7f91b4071920 con 0x7f91b40b30f0 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b2000640 1 -- 172.21.15.86:0/4081123307 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9198003530 con 0x7f91b40b30f0 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b3400640 1 -- 172.21.15.86:0/4081123307 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b40cc3e0 msgr2=0x7f91b4072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b3400640 1 --2- 172.21.15.86:0/4081123307 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b40cc3e0 0x7f91b4072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b3400640 1 -- 172.21.15.86:0/4081123307 >> v1:172.21.15.86:6789/0 conn(0x7f91b40b30f0 legacy=0x7f91b40ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.074+0000 7f91b3400640 1 -- 172.21.15.86:0/4081123307 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b4071c00 con 0x7f91b415b3f0 2024-03-22T04:16:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91b3400640 1 --2- 172.21.15.86:0/4081123307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b415b3f0 0x7f91b40729c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f91a40097c0 tx=0x7f91a401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ec444be94167b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91b2000640 1 -- 172.21.15.86:0/4081123307 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a40b0070 con 0x7f91b415b3f0 2024-03-22T04:16:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91b2000640 1 -- 172.21.15.86:0/4081123307 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a40ab600 con 0x7f91b415b3f0 2024-03-22T04:16:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91b2000640 1 -- 172.21.15.86:0/4081123307 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a40ab9d0 con 0x7f91b415b3f0 2024-03-22T04:16:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 -- 172.21.15.86:0/4081123307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b415b3f0 msgr2=0x7f91b40729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 --2- 172.21.15.86:0/4081123307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b415b3f0 0x7f91b40729c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f91a40097c0 tx=0x7f91a401c410 comp rx=0 tx=0).stop 2024-03-22T04:16:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 -- 172.21.15.86:0/4081123307 shutdown_connections 2024-03-22T04:16:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 --2- 172.21.15.86:0/4081123307 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b40cc3e0 0x7f91b4072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 --2- 172.21.15.86:0/4081123307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b415b3f0 0x7f91b40729c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 -- 172.21.15.86:0/4081123307 >> 172.21.15.86:0/4081123307 conn(0x7f91b40b1df0 msgr2=0x7f91b40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 -- 172.21.15.86:0/4081123307 shutdown_connections 2024-03-22T04:16:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 -- 172.21.15.86:0/4081123307 wait complete. 2024-03-22T04:16:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 Processor -- start 2024-03-22T04:16:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.078+0000 7f91ba000640 1 -- start start 2024-03-22T04:16:40.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91b40b30f0 0x7f91b41fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b40cc3e0 0x7f91b40991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b415b3f0 0x7f91b414e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4168580 con 0x7f91b40b30f0 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4168400 con 0x7f91b40cc3e0 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f91b4168700 con 0x7f91b415b3f0 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91b40b30f0 0x7f91b41fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91b40b30f0 0x7f91b41fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43154/0 (socket says 172.21.15.86:43154) 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 -- 172.21.15.86:0/2606420980 learned_addr learned my addr 172.21.15.86:0/2606420980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b2a00640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b40cc3e0 0x7f91b40991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3e00640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b415b3f0 0x7f91b414e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 -- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b415b3f0 msgr2=0x7f91b414e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b415b3f0 0x7f91b414e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 -- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b40cc3e0 msgr2=0x7f91b40991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b40cc3e0 0x7f91b40991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 -- 172.21.15.86:0/2606420980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b415b8c0 con 0x7f91b40b30f0 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3e00640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b415b3f0 0x7f91b414e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91b3400640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91b40b30f0 0x7f91b41fbb80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f91a401a040 tx=0x7f91a401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ab400640 1 -- 172.21.15.86:0/2606420980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a40b0050 con 0x7f91b40b30f0 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ab400640 1 -- 172.21.15.86:0/2606420980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a40bc070 con 0x7f91b40b30f0 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ab400640 1 -- 172.21.15.86:0/2606420980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4006e20 con 0x7f91b40b30f0 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b4151980 con 0x7f91b40b30f0 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b4153650 con 0x7f91b40b30f0 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b4153160 con 0x7f91b40b30f0 2024-03-22T04:16:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.082+0000 7f91ab400640 1 -- 172.21.15.86:0/2606420980 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f91a40b5830 con 0x7f91b40b30f0 2024-03-22T04:16:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.086+0000 7f91ab400640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9194067b20 0x7f9194069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.086+0000 7f91ab400640 1 -- 172.21.15.86:0/2606420980 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f91a4120760 con 0x7f91b40b30f0 2024-03-22T04:16:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.086+0000 7f91b2a00640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9194067b20 0x7f9194069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.086+0000 7f91b2a00640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9194067b20 0x7f9194069fe0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f91b414dcd0 tx=0x7f91a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:40.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.090+0000 7f91ab400640 1 -- 172.21.15.86:0/2606420980 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f91a4016090 con 0x7f91b40b30f0 2024-03-22T04:16:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.278+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f91b4068650 con 0x7f91b40b30f0 2024-03-22T04:16:40.280 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:16:40.278+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:16:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.282+0000 7f91ab400640 1 -- 172.21.15.86:0/2606420980 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) ==== 161+0+0 (secure 0 0 0) 0x7f91a40f0f90 con 0x7f91b40b30f0 2024-03-22T04:16:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9194067b20 msgr2=0x7f9194069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9194067b20 0x7f9194069fe0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f91b414dcd0 tx=0x7f91a0008040 comp rx=0 tx=0).stop 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91b40b30f0 msgr2=0x7f91b41fbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91b40b30f0 0x7f91b41fbb80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f91a401a040 tx=0x7f91a401ac50 comp rx=0 tx=0).stop 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 shutdown_connections 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9194067b20 0x7f9194069fe0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b40cc3e0 0x7f91b40991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91b415b3f0 0x7f91b414e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 --2- 172.21.15.86:0/2606420980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91b40b30f0 0x7f91b41fbb80 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 >> 172.21.15.86:0/2606420980 conn(0x7f91b40b1df0 msgr2=0x7f91b415bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 shutdown_connections 2024-03-22T04:16:40.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.286+0000 7f91ba000640 1 -- 172.21.15.86:0/2606420980 wait complete. 2024-03-22T04:16:40.295 INFO:tasks.ceph.mds.a.smithi086.stderr: -4234> 2024-03-22T04:15:26.407+0000 7febf8c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:16:40.308 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T04:16:40.347 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:16:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6893200640 1 Processor -- start 2024-03-22T04:16:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6893200640 1 -- start start 2024-03-22T04:16:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6893200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f689415beb0 0x7f689415c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6893200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689406fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6893200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689406df90 con 0x7f6894072170 2024-03-22T04:16:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6893200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f689406e270 con 0x7f689415beb0 2024-03-22T04:16:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6893200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f689406e550 con 0x7f689406dab0 2024-03-22T04:16:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6891e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f689415beb0 0x7f689415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6892800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6894072170 0x7f68940ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40568/0 (socket says 172.21.15.86:40568) 2024-03-22T04:16:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.450+0000 7f6892800640 1 -- 172.21.15.86:0/1188212358 learned_addr learned my addr 172.21.15.86:0/1188212358 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6889400640 1 --2- 172.21.15.86:0/1188212358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689406fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891e00640 1 -- 172.21.15.86:0/1188212358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 msgr2=0x7f689406fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891e00640 1 --2- 172.21.15.86:0/1188212358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689406fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891e00640 1 -- 172.21.15.86:0/1188212358 >> v1:172.21.15.86:6789/0 conn(0x7f6894072170 legacy=0x7f68940ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891e00640 1 -- 172.21.15.86:0/1188212358 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6894071c50 con 0x7f689415beb0 2024-03-22T04:16:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891400640 1 -- 172.21.15.86:0/1188212358 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2566956747 0 0) 0x7f689406df90 con 0x7f6894072170 2024-03-22T04:16:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6889400640 1 --2- 172.21.15.86:0/1188212358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689406fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891e00640 1 --2- 172.21.15.86:0/1188212358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f689415c290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6884009d90 tx=0x7f688401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3944753695c5834b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891400640 1 -- 172.21.15.86:0/1188212358 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840b2070 con 0x7f689415beb0 2024-03-22T04:16:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891400640 1 -- 172.21.15.86:0/1188212358 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68840a59c0 con 0x7f689415beb0 2024-03-22T04:16:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6891400640 1 -- 172.21.15.86:0/1188212358 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840a5d90 con 0x7f689415beb0 2024-03-22T04:16:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6893200640 1 -- 172.21.15.86:0/1188212358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 msgr2=0x7f689415c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6893200640 1 --2- 172.21.15.86:0/1188212358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f689415c290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6884009d90 tx=0x7f688401c8a0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6893200640 1 -- 172.21.15.86:0/1188212358 shutdown_connections 2024-03-22T04:16:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6893200640 1 --2- 172.21.15.86:0/1188212358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689406fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6893200640 1 --2- 172.21.15.86:0/1188212358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f689415c290 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6893200640 1 -- 172.21.15.86:0/1188212358 >> 172.21.15.86:0/1188212358 conn(0x7f68940b2130 msgr2=0x7f68940b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6893200640 1 -- 172.21.15.86:0/1188212358 shutdown_connections 2024-03-22T04:16:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.454+0000 7f6893200640 1 -- 172.21.15.86:0/1188212358 wait complete. 2024-03-22T04:16:40.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 Processor -- start 2024-03-22T04:16:40.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 -- start start 2024-03-22T04:16:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894072170 0x7f689414afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f68940c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68940cd6a0 con 0x7f689415beb0 2024-03-22T04:16:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6894160700 con 0x7f6894072170 2024-03-22T04:16:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f68940cd820 con 0x7f689406dab0 2024-03-22T04:16:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f68940c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f68940c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43192/0 (socket says 172.21.15.86:43192) 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 -- 172.21.15.86:0/196493729 learned_addr learned my addr 172.21.15.86:0/196493729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6891e00640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6889400640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894072170 0x7f689414afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 -- 172.21.15.86:0/196493729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 msgr2=0x7f689414e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689414e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 -- 172.21.15.86:0/196493729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894072170 msgr2=0x7f689414afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894072170 0x7f689414afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 -- 172.21.15.86:0/196493729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68940c7730 con 0x7f689415beb0 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6892800640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f68940c9e30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c480 tx=0x7f688c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f688aa00640 1 -- 172.21.15.86:0/196493729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c018070 con 0x7f689415beb0 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f688aa00640 1 -- 172.21.15.86:0/196493729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688c0074e0 con 0x7f689415beb0 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f688aa00640 1 -- 172.21.15.86:0/196493729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c004a30 con 0x7f689415beb0 2024-03-22T04:16:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68940c7a10 con 0x7f689415beb0 2024-03-22T04:16:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.458+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941513e0 con 0x7f689415beb0 2024-03-22T04:16:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.462+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68940c7f10 con 0x7f689415beb0 2024-03-22T04:16:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.462+0000 7f688aa00640 1 -- 172.21.15.86:0/196493729 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f688c004bd0 con 0x7f689415beb0 2024-03-22T04:16:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.462+0000 7f688aa00640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6870067b20 0x7f6870069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.462+0000 7f688aa00640 1 -- 172.21.15.86:0/196493729 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6273+0+0 (secure 0 0 0) 0x7f688c0854e0 con 0x7f689415beb0 2024-03-22T04:16:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.462+0000 7f6891e00640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6870067b20 0x7f6870069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.462+0000 7f6891e00640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6870067b20 0x7f6870069fe0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f68840af000 tx=0x7f68840a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:40.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.466+0000 7f688aa00640 1 -- 172.21.15.86:0/196493729 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f68940c7f10 con 0x7f689415beb0 2024-03-22T04:16:40.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:40.658+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f68941515e0 con 0x7f689415beb0 2024-03-22T04:16:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.282+0000 7f688aa00640 1 -- 172.21.15.86:0/196493729 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) ==== 104+0+0 (secure 0 0 0) 0x7f68941515e0 con 0x7f689415beb0 2024-03-22T04:16:41.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6870067b20 msgr2=0x7f6870069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:41.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6870067b20 0x7f6870069fe0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f68840af000 tx=0x7f68840a7070 comp rx=0 tx=0).stop 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 msgr2=0x7f68940c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f68940c9e30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c480 tx=0x7f688c00c940 comp rx=0 tx=0).stop 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 shutdown_connections 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6870067b20 0x7f6870069fe0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894072170 0x7f689414afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f689415beb0 0x7f68940c9e30 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 --2- 172.21.15.86:0/196493729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689406dab0 0x7f689414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 >> 172.21.15.86:0/196493729 conn(0x7f68940b2130 msgr2=0x7f68940c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 shutdown_connections 2024-03-22T04:16:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.286+0000 7f6893200640 1 -- 172.21.15.86:0/196493729 wait complete. 2024-03-22T04:16:41.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T04:16:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1a00640 1 Processor -- start 2024-03-22T04:16:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1a00640 1 -- start start 2024-03-22T04:16:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f399c156600 0x7f399c15d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f399c06e450 0x7f399c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f399c15d670 con 0x7f399c06f720 2024-03-22T04:16:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f399c0cc870 con 0x7f399c06e450 2024-03-22T04:16:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f399c0ccac0 con 0x7f399c156600 2024-03-22T04:16:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f399c06e450 0x7f399c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:41.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f3993400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f399c06f720 0x7f399c06dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40576/0 (socket says 172.21.15.86:40576) 2024-03-22T04:16:41.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f3993400640 1 -- 172.21.15.86:0/447118215 learned_addr learned my addr 172.21.15.86:0/447118215 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:41.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f399be00640 1 --2- 172.21.15.86:0/447118215 >> v2:172.21.15.196:3300/0 conn(0x7f399c156600 0x7f399c15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:41.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1000640 1 -- 172.21.15.86:0/447118215 >> v2:172.21.15.196:3300/0 conn(0x7f399c156600 msgr2=0x7f399c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:41.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1000640 1 --2- 172.21.15.86:0/447118215 >> v2:172.21.15.196:3300/0 conn(0x7f399c156600 0x7f399c15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1000640 1 -- 172.21.15.86:0/447118215 >> v1:172.21.15.86:6789/0 conn(0x7f399c06f720 legacy=0x7f399c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:41.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f39a1000640 1 -- 172.21.15.86:0/447118215 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c0ccda0 con 0x7f399c06e450 2024-03-22T04:16:41.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f399be00640 1 --2- 172.21.15.86:0/447118215 >> v2:172.21.15.196:3300/0 conn(0x7f399c156600 0x7f399c15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:41.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.494+0000 7f399b400640 1 -- 172.21.15.86:0/447118215 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2882818826 0 0) 0x7f399c15d670 con 0x7f399c06f720 2024-03-22T04:16:41.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1000640 1 --2- 172.21.15.86:0/447118215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06e450 0x7f399c071d00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f399c068b50 tx=0x7f399401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158f01ece9532d46 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:41.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f399b400640 1 -- 172.21.15.86:0/447118215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39940b2070 con 0x7f399c06e450 2024-03-22T04:16:41.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f399b400640 1 -- 172.21.15.86:0/447118215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39940a5760 con 0x7f399c06e450 2024-03-22T04:16:41.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f399b400640 1 -- 172.21.15.86:0/447118215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39940a5b30 con 0x7f399c06e450 2024-03-22T04:16:41.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1a00640 1 -- 172.21.15.86:0/447118215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06e450 msgr2=0x7f399c071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:41.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1a00640 1 --2- 172.21.15.86:0/447118215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06e450 0x7f399c071d00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f399c068b50 tx=0x7f399401c8a0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1a00640 1 -- 172.21.15.86:0/447118215 shutdown_connections 2024-03-22T04:16:41.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1a00640 1 --2- 172.21.15.86:0/447118215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06e450 0x7f399c071d00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1a00640 1 --2- 172.21.15.86:0/447118215 >> v2:172.21.15.196:3300/0 conn(0x7f399c156600 0x7f399c15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1a00640 1 -- 172.21.15.86:0/447118215 >> 172.21.15.86:0/447118215 conn(0x7f399c06b660 msgr2=0x7f399c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:41.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1a00640 1 -- 172.21.15.86:0/447118215 shutdown_connections 2024-03-22T04:16:41.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.498+0000 7f39a1a00640 1 -- 172.21.15.86:0/447118215 wait complete. 2024-03-22T04:16:41.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 Processor -- start 2024-03-22T04:16:41.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 -- start start 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f399c06e450 0x7f399c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06f720 0x7f399c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f399c156600 0x7f399c144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f399c072a50 con 0x7f399c06f720 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f399c156bf0 con 0x7f399c156600 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f399c072bd0 con 0x7f399c06e450 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06f720 0x7f399c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06f720 0x7f399c145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43200/0 (socket says 172.21.15.86:43200) 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 -- 172.21.15.86:0/1156149651 learned_addr learned my addr 172.21.15.86:0/1156149651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f399be00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f399c06e450 0x7f399c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1000640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f399c156600 0x7f399c144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 -- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f399c06e450 msgr2=0x7f399c0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f399c06e450 0x7f399c0b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 -- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f399c156600 msgr2=0x7f399c144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f399c156600 0x7f399c144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 -- 172.21.15.86:0/1156149651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c154320 con 0x7f399c06f720 2024-03-22T04:16:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1000640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f399c156600 0x7f399c144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3993400640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06f720 0x7f399c145a90 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f39840011f0 tx=0x7f398400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3998c00640 1 -- 172.21.15.86:0/1156149651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984018070 con 0x7f399c06f720 2024-03-22T04:16:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3998c00640 1 -- 172.21.15.86:0/1156149651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39840074e0 con 0x7f399c06f720 2024-03-22T04:16:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399c154600 con 0x7f399c06f720 2024-03-22T04:16:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f3998c00640 1 -- 172.21.15.86:0/1156149651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984004a10 con 0x7f399c06f720 2024-03-22T04:16:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.502+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399c154bc0 con 0x7f399c06f720 2024-03-22T04:16:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.506+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399c06c5f0 con 0x7f399c06f720 2024-03-22T04:16:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.506+0000 7f3998c00640 1 -- 172.21.15.86:0/1156149651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f398401b070 con 0x7f399c06f720 2024-03-22T04:16:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.506+0000 7f3998c00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3978067b20 0x7f3978069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.506+0000 7f3998c00640 1 -- 172.21.15.86:0/1156149651 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6273+0+0 (secure 0 0 0) 0x7f39840854e0 con 0x7f399c06f720 2024-03-22T04:16:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.506+0000 7f399be00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3978067b20 0x7f3978069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.506+0000 7f399be00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3978067b20 0x7f3978069fe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f398c006fd0 tx=0x7f398c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.510+0000 7f3998c00640 1 -- 172.21.15.86:0/1156149651 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3984055ca0 con 0x7f399c06f720 2024-03-22T04:16:41.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:41.666+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f399c14f320 con 0x7f399c06f720 2024-03-22T04:16:42.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.290+0000 7f3998c00640 1 -- 172.21.15.86:0/1156149651 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) ==== 181+0+0 (secure 0 0 0) 0x7f39840555a0 con 0x7f399c06f720 2024-03-22T04:16:42.291 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T04:16:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3978067b20 msgr2=0x7f3978069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3978067b20 0x7f3978069fe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f398c006fd0 tx=0x7f398c008040 comp rx=0 tx=0).stop 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06f720 msgr2=0x7f399c145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06f720 0x7f399c145a90 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f39840011f0 tx=0x7f398400c9b0 comp rx=0 tx=0).stop 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 shutdown_connections 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3978067b20 0x7f3978069fe0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f399c156600 0x7f399c144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f399c06f720 0x7f399c145a90 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 --2- 172.21.15.86:0/1156149651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f399c06e450 0x7f399c0b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 >> 172.21.15.86:0/1156149651 conn(0x7f399c06b660 msgr2=0x7f399c0d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 shutdown_connections 2024-03-22T04:16:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.294+0000 7f39a1a00640 1 -- 172.21.15.86:0/1156149651 wait complete. 2024-03-22T04:16:42.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T04:16:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.454+0000 7f8f01600640 1 Processor -- start 2024-03-22T04:16:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.454+0000 7f8f01600640 1 -- start start 2024-03-22T04:16:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8f01600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8efc144a90 0x7f8efc0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8f01600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8efc144e70 con 0x7f8efc072cf0 2024-03-22T04:16:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8f01600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8efc0b3e00 con 0x7f8efc0b37c0 2024-03-22T04:16:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8f01600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8efc0b40e0 con 0x7f8efc144a90 2024-03-22T04:16:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8efaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8efc144a90 0x7f8efc0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8efaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8efc144a90 0x7f8efc0727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43212/0 (socket says 172.21.15.86:43212) 2024-03-22T04:16:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8efaa00640 1 -- 172.21.15.86:0/934463479 learned_addr learned my addr 172.21.15.86:0/934463479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8efaa00640 1 -- 172.21.15.86:0/934463479 >> v1:172.21.15.196:6789/0 conn(0x7f8efc0b37c0 legacy=0x7f8efc0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8efaa00640 1 -- 172.21.15.86:0/934463479 >> v1:172.21.15.86:6789/0 conn(0x7f8efc072cf0 legacy=0x7f8efc0b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8efaa00640 1 -- 172.21.15.86:0/934463479 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc0cdbd0 con 0x7f8efc144a90 2024-03-22T04:16:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.458+0000 7f8efa000640 1 -- 172.21.15.86:0/934463479 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 265705459 0 0) 0x7f8efc144e70 con 0x7f8efc072cf0 2024-03-22T04:16:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8efaa00640 1 --2- 172.21.15.86:0/934463479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc144a90 0x7f8efc0727b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00bb40 tx=0x7f8eec01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb0481775dcb199 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8efa000640 1 -- 172.21.15.86:0/934463479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0b4070 con 0x7f8efc144a90 2024-03-22T04:16:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8efa000640 1 -- 172.21.15.86:0/934463479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec0a7ae0 con 0x7f8efc144a90 2024-03-22T04:16:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8efa000640 1 -- 172.21.15.86:0/934463479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0af440 con 0x7f8efc144a90 2024-03-22T04:16:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 -- 172.21.15.86:0/934463479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc144a90 msgr2=0x7f8efc0727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 --2- 172.21.15.86:0/934463479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc144a90 0x7f8efc0727b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00bb40 tx=0x7f8eec01ec10 comp rx=0 tx=0).stop 2024-03-22T04:16:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 -- 172.21.15.86:0/934463479 shutdown_connections 2024-03-22T04:16:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 --2- 172.21.15.86:0/934463479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc144a90 0x7f8efc0727b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 -- 172.21.15.86:0/934463479 >> 172.21.15.86:0/934463479 conn(0x7f8efc06b660 msgr2=0x7f8efc06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 -- 172.21.15.86:0/934463479 shutdown_connections 2024-03-22T04:16:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 -- 172.21.15.86:0/934463479 wait complete. 2024-03-22T04:16:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 Processor -- start 2024-03-22T04:16:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.462+0000 7f8f01600640 1 -- start start 2024-03-22T04:16:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc072cf0 0x7f8efc156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8efc0b37c0 0x7f8efc0ac530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8efc144a90 0x7f8efc0aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc0cf680 con 0x7f8efc072cf0 2024-03-22T04:16:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc0cf500 con 0x7f8efc0b37c0 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8efc06d530 con 0x7f8efc144a90 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc072cf0 0x7f8efc156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc072cf0 0x7f8efc156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43214/0 (socket says 172.21.15.86:43214) 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 -- 172.21.15.86:0/1231009458 learned_addr learned my addr 172.21.15.86:0/1231009458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efb400640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8efc144a90 0x7f8efc0aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8ef2000640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8efc0b37c0 0x7f8efc0ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 -- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8efc144a90 msgr2=0x7f8efc0aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8efc144a90 0x7f8efc0aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 -- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8efc0b37c0 msgr2=0x7f8efc0ac530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8efc0b37c0 0x7f8efc0ac530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 -- 172.21.15.86:0/1231009458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc159b60 con 0x7f8efc072cf0 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efb400640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8efc144a90 0x7f8efc0aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8ef2000640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8efc0b37c0 0x7f8efc0ac530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8efaa00640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc072cf0 0x7f8efc156c60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00b2c0 tx=0x7f8eec01d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8ef3400640 1 -- 172.21.15.86:0/1231009458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0bf070 con 0x7f8efc072cf0 2024-03-22T04:16:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8ef3400640 1 -- 172.21.15.86:0/1231009458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec0ba640 con 0x7f8efc072cf0 2024-03-22T04:16:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8efc159e40 con 0x7f8efc072cf0 2024-03-22T04:16:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8ef3400640 1 -- 172.21.15.86:0/1231009458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0b4070 con 0x7f8efc072cf0 2024-03-22T04:16:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8efc15a400 con 0x7f8efc072cf0 2024-03-22T04:16:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8efc074580 con 0x7f8efc072cf0 2024-03-22T04:16:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.466+0000 7f8ef3400640 1 -- 172.21.15.86:0/1231009458 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8eec0c2070 con 0x7f8efc072cf0 2024-03-22T04:16:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.470+0000 7f8ef3400640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ed8067b20 0x7f8ed8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.470+0000 7f8ef3400640 1 -- 172.21.15.86:0/1231009458 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5835+0+0 (secure 0 0 0) 0x7f8eec1206e0 con 0x7f8efc072cf0 2024-03-22T04:16:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.470+0000 7f8ef2000640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ed8067b20 0x7f8ed8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.470+0000 7f8ef2000640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ed8067b20 0x7f8ed8069fe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4007fd0 tx=0x7f8ee4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.474+0000 7f8ef3400640 1 -- 172.21.15.86:0/1231009458 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8eec018800 con 0x7f8efc072cf0 2024-03-22T04:16:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:42.630+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8efc0cc290 con 0x7f8efc072cf0 2024-03-22T04:16:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.434+0000 7f8ef3400640 1 -- 172.21.15.86:0/1231009458 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) ==== 169+0+0 (secure 0 0 0) 0x7f8eec0f1060 con 0x7f8efc072cf0 2024-03-22T04:16:43.435 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T04:16:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ed8067b20 msgr2=0x7f8ed8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ed8067b20 0x7f8ed8069fe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4007fd0 tx=0x7f8ee4009040 comp rx=0 tx=0).stop 2024-03-22T04:16:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc072cf0 msgr2=0x7f8efc156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc072cf0 0x7f8efc156c60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00b2c0 tx=0x7f8eec01d450 comp rx=0 tx=0).stop 2024-03-22T04:16:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 shutdown_connections 2024-03-22T04:16:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8ed8067b20 0x7f8ed8069fe0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8efc0b37c0 0x7f8efc0ac530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8efc144a90 0x7f8efc0aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 --2- 172.21.15.86:0/1231009458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8efc072cf0 0x7f8efc156c60 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 >> 172.21.15.86:0/1231009458 conn(0x7f8efc06b660 msgr2=0x7f8efc0afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 shutdown_connections 2024-03-22T04:16:43.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.438+0000 7f8f01600640 1 -- 172.21.15.86:0/1231009458 wait complete. 2024-03-22T04:16:43.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-22T04:16:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.610+0000 7fd47b600640 1 Processor -- start 2024-03-22T04:16:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.610+0000 7fd47b600640 1 -- start start 2024-03-22T04:16:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.610+0000 7fd47b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4740cb1d0 0x7fd474162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.610+0000 7fd47b600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd4740c7300 0x7fd474162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.610+0000 7fd47b600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4740cc7c0 con 0x7fd4740c6f20 2024-03-22T04:16:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.610+0000 7fd47b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd4740cb720 con 0x7fd4740cb1d0 2024-03-22T04:16:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.610+0000 7fd47b600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd4740cba00 con 0x7fd4740c7300 2024-03-22T04:16:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd46be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4740cb1d0 0x7fd474162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd46be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4740cb1d0 0x7fd474162030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43220/0 (socket says 172.21.15.86:43220) 2024-03-22T04:16:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd46be00640 1 -- 172.21.15.86:0/2548970160 learned_addr learned my addr 172.21.15.86:0/2548970160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd46be00640 1 -- 172.21.15.86:0/2548970160 >> v2:172.21.15.196:3300/0 conn(0x7fd4740c7300 msgr2=0x7fd474162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd46be00640 1 --2- 172.21.15.86:0/2548970160 >> v2:172.21.15.196:3300/0 conn(0x7fd4740c7300 0x7fd474162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd46be00640 1 -- 172.21.15.86:0/2548970160 >> v1:172.21.15.196:6789/0 conn(0x7fd4740c6f20 legacy=0x7fd474072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd46be00640 1 -- 172.21.15.86:0/2548970160 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4740cbce0 con 0x7fd4740cb1d0 2024-03-22T04:16:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd46be00640 1 --2- 172.21.15.86:0/2548970160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740cb1d0 0x7fd474162030 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd4740cad30 tx=0x7fd45c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636208d064daaa4f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd473e00640 1 -- 172.21.15.86:0/2548970160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0b1070 con 0x7fd4740cb1d0 2024-03-22T04:16:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd473e00640 1 -- 172.21.15.86:0/2548970160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c0ac630 con 0x7fd4740cb1d0 2024-03-22T04:16:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.614+0000 7fd473e00640 1 -- 172.21.15.86:0/2548970160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0a8050 con 0x7fd4740cb1d0 2024-03-22T04:16:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- 172.21.15.86:0/2548970160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740cb1d0 msgr2=0x7fd474162030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 --2- 172.21.15.86:0/2548970160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740cb1d0 0x7fd474162030 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd4740cad30 tx=0x7fd45c01c4e0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- 172.21.15.86:0/2548970160 shutdown_connections 2024-03-22T04:16:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 --2- 172.21.15.86:0/2548970160 >> v2:172.21.15.196:3300/0 conn(0x7fd4740c7300 0x7fd474162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 --2- 172.21.15.86:0/2548970160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740cb1d0 0x7fd474162030 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- 172.21.15.86:0/2548970160 >> 172.21.15.86:0/2548970160 conn(0x7fd4740d0730 msgr2=0x7fd4740d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- 172.21.15.86:0/2548970160 shutdown_connections 2024-03-22T04:16:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- 172.21.15.86:0/2548970160 wait complete. 2024-03-22T04:16:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 Processor -- start 2024-03-22T04:16:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- start start 2024-03-22T04:16:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4740c6f20 0x7fd47415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740c7300 0x7fd47415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4740cb1d0 0x7fd4740c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4740c6190 con 0x7fd4740c7300 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4740c6010 con 0x7fd4740cb1d0 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd47b600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd4740c6310 con 0x7fd4740c6f20 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd46be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740c7300 0x7fd47415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd46be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740c7300 0x7fd47415bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43236/0 (socket says 172.21.15.86:43236) 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd46be00640 1 -- 172.21.15.86:0/1504670609 learned_addr learned my addr 172.21.15.86:0/1504670609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd478c00640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4740c6f20 0x7fd47415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd479600640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4740cb1d0 0x7fd4740c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd46be00640 1 -- 172.21.15.86:0/1504670609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4740c6f20 msgr2=0x7fd47415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd46be00640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4740c6f20 0x7fd47415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd46be00640 1 -- 172.21.15.86:0/1504670609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4740cb1d0 msgr2=0x7fd4740c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd46be00640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4740cb1d0 0x7fd4740c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.618+0000 7fd46be00640 1 -- 172.21.15.86:0/1504670609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4740c9020 con 0x7fd4740c7300 2024-03-22T04:16:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd46be00640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740c7300 0x7fd47415bb30 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0af040 tx=0x7fd45c019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd471600640 1 -- 172.21.15.86:0/1504670609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0bc070 con 0x7fd4740c7300 2024-03-22T04:16:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd471600640 1 -- 172.21.15.86:0/1504670609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c0b7650 con 0x7fd4740c7300 2024-03-22T04:16:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4740c9300 con 0x7fd4740c7300 2024-03-22T04:16:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd471600640 1 -- 172.21.15.86:0/1504670609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0b1040 con 0x7fd4740c7300 2024-03-22T04:16:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4740b3ec0 con 0x7fd4740c7300 2024-03-22T04:16:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd474074470 con 0x7fd4740c7300 2024-03-22T04:16:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd471600640 1 -- 172.21.15.86:0/1504670609 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd45c0bf070 con 0x7fd4740c7300 2024-03-22T04:16:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd471600640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd450067b20 0x7fd450069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd471600640 1 -- 172.21.15.86:0/1504670609 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5421+0+0 (secure 0 0 0) 0x7fd45c11f5e0 con 0x7fd4740c7300 2024-03-22T04:16:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.622+0000 7fd478c00640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd450067b20 0x7fd450069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.626+0000 7fd478c00640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd450067b20 0x7fd450069fe0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd4640098e0 tx=0x7fd464009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.626+0000 7fd471600640 1 -- 172.21.15.86:0/1504670609 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd45c0160d0 con 0x7fd4740c7300 2024-03-22T04:16:43.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.782+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd4740af7c0 con 0x7fd4740c7300 2024-03-22T04:16:43.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.786+0000 7fd471600640 1 -- 172.21.15.86:0/1504670609 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7fd45c0efb30 con 0x7fd4740c7300 2024-03-22T04:16:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd450067b20 msgr2=0x7fd450069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd450067b20 0x7fd450069fe0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd4640098e0 tx=0x7fd464009060 comp rx=0 tx=0).stop 2024-03-22T04:16:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740c7300 msgr2=0x7fd47415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740c7300 0x7fd47415bb30 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0af040 tx=0x7fd45c019280 comp rx=0 tx=0).stop 2024-03-22T04:16:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 shutdown_connections 2024-03-22T04:16:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd450067b20 0x7fd450069fe0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4740cb1d0 0x7fd4740c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4740c7300 0x7fd47415bb30 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 --2- 172.21.15.86:0/1504670609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4740c6f20 0x7fd47415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 >> 172.21.15.86:0/1504670609 conn(0x7fd4740d0730 msgr2=0x7fd474161f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 shutdown_connections 2024-03-22T04:16:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.790+0000 7fd47b600640 1 -- 172.21.15.86:0/1504670609 wait complete. 2024-03-22T04:16:43.809 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-22T04:16:43.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.942+0000 7f92d9400640 1 Processor -- start 2024-03-22T04:16:43.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.942+0000 7f92d9400640 1 -- start start 2024-03-22T04:16:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92d9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92d40730d0 0x7f92d40ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92d9400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f92d40cc3e0 0x7f92d40ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92d9400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d4071920 con 0x7f92d415b3f0 2024-03-22T04:16:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92d9400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f92d40716d0 con 0x7f92d40730d0 2024-03-22T04:16:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92d9400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f92d406d990 con 0x7f92d40cc3e0 2024-03-22T04:16:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92c9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92d40730d0 0x7f92d40ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92c9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92d40730d0 0x7f92d40ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43244/0 (socket says 172.21.15.86:43244) 2024-03-22T04:16:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92c9e00640 1 -- 172.21.15.86:0/1334867637 learned_addr learned my addr 172.21.15.86:0/1334867637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92d3200640 1 --2- 172.21.15.86:0/1334867637 >> v2:172.21.15.196:3300/0 conn(0x7f92d40cc3e0 0x7f92d40ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92c9e00640 1 -- 172.21.15.86:0/1334867637 >> v2:172.21.15.196:3300/0 conn(0x7f92d40cc3e0 msgr2=0x7f92d40ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92c9e00640 1 --2- 172.21.15.86:0/1334867637 >> v2:172.21.15.196:3300/0 conn(0x7f92d40cc3e0 0x7f92d40ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92c9e00640 1 -- 172.21.15.86:0/1334867637 >> v1:172.21.15.196:6789/0 conn(0x7f92d415b3f0 legacy=0x7f92d40729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.946+0000 7f92c9e00640 1 -- 172.21.15.86:0/1334867637 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d4071c00 con 0x7f92d40730d0 2024-03-22T04:16:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92c9e00640 1 --2- 172.21.15.86:0/1334867637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d40ade30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f92d4073b20 tx=0x7f92bc01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e995ce6a2df1aa60 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d1e00640 1 -- 172.21.15.86:0/1334867637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0b4070 con 0x7f92d40730d0 2024-03-22T04:16:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d1e00640 1 -- 172.21.15.86:0/1334867637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92bc0a7bb0 con 0x7f92d40730d0 2024-03-22T04:16:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d1e00640 1 -- 172.21.15.86:0/1334867637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0af440 con 0x7f92d40730d0 2024-03-22T04:16:43.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 -- 172.21.15.86:0/1334867637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 msgr2=0x7f92d40ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 --2- 172.21.15.86:0/1334867637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d40ade30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f92d4073b20 tx=0x7f92bc01ebf0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 -- 172.21.15.86:0/1334867637 shutdown_connections 2024-03-22T04:16:43.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 --2- 172.21.15.86:0/1334867637 >> v2:172.21.15.196:3300/0 conn(0x7f92d40cc3e0 0x7f92d40ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 --2- 172.21.15.86:0/1334867637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d40ade30 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 -- 172.21.15.86:0/1334867637 >> 172.21.15.86:0/1334867637 conn(0x7f92d40b1df0 msgr2=0x7f92d40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 -- 172.21.15.86:0/1334867637 shutdown_connections 2024-03-22T04:16:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 -- 172.21.15.86:0/1334867637 wait complete. 2024-03-22T04:16:43.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 Processor -- start 2024-03-22T04:16:43.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.950+0000 7f92d9400640 1 -- start start 2024-03-22T04:16:43.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d414f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92d40cc3e0 0x7f92d414ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92d415b3f0 0x7f92d415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92d406e320 con 0x7f92d40730d0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92d406e1a0 con 0x7f92d415b3f0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f92d406e4a0 con 0x7f92d40cc3e0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d414f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d414f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43250/0 (socket says 172.21.15.86:43250) 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 -- 172.21.15.86:0/1454058195 learned_addr learned my addr 172.21.15.86:0/1454058195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92c9e00640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92d40cc3e0 0x7f92d414ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d3200640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92d415b3f0 0x7f92d415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 -- 172.21.15.86:0/1454058195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92d40cc3e0 msgr2=0x7f92d414ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92d40cc3e0 0x7f92d414ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 -- 172.21.15.86:0/1454058195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92d415b3f0 msgr2=0x7f92d415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92d415b3f0 0x7f92d415d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 -- 172.21.15.86:0/1454058195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d414ea60 con 0x7f92d40730d0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d2800640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d414f820 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f92c400c3a0 tx=0x7f92c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92cb400640 1 -- 172.21.15.86:0/1454058195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40052d0 con 0x7f92d40730d0 2024-03-22T04:16:43.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92cb400640 1 -- 172.21.15.86:0/1454058195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c4005470 con 0x7f92d40730d0 2024-03-22T04:16:43.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d414ed40 con 0x7f92d40730d0 2024-03-22T04:16:43.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92cb400640 1 -- 172.21.15.86:0/1454058195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4007260 con 0x7f92d40730d0 2024-03-22T04:16:43.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d414b0a0 con 0x7f92d40730d0 2024-03-22T04:16:43.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d4068650 con 0x7f92d40730d0 2024-03-22T04:16:43.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.954+0000 7f92cb400640 1 -- 172.21.15.86:0/1454058195 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f92c4007400 con 0x7f92d40730d0 2024-03-22T04:16:43.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.958+0000 7f92cb400640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f92b0067b20 0x7f92b0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:43.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.958+0000 7f92cb400640 1 -- 172.21.15.86:0/1454058195 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5421+0+0 (secure 0 0 0) 0x7f92c40851f0 con 0x7f92d40730d0 2024-03-22T04:16:43.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.958+0000 7f92c9e00640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f92b0067b20 0x7f92b0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:43.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.958+0000 7f92c9e00640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f92b0067b20 0x7f92b0069fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0b1000 tx=0x7f92bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:43.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:43.962+0000 7f92cb400640 1 -- 172.21.15.86:0/1454058195 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f92c40556c0 con 0x7f92d40730d0 2024-03-22T04:16:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.174+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f92d40b2350 con 0x7f92d40730d0 2024-03-22T04:16:44.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.434+0000 7f92cb400640 1 -- 172.21.15.86:0/1454058195 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v322) ==== 166+0+0 (secure 0 0 0) 0x7f92c4059760 con 0x7f92d40730d0 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f92b0067b20 msgr2=0x7f92b0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f92b0067b20 0x7f92b0069fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0b1000 tx=0x7f92bc008040 comp rx=0 tx=0).stop 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 msgr2=0x7f92d414f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d414f820 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f92c400c3a0 tx=0x7f92c400c860 comp rx=0 tx=0).stop 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 shutdown_connections 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f92b0067b20 0x7f92b0069fe0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92d415b3f0 0x7f92d415d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92d40cc3e0 0x7f92d414ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 --2- 172.21.15.86:0/1454058195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d40730d0 0x7f92d414f820 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 >> 172.21.15.86:0/1454058195 conn(0x7f92d40b1df0 msgr2=0x7f92d4157750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 shutdown_connections 2024-03-22T04:16:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.438+0000 7f92d9400640 1 -- 172.21.15.86:0/1454058195 wait complete. 2024-03-22T04:16:44.459 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-22T04:16:44.459 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-22T04:16:44.460 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-22T04:16:44.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:16:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa803800640 1 Processor -- start 2024-03-22T04:16:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa803800640 1 -- start start 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa803800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc161880 0x7fa7fc0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa803800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7fc161c60 0x7fa7fc0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa803800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc06e350 con 0x7fa7fc06d700 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa803800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc15ee80 con 0x7fa7fc161c60 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa803800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc06e630 con 0x7fa7fc161880 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa7fbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7fc161c60 0x7fa7fc0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa7fbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7fc161c60 0x7fa7fc0b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43254/0 (socket says 172.21.15.86:43254) 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa7fbe00640 1 -- 172.21.15.86:0/1976926873 learned_addr learned my addr 172.21.15.86:0/1976926873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa801800640 1 --1- 172.21.15.86:0/1976926873 >> v1:172.21.15.86:6789/0 conn(0x7fa7fc06d700 0x7fa7fc06db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40586/0 (socket says 172.21.15.86:40586) 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa7fbe00640 1 -- 172.21.15.86:0/1976926873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc161880 msgr2=0x7fa7fc0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/1976926873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc161880 0x7fa7fc0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa7fbe00640 1 -- 172.21.15.86:0/1976926873 >> v1:172.21.15.86:6789/0 conn(0x7fa7fc06d700 legacy=0x7fa7fc06db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa7fbe00640 1 -- 172.21.15.86:0/1976926873 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc06e910 con 0x7fa7fc161c60 2024-03-22T04:16:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.598+0000 7fa7fb400640 1 -- 172.21.15.86:0/1976926873 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3959039394 0 0) 0x7fa7fc06e350 con 0x7fa7fc06d700 2024-03-22T04:16:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/1976926873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0b3630 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009870 tx=0x7fa7e801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e173ce7b6e2c3fe server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa7fb400640 1 -- 172.21.15.86:0/1976926873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80aa600 con 0x7fa7fc161c60 2024-03-22T04:16:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa7fb400640 1 -- 172.21.15.86:0/1976926873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e80af070 con 0x7fa7fc161c60 2024-03-22T04:16:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa7fb400640 1 -- 172.21.15.86:0/1976926873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80aa9c0 con 0x7fa7fc161c60 2024-03-22T04:16:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 -- 172.21.15.86:0/1976926873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 msgr2=0x7fa7fc0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 --2- 172.21.15.86:0/1976926873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0b3630 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009870 tx=0x7fa7e801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 -- 172.21.15.86:0/1976926873 shutdown_connections 2024-03-22T04:16:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 --2- 172.21.15.86:0/1976926873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0b3630 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 --2- 172.21.15.86:0/1976926873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc161880 0x7fa7fc0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 -- 172.21.15.86:0/1976926873 >> 172.21.15.86:0/1976926873 conn(0x7fa7fc0b1df0 msgr2=0x7fa7fc0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 -- 172.21.15.86:0/1976926873 shutdown_connections 2024-03-22T04:16:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 -- 172.21.15.86:0/1976926873 wait complete. 2024-03-22T04:16:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 Processor -- start 2024-03-22T04:16:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.602+0000 7fa803800640 1 -- start start 2024-03-22T04:16:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc06d700 0x7fa7fc156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc161880 0x7fa7fc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc160950 con 0x7fa7fc161c60 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1607d0 con 0x7fa7fc06d700 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc160ad0 con 0x7fa7fc161880 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43268/0 (socket says 172.21.15.86:43268) 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 -- 172.21.15.86:0/586545039 learned_addr learned my addr 172.21.15.86:0/586545039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 -- 172.21.15.86:0/586545039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc161880 msgr2=0x7fa7fc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc161880 0x7fa7fc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 -- 172.21.15.86:0/586545039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc06d700 msgr2=0x7fa7fc156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc06d700 0x7fa7fc156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 -- 172.21.15.86:0/586545039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc0730f0 con 0x7fa7fc161c60 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa801800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0d0510 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c700 tx=0x7fa7f400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa7f8c00640 1 -- 172.21.15.86:0/586545039 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4018070 con 0x7fa7fc161c60 2024-03-22T04:16:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa7f8c00640 1 -- 172.21.15.86:0/586545039 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f400ce10 con 0x7fa7fc161c60 2024-03-22T04:16:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa7f8c00640 1 -- 172.21.15.86:0/586545039 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400a860 con 0x7fa7fc161c60 2024-03-22T04:16:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc0ad7f0 con 0x7fa7fc161c60 2024-03-22T04:16:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc0addb0 con 0x7fa7fc161c60 2024-03-22T04:16:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc068650 con 0x7fa7fc161c60 2024-03-22T04:16:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.606+0000 7fa7f8c00640 1 -- 172.21.15.86:0/586545039 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa7f401b070 con 0x7fa7fc161c60 2024-03-22T04:16:44.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.610+0000 7fa7f8c00640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067ba0 0x7fa7dc06a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.610+0000 7fa7f8c00640 1 -- 172.21.15.86:0/586545039 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5421+0+0 (secure 0 0 0) 0x7fa7f40854d0 con 0x7fa7fc161c60 2024-03-22T04:16:44.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.610+0000 7fa800e00640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067ba0 0x7fa7dc06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.610+0000 7fa800e00640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067ba0 0x7fa7dc06a060 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009730 tx=0x7fa7f0006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:44.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.614+0000 7fa7f8c00640 1 -- 172.21.15.86:0/586545039 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa7f4055fe0 con 0x7fa7fc161c60 2024-03-22T04:16:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7fc0cfdb0 con 0x7fa7fc161c60 2024-03-22T04:16:44.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa7f8c00640 1 -- 172.21.15.86:0/586545039 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+13422 (secure 0 0 0) 0x7fa7f40558e0 con 0x7fa7fc161c60 2024-03-22T04:16:44.770 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:44.770 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":84,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:16:43.293130+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/736870197":"2024-03-23T04:16:40.281741+0000","172.21.15.86:6836/736870197":"2024-03-23T04:16:40.281741+0000","172.21.15.86:6837/1130704586":"2024-03-23T04:15:20.393105+0000","172.21.15.86:6836/1130704586":"2024-03-23T04:15:20.393105+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:16:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067ba0 msgr2=0x7fa7dc06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067ba0 0x7fa7dc06a060 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009730 tx=0x7fa7f0006ca0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 msgr2=0x7fa7fc0d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0d0510 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400c700 tx=0x7fa7f400cbc0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 shutdown_connections 2024-03-22T04:16:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067ba0 0x7fa7dc06a060 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc161c60 0x7fa7fc0d0510 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc161880 0x7fa7fc0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 --2- 172.21.15.86:0/586545039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc06d700 0x7fa7fc156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 >> 172.21.15.86:0/586545039 conn(0x7fa7fc0b1df0 msgr2=0x7fa7fc0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 shutdown_connections 2024-03-22T04:16:44.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.770+0000 7fa803800640 1 -- 172.21.15.86:0/586545039 wait complete. 2024-03-22T04:16:44.793 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:16:44.793 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:16:44.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.930+0000 7fb303000640 1 Processor -- start 2024-03-22T04:16:44.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.930+0000 7fb303000640 1 -- start start 2024-03-22T04:16:44.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb303000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc0b2360 0x7fb2fc0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb303000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2fc0b3430 0x7fb2fc15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb303000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2fc15cee0 0x7fb2fc06d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb303000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb2fc0b3890 con 0x7fb2fc0b3430 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb303000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb2fc0b2740 con 0x7fb2fc15cee0 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb303000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb2fc15d710 con 0x7fb2fc0b2360 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2fc0b3430 0x7fb2fc15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2fc0b3430 0x7fb2fc15c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43272/0 (socket says 172.21.15.86:43272) 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 -- 172.21.15.86:0/947462920 learned_addr learned my addr 172.21.15.86:0/947462920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2fbe00640 1 --2- 172.21.15.86:0/947462920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc0b2360 0x7fb2fc0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb301000640 1 --2- 172.21.15.86:0/947462920 >> v2:172.21.15.196:3300/0 conn(0x7fb2fc15cee0 0x7fb2fc06d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 -- 172.21.15.86:0/947462920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc0b2360 msgr2=0x7fb2fc0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 --2- 172.21.15.86:0/947462920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc0b2360 0x7fb2fc0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 -- 172.21.15.86:0/947462920 >> v2:172.21.15.196:3300/0 conn(0x7fb2fc15cee0 msgr2=0x7fb2fc06d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 --2- 172.21.15.86:0/947462920 >> v2:172.21.15.196:3300/0 conn(0x7fb2fc15cee0 0x7fb2fc06d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 -- 172.21.15.86:0/947462920 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc15fe40 con 0x7fb2fc0b3430 2024-03-22T04:16:44.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2f3400640 1 --2- 172.21.15.86:0/947462920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc15c9a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc06f3e0 tx=0x7fb2e401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91fca6427c56e2b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:44.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2fb400640 1 -- 172.21.15.86:0/947462920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40b0070 con 0x7fb2fc0b3430 2024-03-22T04:16:44.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2fb400640 1 -- 172.21.15.86:0/947462920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e40ab600 con 0x7fb2fc0b3430 2024-03-22T04:16:44.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.934+0000 7fb2fb400640 1 -- 172.21.15.86:0/947462920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40ab9d0 con 0x7fb2fc0b3430 2024-03-22T04:16:44.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- 172.21.15.86:0/947462920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 msgr2=0x7fb2fc15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 --2- 172.21.15.86:0/947462920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc15c9a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc06f3e0 tx=0x7fb2e401c230 comp rx=0 tx=0).stop 2024-03-22T04:16:44.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- 172.21.15.86:0/947462920 shutdown_connections 2024-03-22T04:16:44.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 --2- 172.21.15.86:0/947462920 >> v2:172.21.15.196:3300/0 conn(0x7fb2fc15cee0 0x7fb2fc06d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 --2- 172.21.15.86:0/947462920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc15c9a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 --2- 172.21.15.86:0/947462920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc0b2360 0x7fb2fc0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- 172.21.15.86:0/947462920 >> 172.21.15.86:0/947462920 conn(0x7fb2fc0cc940 msgr2=0x7fb2fc0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- 172.21.15.86:0/947462920 shutdown_connections 2024-03-22T04:16:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- 172.21.15.86:0/947462920 wait complete. 2024-03-22T04:16:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 Processor -- start 2024-03-22T04:16:44.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- start start 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2fc0b2360 0x7fb2fc143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc15cee0 0x7fb2fc146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc160c00 con 0x7fb2fc0b3430 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc15d2f0 con 0x7fb2fc0b2360 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb303000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb2fc1437b0 con 0x7fb2fc15cee0 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc14e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc14e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43276/0 (socket says 172.21.15.86:43276) 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2f3400640 1 -- 172.21.15.86:0/3436762316 learned_addr learned my addr 172.21.15.86:0/3436762316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb301000640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc15cee0 0x7fb2fc146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2fbe00640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2fc0b2360 0x7fb2fc143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2f3400640 1 -- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc15cee0 msgr2=0x7fb2fc146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2f3400640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc15cee0 0x7fb2fc146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2f3400640 1 -- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2fc0b2360 msgr2=0x7fb2fc143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2f3400640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2fc0b2360 0x7fb2fc143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2f3400640 1 -- 172.21.15.86:0/3436762316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc14eaf0 con 0x7fb2fc0b3430 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.938+0000 7fb2fbe00640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2fc0b2360 0x7fb2fc143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb2f3400640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc14e050 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb2e401a8e0 tx=0x7fb2e4018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb2f8c00640 1 -- 172.21.15.86:0/3436762316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40b9070 con 0x7fb2fc0b3430 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb2f8c00640 1 -- 172.21.15.86:0/3436762316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e40b4400 con 0x7fb2fc0b3430 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb2f8c00640 1 -- 172.21.15.86:0/3436762316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40b0040 con 0x7fb2fc0b3430 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc14edd0 con 0x7fb2fc0b3430 2024-03-22T04:16:44.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc15e640 con 0x7fb2fc0b3430 2024-03-22T04:16:44.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc074470 con 0x7fb2fc0b3430 2024-03-22T04:16:44.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb2f8c00640 1 -- 172.21.15.86:0/3436762316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb2e40b33a0 con 0x7fb2fc0b3430 2024-03-22T04:16:44.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb2f8c00640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb2d8067b50 0x7fb2d806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb2f8c00640 1 -- 172.21.15.86:0/3436762316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5421+0+0 (secure 0 0 0) 0x7fb2e411f0a0 con 0x7fb2fc0b3430 2024-03-22T04:16:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.942+0000 7fb2fbe00640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb2d8067b50 0x7fb2d806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:44.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.946+0000 7fb2fbe00640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb2d8067b50 0x7fb2d806a010 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec003fd0 tx=0x7fb2ec0039e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:44.946+0000 7fb2f8c00640 1 -- 172.21.15.86:0/3436762316 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb2e4016110 con 0x7fb2fc0b3430 2024-03-22T04:16:45.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.102+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2fc0cce20 con 0x7fb2fc0b3430 2024-03-22T04:16:45.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.102+0000 7fb2f8c00640 1 -- 172.21.15.86:0/3436762316 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fb2e40efaa0 con 0x7fb2fc0b3430 2024-03-22T04:16:45.103 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:16:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb2d8067b50 msgr2=0x7fb2d806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb2d8067b50 0x7fb2d806a010 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec003fd0 tx=0x7fb2ec0039e0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 msgr2=0x7fb2fc14e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc14e050 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb2e401a8e0 tx=0x7fb2e4018fd0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 shutdown_connections 2024-03-22T04:16:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb2d8067b50 0x7fb2d806a010 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2fc15cee0 0x7fb2fc146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2fc0b3430 0x7fb2fc14e050 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 --2- 172.21.15.86:0/3436762316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2fc0b2360 0x7fb2fc143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 >> 172.21.15.86:0/3436762316 conn(0x7fb2fc0cc940 msgr2=0x7fb2fc0cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 shutdown_connections 2024-03-22T04:16:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.106+0000 7fb303000640 1 -- 172.21.15.86:0/3436762316 wait complete. 2024-03-22T04:16:45.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-22T04:16:45.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.258+0000 7f22b5000640 1 Processor -- start 2024-03-22T04:16:45.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.258+0000 7f22b5000640 1 -- start start 2024-03-22T04:16:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22b5000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f22b015b3f0 0x7f22b00729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22b5000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22b00cc3e0 0x7f22b0072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22b5000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00b30f0 0x7f22b00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22b5000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f22b00716d0 con 0x7f22b00cc3e0 2024-03-22T04:16:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22b5000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f22b006d990 con 0x7f22b015b3f0 2024-03-22T04:16:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22b5000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f22b0071920 con 0x7f22b00b30f0 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22a5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22b00cc3e0 0x7f22b0072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22a5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22b00cc3e0 0x7f22b0072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43292/0 (socket says 172.21.15.86:43292) 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22a5400640 1 -- 172.21.15.86:0/1246055870 learned_addr learned my addr 172.21.15.86:0/1246055870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22ade00640 1 --2- 172.21.15.86:0/1246055870 >> v2:172.21.15.196:3300/0 conn(0x7f22b015b3f0 0x7f22b00729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22ae800640 1 --2- 172.21.15.86:0/1246055870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00b30f0 0x7f22b00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22a5400640 1 -- 172.21.15.86:0/1246055870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00b30f0 msgr2=0x7f22b00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22a5400640 1 --2- 172.21.15.86:0/1246055870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00b30f0 0x7f22b00ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22a5400640 1 -- 172.21.15.86:0/1246055870 >> v2:172.21.15.196:3300/0 conn(0x7f22b015b3f0 msgr2=0x7f22b00729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22a5400640 1 --2- 172.21.15.86:0/1246055870 >> v2:172.21.15.196:3300/0 conn(0x7f22b015b3f0 0x7f22b00729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22a5400640 1 -- 172.21.15.86:0/1246055870 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b0071c00 con 0x7f22b00cc3e0 2024-03-22T04:16:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.262+0000 7f22ade00640 1 --2- 172.21.15.86:0/1246055870 >> v2:172.21.15.196:3300/0 conn(0x7f22b015b3f0 0x7f22b00729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22a5400640 1 --2- 172.21.15.86:0/1246055870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00cc3e0 0x7f22b0072f00 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f22b006c510 tx=0x7f229801eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c741278df530d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22ad400640 1 -- 172.21.15.86:0/1246055870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980b4070 con 0x7f22b00cc3e0 2024-03-22T04:16:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22ad400640 1 -- 172.21.15.86:0/1246055870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22980a79f0 con 0x7f22b00cc3e0 2024-03-22T04:16:45.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22ad400640 1 -- 172.21.15.86:0/1246055870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980a7dc0 con 0x7f22b00cc3e0 2024-03-22T04:16:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 -- 172.21.15.86:0/1246055870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00cc3e0 msgr2=0x7f22b0072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 --2- 172.21.15.86:0/1246055870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00cc3e0 0x7f22b0072f00 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f22b006c510 tx=0x7f229801eb60 comp rx=0 tx=0).stop 2024-03-22T04:16:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 -- 172.21.15.86:0/1246055870 shutdown_connections 2024-03-22T04:16:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 --2- 172.21.15.86:0/1246055870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00b30f0 0x7f22b00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 --2- 172.21.15.86:0/1246055870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00cc3e0 0x7f22b0072f00 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 --2- 172.21.15.86:0/1246055870 >> v2:172.21.15.196:3300/0 conn(0x7f22b015b3f0 0x7f22b00729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 -- 172.21.15.86:0/1246055870 >> 172.21.15.86:0/1246055870 conn(0x7f22b00b1df0 msgr2=0x7f22b00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 -- 172.21.15.86:0/1246055870 shutdown_connections 2024-03-22T04:16:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 -- 172.21.15.86:0/1246055870 wait complete. 2024-03-22T04:16:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 Processor -- start 2024-03-22T04:16:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 -- start start 2024-03-22T04:16:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.266+0000 7f22b5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00b30f0 0x7f22b009dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22b5000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00cc3e0 0x7f22b0157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22b5000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b015b3f0 0x7f22b0164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22b5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22b015f1e0 con 0x7f22b00b30f0 2024-03-22T04:16:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22b5000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22b015f060 con 0x7f22b015b3f0 2024-03-22T04:16:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22b5000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f22b015f360 con 0x7f22b00cc3e0 2024-03-22T04:16:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00b30f0 0x7f22b009dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00b30f0 0x7f22b009dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43294/0 (socket says 172.21.15.86:43294) 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 -- 172.21.15.86:0/1268584100 learned_addr learned my addr 172.21.15.86:0/1268584100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22a5400640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00cc3e0 0x7f22b0157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ae800640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b015b3f0 0x7f22b0164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 -- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00cc3e0 msgr2=0x7f22b0157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00cc3e0 0x7f22b0157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 -- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b015b3f0 msgr2=0x7f22b0164680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b015b3f0 0x7f22b0164680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 -- 172.21.15.86:0/1268584100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b0158400 con 0x7f22b00b30f0 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22a5400640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00cc3e0 0x7f22b0157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22ade00640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00b30f0 0x7f22b009dc60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d830 tx=0x7f22a000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22a6a00640 1 -- 172.21.15.86:0/1268584100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0018020 con 0x7f22b00b30f0 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22a6a00640 1 -- 172.21.15.86:0/1268584100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a00096c0 con 0x7f22b00b30f0 2024-03-22T04:16:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22a6a00640 1 -- 172.21.15.86:0/1268584100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0009ab0 con 0x7f22b00b30f0 2024-03-22T04:16:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b01586e0 con 0x7f22b00b30f0 2024-03-22T04:16:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b0163e80 con 0x7f22b00b30f0 2024-03-22T04:16:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b0068650 con 0x7f22b00b30f0 2024-03-22T04:16:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.270+0000 7f22a6a00640 1 -- 172.21.15.86:0/1268584100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f22a001b020 con 0x7f22b00b30f0 2024-03-22T04:16:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.274+0000 7f22a6a00640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f228c067b20 0x7f228c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.274+0000 7f22a6a00640 1 -- 172.21.15.86:0/1268584100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5421+0+0 (secure 0 0 0) 0x7f22a0086640 con 0x7f22b00b30f0 2024-03-22T04:16:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.274+0000 7f22a5400640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f228c067b20 0x7f228c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.274+0000 7f22a5400640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f228c067b20 0x7f228c069fe0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f22980b1000 tx=0x7f2298008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:45.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.278+0000 7f22a6a00640 1 -- 172.21.15.86:0/1268584100 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f22a0022020 con 0x7f22b00b30f0 2024-03-22T04:16:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:45.490+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f22b015eb60 con 0x7f22b00b30f0 2024-03-22T04:16:46.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.442+0000 7f22a6a00640 1 -- 172.21.15.86:0/1268584100 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v324) ==== 170+0+0 (secure 0 0 0) 0x7f22a00570d0 con 0x7f22b00b30f0 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f228c067b20 msgr2=0x7f228c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f228c067b20 0x7f228c069fe0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f22980b1000 tx=0x7f2298008040 comp rx=0 tx=0).stop 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00b30f0 msgr2=0x7f22b009dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00b30f0 0x7f22b009dc60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d830 tx=0x7f22a000dcf0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 shutdown_connections 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f228c067b20 0x7f228c069fe0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b015b3f0 0x7f22b0164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b00cc3e0 0x7f22b0157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 --2- 172.21.15.86:0/1268584100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22b00b30f0 0x7f22b009dc60 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 >> 172.21.15.86:0/1268584100 conn(0x7f22b00b1df0 msgr2=0x7f22b00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 shutdown_connections 2024-03-22T04:16:46.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.446+0000 7f22b5000640 1 -- 172.21.15.86:0/1268584100 wait complete. 2024-03-22T04:16:46.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-22T04:16:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0285e00640 1 Processor -- start 2024-03-22T04:16:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0285e00640 1 -- start start 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0285e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f02800cb1d0 0x7f0280162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0285e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f02800c7300 0x7f0280162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0285e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02800cc7c0 con 0x7f02800c6f20 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0285e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f02800cb720 con 0x7f02800cb1d0 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0285e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f02800cba00 con 0x7f02800c7300 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0276000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f02800cb1d0 0x7f0280162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0276000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f02800cb1d0 0x7f0280162030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57976/0 (socket says 172.21.15.86:57976) 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0276000640 1 -- 172.21.15.86:0/2065369704 learned_addr learned my addr 172.21.15.86:0/2065369704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f027f400640 1 --2- 172.21.15.86:0/2065369704 >> v2:172.21.15.196:3300/0 conn(0x7f02800c7300 0x7f0280162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0276000640 1 -- 172.21.15.86:0/2065369704 >> v2:172.21.15.196:3300/0 conn(0x7f02800c7300 msgr2=0x7f0280162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0276000640 1 --2- 172.21.15.86:0/2065369704 >> v2:172.21.15.196:3300/0 conn(0x7f02800c7300 0x7f0280162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0276000640 1 -- 172.21.15.86:0/2065369704 >> v1:172.21.15.196:6789/0 conn(0x7f02800c6f20 legacy=0x7f0280072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.610+0000 7f0276000640 1 -- 172.21.15.86:0/2065369704 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02800cbce0 con 0x7f02800cb1d0 2024-03-22T04:16:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0276000640 1 --2- 172.21.15.86:0/2065369704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800cb1d0 0x7f0280162030 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f02800cad30 tx=0x7f026801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d86996d696cc8af server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f027e000640 1 -- 172.21.15.86:0/2065369704 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02680b1070 con 0x7f02800cb1d0 2024-03-22T04:16:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f027e000640 1 -- 172.21.15.86:0/2065369704 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02680ac630 con 0x7f02800cb1d0 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f027e000640 1 -- 172.21.15.86:0/2065369704 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02680a8050 con 0x7f02800cb1d0 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0285e00640 1 -- 172.21.15.86:0/2065369704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800cb1d0 msgr2=0x7f0280162030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0285e00640 1 --2- 172.21.15.86:0/2065369704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800cb1d0 0x7f0280162030 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f02800cad30 tx=0x7f026801c4e0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0285e00640 1 -- 172.21.15.86:0/2065369704 shutdown_connections 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0285e00640 1 --2- 172.21.15.86:0/2065369704 >> v2:172.21.15.196:3300/0 conn(0x7f02800c7300 0x7f0280162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0285e00640 1 --2- 172.21.15.86:0/2065369704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800cb1d0 0x7f0280162030 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0285e00640 1 -- 172.21.15.86:0/2065369704 >> 172.21.15.86:0/2065369704 conn(0x7f02800d0730 msgr2=0x7f02800d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0285e00640 1 -- 172.21.15.86:0/2065369704 shutdown_connections 2024-03-22T04:16:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.614+0000 7f0285e00640 1 -- 172.21.15.86:0/2065369704 wait complete. 2024-03-22T04:16:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 Processor -- start 2024-03-22T04:16:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 -- start start 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800c6f20 0x7f028014c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02800c7300 0x7f028015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02800cb1d0 0x7f02801567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02800c6190 con 0x7f02800c6f20 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02800c6010 con 0x7f02800c7300 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f02800c6310 con 0x7f02800cb1d0 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800c6f20 0x7f028014c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800c6f20 0x7f028014c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57986/0 (socket says 172.21.15.86:57986) 2024-03-22T04:16:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 -- 172.21.15.86:0/2810413087 learned_addr learned my addr 172.21.15.86:0/2810413087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0276000640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02800c7300 0x7f028015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027f400640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02800cb1d0 0x7f02801567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 -- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02800cb1d0 msgr2=0x7f02801567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02800cb1d0 0x7f02801567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 -- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02800c7300 msgr2=0x7f028015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02800c7300 0x7f028015d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 -- 172.21.15.86:0/2810413087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02800c83f0 con 0x7f02800c6f20 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027f400640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02800cb1d0 0x7f02801567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f027ea00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800c6f20 0x7f028014c620 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f027000ba50 tx=0x7f027000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0277e00640 1 -- 172.21.15.86:0/2810413087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270019070 con 0x7f02800c6f20 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0277e00640 1 -- 172.21.15.86:0/2810413087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f027000cc20 con 0x7f02800c6f20 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0277e00640 1 -- 172.21.15.86:0/2810413087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270014440 con 0x7f02800c6f20 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02800cd1b0 con 0x7f02800c6f20 2024-03-22T04:16:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.618+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02800cd770 con 0x7f02800c6f20 2024-03-22T04:16:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.622+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0280074470 con 0x7f02800c6f20 2024-03-22T04:16:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.622+0000 7f0277e00640 1 -- 172.21.15.86:0/2810413087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f027001c070 con 0x7f02800c6f20 2024-03-22T04:16:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.622+0000 7f0277e00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f025c067b20 0x7f025c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.622+0000 7f0277e00640 1 -- 172.21.15.86:0/2810413087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5421+0+0 (secure 0 0 0) 0x7f0270085b20 con 0x7f02800c6f20 2024-03-22T04:16:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.622+0000 7f0276000640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f025c067b20 0x7f025c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.622+0000 7f0276000640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f025c067b20 0x7f025c069fe0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f02680ae000 tx=0x7f0268008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.626+0000 7f0277e00640 1 -- 172.21.15.86:0/2810413087 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f02800c83f0 con 0x7f02800c6f20 2024-03-22T04:16:46.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.782+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0280072a90 con 0x7f02800c6f20 2024-03-22T04:16:46.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0277e00640 1 -- 172.21.15.86:0/2810413087 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f0270055f60 con 0x7f02800c6f20 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f025c067b20 msgr2=0x7f025c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f025c067b20 0x7f025c069fe0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f02680ae000 tx=0x7f0268008040 comp rx=0 tx=0).stop 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800c6f20 msgr2=0x7f028014c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800c6f20 0x7f028014c620 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f027000ba50 tx=0x7f027000bf10 comp rx=0 tx=0).stop 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 shutdown_connections 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f025c067b20 0x7f025c069fe0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02800c7300 0x7f028015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f02800cb1d0 0x7f02801567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 --2- 172.21.15.86:0/2810413087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02800c6f20 0x7f028014c620 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:46.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 >> 172.21.15.86:0/2810413087 conn(0x7f02800d0730 msgr2=0x7f02800ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:46.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 shutdown_connections 2024-03-22T04:16:46.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:46.790+0000 7f0285e00640 1 -- 172.21.15.86:0/2810413087 wait complete. 2024-03-22T04:16:46.810 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:16:46.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:16:46.883 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:16:46.967 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:16:46.967 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:16:46.967 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:46.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:16:46.976 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:16:46.976 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:46.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:16:46.985 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:16:46.985 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:47.024 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:16:47.024 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:16:47.024 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:16:47.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:16:47.040 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-22T04:16:47.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:16:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.202+0000 7fd8f1400640 1 Processor -- start 2024-03-22T04:16:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.202+0000 7fd8f1400640 1 -- start start 2024-03-22T04:16:47.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.202+0000 7fd8f1400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec0b6080 0x7fd8ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:47.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.202+0000 7fd8f1400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8ec06f320 0x7fd8ec15c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:47.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.202+0000 7fd8f1400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ec160e10 con 0x7fd8ec13f190 2024-03-22T04:16:47.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.202+0000 7fd8f1400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ec160050 con 0x7fd8ec06f320 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.202+0000 7fd8f1400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec160330 con 0x7fd8ec0b6080 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8eb200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8ec06f320 0x7fd8ec15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8ea800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd8ec13f190 0x7fd8ec0b5970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52906/0 (socket says 172.21.15.86:52906) 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8e1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec0b6080 0x7fd8ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8ea800640 1 -- 172.21.15.86:0/2360334796 learned_addr learned my addr 172.21.15.86:0/2360334796 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8eb200640 1 -- 172.21.15.86:0/2360334796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec0b6080 msgr2=0x7fd8ec06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8eb200640 1 --2- 172.21.15.86:0/2360334796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec0b6080 0x7fd8ec06ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8eb200640 1 -- 172.21.15.86:0/2360334796 >> v1:172.21.15.196:6789/0 conn(0x7fd8ec13f190 legacy=0x7fd8ec0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8eb200640 1 -- 172.21.15.86:0/2360334796 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec160610 con 0x7fd8ec06f320 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8e9e00640 1 -- 172.21.15.86:0/2360334796 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2033112644 0 0) 0x7fd8ec160e10 con 0x7fd8ec13f190 2024-03-22T04:16:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.206+0000 7fd8e1e00640 1 --2- 172.21.15.86:0/2360334796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec0b6080 0x7fd8ec06ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:16:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8eb200640 1 --2- 172.21.15.86:0/2360334796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec06f320 0x7fd8ec15c890 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40097c0 tx=0x7fd8e401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f6245be9c1ab11f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8e9e00640 1 -- 172.21.15.86:0/2360334796 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40b0070 con 0x7fd8ec06f320 2024-03-22T04:16:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8e9e00640 1 -- 172.21.15.86:0/2360334796 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e40ab600 con 0x7fd8ec06f320 2024-03-22T04:16:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8e9e00640 1 -- 172.21.15.86:0/2360334796 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40ab9d0 con 0x7fd8ec06f320 2024-03-22T04:16:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- 172.21.15.86:0/2360334796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec06f320 msgr2=0x7fd8ec15c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 --2- 172.21.15.86:0/2360334796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec06f320 0x7fd8ec15c890 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40097c0 tx=0x7fd8e401c410 comp rx=0 tx=0).stop 2024-03-22T04:16:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- 172.21.15.86:0/2360334796 shutdown_connections 2024-03-22T04:16:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 --2- 172.21.15.86:0/2360334796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec06f320 0x7fd8ec15c890 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 --2- 172.21.15.86:0/2360334796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec0b6080 0x7fd8ec06ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- 172.21.15.86:0/2360334796 >> 172.21.15.86:0/2360334796 conn(0x7fd8ec06b660 msgr2=0x7fd8ec06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- 172.21.15.86:0/2360334796 shutdown_connections 2024-03-22T04:16:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- 172.21.15.86:0/2360334796 wait complete. 2024-03-22T04:16:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 Processor -- start 2024-03-22T04:16:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- start start 2024-03-22T04:16:47.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8ec06f320 0x7fd8ec138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec0b6080 0x7fd8ec1f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec13f190 0x7fd8ec1f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec169270 con 0x7fd8ec06f320 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec15d7f0 con 0x7fd8ec0b6080 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8f1400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec1693f0 con 0x7fd8ec13f190 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.210+0000 7fd8ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8ec06f320 0x7fd8ec138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8ec06f320 0x7fd8ec138f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58000/0 (socket says 172.21.15.86:58000) 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8ea800640 1 -- 172.21.15.86:0/1520567632 learned_addr learned my addr 172.21.15.86:0/1520567632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8eb200640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec13f190 0x7fd8ec1f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8e1e00640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec0b6080 0x7fd8ec1f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8ea800640 1 -- 172.21.15.86:0/1520567632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec13f190 msgr2=0x7fd8ec1f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8ea800640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec13f190 0x7fd8ec1f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8ea800640 1 -- 172.21.15.86:0/1520567632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec0b6080 msgr2=0x7fd8ec1f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8ea800640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec0b6080 0x7fd8ec1f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8ea800640 1 -- 172.21.15.86:0/1520567632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec0b0450 con 0x7fd8ec06f320 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8ea800640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8ec06f320 0x7fd8ec138f10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c7c0 tx=0x7fd8dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8e3400640 1 -- 172.21.15.86:0/1520567632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc018070 con 0x7fd8ec06f320 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8e3400640 1 -- 172.21.15.86:0/1520567632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc00a640 con 0x7fd8ec06f320 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8e3400640 1 -- 172.21.15.86:0/1520567632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc00aa30 con 0x7fd8ec06f320 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec0b0730 con 0x7fd8ec06f320 2024-03-22T04:16:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec0b0cf0 con 0x7fd8ec06f320 2024-03-22T04:16:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec068650 con 0x7fd8ec06f320 2024-03-22T04:16:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8e3400640 1 -- 172.21.15.86:0/1520567632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd8dc01b070 con 0x7fd8ec06f320 2024-03-22T04:16:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8e3400640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8c8067b20 0x7fd8c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.214+0000 7fd8e3400640 1 -- 172.21.15.86:0/1520567632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5421+0+0 (secure 0 0 0) 0x7fd8dc0853a0 con 0x7fd8ec06f320 2024-03-22T04:16:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.218+0000 7fd8e1e00640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8c8067b20 0x7fd8c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.218+0000 7fd8e1e00640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8c8067b20 0x7fd8c8069fe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009580 tx=0x7fd8d4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.218+0000 7fd8e3400640 1 -- 172.21.15.86:0/1520567632 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd8dc055880 con 0x7fd8ec06f320 2024-03-22T04:16:47.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.406+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8ec15c890 con 0x7fd8ec06f320 2024-03-22T04:16:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.406+0000 7fd8e3400640 1 -- 172.21.15.86:0/1520567632 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+3875 (secure 0 0 0) 0x7fd8dc059920 con 0x7fd8ec06f320 2024-03-22T04:16:47.407 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:47.407 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6724,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1704161503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1704161503},{"type":"v1","addr":"172.21.15.196:6834","nonce":1704161503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2938678256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2938678256},{"type":"v1","addr":"172.21.15.86:6835","nonce":2938678256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1780861717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1780861717},{"type":"v1","addr":"172.21.15.196:6835","nonce":1780861717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7407,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4084852779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4084852779},{"type":"v1","addr":"172.21.15.86:6837","nonce":4084852779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[]} 2024-03-22T04:16:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 56 2024-03-22T04:16:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8c8067b20 msgr2=0x7fd8c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8c8067b20 0x7fd8c8069fe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009580 tx=0x7fd8d4009190 comp rx=0 tx=0).stop 2024-03-22T04:16:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8ec06f320 msgr2=0x7fd8ec138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8ec06f320 0x7fd8ec138f10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c7c0 tx=0x7fd8dc00cc80 comp rx=0 tx=0).stop 2024-03-22T04:16:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 shutdown_connections 2024-03-22T04:16:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8c8067b20 0x7fd8c8069fe0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8ec0b6080 0x7fd8ec1f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8ec13f190 0x7fd8ec1f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 --2- 172.21.15.86:0/1520567632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8ec06f320 0x7fd8ec138f10 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 >> 172.21.15.86:0/1520567632 conn(0x7fd8ec06b660 msgr2=0x7fd8ec0af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 shutdown_connections 2024-03-22T04:16:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:47.410+0000 7fd8f1400640 1 -- 172.21.15.86:0/1520567632 wait complete. 2024-03-22T04:16:47.428 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-22T04:16:47.428 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-22T04:16:47.428 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-22T04:16:47.429 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:16:47.429 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-22T04:16:47.429 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-22T04:16:47.429 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-22T04:16:47.429 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:16:47.429 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-22T04:16:47.429 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-22T04:16:47.430 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-22T04:16:47.430 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:16:47.430 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-22T04:16:47.430 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-22T04:16:47.430 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-22T04:16:47.430 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:16:47.441 INFO:tasks.ceph.mds.a:Stopped 2024-03-22T04:16:47.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-22T04:16:47.442 INFO:tasks.ceph.mds.c:Stopped 2024-03-22T04:16:47.442 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-22T04:16:47.445 INFO:tasks.ceph.mds.d:Stopped 2024-03-22T04:16:47.446 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-22T04:16:47.487 INFO:tasks.ceph.mds.a:Started 2024-03-22T04:16:47.487 DEBUG:teuthology.parallel:result is None 2024-03-22T04:16:47.489 INFO:tasks.ceph.mds.c:Started 2024-03-22T04:16:47.489 DEBUG:teuthology.parallel:result is None 2024-03-22T04:16:47.491 INFO:tasks.ceph.mds.d:Started 2024-03-22T04:16:47.492 DEBUG:teuthology.parallel:result is None 2024-03-22T04:16:47.572 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:16:47.572 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-03-22T04:16:47.579 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:16:51.286 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-03-22T04:16:53.443 INFO:tasks.ceph.mds.b:Stopped 2024-03-22T04:16:53.444 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-22T04:16:53.447 INFO:tasks.ceph.mds.b:Started 2024-03-22T04:16:53.447 DEBUG:teuthology.parallel:result is None 2024-03-22T04:16:53.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-22T04:16:53.538 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:16:53.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.594+0000 7f2c6e400640 1 Processor -- start 2024-03-22T04:16:53.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.594+0000 7f2c6e400640 1 -- start start 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c6e400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2c681418f0 0x7f2c680b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c6e400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 0x7f2c680b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c6e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c68142260 con 0x7f2c6815cd10 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c6e400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2c68142820 con 0x7f2c681418f0 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c6e400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2c68142540 con 0x7f2c68141d80 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c67400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2c6815cd10 0x7f2c6815d0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47380/0 (socket says 172.21.15.86:47380) 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c67400640 1 -- 172.21.15.86:0/1470204436 learned_addr learned my addr 172.21.15.86:0/1470204436 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c5ea00640 1 --2- 172.21.15.86:0/1470204436 >> v2:172.21.15.196:3300/0 conn(0x7f2c681418f0 0x7f2c680b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c67e00640 1 --2- 172.21.15.86:0/1470204436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 0x7f2c680b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:53.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c5ea00640 1 -- 172.21.15.86:0/1470204436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 msgr2=0x7f2c680b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:53.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c5ea00640 1 --2- 172.21.15.86:0/1470204436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 0x7f2c680b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:53.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c5ea00640 1 -- 172.21.15.86:0/1470204436 >> v1:172.21.15.86:6789/0 conn(0x7f2c6815cd10 legacy=0x7f2c6815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:53.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.598+0000 7f2c5ea00640 1 -- 172.21.15.86:0/1470204436 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6813f990 con 0x7f2c681418f0 2024-03-22T04:16:53.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c5ea00640 1 --2- 172.21.15.86:0/1470204436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 0x7f2c680b43e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2c580097c0 tx=0x7f2c5801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca79f618f039760 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:53.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c66a00640 1 -- 172.21.15.86:0/1470204436 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580b0070 con 0x7f2c681418f0 2024-03-22T04:16:53.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c66a00640 1 -- 172.21.15.86:0/1470204436 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c580ab600 con 0x7f2c681418f0 2024-03-22T04:16:53.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c66a00640 1 -- 172.21.15.86:0/1470204436 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580ab9d0 con 0x7f2c681418f0 2024-03-22T04:16:53.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c6e400640 1 -- 172.21.15.86:0/1470204436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 msgr2=0x7f2c680b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:53.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c6e400640 1 --2- 172.21.15.86:0/1470204436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 0x7f2c680b43e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2c580097c0 tx=0x7f2c5801c410 comp rx=0 tx=0).stop 2024-03-22T04:16:53.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c6e400640 1 -- 172.21.15.86:0/1470204436 shutdown_connections 2024-03-22T04:16:53.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c6e400640 1 --2- 172.21.15.86:0/1470204436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 0x7f2c680b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:53.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c6e400640 1 --2- 172.21.15.86:0/1470204436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 0x7f2c680b43e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:53.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c6e400640 1 -- 172.21.15.86:0/1470204436 >> 172.21.15.86:0/1470204436 conn(0x7f2c6806ba60 msgr2=0x7f2c6806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:53.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c6e400640 1 -- 172.21.15.86:0/1470204436 shutdown_connections 2024-03-22T04:16:53.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.602+0000 7f2c6e400640 1 -- 172.21.15.86:0/1470204436 wait complete. 2024-03-22T04:16:53.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c6e400640 1 Processor -- start 2024-03-22T04:16:53.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c6e400640 1 -- start start 2024-03-22T04:16:53.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c6e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 0x7f2c6813d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:53.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c6e400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 0x7f2c681538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:53.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c6e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6815cd10 0x7f2c680b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:53.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c6e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68140d30 con 0x7f2c6815cd10 2024-03-22T04:16:53.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c6e400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68140bb0 con 0x7f2c681418f0 2024-03-22T04:16:53.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c6e400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2c68140eb0 con 0x7f2c68141d80 2024-03-22T04:16:53.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6815cd10 0x7f2c680b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6815cd10 0x7f2c680b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58030/0 (socket says 172.21.15.86:58030) 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 -- 172.21.15.86:0/2578128509 learned_addr learned my addr 172.21.15.86:0/2578128509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 0x7f2c6813d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c5ea00640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 0x7f2c681538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 -- 172.21.15.86:0/2578128509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 msgr2=0x7f2c681538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 0x7f2c681538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 -- 172.21.15.86:0/2578128509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 msgr2=0x7f2c6813d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 0x7f2c6813d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 -- 172.21.15.86:0/2578128509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c68138cf0 con 0x7f2c6815cd10 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c67e00640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6815cd10 0x7f2c680b3170 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000efc0 tx=0x7f2c6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c5fe00640 1 -- 172.21.15.86:0/2578128509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60018020 con 0x7f2c6815cd10 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c5fe00640 1 -- 172.21.15.86:0/2578128509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c60014480 con 0x7f2c6815cd10 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.606+0000 7f2c5fe00640 1 -- 172.21.15.86:0/2578128509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60014870 con 0x7f2c6815cd10 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.610+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c68138fd0 con 0x7f2c6815cd10 2024-03-22T04:16:53.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.610+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c68139590 con 0x7f2c6815cd10 2024-03-22T04:16:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.610+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c68068650 con 0x7f2c6815cd10 2024-03-22T04:16:53.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.610+0000 7f2c5fe00640 1 -- 172.21.15.86:0/2578128509 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2c6001b020 con 0x7f2c6815cd10 2024-03-22T04:16:53.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.610+0000 7f2c5fe00640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c44067b20 0x7f2c44069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:53.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.610+0000 7f2c5fe00640 1 -- 172.21.15.86:0/2578128509 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5421+0+0 (secure 0 0 0) 0x7f2c60086c30 con 0x7f2c6815cd10 2024-03-22T04:16:53.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.610+0000 7f2c67400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c44067b20 0x7f2c44069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:53.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.610+0000 7f2c67400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c44067b20 0x7f2c44069fe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2c50000f70 tx=0x7f2c50007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:53.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.614+0000 7f2c5fe00640 1 -- 172.21.15.86:0/2578128509 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2c60022030 con 0x7f2c6815cd10 2024-03-22T04:16:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:53.782+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2c68166f70 con 0x7f2c6815cd10 2024-03-22T04:16:54.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c5fe00640 1 -- 172.21.15.86:0/2578128509 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) ==== 97+0+0 (secure 0 0 0) 0x7f2c60010040 con 0x7f2c6815cd10 2024-03-22T04:16:54.613 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c44067b20 msgr2=0x7f2c44069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c44067b20 0x7f2c44069fe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2c50000f70 tx=0x7f2c50007e90 comp rx=0 tx=0).stop 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6815cd10 msgr2=0x7f2c680b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6815cd10 0x7f2c680b3170 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000efc0 tx=0x7f2c6000c490 comp rx=0 tx=0).stop 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 shutdown_connections 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c44067b20 0x7f2c44069fe0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6815cd10 0x7f2c680b3170 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c68141d80 0x7f2c681538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 --2- 172.21.15.86:0/2578128509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c681418f0 0x7f2c6813d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 >> 172.21.15.86:0/2578128509 conn(0x7f2c6806ba60 msgr2=0x7f2c680b6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 shutdown_connections 2024-03-22T04:16:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.614+0000 7f2c6e400640 1 -- 172.21.15.86:0/2578128509 wait complete. 2024-03-22T04:16:54.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-22T04:16:54.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4489600640 1 Processor -- start 2024-03-22T04:16:54.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4489600640 1 -- start start 2024-03-22T04:16:54.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4489600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840ce6b0 0x7f44840cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4489600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f448415fed0 0x7f4484160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4489600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44840d11c0 con 0x7f44840d0de0 2024-03-22T04:16:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4489600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4484160860 con 0x7f448415fed0 2024-03-22T04:16:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4489600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44840cece0 con 0x7f44840ce6b0 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4482000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840ce6b0 0x7f44840cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4483400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f448415fed0 0x7f4484160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4482a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f44840d0de0 0x7f44840d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52944/0 (socket says 172.21.15.86:52944) 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4482a00640 1 -- 172.21.15.86:0/612635229 learned_addr learned my addr 172.21.15.86:0/612635229 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4481600640 1 -- 172.21.15.86:0/612635229 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3113314728 0 0) 0x7f44840d11c0 con 0x7f44840d0de0 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4481600640 1 -- 172.21.15.86:0/612635229 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4468003530 con 0x7f44840d0de0 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4482000640 1 -- 172.21.15.86:0/612635229 >> v2:172.21.15.196:3300/0 conn(0x7f448415fed0 msgr2=0x7f4484160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4482000640 1 --2- 172.21.15.86:0/612635229 >> v2:172.21.15.196:3300/0 conn(0x7f448415fed0 0x7f4484160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4482000640 1 -- 172.21.15.86:0/612635229 >> v1:172.21.15.196:6789/0 conn(0x7f44840d0de0 legacy=0x7f44840d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4482000640 1 -- 172.21.15.86:0/612635229 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44840cefc0 con 0x7f44840ce6b0 2024-03-22T04:16:54.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.774+0000 7f4483400640 1 --2- 172.21.15.86:0/612635229 >> v2:172.21.15.196:3300/0 conn(0x7f448415fed0 0x7f4484160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:16:54.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4482000640 1 --2- 172.21.15.86:0/612635229 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840ce6b0 0x7f44840cfe70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f44700097c0 tx=0x7f447001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df160510c0f9a5e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:54.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4481600640 1 -- 172.21.15.86:0/612635229 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700b0070 con 0x7f44840ce6b0 2024-03-22T04:16:54.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4481600640 1 -- 172.21.15.86:0/612635229 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44700ab600 con 0x7f44840ce6b0 2024-03-22T04:16:54.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4481600640 1 -- 172.21.15.86:0/612635229 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700ab9d0 con 0x7f44840ce6b0 2024-03-22T04:16:54.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4489600640 1 -- 172.21.15.86:0/612635229 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840ce6b0 msgr2=0x7f44840cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:54.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4489600640 1 --2- 172.21.15.86:0/612635229 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840ce6b0 0x7f44840cfe70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f44700097c0 tx=0x7f447001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4489600640 1 -- 172.21.15.86:0/612635229 shutdown_connections 2024-03-22T04:16:54.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4489600640 1 --2- 172.21.15.86:0/612635229 >> v2:172.21.15.196:3300/0 conn(0x7f448415fed0 0x7f4484160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4489600640 1 --2- 172.21.15.86:0/612635229 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840ce6b0 0x7f44840cfe70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4489600640 1 -- 172.21.15.86:0/612635229 >> 172.21.15.86:0/612635229 conn(0x7f448406f910 msgr2=0x7f448406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:54.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4489600640 1 -- 172.21.15.86:0/612635229 shutdown_connections 2024-03-22T04:16:54.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.778+0000 7f4489600640 1 -- 172.21.15.86:0/612635229 wait complete. 2024-03-22T04:16:54.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 Processor -- start 2024-03-22T04:16:54.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 -- start start 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44840ce6b0 0x7f448406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840d0de0 0x7f448406c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f448415fed0 0x7f4484156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44840d0530 con 0x7f44840ce6b0 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44840d03b0 con 0x7f448415fed0 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4484071e30 con 0x7f44840d0de0 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44840ce6b0 0x7f448406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44840ce6b0 0x7f448406e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58034/0 (socket says 172.21.15.86:58034) 2024-03-22T04:16:54.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 -- 172.21.15.86:0/2282648488 learned_addr learned my addr 172.21.15.86:0/2282648488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482000640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840d0de0 0x7f448406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4483400640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f448415fed0 0x7f4484156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 -- 172.21.15.86:0/2282648488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840d0de0 msgr2=0x7f448406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840d0de0 0x7f448406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 -- 172.21.15.86:0/2282648488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f448415fed0 msgr2=0x7f4484156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f448415fed0 0x7f4484156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 -- 172.21.15.86:0/2282648488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4484153d20 con 0x7f44840ce6b0 2024-03-22T04:16:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4482a00640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44840ce6b0 0x7f448406e3a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f447400cb40 tx=0x7f447400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:54.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f447aa00640 1 -- 172.21.15.86:0/2282648488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474017070 con 0x7f44840ce6b0 2024-03-22T04:16:54.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f447aa00640 1 -- 172.21.15.86:0/2282648488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f447400ac30 con 0x7f44840ce6b0 2024-03-22T04:16:54.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448406ce00 con 0x7f44840ce6b0 2024-03-22T04:16:54.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f447aa00640 1 -- 172.21.15.86:0/2282648488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474012400 con 0x7f44840ce6b0 2024-03-22T04:16:54.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.782+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448406d3c0 con 0x7f44840ce6b0 2024-03-22T04:16:54.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.786+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4484068650 con 0x7f44840ce6b0 2024-03-22T04:16:54.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.786+0000 7f447aa00640 1 -- 172.21.15.86:0/2282648488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f447401a070 con 0x7f44840ce6b0 2024-03-22T04:16:54.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.786+0000 7f447aa00640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4464067b20 0x7f4464069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:54.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.786+0000 7f447aa00640 1 -- 172.21.15.86:0/2282648488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f4474085b30 con 0x7f44840ce6b0 2024-03-22T04:16:54.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.786+0000 7f4482000640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4464067b20 0x7f4464069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:54.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.786+0000 7f4482000640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4464067b20 0x7f4464069fe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f44700ad000 tx=0x7f4470008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:54.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.790+0000 7f447aa00640 1 -- 172.21.15.86:0/2282648488 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4484153d20 con 0x7f44840ce6b0 2024-03-22T04:16:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:54.998+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f44840ccae0 con 0x7f44840ce6b0 2024-03-22T04:16:55.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f447aa00640 1 -- 172.21.15.86:0/2282648488 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f4474055f80 con 0x7f44840ce6b0 2024-03-22T04:16:55.001 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:55.001 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:16:55.001 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-03-22T04:16:55.001 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA==", 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw==", 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q==", 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg==", 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA==", 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA==", 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.007 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg==", 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.008 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA==", 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-03-22T04:16:55.009 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ==", 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow==", 2024-03-22T04:16:55.010 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg==", 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.011 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg==", 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.012 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA==", 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-03-22T04:16:55.013 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg==", 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:16:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w==", 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:16:55.015 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w==", 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:16:55.016 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB4BP1lvLoQORAADwVEr78V9aUc1YblO47Gaw==", 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:16:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g==", 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ==", 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.020 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T04:16:55.020 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:16:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4464067b20 msgr2=0x7f4464069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4464067b20 0x7f4464069fe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f44700ad000 tx=0x7f4470008040 comp rx=0 tx=0).stop 2024-03-22T04:16:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44840ce6b0 msgr2=0x7f448406e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44840ce6b0 0x7f448406e3a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f447400cb40 tx=0x7f447400cf40 comp rx=0 tx=0).stop 2024-03-22T04:16:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 shutdown_connections 2024-03-22T04:16:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4464067b20 0x7f4464069fe0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f448415fed0 0x7f4484156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44840d0de0 0x7f448406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 --2- 172.21.15.86:0/2282648488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44840ce6b0 0x7f448406e3a0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 >> 172.21.15.86:0/2282648488 conn(0x7f448406f910 msgr2=0x7f448415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 shutdown_connections 2024-03-22T04:16:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.002+0000 7f4489600640 1 -- 172.21.15.86:0/2282648488 wait complete. 2024-03-22T04:16:55.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:16:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f000a800640 1 Processor -- start 2024-03-22T04:16:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f000a800640 1 -- start start 2024-03-22T04:16:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f000a800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f000406ef40 0x7f0004142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f000a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f00040ad710 0x7f000415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f000a800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000415c7c0 con 0x7f00040ad330 2024-03-22T04:16:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f000a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f000406dea0 con 0x7f00040ad710 2024-03-22T04:16:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f000a800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f000406e180 con 0x7f000406ef40 2024-03-22T04:16:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f0003e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f00040ad710 0x7f000415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f0003e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f00040ad710 0x7f000415d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58050/0 (socket says 172.21.15.86:58050) 2024-03-22T04:16:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f0003e00640 1 -- 172.21.15.86:0/2844207844 learned_addr learned my addr 172.21.15.86:0/2844207844 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.158+0000 7f0002a00640 1 --2- 172.21.15.86:0/2844207844 >> v2:172.21.15.196:3300/0 conn(0x7f000406ef40 0x7f0004142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f0003e00640 1 -- 172.21.15.86:0/2844207844 >> v2:172.21.15.196:3300/0 conn(0x7f000406ef40 msgr2=0x7f0004142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f0003e00640 1 --2- 172.21.15.86:0/2844207844 >> v2:172.21.15.196:3300/0 conn(0x7f000406ef40 0x7f0004142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f0003e00640 1 -- 172.21.15.86:0/2844207844 >> v1:172.21.15.196:6789/0 conn(0x7f00040ad330 legacy=0x7f000413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f0003e00640 1 -- 172.21.15.86:0/2844207844 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000406e460 con 0x7f00040ad710 2024-03-22T04:16:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f0003e00640 1 --2- 172.21.15.86:0/2844207844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00040ad710 0x7f000415d4a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f000406fbe0 tx=0x7efffc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a3535ca2e1309d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f0002000640 1 -- 172.21.15.86:0/2844207844 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc0aa650 con 0x7f00040ad710 2024-03-22T04:16:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f0002000640 1 -- 172.21.15.86:0/2844207844 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efffc0af070 con 0x7f00040ad710 2024-03-22T04:16:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f0002000640 1 -- 172.21.15.86:0/2844207844 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc0aaa10 con 0x7f00040ad710 2024-03-22T04:16:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f000a800640 1 -- 172.21.15.86:0/2844207844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00040ad710 msgr2=0x7f000415d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f000a800640 1 --2- 172.21.15.86:0/2844207844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00040ad710 0x7f000415d4a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f000406fbe0 tx=0x7efffc01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f000a800640 1 -- 172.21.15.86:0/2844207844 shutdown_connections 2024-03-22T04:16:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f000a800640 1 --2- 172.21.15.86:0/2844207844 >> v2:172.21.15.196:3300/0 conn(0x7f000406ef40 0x7f0004142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f000a800640 1 --2- 172.21.15.86:0/2844207844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00040ad710 0x7f000415d4a0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f000a800640 1 -- 172.21.15.86:0/2844207844 >> 172.21.15.86:0/2844207844 conn(0x7f00040b1df0 msgr2=0x7f00040b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f000a800640 1 -- 172.21.15.86:0/2844207844 shutdown_connections 2024-03-22T04:16:55.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.162+0000 7f000a800640 1 -- 172.21.15.86:0/2844207844 wait complete. 2024-03-22T04:16:55.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 Processor -- start 2024-03-22T04:16:55.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 -- start start 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000406ef40 0x7f000415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00040ad330 0x7f0004137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00040ad710 0x7f000415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00041690e0 con 0x7f000406ef40 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0004168f60 con 0x7f00040ad710 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0004169260 con 0x7f00040ad330 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000406ef40 0x7f000415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000406ef40 0x7f000415b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58058/0 (socket says 172.21.15.86:58058) 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 -- 172.21.15.86:0/141784493 learned_addr learned my addr 172.21.15.86:0/141784493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0002a00640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00040ad330 0x7f0004137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003e00640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00040ad710 0x7f000415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 -- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00040ad330 msgr2=0x7f0004137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00040ad330 0x7f0004137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 -- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00040ad710 msgr2=0x7f000415ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00040ad710 0x7f000415ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 -- 172.21.15.86:0/141784493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000409b260 con 0x7f000406ef40 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0002a00640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00040ad330 0x7f0004137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f0003400640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000406ef40 0x7f000415b300 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efff400c770 tx=0x7efff400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7efffb400640 1 -- 172.21.15.86:0/141784493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4003ba0 con 0x7f000406ef40 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7efffb400640 1 -- 172.21.15.86:0/141784493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff4003d60 con 0x7f000406ef40 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7efffb400640 1 -- 172.21.15.86:0/141784493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff400aa90 con 0x7f000406ef40 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000409b540 con 0x7f000406ef40 2024-03-22T04:16:55.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.166+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000409bb00 con 0x7f000406ef40 2024-03-22T04:16:55.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.170+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0004068650 con 0x7f000406ef40 2024-03-22T04:16:55.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.170+0000 7efffb400640 1 -- 172.21.15.86:0/141784493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7efff400ac30 con 0x7f000406ef40 2024-03-22T04:16:55.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.170+0000 7efffb400640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7effe4067b20 0x7effe4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.170+0000 7efffb400640 1 -- 172.21.15.86:0/141784493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7efff4085d80 con 0x7f000406ef40 2024-03-22T04:16:55.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.170+0000 7f0002a00640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7effe4067b20 0x7effe4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.170+0000 7f0002a00640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7effe4067b20 0x7effe4069fe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f000415b840 tx=0x7efff0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.174+0000 7efffb400640 1 -- 172.21.15.86:0/141784493 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efff40568b0 con 0x7f000406ef40 2024-03-22T04:16:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.326+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00040d0f10 con 0x7f000406ef40 2024-03-22T04:16:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7efffb400640 1 -- 172.21.15.86:0/141784493 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+13165 (secure 0 0 0) 0x7efff40561b0 con 0x7f000406ef40 2024-03-22T04:16:55.329 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:55.329 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:16:54.606469+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:16:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7effe4067b20 msgr2=0x7effe4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7effe4067b20 0x7effe4069fe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f000415b840 tx=0x7efff0008040 comp rx=0 tx=0).stop 2024-03-22T04:16:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000406ef40 msgr2=0x7f000415b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000406ef40 0x7f000415b300 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efff400c770 tx=0x7efff400cc30 comp rx=0 tx=0).stop 2024-03-22T04:16:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 shutdown_connections 2024-03-22T04:16:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7effe4067b20 0x7effe4069fe0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00040ad710 0x7f000415ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00040ad330 0x7f0004137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 --2- 172.21.15.86:0/141784493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000406ef40 0x7f000415b300 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 >> 172.21.15.86:0/141784493 conn(0x7f00040b1df0 msgr2=0x7f0004071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 shutdown_connections 2024-03-22T04:16:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.330+0000 7f000a800640 1 -- 172.21.15.86:0/141784493 wait complete. 2024-03-22T04:16:55.350 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:16:55.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:16:55.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.486+0000 7effc0200640 1 Processor -- start 2024-03-22T04:16:55.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.486+0000 7effc0200640 1 -- start start 2024-03-22T04:16:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effc0200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7effb80b2360 0x7effb80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effc0200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effc0200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb815d710 con 0x7effb80b3430 2024-03-22T04:16:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effc0200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7effb80b2740 con 0x7effb80b2360 2024-03-22T04:16:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effc0200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7effb815d4b0 con 0x7effb815d0b0 2024-03-22T04:16:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbd800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7effb80b2360 0x7effb80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbd800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7effb80b2360 0x7effb80b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58064/0 (socket says 172.21.15.86:58064) 2024-03-22T04:16:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbd800640 1 -- 172.21.15.86:0/2984078306 learned_addr learned my addr 172.21.15.86:0/2984078306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbe200640 1 --2- 172.21.15.86:0/2984078306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbd800640 1 -- 172.21.15.86:0/2984078306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 msgr2=0x7effb80b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbd800640 1 --2- 172.21.15.86:0/2984078306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbd800640 1 -- 172.21.15.86:0/2984078306 >> v1:172.21.15.196:6789/0 conn(0x7effb80b3430 legacy=0x7effb815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbd800640 1 -- 172.21.15.86:0/2984078306 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb815fe40 con 0x7effb80b2360 2024-03-22T04:16:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.490+0000 7effbe200640 1 --2- 172.21.15.86:0/2984078306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effbd800640 1 --2- 172.21.15.86:0/2984078306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80b2ef0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7effac0098a0 tx=0x7effac01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75e3d76bab472ae server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effb7e00640 1 -- 172.21.15.86:0/2984078306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0ab430 con 0x7effb80b2360 2024-03-22T04:16:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effb7e00640 1 -- 172.21.15.86:0/2984078306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac0b0070 con 0x7effb80b2360 2024-03-22T04:16:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effb7e00640 1 -- 172.21.15.86:0/2984078306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0ab7f0 con 0x7effb80b2360 2024-03-22T04:16:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effc0200640 1 -- 172.21.15.86:0/2984078306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 msgr2=0x7effb80b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effc0200640 1 --2- 172.21.15.86:0/2984078306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80b2ef0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7effac0098a0 tx=0x7effac01c540 comp rx=0 tx=0).stop 2024-03-22T04:16:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effc0200640 1 -- 172.21.15.86:0/2984078306 shutdown_connections 2024-03-22T04:16:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effc0200640 1 --2- 172.21.15.86:0/2984078306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effc0200640 1 --2- 172.21.15.86:0/2984078306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80b2ef0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effc0200640 1 -- 172.21.15.86:0/2984078306 >> 172.21.15.86:0/2984078306 conn(0x7effb80cc940 msgr2=0x7effb80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effc0200640 1 -- 172.21.15.86:0/2984078306 shutdown_connections 2024-03-22T04:16:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.494+0000 7effc0200640 1 -- 172.21.15.86:0/2984078306 wait complete. 2024-03-22T04:16:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 Processor -- start 2024-03-22T04:16:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 -- start start 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effb80b3430 0x7effb80aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7effb8160c00 con 0x7effb80b2360 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effb80b5ad0 con 0x7effb80b3430 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7effb80ac090 con 0x7effb815d0b0 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58078/0 (socket says 172.21.15.86:58078) 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 -- 172.21.15.86:0/3373915221 learned_addr learned my addr 172.21.15.86:0/3373915221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbe200640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbce00640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effb80b3430 0x7effb80aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 -- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 msgr2=0x7effb80aaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80aaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 -- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effb80b3430 msgr2=0x7effb80aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effb80b3430 0x7effb80aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 -- 172.21.15.86:0/3373915221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb8142670 con 0x7effb80b2360 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbe200640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effbd800640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80abb50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7effac0ad040 tx=0x7effac0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effb5600640 1 -- 172.21.15.86:0/3373915221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0b4420 con 0x7effb80b2360 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effb5600640 1 -- 172.21.15.86:0/3373915221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac0bb070 con 0x7effb80b2360 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effb5600640 1 -- 172.21.15.86:0/3373915221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0b4800 con 0x7effb80b2360 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb80af310 con 0x7effb80b2360 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb8142c10 con 0x7effb80b2360 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.498+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb8074470 con 0x7effb80b2360 2024-03-22T04:16:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.502+0000 7effb5600640 1 -- 172.21.15.86:0/3373915221 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7effac0b49a0 con 0x7effb80b2360 2024-03-22T04:16:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.502+0000 7effb5600640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff98067b20 0x7eff98069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.502+0000 7effb5600640 1 -- 172.21.15.86:0/3373915221 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7effac11fcb0 con 0x7effb80b2360 2024-03-22T04:16:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.502+0000 7effbce00640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff98067b20 0x7eff98069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.502+0000 7effbce00640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff98067b20 0x7eff98069fe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7effb80b1550 tx=0x7effa4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.506+0000 7effb5600640 1 -- 172.21.15.86:0/3373915221 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7effac016130 con 0x7effb80b2360 2024-03-22T04:16:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.662+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7effb806e9f0 con 0x7effb80b2360 2024-03-22T04:16:55.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.662+0000 7effb5600640 1 -- 172.21.15.86:0/3373915221 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7effac0f07e0 con 0x7effb80b2360 2024-03-22T04:16:55.663 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:16:55.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff98067b20 msgr2=0x7eff98069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff98067b20 0x7eff98069fe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7effb80b1550 tx=0x7effa4009040 comp rx=0 tx=0).stop 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 msgr2=0x7effb80abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80abb50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7effac0ad040 tx=0x7effac0192e0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 shutdown_connections 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7eff98067b20 0x7eff98069fe0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effb80b3430 0x7effb80aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effb815d0b0 0x7effb80aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 --2- 172.21.15.86:0/3373915221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb80b2360 0x7effb80abb50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 >> 172.21.15.86:0/3373915221 conn(0x7effb80cc940 msgr2=0x7effb8071410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 shutdown_connections 2024-03-22T04:16:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.666+0000 7effc0200640 1 -- 172.21.15.86:0/3373915221 wait complete. 2024-03-22T04:16:55.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T04:16:55.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.818+0000 7f4c40400640 1 Processor -- start 2024-03-22T04:16:55.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c40400640 1 -- start start 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c40400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4c3815b3f0 0x7f4c380729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c40400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c380cc3e0 0x7f4c38072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c40400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c38071920 con 0x7f4c380b30f0 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c40400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4c380716d0 con 0x7f4c3815b3f0 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c40400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3806d990 con 0x7f4c380cc3e0 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c3da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4c3815b3f0 0x7f4c380729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c3da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4c3815b3f0 0x7f4c380729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58084/0 (socket says 172.21.15.86:58084) 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c3da00640 1 -- 172.21.15.86:0/1090553953 learned_addr learned my addr 172.21.15.86:0/1090553953 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:55.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c3d000640 1 --2- 172.21.15.86:0/1090553953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c380cc3e0 0x7f4c38072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c3da00640 1 -- 172.21.15.86:0/1090553953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c380cc3e0 msgr2=0x7f4c38072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:55.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c3da00640 1 --2- 172.21.15.86:0/1090553953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c380cc3e0 0x7f4c38072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c3da00640 1 -- 172.21.15.86:0/1090553953 >> v1:172.21.15.86:6789/0 conn(0x7f4c380b30f0 legacy=0x7f4c380ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.822+0000 7f4c3da00640 1 -- 172.21.15.86:0/1090553953 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c38071c00 con 0x7f4c3815b3f0 2024-03-22T04:16:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c3da00640 1 --2- 172.21.15.86:0/1090553953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c3815b3f0 0x7f4c380729c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009e50 tx=0x7f4c2c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef329f148c0d40eb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c37e00640 1 -- 172.21.15.86:0/1090553953 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0a5750 con 0x7f4c3815b3f0 2024-03-22T04:16:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c37e00640 1 -- 172.21.15.86:0/1090553953 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0a58f0 con 0x7f4c3815b3f0 2024-03-22T04:16:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c37e00640 1 -- 172.21.15.86:0/1090553953 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0a5cc0 con 0x7f4c3815b3f0 2024-03-22T04:16:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 -- 172.21.15.86:0/1090553953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c3815b3f0 msgr2=0x7f4c380729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 --2- 172.21.15.86:0/1090553953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c3815b3f0 0x7f4c380729c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009e50 tx=0x7f4c2c01c9a0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 -- 172.21.15.86:0/1090553953 shutdown_connections 2024-03-22T04:16:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 --2- 172.21.15.86:0/1090553953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c380cc3e0 0x7f4c38072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 --2- 172.21.15.86:0/1090553953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c3815b3f0 0x7f4c380729c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 -- 172.21.15.86:0/1090553953 >> 172.21.15.86:0/1090553953 conn(0x7f4c380b1df0 msgr2=0x7f4c380b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 -- 172.21.15.86:0/1090553953 shutdown_connections 2024-03-22T04:16:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 -- 172.21.15.86:0/1090553953 wait complete. 2024-03-22T04:16:55.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 Processor -- start 2024-03-22T04:16:55.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.826+0000 7f4c40400640 1 -- start start 2024-03-22T04:16:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c380b30f0 0x7f4c38099b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c380cc3e0 0x7f4c38153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c3815b3f0 0x7f4c38159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38168580 con 0x7f4c380cc3e0 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38168400 con 0x7f4c380b30f0 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4c38168700 con 0x7f4c3815b3f0 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c380cc3e0 0x7f4c38153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c380cc3e0 0x7f4c38153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58100/0 (socket says 172.21.15.86:58100) 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 -- 172.21.15.86:0/2587208710 learned_addr learned my addr 172.21.15.86:0/2587208710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3e400640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c3815b3f0 0x7f4c38159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3da00640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c380b30f0 0x7f4c38099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 -- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c3815b3f0 msgr2=0x7f4c38159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c3815b3f0 0x7f4c38159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 -- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c380b30f0 msgr2=0x7f4c38099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c380b30f0 0x7f4c38099b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 -- 172.21.15.86:0/2587208710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381545b0 con 0x7f4c380cc3e0 2024-03-22T04:16:55.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3e400640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c3815b3f0 0x7f4c38159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:55.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3da00640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c380b30f0 0x7f4c38099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:55.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c3d000640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c380cc3e0 0x7f4c38153d60 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400d7d0 tx=0x7f4c2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c35600640 1 -- 172.21.15.86:0/2587208710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400b6c0 con 0x7f4c380cc3e0 2024-03-22T04:16:55.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c35600640 1 -- 172.21.15.86:0/2587208710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c24019070 con 0x7f4c380cc3e0 2024-03-22T04:16:55.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c38154890 con 0x7f4c380cc3e0 2024-03-22T04:16:55.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c35600640 1 -- 172.21.15.86:0/2587208710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400ba80 con 0x7f4c380cc3e0 2024-03-22T04:16:55.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3814ed10 con 0x7f4c380cc3e0 2024-03-22T04:16:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c38068650 con 0x7f4c380cc3e0 2024-03-22T04:16:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.830+0000 7f4c35600640 1 -- 172.21.15.86:0/2587208710 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4c2400bc20 con 0x7f4c380cc3e0 2024-03-22T04:16:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.834+0000 7f4c35600640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4c18067b20 0x7f4c18069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.834+0000 7f4c35600640 1 -- 172.21.15.86:0/2587208710 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f4c24086210 con 0x7f4c380cc3e0 2024-03-22T04:16:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.834+0000 7f4c3da00640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4c18067b20 0x7f4c18069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:55.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.834+0000 7f4c3da00640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4c18067b20 0x7f4c18069fe0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0ae000 tx=0x7f4c2c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:55.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.838+0000 7f4c35600640 1 -- 172.21.15.86:0/2587208710 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4c24056d40 con 0x7f4c380cc3e0 2024-03-22T04:16:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.994+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4c38072ef0 con 0x7f4c380cc3e0 2024-03-22T04:16:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.994+0000 7f4c35600640 1 -- 172.21.15.86:0/2587208710 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) ==== 84+0+65 (secure 0 0 0) 0x7f4c24056640 con 0x7f4c380cc3e0 2024-03-22T04:16:55.996 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:55.996 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:16:55.996 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:16:55.996 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T04:16:55.996 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T04:16:55.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:16:55.996 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:16:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4c18067b20 msgr2=0x7f4c18069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4c18067b20 0x7f4c18069fe0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0ae000 tx=0x7f4c2c0a7070 comp rx=0 tx=0).stop 2024-03-22T04:16:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c380cc3e0 msgr2=0x7f4c38153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c380cc3e0 0x7f4c38153d60 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400d7d0 tx=0x7f4c2400dc90 comp rx=0 tx=0).stop 2024-03-22T04:16:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 shutdown_connections 2024-03-22T04:16:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4c18067b20 0x7f4c18069fe0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c3815b3f0 0x7f4c38159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c380cc3e0 0x7f4c38153d60 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 --2- 172.21.15.86:0/2587208710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c380b30f0 0x7f4c38099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 >> 172.21.15.86:0/2587208710 conn(0x7f4c380b1df0 msgr2=0x7f4c380b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 shutdown_connections 2024-03-22T04:16:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:55.998+0000 7f4c40400640 1 -- 172.21.15.86:0/2587208710 wait complete. 2024-03-22T04:16:56.018 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T04:16:56.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T04:16:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.158+0000 7f6238000640 1 Processor -- start 2024-03-22T04:16:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.158+0000 7f6238000640 1 -- start start 2024-03-22T04:16:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6238000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62300b1cd0 0x7f62300b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6238000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300cfa70 0x7f623006d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6238000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62300b25f0 con 0x7f623006dca0 2024-03-22T04:16:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6238000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f62300cea80 con 0x7f62300b1cd0 2024-03-22T04:16:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6238000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f623006e4d0 con 0x7f62300cfa70 2024-03-22T04:16:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6235600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62300b1cd0 0x7f62300b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6236000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f623006dca0 0x7f6230141960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39364/0 (socket says 172.21.15.86:39364) 2024-03-22T04:16:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6236000640 1 -- 172.21.15.86:0/3426135223 learned_addr learned my addr 172.21.15.86:0/3426135223 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:16:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6234c00640 1 --2- 172.21.15.86:0/3426135223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300cfa70 0x7f623006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6235600640 1 -- 172.21.15.86:0/3426135223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300cfa70 msgr2=0x7f623006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6235600640 1 --2- 172.21.15.86:0/3426135223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300cfa70 0x7f623006d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6235600640 1 -- 172.21.15.86:0/3426135223 >> v1:172.21.15.86:6789/0 conn(0x7f623006dca0 legacy=0x7f6230141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f622fe00640 1 -- 172.21.15.86:0/3426135223 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461385881 0 0) 0x7f62300b25f0 con 0x7f623006dca0 2024-03-22T04:16:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6235600640 1 -- 172.21.15.86:0/3426135223 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62300cecb0 con 0x7f62300b1cd0 2024-03-22T04:16:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.162+0000 7f6235600640 1 --2- 172.21.15.86:0/3426135223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62300b1cd0 0x7f62300b20b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f62240097c0 tx=0x7f622401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e827e3bf85f194 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f622fe00640 1 -- 172.21.15.86:0/3426135223 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240b2070 con 0x7f62300b1cd0 2024-03-22T04:16:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f622fe00640 1 -- 172.21.15.86:0/3426135223 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62240ad640 con 0x7f62300b1cd0 2024-03-22T04:16:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f622fe00640 1 -- 172.21.15.86:0/3426135223 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240ada10 con 0x7f62300b1cd0 2024-03-22T04:16:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- 172.21.15.86:0/3426135223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62300b1cd0 msgr2=0x7f62300b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 --2- 172.21.15.86:0/3426135223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62300b1cd0 0x7f62300b20b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f62240097c0 tx=0x7f622401c470 comp rx=0 tx=0).stop 2024-03-22T04:16:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- 172.21.15.86:0/3426135223 shutdown_connections 2024-03-22T04:16:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 --2- 172.21.15.86:0/3426135223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300cfa70 0x7f623006d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 --2- 172.21.15.86:0/3426135223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62300b1cd0 0x7f62300b20b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- 172.21.15.86:0/3426135223 >> 172.21.15.86:0/3426135223 conn(0x7f623006b660 msgr2=0x7f623006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- 172.21.15.86:0/3426135223 shutdown_connections 2024-03-22T04:16:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- 172.21.15.86:0/3426135223 wait complete. 2024-03-22T04:16:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 Processor -- start 2024-03-22T04:16:56.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- start start 2024-03-22T04:16:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623006dca0 0x7f62301301b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300b1cd0 0x7f6230145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62300cfa70 0x7f6230146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f623006e270 con 0x7f623006dca0 2024-03-22T04:16:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f623006e0f0 con 0x7f62300cfa70 2024-03-22T04:16:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6238000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f623013f460 con 0x7f62300b1cd0 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6235600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623006dca0 0x7f62301301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6235600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623006dca0 0x7f62301301b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49752/0 (socket says 172.21.15.86:49752) 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.166+0000 7f6235600640 1 -- 172.21.15.86:0/3323137263 learned_addr learned my addr 172.21.15.86:0/3323137263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6234c00640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300b1cd0 0x7f6230145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6236000640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62300cfa70 0x7f6230146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6235600640 1 -- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300b1cd0 msgr2=0x7f6230145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6235600640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300b1cd0 0x7f6230145f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6235600640 1 -- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62300cfa70 msgr2=0x7f6230146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6235600640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62300cfa70 0x7f6230146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6235600640 1 -- 172.21.15.86:0/3323137263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62301389b0 con 0x7f623006dca0 2024-03-22T04:16:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6234c00640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300b1cd0 0x7f6230145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6236000640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62300cfa70 0x7f6230146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6235600640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623006dca0 0x7f62301301b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f62240b0040 tx=0x7f6224019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f622d600640 1 -- 172.21.15.86:0/3323137263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240b2040 con 0x7f623006dca0 2024-03-22T04:16:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f622d600640 1 -- 172.21.15.86:0/3323137263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62240bc070 con 0x7f623006dca0 2024-03-22T04:16:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6230138c90 con 0x7f623006dca0 2024-03-22T04:16:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f622d600640 1 -- 172.21.15.86:0/3323137263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240b6650 con 0x7f623006dca0 2024-03-22T04:16:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6230139250 con 0x7f623006dca0 2024-03-22T04:16:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6230074580 con 0x7f623006dca0 2024-03-22T04:16:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f622d600640 1 -- 172.21.15.86:0/3323137263 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f62240b67f0 con 0x7f623006dca0 2024-03-22T04:16:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f622d600640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6210067b20 0x7f6210069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f622d600640 1 -- 172.21.15.86:0/3323137263 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f622411fd80 con 0x7f623006dca0 2024-03-22T04:16:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.170+0000 7f6234c00640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6210067b20 0x7f6210069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.174+0000 7f6234c00640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6210067b20 0x7f6210069fe0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6230135ac0 tx=0x7f621c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.174+0000 7f622d600640 1 -- 172.21.15.86:0/3323137263 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6224016090 con 0x7f623006dca0 2024-03-22T04:16:56.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.330+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f62301417d0 con 0x7f623006dca0 2024-03-22T04:16:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f622d600640 1 -- 172.21.15.86:0/3323137263 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) ==== 138+0+0 (secure 0 0 0) 0x7f62240f09f0 con 0x7f623006dca0 2024-03-22T04:16:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T04:16:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6210067b20 msgr2=0x7f6210069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6210067b20 0x7f6210069fe0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6230135ac0 tx=0x7f621c008040 comp rx=0 tx=0).stop 2024-03-22T04:16:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623006dca0 msgr2=0x7f62301301b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623006dca0 0x7f62301301b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f62240b0040 tx=0x7f6224019210 comp rx=0 tx=0).stop 2024-03-22T04:16:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 shutdown_connections 2024-03-22T04:16:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6210067b20 0x7f6210069fe0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62300cfa70 0x7f6230146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62300b1cd0 0x7f6230145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 --2- 172.21.15.86:0/3323137263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623006dca0 0x7f62301301b0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 >> 172.21.15.86:0/3323137263 conn(0x7f623006b660 msgr2=0x7f62301560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 shutdown_connections 2024-03-22T04:16:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.622+0000 7f6238000640 1 -- 172.21.15.86:0/3323137263 wait complete. 2024-03-22T04:16:56.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T04:16:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fdce00640 1 Processor -- start 2024-03-22T04:16:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fdce00640 1 -- start start 2024-03-22T04:16:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fdce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd815c850 0x7f4fd80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fdce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4fd80cf6b0 0x7f4fd806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fdce00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd815fa40 con 0x7f4fd80b5f80 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fdce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd806ed50 con 0x7f4fd80cf6b0 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fdce00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd815fd20 con 0x7f4fd815c850 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4fd80cf6b0 0x7f4fd806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4fd80cf6b0 0x7f4fd806f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49764/0 (socket says 172.21.15.86:49764) 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd6800640 1 -- 172.21.15.86:0/451248633 learned_addr learned my addr 172.21.15.86:0/451248633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fcd400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4fd80b5f80 0x7f4fd80cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39370/0 (socket says 172.21.15.86:39370) 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd5e00640 1 --2- 172.21.15.86:0/451248633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd815c850 0x7f4fd80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd6800640 1 -- 172.21.15.86:0/451248633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd815c850 msgr2=0x7f4fd80b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd6800640 1 --2- 172.21.15.86:0/451248633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd815c850 0x7f4fd80b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd6800640 1 -- 172.21.15.86:0/451248633 >> v1:172.21.15.86:6789/0 conn(0x7f4fd80b5f80 legacy=0x7f4fd80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd6800640 1 -- 172.21.15.86:0/451248633 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd806e700 con 0x7f4fd80cf6b0 2024-03-22T04:16:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd5400640 1 -- 172.21.15.86:0/451248633 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403441558 0 0) 0x7f4fd815fa40 con 0x7f4fd80b5f80 2024-03-22T04:16:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.794+0000 7f4fd5e00640 1 --2- 172.21.15.86:0/451248633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd815c850 0x7f4fd80b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fd6800640 1 --2- 172.21.15.86:0/451248633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd80cf6b0 0x7f4fd806f1e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009e50 tx=0x7f4fd001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6a45d7f8781403 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fd5400640 1 -- 172.21.15.86:0/451248633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00a5850 con 0x7f4fd80cf6b0 2024-03-22T04:16:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fd5400640 1 -- 172.21.15.86:0/451248633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd00a59f0 con 0x7f4fd80cf6b0 2024-03-22T04:16:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fd5400640 1 -- 172.21.15.86:0/451248633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00a5dc0 con 0x7f4fd80cf6b0 2024-03-22T04:16:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 -- 172.21.15.86:0/451248633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd80cf6b0 msgr2=0x7f4fd806f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 --2- 172.21.15.86:0/451248633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd80cf6b0 0x7f4fd806f1e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009e50 tx=0x7f4fd001c970 comp rx=0 tx=0).stop 2024-03-22T04:16:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 -- 172.21.15.86:0/451248633 shutdown_connections 2024-03-22T04:16:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 --2- 172.21.15.86:0/451248633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd80cf6b0 0x7f4fd806f1e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 --2- 172.21.15.86:0/451248633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd815c850 0x7f4fd80b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 -- 172.21.15.86:0/451248633 >> 172.21.15.86:0/451248633 conn(0x7f4fd80b2e50 msgr2=0x7f4fd80b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 -- 172.21.15.86:0/451248633 shutdown_connections 2024-03-22T04:16:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 -- 172.21.15.86:0/451248633 wait complete. 2024-03-22T04:16:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 Processor -- start 2024-03-22T04:16:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.798+0000 7f4fdce00640 1 -- start start 2024-03-22T04:16:56.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd80b5f80 0x7f4fd8146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd80cf6b0 0x7f4fd8145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd815c850 0x7f4fd8147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8073280 con 0x7f4fd815c850 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8073100 con 0x7f4fd80b5f80 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd81471d0 con 0x7f4fd80cf6b0 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd815c850 0x7f4fd8147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd815c850 0x7f4fd8147800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49774/0 (socket says 172.21.15.86:49774) 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 -- 172.21.15.86:0/465623117 learned_addr learned my addr 172.21.15.86:0/465623117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 -- 172.21.15.86:0/465623117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd80cf6b0 msgr2=0x7f4fd8145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd80cf6b0 0x7f4fd8145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 -- 172.21.15.86:0/465623117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd80b5f80 msgr2=0x7f4fd8146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd80b5f80 0x7f4fd8146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 -- 172.21.15.86:0/465623117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd81451a0 con 0x7f4fd815c850 2024-03-22T04:16:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fd6800640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd815c850 0x7f4fd8147800 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00af040 tx=0x7f4fd0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fcea00640 1 -- 172.21.15.86:0/465623117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00bc030 con 0x7f4fd815c850 2024-03-22T04:16:56.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fcea00640 1 -- 172.21.15.86:0/465623117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd00b8680 con 0x7f4fd815c850 2024-03-22T04:16:56.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fcea00640 1 -- 172.21.15.86:0/465623117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00b8a50 con 0x7f4fd815c850 2024-03-22T04:16:56.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd8145480 con 0x7f4fd815c850 2024-03-22T04:16:56.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd8145a40 con 0x7f4fd815c850 2024-03-22T04:16:56.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.802+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd8074470 con 0x7f4fd815c850 2024-03-22T04:16:56.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.806+0000 7f4fcea00640 1 -- 172.21.15.86:0/465623117 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4fd00bf070 con 0x7f4fd815c850 2024-03-22T04:16:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.806+0000 7f4fcea00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fb4067b20 0x7f4fb4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.806+0000 7f4fcea00640 1 -- 172.21.15.86:0/465623117 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5619+0+0 (secure 0 0 0) 0x7f4fd0120f50 con 0x7f4fd815c850 2024-03-22T04:16:56.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.806+0000 7f4fd5e00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fb4067b20 0x7f4fb4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.806+0000 7f4fd5e00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fb4067b20 0x7f4fb4069fe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800a8d0 tx=0x7f4fc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.810+0000 7f4fcea00640 1 -- 172.21.15.86:0/465623117 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4fd00c6070 con 0x7f4fd815c850 2024-03-22T04:16:56.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:56.966+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4fd8073930 con 0x7f4fd815c850 2024-03-22T04:16:57.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fcea00640 1 -- 172.21.15.86:0/465623117 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) ==== 170+0+0 (secure 0 0 0) 0x7f4fd0016620 con 0x7f4fd815c850 2024-03-22T04:16:57.626 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T04:16:57.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fb4067b20 msgr2=0x7f4fb4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:57.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fb4067b20 0x7f4fb4069fe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800a8d0 tx=0x7f4fc8008040 comp rx=0 tx=0).stop 2024-03-22T04:16:57.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd815c850 msgr2=0x7f4fd8147800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:57.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd815c850 0x7f4fd8147800 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00af040 tx=0x7f4fd0019710 comp rx=0 tx=0).stop 2024-03-22T04:16:57.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 shutdown_connections 2024-03-22T04:16:57.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fb4067b20 0x7f4fb4069fe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd815c850 0x7f4fd8147800 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fd80cf6b0 0x7f4fd8145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 --2- 172.21.15.86:0/465623117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd80b5f80 0x7f4fd8146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 >> 172.21.15.86:0/465623117 conn(0x7f4fd80b2e50 msgr2=0x7f4fd80cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:57.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.626+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 shutdown_connections 2024-03-22T04:16:57.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.630+0000 7f4fdce00640 1 -- 172.21.15.86:0/465623117 wait complete. 2024-03-22T04:16:57.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T04:16:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.802+0000 7f8df5400640 1 Processor -- start 2024-03-22T04:16:57.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.802+0000 7f8df5400640 1 -- start start 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.802+0000 7f8df5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df006e7e0 0x7f8df006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.802+0000 7f8df5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8df0072ad0 0x7f8df00b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.802+0000 7f8df5400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df015c770 con 0x7f8df0072650 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.802+0000 7f8df5400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8df0071350 con 0x7f8df0072ad0 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.802+0000 7f8df5400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8df006f100 con 0x7f8df006e7e0 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.802+0000 7f8def200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8df0072ad0 0x7f8df00b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8def200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8df0072ad0 0x7f8df00b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49780/0 (socket says 172.21.15.86:49780) 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8def200640 1 -- 172.21.15.86:0/3879161222 learned_addr learned my addr 172.21.15.86:0/3879161222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8de5e00640 1 --1- 172.21.15.86:0/3879161222 >> v1:172.21.15.86:6789/0 conn(0x7f8df0072650 0x7f8df00b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39380/0 (socket says 172.21.15.86:39380) 2024-03-22T04:16:57.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8dee800640 1 --2- 172.21.15.86:0/3879161222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df006e7e0 0x7f8df006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:57.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8def200640 1 -- 172.21.15.86:0/3879161222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df006e7e0 msgr2=0x7f8df006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:57.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8def200640 1 --2- 172.21.15.86:0/3879161222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df006e7e0 0x7f8df006ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8def200640 1 -- 172.21.15.86:0/3879161222 >> v1:172.21.15.86:6789/0 conn(0x7f8df0072650 legacy=0x7f8df00b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:57.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8def200640 1 -- 172.21.15.86:0/3879161222 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df0073150 con 0x7f8df0072ad0 2024-03-22T04:16:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8def200640 1 --2- 172.21.15.86:0/3879161222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df0072ad0 0x7f8df00b2290 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009870 tx=0x7f8de801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb6181caefb894 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8dede00640 1 -- 172.21.15.86:0/3879161222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80b0070 con 0x7f8df0072ad0 2024-03-22T04:16:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8dede00640 1 -- 172.21.15.86:0/3879161222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de80ab600 con 0x7f8df0072ad0 2024-03-22T04:16:57.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.806+0000 7f8dede00640 1 -- 172.21.15.86:0/3879161222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80ab9d0 con 0x7f8df0072ad0 2024-03-22T04:16:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- 172.21.15.86:0/3879161222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df0072ad0 msgr2=0x7f8df00b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 --2- 172.21.15.86:0/3879161222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df0072ad0 0x7f8df00b2290 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009870 tx=0x7f8de801c360 comp rx=0 tx=0).stop 2024-03-22T04:16:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- 172.21.15.86:0/3879161222 shutdown_connections 2024-03-22T04:16:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 --2- 172.21.15.86:0/3879161222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df0072ad0 0x7f8df00b2290 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 --2- 172.21.15.86:0/3879161222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df006e7e0 0x7f8df006ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- 172.21.15.86:0/3879161222 >> 172.21.15.86:0/3879161222 conn(0x7f8df00cc940 msgr2=0x7f8df00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- 172.21.15.86:0/3879161222 shutdown_connections 2024-03-22T04:16:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- 172.21.15.86:0/3879161222 wait complete. 2024-03-22T04:16:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 Processor -- start 2024-03-22T04:16:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- start start 2024-03-22T04:16:57.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df006e7e0 0x7f8df015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:57.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df0072650 0x7f8df014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:57.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df0072ad0 0x7f8df01613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:57.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0072080 con 0x7f8df006e7e0 2024-03-22T04:16:57.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8df00ce930 con 0x7f8df0072650 2024-03-22T04:16:57.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8df5400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8df015ebc0 con 0x7f8df0072ad0 2024-03-22T04:16:57.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df006e7e0 0x7f8df015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:57.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df006e7e0 0x7f8df015fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49792/0 (socket says 172.21.15.86:49792) 2024-03-22T04:16:57.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 -- 172.21.15.86:0/3664791572 learned_addr learned my addr 172.21.15.86:0/3664791572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 -- 172.21.15.86:0/3664791572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df0072ad0 msgr2=0x7f8df01613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df0072ad0 0x7f8df01613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 -- 172.21.15.86:0/3664791572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df0072650 msgr2=0x7f8df014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df0072650 0x7f8df014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 -- 172.21.15.86:0/3664791572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df015eea0 con 0x7f8df006e7e0 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.810+0000 7f8dee800640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df006e7e0 0x7f8df015fc60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8de0007e40 tx=0x7f8de000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8de7400640 1 -- 172.21.15.86:0/3664791572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0014620 con 0x7f8df006e7e0 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8de7400640 1 -- 172.21.15.86:0/3664791572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de0019070 con 0x7f8df006e7e0 2024-03-22T04:16:57.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df015f180 con 0x7f8df006e7e0 2024-03-22T04:16:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8de7400640 1 -- 172.21.15.86:0/3664791572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0014a00 con 0x7f8df006e7e0 2024-03-22T04:16:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df015f740 con 0x7f8df006e7e0 2024-03-22T04:16:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df0074470 con 0x7f8df006e7e0 2024-03-22T04:16:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8de7400640 1 -- 172.21.15.86:0/3664791572 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8de0014ba0 con 0x7f8df006e7e0 2024-03-22T04:16:57.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8de7400640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dcc067b20 0x7f8dcc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:57.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8de7400640 1 -- 172.21.15.86:0/3664791572 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6001+0+0 (secure 0 0 0) 0x7f8de0087700 con 0x7f8df006e7e0 2024-03-22T04:16:57.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.814+0000 7f8de5e00640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dcc067b20 0x7f8dcc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:57.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.818+0000 7f8de5e00640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dcc067b20 0x7f8dcc069fe0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80018d0 tx=0x7f8dd80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:57.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:57.822+0000 7f8de7400640 1 -- 172.21.15.86:0/3664791572 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8de00580c0 con 0x7f8df006e7e0 2024-03-22T04:16:57.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-03-22T04:16:58.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.018+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8df006df60 con 0x7f8df006e7e0 2024-03-22T04:16:58.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:16:58.250 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:16:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.654+0000 7f8de7400640 1 -- 172.21.15.86:0/3664791572 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v60) ==== 174+0+0 (secure 0 0 0) 0x7f8de00579c0 con 0x7f8df006e7e0 2024-03-22T04:16:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 13 and data pool 14 2024-03-22T04:16:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dcc067b20 msgr2=0x7f8dcc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dcc067b20 0x7f8dcc069fe0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80018d0 tx=0x7f8dd80079b0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df006e7e0 msgr2=0x7f8df015fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df006e7e0 0x7f8df015fc60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8de0007e40 tx=0x7f8de000e4b0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 shutdown_connections 2024-03-22T04:16:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8dcc067b20 0x7f8dcc069fe0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df0072650 0x7f8df014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df0072ad0 0x7f8df01613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 --2- 172.21.15.86:0/3664791572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df006e7e0 0x7f8df015fc60 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 >> 172.21.15.86:0/3664791572 conn(0x7f8df00cc940 msgr2=0x7f8df0157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 shutdown_connections 2024-03-22T04:16:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.658+0000 7f8df5400640 1 -- 172.21.15.86:0/3664791572 wait complete. 2024-03-22T04:16:58.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:16:58.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbe5a00640 1 Processor -- start 2024-03-22T04:16:58.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbe5a00640 1 -- start start 2024-03-22T04:16:58.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbe5a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe0155a00 0x7ffbe00cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:58.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbe5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffbe00d1080 0x7ffbe00d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:58.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbe5a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe015c6f0 con 0x7ffbe0072ee0 2024-03-22T04:16:58.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbe5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe00d1a10 con 0x7ffbe00d1080 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbe5a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe0073480 con 0x7ffbe0155a00 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffbe00d1080 0x7ffbe00d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffbe00d1080 0x7ffbe00d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49800/0 (socket says 172.21.15.86:49800) 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbde000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ffbe0072ee0 0x7ffbe015bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39394/0 (socket says 172.21.15.86:39394) 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdf400640 1 -- 172.21.15.86:0/409995080 learned_addr learned my addr 172.21.15.86:0/409995080 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdea00640 1 --2- 172.21.15.86:0/409995080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe0155a00 0x7ffbe00cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdd600640 1 -- 172.21.15.86:0/409995080 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 732440955 0 0) 0x7ffbe015c6f0 con 0x7ffbe0072ee0 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdd600640 1 -- 172.21.15.86:0/409995080 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbc4005740 con 0x7ffbe0072ee0 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdf400640 1 -- 172.21.15.86:0/409995080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe0155a00 msgr2=0x7ffbe00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdf400640 1 --2- 172.21.15.86:0/409995080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe0155a00 0x7ffbe00cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdf400640 1 -- 172.21.15.86:0/409995080 >> v1:172.21.15.86:6789/0 conn(0x7ffbe0072ee0 legacy=0x7ffbe015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.826+0000 7ffbdf400640 1 -- 172.21.15.86:0/409995080 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe0073760 con 0x7ffbe00d1080 2024-03-22T04:16:58.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbdf400640 1 --2- 172.21.15.86:0/409995080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe00d1080 0x7ffbe00d14d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009870 tx=0x7ffbd801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5d040b55f5d011 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:58.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbdd600640 1 -- 172.21.15.86:0/409995080 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80ab650 con 0x7ffbe00d1080 2024-03-22T04:16:58.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbdd600640 1 -- 172.21.15.86:0/409995080 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd80b0070 con 0x7ffbe00d1080 2024-03-22T04:16:58.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbdd600640 1 -- 172.21.15.86:0/409995080 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80aba10 con 0x7ffbe00d1080 2024-03-22T04:16:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbe5a00640 1 -- 172.21.15.86:0/409995080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe00d1080 msgr2=0x7ffbe00d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/409995080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe00d1080 0x7ffbe00d14d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009870 tx=0x7ffbd801c560 comp rx=0 tx=0).stop 2024-03-22T04:16:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbe5a00640 1 -- 172.21.15.86:0/409995080 shutdown_connections 2024-03-22T04:16:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/409995080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe00d1080 0x7ffbe00d14d0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/409995080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe0155a00 0x7ffbe00cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbe5a00640 1 -- 172.21.15.86:0/409995080 >> 172.21.15.86:0/409995080 conn(0x7ffbe006c770 msgr2=0x7ffbe006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbe5a00640 1 -- 172.21.15.86:0/409995080 shutdown_connections 2024-03-22T04:16:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.830+0000 7ffbe5a00640 1 -- 172.21.15.86:0/409995080 wait complete. 2024-03-22T04:16:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 Processor -- start 2024-03-22T04:16:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 -- start start 2024-03-22T04:16:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbe0072ee0 0x7ffbe013b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe00d1080 0x7ffbe00ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe0155a00 0x7ffbe013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe00cf860 con 0x7ffbe0155a00 2024-03-22T04:16:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe00cf6e0 con 0x7ffbe0072ee0 2024-03-22T04:16:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe006b660 con 0x7ffbe00d1080 2024-03-22T04:16:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe0155a00 0x7ffbe013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe0155a00 0x7ffbe013afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49810/0 (socket says 172.21.15.86:49810) 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 -- 172.21.15.86:0/125943130 learned_addr learned my addr 172.21.15.86:0/125943130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbde000640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe00d1080 0x7ffbe00ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 -- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe00d1080 msgr2=0x7ffbe00ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe00d1080 0x7ffbe00ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdea00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbe0072ee0 0x7ffbe013b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 -- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbe0072ee0 msgr2=0x7ffbe013b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbe0072ee0 0x7ffbe013b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 -- 172.21.15.86:0/125943130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe00ae760 con 0x7ffbe0155a00 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbde000640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe00d1080 0x7ffbe00ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdea00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbe0072ee0 0x7ffbe013b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbdf400640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe0155a00 0x7ffbe013afa0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80ad040 tx=0x7ffbd801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbd6a00640 1 -- 172.21.15.86:0/125943130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80be3f0 con 0x7ffbe0155a00 2024-03-22T04:16:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbd6a00640 1 -- 172.21.15.86:0/125943130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd80be590 con 0x7ffbe0155a00 2024-03-22T04:16:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbd6a00640 1 -- 172.21.15.86:0/125943130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80be980 con 0x7ffbe0155a00 2024-03-22T04:16:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe00aea00 con 0x7ffbe0155a00 2024-03-22T04:16:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe00aefc0 con 0x7ffbe0155a00 2024-03-22T04:16:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.834+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe0068650 con 0x7ffbe0155a00 2024-03-22T04:16:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.838+0000 7ffbd6a00640 1 -- 172.21.15.86:0/125943130 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ffbd80b43f0 con 0x7ffbe0155a00 2024-03-22T04:16:58.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.838+0000 7ffbd6a00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffbc0067b50 0x7ffbc006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:58.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.838+0000 7ffbd6a00640 1 -- 172.21.15.86:0/125943130 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7ffbd811fdf0 con 0x7ffbe0155a00 2024-03-22T04:16:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.838+0000 7ffbdea00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffbc0067b50 0x7ffbc006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.838+0000 7ffbdea00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffbc0067b50 0x7ffbc006a010 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0007440 tx=0x7ffbd000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.842+0000 7ffbd6a00640 1 -- 172.21.15.86:0/125943130 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ffbd80c7020 con 0x7ffbe0155a00 2024-03-22T04:16:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.998+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbe006fd10 con 0x7ffbe0155a00 2024-03-22T04:16:59.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:58.998+0000 7ffbd6a00640 1 -- 172.21.15.86:0/125943130 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16946 (secure 0 0 0) 0x7ffbd8016150 con 0x7ffbe0155a00 2024-03-22T04:16:59.000 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:59.001 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:16:58.020991+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:16:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffbc0067b50 msgr2=0x7ffbc006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffbc0067b50 0x7ffbc006a010 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0007440 tx=0x7ffbd000b040 comp rx=0 tx=0).stop 2024-03-22T04:16:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe0155a00 msgr2=0x7ffbe013afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe0155a00 0x7ffbe013afa0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80ad040 tx=0x7ffbd801ada0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 shutdown_connections 2024-03-22T04:16:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ffbc0067b50 0x7ffbc006a010 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbe0155a00 0x7ffbe013afa0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbe00d1080 0x7ffbe00ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 --2- 172.21.15.86:0/125943130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbe0072ee0 0x7ffbe013b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 >> 172.21.15.86:0/125943130 conn(0x7ffbe006c770 msgr2=0x7ffbe006ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 shutdown_connections 2024-03-22T04:16:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.002+0000 7ffbe5a00640 1 -- 172.21.15.86:0/125943130 wait complete. 2024-03-22T04:16:59.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:16:59.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca85c00640 1 Processor -- start 2024-03-22T04:16:59.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca85c00640 1 -- start start 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca85c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca800700c0 0x7fca80072850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca85c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca800704a0 0x7fca8006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca85c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca8006be40 0x7fca800d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca85c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fca80162740 con 0x7fca800704a0 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca85c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fca80162a20 con 0x7fca800700c0 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca85c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca80162d00 con 0x7fca8006be40 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca76000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca800704a0 0x7fca8006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca76000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca800704a0 0x7fca8006b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49812/0 (socket says 172.21.15.86:49812) 2024-03-22T04:16:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca76000640 1 -- 172.21.15.86:0/1931680037 learned_addr learned my addr 172.21.15.86:0/1931680037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca7f400640 1 --2- 172.21.15.86:0/1931680037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca8006be40 0x7fca800d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca7ea00640 1 --2- 172.21.15.86:0/1931680037 >> v2:172.21.15.196:3300/0 conn(0x7fca800700c0 0x7fca80072850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca76000640 1 -- 172.21.15.86:0/1931680037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca8006be40 msgr2=0x7fca800d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca76000640 1 --2- 172.21.15.86:0/1931680037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca8006be40 0x7fca800d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca76000640 1 -- 172.21.15.86:0/1931680037 >> v2:172.21.15.196:3300/0 conn(0x7fca800700c0 msgr2=0x7fca80072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca76000640 1 --2- 172.21.15.86:0/1931680037 >> v2:172.21.15.196:3300/0 conn(0x7fca800700c0 0x7fca80072850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca76000640 1 -- 172.21.15.86:0/1931680037 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca80162fe0 con 0x7fca800704a0 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca7f400640 1 --2- 172.21.15.86:0/1931680037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca8006be40 0x7fca800d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:59.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.158+0000 7fca7ea00640 1 --2- 172.21.15.86:0/1931680037 >> v2:172.21.15.196:3300/0 conn(0x7fca800700c0 0x7fca80072850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:16:59.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca76000640 1 --2- 172.21.15.86:0/1931680037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800704a0 0x7fca8006b900 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fca8006c740 tx=0x7fca6801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251419c480ffa9a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:59.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca7e000640 1 -- 172.21.15.86:0/1931680037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680a5670 con 0x7fca800704a0 2024-03-22T04:16:59.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca7e000640 1 -- 172.21.15.86:0/1931680037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca680a5810 con 0x7fca800704a0 2024-03-22T04:16:59.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca7e000640 1 -- 172.21.15.86:0/1931680037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680a5be0 con 0x7fca800704a0 2024-03-22T04:16:59.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 -- 172.21.15.86:0/1931680037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800704a0 msgr2=0x7fca8006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 --2- 172.21.15.86:0/1931680037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800704a0 0x7fca8006b900 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fca8006c740 tx=0x7fca6801c9f0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 -- 172.21.15.86:0/1931680037 shutdown_connections 2024-03-22T04:16:59.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 --2- 172.21.15.86:0/1931680037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca8006be40 0x7fca800d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 --2- 172.21.15.86:0/1931680037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800704a0 0x7fca8006b900 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 --2- 172.21.15.86:0/1931680037 >> v2:172.21.15.196:3300/0 conn(0x7fca800700c0 0x7fca80072850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 -- 172.21.15.86:0/1931680037 >> 172.21.15.86:0/1931680037 conn(0x7fca800b2d00 msgr2=0x7fca800b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 -- 172.21.15.86:0/1931680037 shutdown_connections 2024-03-22T04:16:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.162+0000 7fca85c00640 1 -- 172.21.15.86:0/1931680037 wait complete. 2024-03-22T04:16:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 Processor -- start 2024-03-22T04:16:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 -- start start 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8006be40 0x7fca800b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800700c0 0x7fca8015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca800704a0 0x7fca8015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca800cffb0 con 0x7fca800700c0 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca800cfe30 con 0x7fca8006be40 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca80162500 con 0x7fca800704a0 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800700c0 0x7fca8015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800700c0 0x7fca8015c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49826/0 (socket says 172.21.15.86:49826) 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 -- 172.21.15.86:0/3056814471 learned_addr learned my addr 172.21.15.86:0/3056814471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca7ea00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8006be40 0x7fca800b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca7f400640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca800704a0 0x7fca8015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 -- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca800704a0 msgr2=0x7fca8015cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca800704a0 0x7fca8015cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 -- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8006be40 msgr2=0x7fca800b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8006be40 0x7fca800b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 -- 172.21.15.86:0/3056814471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca800b0ef0 con 0x7fca800700c0 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca7f400640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca800704a0 0x7fca8015cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca76000640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800700c0 0x7fca8015c650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fca68009040 tx=0x7fca68019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca77400640 1 -- 172.21.15.86:0/3056814471 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680bc070 con 0x7fca800700c0 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca77400640 1 -- 172.21.15.86:0/3056814471 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca680b8640 con 0x7fca800700c0 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca800b11d0 con 0x7fca800700c0 2024-03-22T04:16:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca77400640 1 -- 172.21.15.86:0/3056814471 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680b8a30 con 0x7fca800700c0 2024-03-22T04:16:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca800b1790 con 0x7fca800700c0 2024-03-22T04:16:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.166+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca80068650 con 0x7fca800700c0 2024-03-22T04:16:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.170+0000 7fca77400640 1 -- 172.21.15.86:0/3056814471 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fca680b8bd0 con 0x7fca800700c0 2024-03-22T04:16:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.170+0000 7fca77400640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca5c067b20 0x7fca5c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.170+0000 7fca77400640 1 -- 172.21.15.86:0/3056814471 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fca68120aa0 con 0x7fca800700c0 2024-03-22T04:16:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.170+0000 7fca7ea00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca5c067b20 0x7fca5c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.170+0000 7fca7ea00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca5c067b20 0x7fca5c069fe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fca7000ab60 tx=0x7fca70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.174+0000 7fca77400640 1 -- 172.21.15.86:0/3056814471 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fca680165e0 con 0x7fca800700c0 2024-03-22T04:16:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca80069b00 con 0x7fca800700c0 2024-03-22T04:16:59.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca77400640 1 -- 172.21.15.86:0/3056814471 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16946 (secure 0 0 0) 0x7fca680f0cc0 con 0x7fca800700c0 2024-03-22T04:16:59.330 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:16:59.330 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:16:58.020991+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:16:59.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca5c067b20 msgr2=0x7fca5c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca5c067b20 0x7fca5c069fe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fca7000ab60 tx=0x7fca70008040 comp rx=0 tx=0).stop 2024-03-22T04:16:59.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800700c0 msgr2=0x7fca8015c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800700c0 0x7fca8015c650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fca68009040 tx=0x7fca68019790 comp rx=0 tx=0).stop 2024-03-22T04:16:59.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 shutdown_connections 2024-03-22T04:16:59.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fca5c067b20 0x7fca5c069fe0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca800704a0 0x7fca8015cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca800700c0 0x7fca8015c650 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 --2- 172.21.15.86:0/3056814471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8006be40 0x7fca800b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 >> 172.21.15.86:0/3056814471 conn(0x7fca800b2d00 msgr2=0x7fca80160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:59.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 shutdown_connections 2024-03-22T04:16:59.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.330+0000 7fca85c00640 1 -- 172.21.15.86:0/3056814471 wait complete. 2024-03-22T04:16:59.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T04:16:59.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb8a2c00640 1 Processor -- start 2024-03-22T04:16:59.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb8a2c00640 1 -- start start 2024-03-22T04:16:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb8a2c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb89c072650 0x7fb89c072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb8a2c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb89c15b760 0x7fb89c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb8a2c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb89c073150 con 0x7fb89c06e7e0 2024-03-22T04:16:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb8a2c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb89c071350 con 0x7fb89c072650 2024-03-22T04:16:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb8a2c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb89c0715a0 con 0x7fb89c15b760 2024-03-22T04:16:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb89b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb89c072650 0x7fb89c072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb89b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb89c072650 0x7fb89c072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49840/0 (socket says 172.21.15.86:49840) 2024-03-22T04:16:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb89b400640 1 -- 172.21.15.86:0/1768006010 learned_addr learned my addr 172.21.15.86:0/1768006010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb8a0c00640 1 --2- 172.21.15.86:0/1768006010 >> v2:172.21.15.196:3300/0 conn(0x7fb89c15b760 0x7fb89c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb89b400640 1 -- 172.21.15.86:0/1768006010 >> v2:172.21.15.196:3300/0 conn(0x7fb89c15b760 msgr2=0x7fb89c15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:16:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb89b400640 1 --2- 172.21.15.86:0/1768006010 >> v2:172.21.15.196:3300/0 conn(0x7fb89c15b760 0x7fb89c15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb89b400640 1 -- 172.21.15.86:0/1768006010 >> v1:172.21.15.196:6789/0 conn(0x7fb89c06e7e0 legacy=0x7fb89c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.490+0000 7fb89b400640 1 -- 172.21.15.86:0/1768006010 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c071880 con 0x7fb89c072650 2024-03-22T04:16:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb89b400640 1 --2- 172.21.15.86:0/1768006010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c072650 0x7fb89c072b00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0cef70 tx=0x7fb88c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e93f6bb8e09f48d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb89aa00640 1 -- 172.21.15.86:0/1768006010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0b4070 con 0x7fb89c072650 2024-03-22T04:16:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb89aa00640 1 -- 172.21.15.86:0/1768006010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88c0a7bc0 con 0x7fb89c072650 2024-03-22T04:16:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb89aa00640 1 -- 172.21.15.86:0/1768006010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0af440 con 0x7fb89c072650 2024-03-22T04:16:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb8a2c00640 1 -- 172.21.15.86:0/1768006010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c072650 msgr2=0x7fb89c072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/1768006010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c072650 0x7fb89c072b00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0cef70 tx=0x7fb88c01eb30 comp rx=0 tx=0).stop 2024-03-22T04:16:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb8a2c00640 1 -- 172.21.15.86:0/1768006010 shutdown_connections 2024-03-22T04:16:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/1768006010 >> v2:172.21.15.196:3300/0 conn(0x7fb89c15b760 0x7fb89c15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/1768006010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c072650 0x7fb89c072b00 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb8a2c00640 1 -- 172.21.15.86:0/1768006010 >> 172.21.15.86:0/1768006010 conn(0x7fb89c0cc940 msgr2=0x7fb89c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:16:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb8a2c00640 1 -- 172.21.15.86:0/1768006010 shutdown_connections 2024-03-22T04:16:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.494+0000 7fb8a2c00640 1 -- 172.21.15.86:0/1768006010 wait complete. 2024-03-22T04:16:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 Processor -- start 2024-03-22T04:16:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 -- start start 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb89c06e7e0 0x7fb89c148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c072650 0x7fb89c135880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c15b760 0x7fb89c14b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c155c20 con 0x7fb89c15b760 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c155aa0 con 0x7fb89c072650 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb89c06d6d0 con 0x7fb89c06e7e0 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c15b760 0x7fb89c14b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c15b760 0x7fb89c14b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49844/0 (socket says 172.21.15.86:49844) 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 -- 172.21.15.86:0/682028987 learned_addr learned my addr 172.21.15.86:0/682028987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb89be00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb89c06e7e0 0x7fb89c148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb89b400640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c072650 0x7fb89c135880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 -- 172.21.15.86:0/682028987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb89c06e7e0 msgr2=0x7fb89c148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb89c06e7e0 0x7fb89c148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 -- 172.21.15.86:0/682028987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c072650 msgr2=0x7fb89c135880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c072650 0x7fb89c135880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 -- 172.21.15.86:0/682028987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c14b860 con 0x7fb89c15b760 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a0c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c15b760 0x7fb89c14b320 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb89400efc0 tx=0x7fb89400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb893e00640 1 -- 172.21.15.86:0/682028987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894018020 con 0x7fb89c15b760 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb893e00640 1 -- 172.21.15.86:0/682028987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894014480 con 0x7fb89c15b760 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c14bb00 con 0x7fb89c15b760 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb893e00640 1 -- 172.21.15.86:0/682028987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894014870 con 0x7fb89c15b760 2024-03-22T04:16:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.498+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c14c0c0 con 0x7fb89c15b760 2024-03-22T04:16:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.502+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89c074470 con 0x7fb89c15b760 2024-03-22T04:16:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.502+0000 7fb893e00640 1 -- 172.21.15.86:0/682028987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb89401b020 con 0x7fb89c15b760 2024-03-22T04:16:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.502+0000 7fb893e00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb878067b20 0x7fb878069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:16:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.502+0000 7fb893e00640 1 -- 172.21.15.86:0/682028987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fb894086a20 con 0x7fb89c15b760 2024-03-22T04:16:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.502+0000 7fb89be00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb878067b20 0x7fb878069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:16:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.502+0000 7fb89be00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb878067b20 0x7fb878069fe0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb884001810 tx=0x7fb884009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:16:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.506+0000 7fb893e00640 1 -- 172.21.15.86:0/682028987 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb894022070 con 0x7fb89c15b760 2024-03-22T04:16:59.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:16:59.702+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb89c15bd90 con 0x7fb89c15b760 2024-03-22T04:17:00.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.666+0000 7fb893e00640 1 -- 172.21.15.86:0/682028987 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) ==== 118+0+0 (secure 0 0 0) 0x7fb894010040 con 0x7fb89c15b760 2024-03-22T04:17:00.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb878067b20 msgr2=0x7fb878069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:00.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb878067b20 0x7fb878069fe0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb884001810 tx=0x7fb884009040 comp rx=0 tx=0).stop 2024-03-22T04:17:00.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c15b760 msgr2=0x7fb89c14b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:00.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c15b760 0x7fb89c14b320 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb89400efc0 tx=0x7fb89400c490 comp rx=0 tx=0).stop 2024-03-22T04:17:00.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 shutdown_connections 2024-03-22T04:17:00.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb878067b20 0x7fb878069fe0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c072650 0x7fb89c135880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb89c15b760 0x7fb89c14b320 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 --2- 172.21.15.86:0/682028987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb89c06e7e0 0x7fb89c148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 >> 172.21.15.86:0/682028987 conn(0x7fb89c0cc940 msgr2=0x7fb89c0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:00.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 shutdown_connections 2024-03-22T04:17:00.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.670+0000 7fb8a2c00640 1 -- 172.21.15.86:0/682028987 wait complete. 2024-03-22T04:17:00.693 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-22T04:17:00.693 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-22T04:17:00.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.834+0000 7f0773000640 1 Processor -- start 2024-03-22T04:17:00.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.834+0000 7f0773000640 1 -- start start 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.834+0000 7f0773000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f076c161cf0 0x7f076c0d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.834+0000 7f0773000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f076c0b32b0 0x7f076c0d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.834+0000 7f0773000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f076c0d51a0 con 0x7f076c0b4240 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.834+0000 7f0773000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f076c0d0670 con 0x7f076c161cf0 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.834+0000 7f0773000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f076c0d4c20 con 0x7f076c0b32b0 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f076c161cf0 0x7f076c0d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f076c161cf0 0x7f076c0d0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49854/0 (socket says 172.21.15.86:49854) 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076be00640 1 -- 172.21.15.86:0/3252027976 learned_addr learned my addr 172.21.15.86:0/3252027976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076b400640 1 --2- 172.21.15.86:0/3252027976 >> v2:172.21.15.196:3300/0 conn(0x7f076c0b32b0 0x7f076c0d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076be00640 1 -- 172.21.15.86:0/3252027976 >> v2:172.21.15.196:3300/0 conn(0x7f076c0b32b0 msgr2=0x7f076c0d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076be00640 1 --2- 172.21.15.86:0/3252027976 >> v2:172.21.15.196:3300/0 conn(0x7f076c0b32b0 0x7f076c0d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076be00640 1 -- 172.21.15.86:0/3252027976 >> v1:172.21.15.196:6789/0 conn(0x7f076c0b4240 legacy=0x7f076c0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:00.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076be00640 1 -- 172.21.15.86:0/3252027976 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c0d4ec0 con 0x7f076c161cf0 2024-03-22T04:17:00.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076b400640 1 --2- 172.21.15.86:0/3252027976 >> v2:172.21.15.196:3300/0 conn(0x7f076c0b32b0 0x7f076c0d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:00.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076be00640 1 --2- 172.21.15.86:0/3252027976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c161cf0 0x7f076c0d0cb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f076c0b3260 tx=0x7f075c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac2fa0642506624 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:00.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076aa00640 1 -- 172.21.15.86:0/3252027976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0b5070 con 0x7f076c161cf0 2024-03-22T04:17:00.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076aa00640 1 -- 172.21.15.86:0/3252027976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075c0a8a10 con 0x7f076c161cf0 2024-03-22T04:17:00.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.838+0000 7f076aa00640 1 -- 172.21.15.86:0/3252027976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0a8de0 con 0x7f076c161cf0 2024-03-22T04:17:00.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- 172.21.15.86:0/3252027976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c161cf0 msgr2=0x7f076c0d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:00.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 --2- 172.21.15.86:0/3252027976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c161cf0 0x7f076c0d0cb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f076c0b3260 tx=0x7f075c01ed10 comp rx=0 tx=0).stop 2024-03-22T04:17:00.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- 172.21.15.86:0/3252027976 shutdown_connections 2024-03-22T04:17:00.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 --2- 172.21.15.86:0/3252027976 >> v2:172.21.15.196:3300/0 conn(0x7f076c0b32b0 0x7f076c0d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 --2- 172.21.15.86:0/3252027976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c161cf0 0x7f076c0d0cb0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- 172.21.15.86:0/3252027976 >> 172.21.15.86:0/3252027976 conn(0x7f076c09d630 msgr2=0x7f076c0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:00.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- 172.21.15.86:0/3252027976 shutdown_connections 2024-03-22T04:17:00.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- 172.21.15.86:0/3252027976 wait complete. 2024-03-22T04:17:00.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 Processor -- start 2024-03-22T04:17:00.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- start start 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c0b32b0 0x7f076c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c0b4240 0x7f076c0cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f076c161cf0 0x7f076c0b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f076c162250 con 0x7f076c0b32b0 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f076c1620d0 con 0x7f076c0b4240 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0773000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f076c0cfee0 con 0x7f076c161cf0 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c0b32b0 0x7f076c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c0b32b0 0x7f076c15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49870/0 (socket says 172.21.15.86:49870) 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 -- 172.21.15.86:0/78788098 learned_addr learned my addr 172.21.15.86:0/78788098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076b400640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c0b4240 0x7f076c0cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0771000640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f076c161cf0 0x7f076c0b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 -- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f076c161cf0 msgr2=0x7f076c0b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:00.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f076c161cf0 0x7f076c0b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 -- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c0b4240 msgr2=0x7f076c0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:00.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c0b4240 0x7f076c0cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:00.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 -- 172.21.15.86:0/78788098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c0ce820 con 0x7f076c0b32b0 2024-03-22T04:17:00.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f0771000640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f076c161cf0 0x7f076c0b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:00.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.842+0000 7f076be00640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c0b32b0 0x7f076c15b1f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f075c01b920 tx=0x7f075c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0763e00640 1 -- 172.21.15.86:0/78788098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0b5070 con 0x7f076c0b32b0 2024-03-22T04:17:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0763e00640 1 -- 172.21.15.86:0/78788098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075c0b0b30 con 0x7f076c0b32b0 2024-03-22T04:17:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076c0cdf80 con 0x7f076c0b32b0 2024-03-22T04:17:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0763e00640 1 -- 172.21.15.86:0/78788098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0bb410 con 0x7f076c0b32b0 2024-03-22T04:17:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076c0ce540 con 0x7f076c0b32b0 2024-03-22T04:17:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f076c074470 con 0x7f076c0b32b0 2024-03-22T04:17:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0763e00640 1 -- 172.21.15.86:0/78788098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f075c0c2060 con 0x7f076c0b32b0 2024-03-22T04:17:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0763e00640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f074c067b20 0x7f074c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f0763e00640 1 -- 172.21.15.86:0/78788098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f075c1230c0 con 0x7f076c0b32b0 2024-03-22T04:17:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f076b400640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f074c067b20 0x7f074c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:00.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.846+0000 7f076b400640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f074c067b20 0x7f074c069fe0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f076c160cc0 tx=0x7f0758008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:00.854+0000 7f0763e00640 1 -- 172.21.15.86:0/78788098 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f075c018870 con 0x7f076c0b32b0 2024-03-22T04:17:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.042+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f076c0cfa60 con 0x7f076c0b32b0 2024-03-22T04:17:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0763e00640 1 -- 172.21.15.86:0/78788098 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) ==== 122+0+0 (secure 0 0 0) 0x7f075c0f3940 con 0x7f076c0b32b0 2024-03-22T04:17:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f074c067b20 msgr2=0x7f074c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f074c067b20 0x7f074c069fe0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f076c160cc0 tx=0x7f0758008040 comp rx=0 tx=0).stop 2024-03-22T04:17:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c0b32b0 msgr2=0x7f076c15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c0b32b0 0x7f076c15b1f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f075c01b920 tx=0x7f075c01bab0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 shutdown_connections 2024-03-22T04:17:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f074c067b20 0x7f074c069fe0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c0b4240 0x7f076c0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f076c161cf0 0x7f076c0b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 --2- 172.21.15.86:0/78788098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f076c0b32b0 0x7f076c15b1f0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 >> 172.21.15.86:0/78788098 conn(0x7f076c09d630 msgr2=0x7f076c0b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 shutdown_connections 2024-03-22T04:17:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.670+0000 7f0773000640 1 -- 172.21.15.86:0/78788098 wait complete. 2024-03-22T04:17:01.693 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-22T04:17:01.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:17:01.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.830+0000 7f8b4be00640 1 Processor -- start 2024-03-22T04:17:01.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.830+0000 7f8b4be00640 1 -- start start 2024-03-22T04:17:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b4be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8b4415e630 0x7f8b440cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b440d03b0 0x7f8b4406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b4be00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b440d07d0 con 0x7f8b4406e2f0 2024-03-22T04:17:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b4be00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8b441623d0 con 0x7f8b4415e630 2024-03-22T04:17:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b4be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8b441620f0 con 0x7f8b440d03b0 2024-03-22T04:17:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b440d03b0 0x7f8b4406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b440d03b0 0x7f8b4406d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:56826/0 (socket says 172.21.15.86:56826) 2024-03-22T04:17:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49e00640 1 -- 172.21.15.86:0/1783117098 learned_addr learned my addr 172.21.15.86:0/1783117098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49400640 1 --2- 172.21.15.86:0/1783117098 >> v2:172.21.15.196:3300/0 conn(0x7f8b4415e630 0x7f8b440cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b43e00640 1 -- 172.21.15.86:0/1783117098 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803244982 0 0) 0x7f8b440d07d0 con 0x7f8b4406e2f0 2024-03-22T04:17:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b43e00640 1 -- 172.21.15.86:0/1783117098 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b2c003530 con 0x7f8b4406e2f0 2024-03-22T04:17:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49400640 1 -- 172.21.15.86:0/1783117098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b440d03b0 msgr2=0x7f8b4406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49400640 1 --2- 172.21.15.86:0/1783117098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b440d03b0 0x7f8b4406d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49400640 1 -- 172.21.15.86:0/1783117098 >> v1:172.21.15.196:6789/0 conn(0x7f8b4406e2f0 legacy=0x7f8b4406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49400640 1 -- 172.21.15.86:0/1783117098 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4415ec10 con 0x7f8b4415e630 2024-03-22T04:17:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.834+0000 7f8b49e00640 1 --2- 172.21.15.86:0/1783117098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b440d03b0 0x7f8b4406d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:01.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b49400640 1 --2- 172.21.15.86:0/1783117098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b4415e630 0x7f8b440cfe70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009d90 tx=0x7f8b3801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=929c2572c84f0e2c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:01.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b43e00640 1 -- 172.21.15.86:0/1783117098 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380b2070 con 0x7f8b4415e630 2024-03-22T04:17:01.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b43e00640 1 -- 172.21.15.86:0/1783117098 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b380a5830 con 0x7f8b4415e630 2024-03-22T04:17:01.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b43e00640 1 -- 172.21.15.86:0/1783117098 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380a5c00 con 0x7f8b4415e630 2024-03-22T04:17:01.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 -- 172.21.15.86:0/1783117098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b4415e630 msgr2=0x7f8b440cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:01.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 --2- 172.21.15.86:0/1783117098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b4415e630 0x7f8b440cfe70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009d90 tx=0x7f8b3801c970 comp rx=0 tx=0).stop 2024-03-22T04:17:01.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 -- 172.21.15.86:0/1783117098 shutdown_connections 2024-03-22T04:17:01.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 --2- 172.21.15.86:0/1783117098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b440d03b0 0x7f8b4406d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 --2- 172.21.15.86:0/1783117098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b4415e630 0x7f8b440cfe70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 -- 172.21.15.86:0/1783117098 >> 172.21.15.86:0/1783117098 conn(0x7f8b4406f910 msgr2=0x7f8b4406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:01.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 -- 172.21.15.86:0/1783117098 shutdown_connections 2024-03-22T04:17:01.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 -- 172.21.15.86:0/1783117098 wait complete. 2024-03-22T04:17:01.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 Processor -- start 2024-03-22T04:17:01.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 -- start start 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4406e2f0 0x7f8b44154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.838+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b440d03b0 0x7f8b440cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4415e630 0x7f8b440cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b4be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4415f9d0 con 0x7f8b4415e630 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b4be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b440d1f60 con 0x7f8b440d03b0 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b4be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8b440cfaf0 con 0x7f8b4406e2f0 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4406e2f0 0x7f8b44154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4415e630 0x7f8b440cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4406e2f0 0x7f8b44154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:56836/0 (socket says 172.21.15.86:56836) 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4415e630 0x7f8b440cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49878/0 (socket says 172.21.15.86:49878) 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49400640 1 -- 172.21.15.86:0/3939670410 learned_addr learned my addr 172.21.15.86:0/3939670410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b48a00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b440d03b0 0x7f8b440cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49e00640 1 -- 172.21.15.86:0/3939670410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4406e2f0 msgr2=0x7f8b44154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49e00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4406e2f0 0x7f8b44154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49e00640 1 -- 172.21.15.86:0/3939670410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b440d03b0 msgr2=0x7f8b440cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49e00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b440d03b0 0x7f8b440cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49e00640 1 -- 172.21.15.86:0/3939670410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b44152fe0 con 0x7f8b4415e630 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b48a00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b440d03b0 0x7f8b440cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b49e00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4415e630 0x7f8b440cd210 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e7f0 tx=0x7f8b3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b41600640 1 -- 172.21.15.86:0/3939670410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c017070 con 0x7f8b4415e630 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b41600640 1 -- 172.21.15.86:0/3939670410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3c004020 con 0x7f8b4415e630 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b441532c0 con 0x7f8b4415e630 2024-03-22T04:17:01.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b41600640 1 -- 172.21.15.86:0/3939670410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0043f0 con 0x7f8b4415e630 2024-03-22T04:17:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b44153880 con 0x7f8b4415e630 2024-03-22T04:17:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b44068650 con 0x7f8b4415e630 2024-03-22T04:17:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.842+0000 7f8b41600640 1 -- 172.21.15.86:0/3939670410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8b3c00c940 con 0x7f8b4415e630 2024-03-22T04:17:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.846+0000 7f8b41600640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b24067b20 0x7f8b24069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.846+0000 7f8b41600640 1 -- 172.21.15.86:0/3939670410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f8b3c086050 con 0x7f8b4415e630 2024-03-22T04:17:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.846+0000 7f8b49400640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b24067b20 0x7f8b24069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.846+0000 7f8b49400640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b24067b20 0x7f8b24069fe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009d90 tx=0x7f8b380a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:01.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:01.850+0000 7f8b41600640 1 -- 172.21.15.86:0/3939670410 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8b3c056310 con 0x7f8b4415e630 2024-03-22T04:17:02.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.038+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b44161a00 con 0x7f8b4415e630 2024-03-22T04:17:02.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.038+0000 7f8b41600640 1 -- 172.21.15.86:0/3939670410 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5185 (secure 0 0 0) 0x7f8b3c05a3b0 con 0x7f8b4415e630 2024-03-22T04:17:02.223 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:02.223 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:01.044333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76469":{"gid":76469,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1097103199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1097103199},{"type":"v1","addr":"172.21.15.196:6835","nonce":1097103199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76469,"qdb_cluster":[76469]},"id":6}]} 2024-03-22T04:17:02.224 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 64 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b24067b20 msgr2=0x7f8b24069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b24067b20 0x7f8b24069fe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009d90 tx=0x7f8b380a7070 comp rx=0 tx=0).stop 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4415e630 msgr2=0x7f8b440cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4415e630 0x7f8b440cd210 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e7f0 tx=0x7f8b3c00ecb0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 shutdown_connections 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b24067b20 0x7f8b24069fe0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b440d03b0 0x7f8b440cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b4415e630 0x7f8b440cd210 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 --2- 172.21.15.86:0/3939670410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4406e2f0 0x7f8b44154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 >> 172.21.15.86:0/3939670410 conn(0x7f8b4406f910 msgr2=0x7f8b440b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 shutdown_connections 2024-03-22T04:17:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.042+0000 7f8b4be00640 1 -- 172.21.15.86:0/3939670410 wait complete. 2024-03-22T04:17:02.226 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:17:02.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f4501800640 1 Processor -- start 2024-03-22T04:17:02.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f4501800640 1 -- start start 2024-03-22T04:17:02.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f4501800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc1444f0 0x7f44fc1692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f4501800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0ae5a0 con 0x7f44fc06cd80 2024-03-22T04:17:02.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f4501800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc144b80 con 0x7f44fc0aff50 2024-03-22T04:17:02.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f4501800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc162aa0 con 0x7f44fc1444f0 2024-03-22T04:17:02.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44fa000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f44fc06cd80 0x7f44fc143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39408/0 (socket says 172.21.15.86:39408) 2024-03-22T04:17:02.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44fa000640 1 -- 172.21.15.86:0/2040542684 learned_addr learned my addr 172.21.15.86:0/2040542684 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:02.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44fb400640 1 --2- 172.21.15.86:0/2040542684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc1444f0 0x7f44fc1692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44f9600640 1 -- 172.21.15.86:0/2040542684 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1119466295 0 0) 0x7f44e8001120 con 0x7f44fc06cd80 2024-03-22T04:17:02.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44f9600640 1 -- 172.21.15.86:0/2040542684 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44e0003530 con 0x7f44fc06cd80 2024-03-22T04:17:02.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44f9600640 1 -- 172.21.15.86:0/2040542684 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3402605404 0 0) 0x7f44fc144b80 con 0x7f44fc0aff50 2024-03-22T04:17:02.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44fb400640 1 -- 172.21.15.86:0/2040542684 >> v1:172.21.15.196:6789/0 conn(0x7f44fc0aff50 legacy=0x7f44fc06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44fb400640 1 -- 172.21.15.86:0/2040542684 >> v1:172.21.15.86:6789/0 conn(0x7f44fc06cd80 legacy=0x7f44fc143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.366+0000 7f44fb400640 1 -- 172.21.15.86:0/2040542684 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc162170 con 0x7f44fc1444f0 2024-03-22T04:17:02.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f44fb400640 1 --2- 172.21.15.86:0/2040542684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc1444f0 0x7f44fc1692f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44f40097f0 tx=0x7f44f401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948e183fa780755 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:02.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f44f9600640 1 -- 172.21.15.86:0/2040542684 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40b0070 con 0x7f44fc1444f0 2024-03-22T04:17:02.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f44f9600640 1 -- 172.21.15.86:0/2040542684 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f40ab600 con 0x7f44fc1444f0 2024-03-22T04:17:02.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f44f9600640 1 -- 172.21.15.86:0/2040542684 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40ab9d0 con 0x7f44fc1444f0 2024-03-22T04:17:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f4501800640 1 -- 172.21.15.86:0/2040542684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc1444f0 msgr2=0x7f44fc1692f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f4501800640 1 --2- 172.21.15.86:0/2040542684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc1444f0 0x7f44fc1692f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44f40097f0 tx=0x7f44f401c450 comp rx=0 tx=0).stop 2024-03-22T04:17:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f4501800640 1 -- 172.21.15.86:0/2040542684 shutdown_connections 2024-03-22T04:17:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f4501800640 1 --2- 172.21.15.86:0/2040542684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc1444f0 0x7f44fc1692f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f4501800640 1 -- 172.21.15.86:0/2040542684 >> 172.21.15.86:0/2040542684 conn(0x7f44fc071fe0 msgr2=0x7f44fc0723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f4501800640 1 -- 172.21.15.86:0/2040542684 shutdown_connections 2024-03-22T04:17:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.370+0000 7f4501800640 1 -- 172.21.15.86:0/2040542684 wait complete. 2024-03-22T04:17:02.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 Processor -- start 2024-03-22T04:17:02.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 -- start start 2024-03-22T04:17:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc06cd80 0x7f44fc1382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0aff50 0x7f44fc13c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc1444f0 0x7f44fc13c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc0b04b0 con 0x7f44fc0aff50 2024-03-22T04:17:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc0b0330 con 0x7f44fc1444f0 2024-03-22T04:17:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc16c460 con 0x7f44fc06cd80 2024-03-22T04:17:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0aff50 0x7f44fc13c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0aff50 0x7f44fc13c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49888/0 (socket says 172.21.15.86:49888) 2024-03-22T04:17:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 -- 172.21.15.86:0/3571063323 learned_addr learned my addr 172.21.15.86:0/3571063323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44faa00640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc06cd80 0x7f44fc1382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fb400640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc1444f0 0x7f44fc13c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 -- 172.21.15.86:0/3571063323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc06cd80 msgr2=0x7f44fc1382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc06cd80 0x7f44fc1382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 -- 172.21.15.86:0/3571063323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc1444f0 msgr2=0x7f44fc13c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc1444f0 0x7f44fc13c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 -- 172.21.15.86:0/3571063323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc1417b0 con 0x7f44fc0aff50 2024-03-22T04:17:02.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44fa000640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0aff50 0x7f44fc13c0a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c760 tx=0x7f44e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44f2a00640 1 -- 172.21.15.86:0/3571063323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8017070 con 0x7f44fc0aff50 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44f2a00640 1 -- 172.21.15.86:0/3571063323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e8005ae0 con 0x7f44fc0aff50 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f44f2a00640 1 -- 172.21.15.86:0/3571063323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8007e30 con 0x7f44fc0aff50 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc13cc10 con 0x7f44fc0aff50 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc13d1d0 con 0x7f44fc0aff50 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.374+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc072b80 con 0x7f44fc0aff50 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.378+0000 7f44f2a00640 1 -- 172.21.15.86:0/3571063323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f44e801a070 con 0x7f44fc0aff50 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.378+0000 7f44f2a00640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.378+0000 7f44f2a00640 1 -- 172.21.15.86:0/3571063323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f44e8085470 con 0x7f44fc0aff50 2024-03-22T04:17:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.378+0000 7f44faa00640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.378+0000 7f44faa00640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0012f0 tx=0x7f44ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.382+0000 7f44f2a00640 1 -- 172.21.15.86:0/3571063323 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f44fc0ae5a0 con 0x7f44fc0aff50 2024-03-22T04:17:02.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.534+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44fc0ae920 con 0x7f44fc0aff50 2024-03-22T04:17:02.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f44f2a00640 1 -- 172.21.15.86:0/3571063323 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16933 (secure 0 0 0) 0x7f44e8055580 con 0x7f44fc0aff50 2024-03-22T04:17:02.538 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:02.538 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:16:59.639901+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:17:02.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 msgr2=0x7f44dc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0012f0 tx=0x7f44ec009040 comp rx=0 tx=0).stop 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0aff50 msgr2=0x7f44fc13c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0aff50 0x7f44fc13c0a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c760 tx=0x7f44e800cc20 comp rx=0 tx=0).stop 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 shutdown_connections 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc1444f0 0x7f44fc13c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0aff50 0x7f44fc13c0a0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 --2- 172.21.15.86:0/3571063323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc06cd80 0x7f44fc1382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 >> 172.21.15.86:0/3571063323 conn(0x7f44fc071fe0 msgr2=0x7f44fc1588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 shutdown_connections 2024-03-22T04:17:02.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.538+0000 7f4501800640 1 -- 172.21.15.86:0/3571063323 wait complete. 2024-03-22T04:17:02.560 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T04:17:02.560 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:17:02.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4cca00640 1 Processor -- start 2024-03-22T04:17:02.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4cca00640 1 -- start start 2024-03-22T04:17:02.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4cca00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd4c806ef40 0x7fd4c8142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4cca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c815c7c0 con 0x7fd4c80ad330 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4cca00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c806dea0 con 0x7fd4c80ad710 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4cca00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c806e180 con 0x7fd4c806ef40 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd4c80ad330 0x7fd4c813fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39424/0 (socket says 172.21.15.86:39424) 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5e00640 1 -- 172.21.15.86:0/347786932 learned_addr learned my addr 172.21.15.86:0/347786932 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4bd400640 1 --2- 172.21.15.86:0/347786932 >> v2:172.21.15.196:3300/0 conn(0x7fd4c806ef40 0x7fd4c8142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1685833747 0 0) 0x7fd4c815c7c0 con 0x7fd4c80ad330 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4ac003530 con 0x7fd4c80ad330 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858276273 0 0) 0x7fd4c806dea0 con 0x7fd4c80ad710 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4c815c7c0 con 0x7fd4c80ad710 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3648688427 0 0) 0x7fd4ac003530 con 0x7fd4c80ad330 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 >> v2:172.21.15.196:3300/0 conn(0x7fd4c806ef40 msgr2=0x7fd4c8142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 --2- 172.21.15.86:0/347786932 >> v2:172.21.15.196:3300/0 conn(0x7fd4c806ef40 0x7fd4c8142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 >> v1:172.21.15.196:6789/0 conn(0x7fd4c80ad710 legacy=0x7fd4c81402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c806e460 con 0x7fd4c80ad330 2024-03-22T04:17:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fd4b80016a0 con 0x7fd4c80ad330 2024-03-22T04:17:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 >> v1:172.21.15.86:6789/0 conn(0x7fd4c80ad330 legacy=0x7fd4c813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ac003b50 0x7fd4ac003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac004810 0x7fd4ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ac0051d0 0x7fd4ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c806e180 con 0x7fd4ac003b50 2024-03-22T04:17:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac00c050 con 0x7fd4ac0051d0 2024-03-22T04:17:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ac00c250 con 0x7fd4ac004810 2024-03-22T04:17:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.698+0000 7fd4c5e00640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ac003b50 0x7fd4ac003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c6800640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ac0051d0 0x7fd4ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4bd400640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac004810 0x7fd4ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5e00640 1 -- 172.21.15.86:0/347786932 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac004810 msgr2=0x7fd4ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5e00640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac004810 0x7fd4ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5e00640 1 -- 172.21.15.86:0/347786932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ac0051d0 msgr2=0x7fd4ac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5e00640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4ac0051d0 0x7fd4ac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5e00640 1 -- 172.21.15.86:0/347786932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac003530 con 0x7fd4ac003b50 2024-03-22T04:17:02.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c6800640 1 -- 172.21.15.86:0/347786932 reap_dead start 2024-03-22T04:17:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5e00640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ac003b50 0x7fd4ac003f50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800c210 tx=0x7fd4b801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cb7320d6ba31f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8007cf0 con 0x7fd4ac003b50 2024-03-22T04:17:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b80a9020 con 0x7fd4ac003b50 2024-03-22T04:17:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4c5400640 1 -- 172.21.15.86:0/347786932 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8005ae0 con 0x7fd4ac003b50 2024-03-22T04:17:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4cca00640 1 -- 172.21.15.86:0/347786932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ac003b50 msgr2=0x7fd4ac003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4cca00640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ac003b50 0x7fd4ac003f50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800c210 tx=0x7fd4b801cbd0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4cca00640 1 -- 172.21.15.86:0/347786932 shutdown_connections 2024-03-22T04:17:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4cca00640 1 --2- 172.21.15.86:0/347786932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4ac003b50 0x7fd4ac003f50 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4cca00640 1 -- 172.21.15.86:0/347786932 >> 172.21.15.86:0/347786932 conn(0x7fd4c80b1df0 msgr2=0x7fd4c80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4cca00640 1 -- 172.21.15.86:0/347786932 shutdown_connections 2024-03-22T04:17:02.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.702+0000 7fd4cca00640 1 -- 172.21.15.86:0/347786932 wait complete. 2024-03-22T04:17:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 Processor -- start 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 -- start start 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac003b50 0x7fd4c815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c809dd10 0x7fd4c81448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4c8144df0 0x7fd4c80b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81403c0 con 0x7fd4c809dd10 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c806db10 con 0x7fd4c8144df0 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c8140540 con 0x7fd4ac003b50 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c809dd10 0x7fd4c81448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c809dd10 0x7fd4c81448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49900/0 (socket says 172.21.15.86:49900) 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 -- 172.21.15.86:0/18541839 learned_addr learned my addr 172.21.15.86:0/18541839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4c5e00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac003b50 0x7fd4c815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4c6800640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4c8144df0 0x7fd4c80b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 -- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac003b50 msgr2=0x7fd4c815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac003b50 0x7fd4c815b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 -- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4c8144df0 msgr2=0x7fd4c80b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4c8144df0 0x7fd4c80b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 -- 172.21.15.86:0/18541839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c81fbe90 con 0x7fd4c809dd10 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4c6800640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4c8144df0 0x7fd4c80b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bd400640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c809dd10 0x7fd4c81448b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0011530 tx=0x7fd4b00119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bea00640 1 -- 172.21.15.86:0/18541839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0015870 con 0x7fd4c809dd10 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bea00640 1 -- 172.21.15.86:0/18541839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b0015a10 con 0x7fd4c809dd10 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c81fb310 con 0x7fd4c809dd10 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4bea00640 1 -- 172.21.15.86:0/18541839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0015dc0 con 0x7fd4c809dd10 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.706+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c81fb8d0 con 0x7fd4c809dd10 2024-03-22T04:17:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.710+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c8068650 con 0x7fd4c809dd10 2024-03-22T04:17:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.710+0000 7fd4bea00640 1 -- 172.21.15.86:0/18541839 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd4b0013400 con 0x7fd4c809dd10 2024-03-22T04:17:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.710+0000 7fd4bea00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4a4067b50 0x7fd4a406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.710+0000 7fd4bea00640 1 -- 172.21.15.86:0/18541839 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fd4b00867f0 con 0x7fd4c809dd10 2024-03-22T04:17:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.710+0000 7fd4c5e00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4a4067b50 0x7fd4a406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:02.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.710+0000 7fd4c5e00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4a4067b50 0x7fd4a406a010 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80aa000 tx=0x7fd4b80b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:02.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.714+0000 7fd4bea00640 1 -- 172.21.15.86:0/18541839 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd4b0019030 con 0x7fd4c809dd10 2024-03-22T04:17:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.866+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4c8072f10 con 0x7fd4a4067b50 2024-03-22T04:17:02.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.870+0000 7fd4bea00640 1 -- 172.21.15.86:0/18541839 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252047 (secure 0 0 0) 0x7fd4b80fa8a0 con 0x7fd4a4067b50 2024-03-22T04:17:02.874 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:17:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4a4067b50 msgr2=0x7fd4a406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4a4067b50 0x7fd4a406a010 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80aa000 tx=0x7fd4b80b0040 comp rx=0 tx=0).stop 2024-03-22T04:17:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c809dd10 msgr2=0x7fd4c81448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c809dd10 0x7fd4c81448b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0011530 tx=0x7fd4b00119f0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 shutdown_connections 2024-03-22T04:17:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd4a4067b50 0x7fd4a406a010 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4c8144df0 0x7fd4c80b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c809dd10 0x7fd4c81448b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 --2- 172.21.15.86:0/18541839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd4ac003b50 0x7fd4c815b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 >> 172.21.15.86:0/18541839 conn(0x7fd4c80b1df0 msgr2=0x7fd4c8071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 shutdown_connections 2024-03-22T04:17:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:02.874+0000 7fd4cca00640 1 -- 172.21.15.86:0/18541839 wait complete. 2024-03-22T04:17:02.894 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-03-22T04:17:01.874119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259540,"kb_used_data":10084,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715180,"statfs":{"total":773094113280,"available":772828344320,"internally_reserved":0,"allocated":10326016,"data_stored":5344640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.247105"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.685929+0000","last_change":"2024-03-22T04:16:58.682586+0000","last_active":"2024-03-22T04:16:58.685929+0000","last_peered":"2024-03-22T04:16:58.685929+0000","last_clean":"2024-03-22T04:16:58.685929+0000","last_became_active":"2024-03-22T04:16:58.682211+0000","last_became_peered":"2024-03-22T04:16:58.682211+0000","last_unstale":"2024-03-22T04:16:58.685929+0000","last_undegraded":"2024-03-22T04:16:58.685929+0000","last_fullsized":"2024-03-22T04:16:58.685929+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:15.690412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671072+0000","last_change":"2024-03-22T04:16:58.670276+0000","last_active":"2024-03-22T04:16:59.671072+0000","last_peered":"2024-03-22T04:16:59.671072+0000","last_clean":"2024-03-22T04:16:59.671072+0000","last_became_active":"2024-03-22T04:16:58.670012+0000","last_became_peered":"2024-03-22T04:16:58.670012+0000","last_unstale":"2024-03-22T04:16:59.671072+0000","last_undegraded":"2024-03-22T04:16:59.671072+0000","last_fullsized":"2024-03-22T04:16:59.671072+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:02.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883386+0000","last_change":"2024-03-22T04:16:58.659168+0000","last_active":"2024-03-22T04:16:59.883386+0000","last_peered":"2024-03-22T04:16:59.883386+0000","last_clean":"2024-03-22T04:16:59.883386+0000","last_became_active":"2024-03-22T04:16:58.657030+0000","last_became_peered":"2024-03-22T04:16:58.657030+0000","last_unstale":"2024-03-22T04:16:59.883386+0000","last_undegraded":"2024-03-22T04:16:59.883386+0000","last_fullsized":"2024-03-22T04:16:59.883386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:32:54.102100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883244+0000","last_change":"2024-03-22T04:16:58.657060+0000","last_active":"2024-03-22T04:16:59.883244+0000","last_peered":"2024-03-22T04:16:59.883244+0000","last_clean":"2024-03-22T04:16:59.883244+0000","last_became_active":"2024-03-22T04:16:58.656743+0000","last_became_peered":"2024-03-22T04:16:58.656743+0000","last_unstale":"2024-03-22T04:16:59.883244+0000","last_undegraded":"2024-03-22T04:16:59.883244+0000","last_fullsized":"2024-03-22T04:16:59.883244+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:15:55.957855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670885+0000","last_change":"2024-03-22T04:16:58.652736+0000","last_active":"2024-03-22T04:16:59.670885+0000","last_peered":"2024-03-22T04:16:59.670885+0000","last_clean":"2024-03-22T04:16:59.670885+0000","last_became_active":"2024-03-22T04:16:58.652273+0000","last_became_peered":"2024-03-22T04:16:58.652273+0000","last_unstale":"2024-03-22T04:16:59.670885+0000","last_undegraded":"2024-03-22T04:16:59.670885+0000","last_fullsized":"2024-03-22T04:16:59.670885+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:57.725363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680957+0000","last_change":"2024-03-22T04:16:58.672863+0000","last_active":"2024-03-22T04:16:58.680957+0000","last_peered":"2024-03-22T04:16:58.680957+0000","last_clean":"2024-03-22T04:16:58.680957+0000","last_became_active":"2024-03-22T04:16:58.672571+0000","last_became_peered":"2024-03-22T04:16:58.672571+0000","last_unstale":"2024-03-22T04:16:58.680957+0000","last_undegraded":"2024-03-22T04:16:58.680957+0000","last_fullsized":"2024-03-22T04:16:58.680957+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:53:42.876217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668164+0000","last_change":"2024-03-22T04:16:58.658200+0000","last_active":"2024-03-22T04:16:59.668164+0000","last_peered":"2024-03-22T04:16:59.668164+0000","last_clean":"2024-03-22T04:16:59.668164+0000","last_became_active":"2024-03-22T04:16:58.657898+0000","last_became_peered":"2024-03-22T04:16:58.657898+0000","last_unstale":"2024-03-22T04:16:59.668164+0000","last_undegraded":"2024-03-22T04:16:59.668164+0000","last_fullsized":"2024-03-22T04:16:59.668164+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:41:20.555405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.667484+0000","last_change":"2024-03-22T04:16:58.657768+0000","last_active":"2024-03-22T04:16:58.667484+0000","last_peered":"2024-03-22T04:16:58.667484+0000","last_clean":"2024-03-22T04:16:58.667484+0000","last_became_active":"2024-03-22T04:16:58.657475+0000","last_became_peered":"2024-03-22T04:16:58.657475+0000","last_unstale":"2024-03-22T04:16:58.667484+0000","last_undegraded":"2024-03-22T04:16:58.667484+0000","last_fullsized":"2024-03-22T04:16:58.667484+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:35.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668260+0000","last_change":"2024-03-22T04:16:58.656824+0000","last_active":"2024-03-22T04:16:59.668260+0000","last_peered":"2024-03-22T04:16:59.668260+0000","last_clean":"2024-03-22T04:16:59.668260+0000","last_became_active":"2024-03-22T04:16:58.656574+0000","last_became_peered":"2024-03-22T04:16:58.656574+0000","last_unstale":"2024-03-22T04:16:59.668260+0000","last_undegraded":"2024-03-22T04:16:59.668260+0000","last_fullsized":"2024-03-22T04:16:59.668260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:03:38.621750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883479+0000","last_change":"2024-03-22T04:16:58.666381+0000","last_active":"2024-03-22T04:16:59.883479+0000","last_peered":"2024-03-22T04:16:59.883479+0000","last_clean":"2024-03-22T04:16:59.883479+0000","last_became_active":"2024-03-22T04:16:58.666126+0000","last_became_peered":"2024-03-22T04:16:58.666126+0000","last_unstale":"2024-03-22T04:16:59.883479+0000","last_undegraded":"2024-03-22T04:16:59.883479+0000","last_fullsized":"2024-03-22T04:16:59.883479+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:50:13.717561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668203+0000","last_change":"2024-03-22T04:16:58.655957+0000","last_active":"2024-03-22T04:16:59.668203+0000","last_peered":"2024-03-22T04:16:59.668203+0000","last_clean":"2024-03-22T04:16:59.668203+0000","last_became_active":"2024-03-22T04:16:58.655689+0000","last_became_peered":"2024-03-22T04:16:58.655689+0000","last_unstale":"2024-03-22T04:16:59.668203+0000","last_undegraded":"2024-03-22T04:16:59.668203+0000","last_fullsized":"2024-03-22T04:16:59.668203+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:21:06.716490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883252+0000","last_change":"2024-03-22T04:16:58.679894+0000","last_active":"2024-03-22T04:16:59.883252+0000","last_peered":"2024-03-22T04:16:59.883252+0000","last_clean":"2024-03-22T04:16:59.883252+0000","last_became_active":"2024-03-22T04:16:58.679537+0000","last_became_peered":"2024-03-22T04:16:58.679537+0000","last_unstale":"2024-03-22T04:16:59.883252+0000","last_undegraded":"2024-03-22T04:16:59.883252+0000","last_fullsized":"2024-03-22T04:16:59.883252+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:37:47.891796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.645910+0000","last_change":"2024-03-22T04:16:58.674007+0000","last_active":"2024-03-22T04:16:59.645910+0000","last_peered":"2024-03-22T04:16:59.645910+0000","last_clean":"2024-03-22T04:16:59.645910+0000","last_became_active":"2024-03-22T04:16:58.673679+0000","last_became_peered":"2024-03-22T04:16:58.673679+0000","last_unstale":"2024-03-22T04:16:59.645910+0000","last_undegraded":"2024-03-22T04:16:59.645910+0000","last_fullsized":"2024-03-22T04:16:59.645910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:46:51.882426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646789+0000","last_change":"2024-03-22T04:16:58.671335+0000","last_active":"2024-03-22T04:16:59.646789+0000","last_peered":"2024-03-22T04:16:59.646789+0000","last_clean":"2024-03-22T04:16:59.646789+0000","last_became_active":"2024-03-22T04:16:58.671015+0000","last_became_peered":"2024-03-22T04:16:58.671015+0000","last_unstale":"2024-03-22T04:16:59.646789+0000","last_undegraded":"2024-03-22T04:16:59.646789+0000","last_fullsized":"2024-03-22T04:16:59.646789+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:57.725363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883094+0000","last_change":"2024-03-22T04:16:58.666983+0000","last_active":"2024-03-22T04:16:59.883094+0000","last_peered":"2024-03-22T04:16:59.883094+0000","last_clean":"2024-03-22T04:16:59.883094+0000","last_became_active":"2024-03-22T04:16:58.665728+0000","last_became_peered":"2024-03-22T04:16:58.665728+0000","last_unstale":"2024-03-22T04:16:59.883094+0000","last_undegraded":"2024-03-22T04:16:59.883094+0000","last_fullsized":"2024-03-22T04:16:59.883094+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:48:05.551943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680975+0000","last_change":"2024-03-22T04:16:58.673197+0000","last_active":"2024-03-22T04:16:58.680975+0000","last_peered":"2024-03-22T04:16:58.680975+0000","last_clean":"2024-03-22T04:16:58.680975+0000","last_became_active":"2024-03-22T04:16:58.672660+0000","last_became_peered":"2024-03-22T04:16:58.672660+0000","last_unstale":"2024-03-22T04:16:58.680975+0000","last_undegraded":"2024-03-22T04:16:58.680975+0000","last_fullsized":"2024-03-22T04:16:58.680975+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:28.357635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.685598+0000","last_change":"2024-03-22T04:16:58.681033+0000","last_active":"2024-03-22T04:16:58.685598+0000","last_peered":"2024-03-22T04:16:58.685598+0000","last_clean":"2024-03-22T04:16:58.685598+0000","last_became_active":"2024-03-22T04:16:58.680675+0000","last_became_peered":"2024-03-22T04:16:58.680675+0000","last_unstale":"2024-03-22T04:16:58.685598+0000","last_undegraded":"2024-03-22T04:16:58.685598+0000","last_fullsized":"2024-03-22T04:16:58.685598+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:13:56.482754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883548+0000","last_change":"2024-03-22T04:16:58.659370+0000","last_active":"2024-03-22T04:16:59.883548+0000","last_peered":"2024-03-22T04:16:59.883548+0000","last_clean":"2024-03-22T04:16:59.883548+0000","last_became_active":"2024-03-22T04:16:58.659040+0000","last_became_peered":"2024-03-22T04:16:58.659040+0000","last_unstale":"2024-03-22T04:16:59.883548+0000","last_undegraded":"2024-03-22T04:16:59.883548+0000","last_fullsized":"2024-03-22T04:16:59.883548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:26:42.516224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.677363+0000","last_change":"2024-03-22T04:16:58.672002+0000","last_active":"2024-03-22T04:16:58.677363+0000","last_peered":"2024-03-22T04:16:58.677363+0000","last_clean":"2024-03-22T04:16:58.677363+0000","last_became_active":"2024-03-22T04:16:58.671740+0000","last_became_peered":"2024-03-22T04:16:58.671740+0000","last_unstale":"2024-03-22T04:16:58.677363+0000","last_undegraded":"2024-03-22T04:16:58.677363+0000","last_fullsized":"2024-03-22T04:16:58.677363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:17.742365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.681061+0000","last_change":"2024-03-22T04:16:58.674240+0000","last_active":"2024-03-22T04:16:58.681061+0000","last_peered":"2024-03-22T04:16:58.681061+0000","last_clean":"2024-03-22T04:16:58.681061+0000","last_became_active":"2024-03-22T04:16:58.673979+0000","last_became_peered":"2024-03-22T04:16:58.673979+0000","last_unstale":"2024-03-22T04:16:58.681061+0000","last_undegraded":"2024-03-22T04:16:58.681061+0000","last_fullsized":"2024-03-22T04:16:58.681061+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:10:27.288562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671045+0000","last_change":"2024-03-22T04:16:58.657152+0000","last_active":"2024-03-22T04:16:59.671045+0000","last_peered":"2024-03-22T04:16:59.671045+0000","last_clean":"2024-03-22T04:16:59.671045+0000","last_became_active":"2024-03-22T04:16:58.656631+0000","last_became_peered":"2024-03-22T04:16:58.656631+0000","last_unstale":"2024-03-22T04:16:59.671045+0000","last_undegraded":"2024-03-22T04:16:59.671045+0000","last_fullsized":"2024-03-22T04:16:59.671045+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:08:00.166390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.661001+0000","last_change":"2024-03-22T04:16:58.656708+0000","last_active":"2024-03-22T04:16:58.661001+0000","last_peered":"2024-03-22T04:16:58.661001+0000","last_clean":"2024-03-22T04:16:58.661001+0000","last_became_active":"2024-03-22T04:16:58.656353+0000","last_became_peered":"2024-03-22T04:16:58.656353+0000","last_unstale":"2024-03-22T04:16:58.661001+0000","last_undegraded":"2024-03-22T04:16:58.661001+0000","last_fullsized":"2024-03-22T04:16:58.661001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:08:00.166390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670759+0000","last_change":"2024-03-22T04:16:58.669779+0000","last_active":"2024-03-22T04:16:59.670759+0000","last_peered":"2024-03-22T04:16:59.670759+0000","last_clean":"2024-03-22T04:16:59.670759+0000","last_became_active":"2024-03-22T04:16:58.668792+0000","last_became_peered":"2024-03-22T04:16:58.668792+0000","last_unstale":"2024-03-22T04:16:59.670759+0000","last_undegraded":"2024-03-22T04:16:59.670759+0000","last_fullsized":"2024-03-22T04:16:59.670759+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:13.073512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.671746+0000","last_change":"2024-03-22T04:16:58.664705+0000","last_active":"2024-03-22T04:16:58.671746+0000","last_peered":"2024-03-22T04:16:58.671746+0000","last_clean":"2024-03-22T04:16:58.671746+0000","last_became_active":"2024-03-22T04:16:58.664362+0000","last_became_peered":"2024-03-22T04:16:58.664362+0000","last_unstale":"2024-03-22T04:16:58.671746+0000","last_undegraded":"2024-03-22T04:16:58.671746+0000","last_fullsized":"2024-03-22T04:16:58.671746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:02.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671170+0000","last_change":"2024-03-22T04:16:58.657736+0000","last_active":"2024-03-22T04:16:59.671170+0000","last_peered":"2024-03-22T04:16:59.671170+0000","last_clean":"2024-03-22T04:16:59.671170+0000","last_became_active":"2024-03-22T04:16:58.657451+0000","last_became_peered":"2024-03-22T04:16:58.657451+0000","last_unstale":"2024-03-22T04:16:59.671170+0000","last_undegraded":"2024-03-22T04:16:59.671170+0000","last_fullsized":"2024-03-22T04:16:59.671170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:46:51.882426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671238+0000","last_change":"2024-03-22T04:16:58.670882+0000","last_active":"2024-03-22T04:16:59.671238+0000","last_peered":"2024-03-22T04:16:59.671238+0000","last_clean":"2024-03-22T04:16:59.671238+0000","last_became_active":"2024-03-22T04:16:58.670518+0000","last_became_peered":"2024-03-22T04:16:58.670518+0000","last_unstale":"2024-03-22T04:16:59.671238+0000","last_undegraded":"2024-03-22T04:16:59.671238+0000","last_fullsized":"2024-03-22T04:16:59.671238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:50.842967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883447+0000","last_change":"2024-03-22T04:16:58.657882+0000","last_active":"2024-03-22T04:16:59.883447+0000","last_peered":"2024-03-22T04:16:59.883447+0000","last_clean":"2024-03-22T04:16:59.883447+0000","last_became_active":"2024-03-22T04:16:58.657483+0000","last_became_peered":"2024-03-22T04:16:58.657483+0000","last_unstale":"2024-03-22T04:16:59.883447+0000","last_undegraded":"2024-03-22T04:16:59.883447+0000","last_fullsized":"2024-03-22T04:16:59.883447+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:59:34.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646904+0000","last_change":"2024-03-22T04:16:58.678361+0000","last_active":"2024-03-22T04:16:59.646904+0000","last_peered":"2024-03-22T04:16:59.646904+0000","last_clean":"2024-03-22T04:16:59.646904+0000","last_became_active":"2024-03-22T04:16:58.677971+0000","last_became_peered":"2024-03-22T04:16:58.677971+0000","last_unstale":"2024-03-22T04:16:59.646904+0000","last_undegraded":"2024-03-22T04:16:59.646904+0000","last_fullsized":"2024-03-22T04:16:59.646904+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:46.047388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671445+0000","last_change":"2024-03-22T04:16:58.669068+0000","last_active":"2024-03-22T04:16:59.671445+0000","last_peered":"2024-03-22T04:16:59.671445+0000","last_clean":"2024-03-22T04:16:59.671445+0000","last_became_active":"2024-03-22T04:16:58.668754+0000","last_became_peered":"2024-03-22T04:16:58.668754+0000","last_unstale":"2024-03-22T04:16:59.671445+0000","last_undegraded":"2024-03-22T04:16:59.671445+0000","last_fullsized":"2024-03-22T04:16:59.671445+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:15.690412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670859+0000","last_change":"2024-03-22T04:16:58.654734+0000","last_active":"2024-03-22T04:16:59.670859+0000","last_peered":"2024-03-22T04:16:59.670859+0000","last_clean":"2024-03-22T04:16:59.670859+0000","last_became_active":"2024-03-22T04:16:58.654441+0000","last_became_peered":"2024-03-22T04:16:58.654441+0000","last_unstale":"2024-03-22T04:16:59.670859+0000","last_undegraded":"2024-03-22T04:16:59.670859+0000","last_fullsized":"2024-03-22T04:16:59.670859+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:25.281798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.677284+0000","last_change":"2024-03-22T04:16:58.669970+0000","last_active":"2024-03-22T04:16:58.677284+0000","last_peered":"2024-03-22T04:16:58.677284+0000","last_clean":"2024-03-22T04:16:58.677284+0000","last_became_active":"2024-03-22T04:16:58.669620+0000","last_became_peered":"2024-03-22T04:16:58.669620+0000","last_unstale":"2024-03-22T04:16:58.677284+0000","last_undegraded":"2024-03-22T04:16:58.677284+0000","last_fullsized":"2024-03-22T04:16:58.677284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:02.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680948+0000","last_change":"2024-03-22T04:16:58.672972+0000","last_active":"2024-03-22T04:16:58.680948+0000","last_peered":"2024-03-22T04:16:58.680948+0000","last_clean":"2024-03-22T04:16:58.680948+0000","last_became_active":"2024-03-22T04:16:58.672223+0000","last_became_peered":"2024-03-22T04:16:58.672223+0000","last_unstale":"2024-03-22T04:16:58.680948+0000","last_undegraded":"2024-03-22T04:16:58.680948+0000","last_fullsized":"2024-03-22T04:16:58.680948+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:37:50.129389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.684725+0000","last_change":"2024-03-22T04:16:58.669199+0000","last_active":"2024-03-22T04:16:58.684725+0000","last_peered":"2024-03-22T04:16:58.684725+0000","last_clean":"2024-03-22T04:16:58.684725+0000","last_became_active":"2024-03-22T04:16:58.668934+0000","last_became_peered":"2024-03-22T04:16:58.668934+0000","last_unstale":"2024-03-22T04:16:58.684725+0000","last_undegraded":"2024-03-22T04:16:58.684725+0000","last_fullsized":"2024-03-22T04:16:58.684725+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:32:09.746284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"92'4","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671362+0000","last_change":"2024-03-22T04:16:57.656478+0000","last_active":"2024-03-22T04:16:59.671362+0000","last_peered":"2024-03-22T04:16:59.671362+0000","last_clean":"2024-03-22T04:16:59.671362+0000","last_became_active":"2024-03-22T04:16:57.656218+0000","last_became_peered":"2024-03-22T04:16:57.656218+0000","last_unstale":"2024-03-22T04:16:59.671362+0000","last_undegraded":"2024-03-22T04:16:59.671362+0000","last_fullsized":"2024-03-22T04:16:59.671362+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:15:40.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680826+0000","last_change":"2024-03-22T04:16:58.670649+0000","last_active":"2024-03-22T04:16:58.680826+0000","last_peered":"2024-03-22T04:16:58.680826+0000","last_clean":"2024-03-22T04:16:58.680826+0000","last_became_active":"2024-03-22T04:16:58.669153+0000","last_became_peered":"2024-03-22T04:16:58.669153+0000","last_unstale":"2024-03-22T04:16:58.680826+0000","last_undegraded":"2024-03-22T04:16:58.680826+0000","last_fullsized":"2024-03-22T04:16:58.680826+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:42:11.903310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646888+0000","last_change":"2024-03-22T04:16:57.643608+0000","last_active":"2024-03-22T04:16:59.646888+0000","last_peered":"2024-03-22T04:16:59.646888+0000","last_clean":"2024-03-22T04:16:59.646888+0000","last_became_active":"2024-03-22T04:16:57.643313+0000","last_became_peered":"2024-03-22T04:16:57.643313+0000","last_unstale":"2024-03-22T04:16:59.646888+0000","last_undegraded":"2024-03-22T04:16:59.646888+0000","last_fullsized":"2024-03-22T04:16:59.646888+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:07:54.462875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668190+0000","last_change":"2024-03-22T04:16:58.681556+0000","last_active":"2024-03-22T04:16:59.668190+0000","last_peered":"2024-03-22T04:16:59.668190+0000","last_clean":"2024-03-22T04:16:59.668190+0000","last_became_active":"2024-03-22T04:16:58.681258+0000","last_became_peered":"2024-03-22T04:16:58.681258+0000","last_unstale":"2024-03-22T04:16:59.668190+0000","last_undegraded":"2024-03-22T04:16:59.668190+0000","last_fullsized":"2024-03-22T04:16:59.668190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:10.340650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671244+0000","last_change":"2024-03-22T04:16:57.655287+0000","last_active":"2024-03-22T04:16:59.671244+0000","last_peered":"2024-03-22T04:16:59.671244+0000","last_clean":"2024-03-22T04:16:59.671244+0000","last_became_active":"2024-03-22T04:16:57.655005+0000","last_became_peered":"2024-03-22T04:16:57.655005+0000","last_unstale":"2024-03-22T04:16:59.671244+0000","last_undegraded":"2024-03-22T04:16:59.671244+0000","last_fullsized":"2024-03-22T04:16:59.671244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:32:40.294292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883490+0000","last_change":"2024-03-22T04:16:58.662679+0000","last_active":"2024-03-22T04:16:59.883490+0000","last_peered":"2024-03-22T04:16:59.883490+0000","last_clean":"2024-03-22T04:16:59.883490+0000","last_became_active":"2024-03-22T04:16:58.662415+0000","last_became_peered":"2024-03-22T04:16:58.662415+0000","last_unstale":"2024-03-22T04:16:59.883490+0000","last_undegraded":"2024-03-22T04:16:59.883490+0000","last_fullsized":"2024-03-22T04:16:59.883490+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:16.247985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646958+0000","last_change":"2024-03-22T04:16:57.652536+0000","last_active":"2024-03-22T04:16:59.646958+0000","last_peered":"2024-03-22T04:16:59.646958+0000","last_clean":"2024-03-22T04:16:59.646958+0000","last_became_active":"2024-03-22T04:16:57.651827+0000","last_became_peered":"2024-03-22T04:16:57.651827+0000","last_unstale":"2024-03-22T04:16:59.646958+0000","last_undegraded":"2024-03-22T04:16:59.646958+0000","last_fullsized":"2024-03-22T04:16:59.646958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:56:57.570918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.645792+0000","last_change":"2024-03-22T04:16:58.675622+0000","last_active":"2024-03-22T04:16:59.645792+0000","last_peered":"2024-03-22T04:16:59.645792+0000","last_clean":"2024-03-22T04:16:59.645792+0000","last_became_active":"2024-03-22T04:16:58.675353+0000","last_became_peered":"2024-03-22T04:16:58.675353+0000","last_unstale":"2024-03-22T04:16:59.645792+0000","last_undegraded":"2024-03-22T04:16:59.645792+0000","last_fullsized":"2024-03-22T04:16:59.645792+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:02.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"92'1","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.683700+0000","last_change":"2024-03-22T04:16:57.642241+0000","last_active":"2024-03-22T04:16:58.683700+0000","last_peered":"2024-03-22T04:16:58.683700+0000","last_clean":"2024-03-22T04:16:58.683700+0000","last_became_active":"2024-03-22T04:16:57.641988+0000","last_became_peered":"2024-03-22T04:16:57.641988+0000","last_unstale":"2024-03-22T04:16:58.683700+0000","last_undegraded":"2024-03-22T04:16:58.683700+0000","last_fullsized":"2024-03-22T04:16:58.683700+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:23:32.537352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668085+0000","last_change":"2024-03-22T04:16:58.657571+0000","last_active":"2024-03-22T04:16:59.668085+0000","last_peered":"2024-03-22T04:16:59.668085+0000","last_clean":"2024-03-22T04:16:59.668085+0000","last_became_active":"2024-03-22T04:16:58.655831+0000","last_became_peered":"2024-03-22T04:16:58.655831+0000","last_unstale":"2024-03-22T04:16:59.668085+0000","last_undegraded":"2024-03-22T04:16:59.668085+0000","last_fullsized":"2024-03-22T04:16:59.668085+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:04:18.293169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646527+0000","last_change":"2024-03-22T04:16:57.654908+0000","last_active":"2024-03-22T04:16:59.646527+0000","last_peered":"2024-03-22T04:16:59.646527+0000","last_clean":"2024-03-22T04:16:59.646527+0000","last_became_active":"2024-03-22T04:16:57.654580+0000","last_became_peered":"2024-03-22T04:16:57.654580+0000","last_unstale":"2024-03-22T04:16:59.646527+0000","last_undegraded":"2024-03-22T04:16:59.646527+0000","last_fullsized":"2024-03-22T04:16:59.646527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:04.318404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.668303+0000","last_change":"2024-03-22T04:16:58.659808+0000","last_active":"2024-03-22T04:16:58.668303+0000","last_peered":"2024-03-22T04:16:58.668303+0000","last_clean":"2024-03-22T04:16:58.668303+0000","last_became_active":"2024-03-22T04:16:58.659525+0000","last_became_peered":"2024-03-22T04:16:58.659525+0000","last_unstale":"2024-03-22T04:16:58.668303+0000","last_undegraded":"2024-03-22T04:16:58.668303+0000","last_fullsized":"2024-03-22T04:16:58.668303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:26:49.963772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883263+0000","last_change":"2024-03-22T04:16:57.645629+0000","last_active":"2024-03-22T04:16:59.883263+0000","last_peered":"2024-03-22T04:16:59.883263+0000","last_clean":"2024-03-22T04:16:59.883263+0000","last_became_active":"2024-03-22T04:16:57.645355+0000","last_became_peered":"2024-03-22T04:16:57.645355+0000","last_unstale":"2024-03-22T04:16:59.883263+0000","last_undegraded":"2024-03-22T04:16:59.883263+0000","last_fullsized":"2024-03-22T04:16:59.883263+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:18:01.826085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.674352+0000","last_change":"2024-03-22T04:16:58.668263+0000","last_active":"2024-03-22T04:16:58.674352+0000","last_peered":"2024-03-22T04:16:58.674352+0000","last_clean":"2024-03-22T04:16:58.674352+0000","last_became_active":"2024-03-22T04:16:58.667678+0000","last_became_peered":"2024-03-22T04:16:58.667678+0000","last_unstale":"2024-03-22T04:16:58.674352+0000","last_undegraded":"2024-03-22T04:16:58.674352+0000","last_fullsized":"2024-03-22T04:16:58.674352+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:46:51.882426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670697+0000","last_change":"2024-03-22T04:16:57.656706+0000","last_active":"2024-03-22T04:16:59.670697+0000","last_peered":"2024-03-22T04:16:59.670697+0000","last_clean":"2024-03-22T04:16:59.670697+0000","last_became_active":"2024-03-22T04:16:57.656425+0000","last_became_peered":"2024-03-22T04:16:57.656425+0000","last_unstale":"2024-03-22T04:16:59.670697+0000","last_undegraded":"2024-03-22T04:16:59.670697+0000","last_fullsized":"2024-03-22T04:16:59.670697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:05:29.717715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671047+0000","last_change":"2024-03-22T04:16:58.679467+0000","last_active":"2024-03-22T04:16:59.671047+0000","last_peered":"2024-03-22T04:16:59.671047+0000","last_clean":"2024-03-22T04:16:59.671047+0000","last_became_active":"2024-03-22T04:16:58.679206+0000","last_became_peered":"2024-03-22T04:16:58.679206+0000","last_unstale":"2024-03-22T04:16:59.671047+0000","last_undegraded":"2024-03-22T04:16:59.671047+0000","last_fullsized":"2024-03-22T04:16:59.671047+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:42.681464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671374+0000","last_change":"2024-03-22T04:16:57.650167+0000","last_active":"2024-03-22T04:16:59.671374+0000","last_peered":"2024-03-22T04:16:59.671374+0000","last_clean":"2024-03-22T04:16:59.671374+0000","last_became_active":"2024-03-22T04:16:57.649028+0000","last_became_peered":"2024-03-22T04:16:57.649028+0000","last_unstale":"2024-03-22T04:16:59.671374+0000","last_undegraded":"2024-03-22T04:16:59.671374+0000","last_fullsized":"2024-03-22T04:16:59.671374+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:48:04.545029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668336+0000","last_change":"2024-03-22T04:16:58.654199+0000","last_active":"2024-03-22T04:16:59.668336+0000","last_peered":"2024-03-22T04:16:59.668336+0000","last_clean":"2024-03-22T04:16:59.668336+0000","last_became_active":"2024-03-22T04:16:58.653953+0000","last_became_peered":"2024-03-22T04:16:58.653953+0000","last_unstale":"2024-03-22T04:16:59.668336+0000","last_undegraded":"2024-03-22T04:16:59.668336+0000","last_fullsized":"2024-03-22T04:16:59.668336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:17:38.009055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883581+0000","last_change":"2024-03-22T04:16:57.654685+0000","last_active":"2024-03-22T04:16:59.883581+0000","last_peered":"2024-03-22T04:16:59.883581+0000","last_clean":"2024-03-22T04:16:59.883581+0000","last_became_active":"2024-03-22T04:16:57.654402+0000","last_became_peered":"2024-03-22T04:16:57.654402+0000","last_unstale":"2024-03-22T04:16:59.883581+0000","last_undegraded":"2024-03-22T04:16:59.883581+0000","last_fullsized":"2024-03-22T04:16:59.883581+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:37:31.501725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.671110+0000","last_change":"2024-03-22T04:16:58.667348+0000","last_active":"2024-03-22T04:16:58.671110+0000","last_peered":"2024-03-22T04:16:58.671110+0000","last_clean":"2024-03-22T04:16:58.671110+0000","last_became_active":"2024-03-22T04:16:58.667043+0000","last_became_peered":"2024-03-22T04:16:58.667043+0000","last_unstale":"2024-03-22T04:16:58.671110+0000","last_undegraded":"2024-03-22T04:16:58.671110+0000","last_fullsized":"2024-03-22T04:16:58.671110+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:17.742365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668394+0000","last_change":"2024-03-22T04:16:57.656122+0000","last_active":"2024-03-22T04:16:59.668394+0000","last_peered":"2024-03-22T04:16:59.668394+0000","last_clean":"2024-03-22T04:16:59.668394+0000","last_became_active":"2024-03-22T04:16:57.655629+0000","last_became_peered":"2024-03-22T04:16:57.655629+0000","last_unstale":"2024-03-22T04:16:59.668394+0000","last_undegraded":"2024-03-22T04:16:59.668394+0000","last_fullsized":"2024-03-22T04:16:59.668394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:26:46.565286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.668218+0000","last_change":"2024-03-22T04:16:58.658233+0000","last_active":"2024-03-22T04:16:58.668218+0000","last_peered":"2024-03-22T04:16:58.668218+0000","last_clean":"2024-03-22T04:16:58.668218+0000","last_became_active":"2024-03-22T04:16:58.657947+0000","last_became_peered":"2024-03-22T04:16:58.657947+0000","last_unstale":"2024-03-22T04:16:58.668218+0000","last_undegraded":"2024-03-22T04:16:58.668218+0000","last_fullsized":"2024-03-22T04:16:58.668218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:45:04.857756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"92'6","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.691407+0000","last_change":"2024-03-22T04:16:57.657660+0000","last_active":"2024-03-22T04:16:58.691407+0000","last_peered":"2024-03-22T04:16:58.691407+0000","last_clean":"2024-03-22T04:16:58.691407+0000","last_became_active":"2024-03-22T04:16:57.657437+0000","last_became_peered":"2024-03-22T04:16:57.657437+0000","last_unstale":"2024-03-22T04:16:58.691407+0000","last_undegraded":"2024-03-22T04:16:58.691407+0000","last_fullsized":"2024-03-22T04:16:58.691407+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:41.674550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.671723+0000","last_change":"2024-03-22T04:16:58.660848+0000","last_active":"2024-03-22T04:16:58.671723+0000","last_peered":"2024-03-22T04:16:58.671723+0000","last_clean":"2024-03-22T04:16:58.671723+0000","last_became_active":"2024-03-22T04:16:58.660480+0000","last_became_peered":"2024-03-22T04:16:58.660480+0000","last_unstale":"2024-03-22T04:16:58.671723+0000","last_undegraded":"2024-03-22T04:16:58.671723+0000","last_fullsized":"2024-03-22T04:16:58.671723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:57.725363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671434+0000","last_change":"2024-03-22T04:16:57.655310+0000","last_active":"2024-03-22T04:16:59.671434+0000","last_peered":"2024-03-22T04:16:59.671434+0000","last_clean":"2024-03-22T04:16:59.671434+0000","last_became_active":"2024-03-22T04:16:57.651465+0000","last_became_peered":"2024-03-22T04:16:57.651465+0000","last_unstale":"2024-03-22T04:16:59.671434+0000","last_undegraded":"2024-03-22T04:16:59.671434+0000","last_fullsized":"2024-03-22T04:16:59.671434+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:28:26.759427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670774+0000","last_change":"2024-03-22T04:16:58.655094+0000","last_active":"2024-03-22T04:16:59.670774+0000","last_peered":"2024-03-22T04:16:59.670774+0000","last_clean":"2024-03-22T04:16:59.670774+0000","last_became_active":"2024-03-22T04:16:58.654815+0000","last_became_peered":"2024-03-22T04:16:58.654815+0000","last_unstale":"2024-03-22T04:16:59.670774+0000","last_undegraded":"2024-03-22T04:16:59.670774+0000","last_fullsized":"2024-03-22T04:16:59.670774+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:23:33.544266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.655010+0000","last_change":"2024-03-22T04:16:57.649758+0000","last_active":"2024-03-22T04:16:58.655010+0000","last_peered":"2024-03-22T04:16:58.655010+0000","last_clean":"2024-03-22T04:16:58.655010+0000","last_became_active":"2024-03-22T04:16:57.649487+0000","last_became_peered":"2024-03-22T04:16:57.649487+0000","last_unstale":"2024-03-22T04:16:58.655010+0000","last_undegraded":"2024-03-22T04:16:58.655010+0000","last_fullsized":"2024-03-22T04:16:58.655010+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:23:32.537352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.685725+0000","last_change":"2024-03-22T04:16:58.677536+0000","last_active":"2024-03-22T04:16:58.685725+0000","last_peered":"2024-03-22T04:16:58.685725+0000","last_clean":"2024-03-22T04:16:58.685725+0000","last_became_active":"2024-03-22T04:16:58.675044+0000","last_became_peered":"2024-03-22T04:16:58.675044+0000","last_unstale":"2024-03-22T04:16:58.685725+0000","last_undegraded":"2024-03-22T04:16:58.685725+0000","last_fullsized":"2024-03-22T04:16:58.685725+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:46.047388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.654139+0000","last_change":"2024-03-22T04:16:57.652081+0000","last_active":"2024-03-22T04:16:58.654139+0000","last_peered":"2024-03-22T04:16:58.654139+0000","last_clean":"2024-03-22T04:16:58.654139+0000","last_became_active":"2024-03-22T04:16:57.651705+0000","last_became_peered":"2024-03-22T04:16:57.651705+0000","last_unstale":"2024-03-22T04:16:58.654139+0000","last_undegraded":"2024-03-22T04:16:58.654139+0000","last_fullsized":"2024-03-22T04:16:58.654139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:12.066597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646061+0000","last_change":"2024-03-22T04:16:58.677326+0000","last_active":"2024-03-22T04:16:59.646061+0000","last_peered":"2024-03-22T04:16:59.646061+0000","last_clean":"2024-03-22T04:16:59.646061+0000","last_became_active":"2024-03-22T04:16:58.677019+0000","last_became_peered":"2024-03-22T04:16:58.677019+0000","last_unstale":"2024-03-22T04:16:59.646061+0000","last_undegraded":"2024-03-22T04:16:59.646061+0000","last_fullsized":"2024-03-22T04:16:59.646061+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:32:09.746284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883026+0000","last_change":"2024-03-22T04:16:57.652413+0000","last_active":"2024-03-22T04:16:59.883026+0000","last_peered":"2024-03-22T04:16:59.883026+0000","last_clean":"2024-03-22T04:16:59.883026+0000","last_became_active":"2024-03-22T04:16:57.648859+0000","last_became_peered":"2024-03-22T04:16:57.648859+0000","last_unstale":"2024-03-22T04:16:59.883026+0000","last_undegraded":"2024-03-22T04:16:59.883026+0000","last_fullsized":"2024-03-22T04:16:59.883026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:41:05.754331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646945+0000","last_change":"2024-03-22T04:16:58.676696+0000","last_active":"2024-03-22T04:16:59.646945+0000","last_peered":"2024-03-22T04:16:59.646945+0000","last_clean":"2024-03-22T04:16:59.646945+0000","last_became_active":"2024-03-22T04:16:58.676428+0000","last_became_peered":"2024-03-22T04:16:58.676428+0000","last_unstale":"2024-03-22T04:16:59.646945+0000","last_undegraded":"2024-03-22T04:16:59.646945+0000","last_fullsized":"2024-03-22T04:16:59.646945+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:15.690412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"92'1","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.679827+0000","last_change":"2024-03-22T04:16:57.643848+0000","last_active":"2024-03-22T04:16:58.679827+0000","last_peered":"2024-03-22T04:16:58.679827+0000","last_clean":"2024-03-22T04:16:58.679827+0000","last_became_active":"2024-03-22T04:16:57.643578+0000","last_became_peered":"2024-03-22T04:16:57.643578+0000","last_unstale":"2024-03-22T04:16:58.679827+0000","last_undegraded":"2024-03-22T04:16:58.679827+0000","last_fullsized":"2024-03-22T04:16:58.679827+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:24.274884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668428+0000","last_change":"2024-03-22T04:16:58.652141+0000","last_active":"2024-03-22T04:16:59.668428+0000","last_peered":"2024-03-22T04:16:59.668428+0000","last_clean":"2024-03-22T04:16:59.668428+0000","last_became_active":"2024-03-22T04:16:58.651862+0000","last_became_peered":"2024-03-22T04:16:58.651862+0000","last_unstale":"2024-03-22T04:16:59.668428+0000","last_undegraded":"2024-03-22T04:16:59.668428+0000","last_fullsized":"2024-03-22T04:16:59.668428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:49:25.282105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646912+0000","last_change":"2024-03-22T04:16:57.653086+0000","last_active":"2024-03-22T04:16:59.646912+0000","last_peered":"2024-03-22T04:16:59.646912+0000","last_clean":"2024-03-22T04:16:59.646912+0000","last_became_active":"2024-03-22T04:16:57.652784+0000","last_became_peered":"2024-03-22T04:16:57.652784+0000","last_unstale":"2024-03-22T04:16:59.646912+0000","last_undegraded":"2024-03-22T04:16:59.646912+0000","last_fullsized":"2024-03-22T04:16:59.646912+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:26:38.381239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671155+0000","last_change":"2024-03-22T04:16:58.681137+0000","last_active":"2024-03-22T04:16:59.671155+0000","last_peered":"2024-03-22T04:16:59.671155+0000","last_clean":"2024-03-22T04:16:59.671155+0000","last_became_active":"2024-03-22T04:16:58.680771+0000","last_became_peered":"2024-03-22T04:16:58.680771+0000","last_unstale":"2024-03-22T04:16:59.671155+0000","last_undegraded":"2024-03-22T04:16:59.671155+0000","last_fullsized":"2024-03-22T04:16:59.671155+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:17.742365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646897+0000","last_change":"2024-03-22T04:16:57.653718+0000","last_active":"2024-03-22T04:16:59.646897+0000","last_peered":"2024-03-22T04:16:59.646897+0000","last_clean":"2024-03-22T04:16:59.646897+0000","last_became_active":"2024-03-22T04:16:57.653417+0000","last_became_peered":"2024-03-22T04:16:57.653417+0000","last_unstale":"2024-03-22T04:16:59.646897+0000","last_undegraded":"2024-03-22T04:16:59.646897+0000","last_fullsized":"2024-03-22T04:16:59.646897+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:41:15.516063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.683878+0000","last_change":"2024-03-22T04:16:58.679301+0000","last_active":"2024-03-22T04:16:58.683878+0000","last_peered":"2024-03-22T04:16:58.683878+0000","last_clean":"2024-03-22T04:16:58.683878+0000","last_became_active":"2024-03-22T04:16:58.677934+0000","last_became_peered":"2024-03-22T04:16:58.677934+0000","last_unstale":"2024-03-22T04:16:58.683878+0000","last_undegraded":"2024-03-22T04:16:58.683878+0000","last_fullsized":"2024-03-22T04:16:58.683878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:37.851619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668075+0000","last_change":"2024-03-22T04:16:57.651216+0000","last_active":"2024-03-22T04:16:59.668075+0000","last_peered":"2024-03-22T04:16:59.668075+0000","last_clean":"2024-03-22T04:16:59.668075+0000","last_became_active":"2024-03-22T04:16:57.650912+0000","last_became_peered":"2024-03-22T04:16:57.650912+0000","last_unstale":"2024-03-22T04:16:59.668075+0000","last_undegraded":"2024-03-22T04:16:59.668075+0000","last_fullsized":"2024-03-22T04:16:59.668075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:23:35.018881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646763+0000","last_change":"2024-03-22T04:16:58.658059+0000","last_active":"2024-03-22T04:16:59.646763+0000","last_peered":"2024-03-22T04:16:59.646763+0000","last_clean":"2024-03-22T04:16:59.646763+0000","last_became_active":"2024-03-22T04:16:58.657660+0000","last_became_peered":"2024-03-22T04:16:58.657660+0000","last_unstale":"2024-03-22T04:16:59.646763+0000","last_undegraded":"2024-03-22T04:16:59.646763+0000","last_fullsized":"2024-03-22T04:16:59.646763+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:25.281798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.650208+0000","last_change":"2024-03-22T04:16:57.650680+0000","last_active":"2024-03-22T04:16:58.650208+0000","last_peered":"2024-03-22T04:16:58.650208+0000","last_clean":"2024-03-22T04:16:58.650208+0000","last_became_active":"2024-03-22T04:16:57.650110+0000","last_became_peered":"2024-03-22T04:16:57.650110+0000","last_unstale":"2024-03-22T04:16:58.650208+0000","last_undegraded":"2024-03-22T04:16:58.650208+0000","last_fullsized":"2024-03-22T04:16:58.650208+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:06:11.062191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.679878+0000","last_change":"2024-03-22T04:16:58.672823+0000","last_active":"2024-03-22T04:16:58.679878+0000","last_peered":"2024-03-22T04:16:58.679878+0000","last_clean":"2024-03-22T04:16:58.679878+0000","last_became_active":"2024-03-22T04:16:58.672484+0000","last_became_peered":"2024-03-22T04:16:58.672484+0000","last_unstale":"2024-03-22T04:16:58.679878+0000","last_undegraded":"2024-03-22T04:16:58.679878+0000","last_fullsized":"2024-03-22T04:16:58.679878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:46:51.882426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646864+0000","last_change":"2024-03-22T04:16:57.653180+0000","last_active":"2024-03-22T04:16:59.646864+0000","last_peered":"2024-03-22T04:16:59.646864+0000","last_clean":"2024-03-22T04:16:59.646864+0000","last_became_active":"2024-03-22T04:16:57.652212+0000","last_became_peered":"2024-03-22T04:16:57.652212+0000","last_unstale":"2024-03-22T04:16:59.646864+0000","last_undegraded":"2024-03-22T04:16:59.646864+0000","last_fullsized":"2024-03-22T04:16:59.646864+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:04.318404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.677571+0000","last_change":"2024-03-22T04:16:58.670538+0000","last_active":"2024-03-22T04:16:58.677571+0000","last_peered":"2024-03-22T04:16:58.677571+0000","last_clean":"2024-03-22T04:16:58.677571+0000","last_became_active":"2024-03-22T04:16:58.668017+0000","last_became_peered":"2024-03-22T04:16:58.668017+0000","last_unstale":"2024-03-22T04:16:58.677571+0000","last_undegraded":"2024-03-22T04:16:58.677571+0000","last_fullsized":"2024-03-22T04:16:58.677571+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:50.842967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.645899+0000","last_change":"2024-03-22T04:16:57.651390+0000","last_active":"2024-03-22T04:16:59.645899+0000","last_peered":"2024-03-22T04:16:59.645899+0000","last_clean":"2024-03-22T04:16:59.645899+0000","last_became_active":"2024-03-22T04:16:57.651089+0000","last_became_peered":"2024-03-22T04:16:57.651089+0000","last_unstale":"2024-03-22T04:16:59.645899+0000","last_undegraded":"2024-03-22T04:16:59.645899+0000","last_fullsized":"2024-03-22T04:16:59.645899+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:41:15.516063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.659686+0000","last_change":"2024-03-22T04:16:58.655696+0000","last_active":"2024-03-22T04:16:58.659686+0000","last_peered":"2024-03-22T04:16:58.659686+0000","last_clean":"2024-03-22T04:16:58.659686+0000","last_became_active":"2024-03-22T04:16:58.655451+0000","last_became_peered":"2024-03-22T04:16:58.655451+0000","last_unstale":"2024-03-22T04:16:58.659686+0000","last_undegraded":"2024-03-22T04:16:58.659686+0000","last_fullsized":"2024-03-22T04:16:58.659686+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:42.681464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668339+0000","last_change":"2024-03-22T04:16:57.645491+0000","last_active":"2024-03-22T04:16:59.668339+0000","last_peered":"2024-03-22T04:16:59.668339+0000","last_clean":"2024-03-22T04:16:59.668339+0000","last_became_active":"2024-03-22T04:16:57.645046+0000","last_became_peered":"2024-03-22T04:16:57.645046+0000","last_unstale":"2024-03-22T04:16:59.668339+0000","last_undegraded":"2024-03-22T04:16:59.668339+0000","last_fullsized":"2024-03-22T04:16:59.668339+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:32:51.874035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.645852+0000","last_change":"2024-03-22T04:16:58.672726+0000","last_active":"2024-03-22T04:16:59.645852+0000","last_peered":"2024-03-22T04:16:59.645852+0000","last_clean":"2024-03-22T04:16:59.645852+0000","last_became_active":"2024-03-22T04:16:58.672394+0000","last_became_peered":"2024-03-22T04:16:58.672394+0000","last_unstale":"2024-03-22T04:16:59.645852+0000","last_undegraded":"2024-03-22T04:16:59.645852+0000","last_fullsized":"2024-03-22T04:16:59.645852+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:17.742365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"93'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670055+0000","last_change":"2024-03-22T04:16:57.655846+0000","last_active":"2024-03-22T04:16:59.670055+0000","last_peered":"2024-03-22T04:16:59.670055+0000","last_clean":"2024-03-22T04:16:59.670055+0000","last_became_active":"2024-03-22T04:16:57.655554+0000","last_became_peered":"2024-03-22T04:16:57.655554+0000","last_unstale":"2024-03-22T04:16:59.670055+0000","last_undegraded":"2024-03-22T04:16:59.670055+0000","last_fullsized":"2024-03-22T04:16:59.670055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:12:52.827841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646420+0000","last_change":"2024-03-22T04:16:58.673870+0000","last_active":"2024-03-22T04:16:59.646420+0000","last_peered":"2024-03-22T04:16:59.646420+0000","last_clean":"2024-03-22T04:16:59.646420+0000","last_became_active":"2024-03-22T04:16:58.673589+0000","last_became_peered":"2024-03-22T04:16:58.673589+0000","last_unstale":"2024-03-22T04:16:59.646420+0000","last_undegraded":"2024-03-22T04:16:59.646420+0000","last_fullsized":"2024-03-22T04:16:59.646420+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:50.842967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.651097+0000","last_change":"2024-03-22T04:16:57.642001+0000","last_active":"2024-03-22T04:16:58.651097+0000","last_peered":"2024-03-22T04:16:58.651097+0000","last_clean":"2024-03-22T04:16:58.651097+0000","last_became_active":"2024-03-22T04:16:57.641733+0000","last_became_peered":"2024-03-22T04:16:57.641733+0000","last_unstale":"2024-03-22T04:16:58.651097+0000","last_undegraded":"2024-03-22T04:16:58.651097+0000","last_fullsized":"2024-03-22T04:16:58.651097+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:56.718448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671304+0000","last_change":"2024-03-22T04:16:58.655490+0000","last_active":"2024-03-22T04:16:59.671304+0000","last_peered":"2024-03-22T04:16:59.671304+0000","last_clean":"2024-03-22T04:16:59.671304+0000","last_became_active":"2024-03-22T04:16:58.655231+0000","last_became_peered":"2024-03-22T04:16:58.655231+0000","last_unstale":"2024-03-22T04:16:59.671304+0000","last_undegraded":"2024-03-22T04:16:59.671304+0000","last_fullsized":"2024-03-22T04:16:59.671304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:46.047388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668342+0000","last_change":"2024-03-22T04:16:57.652632+0000","last_active":"2024-03-22T04:16:59.668342+0000","last_peered":"2024-03-22T04:16:59.668342+0000","last_clean":"2024-03-22T04:16:59.668342+0000","last_became_active":"2024-03-22T04:16:57.652376+0000","last_became_peered":"2024-03-22T04:16:57.652376+0000","last_unstale":"2024-03-22T04:16:59.668342+0000","last_undegraded":"2024-03-22T04:16:59.668342+0000","last_fullsized":"2024-03-22T04:16:59.668342+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:01:27.317630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.681179+0000","last_change":"2024-03-22T04:16:58.677945+0000","last_active":"2024-03-22T04:16:58.681179+0000","last_peered":"2024-03-22T04:16:58.681179+0000","last_clean":"2024-03-22T04:16:58.681179+0000","last_became_active":"2024-03-22T04:16:58.677398+0000","last_became_peered":"2024-03-22T04:16:58.677398+0000","last_unstale":"2024-03-22T04:16:58.681179+0000","last_undegraded":"2024-03-22T04:16:58.681179+0000","last_fullsized":"2024-03-22T04:16:58.681179+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:38:17.444912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.655423+0000","last_change":"2024-03-22T04:16:57.652602+0000","last_active":"2024-03-22T04:16:58.655423+0000","last_peered":"2024-03-22T04:16:58.655423+0000","last_clean":"2024-03-22T04:16:58.655423+0000","last_became_active":"2024-03-22T04:16:57.652261+0000","last_became_peered":"2024-03-22T04:16:57.652261+0000","last_unstale":"2024-03-22T04:16:58.655423+0000","last_undegraded":"2024-03-22T04:16:58.655423+0000","last_fullsized":"2024-03-22T04:16:58.655423+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:24.274884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883166+0000","last_change":"2024-03-22T04:16:58.669636+0000","last_active":"2024-03-22T04:16:59.883166+0000","last_peered":"2024-03-22T04:16:59.883166+0000","last_clean":"2024-03-22T04:16:59.883166+0000","last_became_active":"2024-03-22T04:16:58.669366+0000","last_became_peered":"2024-03-22T04:16:58.669366+0000","last_unstale":"2024-03-22T04:16:59.883166+0000","last_undegraded":"2024-03-22T04:16:59.883166+0000","last_fullsized":"2024-03-22T04:16:59.883166+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:59:54.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670880+0000","last_change":"2024-03-22T04:16:57.658951+0000","last_active":"2024-03-22T04:16:59.670880+0000","last_peered":"2024-03-22T04:16:59.670880+0000","last_clean":"2024-03-22T04:16:59.670880+0000","last_became_active":"2024-03-22T04:16:57.658673+0000","last_became_peered":"2024-03-22T04:16:57.658673+0000","last_unstale":"2024-03-22T04:16:59.670880+0000","last_undegraded":"2024-03-22T04:16:59.670880+0000","last_fullsized":"2024-03-22T04:16:59.670880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:02.561260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.679958+0000","last_change":"2024-03-22T04:16:58.673445+0000","last_active":"2024-03-22T04:16:58.679958+0000","last_peered":"2024-03-22T04:16:58.679958+0000","last_clean":"2024-03-22T04:16:58.679958+0000","last_became_active":"2024-03-22T04:16:58.673195+0000","last_became_peered":"2024-03-22T04:16:58.673195+0000","last_unstale":"2024-03-22T04:16:58.679958+0000","last_undegraded":"2024-03-22T04:16:58.679958+0000","last_fullsized":"2024-03-22T04:16:58.679958+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:32:09.746284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670864+0000","last_change":"2024-03-22T04:16:57.654514+0000","last_active":"2024-03-22T04:16:59.670864+0000","last_peered":"2024-03-22T04:16:59.670864+0000","last_clean":"2024-03-22T04:16:59.670864+0000","last_became_active":"2024-03-22T04:16:57.654071+0000","last_became_peered":"2024-03-22T04:16:57.654071+0000","last_unstale":"2024-03-22T04:16:59.670864+0000","last_undegraded":"2024-03-22T04:16:59.670864+0000","last_fullsized":"2024-03-22T04:16:59.670864+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:07:31.066617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680864+0000","last_change":"2024-03-22T04:16:58.671784+0000","last_active":"2024-03-22T04:16:58.680864+0000","last_peered":"2024-03-22T04:16:58.680864+0000","last_clean":"2024-03-22T04:16:58.680864+0000","last_became_active":"2024-03-22T04:16:58.671416+0000","last_became_peered":"2024-03-22T04:16:58.671416+0000","last_unstale":"2024-03-22T04:16:58.680864+0000","last_undegraded":"2024-03-22T04:16:58.680864+0000","last_fullsized":"2024-03-22T04:16:58.680864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:35:28.012150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670691+0000","last_change":"2024-03-22T04:16:57.659423+0000","last_active":"2024-03-22T04:16:59.670691+0000","last_peered":"2024-03-22T04:16:59.670691+0000","last_clean":"2024-03-22T04:16:59.670691+0000","last_became_active":"2024-03-22T04:16:57.659167+0000","last_became_peered":"2024-03-22T04:16:57.659167+0000","last_unstale":"2024-03-22T04:16:59.670691+0000","last_undegraded":"2024-03-22T04:16:59.670691+0000","last_fullsized":"2024-03-22T04:16:59.670691+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:03:50.782095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671213+0000","last_change":"2024-03-22T04:16:58.679379+0000","last_active":"2024-03-22T04:16:59.671213+0000","last_peered":"2024-03-22T04:16:59.671213+0000","last_clean":"2024-03-22T04:16:59.671213+0000","last_became_active":"2024-03-22T04:16:58.679062+0000","last_became_peered":"2024-03-22T04:16:58.679062+0000","last_unstale":"2024-03-22T04:16:59.671213+0000","last_undegraded":"2024-03-22T04:16:59.671213+0000","last_fullsized":"2024-03-22T04:16:59.671213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:32:09.746284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.662139+0000","last_change":"2024-03-22T04:16:57.657743+0000","last_active":"2024-03-22T04:16:58.662139+0000","last_peered":"2024-03-22T04:16:58.662139+0000","last_clean":"2024-03-22T04:16:58.662139+0000","last_became_active":"2024-03-22T04:16:57.657446+0000","last_became_peered":"2024-03-22T04:16:57.657446+0000","last_unstale":"2024-03-22T04:16:58.662139+0000","last_undegraded":"2024-03-22T04:16:58.662139+0000","last_fullsized":"2024-03-22T04:16:58.662139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:59.159476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":302,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670774+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:16:59.670774+0000","last_peered":"2024-03-22T04:16:59.670774+0000","last_clean":"2024-03-22T04:16:59.670774+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:16:59.670774+0000","last_undegraded":"2024-03-22T04:16:59.670774+0000","last_fullsized":"2024-03-22T04:16:59.670774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509523,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":522395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":517439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1732608,"data_stored":1112763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":524119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509523,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":516307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1744896,"data_stored":1113449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509523,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":522465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":515703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:17:02.898 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:17:03.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1cd600640 1 Processor -- start 2024-03-22T04:17:03.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1cd600640 1 -- start start 2024-03-22T04:17:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1cd600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 0x7fa1c80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1cd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1c80cc3e0 0x7fa1c80ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1cd600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c8071920 con 0x7fa1c815b3f0 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1cd600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c80716d0 con 0x7fa1c80cc3e0 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1cd600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c806d990 con 0x7fa1c80730d0 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1c7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1c80cc3e0 0x7fa1c80ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1c6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa1c815b3f0 0x7fa1c80729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39434/0 (socket says 172.21.15.86:39434) 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1c6a00640 1 -- 172.21.15.86:0/2794397739 learned_addr learned my addr 172.21.15.86:0/2794397739 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.074+0000 7fa1c6000640 1 --2- 172.21.15.86:0/2794397739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 0x7fa1c80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c7400640 1 -- 172.21.15.86:0/2794397739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 msgr2=0x7fa1c80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c7400640 1 --2- 172.21.15.86:0/2794397739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 0x7fa1c80ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c7400640 1 -- 172.21.15.86:0/2794397739 >> v1:172.21.15.86:6789/0 conn(0x7fa1c815b3f0 legacy=0x7fa1c80729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c7400640 1 -- 172.21.15.86:0/2794397739 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c8071c00 con 0x7fa1c80cc3e0 2024-03-22T04:17:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c5600640 1 -- 172.21.15.86:0/2794397739 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250274812 0 0) 0x7fa1c8071920 con 0x7fa1c815b3f0 2024-03-22T04:17:03.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c7400640 1 --2- 172.21.15.86:0/2794397739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c80cc3e0 0x7fa1c80ae370 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009d90 tx=0x7fa1c001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f76f97d9a4d54c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c5600640 1 -- 172.21.15.86:0/2794397739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00b2070 con 0x7fa1c80cc3e0 2024-03-22T04:17:03.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c5600640 1 -- 172.21.15.86:0/2794397739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c00a5920 con 0x7fa1c80cc3e0 2024-03-22T04:17:03.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1c5600640 1 -- 172.21.15.86:0/2794397739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00a5cf0 con 0x7fa1c80cc3e0 2024-03-22T04:17:03.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1cd600640 1 -- 172.21.15.86:0/2794397739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c80cc3e0 msgr2=0x7fa1c80ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1cd600640 1 --2- 172.21.15.86:0/2794397739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c80cc3e0 0x7fa1c80ae370 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009d90 tx=0x7fa1c001c970 comp rx=0 tx=0).stop 2024-03-22T04:17:03.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1cd600640 1 -- 172.21.15.86:0/2794397739 shutdown_connections 2024-03-22T04:17:03.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1cd600640 1 --2- 172.21.15.86:0/2794397739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c80cc3e0 0x7fa1c80ae370 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1cd600640 1 --2- 172.21.15.86:0/2794397739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 0x7fa1c80ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.078+0000 7fa1cd600640 1 -- 172.21.15.86:0/2794397739 >> 172.21.15.86:0/2794397739 conn(0x7fa1c80b1df0 msgr2=0x7fa1c80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:03.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 -- 172.21.15.86:0/2794397739 shutdown_connections 2024-03-22T04:17:03.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 -- 172.21.15.86:0/2794397739 wait complete. 2024-03-22T04:17:03.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 Processor -- start 2024-03-22T04:17:03.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 -- start start 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 0x7fa1c8146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c80cc3e0 0x7fa1c8159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c815b3f0 0x7fa1c809a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c806e2d0 con 0x7fa1c815b3f0 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c806e150 con 0x7fa1c80cc3e0 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1cd600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c806e450 con 0x7fa1c80730d0 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c815b3f0 0x7fa1c809a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c6a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 0x7fa1c8146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c815b3f0 0x7fa1c809a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49926/0 (socket says 172.21.15.86:49926) 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c6000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c80cc3e0 0x7fa1c8159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 -- 172.21.15.86:0/1123688306 learned_addr learned my addr 172.21.15.86:0/1123688306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:03.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 -- 172.21.15.86:0/1123688306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 msgr2=0x7fa1c8146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 0x7fa1c8146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 -- 172.21.15.86:0/1123688306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c80cc3e0 msgr2=0x7fa1c8159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c80cc3e0 0x7fa1c8159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 -- 172.21.15.86:0/1123688306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c81520a0 con 0x7fa1c815b3f0 2024-03-22T04:17:03.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.082+0000 7fa1c7400640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c815b3f0 0x7fa1c809a1e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00b0040 tx=0x7fa1c0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1bea00640 1 -- 172.21.15.86:0/1123688306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00b2050 con 0x7fa1c815b3f0 2024-03-22T04:17:03.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1bea00640 1 -- 172.21.15.86:0/1123688306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c00be070 con 0x7fa1c815b3f0 2024-03-22T04:17:03.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1bea00640 1 -- 172.21.15.86:0/1123688306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00ada90 con 0x7fa1c815b3f0 2024-03-22T04:17:03.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c8152380 con 0x7fa1c815b3f0 2024-03-22T04:17:03.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c8159110 con 0x7fa1c815b3f0 2024-03-22T04:17:03.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c8068650 con 0x7fa1c815b3f0 2024-03-22T04:17:03.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1bea00640 1 -- 172.21.15.86:0/1123688306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa1c00adc30 con 0x7fa1c815b3f0 2024-03-22T04:17:03.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1bea00640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa1a8067b20 0x7fa1a8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1bea00640 1 -- 172.21.15.86:0/1123688306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fa1c01211e0 con 0x7fa1c815b3f0 2024-03-22T04:17:03.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.086+0000 7fa1c6a00640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa1a8067b20 0x7fa1a8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.090+0000 7fa1c6a00640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa1a8067b20 0x7fa1a8069fe0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8000f70 tx=0x7fa1b8007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.090+0000 7fa1bea00640 1 -- 172.21.15.86:0/1123688306 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa1c0016560 con 0x7fa1c815b3f0 2024-03-22T04:17:03.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.242+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa1c8073700 con 0x7fa1a8067b20 2024-03-22T04:17:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.250+0000 7fa1bea00640 1 -- 172.21.15.86:0/1123688306 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252047 (secure 0 0 0) 0x7fa1c8073700 con 0x7fa1a8067b20 2024-03-22T04:17:03.252 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:03.253 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:17:03.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa1a8067b20 msgr2=0x7fa1a8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa1a8067b20 0x7fa1a8069fe0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8000f70 tx=0x7fa1b8007df0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c815b3f0 msgr2=0x7fa1c809a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c815b3f0 0x7fa1c809a1e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00b0040 tx=0x7fa1c0019710 comp rx=0 tx=0).stop 2024-03-22T04:17:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 shutdown_connections 2024-03-22T04:17:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa1a8067b20 0x7fa1a8069fe0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c80cc3e0 0x7fa1c8159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1c815b3f0 0x7fa1c809a1e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 --2- 172.21.15.86:0/1123688306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c80730d0 0x7fa1c8146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 >> 172.21.15.86:0/1123688306 conn(0x7fa1c80b1df0 msgr2=0x7fa1c815be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 shutdown_connections 2024-03-22T04:17:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.254+0000 7fa1cd600640 1 -- 172.21.15.86:0/1123688306 wait complete. 2024-03-22T04:17:03.272 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-03-22T04:17:01.874119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259540,"kb_used_data":10084,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715180,"statfs":{"total":773094113280,"available":772828344320,"internally_reserved":0,"allocated":10326016,"data_stored":5344640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.247105"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.685929+0000","last_change":"2024-03-22T04:16:58.682586+0000","last_active":"2024-03-22T04:16:58.685929+0000","last_peered":"2024-03-22T04:16:58.685929+0000","last_clean":"2024-03-22T04:16:58.685929+0000","last_became_active":"2024-03-22T04:16:58.682211+0000","last_became_peered":"2024-03-22T04:16:58.682211+0000","last_unstale":"2024-03-22T04:16:58.685929+0000","last_undegraded":"2024-03-22T04:16:58.685929+0000","last_fullsized":"2024-03-22T04:16:58.685929+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:15.690412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671072+0000","last_change":"2024-03-22T04:16:58.670276+0000","last_active":"2024-03-22T04:16:59.671072+0000","last_peered":"2024-03-22T04:16:59.671072+0000","last_clean":"2024-03-22T04:16:59.671072+0000","last_became_active":"2024-03-22T04:16:58.670012+0000","last_became_peered":"2024-03-22T04:16:58.670012+0000","last_unstale":"2024-03-22T04:16:59.671072+0000","last_undegraded":"2024-03-22T04:16:59.671072+0000","last_fullsized":"2024-03-22T04:16:59.671072+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:02.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883386+0000","last_change":"2024-03-22T04:16:58.659168+0000","last_active":"2024-03-22T04:16:59.883386+0000","last_peered":"2024-03-22T04:16:59.883386+0000","last_clean":"2024-03-22T04:16:59.883386+0000","last_became_active":"2024-03-22T04:16:58.657030+0000","last_became_peered":"2024-03-22T04:16:58.657030+0000","last_unstale":"2024-03-22T04:16:59.883386+0000","last_undegraded":"2024-03-22T04:16:59.883386+0000","last_fullsized":"2024-03-22T04:16:59.883386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:32:54.102100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883244+0000","last_change":"2024-03-22T04:16:58.657060+0000","last_active":"2024-03-22T04:16:59.883244+0000","last_peered":"2024-03-22T04:16:59.883244+0000","last_clean":"2024-03-22T04:16:59.883244+0000","last_became_active":"2024-03-22T04:16:58.656743+0000","last_became_peered":"2024-03-22T04:16:58.656743+0000","last_unstale":"2024-03-22T04:16:59.883244+0000","last_undegraded":"2024-03-22T04:16:59.883244+0000","last_fullsized":"2024-03-22T04:16:59.883244+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:15:55.957855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670885+0000","last_change":"2024-03-22T04:16:58.652736+0000","last_active":"2024-03-22T04:16:59.670885+0000","last_peered":"2024-03-22T04:16:59.670885+0000","last_clean":"2024-03-22T04:16:59.670885+0000","last_became_active":"2024-03-22T04:16:58.652273+0000","last_became_peered":"2024-03-22T04:16:58.652273+0000","last_unstale":"2024-03-22T04:16:59.670885+0000","last_undegraded":"2024-03-22T04:16:59.670885+0000","last_fullsized":"2024-03-22T04:16:59.670885+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:57.725363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680957+0000","last_change":"2024-03-22T04:16:58.672863+0000","last_active":"2024-03-22T04:16:58.680957+0000","last_peered":"2024-03-22T04:16:58.680957+0000","last_clean":"2024-03-22T04:16:58.680957+0000","last_became_active":"2024-03-22T04:16:58.672571+0000","last_became_peered":"2024-03-22T04:16:58.672571+0000","last_unstale":"2024-03-22T04:16:58.680957+0000","last_undegraded":"2024-03-22T04:16:58.680957+0000","last_fullsized":"2024-03-22T04:16:58.680957+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:53:42.876217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668164+0000","last_change":"2024-03-22T04:16:58.658200+0000","last_active":"2024-03-22T04:16:59.668164+0000","last_peered":"2024-03-22T04:16:59.668164+0000","last_clean":"2024-03-22T04:16:59.668164+0000","last_became_active":"2024-03-22T04:16:58.657898+0000","last_became_peered":"2024-03-22T04:16:58.657898+0000","last_unstale":"2024-03-22T04:16:59.668164+0000","last_undegraded":"2024-03-22T04:16:59.668164+0000","last_fullsized":"2024-03-22T04:16:59.668164+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:41:20.555405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.667484+0000","last_change":"2024-03-22T04:16:58.657768+0000","last_active":"2024-03-22T04:16:58.667484+0000","last_peered":"2024-03-22T04:16:58.667484+0000","last_clean":"2024-03-22T04:16:58.667484+0000","last_became_active":"2024-03-22T04:16:58.657475+0000","last_became_peered":"2024-03-22T04:16:58.657475+0000","last_unstale":"2024-03-22T04:16:58.667484+0000","last_undegraded":"2024-03-22T04:16:58.667484+0000","last_fullsized":"2024-03-22T04:16:58.667484+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:35.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668260+0000","last_change":"2024-03-22T04:16:58.656824+0000","last_active":"2024-03-22T04:16:59.668260+0000","last_peered":"2024-03-22T04:16:59.668260+0000","last_clean":"2024-03-22T04:16:59.668260+0000","last_became_active":"2024-03-22T04:16:58.656574+0000","last_became_peered":"2024-03-22T04:16:58.656574+0000","last_unstale":"2024-03-22T04:16:59.668260+0000","last_undegraded":"2024-03-22T04:16:59.668260+0000","last_fullsized":"2024-03-22T04:16:59.668260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:03:38.621750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883479+0000","last_change":"2024-03-22T04:16:58.666381+0000","last_active":"2024-03-22T04:16:59.883479+0000","last_peered":"2024-03-22T04:16:59.883479+0000","last_clean":"2024-03-22T04:16:59.883479+0000","last_became_active":"2024-03-22T04:16:58.666126+0000","last_became_peered":"2024-03-22T04:16:58.666126+0000","last_unstale":"2024-03-22T04:16:59.883479+0000","last_undegraded":"2024-03-22T04:16:59.883479+0000","last_fullsized":"2024-03-22T04:16:59.883479+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:50:13.717561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668203+0000","last_change":"2024-03-22T04:16:58.655957+0000","last_active":"2024-03-22T04:16:59.668203+0000","last_peered":"2024-03-22T04:16:59.668203+0000","last_clean":"2024-03-22T04:16:59.668203+0000","last_became_active":"2024-03-22T04:16:58.655689+0000","last_became_peered":"2024-03-22T04:16:58.655689+0000","last_unstale":"2024-03-22T04:16:59.668203+0000","last_undegraded":"2024-03-22T04:16:59.668203+0000","last_fullsized":"2024-03-22T04:16:59.668203+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:21:06.716490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883252+0000","last_change":"2024-03-22T04:16:58.679894+0000","last_active":"2024-03-22T04:16:59.883252+0000","last_peered":"2024-03-22T04:16:59.883252+0000","last_clean":"2024-03-22T04:16:59.883252+0000","last_became_active":"2024-03-22T04:16:58.679537+0000","last_became_peered":"2024-03-22T04:16:58.679537+0000","last_unstale":"2024-03-22T04:16:59.883252+0000","last_undegraded":"2024-03-22T04:16:59.883252+0000","last_fullsized":"2024-03-22T04:16:59.883252+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:37:47.891796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.645910+0000","last_change":"2024-03-22T04:16:58.674007+0000","last_active":"2024-03-22T04:16:59.645910+0000","last_peered":"2024-03-22T04:16:59.645910+0000","last_clean":"2024-03-22T04:16:59.645910+0000","last_became_active":"2024-03-22T04:16:58.673679+0000","last_became_peered":"2024-03-22T04:16:58.673679+0000","last_unstale":"2024-03-22T04:16:59.645910+0000","last_undegraded":"2024-03-22T04:16:59.645910+0000","last_fullsized":"2024-03-22T04:16:59.645910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:46:51.882426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646789+0000","last_change":"2024-03-22T04:16:58.671335+0000","last_active":"2024-03-22T04:16:59.646789+0000","last_peered":"2024-03-22T04:16:59.646789+0000","last_clean":"2024-03-22T04:16:59.646789+0000","last_became_active":"2024-03-22T04:16:58.671015+0000","last_became_peered":"2024-03-22T04:16:58.671015+0000","last_unstale":"2024-03-22T04:16:59.646789+0000","last_undegraded":"2024-03-22T04:16:59.646789+0000","last_fullsized":"2024-03-22T04:16:59.646789+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:57.725363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883094+0000","last_change":"2024-03-22T04:16:58.666983+0000","last_active":"2024-03-22T04:16:59.883094+0000","last_peered":"2024-03-22T04:16:59.883094+0000","last_clean":"2024-03-22T04:16:59.883094+0000","last_became_active":"2024-03-22T04:16:58.665728+0000","last_became_peered":"2024-03-22T04:16:58.665728+0000","last_unstale":"2024-03-22T04:16:59.883094+0000","last_undegraded":"2024-03-22T04:16:59.883094+0000","last_fullsized":"2024-03-22T04:16:59.883094+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:48:05.551943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680975+0000","last_change":"2024-03-22T04:16:58.673197+0000","last_active":"2024-03-22T04:16:58.680975+0000","last_peered":"2024-03-22T04:16:58.680975+0000","last_clean":"2024-03-22T04:16:58.680975+0000","last_became_active":"2024-03-22T04:16:58.672660+0000","last_became_peered":"2024-03-22T04:16:58.672660+0000","last_unstale":"2024-03-22T04:16:58.680975+0000","last_undegraded":"2024-03-22T04:16:58.680975+0000","last_fullsized":"2024-03-22T04:16:58.680975+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:57:28.357635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.685598+0000","last_change":"2024-03-22T04:16:58.681033+0000","last_active":"2024-03-22T04:16:58.685598+0000","last_peered":"2024-03-22T04:16:58.685598+0000","last_clean":"2024-03-22T04:16:58.685598+0000","last_became_active":"2024-03-22T04:16:58.680675+0000","last_became_peered":"2024-03-22T04:16:58.680675+0000","last_unstale":"2024-03-22T04:16:58.685598+0000","last_undegraded":"2024-03-22T04:16:58.685598+0000","last_fullsized":"2024-03-22T04:16:58.685598+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:13:56.482754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883548+0000","last_change":"2024-03-22T04:16:58.659370+0000","last_active":"2024-03-22T04:16:59.883548+0000","last_peered":"2024-03-22T04:16:59.883548+0000","last_clean":"2024-03-22T04:16:59.883548+0000","last_became_active":"2024-03-22T04:16:58.659040+0000","last_became_peered":"2024-03-22T04:16:58.659040+0000","last_unstale":"2024-03-22T04:16:59.883548+0000","last_undegraded":"2024-03-22T04:16:59.883548+0000","last_fullsized":"2024-03-22T04:16:59.883548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:26:42.516224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.677363+0000","last_change":"2024-03-22T04:16:58.672002+0000","last_active":"2024-03-22T04:16:58.677363+0000","last_peered":"2024-03-22T04:16:58.677363+0000","last_clean":"2024-03-22T04:16:58.677363+0000","last_became_active":"2024-03-22T04:16:58.671740+0000","last_became_peered":"2024-03-22T04:16:58.671740+0000","last_unstale":"2024-03-22T04:16:58.677363+0000","last_undegraded":"2024-03-22T04:16:58.677363+0000","last_fullsized":"2024-03-22T04:16:58.677363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:17.742365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.681061+0000","last_change":"2024-03-22T04:16:58.674240+0000","last_active":"2024-03-22T04:16:58.681061+0000","last_peered":"2024-03-22T04:16:58.681061+0000","last_clean":"2024-03-22T04:16:58.681061+0000","last_became_active":"2024-03-22T04:16:58.673979+0000","last_became_peered":"2024-03-22T04:16:58.673979+0000","last_unstale":"2024-03-22T04:16:58.681061+0000","last_undegraded":"2024-03-22T04:16:58.681061+0000","last_fullsized":"2024-03-22T04:16:58.681061+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:10:27.288562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671045+0000","last_change":"2024-03-22T04:16:58.657152+0000","last_active":"2024-03-22T04:16:59.671045+0000","last_peered":"2024-03-22T04:16:59.671045+0000","last_clean":"2024-03-22T04:16:59.671045+0000","last_became_active":"2024-03-22T04:16:58.656631+0000","last_became_peered":"2024-03-22T04:16:58.656631+0000","last_unstale":"2024-03-22T04:16:59.671045+0000","last_undegraded":"2024-03-22T04:16:59.671045+0000","last_fullsized":"2024-03-22T04:16:59.671045+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:08:00.166390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.661001+0000","last_change":"2024-03-22T04:16:58.656708+0000","last_active":"2024-03-22T04:16:58.661001+0000","last_peered":"2024-03-22T04:16:58.661001+0000","last_clean":"2024-03-22T04:16:58.661001+0000","last_became_active":"2024-03-22T04:16:58.656353+0000","last_became_peered":"2024-03-22T04:16:58.656353+0000","last_unstale":"2024-03-22T04:16:58.661001+0000","last_undegraded":"2024-03-22T04:16:58.661001+0000","last_fullsized":"2024-03-22T04:16:58.661001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:08:00.166390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670759+0000","last_change":"2024-03-22T04:16:58.669779+0000","last_active":"2024-03-22T04:16:59.670759+0000","last_peered":"2024-03-22T04:16:59.670759+0000","last_clean":"2024-03-22T04:16:59.670759+0000","last_became_active":"2024-03-22T04:16:58.668792+0000","last_became_peered":"2024-03-22T04:16:58.668792+0000","last_unstale":"2024-03-22T04:16:59.670759+0000","last_undegraded":"2024-03-22T04:16:59.670759+0000","last_fullsized":"2024-03-22T04:16:59.670759+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:13.073512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.671746+0000","last_change":"2024-03-22T04:16:58.664705+0000","last_active":"2024-03-22T04:16:58.671746+0000","last_peered":"2024-03-22T04:16:58.671746+0000","last_clean":"2024-03-22T04:16:58.671746+0000","last_became_active":"2024-03-22T04:16:58.664362+0000","last_became_peered":"2024-03-22T04:16:58.664362+0000","last_unstale":"2024-03-22T04:16:58.671746+0000","last_undegraded":"2024-03-22T04:16:58.671746+0000","last_fullsized":"2024-03-22T04:16:58.671746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:02.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671170+0000","last_change":"2024-03-22T04:16:58.657736+0000","last_active":"2024-03-22T04:16:59.671170+0000","last_peered":"2024-03-22T04:16:59.671170+0000","last_clean":"2024-03-22T04:16:59.671170+0000","last_became_active":"2024-03-22T04:16:58.657451+0000","last_became_peered":"2024-03-22T04:16:58.657451+0000","last_unstale":"2024-03-22T04:16:59.671170+0000","last_undegraded":"2024-03-22T04:16:59.671170+0000","last_fullsized":"2024-03-22T04:16:59.671170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:46:51.882426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671238+0000","last_change":"2024-03-22T04:16:58.670882+0000","last_active":"2024-03-22T04:16:59.671238+0000","last_peered":"2024-03-22T04:16:59.671238+0000","last_clean":"2024-03-22T04:16:59.671238+0000","last_became_active":"2024-03-22T04:16:58.670518+0000","last_became_peered":"2024-03-22T04:16:58.670518+0000","last_unstale":"2024-03-22T04:16:59.671238+0000","last_undegraded":"2024-03-22T04:16:59.671238+0000","last_fullsized":"2024-03-22T04:16:59.671238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:50.842967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883447+0000","last_change":"2024-03-22T04:16:58.657882+0000","last_active":"2024-03-22T04:16:59.883447+0000","last_peered":"2024-03-22T04:16:59.883447+0000","last_clean":"2024-03-22T04:16:59.883447+0000","last_became_active":"2024-03-22T04:16:58.657483+0000","last_became_peered":"2024-03-22T04:16:58.657483+0000","last_unstale":"2024-03-22T04:16:59.883447+0000","last_undegraded":"2024-03-22T04:16:59.883447+0000","last_fullsized":"2024-03-22T04:16:59.883447+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:59:34.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646904+0000","last_change":"2024-03-22T04:16:58.678361+0000","last_active":"2024-03-22T04:16:59.646904+0000","last_peered":"2024-03-22T04:16:59.646904+0000","last_clean":"2024-03-22T04:16:59.646904+0000","last_became_active":"2024-03-22T04:16:58.677971+0000","last_became_peered":"2024-03-22T04:16:58.677971+0000","last_unstale":"2024-03-22T04:16:59.646904+0000","last_undegraded":"2024-03-22T04:16:59.646904+0000","last_fullsized":"2024-03-22T04:16:59.646904+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:46.047388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671445+0000","last_change":"2024-03-22T04:16:58.669068+0000","last_active":"2024-03-22T04:16:59.671445+0000","last_peered":"2024-03-22T04:16:59.671445+0000","last_clean":"2024-03-22T04:16:59.671445+0000","last_became_active":"2024-03-22T04:16:58.668754+0000","last_became_peered":"2024-03-22T04:16:58.668754+0000","last_unstale":"2024-03-22T04:16:59.671445+0000","last_undegraded":"2024-03-22T04:16:59.671445+0000","last_fullsized":"2024-03-22T04:16:59.671445+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:15.690412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670859+0000","last_change":"2024-03-22T04:16:58.654734+0000","last_active":"2024-03-22T04:16:59.670859+0000","last_peered":"2024-03-22T04:16:59.670859+0000","last_clean":"2024-03-22T04:16:59.670859+0000","last_became_active":"2024-03-22T04:16:58.654441+0000","last_became_peered":"2024-03-22T04:16:58.654441+0000","last_unstale":"2024-03-22T04:16:59.670859+0000","last_undegraded":"2024-03-22T04:16:59.670859+0000","last_fullsized":"2024-03-22T04:16:59.670859+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:25.281798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.677284+0000","last_change":"2024-03-22T04:16:58.669970+0000","last_active":"2024-03-22T04:16:58.677284+0000","last_peered":"2024-03-22T04:16:58.677284+0000","last_clean":"2024-03-22T04:16:58.677284+0000","last_became_active":"2024-03-22T04:16:58.669620+0000","last_became_peered":"2024-03-22T04:16:58.669620+0000","last_unstale":"2024-03-22T04:16:58.677284+0000","last_undegraded":"2024-03-22T04:16:58.677284+0000","last_fullsized":"2024-03-22T04:16:58.677284+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:02.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680948+0000","last_change":"2024-03-22T04:16:58.672972+0000","last_active":"2024-03-22T04:16:58.680948+0000","last_peered":"2024-03-22T04:16:58.680948+0000","last_clean":"2024-03-22T04:16:58.680948+0000","last_became_active":"2024-03-22T04:16:58.672223+0000","last_became_peered":"2024-03-22T04:16:58.672223+0000","last_unstale":"2024-03-22T04:16:58.680948+0000","last_undegraded":"2024-03-22T04:16:58.680948+0000","last_fullsized":"2024-03-22T04:16:58.680948+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:37:50.129389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.684725+0000","last_change":"2024-03-22T04:16:58.669199+0000","last_active":"2024-03-22T04:16:58.684725+0000","last_peered":"2024-03-22T04:16:58.684725+0000","last_clean":"2024-03-22T04:16:58.684725+0000","last_became_active":"2024-03-22T04:16:58.668934+0000","last_became_peered":"2024-03-22T04:16:58.668934+0000","last_unstale":"2024-03-22T04:16:58.684725+0000","last_undegraded":"2024-03-22T04:16:58.684725+0000","last_fullsized":"2024-03-22T04:16:58.684725+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:32:09.746284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"92'4","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671362+0000","last_change":"2024-03-22T04:16:57.656478+0000","last_active":"2024-03-22T04:16:59.671362+0000","last_peered":"2024-03-22T04:16:59.671362+0000","last_clean":"2024-03-22T04:16:59.671362+0000","last_became_active":"2024-03-22T04:16:57.656218+0000","last_became_peered":"2024-03-22T04:16:57.656218+0000","last_unstale":"2024-03-22T04:16:59.671362+0000","last_undegraded":"2024-03-22T04:16:59.671362+0000","last_fullsized":"2024-03-22T04:16:59.671362+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:15:40.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680826+0000","last_change":"2024-03-22T04:16:58.670649+0000","last_active":"2024-03-22T04:16:58.680826+0000","last_peered":"2024-03-22T04:16:58.680826+0000","last_clean":"2024-03-22T04:16:58.680826+0000","last_became_active":"2024-03-22T04:16:58.669153+0000","last_became_peered":"2024-03-22T04:16:58.669153+0000","last_unstale":"2024-03-22T04:16:58.680826+0000","last_undegraded":"2024-03-22T04:16:58.680826+0000","last_fullsized":"2024-03-22T04:16:58.680826+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:42:11.903310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646888+0000","last_change":"2024-03-22T04:16:57.643608+0000","last_active":"2024-03-22T04:16:59.646888+0000","last_peered":"2024-03-22T04:16:59.646888+0000","last_clean":"2024-03-22T04:16:59.646888+0000","last_became_active":"2024-03-22T04:16:57.643313+0000","last_became_peered":"2024-03-22T04:16:57.643313+0000","last_unstale":"2024-03-22T04:16:59.646888+0000","last_undegraded":"2024-03-22T04:16:59.646888+0000","last_fullsized":"2024-03-22T04:16:59.646888+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:07:54.462875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668190+0000","last_change":"2024-03-22T04:16:58.681556+0000","last_active":"2024-03-22T04:16:59.668190+0000","last_peered":"2024-03-22T04:16:59.668190+0000","last_clean":"2024-03-22T04:16:59.668190+0000","last_became_active":"2024-03-22T04:16:58.681258+0000","last_became_peered":"2024-03-22T04:16:58.681258+0000","last_unstale":"2024-03-22T04:16:59.668190+0000","last_undegraded":"2024-03-22T04:16:59.668190+0000","last_fullsized":"2024-03-22T04:16:59.668190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:22:10.340650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671244+0000","last_change":"2024-03-22T04:16:57.655287+0000","last_active":"2024-03-22T04:16:59.671244+0000","last_peered":"2024-03-22T04:16:59.671244+0000","last_clean":"2024-03-22T04:16:59.671244+0000","last_became_active":"2024-03-22T04:16:57.655005+0000","last_became_peered":"2024-03-22T04:16:57.655005+0000","last_unstale":"2024-03-22T04:16:59.671244+0000","last_undegraded":"2024-03-22T04:16:59.671244+0000","last_fullsized":"2024-03-22T04:16:59.671244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:32:40.294292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883490+0000","last_change":"2024-03-22T04:16:58.662679+0000","last_active":"2024-03-22T04:16:59.883490+0000","last_peered":"2024-03-22T04:16:59.883490+0000","last_clean":"2024-03-22T04:16:59.883490+0000","last_became_active":"2024-03-22T04:16:58.662415+0000","last_became_peered":"2024-03-22T04:16:58.662415+0000","last_unstale":"2024-03-22T04:16:59.883490+0000","last_undegraded":"2024-03-22T04:16:59.883490+0000","last_fullsized":"2024-03-22T04:16:59.883490+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:46:16.247985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646958+0000","last_change":"2024-03-22T04:16:57.652536+0000","last_active":"2024-03-22T04:16:59.646958+0000","last_peered":"2024-03-22T04:16:59.646958+0000","last_clean":"2024-03-22T04:16:59.646958+0000","last_became_active":"2024-03-22T04:16:57.651827+0000","last_became_peered":"2024-03-22T04:16:57.651827+0000","last_unstale":"2024-03-22T04:16:59.646958+0000","last_undegraded":"2024-03-22T04:16:59.646958+0000","last_fullsized":"2024-03-22T04:16:59.646958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:56:57.570918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.645792+0000","last_change":"2024-03-22T04:16:58.675622+0000","last_active":"2024-03-22T04:16:59.645792+0000","last_peered":"2024-03-22T04:16:59.645792+0000","last_clean":"2024-03-22T04:16:59.645792+0000","last_became_active":"2024-03-22T04:16:58.675353+0000","last_became_peered":"2024-03-22T04:16:58.675353+0000","last_unstale":"2024-03-22T04:16:59.645792+0000","last_undegraded":"2024-03-22T04:16:59.645792+0000","last_fullsized":"2024-03-22T04:16:59.645792+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:07:02.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"92'1","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.683700+0000","last_change":"2024-03-22T04:16:57.642241+0000","last_active":"2024-03-22T04:16:58.683700+0000","last_peered":"2024-03-22T04:16:58.683700+0000","last_clean":"2024-03-22T04:16:58.683700+0000","last_became_active":"2024-03-22T04:16:57.641988+0000","last_became_peered":"2024-03-22T04:16:57.641988+0000","last_unstale":"2024-03-22T04:16:58.683700+0000","last_undegraded":"2024-03-22T04:16:58.683700+0000","last_fullsized":"2024-03-22T04:16:58.683700+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:23:32.537352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668085+0000","last_change":"2024-03-22T04:16:58.657571+0000","last_active":"2024-03-22T04:16:59.668085+0000","last_peered":"2024-03-22T04:16:59.668085+0000","last_clean":"2024-03-22T04:16:59.668085+0000","last_became_active":"2024-03-22T04:16:58.655831+0000","last_became_peered":"2024-03-22T04:16:58.655831+0000","last_unstale":"2024-03-22T04:16:59.668085+0000","last_undegraded":"2024-03-22T04:16:59.668085+0000","last_fullsized":"2024-03-22T04:16:59.668085+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:04:18.293169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646527+0000","last_change":"2024-03-22T04:16:57.654908+0000","last_active":"2024-03-22T04:16:59.646527+0000","last_peered":"2024-03-22T04:16:59.646527+0000","last_clean":"2024-03-22T04:16:59.646527+0000","last_became_active":"2024-03-22T04:16:57.654580+0000","last_became_peered":"2024-03-22T04:16:57.654580+0000","last_unstale":"2024-03-22T04:16:59.646527+0000","last_undegraded":"2024-03-22T04:16:59.646527+0000","last_fullsized":"2024-03-22T04:16:59.646527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:04.318404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.668303+0000","last_change":"2024-03-22T04:16:58.659808+0000","last_active":"2024-03-22T04:16:58.668303+0000","last_peered":"2024-03-22T04:16:58.668303+0000","last_clean":"2024-03-22T04:16:58.668303+0000","last_became_active":"2024-03-22T04:16:58.659525+0000","last_became_peered":"2024-03-22T04:16:58.659525+0000","last_unstale":"2024-03-22T04:16:58.668303+0000","last_undegraded":"2024-03-22T04:16:58.668303+0000","last_fullsized":"2024-03-22T04:16:58.668303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:26:49.963772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883263+0000","last_change":"2024-03-22T04:16:57.645629+0000","last_active":"2024-03-22T04:16:59.883263+0000","last_peered":"2024-03-22T04:16:59.883263+0000","last_clean":"2024-03-22T04:16:59.883263+0000","last_became_active":"2024-03-22T04:16:57.645355+0000","last_became_peered":"2024-03-22T04:16:57.645355+0000","last_unstale":"2024-03-22T04:16:59.883263+0000","last_undegraded":"2024-03-22T04:16:59.883263+0000","last_fullsized":"2024-03-22T04:16:59.883263+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:18:01.826085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.674352+0000","last_change":"2024-03-22T04:16:58.668263+0000","last_active":"2024-03-22T04:16:58.674352+0000","last_peered":"2024-03-22T04:16:58.674352+0000","last_clean":"2024-03-22T04:16:58.674352+0000","last_became_active":"2024-03-22T04:16:58.667678+0000","last_became_peered":"2024-03-22T04:16:58.667678+0000","last_unstale":"2024-03-22T04:16:58.674352+0000","last_undegraded":"2024-03-22T04:16:58.674352+0000","last_fullsized":"2024-03-22T04:16:58.674352+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:46:51.882426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670697+0000","last_change":"2024-03-22T04:16:57.656706+0000","last_active":"2024-03-22T04:16:59.670697+0000","last_peered":"2024-03-22T04:16:59.670697+0000","last_clean":"2024-03-22T04:16:59.670697+0000","last_became_active":"2024-03-22T04:16:57.656425+0000","last_became_peered":"2024-03-22T04:16:57.656425+0000","last_unstale":"2024-03-22T04:16:59.670697+0000","last_undegraded":"2024-03-22T04:16:59.670697+0000","last_fullsized":"2024-03-22T04:16:59.670697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:05:29.717715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671047+0000","last_change":"2024-03-22T04:16:58.679467+0000","last_active":"2024-03-22T04:16:59.671047+0000","last_peered":"2024-03-22T04:16:59.671047+0000","last_clean":"2024-03-22T04:16:59.671047+0000","last_became_active":"2024-03-22T04:16:58.679206+0000","last_became_peered":"2024-03-22T04:16:58.679206+0000","last_unstale":"2024-03-22T04:16:59.671047+0000","last_undegraded":"2024-03-22T04:16:59.671047+0000","last_fullsized":"2024-03-22T04:16:59.671047+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:42.681464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671374+0000","last_change":"2024-03-22T04:16:57.650167+0000","last_active":"2024-03-22T04:16:59.671374+0000","last_peered":"2024-03-22T04:16:59.671374+0000","last_clean":"2024-03-22T04:16:59.671374+0000","last_became_active":"2024-03-22T04:16:57.649028+0000","last_became_peered":"2024-03-22T04:16:57.649028+0000","last_unstale":"2024-03-22T04:16:59.671374+0000","last_undegraded":"2024-03-22T04:16:59.671374+0000","last_fullsized":"2024-03-22T04:16:59.671374+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:48:04.545029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668336+0000","last_change":"2024-03-22T04:16:58.654199+0000","last_active":"2024-03-22T04:16:59.668336+0000","last_peered":"2024-03-22T04:16:59.668336+0000","last_clean":"2024-03-22T04:16:59.668336+0000","last_became_active":"2024-03-22T04:16:58.653953+0000","last_became_peered":"2024-03-22T04:16:58.653953+0000","last_unstale":"2024-03-22T04:16:59.668336+0000","last_undegraded":"2024-03-22T04:16:59.668336+0000","last_fullsized":"2024-03-22T04:16:59.668336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:17:38.009055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883581+0000","last_change":"2024-03-22T04:16:57.654685+0000","last_active":"2024-03-22T04:16:59.883581+0000","last_peered":"2024-03-22T04:16:59.883581+0000","last_clean":"2024-03-22T04:16:59.883581+0000","last_became_active":"2024-03-22T04:16:57.654402+0000","last_became_peered":"2024-03-22T04:16:57.654402+0000","last_unstale":"2024-03-22T04:16:59.883581+0000","last_undegraded":"2024-03-22T04:16:59.883581+0000","last_fullsized":"2024-03-22T04:16:59.883581+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:37:31.501725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.671110+0000","last_change":"2024-03-22T04:16:58.667348+0000","last_active":"2024-03-22T04:16:58.671110+0000","last_peered":"2024-03-22T04:16:58.671110+0000","last_clean":"2024-03-22T04:16:58.671110+0000","last_became_active":"2024-03-22T04:16:58.667043+0000","last_became_peered":"2024-03-22T04:16:58.667043+0000","last_unstale":"2024-03-22T04:16:58.671110+0000","last_undegraded":"2024-03-22T04:16:58.671110+0000","last_fullsized":"2024-03-22T04:16:58.671110+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:17.742365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668394+0000","last_change":"2024-03-22T04:16:57.656122+0000","last_active":"2024-03-22T04:16:59.668394+0000","last_peered":"2024-03-22T04:16:59.668394+0000","last_clean":"2024-03-22T04:16:59.668394+0000","last_became_active":"2024-03-22T04:16:57.655629+0000","last_became_peered":"2024-03-22T04:16:57.655629+0000","last_unstale":"2024-03-22T04:16:59.668394+0000","last_undegraded":"2024-03-22T04:16:59.668394+0000","last_fullsized":"2024-03-22T04:16:59.668394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:26:46.565286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.668218+0000","last_change":"2024-03-22T04:16:58.658233+0000","last_active":"2024-03-22T04:16:58.668218+0000","last_peered":"2024-03-22T04:16:58.668218+0000","last_clean":"2024-03-22T04:16:58.668218+0000","last_became_active":"2024-03-22T04:16:58.657947+0000","last_became_peered":"2024-03-22T04:16:58.657947+0000","last_unstale":"2024-03-22T04:16:58.668218+0000","last_undegraded":"2024-03-22T04:16:58.668218+0000","last_fullsized":"2024-03-22T04:16:58.668218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:45:04.857756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"92'6","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.691407+0000","last_change":"2024-03-22T04:16:57.657660+0000","last_active":"2024-03-22T04:16:58.691407+0000","last_peered":"2024-03-22T04:16:58.691407+0000","last_clean":"2024-03-22T04:16:58.691407+0000","last_became_active":"2024-03-22T04:16:57.657437+0000","last_became_peered":"2024-03-22T04:16:57.657437+0000","last_unstale":"2024-03-22T04:16:58.691407+0000","last_undegraded":"2024-03-22T04:16:58.691407+0000","last_fullsized":"2024-03-22T04:16:58.691407+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:41.674550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.671723+0000","last_change":"2024-03-22T04:16:58.660848+0000","last_active":"2024-03-22T04:16:58.671723+0000","last_peered":"2024-03-22T04:16:58.671723+0000","last_clean":"2024-03-22T04:16:58.671723+0000","last_became_active":"2024-03-22T04:16:58.660480+0000","last_became_peered":"2024-03-22T04:16:58.660480+0000","last_unstale":"2024-03-22T04:16:58.671723+0000","last_undegraded":"2024-03-22T04:16:58.671723+0000","last_fullsized":"2024-03-22T04:16:58.671723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:57.725363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671434+0000","last_change":"2024-03-22T04:16:57.655310+0000","last_active":"2024-03-22T04:16:59.671434+0000","last_peered":"2024-03-22T04:16:59.671434+0000","last_clean":"2024-03-22T04:16:59.671434+0000","last_became_active":"2024-03-22T04:16:57.651465+0000","last_became_peered":"2024-03-22T04:16:57.651465+0000","last_unstale":"2024-03-22T04:16:59.671434+0000","last_undegraded":"2024-03-22T04:16:59.671434+0000","last_fullsized":"2024-03-22T04:16:59.671434+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:28:26.759427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670774+0000","last_change":"2024-03-22T04:16:58.655094+0000","last_active":"2024-03-22T04:16:59.670774+0000","last_peered":"2024-03-22T04:16:59.670774+0000","last_clean":"2024-03-22T04:16:59.670774+0000","last_became_active":"2024-03-22T04:16:58.654815+0000","last_became_peered":"2024-03-22T04:16:58.654815+0000","last_unstale":"2024-03-22T04:16:59.670774+0000","last_undegraded":"2024-03-22T04:16:59.670774+0000","last_fullsized":"2024-03-22T04:16:59.670774+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:23:33.544266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.655010+0000","last_change":"2024-03-22T04:16:57.649758+0000","last_active":"2024-03-22T04:16:58.655010+0000","last_peered":"2024-03-22T04:16:58.655010+0000","last_clean":"2024-03-22T04:16:58.655010+0000","last_became_active":"2024-03-22T04:16:57.649487+0000","last_became_peered":"2024-03-22T04:16:57.649487+0000","last_unstale":"2024-03-22T04:16:58.655010+0000","last_undegraded":"2024-03-22T04:16:58.655010+0000","last_fullsized":"2024-03-22T04:16:58.655010+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:23:32.537352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.685725+0000","last_change":"2024-03-22T04:16:58.677536+0000","last_active":"2024-03-22T04:16:58.685725+0000","last_peered":"2024-03-22T04:16:58.685725+0000","last_clean":"2024-03-22T04:16:58.685725+0000","last_became_active":"2024-03-22T04:16:58.675044+0000","last_became_peered":"2024-03-22T04:16:58.675044+0000","last_unstale":"2024-03-22T04:16:58.685725+0000","last_undegraded":"2024-03-22T04:16:58.685725+0000","last_fullsized":"2024-03-22T04:16:58.685725+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:46.047388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.654139+0000","last_change":"2024-03-22T04:16:57.652081+0000","last_active":"2024-03-22T04:16:58.654139+0000","last_peered":"2024-03-22T04:16:58.654139+0000","last_clean":"2024-03-22T04:16:58.654139+0000","last_became_active":"2024-03-22T04:16:57.651705+0000","last_became_peered":"2024-03-22T04:16:57.651705+0000","last_unstale":"2024-03-22T04:16:58.654139+0000","last_undegraded":"2024-03-22T04:16:58.654139+0000","last_fullsized":"2024-03-22T04:16:58.654139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:12.066597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646061+0000","last_change":"2024-03-22T04:16:58.677326+0000","last_active":"2024-03-22T04:16:59.646061+0000","last_peered":"2024-03-22T04:16:59.646061+0000","last_clean":"2024-03-22T04:16:59.646061+0000","last_became_active":"2024-03-22T04:16:58.677019+0000","last_became_peered":"2024-03-22T04:16:58.677019+0000","last_unstale":"2024-03-22T04:16:59.646061+0000","last_undegraded":"2024-03-22T04:16:59.646061+0000","last_fullsized":"2024-03-22T04:16:59.646061+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:32:09.746284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883026+0000","last_change":"2024-03-22T04:16:57.652413+0000","last_active":"2024-03-22T04:16:59.883026+0000","last_peered":"2024-03-22T04:16:59.883026+0000","last_clean":"2024-03-22T04:16:59.883026+0000","last_became_active":"2024-03-22T04:16:57.648859+0000","last_became_peered":"2024-03-22T04:16:57.648859+0000","last_unstale":"2024-03-22T04:16:59.883026+0000","last_undegraded":"2024-03-22T04:16:59.883026+0000","last_fullsized":"2024-03-22T04:16:59.883026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:41:05.754331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646945+0000","last_change":"2024-03-22T04:16:58.676696+0000","last_active":"2024-03-22T04:16:59.646945+0000","last_peered":"2024-03-22T04:16:59.646945+0000","last_clean":"2024-03-22T04:16:59.646945+0000","last_became_active":"2024-03-22T04:16:58.676428+0000","last_became_peered":"2024-03-22T04:16:58.676428+0000","last_unstale":"2024-03-22T04:16:59.646945+0000","last_undegraded":"2024-03-22T04:16:59.646945+0000","last_fullsized":"2024-03-22T04:16:59.646945+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:15.690412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"92'1","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.679827+0000","last_change":"2024-03-22T04:16:57.643848+0000","last_active":"2024-03-22T04:16:58.679827+0000","last_peered":"2024-03-22T04:16:58.679827+0000","last_clean":"2024-03-22T04:16:58.679827+0000","last_became_active":"2024-03-22T04:16:57.643578+0000","last_became_peered":"2024-03-22T04:16:57.643578+0000","last_unstale":"2024-03-22T04:16:58.679827+0000","last_undegraded":"2024-03-22T04:16:58.679827+0000","last_fullsized":"2024-03-22T04:16:58.679827+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:24.274884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668428+0000","last_change":"2024-03-22T04:16:58.652141+0000","last_active":"2024-03-22T04:16:59.668428+0000","last_peered":"2024-03-22T04:16:59.668428+0000","last_clean":"2024-03-22T04:16:59.668428+0000","last_became_active":"2024-03-22T04:16:58.651862+0000","last_became_peered":"2024-03-22T04:16:58.651862+0000","last_unstale":"2024-03-22T04:16:59.668428+0000","last_undegraded":"2024-03-22T04:16:59.668428+0000","last_fullsized":"2024-03-22T04:16:59.668428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:49:25.282105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646912+0000","last_change":"2024-03-22T04:16:57.653086+0000","last_active":"2024-03-22T04:16:59.646912+0000","last_peered":"2024-03-22T04:16:59.646912+0000","last_clean":"2024-03-22T04:16:59.646912+0000","last_became_active":"2024-03-22T04:16:57.652784+0000","last_became_peered":"2024-03-22T04:16:57.652784+0000","last_unstale":"2024-03-22T04:16:59.646912+0000","last_undegraded":"2024-03-22T04:16:59.646912+0000","last_fullsized":"2024-03-22T04:16:59.646912+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:26:38.381239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671155+0000","last_change":"2024-03-22T04:16:58.681137+0000","last_active":"2024-03-22T04:16:59.671155+0000","last_peered":"2024-03-22T04:16:59.671155+0000","last_clean":"2024-03-22T04:16:59.671155+0000","last_became_active":"2024-03-22T04:16:58.680771+0000","last_became_peered":"2024-03-22T04:16:58.680771+0000","last_unstale":"2024-03-22T04:16:59.671155+0000","last_undegraded":"2024-03-22T04:16:59.671155+0000","last_fullsized":"2024-03-22T04:16:59.671155+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:17.742365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646897+0000","last_change":"2024-03-22T04:16:57.653718+0000","last_active":"2024-03-22T04:16:59.646897+0000","last_peered":"2024-03-22T04:16:59.646897+0000","last_clean":"2024-03-22T04:16:59.646897+0000","last_became_active":"2024-03-22T04:16:57.653417+0000","last_became_peered":"2024-03-22T04:16:57.653417+0000","last_unstale":"2024-03-22T04:16:59.646897+0000","last_undegraded":"2024-03-22T04:16:59.646897+0000","last_fullsized":"2024-03-22T04:16:59.646897+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:41:15.516063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.683878+0000","last_change":"2024-03-22T04:16:58.679301+0000","last_active":"2024-03-22T04:16:58.683878+0000","last_peered":"2024-03-22T04:16:58.683878+0000","last_clean":"2024-03-22T04:16:58.683878+0000","last_became_active":"2024-03-22T04:16:58.677934+0000","last_became_peered":"2024-03-22T04:16:58.677934+0000","last_unstale":"2024-03-22T04:16:58.683878+0000","last_undegraded":"2024-03-22T04:16:58.683878+0000","last_fullsized":"2024-03-22T04:16:58.683878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:37.851619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668075+0000","last_change":"2024-03-22T04:16:57.651216+0000","last_active":"2024-03-22T04:16:59.668075+0000","last_peered":"2024-03-22T04:16:59.668075+0000","last_clean":"2024-03-22T04:16:59.668075+0000","last_became_active":"2024-03-22T04:16:57.650912+0000","last_became_peered":"2024-03-22T04:16:57.650912+0000","last_unstale":"2024-03-22T04:16:59.668075+0000","last_undegraded":"2024-03-22T04:16:59.668075+0000","last_fullsized":"2024-03-22T04:16:59.668075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:23:35.018881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646763+0000","last_change":"2024-03-22T04:16:58.658059+0000","last_active":"2024-03-22T04:16:59.646763+0000","last_peered":"2024-03-22T04:16:59.646763+0000","last_clean":"2024-03-22T04:16:59.646763+0000","last_became_active":"2024-03-22T04:16:58.657660+0000","last_became_peered":"2024-03-22T04:16:58.657660+0000","last_unstale":"2024-03-22T04:16:59.646763+0000","last_undegraded":"2024-03-22T04:16:59.646763+0000","last_fullsized":"2024-03-22T04:16:59.646763+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:25.281798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.650208+0000","last_change":"2024-03-22T04:16:57.650680+0000","last_active":"2024-03-22T04:16:58.650208+0000","last_peered":"2024-03-22T04:16:58.650208+0000","last_clean":"2024-03-22T04:16:58.650208+0000","last_became_active":"2024-03-22T04:16:57.650110+0000","last_became_peered":"2024-03-22T04:16:57.650110+0000","last_unstale":"2024-03-22T04:16:58.650208+0000","last_undegraded":"2024-03-22T04:16:58.650208+0000","last_fullsized":"2024-03-22T04:16:58.650208+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:06:11.062191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.679878+0000","last_change":"2024-03-22T04:16:58.672823+0000","last_active":"2024-03-22T04:16:58.679878+0000","last_peered":"2024-03-22T04:16:58.679878+0000","last_clean":"2024-03-22T04:16:58.679878+0000","last_became_active":"2024-03-22T04:16:58.672484+0000","last_became_peered":"2024-03-22T04:16:58.672484+0000","last_unstale":"2024-03-22T04:16:58.679878+0000","last_undegraded":"2024-03-22T04:16:58.679878+0000","last_fullsized":"2024-03-22T04:16:58.679878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:46:51.882426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646864+0000","last_change":"2024-03-22T04:16:57.653180+0000","last_active":"2024-03-22T04:16:59.646864+0000","last_peered":"2024-03-22T04:16:59.646864+0000","last_clean":"2024-03-22T04:16:59.646864+0000","last_became_active":"2024-03-22T04:16:57.652212+0000","last_became_peered":"2024-03-22T04:16:57.652212+0000","last_unstale":"2024-03-22T04:16:59.646864+0000","last_undegraded":"2024-03-22T04:16:59.646864+0000","last_fullsized":"2024-03-22T04:16:59.646864+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:39:04.318404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.677571+0000","last_change":"2024-03-22T04:16:58.670538+0000","last_active":"2024-03-22T04:16:58.677571+0000","last_peered":"2024-03-22T04:16:58.677571+0000","last_clean":"2024-03-22T04:16:58.677571+0000","last_became_active":"2024-03-22T04:16:58.668017+0000","last_became_peered":"2024-03-22T04:16:58.668017+0000","last_unstale":"2024-03-22T04:16:58.677571+0000","last_undegraded":"2024-03-22T04:16:58.677571+0000","last_fullsized":"2024-03-22T04:16:58.677571+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:50.842967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.645899+0000","last_change":"2024-03-22T04:16:57.651390+0000","last_active":"2024-03-22T04:16:59.645899+0000","last_peered":"2024-03-22T04:16:59.645899+0000","last_clean":"2024-03-22T04:16:59.645899+0000","last_became_active":"2024-03-22T04:16:57.651089+0000","last_became_peered":"2024-03-22T04:16:57.651089+0000","last_unstale":"2024-03-22T04:16:59.645899+0000","last_undegraded":"2024-03-22T04:16:59.645899+0000","last_fullsized":"2024-03-22T04:16:59.645899+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:41:15.516063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.659686+0000","last_change":"2024-03-22T04:16:58.655696+0000","last_active":"2024-03-22T04:16:58.659686+0000","last_peered":"2024-03-22T04:16:58.659686+0000","last_clean":"2024-03-22T04:16:58.659686+0000","last_became_active":"2024-03-22T04:16:58.655451+0000","last_became_peered":"2024-03-22T04:16:58.655451+0000","last_unstale":"2024-03-22T04:16:58.659686+0000","last_undegraded":"2024-03-22T04:16:58.659686+0000","last_fullsized":"2024-03-22T04:16:58.659686+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:42.681464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668339+0000","last_change":"2024-03-22T04:16:57.645491+0000","last_active":"2024-03-22T04:16:59.668339+0000","last_peered":"2024-03-22T04:16:59.668339+0000","last_clean":"2024-03-22T04:16:59.668339+0000","last_became_active":"2024-03-22T04:16:57.645046+0000","last_became_peered":"2024-03-22T04:16:57.645046+0000","last_unstale":"2024-03-22T04:16:59.668339+0000","last_undegraded":"2024-03-22T04:16:59.668339+0000","last_fullsized":"2024-03-22T04:16:59.668339+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:32:51.874035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.645852+0000","last_change":"2024-03-22T04:16:58.672726+0000","last_active":"2024-03-22T04:16:59.645852+0000","last_peered":"2024-03-22T04:16:59.645852+0000","last_clean":"2024-03-22T04:16:59.645852+0000","last_became_active":"2024-03-22T04:16:58.672394+0000","last_became_peered":"2024-03-22T04:16:58.672394+0000","last_unstale":"2024-03-22T04:16:59.645852+0000","last_undegraded":"2024-03-22T04:16:59.645852+0000","last_fullsized":"2024-03-22T04:16:59.645852+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:33:17.742365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"93'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670055+0000","last_change":"2024-03-22T04:16:57.655846+0000","last_active":"2024-03-22T04:16:59.670055+0000","last_peered":"2024-03-22T04:16:59.670055+0000","last_clean":"2024-03-22T04:16:59.670055+0000","last_became_active":"2024-03-22T04:16:57.655554+0000","last_became_peered":"2024-03-22T04:16:57.655554+0000","last_unstale":"2024-03-22T04:16:59.670055+0000","last_undegraded":"2024-03-22T04:16:59.670055+0000","last_fullsized":"2024-03-22T04:16:59.670055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:12:52.827841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.646420+0000","last_change":"2024-03-22T04:16:58.673870+0000","last_active":"2024-03-22T04:16:59.646420+0000","last_peered":"2024-03-22T04:16:59.646420+0000","last_clean":"2024-03-22T04:16:59.646420+0000","last_became_active":"2024-03-22T04:16:58.673589+0000","last_became_peered":"2024-03-22T04:16:58.673589+0000","last_unstale":"2024-03-22T04:16:59.646420+0000","last_undegraded":"2024-03-22T04:16:59.646420+0000","last_fullsized":"2024-03-22T04:16:59.646420+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:21:50.842967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.651097+0000","last_change":"2024-03-22T04:16:57.642001+0000","last_active":"2024-03-22T04:16:58.651097+0000","last_peered":"2024-03-22T04:16:58.651097+0000","last_clean":"2024-03-22T04:16:58.651097+0000","last_became_active":"2024-03-22T04:16:57.641733+0000","last_became_peered":"2024-03-22T04:16:57.641733+0000","last_unstale":"2024-03-22T04:16:58.651097+0000","last_undegraded":"2024-03-22T04:16:58.651097+0000","last_fullsized":"2024-03-22T04:16:58.651097+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:05:56.718448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671304+0000","last_change":"2024-03-22T04:16:58.655490+0000","last_active":"2024-03-22T04:16:59.671304+0000","last_peered":"2024-03-22T04:16:59.671304+0000","last_clean":"2024-03-22T04:16:59.671304+0000","last_became_active":"2024-03-22T04:16:58.655231+0000","last_became_peered":"2024-03-22T04:16:58.655231+0000","last_unstale":"2024-03-22T04:16:59.671304+0000","last_undegraded":"2024-03-22T04:16:59.671304+0000","last_fullsized":"2024-03-22T04:16:59.671304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:46.047388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.668342+0000","last_change":"2024-03-22T04:16:57.652632+0000","last_active":"2024-03-22T04:16:59.668342+0000","last_peered":"2024-03-22T04:16:59.668342+0000","last_clean":"2024-03-22T04:16:59.668342+0000","last_became_active":"2024-03-22T04:16:57.652376+0000","last_became_peered":"2024-03-22T04:16:57.652376+0000","last_unstale":"2024-03-22T04:16:59.668342+0000","last_undegraded":"2024-03-22T04:16:59.668342+0000","last_fullsized":"2024-03-22T04:16:59.668342+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:01:27.317630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.681179+0000","last_change":"2024-03-22T04:16:58.677945+0000","last_active":"2024-03-22T04:16:58.681179+0000","last_peered":"2024-03-22T04:16:58.681179+0000","last_clean":"2024-03-22T04:16:58.681179+0000","last_became_active":"2024-03-22T04:16:58.677398+0000","last_became_peered":"2024-03-22T04:16:58.677398+0000","last_unstale":"2024-03-22T04:16:58.681179+0000","last_undegraded":"2024-03-22T04:16:58.681179+0000","last_fullsized":"2024-03-22T04:16:58.681179+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:38:17.444912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.655423+0000","last_change":"2024-03-22T04:16:57.652602+0000","last_active":"2024-03-22T04:16:58.655423+0000","last_peered":"2024-03-22T04:16:58.655423+0000","last_clean":"2024-03-22T04:16:58.655423+0000","last_became_active":"2024-03-22T04:16:57.652261+0000","last_became_peered":"2024-03-22T04:16:57.652261+0000","last_unstale":"2024-03-22T04:16:58.655423+0000","last_undegraded":"2024-03-22T04:16:58.655423+0000","last_fullsized":"2024-03-22T04:16:58.655423+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:47:24.274884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.883166+0000","last_change":"2024-03-22T04:16:58.669636+0000","last_active":"2024-03-22T04:16:59.883166+0000","last_peered":"2024-03-22T04:16:59.883166+0000","last_clean":"2024-03-22T04:16:59.883166+0000","last_became_active":"2024-03-22T04:16:58.669366+0000","last_became_peered":"2024-03-22T04:16:58.669366+0000","last_unstale":"2024-03-22T04:16:59.883166+0000","last_undegraded":"2024-03-22T04:16:59.883166+0000","last_fullsized":"2024-03-22T04:16:59.883166+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:59:54.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670880+0000","last_change":"2024-03-22T04:16:57.658951+0000","last_active":"2024-03-22T04:16:59.670880+0000","last_peered":"2024-03-22T04:16:59.670880+0000","last_clean":"2024-03-22T04:16:59.670880+0000","last_became_active":"2024-03-22T04:16:57.658673+0000","last_became_peered":"2024-03-22T04:16:57.658673+0000","last_unstale":"2024-03-22T04:16:59.670880+0000","last_undegraded":"2024-03-22T04:16:59.670880+0000","last_fullsized":"2024-03-22T04:16:59.670880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:24:02.561260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.679958+0000","last_change":"2024-03-22T04:16:58.673445+0000","last_active":"2024-03-22T04:16:58.679958+0000","last_peered":"2024-03-22T04:16:58.679958+0000","last_clean":"2024-03-22T04:16:58.679958+0000","last_became_active":"2024-03-22T04:16:58.673195+0000","last_became_peered":"2024-03-22T04:16:58.673195+0000","last_unstale":"2024-03-22T04:16:58.679958+0000","last_undegraded":"2024-03-22T04:16:58.679958+0000","last_fullsized":"2024-03-22T04:16:58.679958+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:32:09.746284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670864+0000","last_change":"2024-03-22T04:16:57.654514+0000","last_active":"2024-03-22T04:16:59.670864+0000","last_peered":"2024-03-22T04:16:59.670864+0000","last_clean":"2024-03-22T04:16:59.670864+0000","last_became_active":"2024-03-22T04:16:57.654071+0000","last_became_peered":"2024-03-22T04:16:57.654071+0000","last_unstale":"2024-03-22T04:16:59.670864+0000","last_undegraded":"2024-03-22T04:16:59.670864+0000","last_fullsized":"2024-03-22T04:16:59.670864+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:07:31.066617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.680864+0000","last_change":"2024-03-22T04:16:58.671784+0000","last_active":"2024-03-22T04:16:58.680864+0000","last_peered":"2024-03-22T04:16:58.680864+0000","last_clean":"2024-03-22T04:16:58.680864+0000","last_became_active":"2024-03-22T04:16:58.671416+0000","last_became_peered":"2024-03-22T04:16:58.671416+0000","last_unstale":"2024-03-22T04:16:58.680864+0000","last_undegraded":"2024-03-22T04:16:58.680864+0000","last_fullsized":"2024-03-22T04:16:58.680864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:35:28.012150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670691+0000","last_change":"2024-03-22T04:16:57.659423+0000","last_active":"2024-03-22T04:16:59.670691+0000","last_peered":"2024-03-22T04:16:59.670691+0000","last_clean":"2024-03-22T04:16:59.670691+0000","last_became_active":"2024-03-22T04:16:57.659167+0000","last_became_peered":"2024-03-22T04:16:57.659167+0000","last_unstale":"2024-03-22T04:16:59.670691+0000","last_undegraded":"2024-03-22T04:16:59.670691+0000","last_fullsized":"2024-03-22T04:16:59.670691+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:03:50.782095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.671213+0000","last_change":"2024-03-22T04:16:58.679379+0000","last_active":"2024-03-22T04:16:59.671213+0000","last_peered":"2024-03-22T04:16:59.671213+0000","last_clean":"2024-03-22T04:16:59.671213+0000","last_became_active":"2024-03-22T04:16:58.679062+0000","last_became_peered":"2024-03-22T04:16:58.679062+0000","last_unstale":"2024-03-22T04:16:59.671213+0000","last_undegraded":"2024-03-22T04:16:59.671213+0000","last_fullsized":"2024-03-22T04:16:59.671213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:57.624047+0000","last_clean_scrub_stamp":"2024-03-22T04:16:57.624047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:32:09.746284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-22T04:16:58.662139+0000","last_change":"2024-03-22T04:16:57.657743+0000","last_active":"2024-03-22T04:16:58.662139+0000","last_peered":"2024-03-22T04:16:58.662139+0000","last_clean":"2024-03-22T04:16:58.662139+0000","last_became_active":"2024-03-22T04:16:57.657446+0000","last_became_peered":"2024-03-22T04:16:57.657446+0000","last_unstale":"2024-03-22T04:16:58.662139+0000","last_undegraded":"2024-03-22T04:16:58.662139+0000","last_fullsized":"2024-03-22T04:16:58.662139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:16:56.617133+0000","last_clean_scrub_stamp":"2024-03-22T04:16:56.617133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:59.159476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":302,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-22T04:16:59.670774+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:16:59.670774+0000","last_peered":"2024-03-22T04:16:59.670774+0000","last_clean":"2024-03-22T04:16:59.670774+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:16:59.670774+0000","last_undegraded":"2024-03-22T04:16:59.670774+0000","last_fullsized":"2024-03-22T04:16:59.670774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509523,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":522395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":517439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1732608,"data_stored":1112763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":524119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509523,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":516307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1744896,"data_stored":1113449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509523,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":522465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":515703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:17:03.276 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T04:17:03.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-22T04:17:03.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3485800640 1 Processor -- start 2024-03-22T04:17:03.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3485800640 1 -- start start 2024-03-22T04:17:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3485800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f348015b3d0 0x7f34800cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3485800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34800b4920 0x7f34800cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3485800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34800b4d60 con 0x7f34800d01d0 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3485800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f34800cd380 con 0x7f348015b3d0 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3485800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34800cca50 con 0x7f34800b4920 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3476000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34800b4920 0x7f34800cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f347f400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f34800d01d0 0x7f34800ce370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:53256/0 (socket says 172.21.15.86:53256) 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f347f400640 1 -- 172.21.15.86:0/3617041688 learned_addr learned my addr 172.21.15.86:0/3617041688 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3476000640 1 --2- 172.21.15.86:0/3617041688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34800b4920 0x7f34800cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:56884/0 (socket says 172.21.15.86:56884) 2024-03-22T04:17:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f347ea00640 1 --2- 172.21.15.86:0/3617041688 >> v2:172.21.15.196:3300/0 conn(0x7f348015b3d0 0x7f34800cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3476000640 1 -- 172.21.15.86:0/3617041688 >> v2:172.21.15.196:3300/0 conn(0x7f348015b3d0 msgr2=0x7f34800cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3476000640 1 --2- 172.21.15.86:0/3617041688 >> v2:172.21.15.196:3300/0 conn(0x7f348015b3d0 0x7f34800cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3476000640 1 -- 172.21.15.86:0/3617041688 >> v1:172.21.15.196:6789/0 conn(0x7f34800d01d0 legacy=0x7f34800ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.458+0000 7f3476000640 1 -- 172.21.15.86:0/3617041688 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34800ccd30 con 0x7f34800b4920 2024-03-22T04:17:03.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3476000640 1 --2- 172.21.15.86:0/3617041688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34800b4920 0x7f34800cfc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34680097c0 tx=0x7f346801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2cdffaff966b3a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f347e000640 1 -- 172.21.15.86:0/3617041688 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34680b5070 con 0x7f34800b4920 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f347e000640 1 -- 172.21.15.86:0/3617041688 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34680a8990 con 0x7f34800b4920 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f347e000640 1 -- 172.21.15.86:0/3617041688 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34680a8d60 con 0x7f34800b4920 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3485800640 1 -- 172.21.15.86:0/3617041688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34800b4920 msgr2=0x7f34800cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3485800640 1 --2- 172.21.15.86:0/3617041688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34800b4920 0x7f34800cfc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34680097c0 tx=0x7f346801ec90 comp rx=0 tx=0).stop 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3485800640 1 -- 172.21.15.86:0/3617041688 shutdown_connections 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3485800640 1 --2- 172.21.15.86:0/3617041688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34800b4920 0x7f34800cfc90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3485800640 1 --2- 172.21.15.86:0/3617041688 >> v2:172.21.15.196:3300/0 conn(0x7f348015b3d0 0x7f34800cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3485800640 1 -- 172.21.15.86:0/3617041688 >> 172.21.15.86:0/3617041688 conn(0x7f348006ef50 msgr2=0x7f348006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3485800640 1 -- 172.21.15.86:0/3617041688 shutdown_connections 2024-03-22T04:17:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.462+0000 7f3485800640 1 -- 172.21.15.86:0/3617041688 wait complete. 2024-03-22T04:17:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 Processor -- start 2024-03-22T04:17:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 -- start start 2024-03-22T04:17:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34800b4920 0x7f348015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800d01d0 0x7f3480146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f348015b3d0 0x7f3480146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34800d0660 con 0x7f34800d01d0 2024-03-22T04:17:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34800cd810 con 0x7f34800b4920 2024-03-22T04:17:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34800d07e0 con 0x7f348015b3d0 2024-03-22T04:17:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800d01d0 0x7f3480146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800d01d0 0x7f3480146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49936/0 (socket says 172.21.15.86:49936) 2024-03-22T04:17:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 -- 172.21.15.86:0/1642544959 learned_addr learned my addr 172.21.15.86:0/1642544959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f347f400640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f348015b3d0 0x7f3480146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f347ea00640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34800b4920 0x7f348015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 -- 172.21.15.86:0/1642544959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f348015b3d0 msgr2=0x7f3480146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f348015b3d0 0x7f3480146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 -- 172.21.15.86:0/1642544959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34800b4920 msgr2=0x7f348015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34800b4920 0x7f348015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 -- 172.21.15.86:0/1642544959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34801498f0 con 0x7f34800d01d0 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3476000640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800d01d0 0x7f3480146630 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f34680b3040 tx=0x7f346801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3477400640 1 -- 172.21.15.86:0/1642544959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34680b5040 con 0x7f34800d01d0 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3477400640 1 -- 172.21.15.86:0/1642544959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34680c0070 con 0x7f34800d01d0 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3477400640 1 -- 172.21.15.86:0/1642544959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34680b08e0 con 0x7f34800d01d0 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3480149bd0 con 0x7f34800d01d0 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.466+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348014e420 con 0x7f34800d01d0 2024-03-22T04:17:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.470+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3480074470 con 0x7f34800d01d0 2024-03-22T04:17:03.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.470+0000 7f3477400640 1 -- 172.21.15.86:0/1642544959 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f34680b0a80 con 0x7f34800d01d0 2024-03-22T04:17:03.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.470+0000 7f3477400640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f345c067b20 0x7f345c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.470+0000 7f3477400640 1 -- 172.21.15.86:0/1642544959 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f3468123450 con 0x7f34800d01d0 2024-03-22T04:17:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.470+0000 7f347ea00640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f345c067b20 0x7f345c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.470+0000 7f347ea00640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f345c067b20 0x7f345c069fe0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3470011340 tx=0x7f347000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.474+0000 7f3477400640 1 -- 172.21.15.86:0/1642544959 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f34680187f0 con 0x7f34800d01d0 2024-03-22T04:17:03.607 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:17:03.604+0000 7f495a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:17:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.686+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f348015e3a0 con 0x7f34800d01d0 2024-03-22T04:17:03.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.686+0000 7f3477400640 1 -- 172.21.15.86:0/1642544959 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f34680f35f0 con 0x7f34800d01d0 2024-03-22T04:17:03.688 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T04:17:03.688 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:17:03.688 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-03-22T04:17:03.688 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-03-22T04:17:03.688 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-22T04:17:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-03-22T04:17:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f345c067b20 msgr2=0x7f345c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f345c067b20 0x7f345c069fe0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3470011340 tx=0x7f347000b040 comp rx=0 tx=0).stop 2024-03-22T04:17:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800d01d0 msgr2=0x7f3480146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800d01d0 0x7f3480146630 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f34680b3040 tx=0x7f346801ba30 comp rx=0 tx=0).stop 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 shutdown_connections 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f345c067b20 0x7f345c069fe0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f348015b3d0 0x7f3480146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800d01d0 0x7f3480146630 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 --2- 172.21.15.86:0/1642544959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34800b4920 0x7f348015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 >> 172.21.15.86:0/1642544959 conn(0x7f348006ef50 msgr2=0x7f348006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 shutdown_connections 2024-03-22T04:17:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.690+0000 7f3485800640 1 -- 172.21.15.86:0/1642544959 wait complete. 2024-03-22T04:17:03.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:17:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.842+0000 7fa5e2600640 1 Processor -- start 2024-03-22T04:17:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.842+0000 7fa5e2600640 1 -- start start 2024-03-22T04:17:03.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5e2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5dc0703e0 0x7fa5dc165a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5e2600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5dc0cf7e0 con 0x7fa5dc144420 2024-03-22T04:17:03.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5e2600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5dc0cea20 con 0x7fa5dc06cd80 2024-03-22T04:17:03.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5e2600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa5dc144800 con 0x7fa5dc0703e0 2024-03-22T04:17:03.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5dc0703e0 0x7fa5dc165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5dc0703e0 0x7fa5dc165a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49950/0 (socket says 172.21.15.86:49950) 2024-03-22T04:17:03.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5dbe00640 1 -- 172.21.15.86:0/280825943 learned_addr learned my addr 172.21.15.86:0/280825943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:03.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5dbe00640 1 -- 172.21.15.86:0/280825943 >> v1:172.21.15.196:6789/0 conn(0x7fa5dc06cd80 legacy=0x7fa5dc06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5dbe00640 1 -- 172.21.15.86:0/280825943 >> v1:172.21.15.86:6789/0 conn(0x7fa5dc144420 legacy=0x7fa5dc06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.846+0000 7fa5dbe00640 1 -- 172.21.15.86:0/280825943 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc0ced00 con 0x7fa5dc0703e0 2024-03-22T04:17:03.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5dbe00640 1 --2- 172.21.15.86:0/280825943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc165a00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400d180 tx=0x7fa5d401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bcf65be4b3c7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5daa00640 1 -- 172.21.15.86:0/280825943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40ad070 con 0x7fa5dc0703e0 2024-03-22T04:17:03.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5daa00640 1 -- 172.21.15.86:0/280825943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d40a7b50 con 0x7fa5dc0703e0 2024-03-22T04:17:03.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5daa00640 1 -- 172.21.15.86:0/280825943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40b1400 con 0x7fa5dc0703e0 2024-03-22T04:17:03.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- 172.21.15.86:0/280825943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 msgr2=0x7fa5dc165a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 --2- 172.21.15.86:0/280825943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc165a00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400d180 tx=0x7fa5d401ec10 comp rx=0 tx=0).stop 2024-03-22T04:17:03.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- 172.21.15.86:0/280825943 shutdown_connections 2024-03-22T04:17:03.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 --2- 172.21.15.86:0/280825943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc165a00 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- 172.21.15.86:0/280825943 >> 172.21.15.86:0/280825943 conn(0x7fa5dc071fe0 msgr2=0x7fa5dc0723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:03.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- 172.21.15.86:0/280825943 shutdown_connections 2024-03-22T04:17:03.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- 172.21.15.86:0/280825943 wait complete. 2024-03-22T04:17:03.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 Processor -- start 2024-03-22T04:17:03.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- start start 2024-03-22T04:17:03.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5dc06cd80 0x7fa5dc13bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5dc144420 0x7fa5dc13d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc1680d0 con 0x7fa5dc0703e0 2024-03-22T04:17:03.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc167f50 con 0x7fa5dc144420 2024-03-22T04:17:03.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5e2600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5dc168250 con 0x7fa5dc06cd80 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49966/0 (socket says 172.21.15.86:49966) 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.850+0000 7fa5d2a00640 1 -- 172.21.15.86:0/828142108 learned_addr learned my addr 172.21.15.86:0/828142108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5db400640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5dc06cd80 0x7fa5dc13bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5dbe00640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5dc144420 0x7fa5dc13d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d2a00640 1 -- 172.21.15.86:0/828142108 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5dc06cd80 msgr2=0x7fa5dc13bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d2a00640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5dc06cd80 0x7fa5dc13bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d2a00640 1 -- 172.21.15.86:0/828142108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5dc144420 msgr2=0x7fa5dc13d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d2a00640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5dc144420 0x7fa5dc13d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d2a00640 1 -- 172.21.15.86:0/828142108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc1428a0 con 0x7fa5dc0703e0 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d2a00640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc141ec0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c3a0 tx=0x7fa5c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d3e00640 1 -- 172.21.15.86:0/828142108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40052d0 con 0x7fa5dc0703e0 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc142b80 con 0x7fa5dc0703e0 2024-03-22T04:17:03.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d3e00640 1 -- 172.21.15.86:0/828142108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c4005470 con 0x7fa5dc0703e0 2024-03-22T04:17:03.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d3e00640 1 -- 172.21.15.86:0/828142108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4007260 con 0x7fa5dc0703e0 2024-03-22T04:17:03.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc143140 con 0x7fa5dc0703e0 2024-03-22T04:17:03.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc068650 con 0x7fa5dc0703e0 2024-03-22T04:17:03.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d3e00640 1 -- 172.21.15.86:0/828142108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa5c4007400 con 0x7fa5dc0703e0 2024-03-22T04:17:03.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d3e00640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5b8067b50 0x7fa5b806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:03.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.854+0000 7fa5d3e00640 1 -- 172.21.15.86:0/828142108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fa5c4085620 con 0x7fa5dc0703e0 2024-03-22T04:17:03.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.858+0000 7fa5db400640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5b8067b50 0x7fa5b806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:03.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.858+0000 7fa5db400640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5b8067b50 0x7fa5b806a010 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc007fd0 tx=0x7fa5cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:03.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:03.862+0000 7fa5d3e00640 1 -- 172.21.15.86:0/828142108 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa5c40556a0 con 0x7fa5dc0703e0 2024-03-22T04:17:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.046+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5dc0cff50 con 0x7fa5dc0703e0 2024-03-22T04:17:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.046+0000 7fa5d3e00640 1 -- 172.21.15.86:0/828142108 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5185 (secure 0 0 0) 0x7fa5c4059740 con 0x7fa5dc0703e0 2024-03-22T04:17:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:04.048 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:01.044333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76469":{"gid":76469,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1097103199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1097103199},{"type":"v1","addr":"172.21.15.196:6835","nonce":1097103199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76469,"qdb_cluster":[76469]},"id":6}]} 2024-03-22T04:17:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 64 2024-03-22T04:17:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5b8067b50 msgr2=0x7fa5b806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5b8067b50 0x7fa5b806a010 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc007fd0 tx=0x7fa5cc009040 comp rx=0 tx=0).stop 2024-03-22T04:17:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 msgr2=0x7fa5dc141ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc141ec0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c3a0 tx=0x7fa5c400c860 comp rx=0 tx=0).stop 2024-03-22T04:17:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 shutdown_connections 2024-03-22T04:17:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5b8067b50 0x7fa5b806a010 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5dc144420 0x7fa5dc13d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5dc0703e0 0x7fa5dc141ec0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 --2- 172.21.15.86:0/828142108 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5dc06cd80 0x7fa5dc13bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 >> 172.21.15.86:0/828142108 conn(0x7fa5dc071fe0 msgr2=0x7fa5dc06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 shutdown_connections 2024-03-22T04:17:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.050+0000 7fa5e2600640 1 -- 172.21.15.86:0/828142108 wait complete. 2024-03-22T04:17:04.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:16:58.021326+0000', 'modified': '2024-03-22T04:17:01.044333+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 76469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_76469': {'gid': 76469, 'name': 'b', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/1097103199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1097103199}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1097103199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 76469, 'qdb_cluster': [76469]} 2024-03-22T04:17:04.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:17:04.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T04:17:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f545a400640 1 Processor -- start 2024-03-22T04:17:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f545a400640 1 -- start start 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f545a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f545406c270 0x7f5454162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f545a400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5454162540 0x7f545415fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f545a400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54540b2bd0 con 0x7f5454147e10 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f545a400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f54540b2610 con 0x7f545406c270 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f545a400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f54540b28f0 con 0x7f5454162540 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f5453400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5454147e10 0x7f54540b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39452/0 (socket says 172.21.15.86:39452) 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f5453400640 1 -- 172.21.15.86:0/2213812038 learned_addr learned my addr 172.21.15.86:0/2213812038 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f544be00640 1 --2- 172.21.15.86:0/2213812038 >> v2:172.21.15.86:3300/0 conn(0x7f545406c270 0x7f5454162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f5453e00640 1 --2- 172.21.15.86:0/2213812038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5454162540 0x7f545415fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f5452a00640 1 -- 172.21.15.86:0/2213812038 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 173447506 0 0) 0x7f54540b2bd0 con 0x7f5454147e10 2024-03-22T04:17:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f5452a00640 1 -- 172.21.15.86:0/2213812038 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5438003530 con 0x7f5454147e10 2024-03-22T04:17:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f544be00640 1 -- 172.21.15.86:0/2213812038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5454162540 msgr2=0x7f545415fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f544be00640 1 --2- 172.21.15.86:0/2213812038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5454162540 0x7f545415fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f544be00640 1 -- 172.21.15.86:0/2213812038 >> v1:172.21.15.86:6789/0 conn(0x7f5454147e10 legacy=0x7f54540b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.614+0000 7f544be00640 1 -- 172.21.15.86:0/2213812038 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54540b5ce0 con 0x7f545406c270 2024-03-22T04:17:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f544be00640 1 --2- 172.21.15.86:0/2213812038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545406c270 0x7f5454162000 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f543c0097c0 tx=0x7f543c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50520735b167cef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f5452a00640 1 -- 172.21.15.86:0/2213812038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0b1070 con 0x7f545406c270 2024-03-22T04:17:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f5452a00640 1 -- 172.21.15.86:0/2213812038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543c0ac410 con 0x7f545406c270 2024-03-22T04:17:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f5452a00640 1 -- 172.21.15.86:0/2213812038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0ac7e0 con 0x7f545406c270 2024-03-22T04:17:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 -- 172.21.15.86:0/2213812038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545406c270 msgr2=0x7f5454162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 --2- 172.21.15.86:0/2213812038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545406c270 0x7f5454162000 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f543c0097c0 tx=0x7f543c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 -- 172.21.15.86:0/2213812038 shutdown_connections 2024-03-22T04:17:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 --2- 172.21.15.86:0/2213812038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5454162540 0x7f545415fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 --2- 172.21.15.86:0/2213812038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545406c270 0x7f5454162000 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 -- 172.21.15.86:0/2213812038 >> 172.21.15.86:0/2213812038 conn(0x7f54540b6620 msgr2=0x7f54540b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 -- 172.21.15.86:0/2213812038 shutdown_connections 2024-03-22T04:17:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 -- 172.21.15.86:0/2213812038 wait complete. 2024-03-22T04:17:04.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.618+0000 7f545a400640 1 Processor -- start 2024-03-22T04:17:04.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 -- start start 2024-03-22T04:17:04.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f545406c270 0x7f54540cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5454147e10 0x7f5454142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5454162540 0x7f5454143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5454160c40 con 0x7f5454147e10 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5454160ac0 con 0x7f5454162540 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5454162950 con 0x7f545406c270 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5454147e10 0x7f5454142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5454147e10 0x7f5454142ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49988/0 (socket says 172.21.15.86:49988) 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 -- 172.21.15.86:0/4215501573 learned_addr learned my addr 172.21.15.86:0/4215501573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f5453400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f545406c270 0x7f54540cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f5453e00640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5454162540 0x7f5454143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 -- 172.21.15.86:0/4215501573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f545406c270 msgr2=0x7f54540cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f545406c270 0x7f54540cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 -- 172.21.15.86:0/4215501573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5454162540 msgr2=0x7f5454143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5454162540 0x7f5454143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 -- 172.21.15.86:0/4215501573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54541474e0 con 0x7f5454147e10 2024-03-22T04:17:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544be00640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5454147e10 0x7f5454142ee0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f543c01a0d0 tx=0x7f543c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544b400640 1 -- 172.21.15.86:0/4215501573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0b1040 con 0x7f5454147e10 2024-03-22T04:17:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544b400640 1 -- 172.21.15.86:0/4215501573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543c0bb070 con 0x7f5454147e10 2024-03-22T04:17:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 -- 172.21.15.86:0/4215501573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54540ca6d0 con 0x7f5454147e10 2024-03-22T04:17:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f544b400640 1 -- 172.21.15.86:0/4215501573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c006e20 con 0x7f5454147e10 2024-03-22T04:17:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.622+0000 7f545a400640 1 -- 172.21.15.86:0/4215501573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54540cac90 con 0x7f5454147e10 2024-03-22T04:17:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544b400640 1 -- 172.21.15.86:0/4215501573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f543c0b5630 con 0x7f5454147e10 2024-03-22T04:17:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544b400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5430067b50 0x7f543006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544b400640 1 -- 172.21.15.86:0/4215501573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f543c11fbc0 con 0x7f5454147e10 2024-03-22T04:17:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f5453400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5430067b50 0x7f543006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f545d7c9000 1 Processor -- start 2024-03-22T04:17:04.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f545d7c9000 1 -- start start 2024-03-22T04:17:04.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f5453400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5430067b50 0x7f543006a010 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5444007fd0 tx=0x7f5444009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0213250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0213790 0x5647d01c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01c6c20 0x5647d01c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f545d7c9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647cffc4140 con 0x5647d0213790 2024-03-22T04:17:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f545d7c9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5647d0017b10 con 0x5647d0212e70 2024-03-22T04:17:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f545d7c9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5647cff9ec50 con 0x5647d01c6c20 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0213790 0x5647d01c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0213790 0x5647d01c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49994/0 (socket says 172.21.15.86:49994) 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544be00640 1 -- 172.21.15.86:0/1071458760 learned_addr learned my addr 172.21.15.86:0/1071458760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f5453e00640 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01c6c20 0x5647d01c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f5453400640 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0213250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544be00640 1 -- 172.21.15.86:0/1071458760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01c6c20 msgr2=0x5647d01c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544be00640 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01c6c20 0x5647d01c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544be00640 1 -- 172.21.15.86:0/1071458760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 msgr2=0x5647d0213250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544be00640 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0213250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.626+0000 7f544be00640 1 -- 172.21.15.86:0/1071458760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d005cf20 con 0x5647d0213790 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0213790 0x5647d01c66e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f543c0ec1b0 tx=0x7f543c0ef6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e45ae8c4878b798 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f5443e00640 1 -- 172.21.15.86:0/1071458760 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0f8620 con 0x5647d0213790 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f5443e00640 1 -- 172.21.15.86:0/1071458760 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543c0ee050 con 0x5647d0213790 2024-03-22T04:17:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f5443e00640 1 -- 172.21.15.86:0/1071458760 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0f60b0 con 0x5647d0213790 2024-03-22T04:17:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- 172.21.15.86:0/1071458760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0213790 msgr2=0x5647d01c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0213790 0x5647d01c66e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f543c0ec1b0 tx=0x7f543c0ef6b0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- 172.21.15.86:0/1071458760 shutdown_connections 2024-03-22T04:17:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01c6c20 0x5647d01c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0213790 0x5647d01c66e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 --2- 172.21.15.86:0/1071458760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0213250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- 172.21.15.86:0/1071458760 >> 172.21.15.86:0/1071458760 conn(0x5647d01f3c10 msgr2=0x5647d01f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- 172.21.15.86:0/1071458760 shutdown_connections 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- 172.21.15.86:0/1071458760 wait complete. 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- start start 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 Processor -- start 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d01ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d0213790 0x5647d01cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647cffd1ab0 con 0x5647d0212e70 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5647d0013ac0 con 0x5647d01c6c20 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5647cffc4140 con 0x5647d0213790 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 10 client.0 _ll_register_callbacks cb 0x5647d01b4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50006/0 (socket says 172.21.15.86:50006) 2024-03-22T04:17:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f5453400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d01ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 -- 172.21.15.86:0/4273527407 learned_addr learned my addr 172.21.15.86:0/4273527407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f5453e00640 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d0213790 0x5647d01cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 -- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d0213790 msgr2=0x5647d01cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d0213790 0x5647d01cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 -- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 msgr2=0x5647d01ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d01ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 -- 172.21.15.86:0/4273527407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647cffed570 con 0x5647d0212e70 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f544be00640 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb7b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f543c0097c0 tx=0x7f543c0b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aba7eee68ddbdb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f5440c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f5440c00640 1 -- 172.21.15.86:0/4273527407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0f8620 con 0x5647d0212e70 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f5440c00640 1 -- 172.21.15.86:0/4273527407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f543c10e280 con 0x5647d0212e70 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.630+0000 7f545d7c9000 1 -- 172.21.15.86:0/4273527407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f543c0ee050 con 0x5647d0212e70 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5440c00640 1 -- 172.21.15.86:0/4273527407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f543c0ef1a0 con 0x5647d0212e70 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 10 client.7632 fetch_fsmap learned FSMap version 64 2024-03-22T04:17:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 -- 172.21.15.86:0/4273527407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x5647cffe3880 con 0x5647d0212e70 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5440c00640 1 -- 172.21.15.86:0/4273527407 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543c0ee050 con 0x5647d0212e70 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5440c00640 1 -- 172.21.15.86:0/4273527407 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f543c0b4050 con 0x5647d0212e70 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 10 client.7632 fetch_fsmap finished waiting for FSMap version 64 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 10 client.7632 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 20 client.7632 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] conn(0x5647d01cca60 0x5647d0224120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 4 client.7632 mds_command: new command op to 76469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 -- 172.21.15.86:0/4273527407 --> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647cffb2bc0 con 0x5647d01cca60 2024-03-22T04:17:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5453400640 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] conn(0x5647d01cca60 0x5647d0224120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5453400640 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] conn(0x5647d01cca60 0x5647d0224120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eadb0a27f98fa10 server_cookie=3ec1eeeae152149a in_seq=0 out_seq=0 2024-03-22T04:17:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5440c00640 10 client.7632 ms_handle_connect on v2:172.21.15.196:6834/1097103199 2024-03-22T04:17:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5440c00640 1 -- 172.21.15.86:0/4273527407 <== mds.0 v2:172.21.15.196:6834/1097103199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5647cffb2bc0 con 0x5647d01cca60 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5440c00640 10 client.7632 handle_command_reply: tid=0 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 client.7632 shutdown 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 -- 172.21.15.86:0/4273527407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 msgr2=0x5647d01cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb7b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f543c0097c0 tx=0x7f543c0b6000 comp rx=0 tx=0).stop 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 -- 172.21.15.86:0/4273527407 shutdown_connections 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] conn(0x5647d01cca60 0x5647d0224120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d0213790 0x5647d01cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb7b0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f5440c00640 0 client.7632 ms_handle_reset on v2:172.21.15.196:6834/1097103199 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 --2- 172.21.15.86:0/4273527407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d01ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 -- 172.21.15.86:0/4273527407 >> 172.21.15.86:0/4273527407 conn(0x5647d01f3c10 msgr2=0x5647d01f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 -- 172.21.15.86:0/4273527407 shutdown_connections 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 1 -- 172.21.15.86:0/4273527407 wait complete. 2024-03-22T04:17:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.634+0000 7f545d7c9000 20 client.7632 trim_cache size 0 max 16384 2024-03-22T04:17:04.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 Processor -- start 2024-03-22T04:17:04.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- start start 2024-03-22T04:17:04.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0212350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d0212890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5454202350 con 0x5647d0212e70 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5647d0017b10 con 0x5647d01c6c20 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5647d0057a60 con 0x5647d01f3c10 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50014/0 (socket says 172.21.15.86:50014) 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 -- 172.21.15.86:0/447269373 learned_addr learned my addr 172.21.15.86:0/447269373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453400640 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0212350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f544be00640 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d0212890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 -- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 msgr2=0x5647d0212350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0212350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 -- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 msgr2=0x5647d0212890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d0212890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 -- 172.21.15.86:0/447269373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647cff9fe80 con 0x5647d0212e70 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453400640 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0212350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f544be00640 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d0212890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5453e00640 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb380 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f544c009d40 tx=0x7f544c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2f3281d87fdb56 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5443400640 1 -- 172.21.15.86:0/447269373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c0158c0 con 0x5647d0212e70 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5443400640 1 -- 172.21.15.86:0/447269373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544c015a60 con 0x5647d0212e70 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f5443400640 1 -- 172.21.15.86:0/447269373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c015e50 con 0x5647d0212e70 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- 172.21.15.86:0/447269373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 msgr2=0x5647d01cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb380 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f544c009d40 tx=0x7f544c009750 comp rx=0 tx=0).stop 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- 172.21.15.86:0/447269373 shutdown_connections 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d01c6c20 0x5647d0212890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d0212e70 0x5647d01cb380 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 --2- 172.21.15.86:0/447269373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0212350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- 172.21.15.86:0/447269373 >> 172.21.15.86:0/447269373 conn(0x5647d01c5ee0 msgr2=0x5647d01cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:04.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- 172.21.15.86:0/447269373 shutdown_connections 2024-03-22T04:17:04.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.654+0000 7f545d7c9000 1 -- 172.21.15.86:0/447269373 wait complete. 2024-03-22T04:17:04.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 -- start start 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 Processor -- start 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d01c6c20 0x5647d0282940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0282e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0224ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647d0013ac0 con 0x5647d01c6c20 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5647cffd1ab0 con 0x5647d0212e70 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5647cffc4140 con 0x5647d01f3c10 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 10 client.0 _ll_register_callbacks cb 0x7f54440012e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d01c6c20 0x5647d0282940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d01c6c20 0x5647d0282940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50022/0 (socket says 172.21.15.86:50022) 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 -- 172.21.15.86:0/343605042 learned_addr learned my addr 172.21.15.86:0/343605042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f544be00640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0282e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453e00640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0224ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 -- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 msgr2=0x5647d0282e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0282e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 -- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 msgr2=0x5647d0224ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0224ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 -- 172.21.15.86:0/343605042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647cffab750 con 0x5647d01c6c20 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f544be00640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0282e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453e00640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0224ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5453400640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d01c6c20 0x5647d0282940 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f545409f430 tx=0x7f5444000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ab94a03ca9dd62 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 1 -- 172.21.15.86:0/343605042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54440287e0 con 0x5647d01c6c20 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 1 -- 172.21.15.86:0/343605042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f5444010040 con 0x5647d01c6c20 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 -- 172.21.15.86:0/343605042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544c015a60 con 0x5647d01c6c20 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 1 -- 172.21.15.86:0/343605042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f5444037050 con 0x5647d01c6c20 2024-03-22T04:17:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 10 client.7638 fetch_fsmap learned FSMap version 64 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 -- 172.21.15.86:0/343605042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x5647cffbe450 con 0x5647d01c6c20 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 1 -- 172.21.15.86:0/343605042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544c015a60 con 0x5647d01c6c20 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 1 -- 172.21.15.86:0/343605042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f544403a020 con 0x5647d01c6c20 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 10 client.7638 fetch_fsmap finished waiting for FSMap version 64 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 10 client.7638 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 20 client.7638 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] conn(0x5647d0213790 0x5647d0225420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 4 client.7638 mds_command: new command op to 76469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 -- 172.21.15.86:0/343605042 --> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] -- command(tid 0: {"prefix": "status"}) -- 0x5647d01cf0e0 con 0x5647d0213790 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f544be00640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] conn(0x5647d0213790 0x5647d0225420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f544be00640 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] conn(0x5647d0213790 0x5647d0225420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff4efe21391b530 server_cookie=3909973e652b715f in_seq=0 out_seq=0 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 10 client.7638 ms_handle_connect on v2:172.21.15.196:6834/1097103199 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 1 -- 172.21.15.86:0/343605042 <== mds.0 v2:172.21.15.196:6834/1097103199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5647d01cf0e0 con 0x5647d0213790 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f5440c00640 10 client.7638 handle_command_reply: tid=0 2024-03-22T04:17:04.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.658+0000 7f545d7c9000 1 client.7638 shutdown 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 -- 172.21.15.86:0/343605042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d01c6c20 msgr2=0x5647d0282940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d01c6c20 0x5647d0282940 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f545409f430 tx=0x7f5444000bd0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 -- 172.21.15.86:0/343605042 shutdown_connections 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:6834/1097103199,v1:172.21.15.196:6835/1097103199] conn(0x5647d0213790 0x5647d0225420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5647d0212e70 0x5647d0224ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5647d01f3c10 0x5647d0282e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 --2- 172.21.15.86:0/343605042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647d01c6c20 0x5647d0282940 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f5440c00640 0 client.7638 ms_handle_reset on v2:172.21.15.196:6834/1097103199 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 -- 172.21.15.86:0/343605042 >> 172.21.15.86:0/343605042 conn(0x5647d01c5ee0 msgr2=0x5647d0210f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 -- 172.21.15.86:0/343605042 shutdown_connections 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 1 -- 172.21.15.86:0/343605042 wait complete. 2024-03-22T04:17:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545d7c9000 20 client.7638 trim_cache size 0 max 16384 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 76469, 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.0053853359999998, 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 64, 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 11.118794057000001 2024-03-22T04:17:04.666 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:17:04.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 -- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5430067b50 msgr2=0x7f543006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5430067b50 0x7f543006a010 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5444007fd0 tx=0x7f5444009040 comp rx=0 tx=0).stop 2024-03-22T04:17:04.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 -- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5454147e10 msgr2=0x7f5454142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5454147e10 0x7f5454142ee0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f543c01a0d0 tx=0x7f543c019240 comp rx=0 tx=0).stop 2024-03-22T04:17:04.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 -- 172.21.15.86:0/4215501573 shutdown_connections 2024-03-22T04:17:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5430067b50 0x7f543006a010 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5454162540 0x7f5454143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5454147e10 0x7f5454142ee0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 --2- 172.21.15.86:0/4215501573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f545406c270 0x7f54540cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 -- 172.21.15.86:0/4215501573 >> 172.21.15.86:0/4215501573 conn(0x7f54540b6620 msgr2=0x7f545415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 -- 172.21.15.86:0/4215501573 shutdown_connections 2024-03-22T04:17:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.662+0000 7f545a400640 1 -- 172.21.15.86:0/4215501573 wait complete. 2024-03-22T04:17:04.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:17:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a51200640 1 Processor -- start 2024-03-22T04:17:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a51200640 1 -- start start 2024-03-22T04:17:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a51200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1a4c06bd70 0x7f1a4c0aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a51200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c1601d0 con 0x7f1a4c0af530 2024-03-22T04:17:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a51200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c160490 con 0x7f1a4c0b0810 2024-03-22T04:17:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a51200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1a4c160770 con 0x7f1a4c06bd70 2024-03-22T04:17:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a4a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1a4c0af530 0x7f1a4c0d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39464/0 (socket says 172.21.15.86:39464) 2024-03-22T04:17:04.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a4a800640 1 -- 172.21.15.86:0/1671018710 learned_addr learned my addr 172.21.15.86:0/1671018710 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a41400640 1 --2- 172.21.15.86:0/1671018710 >> v2:172.21.15.196:3300/0 conn(0x7f1a4c06bd70 0x7f1a4c0aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a49400640 1 -- 172.21.15.86:0/1671018710 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710427135 0 0) 0x7f1a4c1601d0 con 0x7f1a4c0af530 2024-03-22T04:17:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a49400640 1 -- 172.21.15.86:0/1671018710 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a30003530 con 0x7f1a4c0af530 2024-03-22T04:17:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a41400640 1 -- 172.21.15.86:0/1671018710 >> v1:172.21.15.196:6789/0 conn(0x7f1a4c0b0810 legacy=0x7f1a4c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a41400640 1 -- 172.21.15.86:0/1671018710 >> v1:172.21.15.86:6789/0 conn(0x7f1a4c0af530 legacy=0x7f1a4c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.818+0000 7f1a41400640 1 -- 172.21.15.86:0/1671018710 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c160a50 con 0x7f1a4c06bd70 2024-03-22T04:17:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a41400640 1 --2- 172.21.15.86:0/1671018710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c0aeff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1a340097c0 tx=0x7f1a3401cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360993b6855c12b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a49400640 1 -- 172.21.15.86:0/1671018710 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340b1070 con 0x7f1a4c06bd70 2024-03-22T04:17:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a49400640 1 -- 172.21.15.86:0/1671018710 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a340ac410 con 0x7f1a4c06bd70 2024-03-22T04:17:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a49400640 1 -- 172.21.15.86:0/1671018710 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340ac7c0 con 0x7f1a4c06bd70 2024-03-22T04:17:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a51200640 1 -- 172.21.15.86:0/1671018710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c06bd70 msgr2=0x7f1a4c0aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a51200640 1 --2- 172.21.15.86:0/1671018710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c0aeff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1a340097c0 tx=0x7f1a3401cf00 comp rx=0 tx=0).stop 2024-03-22T04:17:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a51200640 1 -- 172.21.15.86:0/1671018710 shutdown_connections 2024-03-22T04:17:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a51200640 1 --2- 172.21.15.86:0/1671018710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c0aeff0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a51200640 1 -- 172.21.15.86:0/1671018710 >> 172.21.15.86:0/1671018710 conn(0x7f1a4c0b2d00 msgr2=0x7f1a4c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a51200640 1 -- 172.21.15.86:0/1671018710 shutdown_connections 2024-03-22T04:17:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.822+0000 7f1a51200640 1 -- 172.21.15.86:0/1671018710 wait complete. 2024-03-22T04:17:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 Processor -- start 2024-03-22T04:17:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 -- start start 2024-03-22T04:17:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c0af530 0x7f1a4c14f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a4c0b0810 0x7f1a4c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c15ce00 con 0x7f1a4c06bd70 2024-03-22T04:17:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c15cc80 con 0x7f1a4c0af530 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c15cf80 con 0x7f1a4c0b0810 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50038/0 (socket says 172.21.15.86:50038) 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 -- 172.21.15.86:0/1066575305 learned_addr learned my addr 172.21.15.86:0/1066575305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a41400640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c0af530 0x7f1a4c14f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a4a800640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a4c0b0810 0x7f1a4c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 -- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a4c0b0810 msgr2=0x7f1a4c15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a4c0b0810 0x7f1a4c15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 -- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c0af530 msgr2=0x7f1a4c14f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c0af530 0x7f1a4c14f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 -- 172.21.15.86:0/1066575305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c0c8240 con 0x7f1a4c06bd70 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a41400640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c0af530 0x7f1a4c14f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a49e00640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c153bd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00c730 tx=0x7f1a3c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a42a00640 1 -- 172.21.15.86:0/1066575305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c016020 con 0x7f1a4c06bd70 2024-03-22T04:17:04.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a42a00640 1 -- 172.21.15.86:0/1066575305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3c006d60 con 0x7f1a4c06bd70 2024-03-22T04:17:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c0c84e0 con 0x7f1a4c06bd70 2024-03-22T04:17:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a42a00640 1 -- 172.21.15.86:0/1066575305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c003bc0 con 0x7f1a4c06bd70 2024-03-22T04:17:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c0c8aa0 con 0x7f1a4c06bd70 2024-03-22T04:17:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.826+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a4c068650 con 0x7f1a4c06bd70 2024-03-22T04:17:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.830+0000 7f1a42a00640 1 -- 172.21.15.86:0/1066575305 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1a3c00a850 con 0x7f1a4c06bd70 2024-03-22T04:17:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.830+0000 7f1a42a00640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1a28067b20 0x7f1a28069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.830+0000 7f1a42a00640 1 -- 172.21.15.86:0/1066575305 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f1a3c085ba0 con 0x7f1a4c06bd70 2024-03-22T04:17:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.830+0000 7f1a41400640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1a28067b20 0x7f1a28069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.830+0000 7f1a41400640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1a28067b20 0x7f1a28069fe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1a340ae000 tx=0x7f1a34004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:04.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:04.834+0000 7f1a42a00640 1 -- 172.21.15.86:0/1066575305 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1a3c020020 con 0x7f1a4c06bd70 2024-03-22T04:17:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.022+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1a4c0d02c0 con 0x7f1a4c06bd70 2024-03-22T04:17:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a42a00640 1 -- 172.21.15.86:0/1066575305 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7f1a3c055e60 con 0x7f1a4c06bd70 2024-03-22T04:17:05.024 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:17:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1a28067b20 msgr2=0x7f1a28069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1a28067b20 0x7f1a28069fe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1a340ae000 tx=0x7f1a34004740 comp rx=0 tx=0).stop 2024-03-22T04:17:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a4c06bd70 msgr2=0x7f1a4c153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c153bd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00c730 tx=0x7f1a3c00cbf0 comp rx=0 tx=0).stop 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 shutdown_connections 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1a28067b20 0x7f1a28069fe0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a4c0af530 0x7f1a4c14f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a4c0b0810 0x7f1a4c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 --2- 172.21.15.86:0/1066575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a4c06bd70 0x7f1a4c153bd0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 >> 172.21.15.86:0/1066575305 conn(0x7f1a4c0b2d00 msgr2=0x7f1a4c06e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 shutdown_connections 2024-03-22T04:17:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05.026+0000 7f1a51200640 1 -- 172.21.15.86:0/1066575305 wait complete. 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:17:05.045 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:17:05.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.046 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:17:05.056 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42107sec preferred_lft 42107sec 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:17:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:05.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.059 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:17:05.122 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:17:05.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.124 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:17:05.183 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:17:05.183 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:17:05.183 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:17:05.183 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:17:05.183 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:17:05.184 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:17:05.184 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:17:05.184 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:17:05.184 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:17:05.184 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:17:05.184 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:17:05.184 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:17:05.184 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:17:05.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:17:05.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:17:05.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:17:05.185 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:17:05.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:17:05.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:17:05.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.317 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:17:05.325 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:17:05.325 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:17:05.326 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:17:05.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:17:05.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:17:05.389 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:05.440 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:17:05.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:17:05.440 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:05.495 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:17:05.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:17:05.584 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:17:05.584 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:17:05.584 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:17:05.599 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:17:05.599 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:17:05.599 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:17:05.599 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:17:05.599 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:17:05.609 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:17:05.609 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:17:05.609 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:17:05.609 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:17:05.610 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:17:05.610 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:17:05.610 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:17:05.610 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:17:05.614 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:17:05.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:05.676 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:17:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:17:05.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:17:05.781 INFO:teuthology.orchestra.run.smithi086.stdout:client7423 2024-03-22T04:17:05.781 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:17:05.781 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:17:05.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.790 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-03-22T04:17:05.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.847 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-03-22T04:17:05.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.898 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-03-22T04:17:05.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:05.949 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-03-22T04:17:06.005 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:17:06.005 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:17:06.005 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:17:06.005 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:17:06.005 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-03-22T04:17:06.005 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:17:06.114 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-03-22T04:17:06.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:17:06.123 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:17:06.123 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:17:06.176 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:17:06.176 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:17:06.228 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:17:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e3b000640 1 Processor -- start 2024-03-22T04:17:06.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e3b000640 1 -- start start 2024-03-22T04:17:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e3b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e340ad330 0x7f9e3415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e3b000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 0x7f9e34142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e3b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e3415c7c0 con 0x7f9e3406ef40 2024-03-22T04:17:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e3b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9e3406dea0 con 0x7f9e340ad330 2024-03-22T04:17:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e3b000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3406e180 con 0x7f9e340ad710 2024-03-22T04:17:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e33e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e340ad330 0x7f9e3415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e2b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9e3406ef40 0x7f9e3413fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41500/0 (socket says 172.21.15.86:41500) 2024-03-22T04:17:06.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e2b400640 1 -- 172.21.15.86:0/571654229 learned_addr learned my addr 172.21.15.86:0/571654229 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e39000640 1 --2- 172.21.15.86:0/571654229 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 0x7f9e34142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e33e00640 1 -- 172.21.15.86:0/571654229 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 msgr2=0x7f9e34142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e33e00640 1 --2- 172.21.15.86:0/571654229 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 0x7f9e34142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e33e00640 1 -- 172.21.15.86:0/571654229 >> v1:172.21.15.86:6789/0 conn(0x7f9e3406ef40 legacy=0x7f9e3413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e33e00640 1 -- 172.21.15.86:0/571654229 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3406e460 con 0x7f9e340ad330 2024-03-22T04:17:06.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.406+0000 7f9e33400640 1 -- 172.21.15.86:0/571654229 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2006460335 0 0) 0x7f9e3415c7c0 con 0x7f9e3406ef40 2024-03-22T04:17:06.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e33e00640 1 --2- 172.21.15.86:0/571654229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e3415d4a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9e240097c0 tx=0x7f9e2401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247351b30c24df6e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:06.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e33400640 1 -- 172.21.15.86:0/571654229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e240a7070 con 0x7f9e340ad330 2024-03-22T04:17:06.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e33400640 1 -- 172.21.15.86:0/571654229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e240b0070 con 0x7f9e340ad330 2024-03-22T04:17:06.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e33400640 1 -- 172.21.15.86:0/571654229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e240ab890 con 0x7f9e340ad330 2024-03-22T04:17:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e3b000640 1 -- 172.21.15.86:0/571654229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 msgr2=0x7f9e3415d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e3b000640 1 --2- 172.21.15.86:0/571654229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e3415d4a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9e240097c0 tx=0x7f9e2401c470 comp rx=0 tx=0).stop 2024-03-22T04:17:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e3b000640 1 -- 172.21.15.86:0/571654229 shutdown_connections 2024-03-22T04:17:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e3b000640 1 --2- 172.21.15.86:0/571654229 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 0x7f9e34142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e3b000640 1 --2- 172.21.15.86:0/571654229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e3415d4a0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e3b000640 1 -- 172.21.15.86:0/571654229 >> 172.21.15.86:0/571654229 conn(0x7f9e340b1df0 msgr2=0x7f9e340b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e3b000640 1 -- 172.21.15.86:0/571654229 shutdown_connections 2024-03-22T04:17:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.410+0000 7f9e3b000640 1 -- 172.21.15.86:0/571654229 wait complete. 2024-03-22T04:17:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 Processor -- start 2024-03-22T04:17:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 -- start start 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e3406ef40 0x7f9e3415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e340affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 0x7f9e34098d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e341690e0 con 0x7f9e340ad330 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e34168f60 con 0x7f9e3406ef40 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e34169260 con 0x7f9e340ad710 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e340affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e340affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57256/0 (socket says 172.21.15.86:57256) 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 -- 172.21.15.86:0/2334852476 learned_addr learned my addr 172.21.15.86:0/2334852476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e39000640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 0x7f9e34098d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e33e00640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e3406ef40 0x7f9e3415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 -- 172.21.15.86:0/2334852476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 msgr2=0x7f9e34098d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 0x7f9e34098d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 -- 172.21.15.86:0/2334852476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e3406ef40 msgr2=0x7f9e3415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e3406ef40 0x7f9e3415a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 -- 172.21.15.86:0/2334852476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e340b33e0 con 0x7f9e340ad330 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e2b400640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e340affc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0011f0 tx=0x7f9e1c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e30c00640 1 -- 172.21.15.86:0/2334852476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c018070 con 0x7f9e340ad330 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e30c00640 1 -- 172.21.15.86:0/2334852476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e1c0074e0 con 0x7f9e340ad330 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e30c00640 1 -- 172.21.15.86:0/2334852476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c004a10 con 0x7f9e340ad330 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e340b36c0 con 0x7f9e340ad330 2024-03-22T04:17:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.414+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e340b3c80 con 0x7f9e340ad330 2024-03-22T04:17:06.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.418+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e34068650 con 0x7f9e340ad330 2024-03-22T04:17:06.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.418+0000 7f9e30c00640 1 -- 172.21.15.86:0/2334852476 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9e1c004bb0 con 0x7f9e340ad330 2024-03-22T04:17:06.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.418+0000 7f9e30c00640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e10067b20 0x7f9e10069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.418+0000 7f9e30c00640 1 -- 172.21.15.86:0/2334852476 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f9e1c085470 con 0x7f9e340ad330 2024-03-22T04:17:06.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.418+0000 7f9e33e00640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e10067b20 0x7f9e10069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.418+0000 7f9e33e00640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e10067b20 0x7f9e10069fe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9e240af000 tx=0x7f9e240a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.422+0000 7f9e30c00640 1 -- 172.21.15.86:0/2334852476 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9e1c055600 con 0x7f9e340ad330 2024-03-22T04:17:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.582+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e3415d260 con 0x7f9e340ad330 2024-03-22T04:17:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.582+0000 7f9e30c00640 1 -- 172.21.15.86:0/2334852476 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16933 (secure 0 0 0) 0x7f9e1c0596a0 con 0x7f9e340ad330 2024-03-22T04:17:06.583 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:06.583 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:16:59.639901+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e10067b20 msgr2=0x7f9e10069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e10067b20 0x7f9e10069fe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9e240af000 tx=0x7f9e240a7000 comp rx=0 tx=0).stop 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 msgr2=0x7f9e340affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e340affc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0011f0 tx=0x7f9e1c00c9b0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 shutdown_connections 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e10067b20 0x7f9e10069fe0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e340ad710 0x7f9e34098d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340ad330 0x7f9e340affc0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 --2- 172.21.15.86:0/2334852476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e3406ef40 0x7f9e3415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 >> 172.21.15.86:0/2334852476 conn(0x7f9e340b1df0 msgr2=0x7f9e3414bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 shutdown_connections 2024-03-22T04:17:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.586+0000 7f9e3b000640 1 -- 172.21.15.86:0/2334852476 wait complete. 2024-03-22T04:17:06.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:16:56.341603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:16:56.977675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:17:06.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:17:06.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.746+0000 7fb361c00640 1 Processor -- start 2024-03-22T04:17:06.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.746+0000 7fb361c00640 1 -- start start 2024-03-22T04:17:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.746+0000 7fb361c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb35c0b3600 0x7fb35c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb361c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c15d710 con 0x7fb35c0b2360 2024-03-22T04:17:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb361c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c15fe40 con 0x7fb35c15cee0 2024-03-22T04:17:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb361c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb35c0b2740 con 0x7fb35c0b3600 2024-03-22T04:17:06.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb35aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb35c0b2360 0x7fb35c0b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41512/0 (socket says 172.21.15.86:41512) 2024-03-22T04:17:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb35a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb35c0b3600 0x7fb35c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb35aa00640 1 -- 172.21.15.86:0/2276961343 learned_addr learned my addr 172.21.15.86:0/2276961343 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb35a000640 1 -- 172.21.15.86:0/2276961343 >> v1:172.21.15.196:6789/0 conn(0x7fb35c15cee0 legacy=0x7fb35c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb35a000640 1 -- 172.21.15.86:0/2276961343 >> v1:172.21.15.86:6789/0 conn(0x7fb35c0b2360 legacy=0x7fb35c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb35a000640 1 -- 172.21.15.86:0/2276961343 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c160120 con 0x7fb35c0b3600 2024-03-22T04:17:06.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb359600640 1 -- 172.21.15.86:0/2276961343 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524133951 0 0) 0x7fb35c15d710 con 0x7fb35c0b2360 2024-03-22T04:17:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb35a000640 1 --2- 172.21.15.86:0/2276961343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c0b3600 0x7fb35c15c9a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb3480097c0 tx=0x7fb34801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cdfe154e943ce2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb359600640 1 -- 172.21.15.86:0/2276961343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480a7070 con 0x7fb35c0b3600 2024-03-22T04:17:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb359600640 1 -- 172.21.15.86:0/2276961343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3480b0070 con 0x7fb35c0b3600 2024-03-22T04:17:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.750+0000 7fb359600640 1 -- 172.21.15.86:0/2276961343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480ab890 con 0x7fb35c0b3600 2024-03-22T04:17:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- 172.21.15.86:0/2276961343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c0b3600 msgr2=0x7fb35c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 --2- 172.21.15.86:0/2276961343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c0b3600 0x7fb35c15c9a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb3480097c0 tx=0x7fb34801c470 comp rx=0 tx=0).stop 2024-03-22T04:17:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- 172.21.15.86:0/2276961343 shutdown_connections 2024-03-22T04:17:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 --2- 172.21.15.86:0/2276961343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c0b3600 0x7fb35c15c9a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- 172.21.15.86:0/2276961343 >> 172.21.15.86:0/2276961343 conn(0x7fb35c0cc940 msgr2=0x7fb35c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- 172.21.15.86:0/2276961343 shutdown_connections 2024-03-22T04:17:06.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- 172.21.15.86:0/2276961343 wait complete. 2024-03-22T04:17:06.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 Processor -- start 2024-03-22T04:17:06.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- start start 2024-03-22T04:17:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35c0b2360 0x7fb35c14b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35c0b3600 0x7fb35c151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c15cee0 0x7fb35c14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c15d470 con 0x7fb35c15cee0 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c15d2f0 con 0x7fb35c0b2360 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb361c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c16a1f0 con 0x7fb35c0b3600 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c15cee0 0x7fb35c14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c15cee0 0x7fb35c14e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57278/0 (socket says 172.21.15.86:57278) 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35b400640 1 -- 172.21.15.86:0/4013798839 learned_addr learned my addr 172.21.15.86:0/4013798839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35aa00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35c0b2360 0x7fb35c14b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35a000640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35c0b3600 0x7fb35c151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35b400640 1 -- 172.21.15.86:0/4013798839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35c0b3600 msgr2=0x7fb35c151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35b400640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35c0b3600 0x7fb35c151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35b400640 1 -- 172.21.15.86:0/4013798839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35c0b2360 msgr2=0x7fb35c14b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35b400640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35c0b2360 0x7fb35c14b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.754+0000 7fb35b400640 1 -- 172.21.15.86:0/4013798839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c15dba0 con 0x7fb35c15cee0 2024-03-22T04:17:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb35b400640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c15cee0 0x7fb35c14e650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb35400c730 tx=0x7fb35400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb352a00640 1 -- 172.21.15.86:0/4013798839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354003b80 con 0x7fb35c15cee0 2024-03-22T04:17:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb352a00640 1 -- 172.21.15.86:0/4013798839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb354003d20 con 0x7fb35c15cee0 2024-03-22T04:17:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb352a00640 1 -- 172.21.15.86:0/4013798839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35400aa30 con 0x7fb35c15cee0 2024-03-22T04:17:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c15de80 con 0x7fb35c15cee0 2024-03-22T04:17:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c15e440 con 0x7fb35c15cee0 2024-03-22T04:17:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c074470 con 0x7fb35c15cee0 2024-03-22T04:17:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb352a00640 1 -- 172.21.15.86:0/4013798839 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb35400abd0 con 0x7fb35c15cee0 2024-03-22T04:17:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb352a00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb33c067b20 0x7fb33c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb352a00640 1 -- 172.21.15.86:0/4013798839 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fb354085e00 con 0x7fb35c15cee0 2024-03-22T04:17:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.758+0000 7fb35aa00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb33c067b20 0x7fb33c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:06.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.762+0000 7fb35aa00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb33c067b20 0x7fb33c069fe0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb34c000f70 tx=0x7fb34c007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.762+0000 7fb352a00640 1 -- 172.21.15.86:0/4013798839 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb354056020 con 0x7fb35c15cee0 2024-03-22T04:17:06.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.918+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb35c06e9f0 con 0x7fb35c15cee0 2024-03-22T04:17:06.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.918+0000 7fb352a00640 1 -- 172.21.15.86:0/4013798839 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fb35405a0c0 con 0x7fb35c15cee0 2024-03-22T04:17:06.920 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:17:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb33c067b20 msgr2=0x7fb33c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb33c067b20 0x7fb33c069fe0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb34c000f70 tx=0x7fb34c007240 comp rx=0 tx=0).stop 2024-03-22T04:17:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c15cee0 msgr2=0x7fb35c14e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c15cee0 0x7fb35c14e650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb35400c730 tx=0x7fb35400cbf0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 shutdown_connections 2024-03-22T04:17:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb33c067b20 0x7fb33c069fe0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb35c15cee0 0x7fb35c14e650 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35c0b3600 0x7fb35c151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 --2- 172.21.15.86:0/4013798839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35c0b2360 0x7fb35c14b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 >> 172.21.15.86:0/4013798839 conn(0x7fb35c0cc940 msgr2=0x7fb35c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 shutdown_connections 2024-03-22T04:17:06.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:06.922+0000 7fb361c00640 1 -- 172.21.15.86:0/4013798839 wait complete. 2024-03-22T04:17:06.941 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:17:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.078+0000 7f7e61600640 1 Processor -- start 2024-03-22T04:17:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e61600640 1 -- start start 2024-03-22T04:17:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e61600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e5c072650 0x7f7e5c072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e61600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 0x7f7e5c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e61600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c073150 con 0x7f7e5c06e7e0 2024-03-22T04:17:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e61600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5c071350 con 0x7f7e5c072650 2024-03-22T04:17:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e61600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c0715a0 con 0x7f7e5c15b760 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e5a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e5c072650 0x7f7e5c072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e5a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e5c072650 0x7f7e5c072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57288/0 (socket says 172.21.15.86:57288) 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e5a000640 1 -- 172.21.15.86:0/967145291 learned_addr learned my addr 172.21.15.86:0/967145291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e5b400640 1 --2- 172.21.15.86:0/967145291 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 0x7f7e5c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e5a000640 1 -- 172.21.15.86:0/967145291 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 msgr2=0x7f7e5c15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e5a000640 1 --2- 172.21.15.86:0/967145291 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 0x7f7e5c15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e5a000640 1 -- 172.21.15.86:0/967145291 >> v1:172.21.15.196:6789/0 conn(0x7f7e5c06e7e0 legacy=0x7f7e5c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e5a000640 1 -- 172.21.15.86:0/967145291 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c071880 con 0x7f7e5c072650 2024-03-22T04:17:07.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.082+0000 7f7e59600640 1 -- 172.21.15.86:0/967145291 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546986955 0 0) 0x7f7e5c073150 con 0x7f7e5c06e7e0 2024-03-22T04:17:07.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e5a000640 1 --2- 172.21.15.86:0/967145291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c072b00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800bb10 tx=0x7f7e4801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb52ed45b56687a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e59600640 1 -- 172.21.15.86:0/967145291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480b4070 con 0x7f7e5c072650 2024-03-22T04:17:07.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e59600640 1 -- 172.21.15.86:0/967145291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e480a7c00 con 0x7f7e5c072650 2024-03-22T04:17:07.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e59600640 1 -- 172.21.15.86:0/967145291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480af440 con 0x7f7e5c072650 2024-03-22T04:17:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 -- 172.21.15.86:0/967145291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 msgr2=0x7f7e5c072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 --2- 172.21.15.86:0/967145291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c072b00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800bb10 tx=0x7f7e4801eb70 comp rx=0 tx=0).stop 2024-03-22T04:17:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 -- 172.21.15.86:0/967145291 shutdown_connections 2024-03-22T04:17:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 --2- 172.21.15.86:0/967145291 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 0x7f7e5c15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 --2- 172.21.15.86:0/967145291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c072b00 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 -- 172.21.15.86:0/967145291 >> 172.21.15.86:0/967145291 conn(0x7f7e5c0cc940 msgr2=0x7f7e5c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 -- 172.21.15.86:0/967145291 shutdown_connections 2024-03-22T04:17:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 -- 172.21.15.86:0/967145291 wait complete. 2024-03-22T04:17:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 Processor -- start 2024-03-22T04:17:07.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.086+0000 7f7e61600640 1 -- start start 2024-03-22T04:17:07.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e5c06e7e0 0x7f7e5c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c14c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 0x7f7e5c153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c155c20 con 0x7f7e5c072650 2024-03-22T04:17:07.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c155aa0 con 0x7f7e5c06e7e0 2024-03-22T04:17:07.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c06d6d0 con 0x7f7e5c15b760 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c14c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c14c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57298/0 (socket says 172.21.15.86:57298) 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 -- 172.21.15.86:0/3611707228 learned_addr learned my addr 172.21.15.86:0/3611707228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5aa00640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e5c06e7e0 0x7f7e5c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5b400640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 0x7f7e5c153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 -- 172.21.15.86:0/3611707228 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 msgr2=0x7f7e5c153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 0x7f7e5c153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 -- 172.21.15.86:0/3611707228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e5c06e7e0 msgr2=0x7f7e5c154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e5c06e7e0 0x7f7e5c154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 -- 172.21.15.86:0/3611707228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c153bc0 con 0x7f7e5c072650 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e5a000640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c14c600 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7e4801c7a0 tx=0x7f7e4801b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e52a00640 1 -- 172.21.15.86:0/3611707228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480b4040 con 0x7f7e5c072650 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e52a00640 1 -- 172.21.15.86:0/3611707228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e480bf070 con 0x7f7e5c072650 2024-03-22T04:17:07.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c153ea0 con 0x7f7e5c072650 2024-03-22T04:17:07.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e52a00640 1 -- 172.21.15.86:0/3611707228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480ba660 con 0x7f7e5c072650 2024-03-22T04:17:07.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c154460 con 0x7f7e5c072650 2024-03-22T04:17:07.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c074470 con 0x7f7e5c072650 2024-03-22T04:17:07.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.090+0000 7f7e52a00640 1 -- 172.21.15.86:0/3611707228 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7e480ba800 con 0x7f7e5c072650 2024-03-22T04:17:07.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.094+0000 7f7e52a00640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e3c067b20 0x7f7e3c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.094+0000 7f7e52a00640 1 -- 172.21.15.86:0/3611707228 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f7e48123520 con 0x7f7e5c072650 2024-03-22T04:17:07.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.094+0000 7f7e5aa00640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e3c067b20 0x7f7e3c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.094+0000 7f7e5aa00640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e3c067b20 0x7f7e3c069fe0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c000f70 tx=0x7f7e4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.098+0000 7f7e52a00640 1 -- 172.21.15.86:0/3611707228 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7e48018820 con 0x7f7e5c072650 2024-03-22T04:17:07.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.254+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e5c1551a0 con 0x7f7e5c072650 2024-03-22T04:17:07.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.254+0000 7f7e52a00640 1 -- 172.21.15.86:0/3611707228 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f7e480f3e30 con 0x7f7e5c072650 2024-03-22T04:17:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e3c067b20 msgr2=0x7f7e3c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e3c067b20 0x7f7e3c069fe0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c000f70 tx=0x7f7e4c009040 comp rx=0 tx=0).stop 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 msgr2=0x7f7e5c14c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c14c600 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7e4801c7a0 tx=0x7f7e4801b910 comp rx=0 tx=0).stop 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 shutdown_connections 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e3c067b20 0x7f7e3c069fe0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e5c15b760 0x7f7e5c153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5c072650 0x7f7e5c14c600 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 --2- 172.21.15.86:0/3611707228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e5c06e7e0 0x7f7e5c154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 >> 172.21.15.86:0/3611707228 conn(0x7f7e5c0cc940 msgr2=0x7f7e5c15a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 shutdown_connections 2024-03-22T04:17:07.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.258+0000 7f7e61600640 1 -- 172.21.15.86:0/3611707228 wait complete. 2024-03-22T04:17:07.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:17:07.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.414+0000 7f8084200640 1 Processor -- start 2024-03-22T04:17:07.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.414+0000 7f8084200640 1 -- start start 2024-03-22T04:17:07.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.414+0000 7f8084200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f807c15e7e0 0x7f807c0d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.414+0000 7f8084200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c070560 0x7f807c0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.414+0000 7f8084200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f807c0d15f0 con 0x7f807c0b4710 2024-03-22T04:17:07.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.414+0000 7f8084200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f807c0b4b70 con 0x7f807c15e7e0 2024-03-22T04:17:07.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.414+0000 7f8084200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f807c06f970 con 0x7f807c070560 2024-03-22T04:17:07.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8081800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f807c15e7e0 0x7f807c0d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8081800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f807c15e7e0 0x7f807c0d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57310/0 (socket says 172.21.15.86:57310) 2024-03-22T04:17:07.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8080e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f807c0b4710 0x7f807c06fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:34286/0 (socket says 172.21.15.86:34286) 2024-03-22T04:17:07.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8082200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c070560 0x7f807c0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8080e00640 1 -- 172.21.15.86:0/1663369445 learned_addr learned my addr 172.21.15.86:0/1663369445 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:07.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8081800640 1 -- 172.21.15.86:0/1663369445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c070560 msgr2=0x7f807c0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:07.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8081800640 1 --2- 172.21.15.86:0/1663369445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c070560 0x7f807c0cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8081800640 1 -- 172.21.15.86:0/1663369445 >> v1:172.21.15.196:6789/0 conn(0x7f807c0b4710 legacy=0x7f807c06fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8081800640 1 -- 172.21.15.86:0/1663369445 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c0d1310 con 0x7f807c15e7e0 2024-03-22T04:17:07.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f807be00640 1 -- 172.21.15.86:0/1663369445 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763929550 0 0) 0x7f807c0d15f0 con 0x7f807c0b4710 2024-03-22T04:17:07.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8082200640 1 --2- 172.21.15.86:0/1663369445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c070560 0x7f807c0cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f8081800640 1 --2- 172.21.15.86:0/1663369445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c15e7e0 0x7f807c0d0d30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f80700097c0 tx=0x7f807001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99386c5829fc751 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f807be00640 1 -- 172.21.15.86:0/1663369445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700b0070 con 0x7f807c15e7e0 2024-03-22T04:17:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f807be00640 1 -- 172.21.15.86:0/1663369445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8070004230 con 0x7f807c15e7e0 2024-03-22T04:17:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.418+0000 7f807be00640 1 -- 172.21.15.86:0/1663369445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070003620 con 0x7f807c15e7e0 2024-03-22T04:17:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- 172.21.15.86:0/1663369445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c15e7e0 msgr2=0x7f807c0d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 --2- 172.21.15.86:0/1663369445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c15e7e0 0x7f807c0d0d30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f80700097c0 tx=0x7f807001c350 comp rx=0 tx=0).stop 2024-03-22T04:17:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- 172.21.15.86:0/1663369445 shutdown_connections 2024-03-22T04:17:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 --2- 172.21.15.86:0/1663369445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c070560 0x7f807c0cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 --2- 172.21.15.86:0/1663369445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c15e7e0 0x7f807c0d0d30 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- 172.21.15.86:0/1663369445 >> 172.21.15.86:0/1663369445 conn(0x7f807c0b3bf0 msgr2=0x7f807c0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- 172.21.15.86:0/1663369445 shutdown_connections 2024-03-22T04:17:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- 172.21.15.86:0/1663369445 wait complete. 2024-03-22T04:17:07.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 Processor -- start 2024-03-22T04:17:07.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- start start 2024-03-22T04:17:07.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c070560 0x7f807c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807c0b4710 0x7f807c1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c15e7e0 0x7f807c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0ce230 con 0x7f807c070560 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0ce0b0 con 0x7f807c0b4710 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8084200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f807c0ce3b0 con 0x7f807c15e7e0 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8081800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c070560 0x7f807c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8081800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c070560 0x7f807c0b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57316/0 (socket says 172.21.15.86:57316) 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8081800640 1 -- 172.21.15.86:0/1948600813 learned_addr learned my addr 172.21.15.86:0/1948600813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8080e00640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807c0b4710 0x7f807c1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8082200640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c15e7e0 0x7f807c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8081800640 1 -- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c15e7e0 msgr2=0x7f807c159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8081800640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c15e7e0 0x7f807c159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8081800640 1 -- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807c0b4710 msgr2=0x7f807c1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8081800640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807c0b4710 0x7f807c1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8081800640 1 -- 172.21.15.86:0/1948600813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c0cc260 con 0x7f807c070560 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.422+0000 7f8080e00640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807c0b4710 0x7f807c1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8081800640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c070560 0x7f807c0b00e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8070019f80 tx=0x7f80700190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8079600640 1 -- 172.21.15.86:0/1948600813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700b9070 con 0x7f807c070560 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8079600640 1 -- 172.21.15.86:0/1948600813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8070003e10 con 0x7f807c070560 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8079600640 1 -- 172.21.15.86:0/1948600813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700b0040 con 0x7f807c070560 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c15a150 con 0x7f807c070560 2024-03-22T04:17:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c15a710 con 0x7f807c070560 2024-03-22T04:17:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f807c074470 con 0x7f807c070560 2024-03-22T04:17:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8079600640 1 -- 172.21.15.86:0/1948600813 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f80700bc070 con 0x7f807c070560 2024-03-22T04:17:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8079600640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f805c067b20 0x7f805c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8079600640 1 -- 172.21.15.86:0/1948600813 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f80701202f0 con 0x7f807c070560 2024-03-22T04:17:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.426+0000 7f8080e00640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f805c067b20 0x7f805c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.430+0000 7f8080e00640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f805c067b20 0x7f805c069fe0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8068007fd0 tx=0x7f8068007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.430+0000 7f8079600640 1 -- 172.21.15.86:0/1948600813 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8070016090 con 0x7f807c070560 2024-03-22T04:17:07.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.586+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f807c06fc30 con 0x7f807c070560 2024-03-22T04:17:07.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.586+0000 7f8079600640 1 -- 172.21.15.86:0/1948600813 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f80700f0a40 con 0x7f807c070560 2024-03-22T04:17:07.587 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:17:07.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f805c067b20 msgr2=0x7f805c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f805c067b20 0x7f805c069fe0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8068007fd0 tx=0x7f8068007c20 comp rx=0 tx=0).stop 2024-03-22T04:17:07.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c070560 msgr2=0x7f807c0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c070560 0x7f807c0b00e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8070019f80 tx=0x7f80700190f0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 shutdown_connections 2024-03-22T04:17:07.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f805c067b20 0x7f805c069fe0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807c0b4710 0x7f807c1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f807c15e7e0 0x7f807c159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 --2- 172.21.15.86:0/1948600813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c070560 0x7f807c0b00e0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 >> 172.21.15.86:0/1948600813 conn(0x7f807c0b3bf0 msgr2=0x7f807c06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:07.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 shutdown_connections 2024-03-22T04:17:07.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.590+0000 7f8084200640 1 -- 172.21.15.86:0/1948600813 wait complete. 2024-03-22T04:17:07.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:17:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1fa1e00640 1 Processor -- start 2024-03-22T04:17:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1fa1e00640 1 -- start start 2024-03-22T04:17:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1fa1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f9c0aff50 0x7f1f9c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1fa1e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1f9c06cbb0 0x7f1f9c143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1fa1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c144320 0x7f1f9c06e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1fa1e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9c06d010 con 0x7f1f9c0aff50 2024-03-22T04:17:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1fa1e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9c162aa0 con 0x7f1f9c06cbb0 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1fa1e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c0ae5a0 con 0x7f1f9c144320 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1f9aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f9c0aff50 0x7f1f9c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1f9aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f9c0aff50 0x7f1f9c06c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57326/0 (socket says 172.21.15.86:57326) 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1f9aa00640 1 -- 172.21.15.86:0/4141496503 learned_addr learned my addr 172.21.15.86:0/4141496503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1f92000640 1 --2- 172.21.15.86:0/4141496503 >> v2:172.21.15.196:3300/0 conn(0x7f1f9c06cbb0 0x7f1f9c143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1f9b400640 1 --2- 172.21.15.86:0/4141496503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c144320 0x7f1f9c06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1f9aa00640 1 -- 172.21.15.86:0/4141496503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c144320 msgr2=0x7f1f9c06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.746+0000 7f1f9aa00640 1 --2- 172.21.15.86:0/4141496503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c144320 0x7f1f9c06e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1f9aa00640 1 -- 172.21.15.86:0/4141496503 >> v2:172.21.15.196:3300/0 conn(0x7f1f9c06cbb0 msgr2=0x7f1f9c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1f9aa00640 1 --2- 172.21.15.86:0/4141496503 >> v2:172.21.15.196:3300/0 conn(0x7f1f9c06cbb0 0x7f1f9c143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1f9aa00640 1 -- 172.21.15.86:0/4141496503 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c144b80 con 0x7f1f9c0aff50 2024-03-22T04:17:07.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1f9aa00640 1 --2- 172.21.15.86:0/4141496503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c0aff50 0x7f1f9c06c670 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c148360 tx=0x7f1f8c01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c68bacf2e14da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1f9a000640 1 -- 172.21.15.86:0/4141496503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0b0070 con 0x7f1f9c0aff50 2024-03-22T04:17:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1f9a000640 1 -- 172.21.15.86:0/4141496503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c0ab600 con 0x7f1f9c0aff50 2024-03-22T04:17:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1f9a000640 1 -- 172.21.15.86:0/4141496503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0ab9d0 con 0x7f1f9c0aff50 2024-03-22T04:17:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 -- 172.21.15.86:0/4141496503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c0aff50 msgr2=0x7f1f9c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/4141496503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c0aff50 0x7f1f9c06c670 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c148360 tx=0x7f1f8c01c1a0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 -- 172.21.15.86:0/4141496503 shutdown_connections 2024-03-22T04:17:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/4141496503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c144320 0x7f1f9c06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/4141496503 >> v2:172.21.15.196:3300/0 conn(0x7f1f9c06cbb0 0x7f1f9c143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/4141496503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c0aff50 0x7f1f9c06c670 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 -- 172.21.15.86:0/4141496503 >> 172.21.15.86:0/4141496503 conn(0x7f1f9c071fe0 msgr2=0x7f1f9c0723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 -- 172.21.15.86:0/4141496503 shutdown_connections 2024-03-22T04:17:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.750+0000 7f1fa1e00640 1 -- 172.21.15.86:0/4141496503 wait complete. 2024-03-22T04:17:07.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 Processor -- start 2024-03-22T04:17:07.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 -- start start 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f9c06cbb0 0x7f1f9c13f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c0aff50 0x7f1f9c141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c144320 0x7f1f9c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c162f30 con 0x7f1f9c144320 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c0b0330 con 0x7f1f9c06cbb0 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c15a7d0 con 0x7f1f9c0aff50 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c144320 0x7f1f9c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c144320 0x7f1f9c158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57334/0 (socket says 172.21.15.86:57334) 2024-03-22T04:17:07.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f92000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c0aff50 0x7f1f9c141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f9b400640 1 -- 172.21.15.86:0/3165553717 learned_addr learned my addr 172.21.15.86:0/3165553717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:07.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f9aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f9c06cbb0 0x7f1f9c13f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f92000640 1 -- 172.21.15.86:0/3165553717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f9c06cbb0 msgr2=0x7f1f9c13f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f92000640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f9c06cbb0 0x7f1f9c13f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f92000640 1 -- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c144320 msgr2=0x7f1f9c158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f92000640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c144320 0x7f1f9c158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f92000640 1 -- 172.21.15.86:0/3165553717 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c140ba0 con 0x7f1f9c0aff50 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f9aa00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f9c06cbb0 0x7f1f9c13f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f9b400640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c144320 0x7f1f9c158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f92000640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c0aff50 0x7f1f9c141bd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400e870 tx=0x7f1f8400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f93e00640 1 -- 172.21.15.86:0/3165553717 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84017070 con 0x7f1f9c0aff50 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f93e00640 1 -- 172.21.15.86:0/3165553717 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8400c940 con 0x7f1f9c0aff50 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1f93e00640 1 -- 172.21.15.86:0/3165553717 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400ccf0 con 0x7f1f9c0aff50 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.754+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c140e80 con 0x7f1f9c0aff50 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.758+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c15ace0 con 0x7f1f9c0aff50 2024-03-22T04:17:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.758+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c068650 con 0x7f1f9c0aff50 2024-03-22T04:17:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.758+0000 7f1f93e00640 1 -- 172.21.15.86:0/3165553717 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1f8401a070 con 0x7f1f9c0aff50 2024-03-22T04:17:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.758+0000 7f1f93e00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f78067b50 0x7f1f7806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.758+0000 7f1f93e00640 1 -- 172.21.15.86:0/3165553717 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f1f840862d0 con 0x7f1f9c0aff50 2024-03-22T04:17:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.758+0000 7f1f9aa00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f78067b50 0x7f1f7806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.758+0000 7f1f9aa00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f78067b50 0x7f1f7806a010 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0ad000 tx=0x7f1f8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.766+0000 7f1f93e00640 1 -- 172.21.15.86:0/3165553717 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1f840564f0 con 0x7f1f9c0aff50 2024-03-22T04:17:07.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f9c0b1890 con 0x7f1f9c0aff50 2024-03-22T04:17:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1f93e00640 1 -- 172.21.15.86:0/3165553717 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7f1f8405a590 con 0x7f1f9c0aff50 2024-03-22T04:17:07.937 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:17:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f78067b50 msgr2=0x7f1f7806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f78067b50 0x7f1f7806a010 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0ad000 tx=0x7f1f8c008040 comp rx=0 tx=0).stop 2024-03-22T04:17:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c0aff50 msgr2=0x7f1f9c141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c0aff50 0x7f1f9c141bd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400e870 tx=0x7f1f8400ed30 comp rx=0 tx=0).stop 2024-03-22T04:17:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 shutdown_connections 2024-03-22T04:17:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1f78067b50 0x7f1f7806a010 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f9c144320 0x7f1f9c158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f9c0aff50 0x7f1f9c141bd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/3165553717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f9c06cbb0 0x7f1f9c13f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 >> 172.21.15.86:0/3165553717 conn(0x7f1f9c071fe0 msgr2=0x7f1f9c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 shutdown_connections 2024-03-22T04:17:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:07.938+0000 7f1fa1e00640 1 -- 172.21.15.86:0/3165553717 wait complete. 2024-03-22T04:17:07.958 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:17:07.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:07.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:08.008 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:17:08.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:08.008 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:17:08.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:17:08.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.198+0000 7ff268400640 1 Processor -- start 2024-03-22T04:17:08.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.198+0000 7ff268400640 1 -- start start 2024-03-22T04:17:08.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff268400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 0x7ff26006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff268400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2600b3800 0x7ff26006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff268400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26006e350 con 0x7ff260161880 2024-03-22T04:17:08.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff268400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff26015ee80 con 0x7ff2600b3800 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff268400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff26006e630 con 0x7ff260161c60 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff266400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2600b3800 0x7ff26006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff266400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2600b3800 0x7ff26006dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57348/0 (socket says 172.21.15.86:57348) 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff266400640 1 -- 172.21.15.86:0/3163505075 learned_addr learned my addr 172.21.15.86:0/3163505075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff265000640 1 --2- 172.21.15.86:0/3163505075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 0x7ff26006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff266400640 1 -- 172.21.15.86:0/3163505075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 msgr2=0x7ff26006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff266400640 1 --2- 172.21.15.86:0/3163505075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 0x7ff26006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff266400640 1 -- 172.21.15.86:0/3163505075 >> v1:172.21.15.196:6789/0 conn(0x7ff260161880 legacy=0x7ff2600b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff266400640 1 -- 172.21.15.86:0/3163505075 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26006e910 con 0x7ff2600b3800 2024-03-22T04:17:08.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff266400640 1 --2- 172.21.15.86:0/3163505075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26006dc40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff258009e50 tx=0x7ff25801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0e71138a0d7fd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:08.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff25fe00640 1 -- 172.21.15.86:0/3163505075 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2580a5750 con 0x7ff2600b3800 2024-03-22T04:17:08.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff25fe00640 1 -- 172.21.15.86:0/3163505075 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2580a58f0 con 0x7ff2600b3800 2024-03-22T04:17:08.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.202+0000 7ff25fe00640 1 -- 172.21.15.86:0/3163505075 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2580a5cc0 con 0x7ff2600b3800 2024-03-22T04:17:08.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- 172.21.15.86:0/3163505075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 msgr2=0x7ff26006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 --2- 172.21.15.86:0/3163505075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26006dc40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff258009e50 tx=0x7ff25801c9a0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- 172.21.15.86:0/3163505075 shutdown_connections 2024-03-22T04:17:08.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 --2- 172.21.15.86:0/3163505075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 0x7ff26006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 --2- 172.21.15.86:0/3163505075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26006dc40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- 172.21.15.86:0/3163505075 >> 172.21.15.86:0/3163505075 conn(0x7ff2600b1df0 msgr2=0x7ff2600b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:08.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- 172.21.15.86:0/3163505075 shutdown_connections 2024-03-22T04:17:08.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- 172.21.15.86:0/3163505075 wait complete. 2024-03-22T04:17:08.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 Processor -- start 2024-03-22T04:17:08.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- start start 2024-03-22T04:17:08.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26009c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff260161880 0x7ff2600c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 0x7ff26015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2601608d0 con 0x7ff2600b3800 2024-03-22T04:17:08.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff260160750 con 0x7ff260161880 2024-03-22T04:17:08.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff268400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff260160a50 con 0x7ff260161c60 2024-03-22T04:17:08.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff265a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26009c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff265a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26009c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57356/0 (socket says 172.21.15.86:57356) 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff265a00640 1 -- 172.21.15.86:0/2564485872 learned_addr learned my addr 172.21.15.86:0/2564485872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff266400640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 0x7ff26015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff265000640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff260161880 0x7ff2600c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff265a00640 1 -- 172.21.15.86:0/2564485872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 msgr2=0x7ff26015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff265a00640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 0x7ff26015b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff265a00640 1 -- 172.21.15.86:0/2564485872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff260161880 msgr2=0x7ff2600c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.206+0000 7ff265a00640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff260161880 0x7ff2600c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff265a00640 1 -- 172.21.15.86:0/2564485872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26009cde0 con 0x7ff2600b3800 2024-03-22T04:17:08.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff265a00640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26009c7b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c420 tx=0x7ff25400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:08.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff25d600640 1 -- 172.21.15.86:0/2564485872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540052d0 con 0x7ff2600b3800 2024-03-22T04:17:08.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff25d600640 1 -- 172.21.15.86:0/2564485872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff254005470 con 0x7ff2600b3800 2024-03-22T04:17:08.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff25d600640 1 -- 172.21.15.86:0/2564485872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254007260 con 0x7ff2600b3800 2024-03-22T04:17:08.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26009d0c0 con 0x7ff2600b3800 2024-03-22T04:17:08.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26009d680 con 0x7ff2600b3800 2024-03-22T04:17:08.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2601480b0 con 0x7ff2600b3800 2024-03-22T04:17:08.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff25d600640 1 -- 172.21.15.86:0/2564485872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff254007400 con 0x7ff2600b3800 2024-03-22T04:17:08.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff25d600640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff240067b50 0x7ff24006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff25d600640 1 -- 172.21.15.86:0/2564485872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7ff254085480 con 0x7ff2600b3800 2024-03-22T04:17:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.210+0000 7ff265000640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff240067b50 0x7ff24006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.214+0000 7ff265000640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff240067b50 0x7ff24006a010 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b3680 tx=0x7ff24c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:08.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.214+0000 7ff25d600640 1 -- 172.21.15.86:0/2564485872 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff2540556a0 con 0x7ff2600b3800 2024-03-22T04:17:08.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.402+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff260068650 con 0x7ff2600b3800 2024-03-22T04:17:08.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff25d600640 1 -- 172.21.15.86:0/2564485872 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5185 (secure 0 0 0) 0x7ff254059740 con 0x7ff2600b3800 2024-03-22T04:17:08.404 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:08.405 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:01.044333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76469":{"gid":76469,"name":"b","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1097103199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1097103199},{"type":"v1","addr":"172.21.15.196:6835","nonce":1097103199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76469,"qdb_cluster":[76469]},"id":6}]} 2024-03-22T04:17:08.405 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 64 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff240067b50 msgr2=0x7ff24006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff240067b50 0x7ff24006a010 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b3680 tx=0x7ff24c009190 comp rx=0 tx=0).stop 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 msgr2=0x7ff26009c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26009c7b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c420 tx=0x7ff25400c8e0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 shutdown_connections 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff240067b50 0x7ff24006a010 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff260161880 0x7ff2600c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff260161c60 0x7ff26015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 --2- 172.21.15.86:0/2564485872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2600b3800 0x7ff26009c7b0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 >> 172.21.15.86:0/2564485872 conn(0x7ff2600b1df0 msgr2=0x7ff260072050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:08.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 shutdown_connections 2024-03-22T04:17:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.406+0000 7ff268400640 1 -- 172.21.15.86:0/2564485872 wait complete. 2024-03-22T04:17:08.425 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-22T04:17:08.593 INFO:teuthology.orchestra.run.smithi196.stdout:{"message":"","return_code":0} 2024-03-22T04:17:08.594 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:17:08.594 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:17:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.730+0000 7fece6400640 1 Processor -- start 2024-03-22T04:17:08.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.730+0000 7fece6400640 1 -- start start 2024-03-22T04:17:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fece6400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fece006ef40 0x7fece0142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fece6400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece015c7c0 con 0x7fece00ad710 2024-03-22T04:17:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fece6400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece006dea0 con 0x7fece00ad330 2024-03-22T04:17:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fece6400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fece006e180 con 0x7fece006ef40 2024-03-22T04:17:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fecdfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fece00ad710 0x7fece01402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41514/0 (socket says 172.21.15.86:41514) 2024-03-22T04:17:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fecdfe00640 1 -- 172.21.15.86:0/65926915 learned_addr learned my addr 172.21.15.86:0/65926915 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fecd6a00640 1 --2- 172.21.15.86:0/65926915 >> v2:172.21.15.196:3300/0 conn(0x7fece006ef40 0x7fece0142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fecdea00640 1 -- 172.21.15.86:0/65926915 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2362748131 0 0) 0x7fece015c7c0 con 0x7fece00ad710 2024-03-22T04:17:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fecdea00640 1 -- 172.21.15.86:0/65926915 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecc4003530 con 0x7fece00ad710 2024-03-22T04:17:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fecd6a00640 1 -- 172.21.15.86:0/65926915 >> v1:172.21.15.196:6789/0 conn(0x7fece00ad330 legacy=0x7fece013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fecd6a00640 1 -- 172.21.15.86:0/65926915 >> v1:172.21.15.86:6789/0 conn(0x7fece00ad710 legacy=0x7fece01402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.734+0000 7fecd6a00640 1 -- 172.21.15.86:0/65926915 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece006e460 con 0x7fece006ef40 2024-03-22T04:17:08.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecd6a00640 1 --2- 172.21.15.86:0/65926915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 0x7fece0142690 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fecc80097c0 tx=0x7fecc801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c1d8abf5e8aca68 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:08.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecdea00640 1 -- 172.21.15.86:0/65926915 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc80aa5f0 con 0x7fece006ef40 2024-03-22T04:17:08.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecdea00640 1 -- 172.21.15.86:0/65926915 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc80af070 con 0x7fece006ef40 2024-03-22T04:17:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecdea00640 1 -- 172.21.15.86:0/65926915 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc80aa9b0 con 0x7fece006ef40 2024-03-22T04:17:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- 172.21.15.86:0/65926915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 msgr2=0x7fece0142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 --2- 172.21.15.86:0/65926915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 0x7fece0142690 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fecc80097c0 tx=0x7fecc801c470 comp rx=0 tx=0).stop 2024-03-22T04:17:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- 172.21.15.86:0/65926915 shutdown_connections 2024-03-22T04:17:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 --2- 172.21.15.86:0/65926915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 0x7fece0142690 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- 172.21.15.86:0/65926915 >> 172.21.15.86:0/65926915 conn(0x7fece00b1df0 msgr2=0x7fece00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- 172.21.15.86:0/65926915 shutdown_connections 2024-03-22T04:17:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- 172.21.15.86:0/65926915 wait complete. 2024-03-22T04:17:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 Processor -- start 2024-03-22T04:17:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- start start 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 0x7fece0099560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece00ad330 0x7fece015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fece00ad710 0x7fece0140f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fece006db10 con 0x7fece00ad330 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fece01409d0 con 0x7fece006ef40 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fece6400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fece016a780 con 0x7fece00ad710 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecd6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece00ad330 0x7fece015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecd6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece00ad330 0x7fece015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57364/0 (socket says 172.21.15.86:57364) 2024-03-22T04:17:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecd6a00640 1 -- 172.21.15.86:0/2826663227 learned_addr learned my addr 172.21.15.86:0/2826663227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecdf400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 0x7fece0099560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.738+0000 7fecdfe00640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fece00ad710 0x7fece0140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd6a00640 1 -- 172.21.15.86:0/2826663227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fece00ad710 msgr2=0x7fece0140f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd6a00640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fece00ad710 0x7fece0140f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd6a00640 1 -- 172.21.15.86:0/2826663227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 msgr2=0x7fece0099560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd6a00640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 0x7fece0099560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd6a00640 1 -- 172.21.15.86:0/2826663227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece0159ad0 con 0x7fece00ad330 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd6a00640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece00ad330 0x7fece015ab40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fecc80ac040 tx=0x7fecc801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd7e00640 1 -- 172.21.15.86:0/2826663227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc80be610 con 0x7fece00ad330 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd7e00640 1 -- 172.21.15.86:0/2826663227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc80be7d0 con 0x7fece00ad330 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd7e00640 1 -- 172.21.15.86:0/2826663227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc80bebc0 con 0x7fece00ad330 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece0159db0 con 0x7fece00ad330 2024-03-22T04:17:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece015a370 con 0x7fece00ad330 2024-03-22T04:17:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece015a6a0 con 0x7fece00ad330 2024-03-22T04:17:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd7e00640 1 -- 172.21.15.86:0/2826663227 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fecc80bc020 con 0x7fece00ad330 2024-03-22T04:17:08.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd7e00640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fecbc067b20 0x7fecbc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:08.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecd7e00640 1 -- 172.21.15.86:0/2826663227 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fecc81202f0 con 0x7fece00ad330 2024-03-22T04:17:08.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.742+0000 7fecdf400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fecbc067b20 0x7fecbc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:08.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.746+0000 7fecdf400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fecbc067b20 0x7fecbc069fe0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fece01562e0 tx=0x7fecd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:08.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.746+0000 7fecd7e00640 1 -- 172.21.15.86:0/2826663227 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fecc80c5030 con 0x7fece00ad330 2024-03-22T04:17:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.938+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fece0068650 con 0x7fece00ad330 2024-03-22T04:17:08.937 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:17:08.938+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:17:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.938+0000 7fecd7e00640 1 -- 172.21.15.86:0/2826663227 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) ==== 161+0+0 (secure 0 0 0) 0x7fecc8016090 con 0x7fece00ad330 2024-03-22T04:17:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:17:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fecbc067b20 msgr2=0x7fecbc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fecbc067b20 0x7fecbc069fe0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fece01562e0 tx=0x7fecd0009040 comp rx=0 tx=0).stop 2024-03-22T04:17:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece00ad330 msgr2=0x7fece015ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece00ad330 0x7fece015ab40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fecc80ac040 tx=0x7fecc801acb0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 shutdown_connections 2024-03-22T04:17:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fecbc067b20 0x7fecbc069fe0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fece00ad710 0x7fece0140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece00ad330 0x7fece015ab40 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 --2- 172.21.15.86:0/2826663227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece006ef40 0x7fece0099560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 >> 172.21.15.86:0/2826663227 conn(0x7fece00b1df0 msgr2=0x7fece00b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 shutdown_connections 2024-03-22T04:17:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:08.942+0000 7fece6400640 1 -- 172.21.15.86:0/2826663227 wait complete. 2024-03-22T04:17:08.955 INFO:tasks.ceph.mds.b.smithi196.stderr: -4949> 2024-03-22T04:17:03.604+0000 7f495a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:17:08.961 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-03-22T04:17:09.003 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:17:09.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f816bf3f080 1 Processor -- start 2024-03-22T04:17:09.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f816bf3f080 1 -- start start 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f816bf3f080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a881870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f816bf3f080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de520 0x55ec0a7b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f816bf3f080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de900 0x55ec0a7f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f816bf3f080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec0a7e0890 con 0x55ec0a7af5a0 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f816bf3f080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec0a7e0710 con 0x55ec0a7de520 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f816bf3f080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ec0a7e0a10 con 0x55ec0a7de900 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a881870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de900 0x55ec0a7f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de900 0x55ec0a7f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:44646/0 (socket says 172.21.15.86:44646) 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169c00640 1 -- 172.21.15.86:0/1301544490 learned_addr learned my addr 172.21.15.86:0/1301544490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8163e00640 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de520 0x55ec0a7b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169200640 1 -- 172.21.15.86:0/1301544490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de900 msgr2=0x55ec0a7f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169200640 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de900 0x55ec0a7f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169200640 1 -- 172.21.15.86:0/1301544490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de520 msgr2=0x55ec0a7b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169200640 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de520 0x55ec0a7b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169200640 1 -- 172.21.15.86:0/1301544490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0a6db950 con 0x55ec0a7af5a0 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8163e00640 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de520 0x55ec0a7b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8169200640 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a881870 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f815c0aa000 tx=0x7f815c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c60fc54d529d4ed server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8162a00640 1 -- 172.21.15.86:0/1301544490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c0b0040 con 0x55ec0a7af5a0 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8162a00640 1 -- 172.21.15.86:0/1301544490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815c0aba10 con 0x55ec0a7af5a0 2024-03-22T04:17:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.006+0000 7f8162a00640 1 -- 172.21.15.86:0/1301544490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c0abe00 con 0x55ec0a7af5a0 2024-03-22T04:17:09.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- 172.21.15.86:0/1301544490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 msgr2=0x55ec0a881870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a881870 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f815c0aa000 tx=0x7f815c003030 comp rx=0 tx=0).stop 2024-03-22T04:17:09.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- 172.21.15.86:0/1301544490 shutdown_connections 2024-03-22T04:17:09.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de520 0x55ec0a7b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de900 0x55ec0a7f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 --2- 172.21.15.86:0/1301544490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a881870 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- 172.21.15.86:0/1301544490 >> 172.21.15.86:0/1301544490 conn(0x55ec0a7dbec0 msgr2=0x55ec0a7deda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:09.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- 172.21.15.86:0/1301544490 shutdown_connections 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- 172.21.15.86:0/1301544490 wait complete. 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 Processor -- start 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- start start 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a87fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de520 0x55ec0a7f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de900 0x55ec0a7f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec0a886700 con 0x55ec0a7af5a0 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec0a7f6930 con 0x55ec0a7de900 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ec0a8869a0 con 0x55ec0a7de520 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a87fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a87fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57390/0 (socket says 172.21.15.86:57390) 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 -- 172.21.15.86:0/395111801 learned_addr learned my addr 172.21.15.86:0/395111801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8163e00640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de520 0x55ec0a7f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169c00640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de900 0x55ec0a7f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 -- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de520 msgr2=0x55ec0a7f9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ec0a7de520 0x55ec0a7f9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 -- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de900 msgr2=0x55ec0a7f9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de900 0x55ec0a7f9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 -- 172.21.15.86:0/395111801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0a885160 con 0x55ec0a7af5a0 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169c00640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec0a7de900 0x55ec0a7f9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8169200640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a87fbf0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f815c01bbd0 tx=0x7f815c0180d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8161600640 1 -- 172.21.15.86:0/395111801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c0b0070 con 0x55ec0a7af5a0 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8161600640 1 -- 172.21.15.86:0/395111801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815c0c3070 con 0x55ec0a7af5a0 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f8161600640 1 -- 172.21.15.86:0/395111801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c0b68a0 con 0x55ec0a7af5a0 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec0a885400 con 0x55ec0a7af5a0 2024-03-22T04:17:09.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.010+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0a885940 con 0x55ec0a7af5a0 2024-03-22T04:17:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f8161600640 1 -- 172.21.15.86:0/395111801 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f815c0b6a40 con 0x55ec0a7af5a0 2024-03-22T04:17:09.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f8161600640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8148066530 0x7f81480689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f8161600640 1 -- 172.21.15.86:0/395111801 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7f815c120f60 con 0x55ec0a7af5a0 2024-03-22T04:17:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f8163e00640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8148066530 0x7f81480689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f816bf3f080 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ec0a880130 0x55ec0a7f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x55ec0a88c3d0 con 0x55ec0a880130 2024-03-22T04:17:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f8169c00640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ec0a880130 0x55ec0a7f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f8163e00640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8148066530 0x7f81480689f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8154009850 tx=0x7f815400c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.014+0000 7f8169c00640 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ec0a880130 0x55ec0a7f4d50 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f8169c00640 1 -- 172.21.15.86:0/395111801 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v94'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8164002a30 con 0x55ec0a880130 2024-03-22T04:17:09.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ec0a880130 msgr2=0x55ec0a7f4d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55ec0a880130 0x55ec0a7f4d50 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8148066530 msgr2=0x7f81480689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8148066530 0x7f81480689f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8154009850 tx=0x7f815400c040 comp rx=0 tx=0).stop 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 msgr2=0x55ec0a87fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 --2- 172.21.15.86:0/395111801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec0a7af5a0 0x55ec0a87fbf0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f815c01bbd0 tx=0x7f815c0180d0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f8169c00640 1 -- 172.21.15.86:0/395111801 reap_dead start 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 shutdown_connections 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 >> 172.21.15.86:0/395111801 conn(0x55ec0a7dbec0 msgr2=0x55ec0a880530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 shutdown_connections 2024-03-22T04:17:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.018+0000 7f816bf3f080 1 -- 172.21.15.86:0/395111801 wait complete. 2024-03-22T04:17:09.023 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-03-22T04:17:09.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1935f53400 1 -- start start 2024-03-22T04:17:09.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1935f53400 1 Processor -- start 2024-03-22T04:17:09.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1935f53400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633495f3800 0x56334a3c0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1935f53400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633495f3400 0x56334a3c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1935f53400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633495f3000 0x56334a3c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1935f53400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56334a3d4600 con 0x5633495f3000 2024-03-22T04:17:09.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1935f53400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56334a3d4480 con 0x5633495f3800 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1935f53400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56334a3d4300 con 0x5633495f3400 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1934a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633495f3000 0x56334a3c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1934a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633495f3000 0x56334a3c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57410/0 (socket says 172.21.15.86:57410) 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1934a00640 1 -- 172.21.15.86:0/4055569395 learned_addr learned my addr 172.21.15.86:0/4055569395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1933600640 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633495f3400 0x56334a3c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.106+0000 7f1934000640 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633495f3800 0x56334a3c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934a00640 1 -- 172.21.15.86:0/4055569395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633495f3400 msgr2=0x56334a3c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934a00640 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633495f3400 0x56334a3c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934a00640 1 -- 172.21.15.86:0/4055569395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633495f3800 msgr2=0x56334a3c0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934a00640 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633495f3800 0x56334a3c0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934a00640 1 -- 172.21.15.86:0/4055569395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563349766b40 con 0x5633495f3000 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934a00640 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633495f3000 0x56334a3c0b00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x56334a465320 tx=0x5633496c9560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb861c93b58cf19 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1930e00640 1 -- 172.21.15.86:0/4055569395 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5633496284e0 con 0x5633495f3000 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1930e00640 1 -- 172.21.15.86:0/4055569395 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56334963e540 con 0x5633495f3000 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1930e00640 1 -- 172.21.15.86:0/4055569395 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563349628b60 con 0x5633495f3000 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 -- 172.21.15.86:0/4055569395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563349766d20 con 0x5633495f3000 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 -- 172.21.15.86:0/4055569395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563349766f00 con 0x5633495f3000 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 4 waiting for MDS map... 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1930e00640 1 -- 172.21.15.86:0/4055569395 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x56334a51e600 con 0x5633495f3000 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1930e00640 1 -- 172.21.15.86:0/4055569395 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 65) ==== 2885+0+0 (secure 0 0 0) 0x563349649180 con 0x5633495f3000 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 4 Got MDS map 65 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 10 main: JournalTool::main 2024-03-22T04:17:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 4 main: JournalTool: connecting to RADOS... 2024-03-22T04:17:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 Processor -- start 2024-03-22T04:17:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 -- start start 2024-03-22T04:17:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd000 0x56334a3c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd400 0x56334a3c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56334a3d4180 con 0x56334a4fd400 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563349652480 con 0x56334a4fd000 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1935f53400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x563349652900 con 0x56334a4fd800 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd400 0x56334a3c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd400 0x56334a3c1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57422/0 (socket says 172.21.15.86:57422) 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 -- 172.21.15.86:0/2322787973 learned_addr learned my addr 172.21.15.86:0/2322787973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1933600640 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd000 0x56334a3c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1933600640 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 -- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 msgr2=0x56334a3c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 -- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd000 msgr2=0x56334a3c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd000 0x56334a3c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 -- 172.21.15.86:0/2322787973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563349766d20 con 0x56334a4fd400 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1933600640 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f1934000640 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd400 0x56334a3c1600 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x56334a5646f0 tx=0x56334a501800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87dd1d996c4b5fc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f192e600640 1 -- 172.21.15.86:0/2322787973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5633496284e0 con 0x56334a4fd400 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f192e600640 1 -- 172.21.15.86:0/2322787973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56334963e1c0 con 0x56334a4fd400 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.110+0000 7f192e600640 1 -- 172.21.15.86:0/2322787973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563349628b60 con 0x56334a4fd400 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- 172.21.15.86:0/2322787973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd400 msgr2=0x56334a3c1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd400 0x56334a3c1600 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x56334a5646f0 tx=0x56334a501800 comp rx=0 tx=0).stop 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- 172.21.15.86:0/2322787973 shutdown_connections 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd400 0x56334a3c1600 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 --2- 172.21.15.86:0/2322787973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd000 0x56334a3c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- 172.21.15.86:0/2322787973 >> 172.21.15.86:0/2322787973 conn(0x5633495f2c00 msgr2=0x56334961f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- 172.21.15.86:0/2322787973 shutdown_connections 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- 172.21.15.86:0/2322787973 wait complete. 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 Processor -- start 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- start start 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd000 0x56334a3c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd400 0x56334a3c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56334a3d5380 con 0x56334a4fd000 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56334a3d4180 con 0x56334a4fd400 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56334a3d5500 con 0x56334a4fd800 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd000 0x56334a3c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd000 0x56334a3c1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57436/0 (socket says 172.21.15.86:57436) 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 -- 172.21.15.86:0/3044604818 learned_addr learned my addr 172.21.15.86:0/3044604818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1934000640 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd400 0x56334a3c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 -- 172.21.15.86:0/3044604818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 msgr2=0x56334a3c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 -- 172.21.15.86:0/3044604818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd400 msgr2=0x56334a3c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd400 0x56334a3c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 -- 172.21.15.86:0/3044604818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563349767c20 con 0x56334a4fd000 2024-03-22T04:17:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1933600640 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd000 0x56334a3c1080 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x56334a464990 tx=0x56334a37a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f192c800640 1 -- 172.21.15.86:0/3044604818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5633496289c0 con 0x56334a4fd000 2024-03-22T04:17:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f192c800640 1 -- 172.21.15.86:0/3044604818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56334963ea80 con 0x56334a4fd000 2024-03-22T04:17:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- 172.21.15.86:0/3044604818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563349767e00 con 0x56334a4fd000 2024-03-22T04:17:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f192c800640 1 -- 172.21.15.86:0/3044604818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5633496284e0 con 0x56334a4fd000 2024-03-22T04:17:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 1 -- 172.21.15.86:0/3044604818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633497661e0 con 0x56334a4fd000 2024-03-22T04:17:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.114+0000 7f1935f53400 4 main: JournalTool: resolving pool 13 2024-03-22T04:17:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f192c800640 1 -- 172.21.15.86:0/3044604818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x56334a566c00 con 0x56334a4fd000 2024-03-22T04:17:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f192c800640 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5633498dec00 0x56334a3c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f192c800640 1 -- 172.21.15.86:0/3044604818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x5633498d8600 con 0x56334a4fd000 2024-03-22T04:17:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 4 main: JournalTool: creating IoCtx.. 2024-03-22T04:17:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 4 main: Executing for rank 0 2024-03-22T04:17:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1934000640 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5633498dec00 0x56334a3c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1934000640 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5633498dec00 0x56334a3c2100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x56334a5646f0 tx=0x5633496c9500 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 1 -- start start 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 1 Processor -- start 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633498df000 0x56334a3c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633498df400 0x56334a3c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633498df800 0x56334a3c3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56334a3d5800 con 0x5633498df800 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56334a3d5980 con 0x5633498df400 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1935f53400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x563349652900 con 0x5633498df000 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1933600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633498df800 0x56334a3c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1933600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633498df800 0x56334a3c3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57450/0 (socket says 172.21.15.86:57450) 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1933600640 1 -- 172.21.15.86:0/2517287303 learned_addr learned my addr 172.21.15.86:0/2517287303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1934000640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633498df400 0x56334a3c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.118+0000 7f1934a00640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633498df000 0x56334a3c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1933600640 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633498df000 msgr2=0x56334a3c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1933600640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633498df000 0x56334a3c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1933600640 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633498df400 msgr2=0x56334a3c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1933600640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633498df400 0x56334a3c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1933600640 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563349a1e780 con 0x5633498df800 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1934000640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633498df400 0x56334a3c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1933600640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633498df800 0x56334a3c3180 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x5633499371d0 tx=0x563349a2c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2304458f4ed4ef8d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1928c00640 1 -- 172.21.15.86:0/2517287303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56334996c4e0 con 0x5633498df800 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1928c00640 1 -- 172.21.15.86:0/2517287303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56334963f340 con 0x5633498df800 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1928c00640 1 -- 172.21.15.86:0/2517287303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56334996c680 con 0x5633498df800 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563349a1e960 con 0x5633498df800 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563349a1eb40 con 0x5633498df800 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1935f53400 4 waiting for MDS map... 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1928c00640 1 -- 172.21.15.86:0/2517287303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x563349a2a800 con 0x5633498df800 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1928c00640 1 -- 172.21.15.86:0/2517287303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 65) ==== 2885+0+0 (secure 0 0 0) 0x563349649500 con 0x5633498df800 2024-03-22T04:17:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1935f53400 4 Got MDS map 65 2024-03-22T04:17:09.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5633498dfc00 0x56334a3c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x563349a2f800 con 0x5633498dfc00 2024-03-22T04:17:09.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1934a00640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5633498dfc00 0x56334a3c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.122+0000 7f1934a00640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5633498dfc00 0x56334a3c3700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=dce3cf74ed0649e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1934a00640 1 -- 172.21.15.86:0/2517287303 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x56334a370000 con 0x5633498dfc00 2024-03-22T04:17:09.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563349a2fc00 0x563349a4c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x563349a50000 con 0x563349a2fc00 2024-03-22T04:17:09.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1934000640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563349a2fc00 0x563349a4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1934000640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563349a2fc00 0x563349a4c000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=b1111c0ef0508c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1934000640 1 -- 172.21.15.86:0/2517287303 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x56334a370500 con 0x563349a2fc00 2024-03-22T04:17:09.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1926e00640 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e94) -- 0x563349a52000 con 0x563349a2fc00 2024-03-22T04:17:09.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1926e00640 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e94) -- 0x563349a52400 con 0x5633498dfc00 2024-03-22T04:17:09.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1934000640 1 -- 172.21.15.86:0/2517287303 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x56334a370500 con 0x563349a2fc00 2024-03-22T04:17:09.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1934a00640 1 -- 172.21.15.86:0/2517287303 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56334a370000 con 0x5633498dfc00 2024-03-22T04:17:09.127 INFO:teuthology.orchestra.run.smithi086.stdout:old journal was 4212014~2544 2024-03-22T04:17:09.127 INFO:teuthology.orchestra.run.smithi086.stdout:new journal start will be 8388608 (4174050 bytes past old end) 2024-03-22T04:17:09.127 INFO:teuthology.orchestra.run.smithi086.stdout:writing journal head 2024-03-22T04:17:09.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x563349a50400 con 0x563349a2fc00 2024-03-22T04:17:09.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1934000640 1 -- 172.21.15.86:0/2517287303 <== osd.6 v2:172.21.15.196:6816/1648471396 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v94'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56334a370500 con 0x563349a2fc00 2024-03-22T04:17:09.128 INFO:teuthology.orchestra.run.smithi086.stdout:writing EResetJournal entry 2024-03-22T04:17:09.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x563349a50800 con 0x5633498dfc00 2024-03-22T04:17:09.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563349a50c00 0x563349a4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x563349a51000 con 0x563349a50c00 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563349a51400 0x563349a4cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x563349a51800 con 0x563349a51400 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563349a51c00 0x563349a4d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x563349a52c00 con 0x563349a51c00 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563349a53000 0x563349a4d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x563349a53400 con 0x563349a53000 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x563349a53800 con 0x563349a51400 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.126+0000 7f1934a00640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563349a50c00 0x563349a4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1933600640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563349a51c00 0x563349a4d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1934a00640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563349a53000 0x563349a4d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1934000640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563349a51400 0x563349a4cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1933600640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563349a51c00 0x563349a4d080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=a1c12a9b9aa6968f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1934a00640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563349a50c00 0x563349a4c580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=9908640c87a1b9a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1934a00640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563349a53000 0x563349a4d600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=d669753bf17d03c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1934a00640 1 -- 172.21.15.86:0/2517287303 <== osd.2 v2:172.21.15.86:6816/4132808680 3 ==== osd_op_reply(6 200.00000002 [delete] v94'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56334a370000 con 0x5633498dfc00 2024-03-22T04:17:09.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1934000640 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563349a51400 0x563349a4cb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=4dbd7ab8ee937ad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1933600640 1 -- 172.21.15.86:0/2517287303 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(9 200.00000005 [delete] v94'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56334a370780 con 0x563349a51c00 2024-03-22T04:17:09.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.130+0000 7f1934a00640 1 -- 172.21.15.86:0/2517287303 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(10 200.00000006 [delete] v94'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56334a370000 con 0x563349a53000 2024-03-22T04:17:09.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.134+0000 7f1934a00640 1 -- 172.21.15.86:0/2517287303 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(7 200.00000003 [delete] v94'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56334a370000 con 0x563349a50c00 2024-03-22T04:17:09.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.134+0000 7f1926e00640 1 -- 172.21.15.86:0/2517287303 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x563349a52800 con 0x5633498dfc00 2024-03-22T04:17:09.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.134+0000 7f1934000640 1 -- 172.21.15.86:0/2517287303 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== osd_op_reply(8 200.00000004 [delete] v94'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56334a370500 con 0x563349a51400 2024-03-22T04:17:09.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.134+0000 7f1934000640 1 -- 172.21.15.86:0/2517287303 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== osd_op_reply(11 200.00000007 [delete] v94'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56334a370500 con 0x563349a51400 2024-03-22T04:17:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1934a00640 1 -- 172.21.15.86:0/2517287303 <== osd.2 v2:172.21.15.86:6816/4132808680 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v94'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56334a370000 con 0x5633498dfc00 2024-03-22T04:17:09.137 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-03-22T04:17:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563349a53000 msgr2=0x563349a4d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563349a53000 0x563349a4d600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563349a51400 msgr2=0x563349a4cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563349a51400 0x563349a4cb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5633498dfc00 msgr2=0x56334a3c3700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5633498dfc00 0x56334a3c3700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563349a50c00 msgr2=0x563349a4c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563349a50c00 0x563349a4c580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563349a51c00 msgr2=0x563349a4d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563349a51c00 0x563349a4d080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563349a2fc00 msgr2=0x563349a4c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563349a2fc00 0x563349a4c000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1933600640 1 -- 172.21.15.86:0/2517287303 reap_dead start 2024-03-22T04:17:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1933600640 1 -- 172.21.15.86:0/2517287303 reap_dead start 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633498df800 msgr2=0x56334a3c3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/2517287303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633498df800 0x56334a3c3180 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x5633499371d0 tx=0x563349a2c060 comp rx=0 tx=0).stop 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1933600640 1 -- 172.21.15.86:0/2517287303 reap_dead start 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 shutdown_connections 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 >> 172.21.15.86:0/2517287303 conn(0x56334a4fdc00 msgr2=0x56334961e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 shutdown_connections 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/2517287303 wait complete. 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5633498dec00 msgr2=0x56334a3c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5633498dec00 0x56334a3c2100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x56334a5646f0 tx=0x5633496c9500 comp rx=0 tx=0).stop 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd000 msgr2=0x56334a3c1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd000 0x56334a3c1080 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x56334a464990 tx=0x56334a37a900 comp rx=0 tx=0).stop 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/3044604818 shutdown_connections 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5633498dec00 0x56334a3c2100 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56334a4fd400 0x56334a3c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56334a4fd800 0x56334a3c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/3044604818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56334a4fd000 0x56334a3c1080 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/3044604818 >> 172.21.15.86:0/3044604818 conn(0x5633495f2c00 msgr2=0x56334961f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/3044604818 shutdown_connections 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/3044604818 wait complete. 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/4055569395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633495f3000 msgr2=0x56334a3c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633495f3000 0x56334a3c0b00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x56334a465320 tx=0x5633496c9560 comp rx=0 tx=0).stop 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/4055569395 shutdown_connections 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633495f3000 0x56334a3c0b00 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5633495f3400 0x56334a3c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 --2- 172.21.15.86:0/4055569395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633495f3800 0x56334a3c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/4055569395 >> 172.21.15.86:0/4055569395 conn(0x5633495f3c00 msgr2=0x56334961f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/4055569395 shutdown_connections 2024-03-22T04:17:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.138+0000 7f1935f53400 1 -- 172.21.15.86:0/4055569395 wait complete. 2024-03-22T04:17:09.144 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.120614, result: old journal was 4212014~2544 new journal start will be 8388608 (4174050 bytes past old end) writing journal head writing EResetJournal entry done 2024-03-22T04:17:09.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-22T04:17:09.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0763600640 1 Processor -- start 2024-03-22T04:17:09.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0763600640 1 -- start start 2024-03-22T04:17:09.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0763600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f075c06e7e0 0x7f075c06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0763600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f075c072b00 0x7f075c0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0763600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f075c15c770 con 0x7f075c072650 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0763600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f075c071350 con 0x7f075c072b00 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0763600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f075c06f100 con 0x7f075c06e7e0 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0761600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f075c072b00 0x7f075c0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0761600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f075c072b00 0x7f075c0b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57464/0 (socket says 172.21.15.86:57464) 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0761600640 1 -- 172.21.15.86:0/4232312403 learned_addr learned my addr 172.21.15.86:0/4232312403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0760c00640 1 --2- 172.21.15.86:0/4232312403 >> v2:172.21.15.196:3300/0 conn(0x7f075c06e7e0 0x7f075c06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0761600640 1 -- 172.21.15.86:0/4232312403 >> v2:172.21.15.196:3300/0 conn(0x7f075c06e7e0 msgr2=0x7f075c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0761600640 1 --2- 172.21.15.86:0/4232312403 >> v2:172.21.15.196:3300/0 conn(0x7f075c06e7e0 0x7f075c06ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0761600640 1 -- 172.21.15.86:0/4232312403 >> v1:172.21.15.196:6789/0 conn(0x7f075c072650 legacy=0x7f075c0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.330+0000 7f0761600640 1 -- 172.21.15.86:0/4232312403 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c073150 con 0x7f075c072b00 2024-03-22T04:17:09.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0761600640 1 --2- 172.21.15.86:0/4232312403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c072b00 0x7f075c0b2290 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f075c0ce650 tx=0x7f075401e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab61252a4028484 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f075b400640 1 -- 172.21.15.86:0/4232312403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075400b2d0 con 0x7f075c072b00 2024-03-22T04:17:09.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f075b400640 1 -- 172.21.15.86:0/4232312403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075400b470 con 0x7f075c072b00 2024-03-22T04:17:09.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f075b400640 1 -- 172.21.15.86:0/4232312403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754009820 con 0x7f075c072b00 2024-03-22T04:17:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 -- 172.21.15.86:0/4232312403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c072b00 msgr2=0x7f075c0b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 --2- 172.21.15.86:0/4232312403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c072b00 0x7f075c0b2290 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f075c0ce650 tx=0x7f075401e940 comp rx=0 tx=0).stop 2024-03-22T04:17:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 -- 172.21.15.86:0/4232312403 shutdown_connections 2024-03-22T04:17:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 --2- 172.21.15.86:0/4232312403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c072b00 0x7f075c0b2290 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 --2- 172.21.15.86:0/4232312403 >> v2:172.21.15.196:3300/0 conn(0x7f075c06e7e0 0x7f075c06ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 -- 172.21.15.86:0/4232312403 >> 172.21.15.86:0/4232312403 conn(0x7f075c0cc940 msgr2=0x7f075c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 -- 172.21.15.86:0/4232312403 shutdown_connections 2024-03-22T04:17:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 -- 172.21.15.86:0/4232312403 wait complete. 2024-03-22T04:17:09.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.334+0000 7f0763600640 1 Processor -- start 2024-03-22T04:17:09.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 -- start start 2024-03-22T04:17:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c06e7e0 0x7f075c154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c072650 0x7f075c14de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c072b00 0x7f075c156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f075c072080 con 0x7f075c06e7e0 2024-03-22T04:17:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f075c0ce930 con 0x7f075c072b00 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f075c153040 con 0x7f075c072650 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c06e7e0 0x7f075c154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c06e7e0 0x7f075c154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57474/0 (socket says 172.21.15.86:57474) 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 -- 172.21.15.86:0/2764442002 learned_addr learned my addr 172.21.15.86:0/2764442002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f075be00640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c072650 0x7f075c14de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0761600640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c072b00 0x7f075c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 -- 172.21.15.86:0/2764442002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c072650 msgr2=0x7f075c14de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c072650 0x7f075c14de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 -- 172.21.15.86:0/2764442002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c072b00 msgr2=0x7f075c156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c072b00 0x7f075c156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 -- 172.21.15.86:0/2764442002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c14af80 con 0x7f075c06e7e0 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0760c00640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c06e7e0 0x7f075c154310 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f075000d5f0 tx=0x7f075000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0758c00640 1 -- 172.21.15.86:0/2764442002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500125f0 con 0x7f075c06e7e0 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0758c00640 1 -- 172.21.15.86:0/2764442002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0750019070 con 0x7f075c06e7e0 2024-03-22T04:17:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c153270 con 0x7f075c06e7e0 2024-03-22T04:17:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0758c00640 1 -- 172.21.15.86:0/2764442002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500129b0 con 0x7f075c06e7e0 2024-03-22T04:17:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c153830 con 0x7f075c06e7e0 2024-03-22T04:17:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.338+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c074470 con 0x7f075c06e7e0 2024-03-22T04:17:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.342+0000 7f0758c00640 1 -- 172.21.15.86:0/2764442002 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0750012b50 con 0x7f075c06e7e0 2024-03-22T04:17:09.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.342+0000 7f0758c00640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f073c067b20 0x7f073c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:09.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.342+0000 7f0758c00640 1 -- 172.21.15.86:0/2764442002 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7f0750086480 con 0x7f075c06e7e0 2024-03-22T04:17:09.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.342+0000 7f075be00640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f073c067b20 0x7f073c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.342+0000 7f075be00640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f073c067b20 0x7f073c069fe0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f075c14d360 tx=0x7f0748009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:09.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.346+0000 7f0758c00640 1 -- 172.21.15.86:0/2764442002 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0750056530 con 0x7f075c06e7e0 2024-03-22T04:17:09.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.534+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f075c0cd000 con 0x7f075c06e7e0 2024-03-22T04:17:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0758c00640 1 -- 172.21.15.86:0/2764442002 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v66) ==== 162+0+0 (secure 0 0 0) 0x7f075005a5d0 con 0x7f075c06e7e0 2024-03-22T04:17:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-22T04:17:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f073c067b20 msgr2=0x7f073c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f073c067b20 0x7f073c069fe0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f075c14d360 tx=0x7f0748009060 comp rx=0 tx=0).stop 2024-03-22T04:17:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c06e7e0 msgr2=0x7f075c154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c06e7e0 0x7f075c154310 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f075000d5f0 tx=0x7f075000dab0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 shutdown_connections 2024-03-22T04:17:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f073c067b20 0x7f073c069fe0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c072b00 0x7f075c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c072650 0x7f075c14de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 --2- 172.21.15.86:0/2764442002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f075c06e7e0 0x7f075c154310 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 >> 172.21.15.86:0/2764442002 conn(0x7f075c0cc940 msgr2=0x7f075c1596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 shutdown_connections 2024-03-22T04:17:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:09.942+0000 7f0763600640 1 -- 172.21.15.86:0/2764442002 wait complete. 2024-03-22T04:17:09.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:17:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f82c00640 1 Processor -- start 2024-03-22T04:17:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f82c00640 1 -- start start 2024-03-22T04:17:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f82c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f7c06ef40 0x7f6f7c142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f82c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f7c15c7c0 con 0x7f6f7c0ad710 2024-03-22T04:17:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f82c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f7c06dea0 con 0x7f6f7c0ad330 2024-03-22T04:17:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f82c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7c06e180 con 0x7f6f7c06ef40 2024-03-22T04:17:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f73400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f7c06ef40 0x7f6f7c142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f80c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6f7c0ad710 0x7f6f7c1402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41520/0 (socket says 172.21.15.86:41520) 2024-03-22T04:17:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f80c00640 1 -- 172.21.15.86:0/1712795993 learned_addr learned my addr 172.21.15.86:0/1712795993 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f73400640 1 -- 172.21.15.86:0/1712795993 >> v1:172.21.15.196:6789/0 conn(0x7f6f7c0ad330 legacy=0x7f6f7c13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f73400640 1 -- 172.21.15.86:0/1712795993 >> v1:172.21.15.86:6789/0 conn(0x7f6f7c0ad710 legacy=0x7f6f7c1402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:10.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f73400640 1 -- 172.21.15.86:0/1712795993 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c06e460 con 0x7f6f7c06ef40 2024-03-22T04:17:10.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.102+0000 7f6f7b400640 1 -- 172.21.15.86:0/1712795993 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394231411 0 0) 0x7f6f7c15c7c0 con 0x7f6f7c0ad710 2024-03-22T04:17:10.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f73400640 1 --2- 172.21.15.86:0/1712795993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c142690 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6f640097c0 tx=0x7f6f6401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dd8e0fbbbf02cc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:10.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f7b400640 1 -- 172.21.15.86:0/1712795993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f640aa5f0 con 0x7f6f7c06ef40 2024-03-22T04:17:10.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f7b400640 1 -- 172.21.15.86:0/1712795993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f640af070 con 0x7f6f7c06ef40 2024-03-22T04:17:10.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f7b400640 1 -- 172.21.15.86:0/1712795993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f640aa9b0 con 0x7f6f7c06ef40 2024-03-22T04:17:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f82c00640 1 -- 172.21.15.86:0/1712795993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 msgr2=0x7f6f7c142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f82c00640 1 --2- 172.21.15.86:0/1712795993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c142690 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6f640097c0 tx=0x7f6f6401c470 comp rx=0 tx=0).stop 2024-03-22T04:17:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f82c00640 1 -- 172.21.15.86:0/1712795993 shutdown_connections 2024-03-22T04:17:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f82c00640 1 --2- 172.21.15.86:0/1712795993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c142690 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f82c00640 1 -- 172.21.15.86:0/1712795993 >> 172.21.15.86:0/1712795993 conn(0x7f6f7c0b1df0 msgr2=0x7f6f7c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f82c00640 1 -- 172.21.15.86:0/1712795993 shutdown_connections 2024-03-22T04:17:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.106+0000 7f6f82c00640 1 -- 172.21.15.86:0/1712795993 wait complete. 2024-03-22T04:17:10.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 Processor -- start 2024-03-22T04:17:10.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 -- start start 2024-03-22T04:17:10.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:10.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c0ad330 0x7f6f7c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f7c0ad710 0x7f6f7c0b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c06db10 con 0x7f6f7c06ef40 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c1409d0 con 0x7f6f7c0ad330 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7c16a780 con 0x7f6f7c0ad710 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c1515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57504/0 (socket says 172.21.15.86:57504) 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 -- 172.21.15.86:0/1361324751 learned_addr learned my addr 172.21.15.86:0/1361324751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f80c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f7c0ad710 0x7f6f7c0b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f73400640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c0ad330 0x7f6f7c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 -- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f7c0ad710 msgr2=0x7f6f7c0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f7c0ad710 0x7f6f7c0b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 -- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c0ad330 msgr2=0x7f6f7c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c0ad330 0x7f6f7c0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 -- 172.21.15.86:0/1361324751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c15a1e0 con 0x7f6f7c06ef40 2024-03-22T04:17:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f80c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f7c0ad710 0x7f6f7c0b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f7be00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c1515b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c7f0 tx=0x7f6f6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f78c00640 1 -- 172.21.15.86:0/1361324751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c003b80 con 0x7f6f7c06ef40 2024-03-22T04:17:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f78c00640 1 -- 172.21.15.86:0/1361324751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f6c003d20 con 0x7f6f7c06ef40 2024-03-22T04:17:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7c15a4c0 con 0x7f6f7c06ef40 2024-03-22T04:17:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f78c00640 1 -- 172.21.15.86:0/1361324751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00aa90 con 0x7f6f7c06ef40 2024-03-22T04:17:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.110+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7c15aa80 con 0x7f6f7c06ef40 2024-03-22T04:17:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.114+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7c068650 con 0x7f6f7c06ef40 2024-03-22T04:17:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.114+0000 7f6f78c00640 1 -- 172.21.15.86:0/1361324751 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6f6c00ac30 con 0x7f6f7c06ef40 2024-03-22T04:17:10.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.114+0000 7f6f78c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f58067b20 0x7f6f58069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:10.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.114+0000 7f6f78c00640 1 -- 172.21.15.86:0/1361324751 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7f6f6c0860c0 con 0x7f6f7c06ef40 2024-03-22T04:17:10.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.114+0000 7f6f73400640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f58067b20 0x7f6f58069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:10.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.114+0000 7f6f73400640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f58067b20 0x7f6f58069fe0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6f640ae000 tx=0x7f6f640a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.118+0000 7f6f78c00640 1 -- 172.21.15.86:0/1361324751 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6f6c056850 con 0x7f6f7c06ef40 2024-03-22T04:17:10.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.306+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f7c0cf7d0 con 0x7f6f7c06ef40 2024-03-22T04:17:10.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.306+0000 7f6f78c00640 1 -- 172.21.15.86:0/1361324751 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5177 (secure 0 0 0) 0x7f6f6c056150 con 0x7f6f7c06ef40 2024-03-22T04:17:10.307 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:10.307 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:09.945320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76604":{"gid":76604,"name":"b","rank":0,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.196:6835/1851339627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1851339627},{"type":"v1","addr":"172.21.15.196:6835","nonce":1851339627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-03-22T04:17:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 67 2024-03-22T04:17:10.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f58067b20 msgr2=0x7f6f58069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:10.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f58067b20 0x7f6f58069fe0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6f640ae000 tx=0x7f6f640a7000 comp rx=0 tx=0).stop 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 msgr2=0x7f6f7c1515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c1515b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c7f0 tx=0x7f6f6c00ccb0 comp rx=0 tx=0).stop 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 shutdown_connections 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6f58067b20 0x7f6f58069fe0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c0ad330 0x7f6f7c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f7c0ad710 0x7f6f7c0b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 --2- 172.21.15.86:0/1361324751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f7c06ef40 0x7f6f7c1515b0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 >> 172.21.15.86:0/1361324751 conn(0x7f6f7c0b1df0 msgr2=0x7f6f7c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 shutdown_connections 2024-03-22T04:17:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:10.310+0000 7f6f82c00640 1 -- 172.21.15.86:0/1361324751 wait complete. 2024-03-22T04:17:10.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:16:58.021326+0000', 'modified': '2024-03-22T04:17:09.945320+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 76604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_76604': {'gid': 76604, 'name': 'b', 'rank': 0, 'incarnation': 67, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1851339627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1851339627}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1851339627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:17:10.731 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_76604:up:replay 2024-03-22T04:17:11.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:17:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf76e00640 1 Processor -- start 2024-03-22T04:17:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf76e00640 1 -- start start 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf76e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcf7006cd80 0x7fcf70143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf76e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcf70144320 0x7fcf7015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf76e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf700ae5a0 con 0x7fcf700aff50 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf76e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcf70162aa0 con 0x7fcf70144320 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf76e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcf70144b80 con 0x7fcf7006cd80 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf74e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcf70144320 0x7fcf7015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf74e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcf70144320 0x7fcf7015eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57506/0 (socket says 172.21.15.86:57506) 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf74e00640 1 -- 172.21.15.86:0/1956645625 learned_addr learned my addr 172.21.15.86:0/1956645625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.878+0000 7fcf67e00640 1 --2- 172.21.15.86:0/1956645625 >> v2:172.21.15.196:3300/0 conn(0x7fcf7006cd80 0x7fcf70143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf74e00640 1 -- 172.21.15.86:0/1956645625 >> v2:172.21.15.196:3300/0 conn(0x7fcf7006cd80 msgr2=0x7fcf70143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf74e00640 1 --2- 172.21.15.86:0/1956645625 >> v2:172.21.15.196:3300/0 conn(0x7fcf7006cd80 0x7fcf70143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf74e00640 1 -- 172.21.15.86:0/1956645625 >> v1:172.21.15.196:6789/0 conn(0x7fcf700aff50 legacy=0x7fcf7006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:11.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf74e00640 1 -- 172.21.15.86:0/1956645625 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf70162170 con 0x7fcf70144320 2024-03-22T04:17:11.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf74e00640 1 --2- 172.21.15.86:0/1956645625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf7015eb80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcf70148360 tx=0x7fcf6801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397767bab6996b4d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:11.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf6f400640 1 -- 172.21.15.86:0/1956645625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680a57f0 con 0x7fcf70144320 2024-03-22T04:17:11.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf6f400640 1 -- 172.21.15.86:0/1956645625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf680a5990 con 0x7fcf70144320 2024-03-22T04:17:11.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf6f400640 1 -- 172.21.15.86:0/1956645625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680a5d60 con 0x7fcf70144320 2024-03-22T04:17:11.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf76e00640 1 -- 172.21.15.86:0/1956645625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 msgr2=0x7fcf7015eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:11.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf76e00640 1 --2- 172.21.15.86:0/1956645625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf7015eb80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcf70148360 tx=0x7fcf6801c930 comp rx=0 tx=0).stop 2024-03-22T04:17:11.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf76e00640 1 -- 172.21.15.86:0/1956645625 shutdown_connections 2024-03-22T04:17:11.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf76e00640 1 --2- 172.21.15.86:0/1956645625 >> v2:172.21.15.196:3300/0 conn(0x7fcf7006cd80 0x7fcf70143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:11.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf76e00640 1 --2- 172.21.15.86:0/1956645625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf7015eb80 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:11.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf76e00640 1 -- 172.21.15.86:0/1956645625 >> 172.21.15.86:0/1956645625 conn(0x7fcf70071fe0 msgr2=0x7fcf700723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:11.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.882+0000 7fcf76e00640 1 -- 172.21.15.86:0/1956645625 shutdown_connections 2024-03-22T04:17:11.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 -- 172.21.15.86:0/1956645625 wait complete. 2024-03-22T04:17:11.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 Processor -- start 2024-03-22T04:17:11.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 -- start start 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7006cd80 0x7fcf70147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf700aff50 0x7fcf70139f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf70141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70163090 con 0x7fcf70144320 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70162f10 con 0x7fcf7006cd80 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700b0330 con 0x7fcf700aff50 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf70141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf70141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57518/0 (socket says 172.21.15.86:57518) 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 -- 172.21.15.86:0/2885303065 learned_addr learned my addr 172.21.15.86:0/2885303065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf67e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf700aff50 0x7fcf70139f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf6fe00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7006cd80 0x7fcf70147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:11.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 -- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf700aff50 msgr2=0x7fcf70139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf700aff50 0x7fcf70139f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 -- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7006cd80 msgr2=0x7fcf70147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7006cd80 0x7fcf70147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 -- 172.21.15.86:0/2885303065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf70141780 con 0x7fcf70144320 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf6fe00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7006cd80 0x7fcf70147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf67e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf700aff50 0x7fcf70139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf74e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf70141bd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcf68019540 tx=0x7fcf680196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf6cc00640 1 -- 172.21.15.86:0/2885303065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680bc020 con 0x7fcf70144320 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf6cc00640 1 -- 172.21.15.86:0/2885303065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf680b8670 con 0x7fcf70144320 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf701411c0 con 0x7fcf70144320 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf6cc00640 1 -- 172.21.15.86:0/2885303065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680b8a40 con 0x7fcf70144320 2024-03-22T04:17:11.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.886+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf70155860 con 0x7fcf70144320 2024-03-22T04:17:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.890+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf70068650 con 0x7fcf70144320 2024-03-22T04:17:11.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.890+0000 7fcf6cc00640 1 -- 172.21.15.86:0/2885303065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcf680bf020 con 0x7fcf70144320 2024-03-22T04:17:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.890+0000 7fcf6cc00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf4c067b20 0x7fcf4c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.890+0000 7fcf6cc00640 1 -- 172.21.15.86:0/2885303065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7fcf681211d0 con 0x7fcf70144320 2024-03-22T04:17:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.890+0000 7fcf6fe00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf4c067b20 0x7fcf4c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:11.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.890+0000 7fcf6fe00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf4c067b20 0x7fcf4c069fe0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcf600016f0 tx=0x7fcf60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:11.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:11.894+0000 7fcf6cc00640 1 -- 172.21.15.86:0/2885303065 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcf680c6030 con 0x7fcf70144320 2024-03-22T04:17:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.082+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf7006fc50 con 0x7fcf70144320 2024-03-22T04:17:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf6cc00640 1 -- 172.21.15.86:0/2885303065 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5177 (secure 0 0 0) 0x7fcf680165e0 con 0x7fcf70144320 2024-03-22T04:17:12.085 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:12.085 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:10.952757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76604":{"gid":76604,"name":"b","rank":0,"incarnation":67,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.196:6835/1851339627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1851339627},{"type":"v1","addr":"172.21.15.196:6835","nonce":1851339627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-03-22T04:17:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 69 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf4c067b20 msgr2=0x7fcf4c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf4c067b20 0x7fcf4c069fe0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcf600016f0 tx=0x7fcf60009040 comp rx=0 tx=0).stop 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 msgr2=0x7fcf70141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf70141bd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcf68019540 tx=0x7fcf680196d0 comp rx=0 tx=0).stop 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 shutdown_connections 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcf4c067b20 0x7fcf4c069fe0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf70144320 0x7fcf70141bd0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf700aff50 0x7fcf70139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 --2- 172.21.15.86:0/2885303065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7006cd80 0x7fcf70147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 >> 172.21.15.86:0/2885303065 conn(0x7fcf70071fe0 msgr2=0x7fcf70158340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 shutdown_connections 2024-03-22T04:17:12.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:12.086+0000 7fcf76e00640 1 -- 172.21.15.86:0/2885303065 wait complete. 2024-03-22T04:17:12.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:16:58.021326+0000', 'modified': '2024-03-22T04:17:10.952757+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 76604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_76604': {'gid': 76604, 'name': 'b', 'rank': 0, 'incarnation': 67, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.196:6835/1851339627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1851339627}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1851339627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:17:12.509 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_76604:up:rejoin 2024-03-22T04:17:13.510 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:17:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.658+0000 7fe280a00640 1 Processor -- start 2024-03-22T04:17:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.658+0000 7fe280a00640 1 -- start start 2024-03-22T04:17:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe280a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe27c145b50 0x7fe27c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe280a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c06f060 0x7fe27c06bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe280a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe27c06f600 con 0x7fe27c073680 2024-03-22T04:17:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe280a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe27c15d190 con 0x7fe27c145b50 2024-03-22T04:17:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe280a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe27c15c860 con 0x7fe27c06f060 2024-03-22T04:17:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c06f060 0x7fe27c06bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27aa00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe27c073680 0x7fe27c0cda30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:34368/0 (socket says 172.21.15.86:34368) 2024-03-22T04:17:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27aa00640 1 -- 172.21.15.86:0/696762357 learned_addr learned my addr 172.21.15.86:0/696762357 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27b400640 1 --2- 172.21.15.86:0/696762357 >> v2:172.21.15.196:3300/0 conn(0x7fe27c145b50 0x7fe27c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27a000640 1 -- 172.21.15.86:0/696762357 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3190273983 0 0) 0x7fe27c06f600 con 0x7fe27c073680 2024-03-22T04:17:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27be00640 1 -- 172.21.15.86:0/696762357 >> v2:172.21.15.196:3300/0 conn(0x7fe27c145b50 msgr2=0x7fe27c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27be00640 1 --2- 172.21.15.86:0/696762357 >> v2:172.21.15.196:3300/0 conn(0x7fe27c145b50 0x7fe27c06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27be00640 1 -- 172.21.15.86:0/696762357 >> v1:172.21.15.196:6789/0 conn(0x7fe27c073680 legacy=0x7fe27c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.662+0000 7fe27be00640 1 -- 172.21.15.86:0/696762357 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c15cb40 con 0x7fe27c06f060 2024-03-22T04:17:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe27be00640 1 --2- 172.21.15.86:0/696762357 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c06f060 0x7fe27c06bba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe274009d90 tx=0x7fe27401d280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e5dfff9a12ab622 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe27a000640 1 -- 172.21.15.86:0/696762357 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2740b2070 con 0x7fe27c06f060 2024-03-22T04:17:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe27a000640 1 -- 172.21.15.86:0/696762357 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2740a5820 con 0x7fe27c06f060 2024-03-22T04:17:13.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe27a000640 1 -- 172.21.15.86:0/696762357 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2740a5bf0 con 0x7fe27c06f060 2024-03-22T04:17:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 -- 172.21.15.86:0/696762357 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c06f060 msgr2=0x7fe27c06bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 --2- 172.21.15.86:0/696762357 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c06f060 0x7fe27c06bba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe274009d90 tx=0x7fe27401d280 comp rx=0 tx=0).stop 2024-03-22T04:17:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 -- 172.21.15.86:0/696762357 shutdown_connections 2024-03-22T04:17:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 --2- 172.21.15.86:0/696762357 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c06f060 0x7fe27c06bba0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 --2- 172.21.15.86:0/696762357 >> v2:172.21.15.196:3300/0 conn(0x7fe27c145b50 0x7fe27c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 -- 172.21.15.86:0/696762357 >> 172.21.15.86:0/696762357 conn(0x7fe27c06e540 msgr2=0x7fe27c06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 -- 172.21.15.86:0/696762357 shutdown_connections 2024-03-22T04:17:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 -- 172.21.15.86:0/696762357 wait complete. 2024-03-22T04:17:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 Processor -- start 2024-03-22T04:17:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.666+0000 7fe280a00640 1 -- start start 2024-03-22T04:17:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c06f060 0x7fe27c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe27c073680 0x7fe27c139190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c145b50 0x7fe27c1399c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c168d50 con 0x7fe27c06f060 2024-03-22T04:17:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c15d600 con 0x7fe27c073680 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe27c168ed0 con 0x7fe27c145b50 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c06f060 0x7fe27c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c06f060 0x7fe27c157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57520/0 (socket says 172.21.15.86:57520) 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 -- 172.21.15.86:0/1499593299 learned_addr learned my addr 172.21.15.86:0/1499593299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27aa00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe27c073680 0x7fe27c139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27be00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c145b50 0x7fe27c1399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 -- 172.21.15.86:0/1499593299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c145b50 msgr2=0x7fe27c1399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c145b50 0x7fe27c1399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 -- 172.21.15.86:0/1499593299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe27c073680 msgr2=0x7fe27c139190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe27c073680 0x7fe27c139190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 -- 172.21.15.86:0/1499593299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c141070 con 0x7fe27c06f060 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe27b400640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c06f060 0x7fe27c157b20 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00e8a0 tx=0x7fe26c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe273400640 1 -- 172.21.15.86:0/1499593299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c017070 con 0x7fe27c06f060 2024-03-22T04:17:13.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe273400640 1 -- 172.21.15.86:0/1499593299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26c00c940 con 0x7fe27c06f060 2024-03-22T04:17:13.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c141350 con 0x7fe27c06f060 2024-03-22T04:17:13.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe273400640 1 -- 172.21.15.86:0/1499593299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00ccf0 con 0x7fe27c06f060 2024-03-22T04:17:13.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c158430 con 0x7fe27c06f060 2024-03-22T04:17:13.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe27c074470 con 0x7fe27c06f060 2024-03-22T04:17:13.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.670+0000 7fe273400640 1 -- 172.21.15.86:0/1499593299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe26c01a070 con 0x7fe27c06f060 2024-03-22T04:17:13.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.674+0000 7fe273400640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe25c067b20 0x7fe25c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:13.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.674+0000 7fe273400640 1 -- 172.21.15.86:0/1499593299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7fe26c0862b0 con 0x7fe27c06f060 2024-03-22T04:17:13.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.674+0000 7fe27aa00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe25c067b20 0x7fe25c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:13.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.674+0000 7fe27aa00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe25c067b20 0x7fe25c069fe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe27c136010 tx=0x7fe268007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:13.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.678+0000 7fe273400640 1 -- 172.21.15.86:0/1499593299 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe26c056480 con 0x7fe27c06f060 2024-03-22T04:17:13.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.870+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe27c0b0540 con 0x7fe27c06f060 2024-03-22T04:17:13.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.870+0000 7fe273400640 1 -- 172.21.15.86:0/1499593299 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5186 (secure 0 0 0) 0x7fe26c05a520 con 0x7fe27c06f060 2024-03-22T04:17:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:13.871 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":70,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:12.955122+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76604":{"gid":76604,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.196:6835/1851339627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1851339627},{"type":"v1","addr":"172.21.15.196:6835","nonce":1851339627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76604,"qdb_cluster":[76604]},"id":6}]} 2024-03-22T04:17:13.871 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 70 2024-03-22T04:17:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.870+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe25c067b20 msgr2=0x7fe25c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.870+0000 7fe280a00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe25c067b20 0x7fe25c069fe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe27c136010 tx=0x7fe268007d60 comp rx=0 tx=0).stop 2024-03-22T04:17:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.870+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c06f060 msgr2=0x7fe27c157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.870+0000 7fe280a00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c06f060 0x7fe27c157b20 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00e8a0 tx=0x7fe26c00eca0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.874+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 shutdown_connections 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.874+0000 7fe280a00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe25c067b20 0x7fe25c069fe0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.874+0000 7fe280a00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe27c073680 0x7fe27c139190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.874+0000 7fe280a00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe27c145b50 0x7fe27c1399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.874+0000 7fe280a00640 1 --2- 172.21.15.86:0/1499593299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe27c06f060 0x7fe27c157b20 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.874+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 >> 172.21.15.86:0/1499593299 conn(0x7fe27c06e540 msgr2=0x7fe27c0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.874+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 shutdown_connections 2024-03-22T04:17:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:13.874+0000 7fe280a00640 1 -- 172.21.15.86:0/1499593299 wait complete. 2024-03-22T04:17:14.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:16:58.021326+0000', 'modified': '2024-03-22T04:17:12.955122+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 76604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_76604': {'gid': 76604, 'name': 'b', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.196:6835/1851339627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1851339627}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1851339627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 76604, 'qdb_cluster': [76604]} 2024-03-22T04:17:14.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:17:14.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T04:17:14.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1efce00640 1 Processor -- start 2024-03-22T04:17:14.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1efce00640 1 -- start start 2024-03-22T04:17:14.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1efce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef80ad330 0x7f1ef815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1efce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ef806ef40 0x7f1ef806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1efce00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef815c7c0 con 0x7f1ef80ad710 2024-03-22T04:17:14.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1efce00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef806dc40 con 0x7f1ef806ef40 2024-03-22T04:17:14.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1efce00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef806dea0 con 0x7f1ef80ad330 2024-03-22T04:17:14.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1ef5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef80ad330 0x7f1ef815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1ef5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef80ad330 0x7f1ef815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:44742/0 (socket says 172.21.15.86:44742) 2024-03-22T04:17:14.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1ef5e00640 1 -- 172.21.15.86:0/3171908688 learned_addr learned my addr 172.21.15.86:0/3171908688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:14.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1eed400640 1 --2- 172.21.15.86:0/3171908688 >> v2:172.21.15.196:3300/0 conn(0x7f1ef806ef40 0x7f1ef806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1ef5e00640 1 -- 172.21.15.86:0/3171908688 >> v2:172.21.15.196:3300/0 conn(0x7f1ef806ef40 msgr2=0x7f1ef806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/3171908688 >> v2:172.21.15.196:3300/0 conn(0x7f1ef806ef40 0x7f1ef806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1ef5e00640 1 -- 172.21.15.86:0/3171908688 >> v1:172.21.15.196:6789/0 conn(0x7f1ef80ad710 legacy=0x7f1ef813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.434+0000 7f1ef5e00640 1 -- 172.21.15.86:0/3171908688 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef806e180 con 0x7f1ef80ad330 2024-03-22T04:17:14.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/3171908688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef80ad330 0x7f1ef815d4a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009870 tx=0x7f1ee801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6fd89ec22e06b1c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1ef5400640 1 -- 172.21.15.86:0/3171908688 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80ab640 con 0x7f1ef80ad330 2024-03-22T04:17:14.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1ef5400640 1 -- 172.21.15.86:0/3171908688 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee80b0070 con 0x7f1ef80ad330 2024-03-22T04:17:14.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1ef5400640 1 -- 172.21.15.86:0/3171908688 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80aba00 con 0x7f1ef80ad330 2024-03-22T04:17:14.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1efce00640 1 -- 172.21.15.86:0/3171908688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef80ad330 msgr2=0x7f1ef815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1efce00640 1 --2- 172.21.15.86:0/3171908688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef80ad330 0x7f1ef815d4a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009870 tx=0x7f1ee801c530 comp rx=0 tx=0).stop 2024-03-22T04:17:14.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1efce00640 1 -- 172.21.15.86:0/3171908688 shutdown_connections 2024-03-22T04:17:14.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1efce00640 1 --2- 172.21.15.86:0/3171908688 >> v2:172.21.15.196:3300/0 conn(0x7f1ef806ef40 0x7f1ef806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1efce00640 1 --2- 172.21.15.86:0/3171908688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef80ad330 0x7f1ef815d4a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1efce00640 1 -- 172.21.15.86:0/3171908688 >> 172.21.15.86:0/3171908688 conn(0x7f1ef80b1df0 msgr2=0x7f1ef80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:14.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1efce00640 1 -- 172.21.15.86:0/3171908688 shutdown_connections 2024-03-22T04:17:14.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.438+0000 7f1efce00640 1 -- 172.21.15.86:0/3171908688 wait complete. 2024-03-22T04:17:14.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 Processor -- start 2024-03-22T04:17:14.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 -- start start 2024-03-22T04:17:14.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef806ef40 0x7f1ef81547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef80ad330 0x7f1ef80999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef80ad710 0x7f1ef815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81690e0 con 0x7f1ef80ad330 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8168f60 con 0x7f1ef80ad710 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef8169260 con 0x7f1ef806ef40 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1eed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef80ad330 0x7f1ef80999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1eed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef80ad330 0x7f1ef80999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57522/0 (socket says 172.21.15.86:57522) 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1eed400640 1 -- 172.21.15.86:0/1314445788 learned_addr learned my addr 172.21.15.86:0/1314445788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1ef5e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef806ef40 0x7f1ef81547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1ef6800640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef80ad710 0x7f1ef815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1ef5e00640 1 -- 172.21.15.86:0/1314445788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef80ad710 msgr2=0x7f1ef815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef80ad710 0x7f1ef815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1ef5e00640 1 -- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef80ad330 msgr2=0x7f1ef80999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef80ad330 0x7f1ef80999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1ef5e00640 1 -- 172.21.15.86:0/1314445788 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef8158af0 con 0x7f1ef806ef40 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1eed400640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef80ad330 0x7f1ef80999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:14.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef806ef40 0x7f1ef81547d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80ac000 tx=0x7f1ee801ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1eeea00640 1 -- 172.21.15.86:0/1314445788 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80be3f0 con 0x7f1ef806ef40 2024-03-22T04:17:14.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1eeea00640 1 -- 172.21.15.86:0/1314445788 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee80be590 con 0x7f1ef806ef40 2024-03-22T04:17:14.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1eeea00640 1 -- 172.21.15.86:0/1314445788 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80be980 con 0x7f1ef806ef40 2024-03-22T04:17:14.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 -- 172.21.15.86:0/1314445788 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef8099ef0 con 0x7f1ef806ef40 2024-03-22T04:17:14.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.442+0000 7f1efce00640 1 -- 172.21.15.86:0/1314445788 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef815b1f0 con 0x7f1ef806ef40 2024-03-22T04:17:14.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1eeea00640 1 -- 172.21.15.86:0/1314445788 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1ee80b43f0 con 0x7f1ef806ef40 2024-03-22T04:17:14.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1eeea00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ed4067b20 0x7f1ed4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1eeea00640 1 -- 172.21.15.86:0/1314445788 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7f1ee8120580 con 0x7f1ef806ef40 2024-03-22T04:17:14.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1eed400640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ed4067b20 0x7f1ed4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1f00380000 1 Processor -- start 2024-03-22T04:17:14.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1f00380000 1 -- start start 2024-03-22T04:17:14.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1eed400640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ed4067b20 0x7f1ed4069fe0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80b4e10 tx=0x7f1ee0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1f00380000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 0x556195919250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1f00380000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x5561958cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1f00380000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x5561958cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1f00380000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561956ca140 con 0x5561958ccc20 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1f00380000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55619571db10 con 0x556195918e70 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1f00380000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5561956a4c50 con 0x556195919790 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x5561958cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x5561958cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57538/0 (socket says 172.21.15.86:57538) 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef6800640 1 -- 172.21.15.86:0/3234782505 learned_addr learned my addr 172.21.15.86:0/3234782505 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 0x556195919250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1eed400640 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x5561958cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef6800640 1 -- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 msgr2=0x5561958cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef6800640 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x5561958cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef6800640 1 -- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 msgr2=0x556195919250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef6800640 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 0x556195919250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1ef6800640 1 -- 172.21.15.86:0/3234782505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556195762f20 con 0x5561958ccc20 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.446+0000 7f1eed400640 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x5561958cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:14.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x5561958cd000 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0001f50 tx=0x7f1ef00016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e84630810f36182 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ee6a00640 1 -- 172.21.15.86:0/3234782505 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000fe50 con 0x5561958ccc20 2024-03-22T04:17:14.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ee6a00640 1 -- 172.21.15.86:0/3234782505 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0018070 con 0x5561958ccc20 2024-03-22T04:17:14.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ee6a00640 1 -- 172.21.15.86:0/3234782505 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0003b80 con 0x5561958ccc20 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- 172.21.15.86:0/3234782505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 msgr2=0x5561958cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x5561958cd000 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0001f50 tx=0x7f1ef00016b0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- 172.21.15.86:0/3234782505 shutdown_connections 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x5561958cd000 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x5561958cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 --2- 172.21.15.86:0/3234782505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 0x556195919250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- 172.21.15.86:0/3234782505 >> 172.21.15.86:0/3234782505 conn(0x5561958f9c10 msgr2=0x5561958fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- 172.21.15.86:0/3234782505 shutdown_connections 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- 172.21.15.86:0/3234782505 wait complete. 2024-03-22T04:17:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- start start 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 Processor -- start 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x5561958d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195918e70 0x5561958d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195919790 0x5561958d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561956d7ab0 con 0x556195919790 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556195719ac0 con 0x5561958ccc20 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55619575da60 con 0x556195918e70 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1f00380000 10 client.0 _ll_register_callbacks cb 0x5561958ba380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195919790 0x5561958d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195919790 0x5561958d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57546/0 (socket says 172.21.15.86:57546) 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 -- 172.21.15.86:0/1357519818 learned_addr learned my addr 172.21.15.86:0/1357519818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x5561958d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1eed400640 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195918e70 0x5561958d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 -- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195918e70 msgr2=0x5561958d1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195918e70 0x5561958d1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 -- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 msgr2=0x5561958d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x5561958d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 -- 172.21.15.86:0/1357519818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561956f3570 con 0x556195919790 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x5561958d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1ef6800640 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195919790 0x5561958d2290 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0014000 tx=0x7f1ef0001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6435abf070e552 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.450+0000 7f1edb400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 1 -- 172.21.15.86:0/1357519818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00054e0 con 0x556195919790 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 1 -- 172.21.15.86:0/1357519818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7f1ef0005680 con 0x556195919790 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 -- 172.21.15.86:0/1357519818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef0018070 con 0x556195919790 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 1 -- 172.21.15.86:0/1357519818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f1ef002e050 con 0x556195919790 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 10 client.7755 fetch_fsmap learned FSMap version 70 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 -- 172.21.15.86:0/1357519818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5561956e9880 con 0x556195919790 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 1 -- 172.21.15.86:0/1357519818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0018070 con 0x556195919790 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 1 -- 172.21.15.86:0/1357519818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x7f1ef0031020 con 0x556195919790 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 10 client.7755 fetch_fsmap finished waiting for FSMap version 70 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 10 client.7755 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:17:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 20 client.7755 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5561958d2ca0 0x55619592a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 4 client.7755 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:17:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 -- 172.21.15.86:0/1357519818 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561956b8bc0 con 0x5561958d2ca0 2024-03-22T04:17:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5561958d2ca0 0x55619592a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 10 client.7755 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5561958d2ca0 0x55619592a120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df11e8d2a554bb4c server_cookie=e83278c8726943c5 in_seq=0 out_seq=0 2024-03-22T04:17:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 1 -- 172.21.15.86:0/1357519818 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5561956b8bc0 con 0x5561958d2ca0 2024-03-22T04:17:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 10 client.7755 handle_command_reply: tid=0 2024-03-22T04:17:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 client.7755 shutdown 2024-03-22T04:17:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 -- 172.21.15.86:0/1357519818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195919790 msgr2=0x5561958d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195919790 0x5561958d2290 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0014000 tx=0x7f1ef0001260 comp rx=0 tx=0).stop 2024-03-22T04:17:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 -- 172.21.15.86:0/1357519818 shutdown_connections 2024-03-22T04:17:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5561958d2ca0 0x55619592a120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195919790 0x5561958d2290 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195918e70 0x5561958d1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 --2- 172.21.15.86:0/1357519818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x5561958d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1edb400640 0 client.7755 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 -- 172.21.15.86:0/1357519818 >> 172.21.15.86:0/1357519818 conn(0x5561958f9c10 msgr2=0x5561958fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 -- 172.21.15.86:0/1357519818 shutdown_connections 2024-03-22T04:17:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 1 -- 172.21.15.86:0/1357519818 wait complete. 2024-03-22T04:17:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.454+0000 7f1f00380000 20 client.7755 trim_cache size 0 max 16384 2024-03-22T04:17:14.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 Processor -- start 2024-03-22T04:17:14.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- start start 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x556195917f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195918e70 0x556195918470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619592a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef82011a0 con 0x556195918e70 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8144910 con 0x5561958ccc20 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef8144f60 con 0x556195919790 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195918e70 0x556195918470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195918e70 0x556195918470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57556/0 (socket says 172.21.15.86:57556) 2024-03-22T04:17:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 -- 172.21.15.86:0/1525631549 learned_addr learned my addr 172.21.15.86:0/1525631549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x556195917f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1ef6800640 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619592a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 -- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 msgr2=0x55619592a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619592a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 -- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 msgr2=0x556195917f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x556195917f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 -- 172.21.15.86:0/1525631549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561956a5e80 con 0x556195918e70 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1ef6800640 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619592a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1eed400640 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195918e70 0x556195918470 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0014810 tx=0x7f1ee0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30caafccd28d6d9b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1ee6000640 1 -- 172.21.15.86:0/1525631549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee002c430 con 0x556195918e70 2024-03-22T04:17:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1ee6000640 1 -- 172.21.15.86:0/1525631549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee001b040 con 0x556195918e70 2024-03-22T04:17:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1ee6000640 1 -- 172.21.15.86:0/1525631549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee002c7d0 con 0x556195918e70 2024-03-22T04:17:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- 172.21.15.86:0/1525631549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195918e70 msgr2=0x556195918470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195918e70 0x556195918470 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0014810 tx=0x7f1ee0014000 comp rx=0 tx=0).stop 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- 172.21.15.86:0/1525631549 shutdown_connections 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619592a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556195918e70 0x556195918470 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 --2- 172.21.15.86:0/1525631549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5561958ccc20 0x556195917f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- 172.21.15.86:0/1525631549 >> 172.21.15.86:0/1525631549 conn(0x5561958f9c10 msgr2=0x5561958cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- 172.21.15.86:0/1525631549 shutdown_connections 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- 172.21.15.86:0/1525631549 wait complete. 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 -- start start 2024-03-22T04:17:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.474+0000 7f1f00380000 1 Processor -- start 2024-03-22T04:17:14.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x55619592cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 0x55619592d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619598cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561958d5110 con 0x5561958ccc20 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55619571ce20 con 0x556195918e70 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5561956d7ab0 con 0x556195919790 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 10 client.0 _ll_register_callbacks cb 0x7f1ef0001550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x55619592cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x55619592cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57562/0 (socket says 172.21.15.86:57562) 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 -- 172.21.15.86:0/2600253002 learned_addr learned my addr 172.21.15.86:0/2600253002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1eed400640 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 0x55619592d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef6800640 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619598cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 -- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 msgr2=0x55619598cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619598cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 -- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 msgr2=0x55619592d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 0x55619592d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 -- 172.21.15.86:0/2600253002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561956b1750 con 0x5561958ccc20 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef6800640 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619598cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1ef5e00640 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x55619592cef0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee801c570 tx=0x7f1ee8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4febc654c0a6af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 1 -- 172.21.15.86:0/2600253002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80f9020 con 0x5561958ccc20 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 1 -- 172.21.15.86:0/2600253002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7f1ee80f8b90 con 0x5561958ccc20 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- 172.21.15.86:0/2600253002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee001b040 con 0x5561958ccc20 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 1 -- 172.21.15.86:0/2600253002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f1ee80f8d90 con 0x5561958ccc20 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 10 client.7761 fetch_fsmap learned FSMap version 70 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- 172.21.15.86:0/2600253002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5561956c4450 con 0x5561958ccc20 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 1 -- 172.21.15.86:0/2600253002 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee001b040 con 0x5561958ccc20 2024-03-22T04:17:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 1 -- 172.21.15.86:0/2600253002 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x7f1ee81200b0 con 0x5561958ccc20 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 10 client.7761 fetch_fsmap finished waiting for FSMap version 70 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 10 client.7761 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 20 client.7761 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55619598d8c0 0x55619598dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 4 client.7761 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- 172.21.15.86:0/2600253002 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "status"}) -- 0x55619572b050 con 0x55619598d8c0 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1eed400640 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55619598d8c0 0x55619598dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 10 client.7761 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1eed400640 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55619598d8c0 0x55619598dca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ae28eb5b4ac7f6 server_cookie=6a7daefd18e2035 in_seq=0 out_seq=0 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 1 -- 172.21.15.86:0/2600253002 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55619572b050 con 0x55619598d8c0 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 10 client.7761 handle_command_reply: tid=0 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 client.7761 shutdown 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- 172.21.15.86:0/2600253002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 msgr2=0x55619592cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x55619592cef0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee801c570 tx=0x7f1ee8009240 comp rx=0 tx=0).stop 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- 172.21.15.86:0/2600253002 shutdown_connections 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55619598d8c0 0x55619598dca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556195918e70 0x55619592d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556195919790 0x55619598cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 --2- 172.21.15.86:0/2600253002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561958ccc20 0x55619592cef0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1edb400640 0 client.7761 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- 172.21.15.86:0/2600253002 >> 172.21.15.86:0/2600253002 conn(0x5561958f9c10 msgr2=0x556195916f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- 172.21.15.86:0/2600253002 shutdown_connections 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 1 -- 172.21.15.86:0/2600253002 wait complete. 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.478+0000 7f1f00380000 20 client.7761 trim_cache size 0 max 16384 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:17:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 76604, 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.5327952720000004, 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 70, 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 94, 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 5.4737403870000003 2024-03-22T04:17:14.485 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 -- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ed4067b20 msgr2=0x7f1ed4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ed4067b20 0x7f1ed4069fe0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80b4e10 tx=0x7f1ee0009190 comp rx=0 tx=0).stop 2024-03-22T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 -- 172.21.15.86:0/1314445788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef806ef40 msgr2=0x7f1ef81547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef806ef40 0x7f1ef81547d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80ac000 tx=0x7f1ee801ad70 comp rx=0 tx=0).stop 2024-03-22T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 -- 172.21.15.86:0/1314445788 shutdown_connections 2024-03-22T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1ed4067b20 0x7f1ed4069fe0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef80ad710 0x7f1ef815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef80ad330 0x7f1ef80999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 --2- 172.21.15.86:0/1314445788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef806ef40 0x7f1ef81547d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 -- 172.21.15.86:0/1314445788 >> 172.21.15.86:0/1314445788 conn(0x7f1ef80b1df0 msgr2=0x7f1ef80b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:14.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 -- 172.21.15.86:0/1314445788 shutdown_connections 2024-03-22T04:17:14.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.482+0000 7f1efce00640 1 -- 172.21.15.86:0/1314445788 wait complete. 2024-03-22T04:17:14.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:17:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c44000640 1 Processor -- start 2024-03-22T04:17:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c44000640 1 -- start start 2024-03-22T04:17:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c44000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c44000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0c3c0cc3e0 0x7f0c3c0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c44000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c3c071920 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c44000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3c0716d0 con 0x7f0c3c0cc3e0 2024-03-22T04:17:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c44000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c06d990 con 0x7f0c3c0730d0 2024-03-22T04:17:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c40c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c40c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c0ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:44796/0 (socket says 172.21.15.86:44796) 2024-03-22T04:17:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c40c00640 1 -- 172.21.15.86:0/2662233980 learned_addr learned my addr 172.21.15.86:0/2662233980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c42000640 1 --2- 172.21.15.86:0/2662233980 >> v2:172.21.15.196:3300/0 conn(0x7f0c3c0cc3e0 0x7f0c3c0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c40c00640 1 -- 172.21.15.86:0/2662233980 >> v2:172.21.15.196:3300/0 conn(0x7f0c3c0cc3e0 msgr2=0x7f0c3c0ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c40c00640 1 --2- 172.21.15.86:0/2662233980 >> v2:172.21.15.196:3300/0 conn(0x7f0c3c0cc3e0 0x7f0c3c0ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c40c00640 1 -- 172.21.15.86:0/2662233980 >> v1:172.21.15.196:6789/0 conn(0x7f0c3c15b3f0 legacy=0x7f0c3c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.634+0000 7f0c40c00640 1 -- 172.21.15.86:0/2662233980 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c071c00 con 0x7f0c3c0730d0 2024-03-22T04:17:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c40c00640 1 --2- 172.21.15.86:0/2662233980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c0ade30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009870 tx=0x7f0c2801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c645ba58fe9d55b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c3be00640 1 -- 172.21.15.86:0/2662233980 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c280ab650 con 0x7f0c3c0730d0 2024-03-22T04:17:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c3be00640 1 -- 172.21.15.86:0/2662233980 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c280b0070 con 0x7f0c3c0730d0 2024-03-22T04:17:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c3be00640 1 -- 172.21.15.86:0/2662233980 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c280aba10 con 0x7f0c3c0730d0 2024-03-22T04:17:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c44000640 1 -- 172.21.15.86:0/2662233980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 msgr2=0x7f0c3c0ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c44000640 1 --2- 172.21.15.86:0/2662233980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c0ade30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009870 tx=0x7f0c2801c560 comp rx=0 tx=0).stop 2024-03-22T04:17:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c44000640 1 -- 172.21.15.86:0/2662233980 shutdown_connections 2024-03-22T04:17:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c44000640 1 --2- 172.21.15.86:0/2662233980 >> v2:172.21.15.196:3300/0 conn(0x7f0c3c0cc3e0 0x7f0c3c0ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c44000640 1 --2- 172.21.15.86:0/2662233980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c0ade30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c44000640 1 -- 172.21.15.86:0/2662233980 >> 172.21.15.86:0/2662233980 conn(0x7f0c3c0b1df0 msgr2=0x7f0c3c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c44000640 1 -- 172.21.15.86:0/2662233980 shutdown_connections 2024-03-22T04:17:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.638+0000 7f0c44000640 1 -- 172.21.15.86:0/2662233980 wait complete. 2024-03-22T04:17:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 Processor -- start 2024-03-22T04:17:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 -- start start 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c3c0cc3e0 0x7f0c3c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c3c15b3f0 0x7f0c3c15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c168580 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c168400 con 0x7f0c3c0cc3e0 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c168700 con 0x7f0c3c0730d0 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c41600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c42000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c3c15b3f0 0x7f0c3c15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c41600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:44804/0 (socket says 172.21.15.86:44804) 2024-03-22T04:17:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c41600640 1 -- 172.21.15.86:0/3733046451 learned_addr learned my addr 172.21.15.86:0/3733046451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:14.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c40c00640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c3c0cc3e0 0x7f0c3c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c42000640 1 -- 172.21.15.86:0/3733046451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 msgr2=0x7f0c3c148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c42000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c42000640 1 -- 172.21.15.86:0/3733046451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c3c0cc3e0 msgr2=0x7f0c3c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c42000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c3c0cc3e0 0x7f0c3c15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c42000640 1 -- 172.21.15.86:0/3733046451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c15c2e0 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c42000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c3c15b3f0 0x7f0c3c15bda0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400eb70 tx=0x7f0c3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c39600640 1 -- 172.21.15.86:0/3733046451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34018020 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c39600640 1 -- 172.21.15.86:0/3733046451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3400cd20 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c39600640 1 -- 172.21.15.86:0/3733046451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c340146c0 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c1489c0 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.642+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c15c830 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.646+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c3c068650 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.646+0000 7f0c39600640 1 -- 172.21.15.86:0/3733046451 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0c3401b020 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.646+0000 7f0c39600640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0c1c067b20 0x7f0c1c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:14.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.646+0000 7f0c39600640 1 -- 172.21.15.86:0/3733046451 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7f0c34086be0 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.646+0000 7f0c41600640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0c1c067b20 0x7f0c1c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:14.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.646+0000 7f0c41600640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0c1c067b20 0x7f0c1c069fe0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0c30007fd0 tx=0x7f0c30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:14.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.650+0000 7f0c39600640 1 -- 172.21.15.86:0/3733046451 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0c34022030 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.842+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c3c15ac20 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.842+0000 7f0c39600640 1 -- 172.21.15.86:0/3733046451 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7f0c34010040 con 0x7f0c3c15b3f0 2024-03-22T04:17:14.843 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:17:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0c1c067b20 msgr2=0x7f0c1c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0c1c067b20 0x7f0c1c069fe0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0c30007fd0 tx=0x7f0c30009040 comp rx=0 tx=0).stop 2024-03-22T04:17:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c3c15b3f0 msgr2=0x7f0c3c15bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c3c15b3f0 0x7f0c3c15bda0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400eb70 tx=0x7f0c3400c490 comp rx=0 tx=0).stop 2024-03-22T04:17:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 shutdown_connections 2024-03-22T04:17:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0c1c067b20 0x7f0c1c069fe0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c3c0cc3e0 0x7f0c3c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c3c15b3f0 0x7f0c3c15bda0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 --2- 172.21.15.86:0/3733046451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c3c0730d0 0x7f0c3c148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 >> 172.21.15.86:0/3733046451 conn(0x7f0c3c0b1df0 msgr2=0x7f0c3c0b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 shutdown_connections 2024-03-22T04:17:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:14.846+0000 7f0c44000640 1 -- 172.21.15.86:0/3733046451 wait complete. 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:17:14.864 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:17:14.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:14.864 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:17:14.874 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42097sec preferred_lft 42097sec 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:17:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:17:14.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:14.876 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:17:14.937 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:17:14.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:14.939 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:17:14.948 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:17:14.948+0000 7f0c5f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:17:14.992 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:17:14.992 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:17:14.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:17:14.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:17:14.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:17:14.992 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:17:15.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:17:15.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:17:15.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:15.125 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:17:15.135 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:17:15.135 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:17:15.135 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:17:15.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:15.135 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:17:15.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:17:15.195 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:15.245 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:17:15.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:17:15.246 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:15.295 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:17:15.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:15.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:17:15.383 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:17:15.383 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:17:15.383 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:17:15.398 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:17:15.398 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:17:15.398 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:17:15.398 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:17:15.398 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:17:15.407 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:17:15.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:17:15.407 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:17:15.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:17:15.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:17:15.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:17:15.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:17:15.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:17:15.411 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:17:15.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:15.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:17:15.430 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:17:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:17:15.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:17:15.528 INFO:teuthology.orchestra.run.smithi086.stdout:client76664 2024-03-22T04:17:15.528 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:17:15.528 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:17:15.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:15.537 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-03-22T04:17:15.596 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:17:15.592+0000 7f0c5a400640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-22T04:17:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-03-22T04:17:15.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:17:15.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:15.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-03-22T04:17:15.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa803a00640 1 Processor -- start 2024-03-22T04:17:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa803a00640 1 -- start start 2024-03-22T04:17:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa803a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7fc15e6d0 0x7fa7fc06f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa803a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7fc0afef0 0x7fa7fc0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa803a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc0cef30 con 0x7fa7fc071df0 2024-03-22T04:17:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa803a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc0cf210 con 0x7fa7fc15e6d0 2024-03-22T04:17:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa803a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc06f080 con 0x7fa7fc0afef0 2024-03-22T04:17:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa801000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7fc15e6d0 0x7fa7fc06f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa801a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7fc071df0 0x7fa7fc06b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41528/0 (socket says 172.21.15.86:41528) 2024-03-22T04:17:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa801a00640 1 -- 172.21.15.86:0/2173016697 learned_addr learned my addr 172.21.15.86:0/2173016697 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.778+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/2173016697 >> v2:172.21.15.196:3300/0 conn(0x7fa7fc0afef0 0x7fa7fc0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa801000640 1 -- 172.21.15.86:0/2173016697 >> v2:172.21.15.196:3300/0 conn(0x7fa7fc0afef0 msgr2=0x7fa7fc0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa801000640 1 --2- 172.21.15.86:0/2173016697 >> v2:172.21.15.196:3300/0 conn(0x7fa7fc0afef0 0x7fa7fc0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa801000640 1 -- 172.21.15.86:0/2173016697 >> v1:172.21.15.86:6789/0 conn(0x7fa7fc071df0 legacy=0x7fa7fc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa801000640 1 -- 172.21.15.86:0/2173016697 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc0cec20 con 0x7fa7fc15e6d0 2024-03-22T04:17:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa801000640 1 --2- 172.21.15.86:0/2173016697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc15e6d0 0x7fa7fc06f880 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc072fa0 tx=0x7fa7f001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791b830095fee41d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa7fb400640 1 -- 172.21.15.86:0/2173016697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00b0070 con 0x7fa7fc15e6d0 2024-03-22T04:17:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa7fb400640 1 -- 172.21.15.86:0/2173016697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f00ab600 con 0x7fa7fc15e6d0 2024-03-22T04:17:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa7fb400640 1 -- 172.21.15.86:0/2173016697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00ab9d0 con 0x7fa7fc15e6d0 2024-03-22T04:17:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa803a00640 1 -- 172.21.15.86:0/2173016697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc15e6d0 msgr2=0x7fa7fc06f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa803a00640 1 --2- 172.21.15.86:0/2173016697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc15e6d0 0x7fa7fc06f880 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc072fa0 tx=0x7fa7f001c3f0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa803a00640 1 -- 172.21.15.86:0/2173016697 shutdown_connections 2024-03-22T04:17:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa803a00640 1 --2- 172.21.15.86:0/2173016697 >> v2:172.21.15.196:3300/0 conn(0x7fa7fc0afef0 0x7fa7fc0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa803a00640 1 --2- 172.21.15.86:0/2173016697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc15e6d0 0x7fa7fc06f880 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa803a00640 1 -- 172.21.15.86:0/2173016697 >> 172.21.15.86:0/2173016697 conn(0x7fa7fc0d07e0 msgr2=0x7fa7fc0d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa803a00640 1 -- 172.21.15.86:0/2173016697 shutdown_connections 2024-03-22T04:17:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.782+0000 7fa803a00640 1 -- 172.21.15.86:0/2173016697 wait complete. 2024-03-22T04:17:15.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 Processor -- start 2024-03-22T04:17:15.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 -- start start 2024-03-22T04:17:15.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc071df0 0x7fa7fc15c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc0afef0 0x7fa7fc144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc15e6d0 0x7fa7fc0ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc16bcb0 con 0x7fa7fc071df0 2024-03-22T04:17:15.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc06c390 con 0x7fa7fc0afef0 2024-03-22T04:17:15.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc16be30 con 0x7fa7fc15e6d0 2024-03-22T04:17:15.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc071df0 0x7fa7fc15c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc071df0 0x7fa7fc15c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57592/0 (socket says 172.21.15.86:57592) 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 -- 172.21.15.86:0/2014854986 learned_addr learned my addr 172.21.15.86:0/2014854986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc0afef0 0x7fa7fc144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 -- 172.21.15.86:0/2014854986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc15e6d0 msgr2=0x7fa7fc0ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc15e6d0 0x7fa7fc0ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 -- 172.21.15.86:0/2014854986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc0afef0 msgr2=0x7fa7fc144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc0afef0 0x7fa7fc144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 -- 172.21.15.86:0/2014854986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc0b56e0 con 0x7fa7fc071df0 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa801000640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc071df0 0x7fa7fc15c350 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0019000 tx=0x7fa7f0019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa7f8c00640 1 -- 172.21.15.86:0/2014854986 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00b0040 con 0x7fa7fc071df0 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa7f8c00640 1 -- 172.21.15.86:0/2014854986 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f00b9070 con 0x7fa7fc071df0 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa7f8c00640 1 -- 172.21.15.86:0/2014854986 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00b4650 con 0x7fa7fc071df0 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 -- 172.21.15.86:0/2014854986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc160290 con 0x7fa7fc071df0 2024-03-22T04:17:15.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.786+0000 7fa803a00640 1 -- 172.21.15.86:0/2014854986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc1552c0 con 0x7fa7fc071df0 2024-03-22T04:17:15.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.790+0000 7fa7f8c00640 1 -- 172.21.15.86:0/2014854986 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa7f00b47f0 con 0x7fa7fc071df0 2024-03-22T04:17:15.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.790+0000 7fa7f8c00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067b20 0x7fa7dc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.790+0000 7fa7f8c00640 1 -- 172.21.15.86:0/2014854986 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7fa7f011fdf0 con 0x7fa7fc071df0 2024-03-22T04:17:15.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.790+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067b20 0x7fa7dc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.790+0000 7fa806ec0000 1 Processor -- start 2024-03-22T04:17:15.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.790+0000 7fa806ec0000 1 -- start start 2024-03-22T04:17:15.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.790+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067b20 0x7fa7dc069fe0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0ce740 tx=0x7fa7e8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c53370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c538b0 0x5581b0c09310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c09c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581b0a57e70 con 0x5581b0c09850 2024-03-22T04:17:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581b09f5110 con 0x5581b0c52f90 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5581b0a53e20 con 0x5581b0c538b0 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c09c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c53370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c53370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:43362/0 (socket says 172.21.15.86:43362) 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c09c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57600/0 (socket says 172.21.15.86:57600) 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa7fbe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c538b0 0x5581b0c09310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 -- 172.21.15.86:0/3101006288 learned_addr learned my addr 172.21.15.86:0/3101006288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 -- 172.21.15.86:0/3101006288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c538b0 msgr2=0x5581b0c09310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 --2- 172.21.15.86:0/3101006288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c538b0 0x5581b0c09310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 -- 172.21.15.86:0/3101006288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 msgr2=0x5581b0c09c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 --2- 172.21.15.86:0/3101006288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c09c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 -- 172.21.15.86:0/3101006288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b0afef20 con 0x5581b0c52f90 2024-03-22T04:17:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801a00640 1 --2- 172.21.15.86:0/3101006288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c09c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa801000640 1 --2- 172.21.15.86:0/3101006288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c53370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00efc10 tx=0x7fa7f00ef870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5e69ba8168c714 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa7ed600640 1 -- 172.21.15.86:0/3101006288 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00f4060 con 0x5581b0c52f90 2024-03-22T04:17:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa7ed600640 1 -- 172.21.15.86:0/3101006288 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f00f8890 con 0x5581b0c52f90 2024-03-22T04:17:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa7ed600640 1 -- 172.21.15.86:0/3101006288 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00ec580 con 0x5581b0c52f90 2024-03-22T04:17:15.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- 172.21.15.86:0/3101006288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 msgr2=0x5581b0c53370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- 172.21.15.86:0/3101006288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c53370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00efc10 tx=0x7fa7f00ef870 comp rx=0 tx=0).stop 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- 172.21.15.86:0/3101006288 shutdown_connections 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- 172.21.15.86:0/3101006288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c09c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- 172.21.15.86:0/3101006288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c538b0 0x5581b0c09310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- 172.21.15.86:0/3101006288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c53370 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- 172.21.15.86:0/3101006288 >> 172.21.15.86:0/3101006288 conn(0x5581b0a04bd0 msgr2=0x5581b0c07bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- 172.21.15.86:0/3101006288 shutdown_connections 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- 172.21.15.86:0/3101006288 wait complete. 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- start start 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 Processor -- start 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c53ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c52f90 0x5581b0c543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c54920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581b09f5110 con 0x5581b0c09850 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581b0a15700 con 0x5581b0c538b0 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5581b0a57e70 con 0x5581b0c52f90 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.794+0000 7fa806ec0000 10 client.0 _ll_register_callbacks cb 0x5581b0a00140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c53ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c53ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57604/0 (socket says 172.21.15.86:57604) 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 -- 172.21.15.86:0/180016572 learned_addr learned my addr 172.21.15.86:0/180016572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801a00640 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c54920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c52f90 0x5581b0c543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 -- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c52f90 msgr2=0x5581b0c543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c52f90 0x5581b0c543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 -- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 msgr2=0x5581b0c54920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c54920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 -- 172.21.15.86:0/180016572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b0a34050 con 0x5581b0c09850 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa801000640 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c53ea0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00f4220 tx=0x7fa7f00aefb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b996b189c2bcc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:15.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 1 -- 172.21.15.86:0/180016572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00f4060 con 0x5581b0c09850 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 1 -- 172.21.15.86:0/180016572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7fa7f00f8600 con 0x5581b0c09850 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 -- 172.21.15.86:0/180016572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f00f8890 con 0x5581b0c09850 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 1 -- 172.21.15.86:0/180016572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fa7f0108020 con 0x5581b0c09850 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 10 client.7779 fetch_fsmap learned FSMap version 70 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 -- 172.21.15.86:0/180016572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5581b0a9cb90 con 0x5581b0c09850 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 1 -- 172.21.15.86:0/180016572 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f00f8890 con 0x5581b0c09850 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 1 -- 172.21.15.86:0/180016572 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x7fa7f00b3070 con 0x5581b0c09850 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 10 client.7779 fetch_fsmap finished waiting for FSMap version 70 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 10 client.7779 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 20 client.7779 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5581b0c62d60 0x5581b0c65150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 4 client.7779 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 -- 172.21.15.86:0/180016572 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581b09f2f70 con 0x5581b0c62d60 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5581b0c62d60 0x5581b0c65150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5581b0c62d60 0x5581b0c65150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9fc2ba84d57427 server_cookie=79e7f77e70b50b5b in_seq=0 out_seq=0 2024-03-22T04:17:15.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 10 client.7779 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 1 -- 172.21.15.86:0/180016572 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5581b09f2f70 con 0x5581b0c62d60 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 10 client.7779 handle_command_reply: tid=0 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 client.7779 shutdown 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 -- 172.21.15.86:0/180016572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 msgr2=0x5581b0c53ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c53ea0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00f4220 tx=0x7fa7f00aefb0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 -- 172.21.15.86:0/180016572 shutdown_connections 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5581b0c62d60 0x5581b0c65150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c54920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c52f90 0x5581b0c543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa7e2000640 0 client.7779 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 --2- 172.21.15.86:0/180016572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c09850 0x5581b0c53ea0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 -- 172.21.15.86:0/180016572 >> 172.21.15.86:0/180016572 conn(0x5581b0a04bd0 msgr2=0x5581b0c07bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 -- 172.21.15.86:0/180016572 shutdown_connections 2024-03-22T04:17:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 1 -- 172.21.15.86:0/180016572 wait complete. 2024-03-22T04:17:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.798+0000 7fa806ec0000 20 client.7779 trim_cache size 0 max 16384 2024-03-22T04:17:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 Processor -- start 2024-03-22T04:17:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- start start 2024-03-22T04:17:15.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 0x5581b0c51250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c51790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c538b0 0x5581b0c51cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc204720 con 0x5581b0c538b0 2024-03-22T04:17:15.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc153470 con 0x5581b0c52f90 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc156cb0 con 0x5581b0c09850 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c538b0 0x5581b0c51cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c538b0 0x5581b0c51cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57620/0 (socket says 172.21.15.86:57620) 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 -- 172.21.15.86:0/2332290718 learned_addr learned my addr 172.21.15.86:0/2332290718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801000640 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 0x5581b0c51250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c51790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 -- 172.21.15.86:0/2332290718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 msgr2=0x5581b0c51250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 0x5581b0c51250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 -- 172.21.15.86:0/2332290718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 msgr2=0x5581b0c51790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c51790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 -- 172.21.15.86:0/2332290718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b09dd4e0 con 0x5581b0c538b0 2024-03-22T04:17:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa801a00640 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c538b0 0x5581b0c51cd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4002210 tx=0x7fa7f4001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25e1406737485a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa7ecc00640 1 -- 172.21.15.86:0/2332290718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4019020 con 0x5581b0c538b0 2024-03-22T04:17:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa7ecc00640 1 -- 172.21.15.86:0/2332290718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f4012e60 con 0x5581b0c538b0 2024-03-22T04:17:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa7ecc00640 1 -- 172.21.15.86:0/2332290718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40074e0 con 0x5581b0c538b0 2024-03-22T04:17:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- 172.21.15.86:0/2332290718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c538b0 msgr2=0x5581b0c51cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c538b0 0x5581b0c51cd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4002210 tx=0x7fa7f4001950 comp rx=0 tx=0).stop 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- 172.21.15.86:0/2332290718 shutdown_connections 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c52f90 0x5581b0c51790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c538b0 0x5581b0c51cd0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 --2- 172.21.15.86:0/2332290718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 0x5581b0c51250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- 172.21.15.86:0/2332290718 >> 172.21.15.86:0/2332290718 conn(0x5581b0a04bd0 msgr2=0x5581b0c09c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- 172.21.15.86:0/2332290718 shutdown_connections 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.818+0000 7fa806ec0000 1 -- 172.21.15.86:0/2332290718 wait complete. 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 -- start start 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 Processor -- start 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 0x5581b0c52650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c52f90 0x5581b0c661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581b09fe410 con 0x5581b0c52f90 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc152e20 con 0x5581b0c538b0 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5581b0c01030 con 0x5581b0c09850 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 10 client.0 _ll_register_callbacks cb 0x7fa7f40017f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c52f90 0x5581b0c661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c52f90 0x5581b0c661a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57634/0 (socket says 172.21.15.86:57634) 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 -- 172.21.15.86:0/343724977 learned_addr learned my addr 172.21.15.86:0/343724977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa801000640 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 0x5581b0c52650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa801a00640 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 -- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 msgr2=0x5581b0c52650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 0x5581b0c52650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 -- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 msgr2=0x5581b0c666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 -- 172.21.15.86:0/343724977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b0ad7f80 con 0x5581b0c52f90 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa801a00640 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7fbe00640 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c52f90 0x5581b0c661a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8016040 tx=0x7fa7e8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecb13cbf1d7dd74 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7e2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7e2000640 1 -- 172.21.15.86:0/343724977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8031030 con 0x5581b0c52f90 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7e2000640 1 -- 172.21.15.86:0/343724977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6187+0+0 (secure 0 0 0) 0x7fa7e800f040 con 0x5581b0c52f90 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 -- 172.21.15.86:0/343724977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f4012e60 con 0x5581b0c52f90 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7e2000640 1 -- 172.21.15.86:0/343724977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fa7e803d050 con 0x5581b0c52f90 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 10 client.7785 fetch_fsmap learned FSMap version 70 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 -- 172.21.15.86:0/343724977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5581b09fe670 con 0x5581b0c52f90 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7e2000640 1 -- 172.21.15.86:0/343724977 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f4012e60 con 0x5581b0c52f90 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7e2000640 1 -- 172.21.15.86:0/343724977 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x7fa7e8040020 con 0x5581b0c52f90 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 10 client.7785 fetch_fsmap finished waiting for FSMap version 70 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 10 client.7785 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 20 client.7785 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5581b0c66c20 0x5581b0cc3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 4 client.7785 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-03-22T04:17:15.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa806ec0000 1 -- 172.21.15.86:0/343724977 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x5581b0a653b0 con 0x5581b0c66c20 2024-03-22T04:17:15.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa801000640 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5581b0c66c20 0x5581b0cc3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.827 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:17:15.824+0000 7f0c5a400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-03-22T04:17:05.898187+0000 1=1+0) n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55c18476bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-22T04:17:05.898187+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-22T04:17:15.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa7e2000640 10 client.7785 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.822+0000 7fa801000640 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5581b0c66c20 0x5581b0cc3ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dae3859fb34212d server_cookie=593f6039ba90e0e9 in_seq=0 out_seq=0 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa7e2000640 1 -- 172.21.15.86:0/343724977 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581b0a653b0 con 0x5581b0c66c20 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa7e2000640 10 client.7785 handle_command_reply: tid=0 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 client.7785 shutdown 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 -- 172.21.15.86:0/343724977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c52f90 msgr2=0x5581b0c661a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c52f90 0x5581b0c661a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8016040 tx=0x7fa7e8004740 comp rx=0 tx=0).stop 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 -- 172.21.15.86:0/343724977 shutdown_connections 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x5581b0c66c20 0x5581b0cc3ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581b0c538b0 0x5581b0c666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581b0c52f90 0x5581b0c661a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa7e2000640 0 client.7785 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 --2- 172.21.15.86:0/343724977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5581b0c09850 0x5581b0c52650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 -- 172.21.15.86:0/343724977 >> 172.21.15.86:0/343724977 conn(0x5581b0a04bd0 msgr2=0x5581b0c0a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:15.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 -- 172.21.15.86:0/343724977 shutdown_connections 2024-03-22T04:17:15.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 1 -- 172.21.15.86:0/343724977 wait complete. 2024-03-22T04:17:15.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa806ec0000 20 client.7785 trim_cache size 0 max 16384 2024-03-22T04:17:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:17:15.830 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:17:15.830 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "d639452a-7d30-46a2-ab5e-42b61e6c32de", 2024-03-22T04:17:15.830 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:17:15.830 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:17:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 -- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067b20 msgr2=0x7fa7dc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067b20 0x7fa7dc069fe0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0ce740 tx=0x7fa7e8009190 comp rx=0 tx=0).stop 2024-03-22T04:17:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 -- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc071df0 msgr2=0x7fa7fc15c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc071df0 0x7fa7fc15c350 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0019000 tx=0x7fa7f0019190 comp rx=0 tx=0).stop 2024-03-22T04:17:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 -- 172.21.15.86:0/2014854986 shutdown_connections 2024-03-22T04:17:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa7dc067b20 0x7fa7dc069fe0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7fc0afef0 0x7fa7fc144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7fc15e6d0 0x7fa7fc0ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 --2- 172.21.15.86:0/2014854986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc071df0 0x7fa7fc15c350 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 -- 172.21.15.86:0/2014854986 >> 172.21.15.86:0/2014854986 conn(0x7fa7fc0d07e0 msgr2=0x7fa7fc0b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 -- 172.21.15.86:0/2014854986 shutdown_connections 2024-03-22T04:17:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.826+0000 7fa803a00640 1 -- 172.21.15.86:0/2014854986 wait complete. 2024-03-22T04:17:15.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:15.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-22T04:17:15.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.990+0000 7fd722c00640 1 Processor -- start 2024-03-22T04:17:15.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.990+0000 7fd722c00640 1 -- start start 2024-03-22T04:17:15.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd722c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072b00 0x7fd71c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:15.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd722c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd71c073150 con 0x7fd71c072650 2024-03-22T04:17:15.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd722c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd71c0715a0 con 0x7fd71c06e7e0 2024-03-22T04:17:15.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd722c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c071350 con 0x7fd71c072b00 2024-03-22T04:17:15.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd71b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd71c072650 0x7fd71c15b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35592/0 (socket says 172.21.15.86:35592) 2024-03-22T04:17:15.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd71b400640 1 -- 172.21.15.86:0/1308775925 learned_addr learned my addr 172.21.15.86:0/1308775925 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:15.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd720c00640 1 --2- 172.21.15.86:0/1308775925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072b00 0x7fd71c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:15.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd71aa00640 1 -- 172.21.15.86:0/1308775925 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221045317 0 0) 0x7fd71c073150 con 0x7fd71c072650 2024-03-22T04:17:15.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd71aa00640 1 -- 172.21.15.86:0/1308775925 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd700003530 con 0x7fd71c072650 2024-03-22T04:17:15.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd720c00640 1 -- 172.21.15.86:0/1308775925 >> v1:172.21.15.196:6789/0 conn(0x7fd71c06e7e0 legacy=0x7fd71c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd71aa00640 1 -- 172.21.15.86:0/1308775925 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810501432 0 0) 0x7fd71c0715a0 con 0x7fd71c06e7e0 2024-03-22T04:17:15.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd720c00640 1 -- 172.21.15.86:0/1308775925 >> v1:172.21.15.86:6789/0 conn(0x7fd71c072650 legacy=0x7fd71c15b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.994+0000 7fd720c00640 1 -- 172.21.15.86:0/1308775925 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c071880 con 0x7fd71c072b00 2024-03-22T04:17:15.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd720c00640 1 --2- 172.21.15.86:0/1308775925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072b00 0x7fd71c15be70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd714009dc0 tx=0x7fd71401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=196aff9919b988f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:15.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd71aa00640 1 -- 172.21.15.86:0/1308775925 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7140b2070 con 0x7fd71c072b00 2024-03-22T04:17:15.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd71aa00640 1 -- 172.21.15.86:0/1308775925 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7140a58d0 con 0x7fd71c072b00 2024-03-22T04:17:15.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd71aa00640 1 -- 172.21.15.86:0/1308775925 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7140a5ca0 con 0x7fd71c072b00 2024-03-22T04:17:15.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 -- 172.21.15.86:0/1308775925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072b00 msgr2=0x7fd71c15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:15.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 --2- 172.21.15.86:0/1308775925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072b00 0x7fd71c15be70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd714009dc0 tx=0x7fd71401c900 comp rx=0 tx=0).stop 2024-03-22T04:17:15.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 -- 172.21.15.86:0/1308775925 shutdown_connections 2024-03-22T04:17:15.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 --2- 172.21.15.86:0/1308775925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072b00 0x7fd71c15be70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:15.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 -- 172.21.15.86:0/1308775925 >> 172.21.15.86:0/1308775925 conn(0x7fd71c0cc940 msgr2=0x7fd71c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:15.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 -- 172.21.15.86:0/1308775925 shutdown_connections 2024-03-22T04:17:15.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 -- 172.21.15.86:0/1308775925 wait complete. 2024-03-22T04:17:15.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 Processor -- start 2024-03-22T04:17:15.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:15.998+0000 7fd722c00640 1 -- start start 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd722c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd71c06e7e0 0x7fd71c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd722c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072650 0x7fd71c14de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd722c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072b00 0x7fd71c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd722c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c155c20 con 0x7fd71c072b00 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd722c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c155aa0 con 0x7fd71c06e7e0 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd722c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c169610 con 0x7fd71c072650 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072b00 0x7fd71c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072b00 0x7fd71c153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60236/0 (socket says 172.21.15.86:60236) 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 -- 172.21.15.86:0/3581698769 learned_addr learned my addr 172.21.15.86:0/3581698769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd71b400640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072650 0x7fd71c14de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd71be00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd71c06e7e0 0x7fd71c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 -- 172.21.15.86:0/3581698769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072650 msgr2=0x7fd71c14de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072650 0x7fd71c14de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 -- 172.21.15.86:0/3581698769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd71c06e7e0 msgr2=0x7fd71c0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd71c06e7e0 0x7fd71c0cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 -- 172.21.15.86:0/3581698769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c14e9b0 con 0x7fd71c072b00 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd720c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072b00 0x7fd71c153a20 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd7140b0040 tx=0x7fd7140196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd713e00640 1 -- 172.21.15.86:0/3581698769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7140b2040 con 0x7fd71c072b00 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd713e00640 1 -- 172.21.15.86:0/3581698769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd714003bc0 con 0x7fd71c072b00 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd722c00640 1 -- 172.21.15.86:0/3581698769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71c158120 con 0x7fd71c072b00 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd713e00640 1 -- 172.21.15.86:0/3581698769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714004030 con 0x7fd71c072b00 2024-03-22T04:17:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd722c00640 1 -- 172.21.15.86:0/3581698769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71c14d650 con 0x7fd71c072b00 2024-03-22T04:17:16.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.002+0000 7fd713e00640 1 -- 172.21.15.86:0/3581698769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd7140a5790 con 0x7fd71c072b00 2024-03-22T04:17:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd713e00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6fc067b50 0x7fd6fc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd713e00640 1 -- 172.21.15.86:0/3581698769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6448+0+0 (secure 0 0 0) 0x7fd714121070 con 0x7fd71c072b00 2024-03-22T04:17:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6fc067b50 0x7fd6fc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd7260bf000 1 Processor -- start 2024-03-22T04:17:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd7260bf000 1 -- start start 2024-03-22T04:17:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6fc067b50 0x7fd6fc06a010 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd70c000f70 tx=0x7fd70c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:16.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da45360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da458a0 0x55682d9f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 0x55682d9f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd7260bf000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55682d849c40 con 0x55682da44f80 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd7260bf000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55682d7e6d90 con 0x55682d9f8f50 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd7260bf000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55682d845bf0 con 0x55682da458a0 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da45360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da45360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60246/0 (socket says 172.21.15.86:60246) 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 -- 172.21.15.86:0/4257532155 learned_addr learned my addr 172.21.15.86:0/4257532155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71b400640 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da458a0 0x55682d9f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd720c00640 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 0x55682d9f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 -- 172.21.15.86:0/4257532155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da458a0 msgr2=0x55682d9f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da458a0 0x55682d9f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 -- 172.21.15.86:0/4257532155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 msgr2=0x55682d9f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 0x55682d9f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 -- 172.21.15.86:0/4257532155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682d8f1130 con 0x55682da44f80 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd71be00640 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da45360 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0052e0 tx=0x7fd70c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f040a4fb87202bc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd710c00640 1 -- 172.21.15.86:0/4257532155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c005450 con 0x55682da44f80 2024-03-22T04:17:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd710c00640 1 -- 172.21.15.86:0/4257532155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70c00e710 con 0x55682da44f80 2024-03-22T04:17:16.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.006+0000 7fd710c00640 1 -- 172.21.15.86:0/4257532155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00eac0 con 0x55682da44f80 2024-03-22T04:17:16.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- 172.21.15.86:0/4257532155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 msgr2=0x55682da45360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:16.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da45360 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0052e0 tx=0x7fd70c011000 comp rx=0 tx=0).stop 2024-03-22T04:17:16.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- 172.21.15.86:0/4257532155 shutdown_connections 2024-03-22T04:17:16.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 0x55682d9f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da458a0 0x55682d9f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 --2- 172.21.15.86:0/4257532155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da45360 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- 172.21.15.86:0/4257532155 >> 172.21.15.86:0/4257532155 conn(0x55682d8e2990 msgr2=0x55682da28150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- 172.21.15.86:0/4257532155 shutdown_connections 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- 172.21.15.86:0/4257532155 wait complete. 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- start start 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 Processor -- start 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 0x55682da009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da44f80 0x55682da29380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682da298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55682d805710 con 0x55682da458a0 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55682d807580 con 0x55682d9f8f50 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55682d84e270 con 0x55682da44f80 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 10 client.0 _ll_register_callbacks cb 0x55682d8ab200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682da298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682da298c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60248/0 (socket says 172.21.15.86:60248) 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 -- 172.21.15.86:0/1004320507 learned_addr learned my addr 172.21.15.86:0/1004320507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd71be00640 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 0x55682da009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd71b400640 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da44f80 0x55682da29380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 -- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da44f80 msgr2=0x55682da29380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da44f80 0x55682da29380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 -- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 msgr2=0x55682da009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 0x55682da009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 -- 172.21.15.86:0/1004320507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682d822d30 con 0x55682da458a0 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd720c00640 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682da298c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd71401c940 tx=0x7fd7140f0a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fcaf640acb9681 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd705600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd705600640 1 -- 172.21.15.86:0/1004320507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7140f99a0 con 0x55682da458a0 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd705600640 1 -- 172.21.15.86:0/1004320507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6448+0+0 (secure 0 0 0) 0x7fd7140f1280 con 0x55682da458a0 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- 172.21.15.86:0/1004320507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd70c00e710 con 0x55682da458a0 2024-03-22T04:17:16.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd705600640 1 -- 172.21.15.86:0/1004320507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fd7140f1480 con 0x55682da458a0 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 10 client.7797 fetch_fsmap learned FSMap version 70 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- 172.21.15.86:0/1004320507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55682d88efc0 con 0x55682da458a0 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd705600640 1 -- 172.21.15.86:0/1004320507 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70c00e710 con 0x55682da458a0 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd705600640 1 -- 172.21.15.86:0/1004320507 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x7fd7140f9510 con 0x55682da458a0 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 10 client.7797 fetch_fsmap finished waiting for FSMap version 70 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 10 client.7797 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 20 client.7797 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55682da4b5c0 0x55682da4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 4 client.7797 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.010+0000 7fd7260bf000 1 -- 172.21.15.86:0/1004320507 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55682d7e4dd0 con 0x55682da4b5c0 2024-03-22T04:17:16.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd71be00640 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55682da4b5c0 0x55682da4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd71be00640 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55682da4b5c0 0x55682da4b9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da4c48a8235683f server_cookie=89b47c2529b52a7d in_seq=0 out_seq=0 2024-03-22T04:17:16.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd705600640 10 client.7797 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd705600640 1 -- 172.21.15.86:0/1004320507 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55682d7e4dd0 con 0x55682da4b5c0 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd705600640 10 client.7797 handle_command_reply: tid=0 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 client.7797 shutdown 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 -- 172.21.15.86:0/1004320507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 msgr2=0x55682da298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682da298c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd71401c940 tx=0x7fd7140f0a70 comp rx=0 tx=0).stop 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 -- 172.21.15.86:0/1004320507 shutdown_connections 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55682da4b5c0 0x55682da4b9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682da298c0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682da44f80 0x55682da29380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd705600640 0 client.7797 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1004320507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682d9f8f50 0x55682da009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 -- 172.21.15.86:0/1004320507 >> 172.21.15.86:0/1004320507 conn(0x55682d8e2990 msgr2=0x55682d9f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 -- 172.21.15.86:0/1004320507 shutdown_connections 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 1 -- 172.21.15.86:0/1004320507 wait complete. 2024-03-22T04:17:16.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.014+0000 7fd7260bf000 20 client.7797 trim_cache size 0 max 16384 2024-03-22T04:17:16.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd7260bf000 1 Processor -- start 2024-03-22T04:17:16.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd7260bf000 1 -- start start 2024-03-22T04:17:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 0x55682da51c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da5b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da458a0 0x55682da49ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd7260bf000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c2023f0 con 0x55682da44f80 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd7260bf000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c2013d0 con 0x55682da458a0 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd7260bf000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c156a70 con 0x55682d9f8f50 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd71b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da5b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.030+0000 7fd71b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da5b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60260/0 (socket says 172.21.15.86:60260) 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71b400640 1 -- 172.21.15.86:0/1648415930 learned_addr learned my addr 172.21.15.86:0/1648415930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71be00640 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 0x55682da51c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da458a0 0x55682da49ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71b400640 1 -- 172.21.15.86:0/1648415930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 msgr2=0x55682da51c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71b400640 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 0x55682da51c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71b400640 1 -- 172.21.15.86:0/1648415930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da458a0 msgr2=0x55682da49ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71b400640 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da458a0 0x55682da49ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71b400640 1 -- 172.21.15.86:0/1648415930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682d81f3a0 con 0x55682da44f80 2024-03-22T04:17:16.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71b400640 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da5b680 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd708000f70 tx=0x7fd708000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989f9b314a8f98e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd707e00640 1 -- 172.21.15.86:0/1648415930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708015c10 con 0x55682da44f80 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd707e00640 1 -- 172.21.15.86:0/1648415930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70801a030 con 0x55682da44f80 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd707e00640 1 -- 172.21.15.86:0/1648415930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7080058c0 con 0x55682da44f80 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- 172.21.15.86:0/1648415930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 msgr2=0x55682da5b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da5b680 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd708000f70 tx=0x7fd708000bd0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- 172.21.15.86:0/1648415930 shutdown_connections 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da458a0 0x55682da49ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da44f80 0x55682da5b680 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 --2- 172.21.15.86:0/1648415930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 0x55682da51c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- 172.21.15.86:0/1648415930 >> 172.21.15.86:0/1648415930 conn(0x55682d8e2990 msgr2=0x55682da010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- 172.21.15.86:0/1648415930 shutdown_connections 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- 172.21.15.86:0/1648415930 wait complete. 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- start start 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 Processor -- start 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 0x55682da5b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da44f80 0x55682dab35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682dab3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55682d905810 con 0x55682da458a0 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c200d40 con 0x55682da44f80 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55682d7f0350 con 0x55682d9f8f50 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd7260bf000 10 client.0 _ll_register_callbacks cb 0x7fd7140f0df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682dab3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682dab3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60264/0 (socket says 172.21.15.86:60264) 2024-03-22T04:17:16.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 -- 172.21.15.86:0/3408951610 learned_addr learned my addr 172.21.15.86:0/3408951610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71b400640 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da44f80 0x55682dab35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd71be00640 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 0x55682da5b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 -- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 msgr2=0x55682da5b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 0x55682da5b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 -- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da44f80 msgr2=0x55682dab35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da44f80 0x55682dab35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.034+0000 7fd720c00640 1 -- 172.21.15.86:0/3408951610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682d7cf380 con 0x55682da458a0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd720c00640 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682dab3b00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd7140ed480 tx=0x7fd71411f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a94597598a99b07 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 1 -- 172.21.15.86:0/3408951610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7140f99a0 con 0x55682da458a0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 1 -- 172.21.15.86:0/3408951610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6448+0+0 (secure 0 0 0) 0x7fd714118db0 con 0x55682da458a0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 -- 172.21.15.86:0/3408951610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd70801a030 con 0x55682da458a0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 1 -- 172.21.15.86:0/3408951610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fd7140f0470 con 0x55682da458a0 2024-03-22T04:17:16.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 10 client.7803 fetch_fsmap learned FSMap version 70 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 -- 172.21.15.86:0/3408951610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55682d8cb460 con 0x55682da458a0 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 1 -- 172.21.15.86:0/3408951610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70801a030 con 0x55682da458a0 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 1 -- 172.21.15.86:0/3408951610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 70) ==== 3494+0+0 (secure 0 0 0) 0x7fd7140f0610 con 0x55682da458a0 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 10 client.7803 fetch_fsmap finished waiting for FSMap version 70 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 10 client.7803 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 20 client.7803 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55682da4f030 0x55682da4f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 4 client.7803 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 -- 172.21.15.86:0/3408951610 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55682d7e9160 con 0x55682da4f030 2024-03-22T04:17:16.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd71be00640 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55682da4f030 0x55682da4f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:16.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd71be00640 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55682da4f030 0x55682da4f440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b2d9f6333ff705 server_cookie=d668eb6a2acef8b5 in_seq=0 out_seq=0 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 10 client.7803 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 1 -- 172.21.15.86:0/3408951610 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55682d7e9160 con 0x55682da4f030 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 10 client.7803 handle_command_reply: tid=0 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 client.7803 shutdown 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 -- 172.21.15.86:0/3408951610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 msgr2=0x55682dab3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682dab3b00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd7140ed480 tx=0x7fd71411f110 comp rx=0 tx=0).stop 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 -- 172.21.15.86:0/3408951610 shutdown_connections 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55682da4f030 0x55682da4f440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55682da44f80 0x55682dab35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682da458a0 0x55682dab3b00 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd705600640 0 client.7803 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 --2- 172.21.15.86:0/3408951610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55682d9f8f50 0x55682da5b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 -- 172.21.15.86:0/3408951610 >> 172.21.15.86:0/3408951610 conn(0x55682d8e2990 msgr2=0x55682d9ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:16.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 -- 172.21.15.86:0/3408951610 shutdown_connections 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 1 -- 172.21.15.86:0/3408951610 wait complete. 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd7260bf000 20 client.7803 trim_cache size 0 max 16384 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout: "d639452a-7d30-46a2-ab5e-42b61e6c32de": { 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir", 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "d639452a-7d30-46a2-ab5e-42b61e6c32de", 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive" 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:17:16.043 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:17:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 -- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6fc067b50 msgr2=0x7fd6fc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6fc067b50 0x7fd6fc06a010 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd70c000f70 tx=0x7fd70c009040 comp rx=0 tx=0).stop 2024-03-22T04:17:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 -- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072b00 msgr2=0x7fd71c153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072b00 0x7fd71c153a20 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd7140b0040 tx=0x7fd7140196a0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 -- 172.21.15.86:0/3581698769 shutdown_connections 2024-03-22T04:17:16.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6fc067b50 0x7fd6fc06a010 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072b00 0x7fd71c153a20 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71c072650 0x7fd71c14de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 --2- 172.21.15.86:0/3581698769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd71c06e7e0 0x7fd71c0cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:16.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 -- 172.21.15.86:0/3581698769 >> 172.21.15.86:0/3581698769 conn(0x7fd71c0cc940 msgr2=0x7fd71c15a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:16.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 -- 172.21.15.86:0/3581698769 shutdown_connections 2024-03-22T04:17:16.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:16.038+0000 7fd722c00640 1 -- 172.21.15.86:0/3581698769 wait complete. 2024-03-22T04:17:16.058 INFO:tasks.cephfs.filesystem:scrub status for tag:d639452a-7d30-46a2-ab5e-42b61e6c32de - {'path': '/dir', 'tag': 'd639452a-7d30-46a2-ab5e-42b61e6c32de', 'options': 'recursive'} 2024-03-22T04:17:20.941 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:17:20.942+0000 7f3dd0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-22T04:17:28.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:17:28.305 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:17:46.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:46.060 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-22T04:17:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.205+0000 7f08eae00640 1 Processor -- start 2024-03-22T04:17:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08eae00640 1 -- start start 2024-03-22T04:17:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08eae00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e406e7e0 0x7f08e406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08eae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e415c770 con 0x7f08e4072ad0 2024-03-22T04:17:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08eae00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e40b29a0 con 0x7f08e4072650 2024-03-22T04:17:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08eae00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f08e4071350 con 0x7f08e406e7e0 2024-03-22T04:17:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e8e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f08e4072ad0 0x7f08e40b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54720/0 (socket says 172.21.15.86:54720) 2024-03-22T04:17:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e8e00640 1 -- 172.21.15.86:0/995808132 learned_addr learned my addr 172.21.15.86:0/995808132 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e3e00640 1 --2- 172.21.15.86:0/995808132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e406e7e0 0x7f08e406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e2a00640 1 -- 172.21.15.86:0/995808132 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4204091698 0 0) 0x7f08dc001120 con 0x7f08e4072ad0 2024-03-22T04:17:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e2a00640 1 -- 172.21.15.86:0/995808132 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c8003530 con 0x7f08e4072ad0 2024-03-22T04:17:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e2a00640 1 -- 172.21.15.86:0/995808132 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3129283470 0 0) 0x7f08e40b29a0 con 0x7f08e4072650 2024-03-22T04:17:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e2a00640 1 -- 172.21.15.86:0/995808132 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08dc001120 con 0x7f08e4072650 2024-03-22T04:17:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e3e00640 1 -- 172.21.15.86:0/995808132 >> v1:172.21.15.196:6789/0 conn(0x7f08e4072650 legacy=0x7f08e40b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e3e00640 1 -- 172.21.15.86:0/995808132 >> v1:172.21.15.86:6789/0 conn(0x7f08e4072ad0 legacy=0x7f08e40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.209+0000 7f08e3e00640 1 -- 172.21.15.86:0/995808132 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e406f100 con 0x7f08e406e7e0 2024-03-22T04:17:46.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08e3e00640 1 --2- 172.21.15.86:0/995808132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e406e7e0 0x7f08e406ebc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009dc0 tx=0x7f08d401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9709df23b74b98 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08e2a00640 1 -- 172.21.15.86:0/995808132 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40b2070 con 0x7f08e406e7e0 2024-03-22T04:17:46.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08e2a00640 1 -- 172.21.15.86:0/995808132 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d40a58b0 con 0x7f08e406e7e0 2024-03-22T04:17:46.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08e2a00640 1 -- 172.21.15.86:0/995808132 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40a5c80 con 0x7f08e406e7e0 2024-03-22T04:17:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08eae00640 1 -- 172.21.15.86:0/995808132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e406e7e0 msgr2=0x7f08e406ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08eae00640 1 --2- 172.21.15.86:0/995808132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e406e7e0 0x7f08e406ebc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009dc0 tx=0x7f08d401c900 comp rx=0 tx=0).stop 2024-03-22T04:17:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08eae00640 1 -- 172.21.15.86:0/995808132 shutdown_connections 2024-03-22T04:17:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08eae00640 1 --2- 172.21.15.86:0/995808132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e406e7e0 0x7f08e406ebc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08eae00640 1 -- 172.21.15.86:0/995808132 >> 172.21.15.86:0/995808132 conn(0x7f08e40cc940 msgr2=0x7f08e40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08eae00640 1 -- 172.21.15.86:0/995808132 shutdown_connections 2024-03-22T04:17:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08eae00640 1 -- 172.21.15.86:0/995808132 wait complete. 2024-03-22T04:17:46.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.213+0000 7f08eae00640 1 Processor -- start 2024-03-22T04:17:46.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 -- start start 2024-03-22T04:17:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08e406e7e0 0x7f08e4154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e4072650 0x7f08e40ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e4072ad0 0x7f08e40ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08e40ce930 con 0x7f08e4072650 2024-03-22T04:17:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4071b60 con 0x7f08e406e7e0 2024-03-22T04:17:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f08e4072080 con 0x7f08e4072ad0 2024-03-22T04:17:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e4072650 0x7f08e40ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e4072650 0x7f08e40ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44564/0 (socket says 172.21.15.86:44564) 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08e406e7e0 0x7f08e4154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 -- 172.21.15.86:0/1960831957 learned_addr learned my addr 172.21.15.86:0/1960831957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e8e00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e4072ad0 0x7f08e40ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 -- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e4072ad0 msgr2=0x7f08e40ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e4072ad0 0x7f08e40ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 -- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08e406e7e0 msgr2=0x7f08e4154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08e406e7e0 0x7f08e4154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 -- 172.21.15.86:0/1960831957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e40b4ad0 con 0x7f08e4072650 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08e406e7e0 0x7f08e4154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e8e00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e4072ad0 0x7f08e40ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08e3400640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e4072650 0x7f08e40ca3b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f08d000c460 tx=0x7f08d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08dbe00640 1 -- 172.21.15.86:0/1960831957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0017070 con 0x7f08e4072650 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08dbe00640 1 -- 172.21.15.86:0/1960831957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d00052f0 con 0x7f08e4072650 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08dbe00640 1 -- 172.21.15.86:0/1960831957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0003840 con 0x7f08e4072650 2024-03-22T04:17:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 -- 172.21.15.86:0/1960831957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e41591a0 con 0x7f08e4072650 2024-03-22T04:17:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.217+0000 7f08eae00640 1 -- 172.21.15.86:0/1960831957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e40cb200 con 0x7f08e4072650 2024-03-22T04:17:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08dbe00640 1 -- 172.21.15.86:0/1960831957 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f08d001a070 con 0x7f08e4072650 2024-03-22T04:17:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08dbe00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f08c4067b20 0x7f08c4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08dbe00640 1 -- 172.21.15.86:0/1960831957 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f08d00855a0 con 0x7f08e4072650 2024-03-22T04:17:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f08c4067b20 0x7f08c4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08ee22b000 1 Processor -- start 2024-03-22T04:17:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08ee22b000 1 -- start start 2024-03-22T04:17:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f08c4067b20 0x7f08c4069fe0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f08d40af000 tx=0x7f08d40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c767a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 0x55a10c768330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08ee22b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a10c5b8c40 con 0x55a10c7b3f80 2024-03-22T04:17:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08ee22b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a10c555d90 con 0x55a10c7b48a0 2024-03-22T04:17:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08ee22b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55a10c5b4bf0 con 0x55a10c767f50 2024-03-22T04:17:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44578/0 (socket says 172.21.15.86:44578) 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08e3e00640 1 -- 172.21.15.86:0/2472009306 learned_addr learned my addr 172.21.15.86:0/2472009306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08e8e00640 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 0x55a10c768330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.221+0000 7f08e3400640 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c767a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 -- 172.21.15.86:0/2472009306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 msgr2=0x55a10c768330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 0x55a10c768330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 -- 172.21.15.86:0/2472009306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 msgr2=0x55a10c767a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c767a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 -- 172.21.15.86:0/2472009306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10c660130 con 0x55a10c7b3f80 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7b4360 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000f90 tx=0x7f08d4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16de9f7287f207b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08d8c00640 1 -- 172.21.15.86:0/2472009306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40be030 con 0x55a10c7b3f80 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08d8c00640 1 -- 172.21.15.86:0/2472009306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d40b3040 con 0x55a10c7b3f80 2024-03-22T04:17:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08d8c00640 1 -- 172.21.15.86:0/2472009306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40b2050 con 0x55a10c7b3f80 2024-03-22T04:17:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- 172.21.15.86:0/2472009306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 msgr2=0x55a10c7b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7b4360 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000f90 tx=0x7f08d4004ac0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- 172.21.15.86:0/2472009306 shutdown_connections 2024-03-22T04:17:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c767a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 0x55a10c768330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 --2- 172.21.15.86:0/2472009306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7b4360 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- 172.21.15.86:0/2472009306 >> 172.21.15.86:0/2472009306 conn(0x55a10c651990 msgr2=0x55a10c797150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- 172.21.15.86:0/2472009306 shutdown_connections 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- 172.21.15.86:0/2472009306 wait complete. 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- start start 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 Processor -- start 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c767f50 0x55a10c76fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b3f80 0x55a10c798380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 0x55a10c7988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a10c574710 con 0x55a10c767f50 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a10c576580 con 0x55a10c7b3f80 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55a10c5bd270 con 0x55a10c7b48a0 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08ee22b000 10 client.0 _ll_register_callbacks cb 0x55a10c61a200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c767f50 0x55a10c76fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c767f50 0x55a10c76fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44582/0 (socket says 172.21.15.86:44582) 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 -- 172.21.15.86:0/1327230456 learned_addr learned my addr 172.21.15.86:0/1327230456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3400640 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b3f80 0x55a10c798380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e8e00640 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 0x55a10c7988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 -- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 msgr2=0x55a10c7988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 0x55a10c7988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 -- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b3f80 msgr2=0x55a10c798380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b3f80 0x55a10c798380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.225+0000 7f08e3e00640 1 -- 172.21.15.86:0/1327230456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a10c591d30 con 0x55a10c767f50 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c767f50 0x55a10c76fa00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f08d4019510 tx=0x7f08d401b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e2be4aeeecc36d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 1 -- 172.21.15.86:0/1327230456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40b2070 con 0x55a10c767f50 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 1 -- 172.21.15.86:0/1327230456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f08d40ad440 con 0x55a10c767f50 2024-03-22T04:17:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 -- 172.21.15.86:0/1327230456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d40b3040 con 0x55a10c767f50 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 1 -- 172.21.15.86:0/1327230456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f08d40da050 con 0x55a10c767f50 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 10 client.7815 fetch_fsmap learned FSMap version 71 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 -- 172.21.15.86:0/1327230456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55a10c5fdfc0 con 0x55a10c767f50 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 1 -- 172.21.15.86:0/1327230456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d40b3040 con 0x55a10c767f50 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 1 -- 172.21.15.86:0/1327230456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f08d40dd020 con 0x55a10c767f50 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 10 client.7815 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 10 client.7815 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 20 client.7815 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55a10c7ba5c0 0x55a10c7ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 4 client.7815 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 -- 172.21.15.86:0/1327230456 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a10c553dd0 con 0x55a10c7ba5c0 2024-03-22T04:17:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08e3400640 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55a10c7ba5c0 0x55a10c7ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 10 client.7815 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08e3400640 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55a10c7ba5c0 0x55a10c7ba9d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc84790e8840dc60 server_cookie=3bd5579ff782f4ec in_seq=0 out_seq=0 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 1 -- 172.21.15.86:0/1327230456 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a10c553dd0 con 0x55a10c7ba5c0 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 10 client.7815 handle_command_reply: tid=0 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 client.7815 shutdown 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 -- 172.21.15.86:0/1327230456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c767f50 msgr2=0x55a10c76fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c767f50 0x55a10c76fa00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f08d4019510 tx=0x7f08d401b140 comp rx=0 tx=0).stop 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 -- 172.21.15.86:0/1327230456 shutdown_connections 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55a10c7ba5c0 0x55a10c7ba9d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b3f80 0x55a10c798380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 0x55a10c7988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08cd600640 0 client.7815 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1327230456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c767f50 0x55a10c76fa00 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 -- 172.21.15.86:0/1327230456 >> 172.21.15.86:0/1327230456 conn(0x55a10c651990 msgr2=0x55a10c767110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 -- 172.21.15.86:0/1327230456 shutdown_connections 2024-03-22T04:17:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 1 -- 172.21.15.86:0/1327230456 wait complete. 2024-03-22T04:17:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.229+0000 7f08ee22b000 20 client.7815 trim_cache size 0 max 16384 2024-03-22T04:17:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.245+0000 7f08ee22b000 1 Processor -- start 2024-03-22T04:17:46.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- start start 2024-03-22T04:17:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c767f50 0x55a10c76eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 0x55a10c76e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08e42041d0 con 0x55a10c7b3f80 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4203230 con 0x55a10c767f50 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f08e415df60 con 0x55a10c7b48a0 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44584/0 (socket says 172.21.15.86:44584) 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 -- 172.21.15.86:0/549348201 learned_addr learned my addr 172.21.15.86:0/549348201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e8e00640 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 0x55a10c76e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3e00640 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c767f50 0x55a10c76eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 -- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 msgr2=0x55a10c76e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 0x55a10c76e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 -- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c767f50 msgr2=0x55a10c76eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c767f50 0x55a10c76eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 -- 172.21.15.86:0/549348201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10c58e3a0 con 0x55a10c7b3f80 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3e00640 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c767f50 0x55a10c76eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08e3400640 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7ca680 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f08d0055300 tx=0x7f08d001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d6e8639f0dd421 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08cfe00640 1 -- 172.21.15.86:0/549348201 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d007f070 con 0x55a10c7b3f80 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08cfe00640 1 -- 172.21.15.86:0/549348201 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d0051c70 con 0x55a10c7b3f80 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08cfe00640 1 -- 172.21.15.86:0/549348201 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0083d90 con 0x55a10c7b3f80 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- 172.21.15.86:0/549348201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 msgr2=0x55a10c7ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7ca680 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f08d0055300 tx=0x7f08d001e040 comp rx=0 tx=0).stop 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- 172.21.15.86:0/549348201 shutdown_connections 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c7b48a0 0x55a10c76e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c7ca680 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 --2- 172.21.15.86:0/549348201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c767f50 0x55a10c76eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- 172.21.15.86:0/549348201 >> 172.21.15.86:0/549348201 conn(0x55a10c651990 msgr2=0x55a10c7700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- 172.21.15.86:0/549348201 shutdown_connections 2024-03-22T04:17:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- 172.21.15.86:0/549348201 wait complete. 2024-03-22T04:17:46.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 -- start start 2024-03-22T04:17:46.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.249+0000 7f08ee22b000 1 Processor -- start 2024-03-22T04:17:46.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 0x55a10c82bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c82c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c829770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a10c674810 con 0x55a10c7b3f80 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08e415e5b0 con 0x55a10c7b48a0 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55a10c5b7f50 con 0x55a10c767f50 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 10 client.0 _ll_register_callbacks cb 0x7f08dc007b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c82c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c82c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44592/0 (socket says 172.21.15.86:44592) 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 -- 172.21.15.86:0/1914154881 learned_addr learned my addr 172.21.15.86:0/1914154881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 0x55a10c82bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e8e00640 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c829770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 -- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 msgr2=0x55a10c82bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 0x55a10c82bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 -- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 msgr2=0x55a10c829770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c829770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 -- 172.21.15.86:0/1914154881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a10c53e380 con 0x55a10c7b3f80 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e8e00640 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c829770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3400640 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c82c280 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f08d000ce20 tx=0x7f08d0083640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea60400e809db23d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 1 -- 172.21.15.86:0/1914154881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d007f040 con 0x55a10c7b3f80 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 1 -- 172.21.15.86:0/1914154881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f08d0013db0 con 0x55a10c7b3f80 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- 172.21.15.86:0/1914154881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d0051c70 con 0x55a10c7b3f80 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 1 -- 172.21.15.86:0/1914154881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f08d00850b0 con 0x55a10c7b3f80 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 10 client.7821 fetch_fsmap learned FSMap version 71 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- 172.21.15.86:0/1914154881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55a10c63a460 con 0x55a10c7b3f80 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 1 -- 172.21.15.86:0/1914154881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d0051c70 con 0x55a10c7b3f80 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 1 -- 172.21.15.86:0/1914154881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f08d0085250 con 0x55a10c7b3f80 2024-03-22T04:17:46.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 10 client.7821 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 10 client.7821 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 20 client.7821 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55a10c82a130 0x55a10c82a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 4 client.7821 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- 172.21.15.86:0/1914154881 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a10c558160 con 0x55a10c82a130 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55a10c82a130 0x55a10c82a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 10 client.7821 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08e3e00640 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55a10c82a130 0x55a10c82a510 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=343e8ca01c0b3f23 server_cookie=4510136b32ef5 in_seq=0 out_seq=0 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 1 -- 172.21.15.86:0/1914154881 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a10c558160 con 0x55a10c82a130 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 10 client.7821 handle_command_reply: tid=0 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 client.7821 shutdown 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- 172.21.15.86:0/1914154881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 msgr2=0x55a10c82c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c82c280 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f08d000ce20 tx=0x7f08d0083640 comp rx=0 tx=0).stop 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- 172.21.15.86:0/1914154881 shutdown_connections 2024-03-22T04:17:46.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55a10c82a130 0x55a10c82a510 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a10c7b48a0 0x55a10c829770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a10c7b3f80 0x55a10c82c280 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 --2- 172.21.15.86:0/1914154881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a10c767f50 0x55a10c82bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- 172.21.15.86:0/1914154881 >> 172.21.15.86:0/1914154881 conn(0x55a10c651990 msgr2=0x55a10c7b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08cd600640 0 client.7821 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- 172.21.15.86:0/1914154881 shutdown_connections 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 1 -- 172.21.15.86:0/1914154881 wait complete. 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.253+0000 7f08ee22b000 20 client.7821 trim_cache size 0 max 16384 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:17:46.260 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:17:46.261 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:17:46.261 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:17:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 -- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f08c4067b20 msgr2=0x7f08c4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f08c4067b20 0x7f08c4069fe0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f08d40af000 tx=0x7f08d40a7070 comp rx=0 tx=0).stop 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 -- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e4072650 msgr2=0x7f08e40ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e4072650 0x7f08e40ca3b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f08d000c460 tx=0x7f08d000c920 comp rx=0 tx=0).stop 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 -- 172.21.15.86:0/1960831957 shutdown_connections 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f08c4067b20 0x7f08c4069fe0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08e4072ad0 0x7f08e40ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e4072650 0x7f08e40ca3b0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 --2- 172.21.15.86:0/1960831957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08e406e7e0 0x7f08e4154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 -- 172.21.15.86:0/1960831957 >> 172.21.15.86:0/1960831957 conn(0x7f08e40cc940 msgr2=0x7f08e40b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 -- 172.21.15.86:0/1960831957 shutdown_connections 2024-03-22T04:17:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.257+0000 7f08eae00640 1 -- 172.21.15.86:0/1960831957 wait complete. 2024-03-22T04:17:46.275 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:17:46.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:17:46.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.409+0000 7f1873a00640 1 Processor -- start 2024-03-22T04:17:46.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.409+0000 7f1873a00640 1 -- start start 2024-03-22T04:17:46.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.409+0000 7f1873a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f186c15b3f0 0x7f186c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.409+0000 7f1873a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f186c0cc3e0 0x7f186c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.409+0000 7f1873a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 0x7f186c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.409+0000 7f1873a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f186c0716d0 con 0x7f186c15b3f0 2024-03-22T04:17:46.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.409+0000 7f1873a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f186c06d990 con 0x7f186c0cc3e0 2024-03-22T04:17:46.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.409+0000 7f1873a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f186c071920 con 0x7f186c0b30f0 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f186c15b3f0 0x7f186c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f186c15b3f0 0x7f186c0729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44598/0 (socket says 172.21.15.86:44598) 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 -- 172.21.15.86:0/2741140699 learned_addr learned my addr 172.21.15.86:0/2741140699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871a00640 1 --2- 172.21.15.86:0/2741140699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 0x7f186c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1863e00640 1 --2- 172.21.15.86:0/2741140699 >> v2:172.21.15.196:3300/0 conn(0x7f186c0cc3e0 0x7f186c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 -- 172.21.15.86:0/2741140699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 msgr2=0x7f186c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 --2- 172.21.15.86:0/2741140699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 0x7f186c0ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 -- 172.21.15.86:0/2741140699 >> v2:172.21.15.196:3300/0 conn(0x7f186c0cc3e0 msgr2=0x7f186c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 --2- 172.21.15.86:0/2741140699 >> v2:172.21.15.196:3300/0 conn(0x7f186c0cc3e0 0x7f186c072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 -- 172.21.15.86:0/2741140699 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c071c00 con 0x7f186c15b3f0 2024-03-22T04:17:46.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1863e00640 1 --2- 172.21.15.86:0/2741140699 >> v2:172.21.15.196:3300/0 conn(0x7f186c0cc3e0 0x7f186c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:46.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f1871000640 1 --2- 172.21.15.86:0/2741140699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c0729c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f186c070ed0 tx=0x7f185c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e360e2e64e34c2d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f186be00640 1 -- 172.21.15.86:0/2741140699 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c0b0070 con 0x7f186c15b3f0 2024-03-22T04:17:46.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f186be00640 1 -- 172.21.15.86:0/2741140699 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185c0ab600 con 0x7f186c15b3f0 2024-03-22T04:17:46.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.413+0000 7f186be00640 1 -- 172.21.15.86:0/2741140699 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c0ab9d0 con 0x7f186c15b3f0 2024-03-22T04:17:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- 172.21.15.86:0/2741140699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 msgr2=0x7f186c0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 --2- 172.21.15.86:0/2741140699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c0729c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f186c070ed0 tx=0x7f185c01c490 comp rx=0 tx=0).stop 2024-03-22T04:17:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- 172.21.15.86:0/2741140699 shutdown_connections 2024-03-22T04:17:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 --2- 172.21.15.86:0/2741140699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 0x7f186c0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 --2- 172.21.15.86:0/2741140699 >> v2:172.21.15.196:3300/0 conn(0x7f186c0cc3e0 0x7f186c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 --2- 172.21.15.86:0/2741140699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c0729c0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- 172.21.15.86:0/2741140699 >> 172.21.15.86:0/2741140699 conn(0x7f186c0b1df0 msgr2=0x7f186c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- 172.21.15.86:0/2741140699 shutdown_connections 2024-03-22T04:17:46.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- 172.21.15.86:0/2741140699 wait complete. 2024-03-22T04:17:46.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 Processor -- start 2024-03-22T04:17:46.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- start start 2024-03-22T04:17:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 0x7f186c154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186c0cc3e0 0x7f186c145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f186c15f1e0 con 0x7f186c15b3f0 2024-03-22T04:17:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f186c15f060 con 0x7f186c0cc3e0 2024-03-22T04:17:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1873a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f186c15f360 con 0x7f186c0b30f0 2024-03-22T04:17:46.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44606/0 (socket says 172.21.15.86:44606) 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 -- 172.21.15.86:0/3684510203 learned_addr learned my addr 172.21.15.86:0/3684510203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1863e00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186c0cc3e0 0x7f186c145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871000640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 0x7f186c154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 -- 172.21.15.86:0/3684510203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 msgr2=0x7f186c154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 0x7f186c154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 -- 172.21.15.86:0/3684510203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186c0cc3e0 msgr2=0x7f186c145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186c0cc3e0 0x7f186c145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 -- 172.21.15.86:0/3684510203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c0736a0 con 0x7f186c15b3f0 2024-03-22T04:17:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.417+0000 7f1871a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c15d040 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f186400eed0 tx=0x7f186400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1869600640 1 -- 172.21.15.86:0/3684510203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864018020 con 0x7f186c15b3f0 2024-03-22T04:17:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1869600640 1 -- 172.21.15.86:0/3684510203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864014400 con 0x7f186c15b3f0 2024-03-22T04:17:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1869600640 1 -- 172.21.15.86:0/3684510203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640147f0 con 0x7f186c15b3f0 2024-03-22T04:17:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c154fd0 con 0x7f186c15b3f0 2024-03-22T04:17:46.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c15c560 con 0x7f186c15b3f0 2024-03-22T04:17:46.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c068650 con 0x7f186c15b3f0 2024-03-22T04:17:46.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1869600640 1 -- 172.21.15.86:0/3684510203 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f186401b020 con 0x7f186c15b3f0 2024-03-22T04:17:46.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1869600640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1848067b50 0x7f184806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1869600640 1 -- 172.21.15.86:0/3684510203 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f1864086760 con 0x7f186c15b3f0 2024-03-22T04:17:46.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.421+0000 7f1871000640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1848067b50 0x7f184806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.425+0000 7f1871000640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1848067b50 0x7f184806a010 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f185c0ad000 tx=0x7f185c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.425+0000 7f1869600640 1 -- 172.21.15.86:0/3684510203 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1864022070 con 0x7f186c15b3f0 2024-03-22T04:17:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f186c159db0 con 0x7f186c15b3f0 2024-03-22T04:17:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1869600640 1 -- 172.21.15.86:0/3684510203 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5186 (secure 0 0 0) 0x7f186c154fd0 con 0x7f186c15b3f0 2024-03-22T04:17:46.613 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:46.613 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:19.957152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76604":{"gid":76604,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6835/1851339627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1851339627},{"type":"v1","addr":"172.21.15.196:6835","nonce":1851339627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76604,"qdb_cluster":[76604]},"id":6}]} 2024-03-22T04:17:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 71 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1848067b50 msgr2=0x7f184806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1848067b50 0x7f184806a010 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f185c0ad000 tx=0x7f185c0a6000 comp rx=0 tx=0).stop 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 msgr2=0x7f186c15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c15d040 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f186400eed0 tx=0x7f186400b480 comp rx=0 tx=0).stop 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 shutdown_connections 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1848067b50 0x7f184806a010 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f186c0cc3e0 0x7f186c145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c15b3f0 0x7f186c15d040 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 --2- 172.21.15.86:0/3684510203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f186c0b30f0 0x7f186c154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 >> 172.21.15.86:0/3684510203 conn(0x7f186c0b1df0 msgr2=0x7f186c0b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 shutdown_connections 2024-03-22T04:17:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.613+0000 7f1873a00640 1 -- 172.21.15.86:0/3684510203 wait complete. 2024-03-22T04:17:46.635 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-03-22T04:17:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.773+0000 7fd67f800640 1 Processor -- start 2024-03-22T04:17:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.773+0000 7fd67f800640 1 -- start start 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd67815e7e0 0x7fd6780d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6780b4710 0x7fd67806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67f800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd678070390 0x7fd67809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd6780b4b70 con 0x7fd6780b4710 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67f800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd67806f970 con 0x7fd678070390 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd6780d15f0 con 0x7fd67815e7e0 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6780b4710 0x7fd67806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6780b4710 0x7fd67806fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44614/0 (socket says 172.21.15.86:44614) 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 -- 172.21.15.86:0/1910771326 learned_addr learned my addr 172.21.15.86:0/1910771326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67ce00640 1 --2- 172.21.15.86:0/1910771326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd67815e7e0 0x7fd6780d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67d800640 1 --2- 172.21.15.86:0/1910771326 >> v2:172.21.15.196:3300/0 conn(0x7fd678070390 0x7fd67809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 -- 172.21.15.86:0/1910771326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd67815e7e0 msgr2=0x7fd6780d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 --2- 172.21.15.86:0/1910771326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd67815e7e0 0x7fd6780d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 -- 172.21.15.86:0/1910771326 >> v2:172.21.15.196:3300/0 conn(0x7fd678070390 msgr2=0x7fd67809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 --2- 172.21.15.86:0/1910771326 >> v2:172.21.15.196:3300/0 conn(0x7fd678070390 0x7fd67809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 -- 172.21.15.86:0/1910771326 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd678070820 con 0x7fd6780b4710 2024-03-22T04:17:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd67d800640 1 --2- 172.21.15.86:0/1910771326 >> v2:172.21.15.196:3300/0 conn(0x7fd678070390 0x7fd67809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.777+0000 7fd677e00640 1 --2- 172.21.15.86:0/1910771326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6780b4710 0x7fd67806fe50 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd6780cc650 tx=0x7fd66401c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86df55274523e829 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd677400640 1 -- 172.21.15.86:0/1910771326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6640b0070 con 0x7fd6780b4710 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd677400640 1 -- 172.21.15.86:0/1910771326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6640ab600 con 0x7fd6780b4710 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd677400640 1 -- 172.21.15.86:0/1910771326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6640ab9d0 con 0x7fd6780b4710 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- 172.21.15.86:0/1910771326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6780b4710 msgr2=0x7fd67806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 --2- 172.21.15.86:0/1910771326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6780b4710 0x7fd67806fe50 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd6780cc650 tx=0x7fd66401c110 comp rx=0 tx=0).stop 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- 172.21.15.86:0/1910771326 shutdown_connections 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 --2- 172.21.15.86:0/1910771326 >> v2:172.21.15.196:3300/0 conn(0x7fd678070390 0x7fd67809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 --2- 172.21.15.86:0/1910771326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6780b4710 0x7fd67806fe50 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 --2- 172.21.15.86:0/1910771326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd67815e7e0 0x7fd6780d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- 172.21.15.86:0/1910771326 >> 172.21.15.86:0/1910771326 conn(0x7fd6780b3bf0 msgr2=0x7fd6780b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- 172.21.15.86:0/1910771326 shutdown_connections 2024-03-22T04:17:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- 172.21.15.86:0/1910771326 wait complete. 2024-03-22T04:17:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 Processor -- start 2024-03-22T04:17:46.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- start start 2024-03-22T04:17:46.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd678070390 0x7fd678154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6780b4710 0x7fd67815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd67815e7e0 0x7fd6780c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780cd360 con 0x7fd67815e7e0 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780cce40 con 0x7fd6780b4710 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd6780cd4e0 con 0x7fd678070390 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd67815e7e0 0x7fd6780c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd67815e7e0 0x7fd6780c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44616/0 (socket says 172.21.15.86:44616) 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67d800640 1 -- 172.21.15.86:0/4009797894 learned_addr learned my addr 172.21.15.86:0/4009797894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd67ce00640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd678070390 0x7fd678154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.781+0000 7fd677e00640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6780b4710 0x7fd67815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67d800640 1 -- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd678070390 msgr2=0x7fd678154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67d800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd678070390 0x7fd678154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67d800640 1 -- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6780b4710 msgr2=0x7fd67815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67d800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6780b4710 0x7fd67815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67d800640 1 -- 172.21.15.86:0/4009797894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6780ce5a0 con 0x7fd67815e7e0 2024-03-22T04:17:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd677e00640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6780b4710 0x7fd67815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67d800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd67815e7e0 0x7fd6780c5c10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd670009fa0 tx=0x7fd67000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd674c00640 1 -- 172.21.15.86:0/4009797894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67001a070 con 0x7fd67815e7e0 2024-03-22T04:17:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd674c00640 1 -- 172.21.15.86:0/4009797894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670003bc0 con 0x7fd67815e7e0 2024-03-22T04:17:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67f800640 1 -- 172.21.15.86:0/4009797894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6780b55a0 con 0x7fd67815e7e0 2024-03-22T04:17:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd674c00640 1 -- 172.21.15.86:0/4009797894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670004030 con 0x7fd67815e7e0 2024-03-22T04:17:46.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67f800640 1 -- 172.21.15.86:0/4009797894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6780b5b60 con 0x7fd67815e7e0 2024-03-22T04:17:46.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd674c00640 1 -- 172.21.15.86:0/4009797894 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd67001d070 con 0x7fd67815e7e0 2024-03-22T04:17:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd674c00640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd658067b20 0x7fd658069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd674c00640 1 -- 172.21.15.86:0/4009797894 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fd670086430 con 0x7fd67815e7e0 2024-03-22T04:17:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.785+0000 7fd67ce00640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd658067b20 0x7fd658069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 Processor -- start 2024-03-22T04:17:46.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- start start 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67ce00640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd658067b20 0x7fd658069fe0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd66c007940 tx=0x7fd66c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a7782f0 0x55f60a7786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 0x55f60a72e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a72ed60 0x55f60a72f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f60a57dd80 con 0x55f60a72ed60 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f60a51ace0 con 0x55f60a778c10 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f60a579d30 con 0x55f60a7782f0 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a72ed60 0x55f60a72f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a72ed60 0x55f60a72f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44630/0 (socket says 172.21.15.86:44630) 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 -- 172.21.15.86:0/1824034652 learned_addr learned my addr 172.21.15.86:0/1824034652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67ce00640 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a7782f0 0x55f60a7786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd677e00640 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 0x55f60a72e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 -- 172.21.15.86:0/1824034652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a7782f0 msgr2=0x55f60a7786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a7782f0 0x55f60a7786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 -- 172.21.15.86:0/1824034652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 msgr2=0x55f60a72e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 0x55f60a72e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 -- 172.21.15.86:0/1824034652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f60a558340 con 0x55f60a72ed60 2024-03-22T04:17:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd67d800640 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a72ed60 0x55f60a72f140 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fd670007fd0 tx=0x7fd670056420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc48fabe4594d34 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd669600640 1 -- 172.21.15.86:0/1824034652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670054820 con 0x55f60a72ed60 2024-03-22T04:17:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd669600640 1 -- 172.21.15.86:0/1824034652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670052b90 con 0x55f60a72ed60 2024-03-22T04:17:46.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd669600640 1 -- 172.21.15.86:0/1824034652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67005a170 con 0x55f60a72ed60 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- 172.21.15.86:0/1824034652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a72ed60 msgr2=0x55f60a72f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a72ed60 0x55f60a72f140 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fd670007fd0 tx=0x7fd670056420 comp rx=0 tx=0).stop 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- 172.21.15.86:0/1824034652 shutdown_connections 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 0x55f60a72e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a72ed60 0x55f60a72f140 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 --2- 172.21.15.86:0/1824034652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a7782f0 0x55f60a7786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- 172.21.15.86:0/1824034652 >> 172.21.15.86:0/1824034652 conn(0x55f60a72ad10 msgr2=0x55f60a72d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- 172.21.15.86:0/1824034652 shutdown_connections 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- 172.21.15.86:0/1824034652 wait complete. 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 -- start start 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.789+0000 7fd682d46000 1 Processor -- start 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 0x55f60a7338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a733df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 0x55f60a779200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f60a53b4e0 con 0x55f60a7782f0 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f60a505550 con 0x55f60a778c10 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f60a5fe3b0 con 0x55f60a72ed60 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 10 client.0 _ll_register_callbacks cb 0x55f60a719c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a733df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a733df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44642/0 (socket says 172.21.15.86:44642) 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 -- 172.21.15.86:0/2879541067 learned_addr learned my addr 172.21.15.86:0/2879541067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd67ce00640 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 0x55f60a7338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd67d800640 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 0x55f60a779200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 -- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 msgr2=0x55f60a7338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 0x55f60a7338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 -- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 msgr2=0x55f60a779200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 0x55f60a779200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 -- 172.21.15.86:0/2879541067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60a5c25e0 con 0x55f60a7782f0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd677e00640 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a733df0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd664008010 tx=0x7fd66401a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd28e1f419743d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd65e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd65e000640 1 -- 172.21.15.86:0/2879541067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6640b0040 con 0x55f60a7782f0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd65e000640 1 -- 172.21.15.86:0/2879541067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fd6640c0d80 con 0x55f60a7782f0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 -- 172.21.15.86:0/2879541067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd670052b90 con 0x55f60a7782f0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd65e000640 1 -- 172.21.15.86:0/2879541067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fd6640abcb0 con 0x55f60a7782f0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 10 client.7839 fetch_fsmap learned FSMap version 71 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 -- 172.21.15.86:0/2879541067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55f60a5395a0 con 0x55f60a7782f0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd65e000640 1 -- 172.21.15.86:0/2879541067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670052b90 con 0x55f60a7782f0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd65e000640 1 -- 172.21.15.86:0/2879541067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7fd6640c7050 con 0x55f60a7782f0 2024-03-22T04:17:46.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 10 client.7839 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:17:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 10 client.7839 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:17:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 20 client.7839 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55f60a779c10 0x55f60a789a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 4 client.7839 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:17:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd682d46000 1 -- 172.21.15.86:0/2879541067 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f60a518e40 con 0x55f60a779c10 2024-03-22T04:17:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd67ce00640 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55f60a779c10 0x55f60a789a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd65e000640 10 client.7839 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:46.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.793+0000 7fd67ce00640 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55f60a779c10 0x55f60a789a00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca37d95f0887b93 server_cookie=622b6f351a77f143 in_seq=0 out_seq=0 2024-03-22T04:17:46.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd65e000640 1 -- 172.21.15.86:0/2879541067 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f60a518e40 con 0x55f60a779c10 2024-03-22T04:17:46.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd65e000640 10 client.7839 handle_command_reply: tid=0 2024-03-22T04:17:46.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 client.7839 shutdown 2024-03-22T04:17:46.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 -- 172.21.15.86:0/2879541067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 msgr2=0x55f60a733df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a733df0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd664008010 tx=0x7fd66401a950 comp rx=0 tx=0).stop 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 -- 172.21.15.86:0/2879541067 shutdown_connections 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55f60a779c10 0x55f60a789a00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a778c10 0x55f60a779200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a733df0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd65e000640 0 client.7839 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 --2- 172.21.15.86:0/2879541067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 0x55f60a7338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 -- 172.21.15.86:0/2879541067 >> 172.21.15.86:0/2879541067 conn(0x55f60a72ad10 msgr2=0x55f60a72d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 -- 172.21.15.86:0/2879541067 shutdown_connections 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 1 -- 172.21.15.86:0/2879541067 wait complete. 2024-03-22T04:17:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.797+0000 7fd682d46000 20 client.7839 trim_cache size 0 max 16384 2024-03-22T04:17:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd682d46000 1 Processor -- start 2024-03-22T04:17:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd682d46000 1 -- start start 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd682d46000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 0x55f60a733c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd682d46000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a7782f0 0x55f60a72d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd682d46000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a778c10 0x55f60a72d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd682d46000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd678203460 con 0x55f60a778c10 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd682d46000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780b7660 con 0x55f60a7782f0 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd682d46000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd678202dc0 con 0x55f60a72ed60 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a778c10 0x55f60a72d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a778c10 0x55f60a72d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44644/0 (socket says 172.21.15.86:44644) 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 -- 172.21.15.86:0/2514129610 learned_addr learned my addr 172.21.15.86:0/2514129610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67ce00640 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 0x55f60a733c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd677e00640 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a7782f0 0x55f60a72d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 -- 172.21.15.86:0/2514129610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 msgr2=0x55f60a733c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 0x55f60a733c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 -- 172.21.15.86:0/2514129610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a7782f0 msgr2=0x55f60a72d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a7782f0 0x55f60a72d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 -- 172.21.15.86:0/2514129610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f60a539a50 con 0x55f60a778c10 2024-03-22T04:17:46.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.809+0000 7fd67d800640 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a778c10 0x55f60a72d640 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd670056460 tx=0x7fd670086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8e16cd8ac17653 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd668c00640 1 -- 172.21.15.86:0/2514129610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67006e040 con 0x55f60a778c10 2024-03-22T04:17:46.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd668c00640 1 -- 172.21.15.86:0/2514129610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6700548e0 con 0x55f60a778c10 2024-03-22T04:17:46.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd668c00640 1 -- 172.21.15.86:0/2514129610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700860c0 con 0x55f60a778c10 2024-03-22T04:17:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- 172.21.15.86:0/2514129610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a778c10 msgr2=0x55f60a72d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a778c10 0x55f60a72d640 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd670056460 tx=0x7fd670086050 comp rx=0 tx=0).stop 2024-03-22T04:17:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- 172.21.15.86:0/2514129610 shutdown_connections 2024-03-22T04:17:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a7782f0 0x55f60a72d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a778c10 0x55f60a72d640 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 --2- 172.21.15.86:0/2514129610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a72ed60 0x55f60a733c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- 172.21.15.86:0/2514129610 >> 172.21.15.86:0/2514129610 conn(0x55f60a72ad10 msgr2=0x55f60a7631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- 172.21.15.86:0/2514129610 shutdown_connections 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- 172.21.15.86:0/2514129610 wait complete. 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- start start 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 Processor -- start 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a72ed60 0x55f60a7e7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a789d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a778c10 0x55f60a78a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f60a72ab90 con 0x55f60a7782f0 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f60a57d090 con 0x55f60a72ed60 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f60a524230 con 0x55f60a778c10 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 10 client.0 _ll_register_callbacks cb 0x7fd6640ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a789d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a789d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44654/0 (socket says 172.21.15.86:44654) 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 -- 172.21.15.86:0/3682689598 learned_addr learned my addr 172.21.15.86:0/3682689598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd67ce00640 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a72ed60 0x55f60a7e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd67d800640 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a778c10 0x55f60a78a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 -- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a778c10 msgr2=0x55f60a78a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a778c10 0x55f60a78a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 -- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a72ed60 msgr2=0x55f60a7e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a72ed60 0x55f60a7e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 -- 172.21.15.86:0/3682689598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60a5fe840 con 0x55f60a7782f0 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd677e00640 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a789d10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd6640b5430 tx=0x7fd664008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9c39c6acbd5cc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd65e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd65e000640 1 -- 172.21.15.86:0/3682689598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6640b0040 con 0x55f60a7782f0 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd65e000640 1 -- 172.21.15.86:0/3682689598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fd664004450 con 0x55f60a7782f0 2024-03-22T04:17:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- 172.21.15.86:0/3682689598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f60a58b2c0 con 0x55f60a7782f0 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd65e000640 1 -- 172.21.15.86:0/3682689598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fd6640da050 con 0x55f60a7782f0 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 10 client.7845 fetch_fsmap learned FSMap version 71 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.813+0000 7fd682d46000 1 -- 172.21.15.86:0/3682689598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55f60a5243e0 con 0x55f60a7782f0 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd65e000640 1 -- 172.21.15.86:0/3682689598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f60a58b2c0 con 0x55f60a7782f0 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd65e000640 1 -- 172.21.15.86:0/3682689598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7fd6640dd020 con 0x55f60a7782f0 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 10 client.7845 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 10 client.7845 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 20 client.7845 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55f60a78a790 0x55f60a78ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 4 client.7845 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 -- 172.21.15.86:0/3682689598 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55f60a585610 con 0x55f60a78a790 2024-03-22T04:17:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67ce00640 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55f60a78a790 0x55f60a78ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd65e000640 10 client.7845 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67ce00640 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55f60a78a790 0x55f60a78ab70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c68f40b2b3a35af server_cookie=227bdb2258f0647f in_seq=0 out_seq=0 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd65e000640 1 -- 172.21.15.86:0/3682689598 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+419 (crc 0 0 0) 0x55f60a585610 con 0x55f60a78a790 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd65e000640 10 client.7845 handle_command_reply: tid=0 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 client.7845 shutdown 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 -- 172.21.15.86:0/3682689598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 msgr2=0x55f60a789d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a789d10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd6640b5430 tx=0x7fd664008040 comp rx=0 tx=0).stop 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 -- 172.21.15.86:0/3682689598 shutdown_connections 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55f60a78a790 0x55f60a78ab70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f60a778c10 0x55f60a78a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f60a7782f0 0x55f60a789d10 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 --2- 172.21.15.86:0/3682689598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f60a72ed60 0x55f60a7e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd65e000640 0 client.7845 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 -- 172.21.15.86:0/3682689598 >> 172.21.15.86:0/3682689598 conn(0x55f60a72ad10 msgr2=0x55f60a763a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 -- 172.21.15.86:0/3682689598 shutdown_connections 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 1 -- 172.21.15.86:0/3682689598 wait complete. 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd682d46000 20 client.7845 trim_cache size 0 max 16384 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 237731977, 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627776, 2024-03-22T04:17:46.822 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir" 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "dir_frag", 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 2167748447, 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627776, 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "frag": "*", 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir" 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 2646213554, 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/testdir/hardlink" 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:17:46.823 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:17:46.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 -- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd658067b20 msgr2=0x7fd658069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd658067b20 0x7fd658069fe0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd66c007940 tx=0x7fd66c008040 comp rx=0 tx=0).stop 2024-03-22T04:17:46.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 -- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd67815e7e0 msgr2=0x7fd6780c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd67815e7e0 0x7fd6780c5c10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd670009fa0 tx=0x7fd67000d560 comp rx=0 tx=0).stop 2024-03-22T04:17:46.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 -- 172.21.15.86:0/4009797894 shutdown_connections 2024-03-22T04:17:46.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd658067b20 0x7fd658069fe0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6780b4710 0x7fd67815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd67815e7e0 0x7fd6780c5c10 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 --2- 172.21.15.86:0/4009797894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd678070390 0x7fd678154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 -- 172.21.15.86:0/4009797894 >> 172.21.15.86:0/4009797894 conn(0x7fd6780b3bf0 msgr2=0x7fd678070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 -- 172.21.15.86:0/4009797894 shutdown_connections 2024-03-22T04:17:46.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.817+0000 7fd67f800640 1 -- 172.21.15.86:0/4009797894 wait complete. 2024-03-22T04:17:46.837 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-22T04:17:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa925400640 1 Processor -- start 2024-03-22T04:17:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa925400640 1 -- start start 2024-03-22T04:17:46.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa925400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa920145aa0 0x7fa9200acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa925400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa92006e0e0 0x7fa9200b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa925400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa920145e80 con 0x7fa9200ad4a0 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa925400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa92006e700 con 0x7fa92006e0e0 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa925400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa92006e9e0 con 0x7fa920145aa0 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa91f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa92006e0e0 0x7fa9200b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa915e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa9200ad4a0 0x7fa92006d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54730/0 (socket says 172.21.15.86:54730) 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa915e00640 1 -- 172.21.15.86:0/278235862 learned_addr learned my addr 172.21.15.86:0/278235862 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa91e800640 1 --2- 172.21.15.86:0/278235862 >> v2:172.21.15.196:3300/0 conn(0x7fa920145aa0 0x7fa9200acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa91f200640 1 -- 172.21.15.86:0/278235862 >> v2:172.21.15.196:3300/0 conn(0x7fa920145aa0 msgr2=0x7fa9200acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa91f200640 1 --2- 172.21.15.86:0/278235862 >> v2:172.21.15.196:3300/0 conn(0x7fa920145aa0 0x7fa9200acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa91f200640 1 -- 172.21.15.86:0/278235862 >> v1:172.21.15.86:6789/0 conn(0x7fa9200ad4a0 legacy=0x7fa92006d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa91f200640 1 -- 172.21.15.86:0/278235862 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92006ecc0 con 0x7fa92006e0e0 2024-03-22T04:17:46.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.973+0000 7fa91de00640 1 -- 172.21.15.86:0/278235862 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522051286 0 0) 0x7fa920145e80 con 0x7fa9200ad4a0 2024-03-22T04:17:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa91f200640 1 --2- 172.21.15.86:0/278235862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92006e0e0 0x7fa9200b3360 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa9200d1060 tx=0x7fa91801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a1b8172ff4a703 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa91de00640 1 -- 172.21.15.86:0/278235862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180b2070 con 0x7fa92006e0e0 2024-03-22T04:17:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa91de00640 1 -- 172.21.15.86:0/278235862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9180a57f0 con 0x7fa92006e0e0 2024-03-22T04:17:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa91de00640 1 -- 172.21.15.86:0/278235862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180a5bc0 con 0x7fa92006e0e0 2024-03-22T04:17:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa925400640 1 -- 172.21.15.86:0/278235862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92006e0e0 msgr2=0x7fa9200b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa925400640 1 --2- 172.21.15.86:0/278235862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92006e0e0 0x7fa9200b3360 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa9200d1060 tx=0x7fa91801c930 comp rx=0 tx=0).stop 2024-03-22T04:17:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa925400640 1 -- 172.21.15.86:0/278235862 shutdown_connections 2024-03-22T04:17:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa925400640 1 --2- 172.21.15.86:0/278235862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa92006e0e0 0x7fa9200b3360 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa925400640 1 --2- 172.21.15.86:0/278235862 >> v2:172.21.15.196:3300/0 conn(0x7fa920145aa0 0x7fa9200acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa925400640 1 -- 172.21.15.86:0/278235862 >> 172.21.15.86:0/278235862 conn(0x7fa9200b1c80 msgr2=0x7fa9200b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa925400640 1 -- 172.21.15.86:0/278235862 shutdown_connections 2024-03-22T04:17:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.977+0000 7fa925400640 1 -- 172.21.15.86:0/278235862 wait complete. 2024-03-22T04:17:46.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 Processor -- start 2024-03-22T04:17:46.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 -- start start 2024-03-22T04:17:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92006e0e0 0x7fa92015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9200ad4a0 0x7fa920072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa920145aa0 0x7fa920140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9200b4480 con 0x7fa9200ad4a0 2024-03-22T04:17:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9200b4300 con 0x7fa92006e0e0 2024-03-22T04:17:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa9200b4600 con 0x7fa920145aa0 2024-03-22T04:17:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9200ad4a0 0x7fa920072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9200ad4a0 0x7fa920072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44670/0 (socket says 172.21.15.86:44670) 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 -- 172.21.15.86:0/1377636874 learned_addr learned my addr 172.21.15.86:0/1377636874 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa91e800640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92006e0e0 0x7fa92015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa91f200640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa920145aa0 0x7fa920140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 -- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa920145aa0 msgr2=0x7fa920140030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa920145aa0 0x7fa920140030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 -- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92006e0e0 msgr2=0x7fa92015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92006e0e0 0x7fa92015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 -- 172.21.15.86:0/1377636874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa920071830 con 0x7fa9200ad4a0 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa91f200640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa920145aa0 0x7fa920140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa915e00640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9200ad4a0 0x7fa920072900 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c480 tx=0x7fa90800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa917400640 1 -- 172.21.15.86:0/1377636874 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908018070 con 0x7fa9200ad4a0 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa917400640 1 -- 172.21.15.86:0/1377636874 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa908007500 con 0x7fa9200ad4a0 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa917400640 1 -- 172.21.15.86:0/1377636874 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908004a30 con 0x7fa9200ad4a0 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa920071ad0 con 0x7fa9200ad4a0 2024-03-22T04:17:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa920072090 con 0x7fa9200ad4a0 2024-03-22T04:17:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.981+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa920074550 con 0x7fa9200ad4a0 2024-03-22T04:17:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.985+0000 7fa917400640 1 -- 172.21.15.86:0/1377636874 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa908004bd0 con 0x7fa9200ad4a0 2024-03-22T04:17:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.985+0000 7fa917400640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8fc067b20 0x7fa8fc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.985+0000 7fa917400640 1 -- 172.21.15.86:0/1377636874 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fa908085420 con 0x7fa9200ad4a0 2024-03-22T04:17:46.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.985+0000 7fa91e800640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8fc067b20 0x7fa8fc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:46.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.985+0000 7fa91e800640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8fc067b20 0x7fa8fc069fe0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa92013db30 tx=0x7fa910009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:46.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:46.989+0000 7fa917400640 1 -- 172.21.15.86:0/1377636874 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa908055510 con 0x7fa9200ad4a0 2024-03-22T04:17:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.173+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa92006f990 con 0x7fa9200ad4a0 2024-03-22T04:17:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.173+0000 7fa917400640 1 -- 172.21.15.86:0/1377636874 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fa9080595b0 con 0x7fa9200ad4a0 2024-03-22T04:17:47.175 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:17:47.175 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8fc067b20 msgr2=0x7fa8fc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8fc067b20 0x7fa8fc069fe0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa92013db30 tx=0x7fa910009190 comp rx=0 tx=0).stop 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9200ad4a0 msgr2=0x7fa920072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9200ad4a0 0x7fa920072900 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c480 tx=0x7fa90800c940 comp rx=0 tx=0).stop 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 shutdown_connections 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8fc067b20 0x7fa8fc069fe0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa920145aa0 0x7fa920140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9200ad4a0 0x7fa920072900 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 --2- 172.21.15.86:0/1377636874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92006e0e0 0x7fa92015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 >> 172.21.15.86:0/1377636874 conn(0x7fa9200b1c80 msgr2=0x7fa92009c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 shutdown_connections 2024-03-22T04:17:47.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.177+0000 7fa925400640 1 -- 172.21.15.86:0/1377636874 wait complete. 2024-03-22T04:17:47.197 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-22T04:17:47.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:47.197 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-22T04:17:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49bba00640 1 Processor -- start 2024-03-22T04:17:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49bba00640 1 -- start start 2024-03-22T04:17:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49bba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49b415c440 0x7f49b4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49bba00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f49b406c820 0x7f49b4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49bba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b406ef70 con 0x7f49b4072990 2024-03-22T04:17:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49bba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f49b406f250 con 0x7f49b415c440 2024-03-22T04:17:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49bba00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f49b406f530 con 0x7f49b406c820 2024-03-22T04:17:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49b9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49b415c440 0x7f49b4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49abe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f49b4072990 0x7f49b4160020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54738/0 (socket says 172.21.15.86:54738) 2024-03-22T04:17:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49abe00640 1 -- 172.21.15.86:0/1537667981 learned_addr learned my addr 172.21.15.86:0/1537667981 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49b9a00640 1 --2- 172.21.15.86:0/1537667981 >> v2:172.21.15.196:3300/0 conn(0x7f49b406c820 0x7f49b4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49b9000640 1 -- 172.21.15.86:0/1537667981 >> v2:172.21.15.196:3300/0 conn(0x7f49b406c820 msgr2=0x7f49b4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49b9000640 1 --2- 172.21.15.86:0/1537667981 >> v2:172.21.15.196:3300/0 conn(0x7f49b406c820 0x7f49b4072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49b9000640 1 -- 172.21.15.86:0/1537667981 >> v1:172.21.15.86:6789/0 conn(0x7f49b4072990 legacy=0x7f49b4160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49b9000640 1 -- 172.21.15.86:0/1537667981 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b406f810 con 0x7f49b415c440 2024-03-22T04:17:47.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.337+0000 7f49b3e00640 1 -- 172.21.15.86:0/1537667981 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194690238 0 0) 0x7f49b406ef70 con 0x7f49b4072990 2024-03-22T04:17:47.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49b9000640 1 --2- 172.21.15.86:0/1537667981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b415c440 0x7f49b4071df0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f49b40cd2b0 tx=0x7f49a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6e91f078a70d71 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49b3e00640 1 -- 172.21.15.86:0/1537667981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40b0070 con 0x7f49b415c440 2024-03-22T04:17:47.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49b3e00640 1 -- 172.21.15.86:0/1537667981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a40ab600 con 0x7f49b415c440 2024-03-22T04:17:47.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49b3e00640 1 -- 172.21.15.86:0/1537667981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40ab9d0 con 0x7f49b415c440 2024-03-22T04:17:47.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49bba00640 1 -- 172.21.15.86:0/1537667981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b415c440 msgr2=0x7f49b4071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49bba00640 1 --2- 172.21.15.86:0/1537667981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b415c440 0x7f49b4071df0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f49b40cd2b0 tx=0x7f49a401c3c0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49bba00640 1 -- 172.21.15.86:0/1537667981 shutdown_connections 2024-03-22T04:17:47.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49bba00640 1 --2- 172.21.15.86:0/1537667981 >> v2:172.21.15.196:3300/0 conn(0x7f49b406c820 0x7f49b4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49bba00640 1 --2- 172.21.15.86:0/1537667981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b415c440 0x7f49b4071df0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49bba00640 1 -- 172.21.15.86:0/1537667981 >> 172.21.15.86:0/1537667981 conn(0x7f49b40d07e0 msgr2=0x7f49b40d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49bba00640 1 -- 172.21.15.86:0/1537667981 shutdown_connections 2024-03-22T04:17:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.341+0000 7f49bba00640 1 -- 172.21.15.86:0/1537667981 wait complete. 2024-03-22T04:17:47.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 Processor -- start 2024-03-22T04:17:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 -- start start 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49b406c820 0x7f49b414c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b4072990 0x7f49b406b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49b415c440 0x7f49b406bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41694b0 con 0x7f49b4072990 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4169330 con 0x7f49b406c820 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f49b4169630 con 0x7f49b415c440 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b4072990 0x7f49b406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b4072990 0x7f49b406b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44682/0 (socket says 172.21.15.86:44682) 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 -- 172.21.15.86:0/1495112901 learned_addr learned my addr 172.21.15.86:0/1495112901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49b9000640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49b406c820 0x7f49b414c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49b9a00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49b415c440 0x7f49b406bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 -- 172.21.15.86:0/1495112901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49b415c440 msgr2=0x7f49b406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49b415c440 0x7f49b406bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 -- 172.21.15.86:0/1495112901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49b406c820 msgr2=0x7f49b414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49b406c820 0x7f49b414c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 -- 172.21.15.86:0/1495112901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b415d440 con 0x7f49b4072990 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49abe00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b4072990 0x7f49b406b810 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c4e0 tx=0x7f499c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49b1600640 1 -- 172.21.15.86:0/1495112901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c018070 con 0x7f49b4072990 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49b1600640 1 -- 172.21.15.86:0/1495112901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c007500 con 0x7f49b4072990 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 -- 172.21.15.86:0/1495112901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b415d6e0 con 0x7f49b4072990 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49b1600640 1 -- 172.21.15.86:0/1495112901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c004a30 con 0x7f49b4072990 2024-03-22T04:17:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.345+0000 7f49bba00640 1 -- 172.21.15.86:0/1495112901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b415cfb0 con 0x7f49b4072990 2024-03-22T04:17:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b1600640 1 -- 172.21.15.86:0/1495112901 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f499c01b070 con 0x7f49b4072990 2024-03-22T04:17:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b1600640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4990067b50 0x7f499006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b1600640 1 -- 172.21.15.86:0/1495112901 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f499c085430 con 0x7f49b4072990 2024-03-22T04:17:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b9000640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4990067b50 0x7f499006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49bee55000 1 Processor -- start 2024-03-22T04:17:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49bee55000 1 -- start start 2024-03-22T04:17:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b9000640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4990067b50 0x7f499006a010 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ad000 tx=0x7f49a40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49bee55000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x56274120b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49bee55000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411c1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49bee55000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49bee55000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627410a8830 con 0x5627411c1eb0 2024-03-22T04:17:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49bee55000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56274100fe10 con 0x56274120ae10 2024-03-22T04:17:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49bee55000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x562740face50 con 0x56274120b730 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44694/0 (socket says 172.21.15.86:44694) 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b9a00640 1 -- 172.21.15.86:0/2961231428 learned_addr learned my addr 172.21.15.86:0/2961231428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49abe00640 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.349+0000 7f49b9000640 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x56274120b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9a00640 1 -- 172.21.15.86:0/2961231428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 msgr2=0x5627411c1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9a00640 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411c1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9a00640 1 -- 172.21.15.86:0/2961231428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 msgr2=0x56274120b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9a00640 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x56274120b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9a00640 1 -- 172.21.15.86:0/2961231428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562740f95510 con 0x5627411c1eb0 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9a00640 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c2290 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f49ac011580 tx=0x7f49ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7737a6967056ed89 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49a9600640 1 -- 172.21.15.86:0/2961231428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac015900 con 0x5627411c1eb0 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49a9600640 1 -- 172.21.15.86:0/2961231428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac015aa0 con 0x5627411c1eb0 2024-03-22T04:17:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49a9600640 1 -- 172.21.15.86:0/2961231428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac01d400 con 0x5627411c1eb0 2024-03-22T04:17:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- 172.21.15.86:0/2961231428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 msgr2=0x5627411c2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c2290 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f49ac011580 tx=0x7f49ac009750 comp rx=0 tx=0).stop 2024-03-22T04:17:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- 172.21.15.86:0/2961231428 shutdown_connections 2024-03-22T04:17:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c2290 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 --2- 172.21.15.86:0/2961231428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x56274120b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- 172.21.15.86:0/2961231428 >> 172.21.15.86:0/2961231428 conn(0x5627411bde60 msgr2=0x5627411c0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- 172.21.15.86:0/2961231428 shutdown_connections 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- 172.21.15.86:0/2961231428 wait complete. 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- start start 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 Processor -- start 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627411c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562740fcd640 con 0x5627411c1eb0 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56274100bdc0 con 0x56274120ae10 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x562740fb80b0 con 0x56274120b730 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49bee55000 10 client.0 _ll_register_callbacks cb 0x5627411acd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44708/0 (socket says 172.21.15.86:44708) 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9000640 1 -- 172.21.15.86:0/361031547 learned_addr learned my addr 172.21.15.86:0/361031547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49abe00640 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627411c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9a00640 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9000640 1 -- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 msgr2=0x5627411cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9000640 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9000640 1 -- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 msgr2=0x5627411c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9000640 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627411c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.353+0000 7f49b9000640 1 -- 172.21.15.86:0/361031547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562740fcda40 con 0x5627411c1eb0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49b9000640 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c73c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ae040 tx=0x7f49a401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3711484d42ff4ab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 1 -- 172.21.15.86:0/361031547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40c9020 con 0x5627411c1eb0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 1 -- 172.21.15.86:0/361031547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f49a40b89c0 con 0x5627411c1eb0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 -- 172.21.15.86:0/361031547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49ac015aa0 con 0x5627411c1eb0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 1 -- 172.21.15.86:0/361031547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f49a40b0040 con 0x5627411c1eb0 2024-03-22T04:17:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 10 client.7869 fetch_fsmap learned FSMap version 71 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 -- 172.21.15.86:0/361031547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x562740fc8370 con 0x5627411c1eb0 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 1 -- 172.21.15.86:0/361031547 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac015aa0 con 0x5627411c1eb0 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 1 -- 172.21.15.86:0/361031547 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f49a40b8bc0 con 0x5627411c1eb0 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 10 client.7869 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 10 client.7869 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 20 client.7869 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x56274121a0b0 0x56274121c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 4 client.7869 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 -- 172.21.15.86:0/361031547 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562740faaed0 con 0x56274121a0b0 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49abe00640 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x56274121a0b0 0x56274121c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49abe00640 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x56274121a0b0 0x56274121c4a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9898a390392e50 server_cookie=d24ba4a2ff206851 in_seq=0 out_seq=0 2024-03-22T04:17:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 10 client.7869 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:47.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 1 -- 172.21.15.86:0/361031547 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562740faaed0 con 0x56274121a0b0 2024-03-22T04:17:47.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 10 client.7869 handle_command_reply: tid=0 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 client.7869 shutdown 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 -- 172.21.15.86:0/361031547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 msgr2=0x5627411c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c73c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ae040 tx=0x7f49a401ac00 comp rx=0 tx=0).stop 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 -- 172.21.15.86:0/361031547 shutdown_connections 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x56274121a0b0 0x56274121c4a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627411c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49a2000640 0 client.7869 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 --2- 172.21.15.86:0/361031547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x5627411c73c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 -- 172.21.15.86:0/361031547 >> 172.21.15.86:0/361031547 conn(0x5627411bde60 msgr2=0x5627411c0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 -- 172.21.15.86:0/361031547 shutdown_connections 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 1 -- 172.21.15.86:0/361031547 wait complete. 2024-03-22T04:17:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.357+0000 7f49bee55000 20 client.7869 trim_cache size 0 max 16384 2024-03-22T04:17:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.373+0000 7f49bee55000 1 Processor -- start 2024-03-22T04:17:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.373+0000 7f49bee55000 1 -- start start 2024-03-22T04:17:47.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x56274120a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627411c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49b42048a0 con 0x5627411c1eb0 2024-03-22T04:17:47.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4203880 con 0x56274120ae10 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f49b4202d10 con 0x56274120b730 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x56274120a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x56274120a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44714/0 (socket says 172.21.15.86:44714) 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 -- 172.21.15.86:0/255392917 learned_addr learned my addr 172.21.15.86:0/255392917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49abe00640 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627411c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9a00640 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 -- 172.21.15.86:0/255392917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 msgr2=0x5627411c17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411c17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 -- 172.21.15.86:0/255392917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 msgr2=0x5627411c1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627411c1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 -- 172.21.15.86:0/255392917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562740fad0a0 con 0x5627411c1eb0 2024-03-22T04:17:47.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49b9000640 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x56274120a090 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f49a4019ff0 tx=0x7f49a40c2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f9110f5bbc3d3b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49a8c00640 1 -- 172.21.15.86:0/255392917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40c9040 con 0x5627411c1eb0 2024-03-22T04:17:47.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49a8c00640 1 -- 172.21.15.86:0/255392917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a40b1040 con 0x5627411c1eb0 2024-03-22T04:17:47.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49a8c00640 1 -- 172.21.15.86:0/255392917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40d7880 con 0x5627411c1eb0 2024-03-22T04:17:47.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- 172.21.15.86:0/255392917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 msgr2=0x56274120a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x56274120a090 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f49a4019ff0 tx=0x7f49a40c2000 comp rx=0 tx=0).stop 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- 172.21.15.86:0/255392917 shutdown_connections 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627411c1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56274120b730 0x5627411c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- 172.21.15.86:0/255392917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627411c1eb0 0x56274120a090 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- 172.21.15.86:0/255392917 >> 172.21.15.86:0/255392917 conn(0x5627411bde60 msgr2=0x562741209140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- 172.21.15.86:0/255392917 shutdown_connections 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- 172.21.15.86:0/255392917 wait complete. 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- start start 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 Processor -- start 2024-03-22T04:17:47.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5627411c1eb0 0x56274121f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627412765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56274120b730 0x562741276b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627410913b0 con 0x56274120b730 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49b42031b0 con 0x56274120ae10 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56274100bdc0 con 0x5627411c1eb0 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.377+0000 7f49bee55000 10 client.0 _ll_register_callbacks cb 0x7f49a4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56274120b730 0x562741276b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56274120b730 0x562741276b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44718/0 (socket says 172.21.15.86:44718) 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 -- 172.21.15.86:0/3918950938 learned_addr learned my addr 172.21.15.86:0/3918950938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9000640 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5627411c1eb0 0x56274121f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49abe00640 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627412765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 -- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5627411c1eb0 msgr2=0x56274121f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5627411c1eb0 0x56274121f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 -- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 msgr2=0x5627412765c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627412765c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 -- 172.21.15.86:0/3918950938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562741091840 con 0x56274120b730 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49abe00640 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627412765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9a00640 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56274120b730 0x562741276b00 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x562740fdd1c0 tx=0x7f49ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd41c4e15ccb864 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 1 -- 172.21.15.86:0/3918950938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac02c060 con 0x56274120b730 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 1 -- 172.21.15.86:0/3918950938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f49ac015b10 con 0x56274120b730 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 -- 172.21.15.86:0/3918950938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a40b1040 con 0x56274120b730 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 1 -- 172.21.15.86:0/3918950938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f49ac032050 con 0x56274120b730 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 10 client.7875 fetch_fsmap learned FSMap version 71 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 -- 172.21.15.86:0/3918950938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x562740ccb470 con 0x56274120b730 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 1 -- 172.21.15.86:0/3918950938 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a40b1040 con 0x56274120b730 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 1 -- 172.21.15.86:0/3918950938 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f49ac035020 con 0x56274120b730 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 10 client.7875 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:17:47.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 10 client.7875 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:47.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 20 client.7875 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:47.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x562741277040 0x562741277450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 4 client.7875 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-22T04:17:47.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 -- 172.21.15.86:0/3918950938 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x562740fe96b0 con 0x562741277040 2024-03-22T04:17:47.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9000640 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x562741277040 0x562741277450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 10 client.7875 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:47.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49b9000640 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x562741277040 0x562741277450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354fc40a3a60c485 server_cookie=a7bdf201afc1722f in_seq=0 out_seq=0 2024-03-22T04:17:47.386 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:17:47.380+0000 7f0c5a400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-03-22T04:17:05.898187+0000 1=1+0)->f(v1 m2024-03-22T04:17:05.898187+0000) n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1)->n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55c18476bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-22T04:17:05.898187+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 1 -- 172.21.15.86:0/3918950938 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562740fe96b0 con 0x562741277040 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 10 client.7875 handle_command_reply: tid=0 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 client.7875 shutdown 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 -- 172.21.15.86:0/3918950938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56274120b730 msgr2=0x562741276b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56274120b730 0x562741276b00 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x562740fdd1c0 tx=0x7f49ac014000 comp rx=0 tx=0).stop 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 -- 172.21.15.86:0/3918950938 shutdown_connections 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x562741277040 0x562741277450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56274120ae10 0x5627412765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56274120b730 0x562741276b00 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 --2- 172.21.15.86:0/3918950938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5627411c1eb0 0x56274121f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49a2000640 0 client.7875 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 -- 172.21.15.86:0/3918950938 >> 172.21.15.86:0/3918950938 conn(0x5627411bde60 msgr2=0x5627412099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 -- 172.21.15.86:0/3918950938 shutdown_connections 2024-03-22T04:17:47.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 1 -- 172.21.15.86:0/3918950938 wait complete. 2024-03-22T04:17:47.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bee55000 20 client.7875 trim_cache size 0 max 16384 2024-03-22T04:17:47.388 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:17:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:17:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e30abe24-0214-4280-a5a0-292c17586b0a", 2024-03-22T04:17:47.388 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:17:47.388 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bba00640 1 -- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4990067b50 msgr2=0x7f499006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bba00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4990067b50 0x7f499006a010 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ad000 tx=0x7f49a40a6000 comp rx=0 tx=0).stop 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bba00640 1 -- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b4072990 msgr2=0x7f49b406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bba00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b4072990 0x7f49b406b810 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c4e0 tx=0x7f499c00c9a0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bba00640 1 -- 172.21.15.86:0/1495112901 shutdown_connections 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bba00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4990067b50 0x7f499006a010 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.381+0000 7f49bba00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49b415c440 0x7f49b406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.385+0000 7f49bba00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49b4072990 0x7f49b406b810 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.385+0000 7f49bba00640 1 --2- 172.21.15.86:0/1495112901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49b406c820 0x7f49b414c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.385+0000 7f49bba00640 1 -- 172.21.15.86:0/1495112901 >> 172.21.15.86:0/1495112901 conn(0x7f49b40d07e0 msgr2=0x7f49b4071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.385+0000 7f49bba00640 1 -- 172.21.15.86:0/1495112901 shutdown_connections 2024-03-22T04:17:47.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.385+0000 7f49bba00640 1 -- 172.21.15.86:0/1495112901 wait complete. 2024-03-22T04:17:47.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:17:47.404 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-22T04:17:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e2c400640 1 Processor -- start 2024-03-22T04:17:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e2c400640 1 -- start start 2024-03-22T04:17:47.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e2c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e2415c4b0 0x7f6e2415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e2c400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2415c890 con 0x7f6e240d1060 2024-03-22T04:17:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e2c400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e24073b30 con 0x7f6e24068d60 2024-03-22T04:17:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e2c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e240c7630 con 0x7f6e2415c4b0 2024-03-22T04:17:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e29000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6e240d1060 0x7f6e24072d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54750/0 (socket says 172.21.15.86:54750) 2024-03-22T04:17:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e29000640 1 -- 172.21.15.86:0/3888271505 learned_addr learned my addr 172.21.15.86:0/3888271505 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:17:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e29a00640 1 --2- 172.21.15.86:0/3888271505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e2415c4b0 0x7f6e2415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e23e00640 1 -- 172.21.15.86:0/3888271505 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399723282 0 0) 0x7f6e2415c890 con 0x7f6e240d1060 2024-03-22T04:17:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e23e00640 1 -- 172.21.15.86:0/3888271505 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e0c003530 con 0x7f6e240d1060 2024-03-22T04:17:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e23e00640 1 -- 172.21.15.86:0/3888271505 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368636112 0 0) 0x7f6e24073b30 con 0x7f6e24068d60 2024-03-22T04:17:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e23e00640 1 -- 172.21.15.86:0/3888271505 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e2415c890 con 0x7f6e24068d60 2024-03-22T04:17:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e29a00640 1 -- 172.21.15.86:0/3888271505 >> v1:172.21.15.196:6789/0 conn(0x7f6e24068d60 legacy=0x7f6e24073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e29a00640 1 -- 172.21.15.86:0/3888271505 >> v1:172.21.15.86:6789/0 conn(0x7f6e240d1060 legacy=0x7f6e24072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.541+0000 7f6e29a00640 1 -- 172.21.15.86:0/3888271505 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e24160b50 con 0x7f6e2415c4b0 2024-03-22T04:17:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e29a00640 1 --2- 172.21.15.86:0/3888271505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e2415c4b0 0x7f6e2415ff80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6e180097f0 tx=0x7f6e1801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3f77ee02bd7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e23e00640 1 -- 172.21.15.86:0/3888271505 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180b2070 con 0x7f6e2415c4b0 2024-03-22T04:17:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e23e00640 1 -- 172.21.15.86:0/3888271505 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e180a9070 con 0x7f6e2415c4b0 2024-03-22T04:17:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e23e00640 1 -- 172.21.15.86:0/3888271505 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180ad880 con 0x7f6e2415c4b0 2024-03-22T04:17:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e2c400640 1 -- 172.21.15.86:0/3888271505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e2415c4b0 msgr2=0x7f6e2415ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e2c400640 1 --2- 172.21.15.86:0/3888271505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e2415c4b0 0x7f6e2415ff80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6e180097f0 tx=0x7f6e1801c450 comp rx=0 tx=0).stop 2024-03-22T04:17:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e2c400640 1 -- 172.21.15.86:0/3888271505 shutdown_connections 2024-03-22T04:17:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e2c400640 1 --2- 172.21.15.86:0/3888271505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e2415c4b0 0x7f6e2415ff80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e2c400640 1 -- 172.21.15.86:0/3888271505 >> 172.21.15.86:0/3888271505 conn(0x7f6e240d0330 msgr2=0x7f6e240d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e2c400640 1 -- 172.21.15.86:0/3888271505 shutdown_connections 2024-03-22T04:17:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.545+0000 7f6e2c400640 1 -- 172.21.15.86:0/3888271505 wait complete. 2024-03-22T04:17:47.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 Processor -- start 2024-03-22T04:17:47.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 -- start start 2024-03-22T04:17:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e24068d60 0x7f6e24151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e240d1060 0x7f6e24098e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e2415c4b0 0x7f6e24153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2416aae0 con 0x7f6e2415c4b0 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e241608d0 con 0x7f6e240d1060 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2416ac60 con 0x7f6e24068d60 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e2415c4b0 0x7f6e24153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e29a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e24068d60 0x7f6e24151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e29000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e240d1060 0x7f6e24098e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e2415c4b0 0x7f6e24153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44720/0 (socket says 172.21.15.86:44720) 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2a400640 1 -- 172.21.15.86:0/488781445 learned_addr learned my addr 172.21.15.86:0/488781445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e29a00640 1 -- 172.21.15.86:0/488781445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e240d1060 msgr2=0x7f6e24098e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e29a00640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e240d1060 0x7f6e24098e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e29a00640 1 -- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e2415c4b0 msgr2=0x7f6e24153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e29a00640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e2415c4b0 0x7f6e24153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e29a00640 1 -- 172.21.15.86:0/488781445 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e240993c0 con 0x7f6e24068d60 2024-03-22T04:17:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e29a00640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e24068d60 0x7f6e24151db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e180ac000 tx=0x7f6e180191f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e21600640 1 -- 172.21.15.86:0/488781445 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180b2040 con 0x7f6e24068d60 2024-03-22T04:17:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e21600640 1 -- 172.21.15.86:0/488781445 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e180add30 con 0x7f6e24068d60 2024-03-22T04:17:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 -- 172.21.15.86:0/488781445 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e241524b0 con 0x7f6e24068d60 2024-03-22T04:17:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e21600640 1 -- 172.21.15.86:0/488781445 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180b6670 con 0x7f6e24068d60 2024-03-22T04:17:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.549+0000 7f6e2c400640 1 -- 172.21.15.86:0/488781445 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e24152a70 con 0x7f6e24068d60 2024-03-22T04:17:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.553+0000 7f6e21600640 1 -- 172.21.15.86:0/488781445 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6e180bd060 con 0x7f6e24068d60 2024-03-22T04:17:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.553+0000 7f6e21600640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e04067b20 0x7f6e04069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.553+0000 7f6e21600640 1 -- 172.21.15.86:0/488781445 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f6e1811ec70 con 0x7f6e24068d60 2024-03-22T04:17:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.553+0000 7f6e29000640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e04067b20 0x7f6e04069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.553+0000 7f6e2f9a2000 1 Processor -- start 2024-03-22T04:17:47.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.553+0000 7f6e2f9a2000 1 -- start start 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e04067b20 0x7f6e04069fe0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6e240cca70 tx=0x7f6e10007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee2980360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee29808a0 0x555ee2933a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee2934330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ee2784c40 con 0x555ee29808a0 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ee2721d90 con 0x555ee2933f50 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x555ee2780bf0 con 0x555ee297ff80 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee29808a0 0x555ee2933a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee29808a0 0x555ee2933a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44728/0 (socket says 172.21.15.86:44728) 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 -- 172.21.15.86:0/3623516970 learned_addr learned my addr 172.21.15.86:0/3623516970 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2a400640 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee2934330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29a00640 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee2980360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 -- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 msgr2=0x555ee2980360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee2980360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 -- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 msgr2=0x555ee2934330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee2934330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 -- 172.21.15.86:0/3623516970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ee282c130 con 0x555ee29808a0 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29a00640 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee2980360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e29000640 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee29808a0 0x555ee2933a10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6e10001860 tx=0x7f6e10012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf412f27a3fbeaef server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e16000640 1 -- 172.21.15.86:0/3623516970 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10025070 con 0x555ee29808a0 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e16000640 1 -- 172.21.15.86:0/3623516970 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1001b050 con 0x555ee29808a0 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e16000640 1 -- 172.21.15.86:0/3623516970 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10014de0 con 0x555ee29808a0 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3623516970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee29808a0 msgr2=0x555ee2933a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee29808a0 0x555ee2933a10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6e10001860 tx=0x7f6e10012000 comp rx=0 tx=0).stop 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3623516970 shutdown_connections 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee2934330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee29808a0 0x555ee2933a10 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3623516970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee2980360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3623516970 >> 172.21.15.86:0/3623516970 conn(0x555ee281d990 msgr2=0x555ee2963150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3623516970 shutdown_connections 2024-03-22T04:17:47.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3623516970 wait complete. 2024-03-22T04:17:47.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 -- start start 2024-03-22T04:17:47.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.557+0000 7f6e2f9a2000 1 Processor -- start 2024-03-22T04:17:47.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee2933f50 0x555ee293ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee2964380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee29808a0 0x555ee29648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ee2740710 con 0x555ee2933f50 2024-03-22T04:17:47.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ee2742580 con 0x555ee29808a0 2024-03-22T04:17:47.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x555ee2789270 con 0x555ee297ff80 2024-03-22T04:17:47.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 10 client.0 _ll_register_callbacks cb 0x555ee27e6200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee2933f50 0x555ee293ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee2933f50 0x555ee293ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44732/0 (socket says 172.21.15.86:44732) 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 -- 172.21.15.86:0/2806295768 learned_addr learned my addr 172.21.15.86:0/2806295768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 -- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 msgr2=0x555ee2964380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee2964380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 -- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee29808a0 msgr2=0x555ee29648c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee29808a0 0x555ee29648c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 -- 172.21.15.86:0/2806295768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ee275dd30 con 0x555ee2933f50 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29a00640 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee2933f50 0x555ee293ba40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f6e18097e30 tx=0x7f6e180eecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcb1b6f4bc3d332 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 1 -- 172.21.15.86:0/2806295768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180f52b0 con 0x555ee2933f50 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 1 -- 172.21.15.86:0/2806295768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f6e180ee120 con 0x555ee2933f50 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/2806295768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e1001b050 con 0x555ee2933f50 2024-03-22T04:17:47.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 1 -- 172.21.15.86:0/2806295768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f6e180ee320 con 0x555ee2933f50 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 10 client.7887 fetch_fsmap learned FSMap version 71 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/2806295768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x555ee27c9fc0 con 0x555ee2933f50 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 1 -- 172.21.15.86:0/2806295768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1001b050 con 0x555ee2933f50 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 1 -- 172.21.15.86:0/2806295768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f6e180b5070 con 0x555ee2933f50 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 10 client.7887 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 10 client.7887 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 20 client.7887 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x555ee2986630 0x555ee2986a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 4 client.7887 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/2806295768 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ee271fdd0 con 0x555ee2986630 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29000640 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x555ee2986630 0x555ee2986a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 10 client.7887 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e29000640 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x555ee2986630 0x555ee2986a10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e9e11fe5d0315e server_cookie=df4b823b2f7b8d3c in_seq=0 out_seq=0 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 1 -- 172.21.15.86:0/2806295768 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555ee271fdd0 con 0x555ee2986630 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 10 client.7887 handle_command_reply: tid=0 2024-03-22T04:17:47.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 client.7887 shutdown 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/2806295768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee2933f50 msgr2=0x555ee293ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee2933f50 0x555ee293ba40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f6e18097e30 tx=0x7f6e180eecd0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/2806295768 shutdown_connections 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x555ee2986630 0x555ee2986a10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee29808a0 0x555ee29648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee2964380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e0aa00640 0 client.7887 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/2806295768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee2933f50 0x555ee293ba40 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/2806295768 >> 172.21.15.86:0/2806295768 conn(0x555ee281d990 msgr2=0x555ee2933110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/2806295768 shutdown_connections 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.561+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/2806295768 wait complete. 2024-03-22T04:17:47.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.565+0000 7f6e2f9a2000 20 client.7887 trim_cache size 0 max 16384 2024-03-22T04:17:47.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 Processor -- start 2024-03-22T04:17:47.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- start start 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2996b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee298ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee293a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e242003a0 con 0x555ee281d990 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e240cf7e0 con 0x555ee2933f50 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e240c6570 con 0x555ee297ff80 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2996b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2996b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44748/0 (socket says 172.21.15.86:44748) 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 -- 172.21.15.86:0/4004011316 learned_addr learned my addr 172.21.15.86:0/4004011316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29000640 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee298ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2a400640 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee293a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 -- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 msgr2=0x555ee293a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee293a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 -- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 msgr2=0x555ee298ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee298ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 -- 172.21.15.86:0/4004011316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ee275a3a0 con 0x555ee281d990 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29000640 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee298ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e29a00640 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2996b80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009740 tx=0x7f6e180f0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45f1ab2ae2acb7c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e15600640 1 -- 172.21.15.86:0/4004011316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1810d310 con 0x555ee281d990 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e15600640 1 -- 172.21.15.86:0/4004011316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e180a9050 con 0x555ee281d990 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e15600640 1 -- 172.21.15.86:0/4004011316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180eaa20 con 0x555ee281d990 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/4004011316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 msgr2=0x555ee2996b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2996b80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009740 tx=0x7f6e180f0000 comp rx=0 tx=0).stop 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/4004011316 shutdown_connections 2024-03-22T04:17:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee2933f50 0x555ee298ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee297ff80 0x555ee293a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/4004011316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2996b80 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/4004011316 >> 172.21.15.86:0/4004011316 conn(0x555ee2963400 msgr2=0x555ee293a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/4004011316 shutdown_connections 2024-03-22T04:17:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.581+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/4004011316 wait complete. 2024-03-22T04:17:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 -- start start 2024-03-22T04:17:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 Processor -- start 2024-03-22T04:17:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2989a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee2933f50 0x555ee2989fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee297ff80 0x555ee298a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ee2840810 con 0x555ee281d990 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e240cf110 con 0x555ee297ff80 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x555ee2783f50 con 0x555ee2933f50 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 10 client.0 _ll_register_callbacks cb 0x7f6e18009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2989a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2989a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44754/0 (socket says 172.21.15.86:44754) 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 -- 172.21.15.86:0/3122218796 learned_addr learned my addr 172.21.15.86:0/3122218796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29000640 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee2933f50 0x555ee2989fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2a400640 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee297ff80 0x555ee298a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 -- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee2933f50 msgr2=0x555ee2989fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee2933f50 0x555ee2989fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 -- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee297ff80 msgr2=0x555ee298a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee297ff80 0x555ee298a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 -- 172.21.15.86:0/3122218796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ee270a380 con 0x555ee281d990 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2a400640 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee297ff80 0x555ee298a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29a00640 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2989a70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f6e18097e30 tx=0x7f6e1811cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fed847a85750806 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 1 -- 172.21.15.86:0/3122218796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180ee120 con 0x555ee281d990 2024-03-22T04:17:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 1 -- 172.21.15.86:0/3122218796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f6e180f50b0 con 0x555ee281d990 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3122218796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e180a9050 con 0x555ee281d990 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 1 -- 172.21.15.86:0/3122218796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f6e180fc050 con 0x555ee281d990 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 10 client.7893 fetch_fsmap learned FSMap version 71 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3122218796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x555ee2806460 con 0x555ee281d990 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 1 -- 172.21.15.86:0/3122218796 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e180a9050 con 0x555ee281d990 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 1 -- 172.21.15.86:0/3122218796 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f6e180b5040 con 0x555ee281d990 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 10 client.7893 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 10 client.7893 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 20 client.7893 populate_metadata read hostname 'smithi086' 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x555ee29808a0 0x555ee298aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 4 client.7893 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3122218796 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ee2724160 con 0x555ee29808a0 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29000640 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x555ee29808a0 0x555ee298aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 10 client.7893 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e29000640 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x555ee29808a0 0x555ee298aa30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166794937af8c5c5 server_cookie=77594bd6b1a0b3d1 in_seq=0 out_seq=0 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 1 -- 172.21.15.86:0/3122218796 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x555ee2724160 con 0x555ee29808a0 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e0aa00640 10 client.7893 handle_command_reply: tid=0 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 client.7893 shutdown 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3122218796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 msgr2=0x555ee2989a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.585+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2989a70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f6e18097e30 tx=0x7f6e1811cd10 comp rx=0 tx=0).stop 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3122218796 shutdown_connections 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x555ee29808a0 0x555ee298aa30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ee297ff80 0x555ee298a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ee2933f50 0x555ee2989fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e0aa00640 0 client.7893 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 1 --2- 172.21.15.86:0/3122218796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ee281d990 0x555ee2989a70 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3122218796 >> 172.21.15.86:0/3122218796 conn(0x555ee2963400 msgr2=0x555ee293aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3122218796 shutdown_connections 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 1 -- 172.21.15.86:0/3122218796 wait complete. 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2f9a2000 20 client.7893 trim_cache size 0 max 16384 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "e30abe24-0214-4280-a5a0-292c17586b0a": { 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir", 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e30abe24-0214-4280-a5a0-292c17586b0a", 2024-03-22T04:17:47.594 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-03-22T04:17:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:17:47.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:17:47.595 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:17:47.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 -- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e04067b20 msgr2=0x7f6e04069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e04067b20 0x7f6e04069fe0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6e240cca70 tx=0x7f6e10007d80 comp rx=0 tx=0).stop 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 -- 172.21.15.86:0/488781445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e24068d60 msgr2=0x7f6e24151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e24068d60 0x7f6e24151db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e180ac000 tx=0x7f6e180191f0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 -- 172.21.15.86:0/488781445 shutdown_connections 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e04067b20 0x7f6e04069fe0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e240d1060 0x7f6e24098e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e2415c4b0 0x7f6e24153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 --2- 172.21.15.86:0/488781445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e24068d60 0x7f6e24151db0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 -- 172.21.15.86:0/488781445 >> 172.21.15.86:0/488781445 conn(0x7f6e240d0330 msgr2=0x7f6e240c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 -- 172.21.15.86:0/488781445 shutdown_connections 2024-03-22T04:17:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:17:47.589+0000 7f6e2c400640 1 -- 172.21.15.86:0/488781445 wait complete. 2024-03-22T04:17:47.606 INFO:tasks.cephfs.filesystem:scrub status for tag:e30abe24-0214-4280-a5a0-292c17586b0a - {'path': '/dir', 'tag': 'e30abe24-0214-4280-a5a0-292c17586b0a', 'options': 'recursive,repair'} 2024-03-22T04:17:58.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:17:58.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:18:17.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:17.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-22T04:18:17.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb523400640 1 Processor -- start 2024-03-22T04:18:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb523400640 1 -- start start 2024-03-22T04:18:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb523400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb51c15a830 0x7fb51c0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb523400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071860 0x7fb51c15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb523400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb51c0af900 con 0x7fb51c071d20 2024-03-22T04:18:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb523400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb51c0cfbb0 con 0x7fb51c15a830 2024-03-22T04:18:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb523400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb51c0cf1e0 con 0x7fb51c071860 2024-03-22T04:18:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb51be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071860 0x7fb51c15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb521400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb51c071d20 0x7fb51c072140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:56178/0 (socket says 172.21.15.86:56178) 2024-03-22T04:18:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb521400640 1 -- 172.21.15.86:0/234998803 learned_addr learned my addr 172.21.15.86:0/234998803 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb520a00640 1 --2- 172.21.15.86:0/234998803 >> v2:172.21.15.196:3300/0 conn(0x7fb51c15a830 0x7fb51c0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb51b400640 1 -- 172.21.15.86:0/234998803 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639190592 0 0) 0x7fb51c0af900 con 0x7fb51c071d20 2024-03-22T04:18:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb51b400640 1 -- 172.21.15.86:0/234998803 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb504003530 con 0x7fb51c071d20 2024-03-22T04:18:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb51be00640 1 -- 172.21.15.86:0/234998803 >> v2:172.21.15.196:3300/0 conn(0x7fb51c15a830 msgr2=0x7fb51c0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb51be00640 1 --2- 172.21.15.86:0/234998803 >> v2:172.21.15.196:3300/0 conn(0x7fb51c15a830 0x7fb51c0cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb51be00640 1 -- 172.21.15.86:0/234998803 >> v1:172.21.15.196:6789/0 conn(0x7fb51c071d20 legacy=0x7fb51c072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb51be00640 1 -- 172.21.15.86:0/234998803 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c0ce5d0 con 0x7fb51c071860 2024-03-22T04:18:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.761+0000 7fb520a00640 1 --2- 172.21.15.86:0/234998803 >> v2:172.21.15.196:3300/0 conn(0x7fb51c15a830 0x7fb51c0cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:17.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb51be00640 1 --2- 172.21.15.86:0/234998803 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071860 0x7fb51c15ac10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb50800a960 tx=0x7fb50801d870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=222596c49b04c8a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb51b400640 1 -- 172.21.15.86:0/234998803 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5080b3070 con 0x7fb51c071860 2024-03-22T04:18:17.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb51b400640 1 -- 172.21.15.86:0/234998803 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5080a6790 con 0x7fb51c071860 2024-03-22T04:18:17.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb51b400640 1 -- 172.21.15.86:0/234998803 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50800b040 con 0x7fb51c071860 2024-03-22T04:18:17.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb523400640 1 -- 172.21.15.86:0/234998803 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071860 msgr2=0x7fb51c15ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb523400640 1 --2- 172.21.15.86:0/234998803 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071860 0x7fb51c15ac10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb50800a960 tx=0x7fb50801d870 comp rx=0 tx=0).stop 2024-03-22T04:18:17.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb523400640 1 -- 172.21.15.86:0/234998803 shutdown_connections 2024-03-22T04:18:17.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb523400640 1 --2- 172.21.15.86:0/234998803 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071860 0x7fb51c15ac10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb523400640 1 --2- 172.21.15.86:0/234998803 >> v2:172.21.15.196:3300/0 conn(0x7fb51c15a830 0x7fb51c0cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb523400640 1 -- 172.21.15.86:0/234998803 >> 172.21.15.86:0/234998803 conn(0x7fb51c0cc940 msgr2=0x7fb51c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:17.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb523400640 1 -- 172.21.15.86:0/234998803 shutdown_connections 2024-03-22T04:18:17.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.765+0000 7fb523400640 1 -- 172.21.15.86:0/234998803 wait complete. 2024-03-22T04:18:17.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 Processor -- start 2024-03-22T04:18:17.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 -- start start 2024-03-22T04:18:17.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb51c071860 0x7fb51c150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071d20 0x7fb51c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb51c15a830 0x7fb51c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c06d6d0 con 0x7fb51c071860 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c15b150 con 0x7fb51c15a830 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb51c06d850 con 0x7fb51c071d20 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb51c071860 0x7fb51c150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb51c071860 0x7fb51c150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54966/0 (socket says 172.21.15.86:54966) 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 -- 172.21.15.86:0/978853632 learned_addr learned my addr 172.21.15.86:0/978853632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb521400640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb51c15a830 0x7fb51c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb51be00640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071d20 0x7fb51c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 -- 172.21.15.86:0/978853632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071d20 msgr2=0x7fb51c157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071d20 0x7fb51c157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 -- 172.21.15.86:0/978853632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb51c15a830 msgr2=0x7fb51c15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb51c15a830 0x7fb51c15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 -- 172.21.15.86:0/978853632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c150f90 con 0x7fb51c071860 2024-03-22T04:18:17.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb520a00640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb51c071860 0x7fb51c150a50 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e980 tx=0x7fb51000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb518c00640 1 -- 172.21.15.86:0/978853632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510017070 con 0x7fb51c071860 2024-03-22T04:18:17.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb518c00640 1 -- 172.21.15.86:0/978853632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51000ca00 con 0x7fb51c071860 2024-03-22T04:18:17.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 -- 172.21.15.86:0/978853632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51c155c40 con 0x7fb51c071860 2024-03-22T04:18:17.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb518c00640 1 -- 172.21.15.86:0/978853632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51000cdd0 con 0x7fb51c071860 2024-03-22T04:18:17.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.769+0000 7fb523400640 1 -- 172.21.15.86:0/978853632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51c156200 con 0x7fb51c071860 2024-03-22T04:18:17.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb518c00640 1 -- 172.21.15.86:0/978853632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb51001a070 con 0x7fb51c071860 2024-03-22T04:18:17.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb518c00640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4fc067b50 0x7fb4fc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb518c00640 1 -- 172.21.15.86:0/978853632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fb510086400 con 0x7fb51c071860 2024-03-22T04:18:17.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb51be00640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4fc067b50 0x7fb4fc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb526864000 1 Processor -- start 2024-03-22T04:18:17.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb526864000 1 -- start start 2024-03-22T04:18:17.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb51be00640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4fc067b50 0x7fb4fc06a010 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb5080b0000 tx=0x7fb508008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb526864000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694be0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb526864000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694b93a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb526864000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b94330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb526864000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616949e4c40 con 0x561694be08a0 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb526864000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561694981d90 con 0x561694b93f50 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb526864000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5616949e0bf0 con 0x561694bdff80 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb51be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694b93a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb51be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694b93a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54970/0 (socket says 172.21.15.86:54970) 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb51be00640 1 -- 172.21.15.86:0/51684316 learned_addr learned my addr 172.21.15.86:0/51684316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.773+0000 7fb520a00640 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694be0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b94330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 -- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 msgr2=0x561694be0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694be0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 -- 172.21.15.86:0/51684316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 msgr2=0x561694b93a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694b93a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 -- 172.21.15.86:0/51684316 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561694a8c130 con 0x561694b93f50 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b94330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb5140012f0 tx=0x7fb514007600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e67b8125d2cd43e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb50d600640 1 -- 172.21.15.86:0/51684316 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514011580 con 0x561694b93f50 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb50d600640 1 -- 172.21.15.86:0/51684316 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb514018070 con 0x561694b93f50 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb50d600640 1 -- 172.21.15.86:0/51684316 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514011920 con 0x561694b93f50 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 msgr2=0x561694b94330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b94330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb5140012f0 tx=0x7fb514007600 comp rx=0 tx=0).stop 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- 172.21.15.86:0/51684316 shutdown_connections 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b94330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694b93a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 --2- 172.21.15.86:0/51684316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694be0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- 172.21.15.86:0/51684316 >> 172.21.15.86:0/51684316 conn(0x561694a7d990 msgr2=0x561694bc3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:17.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- 172.21.15.86:0/51684316 shutdown_connections 2024-03-22T04:18:17.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- 172.21.15.86:0/51684316 wait complete. 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- start start 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 Processor -- start 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b9b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694bc4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694bc48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616949a0710 con 0x561694be08a0 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5616949a2580 con 0x561694b93f50 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5616949e9270 con 0x561694bdff80 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb526864000 10 client.0 _ll_register_callbacks cb 0x561694a46200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694bc48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694bc48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54984/0 (socket says 172.21.15.86:54984) 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb521400640 1 -- 172.21.15.86:0/903533834 learned_addr learned my addr 172.21.15.86:0/903533834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb51be00640 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694bc4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.777+0000 7fb520a00640 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b9b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb521400640 1 -- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 msgr2=0x561694bc4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb521400640 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694bc4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb521400640 1 -- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 msgr2=0x561694b9b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb521400640 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b9b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb521400640 1 -- 172.21.15.86:0/903533834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616949bdd30 con 0x561694be08a0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb521400640 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694bc48c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb514007640 tx=0x7fb514001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2003116652cadd9e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 1 -- 172.21.15.86:0/903533834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514011d40 con 0x561694be08a0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 1 -- 172.21.15.86:0/903533834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fb514010bd0 con 0x561694be08a0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 -- 172.21.15.86:0/903533834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb514018070 con 0x561694be08a0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 1 -- 172.21.15.86:0/903533834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fb514010dd0 con 0x561694be08a0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 10 client.7902 fetch_fsmap learned FSMap version 71 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 -- 172.21.15.86:0/903533834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x561694a29fc0 con 0x561694be08a0 2024-03-22T04:18:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 1 -- 172.21.15.86:0/903533834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb514018070 con 0x561694be08a0 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 1 -- 172.21.15.86:0/903533834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7fb514010290 con 0x561694be08a0 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 10 client.7902 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 10 client.7902 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 20 client.7902 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x561694be65c0 0x561694be69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 4 client.7902 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 -- 172.21.15.86:0/903533834 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56169497fdd0 con 0x561694be65c0 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb520a00640 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x561694be65c0 0x561694be69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb520a00640 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x561694be65c0 0x561694be69a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb01b1566c79cb6 server_cookie=8c50291194322f07 in_seq=0 out_seq=0 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 10 client.7902 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 1 -- 172.21.15.86:0/903533834 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56169497fdd0 con 0x561694be65c0 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 10 client.7902 handle_command_reply: tid=0 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 client.7902 shutdown 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 -- 172.21.15.86:0/903533834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 msgr2=0x561694bc48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694bc48c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb514007640 tx=0x7fb514001260 comp rx=0 tx=0).stop 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 -- 172.21.15.86:0/903533834 shutdown_connections 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x561694be65c0 0x561694be69a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694be08a0 0x561694bc48c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694bc4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 --2- 172.21.15.86:0/903533834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694b9b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb502000640 0 client.7902 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 -- 172.21.15.86:0/903533834 >> 172.21.15.86:0/903533834 conn(0x561694a7d990 msgr2=0x561694b93110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 -- 172.21.15.86:0/903533834 shutdown_connections 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 1 -- 172.21.15.86:0/903533834 wait complete. 2024-03-22T04:18:17.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.781+0000 7fb526864000 20 client.7902 trim_cache size 0 max 16384 2024-03-22T04:18:17.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 Processor -- start 2024-03-22T04:18:17.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- start start 2024-03-22T04:18:17.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694a7d990 0x561694bf6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694be53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694be58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c204240 con 0x561694a7d990 2024-03-22T04:18:17.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c203220 con 0x561694b93f50 2024-03-22T04:18:17.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb51c152200 con 0x561694bdff80 2024-03-22T04:18:17.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694a7d990 0x561694bf6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694a7d990 0x561694bf6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54988/0 (socket says 172.21.15.86:54988) 2024-03-22T04:18:17.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 -- 172.21.15.86:0/1998908878 learned_addr learned my addr 172.21.15.86:0/1998908878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:17.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb521400640 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694be58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb51be00640 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694be53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 -- 172.21.15.86:0/1998908878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 msgr2=0x561694be58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694be58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 -- 172.21.15.86:0/1998908878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 msgr2=0x561694be53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694be53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 -- 172.21.15.86:0/1998908878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616949ba3a0 con 0x561694a7d990 2024-03-22T04:18:17.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb520a00640 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694a7d990 0x561694bf6680 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb510009fd0 tx=0x7fb51001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58d0dc2a8d34923 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb50cc00640 1 -- 172.21.15.86:0/1998908878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510021040 con 0x561694a7d990 2024-03-22T04:18:17.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb50cc00640 1 -- 172.21.15.86:0/1998908878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb510054760 con 0x561694a7d990 2024-03-22T04:18:17.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb50cc00640 1 -- 172.21.15.86:0/1998908878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510052ad0 con 0x561694a7d990 2024-03-22T04:18:17.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- 172.21.15.86:0/1998908878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694a7d990 msgr2=0x561694bf6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694a7d990 0x561694bf6680 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb510009fd0 tx=0x7fb51001e040 comp rx=0 tx=0).stop 2024-03-22T04:18:17.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- 172.21.15.86:0/1998908878 shutdown_connections 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694be53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694bdff80 0x561694be58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 --2- 172.21.15.86:0/1998908878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694a7d990 0x561694bf6680 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- 172.21.15.86:0/1998908878 >> 172.21.15.86:0/1998908878 conn(0x561694bc3400 msgr2=0x561694be4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- 172.21.15.86:0/1998908878 shutdown_connections 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- 172.21.15.86:0/1998908878 wait complete. 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 -- start start 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.801+0000 7fb526864000 1 Processor -- start 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694a7d990 0x561694be9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694bea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694bdff80 0x561694bea660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561694aa0810 con 0x561694bdff80 2024-03-22T04:18:17.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c202b80 con 0x561694b93f50 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56169498b350 con 0x561694a7d990 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 10 client.0 _ll_register_callbacks cb 0x7fb504003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694bdff80 0x561694bea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb520a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694a7d990 0x561694be9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694bdff80 0x561694bea660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54996/0 (socket says 172.21.15.86:54996) 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 -- 172.21.15.86:0/571381912 learned_addr learned my addr 172.21.15.86:0/571381912 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb51be00640 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694bea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 -- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694a7d990 msgr2=0x561694be9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694a7d990 0x561694be9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 -- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 msgr2=0x561694bea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694bea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 -- 172.21.15.86:0/571381912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169496a380 con 0x561694bdff80 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb521400640 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694bdff80 0x561694bea660 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb514001230 tx=0x7fb514001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9219b725e429c464 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:17.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 1 -- 172.21.15.86:0/571381912 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514037070 con 0x561694bdff80 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 1 -- 172.21.15.86:0/571381912 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fb514011d40 con 0x561694bdff80 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- 172.21.15.86:0/571381912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510054760 con 0x561694bdff80 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 1 -- 172.21.15.86:0/571381912 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fb514044050 con 0x561694bdff80 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 10 client.7908 fetch_fsmap learned FSMap version 71 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- 172.21.15.86:0/571381912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x561694a66460 con 0x561694bdff80 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 1 -- 172.21.15.86:0/571381912 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb510054760 con 0x561694bdff80 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 1 -- 172.21.15.86:0/571381912 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7fb514047020 con 0x561694bdff80 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 10 client.7908 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 10 client.7908 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 20 client.7908 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x561694be08a0 0x561694beaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 4 client.7908 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- 172.21.15.86:0/571381912 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561694984160 con 0x561694be08a0 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb520a00640 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x561694be08a0 0x561694beaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb520a00640 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x561694be08a0 0x561694beaba0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2455600330cff26f server_cookie=49d8f553de4238a1 in_seq=0 out_seq=0 2024-03-22T04:18:17.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 10 client.7908 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 1 -- 172.21.15.86:0/571381912 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561694984160 con 0x561694be08a0 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 10 client.7908 handle_command_reply: tid=0 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 client.7908 shutdown 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- 172.21.15.86:0/571381912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694bdff80 msgr2=0x561694bea660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694bdff80 0x561694bea660 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb514001230 tx=0x7fb514001a20 comp rx=0 tx=0).stop 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- 172.21.15.86:0/571381912 shutdown_connections 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x561694be08a0 0x561694beaba0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561694b93f50 0x561694bea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561694bdff80 0x561694bea660 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 --2- 172.21.15.86:0/571381912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561694a7d990 0x561694be9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb502000640 0 client.7908 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- 172.21.15.86:0/571381912 >> 172.21.15.86:0/571381912 conn(0x561694bc3400 msgr2=0x561694be5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- 172.21.15.86:0/571381912 shutdown_connections 2024-03-22T04:18:17.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 1 -- 172.21.15.86:0/571381912 wait complete. 2024-03-22T04:18:17.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.805+0000 7fb526864000 20 client.7908 trim_cache size 0 max 16384 2024-03-22T04:18:17.814 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:18:17.814 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:18:17.814 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:18:17.814 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:18:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 -- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4fc067b50 msgr2=0x7fb4fc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4fc067b50 0x7fb4fc06a010 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb5080b0000 tx=0x7fb508008040 comp rx=0 tx=0).stop 2024-03-22T04:18:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 -- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb51c071860 msgr2=0x7fb51c150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb51c071860 0x7fb51c150a50 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e980 tx=0x7fb51000ee40 comp rx=0 tx=0).stop 2024-03-22T04:18:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 -- 172.21.15.86:0/978853632 shutdown_connections 2024-03-22T04:18:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb4fc067b50 0x7fb4fc06a010 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb51c15a830 0x7fb51c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51c071d20 0x7fb51c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 --2- 172.21.15.86:0/978853632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb51c071860 0x7fb51c150a50 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 -- 172.21.15.86:0/978853632 >> 172.21.15.86:0/978853632 conn(0x7fb51c0cc940 msgr2=0x7fb51c0adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 -- 172.21.15.86:0/978853632 shutdown_connections 2024-03-22T04:18:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.809+0000 7fb523400640 1 -- 172.21.15.86:0/978853632 wait complete. 2024-03-22T04:18:17.826 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:18:17.826 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:18:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.961+0000 7f567e000640 1 Processor -- start 2024-03-22T04:18:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.961+0000 7f567e000640 1 -- start start 2024-03-22T04:18:17.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f567e000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 0x7f56780cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f567e000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f567815fed0 0x7f5678160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f567e000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56780d11c0 con 0x7f56780d0de0 2024-03-22T04:18:17.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f567e000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5678160860 con 0x7f567815fed0 2024-03-22T04:18:17.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f567e000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f56780cece0 con 0x7f56780ce6b0 2024-03-22T04:18:17.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5677400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f56780d0de0 0x7f56780d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:56190/0 (socket says 172.21.15.86:56190) 2024-03-22T04:18:17.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5677400640 1 -- 172.21.15.86:0/3540360282 learned_addr learned my addr 172.21.15.86:0/3540360282 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5677e00640 1 --2- 172.21.15.86:0/3540360282 >> v2:172.21.15.196:3300/0 conn(0x7f567815fed0 0x7f5678160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5676a00640 1 --2- 172.21.15.86:0/3540360282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 0x7f56780cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5676000640 1 -- 172.21.15.86:0/3540360282 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3242175101 0 0) 0x7f56780d11c0 con 0x7f56780d0de0 2024-03-22T04:18:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5676000640 1 -- 172.21.15.86:0/3540360282 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f565c003530 con 0x7f56780d0de0 2024-03-22T04:18:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5677e00640 1 -- 172.21.15.86:0/3540360282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 msgr2=0x7f56780cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5677e00640 1 --2- 172.21.15.86:0/3540360282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 0x7f56780cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5677e00640 1 -- 172.21.15.86:0/3540360282 >> v1:172.21.15.196:6789/0 conn(0x7f56780d0de0 legacy=0x7f56780d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.965+0000 7f5677e00640 1 -- 172.21.15.86:0/3540360282 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56780cefc0 con 0x7f567815fed0 2024-03-22T04:18:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f5677e00640 1 --2- 172.21.15.86:0/3540360282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 0x7f5678160320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f56700097c0 tx=0x7f567001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28438daeb507ec8a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f5676000640 1 -- 172.21.15.86:0/3540360282 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700b0070 con 0x7f567815fed0 2024-03-22T04:18:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f5676000640 1 -- 172.21.15.86:0/3540360282 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56700ab600 con 0x7f567815fed0 2024-03-22T04:18:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f5676000640 1 -- 172.21.15.86:0/3540360282 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700ab9b0 con 0x7f567815fed0 2024-03-22T04:18:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- 172.21.15.86:0/3540360282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 msgr2=0x7f5678160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 --2- 172.21.15.86:0/3540360282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 0x7f5678160320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f56700097c0 tx=0x7f567001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- 172.21.15.86:0/3540360282 shutdown_connections 2024-03-22T04:18:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 --2- 172.21.15.86:0/3540360282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 0x7f5678160320 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 --2- 172.21.15.86:0/3540360282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 0x7f56780cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- 172.21.15.86:0/3540360282 >> 172.21.15.86:0/3540360282 conn(0x7f567806f910 msgr2=0x7f567806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- 172.21.15.86:0/3540360282 shutdown_connections 2024-03-22T04:18:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- 172.21.15.86:0/3540360282 wait complete. 2024-03-22T04:18:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 Processor -- start 2024-03-22T04:18:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- start start 2024-03-22T04:18:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 0x7f5678157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56780d0de0 0x7f5678158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 0x7f5678154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56780d0530 con 0x7f56780d0de0 2024-03-22T04:18:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56780d03b0 con 0x7f567815fed0 2024-03-22T04:18:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.969+0000 7f567e000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5678071e30 con 0x7f56780ce6b0 2024-03-22T04:18:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56780d0de0 0x7f5678158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5677e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 0x7f5678154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56780d0de0 0x7f5678158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54998/0 (socket says 172.21.15.86:54998) 2024-03-22T04:18:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 -- 172.21.15.86:0/3317190956 learned_addr learned my addr 172.21.15.86:0/3317190956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5677400640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 0x7f5678157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 -- 172.21.15.86:0/3317190956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 msgr2=0x7f5678157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 0x7f5678157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 -- 172.21.15.86:0/3317190956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 msgr2=0x7f5678154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 0x7f5678154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 -- 172.21.15.86:0/3317190956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56780b5350 con 0x7f56780d0de0 2024-03-22T04:18:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5676a00640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56780d0de0 0x7f5678158270 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f566400e870 tx=0x7f566400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f566f400640 1 -- 172.21.15.86:0/3317190956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664017070 con 0x7f56780d0de0 2024-03-22T04:18:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f566f400640 1 -- 172.21.15.86:0/3317190956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f566400c940 con 0x7f56780d0de0 2024-03-22T04:18:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56780b5630 con 0x7f56780d0de0 2024-03-22T04:18:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f566f400640 1 -- 172.21.15.86:0/3317190956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566400ccf0 con 0x7f56780d0de0 2024-03-22T04:18:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56780b5bf0 con 0x7f56780d0de0 2024-03-22T04:18:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5678068650 con 0x7f56780d0de0 2024-03-22T04:18:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f566f400640 1 -- 172.21.15.86:0/3317190956 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f566401a070 con 0x7f56780d0de0 2024-03-22T04:18:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f566f400640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5658067b50 0x7f565806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f566f400640 1 -- 172.21.15.86:0/3317190956 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f5664086060 con 0x7f56780d0de0 2024-03-22T04:18:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.973+0000 7f5677400640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5658067b50 0x7f565806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.977+0000 7f5677400640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5658067b50 0x7f565806a010 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5668005c10 tx=0x7f5668009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:17.981+0000 7f566f400640 1 -- 172.21.15.86:0/3317190956 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f56640560c0 con 0x7f56780d0de0 2024-03-22T04:18:18.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.165+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56780d30c0 con 0x7f56780d0de0 2024-03-22T04:18:18.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.165+0000 7f566f400640 1 -- 172.21.15.86:0/3317190956 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5186 (secure 0 0 0) 0x7f566405a160 con 0x7f56780d0de0 2024-03-22T04:18:18.166 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:19.957152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76604":{"gid":76604,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6835/1851339627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1851339627},{"type":"v1","addr":"172.21.15.196:6835","nonce":1851339627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76604,"qdb_cluster":[76604]},"id":6}]} 2024-03-22T04:18:18.166 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 71 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5658067b50 msgr2=0x7f565806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5658067b50 0x7f565806a010 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5668005c10 tx=0x7f5668009040 comp rx=0 tx=0).stop 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56780d0de0 msgr2=0x7f5678158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56780d0de0 0x7f5678158270 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f566400e870 tx=0x7f566400ec70 comp rx=0 tx=0).stop 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 shutdown_connections 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5658067b50 0x7f565806a010 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f567815fed0 0x7f5678154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56780d0de0 0x7f5678158270 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 --2- 172.21.15.86:0/3317190956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56780ce6b0 0x7f5678157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 >> 172.21.15.86:0/3317190956 conn(0x7f567806f910 msgr2=0x7f567815a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 shutdown_connections 2024-03-22T04:18:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.169+0000 7f567e000640 1 -- 172.21.15.86:0/3317190956 wait complete. 2024-03-22T04:18:18.188 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-03-22T04:18:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0150400640 1 Processor -- start 2024-03-22T04:18:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0150400640 1 -- start start 2024-03-22T04:18:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0150400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f014815c730 0x7f014809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0150400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01480cda00 con 0x7f01480b3610 2024-03-22T04:18:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0150400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01480cdce0 con 0x7f01480b11c0 2024-03-22T04:18:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0150400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f014809b910 con 0x7f014815c730 2024-03-22T04:18:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f014d000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f01480b3610 0x7f014809afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45912/0 (socket says 172.21.15.86:45912) 2024-03-22T04:18:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f014d000640 1 -- 172.21.15.86:0/2021066246 learned_addr learned my addr 172.21.15.86:0/2021066246 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:18.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f014da00640 1 --2- 172.21.15.86:0/2021066246 >> v2:172.21.15.196:3300/0 conn(0x7f014815c730 0x7f014809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0147e00640 1 -- 172.21.15.86:0/2021066246 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240996451 0 0) 0x7f01480cda00 con 0x7f01480b3610 2024-03-22T04:18:18.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0147e00640 1 -- 172.21.15.86:0/2021066246 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0130003530 con 0x7f01480b3610 2024-03-22T04:18:18.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f0147e00640 1 -- 172.21.15.86:0/2021066246 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173637988 0 0) 0x7f01480cdce0 con 0x7f01480b11c0 2024-03-22T04:18:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f014da00640 1 -- 172.21.15.86:0/2021066246 >> v1:172.21.15.196:6789/0 conn(0x7f01480b11c0 legacy=0x7f01480b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f014da00640 1 -- 172.21.15.86:0/2021066246 >> v1:172.21.15.86:6789/0 conn(0x7f01480b3610 legacy=0x7f014809afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.325+0000 7f014da00640 1 -- 172.21.15.86:0/2021066246 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01480cdfc0 con 0x7f014815c730 2024-03-22T04:18:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f014da00640 1 --2- 172.21.15.86:0/2021066246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 0x7f014809aa70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f013c0097f0 tx=0x7f013c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb89b0d38b5d01a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0147e00640 1 -- 172.21.15.86:0/2021066246 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c0b0070 con 0x7f014815c730 2024-03-22T04:18:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0147e00640 1 -- 172.21.15.86:0/2021066246 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f013c0ab600 con 0x7f014815c730 2024-03-22T04:18:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0147e00640 1 -- 172.21.15.86:0/2021066246 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c0ab9d0 con 0x7f014815c730 2024-03-22T04:18:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0150400640 1 -- 172.21.15.86:0/2021066246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 msgr2=0x7f014809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0150400640 1 --2- 172.21.15.86:0/2021066246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 0x7f014809aa70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f013c0097f0 tx=0x7f013c01c410 comp rx=0 tx=0).stop 2024-03-22T04:18:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0150400640 1 -- 172.21.15.86:0/2021066246 shutdown_connections 2024-03-22T04:18:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0150400640 1 --2- 172.21.15.86:0/2021066246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 0x7f014809aa70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0150400640 1 -- 172.21.15.86:0/2021066246 >> 172.21.15.86:0/2021066246 conn(0x7f01480b3bf0 msgr2=0x7f01480b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0150400640 1 -- 172.21.15.86:0/2021066246 shutdown_connections 2024-03-22T04:18:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.329+0000 7f0150400640 1 -- 172.21.15.86:0/2021066246 wait complete. 2024-03-22T04:18:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 Processor -- start 2024-03-22T04:18:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 -- start start 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01480b11c0 0x7f0148153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01480b3610 0x7f01481597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 0x7f0148159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01480b2e90 con 0x7f01480b3610 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01480b2d10 con 0x7f014815c730 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f014809b6c0 con 0x7f01480b11c0 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01480b3610 0x7f01481597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01480b3610 0x7f01481597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55012/0 (socket says 172.21.15.86:55012) 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 -- 172.21.15.86:0/3850508999 learned_addr learned my addr 172.21.15.86:0/3850508999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014da00640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01480b11c0 0x7f0148153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014e400640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 0x7f0148159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 -- 172.21.15.86:0/3850508999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01480b11c0 msgr2=0x7f0148153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01480b11c0 0x7f0148153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 -- 172.21.15.86:0/3850508999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 msgr2=0x7f0148159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 0x7f0148159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 -- 172.21.15.86:0/3850508999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01480b6820 con 0x7f01480b3610 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f014d000640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01480b3610 0x7f01481597c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f013400c3c0 tx=0x7f013400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0145600640 1 -- 172.21.15.86:0/3850508999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134017070 con 0x7f01480b3610 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0145600640 1 -- 172.21.15.86:0/3850508999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0134005cf0 con 0x7f01480b3610 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 -- 172.21.15.86:0/3850508999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01481571e0 con 0x7f01480b3610 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0145600640 1 -- 172.21.15.86:0/3850508999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340072f0 con 0x7f01480b3610 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.333+0000 7f0150400640 1 -- 172.21.15.86:0/3850508999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01481577a0 con 0x7f01480b3610 2024-03-22T04:18:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0145600640 1 -- 172.21.15.86:0/3850508999 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f013401a070 con 0x7f01480b3610 2024-03-22T04:18:18.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0145600640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0128067b20 0x7f0128069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0145600640 1 -- 172.21.15.86:0/3850508999 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f0134085500 con 0x7f01480b3610 2024-03-22T04:18:18.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f014da00640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0128067b20 0x7f0128069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0153980000 1 Processor -- start 2024-03-22T04:18:18.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0153980000 1 -- start start 2024-03-22T04:18:18.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f014da00640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0128067b20 0x7f0128069fe0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f013c0ad000 tx=0x7f013c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0153980000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa9912a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0153980000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912ac10 0x55fa990e0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0153980000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa990e0d60 0x55fa990e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0153980000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa98f2fd80 con 0x55fa990e0d60 2024-03-22T04:18:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0153980000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa98eccce0 con 0x55fa9912a2f0 2024-03-22T04:18:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f0153980000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55fa98f2bd30 con 0x55fa9912ac10 2024-03-22T04:18:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f014e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa990e0d60 0x55fa990e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f014e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa990e0d60 0x55fa990e1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55024/0 (socket says 172.21.15.86:55024) 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.337+0000 7f014e400640 1 -- 172.21.15.86:0/3199951075 learned_addr learned my addr 172.21.15.86:0/3199951075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014d000640 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912ac10 0x55fa990e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014da00640 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa9912a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 -- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912ac10 msgr2=0x55fa990e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912ac10 0x55fa990e0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 -- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 msgr2=0x55fa9912a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa9912a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 -- 172.21.15.86:0/3199951075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa98f0a340 con 0x55fa990e0d60 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014da00640 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa9912a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa990e0d60 0x55fa990e1140 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0140001e30 tx=0x7f014000f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc6cafc35590468 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f013a000640 1 -- 172.21.15.86:0/3199951075 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014000fcb0 con 0x55fa990e0d60 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f013a000640 1 -- 172.21.15.86:0/3199951075 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140018070 con 0x55fa990e0d60 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f013a000640 1 -- 172.21.15.86:0/3199951075 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140003ba0 con 0x55fa990e0d60 2024-03-22T04:18:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- 172.21.15.86:0/3199951075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa990e0d60 msgr2=0x55fa990e1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa990e0d60 0x55fa990e1140 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0140001e30 tx=0x7f014000f5a0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- 172.21.15.86:0/3199951075 shutdown_connections 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa990e0d60 0x55fa990e1140 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912ac10 0x55fa990e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 --2- 172.21.15.86:0/3199951075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa9912a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- 172.21.15.86:0/3199951075 >> 172.21.15.86:0/3199951075 conn(0x55fa990dcd10 msgr2=0x55fa990df100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- 172.21.15.86:0/3199951075 shutdown_connections 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- 172.21.15.86:0/3199951075 wait complete. 2024-03-22T04:18:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- start start 2024-03-22T04:18:18.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 Processor -- start 2024-03-22T04:18:18.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa990e0d60 0x55fa990e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912a2f0 0x55fa99117570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99117ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa98eed4e0 con 0x55fa9912ac10 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa98eb7550 con 0x55fa990e0d60 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55fa98fb03b0 con 0x55fa9912a2f0 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f0153980000 10 client.0 _ll_register_callbacks cb 0x55fa990cbc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99117ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99117ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55030/0 (socket says 172.21.15.86:55030) 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 -- 172.21.15.86:0/24259364 learned_addr learned my addr 172.21.15.86:0/24259364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014da00640 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa990e0d60 0x55fa990e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014d000640 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912a2f0 0x55fa99117570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 -- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912a2f0 msgr2=0x55fa99117570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912a2f0 0x55fa99117570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 -- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa990e0d60 msgr2=0x55fa990e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa990e0d60 0x55fa990e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.341+0000 7f014e400640 1 -- 172.21.15.86:0/24259364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa98f745e0 con 0x55fa9912ac10 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f014e400640 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99117ab0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0140007b70 tx=0x7f01400015d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d741f433790a5ead server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 1 -- 172.21.15.86:0/24259364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01400050d0 con 0x55fa9912ac10 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 1 -- 172.21.15.86:0/24259364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f0140027dc0 con 0x55fa9912ac10 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 -- 172.21.15.86:0/24259364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0140018070 con 0x55fa9912ac10 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 1 -- 172.21.15.86:0/24259364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f014002e050 con 0x55fa9912ac10 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 10 client.7923 fetch_fsmap learned FSMap version 71 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 -- 172.21.15.86:0/24259364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55fa98eeb5a0 con 0x55fa9912ac10 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 1 -- 172.21.15.86:0/24259364 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140018070 con 0x55fa9912ac10 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 1 -- 172.21.15.86:0/24259364 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f0140031020 con 0x55fa9912ac10 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 10 client.7923 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 10 client.7923 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 20 client.7923 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55fa99139850 0x55fa9913bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 4 client.7923 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 -- 172.21.15.86:0/24259364 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa98ecae40 con 0x55fa99139850 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f014da00640 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55fa99139850 0x55fa9913bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f014da00640 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55fa99139850 0x55fa9913bc40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d7eeda19a4b0726 server_cookie=f6046dc6e1785f03 in_seq=0 out_seq=0 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 10 client.7923 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 1 -- 172.21.15.86:0/24259364 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fa98ecae40 con 0x55fa99139850 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 10 client.7923 handle_command_reply: tid=0 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 client.7923 shutdown 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 -- 172.21.15.86:0/24259364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 msgr2=0x55fa99117ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99117ab0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0140007b70 tx=0x7f01400015d0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 -- 172.21.15.86:0/24259364 shutdown_connections 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55fa99139850 0x55fa9913bc40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99117ab0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa9912a2f0 0x55fa99117570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f012ea00640 0 client.7923 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:18:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 --2- 172.21.15.86:0/24259364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa990e0d60 0x55fa990e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 -- 172.21.15.86:0/24259364 >> 172.21.15.86:0/24259364 conn(0x55fa990dcd10 msgr2=0x55fa990df990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 -- 172.21.15.86:0/24259364 shutdown_connections 2024-03-22T04:18:18.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 1 -- 172.21.15.86:0/24259364 wait complete. 2024-03-22T04:18:18.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.345+0000 7f0153980000 20 client.7923 trim_cache size 0 max 16384 2024-03-22T04:18:18.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 Processor -- start 2024-03-22T04:18:18.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- start start 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 0x55fa99194b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa991950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99195610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0148202ea0 con 0x55fa9912ac10 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0148202210 con 0x55fa9912a2f0 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f01482027d0 con 0x55fa990e0d60 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55032/0 (socket says 172.21.15.86:55032) 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 -- 172.21.15.86:0/3880264581 learned_addr learned my addr 172.21.15.86:0/3880264581 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014d000640 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa991950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014da00640 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 0x55fa99194b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 -- 172.21.15.86:0/3880264581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 msgr2=0x55fa99194b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:18.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 0x55fa99194b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 -- 172.21.15.86:0/3880264581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 msgr2=0x55fa991950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa991950d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 -- 172.21.15.86:0/3880264581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa98eeba50 con 0x55fa9912ac10 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f014e400640 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99195610 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0140001230 tx=0x7f014001f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf3dd8d7ee13791 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0139600640 1 -- 172.21.15.86:0/3880264581 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140037070 con 0x55fa9912ac10 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0139600640 1 -- 172.21.15.86:0/3880264581 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140018040 con 0x55fa9912ac10 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0139600640 1 -- 172.21.15.86:0/3880264581 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140004ba0 con 0x55fa9912ac10 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- 172.21.15.86:0/3880264581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 msgr2=0x55fa99195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99195610 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0140001230 tx=0x7f014001f070 comp rx=0 tx=0).stop 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- 172.21.15.86:0/3880264581 shutdown_connections 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912a2f0 0x55fa991950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912ac10 0x55fa99195610 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 --2- 172.21.15.86:0/3880264581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 0x55fa99194b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- 172.21.15.86:0/3880264581 >> 172.21.15.86:0/3880264581 conn(0x55fa990dcd10 msgr2=0x55fa990df100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- 172.21.15.86:0/3880264581 shutdown_connections 2024-03-22T04:18:18.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.361+0000 7f0153980000 1 -- 172.21.15.86:0/3880264581 wait complete. 2024-03-22T04:18:18.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 -- start start 2024-03-22T04:18:18.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 Processor -- start 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 0x55fa9913df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912a2f0 0x55fa9913e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912ac10 0x55fa99195a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa98efdc20 con 0x55fa9912a2f0 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fa98f3c490 con 0x55fa9912ac10 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55fa98eca2a0 con 0x55fa990e0d60 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 10 client.0 _ll_register_callbacks cb 0x7f013c0ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912a2f0 0x55fa9913e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912a2f0 0x55fa9913e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55036/0 (socket says 172.21.15.86:55036) 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 -- 172.21.15.86:0/3423949597 learned_addr learned my addr 172.21.15.86:0/3423949597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014da00640 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 0x55fa9913df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014e400640 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912ac10 0x55fa99195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 -- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 msgr2=0x55fa9913df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 0x55fa9913df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 -- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912ac10 msgr2=0x55fa99195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912ac10 0x55fa99195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 -- 172.21.15.86:0/3423949597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa98fb0840 con 0x55fa9912a2f0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014d000640 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912a2f0 0x55fa9913e4a0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0134055580 tx=0x7f013401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249cad76420e2ef6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f012ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f012ea00640 1 -- 172.21.15.86:0/3423949597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340833f0 con 0x55fa9912a2f0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f012ea00640 1 -- 172.21.15.86:0/3423949597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f0134007490 con 0x55fa9912a2f0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 -- 172.21.15.86:0/3423949597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0140018040 con 0x55fa9912a2f0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f012ea00640 1 -- 172.21.15.86:0/3423949597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0134007690 con 0x55fa9912a2f0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 10 client.7929 fetch_fsmap learned FSMap version 71 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 -- 172.21.15.86:0/3423949597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55fa98ed63e0 con 0x55fa9912a2f0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f012ea00640 1 -- 172.21.15.86:0/3423949597 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140018040 con 0x55fa9912a2f0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f012ea00640 1 -- 172.21.15.86:0/3423949597 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 71) ==== 3494+0+0 (secure 0 0 0) 0x7f0134054a40 con 0x55fa9912a2f0 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 10 client.7929 fetch_fsmap finished waiting for FSMap version 71 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 10 client.7929 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:18:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 20 client.7929 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55fa99195fb0 0x55fa99196390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 4 client.7929 mds_command: new command op to 76604 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f0153980000 1 -- 172.21.15.86:0/3423949597 --> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55fa98f3d2c0 con 0x55fa99195fb0 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014da00640 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55fa99195fb0 0x55fa99196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f014da00640 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55fa99195fb0 0x55fa99196390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd003e6ec207a825 server_cookie=2a9a513e482ee4eb in_seq=0 out_seq=0 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.365+0000 7f012ea00640 10 client.7929 ms_handle_connect on v2:172.21.15.196:6834/1851339627 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f012ea00640 1 -- 172.21.15.86:0/3423949597 <== mds.0 v2:172.21.15.196:6834/1851339627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x55fa98f3d2c0 con 0x55fa99195fb0 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f012ea00640 10 client.7929 handle_command_reply: tid=0 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 client.7929 shutdown 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 -- 172.21.15.86:0/3423949597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912a2f0 msgr2=0x55fa9913e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912a2f0 0x55fa9913e4a0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0134055580 tx=0x7f013401e040 comp rx=0 tx=0).stop 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 -- 172.21.15.86:0/3423949597 shutdown_connections 2024-03-22T04:18:18.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:6834/1851339627,v1:172.21.15.196:6835/1851339627] conn(0x55fa99195fb0 0x55fa99196390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa9912ac10 0x55fa99195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa9912a2f0 0x55fa9913e4a0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 --2- 172.21.15.86:0/3423949597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa990e0d60 0x55fa9913df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 -- 172.21.15.86:0/3423949597 >> 172.21.15.86:0/3423949597 conn(0x55fa990dcd10 msgr2=0x55fa990df990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f012ea00640 0 client.7929 ms_handle_reset on v2:172.21.15.196:6834/1851339627 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 -- 172.21.15.86:0/3423949597 shutdown_connections 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 1 -- 172.21.15.86:0/3423949597 wait complete. 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0153980000 20 client.7929 trim_cache size 0 max 16384 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-22T04:18:18.372 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 2646213554, 2024-03-22T04:18:18.373 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-03-22T04:18:18.373 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/testdir/hardlink" 2024-03-22T04:18:18.373 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:18:18.373 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:18:18.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 -- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0128067b20 msgr2=0x7f0128069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0128067b20 0x7f0128069fe0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f013c0ad000 tx=0x7f013c0a6000 comp rx=0 tx=0).stop 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 -- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01480b3610 msgr2=0x7f01481597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01480b3610 0x7f01481597c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f013400c3c0 tx=0x7f013400c880 comp rx=0 tx=0).stop 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 -- 172.21.15.86:0/3850508999 shutdown_connections 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0128067b20 0x7f0128069fe0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f014815c730 0x7f0148159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01480b3610 0x7f01481597c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 --2- 172.21.15.86:0/3850508999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01480b11c0 0x7f0148153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 -- 172.21.15.86:0/3850508999 >> 172.21.15.86:0/3850508999 conn(0x7f01480b3bf0 msgr2=0x7f014806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 -- 172.21.15.86:0/3850508999 shutdown_connections 2024-03-22T04:18:18.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.369+0000 7f0150400640 1 -- 172.21.15.86:0/3850508999 wait complete. 2024-03-22T04:18:18.387 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:18:18.387 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:18:18.396 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:18:18.396 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:18:18.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:18:18.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.625+0000 7f0377400640 1 Processor -- start 2024-03-22T04:18:18.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.625+0000 7f0377400640 1 -- start start 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0377400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0370147e10 0x7f03700b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0377400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03700b2bd0 con 0x7f0370162710 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0377400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03700b2610 con 0x7f037006c270 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0377400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f03700b28f0 con 0x7f0370147e10 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0374a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0370147e10 0x7f03700b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0375400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0370162710 0x7f037015fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45926/0 (socket says 172.21.15.86:45926) 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0375400640 1 -- 172.21.15.86:0/2703634349 learned_addr learned my addr 172.21.15.86:0/2703634349 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0374a00640 1 -- 172.21.15.86:0/2703634349 >> v1:172.21.15.196:6789/0 conn(0x7f037006c270 legacy=0x7f0370162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0374a00640 1 -- 172.21.15.86:0/2703634349 >> v1:172.21.15.86:6789/0 conn(0x7f0370162710 legacy=0x7f037015fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f0374a00640 1 -- 172.21.15.86:0/2703634349 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03700b5ce0 con 0x7f0370147e10 2024-03-22T04:18:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.629+0000 7f036f400640 1 -- 172.21.15.86:0/2703634349 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571717637 0 0) 0x7f03700b2bd0 con 0x7f0370162710 2024-03-22T04:18:18.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0374a00640 1 --2- 172.21.15.86:0/2703634349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0370147e10 0x7f03700b4100 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f03600097c0 tx=0x7f036001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbb3eb78938b9c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f036f400640 1 -- 172.21.15.86:0/2703634349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03600b1070 con 0x7f0370147e10 2024-03-22T04:18:18.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f036f400640 1 -- 172.21.15.86:0/2703634349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03600ac410 con 0x7f0370147e10 2024-03-22T04:18:18.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f036f400640 1 -- 172.21.15.86:0/2703634349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03600ac7e0 con 0x7f0370147e10 2024-03-22T04:18:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 -- 172.21.15.86:0/2703634349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0370147e10 msgr2=0x7f03700b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 --2- 172.21.15.86:0/2703634349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0370147e10 0x7f03700b4100 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f03600097c0 tx=0x7f036001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 -- 172.21.15.86:0/2703634349 shutdown_connections 2024-03-22T04:18:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 --2- 172.21.15.86:0/2703634349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0370147e10 0x7f03700b4100 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 -- 172.21.15.86:0/2703634349 >> 172.21.15.86:0/2703634349 conn(0x7f03700b6620 msgr2=0x7f03700b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 -- 172.21.15.86:0/2703634349 shutdown_connections 2024-03-22T04:18:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 -- 172.21.15.86:0/2703634349 wait complete. 2024-03-22T04:18:18.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 Processor -- start 2024-03-22T04:18:18.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.633+0000 7f0377400640 1 -- start start 2024-03-22T04:18:18.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037006c270 0x7f037015e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0370147e10 0x7f03700ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0370162710 0x7f03700b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f037016c160 con 0x7f037006c270 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0370160d10 con 0x7f0370147e10 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f037016c2e0 con 0x7f0370162710 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037006c270 0x7f037015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037006c270 0x7f037015e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55054/0 (socket says 172.21.15.86:55054) 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 -- 172.21.15.86:0/2280725067 learned_addr learned my addr 172.21.15.86:0/2280725067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f036fe00640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0370147e10 0x7f03700ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0375400640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0370162710 0x7f03700b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 -- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0370162710 msgr2=0x7f03700b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0370162710 0x7f03700b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 -- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0370147e10 msgr2=0x7f03700ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0370147e10 0x7f03700ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 -- 172.21.15.86:0/2280725067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03700b0a80 con 0x7f037006c270 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0375400640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0370162710 0x7f03700b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0374a00640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037006c270 0x7f037015e910 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f03600190b0 tx=0x7f0360019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f036cc00640 1 -- 172.21.15.86:0/2280725067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03600b1040 con 0x7f037006c270 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f036cc00640 1 -- 172.21.15.86:0/2280725067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03600bb070 con 0x7f037006c270 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03700b0d60 con 0x7f037006c270 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f036cc00640 1 -- 172.21.15.86:0/2280725067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03600b5630 con 0x7f037006c270 2024-03-22T04:18:18.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03700b1320 con 0x7f037006c270 2024-03-22T04:18:18.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0370068650 con 0x7f037006c270 2024-03-22T04:18:18.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.637+0000 7f036cc00640 1 -- 172.21.15.86:0/2280725067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f03600b57d0 con 0x7f037006c270 2024-03-22T04:18:18.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.641+0000 7f036cc00640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0350067b50 0x7f035006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.641+0000 7f036cc00640 1 -- 172.21.15.86:0/2280725067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f036011fc20 con 0x7f037006c270 2024-03-22T04:18:18.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.641+0000 7f036fe00640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0350067b50 0x7f035006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.641+0000 7f036fe00640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0350067b50 0x7f035006a010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f035c007fd0 tx=0x7f035c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.645+0000 7f036cc00640 1 -- 172.21.15.86:0/2280725067 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0360016090 con 0x7f037006c270 2024-03-22T04:18:18.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.805+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03700ca900 con 0x7f037006c270 2024-03-22T04:18:18.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.805+0000 7f036cc00640 1 -- 172.21.15.86:0/2280725067 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+17497 (secure 0 0 0) 0x7f03600f02d0 con 0x7f037006c270 2024-03-22T04:18:18.806 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:18.806 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":96,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:17:16.972499+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.32","mappings":[{"from":7,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":3}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]},{"pgid":"14.3c","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3e","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1097103199":"2024-03-23T04:17:08.939658+0000","172.21.15.196:6834/1097103199":"2024-03-23T04:17:08.939658+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0350067b50 msgr2=0x7f035006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0350067b50 0x7f035006a010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f035c007fd0 tx=0x7f035c009040 comp rx=0 tx=0).stop 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037006c270 msgr2=0x7f037015e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037006c270 0x7f037015e910 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f03600190b0 tx=0x7f0360019240 comp rx=0 tx=0).stop 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 shutdown_connections 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0350067b50 0x7f035006a010 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0370147e10 0x7f03700ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0370162710 0x7f03700b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 --2- 172.21.15.86:0/2280725067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037006c270 0x7f037015e910 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 >> 172.21.15.86:0/2280725067 conn(0x7f03700b6620 msgr2=0x7f03701695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 shutdown_connections 2024-03-22T04:18:18.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.809+0000 7f0377400640 1 -- 172.21.15.86:0/2280725067 wait complete. 2024-03-22T04:18:18.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:16:56.341603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:16:56.977675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:18:18.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:18:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.965+0000 7fc8d8800640 1 Processor -- start 2024-03-22T04:18:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.965+0000 7fc8d8800640 1 -- start start 2024-03-22T04:18:18.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d8800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d00b2360 0x7fc8d00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d8800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d015d710 con 0x7fc8d00b3430 2024-03-22T04:18:18.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d8800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d015fe40 con 0x7fc8d015d0b0 2024-03-22T04:18:18.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d8800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d00b3890 con 0x7fc8d00b2360 2024-03-22T04:18:18.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc8d00b3430 0x7fc8d015c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45942/0 (socket says 172.21.15.86:45942) 2024-03-22T04:18:18.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d5400640 1 -- 172.21.15.86:0/1243404771 learned_addr learned my addr 172.21.15.86:0/1243404771 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:18.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d5e00640 1 --2- 172.21.15.86:0/1243404771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d00b2360 0x7fc8d00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d4a00640 1 -- 172.21.15.86:0/1243404771 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3113659872 0 0) 0x7fc8d015d710 con 0x7fc8d00b3430 2024-03-22T04:18:18.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d4a00640 1 -- 172.21.15.86:0/1243404771 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8b8003530 con 0x7fc8d00b3430 2024-03-22T04:18:18.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d4a00640 1 -- 172.21.15.86:0/1243404771 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383834353 0 0) 0x7fc8d015fe40 con 0x7fc8d015d0b0 2024-03-22T04:18:18.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d4a00640 1 -- 172.21.15.86:0/1243404771 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8d015d710 con 0x7fc8d015d0b0 2024-03-22T04:18:18.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d5e00640 1 -- 172.21.15.86:0/1243404771 >> v1:172.21.15.196:6789/0 conn(0x7fc8d015d0b0 legacy=0x7fc8d00b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d5e00640 1 -- 172.21.15.86:0/1243404771 >> v1:172.21.15.86:6789/0 conn(0x7fc8d00b3430 legacy=0x7fc8d015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.969+0000 7fc8d5e00640 1 -- 172.21.15.86:0/1243404771 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d0160120 con 0x7fc8d00b2360 2024-03-22T04:18:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d5e00640 1 --2- 172.21.15.86:0/1243404771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d00b2360 0x7fc8d00b2ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40097f0 tx=0x7fc8c401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53d0d14f95798c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d4a00640 1 -- 172.21.15.86:0/1243404771 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40b0070 con 0x7fc8d00b2360 2024-03-22T04:18:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d4a00640 1 -- 172.21.15.86:0/1243404771 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c40ab600 con 0x7fc8d00b2360 2024-03-22T04:18:18.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d4a00640 1 -- 172.21.15.86:0/1243404771 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40ab9d0 con 0x7fc8d00b2360 2024-03-22T04:18:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- 172.21.15.86:0/1243404771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d00b2360 msgr2=0x7fc8d00b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 --2- 172.21.15.86:0/1243404771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d00b2360 0x7fc8d00b2ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40097f0 tx=0x7fc8c401c4e0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- 172.21.15.86:0/1243404771 shutdown_connections 2024-03-22T04:18:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 --2- 172.21.15.86:0/1243404771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d00b2360 0x7fc8d00b2ef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- 172.21.15.86:0/1243404771 >> 172.21.15.86:0/1243404771 conn(0x7fc8d00cc940 msgr2=0x7fc8d00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- 172.21.15.86:0/1243404771 shutdown_connections 2024-03-22T04:18:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- 172.21.15.86:0/1243404771 wait complete. 2024-03-22T04:18:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 Processor -- start 2024-03-22T04:18:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- start start 2024-03-22T04:18:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d00b2360 0x7fc8d00ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d00b3430 0x7fc8d015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d015d0b0 0x7fc8d015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d016a370 con 0x7fc8d00b2360 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d016a1f0 con 0x7fc8d00b3430 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d8800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d016a4f0 con 0x7fc8d015d0b0 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d00b2360 0x7fc8d00ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d00b2360 0x7fc8d00ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55056/0 (socket says 172.21.15.86:55056) 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d5e00640 1 -- 172.21.15.86:0/1690987296 learned_addr learned my addr 172.21.15.86:0/1690987296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d5400640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d00b3430 0x7fc8d015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.973+0000 7fc8d6800640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d015d0b0 0x7fc8d015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d5e00640 1 -- 172.21.15.86:0/1690987296 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d015d0b0 msgr2=0x7fc8d015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d5e00640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d015d0b0 0x7fc8d015be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d5e00640 1 -- 172.21.15.86:0/1690987296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d00b3430 msgr2=0x7fc8d015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d5e00640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d00b3430 0x7fc8d015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d5e00640 1 -- 172.21.15.86:0/1690987296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d00a9b00 con 0x7fc8d00b2360 2024-03-22T04:18:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d5e00640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d00b2360 0x7fc8d00ac040 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40097f0 tx=0x7fc8c401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8ca000640 1 -- 172.21.15.86:0/1690987296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40b0050 con 0x7fc8d00b2360 2024-03-22T04:18:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8ca000640 1 -- 172.21.15.86:0/1690987296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c40bc070 con 0x7fc8d00b2360 2024-03-22T04:18:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d00a9de0 con 0x7fc8d00b2360 2024-03-22T04:18:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8ca000640 1 -- 172.21.15.86:0/1690987296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4006e20 con 0x7fc8d00b2360 2024-03-22T04:18:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d00aa3a0 con 0x7fc8d00b2360 2024-03-22T04:18:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d0074470 con 0x7fc8d00b2360 2024-03-22T04:18:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8ca000640 1 -- 172.21.15.86:0/1690987296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc8c40b5870 con 0x7fc8d00b2360 2024-03-22T04:18:18.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8ca000640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8b4067b20 0x7fc8b4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:18.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8ca000640 1 -- 172.21.15.86:0/1690987296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fc8c4120680 con 0x7fc8d00b2360 2024-03-22T04:18:18.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.977+0000 7fc8d5400640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8b4067b20 0x7fc8b4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:18.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.981+0000 7fc8d5400640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8b4067b20 0x7fc8b4069fe0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b53f0 tx=0x7fc8c0007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:18.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:18.981+0000 7fc8ca000640 1 -- 172.21.15.86:0/1690987296 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc8c40160d0 con 0x7fc8d00b2360 2024-03-22T04:18:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8d00af3a0 con 0x7fc8d00b2360 2024-03-22T04:18:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8ca000640 1 -- 172.21.15.86:0/1690987296 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fc8c40f0770 con 0x7fc8d00b2360 2024-03-22T04:18:19.137 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8b4067b20 msgr2=0x7fc8b4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8b4067b20 0x7fc8b4069fe0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b53f0 tx=0x7fc8c0007d80 comp rx=0 tx=0).stop 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d00b2360 msgr2=0x7fc8d00ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d00b2360 0x7fc8d00ac040 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40097f0 tx=0x7fc8c401ad20 comp rx=0 tx=0).stop 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 shutdown_connections 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc8b4067b20 0x7fc8b4069fe0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d00b3430 0x7fc8d015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8d015d0b0 0x7fc8d015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 --2- 172.21.15.86:0/1690987296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d00b2360 0x7fc8d00ac040 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 >> 172.21.15.86:0/1690987296 conn(0x7fc8d00cc940 msgr2=0x7fc8d00b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:19.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 shutdown_connections 2024-03-22T04:18:19.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.137+0000 7fc8d8800640 1 -- 172.21.15.86:0/1690987296 wait complete. 2024-03-22T04:18:19.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:18:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce84400640 1 Processor -- start 2024-03-22T04:18:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce84400640 1 -- start start 2024-03-22T04:18:19.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce84400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fce7c0b1b80 0x7fce7c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce84400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0cdf10 0x7fce7c0ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce84400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce7c06d6d0 con 0x7fce7c144a90 2024-03-22T04:18:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce84400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fce7c144e70 con 0x7fce7c0b1b80 2024-03-22T04:18:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce84400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce7c06d970 con 0x7fce7c0cdf10 2024-03-22T04:18:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce81a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fce7c144a90 0x7fce7c15f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45950/0 (socket says 172.21.15.86:45950) 2024-03-22T04:18:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce81a00640 1 -- 172.21.15.86:0/2708472029 learned_addr learned my addr 172.21.15.86:0/2708472029 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce81000640 1 --2- 172.21.15.86:0/2708472029 >> v2:172.21.15.196:3300/0 conn(0x7fce7c0b1b80 0x7fce7c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce82400640 1 --2- 172.21.15.86:0/2708472029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0cdf10 0x7fce7c0ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce7be00640 1 -- 172.21.15.86:0/2708472029 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2323442081 0 0) 0x7fce7c06d6d0 con 0x7fce7c144a90 2024-03-22T04:18:19.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce7be00640 1 -- 172.21.15.86:0/2708472029 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce64003530 con 0x7fce7c144a90 2024-03-22T04:18:19.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce82400640 1 -- 172.21.15.86:0/2708472029 >> v2:172.21.15.196:3300/0 conn(0x7fce7c0b1b80 msgr2=0x7fce7c0b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:19.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce82400640 1 --2- 172.21.15.86:0/2708472029 >> v2:172.21.15.196:3300/0 conn(0x7fce7c0b1b80 0x7fce7c0b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce82400640 1 -- 172.21.15.86:0/2708472029 >> v1:172.21.15.86:6789/0 conn(0x7fce7c144a90 legacy=0x7fce7c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce82400640 1 -- 172.21.15.86:0/2708472029 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c06dc50 con 0x7fce7c0cdf10 2024-03-22T04:18:19.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.297+0000 7fce81000640 1 --2- 172.21.15.86:0/2708472029 >> v2:172.21.15.196:3300/0 conn(0x7fce7c0b1b80 0x7fce7c0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce82400640 1 --2- 172.21.15.86:0/2708472029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0cdf10 0x7fce7c0ce360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce74009d90 tx=0x7fce7401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174ea1b8797c8264 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce7be00640 1 -- 172.21.15.86:0/2708472029 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce740b2070 con 0x7fce7c0cdf10 2024-03-22T04:18:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce7be00640 1 -- 172.21.15.86:0/2708472029 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce740a5920 con 0x7fce7c0cdf10 2024-03-22T04:18:19.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce7be00640 1 -- 172.21.15.86:0/2708472029 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce740a5cf0 con 0x7fce7c0cdf10 2024-03-22T04:18:19.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce84400640 1 -- 172.21.15.86:0/2708472029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0cdf10 msgr2=0x7fce7c0ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce84400640 1 --2- 172.21.15.86:0/2708472029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0cdf10 0x7fce7c0ce360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce74009d90 tx=0x7fce7401c970 comp rx=0 tx=0).stop 2024-03-22T04:18:19.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce84400640 1 -- 172.21.15.86:0/2708472029 shutdown_connections 2024-03-22T04:18:19.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce84400640 1 --2- 172.21.15.86:0/2708472029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0cdf10 0x7fce7c0ce360 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce84400640 1 --2- 172.21.15.86:0/2708472029 >> v2:172.21.15.196:3300/0 conn(0x7fce7c0b1b80 0x7fce7c0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce84400640 1 -- 172.21.15.86:0/2708472029 >> 172.21.15.86:0/2708472029 conn(0x7fce7c0cc940 msgr2=0x7fce7c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:19.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce84400640 1 -- 172.21.15.86:0/2708472029 shutdown_connections 2024-03-22T04:18:19.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.301+0000 7fce84400640 1 -- 172.21.15.86:0/2708472029 wait complete. 2024-03-22T04:18:19.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 Processor -- start 2024-03-22T04:18:19.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 -- start start 2024-03-22T04:18:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0b1b80 0x7fce7c1419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce7c0cdf10 0x7fce7c1f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c144a90 0x7fce7c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c0add70 con 0x7fce7c0cdf10 2024-03-22T04:18:19.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c0adbf0 con 0x7fce7c144a90 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce7c16b480 con 0x7fce7c0b1b80 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce7c0cdf10 0x7fce7c1f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce7c0cdf10 0x7fce7c1f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55072/0 (socket says 172.21.15.86:55072) 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 -- 172.21.15.86:0/3854090084 learned_addr learned my addr 172.21.15.86:0/3854090084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81a00640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0b1b80 0x7fce7c1419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce82400640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c144a90 0x7fce7c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 -- 172.21.15.86:0/3854090084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0b1b80 msgr2=0x7fce7c1419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0b1b80 0x7fce7c1419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 -- 172.21.15.86:0/3854090084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c144a90 msgr2=0x7fce7c0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c144a90 0x7fce7c0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 -- 172.21.15.86:0/3854090084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c155ba0 con 0x7fce7c0cdf10 2024-03-22T04:18:19.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce81000640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce7c0cdf10 0x7fce7c1f7920 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fce6800e980 tx=0x7fce6800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce79600640 1 -- 172.21.15.86:0/3854090084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68017070 con 0x7fce7c0cdf10 2024-03-22T04:18:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7c1f7e60 con 0x7fce7c0cdf10 2024-03-22T04:18:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce79600640 1 -- 172.21.15.86:0/3854090084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce6800ca00 con 0x7fce7c0cdf10 2024-03-22T04:18:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce79600640 1 -- 172.21.15.86:0/3854090084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6800cdd0 con 0x7fce7c0cdf10 2024-03-22T04:18:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7c159cb0 con 0x7fce7c0cdf10 2024-03-22T04:18:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.305+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce7c074470 con 0x7fce7c0cdf10 2024-03-22T04:18:19.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.309+0000 7fce79600640 1 -- 172.21.15.86:0/3854090084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fce6801a070 con 0x7fce7c0cdf10 2024-03-22T04:18:19.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.309+0000 7fce79600640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce5c067b20 0x7fce5c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.309+0000 7fce79600640 1 -- 172.21.15.86:0/3854090084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fce680860d0 con 0x7fce7c0cdf10 2024-03-22T04:18:19.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.309+0000 7fce81a00640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce5c067b20 0x7fce5c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.309+0000 7fce81a00640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce5c067b20 0x7fce5c069fe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.313+0000 7fce79600640 1 -- 172.21.15.86:0/3854090084 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fce7c155ba0 con 0x7fce7c0cdf10 2024-03-22T04:18:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.465+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fce7c06ef80 con 0x7fce7c0cdf10 2024-03-22T04:18:19.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.465+0000 7fce79600640 1 -- 172.21.15.86:0/3854090084 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7fce68056140 con 0x7fce7c0cdf10 2024-03-22T04:18:19.466 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:18:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce5c067b20 msgr2=0x7fce5c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce5c067b20 0x7fce5c069fe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).stop 2024-03-22T04:18:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce7c0cdf10 msgr2=0x7fce7c1f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce7c0cdf10 0x7fce7c1f7920 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fce6800e980 tx=0x7fce6800ee40 comp rx=0 tx=0).stop 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 shutdown_connections 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fce5c067b20 0x7fce5c069fe0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c144a90 0x7fce7c0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce7c0cdf10 0x7fce7c1f7920 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 --2- 172.21.15.86:0/3854090084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce7c0b1b80 0x7fce7c1419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 >> 172.21.15.86:0/3854090084 conn(0x7fce7c0cc940 msgr2=0x7fce7c0afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 shutdown_connections 2024-03-22T04:18:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.469+0000 7fce84400640 1 -- 172.21.15.86:0/3854090084 wait complete. 2024-03-22T04:18:19.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:18:19.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.625+0000 7f134d000640 1 Processor -- start 2024-03-22T04:18:19.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.625+0000 7f134d000640 1 -- start start 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f134d000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f134815ea00 0x7f134806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f134d000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480cf310 0x7f13480ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f134d000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13480cc670 con 0x7f13480cecb0 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f134d000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f13480cf110 con 0x7f134815ea00 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f134d000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f134806e120 con 0x7f13480cf310 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f1347400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f134815ea00 0x7f134806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f133ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480cf310 0x7f13480ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f1347e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f13480cecb0 0x7f13480cfdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:56236/0 (socket says 172.21.15.86:56236) 2024-03-22T04:18:19.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f1347e00640 1 -- 172.21.15.86:0/2354462135 learned_addr learned my addr 172.21.15.86:0/2354462135 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f1347400640 1 -- 172.21.15.86:0/2354462135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480cf310 msgr2=0x7f13480ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f1347400640 1 --2- 172.21.15.86:0/2354462135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480cf310 0x7f13480ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f1347400640 1 -- 172.21.15.86:0/2354462135 >> v1:172.21.15.196:6789/0 conn(0x7f13480cecb0 legacy=0x7f13480cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.629+0000 7f1347400640 1 -- 172.21.15.86:0/2354462135 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13480cc950 con 0x7f134815ea00 2024-03-22T04:18:19.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f1347400640 1 --2- 172.21.15.86:0/2354462135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f134815ea00 0x7f134806d7d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1338009dc0 tx=0x7f133801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0d6679df502e251 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f1346a00640 1 -- 172.21.15.86:0/2354462135 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380b2070 con 0x7f134815ea00 2024-03-22T04:18:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f1346a00640 1 -- 172.21.15.86:0/2354462135 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13380a5930 con 0x7f134815ea00 2024-03-22T04:18:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f1346a00640 1 -- 172.21.15.86:0/2354462135 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380a5d00 con 0x7f134815ea00 2024-03-22T04:18:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- 172.21.15.86:0/2354462135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f134815ea00 msgr2=0x7f134806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 --2- 172.21.15.86:0/2354462135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f134815ea00 0x7f134806d7d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1338009dc0 tx=0x7f133801c950 comp rx=0 tx=0).stop 2024-03-22T04:18:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- 172.21.15.86:0/2354462135 shutdown_connections 2024-03-22T04:18:19.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 --2- 172.21.15.86:0/2354462135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480cf310 0x7f13480ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 --2- 172.21.15.86:0/2354462135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f134815ea00 0x7f134806d7d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- 172.21.15.86:0/2354462135 >> 172.21.15.86:0/2354462135 conn(0x7f13480d1ef0 msgr2=0x7f13480d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- 172.21.15.86:0/2354462135 shutdown_connections 2024-03-22T04:18:19.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- 172.21.15.86:0/2354462135 wait complete. 2024-03-22T04:18:19.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 Processor -- start 2024-03-22T04:18:19.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- start start 2024-03-22T04:18:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13480cecb0 0x7f13480cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13480cf310 0x7f1348155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480c99d0 0x7f13480b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13480aed60 con 0x7f13480cecb0 2024-03-22T04:18:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13480aebe0 con 0x7f13480cf310 2024-03-22T04:18:19.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.633+0000 7f134d000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f134816b3b0 con 0x7f13480c99d0 2024-03-22T04:18:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13480cecb0 0x7f13480cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480c99d0 0x7f13480b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13480cecb0 0x7f13480cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55088/0 (socket says 172.21.15.86:55088) 2024-03-22T04:18:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 -- 172.21.15.86:0/3819516547 learned_addr learned my addr 172.21.15.86:0/3819516547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:19.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f133ea00640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13480cf310 0x7f1348155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 -- 172.21.15.86:0/3819516547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480c99d0 msgr2=0x7f13480b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480c99d0 0x7f13480b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 -- 172.21.15.86:0/3819516547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13480cf310 msgr2=0x7f1348155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13480cf310 0x7f1348155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 -- 172.21.15.86:0/3819516547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1348155ba0 con 0x7f13480cecb0 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347e00640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480c99d0 0x7f13480b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f1347400640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13480cecb0 0x7f13480cb830 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f133801a580 tx=0x7f13380196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f133fe00640 1 -- 172.21.15.86:0/3819516547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380b2040 con 0x7f13480cecb0 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f133fe00640 1 -- 172.21.15.86:0/3819516547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13380bf070 con 0x7f13480cecb0 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f133fe00640 1 -- 172.21.15.86:0/3819516547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380b8870 con 0x7f13480cecb0 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13480ca790 con 0x7f13480cecb0 2024-03-22T04:18:19.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13480cad50 con 0x7f13480cecb0 2024-03-22T04:18:19.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134804f900 con 0x7f13480cecb0 2024-03-22T04:18:19.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f133fe00640 1 -- 172.21.15.86:0/3819516547 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f13380a57b0 con 0x7f13480cecb0 2024-03-22T04:18:19.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f133fe00640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1324067b20 0x7f1324069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.637+0000 7f133fe00640 1 -- 172.21.15.86:0/3819516547 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f13381210b0 con 0x7f13480cecb0 2024-03-22T04:18:19.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.641+0000 7f133ea00640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1324067b20 0x7f1324069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.641+0000 7f133ea00640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1324067b20 0x7f1324069fe0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1348169060 tx=0x7f1330009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.641+0000 7f133fe00640 1 -- 172.21.15.86:0/3819516547 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1338016540 con 0x7f13480cecb0 2024-03-22T04:18:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1348159000 con 0x7f13480cecb0 2024-03-22T04:18:19.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f133fe00640 1 -- 172.21.15.86:0/3819516547 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f13380f1110 con 0x7f13480cecb0 2024-03-22T04:18:19.797 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:18:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1324067b20 msgr2=0x7f1324069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1324067b20 0x7f1324069fe0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1348169060 tx=0x7f1330009190 comp rx=0 tx=0).stop 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13480cecb0 msgr2=0x7f13480cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13480cecb0 0x7f13480cb830 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f133801a580 tx=0x7f13380196f0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 shutdown_connections 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1324067b20 0x7f1324069fe0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13480cf310 0x7f1348155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13480c99d0 0x7f13480b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 --2- 172.21.15.86:0/3819516547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13480cecb0 0x7f13480cb830 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.797+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 >> 172.21.15.86:0/3819516547 conn(0x7f13480d1ef0 msgr2=0x7f134815d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.801+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 shutdown_connections 2024-03-22T04:18:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.801+0000 7f134d000640 1 -- 172.21.15.86:0/3819516547 wait complete. 2024-03-22T04:18:19.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:18:19.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2d2000640 1 Processor -- start 2024-03-22T04:18:19.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2d2000640 1 -- start start 2024-03-22T04:18:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2d2000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc147d10 0x7fa2cc0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2d2000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2cc0b3e00 0x7fa2cc06fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2d2000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2cc1621c0 con 0x7fa2cc06ff40 2024-03-22T04:18:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2d2000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa2cc1624a0 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2d2000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa2cc162780 con 0x7fa2cc147d10 2024-03-22T04:18:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2c2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2cc0b3e00 0x7fa2cc06fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2cb400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc147d10 0x7fa2cc0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2cbe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa2cc06ff40 0x7fa2cc0b5090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:56238/0 (socket says 172.21.15.86:56238) 2024-03-22T04:18:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2cbe00640 1 -- 172.21.15.86:0/1283903700 learned_addr learned my addr 172.21.15.86:0/1283903700 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2c2a00640 1 -- 172.21.15.86:0/1283903700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc147d10 msgr2=0x7fa2cc0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2c2a00640 1 --2- 172.21.15.86:0/1283903700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc147d10 0x7fa2cc0b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2c2a00640 1 -- 172.21.15.86:0/1283903700 >> v1:172.21.15.196:6789/0 conn(0x7fa2cc06ff40 legacy=0x7fa2cc0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.957+0000 7fa2c2a00640 1 -- 172.21.15.86:0/1283903700 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc162a60 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.961+0000 7fa2c2a00640 1 --2- 172.21.15.86:0/1283903700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc06fa00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40097c0 tx=0x7fa2b401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1272557256c69f2c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2caa00640 1 -- 172.21.15.86:0/1283903700 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40b1070 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2caa00640 1 -- 172.21.15.86:0/1283903700 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b40ac410 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2caa00640 1 -- 172.21.15.86:0/1283903700 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40ac7e0 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- 172.21.15.86:0/1283903700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc0b3e00 msgr2=0x7fa2cc06fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 --2- 172.21.15.86:0/1283903700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc06fa00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40097c0 tx=0x7fa2b401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- 172.21.15.86:0/1283903700 shutdown_connections 2024-03-22T04:18:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 --2- 172.21.15.86:0/1283903700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc06fa00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 --2- 172.21.15.86:0/1283903700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc147d10 0x7fa2cc0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- 172.21.15.86:0/1283903700 >> 172.21.15.86:0/1283903700 conn(0x7fa2cc0b75b0 msgr2=0x7fa2cc0b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- 172.21.15.86:0/1283903700 shutdown_connections 2024-03-22T04:18:19.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- 172.21.15.86:0/1283903700 wait complete. 2024-03-22T04:18:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 Processor -- start 2024-03-22T04:18:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- start start 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc06ff40 0x7fa2cc13f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc147d10 0x7fa2cc159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc06dc90 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc0b4b50 con 0x7fa2cc147d10 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2d2000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa2cc06de10 con 0x7fa2cc06ff40 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2c2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2c2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55096/0 (socket says 172.21.15.86:55096) 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.965+0000 7fa2c2a00640 1 -- 172.21.15.86:0/3519896032 learned_addr learned my addr 172.21.15.86:0/3519896032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2cbe00640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc147d10 0x7fa2cc159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2cb400640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc06ff40 0x7fa2cc13f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c2a00640 1 -- 172.21.15.86:0/3519896032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc06ff40 msgr2=0x7fa2cc13f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c2a00640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc06ff40 0x7fa2cc13f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c2a00640 1 -- 172.21.15.86:0/3519896032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc147d10 msgr2=0x7fa2cc159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c2a00640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc147d10 0x7fa2cc159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c2a00640 1 -- 172.21.15.86:0/3519896032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc15ed30 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c2a00640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc145f50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa2b401a0d0 tx=0x7fa2b4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c3e00640 1 -- 172.21.15.86:0/3519896032 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40b1040 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c3e00640 1 -- 172.21.15.86:0/3519896032 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b40bb070 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c3e00640 1 -- 172.21.15.86:0/3519896032 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40b5410 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc13fb20 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc1400e0 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2cc068650 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c3e00640 1 -- 172.21.15.86:0/3519896032 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa2b40b55b0 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c3e00640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2a8067b20 0x7fa2a8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2c3e00640 1 -- 172.21.15.86:0/3519896032 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fa2b411fcb0 con 0x7fa2cc0b3e00 2024-03-22T04:18:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.969+0000 7fa2cb400640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2a8067b20 0x7fa2a8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.973+0000 7fa2cb400640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2a8067b20 0x7fa2a8069fe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc011340 tx=0x7fa2bc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:19.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:19.973+0000 7fa2c3e00640 1 -- 172.21.15.86:0/3519896032 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa2b4016090 con 0x7fa2cc0b3e00 2024-03-22T04:18:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.141+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2cc0b2510 con 0x7fa2cc0b3e00 2024-03-22T04:18:20.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.141+0000 7fa2c3e00640 1 -- 172.21.15.86:0/3519896032 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v96) ==== 80+0+124 (secure 0 0 0) 0x7fa2b40efd10 con 0x7fa2cc0b3e00 2024-03-22T04:18:20.142 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.196:6835/1097103199 2024-03-23T04:17:08.939658+0000 2024-03-22T04:18:20.142 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.196:6834/1097103199 2024-03-23T04:17:08.939658+0000 2024-03-22T04:18:20.143 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-03-22T04:18:20.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2a8067b20 msgr2=0x7fa2a8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2a8067b20 0x7fa2a8069fe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc011340 tx=0x7fa2bc00b040 comp rx=0 tx=0).stop 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0b3e00 msgr2=0x7fa2cc145f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc145f50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa2b401a0d0 tx=0x7fa2b4019240 comp rx=0 tx=0).stop 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 shutdown_connections 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2a8067b20 0x7fa2a8069fe0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2cc147d10 0x7fa2cc159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0b3e00 0x7fa2cc145f50 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 --2- 172.21.15.86:0/3519896032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2cc06ff40 0x7fa2cc13f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 >> 172.21.15.86:0/3519896032 conn(0x7fa2cc0b75b0 msgr2=0x7fa2cc15b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 shutdown_connections 2024-03-22T04:18:20.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.145+0000 7fa2d2000640 1 -- 172.21.15.86:0/3519896032 wait complete. 2024-03-22T04:18:20.164 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:18:20.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:20.164 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:18:20.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:18:20.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:20.190 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:18:20.240 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:18:20.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe168000640 1 Processor -- start 2024-03-22T04:18:20.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe168000640 1 -- start start 2024-03-22T04:18:20.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe168000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe1600b4af0 0x7fe160167190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe168000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe160142260 con 0x7fe16015cd10 2024-03-22T04:18:20.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe168000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe160142540 con 0x7fe1601418f0 2024-03-22T04:18:20.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe168000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe160142820 con 0x7fe1600b4af0 2024-03-22T04:18:20.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe165600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe16015cd10 0x7fe16015d0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45952/0 (socket says 172.21.15.86:45952) 2024-03-22T04:18:20.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe165600640 1 -- 172.21.15.86:0/3454922083 learned_addr learned my addr 172.21.15.86:0/3454922083 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:20.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe166000640 1 --2- 172.21.15.86:0/3454922083 >> v2:172.21.15.86:3300/0 conn(0x7fe1600b4af0 0x7fe160167190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe15fe00640 1 -- 172.21.15.86:0/3454922083 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894313300 0 0) 0x7fe160142260 con 0x7fe16015cd10 2024-03-22T04:18:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe15fe00640 1 -- 172.21.15.86:0/3454922083 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe148003530 con 0x7fe16015cd10 2024-03-22T04:18:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe15fe00640 1 -- 172.21.15.86:0/3454922083 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3065172672 0 0) 0x7fe160142540 con 0x7fe1601418f0 2024-03-22T04:18:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe166000640 1 -- 172.21.15.86:0/3454922083 >> v1:172.21.15.196:6789/0 conn(0x7fe1601418f0 legacy=0x7fe1600b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe166000640 1 -- 172.21.15.86:0/3454922083 >> v1:172.21.15.86:6789/0 conn(0x7fe16015cd10 legacy=0x7fe16015d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.417+0000 7fe166000640 1 -- 172.21.15.86:0/3454922083 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16013f990 con 0x7fe1600b4af0 2024-03-22T04:18:20.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe166000640 1 --2- 172.21.15.86:0/3454922083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1600b4af0 0x7fe160167190 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe1601429a0 tx=0x7fe15801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06ba2f953862ee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:20.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe15fe00640 1 -- 172.21.15.86:0/3454922083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b2070 con 0x7fe1600b4af0 2024-03-22T04:18:20.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe15fe00640 1 -- 172.21.15.86:0/3454922083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1580a5890 con 0x7fe1600b4af0 2024-03-22T04:18:20.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe15fe00640 1 -- 172.21.15.86:0/3454922083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580a5c60 con 0x7fe1600b4af0 2024-03-22T04:18:20.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe168000640 1 -- 172.21.15.86:0/3454922083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1600b4af0 msgr2=0x7fe160167190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe168000640 1 --2- 172.21.15.86:0/3454922083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1600b4af0 0x7fe160167190 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe1601429a0 tx=0x7fe15801c8c0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe168000640 1 -- 172.21.15.86:0/3454922083 shutdown_connections 2024-03-22T04:18:20.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe168000640 1 --2- 172.21.15.86:0/3454922083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1600b4af0 0x7fe160167190 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe168000640 1 -- 172.21.15.86:0/3454922083 >> 172.21.15.86:0/3454922083 conn(0x7fe16006ba60 msgr2=0x7fe16006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:20.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe168000640 1 -- 172.21.15.86:0/3454922083 shutdown_connections 2024-03-22T04:18:20.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.421+0000 7fe168000640 1 -- 172.21.15.86:0/3454922083 wait complete. 2024-03-22T04:18:20.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 Processor -- start 2024-03-22T04:18:20.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 -- start start 2024-03-22T04:18:20.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1600b4af0 0x7fe160137dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1601418f0 0x7fe16006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16015cd10 0x7fe1601517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe16016a280 con 0x7fe16015cd10 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe16016a100 con 0x7fe1601418f0 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe16016a400 con 0x7fe1600b4af0 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16015cd10 0x7fe1601517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16015cd10 0x7fe1601517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55124/0 (socket says 172.21.15.86:55124) 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 -- 172.21.15.86:0/895789225 learned_addr learned my addr 172.21.15.86:0/895789225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe165600640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1600b4af0 0x7fe160137dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe164c00640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1601418f0 0x7fe16006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 -- 172.21.15.86:0/895789225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1600b4af0 msgr2=0x7fe160137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1600b4af0 0x7fe160137dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 -- 172.21.15.86:0/895789225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1601418f0 msgr2=0x7fe16006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1601418f0 0x7fe16006dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 -- 172.21.15.86:0/895789225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16013bf80 con 0x7fe16015cd10 2024-03-22T04:18:20.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe166000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16015cd10 0x7fe1601517f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe15801cda0 tx=0x7fe158019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:20.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe15d600640 1 -- 172.21.15.86:0/895789225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580bd020 con 0x7fe16015cd10 2024-03-22T04:18:20.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe15d600640 1 -- 172.21.15.86:0/895789225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1580b8670 con 0x7fe16015cd10 2024-03-22T04:18:20.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe15d600640 1 -- 172.21.15.86:0/895789225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b2040 con 0x7fe16015cd10 2024-03-22T04:18:20.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16013c260 con 0x7fe16015cd10 2024-03-22T04:18:20.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.425+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe160142d20 con 0x7fe16015cd10 2024-03-22T04:18:20.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.429+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe160068650 con 0x7fe16015cd10 2024-03-22T04:18:20.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.429+0000 7fe15d600640 1 -- 172.21.15.86:0/895789225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe1580c0020 con 0x7fe16015cd10 2024-03-22T04:18:20.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.429+0000 7fe15d600640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe140067b20 0x7fe140069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.429+0000 7fe15d600640 1 -- 172.21.15.86:0/895789225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7fe158120fc0 con 0x7fe16015cd10 2024-03-22T04:18:20.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.429+0000 7fe165600640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe140067b20 0x7fe140069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.429+0000 7fe165600640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe140067b20 0x7fe140069fe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe154007fd0 tx=0x7fe154009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:20.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.433+0000 7fe15d600640 1 -- 172.21.15.86:0/895789225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe1580c7030 con 0x7fe16015cd10 2024-03-22T04:18:20.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe160072220 con 0x7fe16015cd10 2024-03-22T04:18:20.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe15d600640 1 -- 172.21.15.86:0/895789225 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5186 (secure 0 0 0) 0x7fe158016540 con 0x7fe16015cd10 2024-03-22T04:18:20.621 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:17:19.957152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76604":{"gid":76604,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6835/1851339627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1851339627},{"type":"v1","addr":"172.21.15.196:6835","nonce":1851339627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76604,"qdb_cluster":[76604]},"id":6}]} 2024-03-22T04:18:20.622 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 71 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe140067b20 msgr2=0x7fe140069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe140067b20 0x7fe140069fe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe154007fd0 tx=0x7fe154009040 comp rx=0 tx=0).stop 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16015cd10 msgr2=0x7fe1601517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16015cd10 0x7fe1601517f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe15801cda0 tx=0x7fe158019660 comp rx=0 tx=0).stop 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 shutdown_connections 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe140067b20 0x7fe140069fe0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1601418f0 0x7fe16006dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe16015cd10 0x7fe1601517f0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 --2- 172.21.15.86:0/895789225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1600b4af0 0x7fe160137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 >> 172.21.15.86:0/895789225 conn(0x7fe16006ba60 msgr2=0x7fe16015d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:20.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 shutdown_connections 2024-03-22T04:18:20.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.621+0000 7fe168000640 1 -- 172.21.15.86:0/895789225 wait complete. 2024-03-22T04:18:20.642 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-22T04:18:20.816 INFO:teuthology.orchestra.run.smithi196.stdout:{"message":"","return_code":0} 2024-03-22T04:18:20.816 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:18:20.817 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:18:20.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.953+0000 7f111e200640 1 Processor -- start 2024-03-22T04:18:20.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.953+0000 7f111e200640 1 -- start start 2024-03-22T04:18:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.953+0000 7f111e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad330 0x7f111815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.953+0000 7f111e200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f111806ef40 0x7f111806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.953+0000 7f111e200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f111815c7c0 con 0x7f11180ad710 2024-03-22T04:18:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.953+0000 7f111e200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f111806dc40 con 0x7f111806ef40 2024-03-22T04:18:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.953+0000 7f111e200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f111806dea0 con 0x7f11180ad330 2024-03-22T04:18:20.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1117400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad330 0x7f111815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1116a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f111806ef40 0x7f111806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1117400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad330 0x7f111815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:40802/0 (socket says 172.21.15.86:40802) 2024-03-22T04:18:20.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1116a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f111806ef40 0x7f111806d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:59870/0 (socket says 172.21.15.86:59870) 2024-03-22T04:18:20.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1117400640 1 -- 172.21.15.86:0/623202004 learned_addr learned my addr 172.21.15.86:0/623202004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:20.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1117400640 1 -- 172.21.15.86:0/623202004 >> v2:172.21.15.196:3300/0 conn(0x7f111806ef40 msgr2=0x7f111806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:20.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1117400640 1 --2- 172.21.15.86:0/623202004 >> v2:172.21.15.196:3300/0 conn(0x7f111806ef40 0x7f111806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1117400640 1 -- 172.21.15.86:0/623202004 >> v1:172.21.15.196:6789/0 conn(0x7f11180ad710 legacy=0x7f111813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1117400640 1 -- 172.21.15.86:0/623202004 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111806e180 con 0x7f11180ad330 2024-03-22T04:18:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1116a00640 1 --2- 172.21.15.86:0/623202004 >> v2:172.21.15.196:3300/0 conn(0x7f111806ef40 0x7f111806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1116000640 1 -- 172.21.15.86:0/623202004 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3619884363 0 0) 0x7f111815c7c0 con 0x7f11180ad710 2024-03-22T04:18:20.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1117400640 1 --2- 172.21.15.86:0/623202004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad330 0x7f111815d4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1108009e50 tx=0x7f110801ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f845949a5f2bd4f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:20.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1116000640 1 -- 172.21.15.86:0/623202004 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080a57e0 con 0x7f11180ad330 2024-03-22T04:18:20.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1116000640 1 -- 172.21.15.86:0/623202004 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11080a5980 con 0x7f11180ad330 2024-03-22T04:18:20.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.957+0000 7f1116000640 1 -- 172.21.15.86:0/623202004 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080a5d50 con 0x7f11180ad330 2024-03-22T04:18:20.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- 172.21.15.86:0/623202004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad330 msgr2=0x7f111815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 --2- 172.21.15.86:0/623202004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad330 0x7f111815d4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1108009e50 tx=0x7f110801ca00 comp rx=0 tx=0).stop 2024-03-22T04:18:20.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- 172.21.15.86:0/623202004 shutdown_connections 2024-03-22T04:18:20.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 --2- 172.21.15.86:0/623202004 >> v2:172.21.15.196:3300/0 conn(0x7f111806ef40 0x7f111806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 --2- 172.21.15.86:0/623202004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad330 0x7f111815d4a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- 172.21.15.86:0/623202004 >> 172.21.15.86:0/623202004 conn(0x7f11180b1df0 msgr2=0x7f11180b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:20.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- 172.21.15.86:0/623202004 shutdown_connections 2024-03-22T04:18:20.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- 172.21.15.86:0/623202004 wait complete. 2024-03-22T04:18:20.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 Processor -- start 2024-03-22T04:18:20.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- start start 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f111806ef40 0x7f11180999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11180ad330 0x7f111815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad710 0x7f111815bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11181690e0 con 0x7f111806ef40 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1118168f60 con 0x7f11180ad330 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f111e200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1118169260 con 0x7f11180ad710 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f111806ef40 0x7f11180999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f111806ef40 0x7f11180999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55138/0 (socket says 172.21.15.86:55138) 2024-03-22T04:18:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117400640 1 -- 172.21.15.86:0/3154561500 learned_addr learned my addr 172.21.15.86:0/3154561500 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117e00640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad710 0x7f111815bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1116a00640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11180ad330 0x7f111815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117400640 1 -- 172.21.15.86:0/3154561500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad710 msgr2=0x7f111815bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117400640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad710 0x7f111815bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117400640 1 -- 172.21.15.86:0/3154561500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11180ad330 msgr2=0x7f111815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117400640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11180ad330 0x7f111815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.961+0000 7f1117400640 1 -- 172.21.15.86:0/3154561500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1118099ef0 con 0x7f111806ef40 2024-03-22T04:18:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f1117400640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f111806ef40 0x7f11180999b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f11080af040 tx=0x7f11080197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f110f400640 1 -- 172.21.15.86:0/3154561500 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080b8670 con 0x7f111806ef40 2024-03-22T04:18:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f110f400640 1 -- 172.21.15.86:0/3154561500 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11080be070 con 0x7f111806ef40 2024-03-22T04:18:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f110f400640 1 -- 172.21.15.86:0/3154561500 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080b8a70 con 0x7f111806ef40 2024-03-22T04:18:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11181441b0 con 0x7f111806ef40 2024-03-22T04:18:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11181433f0 con 0x7f111806ef40 2024-03-22T04:18:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1118068650 con 0x7f111806ef40 2024-03-22T04:18:20.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f110f400640 1 -- 172.21.15.86:0/3154561500 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f11080b8c10 con 0x7f111806ef40 2024-03-22T04:18:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f110f400640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f10f8067b50 0x7f10f806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f110f400640 1 -- 172.21.15.86:0/3154561500 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6448+0+0 (secure 0 0 0) 0x7f11081215b0 con 0x7f111806ef40 2024-03-22T04:18:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.965+0000 7f1116a00640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f10f8067b50 0x7f10f806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:20.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.969+0000 7f1116a00640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f10f8067b50 0x7f10f806a010 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1104009580 tx=0x7f1104009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:20.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:20.973+0000 7f110f400640 1 -- 172.21.15.86:0/3154561500 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f11080c7020 con 0x7f111806ef40 2024-03-22T04:18:21.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.157+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f111815f190 con 0x7f111806ef40 2024-03-22T04:18:21.159 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:18:21.161+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:18:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.161+0000 7f110f400640 1 -- 172.21.15.86:0/3154561500 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) ==== 161+0+0 (secure 0 0 0) 0x7f1108016620 con 0x7f111806ef40 2024-03-22T04:18:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:18:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f10f8067b50 msgr2=0x7f10f806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f10f8067b50 0x7f10f806a010 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1104009580 tx=0x7f1104009190 comp rx=0 tx=0).stop 2024-03-22T04:18:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f111806ef40 msgr2=0x7f11180999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f111806ef40 0x7f11180999b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f11080af040 tx=0x7f11080197a0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 shutdown_connections 2024-03-22T04:18:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f10f8067b50 0x7f10f806a010 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11180ad330 0x7f111815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11180ad710 0x7f111815bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 --2- 172.21.15.86:0/3154561500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f111806ef40 0x7f11180999b0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 >> 172.21.15.86:0/3154561500 conn(0x7f11180b1df0 msgr2=0x7f11180b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 shutdown_connections 2024-03-22T04:18:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.165+0000 7f111e200640 1 -- 172.21.15.86:0/3154561500 wait complete. 2024-03-22T04:18:21.176 INFO:tasks.ceph.mds.b.smithi196.stderr: -4660> 2024-03-22T04:17:14.948+0000 7f0c5f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:18:21.177 INFO:tasks.ceph.mds.b.smithi196.stderr: -4267> 2024-03-22T04:17:15.592+0000 7f0c5a400640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-22T04:18:21.177 INFO:tasks.ceph.mds.b.smithi196.stderr: -4101> 2024-03-22T04:17:15.824+0000 7f0c5a400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-03-22T04:17:05.898187+0000 1=1+0) n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55c18476bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-22T04:17:05.898187+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-22T04:18:21.181 INFO:tasks.ceph.mds.b.smithi196.stderr: -2232> 2024-03-22T04:17:47.380+0000 7f0c5a400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-03-22T04:17:05.898187+0000 1=1+0)->f(v1 m2024-03-22T04:17:05.898187+0000) n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1)->n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55c18476bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-22T04:17:05.898187+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-22T04:17:06.002186+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-22T04:18:21.185 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-22T04:18:21.224 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:18:21.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 -- start start 2024-03-22T04:18:21.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 Processor -- start 2024-03-22T04:18:21.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565447a45800 0x5654487fc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565447a45400 0x5654487fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565447a45000 0x5654487fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565448810780 con 0x565447a45000 2024-03-22T04:18:21.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565448810600 con 0x565447a45800 2024-03-22T04:18:21.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x565448810480 con 0x565447a45400 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565447a45000 0x5654487fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565447a45000 0x5654487fcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55140/0 (socket says 172.21.15.86:55140) 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 -- 172.21.15.86:0/3043307566 learned_addr learned my addr 172.21.15.86:0/3043307566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2a000640 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565447a45400 0x5654487fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565447a45800 0x5654487fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 -- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565447a45400 msgr2=0x5654487fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565447a45400 0x5654487fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 -- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565447a45800 msgr2=0x5654487fc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565447a45800 0x5654487fc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 -- 172.21.15.86:0/3043307566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565447a6a1e0 con 0x565447a45000 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565447a45800 0x5654487fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2b400640 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565447a45000 0x5654487fcb00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x565448896ff0 tx=0x5654487cc7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9399b49cd1dcef54 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c27800640 1 -- 172.21.15.86:0/3043307566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565447a7a9c0 con 0x565447a45000 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c27800640 1 -- 172.21.15.86:0/3043307566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565447a90e00 con 0x565447a45000 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c27800640 1 -- 172.21.15.86:0/3043307566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565447a7a4e0 con 0x565447a45000 2024-03-22T04:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/3043307566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565447a6b2c0 con 0x565447a45000 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/3043307566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x565447a6a3c0 con 0x565447a45000 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.229+0000 7f9c2c87f8c0 4 waiting for MDS map... 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c27800640 1 -- 172.21.15.86:0/3043307566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x565448814000 con 0x565447a45000 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c27800640 1 -- 172.21.15.86:0/3043307566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 72) ==== 2885+0+0 (secure 0 0 0) 0x565447a9b180 con 0x565447a45000 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 4 Got MDS map 72 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 4 datascan.main: Using metadata pool output 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 4 datascan.main: connecting to RADOS... 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 Processor -- start 2024-03-22T04:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- start start 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565448813c00 0x5654487fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654489c0000 0x5654487fd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c0400 0x5654487fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565448810300 con 0x5654489c0000 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565448810000 con 0x5654489c0400 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x565447aa4900 con 0x565448813c00 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654489c0000 0x5654487fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2a000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565448813c00 0x5654487fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654489c0000 0x5654487fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55154/0 (socket says 172.21.15.86:55154) 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2984914507 learned_addr learned my addr 172.21.15.86:0/2984914507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c0400 0x5654487fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2984914507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565448813c00 msgr2=0x5654487fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565448813c00 0x5654487fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2984914507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c0400 msgr2=0x5654487fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c0400 0x5654487fdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2984914507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565447a6a1e0 con 0x5654489c0000 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c0400 0x5654487fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654489c0000 0x5654487fd600 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x565448968ba0 tx=0x565448953c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8b6ba76e6798aa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c25000640 1 -- 172.21.15.86:0/2984914507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565447a7a9c0 con 0x5654489c0000 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c25000640 1 -- 172.21.15.86:0/2984914507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565447a90a80 con 0x5654489c0000 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c25000640 1 -- 172.21.15.86:0/2984914507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565447a7a4e0 con 0x5654489c0000 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2984914507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654489c0000 msgr2=0x5654487fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654489c0000 0x5654487fd600 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x565448968ba0 tx=0x565448953c20 comp rx=0 tx=0).stop 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2984914507 shutdown_connections 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c0400 0x5654487fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654489c0000 0x5654487fd600 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2984914507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565448813c00 0x5654487fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2984914507 >> 172.21.15.86:0/2984914507 conn(0x565447a44c00 msgr2=0x565447a73800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2984914507 shutdown_connections 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.233+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2984914507 wait complete. 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 Processor -- start 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 -- start start 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565448813c00 0x5654487fd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c1800 0x5654487fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654489c1400 0x5654487fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565448810000 con 0x565448813c00 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565448811500 con 0x5654489c1800 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x565448810300 con 0x5654489c1400 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565448813c00 0x5654487fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565448813c00 0x5654487fd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55160/0 (socket says 172.21.15.86:55160) 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 learned_addr learned my addr 172.21.15.86:0/2593922424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c1800 0x5654487fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654489c1400 0x5654487fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654489c1400 msgr2=0x5654487fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654489c1400 0x5654487fdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c1800 msgr2=0x5654487fd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654489c1800 0x5654487fd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565447a6b4a0 con 0x565448813c00 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565448813c00 0x5654487fd080 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x565448969800 tx=0x565447b1f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c23200640 1 -- 172.21.15.86:0/2593922424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565447a7a9c0 con 0x565448813c00 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c23200640 1 -- 172.21.15.86:0/2593922424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565447a90540 con 0x565448813c00 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c23200640 1 -- 172.21.15.86:0/2593922424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565447a7a4e0 con 0x565448813c00 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565447a6ba40 con 0x565448813c00 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565447a6bc20 con 0x565448813c00 2024-03-22T04:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.237+0000 7f9c2c87f8c0 4 datascan.init: resolving metadata pool 13 2024-03-22T04:18:21.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c23200640 1 -- 172.21.15.86:0/2593922424 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x5654489c6c00 con 0x565448813c00 2024-03-22T04:18:21.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c23200640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565447d37400 0x5654487fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c23200640 1 -- 172.21.15.86:0/2593922424 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x565447d28c00 con 0x565448813c00 2024-03-22T04:18:21.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2c87f8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2c87f8c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2c87f8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5654489c1000 0x5654487fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x5654489c0c00 con 0x5654489c1000 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565447d37400 0x5654487fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5654489c1000 0x5654487fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565447d37400 0x5654487fe100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x565448968ba0 tx=0x5654487cca20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5654489c1000 0x5654487fe680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(1 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447d37800 con 0x5654489c1000 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 2 ==== osd_op_reply(2 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c25000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x565447d37c00 0x565447e40000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44000 con 0x565447d37c00 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2b400640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x565447d37c00 0x565447e40000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.241+0000 7f9c2b400640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x565447d37c00 0x565447e40000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(3 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447d37c00 2024-03-22T04:18:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44800 con 0x5654489c1000 2024-03-22T04:18:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 3 ==== osd_op_reply(4 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c25000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x565447e44400 0x565447e40580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e45000 con 0x565447e44400 2024-03-22T04:18:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x565447e44400 0x565447e40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x565447e44400 0x565447e40580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(5 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e44400 2024-03-22T04:18:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44c00 con 0x565447d37c00 2024-03-22T04:18:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== osd_op_reply(6 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447d37c00 2024-03-22T04:18:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c25000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x565447e45400 0x565447e40b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.245+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e45800 con 0x565447e45400 2024-03-22T04:18:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x565447e45400 0x565447e40b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x565447e45400 0x565447e40b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(7 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x565447e45400 2024-03-22T04:18:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44800 con 0x5654489c1000 2024-03-22T04:18:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 4 ==== osd_op_reply(8 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c25000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x565447e45800 0x565447e41080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e45c00 con 0x565447e45800 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2b400640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x565447e45800 0x565447e41080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2b400640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x565447e45800 0x565447e41080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(9 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e45800 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447d37800 con 0x565447e45400 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== osd_op_reply(10 [pgnls start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x565447e45400 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c25000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x565447e56000 0x565447e41600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56400 con 0x565447e56000 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x565447e56000 0x565447e41600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.249+0000 7f9c2aa00640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x565447e56000 0x565447e41600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== osd_op_reply(11 [pgnls start_epoch 97 out=49b] v93'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c24600640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5654489c0c00 0x565447e42100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57400 con 0x5654489c0c00 2024-03-22T04:18:21.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5654489c0c00 0x565447e42100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c2a000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5654489c0c00 0x565447e42100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(12 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c0c00 2024-03-22T04:18:21.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56800 con 0x565447e45400 2024-03-22T04:18:21.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.5 v2:172.21.15.196:6808/2743139076 3 ==== osd_op_reply(13 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x565447e45400 2024-03-22T04:18:21.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57800 con 0x5654489c1000 2024-03-22T04:18:21.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 5 ==== osd_op_reply(14 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56c00 con 0x565447d37c00 2024-03-22T04:18:21.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.6 v2:172.21.15.196:6816/1648471396 3 ==== osd_op_reply(15 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447d37c00 2024-03-22T04:18:21.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57c00 con 0x565447e44400 2024-03-22T04:18:21.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.0 v2:172.21.15.86:6808/2750248043 2 ==== osd_op_reply(16 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e44400 2024-03-22T04:18:21.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.253+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57000 con 0x565447e45800 2024-03-22T04:18:21.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== osd_op_reply(17 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e45800 2024-03-22T04:18:21.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44000 con 0x565447e56000 2024-03-22T04:18:21.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== osd_op_reply(18 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56c00 con 0x565447d37c00 2024-03-22T04:18:21.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.6 v2:172.21.15.196:6816/1648471396 4 ==== osd_op_reply(19 [pgnls start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447d37c00 2024-03-22T04:18:21.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44c00 con 0x565447e44400 2024-03-22T04:18:21.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.0 v2:172.21.15.86:6808/2750248043 3 ==== osd_op_reply(20 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e44400 2024-03-22T04:18:21.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57000 con 0x565447e45800 2024-03-22T04:18:21.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.2 v2:172.21.15.86:6816/4132808680 3 ==== osd_op_reply(21 [pgnls start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e45800 2024-03-22T04:18:21.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e45c00 con 0x565447e45800 2024-03-22T04:18:21.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.2 v2:172.21.15.86:6816/4132808680 4 ==== osd_op_reply(22 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e45800 2024-03-22T04:18:21.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56400 con 0x5654489c1000 2024-03-22T04:18:21.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 6 ==== osd_op_reply(23 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e45000 con 0x5654489c1000 2024-03-22T04:18:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 7 ==== osd_op_reply(24 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57c00 con 0x565447e45400 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.5 v2:172.21.15.196:6808/2743139076 4 ==== osd_op_reply(25 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x565447e45400 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44800 con 0x565447e56000 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 3 ==== osd_op_reply(26 [pgnls start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447d36400 con 0x565447d37c00 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.6 v2:172.21.15.196:6816/1648471396 5 ==== osd_op_reply(27 [pgnls start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447d37c00 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57400 con 0x5654489c0c00 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.7 v2:172.21.15.196:6817/2847924285 2 ==== osd_op_reply(28 [pgnls start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c0c00 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447d37800 con 0x565447e56000 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 4 ==== osd_op_reply(29 [pgnls start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.257+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56800 con 0x5654489c0c00 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.7 v2:172.21.15.196:6817/2847924285 3 ==== osd_op_reply(30 [pgnls start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c0c00 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c25000640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x565447e64000 0x565447e41b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e64400 con 0x565447e64000 2024-03-22T04:18:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c2b400640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x565447e64000 0x565447e41b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c2b400640 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x565447e64000 0x565447e41b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(31 [pgnls start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e64000 2024-03-22T04:18:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57800 con 0x565447e44400 2024-03-22T04:18:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.0 v2:172.21.15.86:6808/2750248043 4 ==== osd_op_reply(32 [pgnls start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e44400 2024-03-22T04:18:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.261+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e64800 con 0x5654489c0c00 2024-03-22T04:18:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.7 v2:172.21.15.196:6817/2847924285 4 ==== osd_op_reply(33 [pgnls start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c0c00 2024-03-22T04:18:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56400 con 0x565447e45800 2024-03-22T04:18:21.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.2 v2:172.21.15.86:6816/4132808680 5 ==== osd_op_reply(34 [pgnls start_epoch 97 out=49b] v93'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e45800 2024-03-22T04:18:21.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e64c00 con 0x565447e56000 2024-03-22T04:18:21.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 5 ==== osd_op_reply(35 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57c00 con 0x5654489c0c00 2024-03-22T04:18:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.7 v2:172.21.15.196:6817/2847924285 5 ==== osd_op_reply(36 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c0c00 2024-03-22T04:18:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e65000 con 0x565447e44400 2024-03-22T04:18:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.0 v2:172.21.15.86:6808/2750248043 5 ==== osd_op_reply(37 [pgnls start_epoch 97 out=49b] v93'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e44400 2024-03-22T04:18:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447d37800 con 0x565447e56000 2024-03-22T04:18:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 6 ==== osd_op_reply(38 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e65400 con 0x565447e64000 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== osd_op_reply(39 [pgnls start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e64000 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e45000 con 0x565447e64000 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.3 v2:172.21.15.86:6822/2837220459 3 ==== osd_op_reply(40 [pgnls start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e64000 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e65800 con 0x565447e45400 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.5 v2:172.21.15.196:6808/2743139076 5 ==== osd_op_reply(41 [pgnls start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x565447e45400 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44000 con 0x5654489c0c00 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.265+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.7 v2:172.21.15.196:6817/2847924285 6 ==== osd_op_reply(42 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c0c00 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56c00 con 0x565447e64000 2024-03-22T04:18:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.3 v2:172.21.15.86:6822/2837220459 4 ==== osd_op_reply(43 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e64000 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44c00 con 0x565447e64000 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.3 v2:172.21.15.86:6822/2837220459 5 ==== osd_op_reply(44 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e64000 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57000 con 0x565447e64000 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.3 v2:172.21.15.86:6822/2837220459 6 ==== osd_op_reply(45 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e64000 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e44800 con 0x565447e45800 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.2 v2:172.21.15.86:6816/4132808680 6 ==== osd_op_reply(46 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e45800 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56400 con 0x5654489c1000 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 8 ==== osd_op_reply(47 [pgnls start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e64400 con 0x565447e44400 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.0 v2:172.21.15.86:6808/2750248043 6 ==== osd_op_reply(48 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e44400 2024-03-22T04:18:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57400 con 0x5654489c1000 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 9 ==== osd_op_reply(49 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e64800 con 0x565447e56000 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 7 ==== osd_op_reply(50 [pgnls start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56800 con 0x565447e45400 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.5 v2:172.21.15.196:6808/2743139076 6 ==== osd_op_reply(51 [pgnls start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x565447e45400 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e65c00 con 0x565447e45400 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.5 v2:172.21.15.196:6808/2743139076 7 ==== osd_op_reply(52 [pgnls start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x565447e45400 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57800 con 0x565447d37c00 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.6 v2:172.21.15.196:6816/1648471396 6 ==== osd_op_reply(53 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447d37c00 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.269+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56c00 con 0x565447e56000 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 8 ==== osd_op_reply(54 [pgnls start_epoch 97 out=49b] v93'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447d36400 con 0x565447d37c00 2024-03-22T04:18:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.6 v2:172.21.15.196:6816/1648471396 7 ==== osd_op_reply(55 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447d37c00 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57000 con 0x565447e64000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.3 v2:172.21.15.86:6822/2837220459 7 ==== osd_op_reply(56 [pgnls start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e64000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e45c00 con 0x565447e56000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 9 ==== osd_op_reply(57 [pgnls start_epoch 97 out=81b] v93'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x5654489c0800 con 0x565447e56000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2c87f8c0 10 datascan.operator(): handling object 1099511627777.0 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5654489c0400 con 0x565447e56000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 11 ==== osd_op_reply(59 10000000001.00000000 [call] v97'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x5654489c0000 con 0x565447e56000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 12 ==== osd_op_reply(60 [pgnls start_epoch 97 out=49b] v93'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e64c00 con 0x5654489c1000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 10 ==== osd_op_reply(61 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.273+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57800 con 0x565447e64000 2024-03-22T04:18:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.3 v2:172.21.15.86:6822/2837220459 8 ==== osd_op_reply(62 [pgnls start_epoch 97 out=49b] v93'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447e64000 2024-03-22T04:18:21.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e65000 con 0x565447e56000 2024-03-22T04:18:21.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 <== osd.1 v2:172.21.15.86:6800/3761623609 13 ==== osd_op_reply(63 [pgnls start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a780 con 0x565447e56000 2024-03-22T04:18:21.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e57c00 con 0x5654489c1000 2024-03-22T04:18:21.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.4 v2:172.21.15.196:6800/655314492 11 ==== osd_op_reply(64 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c1000 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c25000640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e64400 con 0x5654489c0c00 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2a000640 1 -- 172.21.15.86:0/2593922424 <== osd.7 v2:172.21.15.196:6817/2847924285 7 ==== osd_op_reply(65 [pgnls start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a000 con 0x5654489c0c00 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c24600640 1 -- 172.21.15.86:0/2593922424 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x565447e56400 con 0x565447d37c00 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2b400640 1 -- 172.21.15.86:0/2593922424 <== osd.6 v2:172.21.15.196:6816/1648471396 8 ==== osd_op_reply(66 [pgnls start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x565447b1a500 con 0x565447d37c00 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x565447e44400 msgr2=0x565447e40580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x565447e44400 0x565447e40580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x565447e56000 msgr2=0x565447e41600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x565447e56000 0x565447e41600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x565447e45800 msgr2=0x565447e41080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x565447e45800 0x565447e41080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x565447e64000 msgr2=0x565447e41b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x565447e64000 0x565447e41b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5654489c1000 msgr2=0x5654487fe680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5654489c1000 0x5654487fe680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x565447e45400 msgr2=0x565447e40b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x565447e45400 0x565447e40b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x565447d37c00 msgr2=0x565447e40000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x565447d37c00 0x565447e40000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5654489c0c00 msgr2=0x565447e42100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5654489c0c00 0x565447e42100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2aa00640 1 -- 172.21.15.86:0/2593922424 reap_dead start 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565447d37400 msgr2=0x5654487fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565447d37400 0x5654487fe100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x565448968ba0 tx=0x5654487cca20 comp rx=0 tx=0).stop 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565448813c00 msgr2=0x5654487fd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565448813c00 0x5654487fd080 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x565448969800 tx=0x565447b1f4a0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 shutdown_connections 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565447d37400 0x5654487fe100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/2593922424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565448813c00 0x5654487fd080 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 >> 172.21.15.86:0/2593922424 conn(0x565447a44c00 msgr2=0x565447a73800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 shutdown_connections 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/2593922424 wait complete. 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/3043307566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565447a45000 msgr2=0x5654487fcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565447a45000 0x5654487fcb00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x565448896ff0 tx=0x5654487cc7e0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/3043307566 shutdown_connections 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565447a45000 0x5654487fcb00 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565447a45400 0x5654487fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 --2- 172.21.15.86:0/3043307566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565447a45800 0x5654487fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/3043307566 >> 172.21.15.86:0/3043307566 conn(0x565447a45c00 msgr2=0x565447a73000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/3043307566 shutdown_connections 2024-03-22T04:18:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.277+0000 7f9c2c87f8c0 1 -- 172.21.15.86:0/3043307566 wait complete. 2024-03-22T04:18:21.287 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.102956, result: 2024-03-22T04:18:21.288 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-03-22T04:18:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e58a78c0 1 -- start start 2024-03-22T04:18:21.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e58a78c0 1 Processor -- start 2024-03-22T04:18:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642946cb800 0x564295482000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642946cb400 0x564295482580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642946cb000 0x564295482b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564295494180 con 0x5642946cb000 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564295494300 con 0x5642946cb400 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x564295494480 con 0x5642946cb800 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642946cb000 0x564295482b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642946cb000 0x564295482b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55178/0 (socket says 172.21.15.86:55178) 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e4400640 1 -- 172.21.15.86:0/4259330568 learned_addr learned my addr 172.21.15.86:0/4259330568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e3000640 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642946cb400 0x564295482580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.369+0000 7f26e3a00640 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642946cb800 0x564295482000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e4400640 1 -- 172.21.15.86:0/4259330568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642946cb800 msgr2=0x564295482000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e4400640 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642946cb800 0x564295482000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e4400640 1 -- 172.21.15.86:0/4259330568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642946cb400 msgr2=0x564295482580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e4400640 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642946cb400 0x564295482580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e4400640 1 -- 172.21.15.86:0/4259330568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642946f01e0 con 0x5642946cb000 2024-03-22T04:18:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e4400640 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642946cb000 0x564295482b00 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x564295522fc0 tx=0x5642954527e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869aa5610b56c6a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e0800640 1 -- 172.21.15.86:0/4259330568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642947009c0 con 0x5642946cb000 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e0800640 1 -- 172.21.15.86:0/4259330568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564294716e00 con 0x5642946cb000 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e0800640 1 -- 172.21.15.86:0/4259330568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642947004e0 con 0x5642946cb000 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 -- 172.21.15.86:0/4259330568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642946f12c0 con 0x5642946cb000 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 -- 172.21.15.86:0/4259330568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5642946f03c0 con 0x5642946cb000 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 4 waiting for MDS map... 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e0800640 1 -- 172.21.15.86:0/4259330568 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x56429548e000 con 0x5642946cb000 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e0800640 1 -- 172.21.15.86:0/4259330568 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 72) ==== 2885+0+0 (secure 0 0 0) 0x564294721180 con 0x5642946cb000 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 4 Got MDS map 72 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 4 datascan.main: Using metadata pool output 2024-03-22T04:18:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 4 datascan.main: connecting to RADOS... 2024-03-22T04:18:21.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 Processor -- start 2024-03-22T04:18:21.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 -- start start 2024-03-22T04:18:21.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642954abc00 0x564295483080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564295648000 0x564295483600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295648400 0x564295483b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564295494600 con 0x5642954abc00 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564295494780 con 0x564295648000 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x564295494900 con 0x564295648400 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642954abc00 0x564295483080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642954abc00 0x564295483080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55190/0 (socket says 172.21.15.86:55190) 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 -- 172.21.15.86:0/3940018421 learned_addr learned my addr 172.21.15.86:0/3940018421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3a00640 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564295648000 0x564295483600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295648400 0x564295483b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 -- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295648400 msgr2=0x564295483b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295648400 0x564295483b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 -- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564295648000 msgr2=0x564295483600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564295648000 0x564295483600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 -- 172.21.15.86:0/3940018421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642946f01e0 con 0x5642954abc00 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3a00640 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564295648000 0x564295483600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26e3000640 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642954abc00 0x564295483080 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x564295604b10 tx=0x5642955e1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693b496541b39a61 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26de000640 1 -- 172.21.15.86:0/3940018421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642947009c0 con 0x5642954abc00 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26de000640 1 -- 172.21.15.86:0/3940018421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564294716540 con 0x5642954abc00 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.373+0000 7f26de000640 1 -- 172.21.15.86:0/3940018421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642947004e0 con 0x5642954abc00 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- 172.21.15.86:0/3940018421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642954abc00 msgr2=0x564295483080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642954abc00 0x564295483080 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x564295604b10 tx=0x5642955e1b60 comp rx=0 tx=0).stop 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- 172.21.15.86:0/3940018421 shutdown_connections 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564295648000 0x564295483600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295648400 0x564295483b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/3940018421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642954abc00 0x564295483080 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- 172.21.15.86:0/3940018421 >> 172.21.15.86:0/3940018421 conn(0x5642946cac00 msgr2=0x5642946f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- 172.21.15.86:0/3940018421 shutdown_connections 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- 172.21.15.86:0/3940018421 wait complete. 2024-03-22T04:18:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 Processor -- start 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- start start 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642954abc00 0x564295483080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295649800 0x564295483600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564295649400 0x564295483b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564295495c80 con 0x564295649400 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564295495b00 con 0x5642954abc00 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x564295495980 con 0x564295649800 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564295649400 0x564295483b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564295649400 0x564295483b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55202/0 (socket says 172.21.15.86:55202) 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 learned_addr learned my addr 172.21.15.86:0/402127278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642954abc00 0x564295483080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3a00640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295649800 0x564295483600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295649800 msgr2=0x564295483600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564295649800 0x564295483600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642954abc00 msgr2=0x564295483080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642954abc00 0x564295483080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642946f14a0 con 0x564295649400 2024-03-22T04:18:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564295649400 0x564295483b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x5642956057d0 tx=0x5642947a54a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26dc200640 1 -- 172.21.15.86:0/402127278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564294700000 con 0x564295649400 2024-03-22T04:18:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26dc200640 1 -- 172.21.15.86:0/402127278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564294716a80 con 0x564295649400 2024-03-22T04:18:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26dc200640 1 -- 172.21.15.86:0/402127278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642947009c0 con 0x564295649400 2024-03-22T04:18:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5642946f1a40 con 0x564295649400 2024-03-22T04:18:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642946f1c20 con 0x564295649400 2024-03-22T04:18:21.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.377+0000 7f26e58a78c0 4 datascan.init: resolving metadata pool 13 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26dc200640 1 -- 172.21.15.86:0/402127278 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x564295650c00 con 0x564295649400 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26dc200640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5642949c2800 0x564295484100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26dc200640 1 -- 172.21.15.86:0/402127278 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x56429564ea00 con 0x564295649400 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e3a00640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5642949c2800 0x564295484100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x564295649000 0x564295484680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x564295648c00 con 0x564295649000 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x564295649000 0x564295484680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e3a00640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5642949c2800 0x564295484100 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x56429544ecf0 tx=0x564294a4f020 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x564295649000 0x564295484680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x564295648800 0x564295484c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x564295648400 con 0x564295648800 2024-03-22T04:18:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e4400640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x564295648800 0x564295484c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e4400640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x564295648800 0x564295484c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x564295648000 0x564295485180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.381+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x5642949c3800 con 0x564295648000 2024-03-22T04:18:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3a00640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x564295648000 0x564295485180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3a00640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x564295648000 0x564295485180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2000 con 0x564295648000 2024-03-22T04:18:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x564295649000 2024-03-22T04:18:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2800 con 0x564295648000 2024-03-22T04:18:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26de000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5642949c3400 0x564294adc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x5642949c3800 con 0x5642949c3400 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5642949c3400 0x564294adc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5642949c3400 0x564294adc000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2c00 con 0x564295649000 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 3 ==== osd_op_reply(8 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26de000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5642949c3800 0x564294adc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648c00 con 0x5642949c3800 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.385+0000 7f26e4400640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5642949c3800 0x564294adc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e4400640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5642949c3800 0x564294adc580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2000 con 0x564295648000 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 4 ==== osd_op_reply(10 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26de000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x564294ad2c00 0x564294adcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3000 con 0x564294ad2c00 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e3a00640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x564294ad2c00 0x564294adcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e3a00640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x564294ad2c00 0x564294adcb00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(11 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x5642949c3800 2024-03-22T04:18:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3400 con 0x564295648800 2024-03-22T04:18:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 97 out=49b] v93'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26dd600640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x564294ad2800 0x564294add080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3c00 con 0x564294ad2800 2024-03-22T04:18:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x564294ad2800 0x564294add080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e3000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x564294ad2800 0x564294add080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(14 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3800 con 0x5642949c3800 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.389+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x564295648000 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec000 con 0x564295649000 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3400 con 0x5642949c3400 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec400 con 0x564294ad2c00 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3800 con 0x564295648800 2024-03-22T04:18:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec800 con 0x564295649000 2024-03-22T04:18:21.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648400 con 0x5642949c3400 2024-03-22T04:18:21.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aecc00 con 0x564294ad2c00 2024-03-22T04:18:21.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648c00 con 0x564294ad2c00 2024-03-22T04:18:21.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed000 con 0x564295648000 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 6 ==== osd_op_reply(25 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648400 con 0x564295648000 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 7 ==== osd_op_reply(26 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec400 con 0x5642949c3800 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 4 ==== osd_op_reply(27 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.393+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2000 con 0x564295648800 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aecc00 con 0x564295649000 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3000 con 0x564294ad2800 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 2 ==== osd_op_reply(30 [pgnls-filter start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec000 con 0x564295648800 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x564294ad2800 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26de000640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x564294aec800 0x564294add600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed400 con 0x564294aec800 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e4400640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x564294aec800 0x564294add600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e4400640 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x564294aec800 0x564294add600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3c00 con 0x5642949c3400 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 4 ==== osd_op_reply(34 [pgnls-filter start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed800 con 0x564294ad2800 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3400 con 0x564294ad2c00 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 5 ==== osd_op_reply(36 [pgnls-filter start_epoch 97 out=49b] v93'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aedc00 con 0x564295648800 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.397+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec400 con 0x564294ad2800 2024-03-22T04:18:21.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aecc00 con 0x5642949c3400 2024-03-22T04:18:21.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 97 out=49b] v93'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec000 con 0x564295648800 2024-03-22T04:18:21.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 7 ==== osd_op_reply(40 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed800 con 0x564294aec800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== osd_op_reply(41 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed400 con 0x564294aec800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec400 con 0x5642949c3800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aedc00 con 0x564294ad2800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3000 con 0x564294aec800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3800 con 0x564294aec800 2024-03-22T04:18:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x564294aec800 2024-03-22T04:18:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2000 con 0x564294ad2c00 2024-03-22T04:18:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 6 ==== osd_op_reply(48 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.401+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3c00 con 0x564295648000 2024-03-22T04:18:21.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 8 ==== osd_op_reply(49 [pgnls-filter start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3400 con 0x5642949c3400 2024-03-22T04:18:21.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2000 con 0x564295648000 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 9 ==== osd_op_reply(51 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed000 con 0x564295648800 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3c00 con 0x5642949c3800 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec000 con 0x5642949c3800 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3000 con 0x564295649000 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 7 ==== osd_op_reply(55 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed800 con 0x564295648800 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 97 out=49b] v93'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3800 con 0x564295649000 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed400 con 0x564294aec800 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.405+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 97 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x564295648800 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 10 ==== osd_op_reply(59 [pgnls-filter start_epoch 97 out=81b] v97'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x5642949c3c00 con 0x564295648000 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4000 con 0x564295648000 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 11 ==== osd_op_reply(61 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec400 con 0x564295649000 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4400 con 0x564295648000 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 12 ==== osd_op_reply(63 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648c00 con 0x5642949c3400 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 7 ==== osd_op_reply(64 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4800 con 0x564295649000 2024-03-22T04:18:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 10 ==== osd_op_reply(65 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648400 con 0x5642949c3800 2024-03-22T04:18:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4c00 con 0x564295648000 2024-03-22T04:18:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 13 ==== osd_op_reply(67 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648c00 con 0x564294ad2c00 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.409+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5000 con 0x5642949c3800 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 9 ==== osd_op_reply(69 [pgnls-filter start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aecc00 con 0x564295648800 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 11 ==== osd_op_reply(70 [pgnls-filter start_epoch 97 out=49b] v93'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5400 con 0x564294ad2800 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aedc00 con 0x5642949c3800 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5800 con 0x564295648000 2024-03-22T04:18:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 14 ==== osd_op_reply(73 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec400 con 0x564295649000 2024-03-22T04:18:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 11 ==== osd_op_reply(74 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5c00 con 0x5642949c3400 2024-03-22T04:18:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 8 ==== osd_op_reply(75 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3400 con 0x564294ad2c00 2024-03-22T04:18:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4000 con 0x564295648800 2024-03-22T04:18:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3000 con 0x564295649000 2024-03-22T04:18:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 12 ==== osd_op_reply(78 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.413+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648c00 con 0x5642949c3400 2024-03-22T04:18:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 9 ==== osd_op_reply(79 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3800 con 0x564294ad2c00 2024-03-22T04:18:21.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 9 ==== osd_op_reply(80 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648400 con 0x564294ad2c00 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4400 con 0x564295648000 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 15 ==== osd_op_reply(82 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2000 con 0x564295648000 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 16 ==== osd_op_reply(83 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4c00 con 0x5642949c3800 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 11 ==== osd_op_reply(84 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3c00 con 0x564295648800 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 13 ==== osd_op_reply(85 [pgnls-filter start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5800 con 0x564295649000 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 13 ==== osd_op_reply(86 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x564294ad2800 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 8 ==== osd_op_reply(87 [pgnls-filter start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5000 con 0x564295648800 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 14 ==== osd_op_reply(88 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.417+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed000 con 0x564294ad2800 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 9 ==== osd_op_reply(89 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3000 con 0x564294aec800 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 8 ==== osd_op_reply(90 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec000 con 0x5642949c3400 2024-03-22T04:18:21.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 10 ==== osd_op_reply(91 [pgnls-filter start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x564294ad2800 2024-03-22T04:18:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 10 ==== osd_op_reply(92 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed800 con 0x564294ad2c00 2024-03-22T04:18:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 11 ==== osd_op_reply(93 [pgnls-filter start_epoch 97 out=49b] v93'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4800 con 0x564295648800 2024-03-22T04:18:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 15 ==== osd_op_reply(94 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aed400 con 0x564294ad2800 2024-03-22T04:18:21.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 11 ==== osd_op_reply(95 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5400 con 0x5642949c3400 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 11 ==== osd_op_reply(96 [pgnls-filter start_epoch 97 out=49b] v93'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x5642949c3c00 con 0x564295648800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 16 ==== osd_op_reply(97 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5c00 con 0x564294aec800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 9 ==== osd_op_reply(98 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.421+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x5642949c3c00 con 0x564294aec800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 10 ==== osd_op_reply(99 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5800 con 0x5642949c3800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 12 ==== osd_op_reply(100 [pgnls-filter start_epoch 97 out=49b] v93'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4000 con 0x564294ad2800 2024-03-22T04:18:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 12 ==== osd_op_reply(101 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295648c00 con 0x564294aec800 2024-03-22T04:18:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 11 ==== osd_op_reply(102 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4c00 con 0x564294aec800 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 12 ==== osd_op_reply(103 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aec400 con 0x564294aec800 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 13 ==== osd_op_reply(104 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af5000 con 0x564294ad2c00 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.2 v2:172.21.15.86:6816/4132808680 12 ==== osd_op_reply(105 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564294ad2c00 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b06000 con 0x564295648000 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 17 ==== osd_op_reply(106 [pgnls-filter start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294af4800 con 0x5642949c3400 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.0 v2:172.21.15.86:6808/2750248043 12 ==== osd_op_reply(107 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x5642949c3400 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b06400 con 0x564295648000 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 18 ==== osd_op_reply(108 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3c00 con 0x564295648800 2024-03-22T04:18:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 17 ==== osd_op_reply(109 [pgnls-filter start_epoch 97 out=49b] v93'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.425+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b06800 con 0x5642949c3800 2024-03-22T04:18:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 13 ==== osd_op_reply(110 [pgnls-filter start_epoch 97 out=49b] v93'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3000 con 0x5642949c3800 2024-03-22T04:18:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.5 v2:172.21.15.196:6808/2743139076 14 ==== osd_op_reply(111 [pgnls-filter start_epoch 97 out=49b] v93'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x5642949c3800 2024-03-22T04:18:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b06c00 con 0x564295649000 2024-03-22T04:18:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 14 ==== osd_op_reply(112 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aecc00 con 0x564295648800 2024-03-22T04:18:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 97 out=49b] v93'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b07000 con 0x564295649000 2024-03-22T04:18:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 15 ==== osd_op_reply(114 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294aedc00 con 0x564294aec800 2024-03-22T04:18:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 14 ==== osd_op_reply(115 [pgnls-filter start_epoch 97 out=49b] v93'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b07400 con 0x564295648800 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 19 ==== osd_op_reply(116 [pgnls-filter start_epoch 97 out=81b] v97'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 10 datascan.operator(): handling object 10000000001.0 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x564294aec000 con 0x564295648800 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 20 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2400 con 0x564294ad2800 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 13 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-03-22T04:18:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.429+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3c00 con 0x564295648800 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 21 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 20 datascan.get_frag_of: frag is * 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3000 con 0x564294ad2800 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 14 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3400 con 0x564294ad2800 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 15 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 20 datascan.read_dentry: file_head not found in result 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-03-22T04:18:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x564294ad3800 con 0x564294ad2800 2024-03-22T04:18:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 16 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v97'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-22T04:18:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294ad2000 con 0x564295648800 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 22 ==== osd_op_reply(123 [pgnls-filter start_epoch 97 out=49b] v93'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b07800 con 0x564295648000 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 19 ==== osd_op_reply(124 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b06c00 con 0x564294aec800 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.3 v2:172.21.15.86:6822/2837220459 15 ==== osd_op_reply(125 [pgnls-filter start_epoch 97 out=49b] v93'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564294aec800 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b07c00 con 0x564295648800 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e4400640 1 -- 172.21.15.86:0/402127278 <== osd.1 v2:172.21.15.86:6800/3761623609 23 ==== osd_op_reply(126 [pgnls-filter start_epoch 97 out=49b] v93'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0500 con 0x564295648800 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b07000 con 0x564295648000 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 <== osd.4 v2:172.21.15.196:6800/655314492 20 ==== osd_op_reply(127 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0780 con 0x564295648000 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26dd600640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564295528000 con 0x564294ad2800 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.7 v2:172.21.15.196:6817/2847924285 17 ==== osd_op_reply(128 [pgnls-filter start_epoch 97 out=49b] v93'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564294ad2800 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.433+0000 7f26de000640 1 -- 172.21.15.86:0/402127278 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 97 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x564294b06800 con 0x564295649000 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e3000640 1 -- 172.21.15.86:0/402127278 <== osd.6 v2:172.21.15.196:6816/1648471396 16 ==== osd_op_reply(129 [pgnls-filter start_epoch 97 out=49b] v93'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642947a0000 con 0x564295649000 2024-03-22T04:18:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5642949c3400 msgr2=0x564294adc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5642949c3400 0x564294adc000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x564295648800 msgr2=0x564295484c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x564295648800 0x564295484c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x564294ad2c00 msgr2=0x564294adcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x564294ad2c00 0x564294adcb00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x564294aec800 msgr2=0x564294add600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x564294aec800 0x564294add600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x564295648000 msgr2=0x564295485180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x564295648000 0x564295485180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5642949c3800 msgr2=0x564294adc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 reap_dead start 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5642949c3800 0x564294adc580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x564295649000 msgr2=0x564295484680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x564295649000 0x564295484680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x564294ad2800 msgr2=0x564294add080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x564294ad2800 0x564294add080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 reap_dead start 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e3a00640 1 -- 172.21.15.86:0/402127278 reap_dead start 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5642949c2800 msgr2=0x564295484100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5642949c2800 0x564295484100 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x56429544ecf0 tx=0x564294a4f020 comp rx=0 tx=0).stop 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564295649400 msgr2=0x564295483b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564295649400 0x564295483b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x5642956057d0 tx=0x5642947a54a0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 shutdown_connections 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5642949c2800 0x564295484100 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/402127278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564295649400 0x564295483b80 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 >> 172.21.15.86:0/402127278 conn(0x5642946cac00 msgr2=0x5642946f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 shutdown_connections 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/402127278 wait complete. 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/4259330568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642946cb000 msgr2=0x564295482b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642946cb000 0x564295482b00 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x564295522fc0 tx=0x5642954527e0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/4259330568 shutdown_connections 2024-03-22T04:18:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642946cb400 0x564295482580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642946cb000 0x564295482b00 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 --2- 172.21.15.86:0/4259330568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642946cb800 0x564295482000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/4259330568 >> 172.21.15.86:0/4259330568 conn(0x5642946cbc00 msgr2=0x5642946f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/4259330568 shutdown_connections 2024-03-22T04:18:21.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.437+0000 7f26e58a78c0 1 -- 172.21.15.86:0/4259330568 wait complete. 2024-03-22T04:18:21.442 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.154072, result: 2024-03-22T04:18:21.442 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-22T04:18:21.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.525+0000 7fd9d71cd8c0 1 -- start start 2024-03-22T04:18:21.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.525+0000 7fd9d71cd8c0 1 Processor -- start 2024-03-22T04:18:21.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c778ebb000 0x55c779c8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c778ebb400 0x55c779c8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c778ebb800 0x55c779c8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c779ca0180 con 0x55c778ebb400 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c779ca0300 con 0x55c778ebb000 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55c779ca0480 con 0x55c778ebb800 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c778ebb400 0x55c779c8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c778ebb400 0x55c779c8c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55224/0 (socket says 172.21.15.86:55224) 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 -- 172.21.15.86:0/4084950496 learned_addr learned my addr 172.21.15.86:0/4084950496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d5200640 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c778ebb000 0x55c779c8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c778ebb800 0x55c779c8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 -- 172.21.15.86:0/4084950496 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c778ebb800 msgr2=0x55c779c8cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c778ebb800 0x55c779c8cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 -- 172.21.15.86:0/4084950496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c778ebb000 msgr2=0x55c779c8c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c778ebb000 0x55c779c8c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 -- 172.21.15.86:0/4084950496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c778ee01e0 con 0x55c778ebb400 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d4800640 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c778ebb400 0x55c779c8c580 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x55c779d1f470 tx=0x55c778f94fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0ce211c768efae server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d2000640 1 -- 172.21.15.86:0/4084950496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c778ef04e0 con 0x55c778ebb400 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d2000640 1 -- 172.21.15.86:0/4084950496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c778f06e00 con 0x55c778ebb400 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d2000640 1 -- 172.21.15.86:0/4084950496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c778ef0b60 con 0x55c778ebb400 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/4084950496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c778ee14a0 con 0x55c778ebb400 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/4084950496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c778ee1a40 con 0x55c778ebb400 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 4 waiting for MDS map... 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d2000640 1 -- 172.21.15.86:0/4084950496 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x55c779c87000 con 0x55c778ebb400 2024-03-22T04:18:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d2000640 1 -- 172.21.15.86:0/4084950496 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 72) ==== 2885+0+0 (secure 0 0 0) 0x55c778f11180 con 0x55c778ebb400 2024-03-22T04:18:21.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 4 Got MDS map 72 2024-03-22T04:18:21.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 4 datascan.main: Using metadata pool output 2024-03-22T04:18:21.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 4 datascan.main: connecting to RADOS... 2024-03-22T04:18:21.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 Processor -- start 2024-03-22T04:18:21.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- start start 2024-03-22T04:18:21.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 0x55c779c8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c779ca0600 con 0x55c779dd1c00 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c779ca0780 con 0x55c779dd1800 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.529+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55c779ca0900 con 0x55c779dd1400 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55230/0 (socket says 172.21.15.86:55230) 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 -- 172.21.15.86:0/3036595188 learned_addr learned my addr 172.21.15.86:0/3036595188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d5200640 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 0x55c779c8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 -- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 msgr2=0x55c779c8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 0x55c779c8d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 -- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 msgr2=0x55c779c8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 -- 172.21.15.86:0/3036595188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c778ee1c20 con 0x55c779dd1c00 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d5200640 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:21.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x55c779df2660 tx=0x55c778f94060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aade22010fc170b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9cf800640 1 -- 172.21.15.86:0/3036595188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c778ef0340 con 0x55c779dd1c00 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9cf800640 1 -- 172.21.15.86:0/3036595188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c778f06c40 con 0x55c779dd1c00 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9cf800640 1 -- 172.21.15.86:0/3036595188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c778ef04e0 con 0x55c779dd1c00 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/3036595188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 msgr2=0x55c779c8db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x55c779df2660 tx=0x55c778f94060 comp rx=0 tx=0).stop 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/3036595188 shutdown_connections 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/3036595188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 0x55c779c8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/3036595188 >> 172.21.15.86:0/3036595188 conn(0x55c779dd1000 msgr2=0x55c778ee8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/3036595188 shutdown_connections 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/3036595188 wait complete. 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 Processor -- start 2024-03-22T04:18:21.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- start start 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 0x55c779c8d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c779ca1c80 con 0x55c779dd1c00 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c779ca1b00 con 0x55c779dd1800 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d71cd8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55c779ca1980 con 0x55c779dd1400 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55236/0 (socket says 172.21.15.86:55236) 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 learned_addr learned my addr 172.21.15.86:0/2443431658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 0x55c779c8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 msgr2=0x55c779c8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 0x55c779c8d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 msgr2=0x55c779c8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c778ee01e0 con 0x55c779dd1c00 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c779dd1400 0x55c779c8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.533+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c779dd1800 0x55c779c8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d4800640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x55c779df3920 tx=0x55c779dfd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9cda00640 1 -- 172.21.15.86:0/2443431658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c778ef0b60 con 0x55c779dd1c00 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9cda00640 1 -- 172.21.15.86:0/2443431658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c778f06a80 con 0x55c779dd1c00 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c778ee14a0 con 0x55c779dd1c00 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9cda00640 1 -- 172.21.15.86:0/2443431658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c778ef0000 con 0x55c779dd1c00 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c778ee1e00 con 0x55c779dd1c00 2024-03-22T04:18:21.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d71cd8c0 4 datascan.init: resolving metadata pool 13 2024-03-22T04:18:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9cda00640 1 -- 172.21.15.86:0/2443431658 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x55c7791b4c00 con 0x55c779dd1c00 2024-03-22T04:18:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9cda00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55c7791bf400 0x55c779c8e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9cda00640 1 -- 172.21.15.86:0/2443431658 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x55c7791bc600 con 0x55c779dd1c00 2024-03-22T04:18:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d71cd8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-22T04:18:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d71cd8c0 4 datascan.main: resolving metadata pool 13 2024-03-22T04:18:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x55c7791bf800 0x55c779c8e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55c7791bf400 0x55c779c8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7791bfc00 con 0x55c7791bf800 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x55c7791bf800 0x55c779c8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55c7791bf400 0x55c779c8e100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55c779c3e0c0 tx=0x55c778f941e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.537+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x55c7791bf800 0x55c779c8e680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== osd_op_reply(1 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55c7791bfc00 0x55c7792aa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792ae000 con 0x55c7791bfc00 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55c7791bfc00 0x55c7792aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55c7791bfc00 0x55c7792aa000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(2 [pgnls start_epoch 97 out=177b] v97'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55c7792ae000 0x55c7792aa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c779c9b000 con 0x55c7792ae000 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55c7792ae000 0x55c7792aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55c7792ae000 0x55c7792aa580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(3 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7792ae000 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c779c9b000 con 0x55c7791bf800 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== osd_op_reply(4 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c779c9b000 0x55c7792aab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792ae400 con 0x55c779c9b000 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d4800640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c779c9b000 0x55c7792aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d4800640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c779c9b000 0x55c7792aab00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(5 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d4800640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55c7792ae400 0x55c7792ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.541+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7791be800 con 0x55c7792ae400 2024-03-22T04:18:21.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55c7792ae400 0x55c7792ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55c7792ae400 0x55c7792ab600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(6 [pgnls start_epoch 97 out=149b] v96'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7791be800 con 0x55c7792ae400 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== osd_op_reply(7 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55c7791be800 0x55c7792abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792ae800 con 0x55c7791be800 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55c7791be800 0x55c7792abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5200640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55c7791be800 0x55c7792abb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(8 [pgnls start_epoch 97 out=73b] v96'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792ae800 con 0x55c7791bf800 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 3 ==== osd_op_reply(9 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792ae800 con 0x55c7792ae400 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 3 ==== osd_op_reply(10 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55c7792ae800 0x55c7792ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7792ae800 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d4800640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55c7792ae800 0x55c7792ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d4800640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55c7792ae800 0x55c7792ac100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.545+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(11 [pgnls start_epoch 97 out=81b] v96'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c778f90780 con 0x55c7792ae800 2024-03-22T04:18:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7792ae000 2024-03-22T04:18:21.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== osd_op_reply(12 [pgnls start_epoch 97 out=49b] v94'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7792ae000 2024-03-22T04:18:21.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c779c9b000 2024-03-22T04:18:21.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== osd_op_reply(13 [pgnls start_epoch 97 out=77b] v96'6 uv6 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7791bfc00 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 2 ==== osd_op_reply(14 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7791be800 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 2 ==== osd_op_reply(15 [pgnls start_epoch 97 out=73b] v96'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7791be800 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 3 ==== osd_op_reply(16 [pgnls start_epoch 97 out=49b] v94'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7791bf800 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 4 ==== osd_op_reply(17 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7792ae400 2024-03-22T04:18:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 4 ==== osd_op_reply(18 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7791be800 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 4 ==== osd_op_reply(19 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792aec00 con 0x55c7791bfc00 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 3 ==== osd_op_reply(20 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55c7792aec00 0x55c7792ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7792aec00 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.549+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55c7792aec00 0x55c7792ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55c7792aec00 0x55c7792ac680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(21 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792aec00 2024-03-22T04:18:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c779c9b000 2024-03-22T04:18:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 3 ==== osd_op_reply(22 [pgnls start_epoch 97 out=97b] v96'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7791be800 2024-03-22T04:18:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 5 ==== osd_op_reply(23 [pgnls start_epoch 97 out=74b] v92'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7791bf800 2024-03-22T04:18:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 5 ==== osd_op_reply(24 [pgnls start_epoch 97 out=73b] v96'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7792ae400 2024-03-22T04:18:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 5 ==== osd_op_reply(25 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7791bf800 2024-03-22T04:18:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 6 ==== osd_op_reply(26 [pgnls start_epoch 97 out=98b] v96'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c779c9b000 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 4 ==== osd_op_reply(27 [pgnls start_epoch 97 out=76b] v96'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7792aec00 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== osd_op_reply(28 [pgnls start_epoch 97 out=73b] v96'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90500 con 0x55c7792aec00 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c779c9b000 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 5 ==== osd_op_reply(29 [pgnls start_epoch 97 out=73b] v96'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7791bf800 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 7 ==== osd_op_reply(30 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7792aec00 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.2 v2:172.21.15.86:6816/4132808680 3 ==== osd_op_reply(31 [pgnls start_epoch 97 out=73b] v92'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90500 con 0x55c7792aec00 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.553+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7791bf800 2024-03-22T04:18:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 8 ==== osd_op_reply(32 [pgnls start_epoch 97 out=101b] v96'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af800 con 0x55c7791bfc00 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 4 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792afc00 con 0x55c7791bfc00 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 5 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d2000 con 0x55c7791bfc00 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 6 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 20 datascan.scan_links: injected ino detected 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d2400 con 0x55c7791bfc00 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 7 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-22T04:18:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d2800 con 0x55c7791bfc00 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 8 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d2c00 con 0x55c7792ae400 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 6 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3000 con 0x55c7791be800 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-03-22T04:18:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792ae800 2024-03-22T04:18:21.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.4 v2:172.21.15.196:6800/655314492 2 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55c778f90780 con 0x55c7792ae800 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3800 con 0x55c7791be800 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d2000 con 0x55c779c9b000 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 6 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-22T04:18:21.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d2400 con 0x55c7791bf800 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 9 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.557+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d2800 con 0x55c7791bf800 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 10 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-22T04:18:21.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3c00 con 0x55c779c9b000 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af400 con 0x55c7791bf800 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 11 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af800 con 0x55c7791bf800 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 12 ==== osd_op_reply(47 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af800 con 0x55c7791bfc00 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 9 ==== osd_op_reply(48 [pgnls start_epoch 97 out=177b] v97'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af800 con 0x55c7792ae000 2024-03-22T04:18:21.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.5 v2:172.21.15.196:6808/2743139076 3 ==== osd_op_reply(49 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7792ae000 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af800 con 0x55c7791bf800 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 13 ==== osd_op_reply(50 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af800 con 0x55c779c9b000 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 8 ==== osd_op_reply(51 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792ae400 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 7 ==== osd_op_reply(52 [pgnls start_epoch 97 out=149b] v96'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792ae400 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 8 ==== osd_op_reply(53 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791be800 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 8 ==== osd_op_reply(54 [pgnls start_epoch 97 out=73b] v96'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791bf800 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 14 ==== osd_op_reply(55 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.561+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792ae400 2024-03-22T04:18:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 9 ==== osd_op_reply(56 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792ae800 2024-03-22T04:18:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.4 v2:172.21.15.196:6800/655314492 3 ==== osd_op_reply(57 [pgnls start_epoch 97 out=81b] v96'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c778f90780 con 0x55c7792ae800 2024-03-22T04:18:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792ae000 2024-03-22T04:18:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.5 v2:172.21.15.196:6808/2743139076 4 ==== osd_op_reply(58 [pgnls start_epoch 97 out=49b] v94'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7792ae000 2024-03-22T04:18:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c779c9b000 2024-03-22T04:18:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 9 ==== osd_op_reply(59 [pgnls start_epoch 97 out=77b] v96'6 uv6 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791bfc00 2024-03-22T04:18:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 10 ==== osd_op_reply(60 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791be800 2024-03-22T04:18:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 9 ==== osd_op_reply(61 [pgnls start_epoch 97 out=73b] v96'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791be800 2024-03-22T04:18:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 10 ==== osd_op_reply(62 [pgnls start_epoch 97 out=49b] v94'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791bf800 2024-03-22T04:18:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 15 ==== osd_op_reply(63 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792ae400 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 10 ==== osd_op_reply(64 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791be800 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 11 ==== osd_op_reply(65 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791bfc00 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 11 ==== osd_op_reply(66 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.565+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792aec00 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.2 v2:172.21.15.86:6816/4132808680 4 ==== osd_op_reply(67 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792aec00 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c779c9b000 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 10 ==== osd_op_reply(68 [pgnls start_epoch 97 out=97b] v96'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791be800 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 12 ==== osd_op_reply(69 [pgnls start_epoch 97 out=74b] v92'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791bf800 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 16 ==== osd_op_reply(70 [pgnls start_epoch 97 out=73b] v96'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792ae400 2024-03-22T04:18:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 11 ==== osd_op_reply(71 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791bf800 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 17 ==== osd_op_reply(72 [pgnls start_epoch 97 out=98b] v96'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c779c9b000 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 11 ==== osd_op_reply(73 [pgnls start_epoch 97 out=76b] v96'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792aec00 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.2 v2:172.21.15.86:6816/4132808680 5 ==== osd_op_reply(74 [pgnls start_epoch 97 out=73b] v96'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90500 con 0x55c7792aec00 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c779c9b000 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 12 ==== osd_op_reply(75 [pgnls start_epoch 97 out=73b] v96'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791bf800 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 18 ==== osd_op_reply(76 [pgnls start_epoch 97 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7792aec00 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.2 v2:172.21.15.86:6816/4132808680 6 ==== osd_op_reply(77 [pgnls start_epoch 97 out=73b] v92'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c778f90500 con 0x55c7792aec00 2024-03-22T04:18:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 97 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3400 con 0x55c7791bf800 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 19 ==== osd_op_reply(78 [pgnls start_epoch 97 out=101b] v96'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.569+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792afc00 con 0x55c7791bfc00 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 12 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3c00 con 0x55c7791bfc00 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 13 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3000 con 0x55c7791bfc00 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 14 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 20 datascan.scan_links: injected ino detected 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-22T04:18:21.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d3800 con 0x55c7791bfc00 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 15 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d2400 con 0x55c7791bfc00 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 16 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d4000 con 0x55c7792ae400 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.6 v2:172.21.15.196:6816/1648471396 12 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90500 con 0x55c7792ae400 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d4400 con 0x55c7791be800 2024-03-22T04:18:21.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d4800 con 0x55c7792ae800 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.4 v2:172.21.15.196:6800/655314492 4 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55c778f90780 con 0x55c7792ae800 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d4c00 con 0x55c7791be800 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 14 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d5000 con 0x55c779c9b000 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 13 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-22T04:18:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d5400 con 0x55c7791bf800 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 20 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.573+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d5800 con 0x55c7791bf800 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 21 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d5c00 con 0x55c779c9b000 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d4800640 1 -- 172.21.15.86:0/2443431658 <== osd.3 v2:172.21.15.86:6822/2837220459 14 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c778f90780 con 0x55c779c9b000 2024-03-22T04:18:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d4800 con 0x55c7791bf800 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 22 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-03-22T04:18:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d5000 con 0x55c7791bfc00 2024-03-22T04:18:21.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 17 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d5c00 con 0x55c7791bfc00 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 18 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v97'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: processing 1 injected_inos 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d4400 con 0x55c7791bfc00 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 19 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 20 datascan.scan_links: first is now 2 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.577+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x55c7792d4c00 con 0x55c7791bfc00 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.581+0000 7fd9d5c00640 1 -- 172.21.15.86:0/2443431658 <== osd.7 v2:172.21.15.196:6817/2847924285 20 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v97'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55c778f90500 con 0x55c7791bfc00 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.581+0000 7fd9d71cd8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.581+0000 7fd9d71cd8c0 10 datascan.scan_links: updating inotable 2024-03-22T04:18:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.581+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af000 con 0x55c7791bf800 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.581+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 23 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.581+0000 7fd9d71cd8c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.581+0000 7fd9d71cd8c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.581+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af800 con 0x55c7791bf800 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.1 v2:172.21.15.86:6800/3761623609 24 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v97'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55c778f90000 con 0x55c7791bf800 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 10 datascan.scan_links: updating snaptable 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55c7792af400 con 0x55c7791be800 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 <== osd.0 v2:172.21.15.86:6808/2750248043 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55c778f90000 con 0x55c7791be800 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55c7791be800 msgr2=0x55c7792abb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x55c7791be800 0x55c7792abb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x55c7791bf800 msgr2=0x55c779c8e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x55c7791bf800 0x55c779c8e680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55c7792aec00 msgr2=0x55c7792ac680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55c7792aec00 0x55c7792ac680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c779c9b000 msgr2=0x55c7792aab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55c779c9b000 0x55c7792aab00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55c7792ae800 msgr2=0x55c7792ac100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55c7792ae800 0x55c7792ac100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55c7792ae000 msgr2=0x55c7792aa580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55c7792ae000 0x55c7792aa580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55c7792ae400 msgr2=0x55c7792ab600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55c7792ae400 0x55c7792ab600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55c7791bfc00 msgr2=0x55c7792aa000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55c7791bfc00 0x55c7792aa000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 reap_dead start 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 reap_dead start 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 reap_dead start 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d5200640 1 -- 172.21.15.86:0/2443431658 reap_dead start 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55c7791bf400 msgr2=0x55c779c8e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55c7791bf400 0x55c779c8e100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55c779c3e0c0 tx=0x55c778f941e0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 msgr2=0x55c779c8db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x55c779df3920 tx=0x55c779dfd800 comp rx=0 tx=0).stop 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 shutdown_connections 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55c7791bf400 0x55c779c8e100 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/2443431658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c779dd1c00 0x55c779c8db80 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 >> 172.21.15.86:0/2443431658 conn(0x55c779dd1000 msgr2=0x55c778ee8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 shutdown_connections 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/2443431658 wait complete. 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/4084950496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c778ebb400 msgr2=0x55c779c8c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c778ebb400 0x55c779c8c580 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x55c779d1f470 tx=0x55c778f94fc0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/4084950496 shutdown_connections 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c778ebb800 0x55c779c8cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c778ebb400 0x55c779c8c580 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 --2- 172.21.15.86:0/4084950496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c778ebb000 0x55c779c8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/4084950496 >> 172.21.15.86:0/4084950496 conn(0x55c778ebac00 msgr2=0x55c778ee7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/4084950496 shutdown_connections 2024-03-22T04:18:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.585+0000 7fd9d71cd8c0 1 -- 172.21.15.86:0/4084950496 wait complete. 2024-03-22T04:18:21.591 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.149281, result: 2024-03-22T04:18:21.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-22T04:18:21.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea7c00640 1 Processor -- start 2024-03-22T04:18:21.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea7c00640 1 -- start start 2024-03-22T04:18:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea7c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea00d0d30 0x7f7ea00b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea7c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea00d15f0 con 0x7f7ea0070560 2024-03-22T04:18:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea7c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea00d1310 con 0x7f7ea015e7e0 2024-03-22T04:18:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea7c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea006f970 con 0x7f7ea00d0d30 2024-03-22T04:18:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7ea0070560 0x7f7ea00cd360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45964/0 (socket says 172.21.15.86:45964) 2024-03-22T04:18:21.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9fe00640 1 -- 172.21.15.86:0/3438212483 learned_addr learned my addr 172.21.15.86:0/3438212483 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:21.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea5c00640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea00d0d30 0x7f7ea00b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345710888 0 0) 0x7f7ea00d15f0 con 0x7f7ea0070560 2024-03-22T04:18:21.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e88003530 con 0x7f7ea0070560 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4105998082 0 0) 0x7f7e88003530 con 0x7f7ea0070560 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea00d0d30 msgr2=0x7f7ea00b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea00d0d30 0x7f7ea00b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 >> v1:172.21.15.196:6789/0 conn(0x7f7ea015e7e0 legacy=0x7f7ea006fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea00cc880 con 0x7f7ea0070560 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f7e8c001680 con 0x7f7ea0070560 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 >> v1:172.21.15.86:6789/0 conn(0x7f7ea0070560 legacy=0x7f7ea00cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea00d0d30 0x7f7e880039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e88003f30 0x7f7e88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7e880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea006f970 con 0x7f7e88004d50 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8800bbc0 con 0x7f7e88003f30 2024-03-22T04:18:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8800bdc0 con 0x7f7ea00d0d30 2024-03-22T04:18:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea5c00640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7e880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7e9fe00640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e88003f30 0x7f7e88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea5200640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea00d0d30 0x7f7e880039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea5c00640 1 -- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea00d0d30 msgr2=0x7f7e880039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea5c00640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea00d0d30 0x7f7e880039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea5c00640 1 -- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e88003f30 msgr2=0x7f7e88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea5c00640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e88003f30 0x7f7e88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.773+0000 7f7ea5c00640 1 -- 172.21.15.86:0/3438212483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e88003530 con 0x7f7e88004d50 2024-03-22T04:18:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7ea5c00640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7e880051d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009fc0 tx=0x7f7e9801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee9cee2f47654c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980b2070 con 0x7f7e88004d50 2024-03-22T04:18:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e980a5c90 con 0x7f7e88004d50 2024-03-22T04:18:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7e9f400640 1 -- 172.21.15.86:0/3438212483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98009c10 con 0x7f7e88004d50 2024-03-22T04:18:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3438212483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 msgr2=0x7f7e880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7ea7c00640 1 --2- 172.21.15.86:0/3438212483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7e880051d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009fc0 tx=0x7f7e9801cc80 comp rx=0 tx=0).stop 2024-03-22T04:18:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7ea5c00640 1 -- 172.21.15.86:0/3438212483 reap_dead start 2024-03-22T04:18:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3438212483 shutdown_connections 2024-03-22T04:18:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3438212483 >> 172.21.15.86:0/3438212483 conn(0x7f7ea00b3bf0 msgr2=0x7f7ea00b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3438212483 shutdown_connections 2024-03-22T04:18:21.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.777+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3438212483 wait complete. 2024-03-22T04:18:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 Processor -- start 2024-03-22T04:18:21.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 -- start start 2024-03-22T04:18:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7ea015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea014a050 0x7f7ea01f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea006da10 0x7f7ea006de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea00b4710 con 0x7f7e88004d50 2024-03-22T04:18:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea00cce40 con 0x7f7ea014a050 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea00b4890 con 0x7f7ea006da10 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7ea015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7ea015b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55254/0 (socket says 172.21.15.86:55254) 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 -- 172.21.15.86:0/3705234435 learned_addr learned my addr 172.21.15.86:0/3705234435 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7e9fe00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea014a050 0x7f7ea01f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5c00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea006da10 0x7f7ea006de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 -- 172.21.15.86:0/3705234435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea006da10 msgr2=0x7f7ea006de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea006da10 0x7f7ea006de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 -- 172.21.15.86:0/3705234435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea014a050 msgr2=0x7f7ea01f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea014a050 0x7f7ea01f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 -- 172.21.15.86:0/3705234435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea014a8d0 con 0x7f7e88004d50 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea5200640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7ea015b460 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7e94011bd0 tx=0x7f7e9400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7e9cc00640 1 -- 172.21.15.86:0/3705234435 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94019020 con 0x7f7e88004d50 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7e9cc00640 1 -- 172.21.15.86:0/3705234435 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9400fcb0 con 0x7f7e88004d50 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea015ebc0 con 0x7f7e88004d50 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7e9cc00640 1 -- 172.21.15.86:0/3705234435 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94005310 con 0x7f7e88004d50 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.781+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea0161280 con 0x7f7e88004d50 2024-03-22T04:18:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.785+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea0074470 con 0x7f7e88004d50 2024-03-22T04:18:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.785+0000 7f7e9cc00640 1 -- 172.21.15.86:0/3705234435 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7e9401c020 con 0x7f7e88004d50 2024-03-22T04:18:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.785+0000 7f7e9cc00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e80067b50 0x7f7e8006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.785+0000 7f7e9cc00640 1 -- 172.21.15.86:0/3705234435 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f7e94089e20 con 0x7f7e88004d50 2024-03-22T04:18:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.785+0000 7f7e9fe00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e80067b50 0x7f7e8006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.785+0000 7f7e9fe00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e80067b50 0x7f7e8006a010 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7ea014a550 tx=0x7f7e8c000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:21.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.789+0000 7f7e9cc00640 1 -- 172.21.15.86:0/3705234435 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7e94023020 con 0x7f7e88004d50 2024-03-22T04:18:21.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:21.977+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7ea006fc30 con 0x7f7e88004d50 2024-03-22T04:18:22.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7e9cc00640 1 -- 172.21.15.86:0/3705234435 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v73) ==== 162+0+0 (secure 0 0 0) 0x7f7ea014a8d0 con 0x7f7e88004d50 2024-03-22T04:18:22.169 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-22T04:18:22.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e80067b50 msgr2=0x7f7e8006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e80067b50 0x7f7e8006a010 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7ea014a550 tx=0x7f7e8c000ff0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 msgr2=0x7f7ea015b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7ea015b460 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7e94011bd0 tx=0x7f7e9400f5a0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 shutdown_connections 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e80067b50 0x7f7e8006a010 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea014a050 0x7f7ea01f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea006da10 0x7f7ea006de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 --2- 172.21.15.86:0/3705234435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e88004d50 0x7f7ea015b460 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.169+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 >> 172.21.15.86:0/3705234435 conn(0x7f7ea00b3bf0 msgr2=0x7f7ea00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.173+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 shutdown_connections 2024-03-22T04:18:22.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.173+0000 7f7ea7c00640 1 -- 172.21.15.86:0/3705234435 wait complete. 2024-03-22T04:18:22.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:18:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7f00c00640 1 Processor -- start 2024-03-22T04:18:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7f00c00640 1 -- start start 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7f00c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc06ef40 0x7f7efc142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7f00c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7efc15c7c0 con 0x7f7efc0ad710 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7f00c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7efc06dea0 con 0x7f7efc0ad330 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7f00c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc06e180 con 0x7f7efc06ef40 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7efa800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7efc0ad710 0x7f7efc1402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45968/0 (socket says 172.21.15.86:45968) 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7efa800640 1 -- 172.21.15.86:0/1965537115 learned_addr learned my addr 172.21.15.86:0/1965537115 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7ef9400640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc06ef40 0x7f7efc142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3802345482 0 0) 0x7f7efc15c7c0 con 0x7f7efc0ad710 2024-03-22T04:18:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ee0003530 con 0x7f7efc0ad710 2024-03-22T04:18:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170547131 0 0) 0x7f7efc06dea0 con 0x7f7efc0ad330 2024-03-22T04:18:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.329+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7efc15c7c0 con 0x7f7efc0ad330 2024-03-22T04:18:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2167102109 0 0) 0x7f7ee0003530 con 0x7f7efc0ad710 2024-03-22T04:18:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc06ef40 msgr2=0x7f7efc142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc06ef40 0x7f7efc142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 >> v1:172.21.15.196:6789/0 conn(0x7f7efc0ad330 legacy=0x7f7efc13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc06e460 con 0x7f7efc0ad710 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f7ef4001a00 con 0x7f7efc0ad710 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 >> v1:172.21.15.86:6789/0 conn(0x7f7efc0ad710 legacy=0x7f7efc1402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee0003b80 0x7f7ee0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0004810 0x7f7ee0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00051d0 0x7f7ee000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc06e180 con 0x7f7ee0004810 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee000bff0 con 0x7f7ee0003b80 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee000c1f0 con 0x7f7ee00051d0 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9400640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc06ef40 0x7f7efc142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9e00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee0003b80 0x7f7ee0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9400640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0004810 0x7f7ee0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7efa800640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00051d0 0x7f7ee000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9e00640 1 -- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00051d0 msgr2=0x7f7ee000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9e00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00051d0 0x7f7ee000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9e00640 1 -- 172.21.15.86:0/1965537115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0004810 msgr2=0x7f7ee0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9e00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0004810 0x7f7ee0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9e00640 1 -- 172.21.15.86:0/1965537115 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee0003530 con 0x7f7ee0003b80 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7efa800640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00051d0 0x7f7ee000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9400640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0004810 0x7f7ee0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.333+0000 7f7ef9e00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee0003b80 0x7f7ee0003f80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0018c0 tx=0x7f7eec01ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e161917813cb4fe server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec001510 con 0x7f7ee0003b80 2024-03-22T04:18:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec0052d0 con 0x7f7ee0003b80 2024-03-22T04:18:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7ef8a00640 1 -- 172.21.15.86:0/1965537115 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0a8930 con 0x7f7ee0003b80 2024-03-22T04:18:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee0003b80 msgr2=0x7f7ee0003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 --2- 172.21.15.86:0/1965537115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee0003b80 0x7f7ee0003f80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0018c0 tx=0x7f7eec01ecf0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7efa800640 1 -- 172.21.15.86:0/1965537115 reap_dead start 2024-03-22T04:18:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- 172.21.15.86:0/1965537115 shutdown_connections 2024-03-22T04:18:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- 172.21.15.86:0/1965537115 >> 172.21.15.86:0/1965537115 conn(0x7f7efc0b1df0 msgr2=0x7f7efc0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- 172.21.15.86:0/1965537115 shutdown_connections 2024-03-22T04:18:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- 172.21.15.86:0/1965537115 wait complete. 2024-03-22T04:18:22.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 Processor -- start 2024-03-22T04:18:22.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- start start 2024-03-22T04:18:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0003b80 0x7f7efc1f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc150130 0x7f7efc1e6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7efc0b4060 0x7f7efc0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1409d0 con 0x7f7ee0003b80 2024-03-22T04:18:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc06ea20 con 0x7f7efc0b4060 2024-03-22T04:18:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7f00c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc142690 con 0x7f7efc150130 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7ef9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0003b80 0x7f7efc1f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7ef9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0003b80 0x7f7efc1f39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55276/0 (socket says 172.21.15.86:55276) 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.337+0000 7f7ef9e00640 1 -- 172.21.15.86:0/3815292090 learned_addr learned my addr 172.21.15.86:0/3815292090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7efa800640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7efc0b4060 0x7f7efc0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9400640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc150130 0x7f7efc1e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9e00640 1 -- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc150130 msgr2=0x7f7efc1e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9e00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc150130 0x7f7efc1e6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9e00640 1 -- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7efc0b4060 msgr2=0x7f7efc0b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9e00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7efc0b4060 0x7f7efc0b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9e00640 1 -- 172.21.15.86:0/3815292090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc1e6f80 con 0x7f7ee0003b80 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7efa800640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7efc0b4060 0x7f7efc0b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9400640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc150130 0x7f7efc1e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9e00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0003b80 0x7f7efc1f39e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f7eec005040 tx=0x7f7ee0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef2000640 1 -- 172.21.15.86:0/3815292090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0bf070 con 0x7f7ee0003b80 2024-03-22T04:18:22.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef2000640 1 -- 172.21.15.86:0/3815292090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec0014f0 con 0x7f7ee0003b80 2024-03-22T04:18:22.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc1e7260 con 0x7f7ee0003b80 2024-03-22T04:18:22.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef2000640 1 -- 172.21.15.86:0/3815292090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec005480 con 0x7f7ee0003b80 2024-03-22T04:18:22.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc142d30 con 0x7f7ee0003b80 2024-03-22T04:18:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7efc068650 con 0x7f7ee0003b80 2024-03-22T04:18:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef2000640 1 -- 172.21.15.86:0/3815292090 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7eec0c2070 con 0x7f7ee0003b80 2024-03-22T04:18:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef2000640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7edc067b20 0x7f7edc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef2000640 1 -- 172.21.15.86:0/3815292090 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f7eec1229d0 con 0x7f7ee0003b80 2024-03-22T04:18:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.341+0000 7f7ef9400640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7edc067b20 0x7f7edc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.345+0000 7f7ef9400640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7edc067b20 0x7f7edc069fe0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7efc150530 tx=0x7f7ee8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.345+0000 7f7ef2000640 1 -- 172.21.15.86:0/3815292090 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7eec018a00 con 0x7f7ee0003b80 2024-03-22T04:18:22.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.533+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7efc140590 con 0x7f7ee0003b80 2024-03-22T04:18:22.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.533+0000 7f7ef2000640 1 -- 172.21.15.86:0/3815292090 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5177 (secure 0 0 0) 0x7f7eec0f3000 con 0x7f7ee0003b80 2024-03-22T04:18:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:22.534 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:18:22.173727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76850":{"gid":76850,"name":"b","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.196:6835/1227183960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1227183960},{"type":"v1","addr":"172.21.15.196:6835","nonce":1227183960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-03-22T04:18:22.535 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 74 2024-03-22T04:18:22.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7edc067b20 msgr2=0x7f7edc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7edc067b20 0x7f7edc069fe0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7efc150530 tx=0x7f7ee8008040 comp rx=0 tx=0).stop 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0003b80 msgr2=0x7f7efc1f39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0003b80 0x7f7efc1f39e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f7eec005040 tx=0x7f7ee0005840 comp rx=0 tx=0).stop 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 shutdown_connections 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7edc067b20 0x7f7edc069fe0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7efc0b4060 0x7f7efc0b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7efc150130 0x7f7efc1e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 --2- 172.21.15.86:0/3815292090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0003b80 0x7f7efc1f39e0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 >> 172.21.15.86:0/3815292090 conn(0x7f7efc0b1df0 msgr2=0x7f7efc15f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 shutdown_connections 2024-03-22T04:18:22.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:22.537+0000 7f7f00c00640 1 -- 172.21.15.86:0/3815292090 wait complete. 2024-03-22T04:18:22.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:16:58.021326+0000', 'modified': '2024-03-22T04:18:22.173727+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 76850}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_76850': {'gid': 76850, 'name': 'b', 'rank': 0, 'incarnation': 74, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1227183960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1227183960}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1227183960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:18:22.957 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_76850:up:replay 2024-03-22T04:18:23.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:18:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fe0800640 1 Processor -- start 2024-03-22T04:18:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fe0800640 1 -- start start 2024-03-22T04:18:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fe0800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9fd815e630 0x7f9fd80cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fe0800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd806e2f0 0x7f9fd80d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fe0800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd81623d0 con 0x7f9fd806cc80 2024-03-22T04:18:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fe0800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9fd81620f0 con 0x7f9fd815e630 2024-03-22T04:18:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fe0800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd815ec10 con 0x7f9fd806e2f0 2024-03-22T04:18:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fde800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9fd806cc80 0x7f9fd806d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45984/0 (socket says 172.21.15.86:45984) 2024-03-22T04:18:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fde800640 1 -- 172.21.15.86:0/3597952168 learned_addr learned my addr 172.21.15.86:0/3597952168 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdde00640 1 --2- 172.21.15.86:0/3597952168 >> v2:172.21.15.196:3300/0 conn(0x7f9fd815e630 0x7f9fd80cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdd400640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd806e2f0 0x7f9fd80d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139794830 0 0) 0x7f9fd81623d0 con 0x7f9fd806cc80 2024-03-22T04:18:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fc0003530 con 0x7f9fd806cc80 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 768892504 0 0) 0x7f9fc0003530 con 0x7f9fd806cc80 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd806e2f0 msgr2=0x7f9fd80d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd806e2f0 0x7f9fd80d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 >> v2:172.21.15.196:3300/0 conn(0x7f9fd815e630 msgr2=0x7f9fd80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 --2- 172.21.15.86:0/3597952168 >> v2:172.21.15.196:3300/0 conn(0x7f9fd815e630 0x7f9fd80cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd815eef0 con 0x7f9fd806cc80 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f9fd4001a00 con 0x7f9fd806cc80 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 >> v1:172.21.15.86:6789/0 conn(0x7f9fd806cc80 legacy=0x7f9fd806d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.105+0000 7f9fdca00640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd806e2f0 0x7f9fc00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdca00640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fc0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdca00640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fc0004d50 0x7f9fc00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd81620f0 con 0x7f9fc0003f40 2024-03-22T04:18:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd815ec10 con 0x7f9fc0004d50 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc000b9d0 con 0x7f9fd806e2f0 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdd400640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fc0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdde00640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd806e2f0 0x7f9fc00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fde800640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fc0004d50 0x7f9fc00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdd400640 1 -- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd806e2f0 msgr2=0x7f9fc00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdd400640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd806e2f0 0x7f9fc00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdd400640 1 -- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fc0004d50 msgr2=0x7f9fc00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdd400640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fc0004d50 0x7f9fc00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdd400640 1 -- 172.21.15.86:0/3597952168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc0003530 con 0x7f9fc0003f40 2024-03-22T04:18:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdd400640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fc0004810 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009ae0 tx=0x7f9fc801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b22a4e93273e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80b2070 con 0x7f9fc0003f40 2024-03-22T04:18:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc80a5550 con 0x7f9fc0003f40 2024-03-22T04:18:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.109+0000 7f9fdca00640 1 -- 172.21.15.86:0/3597952168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80096b0 con 0x7f9fc0003f40 2024-03-22T04:18:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- 172.21.15.86:0/3597952168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 msgr2=0x7f9fc0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 --2- 172.21.15.86:0/3597952168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fc0004810 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009ae0 tx=0x7f9fc801ca10 comp rx=0 tx=0).stop 2024-03-22T04:18:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fde800640 1 -- 172.21.15.86:0/3597952168 reap_dead start 2024-03-22T04:18:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- 172.21.15.86:0/3597952168 shutdown_connections 2024-03-22T04:18:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- 172.21.15.86:0/3597952168 >> 172.21.15.86:0/3597952168 conn(0x7f9fd806f910 msgr2=0x7f9fd806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- 172.21.15.86:0/3597952168 shutdown_connections 2024-03-22T04:18:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- 172.21.15.86:0/3597952168 wait complete. 2024-03-22T04:18:24.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 Processor -- start 2024-03-22T04:18:24.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- start start 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fd815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd815c220 0x7f9fd815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd80c9370 0x7f9fd80c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd80d1f60 con 0x7f9fc0003f40 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd815f4b0 con 0x7f9fd80c9370 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fe0800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd815f9d0 con 0x7f9fd815c220 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fd815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fd815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55294/0 (socket says 172.21.15.86:55294) 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdde00640 1 -- 172.21.15.86:0/2264034380 learned_addr learned my addr 172.21.15.86:0/2264034380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdd400640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd815c220 0x7f9fd815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fde800640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd80c9370 0x7f9fd80c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdde00640 1 -- 172.21.15.86:0/2264034380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd815c220 msgr2=0x7f9fd815c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:24.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdde00640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd815c220 0x7f9fd815c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdde00640 1 -- 172.21.15.86:0/2264034380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd80c9370 msgr2=0x7f9fd80c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdde00640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd80c9370 0x7f9fd80c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.113+0000 7f9fdde00640 1 -- 172.21.15.86:0/2264034380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd815bd50 con 0x7f9fc0003f40 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fdde00640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fd815b810 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc011ad0 tx=0x7f9fcc011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fd2000640 1 -- 172.21.15.86:0/2264034380 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00faf0 con 0x7f9fc0003f40 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fd2000640 1 -- 172.21.15.86:0/2264034380 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc00fc90 con 0x7f9fc0003f40 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fd2000640 1 -- 172.21.15.86:0/2264034380 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc014440 con 0x7f9fc0003f40 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd80c9f80 con 0x7f9fc0003f40 2024-03-22T04:18:24.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd80ca540 con 0x7f9fc0003f40 2024-03-22T04:18:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd815cbe0 con 0x7f9fc0003f40 2024-03-22T04:18:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fd2000640 1 -- 172.21.15.86:0/2264034380 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9fcc012040 con 0x7f9fc0003f40 2024-03-22T04:18:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fd2000640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9fbc067b50 0x7f9fbc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fd2000640 1 -- 172.21.15.86:0/2264034380 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f9fcc086840 con 0x7f9fc0003f40 2024-03-22T04:18:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.117+0000 7f9fdd400640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9fbc067b50 0x7f9fbc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.121+0000 7f9fdd400640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9fbc067b50 0x7f9fbc06a010 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80af000 tx=0x7f9fc80046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:24.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.125+0000 7f9fd2000640 1 -- 172.21.15.86:0/2264034380 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9fcc019030 con 0x7f9fc0003f40 2024-03-22T04:18:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fd8068650 con 0x7f9fc0003f40 2024-03-22T04:18:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fd2000640 1 -- 172.21.15.86:0/2264034380 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5177 (secure 0 0 0) 0x7f9fcc056f00 con 0x7f9fc0003f40 2024-03-22T04:18:24.313 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:24.313 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:18:23.182782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76850":{"gid":76850,"name":"b","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.196:6835/1227183960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1227183960},{"type":"v1","addr":"172.21.15.196:6835","nonce":1227183960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-03-22T04:18:24.314 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 76 2024-03-22T04:18:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9fbc067b50 msgr2=0x7f9fbc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9fbc067b50 0x7f9fbc06a010 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80af000 tx=0x7f9fc80046d0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 msgr2=0x7f9fd815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fd815b810 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc011ad0 tx=0x7f9fcc011ed0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 shutdown_connections 2024-03-22T04:18:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9fbc067b50 0x7f9fbc06a010 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd80c9370 0x7f9fd80c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd815c220 0x7f9fd815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 --2- 172.21.15.86:0/2264034380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fc0003f40 0x7f9fd815b810 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 >> 172.21.15.86:0/2264034380 conn(0x7f9fd806f910 msgr2=0x7f9fd80b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 shutdown_connections 2024-03-22T04:18:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:24.313+0000 7f9fe0800640 1 -- 172.21.15.86:0/2264034380 wait complete. 2024-03-22T04:18:24.738 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:16:58.021326+0000', 'modified': '2024-03-22T04:18:23.182782+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 76850}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_76850': {'gid': 76850, 'name': 'b', 'rank': 0, 'incarnation': 74, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.196:6835/1227183960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1227183960}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1227183960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:18:24.738 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_76850:up:rejoin 2024-03-22T04:18:25.739 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:18:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.885+0000 7fc60d400640 1 Processor -- start 2024-03-22T04:18:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.885+0000 7fc60d400640 1 -- start start 2024-03-22T04:18:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc60d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc60815e630 0x7fc6080cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc60d400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 0x7fc60806d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc60d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6080d07d0 con 0x7fc60806e2f0 2024-03-22T04:18:25.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc60d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc6081623d0 con 0x7fc60815e630 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc60d400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc6081620f0 con 0x7fc6080d03b0 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc606800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc60815e630 0x7fc6080cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc606800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc60815e630 0x7fc6080cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43178/0 (socket says 172.21.15.86:43178) 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc606800640 1 -- 172.21.15.86:0/786294819 learned_addr learned my addr 172.21.15.86:0/786294819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc605e00640 1 --1- 172.21.15.86:0/786294819 >> v1:172.21.15.86:6789/0 conn(0x7fc60806e2f0 0x7fc60806cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33746/0 (socket says 172.21.15.86:33746) 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc607200640 1 --2- 172.21.15.86:0/786294819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 0x7fc60806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc606800640 1 -- 172.21.15.86:0/786294819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 msgr2=0x7fc60806d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc606800640 1 --2- 172.21.15.86:0/786294819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 0x7fc60806d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc606800640 1 -- 172.21.15.86:0/786294819 >> v1:172.21.15.86:6789/0 conn(0x7fc60806e2f0 legacy=0x7fc60806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:25.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc606800640 1 -- 172.21.15.86:0/786294819 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60815ec10 con 0x7fc60815e630 2024-03-22T04:18:25.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.889+0000 7fc606800640 1 --2- 172.21.15.86:0/786294819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc6080cfe70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009870 tx=0x7fc5f801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28d6e07dc34e92 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc605400640 1 -- 172.21.15.86:0/786294819 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80b2070 con 0x7fc60815e630 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc605400640 1 -- 172.21.15.86:0/786294819 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f80a9070 con 0x7fc60815e630 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc605400640 1 -- 172.21.15.86:0/786294819 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80ad880 con 0x7fc60815e630 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- 172.21.15.86:0/786294819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 msgr2=0x7fc6080cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 --2- 172.21.15.86:0/786294819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc6080cfe70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009870 tx=0x7fc5f801c560 comp rx=0 tx=0).stop 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- 172.21.15.86:0/786294819 shutdown_connections 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 --2- 172.21.15.86:0/786294819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 0x7fc60806d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 --2- 172.21.15.86:0/786294819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc6080cfe70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- 172.21.15.86:0/786294819 >> 172.21.15.86:0/786294819 conn(0x7fc60806f910 msgr2=0x7fc60806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- 172.21.15.86:0/786294819 shutdown_connections 2024-03-22T04:18:25.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- 172.21.15.86:0/786294819 wait complete. 2024-03-22T04:18:25.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 Processor -- start 2024-03-22T04:18:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- start start 2024-03-22T04:18:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc60806e2f0 0x7fc60815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 0x7fc6080c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc60815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc60815f9d0 con 0x7fc60815e630 2024-03-22T04:18:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6080d1f60 con 0x7fc60806e2f0 2024-03-22T04:18:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc60d400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc6080cfaf0 con 0x7fc6080d03b0 2024-03-22T04:18:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc607200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc60815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc607200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc60815fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43192/0 (socket says 172.21.15.86:43192) 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc607200640 1 -- 172.21.15.86:0/3964889273 learned_addr learned my addr 172.21.15.86:0/3964889273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc606800640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc60806e2f0 0x7fc60815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.893+0000 7fc605e00640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 0x7fc6080c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc607200640 1 -- 172.21.15.86:0/3964889273 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 msgr2=0x7fc6080c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc607200640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 0x7fc6080c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc607200640 1 -- 172.21.15.86:0/3964889273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc60806e2f0 msgr2=0x7fc60815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc607200640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc60806e2f0 0x7fc60815ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc607200640 1 -- 172.21.15.86:0/3964889273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6080c7960 con 0x7fc60815e630 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc607200640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc60815fb50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fc60000efc0 tx=0x7fc60000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc5fea00640 1 -- 172.21.15.86:0/3964889273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600018020 con 0x7fc60815e630 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc5fea00640 1 -- 172.21.15.86:0/3964889273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc600014480 con 0x7fc60815e630 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6080c7c40 con 0x7fc60815e630 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc5fea00640 1 -- 172.21.15.86:0/3964889273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600014870 con 0x7fc60815e630 2024-03-22T04:18:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6080c84e0 con 0x7fc60815e630 2024-03-22T04:18:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6080c86e0 con 0x7fc60815e630 2024-03-22T04:18:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc5fea00640 1 -- 172.21.15.86:0/3964889273 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc60001b020 con 0x7fc60815e630 2024-03-22T04:18:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc5fea00640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5e8067b20 0x7fc5e8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc5fea00640 1 -- 172.21.15.86:0/3964889273 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fc600086bb0 con 0x7fc60815e630 2024-03-22T04:18:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.897+0000 7fc606800640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5e8067b20 0x7fc5e8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.901+0000 7fc606800640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5e8067b20 0x7fc5e8069fe0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80af000 tx=0x7fc5f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:25.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:25.901+0000 7fc5fea00640 1 -- 172.21.15.86:0/3964889273 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc600022070 con 0x7fc60815e630 2024-03-22T04:18:26.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6080c7e10 con 0x7fc60815e630 2024-03-22T04:18:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc5fea00640 1 -- 172.21.15.86:0/3964889273 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5186 (secure 0 0 0) 0x7fc600010040 con 0x7fc60815e630 2024-03-22T04:18:26.089 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:26.089 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":77,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:18:25.189133+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76850":{"gid":76850,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.196:6835/1227183960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1227183960},{"type":"v1","addr":"172.21.15.196:6835","nonce":1227183960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76850,"qdb_cluster":[76850]},"id":6}]} 2024-03-22T04:18:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 77 2024-03-22T04:18:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5e8067b20 msgr2=0x7fc5e8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5e8067b20 0x7fc5e8069fe0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80af000 tx=0x7fc5f8008040 comp rx=0 tx=0).stop 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 msgr2=0x7fc60815fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc60815fb50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fc60000efc0 tx=0x7fc60000c490 comp rx=0 tx=0).stop 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 shutdown_connections 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5e8067b20 0x7fc5e8069fe0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60815e630 0x7fc60815fb50 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6080d03b0 0x7fc6080c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 --2- 172.21.15.86:0/3964889273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc60806e2f0 0x7fc60815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.089+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 >> 172.21.15.86:0/3964889273 conn(0x7fc60806f910 msgr2=0x7fc6080ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.093+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 shutdown_connections 2024-03-22T04:18:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.093+0000 7fc60d400640 1 -- 172.21.15.86:0/3964889273 wait complete. 2024-03-22T04:18:26.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:16:58.021326+0000', 'modified': '2024-03-22T04:18:25.189133+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 76850}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_76850': {'gid': 76850, 'name': 'b', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.196:6835/1227183960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1227183960}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1227183960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 76850, 'qdb_cluster': [76850]} 2024-03-22T04:18:26.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:18:26.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T04:18:26.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.649+0000 7f0d7c400640 1 Processor -- start 2024-03-22T04:18:26.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.649+0000 7f0d7c400640 1 -- start start 2024-03-22T04:18:26.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.649+0000 7f0d7c400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d740730d0 0x7f0d740ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.649+0000 7f0d7c400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0d740cc3e0 0x7f0d740ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.649+0000 7f0d7c400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d74071920 con 0x7f0d7415b3f0 2024-03-22T04:18:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.649+0000 7f0d7c400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0d740716d0 con 0x7f0d740730d0 2024-03-22T04:18:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.649+0000 7f0d7c400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0d7406d990 con 0x7f0d740cc3e0 2024-03-22T04:18:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d79000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d740730d0 0x7f0d740ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d79000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d740730d0 0x7f0d740ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43200/0 (socket says 172.21.15.86:43200) 2024-03-22T04:18:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d79a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0d7415b3f0 0x7f0d740729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33754/0 (socket says 172.21.15.86:33754) 2024-03-22T04:18:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d79000640 1 -- 172.21.15.86:0/3251113689 learned_addr learned my addr 172.21.15.86:0/3251113689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d7a400640 1 --2- 172.21.15.86:0/3251113689 >> v2:172.21.15.196:3300/0 conn(0x7f0d740cc3e0 0x7f0d740ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d79000640 1 -- 172.21.15.86:0/3251113689 >> v2:172.21.15.196:3300/0 conn(0x7f0d740cc3e0 msgr2=0x7f0d740ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d79000640 1 --2- 172.21.15.86:0/3251113689 >> v2:172.21.15.196:3300/0 conn(0x7f0d740cc3e0 0x7f0d740ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d79000640 1 -- 172.21.15.86:0/3251113689 >> v1:172.21.15.86:6789/0 conn(0x7f0d7415b3f0 legacy=0x7f0d740729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d79000640 1 -- 172.21.15.86:0/3251113689 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d74071c00 con 0x7f0d740730d0 2024-03-22T04:18:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.653+0000 7f0d73e00640 1 -- 172.21.15.86:0/3251113689 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451932242 0 0) 0x7f0d74071920 con 0x7f0d7415b3f0 2024-03-22T04:18:26.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d79000640 1 --2- 172.21.15.86:0/3251113689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d740ade30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0d74073b20 tx=0x7f0d6801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b228dbe46873b1e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d73e00640 1 -- 172.21.15.86:0/3251113689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680ab610 con 0x7f0d740730d0 2024-03-22T04:18:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d73e00640 1 -- 172.21.15.86:0/3251113689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d680b0070 con 0x7f0d740730d0 2024-03-22T04:18:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d73e00640 1 -- 172.21.15.86:0/3251113689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680ab9d0 con 0x7f0d740730d0 2024-03-22T04:18:26.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- 172.21.15.86:0/3251113689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 msgr2=0x7f0d740ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 --2- 172.21.15.86:0/3251113689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d740ade30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0d74073b20 tx=0x7f0d6801c520 comp rx=0 tx=0).stop 2024-03-22T04:18:26.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- 172.21.15.86:0/3251113689 shutdown_connections 2024-03-22T04:18:26.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 --2- 172.21.15.86:0/3251113689 >> v2:172.21.15.196:3300/0 conn(0x7f0d740cc3e0 0x7f0d740ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 --2- 172.21.15.86:0/3251113689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d740ade30 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- 172.21.15.86:0/3251113689 >> 172.21.15.86:0/3251113689 conn(0x7f0d740b1df0 msgr2=0x7f0d740b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:26.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- 172.21.15.86:0/3251113689 shutdown_connections 2024-03-22T04:18:26.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- 172.21.15.86:0/3251113689 wait complete. 2024-03-22T04:18:26.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 Processor -- start 2024-03-22T04:18:26.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- start start 2024-03-22T04:18:26.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d7414aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d740cc3e0 0x7f0d7414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d7415b3f0 0x7f0d740cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74168580 con 0x7f0d740730d0 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74168400 con 0x7f0d7415b3f0 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.657+0000 7f0d7c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0d74168700 con 0x7f0d740cc3e0 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d7414aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d7414aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43214/0 (socket says 172.21.15.86:43214) 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 -- 172.21.15.86:0/2009821618 learned_addr learned my addr 172.21.15.86:0/2009821618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79000640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d740cc3e0 0x7f0d7414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d7a400640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d7415b3f0 0x7f0d740cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 -- 172.21.15.86:0/2009821618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d740cc3e0 msgr2=0x7f0d7414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d740cc3e0 0x7f0d7414a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 -- 172.21.15.86:0/2009821618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d7415b3f0 msgr2=0x7f0d740cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d7415b3f0 0x7f0d740cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 -- 172.21.15.86:0/2009821618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d740c9f20 con 0x7f0d740730d0 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d79a00640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d7414aef0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f0d640011f0 tx=0x7f0d6400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d71600640 1 -- 172.21.15.86:0/2009821618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64018070 con 0x7f0d740730d0 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d71600640 1 -- 172.21.15.86:0/2009821618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d640074e0 con 0x7f0d740730d0 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d7c400640 1 -- 172.21.15.86:0/2009821618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d740ca1c0 con 0x7f0d740730d0 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d71600640 1 -- 172.21.15.86:0/2009821618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64004a10 con 0x7f0d740730d0 2024-03-22T04:18:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d7c400640 1 -- 172.21.15.86:0/2009821618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d741536b0 con 0x7f0d740730d0 2024-03-22T04:18:26.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d71600640 1 -- 172.21.15.86:0/2009821618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0d6401b070 con 0x7f0d740730d0 2024-03-22T04:18:26.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.661+0000 7f0d71600640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d58067b20 0x7f0d58069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d71600640 1 -- 172.21.15.86:0/2009821618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f0d640854a0 con 0x7f0d740730d0 2024-03-22T04:18:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d79000640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d58067b20 0x7f0d58069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 Processor -- start 2024-03-22T04:18:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- start start 2024-03-22T04:18:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d79000640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d58067b20 0x7f0d58069fe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0d680af000 tx=0x7f0d680a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208eee70 0x5623208ef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x5623208a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623206a0140 con 0x5623208a2c20 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623206f3b10 con 0x5623208ef790 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56232067ac50 con 0x5623208eee70 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x5623208a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x5623208a3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43224/0 (socket says 172.21.15.86:43224) 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 -- 172.21.15.86:0/1961144745 learned_addr learned my addr 172.21.15.86:0/1961144745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d79000640 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d79a00640 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208eee70 0x5623208ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 -- 172.21.15.86:0/1961144745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208eee70 msgr2=0x5623208ef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208eee70 0x5623208ef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 -- 172.21.15.86:0/1961144745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 msgr2=0x5623208a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 -- 172.21.15.86:0/1961144745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562320738f20 con 0x5623208a2c20 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7a400640 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x5623208a3000 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c011550 tx=0x7f0d6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a284fa590690269 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d62000640 1 -- 172.21.15.86:0/1961144745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c015840 con 0x5623208a2c20 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d62000640 1 -- 172.21.15.86:0/1961144745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c0159e0 con 0x5623208a2c20 2024-03-22T04:18:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d62000640 1 -- 172.21.15.86:0/1961144745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c015d90 con 0x5623208a2c20 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/1961144745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 msgr2=0x5623208a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x5623208a3000 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c011550 tx=0x7f0d6c009750 comp rx=0 tx=0).stop 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/1961144745 shutdown_connections 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x5623208a3000 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/1961144745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208eee70 0x5623208ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/1961144745 >> 172.21.15.86:0/1961144745 conn(0x5623208cfc10 msgr2=0x5623208d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/1961144745 shutdown_connections 2024-03-22T04:18:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.665+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/1961144745 wait complete. 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 -- start start 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 Processor -- start 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 0x5623208aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208eee70 0x5623208a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623206adab0 con 0x5623208eee70 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623206efac0 con 0x5623208ef790 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x562320733a60 con 0x5623208a2c20 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 10 client.0 _ll_register_callbacks cb 0x562320890380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208eee70 0x5623208a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208eee70 0x5623208a77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43230/0 (socket says 172.21.15.86:43230) 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 -- 172.21.15.86:0/3398911384 learned_addr learned my addr 172.21.15.86:0/3398911384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79a00640 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 0x5623208aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7a400640 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 -- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 msgr2=0x5623208aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 0x5623208aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 -- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 msgr2=0x5623208a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 -- 172.21.15.86:0/3398911384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623206c9570 con 0x5623208eee70 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7a400640 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79000640 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208eee70 0x5623208a77e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0d6801ca90 tx=0x7f0d6801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e723a1a90c09f514 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d56a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d56a00640 1 -- 172.21.15.86:0/3398911384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680ce060 con 0x5623208eee70 2024-03-22T04:18:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d56a00640 1 -- 172.21.15.86:0/3398911384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f0d680c0590 con 0x5623208eee70 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/3398911384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6c0159e0 con 0x5623208eee70 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d56a00640 1 -- 172.21.15.86:0/3398911384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0d680c0790 con 0x5623208eee70 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 10 client.8049 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/3398911384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5623206bf880 con 0x5623208eee70 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d56a00640 1 -- 172.21.15.86:0/3398911384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c0159e0 con 0x5623208eee70 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d56a00640 1 -- 172.21.15.86:0/3398911384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f0d740b4b10 con 0x5623208eee70 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 10 client.8049 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 10 client.8049 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 20 client.8049 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5623208a8a90 0x562320900120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 4 client.8049 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/3398911384 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56232068ebc0 con 0x5623208a8a90 2024-03-22T04:18:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.669+0000 7f0d79a00640 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5623208a8a90 0x562320900120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d56a00640 10 client.8049 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d79a00640 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5623208a8a90 0x562320900120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855d7e5ce4b23094 server_cookie=d65017f36f6bf335 in_seq=0 out_seq=0 2024-03-22T04:18:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d56a00640 1 -- 172.21.15.86:0/3398911384 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56232068ebc0 con 0x5623208a8a90 2024-03-22T04:18:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d56a00640 10 client.8049 handle_command_reply: tid=0 2024-03-22T04:18:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 client.8049 shutdown 2024-03-22T04:18:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/3398911384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208eee70 msgr2=0x5623208a77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208eee70 0x5623208a77e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0d6801ca90 tx=0x7f0d6801ad60 comp rx=0 tx=0).stop 2024-03-22T04:18:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/3398911384 shutdown_connections 2024-03-22T04:18:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5623208a8a90 0x562320900120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208ef790 0x5623208a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208eee70 0x5623208a77e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d56a00640 0 client.8049 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/3398911384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 0x5623208aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/3398911384 >> 172.21.15.86:0/3398911384 conn(0x5623208cfc10 msgr2=0x5623208d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/3398911384 shutdown_connections 2024-03-22T04:18:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/3398911384 wait complete. 2024-03-22T04:18:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.673+0000 7f0d7f7d0000 20 client.8049 trim_cache size 0 max 16384 2024-03-22T04:18:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 Processor -- start 2024-03-22T04:18:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- start start 2024-03-22T04:18:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208cfc10 0x5623208ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 0x5623208ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 0x5623208a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623208ab110 con 0x5623208cfc10 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74204a30 con 0x5623208eee70 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0d74203a10 con 0x5623208a2c20 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208cfc10 0x5623208ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208cfc10 0x5623208ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43232/0 (socket says 172.21.15.86:43232) 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 -- 172.21.15.86:0/756821435 learned_addr learned my addr 172.21.15.86:0/756821435 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79000640 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 0x5623208ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7a400640 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 0x5623208a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 -- 172.21.15.86:0/756821435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 msgr2=0x5623208ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 0x5623208ee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 -- 172.21.15.86:0/756821435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 msgr2=0x5623208a73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 0x5623208a73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 -- 172.21.15.86:0/756821435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232067be80 con 0x5623208cfc10 2024-03-22T04:18:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d79a00640 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208cfc10 0x5623208ee350 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0d64083680 tx=0x7f0d64009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221c12f9b9d3cee8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d61600640 1 -- 172.21.15.86:0/756821435 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64051b80 con 0x5623208cfc10 2024-03-22T04:18:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d61600640 1 -- 172.21.15.86:0/756821435 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d64059120 con 0x5623208cfc10 2024-03-22T04:18:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d61600640 1 -- 172.21.15.86:0/756821435 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64051d20 con 0x5623208cfc10 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/756821435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208cfc10 msgr2=0x5623208ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208cfc10 0x5623208ee350 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0d64083680 tx=0x7f0d64009040 comp rx=0 tx=0).stop 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/756821435 shutdown_connections 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 0x5623208a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208a2c20 0x5623208ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/756821435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208cfc10 0x5623208ee350 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/756821435 >> 172.21.15.86:0/756821435 conn(0x5623208a1ee0 msgr2=0x5623208a9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/756821435 shutdown_connections 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/756821435 wait complete. 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 -- start start 2024-03-22T04:18:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.693+0000 7f0d7f7d0000 1 Processor -- start 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x56232095e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208cfc10 0x56232095ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 0x562320900ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74203360 con 0x5623208a2c20 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741549e0 con 0x5623208eee70 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5623206f2e20 con 0x5623208cfc10 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 10 client.0 _ll_register_callbacks cb 0x7f0d680ab510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x56232095e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x56232095e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43240/0 (socket says 172.21.15.86:43240) 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 -- 172.21.15.86:0/2879542888 learned_addr learned my addr 172.21.15.86:0/2879542888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79000640 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208cfc10 0x56232095ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7a400640 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 0x562320900ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 -- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208cfc10 msgr2=0x56232095ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208cfc10 0x56232095ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 -- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 msgr2=0x562320900ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 0x562320900ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 -- 172.21.15.86:0/2879542888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562320687750 con 0x5623208a2c20 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79a00640 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x56232095e7d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400ceb0 tx=0x7f0d64058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6f1c737db012fa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 1 -- 172.21.15.86:0/2879542888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64051b80 con 0x5623208a2c20 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 1 -- 172.21.15.86:0/2879542888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f0d6405d950 con 0x5623208a2c20 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/2879542888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d64059120 con 0x5623208a2c20 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 1 -- 172.21.15.86:0/2879542888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0d64051d20 con 0x5623208a2c20 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 10 client.8055 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/2879542888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x56232069a450 con 0x5623208a2c20 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 1 -- 172.21.15.86:0/2879542888 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d64059120 con 0x5623208a2c20 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 1 -- 172.21.15.86:0/2879542888 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f0d64004bb0 con 0x5623208a2c20 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 10 client.8055 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 10 client.8055 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 20 client.8055 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5623208ef790 0x562320901420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 4 client.8055 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/2879542888 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "status"}) -- 0x562320701050 con 0x5623208ef790 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79000640 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5623208ef790 0x562320901420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 10 client.8055 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d79000640 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5623208ef790 0x562320901420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda8edc344839b57 server_cookie=6e926185de397c66 in_seq=0 out_seq=0 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 1 -- 172.21.15.86:0/2879542888 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x562320701050 con 0x5623208ef790 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 10 client.8055 handle_command_reply: tid=0 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 client.8055 shutdown 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/2879542888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 msgr2=0x56232095e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x56232095e7d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400ceb0 tx=0x7f0d64058040 comp rx=0 tx=0).stop 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/2879542888 shutdown_connections 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5623208ef790 0x562320901420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623208eee70 0x562320900ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5623208cfc10 0x56232095ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d56a00640 0 client.8055 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 --2- 172.21.15.86:0/2879542888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623208a2c20 0x56232095e7d0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/2879542888 >> 172.21.15.86:0/2879542888 conn(0x5623208a1ee0 msgr2=0x5623208ecf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/2879542888 shutdown_connections 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.697+0000 7f0d7f7d0000 1 -- 172.21.15.86:0/2879542888 wait complete. 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7f7d0000 20 client.8055 trim_cache size 0 max 16384 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 76850, 2024-03-22T04:18:26.705 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:18:26.706 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:18:26.706 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:18:26.706 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.521440524, 2024-03-22T04:18:26.706 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 77, 2024-03-22T04:18:26.706 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 97, 2024-03-22T04:18:26.706 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 97, 2024-03-22T04:18:26.706 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 5.4714982890000003 2024-03-22T04:18:26.706 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:18:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 -- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d58067b20 msgr2=0x7f0d58069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d58067b20 0x7f0d58069fe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0d680af000 tx=0x7f0d680a7000 comp rx=0 tx=0).stop 2024-03-22T04:18:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 -- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 msgr2=0x7f0d7414aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d7414aef0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f0d640011f0 tx=0x7f0d6400c9b0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 -- 172.21.15.86:0/2009821618 shutdown_connections 2024-03-22T04:18:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0d58067b20 0x7f0d58069fe0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d7415b3f0 0x7f0d740cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d740cc3e0 0x7f0d7414a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 --2- 172.21.15.86:0/2009821618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d740730d0 0x7f0d7414aef0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 -- 172.21.15.86:0/2009821618 >> 172.21.15.86:0/2009821618 conn(0x7f0d740b1df0 msgr2=0x7f0d74155610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 -- 172.21.15.86:0/2009821618 shutdown_connections 2024-03-22T04:18:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.701+0000 7f0d7c400640 1 -- 172.21.15.86:0/2009821618 wait complete. 2024-03-22T04:18:26.719 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:18:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.853+0000 7f6454400640 1 Processor -- start 2024-03-22T04:18:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.853+0000 7f6454400640 1 -- start start 2024-03-22T04:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6454400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f644c0aff50 0x7f644c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6454400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f644c06cbb0 0x7f644c143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6454400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 0x7f644c06e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6454400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f644c06d010 con 0x7f644c0aff50 2024-03-22T04:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6454400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f644c162aa0 con 0x7f644c06cbb0 2024-03-22T04:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6454400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f644c0ae5a0 con 0x7f644c144320 2024-03-22T04:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f644c0aff50 0x7f644c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f644c0aff50 0x7f644c06c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43242/0 (socket says 172.21.15.86:43242) 2024-03-22T04:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451a00640 1 -- 172.21.15.86:0/2913205205 learned_addr learned my addr 172.21.15.86:0/2913205205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6452400640 1 --2- 172.21.15.86:0/2913205205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 0x7f644c06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451000640 1 --2- 172.21.15.86:0/2913205205 >> v2:172.21.15.196:3300/0 conn(0x7f644c06cbb0 0x7f644c143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451a00640 1 -- 172.21.15.86:0/2913205205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 msgr2=0x7f644c06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451a00640 1 --2- 172.21.15.86:0/2913205205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 0x7f644c06e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451a00640 1 -- 172.21.15.86:0/2913205205 >> v2:172.21.15.196:3300/0 conn(0x7f644c06cbb0 msgr2=0x7f644c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451a00640 1 --2- 172.21.15.86:0/2913205205 >> v2:172.21.15.196:3300/0 conn(0x7f644c06cbb0 0x7f644c143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.857+0000 7f6451a00640 1 -- 172.21.15.86:0/2913205205 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c144b80 con 0x7f644c0aff50 2024-03-22T04:18:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6451a00640 1 --2- 172.21.15.86:0/2913205205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c06c670 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f644c148360 tx=0x7f644001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496205575f12a3aa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f644be00640 1 -- 172.21.15.86:0/2913205205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64400a6500 con 0x7f644c0aff50 2024-03-22T04:18:26.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f644be00640 1 -- 172.21.15.86:0/2913205205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64400a66a0 con 0x7f644c0aff50 2024-03-22T04:18:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f644be00640 1 -- 172.21.15.86:0/2913205205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64400a6a70 con 0x7f644c0aff50 2024-03-22T04:18:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 -- 172.21.15.86:0/2913205205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 msgr2=0x7f644c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 --2- 172.21.15.86:0/2913205205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c06c670 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f644c148360 tx=0x7f644001c670 comp rx=0 tx=0).stop 2024-03-22T04:18:26.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 -- 172.21.15.86:0/2913205205 shutdown_connections 2024-03-22T04:18:26.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 --2- 172.21.15.86:0/2913205205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 0x7f644c06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 --2- 172.21.15.86:0/2913205205 >> v2:172.21.15.196:3300/0 conn(0x7f644c06cbb0 0x7f644c143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 --2- 172.21.15.86:0/2913205205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c06c670 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 -- 172.21.15.86:0/2913205205 >> 172.21.15.86:0/2913205205 conn(0x7f644c071fe0 msgr2=0x7f644c0723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:26.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 -- 172.21.15.86:0/2913205205 shutdown_connections 2024-03-22T04:18:26.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 -- 172.21.15.86:0/2913205205 wait complete. 2024-03-22T04:18:26.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 Processor -- start 2024-03-22T04:18:26.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.861+0000 7f6454400640 1 -- start start 2024-03-22T04:18:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c06cbb0 0x7f644c141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c13f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 0x7f644c1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f644c162f30 con 0x7f644c0aff50 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f644c0b0330 con 0x7f644c06cbb0 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f644c140cf0 con 0x7f644c144320 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c13f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c13f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43248/0 (socket says 172.21.15.86:43248) 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 -- 172.21.15.86:0/1120667638 learned_addr learned my addr 172.21.15.86:0/1120667638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451a00640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c06cbb0 0x7f644c141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6452400640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 0x7f644c1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 -- 172.21.15.86:0/1120667638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 msgr2=0x7f644c1423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 0x7f644c1423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 -- 172.21.15.86:0/1120667638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c06cbb0 msgr2=0x7f644c141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c06cbb0 0x7f644c141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 -- 172.21.15.86:0/1120667638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c1400e0 con 0x7f644c0aff50 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6451000640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c13f770 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f643800e930 tx=0x7f643800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6449600640 1 -- 172.21.15.86:0/1120667638 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438017070 con 0x7f644c0aff50 2024-03-22T04:18:26.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6449600640 1 -- 172.21.15.86:0/1120667638 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643800ca00 con 0x7f644c0aff50 2024-03-22T04:18:26.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c1403c0 con 0x7f644c0aff50 2024-03-22T04:18:26.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6449600640 1 -- 172.21.15.86:0/1120667638 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643800cdb0 con 0x7f644c0aff50 2024-03-22T04:18:26.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c140980 con 0x7f644c0aff50 2024-03-22T04:18:26.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.865+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c068650 con 0x7f644c0aff50 2024-03-22T04:18:26.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.869+0000 7f6449600640 1 -- 172.21.15.86:0/1120667638 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f643801a070 con 0x7f644c0aff50 2024-03-22T04:18:26.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.869+0000 7f6449600640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f642c067b50 0x7f642c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:26.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.869+0000 7f6449600640 1 -- 172.21.15.86:0/1120667638 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f6438086580 con 0x7f644c0aff50 2024-03-22T04:18:26.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.869+0000 7f6451a00640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f642c067b50 0x7f642c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:26.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.869+0000 7f6451a00640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f642c067b50 0x7f642c06a010 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f64400ae000 tx=0x7f64400b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:26.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:26.873+0000 7f6449600640 1 -- 172.21.15.86:0/1120667638 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f64380566b0 con 0x7f644c0aff50 2024-03-22T04:18:27.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.065+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f644c140e70 con 0x7f644c0aff50 2024-03-22T04:18:27.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.065+0000 7f6449600640 1 -- 172.21.15.86:0/1120667638 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) ==== 53+0+73 (secure 0 0 0) 0x7f643805a750 con 0x7f644c0aff50 2024-03-22T04:18:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:18:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f642c067b50 msgr2=0x7f642c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f642c067b50 0x7f642c06a010 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f64400ae000 tx=0x7f64400b1040 comp rx=0 tx=0).stop 2024-03-22T04:18:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 msgr2=0x7f644c13f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c13f770 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f643800e930 tx=0x7f643800edf0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 shutdown_connections 2024-03-22T04:18:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f642c067b50 0x7f642c06a010 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c144320 0x7f644c1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f644c0aff50 0x7f644c13f770 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 --2- 172.21.15.86:0/1120667638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c06cbb0 0x7f644c141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 >> 172.21.15.86:0/1120667638 conn(0x7f644c071fe0 msgr2=0x7f644c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 shutdown_connections 2024-03-22T04:18:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.069+0000 7f6454400640 1 -- 172.21.15.86:0/1120667638 wait complete. 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:18:27.087 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:18:27.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:27.088 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:18:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:18:27.097 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:18:27.097 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:18:27.097 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:18:27.097 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42025sec preferred_lft 42025sec 2024-03-22T04:18:27.098 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:18:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:18:27.100 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:18:27.100 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:18:27.100 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:18:27.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:27.100 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:18:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:18:27.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:27.162 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:18:27.179 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:18:27.179+0000 7efec8200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:18:27.215 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:18:27.215 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:18:27.215 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:18:27.215 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:18:27.215 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:18:27.215 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:18:27.216 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:18:27.216 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:18:27.216 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:18:27.216 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:18:27.216 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:18:27.216 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:18:27.216 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:18:27.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:18:27.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:18:27.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:18:27.216 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:18:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:18:27.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:18:27.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:27.352 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:18:27.361 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:18:27.361 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:18:27.361 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:18:27.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:27.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:18:27.426 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:18:27.426 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:18:27.479 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:18:27.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:18:27.479 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:18:27.530 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:18:27.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:27.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:18:27.618 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:18:27.618 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:18:27.618 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:18:27.633 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:18:27.633 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:18:27.633 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:18:27.633 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:18:27.633 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:18:27.642 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:18:27.642 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:18:27.642 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:18:27.642 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:18:27.642 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:18:27.642 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:18:27.642 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:18:27.642 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:18:27.646 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:18:27.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:27.646 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:18:27.664 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:18:27.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:18:27.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:18:27.766 INFO:teuthology.orchestra.run.smithi086.stdout:client76925 2024-03-22T04:18:27.766 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:18:27.766 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:18:27.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:27.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-22T04:18:27.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.957+0000 7f9970200640 1 Processor -- start 2024-03-22T04:18:27.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.957+0000 7f9970200640 1 -- start start 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9970200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9968144a90 0x7f99680727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9970200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9968072cf0 0x7f99680b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9970200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 0x7f9968071620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9970200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9968144e70 con 0x7f9968072cf0 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9970200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f99680b3e00 con 0x7f9968144a90 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9970200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99680b40e0 con 0x7f99680b35f0 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9968072cf0 0x7f99680b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9968072cf0 0x7f99680b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43264/0 (socket says 172.21.15.86:43264) 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 -- 172.21.15.86:0/782360471 learned_addr learned my addr 172.21.15.86:0/782360471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:27.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996e200640 1 --2- 172.21.15.86:0/782360471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 0x7f9968071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996d800640 1 --2- 172.21.15.86:0/782360471 >> v2:172.21.15.196:3300/0 conn(0x7f9968144a90 0x7f99680727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 -- 172.21.15.86:0/782360471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 msgr2=0x7f9968071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:27.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 --2- 172.21.15.86:0/782360471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 0x7f9968071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 -- 172.21.15.86:0/782360471 >> v2:172.21.15.196:3300/0 conn(0x7f9968144a90 msgr2=0x7f99680727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:27.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 --2- 172.21.15.86:0/782360471 >> v2:172.21.15.196:3300/0 conn(0x7f9968144a90 0x7f99680727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 -- 172.21.15.86:0/782360471 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99680cdbd0 con 0x7f9968072cf0 2024-03-22T04:18:27.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f996ce00640 1 --2- 172.21.15.86:0/782360471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f99680b30b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f996806f910 tx=0x7f995401e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49aa0b594f2bf88 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:27.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9967e00640 1 -- 172.21.15.86:0/782360471 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540b4070 con 0x7f9968072cf0 2024-03-22T04:18:27.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9967e00640 1 -- 172.21.15.86:0/782360471 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99540a78d0 con 0x7f9968072cf0 2024-03-22T04:18:27.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.961+0000 7f9967e00640 1 -- 172.21.15.86:0/782360471 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540a7ca0 con 0x7f9968072cf0 2024-03-22T04:18:27.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- 172.21.15.86:0/782360471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 msgr2=0x7f99680b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:27.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 --2- 172.21.15.86:0/782360471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f99680b30b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f996806f910 tx=0x7f995401e930 comp rx=0 tx=0).stop 2024-03-22T04:18:27.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- 172.21.15.86:0/782360471 shutdown_connections 2024-03-22T04:18:27.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 --2- 172.21.15.86:0/782360471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 0x7f9968071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 --2- 172.21.15.86:0/782360471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f99680b30b0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 --2- 172.21.15.86:0/782360471 >> v2:172.21.15.196:3300/0 conn(0x7f9968144a90 0x7f99680727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- 172.21.15.86:0/782360471 >> 172.21.15.86:0/782360471 conn(0x7f996806b660 msgr2=0x7f996806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:27.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- 172.21.15.86:0/782360471 shutdown_connections 2024-03-22T04:18:27.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- 172.21.15.86:0/782360471 wait complete. 2024-03-22T04:18:27.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 Processor -- start 2024-03-22T04:18:27.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- start start 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f996813a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 0x7f996813b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9968144a90 0x7f996813ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f996806d6b0 con 0x7f9968072cf0 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f996806d530 con 0x7f9968144a90 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f9970200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f996806d830 con 0x7f99680b35f0 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f996d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f996813a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f996d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f996813a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43276/0 (socket says 172.21.15.86:43276) 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f996d800640 1 -- 172.21.15.86:0/3599564977 learned_addr learned my addr 172.21.15.86:0/3599564977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f996ce00640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 0x7f996813b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.965+0000 7f996e200640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9968144a90 0x7f996813ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f996d800640 1 -- 172.21.15.86:0/3599564977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 msgr2=0x7f996813b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:27.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f996d800640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 0x7f996813b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f996d800640 1 -- 172.21.15.86:0/3599564977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9968144a90 msgr2=0x7f996813ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f996d800640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9968144a90 0x7f996813ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f996d800640 1 -- 172.21.15.86:0/3599564977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996813b9d0 con 0x7f9968072cf0 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f996d800640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f996813a8f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f995c00e870 tx=0x7f995c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f9965600640 1 -- 172.21.15.86:0/3599564977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c017070 con 0x7f9968072cf0 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f9965600640 1 -- 172.21.15.86:0/3599564977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c00c8e0 con 0x7f9968072cf0 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f9965600640 1 -- 172.21.15.86:0/3599564977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00cc90 con 0x7f9968072cf0 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f9970200640 1 -- 172.21.15.86:0/3599564977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996813bcb0 con 0x7f9968072cf0 2024-03-22T04:18:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f9970200640 1 -- 172.21.15.86:0/3599564977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996813c270 con 0x7f9968072cf0 2024-03-22T04:18:27.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f9965600640 1 -- 172.21.15.86:0/3599564977 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f995c01a070 con 0x7f9968072cf0 2024-03-22T04:18:27.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f9965600640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9948067b20 0x7f9948069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f9965600640 1 -- 172.21.15.86:0/3599564977 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f995c0863c0 con 0x7f9968072cf0 2024-03-22T04:18:27.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.969+0000 7f996ce00640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9948067b20 0x7f9948069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 Processor -- start 2024-03-22T04:18:27.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- start start 2024-03-22T04:18:27.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9948067b20 0x7f9948069fe0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f99540b1000 tx=0x7f9954008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 0x55677288c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x556772842970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556772842eb0 0x556772843290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556772729830 con 0x55677288c730 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556772690e10 con 0x55677288be10 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55677262de50 con 0x556772842eb0 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x556772842970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x556772842970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43280/0 (socket says 172.21.15.86:43280) 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 -- 172.21.15.86:0/755007578 learned_addr learned my addr 172.21.15.86:0/755007578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996d800640 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 0x55677288c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996e200640 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556772842eb0 0x556772843290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 -- 172.21.15.86:0/755007578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556772842eb0 msgr2=0x556772843290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556772842eb0 0x556772843290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 -- 172.21.15.86:0/755007578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 msgr2=0x55677288c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 0x55677288c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 -- 172.21.15.86:0/755007578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556772616510 con 0x55677288c730 2024-03-22T04:18:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f996ce00640 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x556772842970 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f995400b2c0 tx=0x7f9954004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1482739aaf18cc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f995a000640 1 -- 172.21.15.86:0/755007578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540ca070 con 0x55677288c730 2024-03-22T04:18:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f995a000640 1 -- 172.21.15.86:0/755007578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99540ba450 con 0x55677288c730 2024-03-22T04:18:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f995a000640 1 -- 172.21.15.86:0/755007578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540b4040 con 0x55677288c730 2024-03-22T04:18:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- 172.21.15.86:0/755007578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 msgr2=0x556772842970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x556772842970 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f995400b2c0 tx=0x7f9954004740 comp rx=0 tx=0).stop 2024-03-22T04:18:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- 172.21.15.86:0/755007578 shutdown_connections 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556772842eb0 0x556772843290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x556772842970 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 --2- 172.21.15.86:0/755007578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 0x55677288c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- 172.21.15.86:0/755007578 >> 172.21.15.86:0/755007578 conn(0x55677283ee60 msgr2=0x556772841250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- 172.21.15.86:0/755007578 shutdown_connections 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- 172.21.15.86:0/755007578 wait complete. 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 -- start start 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.973+0000 7f99736e1000 1 Processor -- start 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x5567728483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 0x556772848930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288c730 0x55677284d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55677264e640 con 0x556772842eb0 2024-03-22T04:18:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55677268cdc0 con 0x55677288be10 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55677283ece0 con 0x55677288c730 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 10 client.0 _ll_register_callbacks cb 0x55677282dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x5567728483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x5567728483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43284/0 (socket says 172.21.15.86:43284) 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 -- 172.21.15.86:0/3604182931 learned_addr learned my addr 172.21.15.86:0/3604182931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996e200640 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288c730 0x55677284d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996ce00640 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 0x556772848930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 -- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288c730 msgr2=0x55677284d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288c730 0x55677284d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 -- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 msgr2=0x556772848930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 0x556772848930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 -- 172.21.15.86:0/3604182931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55677264ea40 con 0x556772842eb0 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996d800640 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x5567728483f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c520 tx=0x7f995c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e54eb46d9296e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 1 -- 172.21.15.86:0/3604182931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c06e020 con 0x556772842eb0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 1 -- 172.21.15.86:0/3604182931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f995c05a170 con 0x556772842eb0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 -- 172.21.15.86:0/3604182931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99540ba450 con 0x556772842eb0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 1 -- 172.21.15.86:0/3604182931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f995c021040 con 0x556772842eb0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 10 client.8073 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 -- 172.21.15.86:0/3604182931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x556772649370 con 0x556772842eb0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 1 -- 172.21.15.86:0/3604182931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99540ba450 con 0x556772842eb0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 1 -- 172.21.15.86:0/3604182931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f995c0840d0 con 0x556772842eb0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 10 client.8073 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 10 client.8073 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 20 client.8073 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55677289b0b0 0x55677289d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 4 client.8073 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 -- 172.21.15.86:0/3604182931 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55677262bed0 con 0x55677289b0b0 2024-03-22T04:18:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996ce00640 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55677289b0b0 0x55677289d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 10 client.8073 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f996ce00640 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55677289b0b0 0x55677289d4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e0f0ccc440f6ed server_cookie=5a855cc66907c854 in_seq=0 out_seq=0 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 1 -- 172.21.15.86:0/3604182931 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55677262bed0 con 0x55677289b0b0 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f994ea00640 10 client.8073 handle_command_reply: tid=0 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.977+0000 7f99736e1000 1 client.8073 shutdown 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 -- 172.21.15.86:0/3604182931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 msgr2=0x5567728483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x5567728483f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c520 tx=0x7f995c00cf70 comp rx=0 tx=0).stop 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 -- 172.21.15.86:0/3604182931 shutdown_connections 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55677289b0b0 0x55677289d4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288be10 0x556772848930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288c730 0x55677284d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 --2- 172.21.15.86:0/3604182931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x5567728483f0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f994ea00640 0 client.8073 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 -- 172.21.15.86:0/3604182931 >> 172.21.15.86:0/3604182931 conn(0x55677283ee60 msgr2=0x556772841ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 -- 172.21.15.86:0/3604182931 shutdown_connections 2024-03-22T04:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 1 -- 172.21.15.86:0/3604182931 wait complete. 2024-03-22T04:18:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.981+0000 7f99736e1000 20 client.8073 trim_cache size 0 max 16384 2024-03-22T04:18:27.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f99736e1000 1 Processor -- start 2024-03-22T04:18:27.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f99736e1000 1 -- start start 2024-03-22T04:18:27.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f99736e1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556772842eb0 0x55677288ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f99736e1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 0x556772842260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f99736e1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x5567728427a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f99736e1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9968204560 con 0x55677288c730 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f99736e1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9968203540 con 0x556772842eb0 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f99736e1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9968202840 con 0x55677288be10 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x5567728427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x5567728427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43296/0 (socket says 172.21.15.86:43296) 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 -- 172.21.15.86:0/1633870737 learned_addr learned my addr 172.21.15.86:0/1633870737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996d800640 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556772842eb0 0x55677288ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996ce00640 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 0x556772842260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 -- 172.21.15.86:0/1633870737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 msgr2=0x556772842260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 0x556772842260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 -- 172.21.15.86:0/1633870737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556772842eb0 msgr2=0x55677288ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556772842eb0 0x55677288ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 -- 172.21.15.86:0/1633870737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677262e0a0 con 0x55677288c730 2024-03-22T04:18:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f996e200640 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x5567728427a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f9960003d00 tx=0x7f996000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2052d1f6b9627935 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f9959600640 1 -- 172.21.15.86:0/1633870737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996001c070 con 0x55677288c730 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f9959600640 1 -- 172.21.15.86:0/1633870737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960018860 con 0x55677288c730 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:27.997+0000 7f9959600640 1 -- 172.21.15.86:0/1633870737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960018c50 con 0x55677288c730 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- 172.21.15.86:0/1633870737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 msgr2=0x5567728427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x5567728427a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f9960003d00 tx=0x7f996000b040 comp rx=0 tx=0).stop 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- 172.21.15.86:0/1633870737 shutdown_connections 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55677288c730 0x5567728427a0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 0x556772842260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 --2- 172.21.15.86:0/1633870737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556772842eb0 0x55677288ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- 172.21.15.86:0/1633870737 >> 172.21.15.86:0/1633870737 conn(0x55677283ee60 msgr2=0x55677288a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- 172.21.15.86:0/1633870737 shutdown_connections 2024-03-22T04:18:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- 172.21.15.86:0/1633870737 wait complete. 2024-03-22T04:18:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- start start 2024-03-22T04:18:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 Processor -- start 2024-03-22T04:18:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x55677289fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 0x5567728fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288c730 0x5567728fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55677274cf90 con 0x556772842eb0 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9968202e90 con 0x55677288c730 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55677268cdc0 con 0x55677288be10 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 10 client.0 _ll_register_callbacks cb 0x7f99540b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x55677289fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x55677289fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43310/0 (socket says 172.21.15.86:43310) 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 -- 172.21.15.86:0/3969765985 learned_addr learned my addr 172.21.15.86:0/3969765985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996ce00640 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 0x5567728fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996e200640 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288c730 0x5567728fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 -- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 msgr2=0x5567728fb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 0x5567728fb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 -- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288c730 msgr2=0x5567728fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288c730 0x5567728fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 -- 172.21.15.86:0/3969765985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556772712840 con 0x556772842eb0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f996d800640 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x55677289fbb0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f995c004780 tx=0x7f995c084470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b59b82b912f45d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f994ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f994ea00640 1 -- 172.21.15.86:0/3969765985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c06e040 con 0x556772842eb0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f994ea00640 1 -- 172.21.15.86:0/3969765985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f995c06e3f0 con 0x556772842eb0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- 172.21.15.86:0/3969765985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9960018860 con 0x556772842eb0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f994ea00640 1 -- 172.21.15.86:0/3969765985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f995c055d90 con 0x556772842eb0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 10 client.8079 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f99736e1000 1 -- 172.21.15.86:0/3969765985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55677234c470 con 0x556772842eb0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f994ea00640 1 -- 172.21.15.86:0/3969765985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960018860 con 0x556772842eb0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.001+0000 7f994ea00640 1 -- 172.21.15.86:0/3969765985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f995c055880 con 0x556772842eb0 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 10 client.8079 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 10 client.8079 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 20 client.8079 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5567728fc220 0x5567728f8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 4 client.8079 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 -- 172.21.15.86:0/3969765985 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55677266a6b0 con 0x5567728fc220 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f996ce00640 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5567728fc220 0x5567728f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f996ce00640 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5567728fc220 0x5567728f8200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ff2fc43d376d98 server_cookie=5d1bc66fa094127b in_seq=0 out_seq=0 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f994ea00640 10 client.8079 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f994ea00640 1 -- 172.21.15.86:0/3969765985 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55677266a6b0 con 0x5567728fc220 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f994ea00640 10 client.8079 handle_command_reply: tid=0 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 client.8079 shutdown 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 -- 172.21.15.86:0/3969765985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 msgr2=0x55677289fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x55677289fbb0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f995c004780 tx=0x7f995c084470 comp rx=0 tx=0).stop 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 -- 172.21.15.86:0/3969765985 shutdown_connections 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5567728fc220 0x5567728f8200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55677288c730 0x5567728fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55677288be10 0x5567728fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f994ea00640 0 client.8079 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:28.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 --2- 172.21.15.86:0/3969765985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556772842eb0 0x55677289fbb0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 -- 172.21.15.86:0/3969765985 >> 172.21.15.86:0/3969765985 conn(0x55677283ee60 msgr2=0x55677288a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 -- 172.21.15.86:0/3969765985 shutdown_connections 2024-03-22T04:18:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 1 -- 172.21.15.86:0/3969765985 wait complete. 2024-03-22T04:18:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f99736e1000 20 client.8079 trim_cache size 0 max 16384 2024-03-22T04:18:28.009 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:18:28.009 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:18:28.009 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "cc34364c-b460-4a01-811b-c216125c10b8", 2024-03-22T04:18:28.009 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:18:28.010 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:18:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 -- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9948067b20 msgr2=0x7f9948069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9948067b20 0x7f9948069fe0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f99540b1000 tx=0x7f9954008040 comp rx=0 tx=0).stop 2024-03-22T04:18:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 -- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 msgr2=0x7f996813a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f996813a8f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f995c00e870 tx=0x7f995c00ec70 comp rx=0 tx=0).stop 2024-03-22T04:18:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 -- 172.21.15.86:0/3599564977 shutdown_connections 2024-03-22T04:18:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9948067b20 0x7f9948069fe0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9968144a90 0x7f996813ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99680b35f0 0x7f996813b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 --2- 172.21.15.86:0/3599564977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9968072cf0 0x7f996813a8f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 -- 172.21.15.86:0/3599564977 >> 172.21.15.86:0/3599564977 conn(0x7f996806b660 msgr2=0x7f99680af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 -- 172.21.15.86:0/3599564977 shutdown_connections 2024-03-22T04:18:28.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.005+0000 7f9970200640 1 -- 172.21.15.86:0/3599564977 wait complete. 2024-03-22T04:18:28.011 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:18:28.003+0000 7efec3200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5652e579e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-22T04:18:28.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:28.025 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-22T04:18:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.161+0000 7f3d88200640 1 Processor -- start 2024-03-22T04:18:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.161+0000 7f3d88200640 1 -- start start 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d88200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d800b3600 0x7f3d8015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d88200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3d8015cee0 0x7f3d800b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d88200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8015d710 con 0x7f3d800b2360 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d88200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3d800b2740 con 0x7f3d8015cee0 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d88200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8015fe40 con 0x7f3d800b3600 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d86200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3d8015cee0 0x7f3d800b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d84e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d800b3600 0x7f3d8015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d86200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3d8015cee0 0x7f3d800b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43316/0 (socket says 172.21.15.86:43316) 2024-03-22T04:18:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d86200640 1 -- 172.21.15.86:0/1976505851 learned_addr learned my addr 172.21.15.86:0/1976505851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:28.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d86200640 1 -- 172.21.15.86:0/1976505851 >> v2:172.21.15.196:3300/0 conn(0x7f3d800b3600 msgr2=0x7f3d8015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d86200640 1 --2- 172.21.15.86:0/1976505851 >> v2:172.21.15.196:3300/0 conn(0x7f3d800b3600 0x7f3d8015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d86200640 1 -- 172.21.15.86:0/1976505851 >> v1:172.21.15.196:6789/0 conn(0x7f3d800b2360 legacy=0x7f3d800b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d86200640 1 -- 172.21.15.86:0/1976505851 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d80160120 con 0x7f3d8015cee0 2024-03-22T04:18:28.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.165+0000 7f3d84e00640 1 --2- 172.21.15.86:0/1976505851 >> v2:172.21.15.196:3300/0 conn(0x7f3d800b3600 0x7f3d8015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:28.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d86200640 1 --2- 172.21.15.86:0/1976505851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d8015cee0 0x7f3d800b4d80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3d800b2c00 tx=0x7f3d7801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7067572adf7beef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d7fe00640 1 -- 172.21.15.86:0/1976505851 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780a5880 con 0x7f3d8015cee0 2024-03-22T04:18:28.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d7fe00640 1 -- 172.21.15.86:0/1976505851 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d780a5a20 con 0x7f3d8015cee0 2024-03-22T04:18:28.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d7fe00640 1 -- 172.21.15.86:0/1976505851 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780a5df0 con 0x7f3d8015cee0 2024-03-22T04:18:28.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- 172.21.15.86:0/1976505851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d8015cee0 msgr2=0x7f3d800b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 --2- 172.21.15.86:0/1976505851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d8015cee0 0x7f3d800b4d80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3d800b2c00 tx=0x7f3d7801c9c0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- 172.21.15.86:0/1976505851 shutdown_connections 2024-03-22T04:18:28.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 --2- 172.21.15.86:0/1976505851 >> v2:172.21.15.196:3300/0 conn(0x7f3d800b3600 0x7f3d8015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 --2- 172.21.15.86:0/1976505851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d8015cee0 0x7f3d800b4d80 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- 172.21.15.86:0/1976505851 >> 172.21.15.86:0/1976505851 conn(0x7f3d800cc940 msgr2=0x7f3d800ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- 172.21.15.86:0/1976505851 shutdown_connections 2024-03-22T04:18:28.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- 172.21.15.86:0/1976505851 wait complete. 2024-03-22T04:18:28.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 Processor -- start 2024-03-22T04:18:28.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- start start 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d800b2360 0x7f3d801fa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d800b3600 0x7f3d8014c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d8015cee0 0x7f3d80149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8015d4d0 con 0x7f3d800b2360 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8015d350 con 0x7f3d8015cee0 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d88200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3d800b5ad0 con 0x7f3d800b3600 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.169+0000 7f3d85800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d800b2360 0x7f3d801fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d85800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d800b2360 0x7f3d801fa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43318/0 (socket says 172.21.15.86:43318) 2024-03-22T04:18:28.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d85800640 1 -- 172.21.15.86:0/115029525 learned_addr learned my addr 172.21.15.86:0/115029525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d84e00640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d800b3600 0x7f3d8014c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d86200640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d8015cee0 0x7f3d80149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d85800640 1 -- 172.21.15.86:0/115029525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d800b3600 msgr2=0x7f3d8014c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d85800640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d800b3600 0x7f3d8014c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d85800640 1 -- 172.21.15.86:0/115029525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d8015cee0 msgr2=0x7f3d80149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d85800640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d8015cee0 0x7f3d80149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d85800640 1 -- 172.21.15.86:0/115029525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d80149480 con 0x7f3d800b2360 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d85800640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d800b2360 0x7f3d801fa660 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c450 tx=0x7f3d7400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d7d600640 1 -- 172.21.15.86:0/115029525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74016020 con 0x7f3d800b2360 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d7d600640 1 -- 172.21.15.86:0/115029525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d740052d0 con 0x7f3d800b2360 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d88200640 1 -- 172.21.15.86:0/115029525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d801faba0 con 0x7f3d800b2360 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d7d600640 1 -- 172.21.15.86:0/115029525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74007260 con 0x7f3d800b2360 2024-03-22T04:18:28.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d88200640 1 -- 172.21.15.86:0/115029525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8014d000 con 0x7f3d800b2360 2024-03-22T04:18:28.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.173+0000 7f3d7d600640 1 -- 172.21.15.86:0/115029525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3d74007400 con 0x7f3d800b2360 2024-03-22T04:18:28.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d7d600640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d60067b50 0x7f3d6006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d7d600640 1 -- 172.21.15.86:0/115029525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f3d74085ad0 con 0x7f3d800b2360 2024-03-22T04:18:28.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d84e00640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d60067b50 0x7f3d6006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 Processor -- start 2024-03-22T04:18:28.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- start start 2024-03-22T04:18:28.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d84e00640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d60067b50 0x7f3d6006a010 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009580 tx=0x7f3d6c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98acaa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98acaa8a0 0x55f98ac5da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f98aaaec40 con 0x55f98aca9f80 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f98aa4bd90 con 0x55f98acaa8a0 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f98aaaabf0 con 0x55f98ac5df50 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98acaa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98acaa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43330/0 (socket says 172.21.15.86:43330) 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 -- 172.21.15.86:0/1639045407 learned_addr learned my addr 172.21.15.86:0/1639045407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d86200640 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac5e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d84e00640 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98acaa8a0 0x55f98ac5da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 -- 172.21.15.86:0/1639045407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 msgr2=0x55f98ac5e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac5e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 -- 172.21.15.86:0/1639045407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98acaa8a0 msgr2=0x55f98ac5da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98acaa8a0 0x55f98ac5da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 -- 172.21.15.86:0/1639045407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f98ab56130 con 0x55f98aca9f80 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d85800640 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98acaa360 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3d7405dee0 tx=0x7f3d74011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea54452ddbc64bcc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d72000640 1 -- 172.21.15.86:0/1639045407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7405e410 con 0x55f98aca9f80 2024-03-22T04:18:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d72000640 1 -- 172.21.15.86:0/1639045407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7405c070 con 0x55f98aca9f80 2024-03-22T04:18:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d72000640 1 -- 172.21.15.86:0/1639045407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d740521b0 con 0x55f98aca9f80 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- 172.21.15.86:0/1639045407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 msgr2=0x55f98acaa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98acaa360 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3d7405dee0 tx=0x7f3d74011000 comp rx=0 tx=0).stop 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- 172.21.15.86:0/1639045407 shutdown_connections 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98acaa8a0 0x55f98ac5da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac5e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 --2- 172.21.15.86:0/1639045407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98acaa360 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- 172.21.15.86:0/1639045407 >> 172.21.15.86:0/1639045407 conn(0x55f98ab47990 msgr2=0x55f98ac8d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- 172.21.15.86:0/1639045407 shutdown_connections 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- 172.21.15.86:0/1639045407 wait complete. 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 -- start start 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.177+0000 7f3d8b610000 1 Processor -- start 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98ac659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98ac8e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98acaa8a0 0x55f98ac8e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f98aa6a710 con 0x55f98ac5df50 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f98aa6c580 con 0x55f98aca9f80 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f98aab3270 con 0x55f98acaa8a0 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 10 client.0 _ll_register_callbacks cb 0x55f98ab10200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98ac659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98ac659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43334/0 (socket says 172.21.15.86:43334) 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 -- 172.21.15.86:0/3348543589 learned_addr learned my addr 172.21.15.86:0/3348543589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d84e00640 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98ac8e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d86200640 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98acaa8a0 0x55f98ac8e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 -- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98acaa8a0 msgr2=0x55f98ac8e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98acaa8a0 0x55f98ac8e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 -- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 msgr2=0x55f98ac8e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98ac8e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 -- 172.21.15.86:0/3348543589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f98aa87d30 con 0x55f98ac5df50 2024-03-22T04:18:28.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d85800640 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98ac659f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3d74055a80 tx=0x7f3d74083b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8947e6d3f9e98da4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d66a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d66a00640 1 -- 172.21.15.86:0/3348543589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74071070 con 0x55f98ac5df50 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d66a00640 1 -- 172.21.15.86:0/3348543589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f3d740521b0 con 0x55f98ac5df50 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 -- 172.21.15.86:0/3348543589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7405c070 con 0x55f98ac5df50 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d66a00640 1 -- 172.21.15.86:0/3348543589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f3d7407c050 con 0x55f98ac5df50 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 10 client.8091 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 -- 172.21.15.86:0/3348543589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55f98aaf3fc0 con 0x55f98ac5df50 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d66a00640 1 -- 172.21.15.86:0/3348543589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7405c070 con 0x55f98ac5df50 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d66a00640 1 -- 172.21.15.86:0/3348543589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f3d7407d240 con 0x55f98ac5df50 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 10 client.8091 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 10 client.8091 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 20 client.8091 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55f98acb05c0 0x55f98acb09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 4 client.8091 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:18:28.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d8b610000 1 -- 172.21.15.86:0/3348543589 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f98aa49dd0 con 0x55f98acb05c0 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d84e00640 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55f98acb05c0 0x55f98acb09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d66a00640 10 client.8091 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.181+0000 7f3d84e00640 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55f98acb05c0 0x55f98acb09a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a10b6c9d661c57 server_cookie=37b83ee7637361df in_seq=0 out_seq=0 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d66a00640 1 -- 172.21.15.86:0/3348543589 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f98aa49dd0 con 0x55f98acb05c0 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d66a00640 10 client.8091 handle_command_reply: tid=0 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 client.8091 shutdown 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 -- 172.21.15.86:0/3348543589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 msgr2=0x55f98ac659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98ac659f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3d74055a80 tx=0x7f3d74083b80 comp rx=0 tx=0).stop 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 -- 172.21.15.86:0/3348543589 shutdown_connections 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55f98acb05c0 0x55f98acb09a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98ac8e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98acaa8a0 0x55f98ac8e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3348543589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98ac659f0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d66a00640 0 client.8091 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 -- 172.21.15.86:0/3348543589 >> 172.21.15.86:0/3348543589 conn(0x55f98ab47990 msgr2=0x55f98ac5d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 -- 172.21.15.86:0/3348543589 shutdown_connections 2024-03-22T04:18:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 1 -- 172.21.15.86:0/3348543589 wait complete. 2024-03-22T04:18:28.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.185+0000 7f3d8b610000 20 client.8091 trim_cache size 0 max 16384 2024-03-22T04:18:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d8b610000 1 Processor -- start 2024-03-22T04:18:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d8b610000 1 -- start start 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98ab47990 0x55f98acc0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98ac64e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d8b610000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80204330 con 0x55f98aca9f80 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d8b610000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80203340 con 0x55f98ab47990 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d8b610000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8015ec70 con 0x55f98ac5df50 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d86200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98ac64e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d86200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98ac64e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43340/0 (socket says 172.21.15.86:43340) 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d86200640 1 -- 172.21.15.86:0/3017902972 learned_addr learned my addr 172.21.15.86:0/3017902972 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d84e00640 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d85800640 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98ab47990 0x55f98acc0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d86200640 1 -- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 msgr2=0x55f98ac648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d86200640 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d86200640 1 -- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98ab47990 msgr2=0x55f98acc0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d86200640 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98ab47990 0x55f98acc0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d86200640 1 -- 172.21.15.86:0/3017902972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f98aa843a0 con 0x55f98aca9f80 2024-03-22T04:18:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.201+0000 7f3d84e00640 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d86200640 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98ac64e10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3d7801cea0 tx=0x7f3d780a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6884fce743dd033a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d71600640 1 -- 172.21.15.86:0/3017902972 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780a5880 con 0x55f98aca9f80 2024-03-22T04:18:28.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d71600640 1 -- 172.21.15.86:0/3017902972 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d780a5a20 con 0x55f98aca9f80 2024-03-22T04:18:28.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d71600640 1 -- 172.21.15.86:0/3017902972 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780a5df0 con 0x55f98aca9f80 2024-03-22T04:18:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 -- 172.21.15.86:0/3017902972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 msgr2=0x55f98ac64e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98ac64e10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3d7801cea0 tx=0x7f3d780a7070 comp rx=0 tx=0).stop 2024-03-22T04:18:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 -- 172.21.15.86:0/3017902972 shutdown_connections 2024-03-22T04:18:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98aca9f80 0x55f98ac64e10 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ac5df50 0x55f98ac648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 --2- 172.21.15.86:0/3017902972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98ab47990 0x55f98acc0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 -- 172.21.15.86:0/3017902972 >> 172.21.15.86:0/3017902972 conn(0x55f98ac8d400 msgr2=0x55f98ac641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 -- 172.21.15.86:0/3017902972 shutdown_connections 2024-03-22T04:18:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 -- 172.21.15.86:0/3017902972 wait complete. 2024-03-22T04:18:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.205+0000 7f3d8b610000 1 -- start start 2024-03-22T04:18:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 Processor -- start 2024-03-22T04:18:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ab47990 0x55f98acb6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98acb6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98acb3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8015f2c0 con 0x55f98ac5df50 2024-03-22T04:18:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f98aab3270 con 0x55f98aca9f80 2024-03-22T04:18:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f98ac57e50 con 0x55f98ab47990 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 10 client.0 _ll_register_callbacks cb 0x7f3d74085040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98acb6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98acb6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43342/0 (socket says 172.21.15.86:43342) 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 -- 172.21.15.86:0/2147908201 learned_addr learned my addr 172.21.15.86:0/2147908201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d86200640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98acb3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d85800640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ab47990 0x55f98acb6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 -- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ab47990 msgr2=0x55f98acb6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ab47990 0x55f98acb6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 -- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 msgr2=0x55f98acb3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98acb3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 -- 172.21.15.86:0/2147908201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f98aa34380 con 0x55f98ac5df50 2024-03-22T04:18:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d86200640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98acb3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d85800640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ab47990 0x55f98acb6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d84e00640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98acb6c70 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f3d801484b0 tx=0x7f3d6c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc56237e8546db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 1 -- 172.21.15.86:0/2147908201 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c030070 con 0x55f98ac5df50 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 1 -- 172.21.15.86:0/2147908201 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f3d6c02cb10 con 0x55f98ac5df50 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 -- 172.21.15.86:0/2147908201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d780a5a20 con 0x55f98ac5df50 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 1 -- 172.21.15.86:0/2147908201 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f3d6c03b050 con 0x55f98ac5df50 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 10 client.8097 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 -- 172.21.15.86:0/2147908201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55f98ab30460 con 0x55f98ac5df50 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 1 -- 172.21.15.86:0/2147908201 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d780a5a20 con 0x55f98ac5df50 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 1 -- 172.21.15.86:0/2147908201 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f3d6c03e020 con 0x55f98ac5df50 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 10 client.8097 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:28.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 10 client.8097 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:18:28.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 20 client.8097 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:28.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55f98acaa8a0 0x55f98acb3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:28.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 4 client.8097 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:18:28.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 -- 172.21.15.86:0/2147908201 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f98aa4e160 con 0x55f98acaa8a0 2024-03-22T04:18:28.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d85800640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55f98acaa8a0 0x55f98acb3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:28.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d85800640 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55f98acaa8a0 0x55f98acb3f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28d2e1fa3698ed70 server_cookie=57bf2ce8313f5550 in_seq=0 out_seq=0 2024-03-22T04:18:28.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 10 client.8097 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:28.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 1 -- 172.21.15.86:0/2147908201 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55f98aa4e160 con 0x55f98acaa8a0 2024-03-22T04:18:28.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d66a00640 10 client.8097 handle_command_reply: tid=0 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.209+0000 7f3d8b610000 1 client.8097 shutdown 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 -- 172.21.15.86:0/2147908201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 msgr2=0x55f98acb6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98acb6c70 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f3d801484b0 tx=0x7f3d6c004740 comp rx=0 tx=0).stop 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 -- 172.21.15.86:0/2147908201 shutdown_connections 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x55f98acaa8a0 0x55f98acb3f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f98aca9f80 0x55f98acb3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f98ac5df50 0x55f98acb6c70 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d66a00640 0 client.8097 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 --2- 172.21.15.86:0/2147908201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f98ab47990 0x55f98acb6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 -- 172.21.15.86:0/2147908201 >> 172.21.15.86:0/2147908201 conn(0x55f98ac8d400 msgr2=0x55f98ac64a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 -- 172.21.15.86:0/2147908201 shutdown_connections 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 1 -- 172.21.15.86:0/2147908201 wait complete. 2024-03-22T04:18:28.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d8b610000 20 client.8097 trim_cache size 0 max 16384 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout: "cc34364c-b460-4a01-811b-c216125c10b8": { 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir", 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "cc34364c-b460-4a01-811b-c216125c10b8", 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:18:28.216 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:18:28.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 -- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d60067b50 msgr2=0x7f3d6006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d60067b50 0x7f3d6006a010 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009580 tx=0x7f3d6c009190 comp rx=0 tx=0).stop 2024-03-22T04:18:28.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 -- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d800b2360 msgr2=0x7f3d801fa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:28.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d800b2360 0x7f3d801fa660 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c450 tx=0x7f3d7400c910 comp rx=0 tx=0).stop 2024-03-22T04:18:28.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 -- 172.21.15.86:0/115029525 shutdown_connections 2024-03-22T04:18:28.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d60067b50 0x7f3d6006a010 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d8015cee0 0x7f3d80149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d800b3600 0x7f3d8014c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 --2- 172.21.15.86:0/115029525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d800b2360 0x7f3d801fa660 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:28.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 -- 172.21.15.86:0/115029525 >> 172.21.15.86:0/115029525 conn(0x7f3d800cc940 msgr2=0x7f3d800cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:28.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 -- 172.21.15.86:0/115029525 shutdown_connections 2024-03-22T04:18:28.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:28.213+0000 7f3d88200640 1 -- 172.21.15.86:0/115029525 wait complete. 2024-03-22T04:18:28.231 INFO:tasks.cephfs.filesystem:scrub status for tag:cc34364c-b460-4a01-811b-c216125c10b8 - {'path': '/dir', 'tag': 'cc34364c-b460-4a01-811b-c216125c10b8', 'options': 'recursive,repair'} 2024-03-22T04:18:28.652 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:18:28.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:18:58.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:18:58.232 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-22T04:18:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab96c00640 1 Processor -- start 2024-03-22T04:18:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab96c00640 1 -- start start 2024-03-22T04:18:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab96c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 0x7fab900b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab96c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab9015d0b0 0x7fab900b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab96c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab9015d710 con 0x7fab900b3430 2024-03-22T04:18:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab96c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fab900b3890 con 0x7fab9015d0b0 2024-03-22T04:18:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab96c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab900b2740 con 0x7fab900b2360 2024-03-22T04:18:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab94c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab9015d0b0 0x7fab900b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab8f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fab900b3430 0x7fab9015c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59578/0 (socket says 172.21.15.86:59578) 2024-03-22T04:18:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab8f400640 1 -- 172.21.15.86:0/1536762212 learned_addr learned my addr 172.21.15.86:0/1536762212 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.381+0000 7fab8fe00640 1 --2- 172.21.15.86:0/1536762212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 0x7fab900b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab94c00640 1 -- 172.21.15.86:0/1536762212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 msgr2=0x7fab900b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab94c00640 1 --2- 172.21.15.86:0/1536762212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 0x7fab900b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab94c00640 1 -- 172.21.15.86:0/1536762212 >> v1:172.21.15.86:6789/0 conn(0x7fab900b3430 legacy=0x7fab9015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab94c00640 1 -- 172.21.15.86:0/1536762212 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9015fe40 con 0x7fab9015d0b0 2024-03-22T04:18:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab8ea00640 1 -- 172.21.15.86:0/1536762212 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728169929 0 0) 0x7fab9015d710 con 0x7fab900b3430 2024-03-22T04:18:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab94c00640 1 --2- 172.21.15.86:0/1536762212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9015d0b0 0x7fab900b4d80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fab880097c0 tx=0x7fab8801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850a84c1d6fbb871 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab8ea00640 1 -- 172.21.15.86:0/1536762212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab880b2070 con 0x7fab9015d0b0 2024-03-22T04:18:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab8ea00640 1 -- 172.21.15.86:0/1536762212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab880ad640 con 0x7fab9015d0b0 2024-03-22T04:18:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab8ea00640 1 -- 172.21.15.86:0/1536762212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab880ada10 con 0x7fab9015d0b0 2024-03-22T04:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab96c00640 1 -- 172.21.15.86:0/1536762212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9015d0b0 msgr2=0x7fab900b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab96c00640 1 --2- 172.21.15.86:0/1536762212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9015d0b0 0x7fab900b4d80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fab880097c0 tx=0x7fab8801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab96c00640 1 -- 172.21.15.86:0/1536762212 shutdown_connections 2024-03-22T04:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab96c00640 1 --2- 172.21.15.86:0/1536762212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9015d0b0 0x7fab900b4d80 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab96c00640 1 --2- 172.21.15.86:0/1536762212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 0x7fab900b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab96c00640 1 -- 172.21.15.86:0/1536762212 >> 172.21.15.86:0/1536762212 conn(0x7fab900cc940 msgr2=0x7fab900ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab96c00640 1 -- 172.21.15.86:0/1536762212 shutdown_connections 2024-03-22T04:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.385+0000 7fab96c00640 1 -- 172.21.15.86:0/1536762212 wait complete. 2024-03-22T04:18:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 Processor -- start 2024-03-22T04:18:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 -- start start 2024-03-22T04:18:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 0x7fab90144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab900b3430 0x7fab901506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab9015d0b0 0x7fab90145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab90160c00 con 0x7fab900b3430 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab900b5ad0 con 0x7fab9015d0b0 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab9015b8c0 con 0x7fab900b2360 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab900b3430 0x7fab901506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab900b3430 0x7fab901506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43742/0 (socket says 172.21.15.86:43742) 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 -- 172.21.15.86:0/107483500 learned_addr learned my addr 172.21.15.86:0/107483500 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8fe00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 0x7fab90144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab94c00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab9015d0b0 0x7fab90145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 -- 172.21.15.86:0/107483500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 msgr2=0x7fab90144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 0x7fab90144cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 -- 172.21.15.86:0/107483500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab9015d0b0 msgr2=0x7fab90145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab9015d0b0 0x7fab90145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 -- 172.21.15.86:0/107483500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab900b1b00 con 0x7fab900b3430 2024-03-22T04:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab8f400640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab900b3430 0x7fab901506d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0011f0 tx=0x7fab7c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab87e00640 1 -- 172.21.15.86:0/107483500 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c018070 con 0x7fab900b3430 2024-03-22T04:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab87e00640 1 -- 172.21.15.86:0/107483500 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7c0074e0 con 0x7fab900b3430 2024-03-22T04:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 -- 172.21.15.86:0/107483500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab901451f0 con 0x7fab900b3430 2024-03-22T04:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab87e00640 1 -- 172.21.15.86:0/107483500 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c004a10 con 0x7fab900b3430 2024-03-22T04:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.389+0000 7fab96c00640 1 -- 172.21.15.86:0/107483500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9015bb10 con 0x7fab900b3430 2024-03-22T04:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.393+0000 7fab87e00640 1 -- 172.21.15.86:0/107483500 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fab7c01b070 con 0x7fab900b3430 2024-03-22T04:18:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.393+0000 7fab87e00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab70067b50 0x7fab7006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.393+0000 7fab87e00640 1 -- 172.21.15.86:0/107483500 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fab7c085430 con 0x7fab900b3430 2024-03-22T04:18:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.393+0000 7fab8fe00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab70067b50 0x7fab7006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.393+0000 7fab9a18e000 1 Processor -- start 2024-03-22T04:18:58.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.393+0000 7fab9a18e000 1 -- start start 2024-03-22T04:18:58.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.393+0000 7fab8fe00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab70067b50 0x7fab7006a010 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fab9015fcd0 tx=0x7fab8000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8da4f80 0x557ed8da5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 0x557ed8d58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8d58f50 0x557ed8d59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ed8ba9c40 con 0x557ed8da4f80 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ed8b46d90 con 0x557ed8d58f50 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x557ed8ba5bf0 con 0x557ed8da58a0 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8da4f80 0x557ed8da5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8da4f80 0x557ed8da5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43754/0 (socket says 172.21.15.86:43754) 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 -- 172.21.15.86:0/3802171469 learned_addr learned my addr 172.21.15.86:0/3802171469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8f400640 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 0x557ed8d58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab94c00640 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8d58f50 0x557ed8d59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 -- 172.21.15.86:0/3802171469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 msgr2=0x557ed8d58a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 0x557ed8d58a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 -- 172.21.15.86:0/3802171469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8d58f50 msgr2=0x557ed8d59330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8d58f50 0x557ed8d59330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 -- 172.21.15.86:0/3802171469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed8c51130 con 0x557ed8da4f80 2024-03-22T04:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8da4f80 0x557ed8da5360 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fab80003d40 tx=0x7fab80014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dc4c2d0eaa5730 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab84c00640 1 -- 172.21.15.86:0/3802171469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8001d050 con 0x557ed8da4f80 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab84c00640 1 -- 172.21.15.86:0/3802171469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab80027070 con 0x557ed8da4f80 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab84c00640 1 -- 172.21.15.86:0/3802171469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80015a80 con 0x557ed8da4f80 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- 172.21.15.86:0/3802171469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8da4f80 msgr2=0x557ed8da5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8da4f80 0x557ed8da5360 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fab80003d40 tx=0x7fab80014000 comp rx=0 tx=0).stop 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- 172.21.15.86:0/3802171469 shutdown_connections 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8d58f50 0x557ed8d59330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 0x557ed8d58a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3802171469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8da4f80 0x557ed8da5360 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- 172.21.15.86:0/3802171469 >> 172.21.15.86:0/3802171469 conn(0x557ed8c42990 msgr2=0x557ed8d88150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- 172.21.15.86:0/3802171469 shutdown_connections 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- 172.21.15.86:0/3802171469 wait complete. 2024-03-22T04:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 -- start start 2024-03-22T04:18:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.397+0000 7fab9a18e000 1 Processor -- start 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8d60a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 0x557ed8d89380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8d898c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ed8b65710 con 0x557ed8d58f50 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ed8b67580 con 0x557ed8da58a0 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x557ed8bae270 con 0x557ed8da4f80 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 10 client.0 _ll_register_callbacks cb 0x557ed8c0b200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8d60a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8d60a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43770/0 (socket says 172.21.15.86:43770) 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 -- 172.21.15.86:0/3881026783 learned_addr learned my addr 172.21.15.86:0/3881026783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8f400640 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 0x557ed8d89380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab94c00640 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8d898c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 -- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 msgr2=0x557ed8d89380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 0x557ed8d89380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 -- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 msgr2=0x557ed8d898c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8d898c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 -- 172.21.15.86:0/3881026783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed8b82d30 con 0x557ed8d58f50 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8d60a40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fab8000afd0 tx=0x7fab80004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b94ff9297fe8f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 1 -- 172.21.15.86:0/3881026783 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80022640 con 0x557ed8d58f50 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 -- 172.21.15.86:0/3881026783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab80027070 con 0x557ed8d58f50 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 1 -- 172.21.15.86:0/3881026783 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fab800227e0 con 0x557ed8d58f50 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 1 -- 172.21.15.86:0/3881026783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fab8001d020 con 0x557ed8d58f50 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 10 client.8112 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 -- 172.21.15.86:0/3881026783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x557ed8beefc0 con 0x557ed8d58f50 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 1 -- 172.21.15.86:0/3881026783 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab80027070 con 0x557ed8d58f50 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 1 -- 172.21.15.86:0/3881026783 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7fab80022bf0 con 0x557ed8d58f50 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 10 client.8112 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 10 client.8112 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 20 client.8112 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x557ed8dab5c0 0x557ed8dab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 4 client.8112 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 -- 172.21.15.86:0/3881026783 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed8b44dd0 con 0x557ed8dab5c0 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8f400640 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x557ed8dab5c0 0x557ed8dab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab8f400640 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x557ed8dab5c0 0x557ed8dab9d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916a8ea08fcbbc99 server_cookie=e1296c3fc86181f3 in_seq=0 out_seq=0 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 10 client.8112 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 1 -- 172.21.15.86:0/3881026783 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557ed8b44dd0 con 0x557ed8dab5c0 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab79600640 10 client.8112 handle_command_reply: tid=0 2024-03-22T04:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 client.8112 shutdown 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 -- 172.21.15.86:0/3881026783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 msgr2=0x557ed8d60a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.401+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8d60a40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fab8000afd0 tx=0x7fab80004a50 comp rx=0 tx=0).stop 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 1 -- 172.21.15.86:0/3881026783 shutdown_connections 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x557ed8dab5c0 0x557ed8dab9d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8d898c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 0x557ed8d89380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3881026783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8d60a40 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab79600640 0 client.8112 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 1 -- 172.21.15.86:0/3881026783 >> 172.21.15.86:0/3881026783 conn(0x557ed8c42990 msgr2=0x557ed8d58110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 1 -- 172.21.15.86:0/3881026783 shutdown_connections 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 1 -- 172.21.15.86:0/3881026783 wait complete. 2024-03-22T04:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.405+0000 7fab9a18e000 20 client.8112 trim_cache size 0 max 16384 2024-03-22T04:18:58.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 Processor -- start 2024-03-22T04:18:58.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- start start 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8db19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da4f80 0x557ed8dbb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 0x557ed8daa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab902041e0 con 0x557ed8d58f50 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab902031c0 con 0x557ed8da4f80 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab900b0ad0 con 0x557ed8da58a0 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8db19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8db19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43780/0 (socket says 172.21.15.86:43780) 2024-03-22T04:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 -- 172.21.15.86:0/3231124858 learned_addr learned my addr 172.21.15.86:0/3231124858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8f400640 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da4f80 0x557ed8dbb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab94c00640 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 0x557ed8daa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 -- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 msgr2=0x557ed8daa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 0x557ed8daa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 -- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da4f80 msgr2=0x557ed8dbb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da4f80 0x557ed8dbb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 -- 172.21.15.86:0/3231124858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed8b7f3a0 con 0x557ed8d58f50 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8f400640 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da4f80 0x557ed8dbb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab8fe00640 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8db19f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fab80009bf0 tx=0x7fab8002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7ecbcb5ef2d9dd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab7be00640 1 -- 172.21.15.86:0/3231124858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80047070 con 0x557ed8d58f50 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab7be00640 1 -- 172.21.15.86:0/3231124858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab80027050 con 0x557ed8d58f50 2024-03-22T04:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab7be00640 1 -- 172.21.15.86:0/3231124858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80022640 con 0x557ed8d58f50 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- 172.21.15.86:0/3231124858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 msgr2=0x557ed8db19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8db19f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fab80009bf0 tx=0x7fab8002d000 comp rx=0 tx=0).stop 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- 172.21.15.86:0/3231124858 shutdown_connections 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da4f80 0x557ed8dbb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da58a0 0x557ed8daa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 --2- 172.21.15.86:0/3231124858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8db19f0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- 172.21.15.86:0/3231124858 >> 172.21.15.86:0/3231124858 conn(0x557ed8c42990 msgr2=0x557ed8da9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- 172.21.15.86:0/3231124858 shutdown_connections 2024-03-22T04:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.421+0000 7fab9a18e000 1 -- 172.21.15.86:0/3231124858 wait complete. 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 -- start start 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 Processor -- start 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8e1cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 0x557ed8e1d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8dad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ed8c65810 con 0x557ed8d58f50 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab90202b10 con 0x557ed8da58a0 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x557ed8b65710 con 0x557ed8da4f80 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 10 client.0 _ll_register_callbacks cb 0x7fab800095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8e1cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8e1cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43794/0 (socket says 172.21.15.86:43794) 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 -- 172.21.15.86:0/2393703429 learned_addr learned my addr 172.21.15.86:0/2393703429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8f400640 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 0x557ed8e1d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab94c00640 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8dad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 -- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 msgr2=0x557ed8e1d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 0x557ed8e1d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 -- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 msgr2=0x557ed8dad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8dad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 -- 172.21.15.86:0/2393703429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed8b2f380 con 0x557ed8d58f50 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab94c00640 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8dad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8fe00640 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8e1cc60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fab80009720 tx=0x7fab80018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec0c1cc86e65aa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab79600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab79600640 1 -- 172.21.15.86:0/2393703429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80047040 con 0x557ed8d58f50 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab79600640 1 -- 172.21.15.86:0/2393703429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fab800174d0 con 0x557ed8d58f50 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 -- 172.21.15.86:0/2393703429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab80027050 con 0x557ed8d58f50 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab79600640 1 -- 172.21.15.86:0/2393703429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fab80058050 con 0x557ed8d58f50 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 10 client.8118 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 -- 172.21.15.86:0/2393703429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x557ed8c2b460 con 0x557ed8d58f50 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab79600640 1 -- 172.21.15.86:0/2393703429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab80027050 con 0x557ed8d58f50 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab79600640 1 -- 172.21.15.86:0/2393703429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7fab8005b020 con 0x557ed8d58f50 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 10 client.8118 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 10 client.8118 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 20 client.8118 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x557ed8dae3a0 0x557ed8dae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 4 client.8118 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab9a18e000 1 -- 172.21.15.86:0/2393703429 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ed8b49160 con 0x557ed8dae3a0 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.425+0000 7fab8f400640 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x557ed8dae3a0 0x557ed8dae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab79600640 10 client.8118 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab8f400640 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x557ed8dae3a0 0x557ed8dae7b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf874eb91a687fc6 server_cookie=20504cf76e9948d7 in_seq=0 out_seq=0 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab79600640 1 -- 172.21.15.86:0/2393703429 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557ed8b49160 con 0x557ed8dae3a0 2024-03-22T04:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab79600640 10 client.8118 handle_command_reply: tid=0 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 client.8118 shutdown 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 -- 172.21.15.86:0/2393703429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 msgr2=0x557ed8e1cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8e1cc60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fab80009720 tx=0x7fab80018000 comp rx=0 tx=0).stop 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 -- 172.21.15.86:0/2393703429 shutdown_connections 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x557ed8dae3a0 0x557ed8dae7b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed8da58a0 0x557ed8dad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab79600640 0 client.8118 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557ed8da4f80 0x557ed8e1d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 --2- 172.21.15.86:0/2393703429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed8d58f50 0x557ed8e1cc60 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 -- 172.21.15.86:0/2393703429 >> 172.21.15.86:0/2393703429 conn(0x557ed8c42990 msgr2=0x557ed8d88550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 -- 172.21.15.86:0/2393703429 shutdown_connections 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 1 -- 172.21.15.86:0/2393703429 wait complete. 2024-03-22T04:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab9a18e000 20 client.8118 trim_cache size 0 max 16384 2024-03-22T04:18:58.433 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:18:58.433 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:18:58.433 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:18:58.433 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 -- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab70067b50 msgr2=0x7fab7006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab70067b50 0x7fab7006a010 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fab9015fcd0 tx=0x7fab8000b040 comp rx=0 tx=0).stop 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 -- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab900b3430 msgr2=0x7fab901506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab900b3430 0x7fab901506d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0011f0 tx=0x7fab7c00c9b0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 -- 172.21.15.86:0/107483500 shutdown_connections 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab70067b50 0x7fab7006a010 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab9015d0b0 0x7fab90145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab900b3430 0x7fab901506d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 --2- 172.21.15.86:0/107483500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab900b2360 0x7fab90144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 -- 172.21.15.86:0/107483500 >> 172.21.15.86:0/107483500 conn(0x7fab900cc940 msgr2=0x7fab90151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:58.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 -- 172.21.15.86:0/107483500 shutdown_connections 2024-03-22T04:18:58.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.429+0000 7fab96c00640 1 -- 172.21.15.86:0/107483500 wait complete. 2024-03-22T04:18:58.447 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:18:58.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:18:58.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.581+0000 7fadce200640 1 Processor -- start 2024-03-22T04:18:58.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.581+0000 7fadce200640 1 -- start start 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadce200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadce200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fadc80b3800 0x7fadc806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadce200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc806e350 con 0x7fadc8161880 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadce200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fadc815ee80 con 0x7fadc80b3800 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadce200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fadc806e630 con 0x7fadc8161c60 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc7400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fadc8161880 0x7fadc80b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:49796/0 (socket says 172.21.15.86:49796) 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc7400640 1 -- 172.21.15.86:0/263439701 learned_addr learned my addr 172.21.15.86:0/263439701 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadbea00640 1 --2- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc7e00640 1 --2- 172.21.15.86:0/263439701 >> v2:172.21.15.196:3300/0 conn(0x7fadc80b3800 0x7fadc806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc6a00640 1 -- 172.21.15.86:0/263439701 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1414031601 0 0) 0x7fadc806e350 con 0x7fadc8161880 2024-03-22T04:18:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc6a00640 1 -- 172.21.15.86:0/263439701 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadac003530 con 0x7fadc8161880 2024-03-22T04:18:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc7e00640 1 -- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 msgr2=0x7fadc806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc7e00640 1 --2- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc7e00640 1 -- 172.21.15.86:0/263439701 >> v1:172.21.15.196:6789/0 conn(0x7fadc8161880 legacy=0x7fadc80b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadc7e00640 1 -- 172.21.15.86:0/263439701 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc806e910 con 0x7fadc80b3800 2024-03-22T04:18:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.585+0000 7fadbea00640 1 --2- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadc7e00640 1 --2- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc80b3800 0x7fadc806dc40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009d90 tx=0x7fadc001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe822d97aef9cd23 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadc6a00640 1 -- 172.21.15.86:0/263439701 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc00b2070 con 0x7fadc80b3800 2024-03-22T04:18:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadc6a00640 1 -- 172.21.15.86:0/263439701 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc00a5890 con 0x7fadc80b3800 2024-03-22T04:18:58.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadc6a00640 1 -- 172.21.15.86:0/263439701 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc00a5c60 con 0x7fadc80b3800 2024-03-22T04:18:58.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 -- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc80b3800 msgr2=0x7fadc806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 --2- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc80b3800 0x7fadc806dc40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009d90 tx=0x7fadc001c8e0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 -- 172.21.15.86:0/263439701 shutdown_connections 2024-03-22T04:18:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 --2- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc80b3800 0x7fadc806dc40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 --2- 172.21.15.86:0/263439701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 -- 172.21.15.86:0/263439701 >> 172.21.15.86:0/263439701 conn(0x7fadc80b1df0 msgr2=0x7fadc80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 -- 172.21.15.86:0/263439701 shutdown_connections 2024-03-22T04:18:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 -- 172.21.15.86:0/263439701 wait complete. 2024-03-22T04:18:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 Processor -- start 2024-03-22T04:18:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 -- start start 2024-03-22T04:18:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.589+0000 7fadce200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc80b3800 0x7fadc809cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadce200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc8161880 0x7fadc8141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadce200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc815a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadce200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81608d0 con 0x7fadc80b3800 2024-03-22T04:18:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadce200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8160750 con 0x7fadc8161880 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadce200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fadc8160a50 con 0x7fadc8161c60 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc80b3800 0x7fadc809cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc80b3800 0x7fadc809cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43806/0 (socket says 172.21.15.86:43806) 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 -- 172.21.15.86:0/3429563589 learned_addr learned my addr 172.21.15.86:0/3429563589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadbea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc8161880 0x7fadc8141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7e00640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 -- 172.21.15.86:0/3429563589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 msgr2=0x7fadc815a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc815a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 -- 172.21.15.86:0/3429563589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc8161880 msgr2=0x7fadc8141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc8161880 0x7fadc8141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 -- 172.21.15.86:0/3429563589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc815a690 con 0x7fadc80b3800 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadbea00640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc8161880 0x7fadc8141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadc7400640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc80b3800 0x7fadc809cc10 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fadb800c450 tx=0x7fadb800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadbfe00640 1 -- 172.21.15.86:0/3429563589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8017070 con 0x7fadc80b3800 2024-03-22T04:18:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadbfe00640 1 -- 172.21.15.86:0/3429563589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb8005cf0 con 0x7fadc80b3800 2024-03-22T04:18:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc809be50 con 0x7fadc80b3800 2024-03-22T04:18:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadbfe00640 1 -- 172.21.15.86:0/3429563589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8007290 con 0x7fadc80b3800 2024-03-22T04:18:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc809c410 con 0x7fadc80b3800 2024-03-22T04:18:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc8068650 con 0x7fadc80b3800 2024-03-22T04:18:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadbfe00640 1 -- 172.21.15.86:0/3429563589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fadb801a070 con 0x7fadc80b3800 2024-03-22T04:18:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.593+0000 7fadbfe00640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fada4067b20 0x7fada4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.597+0000 7fadbfe00640 1 -- 172.21.15.86:0/3429563589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fadb8085530 con 0x7fadc80b3800 2024-03-22T04:18:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.597+0000 7fadbea00640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fada4067b20 0x7fada4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.597+0000 7fadbea00640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fada4067b20 0x7fada4069fe0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fadc809e1f0 tx=0x7fadb0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.601+0000 7fadbfe00640 1 -- 172.21.15.86:0/3429563589 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fadb80555b0 con 0x7fadc80b3800 2024-03-22T04:18:58.738 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:18:58.742 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:18:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadc80cfdb0 con 0x7fadc80b3800 2024-03-22T04:18:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadbfe00640 1 -- 172.21.15.86:0/3429563589 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5186 (secure 0 0 0) 0x7fadb8059650 con 0x7fadc80b3800 2024-03-22T04:18:58.792 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:58.792 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":77,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:18:25.189133+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76850":{"gid":76850,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.196:6835/1227183960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1227183960},{"type":"v1","addr":"172.21.15.196:6835","nonce":1227183960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76850,"qdb_cluster":[76850]},"id":6}]} 2024-03-22T04:18:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 77 2024-03-22T04:18:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fada4067b20 msgr2=0x7fada4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fada4067b20 0x7fada4069fe0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fadc809e1f0 tx=0x7fadb0009190 comp rx=0 tx=0).stop 2024-03-22T04:18:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc80b3800 msgr2=0x7fadc809cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc80b3800 0x7fadc809cc10 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fadb800c450 tx=0x7fadb800c910 comp rx=0 tx=0).stop 2024-03-22T04:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 shutdown_connections 2024-03-22T04:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fada4067b20 0x7fada4069fe0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc8161880 0x7fadc8141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc8161c60 0x7fadc815a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 --2- 172.21.15.86:0/3429563589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadc80b3800 0x7fadc809cc10 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 >> 172.21.15.86:0/3429563589 conn(0x7fadc80b1df0 msgr2=0x7fadc815ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 shutdown_connections 2024-03-22T04:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.793+0000 7fadce200640 1 -- 172.21.15.86:0/3429563589 wait complete. 2024-03-22T04:18:58.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-03-22T04:18:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7f200640 1 Processor -- start 2024-03-22T04:18:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7f200640 1 -- start start 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7f200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7e8006f1a0 0x7f7e800d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e8015c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8015fbe0 con 0x7f7e80156620 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7f200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7e800ad4b0 con 0x7f7e8006f1a0 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e80156a00 con 0x7f7e800d0640 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e77e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7e8006f1a0 0x7f7e800d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e8015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7de00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7e80156620 0x7f7e800acda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:49802/0 (socket says 172.21.15.86:49802) 2024-03-22T04:18:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7de00640 1 -- 172.21.15.86:0/1077550347 learned_addr learned my addr 172.21.15.86:0/1077550347 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:18:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1077550347 >> v2:172.21.15.196:3300/0 conn(0x7f7e8006f1a0 0x7f7e800d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:55070/0 (socket says 172.21.15.86:55070) 2024-03-22T04:18:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7d400640 1 -- 172.21.15.86:0/1077550347 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3058973029 0 0) 0x7f7e8015fbe0 con 0x7f7e80156620 2024-03-22T04:18:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7d400640 1 -- 172.21.15.86:0/1077550347 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e64003530 con 0x7f7e80156620 2024-03-22T04:18:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e77e00640 1 -- 172.21.15.86:0/1077550347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 msgr2=0x7f7e8015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1077550347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e8015c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e77e00640 1 -- 172.21.15.86:0/1077550347 >> v1:172.21.15.196:6789/0 conn(0x7f7e80156620 legacy=0x7f7e800acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e77e00640 1 -- 172.21.15.86:0/1077550347 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8015fe50 con 0x7f7e8006f1a0 2024-03-22T04:18:59.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:58.997+0000 7f7e7e800640 1 --2- 172.21.15.86:0/1077550347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e8015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1077550347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800d0100 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7e680097c0 tx=0x7f7e6801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5ffca41c0794d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7d400640 1 -- 172.21.15.86:0/1077550347 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680b2070 con 0x7f7e8006f1a0 2024-03-22T04:18:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7d400640 1 -- 172.21.15.86:0/1077550347 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e680ad640 con 0x7f7e8006f1a0 2024-03-22T04:18:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7d400640 1 -- 172.21.15.86:0/1077550347 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680ada10 con 0x7f7e8006f1a0 2024-03-22T04:18:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7f200640 1 -- 172.21.15.86:0/1077550347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 msgr2=0x7f7e800d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1077550347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800d0100 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7e680097c0 tx=0x7f7e6801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7f200640 1 -- 172.21.15.86:0/1077550347 shutdown_connections 2024-03-22T04:18:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1077550347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e8015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1077550347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800d0100 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7f200640 1 -- 172.21.15.86:0/1077550347 >> 172.21.15.86:0/1077550347 conn(0x7f7e8006ba60 msgr2=0x7f7e8006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7f200640 1 -- 172.21.15.86:0/1077550347 shutdown_connections 2024-03-22T04:18:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.001+0000 7f7e7f200640 1 -- 172.21.15.86:0/1077550347 wait complete. 2024-03-22T04:18:59.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 Processor -- start 2024-03-22T04:18:59.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 -- start start 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e801460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e80156620 0x7f7e8009ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8006d480 con 0x7f7e80156620 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8006d300 con 0x7f7e8006f1a0 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8015d2a0 con 0x7f7e800d0640 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e80156620 0x7f7e8009ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e80156620 0x7f7e8009ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43818/0 (socket says 172.21.15.86:43818) 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:55072/0 (socket says 172.21.15.86:55072) 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7e800640 1 -- 172.21.15.86:0/1320904762 learned_addr learned my addr 172.21.15.86:0/1320904762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e801460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7de00640 1 -- 172.21.15.86:0/1320904762 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 msgr2=0x7f7e801460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e801460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7de00640 1 -- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e80156620 msgr2=0x7f7e8009ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e80156620 0x7f7e8009ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7de00640 1 -- 172.21.15.86:0/1320904762 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e800b3af0 con 0x7f7e8006f1a0 2024-03-22T04:18:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800b35b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c390 tx=0x7f7e70001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e76000640 1 -- 172.21.15.86:0/1320904762 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70017070 con 0x7f7e8006f1a0 2024-03-22T04:18:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e76000640 1 -- 172.21.15.86:0/1320904762 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e70005cf0 con 0x7f7e8006f1a0 2024-03-22T04:18:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 -- 172.21.15.86:0/1320904762 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e800afa60 con 0x7f7e8006f1a0 2024-03-22T04:18:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e76000640 1 -- 172.21.15.86:0/1320904762 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e700072f0 con 0x7f7e8006f1a0 2024-03-22T04:18:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.005+0000 7f7e7f200640 1 -- 172.21.15.86:0/1320904762 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e800b0020 con 0x7f7e8006f1a0 2024-03-22T04:18:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.009+0000 7f7e76000640 1 -- 172.21.15.86:0/1320904762 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7e7001a070 con 0x7f7e8006f1a0 2024-03-22T04:18:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.009+0000 7f7e76000640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e5c067b20 0x7f7e5c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.009+0000 7f7e76000640 1 -- 172.21.15.86:0/1320904762 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f7e70085420 con 0x7f7e8006f1a0 2024-03-22T04:18:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.009+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e5c067b20 0x7f7e5c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.009+0000 7f7e883d4000 1 Processor -- start 2024-03-22T04:18:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- start start 2024-03-22T04:18:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e5c067b20 0x7f7e5c069fe0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7e680af000 tx=0x7f7e68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4c062f0 0x5642e4c066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4bbc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4bbcd60 0x5642e4bbd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642e4a0bd80 con 0x5642e4c062f0 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642e49a8ce0 con 0x5642e4c06c10 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5642e4a07d30 con 0x5642e4bbcd60 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4c062f0 0x5642e4c066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4c062f0 0x5642e4c066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43834/0 (socket says 172.21.15.86:43834) 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 -- 172.21.15.86:0/1313592530 learned_addr learned my addr 172.21.15.86:0/1313592530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4bbc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7e800640 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4bbcd60 0x5642e4bbd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 -- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4bbcd60 msgr2=0x5642e4bbd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4bbcd60 0x5642e4bbd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 -- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 msgr2=0x5642e4bbc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4bbc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 -- 172.21.15.86:0/1313592530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e49e6340 con 0x5642e4c062f0 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4bbc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4c062f0 0x5642e4c066d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7e700836a0 tx=0x7f7e700010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3734be47f3cdb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e6ea00640 1 -- 172.21.15.86:0/1313592530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7005de00 con 0x5642e4c062f0 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e6ea00640 1 -- 172.21.15.86:0/1313592530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e70053830 con 0x5642e4c062f0 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e6ea00640 1 -- 172.21.15.86:0/1313592530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70085070 con 0x5642e4c062f0 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- 172.21.15.86:0/1313592530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4c062f0 msgr2=0x5642e4c066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4c062f0 0x5642e4c066d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7e700836a0 tx=0x7f7e700010b0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- 172.21.15.86:0/1313592530 shutdown_connections 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4bbc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4bbcd60 0x5642e4bbd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 --2- 172.21.15.86:0/1313592530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4c062f0 0x5642e4c066d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- 172.21.15.86:0/1313592530 >> 172.21.15.86:0/1313592530 conn(0x5642e4bb8d10 msgr2=0x5642e4bbb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- 172.21.15.86:0/1313592530 shutdown_connections 2024-03-22T04:18:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- 172.21.15.86:0/1313592530 wait complete. 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 -- start start 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.013+0000 7f7e883d4000 1 Processor -- start 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bc1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bc70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4bc75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642e49c94e0 con 0x5642e4bbcd60 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642e4a8c3b0 con 0x5642e4c062f0 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5642e4bb8b90 con 0x5642e4c06c10 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 10 client.0 _ll_register_callbacks cb 0x5642e4ba7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bc1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bc1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43846/0 (socket says 172.21.15.86:43846) 2024-03-22T04:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 -- 172.21.15.86:0/2235035839 learned_addr learned my addr 172.21.15.86:0/2235035839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e77e00640 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bc70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7e800640 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4bc75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 -- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 msgr2=0x5642e4bc75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4bc75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 -- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 msgr2=0x5642e4bc70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bc70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 -- 172.21.15.86:0/2235035839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e4a505e0 con 0x5642e4bbcd60 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7e800640 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4bc75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e7de00640 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bc1c40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7e70051990 tx=0x7f7e70000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3cde5f056655d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 1 -- 172.21.15.86:0/2235035839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70085280 con 0x5642e4bbcd60 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 1 -- 172.21.15.86:0/2235035839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f7e7001de30 con 0x5642e4bbcd60 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 -- 172.21.15.86:0/2235035839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e70053830 con 0x5642e4bbcd60 2024-03-22T04:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 1 -- 172.21.15.86:0/2235035839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f7e700591b0 con 0x5642e4bbcd60 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 10 client.8130 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 -- 172.21.15.86:0/2235035839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5642e49c75a0 con 0x5642e4bbcd60 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 1 -- 172.21.15.86:0/2235035839 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e70053830 con 0x5642e4bbcd60 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 1 -- 172.21.15.86:0/2235035839 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f7e70007490 con 0x5642e4bbcd60 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 10 client.8130 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 10 client.8130 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 20 client.8130 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5642e4c15a60 0x5642e4c17e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 4 client.8130 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 -- 172.21.15.86:0/2235035839 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642e49a6e40 con 0x5642e4c15a60 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e77e00640 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5642e4c15a60 0x5642e4c17e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e77e00640 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5642e4c15a60 0x5642e4c17e50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2214b6ff73a208f7 server_cookie=60cfac0b411307ba in_seq=0 out_seq=0 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 10 client.8130 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 1 -- 172.21.15.86:0/2235035839 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5642e49a6e40 con 0x5642e4c15a60 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e63400640 10 client.8130 handle_command_reply: tid=0 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 client.8130 shutdown 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 -- 172.21.15.86:0/2235035839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 msgr2=0x5642e4bc1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bc1c40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7e70051990 tx=0x7f7e70000bd0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 -- 172.21.15.86:0/2235035839 shutdown_connections 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.017+0000 7f7e883d4000 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5642e4c15a60 0x5642e4c17e50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.021+0000 7f7e883d4000 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bc70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.021+0000 7f7e883d4000 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4bc75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.021+0000 7f7e63400640 0 client.8130 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.021+0000 7f7e883d4000 1 --2- 172.21.15.86:0/2235035839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bc1c40 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.021+0000 7f7e883d4000 1 -- 172.21.15.86:0/2235035839 >> 172.21.15.86:0/2235035839 conn(0x5642e4bb8d10 msgr2=0x5642e4bbb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.021+0000 7f7e883d4000 1 -- 172.21.15.86:0/2235035839 shutdown_connections 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.021+0000 7f7e883d4000 1 -- 172.21.15.86:0/2235035839 wait complete. 2024-03-22T04:18:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.021+0000 7f7e883d4000 20 client.8130 trim_cache size 0 max 16384 2024-03-22T04:18:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 Processor -- start 2024-03-22T04:18:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 -- start start 2024-03-22T04:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bbc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bbc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4c18e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80204030 con 0x5642e4bbcd60 2024-03-22T04:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80203040 con 0x5642e4c062f0 2024-03-22T04:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e80202330 con 0x5642e4c06c10 2024-03-22T04:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bbc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bbc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43858/0 (socket says 172.21.15.86:43858) 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 -- 172.21.15.86:0/1479590883 learned_addr learned my addr 172.21.15.86:0/1479590883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bbc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7e800640 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4c18e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 -- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 msgr2=0x5642e4c18e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4c18e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 -- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 msgr2=0x5642e4bbc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bbc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 -- 172.21.15.86:0/1479590883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e49c7a50 con 0x5642e4bbcd60 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e77e00640 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bbc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e7de00640 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bbc0a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7e70059790 tx=0x7f7e7007e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f347f453a4d72651 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e6e000640 1 -- 172.21.15.86:0/1479590883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70083200 con 0x5642e4bbcd60 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e6e000640 1 -- 172.21.15.86:0/1479590883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e70007490 con 0x5642e4bbcd60 2024-03-22T04:18:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e6e000640 1 -- 172.21.15.86:0/1479590883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70054bc0 con 0x5642e4bbcd60 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 -- 172.21.15.86:0/1479590883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 msgr2=0x5642e4bbc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bbc0a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7e70059790 tx=0x7f7e7007e040 comp rx=0 tx=0).stop 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 -- 172.21.15.86:0/1479590883 shutdown_connections 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c062f0 0x5642e4bbc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c06c10 0x5642e4c18e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 --2- 172.21.15.86:0/1479590883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4bbc0a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.033+0000 7f7e883d4000 1 -- 172.21.15.86:0/1479590883 >> 172.21.15.86:0/1479590883 conn(0x5642e4bb8d10 msgr2=0x5642e4bbb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- 172.21.15.86:0/1479590883 shutdown_connections 2024-03-22T04:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- 172.21.15.86:0/1479590883 wait complete. 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- start start 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 Processor -- start 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4c1afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c062f0 0x5642e4c61b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4c620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642e49d9c20 con 0x5642e4bbcd60 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80202970 con 0x5642e4c06c10 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5642e4a0b090 con 0x5642e4c062f0 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 10 client.0 _ll_register_callbacks cb 0x7f7e70007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4c1afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4c1afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43872/0 (socket says 172.21.15.86:43872) 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 -- 172.21.15.86:0/525346512 learned_addr learned my addr 172.21.15.86:0/525346512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e77e00640 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c062f0 0x5642e4c61b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7e800640 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4c620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 -- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c062f0 msgr2=0x5642e4c61b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c062f0 0x5642e4c61b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 -- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 msgr2=0x5642e4c620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4c620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 -- 172.21.15.86:0/525346512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e4a8c840 con 0x5642e4bbcd60 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e7de00640 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4c1afd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7e70059790 tx=0x7f7e70000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a57429a4472106 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e63400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e63400640 1 -- 172.21.15.86:0/525346512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70083200 con 0x5642e4bbcd60 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e63400640 1 -- 172.21.15.86:0/525346512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f7e70080b40 con 0x5642e4bbcd60 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- 172.21.15.86:0/525346512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e70007490 con 0x5642e4bbcd60 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e63400640 1 -- 172.21.15.86:0/525346512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f7e70080d80 con 0x5642e4bbcd60 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 10 client.8136 fetch_fsmap learned FSMap version 77 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- 172.21.15.86:0/525346512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5642e49b23e0 con 0x5642e4bbcd60 2024-03-22T04:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e63400640 1 -- 172.21.15.86:0/525346512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e70007490 con 0x5642e4bbcd60 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e63400640 1 -- 172.21.15.86:0/525346512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 77) ==== 3494+0+0 (secure 0 0 0) 0x7f7e700543f0 con 0x5642e4bbcd60 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 10 client.8136 fetch_fsmap finished waiting for FSMap version 77 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 10 client.8136 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 20 client.8136 populate_metadata read hostname 'smithi086' 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5642e4c628b0 0x5642e4c62c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 4 client.8136 mds_command: new command op to 76850 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e883d4000 1 -- 172.21.15.86:0/525346512 --> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5642e4a192c0 con 0x5642e4c628b0 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.037+0000 7f7e77e00640 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5642e4c628b0 0x5642e4c62c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e77e00640 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5642e4c628b0 0x5642e4c62c90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0069a3802d6a20f server_cookie=ee6f1618446e71a5 in_seq=0 out_seq=0 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e63400640 10 client.8136 ms_handle_connect on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e63400640 1 -- 172.21.15.86:0/525346512 <== mds.0 v2:172.21.15.196:6834/1227183960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642e4a192c0 con 0x5642e4c628b0 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e63400640 10 client.8136 handle_command_reply: tid=0 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 client.8136 shutdown 2024-03-22T04:18:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 -- 172.21.15.86:0/525346512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 msgr2=0x5642e4c1afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4c1afd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7e70059790 tx=0x7f7e70000ee0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 -- 172.21.15.86:0/525346512 shutdown_connections 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:6834/1227183960,v1:172.21.15.196:6835/1227183960] conn(0x5642e4c628b0 0x5642e4c62c90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642e4c06c10 0x5642e4c620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5642e4c062f0 0x5642e4c61b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 --2- 172.21.15.86:0/525346512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642e4bbcd60 0x5642e4c1afd0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 -- 172.21.15.86:0/525346512 >> 172.21.15.86:0/525346512 conn(0x5642e4bb8d10 msgr2=0x5642e4bbc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e63400640 0 client.8136 ms_handle_reset on v2:172.21.15.196:6834/1227183960 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 -- 172.21.15.86:0/525346512 shutdown_connections 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 1 -- 172.21.15.86:0/525346512 wait complete. 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e883d4000 20 client.8136 trim_cache size 0 max 16384 2024-03-22T04:18:59.044 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:59.045 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-22T04:18:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 -- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e5c067b20 msgr2=0x7f7e5c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e5c067b20 0x7f7e5c069fe0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7e680af000 tx=0x7f7e68008040 comp rx=0 tx=0).stop 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 -- 172.21.15.86:0/1320904762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 msgr2=0x7f7e800b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800b35b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c390 tx=0x7f7e70001e80 comp rx=0 tx=0).stop 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 -- 172.21.15.86:0/1320904762 shutdown_connections 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e5c067b20 0x7f7e5c069fe0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e80156620 0x7f7e8009ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e800d0640 0x7f7e801460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 --2- 172.21.15.86:0/1320904762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8006f1a0 0x7f7e800b35b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 -- 172.21.15.86:0/1320904762 >> 172.21.15.86:0/1320904762 conn(0x7f7e8006ba60 msgr2=0x7f7e80155270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 -- 172.21.15.86:0/1320904762 shutdown_connections 2024-03-22T04:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.041+0000 7f7e7f200640 1 -- 172.21.15.86:0/1320904762 wait complete. 2024-03-22T04:18:59.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-22T04:18:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.193+0000 7f17ab600640 1 Processor -- start 2024-03-22T04:18:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.193+0000 7f17ab600640 1 -- start start 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17ab600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f17a415ecb0 0x7f17a406d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17ab600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415b380 0x7f17a415f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17ab600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f17a40b2f80 0x7f17a40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17ab600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f17a4072f90 con 0x7f17a40b2f80 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17ab600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f17a40721d0 con 0x7f17a415ecb0 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17ab600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f17a40724b0 con 0x7f17a415b380 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a9600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f17a40b2f80 0x7f17a40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a9600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f17a40b2f80 0x7f17a40b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43878/0 (socket says 172.21.15.86:43878) 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a9600640 1 -- 172.21.15.86:0/1712299530 learned_addr learned my addr 172.21.15.86:0/1712299530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a8c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f17a415ecb0 0x7f17a406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a3e00640 1 --2- 172.21.15.86:0/1712299530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415b380 0x7f17a415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a9600640 1 -- 172.21.15.86:0/1712299530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415b380 msgr2=0x7f17a415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a9600640 1 --2- 172.21.15.86:0/1712299530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415b380 0x7f17a415f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a9600640 1 -- 172.21.15.86:0/1712299530 >> v2:172.21.15.196:3300/0 conn(0x7f17a415ecb0 msgr2=0x7f17a406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a9600640 1 --2- 172.21.15.86:0/1712299530 >> v2:172.21.15.196:3300/0 conn(0x7f17a415ecb0 0x7f17a406d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.197+0000 7f17a9600640 1 -- 172.21.15.86:0/1712299530 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a4072790 con 0x7f17a40b2f80 2024-03-22T04:18:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17a9600640 1 --2- 172.21.15.86:0/1712299530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a40b3410 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f17a406fb10 tx=0x7f179c0213e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61e9dd40fa08767 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17a3400640 1 -- 172.21.15.86:0/1712299530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c0b6070 con 0x7f17a40b2f80 2024-03-22T04:18:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17a3400640 1 -- 172.21.15.86:0/1712299530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f179c00b2d0 con 0x7f17a40b2f80 2024-03-22T04:18:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17a3400640 1 -- 172.21.15.86:0/1712299530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c00b480 con 0x7f17a40b2f80 2024-03-22T04:18:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- 172.21.15.86:0/1712299530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 msgr2=0x7f17a40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 --2- 172.21.15.86:0/1712299530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a40b3410 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f17a406fb10 tx=0x7f179c0213e0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- 172.21.15.86:0/1712299530 shutdown_connections 2024-03-22T04:18:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 --2- 172.21.15.86:0/1712299530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a40b3410 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 --2- 172.21.15.86:0/1712299530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415b380 0x7f17a415f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 --2- 172.21.15.86:0/1712299530 >> v2:172.21.15.196:3300/0 conn(0x7f17a415ecb0 0x7f17a406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- 172.21.15.86:0/1712299530 >> 172.21.15.86:0/1712299530 conn(0x7f17a40b1c80 msgr2=0x7f17a40b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- 172.21.15.86:0/1712299530 shutdown_connections 2024-03-22T04:18:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- 172.21.15.86:0/1712299530 wait complete. 2024-03-22T04:18:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 Processor -- start 2024-03-22T04:18:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- start start 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a41f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17a415b380 0x7f17a41f1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415ecb0 0x7f17a415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17a40b3950 con 0x7f17a40b2f80 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17a415f5d0 con 0x7f17a415b380 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17ab600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f17a40b3ad0 con 0x7f17a415ecb0 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.201+0000 7f17a8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a41f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a41f1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43880/0 (socket says 172.21.15.86:43880) 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a8c00640 1 -- 172.21.15.86:0/1130482707 learned_addr learned my addr 172.21.15.86:0/1130482707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a3e00640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17a415b380 0x7f17a41f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a9600640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415ecb0 0x7f17a415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a8c00640 1 -- 172.21.15.86:0/1130482707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415ecb0 msgr2=0x7f17a415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a8c00640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415ecb0 0x7f17a415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a8c00640 1 -- 172.21.15.86:0/1130482707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17a415b380 msgr2=0x7f17a41f1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a8c00640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17a415b380 0x7f17a41f1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a8c00640 1 -- 172.21.15.86:0/1130482707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a415c6c0 con 0x7f17a40b2f80 2024-03-22T04:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a8c00640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a41f1bb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f179400e870 tx=0x7f179400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a0c00640 1 -- 172.21.15.86:0/1130482707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794017070 con 0x7f17a40b2f80 2024-03-22T04:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a0c00640 1 -- 172.21.15.86:0/1130482707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f179400c8e0 con 0x7f17a40b2f80 2024-03-22T04:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a4158b90 con 0x7f17a40b2f80 2024-03-22T04:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a0c00640 1 -- 172.21.15.86:0/1130482707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179400cc90 con 0x7f17a40b2f80 2024-03-22T04:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a4159150 con 0x7f17a40b2f80 2024-03-22T04:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a4158980 con 0x7f17a40b2f80 2024-03-22T04:18:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a0c00640 1 -- 172.21.15.86:0/1130482707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f179401a070 con 0x7f17a40b2f80 2024-03-22T04:18:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a0c00640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1780067b50 0x7f178006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a0c00640 1 -- 172.21.15.86:0/1130482707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f1794085ff0 con 0x7f17a40b2f80 2024-03-22T04:18:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.205+0000 7f17a3e00640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1780067b50 0x7f178006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.209+0000 7f17a3e00640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1780067b50 0x7f178006a010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f17a415ab20 tx=0x7f178c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.213+0000 7f17a0c00640 1 -- 172.21.15.86:0/1130482707 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f17940560c0 con 0x7f17a40b2f80 2024-03-22T04:18:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.397+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f17a4074550 con 0x7f17a40b2f80 2024-03-22T04:18:59.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.397+0000 7f17a0c00640 1 -- 172.21.15.86:0/1130482707 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f179405a160 con 0x7f17a40b2f80 2024-03-22T04:18:59.398 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:59.399 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T04:18:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1780067b50 msgr2=0x7f178006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1780067b50 0x7f178006a010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f17a415ab20 tx=0x7f178c009190 comp rx=0 tx=0).stop 2024-03-22T04:18:59.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 msgr2=0x7f17a41f1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a41f1bb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f179400e870 tx=0x7f179400ec70 comp rx=0 tx=0).stop 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 shutdown_connections 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1780067b50 0x7f178006a010 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17a415b380 0x7f17a41f1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17a415ecb0 0x7f17a415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 --2- 172.21.15.86:0/1130482707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a40b2f80 0x7f17a41f1bb0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 >> 172.21.15.86:0/1130482707 conn(0x7f17a40b1c80 msgr2=0x7f17a4154180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 shutdown_connections 2024-03-22T04:18:59.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.401+0000 7f17ab600640 1 -- 172.21.15.86:0/1130482707 wait complete. 2024-03-22T04:18:59.419 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-22T04:18:59.419 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:18:59.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:18:59.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:18:59.514 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:18:59.514 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:18:59.514 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:18:59.514 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:18:59.523 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:18:59.523 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:18:59.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:18:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.749+0000 7f3221a00640 1 Processor -- start 2024-03-22T04:18:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.749+0000 7f3221a00640 1 -- start start 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f3221a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f321c15b3f0 0x7f321c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f3221a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f321c0ae540 0x7f321c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f3221a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c071920 con 0x7f321c0cc3e0 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f3221a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f321c0716d0 con 0x7f321c0ae540 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f3221a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f321c06d990 con 0x7f321c15b3f0 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f321b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f321c0ae540 0x7f321c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f321b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f321c0ae540 0x7f321c072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43892/0 (socket says 172.21.15.86:43892) 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f321b400640 1 -- 172.21.15.86:0/1988335501 learned_addr learned my addr 172.21.15.86:0/1988335501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f321aa00640 1 --2- 172.21.15.86:0/1988335501 >> v2:172.21.15.196:3300/0 conn(0x7f321c15b3f0 0x7f321c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f321b400640 1 -- 172.21.15.86:0/1988335501 >> v2:172.21.15.196:3300/0 conn(0x7f321c15b3f0 msgr2=0x7f321c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:18:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f321b400640 1 --2- 172.21.15.86:0/1988335501 >> v2:172.21.15.196:3300/0 conn(0x7f321c15b3f0 0x7f321c0729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f321b400640 1 -- 172.21.15.86:0/1988335501 >> v1:172.21.15.196:6789/0 conn(0x7f321c0cc3e0 legacy=0x7f321c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.753+0000 7f321b400640 1 -- 172.21.15.86:0/1988335501 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c071c00 con 0x7f321c0ae540 2024-03-22T04:18:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f321b400640 1 --2- 172.21.15.86:0/1988335501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c072f00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f321c06c510 tx=0x7f321401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53f37169d7926b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f321a000640 1 -- 172.21.15.86:0/1988335501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140a5710 con 0x7f321c0ae540 2024-03-22T04:18:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f321a000640 1 -- 172.21.15.86:0/1988335501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32140a58b0 con 0x7f321c0ae540 2024-03-22T04:18:59.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f321a000640 1 -- 172.21.15.86:0/1988335501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140a5c80 con 0x7f321c0ae540 2024-03-22T04:18:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- 172.21.15.86:0/1988335501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 msgr2=0x7f321c072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 --2- 172.21.15.86:0/1988335501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c072f00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f321c06c510 tx=0x7f321401c960 comp rx=0 tx=0).stop 2024-03-22T04:18:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- 172.21.15.86:0/1988335501 shutdown_connections 2024-03-22T04:18:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 --2- 172.21.15.86:0/1988335501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c072f00 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 --2- 172.21.15.86:0/1988335501 >> v2:172.21.15.196:3300/0 conn(0x7f321c15b3f0 0x7f321c0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- 172.21.15.86:0/1988335501 >> 172.21.15.86:0/1988335501 conn(0x7f321c0b1df0 msgr2=0x7f321c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- 172.21.15.86:0/1988335501 shutdown_connections 2024-03-22T04:18:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- 172.21.15.86:0/1988335501 wait complete. 2024-03-22T04:18:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 Processor -- start 2024-03-22T04:18:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- start start 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321c0cc3e0 0x7f321c1640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c15b3f0 0x7f321c14f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f321c168580 con 0x7f321c0ae540 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f321c168400 con 0x7f321c15b3f0 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f3221a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f321c168700 con 0x7f321c0cc3e0 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.757+0000 7f321aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c14ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43906/0 (socket says 172.21.15.86:43906) 2024-03-22T04:18:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321aa00640 1 -- 172.21.15.86:0/711452487 learned_addr learned my addr 172.21.15.86:0/711452487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321b400640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c15b3f0 0x7f321c14f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f3212000640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321c0cc3e0 0x7f321c1640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321aa00640 1 -- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321c0cc3e0 msgr2=0x7f321c1640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321aa00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321c0cc3e0 0x7f321c1640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321aa00640 1 -- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c15b3f0 msgr2=0x7f321c14f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321aa00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c15b3f0 0x7f321c14f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321aa00640 1 -- 172.21.15.86:0/711452487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c164b60 con 0x7f321c0ae540 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321b400640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c15b3f0 0x7f321c14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f321aa00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c14ab60 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f320c00e960 tx=0x7f320c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f3213400640 1 -- 172.21.15.86:0/711452487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c017070 con 0x7f321c0ae540 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f3213400640 1 -- 172.21.15.86:0/711452487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c00ca00 con 0x7f321c0ae540 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f3213400640 1 -- 172.21.15.86:0/711452487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c00cdb0 con 0x7f321c0ae540 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c164e40 con 0x7f321c0ae540 2024-03-22T04:18:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c165400 con 0x7f321c0ae540 2024-03-22T04:18:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c068650 con 0x7f321c0ae540 2024-03-22T04:18:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.761+0000 7f3213400640 1 -- 172.21.15.86:0/711452487 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f320c01a070 con 0x7f321c0ae540 2024-03-22T04:18:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.765+0000 7f3213400640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31f8067b50 0x7f31f806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:18:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.765+0000 7f3213400640 1 -- 172.21.15.86:0/711452487 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f320c0861e0 con 0x7f321c0ae540 2024-03-22T04:18:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.765+0000 7f3212000640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31f8067b50 0x7f31f806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:18:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.765+0000 7f3212000640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31f8067b50 0x7f31f806a010 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f321c1514a0 tx=0x7f3204009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:18:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.769+0000 7f3213400640 1 -- 172.21.15.86:0/711452487 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f320c056250 con 0x7f321c0ae540 2024-03-22T04:18:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.925+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f321c158d50 con 0x7f321c0ae540 2024-03-22T04:18:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.925+0000 7f3213400640 1 -- 172.21.15.86:0/711452487 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17629 (secure 0 0 0) 0x7f320c05a2f0 con 0x7f321c0ae540 2024-03-22T04:18:59.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:18:59.927 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":97,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:18:21.162952+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.32","mappings":[{"from":7,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":3}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]},{"pgid":"14.3c","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3e","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6834/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6835/1097103199":"2024-03-23T04:17:08.939658+0000","172.21.15.196:6834/1097103199":"2024-03-23T04:17:08.939658+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31f8067b50 msgr2=0x7f31f806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31f8067b50 0x7f31f806a010 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f321c1514a0 tx=0x7f3204009040 comp rx=0 tx=0).stop 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 msgr2=0x7f321c14ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c14ab60 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f320c00e960 tx=0x7f320c00ee20 comp rx=0 tx=0).stop 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 shutdown_connections 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f31f8067b50 0x7f31f806a010 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c15b3f0 0x7f321c14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f321c0cc3e0 0x7f321c1640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 --2- 172.21.15.86:0/711452487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0ae540 0x7f321c14ab60 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 >> 172.21.15.86:0/711452487 conn(0x7f321c0b1df0 msgr2=0x7f321c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:18:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 shutdown_connections 2024-03-22T04:18:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:18:59.929+0000 7f3221a00640 1 -- 172.21.15.86:0/711452487 wait complete. 2024-03-22T04:18:59.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:16:56.341603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:16:56.977675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:18:59.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:19:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.089+0000 7f0b5bc00640 1 Processor -- start 2024-03-22T04:19:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.089+0000 7f0b5bc00640 1 -- start start 2024-03-22T04:19:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b5bc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b54072b00 0x7f0b5415be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b5bc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b54073150 con 0x7f0b54072650 2024-03-22T04:19:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b5bc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b540715a0 con 0x7f0b5406e7e0 2024-03-22T04:19:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b5bc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0b54071350 con 0x7f0b54072b00 2024-03-22T04:19:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b4be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0b54072650 0x7f0b5415b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59582/0 (socket says 172.21.15.86:59582) 2024-03-22T04:19:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b4be00640 1 -- 172.21.15.86:0/1989022116 learned_addr learned my addr 172.21.15.86:0/1989022116 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b59c00640 1 --2- 172.21.15.86:0/1989022116 >> v2:172.21.15.86:3300/0 conn(0x7f0b54072b00 0x7f0b5415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b59c00640 1 -- 172.21.15.86:0/1989022116 >> v1:172.21.15.196:6789/0 conn(0x7f0b5406e7e0 legacy=0x7f0b5406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b59c00640 1 -- 172.21.15.86:0/1989022116 >> v1:172.21.15.86:6789/0 conn(0x7f0b54072650 legacy=0x7f0b5415b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.093+0000 7f0b59c00640 1 -- 172.21.15.86:0/1989022116 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b54071880 con 0x7f0b54072b00 2024-03-22T04:19:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b59c00640 1 --2- 172.21.15.86:0/1989022116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5415be70 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009d90 tx=0x7f0b4c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e63ce351b60c5db server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b53e00640 1 -- 172.21.15.86:0/1989022116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0b2070 con 0x7f0b54072b00 2024-03-22T04:19:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b53e00640 1 -- 172.21.15.86:0/1989022116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b4c0a58b0 con 0x7f0b54072b00 2024-03-22T04:19:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b53e00640 1 -- 172.21.15.86:0/1989022116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0a5c80 con 0x7f0b54072b00 2024-03-22T04:19:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 -- 172.21.15.86:0/1989022116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 msgr2=0x7f0b5415be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 --2- 172.21.15.86:0/1989022116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5415be70 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009d90 tx=0x7f0b4c01c8e0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 -- 172.21.15.86:0/1989022116 shutdown_connections 2024-03-22T04:19:00.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 --2- 172.21.15.86:0/1989022116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5415be70 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 -- 172.21.15.86:0/1989022116 >> 172.21.15.86:0/1989022116 conn(0x7f0b540cc940 msgr2=0x7f0b540ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:00.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 -- 172.21.15.86:0/1989022116 shutdown_connections 2024-03-22T04:19:00.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 -- 172.21.15.86:0/1989022116 wait complete. 2024-03-22T04:19:00.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 Processor -- start 2024-03-22T04:19:00.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 -- start start 2024-03-22T04:19:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.097+0000 7f0b5bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b5406e7e0 0x7f0b54144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b5bc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b54072650 0x7f0b5414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b5bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5414d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b5bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b54155c20 con 0x7f0b54072b00 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b5bc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b54155aa0 con 0x7f0b54072650 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b5bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0b54169610 con 0x7f0b5406e7e0 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5414d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5414d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43922/0 (socket says 172.21.15.86:43922) 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 -- 172.21.15.86:0/932621804 learned_addr learned my addr 172.21.15.86:0/932621804 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b4be00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b54072650 0x7f0b5414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59200640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b5406e7e0 0x7f0b54144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 -- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b5406e7e0 msgr2=0x7f0b54144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b5406e7e0 0x7f0b54144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 -- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b54072650 msgr2=0x7f0b5414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b54072650 0x7f0b5414d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 -- 172.21.15.86:0/932621804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b541535a0 con 0x7f0b54072b00 2024-03-22T04:19:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b4be00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b54072650 0x7f0b5414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b59c00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5414d580 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c01a510 tx=0x7f0b4c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b51600640 1 -- 172.21.15.86:0/932621804 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0b2050 con 0x7f0b54072b00 2024-03-22T04:19:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b51600640 1 -- 172.21.15.86:0/932621804 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b4c0be070 con 0x7f0b54072b00 2024-03-22T04:19:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b54153880 con 0x7f0b54072b00 2024-03-22T04:19:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b51600640 1 -- 172.21.15.86:0/932621804 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0ada50 con 0x7f0b54072b00 2024-03-22T04:19:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b54153e40 con 0x7f0b54072b00 2024-03-22T04:19:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b54154040 con 0x7f0b54072b00 2024-03-22T04:19:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.101+0000 7f0b51600640 1 -- 172.21.15.86:0/932621804 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0b4c0adbf0 con 0x7f0b54072b00 2024-03-22T04:19:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.105+0000 7f0b51600640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b30067b50 0x7f0b3006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.105+0000 7f0b51600640 1 -- 172.21.15.86:0/932621804 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f0b4c120d10 con 0x7f0b54072b00 2024-03-22T04:19:00.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.105+0000 7f0b59200640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b30067b50 0x7f0b3006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.105+0000 7f0b59200640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b30067b50 0x7f0b3006a010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0b541f8c60 tx=0x7f0b44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.109+0000 7f0b51600640 1 -- 172.21.15.86:0/932621804 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0b54154040 con 0x7f0b54072b00 2024-03-22T04:19:00.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b540cf7b0 con 0x7f0b54072b00 2024-03-22T04:19:00.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b51600640 1 -- 172.21.15.86:0/932621804 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f0b4c126020 con 0x7f0b54072b00 2024-03-22T04:19:00.264 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:19:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b30067b50 msgr2=0x7f0b3006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b30067b50 0x7f0b3006a010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0b541f8c60 tx=0x7f0b44009040 comp rx=0 tx=0).stop 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 msgr2=0x7f0b5414d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5414d580 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c01a510 tx=0x7f0b4c019680 comp rx=0 tx=0).stop 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 shutdown_connections 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0b30067b50 0x7f0b3006a010 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b54072650 0x7f0b5414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b54072b00 0x7f0b5414d580 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 --2- 172.21.15.86:0/932621804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b5406e7e0 0x7f0b54144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 >> 172.21.15.86:0/932621804 conn(0x7f0b540cc940 msgr2=0x7f0b541517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 shutdown_connections 2024-03-22T04:19:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.265+0000 7f0b5bc00640 1 -- 172.21.15.86:0/932621804 wait complete. 2024-03-22T04:19:00.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:19:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6132e00640 1 Processor -- start 2024-03-22T04:19:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6132e00640 1 -- start start 2024-03-22T04:19:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6132e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f612c144a90 0x7f612c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6132e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c0b1b80 0x7f612c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6132e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f612c1620e0 0x7f612c162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6132e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f612c144e70 con 0x7f612c1620e0 2024-03-22T04:19:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6132e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f612c15fab0 con 0x7f612c144a90 2024-03-22T04:19:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6132e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f612c06d6d0 con 0x7f612c0b1b80 2024-03-22T04:19:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6130e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f612c1620e0 0x7f612c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6130e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f612c1620e0 0x7f612c162530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43938/0 (socket says 172.21.15.86:43938) 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6123400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c0b1b80 0x7f612c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6130e00640 1 -- 172.21.15.86:0/1632476127 learned_addr learned my addr 172.21.15.86:0/1632476127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f612be00640 1 --2- 172.21.15.86:0/1632476127 >> v2:172.21.15.196:3300/0 conn(0x7f612c144a90 0x7f612c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6130e00640 1 -- 172.21.15.86:0/1632476127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c0b1b80 msgr2=0x7f612c0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6130e00640 1 --2- 172.21.15.86:0/1632476127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c0b1b80 0x7f612c0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6130e00640 1 -- 172.21.15.86:0/1632476127 >> v2:172.21.15.196:3300/0 conn(0x7f612c144a90 msgr2=0x7f612c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6130e00640 1 --2- 172.21.15.86:0/1632476127 >> v2:172.21.15.196:3300/0 conn(0x7f612c144a90 0x7f612c15f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.425+0000 7f6130e00640 1 -- 172.21.15.86:0/1632476127 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c06d970 con 0x7f612c1620e0 2024-03-22T04:19:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6130e00640 1 --2- 172.21.15.86:0/1632476127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c1620e0 0x7f612c162530 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f612c06f830 tx=0x7f612401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb8da29db155be3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f612b400640 1 -- 172.21.15.86:0/1632476127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61240a5760 con 0x7f612c1620e0 2024-03-22T04:19:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f612b400640 1 -- 172.21.15.86:0/1632476127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61240a5900 con 0x7f612c1620e0 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f612b400640 1 -- 172.21.15.86:0/1632476127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61240a5cd0 con 0x7f612c1620e0 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 -- 172.21.15.86:0/1632476127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c1620e0 msgr2=0x7f612c162530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 --2- 172.21.15.86:0/1632476127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c1620e0 0x7f612c162530 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f612c06f830 tx=0x7f612401c9f0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 -- 172.21.15.86:0/1632476127 shutdown_connections 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 --2- 172.21.15.86:0/1632476127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c1620e0 0x7f612c162530 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 --2- 172.21.15.86:0/1632476127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c0b1b80 0x7f612c0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 --2- 172.21.15.86:0/1632476127 >> v2:172.21.15.196:3300/0 conn(0x7f612c144a90 0x7f612c15f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 -- 172.21.15.86:0/1632476127 >> 172.21.15.86:0/1632476127 conn(0x7f612c0cc940 msgr2=0x7f612c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 -- 172.21.15.86:0/1632476127 shutdown_connections 2024-03-22T04:19:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.429+0000 7f6132e00640 1 -- 172.21.15.86:0/1632476127 wait complete. 2024-03-22T04:19:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 Processor -- start 2024-03-22T04:19:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 -- start start 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c0b1b80 0x7f612c159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c144a90 0x7f612c157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612c1620e0 0x7f612c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f612c162e80 con 0x7f612c0b1b80 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f612c06e4f0 con 0x7f612c1620e0 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f612c0adbf0 con 0x7f612c144a90 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c0b1b80 0x7f612c159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c0b1b80 0x7f612c159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43942/0 (socket says 172.21.15.86:43942) 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 -- 172.21.15.86:0/728126818 learned_addr learned my addr 172.21.15.86:0/728126818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6123400640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c144a90 0x7f612c157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6130e00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612c1620e0 0x7f612c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 -- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c144a90 msgr2=0x7f612c157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c144a90 0x7f612c157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 -- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612c1620e0 msgr2=0x7f612c155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612c1620e0 0x7f612c155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 -- 172.21.15.86:0/728126818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c141e50 con 0x7f612c0b1b80 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6130e00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612c1620e0 0x7f612c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f612be00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c0b1b80 0x7f612c159ba0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e8a0 tx=0x7f611c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6128c00640 1 -- 172.21.15.86:0/728126818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c017070 con 0x7f612c0b1b80 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6128c00640 1 -- 172.21.15.86:0/728126818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c00c940 con 0x7f612c0b1b80 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6128c00640 1 -- 172.21.15.86:0/728126818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00ccf0 con 0x7f612c0b1b80 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c141890 con 0x7f612c0b1b80 2024-03-22T04:19:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c155670 con 0x7f612c0b1b80 2024-03-22T04:19:00.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.433+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c074470 con 0x7f612c0b1b80 2024-03-22T04:19:00.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.437+0000 7f6128c00640 1 -- 172.21.15.86:0/728126818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f611c01a070 con 0x7f612c0b1b80 2024-03-22T04:19:00.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.437+0000 7f6128c00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.437+0000 7f6128c00640 1 -- 172.21.15.86:0/728126818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f611c086330 con 0x7f612c0b1b80 2024-03-22T04:19:00.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.437+0000 7f6123400640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.437+0000 7f6123400640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f612c143cc0 tx=0x7f6114009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.441+0000 7f6128c00640 1 -- 172.21.15.86:0/728126818 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f611c056440 con 0x7f612c0b1b80 2024-03-22T04:19:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.593+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f612c161de0 con 0x7f612c0b1b80 2024-03-22T04:19:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6128c00640 1 -- 172.21.15.86:0/728126818 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f611c05a4e0 con 0x7f612c0b1b80 2024-03-22T04:19:00.595 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:19:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 msgr2=0x7f6108069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f612c143cc0 tx=0x7f6114009190 comp rx=0 tx=0).stop 2024-03-22T04:19:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c0b1b80 msgr2=0x7f612c159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c0b1b80 0x7f612c159ba0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e8a0 tx=0x7f611c00eca0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 shutdown_connections 2024-03-22T04:19:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6108067b20 0x7f6108069fe0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f612c1620e0 0x7f612c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f612c144a90 0x7f612c157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 --2- 172.21.15.86:0/728126818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f612c0b1b80 0x7f612c159ba0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 >> 172.21.15.86:0/728126818 conn(0x7f612c0cc940 msgr2=0x7f612c15b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 shutdown_connections 2024-03-22T04:19:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.597+0000 7f6132e00640 1 -- 172.21.15.86:0/728126818 wait complete. 2024-03-22T04:19:00.615 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:19:00.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.753+0000 7fde98e00640 1 Processor -- start 2024-03-22T04:19:00.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.753+0000 7fde98e00640 1 -- start start 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.753+0000 7fde98e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde940b2360 0x7fde940b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.753+0000 7fde98e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde9415d0b0 0x7fde940b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.753+0000 7fde98e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9415d710 con 0x7fde940b3430 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.753+0000 7fde98e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fde940b2740 con 0x7fde940b2360 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.753+0000 7fde98e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fde9415d4b0 con 0x7fde9415d0b0 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde940b2360 0x7fde940b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde940b2360 0x7fde940b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43958/0 (socket says 172.21.15.86:43958) 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91e00640 1 -- 172.21.15.86:0/3342371128 learned_addr learned my addr 172.21.15.86:0/3342371128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde92800640 1 --2- 172.21.15.86:0/3342371128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde9415d0b0 0x7fde940b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91e00640 1 -- 172.21.15.86:0/3342371128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde9415d0b0 msgr2=0x7fde940b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91e00640 1 --2- 172.21.15.86:0/3342371128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde9415d0b0 0x7fde940b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91e00640 1 -- 172.21.15.86:0/3342371128 >> v1:172.21.15.86:6789/0 conn(0x7fde940b3430 legacy=0x7fde9415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91e00640 1 -- 172.21.15.86:0/3342371128 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9415fe40 con 0x7fde940b2360 2024-03-22T04:19:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91e00640 1 --2- 172.21.15.86:0/3342371128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde940b2360 0x7fde940b2ef0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fde840098a0 tx=0x7fde8401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94b154c7b9d18de server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91400640 1 -- 172.21.15.86:0/3342371128 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840ab430 con 0x7fde940b2360 2024-03-22T04:19:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91400640 1 -- 172.21.15.86:0/3342371128 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde840b0070 con 0x7fde940b2360 2024-03-22T04:19:00.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.757+0000 7fde91400640 1 -- 172.21.15.86:0/3342371128 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840ab7f0 con 0x7fde940b2360 2024-03-22T04:19:00.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- 172.21.15.86:0/3342371128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde940b2360 msgr2=0x7fde940b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 --2- 172.21.15.86:0/3342371128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde940b2360 0x7fde940b2ef0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fde840098a0 tx=0x7fde8401c540 comp rx=0 tx=0).stop 2024-03-22T04:19:00.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- 172.21.15.86:0/3342371128 shutdown_connections 2024-03-22T04:19:00.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 --2- 172.21.15.86:0/3342371128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde9415d0b0 0x7fde940b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 --2- 172.21.15.86:0/3342371128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde940b2360 0x7fde940b2ef0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- 172.21.15.86:0/3342371128 >> 172.21.15.86:0/3342371128 conn(0x7fde940cc940 msgr2=0x7fde940ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:00.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- 172.21.15.86:0/3342371128 shutdown_connections 2024-03-22T04:19:00.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- 172.21.15.86:0/3342371128 wait complete. 2024-03-22T04:19:00.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 Processor -- start 2024-03-22T04:19:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- start start 2024-03-22T04:19:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde940b2360 0x7fde941526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde940b3430 0x7fde9415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415d0b0 0x7fde9415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde94160c00 con 0x7fde9415d0b0 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde940b5ad0 con 0x7fde940b3430 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde98e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fde940aef40 con 0x7fde940b2360 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415d0b0 0x7fde9415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415d0b0 0x7fde9415b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43974/0 (socket says 172.21.15.86:43974) 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 -- 172.21.15.86:0/3310099467 learned_addr learned my addr 172.21.15.86:0/3310099467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde89400640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde940b3430 0x7fde9415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde91e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde940b2360 0x7fde941526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 -- 172.21.15.86:0/3310099467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde940b2360 msgr2=0x7fde941526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde940b2360 0x7fde941526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 -- 172.21.15.86:0/3310099467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde940b3430 msgr2=0x7fde9415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde940b3430 0x7fde9415b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 -- 172.21.15.86:0/3310099467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde94072130 con 0x7fde9415d0b0 2024-03-22T04:19:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.761+0000 7fde92800640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415d0b0 0x7fde9415b5e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009d30 tx=0x7fde8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde8aa00640 1 -- 172.21.15.86:0/3310099467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c018020 con 0x7fde9415d0b0 2024-03-22T04:19:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde8aa00640 1 -- 172.21.15.86:0/3310099467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c014480 con 0x7fde9415d0b0 2024-03-22T04:19:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde8aa00640 1 -- 172.21.15.86:0/3310099467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c014870 con 0x7fde9415d0b0 2024-03-22T04:19:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde94072410 con 0x7fde9415d0b0 2024-03-22T04:19:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde94071d70 con 0x7fde9415d0b0 2024-03-22T04:19:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde94074470 con 0x7fde9415d0b0 2024-03-22T04:19:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde8aa00640 1 -- 172.21.15.86:0/3310099467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fde8c01b020 con 0x7fde9415d0b0 2024-03-22T04:19:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde8aa00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde70067b50 0x7fde7006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde8aa00640 1 -- 172.21.15.86:0/3310099467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fde8c086ce0 con 0x7fde9415d0b0 2024-03-22T04:19:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.765+0000 7fde91e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde70067b50 0x7fde7006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.769+0000 7fde91e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde70067b50 0x7fde7006a010 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fde840af000 tx=0x7fde84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.769+0000 7fde8aa00640 1 -- 172.21.15.86:0/3310099467 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fde8c022030 con 0x7fde9415d0b0 2024-03-22T04:19:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.925+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde9406e9f0 con 0x7fde9415d0b0 2024-03-22T04:19:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.925+0000 7fde8aa00640 1 -- 172.21.15.86:0/3310099467 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fde8c010040 con 0x7fde9415d0b0 2024-03-22T04:19:00.926 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde70067b50 msgr2=0x7fde7006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde70067b50 0x7fde7006a010 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fde840af000 tx=0x7fde84008040 comp rx=0 tx=0).stop 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415d0b0 msgr2=0x7fde9415b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415d0b0 0x7fde9415b5e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009d30 tx=0x7fde8c00c490 comp rx=0 tx=0).stop 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 shutdown_connections 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fde70067b50 0x7fde7006a010 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde940b3430 0x7fde9415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415d0b0 0x7fde9415b5e0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 --2- 172.21.15.86:0/3310099467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde940b2360 0x7fde941526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 >> 172.21.15.86:0/3310099467 conn(0x7fde940cc940 msgr2=0x7fde941562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 shutdown_connections 2024-03-22T04:19:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:00.929+0000 7fde98e00640 1 -- 172.21.15.86:0/3310099467 wait complete. 2024-03-22T04:19:00.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:19:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3417200640 1 Processor -- start 2024-03-22T04:19:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3417200640 1 -- start start 2024-03-22T04:19:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3417200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f341815f860 0x7f34180ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3417200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3418073cf0 con 0x7f34180cdb00 2024-03-22T04:19:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3417200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34180ce280 con 0x7f341809b180 2024-03-22T04:19:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3417200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f34180ce560 con 0x7f341815f860 2024-03-22T04:19:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f340d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f34180cdb00 0x7f341809aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59596/0 (socket says 172.21.15.86:59596) 2024-03-22T04:19:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f340d400640 1 -- 172.21.15.86:0/416014605 learned_addr learned my addr 172.21.15.86:0/416014605 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:01.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415e00640 1 --2- 172.21.15.86:0/416014605 >> v2:172.21.15.196:3300/0 conn(0x7f341815f860 0x7f34180ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415400640 1 -- 172.21.15.86:0/416014605 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231531031 0 0) 0x7f3418073cf0 con 0x7f34180cdb00 2024-03-22T04:19:01.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415400640 1 -- 172.21.15.86:0/416014605 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33fc003530 con 0x7f34180cdb00 2024-03-22T04:19:01.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415400640 1 -- 172.21.15.86:0/416014605 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440515960 0 0) 0x7f34180ce280 con 0x7f341809b180 2024-03-22T04:19:01.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415400640 1 -- 172.21.15.86:0/416014605 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3418073cf0 con 0x7f341809b180 2024-03-22T04:19:01.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415400640 1 -- 172.21.15.86:0/416014605 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3665387961 0 0) 0x7f33fc003530 con 0x7f34180cdb00 2024-03-22T04:19:01.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415e00640 1 -- 172.21.15.86:0/416014605 >> v1:172.21.15.196:6789/0 conn(0x7f341809b180 legacy=0x7f34180b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415e00640 1 -- 172.21.15.86:0/416014605 >> v1:172.21.15.86:6789/0 conn(0x7f34180cdb00 legacy=0x7f341809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.089+0000 7f3415e00640 1 -- 172.21.15.86:0/416014605 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34180ce840 con 0x7f341815f860 2024-03-22T04:19:01.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3415e00640 1 --2- 172.21.15.86:0/416014605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 0x7f34180ccb10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34080097c0 tx=0x7f340801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b390b88ef6494bb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3415400640 1 -- 172.21.15.86:0/416014605 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34080b1070 con 0x7f341815f860 2024-03-22T04:19:01.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3415400640 1 -- 172.21.15.86:0/416014605 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34080ac440 con 0x7f341815f860 2024-03-22T04:19:01.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3415400640 1 -- 172.21.15.86:0/416014605 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400001680 con 0x7f341815f860 2024-03-22T04:19:01.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 -- 172.21.15.86:0/416014605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 msgr2=0x7f34180ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 --2- 172.21.15.86:0/416014605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 0x7f34180ccb10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34080097c0 tx=0x7f340801c390 comp rx=0 tx=0).stop 2024-03-22T04:19:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 -- 172.21.15.86:0/416014605 shutdown_connections 2024-03-22T04:19:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 --2- 172.21.15.86:0/416014605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 0x7f34180ccb10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 -- 172.21.15.86:0/416014605 >> 172.21.15.86:0/416014605 conn(0x7f34180b3d60 msgr2=0x7f34180b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 -- 172.21.15.86:0/416014605 shutdown_connections 2024-03-22T04:19:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 -- 172.21.15.86:0/416014605 wait complete. 2024-03-22T04:19:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 Processor -- start 2024-03-22T04:19:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.093+0000 7f3417200640 1 -- start start 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341809b180 0x7f34180c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34180cdb00 0x7f34180c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 0x7f34180cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f341815fdc0 con 0x7f341809b180 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f341815fc40 con 0x7f341815f860 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34180b30f0 con 0x7f34180cdb00 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341809b180 0x7f34180c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341809b180 0x7f34180c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43976/0 (socket says 172.21.15.86:43976) 2024-03-22T04:19:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 -- 172.21.15.86:0/151161316 learned_addr learned my addr 172.21.15.86:0/151161316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f340d400640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34180cdb00 0x7f34180c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3416800640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 0x7f34180cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 -- 172.21.15.86:0/151161316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34180cdb00 msgr2=0x7f34180c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34180cdb00 0x7f34180c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 -- 172.21.15.86:0/151161316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 msgr2=0x7f34180cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 0x7f34180cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 -- 172.21.15.86:0/151161316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34180b58c0 con 0x7f341809b180 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3415e00640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341809b180 0x7f34180c9340 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f34080097c0 tx=0x7f3408019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f340ea00640 1 -- 172.21.15.86:0/151161316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34080b9070 con 0x7f341809b180 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f340ea00640 1 -- 172.21.15.86:0/151161316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408004020 con 0x7f341809b180 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34180b5ba0 con 0x7f341809b180 2024-03-22T04:19:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f340ea00640 1 -- 172.21.15.86:0/151161316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34080b1040 con 0x7f341809b180 2024-03-22T04:19:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34180cb830 con 0x7f341809b180 2024-03-22T04:19:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.097+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3418074470 con 0x7f341809b180 2024-03-22T04:19:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.101+0000 7f340ea00640 1 -- 172.21.15.86:0/151161316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3408003620 con 0x7f341809b180 2024-03-22T04:19:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.101+0000 7f340ea00640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f33f4067b50 0x7f33f406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.101+0000 7f340ea00640 1 -- 172.21.15.86:0/151161316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f340811fe40 con 0x7f341809b180 2024-03-22T04:19:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.101+0000 7f340d400640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f33f4067b50 0x7f33f406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.101+0000 7f340d400640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f33f4067b50 0x7f33f406a010 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f34181faa10 tx=0x7f3400009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.105+0000 7f340ea00640 1 -- 172.21.15.86:0/151161316 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3408015fb0 con 0x7f341809b180 2024-03-22T04:19:01.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.269+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34180d0960 con 0x7f341809b180 2024-03-22T04:19:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.269+0000 7f340ea00640 1 -- 172.21.15.86:0/151161316 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v97) ==== 80+0+248 (secure 0 0 0) 0x7f34080f0500 con 0x7f341809b180 2024-03-22T04:19:01.270 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.196:6835/1851339627 2024-03-23T04:18:21.162930+0000 2024-03-22T04:19:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.196:6834/1851339627 2024-03-23T04:18:21.162930+0000 2024-03-22T04:19:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.196:6835/1097103199 2024-03-23T04:17:08.939658+0000 2024-03-22T04:19:01.271 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.196:6834/1097103199 2024-03-23T04:17:08.939658+0000 2024-03-22T04:19:01.272 INFO:teuthology.orchestra.run.smithi086.stderr:listed 4 entries 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f33f4067b50 msgr2=0x7f33f406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f33f4067b50 0x7f33f406a010 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f34181faa10 tx=0x7f3400009040 comp rx=0 tx=0).stop 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341809b180 msgr2=0x7f34180c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341809b180 0x7f34180c9340 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f34080097c0 tx=0x7f3408019130 comp rx=0 tx=0).stop 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 shutdown_connections 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f33f4067b50 0x7f33f406a010 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f341815f860 0x7f34180cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34180cdb00 0x7f34180c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 --2- 172.21.15.86:0/151161316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f341809b180 0x7f34180c9340 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 >> 172.21.15.86:0/151161316 conn(0x7f34180b3d60 msgr2=0x7f341815a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:01.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 shutdown_connections 2024-03-22T04:19:01.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.273+0000 7f3417200640 1 -- 172.21.15.86:0/151161316 wait complete. 2024-03-22T04:19:01.292 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:19:01.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:01.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:01.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:19:01.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:01.318 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:19:01.367 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:19:01.367 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:19:01.367 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:01.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:19:01.376 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:19:01.376 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:01.376 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:01.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:19:01.418 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:19:01.418 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:19:01.418 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:01.426 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:19:01.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7f6e00640 1 Processor -- start 2024-03-22T04:19:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7f6e00640 1 -- start start 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7f6e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd7f00aff50 0x7fd7f006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7f6e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7f006cbb0 0x7fd7f0143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7f6e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f00ae5a0 con 0x7fd7f0144320 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7f6e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f006d010 con 0x7fd7f006cbb0 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7f6e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f0162aa0 con 0x7fd7f00aff50 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7ef400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7f006cbb0 0x7fd7f0143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7ef400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7f006cbb0 0x7fd7f0143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43984/0 (socket says 172.21.15.86:43984) 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.597+0000 7fd7ef400640 1 -- 172.21.15.86:0/3873376377 learned_addr learned my addr 172.21.15.86:0/3873376377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7efe00640 1 --2- 172.21.15.86:0/3873376377 >> v2:172.21.15.196:3300/0 conn(0x7fd7f00aff50 0x7fd7f006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7ef400640 1 -- 172.21.15.86:0/3873376377 >> v2:172.21.15.196:3300/0 conn(0x7fd7f00aff50 msgr2=0x7fd7f006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7ef400640 1 --2- 172.21.15.86:0/3873376377 >> v2:172.21.15.196:3300/0 conn(0x7fd7f00aff50 0x7fd7f006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7ef400640 1 -- 172.21.15.86:0/3873376377 >> v1:172.21.15.196:6789/0 conn(0x7fd7f0144320 legacy=0x7fd7f006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7ef400640 1 -- 172.21.15.86:0/3873376377 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f0144b80 con 0x7fd7f006cbb0 2024-03-22T04:19:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7ef400640 1 --2- 172.21.15.86:0/3873376377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f006cbb0 0x7fd7f0143de0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0073280 tx=0x7fd7dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f0a12f480daf83 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7eea00640 1 -- 172.21.15.86:0/3873376377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0b0070 con 0x7fd7f006cbb0 2024-03-22T04:19:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7eea00640 1 -- 172.21.15.86:0/3873376377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7dc0ab600 con 0x7fd7f006cbb0 2024-03-22T04:19:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7eea00640 1 -- 172.21.15.86:0/3873376377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0ab9d0 con 0x7fd7f006cbb0 2024-03-22T04:19:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7f6e00640 1 -- 172.21.15.86:0/3873376377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f006cbb0 msgr2=0x7fd7f0143de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.601+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/3873376377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f006cbb0 0x7fd7f0143de0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0073280 tx=0x7fd7dc01c490 comp rx=0 tx=0).stop 2024-03-22T04:19:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- 172.21.15.86:0/3873376377 shutdown_connections 2024-03-22T04:19:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/3873376377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f006cbb0 0x7fd7f0143de0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/3873376377 >> v2:172.21.15.196:3300/0 conn(0x7fd7f00aff50 0x7fd7f006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- 172.21.15.86:0/3873376377 >> 172.21.15.86:0/3873376377 conn(0x7fd7f0071fe0 msgr2=0x7fd7f00723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- 172.21.15.86:0/3873376377 shutdown_connections 2024-03-22T04:19:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- 172.21.15.86:0/3873376377 wait complete. 2024-03-22T04:19:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 Processor -- start 2024-03-22T04:19:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- start start 2024-03-22T04:19:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7f006cbb0 0x7fd7f01566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f00aff50 0x7fd7f00ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f0144320 0x7fd7f00aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0163090 con 0x7fd7f0144320 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0162f10 con 0x7fd7f00aff50 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f00b0330 con 0x7fd7f006cbb0 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f0144320 0x7fd7f00aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f0144320 0x7fd7f00aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43996/0 (socket says 172.21.15.86:43996) 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 -- 172.21.15.86:0/2225820072 learned_addr learned my addr 172.21.15.86:0/2225820072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7ef400640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f00aff50 0x7fd7f00ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7efe00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7f006cbb0 0x7fd7f01566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 -- 172.21.15.86:0/2225820072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7f006cbb0 msgr2=0x7fd7f01566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7f006cbb0 0x7fd7f01566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 -- 172.21.15.86:0/2225820072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f00aff50 msgr2=0x7fd7f00ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f00aff50 0x7fd7f00ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 -- 172.21.15.86:0/2225820072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f00b6a00 con 0x7fd7f0144320 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f4e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f0144320 0x7fd7f00aecc0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800c340 tx=0x7fd7e800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7e7e00640 1 -- 172.21.15.86:0/2225820072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8005220 con 0x7fd7f0144320 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7e7e00640 1 -- 172.21.15.86:0/2225820072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e80053c0 con 0x7fd7f0144320 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f01405c0 con 0x7fd7f0144320 2024-03-22T04:19:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.605+0000 7fd7e7e00640 1 -- 172.21.15.86:0/2225820072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8007270 con 0x7fd7f0144320 2024-03-22T04:19:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.609+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f0140b80 con 0x7fd7f0144320 2024-03-22T04:19:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.609+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f0068650 con 0x7fd7f0144320 2024-03-22T04:19:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.609+0000 7fd7e7e00640 1 -- 172.21.15.86:0/2225820072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd7e8019020 con 0x7fd7f0144320 2024-03-22T04:19:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.609+0000 7fd7e7e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd7d0067b20 0x7fd7d0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.609+0000 7fd7e7e00640 1 -- 172.21.15.86:0/2225820072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fd7e8085350 con 0x7fd7f0144320 2024-03-22T04:19:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.609+0000 7fd7efe00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd7d0067b20 0x7fd7d0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.613+0000 7fd7efe00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd7d0067b20 0x7fd7d0069fe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0003b90 tx=0x7fd7e00098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.613+0000 7fd7e7e00640 1 -- 172.21.15.86:0/2225820072 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd7f00ae5a0 con 0x7fd7f0144320 2024-03-22T04:19:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.801+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f006d1a0 con 0x7fd7f0144320 2024-03-22T04:19:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.801+0000 7fd7e7e00640 1 -- 172.21.15.86:0/2225820072 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5186 (secure 0 0 0) 0x7fd7e8055460 con 0x7fd7f0144320 2024-03-22T04:19:01.801 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:01.802 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":77,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:18:25.189133+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76850":{"gid":76850,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.196:6835/1227183960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1227183960},{"type":"v1","addr":"172.21.15.196:6835","nonce":1227183960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76850,"qdb_cluster":[76850]},"id":6}]} 2024-03-22T04:19:01.802 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 77 2024-03-22T04:19:01.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd7d0067b20 msgr2=0x7fd7d0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd7d0067b20 0x7fd7d0069fe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0003b90 tx=0x7fd7e00098d0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f0144320 msgr2=0x7fd7f00aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f0144320 0x7fd7f00aecc0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800c340 tx=0x7fd7e800c800 comp rx=0 tx=0).stop 2024-03-22T04:19:01.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 shutdown_connections 2024-03-22T04:19:01.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd7d0067b20 0x7fd7d0069fe0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7f00aff50 0x7fd7f00ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7f0144320 0x7fd7f00aecc0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 --2- 172.21.15.86:0/2225820072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7f006cbb0 0x7fd7f01566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 >> 172.21.15.86:0/2225820072 conn(0x7fd7f0071fe0 msgr2=0x7fd7f01604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:01.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 shutdown_connections 2024-03-22T04:19:01.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.805+0000 7fd7f6e00640 1 -- 172.21.15.86:0/2225820072 wait complete. 2024-03-22T04:19:01.822 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:01.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.957+0000 7fcb01c00640 1 Processor -- start 2024-03-22T04:19:01.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.957+0000 7fcb01c00640 1 -- start start 2024-03-22T04:19:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcb01c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcafc0730d0 0x7fcafc0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0cc3e0 0x7fcafc0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcb01c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcafc071920 con 0x7fcafc15b3f0 2024-03-22T04:19:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcb01c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcafc0716d0 con 0x7fcafc0730d0 2024-03-22T04:19:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcb01c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcafc06d990 con 0x7fcafc0cc3e0 2024-03-22T04:19:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcafaa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcafc15b3f0 0x7fcafc0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59612/0 (socket says 172.21.15.86:59612) 2024-03-22T04:19:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcafaa00640 1 -- 172.21.15.86:0/250794722 learned_addr learned my addr 172.21.15.86:0/250794722 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcaf2000640 1 --2- 172.21.15.86:0/250794722 >> v2:172.21.15.86:3300/0 conn(0x7fcafc0730d0 0x7fcafc0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcafb400640 1 --2- 172.21.15.86:0/250794722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0cc3e0 0x7fcafc0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcafa000640 1 -- 172.21.15.86:0/250794722 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446237754 0 0) 0x7fcafc071920 con 0x7fcafc15b3f0 2024-03-22T04:19:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcafa000640 1 -- 172.21.15.86:0/250794722 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcae0003530 con 0x7fcafc15b3f0 2024-03-22T04:19:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcaf2000640 1 -- 172.21.15.86:0/250794722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0cc3e0 msgr2=0x7fcafc0ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcaf2000640 1 --2- 172.21.15.86:0/250794722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0cc3e0 0x7fcafc0ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcaf2000640 1 -- 172.21.15.86:0/250794722 >> v1:172.21.15.86:6789/0 conn(0x7fcafc15b3f0 legacy=0x7fcafc0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.961+0000 7fcaf2000640 1 -- 172.21.15.86:0/250794722 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc071c00 con 0x7fcafc0730d0 2024-03-22T04:19:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcaf2000640 1 --2- 172.21.15.86:0/250794722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc0730d0 0x7fcafc0ade30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fcae40097c0 tx=0x7fcae401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d09e43ef4ce076 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcafa000640 1 -- 172.21.15.86:0/250794722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40b2070 con 0x7fcafc0730d0 2024-03-22T04:19:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcafa000640 1 -- 172.21.15.86:0/250794722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae40ad640 con 0x7fcafc0730d0 2024-03-22T04:19:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcafa000640 1 -- 172.21.15.86:0/250794722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40ada10 con 0x7fcafc0730d0 2024-03-22T04:19:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 -- 172.21.15.86:0/250794722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc0730d0 msgr2=0x7fcafc0ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 --2- 172.21.15.86:0/250794722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc0730d0 0x7fcafc0ade30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fcae40097c0 tx=0x7fcae401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 -- 172.21.15.86:0/250794722 shutdown_connections 2024-03-22T04:19:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 --2- 172.21.15.86:0/250794722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0cc3e0 0x7fcafc0ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 --2- 172.21.15.86:0/250794722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc0730d0 0x7fcafc0ade30 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 -- 172.21.15.86:0/250794722 >> 172.21.15.86:0/250794722 conn(0x7fcafc0b1df0 msgr2=0x7fcafc0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:01.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 -- 172.21.15.86:0/250794722 shutdown_connections 2024-03-22T04:19:01.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 -- 172.21.15.86:0/250794722 wait complete. 2024-03-22T04:19:01.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 Processor -- start 2024-03-22T04:19:01.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 -- start start 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0730d0 0x7fcafc15e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.965+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcafc0cc3e0 0x7fcafc14eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc15b3f0 0x7fcafc1646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcb01c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc168580 con 0x7fcafc15b3f0 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcb01c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc168400 con 0x7fcafc0cc3e0 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcb01c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcafc168700 con 0x7fcafc0730d0 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc15b3f0 0x7fcafc1646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc15b3f0 0x7fcafc1646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44022/0 (socket says 172.21.15.86:44022) 2024-03-22T04:19:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 -- 172.21.15.86:0/1978210163 learned_addr learned my addr 172.21.15.86:0/1978210163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:01.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafaa00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0730d0 0x7fcafc15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcaf2000640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcafc0cc3e0 0x7fcafc14eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 -- 172.21.15.86:0/1978210163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0730d0 msgr2=0x7fcafc15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0730d0 0x7fcafc15e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 -- 172.21.15.86:0/1978210163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcafc0cc3e0 msgr2=0x7fcafc14eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcafc0cc3e0 0x7fcafc14eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 -- 172.21.15.86:0/1978210163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc15d660 con 0x7fcafc15b3f0 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcafb400640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc15b3f0 0x7fcafc1646f0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400e930 tx=0x7fcaf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcaf3400640 1 -- 172.21.15.86:0/1978210163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4017070 con 0x7fcafc15b3f0 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcaf3400640 1 -- 172.21.15.86:0/1978210163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf400c9a0 con 0x7fcafc15b3f0 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc15d940 con 0x7fcafc15b3f0 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcaf3400640 1 -- 172.21.15.86:0/1978210163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400cd70 con 0x7fcafc15b3f0 2024-03-22T04:19:01.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc15df00 con 0x7fcafc15b3f0 2024-03-22T04:19:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc068650 con 0x7fcafc15b3f0 2024-03-22T04:19:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.969+0000 7fcaf3400640 1 -- 172.21.15.86:0/1978210163 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcaf401a070 con 0x7fcafc15b3f0 2024-03-22T04:19:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.973+0000 7fcaf3400640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcad8067b20 0x7fcad8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.973+0000 7fcaf3400640 1 -- 172.21.15.86:0/1978210163 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fcaf4086c60 con 0x7fcafc15b3f0 2024-03-22T04:19:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.973+0000 7fcafaa00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcad8067b20 0x7fcad8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:01.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.973+0000 7fcafaa00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcad8067b20 0x7fcad8069fe0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcaec005c10 tx=0x7fcaec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:01.977+0000 7fcaf3400640 1 -- 172.21.15.86:0/1978210163 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcaf40566b0 con 0x7fcafc15b3f0 2024-03-22T04:19:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.133+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcafc154c80 con 0x7fcafc15b3f0 2024-03-22T04:19:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.133+0000 7fcaf3400640 1 -- 172.21.15.86:0/1978210163 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17629 (secure 0 0 0) 0x7fcaf405a750 con 0x7fcafc15b3f0 2024-03-22T04:19:02.135 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:02.135 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":97,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:18:21.162952+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.32","mappings":[{"from":7,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":3}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]},{"pgid":"14.3c","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3e","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6834/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6835/1097103199":"2024-03-23T04:17:08.939658+0000","172.21.15.196:6834/1097103199":"2024-03-23T04:17:08.939658+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcad8067b20 msgr2=0x7fcad8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcad8067b20 0x7fcad8069fe0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcaec005c10 tx=0x7fcaec009040 comp rx=0 tx=0).stop 2024-03-22T04:19:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc15b3f0 msgr2=0x7fcafc1646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc15b3f0 0x7fcafc1646f0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400e930 tx=0x7fcaf400edf0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 shutdown_connections 2024-03-22T04:19:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcad8067b20 0x7fcad8069fe0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcafc0cc3e0 0x7fcafc14eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcafc15b3f0 0x7fcafc1646f0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 --2- 172.21.15.86:0/1978210163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcafc0730d0 0x7fcafc15e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 >> 172.21.15.86:0/1978210163 conn(0x7fcafc0b1df0 msgr2=0x7fcafc155480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 shutdown_connections 2024-03-22T04:19:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.137+0000 7fcb01c00640 1 -- 172.21.15.86:0/1978210163 wait complete. 2024-03-22T04:19:02.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:16:56.341603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:16:56.977675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:19:02.159 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:19:02.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.297+0000 7fc9e8000640 1 Processor -- start 2024-03-22T04:19:02.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.297+0000 7fc9e8000640 1 -- start start 2024-03-22T04:19:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.297+0000 7fc9e8000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0072b70 0x7fc9e0160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.297+0000 7fc9e8000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e0145f70 con 0x7fc9e0145b90 2024-03-22T04:19:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.297+0000 7fc9e8000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e0160850 con 0x7fc9e015fd90 2024-03-22T04:19:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.297+0000 7fc9e8000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e0160b30 con 0x7fc9e0072b70 2024-03-22T04:19:02.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.297+0000 7fc9e5600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc9e0145b90 0x7fc9e00d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59616/0 (socket says 172.21.15.86:59616) 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.297+0000 7fc9e5600640 1 -- 172.21.15.86:0/2282027440 learned_addr learned my addr 172.21.15.86:0/2282027440 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9e6000640 1 --2- 172.21.15.86:0/2282027440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0072b70 0x7fc9e0160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9dfe00640 1 -- 172.21.15.86:0/2282027440 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1558627388 0 0) 0x7fc9e0145f70 con 0x7fc9e0145b90 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9dfe00640 1 -- 172.21.15.86:0/2282027440 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c8003530 con 0x7fc9e0145b90 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9dfe00640 1 -- 172.21.15.86:0/2282027440 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296584606 0 0) 0x7fc9e0160850 con 0x7fc9e015fd90 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9dfe00640 1 -- 172.21.15.86:0/2282027440 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9e0145f70 con 0x7fc9e015fd90 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9e6000640 1 -- 172.21.15.86:0/2282027440 >> v1:172.21.15.196:6789/0 conn(0x7fc9e015fd90 legacy=0x7fc9e0072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9e6000640 1 -- 172.21.15.86:0/2282027440 >> v1:172.21.15.86:6789/0 conn(0x7fc9e0145b90 legacy=0x7fc9e00d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9e6000640 1 -- 172.21.15.86:0/2282027440 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00cfd60 con 0x7fc9e0072b70 2024-03-22T04:19:02.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9e6000640 1 --2- 172.21.15.86:0/2282027440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0072b70 0x7fc9e0160220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009dc0 tx=0x7fc9d801c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a440f1f4b6b9554 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9dfe00640 1 -- 172.21.15.86:0/2282027440 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80b2070 con 0x7fc9e0072b70 2024-03-22T04:19:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9dfe00640 1 -- 172.21.15.86:0/2282027440 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d80a59b0 con 0x7fc9e0072b70 2024-03-22T04:19:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.301+0000 7fc9dfe00640 1 -- 172.21.15.86:0/2282027440 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80a5d80 con 0x7fc9e0072b70 2024-03-22T04:19:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- 172.21.15.86:0/2282027440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0072b70 msgr2=0x7fc9e0160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 --2- 172.21.15.86:0/2282027440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0072b70 0x7fc9e0160220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009dc0 tx=0x7fc9d801c8d0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- 172.21.15.86:0/2282027440 shutdown_connections 2024-03-22T04:19:02.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 --2- 172.21.15.86:0/2282027440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0072b70 0x7fc9e0160220 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- 172.21.15.86:0/2282027440 >> 172.21.15.86:0/2282027440 conn(0x7fc9e00d0fa0 msgr2=0x7fc9e00d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:02.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- 172.21.15.86:0/2282027440 shutdown_connections 2024-03-22T04:19:02.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- 172.21.15.86:0/2282027440 wait complete. 2024-03-22T04:19:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 Processor -- start 2024-03-22T04:19:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- start start 2024-03-22T04:19:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9e0072b70 0x7fc9e00b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0145b90 0x7fc9e015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9e015fd90 0x7fc9e015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e016a140 con 0x7fc9e0072b70 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00d0c70 con 0x7fc9e015fd90 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e8000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e016a2c0 con 0x7fc9e0145b90 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9e0072b70 0x7fc9e00b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9e0072b70 0x7fc9e00b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44032/0 (socket says 172.21.15.86:44032) 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e5600640 1 -- 172.21.15.86:0/101200481 learned_addr learned my addr 172.21.15.86:0/101200481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e4c00640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0145b90 0x7fc9e015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e6000640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9e015fd90 0x7fc9e015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e5600640 1 -- 172.21.15.86:0/101200481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0145b90 msgr2=0x7fc9e015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e5600640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0145b90 0x7fc9e015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e5600640 1 -- 172.21.15.86:0/101200481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9e015fd90 msgr2=0x7fc9e015b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e5600640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9e015fd90 0x7fc9e015b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.305+0000 7fc9e5600640 1 -- 172.21.15.86:0/101200481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e015f780 con 0x7fc9e0072b70 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9e5600640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9e0072b70 0x7fc9e00b5a60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c4d0 tx=0x7fc9d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9dd600640 1 -- 172.21.15.86:0/101200481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4017070 con 0x7fc9e0072b70 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9dd600640 1 -- 172.21.15.86:0/101200481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4005cf0 con 0x7fc9e0072b70 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9dd600640 1 -- 172.21.15.86:0/101200481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40072b0 con 0x7fc9e0072b70 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e00b4ca0 con 0x7fc9e0072b70 2024-03-22T04:19:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e00b5260 con 0x7fc9e0072b70 2024-03-22T04:19:02.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e0074470 con 0x7fc9e0072b70 2024-03-22T04:19:02.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9dd600640 1 -- 172.21.15.86:0/101200481 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc9d401a070 con 0x7fc9e0072b70 2024-03-22T04:19:02.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9dd600640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9c0067b20 0x7fc9c0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9dd600640 1 -- 172.21.15.86:0/101200481 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fc9d4085520 con 0x7fc9e0072b70 2024-03-22T04:19:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.309+0000 7fc9e4c00640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9c0067b20 0x7fc9c0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.313+0000 7fc9e4c00640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9c0067b20 0x7fc9c0069fe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0068980 tx=0x7fc9cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.313+0000 7fc9dd600640 1 -- 172.21.15.86:0/101200481 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc9d40555a0 con 0x7fc9e0072b70 2024-03-22T04:19:02.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.469+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc9e006e5e0 con 0x7fc9e0072b70 2024-03-22T04:19:02.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.469+0000 7fc9dd600640 1 -- 172.21.15.86:0/101200481 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fc9d4059640 con 0x7fc9e0072b70 2024-03-22T04:19:02.470 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:19:02.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9c0067b20 msgr2=0x7fc9c0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9c0067b20 0x7fc9c0069fe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0068980 tx=0x7fc9cc009040 comp rx=0 tx=0).stop 2024-03-22T04:19:02.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9e0072b70 msgr2=0x7fc9e00b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9e0072b70 0x7fc9e00b5a60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c4d0 tx=0x7fc9d400c990 comp rx=0 tx=0).stop 2024-03-22T04:19:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 shutdown_connections 2024-03-22T04:19:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9c0067b20 0x7fc9c0069fe0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9e015fd90 0x7fc9e015b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9e0145b90 0x7fc9e015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 --2- 172.21.15.86:0/101200481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9e0072b70 0x7fc9e00b5a60 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 >> 172.21.15.86:0/101200481 conn(0x7fc9e00d0fa0 msgr2=0x7fc9e0144c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 shutdown_connections 2024-03-22T04:19:02.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.473+0000 7fc9e8000640 1 -- 172.21.15.86:0/101200481 wait complete. 2024-03-22T04:19:02.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:19:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.629+0000 7f8457200640 1 Processor -- start 2024-03-22T04:19:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.629+0000 7f8457200640 1 -- start start 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f8457200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84580cd2a0 0x7f84580cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f8457200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f845806fdc0 0x7f8458070210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f8457200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f845815c760 con 0x7f845815c380 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f8457200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f845806f930 con 0x7f84580cd2a0 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f8457200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f84580d0940 con 0x7f845806fdc0 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f844d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84580cd2a0 0x7f84580cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f844d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84580cd2a0 0x7f84580cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44044/0 (socket says 172.21.15.86:44044) 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f8455e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f845815c380 0x7f84580ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59620/0 (socket says 172.21.15.86:59620) 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f844d400640 1 -- 172.21.15.86:0/423770514 learned_addr learned my addr 172.21.15.86:0/423770514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f8456800640 1 --2- 172.21.15.86:0/423770514 >> v2:172.21.15.196:3300/0 conn(0x7f845806fdc0 0x7f8458070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f844d400640 1 -- 172.21.15.86:0/423770514 >> v2:172.21.15.196:3300/0 conn(0x7f845806fdc0 msgr2=0x7f8458070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f844d400640 1 --2- 172.21.15.86:0/423770514 >> v2:172.21.15.196:3300/0 conn(0x7f845806fdc0 0x7f8458070210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f844d400640 1 -- 172.21.15.86:0/423770514 >> v1:172.21.15.86:6789/0 conn(0x7f845815c380 legacy=0x7f84580ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.633+0000 7f844d400640 1 -- 172.21.15.86:0/423770514 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84580d0c20 con 0x7f84580cd2a0 2024-03-22T04:19:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f844d400640 1 --2- 172.21.15.86:0/423770514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f84580cd750 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f8458072e70 tx=0x7f844001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c7971a2204a79e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8455400640 1 -- 172.21.15.86:0/423770514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84400ab430 con 0x7f84580cd2a0 2024-03-22T04:19:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8455400640 1 -- 172.21.15.86:0/423770514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84400b0070 con 0x7f84580cd2a0 2024-03-22T04:19:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8455400640 1 -- 172.21.15.86:0/423770514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84400ab7f0 con 0x7f84580cd2a0 2024-03-22T04:19:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- 172.21.15.86:0/423770514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 msgr2=0x7f84580cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 --2- 172.21.15.86:0/423770514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f84580cd750 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f8458072e70 tx=0x7f844001c520 comp rx=0 tx=0).stop 2024-03-22T04:19:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- 172.21.15.86:0/423770514 shutdown_connections 2024-03-22T04:19:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 --2- 172.21.15.86:0/423770514 >> v2:172.21.15.196:3300/0 conn(0x7f845806fdc0 0x7f8458070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 --2- 172.21.15.86:0/423770514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f84580cd750 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- 172.21.15.86:0/423770514 >> 172.21.15.86:0/423770514 conn(0x7f84580d1f80 msgr2=0x7f84580d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- 172.21.15.86:0/423770514 shutdown_connections 2024-03-22T04:19:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- 172.21.15.86:0/423770514 wait complete. 2024-03-22T04:19:02.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 Processor -- start 2024-03-22T04:19:02.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- start start 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845806fdc0 0x7f8458148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f845806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815c380 0x7f845806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84580d1080 con 0x7f84580cd2a0 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84580d0f00 con 0x7f845806fdc0 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8457200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8458168eb0 con 0x7f845815c380 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f844d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f845806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f844d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f845806c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44052/0 (socket says 172.21.15.86:44052) 2024-03-22T04:19:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f844d400640 1 -- 172.21.15.86:0/4272844415 learned_addr learned my addr 172.21.15.86:0/4272844415 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.637+0000 7f8455e00640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845806fdc0 0x7f8458148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f8456800640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815c380 0x7f845806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844d400640 1 -- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815c380 msgr2=0x7f845806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844d400640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815c380 0x7f845806ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844d400640 1 -- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845806fdc0 msgr2=0x7f8458148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844d400640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845806fdc0 0x7f8458148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844d400640 1 -- 172.21.15.86:0/4272844415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8458153b90 con 0x7f84580cd2a0 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f8456800640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815c380 0x7f845806ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844d400640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f845806c770 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f84400ad040 tx=0x7f844001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844ea00640 1 -- 172.21.15.86:0/4272844415 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84400be5f0 con 0x7f84580cd2a0 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844ea00640 1 -- 172.21.15.86:0/4272844415 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84400be790 con 0x7f84580cd2a0 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844ea00640 1 -- 172.21.15.86:0/4272844415 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84400beb60 con 0x7f84580cd2a0 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845806d2e0 con 0x7f84580cd2a0 2024-03-22T04:19:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845806d8a0 con 0x7f84580cd2a0 2024-03-22T04:19:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845804f900 con 0x7f84580cd2a0 2024-03-22T04:19:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844ea00640 1 -- 172.21.15.86:0/4272844415 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f84400b43f0 con 0x7f84580cd2a0 2024-03-22T04:19:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844ea00640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8434067b20 0x7f8434069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f844ea00640 1 -- 172.21.15.86:0/4272844415 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f84400a8030 con 0x7f84580cd2a0 2024-03-22T04:19:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.641+0000 7f8455e00640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8434067b20 0x7f8434069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.645+0000 7f8455e00640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8434067b20 0x7f8434069fe0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f84480012f0 tx=0x7f84480016e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.645+0000 7f844ea00640 1 -- 172.21.15.86:0/4272844415 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8440016110 con 0x7f84580cd2a0 2024-03-22T04:19:02.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f84580d2bf0 con 0x7f84580cd2a0 2024-03-22T04:19:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f844ea00640 1 -- 172.21.15.86:0/4272844415 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f84400f03a0 con 0x7f84580cd2a0 2024-03-22T04:19:02.801 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8434067b20 msgr2=0x7f8434069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8434067b20 0x7f8434069fe0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f84480012f0 tx=0x7f84480016e0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 msgr2=0x7f845806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f845806c770 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f84400ad040 tx=0x7f844001ad60 comp rx=0 tx=0).stop 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 shutdown_connections 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8434067b20 0x7f8434069fe0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815c380 0x7f845806ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84580cd2a0 0x7f845806c770 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 --2- 172.21.15.86:0/4272844415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f845806fdc0 0x7f8458148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.801+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 >> 172.21.15.86:0/4272844415 conn(0x7f84580d1f80 msgr2=0x7f84580cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:02.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.805+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 shutdown_connections 2024-03-22T04:19:02.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.805+0000 7f8457200640 1 -- 172.21.15.86:0/4272844415 wait complete. 2024-03-22T04:19:02.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:19:02.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc8400640 1 Processor -- start 2024-03-22T04:19:02.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc8400640 1 -- start start 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc8400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1dc006e7e0 0x7f1dc006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc8400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc0072b00 0x7f1dc00b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc8400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc015c770 con 0x7f1dc0072650 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc8400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc0071350 con 0x7f1dc006e7e0 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc8400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc006f100 con 0x7f1dc0072b00 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1dc006e7e0 0x7f1dc006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc6400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc0072b00 0x7f1dc00b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc5000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1dc0072650 0x7f1dc00b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59622/0 (socket says 172.21.15.86:59622) 2024-03-22T04:19:02.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc5000640 1 -- 172.21.15.86:0/1715729077 learned_addr learned my addr 172.21.15.86:0/1715729077 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:02.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc5a00640 1 -- 172.21.15.86:0/1715729077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc0072b00 msgr2=0x7f1dc00b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:02.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc5a00640 1 --2- 172.21.15.86:0/1715729077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc0072b00 0x7f1dc00b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc5a00640 1 -- 172.21.15.86:0/1715729077 >> v1:172.21.15.86:6789/0 conn(0x7f1dc0072650 legacy=0x7f1dc00b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc5a00640 1 -- 172.21.15.86:0/1715729077 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc0073150 con 0x7f1dc006e7e0 2024-03-22T04:19:02.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dc6400640 1 --2- 172.21.15.86:0/1715729077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc0072b00 0x7f1dc00b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:02.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.961+0000 7f1dbfe00640 1 -- 172.21.15.86:0/1715729077 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308748657 0 0) 0x7f1dac001120 con 0x7f1dc0072650 2024-03-22T04:19:02.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc5a00640 1 --2- 172.21.15.86:0/1715729077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc006e7e0 0x7f1dc006ebc0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009d90 tx=0x7f1db401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819b1090504eca1c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dbfe00640 1 -- 172.21.15.86:0/1715729077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40b2070 con 0x7f1dc006e7e0 2024-03-22T04:19:02.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dbfe00640 1 -- 172.21.15.86:0/1715729077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db40a59a0 con 0x7f1dc006e7e0 2024-03-22T04:19:02.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dbfe00640 1 -- 172.21.15.86:0/1715729077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40a5d70 con 0x7f1dc006e7e0 2024-03-22T04:19:02.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc8400640 1 -- 172.21.15.86:0/1715729077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc006e7e0 msgr2=0x7f1dc006ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc8400640 1 --2- 172.21.15.86:0/1715729077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc006e7e0 0x7f1dc006ebc0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009d90 tx=0x7f1db401c8e0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc8400640 1 -- 172.21.15.86:0/1715729077 shutdown_connections 2024-03-22T04:19:02.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc8400640 1 --2- 172.21.15.86:0/1715729077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc0072b00 0x7f1dc00b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc8400640 1 --2- 172.21.15.86:0/1715729077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc006e7e0 0x7f1dc006ebc0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc8400640 1 -- 172.21.15.86:0/1715729077 >> 172.21.15.86:0/1715729077 conn(0x7f1dc00cc940 msgr2=0x7f1dc00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:02.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc8400640 1 -- 172.21.15.86:0/1715729077 shutdown_connections 2024-03-22T04:19:02.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.965+0000 7f1dc8400640 1 -- 172.21.15.86:0/1715729077 wait complete. 2024-03-22T04:19:02.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 Processor -- start 2024-03-22T04:19:02.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 -- start start 2024-03-22T04:19:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc006e7e0 0x7f1dc0153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dc0072650 0x7f1dc01531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc0072b00 0x7f1dc01594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0072080 con 0x7f1dc0072b00 2024-03-22T04:19:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00ce930 con 0x7f1dc0072650 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc00c67f0 con 0x7f1dc006e7e0 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc0072b00 0x7f1dc01594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc0072b00 0x7f1dc01594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44064/0 (socket says 172.21.15.86:44064) 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 -- 172.21.15.86:0/3681460065 learned_addr learned my addr 172.21.15.86:0/3681460065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc5a00640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc006e7e0 0x7f1dc0153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc5000640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dc0072650 0x7f1dc01531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 -- 172.21.15.86:0/3681460065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc006e7e0 msgr2=0x7f1dc0153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc006e7e0 0x7f1dc0153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 -- 172.21.15.86:0/3681460065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dc0072650 msgr2=0x7f1dc01531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dc0072650 0x7f1dc01531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 -- 172.21.15.86:0/3681460065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc00c5be0 con 0x7f1dc0072b00 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc6400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc0072b00 0x7f1dc01594f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f1db800e930 tx=0x7f1db800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dbd600640 1 -- 172.21.15.86:0/3681460065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8017070 con 0x7f1dc0072b00 2024-03-22T04:19:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dbd600640 1 -- 172.21.15.86:0/3681460065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8004020 con 0x7f1dc0072b00 2024-03-22T04:19:02.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dbd600640 1 -- 172.21.15.86:0/3681460065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8004430 con 0x7f1dc0072b00 2024-03-22T04:19:02.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc00c5f20 con 0x7f1dc0072b00 2024-03-22T04:19:02.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc0157da0 con 0x7f1dc0072b00 2024-03-22T04:19:02.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.969+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc0074470 con 0x7f1dc0072b00 2024-03-22T04:19:02.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.973+0000 7f1dbd600640 1 -- 172.21.15.86:0/3681460065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1db800c940 con 0x7f1dc0072b00 2024-03-22T04:19:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.973+0000 7f1dbd600640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1da0067b20 0x7f1da0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.973+0000 7f1dbd600640 1 -- 172.21.15.86:0/3681460065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f1db80860d0 con 0x7f1dc0072b00 2024-03-22T04:19:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.973+0000 7f1dc5a00640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1da0067b20 0x7f1da0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.973+0000 7f1dc5a00640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1da0067b20 0x7f1da0069fe0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1db40af000 tx=0x7f1db40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:02.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:02.977+0000 7f1dbd600640 1 -- 172.21.15.86:0/3681460065 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1dc00c5f20 con 0x7f1dc0072b00 2024-03-22T04:19:03.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.129+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1dc00c6100 con 0x7f1dc0072b00 2024-03-22T04:19:03.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dbd600640 1 -- 172.21.15.86:0/3681460065 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f1db80560b0 con 0x7f1dc0072b00 2024-03-22T04:19:03.131 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:19:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1da0067b20 msgr2=0x7f1da0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1da0067b20 0x7f1da0069fe0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1db40af000 tx=0x7f1db40a7070 comp rx=0 tx=0).stop 2024-03-22T04:19:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc0072b00 msgr2=0x7f1dc01594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc0072b00 0x7f1dc01594f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f1db800e930 tx=0x7f1db800edf0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 shutdown_connections 2024-03-22T04:19:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1da0067b20 0x7f1da0069fe0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dc0072650 0x7f1dc01531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc0072b00 0x7f1dc01594f0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 --2- 172.21.15.86:0/3681460065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dc006e7e0 0x7f1dc0153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 >> 172.21.15.86:0/3681460065 conn(0x7f1dc00cc940 msgr2=0x7f1dc00b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 shutdown_connections 2024-03-22T04:19:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.133+0000 7f1dc8400640 1 -- 172.21.15.86:0/3681460065 wait complete. 2024-03-22T04:19:03.153 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b94800640 1 Processor -- start 2024-03-22T04:19:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b94800640 1 -- start start 2024-03-22T04:19:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b94800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b8c0ce6b0 0x7f3b8c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b94800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 0x7f3b8c160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b94800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b8c0d11c0 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b94800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3b8c160860 con 0x7f3b8c0ce6b0 2024-03-22T04:19:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b94800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8c0cece0 con 0x7f3b8c15fed0 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b91e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3b8c0d0de0 0x7f3b8c0d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59630/0 (socket says 172.21.15.86:59630) 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b91e00640 1 -- 172.21.15.86:0/1806176019 learned_addr learned my addr 172.21.15.86:0/1806176019 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b91400640 1 --2- 172.21.15.86:0/1806176019 >> v2:172.21.15.86:3300/0 conn(0x7f3b8c0ce6b0 0x7f3b8c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b92800640 1 --2- 172.21.15.86:0/1806176019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 0x7f3b8c160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b90a00640 1 -- 172.21.15.86:0/1806176019 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877766241 0 0) 0x7f3b8c0d11c0 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b90a00640 1 -- 172.21.15.86:0/1806176019 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b74003530 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b91400640 1 -- 172.21.15.86:0/1806176019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 msgr2=0x7f3b8c160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b91400640 1 --2- 172.21.15.86:0/1806176019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 0x7f3b8c160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b91400640 1 -- 172.21.15.86:0/1806176019 >> v1:172.21.15.86:6789/0 conn(0x7f3b8c0d0de0 legacy=0x7f3b8c0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.289+0000 7f3b91400640 1 -- 172.21.15.86:0/1806176019 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c0cefc0 con 0x7f3b8c0ce6b0 2024-03-22T04:19:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b91400640 1 --2- 172.21.15.86:0/1806176019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0ce6b0 0x7f3b8c0cfe70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0097c0 tx=0x7f3b7c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955021792a4cfcad server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b90a00640 1 -- 172.21.15.86:0/1806176019 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b0070 con 0x7f3b8c0ce6b0 2024-03-22T04:19:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b90a00640 1 -- 172.21.15.86:0/1806176019 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0ab600 con 0x7f3b8c0ce6b0 2024-03-22T04:19:03.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b90a00640 1 -- 172.21.15.86:0/1806176019 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0ab9d0 con 0x7f3b8c0ce6b0 2024-03-22T04:19:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b94800640 1 -- 172.21.15.86:0/1806176019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0ce6b0 msgr2=0x7f3b8c0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b94800640 1 --2- 172.21.15.86:0/1806176019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0ce6b0 0x7f3b8c0cfe70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0097c0 tx=0x7f3b7c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b94800640 1 -- 172.21.15.86:0/1806176019 shutdown_connections 2024-03-22T04:19:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b94800640 1 --2- 172.21.15.86:0/1806176019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 0x7f3b8c160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b94800640 1 --2- 172.21.15.86:0/1806176019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0ce6b0 0x7f3b8c0cfe70 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b94800640 1 -- 172.21.15.86:0/1806176019 >> 172.21.15.86:0/1806176019 conn(0x7f3b8c06f910 msgr2=0x7f3b8c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b94800640 1 -- 172.21.15.86:0/1806176019 shutdown_connections 2024-03-22T04:19:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.293+0000 7f3b94800640 1 -- 172.21.15.86:0/1806176019 wait complete. 2024-03-22T04:19:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 Processor -- start 2024-03-22T04:19:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 -- start start 2024-03-22T04:19:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0ce6b0 0x7f3b8c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0d0de0 0x7f3b8c0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 0x7f3b8c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c0d0530 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c0d03b0 con 0x7f3b8c0ce6b0 2024-03-22T04:19:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8c071e30 con 0x7f3b8c15fed0 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0d0de0 0x7f3b8c0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0d0de0 0x7f3b8c0c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44074/0 (socket says 172.21.15.86:44074) 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 -- 172.21.15.86:0/3099586903 learned_addr learned my addr 172.21.15.86:0/3099586903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b92800640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 0x7f3b8c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91e00640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0ce6b0 0x7f3b8c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 -- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 msgr2=0x7f3b8c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 0x7f3b8c0c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 -- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0ce6b0 msgr2=0x7f3b8c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0ce6b0 0x7f3b8c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 -- 172.21.15.86:0/3099586903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c06cc50 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91e00640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0ce6b0 0x7f3b8c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b91400640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0d0de0 0x7f3b8c0c9370 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0ae040 tx=0x7f3b7c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b86000640 1 -- 172.21.15.86:0/3099586903 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b0040 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b86000640 1 -- 172.21.15.86:0/3099586903 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0b9070 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b86000640 1 -- 172.21.15.86:0/3099586903 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b4690 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c06cf30 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c06d4f0 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.297+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8c068650 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.301+0000 7f3b86000640 1 -- 172.21.15.86:0/3099586903 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3b7c0b4830 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.301+0000 7f3b86000640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3b70067b20 0x7f3b70069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.301+0000 7f3b86000640 1 -- 172.21.15.86:0/3099586903 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f3b7c11ff10 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.301+0000 7f3b91e00640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3b70067b20 0x7f3b70069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.301+0000 7f3b91e00640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3b70067b20 0x7f3b70069fe0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3b80005890 tx=0x7f3b80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.305+0000 7f3b86000640 1 -- 172.21.15.86:0/3099586903 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3b7c016090 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.489+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b8c0cca00 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.489+0000 7f3b86000640 1 -- 172.21.15.86:0/3099586903 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5186 (secure 0 0 0) 0x7f3b7c0f0540 con 0x7f3b8c0d0de0 2024-03-22T04:19:03.491 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:03.491 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":77,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:18:25.189133+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76850":{"gid":76850,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.196:6835/1227183960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1227183960},{"type":"v1","addr":"172.21.15.196:6835","nonce":1227183960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76850,"qdb_cluster":[76850]},"id":6}]} 2024-03-22T04:19:03.491 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 77 2024-03-22T04:19:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3b70067b20 msgr2=0x7f3b70069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3b70067b20 0x7f3b70069fe0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3b80005890 tx=0x7f3b80009040 comp rx=0 tx=0).stop 2024-03-22T04:19:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0d0de0 msgr2=0x7f3b8c0c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0d0de0 0x7f3b8c0c9370 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0ae040 tx=0x7f3b7c019240 comp rx=0 tx=0).stop 2024-03-22T04:19:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 shutdown_connections 2024-03-22T04:19:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3b70067b20 0x7f3b70069fe0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c15fed0 0x7f3b8c0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c0d0de0 0x7f3b8c0c9370 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 --2- 172.21.15.86:0/3099586903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0ce6b0 0x7f3b8c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 >> 172.21.15.86:0/3099586903 conn(0x7f3b8c06f910 msgr2=0x7f3b8c15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 shutdown_connections 2024-03-22T04:19:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.493+0000 7f3b94800640 1 -- 172.21.15.86:0/3099586903 wait complete. 2024-03-22T04:19:03.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:03.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09b4c00640 1 Processor -- start 2024-03-22T04:19:03.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09b4c00640 1 -- start start 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09b4c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f09b015f860 0x7f09b00ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09b4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09b00cdb00 0x7f09b009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09b4c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b009afb0 0x7f09b00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09b4c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f09b0073cf0 con 0x7f09b00cdb00 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09b4c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f09b009b390 con 0x7f09b015f860 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09b4c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f09b00ce280 con 0x7f09b009afb0 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09aea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09b00cdb00 0x7f09b009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09aea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09b00cdb00 0x7f09b009aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44082/0 (socket says 172.21.15.86:44082) 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09aea00640 1 -- 172.21.15.86:0/738711916 learned_addr learned my addr 172.21.15.86:0/738711916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09af400640 1 --2- 172.21.15.86:0/738711916 >> v2:172.21.15.196:3300/0 conn(0x7f09b015f860 0x7f09b00ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09afe00640 1 --2- 172.21.15.86:0/738711916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b009afb0 0x7f09b00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09aea00640 1 -- 172.21.15.86:0/738711916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b009afb0 msgr2=0x7f09b00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09aea00640 1 --2- 172.21.15.86:0/738711916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b009afb0 0x7f09b00d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09aea00640 1 -- 172.21.15.86:0/738711916 >> v2:172.21.15.196:3300/0 conn(0x7f09b015f860 msgr2=0x7f09b00ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09aea00640 1 --2- 172.21.15.86:0/738711916 >> v2:172.21.15.196:3300/0 conn(0x7f09b015f860 0x7f09b00ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.649+0000 7f09aea00640 1 -- 172.21.15.86:0/738711916 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b00ce560 con 0x7f09b00cdb00 2024-03-22T04:19:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09aea00640 1 --2- 172.21.15.86:0/738711916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 0x7f09b009aa70 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f09b00733d0 tx=0x7f099c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642afc52738cd657 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09ae000640 1 -- 172.21.15.86:0/738711916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0b0070 con 0x7f09b00cdb00 2024-03-22T04:19:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09ae000640 1 -- 172.21.15.86:0/738711916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099c0ab600 con 0x7f09b00cdb00 2024-03-22T04:19:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09ae000640 1 -- 172.21.15.86:0/738711916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0ab9d0 con 0x7f09b00cdb00 2024-03-22T04:19:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 -- 172.21.15.86:0/738711916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 msgr2=0x7f09b009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 --2- 172.21.15.86:0/738711916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 0x7f09b009aa70 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f09b00733d0 tx=0x7f099c01c230 comp rx=0 tx=0).stop 2024-03-22T04:19:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 -- 172.21.15.86:0/738711916 shutdown_connections 2024-03-22T04:19:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 --2- 172.21.15.86:0/738711916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b009afb0 0x7f09b00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 --2- 172.21.15.86:0/738711916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 0x7f09b009aa70 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 --2- 172.21.15.86:0/738711916 >> v2:172.21.15.196:3300/0 conn(0x7f09b015f860 0x7f09b00ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 -- 172.21.15.86:0/738711916 >> 172.21.15.86:0/738711916 conn(0x7f09b00b3d60 msgr2=0x7f09b00b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 -- 172.21.15.86:0/738711916 shutdown_connections 2024-03-22T04:19:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 -- 172.21.15.86:0/738711916 wait complete. 2024-03-22T04:19:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 Processor -- start 2024-03-22T04:19:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.653+0000 7f09b4c00640 1 -- start start 2024-03-22T04:19:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b009afb0 0x7f09b0165d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 0x7f09b0166850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b015f860 0x7f09b01f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09b015fc40 con 0x7f09b00cdb00 2024-03-22T04:19:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09b00cdf70 con 0x7f09b009afb0 2024-03-22T04:19:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f09b015fdc0 con 0x7f09b015f860 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09aea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 0x7f09b0166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b009afb0 0x7f09b0165d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b009afb0 0x7f09b0165d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:55286/0 (socket says 172.21.15.86:55286) 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09aea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 0x7f09b0166850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44090/0 (socket says 172.21.15.86:44090) 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 -- 172.21.15.86:0/3693705827 learned_addr learned my addr 172.21.15.86:0/3693705827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09afe00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b015f860 0x7f09b01f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 -- 172.21.15.86:0/3693705827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b015f860 msgr2=0x7f09b01f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b015f860 0x7f09b01f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 -- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 msgr2=0x7f09b0166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 0x7f09b0166850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 -- 172.21.15.86:0/3693705827 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b0164fa0 con 0x7f09b009afb0 2024-03-22T04:19:03.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09af400640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b009afb0 0x7f09b0165d60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f09a000e930 tx=0x7f09a000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09a7400640 1 -- 172.21.15.86:0/3693705827 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0017070 con 0x7f09b009afb0 2024-03-22T04:19:03.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09a7400640 1 -- 172.21.15.86:0/3693705827 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a000ca00 con 0x7f09b009afb0 2024-03-22T04:19:03.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b0165280 con 0x7f09b009afb0 2024-03-22T04:19:03.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09a7400640 1 -- 172.21.15.86:0/3693705827 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a000cdb0 con 0x7f09b009afb0 2024-03-22T04:19:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b0165840 con 0x7f09b009afb0 2024-03-22T04:19:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b0074470 con 0x7f09b009afb0 2024-03-22T04:19:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.657+0000 7f09a7400640 1 -- 172.21.15.86:0/3693705827 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f09a001a070 con 0x7f09b009afb0 2024-03-22T04:19:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.661+0000 7f09a7400640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0990067b20 0x7f0990069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.661+0000 7f09a7400640 1 -- 172.21.15.86:0/3693705827 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f09a0086650 con 0x7f09b009afb0 2024-03-22T04:19:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.661+0000 7f09aea00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0990067b20 0x7f0990069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.661+0000 7f09aea00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0990067b20 0x7f0990069fe0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f099c0ad000 tx=0x7f099c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.665+0000 7f09a7400640 1 -- 172.21.15.86:0/3693705827 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f09a00566f0 con 0x7f09b009afb0 2024-03-22T04:19:03.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.821+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09b015c7b0 con 0x7f09b009afb0 2024-03-22T04:19:03.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.821+0000 7f09a7400640 1 -- 172.21.15.86:0/3693705827 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17629 (secure 0 0 0) 0x7f09a005a790 con 0x7f09b009afb0 2024-03-22T04:19:03.821 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:03.821 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":97,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:18:21.162952+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.32","mappings":[{"from":7,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":3}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]},{"pgid":"14.3c","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3e","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6834/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6835/1097103199":"2024-03-23T04:17:08.939658+0000","172.21.15.196:6834/1097103199":"2024-03-23T04:17:08.939658+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:03.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.821+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0990067b20 msgr2=0x7f0990069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.821+0000 7f09b4c00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0990067b20 0x7f0990069fe0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f099c0ad000 tx=0x7f099c008040 comp rx=0 tx=0).stop 2024-03-22T04:19:03.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.821+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b009afb0 msgr2=0x7f09b0165d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.821+0000 7f09b4c00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b009afb0 0x7f09b0165d60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f09a000e930 tx=0x7f09a000edf0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.825+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 shutdown_connections 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.825+0000 7f09b4c00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0990067b20 0x7f0990069fe0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.825+0000 7f09b4c00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b015f860 0x7f09b01f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.825+0000 7f09b4c00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09b00cdb00 0x7f09b0166850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.825+0000 7f09b4c00640 1 --2- 172.21.15.86:0/3693705827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b009afb0 0x7f09b0165d60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.825+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 >> 172.21.15.86:0/3693705827 conn(0x7f09b00b3d60 msgr2=0x7f09b00b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.825+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 shutdown_connections 2024-03-22T04:19:03.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.825+0000 7f09b4c00640 1 -- 172.21.15.86:0/3693705827 wait complete. 2024-03-22T04:19:03.842 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:19:03.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:03.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.977+0000 7fd70ac00640 1 Processor -- start 2024-03-22T04:19:03.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd70ac00640 1 -- start start 2024-03-22T04:19:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd70ac00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd7040ad330 0x7fd70415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd70ac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70406ef40 0x7fd70406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd70ac00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd70415c7c0 con 0x7fd7040ad710 2024-03-22T04:19:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd70ac00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd70406dc40 con 0x7fd7040ad330 2024-03-22T04:19:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd70ac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd70406dea0 con 0x7fd70406ef40 2024-03-22T04:19:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70406ef40 0x7fd70406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd7040ad330 0x7fd70415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70406ef40 0x7fd70406d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:49422/0 (socket says 172.21.15.86:49422) 2024-03-22T04:19:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703400640 1 -- 172.21.15.86:0/1710211957 learned_addr learned my addr 172.21.15.86:0/1710211957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd708c00640 1 --1- 172.21.15.86:0/1710211957 >> v1:172.21.15.196:6789/0 conn(0x7fd7040ad710 0x7fd70413fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:49854/0 (socket says 172.21.15.86:49854) 2024-03-22T04:19:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703400640 1 -- 172.21.15.86:0/1710211957 >> v2:172.21.15.196:3300/0 conn(0x7fd7040ad330 msgr2=0x7fd70415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703400640 1 --2- 172.21.15.86:0/1710211957 >> v2:172.21.15.196:3300/0 conn(0x7fd7040ad330 0x7fd70415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703400640 1 -- 172.21.15.86:0/1710211957 >> v1:172.21.15.196:6789/0 conn(0x7fd7040ad710 legacy=0x7fd70413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703400640 1 -- 172.21.15.86:0/1710211957 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70406e180 con 0x7fd70406ef40 2024-03-22T04:19:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.981+0000 7fd703e00640 1 --2- 172.21.15.86:0/1710211957 >> v2:172.21.15.196:3300/0 conn(0x7fd7040ad330 0x7fd70415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd703400640 1 --2- 172.21.15.86:0/1710211957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70406ef40 0x7fd70406d700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009870 tx=0x7fd6f001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec7e0b315c2d5587 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd702a00640 1 -- 172.21.15.86:0/1710211957 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00ab640 con 0x7fd70406ef40 2024-03-22T04:19:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd702a00640 1 -- 172.21.15.86:0/1710211957 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f00b0070 con 0x7fd70406ef40 2024-03-22T04:19:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd702a00640 1 -- 172.21.15.86:0/1710211957 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00aba00 con 0x7fd70406ef40 2024-03-22T04:19:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 -- 172.21.15.86:0/1710211957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70406ef40 msgr2=0x7fd70406d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 --2- 172.21.15.86:0/1710211957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70406ef40 0x7fd70406d700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009870 tx=0x7fd6f001c530 comp rx=0 tx=0).stop 2024-03-22T04:19:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 -- 172.21.15.86:0/1710211957 shutdown_connections 2024-03-22T04:19:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 --2- 172.21.15.86:0/1710211957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70406ef40 0x7fd70406d700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 --2- 172.21.15.86:0/1710211957 >> v2:172.21.15.196:3300/0 conn(0x7fd7040ad330 0x7fd70415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 -- 172.21.15.86:0/1710211957 >> 172.21.15.86:0/1710211957 conn(0x7fd7040b1df0 msgr2=0x7fd7040b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:03.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 -- 172.21.15.86:0/1710211957 shutdown_connections 2024-03-22T04:19:03.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 -- 172.21.15.86:0/1710211957 wait complete. 2024-03-22T04:19:03.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.985+0000 7fd70ac00640 1 Processor -- start 2024-03-22T04:19:03.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 -- start start 2024-03-22T04:19:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70406ef40 0x7fd704164510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7040ad330 0x7fd7041fb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7040ad710 0x7fd7041fb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7041690e0 con 0x7fd7040ad710 2024-03-22T04:19:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd704168f60 con 0x7fd70406ef40 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd704169260 con 0x7fd7040ad330 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7040ad710 0x7fd7041fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd703e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70406ef40 0x7fd704164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7040ad710 0x7fd7041fb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44106/0 (socket says 172.21.15.86:44106) 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 -- 172.21.15.86:0/455929691 learned_addr learned my addr 172.21.15.86:0/455929691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd703400640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7040ad330 0x7fd7041fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 -- 172.21.15.86:0/455929691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7040ad330 msgr2=0x7fd7041fb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7040ad330 0x7fd7041fb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 -- 172.21.15.86:0/455929691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70406ef40 msgr2=0x7fd704164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70406ef40 0x7fd704164510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 -- 172.21.15.86:0/455929691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70406ffe0 con 0x7fd7040ad710 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd703e00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70406ef40 0x7fd704164510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd708c00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7040ad710 0x7fd7041fb740 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00c7d0 tx=0x7fd6fc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd6fbe00640 1 -- 172.21.15.86:0/455929691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc017070 con 0x7fd7040ad710 2024-03-22T04:19:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd6fbe00640 1 -- 172.21.15.86:0/455929691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6fc00a640 con 0x7fd7040ad710 2024-03-22T04:19:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7040736e0 con 0x7fd7040ad710 2024-03-22T04:19:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd6fbe00640 1 -- 172.21.15.86:0/455929691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00a9f0 con 0x7fd7040ad710 2024-03-22T04:19:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd704158ea0 con 0x7fd7040ad710 2024-03-22T04:19:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.989+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd704159790 con 0x7fd7040ad710 2024-03-22T04:19:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.993+0000 7fd6fbe00640 1 -- 172.21.15.86:0/455929691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd6fc01a070 con 0x7fd7040ad710 2024-03-22T04:19:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.993+0000 7fd6fbe00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6e4067b20 0x7fd6e4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.993+0000 7fd6fbe00640 1 -- 172.21.15.86:0/455929691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fd6fc0857b0 con 0x7fd7040ad710 2024-03-22T04:19:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.993+0000 7fd703e00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6e4067b20 0x7fd6e4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.993+0000 7fd703e00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6e4067b20 0x7fd6e4069fe0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009580 tx=0x7fd6f4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:03.997+0000 7fd6fbe00640 1 -- 172.21.15.86:0/455929691 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd6fc055820 con 0x7fd7040ad710 2024-03-22T04:19:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.185+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd704068650 con 0x7fd7040ad710 2024-03-22T04:19:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.185+0000 7fd6fbe00640 1 -- 172.21.15.86:0/455929691 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5186 (secure 0 0 0) 0x7fd6fc0598c0 con 0x7fd7040ad710 2024-03-22T04:19:04.186 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:04.186 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":77,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:18:25.189133+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76850":{"gid":76850,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.196:6835/1227183960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1227183960},{"type":"v1","addr":"172.21.15.196:6835","nonce":1227183960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76850,"qdb_cluster":[76850]},"id":6}]} 2024-03-22T04:19:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 77 2024-03-22T04:19:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6e4067b20 msgr2=0x7fd6e4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6e4067b20 0x7fd6e4069fe0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009580 tx=0x7fd6f4009190 comp rx=0 tx=0).stop 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7040ad710 msgr2=0x7fd7041fb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7040ad710 0x7fd7041fb740 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00c7d0 tx=0x7fd6fc00cc90 comp rx=0 tx=0).stop 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 shutdown_connections 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6e4067b20 0x7fd6e4069fe0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7040ad710 0x7fd7041fb740 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7040ad330 0x7fd7041fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 --2- 172.21.15.86:0/455929691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70406ef40 0x7fd704164510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 >> 172.21.15.86:0/455929691 conn(0x7fd7040b1df0 msgr2=0x7fd7040b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 shutdown_connections 2024-03-22T04:19:04.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.189+0000 7fd70ac00640 1 -- 172.21.15.86:0/455929691 wait complete. 2024-03-22T04:19:04.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfca600640 1 Processor -- start 2024-03-22T04:19:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfca600640 1 -- start start 2024-03-22T04:19:04.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfca600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40d0580 0x7fbfc4071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfca600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc41623d0 con 0x7fbfc415e630 2024-03-22T04:19:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfca600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc41620f0 con 0x7fbfc406e2f0 2024-03-22T04:19:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfca600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc415ec10 con 0x7fbfc40d0580 2024-03-22T04:19:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbfc415e630 0x7fbfc40cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59638/0 (socket says 172.21.15.86:59638) 2024-03-22T04:19:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc3400640 1 -- 172.21.15.86:0/484914673 learned_addr learned my addr 172.21.15.86:0/484914673 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc2a00640 1 --2- 172.21.15.86:0/484914673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40d0580 0x7fbfc4071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc2000640 1 -- 172.21.15.86:0/484914673 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31865383 0 0) 0x7fbfc41623d0 con 0x7fbfc415e630 2024-03-22T04:19:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc2000640 1 -- 172.21.15.86:0/484914673 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfa8003530 con 0x7fbfc415e630 2024-03-22T04:19:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc2a00640 1 -- 172.21.15.86:0/484914673 >> v1:172.21.15.196:6789/0 conn(0x7fbfc406e2f0 legacy=0x7fbfc4071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc2a00640 1 -- 172.21.15.86:0/484914673 >> v1:172.21.15.86:6789/0 conn(0x7fbfc415e630 legacy=0x7fbfc40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc2000640 1 -- 172.21.15.86:0/484914673 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802070068 0 0) 0x7fbfc41620f0 con 0x7fbfc406e2f0 2024-03-22T04:19:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.341+0000 7fbfc2a00640 1 -- 172.21.15.86:0/484914673 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc415eef0 con 0x7fbfc40d0580 2024-03-22T04:19:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfc2a00640 1 --2- 172.21.15.86:0/484914673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40d0580 0x7fbfc4071100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00097f0 tx=0x7fbfb001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd2eefdc347d0f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfc2000640 1 -- 172.21.15.86:0/484914673 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00b0070 con 0x7fbfc40d0580 2024-03-22T04:19:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfc2000640 1 -- 172.21.15.86:0/484914673 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb00ab600 con 0x7fbfc40d0580 2024-03-22T04:19:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfc2000640 1 -- 172.21.15.86:0/484914673 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00ab9d0 con 0x7fbfc40d0580 2024-03-22T04:19:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfca600640 1 -- 172.21.15.86:0/484914673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40d0580 msgr2=0x7fbfc4071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfca600640 1 --2- 172.21.15.86:0/484914673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40d0580 0x7fbfc4071100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00097f0 tx=0x7fbfb001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfca600640 1 -- 172.21.15.86:0/484914673 shutdown_connections 2024-03-22T04:19:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfca600640 1 --2- 172.21.15.86:0/484914673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40d0580 0x7fbfc4071100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfca600640 1 -- 172.21.15.86:0/484914673 >> 172.21.15.86:0/484914673 conn(0x7fbfc406f910 msgr2=0x7fbfc406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfca600640 1 -- 172.21.15.86:0/484914673 shutdown_connections 2024-03-22T04:19:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.345+0000 7fbfca600640 1 -- 172.21.15.86:0/484914673 wait complete. 2024-03-22T04:19:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 Processor -- start 2024-03-22T04:19:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 -- start start 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfc406e2f0 0x7fbfc41565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc40d0580 0x7fbfc40c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc415e630 0x7fbfc40c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc40cfaf0 con 0x7fbfc406e2f0 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc415f9d0 con 0x7fbfc40d0580 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc40cfc70 con 0x7fbfc415e630 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfc406e2f0 0x7fbfc41565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfc406e2f0 0x7fbfc41565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44118/0 (socket says 172.21.15.86:44118) 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 -- 172.21.15.86:0/2042767134 learned_addr learned my addr 172.21.15.86:0/2042767134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc2a00640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc40d0580 0x7fbfc40c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3e00640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc415e630 0x7fbfc40c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 -- 172.21.15.86:0/2042767134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc415e630 msgr2=0x7fbfc40c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc415e630 0x7fbfc40c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 -- 172.21.15.86:0/2042767134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc40d0580 msgr2=0x7fbfc40c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc40d0580 0x7fbfc40c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 -- 172.21.15.86:0/2042767134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc415dd40 con 0x7fbfc406e2f0 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfc3400640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfc406e2f0 0x7fbfc41565a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400c480 tx=0x7fbfb400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfbb400640 1 -- 172.21.15.86:0/2042767134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4017070 con 0x7fbfc406e2f0 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfbb400640 1 -- 172.21.15.86:0/2042767134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4005cf0 con 0x7fbfc406e2f0 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc40c7790 con 0x7fbfc406e2f0 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfbb400640 1 -- 172.21.15.86:0/2042767134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4007290 con 0x7fbfc406e2f0 2024-03-22T04:19:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc41543e0 con 0x7fbfc406e2f0 2024-03-22T04:19:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.349+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc4068650 con 0x7fbfc406e2f0 2024-03-22T04:19:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.353+0000 7fbfbb400640 1 -- 172.21.15.86:0/2042767134 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbfb401a070 con 0x7fbfc406e2f0 2024-03-22T04:19:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.353+0000 7fbfbb400640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbfa4067b20 0x7fbfa4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.353+0000 7fbfbb400640 1 -- 172.21.15.86:0/2042767134 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fbfb40855e0 con 0x7fbfc406e2f0 2024-03-22T04:19:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.353+0000 7fbfc2a00640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbfa4067b20 0x7fbfa4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.353+0000 7fbfc2a00640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbfa4067b20 0x7fbfa4069fe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00ad000 tx=0x7fbfb00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.357+0000 7fbfbb400640 1 -- 172.21.15.86:0/2042767134 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbfb4055650 con 0x7fbfc406e2f0 2024-03-22T04:19:04.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.541+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfc40c7b20 con 0x7fbfc406e2f0 2024-03-22T04:19:04.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfbb400640 1 -- 172.21.15.86:0/2042767134 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5186 (secure 0 0 0) 0x7fbfb40596f0 con 0x7fbfc406e2f0 2024-03-22T04:19:04.544 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:04.544 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":77,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:16:58.021326+0000","modified":"2024-03-22T04:18:25.189133+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":76850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_76850":{"gid":76850,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.196:6835/1227183960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1227183960},{"type":"v1","addr":"172.21.15.196:6835","nonce":1227183960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":76850,"qdb_cluster":[76850]},"id":6}]} 2024-03-22T04:19:04.544 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 77 2024-03-22T04:19:04.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbfa4067b20 msgr2=0x7fbfa4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbfa4067b20 0x7fbfa4069fe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00ad000 tx=0x7fbfb00a6000 comp rx=0 tx=0).stop 2024-03-22T04:19:04.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfc406e2f0 msgr2=0x7fbfc41565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfc406e2f0 0x7fbfc41565a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400c480 tx=0x7fbfb400c940 comp rx=0 tx=0).stop 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 shutdown_connections 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbfa4067b20 0x7fbfa4069fe0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc40d0580 0x7fbfc40c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc415e630 0x7fbfc40c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 --2- 172.21.15.86:0/2042767134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfc406e2f0 0x7fbfc41565a0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 >> 172.21.15.86:0/2042767134 conn(0x7fbfc406f910 msgr2=0x7fbfc40b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 shutdown_connections 2024-03-22T04:19:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.545+0000 7fbfca600640 1 -- 172.21.15.86:0/2042767134 wait complete. 2024-03-22T04:19:04.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:04.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.697+0000 7fa8e1e00640 1 Processor -- start 2024-03-22T04:19:04.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.697+0000 7fa8e1e00640 1 -- start start 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8e1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15b3f0 0x7fa8dc0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8e1e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa8dc0ae540 0x7fa8dc072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8e1e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8dc071920 con 0x7fa8dc0cc3e0 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8e1e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa8dc0716d0 con 0x7fa8dc0ae540 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8e1e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa8dc06d990 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8da000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa8dc0cc3e0 0x7fa8dc0ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59640/0 (socket says 172.21.15.86:59640) 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8da000640 1 -- 172.21.15.86:0/307826846 learned_addr learned my addr 172.21.15.86:0/307826846 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8daa00640 1 --2- 172.21.15.86:0/307826846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15b3f0 0x7fa8dc0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8db400640 1 --2- 172.21.15.86:0/307826846 >> v2:172.21.15.196:3300/0 conn(0x7fa8dc0ae540 0x7fa8dc072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8d9600640 1 -- 172.21.15.86:0/307826846 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313568247 0 0) 0x7fa8dc071920 con 0x7fa8dc0cc3e0 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8d9600640 1 -- 172.21.15.86:0/307826846 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8c0003530 con 0x7fa8dc0cc3e0 2024-03-22T04:19:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8daa00640 1 -- 172.21.15.86:0/307826846 >> v2:172.21.15.196:3300/0 conn(0x7fa8dc0ae540 msgr2=0x7fa8dc072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:04.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8daa00640 1 --2- 172.21.15.86:0/307826846 >> v2:172.21.15.196:3300/0 conn(0x7fa8dc0ae540 0x7fa8dc072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8daa00640 1 -- 172.21.15.86:0/307826846 >> v1:172.21.15.86:6789/0 conn(0x7fa8dc0cc3e0 legacy=0x7fa8dc0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.701+0000 7fa8daa00640 1 -- 172.21.15.86:0/307826846 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc071c00 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8daa00640 1 --2- 172.21.15.86:0/307826846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15b3f0 0x7fa8dc0729c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40097c0 tx=0x7fa8c401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c2d407c2d156c94 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8d9600640 1 -- 172.21.15.86:0/307826846 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40b0070 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8d9600640 1 -- 172.21.15.86:0/307826846 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c40ab600 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8d9600640 1 -- 172.21.15.86:0/307826846 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40ab9b0 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 -- 172.21.15.86:0/307826846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15b3f0 msgr2=0x7fa8dc0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/307826846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15b3f0 0x7fa8dc0729c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40097c0 tx=0x7fa8c401c410 comp rx=0 tx=0).stop 2024-03-22T04:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 -- 172.21.15.86:0/307826846 shutdown_connections 2024-03-22T04:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/307826846 >> v2:172.21.15.196:3300/0 conn(0x7fa8dc0ae540 0x7fa8dc072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/307826846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15b3f0 0x7fa8dc0729c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 -- 172.21.15.86:0/307826846 >> 172.21.15.86:0/307826846 conn(0x7fa8dc0b1df0 msgr2=0x7fa8dc0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 -- 172.21.15.86:0/307826846 shutdown_connections 2024-03-22T04:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 -- 172.21.15.86:0/307826846 wait complete. 2024-03-22T04:19:04.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 Processor -- start 2024-03-22T04:19:04.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.705+0000 7fa8e1e00640 1 -- start start 2024-03-22T04:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc0ae540 0x7fa8dc14cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc0cc3e0 0x7fa8dc156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8dc15b3f0 0x7fa8dc157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc06e320 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc06e1a0 con 0x7fa8dc0cc3e0 2024-03-22T04:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa8dc06e4a0 con 0x7fa8dc0ae540 2024-03-22T04:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8dc15b3f0 0x7fa8dc157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8da000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc0cc3e0 0x7fa8dc156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8dc15b3f0 0x7fa8dc157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44122/0 (socket says 172.21.15.86:44122) 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 -- 172.21.15.86:0/755190491 learned_addr learned my addr 172.21.15.86:0/755190491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8daa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc0ae540 0x7fa8dc14cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 -- 172.21.15.86:0/755190491 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc0ae540 msgr2=0x7fa8dc14cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc0ae540 0x7fa8dc14cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 -- 172.21.15.86:0/755190491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc0cc3e0 msgr2=0x7fa8dc156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc0cc3e0 0x7fa8dc156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 -- 172.21.15.86:0/755190491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc150680 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8db400640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8dc15b3f0 0x7fa8dc157090 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400ef90 tx=0x7fa8d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8d3400640 1 -- 172.21.15.86:0/755190491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4018020 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8d3400640 1 -- 172.21.15.86:0/755190491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d4014480 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8d3400640 1 -- 172.21.15.86:0/755190491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4014870 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc168780 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc168d40 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8dc068650 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.709+0000 7fa8d3400640 1 -- 172.21.15.86:0/755190491 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa8d401b020 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.713+0000 7fa8d3400640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8b8067b20 0x7fa8b8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:04.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.713+0000 7fa8d3400640 1 -- 172.21.15.86:0/755190491 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7fa8d4086f70 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.713+0000 7fa8daa00640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8b8067b20 0x7fa8b8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:04.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.713+0000 7fa8daa00640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8b8067b20 0x7fa8b8069fe0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40ad000 tx=0x7fa8c40a4f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:04.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.717+0000 7fa8d3400640 1 -- 172.21.15.86:0/755190491 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa8d4022030 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.873+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8dc153f80 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.873+0000 7fa8d3400640 1 -- 172.21.15.86:0/755190491 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+17629 (secure 0 0 0) 0x7fa8d4010040 con 0x7fa8dc15b3f0 2024-03-22T04:19:04.874 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:04.874 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":97,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:18:21.162952+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:16:56.341603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-03-22T04:16:56.977675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.32","mappings":[{"from":7,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":3}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]},{"pgid":"14.3c","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3e","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6834/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6835/1097103199":"2024-03-23T04:17:08.939658+0000","172.21.15.196:6834/1097103199":"2024-03-23T04:17:08.939658+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8b8067b20 msgr2=0x7fa8b8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8b8067b20 0x7fa8b8069fe0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40ad000 tx=0x7fa8c40a4f90 comp rx=0 tx=0).stop 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8dc15b3f0 msgr2=0x7fa8dc157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8dc15b3f0 0x7fa8dc157090 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400ef90 tx=0x7fa8d400c490 comp rx=0 tx=0).stop 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 shutdown_connections 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8b8067b20 0x7fa8b8069fe0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc0cc3e0 0x7fa8dc156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8dc15b3f0 0x7fa8dc157090 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 --2- 172.21.15.86:0/755190491 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc0ae540 0x7fa8dc14cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 >> 172.21.15.86:0/755190491 conn(0x7fa8dc0b1df0 msgr2=0x7fa8dc153230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 shutdown_connections 2024-03-22T04:19:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:04.877+0000 7fa8e1e00640 1 -- 172.21.15.86:0/755190491 wait complete. 2024-03-22T04:19:04.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:19:05.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4333800640 1 Processor -- start 2024-03-22T04:19:05.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4333800640 1 -- start start 2024-03-22T04:19:05.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4333800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f432c0d0de0 0x7f432c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4333800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4333800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f432c0d1dc0 0x7f432c15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4333800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0d11c0 con 0x7f432c0d1dc0 2024-03-22T04:19:05.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4333800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f432c0cece0 con 0x7f432c0d0de0 2024-03-22T04:19:05.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4333800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f432c0cefc0 con 0x7f432c0d19a0 2024-03-22T04:19:05.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4331800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f432c0d1dc0 0x7f432c15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f432be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4330e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f432c0d0de0 0x7f432c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f432be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c0cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:49476/0 (socket says 172.21.15.86:49476) 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4331800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f432c0d1dc0 0x7f432c15fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44134/0 (socket says 172.21.15.86:44134) 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f432be00640 1 -- 172.21.15.86:0/919666472 learned_addr learned my addr 172.21.15.86:0/919666472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f432be00640 1 -- 172.21.15.86:0/919666472 >> v2:172.21.15.196:3300/0 conn(0x7f432c0d0de0 msgr2=0x7f432c0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f432be00640 1 --2- 172.21.15.86:0/919666472 >> v2:172.21.15.196:3300/0 conn(0x7f432c0d0de0 0x7f432c0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f432be00640 1 -- 172.21.15.86:0/919666472 >> v2:172.21.15.86:3300/0 conn(0x7f432c0d1dc0 msgr2=0x7f432c15fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f432be00640 1 --2- 172.21.15.86:0/919666472 >> v2:172.21.15.86:3300/0 conn(0x7f432c0d1dc0 0x7f432c15fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f432be00640 1 -- 172.21.15.86:0/919666472 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c0cf2a0 con 0x7f432c0d19a0 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4330e00640 1 --2- 172.21.15.86:0/919666472 >> v2:172.21.15.196:3300/0 conn(0x7f432c0d0de0 0x7f432c0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:05.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.033+0000 7f4331800640 1 --2- 172.21.15.86:0/919666472 >> v2:172.21.15.86:3300/0 conn(0x7f432c0d1dc0 0x7f432c15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:05.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f432be00640 1 --2- 172.21.15.86:0/919666472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c0cfe70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f432c06f4c0 tx=0x7f431801c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1273da829e9cbf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:05.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f432b400640 1 -- 172.21.15.86:0/919666472 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180b0070 con 0x7f432c0d19a0 2024-03-22T04:19:05.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f432b400640 1 -- 172.21.15.86:0/919666472 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43180ab600 con 0x7f432c0d19a0 2024-03-22T04:19:05.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f432b400640 1 -- 172.21.15.86:0/919666472 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180ab9d0 con 0x7f432c0d19a0 2024-03-22T04:19:05.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 -- 172.21.15.86:0/919666472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 msgr2=0x7f432c0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 --2- 172.21.15.86:0/919666472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c0cfe70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f432c06f4c0 tx=0x7f431801c520 comp rx=0 tx=0).stop 2024-03-22T04:19:05.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 -- 172.21.15.86:0/919666472 shutdown_connections 2024-03-22T04:19:05.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 --2- 172.21.15.86:0/919666472 >> v2:172.21.15.86:3300/0 conn(0x7f432c0d1dc0 0x7f432c15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 --2- 172.21.15.86:0/919666472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c0cfe70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 --2- 172.21.15.86:0/919666472 >> v2:172.21.15.196:3300/0 conn(0x7f432c0d0de0 0x7f432c0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 -- 172.21.15.86:0/919666472 >> 172.21.15.86:0/919666472 conn(0x7f432c06f910 msgr2=0x7f432c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:05.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 -- 172.21.15.86:0/919666472 shutdown_connections 2024-03-22T04:19:05.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.037+0000 7f4333800640 1 -- 172.21.15.86:0/919666472 wait complete. 2024-03-22T04:19:05.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 Processor -- start 2024-03-22T04:19:05.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 -- start start 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f432c0d0de0 0x7f432c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c162020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c0d1dc0 0x7f432c162560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f432c071fb0 con 0x7f432c0d1dc0 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f432c071e30 con 0x7f432c0d0de0 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f432c15c8f0 con 0x7f432c0d19a0 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f432be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4331800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c0d1dc0 0x7f432c162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4330e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f432c0d0de0 0x7f432c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f432be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c162020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:49492/0 (socket says 172.21.15.86:49492) 2024-03-22T04:19:05.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f432be00640 1 -- 172.21.15.86:0/1207962998 learned_addr learned my addr 172.21.15.86:0/1207962998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4331800640 1 -- 172.21.15.86:0/1207962998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 msgr2=0x7f432c162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4331800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4331800640 1 -- 172.21.15.86:0/1207962998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f432c0d0de0 msgr2=0x7f432c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4331800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f432c0d0de0 0x7f432c1597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4331800640 1 -- 172.21.15.86:0/1207962998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c164080 con 0x7f432c0d1dc0 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4330e00640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f432c0d0de0 0x7f432c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4331800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c0d1dc0 0x7f432c162560 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f432400ef90 tx=0x7f432400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4328c00640 1 -- 172.21.15.86:0/1207962998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324018020 con 0x7f432c0d1dc0 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4328c00640 1 -- 172.21.15.86:0/1207962998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4324014480 con 0x7f432c0d1dc0 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4328c00640 1 -- 172.21.15.86:0/1207962998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324014870 con 0x7f432c0d1dc0 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c159d30 con 0x7f432c0d1dc0 2024-03-22T04:19:05.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.041+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c0b7460 con 0x7f432c0d1dc0 2024-03-22T04:19:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.045+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c164250 con 0x7f432c0d1dc0 2024-03-22T04:19:05.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.045+0000 7f4328c00640 1 -- 172.21.15.86:0/1207962998 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f432401b020 con 0x7f432c0d1dc0 2024-03-22T04:19:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.045+0000 7f4328c00640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f430c067b50 0x7f430c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.045+0000 7f4328c00640 1 -- 172.21.15.86:0/1207962998 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6534+0+0 (secure 0 0 0) 0x7f4324086e70 con 0x7f432c0d1dc0 2024-03-22T04:19:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.045+0000 7f4330e00640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f430c067b50 0x7f430c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.045+0000 7f4330e00640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f430c067b50 0x7f430c06a010 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f431c0095a0 tx=0x7f431c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:05.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.049+0000 7f4328c00640 1 -- 172.21.15.86:0/1207962998 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4324022030 con 0x7f432c0d1dc0 2024-03-22T04:19:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.237+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f432c068650 con 0x7f432c0d1dc0 2024-03-22T04:19:05.239 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:19:05.241+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:19:05.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.241+0000 7f4328c00640 1 -- 172.21.15.86:0/1207962998 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7f4324010040 con 0x7f432c0d1dc0 2024-03-22T04:19:05.243 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:19:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f430c067b50 msgr2=0x7f430c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f430c067b50 0x7f430c06a010 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f431c0095a0 tx=0x7f431c009190 comp rx=0 tx=0).stop 2024-03-22T04:19:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c0d1dc0 msgr2=0x7f432c162560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c0d1dc0 0x7f432c162560 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f432400ef90 tx=0x7f432400c490 comp rx=0 tx=0).stop 2024-03-22T04:19:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 shutdown_connections 2024-03-22T04:19:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f430c067b50 0x7f430c06a010 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f432c0d1dc0 0x7f432c162560 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f432c0d19a0 0x7f432c162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 --2- 172.21.15.86:0/1207962998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f432c0d0de0 0x7f432c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 >> 172.21.15.86:0/1207962998 conn(0x7f432c06f910 msgr2=0x7f432c15a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 shutdown_connections 2024-03-22T04:19:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.245+0000 7f4333800640 1 -- 172.21.15.86:0/1207962998 wait complete. 2024-03-22T04:19:05.254 INFO:tasks.ceph.mds.b.smithi196.stderr: -2402> 2024-03-22T04:18:27.179+0000 7efec8200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:19:05.254 INFO:tasks.ceph.mds.b.smithi196.stderr: -1952> 2024-03-22T04:18:28.003+0000 7efec3200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5652e579e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-22T04:19:05.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T04:19:05.298 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:19:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.401+0000 7fbe33600640 1 Processor -- start 2024-03-22T04:19:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.401+0000 7fbe33600640 1 -- start start 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe33600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbe2c0b3600 0x7fbe2c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe33600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2c15d710 con 0x7fbe2c15cee0 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe33600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2c15fe40 con 0x7fbe2c0b2360 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe33600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2c0b2740 con 0x7fbe2c0b3600 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe31600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbe2c15cee0 0x7fbe2c0b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59654/0 (socket says 172.21.15.86:59654) 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe31600640 1 -- 172.21.15.86:0/40163559 learned_addr learned my addr 172.21.15.86:0/40163559 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe2be00640 1 --2- 172.21.15.86:0/40163559 >> v2:172.21.15.196:3300/0 conn(0x7fbe2c0b3600 0x7fbe2c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe2b400640 1 -- 172.21.15.86:0/40163559 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358914710 0 0) 0x7fbe2c15d710 con 0x7fbe2c15cee0 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe2b400640 1 -- 172.21.15.86:0/40163559 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe10003530 con 0x7fbe2c15cee0 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe2be00640 1 -- 172.21.15.86:0/40163559 >> v1:172.21.15.196:6789/0 conn(0x7fbe2c0b2360 legacy=0x7fbe2c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe2be00640 1 -- 172.21.15.86:0/40163559 >> v1:172.21.15.86:6789/0 conn(0x7fbe2c15cee0 legacy=0x7fbe2c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe2b400640 1 -- 172.21.15.86:0/40163559 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1017311589 0 0) 0x7fbe2c15fe40 con 0x7fbe2c0b2360 2024-03-22T04:19:05.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.405+0000 7fbe2be00640 1 -- 172.21.15.86:0/40163559 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c160120 con 0x7fbe2c0b3600 2024-03-22T04:19:05.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe2be00640 1 --2- 172.21.15.86:0/40163559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b3600 0x7fbe2c15c9a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbe180097c0 tx=0x7fbe1801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7673762b54cdb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe2b400640 1 -- 172.21.15.86:0/40163559 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe180a7070 con 0x7fbe2c0b3600 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe2b400640 1 -- 172.21.15.86:0/40163559 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe180b0070 con 0x7fbe2c0b3600 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe2b400640 1 -- 172.21.15.86:0/40163559 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe180ab890 con 0x7fbe2c0b3600 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 -- 172.21.15.86:0/40163559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b3600 msgr2=0x7fbe2c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 --2- 172.21.15.86:0/40163559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b3600 0x7fbe2c15c9a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbe180097c0 tx=0x7fbe1801c470 comp rx=0 tx=0).stop 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 -- 172.21.15.86:0/40163559 shutdown_connections 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 --2- 172.21.15.86:0/40163559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b3600 0x7fbe2c15c9a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 -- 172.21.15.86:0/40163559 >> 172.21.15.86:0/40163559 conn(0x7fbe2c0cc940 msgr2=0x7fbe2c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 -- 172.21.15.86:0/40163559 shutdown_connections 2024-03-22T04:19:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 -- 172.21.15.86:0/40163559 wait complete. 2024-03-22T04:19:05.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 Processor -- start 2024-03-22T04:19:05.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.409+0000 7fbe33600640 1 -- start start 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b2360 0x7fbe2c0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe2c0b3600 0x7fbe2c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe2c15cee0 0x7fbe2c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c15d470 con 0x7fbe2c15cee0 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c15d2f0 con 0x7fbe2c0b2360 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c16a1f0 con 0x7fbe2c0b3600 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe2c15cee0 0x7fbe2c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe2c15cee0 0x7fbe2c0b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44152/0 (socket says 172.21.15.86:44152) 2024-03-22T04:19:05.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 -- 172.21.15.86:0/960132603 learned_addr learned my addr 172.21.15.86:0/960132603 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe2be00640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe2c0b3600 0x7fbe2c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe30c00640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b2360 0x7fbe2c0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 -- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe2c0b3600 msgr2=0x7fbe2c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe2c0b3600 0x7fbe2c0b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 -- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b2360 msgr2=0x7fbe2c0afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b2360 0x7fbe2c0afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 -- 172.21.15.86:0/960132603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c0d04e0 con 0x7fbe2c15cee0 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe2be00640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe2c0b3600 0x7fbe2c0b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe31600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe2c15cee0 0x7fbe2c0b05c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400c3f0 tx=0x7fbe2400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe28c00640 1 -- 172.21.15.86:0/960132603 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24017070 con 0x7fbe2c15cee0 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe28c00640 1 -- 172.21.15.86:0/960132603 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe24005cf0 con 0x7fbe2c15cee0 2024-03-22T04:19:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe28c00640 1 -- 172.21.15.86:0/960132603 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24007310 con 0x7fbe2c15cee0 2024-03-22T04:19:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c0731f0 con 0x7fbe2c15cee0 2024-03-22T04:19:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c0b0fd0 con 0x7fbe2c15cee0 2024-03-22T04:19:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2c0b0b00 con 0x7fbe2c15cee0 2024-03-22T04:19:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.413+0000 7fbe28c00640 1 -- 172.21.15.86:0/960132603 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbe240074b0 con 0x7fbe2c15cee0 2024-03-22T04:19:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.417+0000 7fbe28c00640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe0c067b20 0x7fbe0c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.417+0000 7fbe28c00640 1 -- 172.21.15.86:0/960132603 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6620+0+0 (secure 0 0 0) 0x7fbe24085620 con 0x7fbe2c15cee0 2024-03-22T04:19:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.417+0000 7fbe30c00640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe0c067b20 0x7fbe0c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.417+0000 7fbe30c00640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe0c067b20 0x7fbe0c069fe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0012f0 tx=0x7fbe1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.421+0000 7fbe28c00640 1 -- 172.21.15.86:0/960132603 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbe24055c90 con 0x7fbe2c15cee0 2024-03-22T04:19:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:05.609+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbe2c074470 con 0x7fbe2c15cee0 2024-03-22T04:19:06.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.249+0000 7fbe28c00640 1 -- 172.21.15.86:0/960132603 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7fbe24055590 con 0x7fbe2c15cee0 2024-03-22T04:19:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe0c067b20 msgr2=0x7fbe0c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe0c067b20 0x7fbe0c069fe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0012f0 tx=0x7fbe1c009040 comp rx=0 tx=0).stop 2024-03-22T04:19:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe2c15cee0 msgr2=0x7fbe2c0b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe2c15cee0 0x7fbe2c0b05c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400c3f0 tx=0x7fbe2400c8b0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 shutdown_connections 2024-03-22T04:19:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbe0c067b20 0x7fbe0c069fe0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe2c15cee0 0x7fbe2c0b05c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe2c0b3600 0x7fbe2c0b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 --2- 172.21.15.86:0/960132603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe2c0b2360 0x7fbe2c0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 >> 172.21.15.86:0/960132603 conn(0x7fbe2c0cc940 msgr2=0x7fbe2c153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 shutdown_connections 2024-03-22T04:19:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.253+0000 7fbe33600640 1 -- 172.21.15.86:0/960132603 wait complete. 2024-03-22T04:19:06.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T04:19:06.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593fe00640 1 Processor -- start 2024-03-22T04:19:06.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593fe00640 1 -- start start 2024-03-22T04:19:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5940145aa0 0x7f59400acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59400ae8c0 0x7f59400aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593fe00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594006e0a0 con 0x7f594015fc00 2024-03-22T04:19:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593fe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f594006ecb0 con 0x7f5940145aa0 2024-03-22T04:19:06.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f594006e380 con 0x7f59400ae8c0 2024-03-22T04:19:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5940145aa0 0x7f59400acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f5936000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59400ae8c0 0x7f59400aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:06.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593f400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f594015fc00 0x7f5940160090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52196/0 (socket says 172.21.15.86:52196) 2024-03-22T04:19:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593f400640 1 -- 172.21.15.86:0/2915066280 learned_addr learned my addr 172.21.15.86:0/2915066280 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f5936000640 1 -- 172.21.15.86:0/2915066280 >> v2:172.21.15.196:3300/0 conn(0x7f5940145aa0 msgr2=0x7f59400acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593e000640 1 -- 172.21.15.86:0/2915066280 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991273069 0 0) 0x7f594006e0a0 con 0x7f594015fc00 2024-03-22T04:19:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f5936000640 1 --2- 172.21.15.86:0/2915066280 >> v2:172.21.15.196:3300/0 conn(0x7f5940145aa0 0x7f59400acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f5936000640 1 -- 172.21.15.86:0/2915066280 >> v1:172.21.15.196:6789/0 conn(0x7f594015fc00 legacy=0x7f5940160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f5936000640 1 -- 172.21.15.86:0/2915066280 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594006e660 con 0x7f59400ae8c0 2024-03-22T04:19:06.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.413+0000 7f593ea00640 1 --2- 172.21.15.86:0/2915066280 >> v2:172.21.15.196:3300/0 conn(0x7f5940145aa0 0x7f59400acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f5936000640 1 --2- 172.21.15.86:0/2915066280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59400ae8c0 0x7f59400aed00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59280097c0 tx=0x7f592801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749fd83c81dd36dc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593e000640 1 -- 172.21.15.86:0/2915066280 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59280b0070 con 0x7f59400ae8c0 2024-03-22T04:19:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593e000640 1 -- 172.21.15.86:0/2915066280 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59280ab600 con 0x7f59400ae8c0 2024-03-22T04:19:06.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593e000640 1 -- 172.21.15.86:0/2915066280 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59280ab9d0 con 0x7f59400ae8c0 2024-03-22T04:19:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 -- 172.21.15.86:0/2915066280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59400ae8c0 msgr2=0x7f59400aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 --2- 172.21.15.86:0/2915066280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59400ae8c0 0x7f59400aed00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59280097c0 tx=0x7f592801c410 comp rx=0 tx=0).stop 2024-03-22T04:19:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 -- 172.21.15.86:0/2915066280 shutdown_connections 2024-03-22T04:19:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 --2- 172.21.15.86:0/2915066280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59400ae8c0 0x7f59400aed00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 --2- 172.21.15.86:0/2915066280 >> v2:172.21.15.196:3300/0 conn(0x7f5940145aa0 0x7f59400acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 -- 172.21.15.86:0/2915066280 >> 172.21.15.86:0/2915066280 conn(0x7f59400cfe20 msgr2=0x7f59400d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:06.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 -- 172.21.15.86:0/2915066280 shutdown_connections 2024-03-22T04:19:06.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 -- 172.21.15.86:0/2915066280 wait complete. 2024-03-22T04:19:06.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 Processor -- start 2024-03-22T04:19:06.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.417+0000 7f593fe00640 1 -- start start 2024-03-22T04:19:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59400ae8c0 0x7f5940165d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5940145aa0 0x7f5940159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f594015fc00 0x7f5940142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59400ad460 con 0x7f5940145aa0 2024-03-22T04:19:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59400ad2e0 con 0x7f59400ae8c0 2024-03-22T04:19:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5940169580 con 0x7f594015fc00 2024-03-22T04:19:06.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5940145aa0 0x7f5940159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5940145aa0 0x7f5940159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55970/0 (socket says 172.21.15.86:55970) 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 -- 172.21.15.86:0/2594302634 learned_addr learned my addr 172.21.15.86:0/2594302634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593ea00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59400ae8c0 0x7f5940165d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593f400640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f594015fc00 0x7f5940142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 -- 172.21.15.86:0/2594302634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f594015fc00 msgr2=0x7f5940142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f594015fc00 0x7f5940142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 -- 172.21.15.86:0/2594302634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59400ae8c0 msgr2=0x7f5940165d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59400ae8c0 0x7f5940165d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 -- 172.21.15.86:0/2594302634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594015a170 con 0x7f5940145aa0 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5936000640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5940145aa0 0x7f5940159c30 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f59280ae040 tx=0x7f59280191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5937400640 1 -- 172.21.15.86:0/2594302634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59280b0040 con 0x7f5940145aa0 2024-03-22T04:19:06.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5937400640 1 -- 172.21.15.86:0/2594302634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59280b43f0 con 0x7f5940145aa0 2024-03-22T04:19:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5940144b00 con 0x7f5940145aa0 2024-03-22T04:19:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f5937400640 1 -- 172.21.15.86:0/2594302634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59280b4800 con 0x7f5940145aa0 2024-03-22T04:19:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59401450c0 con 0x7f5940145aa0 2024-03-22T04:19:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.421+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5940074470 con 0x7f5940145aa0 2024-03-22T04:19:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.425+0000 7f5937400640 1 -- 172.21.15.86:0/2594302634 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f59280b49a0 con 0x7f5940145aa0 2024-03-22T04:19:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.425+0000 7f5937400640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f591c067b20 0x7f591c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.425+0000 7f5937400640 1 -- 172.21.15.86:0/2594302634 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6620+0+0 (secure 0 0 0) 0x7f592811fcd0 con 0x7f5940145aa0 2024-03-22T04:19:06.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.425+0000 7f593ea00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f591c067b20 0x7f591c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:06.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.425+0000 7f593ea00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f591c067b20 0x7f591c069fe0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5930009580 tx=0x7f5930009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:06.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.429+0000 7f5937400640 1 -- 172.21.15.86:0/2594302634 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5928016090 con 0x7f5940145aa0 2024-03-22T04:19:06.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:06.584+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5940166620 con 0x7f5940145aa0 2024-03-22T04:19:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.252+0000 7f5937400640 1 -- 172.21.15.86:0/2594302634 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7f59280f0340 con 0x7f5940145aa0 2024-03-22T04:19:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f591c067b20 msgr2=0x7f591c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f591c067b20 0x7f591c069fe0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5930009580 tx=0x7f5930009190 comp rx=0 tx=0).stop 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5940145aa0 msgr2=0x7f5940159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5940145aa0 0x7f5940159c30 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f59280ae040 tx=0x7f59280191b0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 shutdown_connections 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f591c067b20 0x7f591c069fe0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f594015fc00 0x7f5940142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5940145aa0 0x7f5940159c30 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 --2- 172.21.15.86:0/2594302634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59400ae8c0 0x7f5940165d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 >> 172.21.15.86:0/2594302634 conn(0x7f59400cfe20 msgr2=0x7f5940142480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 shutdown_connections 2024-03-22T04:19:07.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.256+0000 7f593fe00640 1 -- 172.21.15.86:0/2594302634 wait complete. 2024-03-22T04:19:07.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T04:19:07.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.416+0000 7f5270200640 1 Processor -- start 2024-03-22T04:19:07.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.416+0000 7f5270200640 1 -- start start 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f5270200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f52680b1cd0 0x7f52680b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f5270200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52680cfa70 0x7f526806d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f5270200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f526806dca0 0x7f5268141960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f5270200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52680cea80 con 0x7f52680cfa70 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f5270200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f526806e4d0 con 0x7f52680b1cd0 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f5270200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f52680b25f0 con 0x7f526806dca0 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52680cfa70 0x7f526806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52680cfa70 0x7f526806d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55982/0 (socket says 172.21.15.86:55982) 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 -- 172.21.15.86:0/1806831173 learned_addr learned my addr 172.21.15.86:0/1806831173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526d800640 1 --2- 172.21.15.86:0/1806831173 >> v2:172.21.15.196:3300/0 conn(0x7f52680b1cd0 0x7f52680b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526e200640 1 --2- 172.21.15.86:0/1806831173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f526806dca0 0x7f5268141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 -- 172.21.15.86:0/1806831173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f526806dca0 msgr2=0x7f5268141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 --2- 172.21.15.86:0/1806831173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f526806dca0 0x7f5268141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 -- 172.21.15.86:0/1806831173 >> v2:172.21.15.196:3300/0 conn(0x7f52680b1cd0 msgr2=0x7f52680b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 --2- 172.21.15.86:0/1806831173 >> v2:172.21.15.196:3300/0 conn(0x7f52680b1cd0 0x7f52680b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 -- 172.21.15.86:0/1806831173 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52680cecb0 con 0x7f52680cfa70 2024-03-22T04:19:07.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.420+0000 7f526ce00640 1 --2- 172.21.15.86:0/1806831173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52680cfa70 0x7f526806d760 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5268075b20 tx=0x7f525401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7de3e11c015926 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:07.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5267e00640 1 -- 172.21.15.86:0/1806831173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52540b0070 con 0x7f52680cfa70 2024-03-22T04:19:07.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5267e00640 1 -- 172.21.15.86:0/1806831173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52540ab600 con 0x7f52680cfa70 2024-03-22T04:19:07.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5267e00640 1 -- 172.21.15.86:0/1806831173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52540ab9d0 con 0x7f52680cfa70 2024-03-22T04:19:07.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- 172.21.15.86:0/1806831173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52680cfa70 msgr2=0x7f526806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:07.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 --2- 172.21.15.86:0/1806831173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52680cfa70 0x7f526806d760 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5268075b20 tx=0x7f525401c230 comp rx=0 tx=0).stop 2024-03-22T04:19:07.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- 172.21.15.86:0/1806831173 shutdown_connections 2024-03-22T04:19:07.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 --2- 172.21.15.86:0/1806831173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f526806dca0 0x7f5268141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 --2- 172.21.15.86:0/1806831173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52680cfa70 0x7f526806d760 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 --2- 172.21.15.86:0/1806831173 >> v2:172.21.15.196:3300/0 conn(0x7f52680b1cd0 0x7f52680b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- 172.21.15.86:0/1806831173 >> 172.21.15.86:0/1806831173 conn(0x7f526806b660 msgr2=0x7f526806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:07.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- 172.21.15.86:0/1806831173 shutdown_connections 2024-03-22T04:19:07.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- 172.21.15.86:0/1806831173 wait complete. 2024-03-22T04:19:07.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 Processor -- start 2024-03-22T04:19:07.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- start start 2024-03-22T04:19:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526806dca0 0x7f5268145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52680b1cd0 0x7f5268072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52680cfa70 0x7f5268155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f526813f5e0 con 0x7f526806dca0 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f526813f460 con 0x7f52680cfa70 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f5270200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f526806e0b0 con 0x7f52680b1cd0 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f526d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526806dca0 0x7f5268145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f526d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526806dca0 0x7f5268145510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55988/0 (socket says 172.21.15.86:55988) 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f526d800640 1 -- 172.21.15.86:0/2792135821 learned_addr learned my addr 172.21.15.86:0/2792135821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f526ce00640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52680b1cd0 0x7f5268072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.424+0000 7f526e200640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52680cfa70 0x7f5268155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f526d800640 1 -- 172.21.15.86:0/2792135821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52680b1cd0 msgr2=0x7f5268072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f526d800640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52680b1cd0 0x7f5268072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f526d800640 1 -- 172.21.15.86:0/2792135821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52680cfa70 msgr2=0x7f5268155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f526d800640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52680cfa70 0x7f5268155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f526d800640 1 -- 172.21.15.86:0/2792135821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5268072570 con 0x7f526806dca0 2024-03-22T04:19:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f526d800640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526806dca0 0x7f5268145510 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f525c007ef0 tx=0x7f525c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5265600640 1 -- 172.21.15.86:0/2792135821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c014620 con 0x7f526806dca0 2024-03-22T04:19:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5265600640 1 -- 172.21.15.86:0/2792135821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525c019070 con 0x7f526806dca0 2024-03-22T04:19:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5265600640 1 -- 172.21.15.86:0/2792135821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c014a00 con 0x7f526806dca0 2024-03-22T04:19:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5268072810 con 0x7f526806dca0 2024-03-22T04:19:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5268072dd0 con 0x7f526806dca0 2024-03-22T04:19:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5268074580 con 0x7f526806dca0 2024-03-22T04:19:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5265600640 1 -- 172.21.15.86:0/2792135821 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f525c014ba0 con 0x7f526806dca0 2024-03-22T04:19:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5265600640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5248067b20 0x7f5248069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f5265600640 1 -- 172.21.15.86:0/2792135821 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6153+0+0 (secure 0 0 0) 0x7f525c086830 con 0x7f526806dca0 2024-03-22T04:19:07.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.428+0000 7f526ce00640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5248067b20 0x7f5248069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:07.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.432+0000 7f526ce00640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5248067b20 0x7f5248069fe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f52540ad000 tx=0x7f5254008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:07.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.432+0000 7f5265600640 1 -- 172.21.15.86:0/2792135821 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f525c056ac0 con 0x7f526806dca0 2024-03-22T04:19:07.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:07.592+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f52680d09f0 con 0x7f526806dca0 2024-03-22T04:19:08.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.260+0000 7f5265600640 1 -- 172.21.15.86:0/2792135821 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7f525c05ab60 con 0x7f526806dca0 2024-03-22T04:19:08.262 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T04:19:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5248067b20 msgr2=0x7f5248069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5248067b20 0x7f5248069fe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f52540ad000 tx=0x7f5254008040 comp rx=0 tx=0).stop 2024-03-22T04:19:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526806dca0 msgr2=0x7f5268145510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526806dca0 0x7f5268145510 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f525c007ef0 tx=0x7f525c00e4b0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 shutdown_connections 2024-03-22T04:19:08.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5248067b20 0x7f5248069fe0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52680cfa70 0x7f5268155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52680b1cd0 0x7f5268072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 --2- 172.21.15.86:0/2792135821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f526806dca0 0x7f5268145510 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 >> 172.21.15.86:0/2792135821 conn(0x7f526806b660 msgr2=0x7f52680b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:08.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 shutdown_connections 2024-03-22T04:19:08.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.264+0000 7f5270200640 1 -- 172.21.15.86:0/2792135821 wait complete. 2024-03-22T04:19:08.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-22T04:19:08.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.428+0000 7f3b1ca00640 1 Processor -- start 2024-03-22T04:19:08.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b1ca00640 1 -- start start 2024-03-22T04:19:08.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b1ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b18147d20 0x7f3b1806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b1ca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b1806bba0 0x7f3b1806fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b1ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b18070770 con 0x7f3b1806ff80 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b1ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3b1806c010 con 0x7f3b18147d20 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b1ca00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b18148100 con 0x7f3b1806bba0 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b15e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b18147d20 0x7f3b1806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b15e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b18147d20 0x7f3b1806b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55992/0 (socket says 172.21.15.86:55992) 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b15e00640 1 -- 172.21.15.86:0/2562907268 learned_addr learned my addr 172.21.15.86:0/2562907268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b16800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3b1806ff80 0x7f3b18161f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:56702/0 (socket says 172.21.15.86:56702) 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b15400640 1 --2- 172.21.15.86:0/2562907268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b1806bba0 0x7f3b1806fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b15e00640 1 -- 172.21.15.86:0/2562907268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b1806bba0 msgr2=0x7f3b1806fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b15e00640 1 --2- 172.21.15.86:0/2562907268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b1806bba0 0x7f3b1806fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b15e00640 1 -- 172.21.15.86:0/2562907268 >> v1:172.21.15.86:6789/0 conn(0x7f3b1806ff80 legacy=0x7f3b18161f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b15e00640 1 -- 172.21.15.86:0/2562907268 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b18073470 con 0x7f3b18147d20 2024-03-22T04:19:08.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.432+0000 7f3b14a00640 1 -- 172.21.15.86:0/2562907268 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800465896 0 0) 0x7f3b18070770 con 0x7f3b1806ff80 2024-03-22T04:19:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b15e00640 1 --2- 172.21.15.86:0/2562907268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18147d20 0x7f3b1806b660 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa61592310d976e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b14a00640 1 -- 172.21.15.86:0/2562907268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b080a57f0 con 0x7f3b18147d20 2024-03-22T04:19:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b14a00640 1 -- 172.21.15.86:0/2562907268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b080a5990 con 0x7f3b18147d20 2024-03-22T04:19:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b14a00640 1 -- 172.21.15.86:0/2562907268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b080a5d60 con 0x7f3b18147d20 2024-03-22T04:19:08.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 -- 172.21.15.86:0/2562907268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18147d20 msgr2=0x7f3b1806b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/2562907268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18147d20 0x7f3b1806b660 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0801ca10 comp rx=0 tx=0).stop 2024-03-22T04:19:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 -- 172.21.15.86:0/2562907268 shutdown_connections 2024-03-22T04:19:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/2562907268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b1806bba0 0x7f3b1806fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/2562907268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b18147d20 0x7f3b1806b660 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 -- 172.21.15.86:0/2562907268 >> 172.21.15.86:0/2562907268 conn(0x7f3b180d0fa0 msgr2=0x7f3b180d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 -- 172.21.15.86:0/2562907268 shutdown_connections 2024-03-22T04:19:08.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 -- 172.21.15.86:0/2562907268 wait complete. 2024-03-22T04:19:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 Processor -- start 2024-03-22T04:19:08.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.436+0000 7f3b1ca00640 1 -- start start 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b1806bba0 0x7f3b180b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1806ff80 0x7f3b180b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b18147d20 0x7f3b1813c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b180ce260 con 0x7f3b1806ff80 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b180cdf80 con 0x7f3b1806bba0 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b180703d0 con 0x7f3b18147d20 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1806ff80 0x7f3b180b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1806ff80 0x7f3b180b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56000/0 (socket says 172.21.15.86:56000) 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 -- 172.21.15.86:0/1511603261 learned_addr learned my addr 172.21.15.86:0/1511603261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b16800640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b18147d20 0x7f3b1813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15e00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b1806bba0 0x7f3b180b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 -- 172.21.15.86:0/1511603261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b18147d20 msgr2=0x7f3b1813c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b18147d20 0x7f3b1813c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 -- 172.21.15.86:0/1511603261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b1806bba0 msgr2=0x7f3b180b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b1806bba0 0x7f3b180b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 -- 172.21.15.86:0/1511603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b181f7240 con 0x7f3b1806ff80 2024-03-22T04:19:08.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b15400640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1806ff80 0x7f3b180b2030 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400d800 tx=0x7f3b0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b0e000640 1 -- 172.21.15.86:0/1511603261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400b6c0 con 0x7f3b1806ff80 2024-03-22T04:19:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b0e000640 1 -- 172.21.15.86:0/1511603261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04019070 con 0x7f3b1806ff80 2024-03-22T04:19:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b181f7520 con 0x7f3b1806ff80 2024-03-22T04:19:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b0e000640 1 -- 172.21.15.86:0/1511603261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400ba60 con 0x7f3b1806ff80 2024-03-22T04:19:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b181f7ae0 con 0x7f3b1806ff80 2024-03-22T04:19:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b18074470 con 0x7f3b1806ff80 2024-03-22T04:19:08.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.440+0000 7f3b0e000640 1 -- 172.21.15.86:0/1511603261 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3b0400bc00 con 0x7f3b1806ff80 2024-03-22T04:19:08.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.444+0000 7f3b0e000640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3af8067b20 0x7f3af8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.444+0000 7f3b0e000640 1 -- 172.21.15.86:0/1511603261 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5507+0+0 (secure 0 0 0) 0x7f3b040863a0 con 0x7f3b1806ff80 2024-03-22T04:19:08.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.444+0000 7f3b15e00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3af8067b20 0x7f3af8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.444+0000 7f3b15e00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3af8067b20 0x7f3af8069fe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3b080ae000 tx=0x7f3b080a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:08.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.448+0000 7f3b0e000640 1 -- 172.21.15.86:0/1511603261 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3b04056dd0 con 0x7f3b1806ff80 2024-03-22T04:19:08.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.600+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3b181690f0 con 0x7f3b1806ff80 2024-03-22T04:19:08.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.604+0000 7f3b0e000640 1 -- 172.21.15.86:0/1511603261 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f3b040566d0 con 0x7f3b1806ff80 2024-03-22T04:19:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3af8067b20 msgr2=0x7f3af8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3af8067b20 0x7f3af8069fe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3b080ae000 tx=0x7f3b080a7070 comp rx=0 tx=0).stop 2024-03-22T04:19:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1806ff80 msgr2=0x7f3b180b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1806ff80 0x7f3b180b2030 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400d800 tx=0x7f3b0400dcc0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 shutdown_connections 2024-03-22T04:19:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3af8067b20 0x7f3af8069fe0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b18147d20 0x7f3b1813c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1806ff80 0x7f3b180b2030 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 --2- 172.21.15.86:0/1511603261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b1806bba0 0x7f3b180b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 >> 172.21.15.86:0/1511603261 conn(0x7f3b180d0fa0 msgr2=0x7f3b180cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 shutdown_connections 2024-03-22T04:19:08.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.608+0000 7f3b1ca00640 1 -- 172.21.15.86:0/1511603261 wait complete. 2024-03-22T04:19:08.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-22T04:19:08.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.768+0000 7f27f7200640 1 Processor -- start 2024-03-22T04:19:08.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.768+0000 7f27f7200640 1 -- start start 2024-03-22T04:19:08.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f7200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f27f00730d0 0x7f27f00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f7200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f00cc3e0 0x7f27f00ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f7200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f0071920 con 0x7f27f015b3f0 2024-03-22T04:19:08.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f7200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f27f00716d0 con 0x7f27f00730d0 2024-03-22T04:19:08.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f7200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f27f006d990 con 0x7f27f00cc3e0 2024-03-22T04:19:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f5200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f00cc3e0 0x7f27f00ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27e7400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f27f00730d0 0x7f27f00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f5200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f00cc3e0 0x7f27f00ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:49894/0 (socket says 172.21.15.86:49894) 2024-03-22T04:19:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f5200640 1 -- 172.21.15.86:0/3651853716 learned_addr learned my addr 172.21.15.86:0/3651853716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27efe00640 1 --1- 172.21.15.86:0/3651853716 >> v1:172.21.15.196:6789/0 conn(0x7f27f015b3f0 0x7f27f00729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52206/0 (socket says 172.21.15.86:52206) 2024-03-22T04:19:08.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f5200640 1 -- 172.21.15.86:0/3651853716 >> v2:172.21.15.196:3300/0 conn(0x7f27f00730d0 msgr2=0x7f27f00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f5200640 1 --2- 172.21.15.86:0/3651853716 >> v2:172.21.15.196:3300/0 conn(0x7f27f00730d0 0x7f27f00ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f5200640 1 -- 172.21.15.86:0/3651853716 >> v1:172.21.15.196:6789/0 conn(0x7f27f015b3f0 legacy=0x7f27f00729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27f5200640 1 -- 172.21.15.86:0/3651853716 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f0071c00 con 0x7f27f00cc3e0 2024-03-22T04:19:08.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.772+0000 7f27e7400640 1 --2- 172.21.15.86:0/3651853716 >> v2:172.21.15.196:3300/0 conn(0x7f27f00730d0 0x7f27f00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f5200640 1 --2- 172.21.15.86:0/3651853716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f00cc3e0 0x7f27f00ae370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009e50 tx=0x7f27e801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ffd1a19c3ba514 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27ef400640 1 -- 172.21.15.86:0/3651853716 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80a57f0 con 0x7f27f00cc3e0 2024-03-22T04:19:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27ef400640 1 -- 172.21.15.86:0/3651853716 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e80a5990 con 0x7f27f00cc3e0 2024-03-22T04:19:08.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27ef400640 1 -- 172.21.15.86:0/3651853716 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80a5d60 con 0x7f27f00cc3e0 2024-03-22T04:19:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- 172.21.15.86:0/3651853716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f00cc3e0 msgr2=0x7f27f00ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 --2- 172.21.15.86:0/3651853716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f00cc3e0 0x7f27f00ae370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009e50 tx=0x7f27e801ca30 comp rx=0 tx=0).stop 2024-03-22T04:19:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- 172.21.15.86:0/3651853716 shutdown_connections 2024-03-22T04:19:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 --2- 172.21.15.86:0/3651853716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f00cc3e0 0x7f27f00ae370 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 --2- 172.21.15.86:0/3651853716 >> v2:172.21.15.196:3300/0 conn(0x7f27f00730d0 0x7f27f00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- 172.21.15.86:0/3651853716 >> 172.21.15.86:0/3651853716 conn(0x7f27f00b1df0 msgr2=0x7f27f00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- 172.21.15.86:0/3651853716 shutdown_connections 2024-03-22T04:19:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- 172.21.15.86:0/3651853716 wait complete. 2024-03-22T04:19:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 Processor -- start 2024-03-22T04:19:08.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- start start 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f00730d0 0x7f27f00b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f00cc3e0 0x7f27f015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f015b3f0 0x7f27f015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0166570 con 0x7f27f00cc3e0 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01663f0 con 0x7f27f00730d0 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.776+0000 7f27f7200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f27f01666f0 con 0x7f27f015b3f0 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27efe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f00730d0 0x7f27f00b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f00cc3e0 0x7f27f015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f00cc3e0 0x7f27f015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56012/0 (socket says 172.21.15.86:56012) 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 -- 172.21.15.86:0/3479021235 learned_addr learned my addr 172.21.15.86:0/3479021235 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27f5200640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f015b3f0 0x7f27f015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 -- 172.21.15.86:0/3479021235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f015b3f0 msgr2=0x7f27f015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f015b3f0 0x7f27f015da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 -- 172.21.15.86:0/3479021235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f00730d0 msgr2=0x7f27f00b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f00730d0 0x7f27f00b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 -- 172.21.15.86:0/3479021235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f015b7d0 con 0x7f27f00cc3e0 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27e7400640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f00cc3e0 0x7f27f015d4c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ec10 tx=0x7f27d800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27ecc00640 1 -- 172.21.15.86:0/3479021235 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8017070 con 0x7f27f00cc3e0 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27ecc00640 1 -- 172.21.15.86:0/3479021235 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d800cc30 con 0x7f27f00cc3e0 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27ecc00640 1 -- 172.21.15.86:0/3479021235 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8013470 con 0x7f27f00cc3e0 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f015bab0 con 0x7f27f00cc3e0 2024-03-22T04:19:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f015c070 con 0x7f27f00cc3e0 2024-03-22T04:19:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f0068650 con 0x7f27f00cc3e0 2024-03-22T04:19:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.780+0000 7f27ecc00640 1 -- 172.21.15.86:0/3479021235 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f27d801a070 con 0x7f27f00cc3e0 2024-03-22T04:19:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.784+0000 7f27ecc00640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27cc067b20 0x7f27cc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.784+0000 7f27ecc00640 1 -- 172.21.15.86:0/3479021235 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5507+0+0 (secure 0 0 0) 0x7f27d8086110 con 0x7f27f00cc3e0 2024-03-22T04:19:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.784+0000 7f27efe00640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27cc067b20 0x7f27cc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.784+0000 7f27efe00640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27cc067b20 0x7f27cc069fe0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f27e0000f70 tx=0x7f27e0007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:08.788+0000 7f27ecc00640 1 -- 172.21.15.86:0/3479021235 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f27d8056590 con 0x7f27f00cc3e0 2024-03-22T04:19:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.000+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f27f006f2c0 con 0x7f27f00cc3e0 2024-03-22T04:19:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27ecc00640 1 -- 172.21.15.86:0/3479021235 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v425) ==== 167+0+0 (secure 0 0 0) 0x7f27d805a630 con 0x7f27f00cc3e0 2024-03-22T04:19:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27cc067b20 msgr2=0x7f27cc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27cc067b20 0x7f27cc069fe0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f27e0000f70 tx=0x7f27e0007e90 comp rx=0 tx=0).stop 2024-03-22T04:19:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f00cc3e0 msgr2=0x7f27f015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f00cc3e0 0x7f27f015d4c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ec10 tx=0x7f27d800c520 comp rx=0 tx=0).stop 2024-03-22T04:19:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 shutdown_connections 2024-03-22T04:19:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f27cc067b20 0x7f27cc069fe0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27f015b3f0 0x7f27f015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f00cc3e0 0x7f27f015d4c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 --2- 172.21.15.86:0/3479021235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f00730d0 0x7f27f00b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 >> 172.21.15.86:0/3479021235 conn(0x7f27f00b1df0 msgr2=0x7f27f01542b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:09.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 shutdown_connections 2024-03-22T04:19:09.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.264+0000 7f27f7200640 1 -- 172.21.15.86:0/3479021235 wait complete. 2024-03-22T04:19:09.284 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-22T04:19:09.285 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-22T04:19:09.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f5639e00640 1 Processor -- start 2024-03-22T04:19:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f5639e00640 1 -- start start 2024-03-22T04:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f5639e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f563406ef40 0x7f5634142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f5639e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f56340ad710 0x7f563415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f5639e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563415c7c0 con 0x7f56340ad330 2024-03-22T04:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f5639e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f563406dea0 con 0x7f563406ef40 2024-03-22T04:19:09.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f5639e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f563406e180 con 0x7f56340ad710 2024-03-22T04:19:09.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f562a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f563406ef40 0x7f5634142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f562a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f563406ef40 0x7f5634142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56018/0 (socket says 172.21.15.86:56018) 2024-03-22T04:19:09.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f562a000640 1 -- 172.21.15.86:0/1304564395 learned_addr learned my addr 172.21.15.86:0/1304564395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:09.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.420+0000 7f5633400640 1 --2- 172.21.15.86:0/1304564395 >> v2:172.21.15.196:3300/0 conn(0x7f56340ad710 0x7f563415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f562a000640 1 -- 172.21.15.86:0/1304564395 >> v2:172.21.15.196:3300/0 conn(0x7f56340ad710 msgr2=0x7f563415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:09.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f562a000640 1 --2- 172.21.15.86:0/1304564395 >> v2:172.21.15.196:3300/0 conn(0x7f56340ad710 0x7f563415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f562a000640 1 -- 172.21.15.86:0/1304564395 >> v1:172.21.15.196:6789/0 conn(0x7f56340ad330 legacy=0x7f563413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f562a000640 1 -- 172.21.15.86:0/1304564395 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563406e460 con 0x7f563406ef40 2024-03-22T04:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5633400640 1 --2- 172.21.15.86:0/1304564395 >> v2:172.21.15.196:3300/0 conn(0x7f56340ad710 0x7f563415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f562a000640 1 --2- 172.21.15.86:0/1304564395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563406ef40 0x7f5634142690 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f56340aed10 tx=0x7f561c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a0b773a656c611 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5632000640 1 -- 172.21.15.86:0/1304564395 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0aa650 con 0x7f563406ef40 2024-03-22T04:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5632000640 1 -- 172.21.15.86:0/1304564395 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561c0af070 con 0x7f563406ef40 2024-03-22T04:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5632000640 1 -- 172.21.15.86:0/1304564395 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0aaa10 con 0x7f563406ef40 2024-03-22T04:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5639e00640 1 -- 172.21.15.86:0/1304564395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563406ef40 msgr2=0x7f5634142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5639e00640 1 --2- 172.21.15.86:0/1304564395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563406ef40 0x7f5634142690 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f56340aed10 tx=0x7f561c01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5639e00640 1 -- 172.21.15.86:0/1304564395 shutdown_connections 2024-03-22T04:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5639e00640 1 --2- 172.21.15.86:0/1304564395 >> v2:172.21.15.196:3300/0 conn(0x7f56340ad710 0x7f563415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5639e00640 1 --2- 172.21.15.86:0/1304564395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563406ef40 0x7f5634142690 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5639e00640 1 -- 172.21.15.86:0/1304564395 >> 172.21.15.86:0/1304564395 conn(0x7f56340b1df0 msgr2=0x7f56340b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5639e00640 1 -- 172.21.15.86:0/1304564395 shutdown_connections 2024-03-22T04:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.424+0000 7f5639e00640 1 -- 172.21.15.86:0/1304564395 wait complete. 2024-03-22T04:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 Processor -- start 2024-03-22T04:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 -- start start 2024-03-22T04:19:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f563406ef40 0x7f563409dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56340ad330 0x7f56340999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56340ad710 0x7f5634165cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56341690e0 con 0x7f56340ad330 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5634168f60 con 0x7f563406ef40 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5634169260 con 0x7f56340ad710 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56340ad330 0x7f56340999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56340ad330 0x7f56340999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56034/0 (socket says 172.21.15.86:56034) 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 -- 172.21.15.86:0/2614906381 learned_addr learned my addr 172.21.15.86:0/2614906381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5633400640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56340ad710 0x7f5634165cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5632a00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f563406ef40 0x7f563409dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 -- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56340ad710 msgr2=0x7f5634165cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56340ad710 0x7f5634165cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 -- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f563406ef40 msgr2=0x7f563409dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f563406ef40 0x7f563409dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 -- 172.21.15.86:0/2614906381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5634099ef0 con 0x7f56340ad330 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5633400640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56340ad710 0x7f5634165cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5632a00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f563406ef40 0x7f563409dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562a000640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56340ad330 0x7f56340999b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f561c019100 tx=0x7f561c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562be00640 1 -- 172.21.15.86:0/2614906381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0bd5f0 con 0x7f56340ad330 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562be00640 1 -- 172.21.15.86:0/2614906381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561c0a6040 con 0x7f56340ad330 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5634166200 con 0x7f56340ad330 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.428+0000 7f562be00640 1 -- 172.21.15.86:0/2614906381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0bd9d0 con 0x7f56340ad330 2024-03-22T04:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.432+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5634166780 con 0x7f56340ad330 2024-03-22T04:19:09.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.432+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634068650 con 0x7f56340ad330 2024-03-22T04:19:09.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.432+0000 7f562be00640 1 -- 172.21.15.86:0/2614906381 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f561c0b33f0 con 0x7f56340ad330 2024-03-22T04:19:09.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.432+0000 7f562be00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5610067b20 0x7f5610069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.432+0000 7f562be00640 1 -- 172.21.15.86:0/2614906381 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5507+0+0 (secure 0 0 0) 0x7f561c120cd0 con 0x7f56340ad330 2024-03-22T04:19:09.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.432+0000 7f5632a00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5610067b20 0x7f5610069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.432+0000 7f5632a00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5610067b20 0x7f5610069fe0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f56240012f0 tx=0x7f5624009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:09.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.436+0000 7f562be00640 1 -- 172.21.15.86:0/2614906381 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f561c0c3060 con 0x7f56340ad330 2024-03-22T04:19:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.592+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f563415d180 con 0x7f56340ad330 2024-03-22T04:19:09.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f562be00640 1 -- 172.21.15.86:0/2614906381 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+13561 (secure 0 0 0) 0x7f561c016110 con 0x7f56340ad330 2024-03-22T04:19:09.595 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":100,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:19:08.257589+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1227183960":"2024-03-23T04:19:05.242869+0000","172.21.15.196:6834/1227183960":"2024-03-23T04:19:05.242869+0000","172.21.15.196:6835/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6834/1851339627":"2024-03-23T04:18:21.162930+0000","172.21.15.196:6835/1097103199":"2024-03-23T04:17:08.939658+0000","172.21.15.196:6834/1097103199":"2024-03-23T04:17:08.939658+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5610067b20 msgr2=0x7f5610069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5610067b20 0x7f5610069fe0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f56240012f0 tx=0x7f5624009040 comp rx=0 tx=0).stop 2024-03-22T04:19:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56340ad330 msgr2=0x7f56340999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56340ad330 0x7f56340999b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f561c019100 tx=0x7f561c01ad30 comp rx=0 tx=0).stop 2024-03-22T04:19:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 shutdown_connections 2024-03-22T04:19:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5610067b20 0x7f5610069fe0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56340ad710 0x7f5634165cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56340ad330 0x7f56340999b0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 --2- 172.21.15.86:0/2614906381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f563406ef40 0x7f563409dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 >> 172.21.15.86:0/2614906381 conn(0x7f56340b1df0 msgr2=0x7f56340731a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:09.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 shutdown_connections 2024-03-22T04:19:09.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.596+0000 7f5639e00640 1 -- 172.21.15.86:0/2614906381 wait complete. 2024-03-22T04:19:09.616 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:19:09.616 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:19:09.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e45000640 1 Processor -- start 2024-03-22T04:19:09.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e45000640 1 -- start start 2024-03-22T04:19:09.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e45000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e4015d0b0 0x7f9e401671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e45000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4015d710 con 0x7f9e400b2360 2024-03-22T04:19:09.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e45000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4015fe40 con 0x7f9e400b3600 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e45000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9e400b2740 con 0x7f9e4015d0b0 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9e400b2360 0x7f9e400b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:56710/0 (socket says 172.21.15.86:56710) 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3de00640 1 -- 172.21.15.86:0/3389851894 learned_addr learned my addr 172.21.15.86:0/3389851894 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3e800640 1 --2- 172.21.15.86:0/3389851894 >> v2:172.21.15.196:3300/0 conn(0x7f9e4015d0b0 0x7f9e401671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3d400640 1 -- 172.21.15.86:0/3389851894 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237701561 0 0) 0x7f9e4015d710 con 0x7f9e400b2360 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3d400640 1 -- 172.21.15.86:0/3389851894 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e24003530 con 0x7f9e400b2360 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3d400640 1 -- 172.21.15.86:0/3389851894 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323154874 0 0) 0x7f9e4015fe40 con 0x7f9e400b3600 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3d400640 1 -- 172.21.15.86:0/3389851894 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e4015d710 con 0x7f9e400b3600 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3d400640 1 -- 172.21.15.86:0/3389851894 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2542619403 0 0) 0x7f9e24003530 con 0x7f9e400b2360 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3e800640 1 -- 172.21.15.86:0/3389851894 >> v1:172.21.15.196:6789/0 conn(0x7f9e400b3600 legacy=0x7f9e4015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3e800640 1 -- 172.21.15.86:0/3389851894 >> v1:172.21.15.86:6789/0 conn(0x7f9e400b2360 legacy=0x7f9e400b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.756+0000 7f9e3e800640 1 -- 172.21.15.86:0/3389851894 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e40160120 con 0x7f9e4015d0b0 2024-03-22T04:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e3e800640 1 --2- 172.21.15.86:0/3389851894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 0x7f9e401671d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009d90 tx=0x7f9e3801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e11e1e937c5d86f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e3d400640 1 -- 172.21.15.86:0/3389851894 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e380b2070 con 0x7f9e4015d0b0 2024-03-22T04:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e3d400640 1 -- 172.21.15.86:0/3389851894 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e380a55c0 con 0x7f9e4015d0b0 2024-03-22T04:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e3d400640 1 -- 172.21.15.86:0/3389851894 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30001680 con 0x7f9e4015d0b0 2024-03-22T04:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e45000640 1 -- 172.21.15.86:0/3389851894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 msgr2=0x7f9e401671d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e45000640 1 --2- 172.21.15.86:0/3389851894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 0x7f9e401671d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009d90 tx=0x7f9e3801c890 comp rx=0 tx=0).stop 2024-03-22T04:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e45000640 1 -- 172.21.15.86:0/3389851894 shutdown_connections 2024-03-22T04:19:09.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e45000640 1 --2- 172.21.15.86:0/3389851894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 0x7f9e401671d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e45000640 1 -- 172.21.15.86:0/3389851894 >> 172.21.15.86:0/3389851894 conn(0x7f9e400cc940 msgr2=0x7f9e400ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:09.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e45000640 1 -- 172.21.15.86:0/3389851894 shutdown_connections 2024-03-22T04:19:09.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.760+0000 7f9e45000640 1 -- 172.21.15.86:0/3389851894 wait complete. 2024-03-22T04:19:09.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 Processor -- start 2024-03-22T04:19:09.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 -- start start 2024-03-22T04:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e400b2360 0x7f9e40148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e400b3600 0x7f9e4014dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 0x7f9e40145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40160c00 con 0x7f9e400b3600 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40169f30 con 0x7f9e4015d0b0 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e40146330 con 0x7f9e400b2360 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e400b3600 0x7f9e4014dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e400b3600 0x7f9e4014dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56040/0 (socket says 172.21.15.86:56040) 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 -- 172.21.15.86:0/531835234 learned_addr learned my addr 172.21.15.86:0/531835234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e3de00640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e400b2360 0x7f9e40148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e3e800640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 0x7f9e40145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 -- 172.21.15.86:0/531835234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e400b2360 msgr2=0x7f9e40148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e400b2360 0x7f9e40148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 -- 172.21.15.86:0/531835234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 msgr2=0x7f9e40145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 0x7f9e40145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 -- 172.21.15.86:0/531835234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e400ceef0 con 0x7f9e400b3600 2024-03-22T04:19:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e35400640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e400b3600 0x7f9e4014dc60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800c490 tx=0x7f9e2800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e36a00640 1 -- 172.21.15.86:0/531835234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e280052d0 con 0x7f9e400b3600 2024-03-22T04:19:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e36a00640 1 -- 172.21.15.86:0/531835234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e28005470 con 0x7f9e400b3600 2024-03-22T04:19:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4014e390 con 0x7f9e400b3600 2024-03-22T04:19:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e36a00640 1 -- 172.21.15.86:0/531835234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28012870 con 0x7f9e400b3600 2024-03-22T04:19:09.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4014c990 con 0x7f9e400b3600 2024-03-22T04:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.764+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e40074470 con 0x7f9e400b3600 2024-03-22T04:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.768+0000 7f9e36a00640 1 -- 172.21.15.86:0/531835234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9e28012a10 con 0x7f9e400b3600 2024-03-22T04:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.768+0000 7f9e36a00640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e1c067b20 0x7f9e1c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.768+0000 7f9e36a00640 1 -- 172.21.15.86:0/531835234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5507+0+0 (secure 0 0 0) 0x7f9e280859f0 con 0x7f9e400b3600 2024-03-22T04:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.768+0000 7f9e3de00640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e1c067b20 0x7f9e1c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.768+0000 7f9e3de00640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e1c067b20 0x7f9e1c069fe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9e300011f0 tx=0x7f9e30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.772+0000 7f9e36a00640 1 -- 172.21.15.86:0/531835234 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9e28056550 con 0x7f9e400b3600 2024-03-22T04:19:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.928+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e40160d80 con 0x7f9e400b3600 2024-03-22T04:19:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.928+0000 7f9e36a00640 1 -- 172.21.15.86:0/531835234 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f9e28055e50 con 0x7f9e400b3600 2024-03-22T04:19:09.931 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:19:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e1c067b20 msgr2=0x7f9e1c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e1c067b20 0x7f9e1c069fe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9e300011f0 tx=0x7f9e30009040 comp rx=0 tx=0).stop 2024-03-22T04:19:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e400b3600 msgr2=0x7f9e4014dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e400b3600 0x7f9e4014dc60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800c490 tx=0x7f9e2800c950 comp rx=0 tx=0).stop 2024-03-22T04:19:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 shutdown_connections 2024-03-22T04:19:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9e1c067b20 0x7f9e1c069fe0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e4015d0b0 0x7f9e40145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e400b3600 0x7f9e4014dc60 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 --2- 172.21.15.86:0/531835234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e400b2360 0x7f9e40148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 >> 172.21.15.86:0/531835234 conn(0x7f9e400cc940 msgr2=0x7f9e400b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 shutdown_connections 2024-03-22T04:19:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:09.932+0000 7f9e45000640 1 -- 172.21.15.86:0/531835234 wait complete. 2024-03-22T04:19:09.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-22T04:19:10.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85ba600640 1 Processor -- start 2024-03-22T04:19:10.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85ba600640 1 -- start start 2024-03-22T04:19:10.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85ba600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:10.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85ba600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f85b406ef40 0x7f85b406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:10.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85ba600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b415c7c0 con 0x7f85b40ad710 2024-03-22T04:19:10.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85ba600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f85b406dc40 con 0x7f85b406ef40 2024-03-22T04:19:10.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85ba600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f85b406dea0 con 0x7f85b40ad330 2024-03-22T04:19:10.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f85b406ef40 0x7f85b406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:10.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b3400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:10.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f85b406ef40 0x7f85b406d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:34128/0 (socket says 172.21.15.86:34128) 2024-03-22T04:19:10.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b2a00640 1 -- 172.21.15.86:0/3407713046 learned_addr learned my addr 172.21.15.86:0/3407713046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:10.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b2a00640 1 -- 172.21.15.86:0/3407713046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 msgr2=0x7f85b415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:10.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b2a00640 1 --2- 172.21.15.86:0/3407713046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:10.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b2a00640 1 -- 172.21.15.86:0/3407713046 >> v1:172.21.15.196:6789/0 conn(0x7f85b40ad710 legacy=0x7f85b413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:10.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b2a00640 1 -- 172.21.15.86:0/3407713046 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b406e180 con 0x7f85b406ef40 2024-03-22T04:19:10.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b2000640 1 -- 172.21.15.86:0/3407713046 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2801090163 0 0) 0x7f85b415c7c0 con 0x7f85b40ad710 2024-03-22T04:19:10.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.092+0000 7f85b3400640 1 --2- 172.21.15.86:0/3407713046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:10.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85b2a00640 1 --2- 172.21.15.86:0/3407713046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b406ef40 0x7f85b406d700 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009870 tx=0x7f85a001c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb84d56a8d9ef401 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:10.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85b2000640 1 -- 172.21.15.86:0/3407713046 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00ab640 con 0x7f85b406ef40 2024-03-22T04:19:10.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85b2000640 1 -- 172.21.15.86:0/3407713046 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a00b0070 con 0x7f85b406ef40 2024-03-22T04:19:10.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85b2000640 1 -- 172.21.15.86:0/3407713046 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00aba00 con 0x7f85b406ef40 2024-03-22T04:19:10.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85ba600640 1 -- 172.21.15.86:0/3407713046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b406ef40 msgr2=0x7f85b406d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:10.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85ba600640 1 --2- 172.21.15.86:0/3407713046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b406ef40 0x7f85b406d700 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009870 tx=0x7f85a001c530 comp rx=0 tx=0).stop 2024-03-22T04:19:10.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85ba600640 1 -- 172.21.15.86:0/3407713046 shutdown_connections 2024-03-22T04:19:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85ba600640 1 --2- 172.21.15.86:0/3407713046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b406ef40 0x7f85b406d700 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85ba600640 1 --2- 172.21.15.86:0/3407713046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85ba600640 1 -- 172.21.15.86:0/3407713046 >> 172.21.15.86:0/3407713046 conn(0x7f85b40b1df0 msgr2=0x7f85b40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85ba600640 1 -- 172.21.15.86:0/3407713046 shutdown_connections 2024-03-22T04:19:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.096+0000 7f85ba600640 1 -- 172.21.15.86:0/3407713046 wait complete. 2024-03-22T04:19:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 Processor -- start 2024-03-22T04:19:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 -- start start 2024-03-22T04:19:10.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b406ef40 0x7f85b4143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:10.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b4099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b40ad710 0x7f85b409a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85b41690e0 con 0x7f85b406ef40 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4168f60 con 0x7f85b40ad710 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f85b4169260 con 0x7f85b40ad330 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b406ef40 0x7f85b4143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b406ef40 0x7f85b4143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56044/0 (socket says 172.21.15.86:56044) 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 -- 172.21.15.86:0/104425258 learned_addr learned my addr 172.21.15.86:0/104425258 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3e00640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b40ad710 0x7f85b409a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b2a00640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b4099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 -- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 msgr2=0x7f85b4099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b4099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 -- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b40ad710 msgr2=0x7f85b409a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b40ad710 0x7f85b409a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:10.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 -- 172.21.15.86:0/104425258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b4144280 con 0x7f85b406ef40 2024-03-22T04:19:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b2a00640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b4099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3e00640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b40ad710 0x7f85b409a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85b3400640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b406ef40 0x7f85b4143c90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e980 tx=0x7f85a400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ab400640 1 -- 172.21.15.86:0/104425258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4017070 con 0x7f85b406ef40 2024-03-22T04:19:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ab400640 1 -- 172.21.15.86:0/104425258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a400ca00 con 0x7f85b406ef40 2024-03-22T04:19:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b4144560 con 0x7f85b406ef40 2024-03-22T04:19:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ab400640 1 -- 172.21.15.86:0/104425258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a400cdd0 con 0x7f85b406ef40 2024-03-22T04:19:10.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b4144b20 con 0x7f85b406ef40 2024-03-22T04:19:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.100+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b4068650 con 0x7f85b406ef40 2024-03-22T04:19:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.104+0000 7f85ab400640 1 -- 172.21.15.86:0/104425258 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f85a401a070 con 0x7f85b406ef40 2024-03-22T04:19:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.104+0000 7f85ab400640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8594067b50 0x7f859406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.104+0000 7f85ab400640 1 -- 172.21.15.86:0/104425258 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5507+0+0 (secure 0 0 0) 0x7f85a4086160 con 0x7f85b406ef40 2024-03-22T04:19:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.104+0000 7f85b2a00640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8594067b50 0x7f859406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:10.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.104+0000 7f85b2a00640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8594067b50 0x7f859406a010 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f85a00af000 tx=0x7f85a00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.108+0000 7f85ab400640 1 -- 172.21.15.86:0/104425258 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f85a4056b90 con 0x7f85b406ef40 2024-03-22T04:19:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:10.320+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f85b415d6c0 con 0x7f85b406ef40 2024-03-22T04:19:11.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ab400640 1 -- 172.21.15.86:0/104425258 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v427) ==== 149+0+0 (secure 0 0 0) 0x7f85a4056490 con 0x7f85b406ef40 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8594067b50 msgr2=0x7f859406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8594067b50 0x7f859406a010 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f85a00af000 tx=0x7f85a00a7000 comp rx=0 tx=0).stop 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b406ef40 msgr2=0x7f85b4143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b406ef40 0x7f85b4143c90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e980 tx=0x7f85a400ee40 comp rx=0 tx=0).stop 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 shutdown_connections 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8594067b50 0x7f859406a010 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85b40ad710 0x7f85b409a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85b40ad330 0x7f85b4099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 --2- 172.21.15.86:0/104425258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b406ef40 0x7f85b4143c90 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 >> 172.21.15.86:0/104425258 conn(0x7f85b40b1df0 msgr2=0x7f85b40729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 shutdown_connections 2024-03-22T04:19:11.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.272+0000 7f85ba600640 1 -- 172.21.15.86:0/104425258 wait complete. 2024-03-22T04:19:11.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-22T04:19:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.436+0000 7fcb24800640 1 Processor -- start 2024-03-22T04:19:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.436+0000 7fcb24800640 1 -- start start 2024-03-22T04:19:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb24800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b3600 0x7fcb1c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb24800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb1c15d710 con 0x7fcb1c15cee0 2024-03-22T04:19:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb24800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb1c15fe40 con 0x7fcb1c0b2360 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb24800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1c0b2740 con 0x7fcb1c0b3600 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb22800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcb1c15cee0 0x7fcb1c0b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:56712/0 (socket says 172.21.15.86:56712) 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb22800640 1 -- 172.21.15.86:0/3566233116 learned_addr learned my addr 172.21.15.86:0/3566233116 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb21400640 1 --2- 172.21.15.86:0/3566233116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b3600 0x7fcb1c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb20a00640 1 -- 172.21.15.86:0/3566233116 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066338756 0 0) 0x7fcb1c15d710 con 0x7fcb1c15cee0 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb20a00640 1 -- 172.21.15.86:0/3566233116 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb04003530 con 0x7fcb1c15cee0 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb20a00640 1 -- 172.21.15.86:0/3566233116 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418460558 0 0) 0x7fcb1c15fe40 con 0x7fcb1c0b2360 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb20a00640 1 -- 172.21.15.86:0/3566233116 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb1c15d710 con 0x7fcb1c0b2360 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb21400640 1 -- 172.21.15.86:0/3566233116 >> v1:172.21.15.196:6789/0 conn(0x7fcb1c0b2360 legacy=0x7fcb1c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb20a00640 1 -- 172.21.15.86:0/3566233116 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2874134773 0 0) 0x7fcb04003530 con 0x7fcb1c15cee0 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb21400640 1 -- 172.21.15.86:0/3566233116 >> v1:172.21.15.86:6789/0 conn(0x7fcb1c15cee0 legacy=0x7fcb1c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.440+0000 7fcb21400640 1 -- 172.21.15.86:0/3566233116 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c160120 con 0x7fcb1c0b3600 2024-03-22T04:19:11.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb21400640 1 --2- 172.21.15.86:0/3566233116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b3600 0x7fcb1c15c9a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0097f0 tx=0x7fcb0c01c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=babd0aad371eab64 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:11.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb20a00640 1 -- 172.21.15.86:0/3566233116 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0b0070 con 0x7fcb1c0b3600 2024-03-22T04:19:11.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb20a00640 1 -- 172.21.15.86:0/3566233116 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0c004030 con 0x7fcb1c0b3600 2024-03-22T04:19:11.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb20a00640 1 -- 172.21.15.86:0/3566233116 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18001a00 con 0x7fcb1c0b3600 2024-03-22T04:19:11.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 -- 172.21.15.86:0/3566233116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b3600 msgr2=0x7fcb1c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 --2- 172.21.15.86:0/3566233116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b3600 0x7fcb1c15c9a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0097f0 tx=0x7fcb0c01c3d0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 -- 172.21.15.86:0/3566233116 shutdown_connections 2024-03-22T04:19:11.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 --2- 172.21.15.86:0/3566233116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b3600 0x7fcb1c15c9a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 -- 172.21.15.86:0/3566233116 >> 172.21.15.86:0/3566233116 conn(0x7fcb1c0cc940 msgr2=0x7fcb1c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:11.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 -- 172.21.15.86:0/3566233116 shutdown_connections 2024-03-22T04:19:11.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 -- 172.21.15.86:0/3566233116 wait complete. 2024-03-22T04:19:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 Processor -- start 2024-03-22T04:19:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.444+0000 7fcb24800640 1 -- start start 2024-03-22T04:19:11.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b2360 0x7fcb1c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb1c0b3600 0x7fcb1c137b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c15cee0 0x7fcb1c155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c15d490 con 0x7fcb1c0b3600 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c15d310 con 0x7fcb1c15cee0 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1c16a1f0 con 0x7fcb1c0b2360 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb1c0b3600 0x7fcb1c137b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb1c0b3600 0x7fcb1c137b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56048/0 (socket says 172.21.15.86:56048) 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 -- 172.21.15.86:0/2895967707 learned_addr learned my addr 172.21.15.86:0/2895967707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21e00640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b2360 0x7fcb1c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb22800640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c15cee0 0x7fcb1c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 -- 172.21.15.86:0/2895967707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b2360 msgr2=0x7fcb1c14de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b2360 0x7fcb1c14de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 -- 172.21.15.86:0/2895967707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c15cee0 msgr2=0x7fcb1c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c15cee0 0x7fcb1c155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 -- 172.21.15.86:0/2895967707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c138ae0 con 0x7fcb1c0b3600 2024-03-22T04:19:11.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb21400640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb1c0b3600 0x7fcb1c137b50 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0097f0 tx=0x7fcb0c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:11.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb16000640 1 -- 172.21.15.86:0/2895967707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0b0040 con 0x7fcb1c0b3600 2024-03-22T04:19:11.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb16000640 1 -- 172.21.15.86:0/2895967707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0c0037e0 con 0x7fcb1c0b3600 2024-03-22T04:19:11.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb16000640 1 -- 172.21.15.86:0/2895967707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c003bb0 con 0x7fcb1c0b3600 2024-03-22T04:19:11.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c138520 con 0x7fcb1c0b3600 2024-03-22T04:19:11.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c14d640 con 0x7fcb1c0b3600 2024-03-22T04:19:11.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1c14d430 con 0x7fcb1c0b3600 2024-03-22T04:19:11.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.448+0000 7fcb16000640 1 -- 172.21.15.86:0/2895967707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcb0c0bd030 con 0x7fcb1c0b3600 2024-03-22T04:19:11.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.452+0000 7fcb16000640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb00067b20 0x7fcb00069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:11.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.452+0000 7fcb16000640 1 -- 172.21.15.86:0/2895967707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5507+0+0 (secure 0 0 0) 0x7fcb0c11f260 con 0x7fcb1c0b3600 2024-03-22T04:19:11.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.452+0000 7fcb21e00640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb00067b20 0x7fcb00069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:11.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.452+0000 7fcb21e00640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb00067b20 0x7fcb00069fe0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcb100011f0 tx=0x7fcb10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:11.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.456+0000 7fcb16000640 1 -- 172.21.15.86:0/2895967707 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcb0c0c4060 con 0x7fcb1c0b3600 2024-03-22T04:19:11.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.608+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcb1c074470 con 0x7fcb1c0b3600 2024-03-22T04:19:11.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb16000640 1 -- 172.21.15.86:0/2895967707 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fcb0c015ff0 con 0x7fcb1c0b3600 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb00067b20 msgr2=0x7fcb00069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb00067b20 0x7fcb00069fe0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcb100011f0 tx=0x7fcb10009040 comp rx=0 tx=0).stop 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb1c0b3600 msgr2=0x7fcb1c137b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb1c0b3600 0x7fcb1c137b50 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0097f0 tx=0x7fcb0c019170 comp rx=0 tx=0).stop 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 shutdown_connections 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcb00067b20 0x7fcb00069fe0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c15cee0 0x7fcb1c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb1c0b3600 0x7fcb1c137b50 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 --2- 172.21.15.86:0/2895967707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb1c0b2360 0x7fcb1c14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 >> 172.21.15.86:0/2895967707 conn(0x7fcb1c0cc940 msgr2=0x7fcb1c0b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 shutdown_connections 2024-03-22T04:19:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:11.616+0000 7fcb24800640 1 -- 172.21.15.86:0/2895967707 wait complete. 2024-03-22T04:19:11.637 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:19:11.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:19:11.729 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:19:11.822 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:19:11.822 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:11.822 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:11.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:19:11.832 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:19:11.832 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:11.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:19:11.841 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:11.841 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:11.882 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:19:11.882 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:19:11.882 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:11.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:19:11.891 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-22T04:19:11.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:12.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ee00640 1 Processor -- start 2024-03-22T04:19:12.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ee00640 1 -- start start 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ee00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f476813f190 0x7f47680b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f476806f4f0 0x7f476815c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ee00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476806f960 con 0x7f47680b5eb0 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4768160e10 con 0x7f476806f4f0 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ee00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4768160050 con 0x7f476813f190 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f476806f4f0 0x7f476815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f476806f4f0 0x7f476815c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56056/0 (socket says 172.21.15.86:56056) 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f476ce00640 1 -- 172.21.15.86:0/4200643278 learned_addr learned my addr 172.21.15.86:0/4200643278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.056+0000 7f4767e00640 1 --2- 172.21.15.86:0/4200643278 >> v2:172.21.15.196:3300/0 conn(0x7f476813f190 0x7f47680b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ce00640 1 -- 172.21.15.86:0/4200643278 >> v2:172.21.15.196:3300/0 conn(0x7f476813f190 msgr2=0x7f47680b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ce00640 1 --2- 172.21.15.86:0/4200643278 >> v2:172.21.15.196:3300/0 conn(0x7f476813f190 0x7f47680b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ce00640 1 -- 172.21.15.86:0/4200643278 >> v1:172.21.15.196:6789/0 conn(0x7f47680b5eb0 legacy=0x7f476806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:12.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ce00640 1 -- 172.21.15.86:0/4200643278 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4768160330 con 0x7f476806f4f0 2024-03-22T04:19:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ce00640 1 --2- 172.21.15.86:0/4200643278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f476815c890 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4768072e30 tx=0x7f476001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f61b55492094b6a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f4767400640 1 -- 172.21.15.86:0/4200643278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600b0070 con 0x7f476806f4f0 2024-03-22T04:19:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f4767400640 1 -- 172.21.15.86:0/4200643278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47600ab600 con 0x7f476806f4f0 2024-03-22T04:19:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f4767400640 1 -- 172.21.15.86:0/4200643278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600ab9d0 con 0x7f476806f4f0 2024-03-22T04:19:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ee00640 1 -- 172.21.15.86:0/4200643278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 msgr2=0x7f476815c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ee00640 1 --2- 172.21.15.86:0/4200643278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f476815c890 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4768072e30 tx=0x7f476001c450 comp rx=0 tx=0).stop 2024-03-22T04:19:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ee00640 1 -- 172.21.15.86:0/4200643278 shutdown_connections 2024-03-22T04:19:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ee00640 1 --2- 172.21.15.86:0/4200643278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f476815c890 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ee00640 1 --2- 172.21.15.86:0/4200643278 >> v2:172.21.15.196:3300/0 conn(0x7f476813f190 0x7f47680b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ee00640 1 -- 172.21.15.86:0/4200643278 >> 172.21.15.86:0/4200643278 conn(0x7f476806b660 msgr2=0x7f476806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ee00640 1 -- 172.21.15.86:0/4200643278 shutdown_connections 2024-03-22T04:19:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.060+0000 7f476ee00640 1 -- 172.21.15.86:0/4200643278 wait complete. 2024-03-22T04:19:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 Processor -- start 2024-03-22T04:19:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 -- start start 2024-03-22T04:19:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f4768139f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47680b5eb0 0x7f476813dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476813f190 0x7f4768137c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4768169270 con 0x7f476806f4f0 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f476815d7f0 con 0x7f476813f190 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f47681693f0 con 0x7f47680b5eb0 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f4768139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f4768139f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56070/0 (socket says 172.21.15.86:56070) 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 -- 172.21.15.86:0/499879572 learned_addr learned my addr 172.21.15.86:0/499879572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ce00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476813f190 0x7f4768137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f475f400640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47680b5eb0 0x7f476813dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 -- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47680b5eb0 msgr2=0x7f476813dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47680b5eb0 0x7f476813dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 -- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476813f190 msgr2=0x7f4768137c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476813f190 0x7f4768137c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 -- 172.21.15.86:0/499879572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476813a5e0 con 0x7f476806f4f0 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f475f400640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47680b5eb0 0x7f476813dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4767e00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f4768139f00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f475800d5f0 tx=0x7f475800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4764c00640 1 -- 172.21.15.86:0/499879572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47580125f0 con 0x7f476806f4f0 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4764c00640 1 -- 172.21.15.86:0/499879572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4758019070 con 0x7f476806f4f0 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476813a8c0 con 0x7f476806f4f0 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.064+0000 7f4764c00640 1 -- 172.21.15.86:0/499879572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47580129b0 con 0x7f476806f4f0 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.068+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4768138180 con 0x7f476806f4f0 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.068+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4768068650 con 0x7f476806f4f0 2024-03-22T04:19:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.068+0000 7f4764c00640 1 -- 172.21.15.86:0/499879572 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4758012b50 con 0x7f476806f4f0 2024-03-22T04:19:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.068+0000 7f4764c00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4744067b20 0x7f4744069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.068+0000 7f4764c00640 1 -- 172.21.15.86:0/499879572 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5507+0+0 (secure 0 0 0) 0x7f4758086150 con 0x7f476806f4f0 2024-03-22T04:19:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.068+0000 7f475f400640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4744067b20 0x7f4744069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.068+0000 7f475f400640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4744067b20 0x7f4744069fe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f47680d0270 tx=0x7f4750009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.072+0000 7f4764c00640 1 -- 172.21.15.86:0/499879572 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4758056c90 con 0x7f476806f4f0 2024-03-22T04:19:12.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f476806e490 con 0x7f476806f4f0 2024-03-22T04:19:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f4764c00640 1 -- 172.21.15.86:0/499879572 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+3866 (secure 0 0 0) 0x7f4758056590 con 0x7f476806f4f0 2024-03-22T04:19:12.261 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:12.261 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4054094517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4054094517},{"type":"v1","addr":"172.21.15.196:6833","nonce":4054094517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7488,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/83419932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":83419932},{"type":"v1","addr":"172.21.15.86:6836","nonce":83419932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/556835327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":556835327},{"type":"v1","addr":"172.21.15.86:6837","nonce":556835327}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":8050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/4278934706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":4278934706},{"type":"v1","addr":"172.21.15.196:6835","nonce":4278934706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[]} 2024-03-22T04:19:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 79 2024-03-22T04:19:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4744067b20 msgr2=0x7f4744069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4744067b20 0x7f4744069fe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f47680d0270 tx=0x7f4750009040 comp rx=0 tx=0).stop 2024-03-22T04:19:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 msgr2=0x7f4768139f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f4768139f00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f475800d5f0 tx=0x7f475800dab0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 shutdown_connections 2024-03-22T04:19:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4744067b20 0x7f4744069fe0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f476813f190 0x7f4768137c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47680b5eb0 0x7f476813dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 --2- 172.21.15.86:0/499879572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476806f4f0 0x7f4768139f00 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.260+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 >> 172.21.15.86:0/499879572 conn(0x7f476806b660 msgr2=0x7f476815ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.264+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 shutdown_connections 2024-03-22T04:19:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.264+0000 7f476ee00640 1 -- 172.21.15.86:0/499879572 wait complete. 2024-03-22T04:19:12.283 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-22T04:19:12.283 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-22T04:19:12.283 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-22T04:19:12.283 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:19:12.283 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-22T04:19:12.283 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-22T04:19:12.284 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-22T04:19:12.284 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:19:12.284 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-22T04:19:12.284 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-22T04:19:12.284 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-22T04:19:12.284 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:19:12.284 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-22T04:19:12.284 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-22T04:19:12.285 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-22T04:19:12.285 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:19:12.295 INFO:tasks.ceph.mds.a:Stopped 2024-03-22T04:19:12.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-22T04:19:12.295 INFO:tasks.ceph.mds.c:Stopped 2024-03-22T04:19:12.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-22T04:19:12.300 INFO:tasks.ceph.mds.b:Stopped 2024-03-22T04:19:12.300 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-22T04:19:12.301 INFO:tasks.ceph.mds.d:Stopped 2024-03-22T04:19:12.301 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-22T04:19:12.341 INFO:tasks.ceph.mds.a:Started 2024-03-22T04:19:12.342 DEBUG:teuthology.parallel:result is None 2024-03-22T04:19:12.343 INFO:tasks.ceph.mds.c:Started 2024-03-22T04:19:12.343 DEBUG:teuthology.parallel:result is None 2024-03-22T04:19:12.346 INFO:tasks.ceph.mds.b:Started 2024-03-22T04:19:12.346 DEBUG:teuthology.parallel:result is None 2024-03-22T04:19:12.347 INFO:tasks.ceph.mds.d:Started 2024-03-22T04:19:12.348 DEBUG:teuthology.parallel:result is None 2024-03-22T04:19:12.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-22T04:19:12.428 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:19:12.428 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-03-22T04:19:12.437 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:19:12.438 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:19:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb16ce00640 1 Processor -- start 2024-03-22T04:19:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb16ce00640 1 -- start start 2024-03-22T04:19:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb16ce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb16815b3f0 0x7fb1680729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 0x7fb168072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb16ce00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb168071920 con 0x7fb1680cc3e0 2024-03-22T04:19:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb16ce00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb1680716d0 con 0x7fb16815b3f0 2024-03-22T04:19:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb16ce00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb16806d990 con 0x7fb1680ae540 2024-03-22T04:19:12.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb165e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb16815b3f0 0x7fb1680729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb166800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 0x7fb168072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb165e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb16815b3f0 0x7fb1680729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:34216/0 (socket says 172.21.15.86:34216) 2024-03-22T04:19:12.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.492+0000 7fb165e00640 1 -- 172.21.15.86:0/3779668628 learned_addr learned my addr 172.21.15.86:0/3779668628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:12.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb165e00640 1 -- 172.21.15.86:0/3779668628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 msgr2=0x7fb168072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:12.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb165e00640 1 --2- 172.21.15.86:0/3779668628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 0x7fb168072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb165e00640 1 -- 172.21.15.86:0/3779668628 >> v1:172.21.15.196:6789/0 conn(0x7fb1680cc3e0 legacy=0x7fb1680ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb165e00640 1 -- 172.21.15.86:0/3779668628 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb168071c00 con 0x7fb16815b3f0 2024-03-22T04:19:12.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb165e00640 1 --2- 172.21.15.86:0/3779668628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 0x7fb1680729c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb158009e50 tx=0x7fb15801c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61db83859dff86c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:12.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb164a00640 1 -- 172.21.15.86:0/3779668628 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580a5750 con 0x7fb16815b3f0 2024-03-22T04:19:12.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb164a00640 1 -- 172.21.15.86:0/3779668628 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1580a58f0 con 0x7fb16815b3f0 2024-03-22T04:19:12.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb164a00640 1 -- 172.21.15.86:0/3779668628 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580a5cc0 con 0x7fb16815b3f0 2024-03-22T04:19:12.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb16ce00640 1 -- 172.21.15.86:0/3779668628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 msgr2=0x7fb1680729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.496+0000 7fb16ce00640 1 --2- 172.21.15.86:0/3779668628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 0x7fb1680729c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb158009e50 tx=0x7fb15801c9a0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 -- 172.21.15.86:0/3779668628 shutdown_connections 2024-03-22T04:19:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 --2- 172.21.15.86:0/3779668628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 0x7fb168072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 --2- 172.21.15.86:0/3779668628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 0x7fb1680729c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 -- 172.21.15.86:0/3779668628 >> 172.21.15.86:0/3779668628 conn(0x7fb1680b1df0 msgr2=0x7fb1680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 -- 172.21.15.86:0/3779668628 shutdown_connections 2024-03-22T04:19:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 -- 172.21.15.86:0/3779668628 wait complete. 2024-03-22T04:19:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 Processor -- start 2024-03-22T04:19:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 -- start start 2024-03-22T04:19:12.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 0x7fb168153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1680cc3e0 0x7fb1681510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 0x7fb168099760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb168168580 con 0x7fb1680cc3e0 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb168168400 con 0x7fb16815b3f0 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb16ce00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb168168700 con 0x7fb1680ae540 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1680cc3e0 0x7fb1681510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1680cc3e0 0x7fb1681510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56104/0 (socket says 172.21.15.86:56104) 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 -- 172.21.15.86:0/774627497 learned_addr learned my addr 172.21.15.86:0/774627497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb166800640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 0x7fb168099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165e00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 0x7fb168153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 -- 172.21.15.86:0/774627497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 msgr2=0x7fb168153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 0x7fb168153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 -- 172.21.15.86:0/774627497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 msgr2=0x7fb168099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 0x7fb168099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 -- 172.21.15.86:0/774627497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb168154360 con 0x7fb1680cc3e0 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb165400640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1680cc3e0 0x7fb1681510f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c3a0 tx=0x7fb15400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.500+0000 7fb15e000640 1 -- 172.21.15.86:0/774627497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1540052d0 con 0x7fb1680cc3e0 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb15e000640 1 -- 172.21.15.86:0/774627497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb154005470 con 0x7fb1680cc3e0 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb15e000640 1 -- 172.21.15.86:0/774627497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154007260 con 0x7fb1680cc3e0 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb168154640 con 0x7fb1680cc3e0 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb168099ca0 con 0x7fb1680cc3e0 2024-03-22T04:19:12.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb168068650 con 0x7fb1680cc3e0 2024-03-22T04:19:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb15e000640 1 -- 172.21.15.86:0/774627497 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb154007400 con 0x7fb1680cc3e0 2024-03-22T04:19:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb15e000640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb148067b50 0x7fb14806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb15e000640 1 -- 172.21.15.86:0/774627497 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5507+0+0 (secure 0 0 0) 0x7fb1540851c0 con 0x7fb1680cc3e0 2024-03-22T04:19:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb165e00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb148067b50 0x7fb14806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.504+0000 7fb165e00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb148067b50 0x7fb14806a010 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb1580ae000 tx=0x7fb1580a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.508+0000 7fb15e000640 1 -- 172.21.15.86:0/774627497 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb154055630 con 0x7fb1680cc3e0 2024-03-22T04:19:12.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:12.680+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb1680b0af0 con 0x7fb1680cc3e0 2024-03-22T04:19:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.632+0000 7fb15e000640 1 -- 172.21.15.86:0/774627497 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) ==== 97+0+0 (secure 0 0 0) 0x7fb1540596d0 con 0x7fb1680cc3e0 2024-03-22T04:19:13.633 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-03-22T04:19:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb148067b50 msgr2=0x7fb14806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb148067b50 0x7fb14806a010 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb1580ae000 tx=0x7fb1580a7070 comp rx=0 tx=0).stop 2024-03-22T04:19:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1680cc3e0 msgr2=0x7fb1681510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1680cc3e0 0x7fb1681510f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c3a0 tx=0x7fb15400c860 comp rx=0 tx=0).stop 2024-03-22T04:19:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 shutdown_connections 2024-03-22T04:19:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb148067b50 0x7fb14806a010 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16815b3f0 0x7fb168099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1680cc3e0 0x7fb1681510f0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 --2- 172.21.15.86:0/774627497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1680ae540 0x7fb168153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 >> 172.21.15.86:0/774627497 conn(0x7fb1680b1df0 msgr2=0x7fb1680b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 shutdown_connections 2024-03-22T04:19:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.636+0000 7fb16ce00640 1 -- 172.21.15.86:0/774627497 wait complete. 2024-03-22T04:19:13.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-22T04:19:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.792+0000 7f1971c00640 1 Processor -- start 2024-03-22T04:19:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f1971c00640 1 -- start start 2024-03-22T04:19:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f1971c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0730d0 0x7f196c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f1971c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f196c0cc3e0 0x7f196c0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f1971c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196c071920 con 0x7f196c15b3f0 2024-03-22T04:19:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f1971c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f196c0716d0 con 0x7f196c0cc3e0 2024-03-22T04:19:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f1971c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f196c06d990 con 0x7f196c0730d0 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f196b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f196c0cc3e0 0x7f196c0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f1962000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0730d0 0x7f196c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f196b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f196c0cc3e0 0x7f196c0ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:34234/0 (socket says 172.21.15.86:34234) 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f1962000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0730d0 0x7f196c0ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:50066/0 (socket says 172.21.15.86:50066) 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f196b400640 1 -- 172.21.15.86:0/2179413688 learned_addr learned my addr 172.21.15.86:0/2179413688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f196aa00640 1 --1- 172.21.15.86:0/2179413688 >> v1:172.21.15.196:6789/0 conn(0x7f196c15b3f0 0x7f196c0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52276/0 (socket says 172.21.15.86:52276) 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f196b400640 1 -- 172.21.15.86:0/2179413688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0730d0 msgr2=0x7f196c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f196b400640 1 --2- 172.21.15.86:0/2179413688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0730d0 0x7f196c0ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f196b400640 1 -- 172.21.15.86:0/2179413688 >> v1:172.21.15.196:6789/0 conn(0x7f196c15b3f0 legacy=0x7f196c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:13.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.796+0000 7f196b400640 1 -- 172.21.15.86:0/2179413688 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c071c00 con 0x7f196c0cc3e0 2024-03-22T04:19:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f196b400640 1 --2- 172.21.15.86:0/2179413688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0cc3e0 0x7f196c0ae370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f196400d150 tx=0x7f196401ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bfcd46f37cb3790 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f196a000640 1 -- 172.21.15.86:0/2179413688 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196400b2d0 con 0x7f196c0cc3e0 2024-03-22T04:19:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f196a000640 1 -- 172.21.15.86:0/2179413688 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196400b470 con 0x7f196c0cc3e0 2024-03-22T04:19:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f196a000640 1 -- 172.21.15.86:0/2179413688 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640a7c10 con 0x7f196c0cc3e0 2024-03-22T04:19:13.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f1971c00640 1 -- 172.21.15.86:0/2179413688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0cc3e0 msgr2=0x7f196c0ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:13.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f1971c00640 1 --2- 172.21.15.86:0/2179413688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0cc3e0 0x7f196c0ae370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f196400d150 tx=0x7f196401ec30 comp rx=0 tx=0).stop 2024-03-22T04:19:13.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f1971c00640 1 -- 172.21.15.86:0/2179413688 shutdown_connections 2024-03-22T04:19:13.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f1971c00640 1 --2- 172.21.15.86:0/2179413688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0cc3e0 0x7f196c0ae370 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f1971c00640 1 --2- 172.21.15.86:0/2179413688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0730d0 0x7f196c0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f1971c00640 1 -- 172.21.15.86:0/2179413688 >> 172.21.15.86:0/2179413688 conn(0x7f196c0b1df0 msgr2=0x7f196c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:13.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f1971c00640 1 -- 172.21.15.86:0/2179413688 shutdown_connections 2024-03-22T04:19:13.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.800+0000 7f1971c00640 1 -- 172.21.15.86:0/2179413688 wait complete. 2024-03-22T04:19:13.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 Processor -- start 2024-03-22T04:19:13.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 -- start start 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0730d0 0x7f196c158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0cc3e0 0x7f196c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c15b3f0 0x7f196c1f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f196c06e2d0 con 0x7f196c15b3f0 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f196c06e150 con 0x7f196c0730d0 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f196c06e450 con 0x7f196c0cc3e0 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c15b3f0 0x7f196c1f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c15b3f0 0x7f196c1f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56114/0 (socket says 172.21.15.86:56114) 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 -- 172.21.15.86:0/1349196043 learned_addr learned my addr 172.21.15.86:0/1349196043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196aa00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0730d0 0x7f196c158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1962000640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0cc3e0 0x7f196c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 -- 172.21.15.86:0/1349196043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0cc3e0 msgr2=0x7f196c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0cc3e0 0x7f196c15a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 -- 172.21.15.86:0/1349196043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0730d0 msgr2=0x7f196c158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0730d0 0x7f196c158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 -- 172.21.15.86:0/1349196043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c1f8120 con 0x7f196c15b3f0 2024-03-22T04:19:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f196b400640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c15b3f0 0x7f196c1f78d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f19640b1040 tx=0x7f196401b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:13.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1963400640 1 -- 172.21.15.86:0/1349196043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964009800 con 0x7f196c15b3f0 2024-03-22T04:19:13.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1963400640 1 -- 172.21.15.86:0/1349196043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19640be070 con 0x7f196c15b3f0 2024-03-22T04:19:13.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1963400640 1 -- 172.21.15.86:0/1349196043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964009be0 con 0x7f196c15b3f0 2024-03-22T04:19:13.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c1f8400 con 0x7f196c15b3f0 2024-03-22T04:19:13.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c1f7e10 con 0x7f196c15b3f0 2024-03-22T04:19:13.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.804+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c068650 con 0x7f196c15b3f0 2024-03-22T04:19:13.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.808+0000 7f1963400640 1 -- 172.21.15.86:0/1349196043 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f19640b6070 con 0x7f196c15b3f0 2024-03-22T04:19:13.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.808+0000 7f1963400640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.808+0000 7f1963400640 1 -- 172.21.15.86:0/1349196043 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5249+0+0 (secure 0 0 0) 0x7f1964120df0 con 0x7f196c15b3f0 2024-03-22T04:19:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.808+0000 7f196aa00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.808+0000 7f196aa00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f195c000f70 tx=0x7f195c007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:13.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:13.812+0000 7f1963400640 1 -- 172.21.15.86:0/1349196043 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1964018820 con 0x7f196c15b3f0 2024-03-22T04:19:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.020+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f196c15aab0 con 0x7f196c15b3f0 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1963400640 1 -- 172.21.15.86:0/1349196043 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f19640f1920 con 0x7f196c15b3f0 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA==", 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:19:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw==", 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q==", 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg==", 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA==", 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA==", 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg==", 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:19:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA==", 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ==", 2024-03-22T04:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow==", 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg==", 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg==", 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:19:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA==", 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg==", 2024-03-22T04:19:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w==", 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:19:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w==", 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB4BP1lvLoQORAADwVEr78V9aUc1YblO47Gaw==", 2024-03-22T04:19:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g==", 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ==", 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T04:19:14.035 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:19:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 msgr2=0x7f1948069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f195c000f70 tx=0x7f195c007cf0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c15b3f0 msgr2=0x7f196c1f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c15b3f0 0x7f196c1f78d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f19640b1040 tx=0x7f196401b9d0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 shutdown_connections 2024-03-22T04:19:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1948067b20 0x7f1948069fe0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196c15b3f0 0x7f196c1f78d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f196c0cc3e0 0x7f196c15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 --2- 172.21.15.86:0/1349196043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f196c0730d0 0x7f196c158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 >> 172.21.15.86:0/1349196043 conn(0x7f196c0b1df0 msgr2=0x7f196c14d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 shutdown_connections 2024-03-22T04:19:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.024+0000 7f1971c00640 1 -- 172.21.15.86:0/1349196043 wait complete. 2024-03-22T04:19:14.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:14.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.176+0000 7f3177e00640 1 Processor -- start 2024-03-22T04:19:14.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177e00640 1 -- start start 2024-03-22T04:19:14.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f31780724b0 0x7f317815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780affc0 0x7f31780cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31780cd140 con 0x7f31780afbe0 2024-03-22T04:19:14.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f317806de00 con 0x7f31780724b0 2024-03-22T04:19:14.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f31780cd420 con 0x7f31780affc0 2024-03-22T04:19:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780affc0 0x7f31780cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3176000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f31780724b0 0x7f317815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780affc0 0x7f31780cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:50076/0 (socket says 172.21.15.86:50076) 2024-03-22T04:19:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3176a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f31780afbe0 0x7f3178071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:52290/0 (socket says 172.21.15.86:52290) 2024-03-22T04:19:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3176000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f31780724b0 0x7f317815d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:34242/0 (socket says 172.21.15.86:34242) 2024-03-22T04:19:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177400640 1 -- 172.21.15.86:0/891398398 learned_addr learned my addr 172.21.15.86:0/891398398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3175600640 1 -- 172.21.15.86:0/891398398 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2944027434 0 0) 0x7f31780cd140 con 0x7f31780afbe0 2024-03-22T04:19:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3175600640 1 -- 172.21.15.86:0/891398398 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f315c003530 con 0x7f31780afbe0 2024-03-22T04:19:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3176000640 1 -- 172.21.15.86:0/891398398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780affc0 msgr2=0x7f31780cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3176000640 1 --2- 172.21.15.86:0/891398398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780affc0 0x7f31780cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3176000640 1 -- 172.21.15.86:0/891398398 >> v1:172.21.15.196:6789/0 conn(0x7f31780afbe0 legacy=0x7f3178071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3176000640 1 -- 172.21.15.86:0/891398398 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31780cd700 con 0x7f31780724b0 2024-03-22T04:19:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.180+0000 7f3177400640 1 --2- 172.21.15.86:0/891398398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780affc0 0x7f31780cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3176000640 1 --2- 172.21.15.86:0/891398398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780724b0 0x7f317815d300 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f31640097c0 tx=0x7f316401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f9b56464034f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3175600640 1 -- 172.21.15.86:0/891398398 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31640b0070 con 0x7f31780724b0 2024-03-22T04:19:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3175600640 1 -- 172.21.15.86:0/891398398 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31640ab600 con 0x7f31780724b0 2024-03-22T04:19:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3175600640 1 -- 172.21.15.86:0/891398398 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31640ab9d0 con 0x7f31780724b0 2024-03-22T04:19:14.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3177e00640 1 -- 172.21.15.86:0/891398398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780724b0 msgr2=0x7f317815d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3177e00640 1 --2- 172.21.15.86:0/891398398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780724b0 0x7f317815d300 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f31640097c0 tx=0x7f316401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3177e00640 1 -- 172.21.15.86:0/891398398 shutdown_connections 2024-03-22T04:19:14.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3177e00640 1 --2- 172.21.15.86:0/891398398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780affc0 0x7f31780cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3177e00640 1 --2- 172.21.15.86:0/891398398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780724b0 0x7f317815d300 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3177e00640 1 -- 172.21.15.86:0/891398398 >> 172.21.15.86:0/891398398 conn(0x7f317806b660 msgr2=0x7f317806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3177e00640 1 -- 172.21.15.86:0/891398398 shutdown_connections 2024-03-22T04:19:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.184+0000 7f3177e00640 1 -- 172.21.15.86:0/891398398 wait complete. 2024-03-22T04:19:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 Processor -- start 2024-03-22T04:19:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 -- start start 2024-03-22T04:19:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31780724b0 0x7f31780b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780afbe0 0x7f31781493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780affc0 0x7f317815b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f317806ee30 con 0x7f31780724b0 2024-03-22T04:19:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f317806ecb0 con 0x7f31780affc0 2024-03-22T04:19:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f317806efb0 con 0x7f31780afbe0 2024-03-22T04:19:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31780724b0 0x7f31780b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31780724b0 0x7f31780b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56122/0 (socket says 172.21.15.86:56122) 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 -- 172.21.15.86:0/410801878 learned_addr learned my addr 172.21.15.86:0/410801878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176000640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780afbe0 0x7f31781493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177400640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780affc0 0x7f317815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 -- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780afbe0 msgr2=0x7f31781493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780afbe0 0x7f31781493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 -- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780affc0 msgr2=0x7f317815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780affc0 0x7f317815b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 -- 172.21.15.86:0/410801878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31781503c0 con 0x7f31780724b0 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177400640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780affc0 0x7f317815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3176a00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31780724b0 0x7f31780b4a80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f316800c7c0 tx=0x7f316800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f316ea00640 1 -- 172.21.15.86:0/410801878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168017070 con 0x7f31780724b0 2024-03-22T04:19:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f316ea00640 1 -- 172.21.15.86:0/410801878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316800a640 con 0x7f31780724b0 2024-03-22T04:19:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31781506a0 con 0x7f31780724b0 2024-03-22T04:19:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f316ea00640 1 -- 172.21.15.86:0/410801878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316800aa30 con 0x7f31780724b0 2024-03-22T04:19:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3178150c60 con 0x7f31780724b0 2024-03-22T04:19:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.188+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3178074470 con 0x7f31780724b0 2024-03-22T04:19:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.192+0000 7f316ea00640 1 -- 172.21.15.86:0/410801878 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f316801a070 con 0x7f31780724b0 2024-03-22T04:19:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.192+0000 7f316ea00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3158067b20 0x7f3158069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.192+0000 7f316ea00640 1 -- 172.21.15.86:0/410801878 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5249+0+0 (secure 0 0 0) 0x7f3168085470 con 0x7f31780724b0 2024-03-22T04:19:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.192+0000 7f3176000640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3158067b20 0x7f3158069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.192+0000 7f3176000640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3158067b20 0x7f3158069fe0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f31640ad000 tx=0x7f3164008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.196+0000 7f316ea00640 1 -- 172.21.15.86:0/410801878 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3168055960 con 0x7f31780724b0 2024-03-22T04:19:14.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.352+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31780b1530 con 0x7f31780724b0 2024-03-22T04:19:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.352+0000 7f316ea00640 1 -- 172.21.15.86:0/410801878 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+13166 (secure 0 0 0) 0x7f3168059a00 con 0x7f31780724b0 2024-03-22T04:19:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:14.354 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:19:13.634148+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3158067b20 msgr2=0x7f3158069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3158067b20 0x7f3158069fe0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f31640ad000 tx=0x7f3164008040 comp rx=0 tx=0).stop 2024-03-22T04:19:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31780724b0 msgr2=0x7f31780b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31780724b0 0x7f31780b4a80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f316800c7c0 tx=0x7f316800cc80 comp rx=0 tx=0).stop 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 shutdown_connections 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3158067b20 0x7f3158069fe0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31780affc0 0x7f317815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f31780afbe0 0x7f31781493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 --2- 172.21.15.86:0/410801878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31780724b0 0x7f31780b4a80 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 >> 172.21.15.86:0/410801878 conn(0x7f317806b660 msgr2=0x7f31780b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 shutdown_connections 2024-03-22T04:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.356+0000 7f3177e00640 1 -- 172.21.15.86:0/410801878 wait complete. 2024-03-22T04:19:14.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:19:14.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:19:14.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.512+0000 7f25fcc00640 1 Processor -- start 2024-03-22T04:19:14.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.512+0000 7f25fcc00640 1 -- start start 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25fcc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f25f806e7e0 0x7f25f806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25fcc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25f8072650 0x7f25f8072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25fcc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f80b1b80 0x7f25f80b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25fcc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f25f8071350 con 0x7f25f8072650 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25fcc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f25f806f100 con 0x7f25f806e7e0 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25fcc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f25f8073150 con 0x7f25f80b1b80 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25ed400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25f8072650 0x7f25f8072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25ed400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25f8072650 0x7f25f8072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56128/0 (socket says 172.21.15.86:56128) 2024-03-22T04:19:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25ed400640 1 -- 172.21.15.86:0/4068636426 learned_addr learned my addr 172.21.15.86:0/4068636426 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25f5e00640 1 --2- 172.21.15.86:0/4068636426 >> v2:172.21.15.196:3300/0 conn(0x7f25f806e7e0 0x7f25f806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25f6800640 1 --2- 172.21.15.86:0/4068636426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f80b1b80 0x7f25f80b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25ed400640 1 -- 172.21.15.86:0/4068636426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f80b1b80 msgr2=0x7f25f80b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25ed400640 1 --2- 172.21.15.86:0/4068636426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f80b1b80 0x7f25f80b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25ed400640 1 -- 172.21.15.86:0/4068636426 >> v2:172.21.15.196:3300/0 conn(0x7f25f806e7e0 msgr2=0x7f25f806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25ed400640 1 --2- 172.21.15.86:0/4068636426 >> v2:172.21.15.196:3300/0 conn(0x7f25f806e7e0 0x7f25f806ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25ed400640 1 -- 172.21.15.86:0/4068636426 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f80715a0 con 0x7f25f8072650 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25f5e00640 1 --2- 172.21.15.86:0/4068636426 >> v2:172.21.15.196:3300/0 conn(0x7f25f806e7e0 0x7f25f806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:19:14.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.516+0000 7f25f6800640 1 --2- 172.21.15.86:0/4068636426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f80b1b80 0x7f25f80b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:19:14.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25ed400640 1 --2- 172.21.15.86:0/4068636426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f8072650 0x7f25f8072b00 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f25f80ce650 tx=0x7f25e001c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d02ae45c20a5dd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25f5400640 1 -- 172.21.15.86:0/4068636426 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e00b0070 con 0x7f25f8072650 2024-03-22T04:19:14.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25f5400640 1 -- 172.21.15.86:0/4068636426 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e00ab600 con 0x7f25f8072650 2024-03-22T04:19:14.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25f5400640 1 -- 172.21.15.86:0/4068636426 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e00ab9d0 con 0x7f25f8072650 2024-03-22T04:19:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 -- 172.21.15.86:0/4068636426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f8072650 msgr2=0x7f25f8072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 --2- 172.21.15.86:0/4068636426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f8072650 0x7f25f8072b00 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f25f80ce650 tx=0x7f25e001c280 comp rx=0 tx=0).stop 2024-03-22T04:19:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 -- 172.21.15.86:0/4068636426 shutdown_connections 2024-03-22T04:19:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 --2- 172.21.15.86:0/4068636426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f80b1b80 0x7f25f80b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 --2- 172.21.15.86:0/4068636426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f8072650 0x7f25f8072b00 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 --2- 172.21.15.86:0/4068636426 >> v2:172.21.15.196:3300/0 conn(0x7f25f806e7e0 0x7f25f806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 -- 172.21.15.86:0/4068636426 >> 172.21.15.86:0/4068636426 conn(0x7f25f80cc940 msgr2=0x7f25f80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 -- 172.21.15.86:0/4068636426 shutdown_connections 2024-03-22T04:19:14.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 -- 172.21.15.86:0/4068636426 wait complete. 2024-03-22T04:19:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 Processor -- start 2024-03-22T04:19:14.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.520+0000 7f25fcc00640 1 -- start start 2024-03-22T04:19:14.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f806e7e0 0x7f25f815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f8072650 0x7f25f815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f80b1b80 0x7f25f80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8072080 con 0x7f25f806e7e0 2024-03-22T04:19:14.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f25f80ce930 con 0x7f25f80b1b80 2024-03-22T04:19:14.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f25f8158a70 con 0x7f25f8072650 2024-03-22T04:19:14.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f806e7e0 0x7f25f815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f806e7e0 0x7f25f815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56130/0 (socket says 172.21.15.86:56130) 2024-03-22T04:19:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 -- 172.21.15.86:0/2781769795 learned_addr learned my addr 172.21.15.86:0/2781769795 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 -- 172.21.15.86:0/2781769795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f8072650 msgr2=0x7f25f815ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f8072650 0x7f25f815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 -- 172.21.15.86:0/2781769795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f80b1b80 msgr2=0x7f25f80cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f80b1b80 0x7f25f80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 -- 172.21.15.86:0/2781769795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f8158230 con 0x7f25f806e7e0 2024-03-22T04:19:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25f5e00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f806e7e0 0x7f25f815a5e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f25e800c820 tx=0x7f25e800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25eea00640 1 -- 172.21.15.86:0/2781769795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8015070 con 0x7f25f806e7e0 2024-03-22T04:19:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25eea00640 1 -- 172.21.15.86:0/2781769795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e8007a30 con 0x7f25f806e7e0 2024-03-22T04:19:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25eea00640 1 -- 172.21.15.86:0/2781769795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8007e20 con 0x7f25f806e7e0 2024-03-22T04:19:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f81584b0 con 0x7f25f806e7e0 2024-03-22T04:19:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f8140540 con 0x7f25f806e7e0 2024-03-22T04:19:14.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f8074470 con 0x7f25f806e7e0 2024-03-22T04:19:14.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.524+0000 7f25eea00640 1 -- 172.21.15.86:0/2781769795 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f25e8018070 con 0x7f25f806e7e0 2024-03-22T04:19:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.528+0000 7f25eea00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f25d4067b20 0x7f25d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.528+0000 7f25eea00640 1 -- 172.21.15.86:0/2781769795 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5249+0+0 (secure 0 0 0) 0x7f25e8085c60 con 0x7f25f806e7e0 2024-03-22T04:19:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.528+0000 7f25ed400640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f25d4067b20 0x7f25d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.528+0000 7f25ed400640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f25d4067b20 0x7f25d4069fe0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f25e00ad000 tx=0x7f25e0004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.532+0000 7f25eea00640 1 -- 172.21.15.86:0/2781769795 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f25f8158230 con 0x7f25f806e7e0 2024-03-22T04:19:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.684+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25f806c670 con 0x7f25f806e7e0 2024-03-22T04:19:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.684+0000 7f25eea00640 1 -- 172.21.15.86:0/2781769795 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f25e80567a0 con 0x7f25f806e7e0 2024-03-22T04:19:14.687 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:19:14.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f25d4067b20 msgr2=0x7f25d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f25d4067b20 0x7f25d4069fe0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f25e00ad000 tx=0x7f25e0004740 comp rx=0 tx=0).stop 2024-03-22T04:19:14.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f806e7e0 msgr2=0x7f25f815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f806e7e0 0x7f25f815a5e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f25e800c820 tx=0x7f25e800cce0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 shutdown_connections 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f25d4067b20 0x7f25d4069fe0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f80b1b80 0x7f25f80cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f8072650 0x7f25f815ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 --2- 172.21.15.86:0/2781769795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f806e7e0 0x7f25f815a5e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 >> 172.21.15.86:0/2781769795 conn(0x7f25f80cc940 msgr2=0x7f25f8152ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 shutdown_connections 2024-03-22T04:19:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.688+0000 7f25fcc00640 1 -- 172.21.15.86:0/2781769795 wait complete. 2024-03-22T04:19:14.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T04:19:14.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0596000640 1 Processor -- start 2024-03-22T04:19:14.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0596000640 1 -- start start 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0596000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f059015be40 0x7f059015f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0596000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f059006f060 0x7f059006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0596000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f059006be00 con 0x7f05900b0040 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0596000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f059006bba0 con 0x7f059006f060 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0596000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f059006c0e0 con 0x7f059015be40 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0587e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f059006f060 0x7f059006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0587e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f059006f060 0x7f059006b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56138/0 (socket says 172.21.15.86:56138) 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0587e00640 1 -- 172.21.15.86:0/884685027 learned_addr learned my addr 172.21.15.86:0/884685027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0594c00640 1 --2- 172.21.15.86:0/884685027 >> v2:172.21.15.196:3300/0 conn(0x7f059015be40 0x7f059015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0587e00640 1 -- 172.21.15.86:0/884685027 >> v2:172.21.15.196:3300/0 conn(0x7f059015be40 msgr2=0x7f059015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0587e00640 1 --2- 172.21.15.86:0/884685027 >> v2:172.21.15.196:3300/0 conn(0x7f059015be40 0x7f059015f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0587e00640 1 -- 172.21.15.86:0/884685027 >> v1:172.21.15.196:6789/0 conn(0x7f05900b0040 legacy=0x7f05900b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0587e00640 1 -- 172.21.15.86:0/884685027 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059015ec60 con 0x7f059006f060 2024-03-22T04:19:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.844+0000 7f0594c00640 1 --2- 172.21.15.86:0/884685027 >> v2:172.21.15.196:3300/0 conn(0x7f059015be40 0x7f059015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:19:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0587e00640 1 --2- 172.21.15.86:0/884685027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059006f060 0x7f059006b660 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f05900502d0 tx=0x7f057801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0c23bc27cd3e64 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f058fe00640 1 -- 172.21.15.86:0/884685027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05780b1070 con 0x7f059006f060 2024-03-22T04:19:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f058fe00640 1 -- 172.21.15.86:0/884685027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05780a8050 con 0x7f059006f060 2024-03-22T04:19:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f058fe00640 1 -- 172.21.15.86:0/884685027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05780ac860 con 0x7f059006f060 2024-03-22T04:19:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 -- 172.21.15.86:0/884685027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059006f060 msgr2=0x7f059006b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 --2- 172.21.15.86:0/884685027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059006f060 0x7f059006b660 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f05900502d0 tx=0x7f057801c4b0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 -- 172.21.15.86:0/884685027 shutdown_connections 2024-03-22T04:19:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 --2- 172.21.15.86:0/884685027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059006f060 0x7f059006b660 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 --2- 172.21.15.86:0/884685027 >> v2:172.21.15.196:3300/0 conn(0x7f059015be40 0x7f059015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 -- 172.21.15.86:0/884685027 >> 172.21.15.86:0/884685027 conn(0x7f059006e540 msgr2=0x7f059006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 -- 172.21.15.86:0/884685027 shutdown_connections 2024-03-22T04:19:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 -- 172.21.15.86:0/884685027 wait complete. 2024-03-22T04:19:14.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 Processor -- start 2024-03-22T04:19:14.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.848+0000 7f0596000640 1 -- start start 2024-03-22T04:19:14.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f059006f060 0x7f059014ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05900b0040 0x7f05900cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f059015be40 0x7f05900ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05900b49e0 con 0x7f05900b0040 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05900b4860 con 0x7f059006f060 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f05900b4b60 con 0x7f059015be40 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05900b0040 0x7f05900cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05900b0040 0x7f05900cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56142/0 (socket says 172.21.15.86:56142) 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 -- 172.21.15.86:0/620584463 learned_addr learned my addr 172.21.15.86:0/620584463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 -- 172.21.15.86:0/620584463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f059015be40 msgr2=0x7f05900ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f059015be40 0x7f05900ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 -- 172.21.15.86:0/620584463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f059006f060 msgr2=0x7f059014ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f059006f060 0x7f059014ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 -- 172.21.15.86:0/620584463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059014f240 con 0x7f05900b0040 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0587e00640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05900b0040 0x7f05900cf180 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f057801a0e0 tx=0x7f057801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f058d600640 1 -- 172.21.15.86:0/620584463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05780b1040 con 0x7f05900b0040 2024-03-22T04:19:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f058d600640 1 -- 172.21.15.86:0/620584463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05780bc070 con 0x7f05900b0040 2024-03-22T04:19:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f058d600640 1 -- 172.21.15.86:0/620584463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05780b7650 con 0x7f05900b0040 2024-03-22T04:19:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05900cfc20 con 0x7f05900b0040 2024-03-22T04:19:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05901536a0 con 0x7f05900b0040 2024-03-22T04:19:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0590074470 con 0x7f05900b0040 2024-03-22T04:19:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.852+0000 7f058d600640 1 -- 172.21.15.86:0/620584463 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f05780b77f0 con 0x7f05900b0040 2024-03-22T04:19:14.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.856+0000 7f058d600640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f056c067b20 0x7f056c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:14.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.856+0000 7f058d600640 1 -- 172.21.15.86:0/620584463 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5249+0+0 (secure 0 0 0) 0x7f05781201c0 con 0x7f05900b0040 2024-03-22T04:19:14.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.856+0000 7f0594c00640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f056c067b20 0x7f056c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:14.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.856+0000 7f0594c00640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f056c067b20 0x7f056c069fe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f058000a8d0 tx=0x7f0580008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:14.860+0000 7f058d600640 1 -- 172.21.15.86:0/620584463 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f05780160d0 con 0x7f05900b0040 2024-03-22T04:19:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.016+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f05900704b0 con 0x7f05900b0040 2024-03-22T04:19:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.016+0000 7f058d600640 1 -- 172.21.15.86:0/620584463 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) ==== 84+0+65 (secure 0 0 0) 0x7f05780f0cf0 con 0x7f05900b0040 2024-03-22T04:19:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:15.019 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:19:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:19:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T04:19:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T04:19:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:15.019 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:19:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f056c067b20 msgr2=0x7f056c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f056c067b20 0x7f056c069fe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f058000a8d0 tx=0x7f0580008040 comp rx=0 tx=0).stop 2024-03-22T04:19:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05900b0040 msgr2=0x7f05900cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05900b0040 0x7f05900cf180 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f057801a0e0 tx=0x7f057801acf0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 shutdown_connections 2024-03-22T04:19:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f056c067b20 0x7f056c069fe0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f059015be40 0x7f05900ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05900b0040 0x7f05900cf180 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 --2- 172.21.15.86:0/620584463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f059006f060 0x7f059014ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 >> 172.21.15.86:0/620584463 conn(0x7f059006e540 msgr2=0x7f059015b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 shutdown_connections 2024-03-22T04:19:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.020+0000 7f0596000640 1 -- 172.21.15.86:0/620584463 wait complete. 2024-03-22T04:19:15.040 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T04:19:15.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T04:19:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f582d600640 1 Processor -- start 2024-03-22T04:19:15.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f582d600640 1 -- start start 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f582d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58280b1cd0 0x7f58280b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f582d600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f582806de70 0x7f582813f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f582d600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58280b25f0 con 0x7f58280cfa70 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f582d600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f58280cea80 con 0x7f58280b1cd0 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f582d600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f582806e4d0 con 0x7f582806de70 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f5826a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58280b1cd0 0x7f58280b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f5826a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58280b1cd0 0x7f58280b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56146/0 (socket says 172.21.15.86:56146) 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f5826a00640 1 -- 172.21.15.86:0/1712661465 learned_addr learned my addr 172.21.15.86:0/1712661465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f5827400640 1 --2- 172.21.15.86:0/1712661465 >> v2:172.21.15.196:3300/0 conn(0x7f582806de70 0x7f582813f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f5826a00640 1 -- 172.21.15.86:0/1712661465 >> v2:172.21.15.196:3300/0 conn(0x7f582806de70 msgr2=0x7f582813f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f5826a00640 1 --2- 172.21.15.86:0/1712661465 >> v2:172.21.15.196:3300/0 conn(0x7f582806de70 0x7f582813f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f5826a00640 1 -- 172.21.15.86:0/1712661465 >> v1:172.21.15.196:6789/0 conn(0x7f58280cfa70 legacy=0x7f582806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.180+0000 7f5826a00640 1 -- 172.21.15.86:0/1712661465 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58280cecb0 con 0x7f58280b1cd0 2024-03-22T04:19:15.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f5826a00640 1 --2- 172.21.15.86:0/1712661465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58280b1cd0 0x7f58280b20b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5828156d90 tx=0x7f581801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aed267a033ee0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:15.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f5825600640 1 -- 172.21.15.86:0/1712661465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180ab610 con 0x7f58280b1cd0 2024-03-22T04:19:15.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f5825600640 1 -- 172.21.15.86:0/1712661465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58180b0070 con 0x7f58280b1cd0 2024-03-22T04:19:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f5825600640 1 -- 172.21.15.86:0/1712661465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180ab9d0 con 0x7f58280b1cd0 2024-03-22T04:19:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f582d600640 1 -- 172.21.15.86:0/1712661465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58280b1cd0 msgr2=0x7f58280b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f582d600640 1 --2- 172.21.15.86:0/1712661465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58280b1cd0 0x7f58280b20b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5828156d90 tx=0x7f581801c520 comp rx=0 tx=0).stop 2024-03-22T04:19:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f582d600640 1 -- 172.21.15.86:0/1712661465 shutdown_connections 2024-03-22T04:19:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f582d600640 1 --2- 172.21.15.86:0/1712661465 >> v2:172.21.15.196:3300/0 conn(0x7f582806de70 0x7f582813f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f582d600640 1 --2- 172.21.15.86:0/1712661465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58280b1cd0 0x7f58280b20b0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f582d600640 1 -- 172.21.15.86:0/1712661465 >> 172.21.15.86:0/1712661465 conn(0x7f582806b660 msgr2=0x7f582806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:15.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f582d600640 1 -- 172.21.15.86:0/1712661465 shutdown_connections 2024-03-22T04:19:15.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.184+0000 7f582d600640 1 -- 172.21.15.86:0/1712661465 wait complete. 2024-03-22T04:19:15.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 Processor -- start 2024-03-22T04:19:15.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 -- start start 2024-03-22T04:19:15.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f582806de70 0x7f5828156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:15.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58280b1cd0 0x7f58280b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58280cfa70 0x7f58281393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f582813f5e0 con 0x7f582806de70 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f582813f460 con 0x7f58280cfa70 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5828168d80 con 0x7f58280b1cd0 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f582806de70 0x7f5828156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f582806de70 0x7f5828156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56150/0 (socket says 172.21.15.86:56150) 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 -- 172.21.15.86:0/1009078809 learned_addr learned my addr 172.21.15.86:0/1009078809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826000640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58280b1cd0 0x7f58280b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5827400640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58280cfa70 0x7f58281393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 -- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58280b1cd0 msgr2=0x7f58280b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58280b1cd0 0x7f58280b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 -- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58280cfa70 msgr2=0x7f58281393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58280cfa70 0x7f58281393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:15.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 -- 172.21.15.86:0/1009078809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5828157490 con 0x7f582806de70 2024-03-22T04:19:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826000640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58280b1cd0 0x7f58280b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5827400640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58280cfa70 0x7f58281393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f5826a00640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f582806de70 0x7f5828156e00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5818019130 tx=0x7f581801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f581ea00640 1 -- 172.21.15.86:0/1009078809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180be5f0 con 0x7f582806de70 2024-03-22T04:19:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f581ea00640 1 -- 172.21.15.86:0/1009078809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58180be790 con 0x7f582806de70 2024-03-22T04:19:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5828157770 con 0x7f582806de70 2024-03-22T04:19:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f581ea00640 1 -- 172.21.15.86:0/1009078809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180beb40 con 0x7f582806de70 2024-03-22T04:19:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5828157d30 con 0x7f582806de70 2024-03-22T04:19:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.188+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5828074580 con 0x7f582806de70 2024-03-22T04:19:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.192+0000 7f581ea00640 1 -- 172.21.15.86:0/1009078809 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5828140620 con 0x7f582806de70 2024-03-22T04:19:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.192+0000 7f581ea00640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5808067b20 0x7f5808069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.192+0000 7f581ea00640 1 -- 172.21.15.86:0/1009078809 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5249+0+0 (secure 0 0 0) 0x7f5818120570 con 0x7f582806de70 2024-03-22T04:19:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.192+0000 7f5826000640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5808067b20 0x7f5808069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:15.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.192+0000 7f5826000640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5808067b20 0x7f5808069fe0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f582806d8d0 tx=0x7f5814009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.196+0000 7f581ea00640 1 -- 172.21.15.86:0/1009078809 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f58180c4060 con 0x7f582806de70 2024-03-22T04:19:15.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:15.352+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f582806d900 con 0x7f582806de70 2024-03-22T04:19:16.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.288+0000 7f581ea00640 1 -- 172.21.15.86:0/1009078809 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v106) ==== 138+0+0 (secure 0 0 0) 0x7f58180a8050 con 0x7f582806de70 2024-03-22T04:19:16.289 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T04:19:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5808067b20 msgr2=0x7f5808069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5808067b20 0x7f5808069fe0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f582806d8d0 tx=0x7f5814009040 comp rx=0 tx=0).stop 2024-03-22T04:19:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f582806de70 msgr2=0x7f5828156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f582806de70 0x7f5828156e00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5818019130 tx=0x7f581801ad60 comp rx=0 tx=0).stop 2024-03-22T04:19:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 shutdown_connections 2024-03-22T04:19:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5808067b20 0x7f5808069fe0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58280cfa70 0x7f58281393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58280b1cd0 0x7f58280b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 --2- 172.21.15.86:0/1009078809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f582806de70 0x7f5828156e00 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 >> 172.21.15.86:0/1009078809 conn(0x7f582806b660 msgr2=0x7f58280b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 shutdown_connections 2024-03-22T04:19:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.292+0000 7f582d600640 1 -- 172.21.15.86:0/1009078809 wait complete. 2024-03-22T04:19:16.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T04:19:16.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.460+0000 7f8e4d600640 1 Processor -- start 2024-03-22T04:19:16.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.460+0000 7f8e4d600640 1 -- start start 2024-03-22T04:19:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.460+0000 7f8e4d600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815c010 0x7f8e480d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.460+0000 7f8e4d600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8e480d28f0 0x7f8e4815f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.460+0000 7f8e4d600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4815ee60 con 0x7f8e4815e980 2024-03-22T04:19:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.460+0000 7f8e4d600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8e48161f50 con 0x7f8e480d28f0 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.460+0000 7f8e4d600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4815f140 con 0x7f8e4815c010 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e46a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815c010 0x7f8e480d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e3fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8e480d28f0 0x7f8e4815f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e46a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815c010 0x7f8e480d23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:38774/0 (socket says 172.21.15.86:38774) 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e46a00640 1 -- 172.21.15.86:0/3056330743 learned_addr learned my addr 172.21.15.86:0/3056330743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e3fe00640 1 --2- 172.21.15.86:0/3056330743 >> v2:172.21.15.196:3300/0 conn(0x7f8e480d28f0 0x7f8e4815f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:47286/0 (socket says 172.21.15.86:47286) 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e46a00640 1 -- 172.21.15.86:0/3056330743 >> v2:172.21.15.196:3300/0 conn(0x7f8e480d28f0 msgr2=0x7f8e4815f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e46a00640 1 --2- 172.21.15.86:0/3056330743 >> v2:172.21.15.196:3300/0 conn(0x7f8e480d28f0 0x7f8e4815f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e46a00640 1 -- 172.21.15.86:0/3056330743 >> v1:172.21.15.196:6789/0 conn(0x7f8e4815e980 legacy=0x7f8e4806dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e46a00640 1 -- 172.21.15.86:0/3056330743 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4815f420 con 0x7f8e4815c010 2024-03-22T04:19:16.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e3fe00640 1 --2- 172.21.15.86:0/3056330743 >> v2:172.21.15.196:3300/0 conn(0x7f8e480d28f0 0x7f8e4815f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:16.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.464+0000 7f8e46a00640 1 --2- 172.21.15.86:0/3056330743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815c010 0x7f8e480d23b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009870 tx=0x7f8e3801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e754c149bd6c3b2b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:16.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e46000640 1 -- 172.21.15.86:0/3056330743 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380a9610 con 0x7f8e4815c010 2024-03-22T04:19:16.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e46000640 1 -- 172.21.15.86:0/3056330743 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e380a97b0 con 0x7f8e4815c010 2024-03-22T04:19:16.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e46000640 1 -- 172.21.15.86:0/3056330743 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380a9b80 con 0x7f8e4815c010 2024-03-22T04:19:16.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- 172.21.15.86:0/3056330743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815c010 msgr2=0x7f8e480d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:16.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 --2- 172.21.15.86:0/3056330743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815c010 0x7f8e480d23b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009870 tx=0x7f8e3801c530 comp rx=0 tx=0).stop 2024-03-22T04:19:16.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- 172.21.15.86:0/3056330743 shutdown_connections 2024-03-22T04:19:16.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 --2- 172.21.15.86:0/3056330743 >> v2:172.21.15.196:3300/0 conn(0x7f8e480d28f0 0x7f8e4815f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 --2- 172.21.15.86:0/3056330743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815c010 0x7f8e480d23b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- 172.21.15.86:0/3056330743 >> 172.21.15.86:0/3056330743 conn(0x7f8e4806b660 msgr2=0x7f8e4806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:16.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- 172.21.15.86:0/3056330743 shutdown_connections 2024-03-22T04:19:16.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- 172.21.15.86:0/3056330743 wait complete. 2024-03-22T04:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 Processor -- start 2024-03-22T04:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- start start 2024-03-22T04:19:16.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4815c010 0x7f8e480b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:16.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815e980 0x7f8e481574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e481582b0 0x7f8e480b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4816b680 con 0x7f8e481582b0 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4816b500 con 0x7f8e4815c010 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e4d600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4816b800 con 0x7f8e4815e980 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e47400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e481582b0 0x7f8e480b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e47400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e481582b0 0x7f8e480b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39896/0 (socket says 172.21.15.86:39896) 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e47400640 1 -- 172.21.15.86:0/652522897 learned_addr learned my addr 172.21.15.86:0/652522897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e3fe00640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815e980 0x7f8e481574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.468+0000 7f8e46a00640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4815c010 0x7f8e480b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e47400640 1 -- 172.21.15.86:0/652522897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815e980 msgr2=0x7f8e481574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e47400640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815e980 0x7f8e481574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e47400640 1 -- 172.21.15.86:0/652522897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4815c010 msgr2=0x7f8e480b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e47400640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4815c010 0x7f8e480b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:16.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e47400640 1 -- 172.21.15.86:0/652522897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e481579e0 con 0x7f8e481582b0 2024-03-22T04:19:16.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e47400640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e481582b0 0x7f8e480b3110 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000c730 tx=0x7f8e4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:16.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e3ea00640 1 -- 172.21.15.86:0/652522897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40003b80 con 0x7f8e481582b0 2024-03-22T04:19:16.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e3ea00640 1 -- 172.21.15.86:0/652522897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e40003d20 con 0x7f8e481582b0 2024-03-22T04:19:16.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e480b7b50 con 0x7f8e481582b0 2024-03-22T04:19:16.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e3ea00640 1 -- 172.21.15.86:0/652522897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40005310 con 0x7f8e481582b0 2024-03-22T04:19:16.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e480ada90 con 0x7f8e481582b0 2024-03-22T04:19:16.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e480b8070 con 0x7f8e481582b0 2024-03-22T04:19:16.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.472+0000 7f8e3ea00640 1 -- 172.21.15.86:0/652522897 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8e400056c0 con 0x7f8e481582b0 2024-03-22T04:19:16.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.476+0000 7f8e3ea00640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e24067b20 0x7f8e24069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:16.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.476+0000 7f8e3ea00640 1 -- 172.21.15.86:0/652522897 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5619+0+0 (secure 0 0 0) 0x7f8e40086700 con 0x7f8e481582b0 2024-03-22T04:19:16.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.476+0000 7f8e46a00640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e24067b20 0x7f8e24069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:16.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.476+0000 7f8e46a00640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e24067b20 0x7f8e24069fe0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8e380ae000 tx=0x7f8e380b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:16.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.476+0000 7f8e3ea00640 1 -- 172.21.15.86:0/652522897 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8e480b8070 con 0x7f8e481582b0 2024-03-22T04:19:16.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:16.632+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8e48074470 con 0x7f8e481582b0 2024-03-22T04:19:17.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.292+0000 7f8e3ea00640 1 -- 172.21.15.86:0/652522897 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v107) ==== 170+0+0 (secure 0 0 0) 0x7f8e481579e0 con 0x7f8e481582b0 2024-03-22T04:19:17.293 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e24067b20 msgr2=0x7f8e24069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e24067b20 0x7f8e24069fe0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8e380ae000 tx=0x7f8e380b3040 comp rx=0 tx=0).stop 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e481582b0 msgr2=0x7f8e480b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e481582b0 0x7f8e480b3110 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000c730 tx=0x7f8e4000cbf0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 shutdown_connections 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e24067b20 0x7f8e24069fe0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e481582b0 0x7f8e480b3110 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815e980 0x7f8e481574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 --2- 172.21.15.86:0/652522897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4815c010 0x7f8e480b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 >> 172.21.15.86:0/652522897 conn(0x7f8e4806b660 msgr2=0x7f8e481613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:17.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 shutdown_connections 2024-03-22T04:19:17.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.296+0000 7f8e4d600640 1 -- 172.21.15.86:0/652522897 wait complete. 2024-03-22T04:19:17.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T04:19:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59fa000640 1 Processor -- start 2024-03-22T04:19:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59fa000640 1 -- start start 2024-03-22T04:19:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59fa000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f59f4072650 0x7f59f4072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59fa000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f4073150 con 0x7f59f415b760 2024-03-22T04:19:17.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59fa000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f40715a0 con 0x7f59f406e7e0 2024-03-22T04:19:17.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59fa000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f59f4071350 con 0x7f59f4072650 2024-03-22T04:19:17.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59f3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f59f415b760 0x7f59f415bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44340/0 (socket says 172.21.15.86:44340) 2024-03-22T04:19:17.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59f3e00640 1 -- 172.21.15.86:0/3392787133 learned_addr learned my addr 172.21.15.86:0/3392787133 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:17.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59f2a00640 1 --2- 172.21.15.86:0/3392787133 >> v2:172.21.15.196:3300/0 conn(0x7f59f4072650 0x7f59f4072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59f2000640 1 -- 172.21.15.86:0/3392787133 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550002575 0 0) 0x7f59f4073150 con 0x7f59f415b760 2024-03-22T04:19:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59f2000640 1 -- 172.21.15.86:0/3392787133 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59d8003530 con 0x7f59f415b760 2024-03-22T04:19:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59f2a00640 1 -- 172.21.15.86:0/3392787133 >> v1:172.21.15.196:6789/0 conn(0x7f59f406e7e0 legacy=0x7f59f406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59f2a00640 1 -- 172.21.15.86:0/3392787133 >> v1:172.21.15.86:6789/0 conn(0x7f59f415b760 legacy=0x7f59f415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.476+0000 7f59f2a00640 1 -- 172.21.15.86:0/3392787133 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f4071880 con 0x7f59f4072650 2024-03-22T04:19:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59f2a00640 1 --2- 172.21.15.86:0/3392787133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f4072650 0x7f59f4072b00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097c0 tx=0x7f59e001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78cac9792fb92dc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59f2000640 1 -- 172.21.15.86:0/3392787133 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e00b1070 con 0x7f59f4072650 2024-03-22T04:19:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59f2000640 1 -- 172.21.15.86:0/3392787133 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e00a8050 con 0x7f59f4072650 2024-03-22T04:19:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59f2000640 1 -- 172.21.15.86:0/3392787133 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e00ac860 con 0x7f59f4072650 2024-03-22T04:19:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59fa000640 1 -- 172.21.15.86:0/3392787133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f4072650 msgr2=0x7f59f4072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59fa000640 1 --2- 172.21.15.86:0/3392787133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f4072650 0x7f59f4072b00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097c0 tx=0x7f59e001c3e0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59fa000640 1 -- 172.21.15.86:0/3392787133 shutdown_connections 2024-03-22T04:19:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59fa000640 1 --2- 172.21.15.86:0/3392787133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f4072650 0x7f59f4072b00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59fa000640 1 -- 172.21.15.86:0/3392787133 >> 172.21.15.86:0/3392787133 conn(0x7f59f40cc940 msgr2=0x7f59f40ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59fa000640 1 -- 172.21.15.86:0/3392787133 shutdown_connections 2024-03-22T04:19:17.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.480+0000 7f59fa000640 1 -- 172.21.15.86:0/3392787133 wait complete. 2024-03-22T04:19:17.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 Processor -- start 2024-03-22T04:19:17.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 -- start start 2024-03-22T04:19:17.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59f406e7e0 0x7f59f414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:17.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59f4072650 0x7f59f4157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:17.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f415b760 0x7f59f414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:17.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4072080 con 0x7f59f406e7e0 2024-03-22T04:19:17.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4155aa0 con 0x7f59f415b760 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f59f4156940 con 0x7f59f4072650 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59f406e7e0 0x7f59f414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59f406e7e0 0x7f59f414cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39904/0 (socket says 172.21.15.86:39904) 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 -- 172.21.15.86:0/3179115037 learned_addr learned my addr 172.21.15.86:0/3179115037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f2a00640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59f4072650 0x7f59f4157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3e00640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f415b760 0x7f59f414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 -- 172.21.15.86:0/3179115037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59f4072650 msgr2=0x7f59f4157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59f4072650 0x7f59f4157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 -- 172.21.15.86:0/3179115037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f415b760 msgr2=0x7f59f414e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:17.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f415b760 0x7f59f414e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 -- 172.21.15.86:0/3179115037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f4156b70 con 0x7f59f406e7e0 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59f3400640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59f406e7e0 0x7f59f414cca0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f59e400c500 tx=0x7f59e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59eb400640 1 -- 172.21.15.86:0/3179115037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4016020 con 0x7f59f406e7e0 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59eb400640 1 -- 172.21.15.86:0/3179115037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4005220 con 0x7f59f406e7e0 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59eb400640 1 -- 172.21.15.86:0/3179115037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4007030 con 0x7f59f406e7e0 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f414d1e0 con 0x7f59f406e7e0 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f41570f0 con 0x7f59f406e7e0 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.484+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f4074470 con 0x7f59f406e7e0 2024-03-22T04:19:17.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.488+0000 7f59eb400640 1 -- 172.21.15.86:0/3179115037 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f59e40071f0 con 0x7f59f406e7e0 2024-03-22T04:19:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.488+0000 7f59eb400640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f59d4067b20 0x7f59d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.488+0000 7f59eb400640 1 -- 172.21.15.86:0/3179115037 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6001+0+0 (secure 0 0 0) 0x7f59e4086690 con 0x7f59f406e7e0 2024-03-22T04:19:17.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.488+0000 7f59f2a00640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f59d4067b20 0x7f59d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:17.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.488+0000 7f59f2a00640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f59d4067b20 0x7f59d4069fe0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f59e00ae000 tx=0x7f59e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:17.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.492+0000 7f59eb400640 1 -- 172.21.15.86:0/3179115037 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f59e4020020 con 0x7f59f406e7e0 2024-03-22T04:19:17.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:17.680+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f59f4072200 con 0x7f59f406e7e0 2024-03-22T04:19:18.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.316+0000 7f59eb400640 1 -- 172.21.15.86:0/3179115037 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v81) ==== 174+0+0 (secure 0 0 0) 0x7f59e40569b0 con 0x7f59f406e7e0 2024-03-22T04:19:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 15 and data pool 16 2024-03-22T04:19:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f59d4067b20 msgr2=0x7f59d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f59d4067b20 0x7f59d4069fe0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f59e00ae000 tx=0x7f59e0008040 comp rx=0 tx=0).stop 2024-03-22T04:19:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59f406e7e0 msgr2=0x7f59f414cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59f406e7e0 0x7f59f414cca0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f59e400c500 tx=0x7f59e400c9c0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 shutdown_connections 2024-03-22T04:19:18.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f59d4067b20 0x7f59d4069fe0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f415b760 0x7f59f414e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59f4072650 0x7f59f4157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 --2- 172.21.15.86:0/3179115037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59f406e7e0 0x7f59f414cca0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.324+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 >> 172.21.15.86:0/3179115037 conn(0x7f59f40cc940 msgr2=0x7f59f4154970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.328+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 shutdown_connections 2024-03-22T04:19:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.328+0000 7f59fa000640 1 -- 172.21.15.86:0/3179115037 wait complete. 2024-03-22T04:19:18.358 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:18.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.492+0000 7f555ec00640 1 Processor -- start 2024-03-22T04:19:18.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.492+0000 7f555ec00640 1 -- start start 2024-03-22T04:19:18.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555ec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f55580aff50 0x7f555806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f555806cbb0 0x7f5558143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555ec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5558144320 0x7f555806e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555ec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f555806d010 con 0x7f5558144320 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555ec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5558162aa0 con 0x7f55580aff50 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f55580ae5a0 con 0x7f555806cbb0 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5558144320 0x7f555806e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5558144320 0x7f555806e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39906/0 (socket says 172.21.15.86:39906) 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 -- 172.21.15.86:0/2554515902 learned_addr learned my addr 172.21.15.86:0/2554515902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f5557400640 1 --2- 172.21.15.86:0/2554515902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f555806cbb0 0x7f5558143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f5557e00640 1 --2- 172.21.15.86:0/2554515902 >> v2:172.21.15.196:3300/0 conn(0x7f55580aff50 0x7f555806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 -- 172.21.15.86:0/2554515902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f555806cbb0 msgr2=0x7f5558143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 --2- 172.21.15.86:0/2554515902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f555806cbb0 0x7f5558143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 -- 172.21.15.86:0/2554515902 >> v2:172.21.15.196:3300/0 conn(0x7f55580aff50 msgr2=0x7f555806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 --2- 172.21.15.86:0/2554515902 >> v2:172.21.15.196:3300/0 conn(0x7f55580aff50 0x7f555806c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 -- 172.21.15.86:0/2554515902 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5558144b80 con 0x7f5558144320 2024-03-22T04:19:18.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f5557e00640 1 --2- 172.21.15.86:0/2554515902 >> v2:172.21.15.196:3300/0 conn(0x7f55580aff50 0x7f555806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:18.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.496+0000 7f555cc00640 1 --2- 172.21.15.86:0/2554515902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5558144320 0x7f555806e220 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f55580ae750 tx=0x7f555001c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd17482292a898dd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:18.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f5556a00640 1 -- 172.21.15.86:0/2554515902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500b0070 con 0x7f5558144320 2024-03-22T04:19:18.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f5556a00640 1 -- 172.21.15.86:0/2554515902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55500ab600 con 0x7f5558144320 2024-03-22T04:19:18.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f5556a00640 1 -- 172.21.15.86:0/2554515902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500ab9d0 con 0x7f5558144320 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- 172.21.15.86:0/2554515902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5558144320 msgr2=0x7f555806e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 --2- 172.21.15.86:0/2554515902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5558144320 0x7f555806e220 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f55580ae750 tx=0x7f555001c1a0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- 172.21.15.86:0/2554515902 shutdown_connections 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 --2- 172.21.15.86:0/2554515902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5558144320 0x7f555806e220 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 --2- 172.21.15.86:0/2554515902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f555806cbb0 0x7f5558143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 --2- 172.21.15.86:0/2554515902 >> v2:172.21.15.196:3300/0 conn(0x7f55580aff50 0x7f555806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- 172.21.15.86:0/2554515902 >> 172.21.15.86:0/2554515902 conn(0x7f5558071fe0 msgr2=0x7f55580723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- 172.21.15.86:0/2554515902 shutdown_connections 2024-03-22T04:19:18.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- 172.21.15.86:0/2554515902 wait complete. 2024-03-22T04:19:18.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 Processor -- start 2024-03-22T04:19:18.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- start start 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806cbb0 0x7f5558141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580aff50 0x7f55580b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5558144320 0x7f5558156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5558163090 con 0x7f555806cbb0 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5558162f10 con 0x7f5558144320 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5558144790 con 0x7f55580aff50 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f5557e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806cbb0 0x7f5558141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f5557e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806cbb0 0x7f5558141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39922/0 (socket says 172.21.15.86:39922) 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f5557e00640 1 -- 172.21.15.86:0/3083129166 learned_addr learned my addr 172.21.15.86:0/3083129166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f5557400640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580aff50 0x7f55580b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.500+0000 7f555cc00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5558144320 0x7f5558156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f5557e00640 1 -- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580aff50 msgr2=0x7f55580b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f5557e00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580aff50 0x7f55580b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f5557e00640 1 -- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5558144320 msgr2=0x7f5558156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f5557e00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5558144320 0x7f5558156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f5557e00640 1 -- 172.21.15.86:0/3083129166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5558142480 con 0x7f555806cbb0 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f555cc00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5558144320 0x7f5558156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f5557e00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806cbb0 0x7f5558141ba0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5548009fc0 tx=0x7f554800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f554fe00640 1 -- 172.21.15.86:0/3083129166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554801a070 con 0x7f555806cbb0 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f554fe00640 1 -- 172.21.15.86:0/3083129166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5548003ba0 con 0x7f555806cbb0 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5558142760 con 0x7f555806cbb0 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f554fe00640 1 -- 172.21.15.86:0/3083129166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548004030 con 0x7f555806cbb0 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5558142d20 con 0x7f555806cbb0 2024-03-22T04:19:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5558068650 con 0x7f555806cbb0 2024-03-22T04:19:18.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f554fe00640 1 -- 172.21.15.86:0/3083129166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f554801d070 con 0x7f555806cbb0 2024-03-22T04:19:18.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f554fe00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5538067b20 0x7f5538069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f554fe00640 1 -- 172.21.15.86:0/3083129166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f5548086370 con 0x7f555806cbb0 2024-03-22T04:19:18.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.504+0000 7f5557400640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5538067b20 0x7f5538069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.508+0000 7f5557400640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5538067b20 0x7f5538069fe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5558068e10 tx=0x7f5544009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:18.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.508+0000 7f554fe00640 1 -- 172.21.15.86:0/3083129166 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5548010070 con 0x7f555806cbb0 2024-03-22T04:19:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.664+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55580d0080 con 0x7f555806cbb0 2024-03-22T04:19:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f554fe00640 1 -- 172.21.15.86:0/3083129166 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16913 (secure 0 0 0) 0x7f5548056500 con 0x7f555806cbb0 2024-03-22T04:19:18.668 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:18.668 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:19:17.684241+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:19:15.365871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-22T04:19:16.647234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5538067b20 msgr2=0x7f5538069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5538067b20 0x7f5538069fe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5558068e10 tx=0x7f5544009190 comp rx=0 tx=0).stop 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806cbb0 msgr2=0x7f5558141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806cbb0 0x7f5558141ba0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5548009fc0 tx=0x7f554800d560 comp rx=0 tx=0).stop 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 shutdown_connections 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5538067b20 0x7f5538069fe0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5558144320 0x7f5558156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580aff50 0x7f55580b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 --2- 172.21.15.86:0/3083129166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555806cbb0 0x7f5558141ba0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 >> 172.21.15.86:0/3083129166 conn(0x7f5558071fe0 msgr2=0x7f55580b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 shutdown_connections 2024-03-22T04:19:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.668+0000 7f555ec00640 1 -- 172.21.15.86:0/3083129166 wait complete. 2024-03-22T04:19:18.690 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9875400640 1 Processor -- start 2024-03-22T04:19:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9875400640 1 -- start start 2024-03-22T04:19:18.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9875400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9870147e10 0x7f98700b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9875400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f987006c270 0x7f9870162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9875400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9870162540 0x7f987015e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9875400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98700b2bd0 con 0x7f9870162540 2024-03-22T04:19:18.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9875400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f98700b2610 con 0x7f9870147e10 2024-03-22T04:19:18.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9875400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f98700b28f0 con 0x7f987006c270 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9870162540 0x7f987015e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986e800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9870147e10 0x7f98700b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9870162540 0x7f987015e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39928/0 (socket says 172.21.15.86:39928) 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986f200640 1 -- 172.21.15.86:0/3210334577 learned_addr learned my addr 172.21.15.86:0/3210334577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f9865e00640 1 --2- 172.21.15.86:0/3210334577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f987006c270 0x7f9870162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986f200640 1 -- 172.21.15.86:0/3210334577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f987006c270 msgr2=0x7f9870162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986f200640 1 --2- 172.21.15.86:0/3210334577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f987006c270 0x7f9870162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986f200640 1 -- 172.21.15.86:0/3210334577 >> v2:172.21.15.196:3300/0 conn(0x7f9870147e10 msgr2=0x7f98700b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986f200640 1 --2- 172.21.15.86:0/3210334577 >> v2:172.21.15.196:3300/0 conn(0x7f9870147e10 0x7f98700b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986f200640 1 -- 172.21.15.86:0/3210334577 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98700b5ce0 con 0x7f9870162540 2024-03-22T04:19:18.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.824+0000 7f986e800640 1 --2- 172.21.15.86:0/3210334577 >> v2:172.21.15.196:3300/0 conn(0x7f9870147e10 0x7f98700b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:18.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f986f200640 1 --2- 172.21.15.86:0/3210334577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870162540 0x7f987015e9b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9870072c70 tx=0x7f986801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12cbd64fbdc20b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:18.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f986de00640 1 -- 172.21.15.86:0/3210334577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680a5780 con 0x7f9870162540 2024-03-22T04:19:18.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f986de00640 1 -- 172.21.15.86:0/3210334577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98680a5920 con 0x7f9870162540 2024-03-22T04:19:18.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f986de00640 1 -- 172.21.15.86:0/3210334577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680a5cf0 con 0x7f9870162540 2024-03-22T04:19:18.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 -- 172.21.15.86:0/3210334577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870162540 msgr2=0x7f987015e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 --2- 172.21.15.86:0/3210334577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870162540 0x7f987015e9b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9870072c70 tx=0x7f986801c9f0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 -- 172.21.15.86:0/3210334577 shutdown_connections 2024-03-22T04:19:18.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 --2- 172.21.15.86:0/3210334577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870162540 0x7f987015e9b0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 --2- 172.21.15.86:0/3210334577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f987006c270 0x7f9870162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 --2- 172.21.15.86:0/3210334577 >> v2:172.21.15.196:3300/0 conn(0x7f9870147e10 0x7f98700b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 -- 172.21.15.86:0/3210334577 >> 172.21.15.86:0/3210334577 conn(0x7f98700b6620 msgr2=0x7f98700b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:18.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 -- 172.21.15.86:0/3210334577 shutdown_connections 2024-03-22T04:19:18.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.828+0000 7f9875400640 1 -- 172.21.15.86:0/3210334577 wait complete. 2024-03-22T04:19:18.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 Processor -- start 2024-03-22T04:19:18.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 -- start start 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f987006c270 0x7f987013efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9870147e10 0x7f9870140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9870162540 0x7f987013c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f987015feb0 con 0x7f987006c270 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f987015fd30 con 0x7f9870147e10 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9870160030 con 0x7f9870162540 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f987006c270 0x7f987013efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f987006c270 0x7f987013efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39934/0 (socket says 172.21.15.86:39934) 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 -- 172.21.15.86:0/2830932255 learned_addr learned my addr 172.21.15.86:0/2830932255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9865e00640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9870147e10 0x7f9870140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986f200640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9870162540 0x7f987013c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 -- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9870162540 msgr2=0x7f987013c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9870162540 0x7f987013c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 -- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9870147e10 msgr2=0x7f9870140600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9870147e10 0x7f9870140600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 -- 172.21.15.86:0/2830932255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98700cb2e0 con 0x7f987006c270 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9865e00640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9870147e10 0x7f9870140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f986e800640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f987006c270 0x7f987013efd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f986000ebe0 tx=0x7f986000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9867400640 1 -- 172.21.15.86:0/2830932255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017070 con 0x7f987006c270 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9867400640 1 -- 172.21.15.86:0/2830932255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986000cc30 con 0x7f987006c270 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98700cb5c0 con 0x7f987006c270 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9867400640 1 -- 172.21.15.86:0/2830932255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860013470 con 0x7f987006c270 2024-03-22T04:19:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98700cbb80 con 0x7f987006c270 2024-03-22T04:19:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.832+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987013f630 con 0x7f987006c270 2024-03-22T04:19:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.836+0000 7f9867400640 1 -- 172.21.15.86:0/2830932255 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f986001a070 con 0x7f987006c270 2024-03-22T04:19:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.836+0000 7f9867400640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f984c067b20 0x7f984c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.836+0000 7f9867400640 1 -- 172.21.15.86:0/2830932255 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f9860086390 con 0x7f987006c270 2024-03-22T04:19:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.836+0000 7f9865e00640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f984c067b20 0x7f984c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.836+0000 7f9865e00640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f984c067b20 0x7f984c069fe0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98700ccea0 tx=0x7f9858008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:18.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.840+0000 7f9867400640 1 -- 172.21.15.86:0/2830932255 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9860056c10 con 0x7f987006c270 2024-03-22T04:19:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.996+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9870068650 con 0x7f987006c270 2024-03-22T04:19:18.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:18.996+0000 7f9867400640 1 -- 172.21.15.86:0/2830932255 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16913 (secure 0 0 0) 0x7f9860056510 con 0x7f987006c270 2024-03-22T04:19:18.997 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:18.998 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:19:17.684241+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:19:15.365871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-22T04:19:16.647234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:18.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f984c067b20 msgr2=0x7f984c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f984c067b20 0x7f984c069fe0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98700ccea0 tx=0x7f9858008040 comp rx=0 tx=0).stop 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f987006c270 msgr2=0x7f987013efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f987006c270 0x7f987013efd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f986000ebe0 tx=0x7f986000c520 comp rx=0 tx=0).stop 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 shutdown_connections 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f984c067b20 0x7f984c069fe0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9870147e10 0x7f9870140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9870162540 0x7f987013c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 --2- 172.21.15.86:0/2830932255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f987006c270 0x7f987013efd0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 >> 172.21.15.86:0/2830932255 conn(0x7f98700b6620 msgr2=0x7f98700b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 shutdown_connections 2024-03-22T04:19:19.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.000+0000 7f9875400640 1 -- 172.21.15.86:0/2830932255 wait complete. 2024-03-22T04:19:19.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T04:19:19.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.156+0000 7f0de4c00640 1 Processor -- start 2024-03-22T04:19:19.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.156+0000 7f0de4c00640 1 -- start start 2024-03-22T04:19:19.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0de4c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0de00b3600 0x7f0de015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:19.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0de4c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de00b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:19.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0de4c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de015d710 con 0x7f0de00b2360 2024-03-22T04:19:19.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0de4c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0de00b2740 con 0x7f0de00b3600 2024-03-22T04:19:19.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0de4c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0de015fe40 con 0x7f0de015cee0 2024-03-22T04:19:19.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0ddd400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0de00b3600 0x7f0de015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0dde800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de00b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0ddd400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0de00b3600 0x7f0de015c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:47354/0 (socket says 172.21.15.86:47354) 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0ddde00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f0de00b2360 0x7f0de00b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:35998/0 (socket says 172.21.15.86:35998) 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0ddd400640 1 -- 172.21.15.86:0/4169148056 learned_addr learned my addr 172.21.15.86:0/4169148056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0dde800640 1 -- 172.21.15.86:0/4169148056 >> v2:172.21.15.196:3300/0 conn(0x7f0de00b3600 msgr2=0x7f0de015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0dde800640 1 --2- 172.21.15.86:0/4169148056 >> v2:172.21.15.196:3300/0 conn(0x7f0de00b3600 0x7f0de015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0dde800640 1 -- 172.21.15.86:0/4169148056 >> v1:172.21.15.196:6789/0 conn(0x7f0de00b2360 legacy=0x7f0de00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0dde800640 1 -- 172.21.15.86:0/4169148056 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de0160120 con 0x7f0de015cee0 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0ddd400640 1 --2- 172.21.15.86:0/4169148056 >> v2:172.21.15.196:3300/0 conn(0x7f0de00b3600 0x7f0de015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:19.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.160+0000 7f0ddca00640 1 -- 172.21.15.86:0/4169148056 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1802939003 0 0) 0x7f0de015d710 con 0x7f0de00b2360 2024-03-22T04:19:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0dde800640 1 --2- 172.21.15.86:0/4169148056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de00b4d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80097c0 tx=0x7f0dd801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b46700ecdcb72f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0ddca00640 1 -- 172.21.15.86:0/4169148056 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80a7070 con 0x7f0de015cee0 2024-03-22T04:19:19.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0ddca00640 1 -- 172.21.15.86:0/4169148056 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd80b0070 con 0x7f0de015cee0 2024-03-22T04:19:19.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0ddca00640 1 -- 172.21.15.86:0/4169148056 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80ab890 con 0x7f0de015cee0 2024-03-22T04:19:19.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 -- 172.21.15.86:0/4169148056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 msgr2=0x7f0de00b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:19.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 --2- 172.21.15.86:0/4169148056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de00b4d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80097c0 tx=0x7f0dd801c470 comp rx=0 tx=0).stop 2024-03-22T04:19:19.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 -- 172.21.15.86:0/4169148056 shutdown_connections 2024-03-22T04:19:19.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 --2- 172.21.15.86:0/4169148056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de00b4d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 --2- 172.21.15.86:0/4169148056 >> v2:172.21.15.196:3300/0 conn(0x7f0de00b3600 0x7f0de015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 -- 172.21.15.86:0/4169148056 >> 172.21.15.86:0/4169148056 conn(0x7f0de00cc940 msgr2=0x7f0de00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:19.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 -- 172.21.15.86:0/4169148056 shutdown_connections 2024-03-22T04:19:19.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 -- 172.21.15.86:0/4169148056 wait complete. 2024-03-22T04:19:19.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 Processor -- start 2024-03-22T04:19:19.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.164+0000 7f0de4c00640 1 -- start start 2024-03-22T04:19:19.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de00b2360 0x7f0de015f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:19.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0de00b3600 0x7f0de015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:19.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:19.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0de015d4d0 con 0x7f0de00b3600 2024-03-22T04:19:19.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0de015d350 con 0x7f0de00b2360 2024-03-22T04:19:19.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0de00b5ad0 con 0x7f0de015cee0 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0de00b3600 0x7f0de015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddde00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de00b2360 0x7f0de015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0de00b3600 0x7f0de015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39944/0 (socket says 172.21.15.86:39944) 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 -- 172.21.15.86:0/1200731265 learned_addr learned my addr 172.21.15.86:0/1200731265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0dde800640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 -- 172.21.15.86:0/1200731265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 msgr2=0x7f0de015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 -- 172.21.15.86:0/1200731265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de00b2360 msgr2=0x7f0de015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de00b2360 0x7f0de015f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 -- 172.21.15.86:0/1200731265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de015d710 con 0x7f0de00b3600 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0ddd400640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0de00b3600 0x7f0de015b320 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00e9d0 tx=0x7f0dcc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0dd6000640 1 -- 172.21.15.86:0/1200731265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc017070 con 0x7f0de00b3600 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0dd6000640 1 -- 172.21.15.86:0/1200731265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dcc00ca60 con 0x7f0de00b3600 2024-03-22T04:19:19.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0dd6000640 1 -- 172.21.15.86:0/1200731265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00ce10 con 0x7f0de00b3600 2024-03-22T04:19:19.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de0156400 con 0x7f0de00b3600 2024-03-22T04:19:19.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de0139860 con 0x7f0de00b3600 2024-03-22T04:19:19.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de0074470 con 0x7f0de00b3600 2024-03-22T04:19:19.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.168+0000 7f0dd6000640 1 -- 172.21.15.86:0/1200731265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f0dcc01a070 con 0x7f0de00b3600 2024-03-22T04:19:19.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.172+0000 7f0dd6000640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0dc0067b20 0x7f0dc0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:19.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.172+0000 7f0dd6000640 1 -- 172.21.15.86:0/1200731265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6101+0+0 (secure 0 0 0) 0x7f0dcc085e80 con 0x7f0de00b3600 2024-03-22T04:19:19.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.172+0000 7f0ddde00640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0dc0067b20 0x7f0dc0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:19.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.172+0000 7f0ddde00640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0dc0067b20 0x7f0dc0069fe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0004fd0 tx=0x7f0dd0001770 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:19.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.176+0000 7f0dd6000640 1 -- 172.21.15.86:0/1200731265 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0de015d710 con 0x7f0de00b3600 2024-03-22T04:19:19.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:19.364+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0de006e9f0 con 0x7f0de00b3600 2024-03-22T04:19:20.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0dd6000640 1 -- 172.21.15.86:0/1200731265 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7f0dcc056140 con 0x7f0de00b3600 2024-03-22T04:19:20.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0dc0067b20 msgr2=0x7f0dc0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:20.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0dc0067b20 0x7f0dc0069fe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0004fd0 tx=0x7f0dd0001770 comp rx=0 tx=0).stop 2024-03-22T04:19:20.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0de00b3600 msgr2=0x7f0de015b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:20.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0de00b3600 0x7f0de015b320 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00e9d0 tx=0x7f0dcc00ee90 comp rx=0 tx=0).stop 2024-03-22T04:19:20.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 shutdown_connections 2024-03-22T04:19:20.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0dc0067b20 0x7f0dc0069fe0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:20.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0de015cee0 0x7f0de015b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:20.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0de00b3600 0x7f0de015b320 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:20.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 --2- 172.21.15.86:0/1200731265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de00b2360 0x7f0de015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:20.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 >> 172.21.15.86:0/1200731265 conn(0x7f0de00cc940 msgr2=0x7f0de014f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:20.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 shutdown_connections 2024-03-22T04:19:20.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.328+0000 7f0de4c00640 1 -- 172.21.15.86:0/1200731265 wait complete. 2024-03-22T04:19:20.350 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-22T04:19:20.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-22T04:19:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e38a00640 1 Processor -- start 2024-03-22T04:19:20.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e38a00640 1 -- start start 2024-03-22T04:19:20.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e38a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e3406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:20.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e38a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e3415c770 con 0x7f7e34072650 2024-03-22T04:19:20.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e38a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e340b29a0 con 0x7f7e34072ad0 2024-03-22T04:19:20.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e38a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e34071350 con 0x7f7e3406e7e0 2024-03-22T04:19:20.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e31400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7e34072650 0x7f7e340b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44342/0 (socket says 172.21.15.86:44342) 2024-03-22T04:19:20.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e31400640 1 -- 172.21.15.86:0/3310592336 learned_addr learned my addr 172.21.15.86:0/3310592336 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:20.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e31e00640 1 --2- 172.21.15.86:0/3310592336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e3406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:20.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.488+0000 7f7e30a00640 1 -- 172.21.15.86:0/3310592336 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931680398 0 0) 0x7f7e20001120 con 0x7f7e34072650 2024-03-22T04:19:20.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e30a00640 1 -- 172.21.15.86:0/3310592336 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e18003530 con 0x7f7e34072650 2024-03-22T04:19:20.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e30a00640 1 -- 172.21.15.86:0/3310592336 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128230464 0 0) 0x7f7e340b29a0 con 0x7f7e34072ad0 2024-03-22T04:19:20.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e30a00640 1 -- 172.21.15.86:0/3310592336 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e20001120 con 0x7f7e34072ad0 2024-03-22T04:19:20.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e31e00640 1 -- 172.21.15.86:0/3310592336 >> v1:172.21.15.196:6789/0 conn(0x7f7e34072ad0 legacy=0x7f7e340b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e31e00640 1 -- 172.21.15.86:0/3310592336 >> v1:172.21.15.86:6789/0 conn(0x7f7e34072650 legacy=0x7f7e340b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e31e00640 1 -- 172.21.15.86:0/3310592336 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3406f100 con 0x7f7e3406e7e0 2024-03-22T04:19:20.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e31e00640 1 --2- 172.21.15.86:0/3310592336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e3406ebc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e240097f0 tx=0x7f7e2401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b400e35c7ca2ba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e30a00640 1 -- 172.21.15.86:0/3310592336 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240b0070 con 0x7f7e3406e7e0 2024-03-22T04:19:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e30a00640 1 -- 172.21.15.86:0/3310592336 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e240ab600 con 0x7f7e3406e7e0 2024-03-22T04:19:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e30a00640 1 -- 172.21.15.86:0/3310592336 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240ab9d0 con 0x7f7e3406e7e0 2024-03-22T04:19:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e38a00640 1 -- 172.21.15.86:0/3310592336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 msgr2=0x7f7e3406ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e38a00640 1 --2- 172.21.15.86:0/3310592336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e3406ebc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e240097f0 tx=0x7f7e2401c450 comp rx=0 tx=0).stop 2024-03-22T04:19:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e38a00640 1 -- 172.21.15.86:0/3310592336 shutdown_connections 2024-03-22T04:19:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e38a00640 1 --2- 172.21.15.86:0/3310592336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e3406ebc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e38a00640 1 -- 172.21.15.86:0/3310592336 >> 172.21.15.86:0/3310592336 conn(0x7f7e340cc940 msgr2=0x7f7e340ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e38a00640 1 -- 172.21.15.86:0/3310592336 shutdown_connections 2024-03-22T04:19:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.492+0000 7f7e38a00640 1 -- 172.21.15.86:0/3310592336 wait complete. 2024-03-22T04:19:20.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 Processor -- start 2024-03-22T04:19:20.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 -- start start 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e34145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34072650 0x7f7e341564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e34072ad0 0x7f7e3415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e340ce930 con 0x7f7e34072650 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34071b60 con 0x7f7e34072ad0 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e34072080 con 0x7f7e3406e7e0 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34072650 0x7f7e341564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34072650 0x7f7e341564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39950/0 (socket says 172.21.15.86:39950) 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 -- 172.21.15.86:0/881435623 learned_addr learned my addr 172.21.15.86:0/881435623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31e00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e34145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e32800640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e34072ad0 0x7f7e3415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 -- 172.21.15.86:0/881435623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 msgr2=0x7f7e34145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e34145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 -- 172.21.15.86:0/881435623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e34072ad0 msgr2=0x7f7e3415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e34072ad0 0x7f7e3415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 -- 172.21.15.86:0/881435623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e341460e0 con 0x7f7e34072650 2024-03-22T04:19:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e31400640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34072650 0x7f7e341564b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000c700 tx=0x7f7e2000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e2a000640 1 -- 172.21.15.86:0/881435623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20017070 con 0x7f7e34072650 2024-03-22T04:19:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e2a000640 1 -- 172.21.15.86:0/881435623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e20005ae0 con 0x7f7e34072650 2024-03-22T04:19:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e34146370 con 0x7f7e34072650 2024-03-22T04:19:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e2a000640 1 -- 172.21.15.86:0/881435623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20007070 con 0x7f7e34072650 2024-03-22T04:19:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.496+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e34146930 con 0x7f7e34072650 2024-03-22T04:19:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.500+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e34074470 con 0x7f7e34072650 2024-03-22T04:19:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.500+0000 7f7e2a000640 1 -- 172.21.15.86:0/881435623 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7e2001a070 con 0x7f7e34072650 2024-03-22T04:19:20.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.500+0000 7f7e2a000640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e14067b20 0x7f7e14069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:20.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.500+0000 7f7e2a000640 1 -- 172.21.15.86:0/881435623 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f7e200855d0 con 0x7f7e34072650 2024-03-22T04:19:20.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.500+0000 7f7e31e00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e14067b20 0x7f7e14069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.500+0000 7f7e31e00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e14067b20 0x7f7e14069fe0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7e340cd8c0 tx=0x7f7e240a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.504+0000 7f7e2a000640 1 -- 172.21.15.86:0/881435623 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7e3415c770 con 0x7f7e34072650 2024-03-22T04:19:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:20.692+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7e3406d800 con 0x7f7e34072650 2024-03-22T04:19:21.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.332+0000 7f7e2a000640 1 -- 172.21.15.86:0/881435623 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7f7e20055760 con 0x7f7e34072650 2024-03-22T04:19:21.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e14067b20 msgr2=0x7f7e14069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e14067b20 0x7f7e14069fe0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7e340cd8c0 tx=0x7f7e240a6000 comp rx=0 tx=0).stop 2024-03-22T04:19:21.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34072650 msgr2=0x7f7e341564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34072650 0x7f7e341564b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000c700 tx=0x7f7e2000cbc0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 shutdown_connections 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e14067b20 0x7f7e14069fe0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e34072ad0 0x7f7e3415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34072650 0x7f7e341564b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 --2- 172.21.15.86:0/881435623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e3406e7e0 0x7f7e34145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 >> 172.21.15.86:0/881435623 conn(0x7f7e340cc940 msgr2=0x7f7e341597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 shutdown_connections 2024-03-22T04:19:21.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.336+0000 7f7e38a00640 1 -- 172.21.15.86:0/881435623 wait complete. 2024-03-22T04:19:21.357 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-22T04:19:21.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.492+0000 7f1ba3200640 1 Processor -- start 2024-03-22T04:19:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.492+0000 7f1ba3200640 1 -- start start 2024-03-22T04:19:21.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba3200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b9c0730d0 0x7f1b9c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b9c0cc3e0 0x7f1b9c0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9c071920 con 0x7f1b9c15b3f0 2024-03-22T04:19:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c0716d0 con 0x7f1b9c0cc3e0 2024-03-22T04:19:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba3200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c06d990 con 0x7f1b9c0730d0 2024-03-22T04:19:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1b9be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1b9c15b3f0 0x7f1b9c0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44356/0 (socket says 172.21.15.86:44356) 2024-03-22T04:19:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1b9be00640 1 -- 172.21.15.86:0/114707109 learned_addr learned my addr 172.21.15.86:0/114707109 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba1200640 1 --2- 172.21.15.86:0/114707109 >> v2:172.21.15.86:3300/0 conn(0x7f1b9c0cc3e0 0x7f1b9c0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1b93400640 1 --2- 172.21.15.86:0/114707109 >> v2:172.21.15.196:3300/0 conn(0x7f1b9c0730d0 0x7f1b9c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba1200640 1 -- 172.21.15.86:0/114707109 >> v2:172.21.15.196:3300/0 conn(0x7f1b9c0730d0 msgr2=0x7f1b9c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1b9b400640 1 -- 172.21.15.86:0/114707109 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991062910 0 0) 0x7f1b9c071920 con 0x7f1b9c15b3f0 2024-03-22T04:19:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba1200640 1 --2- 172.21.15.86:0/114707109 >> v2:172.21.15.196:3300/0 conn(0x7f1b9c0730d0 0x7f1b9c0ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba1200640 1 -- 172.21.15.86:0/114707109 >> v1:172.21.15.86:6789/0 conn(0x7f1b9c15b3f0 legacy=0x7f1b9c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.496+0000 7f1ba1200640 1 -- 172.21.15.86:0/114707109 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c071c00 con 0x7f1b9c0cc3e0 2024-03-22T04:19:21.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba1200640 1 --2- 172.21.15.86:0/114707109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cc3e0 0x7f1b9c0ae370 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c06f270 tx=0x7f1b9401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f934b005ff54146 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:21.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1b9b400640 1 -- 172.21.15.86:0/114707109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940b2070 con 0x7f1b9c0cc3e0 2024-03-22T04:19:21.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1b9b400640 1 -- 172.21.15.86:0/114707109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b940a58e0 con 0x7f1b9c0cc3e0 2024-03-22T04:19:21.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1b9b400640 1 -- 172.21.15.86:0/114707109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940a5cb0 con 0x7f1b9c0cc3e0 2024-03-22T04:19:21.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 -- 172.21.15.86:0/114707109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cc3e0 msgr2=0x7f1b9c0ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 --2- 172.21.15.86:0/114707109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cc3e0 0x7f1b9c0ae370 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c06f270 tx=0x7f1b9401c930 comp rx=0 tx=0).stop 2024-03-22T04:19:21.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 -- 172.21.15.86:0/114707109 shutdown_connections 2024-03-22T04:19:21.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 --2- 172.21.15.86:0/114707109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cc3e0 0x7f1b9c0ae370 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 --2- 172.21.15.86:0/114707109 >> v2:172.21.15.196:3300/0 conn(0x7f1b9c0730d0 0x7f1b9c0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 -- 172.21.15.86:0/114707109 >> 172.21.15.86:0/114707109 conn(0x7f1b9c0b1df0 msgr2=0x7f1b9c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:21.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 -- 172.21.15.86:0/114707109 shutdown_connections 2024-03-22T04:19:21.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 -- 172.21.15.86:0/114707109 wait complete. 2024-03-22T04:19:21.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 Processor -- start 2024-03-22T04:19:21.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 -- start start 2024-03-22T04:19:21.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0730d0 0x7f1b9c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c0cc3e0 0x7f1b9c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.500+0000 7f1ba3200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b9c15b3f0 0x7f1b9c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1ba3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c166570 con 0x7f1b9c0730d0 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1ba3200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1663f0 con 0x7f1b9c0cc3e0 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1ba3200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c1666f0 con 0x7f1b9c15b3f0 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0730d0 0x7f1b9c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0730d0 0x7f1b9c15a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39972/0 (socket says 172.21.15.86:39972) 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 -- 172.21.15.86:0/1021195673 learned_addr learned my addr 172.21.15.86:0/1021195673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b93400640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c0cc3e0 0x7f1b9c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1ba1200640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b9c15b3f0 0x7f1b9c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 -- 172.21.15.86:0/1021195673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b9c15b3f0 msgr2=0x7f1b9c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b9c15b3f0 0x7f1b9c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 -- 172.21.15.86:0/1021195673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c0cc3e0 msgr2=0x7f1b9c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c0cc3e0 0x7f1b9c15d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 -- 172.21.15.86:0/1021195673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c14ef00 con 0x7f1b9c0730d0 2024-03-22T04:19:21.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b9be00640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0730d0 0x7f1b9c15a350 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c480 tx=0x7f1b8c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:21.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b98c00640 1 -- 172.21.15.86:0/1021195673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c018070 con 0x7f1b9c0730d0 2024-03-22T04:19:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b98c00640 1 -- 172.21.15.86:0/1021195673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c007500 con 0x7f1b9c0730d0 2024-03-22T04:19:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b98c00640 1 -- 172.21.15.86:0/1021195673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c004a30 con 0x7f1b9c0730d0 2024-03-22T04:19:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c14f1e0 con 0x7f1b9c0730d0 2024-03-22T04:19:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c14f7a0 con 0x7f1b9c0730d0 2024-03-22T04:19:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c068650 con 0x7f1b9c0730d0 2024-03-22T04:19:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.504+0000 7f1b98c00640 1 -- 172.21.15.86:0/1021195673 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1b8c004bd0 con 0x7f1b9c0730d0 2024-03-22T04:19:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.508+0000 7f1b98c00640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b78067b20 0x7f1b78069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.508+0000 7f1b98c00640 1 -- 172.21.15.86:0/1021195673 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f1b8c085200 con 0x7f1b9c0730d0 2024-03-22T04:19:21.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.508+0000 7f1b93400640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b78067b20 0x7f1b78069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.508+0000 7f1b93400640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b78067b20 0x7f1b78069fe0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0b12d0 tx=0x7f1b84009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:21.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.508+0000 7f1b98c00640 1 -- 172.21.15.86:0/1021195673 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1b8c0554c0 con 0x7f1b9c0730d0 2024-03-22T04:19:21.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b9c15ea10 con 0x7f1b9c0730d0 2024-03-22T04:19:21.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1b98c00640 1 -- 172.21.15.86:0/1021195673 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5192 (secure 0 0 0) 0x7f1b8c059560 con 0x7f1b9c0730d0 2024-03-22T04:19:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:21.696 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":77159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/269054026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":269054026},{"type":"v1","addr":"172.21.15.196:6834","nonce":269054026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:19:20.698020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77162":{"gid":77162,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2259098413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2259098413},{"type":"v1","addr":"172.21.15.196:6835","nonce":2259098413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77162,"qdb_cluster":[77162]},"id":7}]} 2024-03-22T04:19:21.696 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 84 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b78067b20 msgr2=0x7f1b78069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b78067b20 0x7f1b78069fe0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0b12d0 tx=0x7f1b84009190 comp rx=0 tx=0).stop 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0730d0 msgr2=0x7f1b9c15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0730d0 0x7f1b9c15a350 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c480 tx=0x7f1b8c00c940 comp rx=0 tx=0).stop 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 shutdown_connections 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1b78067b20 0x7f1b78069fe0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c0cc3e0 0x7f1b9c15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b9c15b3f0 0x7f1b9c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 --2- 172.21.15.86:0/1021195673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0730d0 0x7f1b9c15a350 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 >> 172.21.15.86:0/1021195673 conn(0x7f1b9c0b1df0 msgr2=0x7f1b9c0b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 shutdown_connections 2024-03-22T04:19:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.696+0000 7f1ba3200640 1 -- 172.21.15.86:0/1021195673 wait complete. 2024-03-22T04:19:21.716 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:21.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06b3200640 1 Processor -- start 2024-03-22T04:19:21.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06b3200640 1 -- start start 2024-03-22T04:19:21.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06b3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06ac0d0580 0x7f06ac071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06b3200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac06e2f0 0x7f06ac071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06b3200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06ac1623d0 con 0x7f06ac15e630 2024-03-22T04:19:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06b3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f06ac1620f0 con 0x7f06ac0d0580 2024-03-22T04:19:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06b3200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac15ec10 con 0x7f06ac06e2f0 2024-03-22T04:19:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06a3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06ac0d0580 0x7f06ac071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06a3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06ac0d0580 0x7f06ac071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39988/0 (socket says 172.21.15.86:39988) 2024-03-22T04:19:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06a3400640 1 -- 172.21.15.86:0/1830571395 learned_addr learned my addr 172.21.15.86:0/1830571395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:21.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06b1200640 1 --2- 172.21.15.86:0/1830571395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac06e2f0 0x7f06ac071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06a3400640 1 -- 172.21.15.86:0/1830571395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac06e2f0 msgr2=0x7f06ac071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:21.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06a3400640 1 --2- 172.21.15.86:0/1830571395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac06e2f0 0x7f06ac071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06a3400640 1 -- 172.21.15.86:0/1830571395 >> v1:172.21.15.196:6789/0 conn(0x7f06ac15e630 legacy=0x7f06ac0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.852+0000 7f06a3400640 1 -- 172.21.15.86:0/1830571395 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac15eef0 con 0x7f06ac0d0580 2024-03-22T04:19:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06a3400640 1 --2- 172.21.15.86:0/1830571395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac0d0580 0x7f06ac071100 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0694009870 tx=0x7f069401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d0f7253ae320a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06ab400640 1 -- 172.21.15.86:0/1830571395 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940b2070 con 0x7f06ac0d0580 2024-03-22T04:19:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06ab400640 1 -- 172.21.15.86:0/1830571395 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06940a9070 con 0x7f06ac0d0580 2024-03-22T04:19:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06ab400640 1 -- 172.21.15.86:0/1830571395 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940ad880 con 0x7f06ac0d0580 2024-03-22T04:19:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06b3200640 1 -- 172.21.15.86:0/1830571395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac0d0580 msgr2=0x7f06ac071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06b3200640 1 --2- 172.21.15.86:0/1830571395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac0d0580 0x7f06ac071100 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0694009870 tx=0x7f069401c560 comp rx=0 tx=0).stop 2024-03-22T04:19:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06b3200640 1 -- 172.21.15.86:0/1830571395 shutdown_connections 2024-03-22T04:19:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06b3200640 1 --2- 172.21.15.86:0/1830571395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac06e2f0 0x7f06ac071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06b3200640 1 --2- 172.21.15.86:0/1830571395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac0d0580 0x7f06ac071100 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06b3200640 1 -- 172.21.15.86:0/1830571395 >> 172.21.15.86:0/1830571395 conn(0x7f06ac06f910 msgr2=0x7f06ac06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06b3200640 1 -- 172.21.15.86:0/1830571395 shutdown_connections 2024-03-22T04:19:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.856+0000 7f06b3200640 1 -- 172.21.15.86:0/1830571395 wait complete. 2024-03-22T04:19:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 Processor -- start 2024-03-22T04:19:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 -- start start 2024-03-22T04:19:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac06e2f0 0x7f06ac15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06ac0d0580 0x7f06ac161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac15e630 0x7f06ac161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac071f90 con 0x7f06ac06e2f0 2024-03-22T04:19:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac15f9d0 con 0x7f06ac0d0580 2024-03-22T04:19:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac0cfaf0 con 0x7f06ac15e630 2024-03-22T04:19:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac06e2f0 0x7f06ac15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac06e2f0 0x7f06ac15a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40000/0 (socket says 172.21.15.86:40000) 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 -- 172.21.15.86:0/876877473 learned_addr learned my addr 172.21.15.86:0/876877473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06a3400640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06ac0d0580 0x7f06ac161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b1200640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac15e630 0x7f06ac161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 -- 172.21.15.86:0/876877473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac15e630 msgr2=0x7f06ac161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac15e630 0x7f06ac161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 -- 172.21.15.86:0/876877473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06ac0d0580 msgr2=0x7f06ac161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06ac0d0580 0x7f06ac161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 -- 172.21.15.86:0/876877473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac0c5e30 con 0x7f06ac06e2f0 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06abe00640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac06e2f0 0x7f06ac15a4d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c420 tx=0x7f069c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06a8c00640 1 -- 172.21.15.86:0/876877473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0052d0 con 0x7f06ac06e2f0 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06a8c00640 1 -- 172.21.15.86:0/876877473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c005470 con 0x7f06ac06e2f0 2024-03-22T04:19:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06a8c00640 1 -- 172.21.15.86:0/876877473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c007260 con 0x7f06ac06e2f0 2024-03-22T04:19:21.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac0c6110 con 0x7f06ac06e2f0 2024-03-22T04:19:21.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.860+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac0c66d0 con 0x7f06ac06e2f0 2024-03-22T04:19:21.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.864+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ac068650 con 0x7f06ac06e2f0 2024-03-22T04:19:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.864+0000 7f06a8c00640 1 -- 172.21.15.86:0/876877473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f069c007400 con 0x7f06ac06e2f0 2024-03-22T04:19:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.864+0000 7f06a8c00640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0688067b50 0x7f068806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.864+0000 7f06a8c00640 1 -- 172.21.15.86:0/876877473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f069c0853e0 con 0x7f06ac06e2f0 2024-03-22T04:19:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.864+0000 7f06a3400640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0688067b50 0x7f068806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.864+0000 7f06a3400640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0688067b50 0x7f068806a010 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f06940af000 tx=0x7f0694008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:21.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:21.868+0000 7f06a8c00640 1 -- 172.21.15.86:0/876877473 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f069c0556a0 con 0x7f06ac06e2f0 2024-03-22T04:19:22.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.024+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06ac15de60 con 0x7f06ac06e2f0 2024-03-22T04:19:22.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06a8c00640 1 -- 172.21.15.86:0/876877473 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16900 (secure 0 0 0) 0x7f069c059740 con 0x7f06ac06e2f0 2024-03-22T04:19:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:22.027 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":109,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:19:19.318781+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:19:15.365871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-22T04:19:16.647234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0688067b50 msgr2=0x7f068806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0688067b50 0x7f068806a010 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f06940af000 tx=0x7f0694008040 comp rx=0 tx=0).stop 2024-03-22T04:19:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac06e2f0 msgr2=0x7f06ac15a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac06e2f0 0x7f06ac15a4d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c420 tx=0x7f069c00c8e0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 shutdown_connections 2024-03-22T04:19:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0688067b50 0x7f068806a010 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06ac0d0580 0x7f06ac161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06ac15e630 0x7f06ac161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 --2- 172.21.15.86:0/876877473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06ac06e2f0 0x7f06ac15a4d0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 >> 172.21.15.86:0/876877473 conn(0x7f06ac06f910 msgr2=0x7f06ac0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 shutdown_connections 2024-03-22T04:19:22.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.028+0000 7f06b3200640 1 -- 172.21.15.86:0/876877473 wait complete. 2024-03-22T04:19:22.048 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T04:19:22.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:19:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.180+0000 7fe247e00640 1 Processor -- start 2024-03-22T04:19:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe247e00640 1 -- start start 2024-03-22T04:19:22.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe247e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe248161950 0x7fe2480d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe247e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe248161d30 0x7fe2480d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe247e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe248162f00 con 0x7fe2480d50c0 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe247e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe24806da00 con 0x7fe248161d30 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe247e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe2480d2a60 con 0x7fe248161950 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe23fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe248161d30 0x7fe2480d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe23fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe248161d30 0x7fe2480d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40008/0 (socket says 172.21.15.86:40008) 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe23fe00640 1 -- 172.21.15.86:0/3463478 learned_addr learned my addr 172.21.15.86:0/3463478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe247400640 1 --1- 172.21.15.86:0/3463478 >> v1:172.21.15.86:6789/0 conn(0x7fe2480d50c0 0x7fe248162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44364/0 (socket says 172.21.15.86:44364) 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe246a00640 1 --2- 172.21.15.86:0/3463478 >> v2:172.21.15.196:3300/0 conn(0x7fe248161950 0x7fe2480d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe23fe00640 1 -- 172.21.15.86:0/3463478 >> v2:172.21.15.196:3300/0 conn(0x7fe248161950 msgr2=0x7fe2480d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe23fe00640 1 --2- 172.21.15.86:0/3463478 >> v2:172.21.15.196:3300/0 conn(0x7fe248161950 0x7fe2480d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe23fe00640 1 -- 172.21.15.86:0/3463478 >> v1:172.21.15.86:6789/0 conn(0x7fe2480d50c0 legacy=0x7fe248162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe23fe00640 1 -- 172.21.15.86:0/3463478 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24806e340 con 0x7fe248161d30 2024-03-22T04:19:22.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.184+0000 7fe246000640 1 -- 172.21.15.86:0/3463478 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979329247 0 0) 0x7fe248162f00 con 0x7fe2480d50c0 2024-03-22T04:19:22.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe23fe00640 1 --2- 172.21.15.86:0/3463478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480d2ef0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe24808ed10 tx=0x7fe23001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd31269356f676fc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe246000640 1 -- 172.21.15.86:0/3463478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2300b0070 con 0x7fe248161d30 2024-03-22T04:19:22.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe246000640 1 -- 172.21.15.86:0/3463478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2300ab600 con 0x7fe248161d30 2024-03-22T04:19:22.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe246000640 1 -- 172.21.15.86:0/3463478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2300ab9d0 con 0x7fe248161d30 2024-03-22T04:19:22.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 -- 172.21.15.86:0/3463478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 msgr2=0x7fe2480d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 --2- 172.21.15.86:0/3463478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480d2ef0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe24808ed10 tx=0x7fe23001c490 comp rx=0 tx=0).stop 2024-03-22T04:19:22.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 -- 172.21.15.86:0/3463478 shutdown_connections 2024-03-22T04:19:22.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 --2- 172.21.15.86:0/3463478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480d2ef0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 --2- 172.21.15.86:0/3463478 >> v2:172.21.15.196:3300/0 conn(0x7fe248161950 0x7fe2480d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 -- 172.21.15.86:0/3463478 >> 172.21.15.86:0/3463478 conn(0x7fe24808c000 msgr2=0x7fe2480d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:22.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 -- 172.21.15.86:0/3463478 shutdown_connections 2024-03-22T04:19:22.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 -- 172.21.15.86:0/3463478 wait complete. 2024-03-22T04:19:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 Processor -- start 2024-03-22T04:19:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.188+0000 7fe247e00640 1 -- start start 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2480d50c0 0x7fe2480c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe248161950 0x7fe248159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2480b2640 con 0x7fe248161d30 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2480b24c0 con 0x7fe2480d50c0 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe2480b27c0 con 0x7fe248161950 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe246a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2480d50c0 0x7fe2480c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480b7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40012/0 (socket says 172.21.15.86:40012) 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 -- 172.21.15.86:0/3456713194 learned_addr learned my addr 172.21.15.86:0/3456713194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe23fe00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe248161950 0x7fe248159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 -- 172.21.15.86:0/3456713194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe248161950 msgr2=0x7fe248159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe248161950 0x7fe248159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 -- 172.21.15.86:0/3456713194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2480d50c0 msgr2=0x7fe2480c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2480d50c0 0x7fe2480c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 -- 172.21.15.86:0/3456713194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2480cc400 con 0x7fe248161d30 2024-03-22T04:19:22.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247400640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480b7040 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe24000c7a0 tx=0x7fe24000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe23ea00640 1 -- 172.21.15.86:0/3456713194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240018070 con 0x7fe248161d30 2024-03-22T04:19:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe23ea00640 1 -- 172.21.15.86:0/3456713194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24000a640 con 0x7fe248161d30 2024-03-22T04:19:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe23ea00640 1 -- 172.21.15.86:0/3456713194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24000aab0 con 0x7fe248161d30 2024-03-22T04:19:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2480ca260 con 0x7fe248161d30 2024-03-22T04:19:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2480ca820 con 0x7fe248161d30 2024-03-22T04:19:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe248074470 con 0x7fe248161d30 2024-03-22T04:19:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.192+0000 7fe23ea00640 1 -- 172.21.15.86:0/3456713194 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe24000ac50 con 0x7fe248161d30 2024-03-22T04:19:22.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.196+0000 7fe23ea00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe224067b50 0x7fe22406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.196+0000 7fe23ea00640 1 -- 172.21.15.86:0/3456713194 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fe240085600 con 0x7fe248161d30 2024-03-22T04:19:22.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.196+0000 7fe246a00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe224067b50 0x7fe22406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.196+0000 7fe246a00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe224067b50 0x7fe22406a010 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe238009b60 tx=0x7fe238009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.200+0000 7fe23ea00640 1 -- 172.21.15.86:0/3456713194 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe240055790 con 0x7fe248161d30 2024-03-22T04:19:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.348+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2480d1b00 con 0x7fe224067b50 2024-03-22T04:19:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.356+0000 7fe23ea00640 1 -- 172.21.15.86:0/3456713194 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252476 (secure 0 0 0) 0x7fe2480d1b00 con 0x7fe224067b50 2024-03-22T04:19:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:22.361 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:19:22.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe224067b50 msgr2=0x7fe22406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe224067b50 0x7fe22406a010 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe238009b60 tx=0x7fe238009190 comp rx=0 tx=0).stop 2024-03-22T04:19:22.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 msgr2=0x7fe2480b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480b7040 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe24000c7a0 tx=0x7fe24000cc60 comp rx=0 tx=0).stop 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 shutdown_connections 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe224067b50 0x7fe22406a010 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe248161d30 0x7fe2480b7040 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe248161950 0x7fe248159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 --2- 172.21.15.86:0/3456713194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2480d50c0 0x7fe2480c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 >> 172.21.15.86:0/3456713194 conn(0x7fe24808c000 msgr2=0x7fe2480b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 shutdown_connections 2024-03-22T04:19:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.360+0000 7fe247e00640 1 -- 172.21.15.86:0/3456713194 wait complete. 2024-03-22T04:19:22.379 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-03-22T04:19:21.909923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261196,"kb_used_data":11708,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713524,"statfs":{"total":773094113280,"available":772826648576,"internally_reserved":0,"allocated":11988992,"data_stored":6174816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.616093"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.332968+0000","last_change":"2024-03-22T04:19:18.328825+0000","last_active":"2024-03-22T04:19:18.332968+0000","last_peered":"2024-03-22T04:19:18.332968+0000","last_clean":"2024-03-22T04:19:18.332968+0000","last_became_active":"2024-03-22T04:19:18.328524+0000","last_became_peered":"2024-03-22T04:19:18.328524+0000","last_unstale":"2024-03-22T04:19:18.332968+0000","last_undegraded":"2024-03-22T04:19:18.332968+0000","last_fullsized":"2024-03-22T04:19:18.332968+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:04.949643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778610+0000","last_change":"2024-03-22T04:19:18.333041+0000","last_active":"2024-03-22T04:19:19.778610+0000","last_peered":"2024-03-22T04:19:19.778610+0000","last_clean":"2024-03-22T04:19:19.778610+0000","last_became_active":"2024-03-22T04:19:18.332405+0000","last_became_peered":"2024-03-22T04:19:18.332405+0000","last_unstale":"2024-03-22T04:19:19.778610+0000","last_undegraded":"2024-03-22T04:19:19.778610+0000","last_fullsized":"2024-03-22T04:19:19.778610+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:53:23.864360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323957+0000","last_change":"2024-03-22T04:19:18.324516+0000","last_active":"2024-03-22T04:19:19.323957+0000","last_peered":"2024-03-22T04:19:19.323957+0000","last_clean":"2024-03-22T04:19:19.323957+0000","last_became_active":"2024-03-22T04:19:18.324177+0000","last_became_peered":"2024-03-22T04:19:18.324177+0000","last_unstale":"2024-03-22T04:19:19.323957+0000","last_undegraded":"2024-03-22T04:19:19.323957+0000","last_fullsized":"2024-03-22T04:19:19.323957+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:58:15.390106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.336238+0000","last_change":"2024-03-22T04:19:18.330635+0000","last_active":"2024-03-22T04:19:18.336238+0000","last_peered":"2024-03-22T04:19:18.336238+0000","last_clean":"2024-03-22T04:19:18.336238+0000","last_became_active":"2024-03-22T04:19:18.330004+0000","last_became_peered":"2024-03-22T04:19:18.330004+0000","last_unstale":"2024-03-22T04:19:18.336238+0000","last_undegraded":"2024-03-22T04:19:18.336238+0000","last_fullsized":"2024-03-22T04:19:18.336238+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:03:00.854292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.336247+0000","last_change":"2024-03-22T04:19:18.330163+0000","last_active":"2024-03-22T04:19:18.336247+0000","last_peered":"2024-03-22T04:19:18.336247+0000","last_clean":"2024-03-22T04:19:18.336247+0000","last_became_active":"2024-03-22T04:19:18.329885+0000","last_became_peered":"2024-03-22T04:19:18.329885+0000","last_unstale":"2024-03-22T04:19:18.336247+0000","last_undegraded":"2024-03-22T04:19:18.336247+0000","last_fullsized":"2024-03-22T04:19:18.336247+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:23:11.579349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322782+0000","last_change":"2024-03-22T04:19:18.321339+0000","last_active":"2024-03-22T04:19:19.322782+0000","last_peered":"2024-03-22T04:19:19.322782+0000","last_clean":"2024-03-22T04:19:19.322782+0000","last_became_active":"2024-03-22T04:19:18.321082+0000","last_became_peered":"2024-03-22T04:19:18.321082+0000","last_unstale":"2024-03-22T04:19:19.322782+0000","last_undegraded":"2024-03-22T04:19:19.322782+0000","last_fullsized":"2024-03-22T04:19:19.322782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:51.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322622+0000","last_change":"2024-03-22T04:19:18.320954+0000","last_active":"2024-03-22T04:19:19.322622+0000","last_peered":"2024-03-22T04:19:19.322622+0000","last_clean":"2024-03-22T04:19:19.322622+0000","last_became_active":"2024-03-22T04:19:18.320646+0000","last_became_peered":"2024-03-22T04:19:18.320646+0000","last_unstale":"2024-03-22T04:19:19.322622+0000","last_undegraded":"2024-03-22T04:19:19.322622+0000","last_fullsized":"2024-03-22T04:19:19.322622+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:58:15.390106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323209+0000","last_change":"2024-03-22T04:19:18.339388+0000","last_active":"2024-03-22T04:19:19.323209+0000","last_peered":"2024-03-22T04:19:19.323209+0000","last_clean":"2024-03-22T04:19:19.323209+0000","last_became_active":"2024-03-22T04:19:18.338943+0000","last_became_peered":"2024-03-22T04:19:18.338943+0000","last_unstale":"2024-03-22T04:19:19.323209+0000","last_undegraded":"2024-03-22T04:19:19.323209+0000","last_fullsized":"2024-03-22T04:19:19.323209+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:06:17.232427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779190+0000","last_change":"2024-03-22T04:19:18.337616+0000","last_active":"2024-03-22T04:19:19.779190+0000","last_peered":"2024-03-22T04:19:19.779190+0000","last_clean":"2024-03-22T04:19:19.779190+0000","last_became_active":"2024-03-22T04:19:18.336011+0000","last_became_peered":"2024-03-22T04:19:18.336011+0000","last_unstale":"2024-03-22T04:19:19.779190+0000","last_undegraded":"2024-03-22T04:19:19.779190+0000","last_fullsized":"2024-03-22T04:19:19.779190+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:01:10.519103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324496+0000","last_change":"2024-03-22T04:19:18.339319+0000","last_active":"2024-03-22T04:19:19.324496+0000","last_peered":"2024-03-22T04:19:19.324496+0000","last_clean":"2024-03-22T04:19:19.324496+0000","last_became_active":"2024-03-22T04:19:18.339026+0000","last_became_peered":"2024-03-22T04:19:18.339026+0000","last_unstale":"2024-03-22T04:19:19.324496+0000","last_undegraded":"2024-03-22T04:19:19.324496+0000","last_fullsized":"2024-03-22T04:19:19.324496+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:15:06.596648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779054+0000","last_change":"2024-03-22T04:19:18.332716+0000","last_active":"2024-03-22T04:19:19.779054+0000","last_peered":"2024-03-22T04:19:19.779054+0000","last_clean":"2024-03-22T04:19:19.779054+0000","last_became_active":"2024-03-22T04:19:18.332433+0000","last_became_peered":"2024-03-22T04:19:18.332433+0000","last_unstale":"2024-03-22T04:19:19.779054+0000","last_undegraded":"2024-03-22T04:19:19.779054+0000","last_fullsized":"2024-03-22T04:19:19.779054+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:32:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322941+0000","last_change":"2024-03-22T04:19:18.336750+0000","last_active":"2024-03-22T04:19:19.322941+0000","last_peered":"2024-03-22T04:19:19.322941+0000","last_clean":"2024-03-22T04:19:19.322941+0000","last_became_active":"2024-03-22T04:19:18.336489+0000","last_became_peered":"2024-03-22T04:19:18.336489+0000","last_unstale":"2024-03-22T04:19:19.322941+0000","last_undegraded":"2024-03-22T04:19:19.322941+0000","last_fullsized":"2024-03-22T04:19:19.322941+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:06:17.232427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.331049+0000","last_change":"2024-03-22T04:19:18.325476+0000","last_active":"2024-03-22T04:19:18.331049+0000","last_peered":"2024-03-22T04:19:18.331049+0000","last_clean":"2024-03-22T04:19:18.331049+0000","last_became_active":"2024-03-22T04:19:18.325137+0000","last_became_peered":"2024-03-22T04:19:18.325137+0000","last_unstale":"2024-03-22T04:19:18.331049+0000","last_undegraded":"2024-03-22T04:19:18.331049+0000","last_fullsized":"2024-03-22T04:19:18.331049+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:15.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323225+0000","last_change":"2024-03-22T04:19:18.336729+0000","last_active":"2024-03-22T04:19:19.323225+0000","last_peered":"2024-03-22T04:19:19.323225+0000","last_clean":"2024-03-22T04:19:19.323225+0000","last_became_active":"2024-03-22T04:19:18.336469+0000","last_became_peered":"2024-03-22T04:19:18.336469+0000","last_unstale":"2024-03-22T04:19:19.323225+0000","last_undegraded":"2024-03-22T04:19:19.323225+0000","last_fullsized":"2024-03-22T04:19:19.323225+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:49:40.884723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778479+0000","last_change":"2024-03-22T04:19:18.329014+0000","last_active":"2024-03-22T04:19:19.778479+0000","last_peered":"2024-03-22T04:19:19.778479+0000","last_clean":"2024-03-22T04:19:19.778479+0000","last_became_active":"2024-03-22T04:19:18.328731+0000","last_became_peered":"2024-03-22T04:19:18.328731+0000","last_unstale":"2024-03-22T04:19:19.778479+0000","last_undegraded":"2024-03-22T04:19:19.778479+0000","last_fullsized":"2024-03-22T04:19:19.778479+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:27:36.778603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742592+0000","last_change":"2024-03-22T04:19:18.331919+0000","last_active":"2024-03-22T04:19:19.742592+0000","last_peered":"2024-03-22T04:19:19.742592+0000","last_clean":"2024-03-22T04:19:19.742592+0000","last_became_active":"2024-03-22T04:19:18.330483+0000","last_became_peered":"2024-03-22T04:19:18.330483+0000","last_unstale":"2024-03-22T04:19:19.742592+0000","last_undegraded":"2024-03-22T04:19:19.742592+0000","last_fullsized":"2024-03-22T04:19:19.742592+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:07:45.953360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324254+0000","last_change":"2024-03-22T04:19:18.319147+0000","last_active":"2024-03-22T04:19:19.324254+0000","last_peered":"2024-03-22T04:19:19.324254+0000","last_clean":"2024-03-22T04:19:19.324254+0000","last_became_active":"2024-03-22T04:19:18.318830+0000","last_became_peered":"2024-03-22T04:19:18.318830+0000","last_unstale":"2024-03-22T04:19:19.324254+0000","last_undegraded":"2024-03-22T04:19:19.324254+0000","last_fullsized":"2024-03-22T04:19:19.324254+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:51.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779193+0000","last_change":"2024-03-22T04:19:18.333771+0000","last_active":"2024-03-22T04:19:19.779193+0000","last_peered":"2024-03-22T04:19:19.779193+0000","last_clean":"2024-03-22T04:19:19.779193+0000","last_became_active":"2024-03-22T04:19:18.332724+0000","last_became_peered":"2024-03-22T04:19:18.332724+0000","last_unstale":"2024-03-22T04:19:19.779193+0000","last_undegraded":"2024-03-22T04:19:19.779193+0000","last_fullsized":"2024-03-22T04:19:19.779193+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:09.202859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.330525+0000","last_change":"2024-03-22T04:19:18.323710+0000","last_active":"2024-03-22T04:19:18.330525+0000","last_peered":"2024-03-22T04:19:18.330525+0000","last_clean":"2024-03-22T04:19:18.330525+0000","last_became_active":"2024-03-22T04:19:18.323431+0000","last_became_peered":"2024-03-22T04:19:18.323431+0000","last_unstale":"2024-03-22T04:19:18.330525+0000","last_undegraded":"2024-03-22T04:19:18.330525+0000","last_fullsized":"2024-03-22T04:19:18.330525+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:15:02.584209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742923+0000","last_change":"2024-03-22T04:19:18.331951+0000","last_active":"2024-03-22T04:19:19.742923+0000","last_peered":"2024-03-22T04:19:19.742923+0000","last_clean":"2024-03-22T04:19:19.742923+0000","last_became_active":"2024-03-22T04:19:18.331658+0000","last_became_peered":"2024-03-22T04:19:18.331658+0000","last_unstale":"2024-03-22T04:19:19.742923+0000","last_undegraded":"2024-03-22T04:19:19.742923+0000","last_fullsized":"2024-03-22T04:19:19.742923+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:42:29.640947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323984+0000","last_change":"2024-03-22T04:19:18.331587+0000","last_active":"2024-03-22T04:19:19.323984+0000","last_peered":"2024-03-22T04:19:19.323984+0000","last_clean":"2024-03-22T04:19:19.323984+0000","last_became_active":"2024-03-22T04:19:18.329754+0000","last_became_peered":"2024-03-22T04:19:18.329754+0000","last_unstale":"2024-03-22T04:19:19.323984+0000","last_undegraded":"2024-03-22T04:19:19.323984+0000","last_fullsized":"2024-03-22T04:19:19.323984+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:42:29.640947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778553+0000","last_change":"2024-03-22T04:19:18.343320+0000","last_active":"2024-03-22T04:19:19.778553+0000","last_peered":"2024-03-22T04:19:19.778553+0000","last_clean":"2024-03-22T04:19:19.778553+0000","last_became_active":"2024-03-22T04:19:18.342992+0000","last_became_peered":"2024-03-22T04:19:18.342992+0000","last_unstale":"2024-03-22T04:19:19.778553+0000","last_undegraded":"2024-03-22T04:19:19.778553+0000","last_fullsized":"2024-03-22T04:19:19.778553+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:05:01.060782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323499+0000","last_change":"2024-03-22T04:19:18.333136+0000","last_active":"2024-03-22T04:19:19.323499+0000","last_peered":"2024-03-22T04:19:19.323499+0000","last_clean":"2024-03-22T04:19:19.323499+0000","last_became_active":"2024-03-22T04:19:18.332720+0000","last_became_peered":"2024-03-22T04:19:18.332720+0000","last_unstale":"2024-03-22T04:19:19.323499+0000","last_undegraded":"2024-03-22T04:19:19.323499+0000","last_fullsized":"2024-03-22T04:19:19.323499+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:15:35.873149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323243+0000","last_change":"2024-03-22T04:19:18.334522+0000","last_active":"2024-03-22T04:19:19.323243+0000","last_peered":"2024-03-22T04:19:19.323243+0000","last_clean":"2024-03-22T04:19:19.323243+0000","last_became_active":"2024-03-22T04:19:18.334225+0000","last_became_peered":"2024-03-22T04:19:18.334225+0000","last_unstale":"2024-03-22T04:19:19.323243+0000","last_undegraded":"2024-03-22T04:19:19.323243+0000","last_fullsized":"2024-03-22T04:19:19.323243+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:17:34.785225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742975+0000","last_change":"2024-03-22T04:19:18.333501+0000","last_active":"2024-03-22T04:19:19.742975+0000","last_peered":"2024-03-22T04:19:19.742975+0000","last_clean":"2024-03-22T04:19:19.742975+0000","last_became_active":"2024-03-22T04:19:18.333181+0000","last_became_peered":"2024-03-22T04:19:18.333181+0000","last_unstale":"2024-03-22T04:19:19.742975+0000","last_undegraded":"2024-03-22T04:19:19.742975+0000","last_fullsized":"2024-03-22T04:19:19.742975+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:51.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742928+0000","last_change":"2024-03-22T04:19:18.332928+0000","last_active":"2024-03-22T04:19:19.742928+0000","last_peered":"2024-03-22T04:19:19.742928+0000","last_clean":"2024-03-22T04:19:19.742928+0000","last_became_active":"2024-03-22T04:19:18.332641+0000","last_became_peered":"2024-03-22T04:19:18.332641+0000","last_unstale":"2024-03-22T04:19:19.742928+0000","last_undegraded":"2024-03-22T04:19:19.742928+0000","last_fullsized":"2024-03-22T04:19:19.742928+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:58:15.390106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742860+0000","last_change":"2024-03-22T04:19:18.330855+0000","last_active":"2024-03-22T04:19:19.742860+0000","last_peered":"2024-03-22T04:19:19.742860+0000","last_clean":"2024-03-22T04:19:19.742860+0000","last_became_active":"2024-03-22T04:19:18.329496+0000","last_became_peered":"2024-03-22T04:19:18.329496+0000","last_unstale":"2024-03-22T04:19:19.742860+0000","last_undegraded":"2024-03-22T04:19:19.742860+0000","last_fullsized":"2024-03-22T04:19:19.742860+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:40.104909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322998+0000","last_change":"2024-03-22T04:19:18.326025+0000","last_active":"2024-03-22T04:19:19.322998+0000","last_peered":"2024-03-22T04:19:19.322998+0000","last_clean":"2024-03-22T04:19:19.322998+0000","last_became_active":"2024-03-22T04:19:18.325745+0000","last_became_peered":"2024-03-22T04:19:18.325745+0000","last_unstale":"2024-03-22T04:19:19.322998+0000","last_undegraded":"2024-03-22T04:19:19.322998+0000","last_fullsized":"2024-03-22T04:19:19.322998+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:17:34.785225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324144+0000","last_change":"2024-03-22T04:19:18.342111+0000","last_active":"2024-03-22T04:19:19.324144+0000","last_peered":"2024-03-22T04:19:19.324144+0000","last_clean":"2024-03-22T04:19:19.324144+0000","last_became_active":"2024-03-22T04:19:18.341836+0000","last_became_peered":"2024-03-22T04:19:18.341836+0000","last_unstale":"2024-03-22T04:19:19.324144+0000","last_undegraded":"2024-03-22T04:19:19.324144+0000","last_fullsized":"2024-03-22T04:19:19.324144+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:24:42.122978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778573+0000","last_change":"2024-03-22T04:19:18.330796+0000","last_active":"2024-03-22T04:19:19.778573+0000","last_peered":"2024-03-22T04:19:19.778573+0000","last_clean":"2024-03-22T04:19:19.778573+0000","last_became_active":"2024-03-22T04:19:18.330538+0000","last_became_peered":"2024-03-22T04:19:18.330538+0000","last_unstale":"2024-03-22T04:19:19.778573+0000","last_undegraded":"2024-03-22T04:19:19.778573+0000","last_fullsized":"2024-03-22T04:19:19.778573+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:53:32.642446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322658+0000","last_change":"2024-03-22T04:19:18.335946+0000","last_active":"2024-03-22T04:19:19.322658+0000","last_peered":"2024-03-22T04:19:19.322658+0000","last_clean":"2024-03-22T04:19:19.322658+0000","last_became_active":"2024-03-22T04:19:18.335631+0000","last_became_peered":"2024-03-22T04:19:18.335631+0000","last_unstale":"2024-03-22T04:19:19.322658+0000","last_undegraded":"2024-03-22T04:19:19.322658+0000","last_fullsized":"2024-03-22T04:19:19.322658+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:42:29.640947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778427+0000","last_change":"2024-03-22T04:19:18.339420+0000","last_active":"2024-03-22T04:19:19.778427+0000","last_peered":"2024-03-22T04:19:19.778427+0000","last_clean":"2024-03-22T04:19:19.778427+0000","last_became_active":"2024-03-22T04:19:18.336603+0000","last_became_peered":"2024-03-22T04:19:18.336603+0000","last_unstale":"2024-03-22T04:19:19.778427+0000","last_undegraded":"2024-03-22T04:19:19.778427+0000","last_fullsized":"2024-03-22T04:19:19.778427+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:41:53.294167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323210+0000","last_change":"2024-03-22T04:19:18.319201+0000","last_active":"2024-03-22T04:19:19.323210+0000","last_peered":"2024-03-22T04:19:19.323210+0000","last_clean":"2024-03-22T04:19:19.323210+0000","last_became_active":"2024-03-22T04:19:18.318909+0000","last_became_peered":"2024-03-22T04:19:18.318909+0000","last_unstale":"2024-03-22T04:19:19.323210+0000","last_undegraded":"2024-03-22T04:19:19.323210+0000","last_fullsized":"2024-03-22T04:19:19.323210+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:40.104909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779207+0000","last_change":"2024-03-22T04:19:17.325200+0000","last_active":"2024-03-22T04:19:19.779207+0000","last_peered":"2024-03-22T04:19:19.779207+0000","last_clean":"2024-03-22T04:19:19.779207+0000","last_became_active":"2024-03-22T04:19:17.324942+0000","last_became_peered":"2024-03-22T04:19:17.324942+0000","last_unstale":"2024-03-22T04:19:19.779207+0000","last_undegraded":"2024-03-22T04:19:19.779207+0000","last_fullsized":"2024-03-22T04:19:19.779207+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:57.284496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323557+0000","last_change":"2024-03-22T04:19:18.333628+0000","last_active":"2024-03-22T04:19:19.323557+0000","last_peered":"2024-03-22T04:19:19.323557+0000","last_clean":"2024-03-22T04:19:19.323557+0000","last_became_active":"2024-03-22T04:19:18.332722+0000","last_became_peered":"2024-03-22T04:19:18.332722+0000","last_unstale":"2024-03-22T04:19:19.323557+0000","last_undegraded":"2024-03-22T04:19:19.323557+0000","last_fullsized":"2024-03-22T04:19:19.323557+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:34:25.635331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324154+0000","last_change":"2024-03-22T04:19:17.315628+0000","last_active":"2024-03-22T04:19:19.324154+0000","last_peered":"2024-03-22T04:19:19.324154+0000","last_clean":"2024-03-22T04:19:19.324154+0000","last_became_active":"2024-03-22T04:19:17.315012+0000","last_became_peered":"2024-03-22T04:19:17.315012+0000","last_unstale":"2024-03-22T04:19:19.324154+0000","last_undegraded":"2024-03-22T04:19:19.324154+0000","last_fullsized":"2024-03-22T04:19:19.324154+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:08.196395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778767+0000","last_change":"2024-03-22T04:19:18.334880+0000","last_active":"2024-03-22T04:19:19.778767+0000","last_peered":"2024-03-22T04:19:19.778767+0000","last_clean":"2024-03-22T04:19:19.778767+0000","last_became_active":"2024-03-22T04:19:18.334604+0000","last_became_peered":"2024-03-22T04:19:18.334604+0000","last_unstale":"2024-03-22T04:19:19.778767+0000","last_undegraded":"2024-03-22T04:19:19.778767+0000","last_fullsized":"2024-03-22T04:19:19.778767+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:43:48.533737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779060+0000","last_change":"2024-03-22T04:19:17.321241+0000","last_active":"2024-03-22T04:19:19.779060+0000","last_peered":"2024-03-22T04:19:19.779060+0000","last_clean":"2024-03-22T04:19:19.779060+0000","last_became_active":"2024-03-22T04:19:17.320943+0000","last_became_peered":"2024-03-22T04:19:17.320943+0000","last_unstale":"2024-03-22T04:19:19.779060+0000","last_undegraded":"2024-03-22T04:19:19.779060+0000","last_fullsized":"2024-03-22T04:19:19.779060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:34.927515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778492+0000","last_change":"2024-03-22T04:19:18.327481+0000","last_active":"2024-03-22T04:19:19.778492+0000","last_peered":"2024-03-22T04:19:19.778492+0000","last_clean":"2024-03-22T04:19:19.778492+0000","last_became_active":"2024-03-22T04:19:18.327209+0000","last_became_peered":"2024-03-22T04:19:18.327209+0000","last_unstale":"2024-03-22T04:19:19.778492+0000","last_undegraded":"2024-03-22T04:19:19.778492+0000","last_fullsized":"2024-03-22T04:19:19.778492+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:03:04.238555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324418+0000","last_change":"2024-03-22T04:19:17.328186+0000","last_active":"2024-03-22T04:19:19.324418+0000","last_peered":"2024-03-22T04:19:19.324418+0000","last_clean":"2024-03-22T04:19:19.324418+0000","last_became_active":"2024-03-22T04:19:17.327961+0000","last_became_peered":"2024-03-22T04:19:17.327961+0000","last_unstale":"2024-03-22T04:19:19.324418+0000","last_undegraded":"2024-03-22T04:19:19.324418+0000","last_fullsized":"2024-03-22T04:19:19.324418+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:55.072203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.328842+0000","last_change":"2024-03-22T04:19:18.319299+0000","last_active":"2024-03-22T04:19:18.328842+0000","last_peered":"2024-03-22T04:19:18.328842+0000","last_clean":"2024-03-22T04:19:18.328842+0000","last_became_active":"2024-03-22T04:19:18.318985+0000","last_became_peered":"2024-03-22T04:19:18.318985+0000","last_unstale":"2024-03-22T04:19:18.328842+0000","last_undegraded":"2024-03-22T04:19:18.328842+0000","last_fullsized":"2024-03-22T04:19:18.328842+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:36.023100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778571+0000","last_change":"2024-03-22T04:19:17.323558+0000","last_active":"2024-03-22T04:19:19.778571+0000","last_peered":"2024-03-22T04:19:19.778571+0000","last_clean":"2024-03-22T04:19:19.778571+0000","last_became_active":"2024-03-22T04:19:17.323233+0000","last_became_peered":"2024-03-22T04:19:17.323233+0000","last_unstale":"2024-03-22T04:19:19.778571+0000","last_undegraded":"2024-03-22T04:19:19.778571+0000","last_fullsized":"2024-03-22T04:19:19.778571+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:23:20.948143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.333015+0000","last_change":"2024-03-22T04:19:18.328570+0000","last_active":"2024-03-22T04:19:18.333015+0000","last_peered":"2024-03-22T04:19:18.333015+0000","last_clean":"2024-03-22T04:19:18.333015+0000","last_became_active":"2024-03-22T04:19:18.328296+0000","last_became_peered":"2024-03-22T04:19:18.328296+0000","last_unstale":"2024-03-22T04:19:18.333015+0000","last_undegraded":"2024-03-22T04:19:18.333015+0000","last_fullsized":"2024-03-22T04:19:18.333015+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:30:33.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742522+0000","last_change":"2024-03-22T04:19:17.323878+0000","last_active":"2024-03-22T04:19:19.742522+0000","last_peered":"2024-03-22T04:19:19.742522+0000","last_clean":"2024-03-22T04:19:19.742522+0000","last_became_active":"2024-03-22T04:19:17.323608+0000","last_became_peered":"2024-03-22T04:19:17.323608+0000","last_unstale":"2024-03-22T04:19:19.742522+0000","last_undegraded":"2024-03-22T04:19:19.742522+0000","last_fullsized":"2024-03-22T04:19:19.742522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:43:47.527273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778981+0000","last_change":"2024-03-22T04:19:18.333147+0000","last_active":"2024-03-22T04:19:19.778981+0000","last_peered":"2024-03-22T04:19:19.778981+0000","last_clean":"2024-03-22T04:19:19.778981+0000","last_became_active":"2024-03-22T04:19:18.332870+0000","last_became_peered":"2024-03-22T04:19:18.332870+0000","last_unstale":"2024-03-22T04:19:19.778981+0000","last_undegraded":"2024-03-22T04:19:19.778981+0000","last_fullsized":"2024-03-22T04:19:19.778981+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:23:27.050207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742545+0000","last_change":"2024-03-22T04:19:17.324924+0000","last_active":"2024-03-22T04:19:19.742545+0000","last_peered":"2024-03-22T04:19:19.742545+0000","last_clean":"2024-03-22T04:19:19.742545+0000","last_became_active":"2024-03-22T04:19:17.324617+0000","last_became_peered":"2024-03-22T04:19:17.324617+0000","last_unstale":"2024-03-22T04:19:19.742545+0000","last_undegraded":"2024-03-22T04:19:19.742545+0000","last_fullsized":"2024-03-22T04:19:19.742545+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:27:35.772140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.339782+0000","last_change":"2024-03-22T04:19:18.337118+0000","last_active":"2024-03-22T04:19:18.339782+0000","last_peered":"2024-03-22T04:19:18.339782+0000","last_clean":"2024-03-22T04:19:18.339782+0000","last_became_active":"2024-03-22T04:19:18.336842+0000","last_became_peered":"2024-03-22T04:19:18.336842+0000","last_unstale":"2024-03-22T04:19:18.339782+0000","last_undegraded":"2024-03-22T04:19:18.339782+0000","last_fullsized":"2024-03-22T04:19:18.339782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:55.930055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778539+0000","last_change":"2024-03-22T04:19:17.324483+0000","last_active":"2024-03-22T04:19:19.778539+0000","last_peered":"2024-03-22T04:19:19.778539+0000","last_clean":"2024-03-22T04:19:19.778539+0000","last_became_active":"2024-03-22T04:19:17.324174+0000","last_became_peered":"2024-03-22T04:19:17.324174+0000","last_unstale":"2024-03-22T04:19:19.778539+0000","last_undegraded":"2024-03-22T04:19:19.778539+0000","last_fullsized":"2024-03-22T04:19:19.778539+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:25.379236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.332825+0000","last_change":"2024-03-22T04:19:18.326890+0000","last_active":"2024-03-22T04:19:18.332825+0000","last_peered":"2024-03-22T04:19:18.332825+0000","last_clean":"2024-03-22T04:19:18.332825+0000","last_became_active":"2024-03-22T04:19:18.324893+0000","last_became_peered":"2024-03-22T04:19:18.324893+0000","last_unstale":"2024-03-22T04:19:18.332825+0000","last_undegraded":"2024-03-22T04:19:18.332825+0000","last_fullsized":"2024-03-22T04:19:18.332825+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:01:00.508043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323353+0000","last_change":"2024-03-22T04:19:17.321789+0000","last_active":"2024-03-22T04:19:19.323353+0000","last_peered":"2024-03-22T04:19:19.323353+0000","last_clean":"2024-03-22T04:19:19.323353+0000","last_became_active":"2024-03-22T04:19:17.321310+0000","last_became_peered":"2024-03-22T04:19:17.321310+0000","last_unstale":"2024-03-22T04:19:19.323353+0000","last_undegraded":"2024-03-22T04:19:19.323353+0000","last_fullsized":"2024-03-22T04:19:19.323353+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:53:22.857897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.339727+0000","last_change":"2024-03-22T04:19:18.335630+0000","last_active":"2024-03-22T04:19:18.339727+0000","last_peered":"2024-03-22T04:19:18.339727+0000","last_clean":"2024-03-22T04:19:18.339727+0000","last_became_active":"2024-03-22T04:19:18.335380+0000","last_became_peered":"2024-03-22T04:19:18.335380+0000","last_unstale":"2024-03-22T04:19:18.339727+0000","last_undegraded":"2024-03-22T04:19:18.339727+0000","last_fullsized":"2024-03-22T04:19:18.339727+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:44:13.800238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324531+0000","last_change":"2024-03-22T04:19:17.324485+0000","last_active":"2024-03-22T04:19:19.324531+0000","last_peered":"2024-03-22T04:19:19.324531+0000","last_clean":"2024-03-22T04:19:19.324531+0000","last_became_active":"2024-03-22T04:19:17.324207+0000","last_became_peered":"2024-03-22T04:19:17.324207+0000","last_unstale":"2024-03-22T04:19:19.324531+0000","last_undegraded":"2024-03-22T04:19:19.324531+0000","last_fullsized":"2024-03-22T04:19:19.324531+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:01:09.512640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323897+0000","last_change":"2024-03-22T04:19:18.340800+0000","last_active":"2024-03-22T04:19:19.323897+0000","last_peered":"2024-03-22T04:19:19.323897+0000","last_clean":"2024-03-22T04:19:19.323897+0000","last_became_active":"2024-03-22T04:19:18.340538+0000","last_became_peered":"2024-03-22T04:19:18.340538+0000","last_unstale":"2024-03-22T04:19:19.323897+0000","last_undegraded":"2024-03-22T04:19:19.323897+0000","last_fullsized":"2024-03-22T04:19:19.323897+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:34:44.461984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323173+0000","last_change":"2024-03-22T04:19:17.325079+0000","last_active":"2024-03-22T04:19:19.323173+0000","last_peered":"2024-03-22T04:19:19.323173+0000","last_clean":"2024-03-22T04:19:19.323173+0000","last_became_active":"2024-03-22T04:19:17.324808+0000","last_became_peered":"2024-03-22T04:19:17.324808+0000","last_unstale":"2024-03-22T04:19:19.323173+0000","last_undegraded":"2024-03-22T04:19:19.323173+0000","last_fullsized":"2024-03-22T04:19:19.323173+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:08.196395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778617+0000","last_change":"2024-03-22T04:19:18.331742+0000","last_active":"2024-03-22T04:19:19.778617+0000","last_peered":"2024-03-22T04:19:19.778617+0000","last_clean":"2024-03-22T04:19:19.778617+0000","last_became_active":"2024-03-22T04:19:18.331422+0000","last_became_peered":"2024-03-22T04:19:18.331422+0000","last_unstale":"2024-03-22T04:19:19.778617+0000","last_undegraded":"2024-03-22T04:19:19.778617+0000","last_fullsized":"2024-03-22T04:19:19.778617+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:06.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778877+0000","last_change":"2024-03-22T04:19:17.323871+0000","last_active":"2024-03-22T04:19:19.778877+0000","last_peered":"2024-03-22T04:19:19.778877+0000","last_clean":"2024-03-22T04:19:19.778877+0000","last_became_active":"2024-03-22T04:19:17.323633+0000","last_became_peered":"2024-03-22T04:19:17.323633+0000","last_unstale":"2024-03-22T04:19:19.778877+0000","last_undegraded":"2024-03-22T04:19:19.778877+0000","last_fullsized":"2024-03-22T04:19:19.778877+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:48:54.879336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778792+0000","last_change":"2024-03-22T04:19:18.333806+0000","last_active":"2024-03-22T04:19:19.778792+0000","last_peered":"2024-03-22T04:19:19.778792+0000","last_clean":"2024-03-22T04:19:19.778792+0000","last_became_active":"2024-03-22T04:19:18.333455+0000","last_became_peered":"2024-03-22T04:19:18.333455+0000","last_unstale":"2024-03-22T04:19:19.778792+0000","last_undegraded":"2024-03-22T04:19:19.778792+0000","last_fullsized":"2024-03-22T04:19:19.778792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:47.587745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.353342+0000","last_change":"2024-03-22T04:19:17.317443+0000","last_active":"2024-03-22T04:19:18.353342+0000","last_peered":"2024-03-22T04:19:18.353342+0000","last_clean":"2024-03-22T04:19:18.353342+0000","last_became_active":"2024-03-22T04:19:17.313265+0000","last_became_peered":"2024-03-22T04:19:17.313265+0000","last_unstale":"2024-03-22T04:19:18.353342+0000","last_undegraded":"2024-03-22T04:19:18.353342+0000","last_fullsized":"2024-03-22T04:19:18.353342+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:23:47.191984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779036+0000","last_change":"2024-03-22T04:19:18.334614+0000","last_active":"2024-03-22T04:19:19.779036+0000","last_peered":"2024-03-22T04:19:19.779036+0000","last_clean":"2024-03-22T04:19:19.779036+0000","last_became_active":"2024-03-22T04:19:18.334364+0000","last_became_peered":"2024-03-22T04:19:18.334364+0000","last_unstale":"2024-03-22T04:19:19.779036+0000","last_undegraded":"2024-03-22T04:19:19.779036+0000","last_fullsized":"2024-03-22T04:19:19.779036+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:43.362064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323986+0000","last_change":"2024-03-22T04:19:17.322055+0000","last_active":"2024-03-22T04:19:19.323986+0000","last_peered":"2024-03-22T04:19:19.323986+0000","last_clean":"2024-03-22T04:19:19.323986+0000","last_became_active":"2024-03-22T04:19:17.321727+0000","last_became_peered":"2024-03-22T04:19:17.321727+0000","last_unstale":"2024-03-22T04:19:19.323986+0000","last_undegraded":"2024-03-22T04:19:19.323986+0000","last_fullsized":"2024-03-22T04:19:19.323986+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:35:17.288838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779090+0000","last_change":"2024-03-22T04:19:18.345224+0000","last_active":"2024-03-22T04:19:19.779090+0000","last_peered":"2024-03-22T04:19:19.779090+0000","last_clean":"2024-03-22T04:19:19.779090+0000","last_became_active":"2024-03-22T04:19:18.344383+0000","last_became_peered":"2024-03-22T04:19:18.344383+0000","last_unstale":"2024-03-22T04:19:19.779090+0000","last_undegraded":"2024-03-22T04:19:19.779090+0000","last_fullsized":"2024-03-22T04:19:19.779090+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:38:25.882112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.313883+0000","last_change":"2024-03-22T04:19:17.322172+0000","last_active":"2024-03-22T04:19:18.313883+0000","last_peered":"2024-03-22T04:19:18.313883+0000","last_clean":"2024-03-22T04:19:18.313883+0000","last_became_active":"2024-03-22T04:19:17.321881+0000","last_became_peered":"2024-03-22T04:19:17.321881+0000","last_unstale":"2024-03-22T04:19:18.313883+0000","last_undegraded":"2024-03-22T04:19:18.313883+0000","last_fullsized":"2024-03-22T04:19:18.313883+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:30:49.749150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324395+0000","last_change":"2024-03-22T04:19:18.318922+0000","last_active":"2024-03-22T04:19:19.324395+0000","last_peered":"2024-03-22T04:19:19.324395+0000","last_clean":"2024-03-22T04:19:19.324395+0000","last_became_active":"2024-03-22T04:19:18.318590+0000","last_became_peered":"2024-03-22T04:19:18.318590+0000","last_unstale":"2024-03-22T04:19:19.324395+0000","last_undegraded":"2024-03-22T04:19:19.324395+0000","last_fullsized":"2024-03-22T04:19:19.324395+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:20:51.390996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778758+0000","last_change":"2024-03-22T04:19:17.318722+0000","last_active":"2024-03-22T04:19:19.778758+0000","last_peered":"2024-03-22T04:19:19.778758+0000","last_clean":"2024-03-22T04:19:19.778758+0000","last_became_active":"2024-03-22T04:19:17.318429+0000","last_became_peered":"2024-03-22T04:19:17.318429+0000","last_unstale":"2024-03-22T04:19:19.778758+0000","last_undegraded":"2024-03-22T04:19:19.778758+0000","last_fullsized":"2024-03-22T04:19:19.778758+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:32:16.927300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323339+0000","last_change":"2024-03-22T04:19:18.337207+0000","last_active":"2024-03-22T04:19:19.323339+0000","last_peered":"2024-03-22T04:19:19.323339+0000","last_clean":"2024-03-22T04:19:19.323339+0000","last_became_active":"2024-03-22T04:19:18.336144+0000","last_became_peered":"2024-03-22T04:19:18.336144+0000","last_unstale":"2024-03-22T04:19:19.323339+0000","last_undegraded":"2024-03-22T04:19:19.323339+0000","last_fullsized":"2024-03-22T04:19:19.323339+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:49:40.884723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323340+0000","last_change":"2024-03-22T04:19:17.321128+0000","last_active":"2024-03-22T04:19:19.323340+0000","last_peered":"2024-03-22T04:19:19.323340+0000","last_clean":"2024-03-22T04:19:19.323340+0000","last_became_active":"2024-03-22T04:19:17.320791+0000","last_became_peered":"2024-03-22T04:19:17.320791+0000","last_unstale":"2024-03-22T04:19:19.323340+0000","last_undegraded":"2024-03-22T04:19:19.323340+0000","last_fullsized":"2024-03-22T04:19:19.323340+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:42.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742726+0000","last_change":"2024-03-22T04:19:18.331367+0000","last_active":"2024-03-22T04:19:19.742726+0000","last_peered":"2024-03-22T04:19:19.742726+0000","last_clean":"2024-03-22T04:19:19.742726+0000","last_became_active":"2024-03-22T04:19:18.331064+0000","last_became_peered":"2024-03-22T04:19:18.331064+0000","last_unstale":"2024-03-22T04:19:19.742726+0000","last_undegraded":"2024-03-22T04:19:19.742726+0000","last_fullsized":"2024-03-22T04:19:19.742726+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:12:34.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742919+0000","last_change":"2024-03-22T04:19:17.322455+0000","last_active":"2024-03-22T04:19:19.742919+0000","last_peered":"2024-03-22T04:19:19.742919+0000","last_clean":"2024-03-22T04:19:19.742919+0000","last_became_active":"2024-03-22T04:19:17.321998+0000","last_became_peered":"2024-03-22T04:19:17.321998+0000","last_unstale":"2024-03-22T04:19:19.742919+0000","last_undegraded":"2024-03-22T04:19:19.742919+0000","last_fullsized":"2024-03-22T04:19:19.742919+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:46.581281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778548+0000","last_change":"2024-03-22T04:19:18.343245+0000","last_active":"2024-03-22T04:19:19.778548+0000","last_peered":"2024-03-22T04:19:19.778548+0000","last_clean":"2024-03-22T04:19:19.778548+0000","last_became_active":"2024-03-22T04:19:18.342952+0000","last_became_peered":"2024-03-22T04:19:18.342952+0000","last_unstale":"2024-03-22T04:19:19.778548+0000","last_undegraded":"2024-03-22T04:19:19.778548+0000","last_fullsized":"2024-03-22T04:19:19.778548+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:43:04.884756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323030+0000","last_change":"2024-03-22T04:19:17.317197+0000","last_active":"2024-03-22T04:19:19.323030+0000","last_peered":"2024-03-22T04:19:19.323030+0000","last_clean":"2024-03-22T04:19:19.323030+0000","last_became_active":"2024-03-22T04:19:17.316886+0000","last_became_peered":"2024-03-22T04:19:17.316886+0000","last_unstale":"2024-03-22T04:19:19.323030+0000","last_undegraded":"2024-03-22T04:19:19.323030+0000","last_fullsized":"2024-03-22T04:19:19.323030+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:43:47.527273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.338567+0000","last_change":"2024-03-22T04:19:18.334625+0000","last_active":"2024-03-22T04:19:18.338567+0000","last_peered":"2024-03-22T04:19:18.338567+0000","last_clean":"2024-03-22T04:19:18.338567+0000","last_became_active":"2024-03-22T04:19:18.333628+0000","last_became_peered":"2024-03-22T04:19:18.333628+0000","last_unstale":"2024-03-22T04:19:18.338567+0000","last_undegraded":"2024-03-22T04:19:18.338567+0000","last_fullsized":"2024-03-22T04:19:18.338567+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:26:10.608228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324077+0000","last_change":"2024-03-22T04:19:17.312371+0000","last_active":"2024-03-22T04:19:19.324077+0000","last_peered":"2024-03-22T04:19:19.324077+0000","last_clean":"2024-03-22T04:19:19.324077+0000","last_became_active":"2024-03-22T04:19:17.312096+0000","last_became_peered":"2024-03-22T04:19:17.312096+0000","last_unstale":"2024-03-22T04:19:19.324077+0000","last_undegraded":"2024-03-22T04:19:19.324077+0000","last_fullsized":"2024-03-22T04:19:19.324077+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:42.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779013+0000","last_change":"2024-03-22T04:19:18.340853+0000","last_active":"2024-03-22T04:19:19.779013+0000","last_peered":"2024-03-22T04:19:19.779013+0000","last_clean":"2024-03-22T04:19:19.779013+0000","last_became_active":"2024-03-22T04:19:18.340228+0000","last_became_peered":"2024-03-22T04:19:18.340228+0000","last_unstale":"2024-03-22T04:19:19.779013+0000","last_undegraded":"2024-03-22T04:19:19.779013+0000","last_fullsized":"2024-03-22T04:19:19.779013+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:02.043886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323504+0000","last_change":"2024-03-22T04:19:17.320395+0000","last_active":"2024-03-22T04:19:19.323504+0000","last_peered":"2024-03-22T04:19:19.323504+0000","last_clean":"2024-03-22T04:19:19.323504+0000","last_became_active":"2024-03-22T04:19:17.320106+0000","last_became_peered":"2024-03-22T04:19:17.320106+0000","last_unstale":"2024-03-22T04:19:19.323504+0000","last_undegraded":"2024-03-22T04:19:19.323504+0000","last_fullsized":"2024-03-22T04:19:19.323504+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:43:47.527273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778909+0000","last_change":"2024-03-22T04:19:18.340689+0000","last_active":"2024-03-22T04:19:19.778909+0000","last_peered":"2024-03-22T04:19:19.778909+0000","last_clean":"2024-03-22T04:19:19.778909+0000","last_became_active":"2024-03-22T04:19:18.340033+0000","last_became_peered":"2024-03-22T04:19:18.340033+0000","last_unstale":"2024-03-22T04:19:19.778909+0000","last_undegraded":"2024-03-22T04:19:19.778909+0000","last_fullsized":"2024-03-22T04:19:19.778909+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:49:31.761279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322759+0000","last_change":"2024-03-22T04:19:17.325983+0000","last_active":"2024-03-22T04:19:19.322759+0000","last_peered":"2024-03-22T04:19:19.322759+0000","last_clean":"2024-03-22T04:19:19.322759+0000","last_became_active":"2024-03-22T04:19:17.325466+0000","last_became_peered":"2024-03-22T04:19:17.325466+0000","last_unstale":"2024-03-22T04:19:19.322759+0000","last_undegraded":"2024-03-22T04:19:19.322759+0000","last_fullsized":"2024-03-22T04:19:19.322759+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:46.581281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.339800+0000","last_change":"2024-03-22T04:19:18.337141+0000","last_active":"2024-03-22T04:19:18.339800+0000","last_peered":"2024-03-22T04:19:18.339800+0000","last_clean":"2024-03-22T04:19:18.339800+0000","last_became_active":"2024-03-22T04:19:18.336867+0000","last_became_peered":"2024-03-22T04:19:18.336867+0000","last_unstale":"2024-03-22T04:19:18.339800+0000","last_undegraded":"2024-03-22T04:19:18.339800+0000","last_fullsized":"2024-03-22T04:19:18.339800+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:13:07.271663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323482+0000","last_change":"2024-03-22T04:19:17.326510+0000","last_active":"2024-03-22T04:19:19.323482+0000","last_peered":"2024-03-22T04:19:19.323482+0000","last_clean":"2024-03-22T04:19:19.323482+0000","last_became_active":"2024-03-22T04:19:17.325991+0000","last_became_peered":"2024-03-22T04:19:17.325991+0000","last_unstale":"2024-03-22T04:19:19.323482+0000","last_undegraded":"2024-03-22T04:19:19.323482+0000","last_fullsized":"2024-03-22T04:19:19.323482+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:47:50.510343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778688+0000","last_change":"2024-03-22T04:19:18.339601+0000","last_active":"2024-03-22T04:19:19.778688+0000","last_peered":"2024-03-22T04:19:19.778688+0000","last_clean":"2024-03-22T04:19:19.778688+0000","last_became_active":"2024-03-22T04:19:18.339312+0000","last_became_peered":"2024-03-22T04:19:18.339312+0000","last_unstale":"2024-03-22T04:19:19.778688+0000","last_undegraded":"2024-03-22T04:19:19.778688+0000","last_fullsized":"2024-03-22T04:19:19.778688+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:35.933979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324474+0000","last_change":"2024-03-22T04:19:17.323198+0000","last_active":"2024-03-22T04:19:19.324474+0000","last_peered":"2024-03-22T04:19:19.324474+0000","last_clean":"2024-03-22T04:19:19.324474+0000","last_became_active":"2024-03-22T04:19:17.322902+0000","last_became_peered":"2024-03-22T04:19:17.322902+0000","last_unstale":"2024-03-22T04:19:19.324474+0000","last_undegraded":"2024-03-22T04:19:19.324474+0000","last_fullsized":"2024-03-22T04:19:19.324474+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:35:17.288838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778772+0000","last_change":"2024-03-22T04:19:18.334033+0000","last_active":"2024-03-22T04:19:19.778772+0000","last_peered":"2024-03-22T04:19:19.778772+0000","last_clean":"2024-03-22T04:19:19.778772+0000","last_became_active":"2024-03-22T04:19:18.333788+0000","last_became_peered":"2024-03-22T04:19:18.333788+0000","last_unstale":"2024-03-22T04:19:19.778772+0000","last_undegraded":"2024-03-22T04:19:19.778772+0000","last_fullsized":"2024-03-22T04:19:19.778772+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:18:30.823672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"109'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.339694+0000","last_change":"2024-03-22T04:19:17.315344+0000","last_active":"2024-03-22T04:19:19.339694+0000","last_peered":"2024-03-22T04:19:19.339694+0000","last_clean":"2024-03-22T04:19:19.339694+0000","last_became_active":"2024-03-22T04:19:17.314614+0000","last_became_peered":"2024-03-22T04:19:17.314614+0000","last_unstale":"2024-03-22T04:19:19.339694+0000","last_undegraded":"2024-03-22T04:19:19.339694+0000","last_fullsized":"2024-03-22T04:19:19.339694+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:01:09.512640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323558+0000","last_change":"2024-03-22T04:19:18.339080+0000","last_active":"2024-03-22T04:19:19.323558+0000","last_peered":"2024-03-22T04:19:19.323558+0000","last_clean":"2024-03-22T04:19:19.323558+0000","last_became_active":"2024-03-22T04:19:18.338695+0000","last_became_peered":"2024-03-22T04:19:18.338695+0000","last_unstale":"2024-03-22T04:19:19.323558+0000","last_undegraded":"2024-03-22T04:19:19.323558+0000","last_fullsized":"2024-03-22T04:19:19.323558+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:49.253227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779212+0000","last_change":"2024-03-22T04:19:17.318521+0000","last_active":"2024-03-22T04:19:19.779212+0000","last_peered":"2024-03-22T04:19:19.779212+0000","last_clean":"2024-03-22T04:19:19.779212+0000","last_became_active":"2024-03-22T04:19:17.318203+0000","last_became_peered":"2024-03-22T04:19:17.318203+0000","last_unstale":"2024-03-22T04:19:19.779212+0000","last_undegraded":"2024-03-22T04:19:19.779212+0000","last_fullsized":"2024-03-22T04:19:19.779212+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:49:47.221788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742784+0000","last_change":"2024-03-22T04:19:18.333376+0000","last_active":"2024-03-22T04:19:19.742784+0000","last_peered":"2024-03-22T04:19:19.742784+0000","last_clean":"2024-03-22T04:19:19.742784+0000","last_became_active":"2024-03-22T04:19:18.333117+0000","last_became_peered":"2024-03-22T04:19:18.333117+0000","last_unstale":"2024-03-22T04:19:19.742784+0000","last_undegraded":"2024-03-22T04:19:19.742784+0000","last_fullsized":"2024-03-22T04:19:19.742784+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:54:28.206493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.316566+0000","last_change":"2024-03-22T04:19:17.318663+0000","last_active":"2024-03-22T04:19:18.316566+0000","last_peered":"2024-03-22T04:19:18.316566+0000","last_clean":"2024-03-22T04:19:18.316566+0000","last_became_active":"2024-03-22T04:19:17.318400+0000","last_became_peered":"2024-03-22T04:19:17.318400+0000","last_unstale":"2024-03-22T04:19:18.316566+0000","last_undegraded":"2024-03-22T04:19:18.316566+0000","last_fullsized":"2024-03-22T04:19:18.316566+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:50.944233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.339340+0000","last_change":"2024-03-22T04:19:18.335499+0000","last_active":"2024-03-22T04:19:18.339340+0000","last_peered":"2024-03-22T04:19:18.339340+0000","last_clean":"2024-03-22T04:19:18.339340+0000","last_became_active":"2024-03-22T04:19:18.333965+0000","last_became_peered":"2024-03-22T04:19:18.333965+0000","last_unstale":"2024-03-22T04:19:18.339340+0000","last_undegraded":"2024-03-22T04:19:18.339340+0000","last_fullsized":"2024-03-22T04:19:18.339340+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:20:47.281468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778695+0000","last_change":"2024-03-22T04:19:17.322774+0000","last_active":"2024-03-22T04:19:19.778695+0000","last_peered":"2024-03-22T04:19:19.778695+0000","last_clean":"2024-03-22T04:19:19.778695+0000","last_became_active":"2024-03-22T04:19:17.322447+0000","last_became_peered":"2024-03-22T04:19:17.322447+0000","last_unstale":"2024-03-22T04:19:19.778695+0000","last_undegraded":"2024-03-22T04:19:19.778695+0000","last_fullsized":"2024-03-22T04:19:19.778695+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.660361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.338992+0000","last_change":"2024-03-22T04:19:18.335356+0000","last_active":"2024-03-22T04:19:18.338992+0000","last_peered":"2024-03-22T04:19:18.338992+0000","last_clean":"2024-03-22T04:19:18.338992+0000","last_became_active":"2024-03-22T04:19:18.334300+0000","last_became_peered":"2024-03-22T04:19:18.334300+0000","last_unstale":"2024-03-22T04:19:18.338992+0000","last_undegraded":"2024-03-22T04:19:18.338992+0000","last_fullsized":"2024-03-22T04:19:18.338992+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:49:36.153576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323773+0000","last_change":"2024-03-22T04:19:17.313969+0000","last_active":"2024-03-22T04:19:19.323773+0000","last_peered":"2024-03-22T04:19:19.323773+0000","last_clean":"2024-03-22T04:19:19.323773+0000","last_became_active":"2024-03-22T04:19:17.313399+0000","last_became_peered":"2024-03-22T04:19:17.313399+0000","last_unstale":"2024-03-22T04:19:19.323773+0000","last_undegraded":"2024-03-22T04:19:19.323773+0000","last_fullsized":"2024-03-22T04:19:19.323773+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:46.581281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323330+0000","last_change":"2024-03-22T04:19:18.328695+0000","last_active":"2024-03-22T04:19:19.323330+0000","last_peered":"2024-03-22T04:19:19.323330+0000","last_clean":"2024-03-22T04:19:19.323330+0000","last_became_active":"2024-03-22T04:19:18.328379+0000","last_became_peered":"2024-03-22T04:19:18.328379+0000","last_unstale":"2024-03-22T04:19:19.323330+0000","last_undegraded":"2024-03-22T04:19:19.323330+0000","last_fullsized":"2024-03-22T04:19:19.323330+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:15:35.873149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742590+0000","last_change":"2024-03-22T04:19:17.322375+0000","last_active":"2024-03-22T04:19:19.742590+0000","last_peered":"2024-03-22T04:19:19.742590+0000","last_clean":"2024-03-22T04:19:19.742590+0000","last_became_active":"2024-03-22T04:19:17.322125+0000","last_became_peered":"2024-03-22T04:19:17.322125+0000","last_unstale":"2024-03-22T04:19:19.742590+0000","last_undegraded":"2024-03-22T04:19:19.742590+0000","last_fullsized":"2024-03-22T04:19:19.742590+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:53:22.857897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779286+0000","last_change":"2024-03-22T04:19:18.339401+0000","last_active":"2024-03-22T04:19:19.779286+0000","last_peered":"2024-03-22T04:19:19.779286+0000","last_clean":"2024-03-22T04:19:19.779286+0000","last_became_active":"2024-03-22T04:19:18.339105+0000","last_became_peered":"2024-03-22T04:19:18.339105+0000","last_unstale":"2024-03-22T04:19:19.779286+0000","last_undegraded":"2024-03-22T04:19:19.779286+0000","last_fullsized":"2024-03-22T04:19:19.779286+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:56.078667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323271+0000","last_change":"2024-03-22T04:19:17.320664+0000","last_active":"2024-03-22T04:19:19.323271+0000","last_peered":"2024-03-22T04:19:19.323271+0000","last_clean":"2024-03-22T04:19:19.323271+0000","last_became_active":"2024-03-22T04:19:17.320369+0000","last_became_peered":"2024-03-22T04:19:17.320369+0000","last_unstale":"2024-03-22T04:19:19.323271+0000","last_undegraded":"2024-03-22T04:19:19.323271+0000","last_fullsized":"2024-03-22T04:19:19.323271+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:55.072203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322802+0000","last_change":"2024-03-22T04:19:18.330608+0000","last_active":"2024-03-22T04:19:19.322802+0000","last_peered":"2024-03-22T04:19:19.322802+0000","last_clean":"2024-03-22T04:19:19.322802+0000","last_became_active":"2024-03-22T04:19:18.330220+0000","last_became_peered":"2024-03-22T04:19:18.330220+0000","last_unstale":"2024-03-22T04:19:19.322802+0000","last_undegraded":"2024-03-22T04:19:19.322802+0000","last_fullsized":"2024-03-22T04:19:19.322802+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:20:51.390996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.351443+0000","last_change":"2024-03-22T04:19:17.323564+0000","last_active":"2024-03-22T04:19:18.351443+0000","last_peered":"2024-03-22T04:19:18.351443+0000","last_clean":"2024-03-22T04:19:18.351443+0000","last_became_active":"2024-03-22T04:19:17.323148+0000","last_became_peered":"2024-03-22T04:19:17.323148+0000","last_unstale":"2024-03-22T04:19:18.351443+0000","last_undegraded":"2024-03-22T04:19:18.351443+0000","last_fullsized":"2024-03-22T04:19:18.351443+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:38.173133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":332,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742255+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:19:19.742255+0000","last_peered":"2024-03-22T04:19:19.742255+0000","last_clean":"2024-03-22T04:19:19.742255+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:19:19.742255+0000","last_undegraded":"2024-03-22T04:19:19.742255+0000","last_fullsized":"2024-03-22T04:19:19.742255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509551,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":625146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":626106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1945600,"data_stored":1214986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509551,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":626106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":624606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509551,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1945600,"data_stored":1214954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":625076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":617836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:19:22.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:19:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff39d400640 1 Processor -- start 2024-03-22T04:19:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff39d400640 1 -- start start 2024-03-22T04:19:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff39d400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39806ef40 0x7ff398142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff39d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3980ad710 0x7ff39815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff39d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39815c7c0 con 0x7ff3980ad330 2024-03-22T04:19:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff39d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff39806dea0 con 0x7ff3980ad710 2024-03-22T04:19:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff39d400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff39806e180 con 0x7ff39806ef40 2024-03-22T04:19:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff397200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3980ad710 0x7ff39815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff396800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff3980ad330 0x7ff39813fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44374/0 (socket says 172.21.15.86:44374) 2024-03-22T04:19:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff396800640 1 -- 172.21.15.86:0/742836156 learned_addr learned my addr 172.21.15.86:0/742836156 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.560+0000 7ff38de00640 1 --2- 172.21.15.86:0/742836156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39806ef40 0x7ff398142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff397200640 1 -- 172.21.15.86:0/742836156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39806ef40 msgr2=0x7ff398142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff397200640 1 --2- 172.21.15.86:0/742836156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39806ef40 0x7ff398142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff397200640 1 -- 172.21.15.86:0/742836156 >> v1:172.21.15.86:6789/0 conn(0x7ff3980ad330 legacy=0x7ff39813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff397200640 1 -- 172.21.15.86:0/742836156 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39806e460 con 0x7ff3980ad710 2024-03-22T04:19:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff395e00640 1 -- 172.21.15.86:0/742836156 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960159337 0 0) 0x7ff39815c7c0 con 0x7ff3980ad330 2024-03-22T04:19:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff397200640 1 --2- 172.21.15.86:0/742836156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3980ad710 0x7ff39815d4a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff390009d90 tx=0x7ff39001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf7b6f9823de62b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff395e00640 1 -- 172.21.15.86:0/742836156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3900b2070 con 0x7ff3980ad710 2024-03-22T04:19:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff395e00640 1 -- 172.21.15.86:0/742836156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3900a5a00 con 0x7ff3980ad710 2024-03-22T04:19:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff395e00640 1 -- 172.21.15.86:0/742836156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3900a5dd0 con 0x7ff3980ad710 2024-03-22T04:19:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff39d400640 1 -- 172.21.15.86:0/742836156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3980ad710 msgr2=0x7ff39815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff39d400640 1 --2- 172.21.15.86:0/742836156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3980ad710 0x7ff39815d4a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff390009d90 tx=0x7ff39001c940 comp rx=0 tx=0).stop 2024-03-22T04:19:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff39d400640 1 -- 172.21.15.86:0/742836156 shutdown_connections 2024-03-22T04:19:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff39d400640 1 --2- 172.21.15.86:0/742836156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3980ad710 0x7ff39815d4a0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.564+0000 7ff39d400640 1 --2- 172.21.15.86:0/742836156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39806ef40 0x7ff398142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 -- 172.21.15.86:0/742836156 >> 172.21.15.86:0/742836156 conn(0x7ff3980b1df0 msgr2=0x7ff3980b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 -- 172.21.15.86:0/742836156 shutdown_connections 2024-03-22T04:19:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 -- 172.21.15.86:0/742836156 wait complete. 2024-03-22T04:19:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 Processor -- start 2024-03-22T04:19:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 -- start start 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff39806ef40 0x7ff39815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980ad330 0x7ff398154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3980ad710 0x7ff3981592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981690e0 con 0x7ff39806ef40 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff398168f60 con 0x7ff3980ad330 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff39d400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff398169260 con 0x7ff3980ad710 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff39806ef40 0x7ff39815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff39806ef40 0x7ff39815dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40034/0 (socket says 172.21.15.86:40034) 2024-03-22T04:19:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 -- 172.21.15.86:0/364279324 learned_addr learned my addr 172.21.15.86:0/364279324 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff397200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3980ad710 0x7ff3981592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff38de00640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980ad330 0x7ff398154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 -- 172.21.15.86:0/364279324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3980ad710 msgr2=0x7ff3981592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3980ad710 0x7ff3981592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 -- 172.21.15.86:0/364279324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980ad330 msgr2=0x7ff398154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980ad330 0x7ff398154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 -- 172.21.15.86:0/364279324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff398145fb0 con 0x7ff39806ef40 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff396800640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff39806ef40 0x7ff39815dc80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c480 tx=0x7ff38800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.568+0000 7ff38f400640 1 -- 172.21.15.86:0/364279324 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388018070 con 0x7ff39806ef40 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff38f400640 1 -- 172.21.15.86:0/364279324 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388007500 con 0x7ff39806ef40 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff398158ae0 con 0x7ff39806ef40 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff38f400640 1 -- 172.21.15.86:0/364279324 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388004a30 con 0x7ff39806ef40 2024-03-22T04:19:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff398155350 con 0x7ff39806ef40 2024-03-22T04:19:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff398068650 con 0x7ff39806ef40 2024-03-22T04:19:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff38f400640 1 -- 172.21.15.86:0/364279324 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff38801b070 con 0x7ff39806ef40 2024-03-22T04:19:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff38f400640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff374067b50 0x7ff37406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff38f400640 1 -- 172.21.15.86:0/364279324 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7ff3880852a0 con 0x7ff39806ef40 2024-03-22T04:19:22.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff38de00640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff374067b50 0x7ff37406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.572+0000 7ff38de00640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff374067b50 0x7ff37406a010 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff3980b53e0 tx=0x7ff380009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.576+0000 7ff38f400640 1 -- 172.21.15.86:0/364279324 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff388055560 con 0x7ff39806ef40 2024-03-22T04:19:22.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.728+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff39815f790 con 0x7ff374067b50 2024-03-22T04:19:22.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff38f400640 1 -- 172.21.15.86:0/364279324 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252476 (secure 0 0 0) 0x7ff39815f790 con 0x7ff374067b50 2024-03-22T04:19:22.737 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:22.739 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff374067b50 msgr2=0x7ff37406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff374067b50 0x7ff37406a010 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff3980b53e0 tx=0x7ff380009190 comp rx=0 tx=0).stop 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff39806ef40 msgr2=0x7ff39815dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff39806ef40 0x7ff39815dc80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c480 tx=0x7ff38800c940 comp rx=0 tx=0).stop 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 shutdown_connections 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff374067b50 0x7ff37406a010 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980ad330 0x7ff398154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3980ad710 0x7ff3981592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 --2- 172.21.15.86:0/364279324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff39806ef40 0x7ff39815dc80 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.736+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 >> 172.21.15.86:0/364279324 conn(0x7ff3980b1df0 msgr2=0x7ff39815e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.740+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 shutdown_connections 2024-03-22T04:19:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.740+0000 7ff39d400640 1 -- 172.21.15.86:0/364279324 wait complete. 2024-03-22T04:19:22.757 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-03-22T04:19:21.909923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261196,"kb_used_data":11708,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713524,"statfs":{"total":773094113280,"available":772826648576,"internally_reserved":0,"allocated":11988992,"data_stored":6174816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.616093"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.332968+0000","last_change":"2024-03-22T04:19:18.328825+0000","last_active":"2024-03-22T04:19:18.332968+0000","last_peered":"2024-03-22T04:19:18.332968+0000","last_clean":"2024-03-22T04:19:18.332968+0000","last_became_active":"2024-03-22T04:19:18.328524+0000","last_became_peered":"2024-03-22T04:19:18.328524+0000","last_unstale":"2024-03-22T04:19:18.332968+0000","last_undegraded":"2024-03-22T04:19:18.332968+0000","last_fullsized":"2024-03-22T04:19:18.332968+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:07:04.949643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778610+0000","last_change":"2024-03-22T04:19:18.333041+0000","last_active":"2024-03-22T04:19:19.778610+0000","last_peered":"2024-03-22T04:19:19.778610+0000","last_clean":"2024-03-22T04:19:19.778610+0000","last_became_active":"2024-03-22T04:19:18.332405+0000","last_became_peered":"2024-03-22T04:19:18.332405+0000","last_unstale":"2024-03-22T04:19:19.778610+0000","last_undegraded":"2024-03-22T04:19:19.778610+0000","last_fullsized":"2024-03-22T04:19:19.778610+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:53:23.864360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323957+0000","last_change":"2024-03-22T04:19:18.324516+0000","last_active":"2024-03-22T04:19:19.323957+0000","last_peered":"2024-03-22T04:19:19.323957+0000","last_clean":"2024-03-22T04:19:19.323957+0000","last_became_active":"2024-03-22T04:19:18.324177+0000","last_became_peered":"2024-03-22T04:19:18.324177+0000","last_unstale":"2024-03-22T04:19:19.323957+0000","last_undegraded":"2024-03-22T04:19:19.323957+0000","last_fullsized":"2024-03-22T04:19:19.323957+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:58:15.390106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.336238+0000","last_change":"2024-03-22T04:19:18.330635+0000","last_active":"2024-03-22T04:19:18.336238+0000","last_peered":"2024-03-22T04:19:18.336238+0000","last_clean":"2024-03-22T04:19:18.336238+0000","last_became_active":"2024-03-22T04:19:18.330004+0000","last_became_peered":"2024-03-22T04:19:18.330004+0000","last_unstale":"2024-03-22T04:19:18.336238+0000","last_undegraded":"2024-03-22T04:19:18.336238+0000","last_fullsized":"2024-03-22T04:19:18.336238+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:03:00.854292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.336247+0000","last_change":"2024-03-22T04:19:18.330163+0000","last_active":"2024-03-22T04:19:18.336247+0000","last_peered":"2024-03-22T04:19:18.336247+0000","last_clean":"2024-03-22T04:19:18.336247+0000","last_became_active":"2024-03-22T04:19:18.329885+0000","last_became_peered":"2024-03-22T04:19:18.329885+0000","last_unstale":"2024-03-22T04:19:18.336247+0000","last_undegraded":"2024-03-22T04:19:18.336247+0000","last_fullsized":"2024-03-22T04:19:18.336247+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:23:11.579349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322782+0000","last_change":"2024-03-22T04:19:18.321339+0000","last_active":"2024-03-22T04:19:19.322782+0000","last_peered":"2024-03-22T04:19:19.322782+0000","last_clean":"2024-03-22T04:19:19.322782+0000","last_became_active":"2024-03-22T04:19:18.321082+0000","last_became_peered":"2024-03-22T04:19:18.321082+0000","last_unstale":"2024-03-22T04:19:19.322782+0000","last_undegraded":"2024-03-22T04:19:19.322782+0000","last_fullsized":"2024-03-22T04:19:19.322782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:51.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322622+0000","last_change":"2024-03-22T04:19:18.320954+0000","last_active":"2024-03-22T04:19:19.322622+0000","last_peered":"2024-03-22T04:19:19.322622+0000","last_clean":"2024-03-22T04:19:19.322622+0000","last_became_active":"2024-03-22T04:19:18.320646+0000","last_became_peered":"2024-03-22T04:19:18.320646+0000","last_unstale":"2024-03-22T04:19:19.322622+0000","last_undegraded":"2024-03-22T04:19:19.322622+0000","last_fullsized":"2024-03-22T04:19:19.322622+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:58:15.390106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323209+0000","last_change":"2024-03-22T04:19:18.339388+0000","last_active":"2024-03-22T04:19:19.323209+0000","last_peered":"2024-03-22T04:19:19.323209+0000","last_clean":"2024-03-22T04:19:19.323209+0000","last_became_active":"2024-03-22T04:19:18.338943+0000","last_became_peered":"2024-03-22T04:19:18.338943+0000","last_unstale":"2024-03-22T04:19:19.323209+0000","last_undegraded":"2024-03-22T04:19:19.323209+0000","last_fullsized":"2024-03-22T04:19:19.323209+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:06:17.232427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779190+0000","last_change":"2024-03-22T04:19:18.337616+0000","last_active":"2024-03-22T04:19:19.779190+0000","last_peered":"2024-03-22T04:19:19.779190+0000","last_clean":"2024-03-22T04:19:19.779190+0000","last_became_active":"2024-03-22T04:19:18.336011+0000","last_became_peered":"2024-03-22T04:19:18.336011+0000","last_unstale":"2024-03-22T04:19:19.779190+0000","last_undegraded":"2024-03-22T04:19:19.779190+0000","last_fullsized":"2024-03-22T04:19:19.779190+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:01:10.519103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324496+0000","last_change":"2024-03-22T04:19:18.339319+0000","last_active":"2024-03-22T04:19:19.324496+0000","last_peered":"2024-03-22T04:19:19.324496+0000","last_clean":"2024-03-22T04:19:19.324496+0000","last_became_active":"2024-03-22T04:19:18.339026+0000","last_became_peered":"2024-03-22T04:19:18.339026+0000","last_unstale":"2024-03-22T04:19:19.324496+0000","last_undegraded":"2024-03-22T04:19:19.324496+0000","last_fullsized":"2024-03-22T04:19:19.324496+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:15:06.596648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779054+0000","last_change":"2024-03-22T04:19:18.332716+0000","last_active":"2024-03-22T04:19:19.779054+0000","last_peered":"2024-03-22T04:19:19.779054+0000","last_clean":"2024-03-22T04:19:19.779054+0000","last_became_active":"2024-03-22T04:19:18.332433+0000","last_became_peered":"2024-03-22T04:19:18.332433+0000","last_unstale":"2024-03-22T04:19:19.779054+0000","last_undegraded":"2024-03-22T04:19:19.779054+0000","last_fullsized":"2024-03-22T04:19:19.779054+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:32:30.739275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322941+0000","last_change":"2024-03-22T04:19:18.336750+0000","last_active":"2024-03-22T04:19:19.322941+0000","last_peered":"2024-03-22T04:19:19.322941+0000","last_clean":"2024-03-22T04:19:19.322941+0000","last_became_active":"2024-03-22T04:19:18.336489+0000","last_became_peered":"2024-03-22T04:19:18.336489+0000","last_unstale":"2024-03-22T04:19:19.322941+0000","last_undegraded":"2024-03-22T04:19:19.322941+0000","last_fullsized":"2024-03-22T04:19:19.322941+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:06:17.232427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.331049+0000","last_change":"2024-03-22T04:19:18.325476+0000","last_active":"2024-03-22T04:19:18.331049+0000","last_peered":"2024-03-22T04:19:18.331049+0000","last_clean":"2024-03-22T04:19:18.331049+0000","last_became_active":"2024-03-22T04:19:18.325137+0000","last_became_peered":"2024-03-22T04:19:18.325137+0000","last_unstale":"2024-03-22T04:19:18.331049+0000","last_undegraded":"2024-03-22T04:19:18.331049+0000","last_fullsized":"2024-03-22T04:19:18.331049+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:15.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323225+0000","last_change":"2024-03-22T04:19:18.336729+0000","last_active":"2024-03-22T04:19:19.323225+0000","last_peered":"2024-03-22T04:19:19.323225+0000","last_clean":"2024-03-22T04:19:19.323225+0000","last_became_active":"2024-03-22T04:19:18.336469+0000","last_became_peered":"2024-03-22T04:19:18.336469+0000","last_unstale":"2024-03-22T04:19:19.323225+0000","last_undegraded":"2024-03-22T04:19:19.323225+0000","last_fullsized":"2024-03-22T04:19:19.323225+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:49:40.884723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778479+0000","last_change":"2024-03-22T04:19:18.329014+0000","last_active":"2024-03-22T04:19:19.778479+0000","last_peered":"2024-03-22T04:19:19.778479+0000","last_clean":"2024-03-22T04:19:19.778479+0000","last_became_active":"2024-03-22T04:19:18.328731+0000","last_became_peered":"2024-03-22T04:19:18.328731+0000","last_unstale":"2024-03-22T04:19:19.778479+0000","last_undegraded":"2024-03-22T04:19:19.778479+0000","last_fullsized":"2024-03-22T04:19:19.778479+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:27:36.778603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742592+0000","last_change":"2024-03-22T04:19:18.331919+0000","last_active":"2024-03-22T04:19:19.742592+0000","last_peered":"2024-03-22T04:19:19.742592+0000","last_clean":"2024-03-22T04:19:19.742592+0000","last_became_active":"2024-03-22T04:19:18.330483+0000","last_became_peered":"2024-03-22T04:19:18.330483+0000","last_unstale":"2024-03-22T04:19:19.742592+0000","last_undegraded":"2024-03-22T04:19:19.742592+0000","last_fullsized":"2024-03-22T04:19:19.742592+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:07:45.953360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324254+0000","last_change":"2024-03-22T04:19:18.319147+0000","last_active":"2024-03-22T04:19:19.324254+0000","last_peered":"2024-03-22T04:19:19.324254+0000","last_clean":"2024-03-22T04:19:19.324254+0000","last_became_active":"2024-03-22T04:19:18.318830+0000","last_became_peered":"2024-03-22T04:19:18.318830+0000","last_unstale":"2024-03-22T04:19:19.324254+0000","last_undegraded":"2024-03-22T04:19:19.324254+0000","last_fullsized":"2024-03-22T04:19:19.324254+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:51.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779193+0000","last_change":"2024-03-22T04:19:18.333771+0000","last_active":"2024-03-22T04:19:19.779193+0000","last_peered":"2024-03-22T04:19:19.779193+0000","last_clean":"2024-03-22T04:19:19.779193+0000","last_became_active":"2024-03-22T04:19:18.332724+0000","last_became_peered":"2024-03-22T04:19:18.332724+0000","last_unstale":"2024-03-22T04:19:19.779193+0000","last_undegraded":"2024-03-22T04:19:19.779193+0000","last_fullsized":"2024-03-22T04:19:19.779193+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:09.202859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.330525+0000","last_change":"2024-03-22T04:19:18.323710+0000","last_active":"2024-03-22T04:19:18.330525+0000","last_peered":"2024-03-22T04:19:18.330525+0000","last_clean":"2024-03-22T04:19:18.330525+0000","last_became_active":"2024-03-22T04:19:18.323431+0000","last_became_peered":"2024-03-22T04:19:18.323431+0000","last_unstale":"2024-03-22T04:19:18.330525+0000","last_undegraded":"2024-03-22T04:19:18.330525+0000","last_fullsized":"2024-03-22T04:19:18.330525+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:15:02.584209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742923+0000","last_change":"2024-03-22T04:19:18.331951+0000","last_active":"2024-03-22T04:19:19.742923+0000","last_peered":"2024-03-22T04:19:19.742923+0000","last_clean":"2024-03-22T04:19:19.742923+0000","last_became_active":"2024-03-22T04:19:18.331658+0000","last_became_peered":"2024-03-22T04:19:18.331658+0000","last_unstale":"2024-03-22T04:19:19.742923+0000","last_undegraded":"2024-03-22T04:19:19.742923+0000","last_fullsized":"2024-03-22T04:19:19.742923+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:42:29.640947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323984+0000","last_change":"2024-03-22T04:19:18.331587+0000","last_active":"2024-03-22T04:19:19.323984+0000","last_peered":"2024-03-22T04:19:19.323984+0000","last_clean":"2024-03-22T04:19:19.323984+0000","last_became_active":"2024-03-22T04:19:18.329754+0000","last_became_peered":"2024-03-22T04:19:18.329754+0000","last_unstale":"2024-03-22T04:19:19.323984+0000","last_undegraded":"2024-03-22T04:19:19.323984+0000","last_fullsized":"2024-03-22T04:19:19.323984+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:42:29.640947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778553+0000","last_change":"2024-03-22T04:19:18.343320+0000","last_active":"2024-03-22T04:19:19.778553+0000","last_peered":"2024-03-22T04:19:19.778553+0000","last_clean":"2024-03-22T04:19:19.778553+0000","last_became_active":"2024-03-22T04:19:18.342992+0000","last_became_peered":"2024-03-22T04:19:18.342992+0000","last_unstale":"2024-03-22T04:19:19.778553+0000","last_undegraded":"2024-03-22T04:19:19.778553+0000","last_fullsized":"2024-03-22T04:19:19.778553+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:05:01.060782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323499+0000","last_change":"2024-03-22T04:19:18.333136+0000","last_active":"2024-03-22T04:19:19.323499+0000","last_peered":"2024-03-22T04:19:19.323499+0000","last_clean":"2024-03-22T04:19:19.323499+0000","last_became_active":"2024-03-22T04:19:18.332720+0000","last_became_peered":"2024-03-22T04:19:18.332720+0000","last_unstale":"2024-03-22T04:19:19.323499+0000","last_undegraded":"2024-03-22T04:19:19.323499+0000","last_fullsized":"2024-03-22T04:19:19.323499+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:15:35.873149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323243+0000","last_change":"2024-03-22T04:19:18.334522+0000","last_active":"2024-03-22T04:19:19.323243+0000","last_peered":"2024-03-22T04:19:19.323243+0000","last_clean":"2024-03-22T04:19:19.323243+0000","last_became_active":"2024-03-22T04:19:18.334225+0000","last_became_peered":"2024-03-22T04:19:18.334225+0000","last_unstale":"2024-03-22T04:19:19.323243+0000","last_undegraded":"2024-03-22T04:19:19.323243+0000","last_fullsized":"2024-03-22T04:19:19.323243+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:17:34.785225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742975+0000","last_change":"2024-03-22T04:19:18.333501+0000","last_active":"2024-03-22T04:19:19.742975+0000","last_peered":"2024-03-22T04:19:19.742975+0000","last_clean":"2024-03-22T04:19:19.742975+0000","last_became_active":"2024-03-22T04:19:18.333181+0000","last_became_peered":"2024-03-22T04:19:18.333181+0000","last_unstale":"2024-03-22T04:19:19.742975+0000","last_undegraded":"2024-03-22T04:19:19.742975+0000","last_fullsized":"2024-03-22T04:19:19.742975+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:51.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742928+0000","last_change":"2024-03-22T04:19:18.332928+0000","last_active":"2024-03-22T04:19:19.742928+0000","last_peered":"2024-03-22T04:19:19.742928+0000","last_clean":"2024-03-22T04:19:19.742928+0000","last_became_active":"2024-03-22T04:19:18.332641+0000","last_became_peered":"2024-03-22T04:19:18.332641+0000","last_unstale":"2024-03-22T04:19:19.742928+0000","last_undegraded":"2024-03-22T04:19:19.742928+0000","last_fullsized":"2024-03-22T04:19:19.742928+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:58:15.390106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742860+0000","last_change":"2024-03-22T04:19:18.330855+0000","last_active":"2024-03-22T04:19:19.742860+0000","last_peered":"2024-03-22T04:19:19.742860+0000","last_clean":"2024-03-22T04:19:19.742860+0000","last_became_active":"2024-03-22T04:19:18.329496+0000","last_became_peered":"2024-03-22T04:19:18.329496+0000","last_unstale":"2024-03-22T04:19:19.742860+0000","last_undegraded":"2024-03-22T04:19:19.742860+0000","last_fullsized":"2024-03-22T04:19:19.742860+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:40.104909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322998+0000","last_change":"2024-03-22T04:19:18.326025+0000","last_active":"2024-03-22T04:19:19.322998+0000","last_peered":"2024-03-22T04:19:19.322998+0000","last_clean":"2024-03-22T04:19:19.322998+0000","last_became_active":"2024-03-22T04:19:18.325745+0000","last_became_peered":"2024-03-22T04:19:18.325745+0000","last_unstale":"2024-03-22T04:19:19.322998+0000","last_undegraded":"2024-03-22T04:19:19.322998+0000","last_fullsized":"2024-03-22T04:19:19.322998+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:17:34.785225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324144+0000","last_change":"2024-03-22T04:19:18.342111+0000","last_active":"2024-03-22T04:19:19.324144+0000","last_peered":"2024-03-22T04:19:19.324144+0000","last_clean":"2024-03-22T04:19:19.324144+0000","last_became_active":"2024-03-22T04:19:18.341836+0000","last_became_peered":"2024-03-22T04:19:18.341836+0000","last_unstale":"2024-03-22T04:19:19.324144+0000","last_undegraded":"2024-03-22T04:19:19.324144+0000","last_fullsized":"2024-03-22T04:19:19.324144+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:24:42.122978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778573+0000","last_change":"2024-03-22T04:19:18.330796+0000","last_active":"2024-03-22T04:19:19.778573+0000","last_peered":"2024-03-22T04:19:19.778573+0000","last_clean":"2024-03-22T04:19:19.778573+0000","last_became_active":"2024-03-22T04:19:18.330538+0000","last_became_peered":"2024-03-22T04:19:18.330538+0000","last_unstale":"2024-03-22T04:19:19.778573+0000","last_undegraded":"2024-03-22T04:19:19.778573+0000","last_fullsized":"2024-03-22T04:19:19.778573+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:53:32.642446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322658+0000","last_change":"2024-03-22T04:19:18.335946+0000","last_active":"2024-03-22T04:19:19.322658+0000","last_peered":"2024-03-22T04:19:19.322658+0000","last_clean":"2024-03-22T04:19:19.322658+0000","last_became_active":"2024-03-22T04:19:18.335631+0000","last_became_peered":"2024-03-22T04:19:18.335631+0000","last_unstale":"2024-03-22T04:19:19.322658+0000","last_undegraded":"2024-03-22T04:19:19.322658+0000","last_fullsized":"2024-03-22T04:19:19.322658+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:42:29.640947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778427+0000","last_change":"2024-03-22T04:19:18.339420+0000","last_active":"2024-03-22T04:19:19.778427+0000","last_peered":"2024-03-22T04:19:19.778427+0000","last_clean":"2024-03-22T04:19:19.778427+0000","last_became_active":"2024-03-22T04:19:18.336603+0000","last_became_peered":"2024-03-22T04:19:18.336603+0000","last_unstale":"2024-03-22T04:19:19.778427+0000","last_undegraded":"2024-03-22T04:19:19.778427+0000","last_fullsized":"2024-03-22T04:19:19.778427+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:41:53.294167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323210+0000","last_change":"2024-03-22T04:19:18.319201+0000","last_active":"2024-03-22T04:19:19.323210+0000","last_peered":"2024-03-22T04:19:19.323210+0000","last_clean":"2024-03-22T04:19:19.323210+0000","last_became_active":"2024-03-22T04:19:18.318909+0000","last_became_peered":"2024-03-22T04:19:18.318909+0000","last_unstale":"2024-03-22T04:19:19.323210+0000","last_undegraded":"2024-03-22T04:19:19.323210+0000","last_fullsized":"2024-03-22T04:19:19.323210+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:39:40.104909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779207+0000","last_change":"2024-03-22T04:19:17.325200+0000","last_active":"2024-03-22T04:19:19.779207+0000","last_peered":"2024-03-22T04:19:19.779207+0000","last_clean":"2024-03-22T04:19:19.779207+0000","last_became_active":"2024-03-22T04:19:17.324942+0000","last_became_peered":"2024-03-22T04:19:17.324942+0000","last_unstale":"2024-03-22T04:19:19.779207+0000","last_undegraded":"2024-03-22T04:19:19.779207+0000","last_fullsized":"2024-03-22T04:19:19.779207+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:57.284496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323557+0000","last_change":"2024-03-22T04:19:18.333628+0000","last_active":"2024-03-22T04:19:19.323557+0000","last_peered":"2024-03-22T04:19:19.323557+0000","last_clean":"2024-03-22T04:19:19.323557+0000","last_became_active":"2024-03-22T04:19:18.332722+0000","last_became_peered":"2024-03-22T04:19:18.332722+0000","last_unstale":"2024-03-22T04:19:19.323557+0000","last_undegraded":"2024-03-22T04:19:19.323557+0000","last_fullsized":"2024-03-22T04:19:19.323557+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:34:25.635331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324154+0000","last_change":"2024-03-22T04:19:17.315628+0000","last_active":"2024-03-22T04:19:19.324154+0000","last_peered":"2024-03-22T04:19:19.324154+0000","last_clean":"2024-03-22T04:19:19.324154+0000","last_became_active":"2024-03-22T04:19:17.315012+0000","last_became_peered":"2024-03-22T04:19:17.315012+0000","last_unstale":"2024-03-22T04:19:19.324154+0000","last_undegraded":"2024-03-22T04:19:19.324154+0000","last_fullsized":"2024-03-22T04:19:19.324154+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:08.196395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778767+0000","last_change":"2024-03-22T04:19:18.334880+0000","last_active":"2024-03-22T04:19:19.778767+0000","last_peered":"2024-03-22T04:19:19.778767+0000","last_clean":"2024-03-22T04:19:19.778767+0000","last_became_active":"2024-03-22T04:19:18.334604+0000","last_became_peered":"2024-03-22T04:19:18.334604+0000","last_unstale":"2024-03-22T04:19:19.778767+0000","last_undegraded":"2024-03-22T04:19:19.778767+0000","last_fullsized":"2024-03-22T04:19:19.778767+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:43:48.533737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779060+0000","last_change":"2024-03-22T04:19:17.321241+0000","last_active":"2024-03-22T04:19:19.779060+0000","last_peered":"2024-03-22T04:19:19.779060+0000","last_clean":"2024-03-22T04:19:19.779060+0000","last_became_active":"2024-03-22T04:19:17.320943+0000","last_became_peered":"2024-03-22T04:19:17.320943+0000","last_unstale":"2024-03-22T04:19:19.779060+0000","last_undegraded":"2024-03-22T04:19:19.779060+0000","last_fullsized":"2024-03-22T04:19:19.779060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:34.927515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778492+0000","last_change":"2024-03-22T04:19:18.327481+0000","last_active":"2024-03-22T04:19:19.778492+0000","last_peered":"2024-03-22T04:19:19.778492+0000","last_clean":"2024-03-22T04:19:19.778492+0000","last_became_active":"2024-03-22T04:19:18.327209+0000","last_became_peered":"2024-03-22T04:19:18.327209+0000","last_unstale":"2024-03-22T04:19:19.778492+0000","last_undegraded":"2024-03-22T04:19:19.778492+0000","last_fullsized":"2024-03-22T04:19:19.778492+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:03:04.238555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324418+0000","last_change":"2024-03-22T04:19:17.328186+0000","last_active":"2024-03-22T04:19:19.324418+0000","last_peered":"2024-03-22T04:19:19.324418+0000","last_clean":"2024-03-22T04:19:19.324418+0000","last_became_active":"2024-03-22T04:19:17.327961+0000","last_became_peered":"2024-03-22T04:19:17.327961+0000","last_unstale":"2024-03-22T04:19:19.324418+0000","last_undegraded":"2024-03-22T04:19:19.324418+0000","last_fullsized":"2024-03-22T04:19:19.324418+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:55.072203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.328842+0000","last_change":"2024-03-22T04:19:18.319299+0000","last_active":"2024-03-22T04:19:18.328842+0000","last_peered":"2024-03-22T04:19:18.328842+0000","last_clean":"2024-03-22T04:19:18.328842+0000","last_became_active":"2024-03-22T04:19:18.318985+0000","last_became_peered":"2024-03-22T04:19:18.318985+0000","last_unstale":"2024-03-22T04:19:18.328842+0000","last_undegraded":"2024-03-22T04:19:18.328842+0000","last_fullsized":"2024-03-22T04:19:18.328842+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:38:36.023100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778571+0000","last_change":"2024-03-22T04:19:17.323558+0000","last_active":"2024-03-22T04:19:19.778571+0000","last_peered":"2024-03-22T04:19:19.778571+0000","last_clean":"2024-03-22T04:19:19.778571+0000","last_became_active":"2024-03-22T04:19:17.323233+0000","last_became_peered":"2024-03-22T04:19:17.323233+0000","last_unstale":"2024-03-22T04:19:19.778571+0000","last_undegraded":"2024-03-22T04:19:19.778571+0000","last_fullsized":"2024-03-22T04:19:19.778571+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:23:20.948143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.333015+0000","last_change":"2024-03-22T04:19:18.328570+0000","last_active":"2024-03-22T04:19:18.333015+0000","last_peered":"2024-03-22T04:19:18.333015+0000","last_clean":"2024-03-22T04:19:18.333015+0000","last_became_active":"2024-03-22T04:19:18.328296+0000","last_became_peered":"2024-03-22T04:19:18.328296+0000","last_unstale":"2024-03-22T04:19:18.333015+0000","last_undegraded":"2024-03-22T04:19:18.333015+0000","last_fullsized":"2024-03-22T04:19:18.333015+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:30:33.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742522+0000","last_change":"2024-03-22T04:19:17.323878+0000","last_active":"2024-03-22T04:19:19.742522+0000","last_peered":"2024-03-22T04:19:19.742522+0000","last_clean":"2024-03-22T04:19:19.742522+0000","last_became_active":"2024-03-22T04:19:17.323608+0000","last_became_peered":"2024-03-22T04:19:17.323608+0000","last_unstale":"2024-03-22T04:19:19.742522+0000","last_undegraded":"2024-03-22T04:19:19.742522+0000","last_fullsized":"2024-03-22T04:19:19.742522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:43:47.527273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778981+0000","last_change":"2024-03-22T04:19:18.333147+0000","last_active":"2024-03-22T04:19:19.778981+0000","last_peered":"2024-03-22T04:19:19.778981+0000","last_clean":"2024-03-22T04:19:19.778981+0000","last_became_active":"2024-03-22T04:19:18.332870+0000","last_became_peered":"2024-03-22T04:19:18.332870+0000","last_unstale":"2024-03-22T04:19:19.778981+0000","last_undegraded":"2024-03-22T04:19:19.778981+0000","last_fullsized":"2024-03-22T04:19:19.778981+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:23:27.050207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742545+0000","last_change":"2024-03-22T04:19:17.324924+0000","last_active":"2024-03-22T04:19:19.742545+0000","last_peered":"2024-03-22T04:19:19.742545+0000","last_clean":"2024-03-22T04:19:19.742545+0000","last_became_active":"2024-03-22T04:19:17.324617+0000","last_became_peered":"2024-03-22T04:19:17.324617+0000","last_unstale":"2024-03-22T04:19:19.742545+0000","last_undegraded":"2024-03-22T04:19:19.742545+0000","last_fullsized":"2024-03-22T04:19:19.742545+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:27:35.772140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.339782+0000","last_change":"2024-03-22T04:19:18.337118+0000","last_active":"2024-03-22T04:19:18.339782+0000","last_peered":"2024-03-22T04:19:18.339782+0000","last_clean":"2024-03-22T04:19:18.339782+0000","last_became_active":"2024-03-22T04:19:18.336842+0000","last_became_peered":"2024-03-22T04:19:18.336842+0000","last_unstale":"2024-03-22T04:19:18.339782+0000","last_undegraded":"2024-03-22T04:19:18.339782+0000","last_fullsized":"2024-03-22T04:19:18.339782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:55.930055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778539+0000","last_change":"2024-03-22T04:19:17.324483+0000","last_active":"2024-03-22T04:19:19.778539+0000","last_peered":"2024-03-22T04:19:19.778539+0000","last_clean":"2024-03-22T04:19:19.778539+0000","last_became_active":"2024-03-22T04:19:17.324174+0000","last_became_peered":"2024-03-22T04:19:17.324174+0000","last_unstale":"2024-03-22T04:19:19.778539+0000","last_undegraded":"2024-03-22T04:19:19.778539+0000","last_fullsized":"2024-03-22T04:19:19.778539+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:25.379236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.332825+0000","last_change":"2024-03-22T04:19:18.326890+0000","last_active":"2024-03-22T04:19:18.332825+0000","last_peered":"2024-03-22T04:19:18.332825+0000","last_clean":"2024-03-22T04:19:18.332825+0000","last_became_active":"2024-03-22T04:19:18.324893+0000","last_became_peered":"2024-03-22T04:19:18.324893+0000","last_unstale":"2024-03-22T04:19:18.332825+0000","last_undegraded":"2024-03-22T04:19:18.332825+0000","last_fullsized":"2024-03-22T04:19:18.332825+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:01:00.508043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323353+0000","last_change":"2024-03-22T04:19:17.321789+0000","last_active":"2024-03-22T04:19:19.323353+0000","last_peered":"2024-03-22T04:19:19.323353+0000","last_clean":"2024-03-22T04:19:19.323353+0000","last_became_active":"2024-03-22T04:19:17.321310+0000","last_became_peered":"2024-03-22T04:19:17.321310+0000","last_unstale":"2024-03-22T04:19:19.323353+0000","last_undegraded":"2024-03-22T04:19:19.323353+0000","last_fullsized":"2024-03-22T04:19:19.323353+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:53:22.857897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.339727+0000","last_change":"2024-03-22T04:19:18.335630+0000","last_active":"2024-03-22T04:19:18.339727+0000","last_peered":"2024-03-22T04:19:18.339727+0000","last_clean":"2024-03-22T04:19:18.339727+0000","last_became_active":"2024-03-22T04:19:18.335380+0000","last_became_peered":"2024-03-22T04:19:18.335380+0000","last_unstale":"2024-03-22T04:19:18.339727+0000","last_undegraded":"2024-03-22T04:19:18.339727+0000","last_fullsized":"2024-03-22T04:19:18.339727+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:44:13.800238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324531+0000","last_change":"2024-03-22T04:19:17.324485+0000","last_active":"2024-03-22T04:19:19.324531+0000","last_peered":"2024-03-22T04:19:19.324531+0000","last_clean":"2024-03-22T04:19:19.324531+0000","last_became_active":"2024-03-22T04:19:17.324207+0000","last_became_peered":"2024-03-22T04:19:17.324207+0000","last_unstale":"2024-03-22T04:19:19.324531+0000","last_undegraded":"2024-03-22T04:19:19.324531+0000","last_fullsized":"2024-03-22T04:19:19.324531+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:01:09.512640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323897+0000","last_change":"2024-03-22T04:19:18.340800+0000","last_active":"2024-03-22T04:19:19.323897+0000","last_peered":"2024-03-22T04:19:19.323897+0000","last_clean":"2024-03-22T04:19:19.323897+0000","last_became_active":"2024-03-22T04:19:18.340538+0000","last_became_peered":"2024-03-22T04:19:18.340538+0000","last_unstale":"2024-03-22T04:19:19.323897+0000","last_undegraded":"2024-03-22T04:19:19.323897+0000","last_fullsized":"2024-03-22T04:19:19.323897+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:34:44.461984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323173+0000","last_change":"2024-03-22T04:19:17.325079+0000","last_active":"2024-03-22T04:19:19.323173+0000","last_peered":"2024-03-22T04:19:19.323173+0000","last_clean":"2024-03-22T04:19:19.323173+0000","last_became_active":"2024-03-22T04:19:17.324808+0000","last_became_peered":"2024-03-22T04:19:17.324808+0000","last_unstale":"2024-03-22T04:19:19.323173+0000","last_undegraded":"2024-03-22T04:19:19.323173+0000","last_fullsized":"2024-03-22T04:19:19.323173+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:31:08.196395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778617+0000","last_change":"2024-03-22T04:19:18.331742+0000","last_active":"2024-03-22T04:19:19.778617+0000","last_peered":"2024-03-22T04:19:19.778617+0000","last_clean":"2024-03-22T04:19:19.778617+0000","last_became_active":"2024-03-22T04:19:18.331422+0000","last_became_peered":"2024-03-22T04:19:18.331422+0000","last_unstale":"2024-03-22T04:19:19.778617+0000","last_undegraded":"2024-03-22T04:19:19.778617+0000","last_fullsized":"2024-03-22T04:19:19.778617+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:48:06.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778877+0000","last_change":"2024-03-22T04:19:17.323871+0000","last_active":"2024-03-22T04:19:19.778877+0000","last_peered":"2024-03-22T04:19:19.778877+0000","last_clean":"2024-03-22T04:19:19.778877+0000","last_became_active":"2024-03-22T04:19:17.323633+0000","last_became_peered":"2024-03-22T04:19:17.323633+0000","last_unstale":"2024-03-22T04:19:19.778877+0000","last_undegraded":"2024-03-22T04:19:19.778877+0000","last_fullsized":"2024-03-22T04:19:19.778877+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:48:54.879336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778792+0000","last_change":"2024-03-22T04:19:18.333806+0000","last_active":"2024-03-22T04:19:19.778792+0000","last_peered":"2024-03-22T04:19:19.778792+0000","last_clean":"2024-03-22T04:19:19.778792+0000","last_became_active":"2024-03-22T04:19:18.333455+0000","last_became_peered":"2024-03-22T04:19:18.333455+0000","last_unstale":"2024-03-22T04:19:19.778792+0000","last_undegraded":"2024-03-22T04:19:19.778792+0000","last_fullsized":"2024-03-22T04:19:19.778792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:47.587745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.353342+0000","last_change":"2024-03-22T04:19:17.317443+0000","last_active":"2024-03-22T04:19:18.353342+0000","last_peered":"2024-03-22T04:19:18.353342+0000","last_clean":"2024-03-22T04:19:18.353342+0000","last_became_active":"2024-03-22T04:19:17.313265+0000","last_became_peered":"2024-03-22T04:19:17.313265+0000","last_unstale":"2024-03-22T04:19:18.353342+0000","last_undegraded":"2024-03-22T04:19:18.353342+0000","last_fullsized":"2024-03-22T04:19:18.353342+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:23:47.191984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779036+0000","last_change":"2024-03-22T04:19:18.334614+0000","last_active":"2024-03-22T04:19:19.779036+0000","last_peered":"2024-03-22T04:19:19.779036+0000","last_clean":"2024-03-22T04:19:19.779036+0000","last_became_active":"2024-03-22T04:19:18.334364+0000","last_became_peered":"2024-03-22T04:19:18.334364+0000","last_unstale":"2024-03-22T04:19:19.779036+0000","last_undegraded":"2024-03-22T04:19:19.779036+0000","last_fullsized":"2024-03-22T04:19:19.779036+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:43.362064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323986+0000","last_change":"2024-03-22T04:19:17.322055+0000","last_active":"2024-03-22T04:19:19.323986+0000","last_peered":"2024-03-22T04:19:19.323986+0000","last_clean":"2024-03-22T04:19:19.323986+0000","last_became_active":"2024-03-22T04:19:17.321727+0000","last_became_peered":"2024-03-22T04:19:17.321727+0000","last_unstale":"2024-03-22T04:19:19.323986+0000","last_undegraded":"2024-03-22T04:19:19.323986+0000","last_fullsized":"2024-03-22T04:19:19.323986+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:35:17.288838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779090+0000","last_change":"2024-03-22T04:19:18.345224+0000","last_active":"2024-03-22T04:19:19.779090+0000","last_peered":"2024-03-22T04:19:19.779090+0000","last_clean":"2024-03-22T04:19:19.779090+0000","last_became_active":"2024-03-22T04:19:18.344383+0000","last_became_peered":"2024-03-22T04:19:18.344383+0000","last_unstale":"2024-03-22T04:19:19.779090+0000","last_undegraded":"2024-03-22T04:19:19.779090+0000","last_fullsized":"2024-03-22T04:19:19.779090+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:38:25.882112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.313883+0000","last_change":"2024-03-22T04:19:17.322172+0000","last_active":"2024-03-22T04:19:18.313883+0000","last_peered":"2024-03-22T04:19:18.313883+0000","last_clean":"2024-03-22T04:19:18.313883+0000","last_became_active":"2024-03-22T04:19:17.321881+0000","last_became_peered":"2024-03-22T04:19:17.321881+0000","last_unstale":"2024-03-22T04:19:18.313883+0000","last_undegraded":"2024-03-22T04:19:18.313883+0000","last_fullsized":"2024-03-22T04:19:18.313883+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:30:49.749150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324395+0000","last_change":"2024-03-22T04:19:18.318922+0000","last_active":"2024-03-22T04:19:19.324395+0000","last_peered":"2024-03-22T04:19:19.324395+0000","last_clean":"2024-03-22T04:19:19.324395+0000","last_became_active":"2024-03-22T04:19:18.318590+0000","last_became_peered":"2024-03-22T04:19:18.318590+0000","last_unstale":"2024-03-22T04:19:19.324395+0000","last_undegraded":"2024-03-22T04:19:19.324395+0000","last_fullsized":"2024-03-22T04:19:19.324395+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:20:51.390996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778758+0000","last_change":"2024-03-22T04:19:17.318722+0000","last_active":"2024-03-22T04:19:19.778758+0000","last_peered":"2024-03-22T04:19:19.778758+0000","last_clean":"2024-03-22T04:19:19.778758+0000","last_became_active":"2024-03-22T04:19:17.318429+0000","last_became_peered":"2024-03-22T04:19:17.318429+0000","last_unstale":"2024-03-22T04:19:19.778758+0000","last_undegraded":"2024-03-22T04:19:19.778758+0000","last_fullsized":"2024-03-22T04:19:19.778758+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:32:16.927300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323339+0000","last_change":"2024-03-22T04:19:18.337207+0000","last_active":"2024-03-22T04:19:19.323339+0000","last_peered":"2024-03-22T04:19:19.323339+0000","last_clean":"2024-03-22T04:19:19.323339+0000","last_became_active":"2024-03-22T04:19:18.336144+0000","last_became_peered":"2024-03-22T04:19:18.336144+0000","last_unstale":"2024-03-22T04:19:19.323339+0000","last_undegraded":"2024-03-22T04:19:19.323339+0000","last_fullsized":"2024-03-22T04:19:19.323339+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:49:40.884723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323340+0000","last_change":"2024-03-22T04:19:17.321128+0000","last_active":"2024-03-22T04:19:19.323340+0000","last_peered":"2024-03-22T04:19:19.323340+0000","last_clean":"2024-03-22T04:19:19.323340+0000","last_became_active":"2024-03-22T04:19:17.320791+0000","last_became_peered":"2024-03-22T04:19:17.320791+0000","last_unstale":"2024-03-22T04:19:19.323340+0000","last_undegraded":"2024-03-22T04:19:19.323340+0000","last_fullsized":"2024-03-22T04:19:19.323340+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:42.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742726+0000","last_change":"2024-03-22T04:19:18.331367+0000","last_active":"2024-03-22T04:19:19.742726+0000","last_peered":"2024-03-22T04:19:19.742726+0000","last_clean":"2024-03-22T04:19:19.742726+0000","last_became_active":"2024-03-22T04:19:18.331064+0000","last_became_peered":"2024-03-22T04:19:18.331064+0000","last_unstale":"2024-03-22T04:19:19.742726+0000","last_undegraded":"2024-03-22T04:19:19.742726+0000","last_fullsized":"2024-03-22T04:19:19.742726+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:12:34.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742919+0000","last_change":"2024-03-22T04:19:17.322455+0000","last_active":"2024-03-22T04:19:19.742919+0000","last_peered":"2024-03-22T04:19:19.742919+0000","last_clean":"2024-03-22T04:19:19.742919+0000","last_became_active":"2024-03-22T04:19:17.321998+0000","last_became_peered":"2024-03-22T04:19:17.321998+0000","last_unstale":"2024-03-22T04:19:19.742919+0000","last_undegraded":"2024-03-22T04:19:19.742919+0000","last_fullsized":"2024-03-22T04:19:19.742919+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:46.581281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778548+0000","last_change":"2024-03-22T04:19:18.343245+0000","last_active":"2024-03-22T04:19:19.778548+0000","last_peered":"2024-03-22T04:19:19.778548+0000","last_clean":"2024-03-22T04:19:19.778548+0000","last_became_active":"2024-03-22T04:19:18.342952+0000","last_became_peered":"2024-03-22T04:19:18.342952+0000","last_unstale":"2024-03-22T04:19:19.778548+0000","last_undegraded":"2024-03-22T04:19:19.778548+0000","last_fullsized":"2024-03-22T04:19:19.778548+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:43:04.884756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323030+0000","last_change":"2024-03-22T04:19:17.317197+0000","last_active":"2024-03-22T04:19:19.323030+0000","last_peered":"2024-03-22T04:19:19.323030+0000","last_clean":"2024-03-22T04:19:19.323030+0000","last_became_active":"2024-03-22T04:19:17.316886+0000","last_became_peered":"2024-03-22T04:19:17.316886+0000","last_unstale":"2024-03-22T04:19:19.323030+0000","last_undegraded":"2024-03-22T04:19:19.323030+0000","last_fullsized":"2024-03-22T04:19:19.323030+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:43:47.527273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.338567+0000","last_change":"2024-03-22T04:19:18.334625+0000","last_active":"2024-03-22T04:19:18.338567+0000","last_peered":"2024-03-22T04:19:18.338567+0000","last_clean":"2024-03-22T04:19:18.338567+0000","last_became_active":"2024-03-22T04:19:18.333628+0000","last_became_peered":"2024-03-22T04:19:18.333628+0000","last_unstale":"2024-03-22T04:19:18.338567+0000","last_undegraded":"2024-03-22T04:19:18.338567+0000","last_fullsized":"2024-03-22T04:19:18.338567+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:26:10.608228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324077+0000","last_change":"2024-03-22T04:19:17.312371+0000","last_active":"2024-03-22T04:19:19.324077+0000","last_peered":"2024-03-22T04:19:19.324077+0000","last_clean":"2024-03-22T04:19:19.324077+0000","last_became_active":"2024-03-22T04:19:17.312096+0000","last_became_peered":"2024-03-22T04:19:17.312096+0000","last_unstale":"2024-03-22T04:19:19.324077+0000","last_undegraded":"2024-03-22T04:19:19.324077+0000","last_fullsized":"2024-03-22T04:19:19.324077+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:53:42.355600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779013+0000","last_change":"2024-03-22T04:19:18.340853+0000","last_active":"2024-03-22T04:19:19.779013+0000","last_peered":"2024-03-22T04:19:19.779013+0000","last_clean":"2024-03-22T04:19:19.779013+0000","last_became_active":"2024-03-22T04:19:18.340228+0000","last_became_peered":"2024-03-22T04:19:18.340228+0000","last_unstale":"2024-03-22T04:19:19.779013+0000","last_undegraded":"2024-03-22T04:19:19.779013+0000","last_fullsized":"2024-03-22T04:19:19.779013+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:04:02.043886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323504+0000","last_change":"2024-03-22T04:19:17.320395+0000","last_active":"2024-03-22T04:19:19.323504+0000","last_peered":"2024-03-22T04:19:19.323504+0000","last_clean":"2024-03-22T04:19:19.323504+0000","last_became_active":"2024-03-22T04:19:17.320106+0000","last_became_peered":"2024-03-22T04:19:17.320106+0000","last_unstale":"2024-03-22T04:19:19.323504+0000","last_undegraded":"2024-03-22T04:19:19.323504+0000","last_fullsized":"2024-03-22T04:19:19.323504+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:43:47.527273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778909+0000","last_change":"2024-03-22T04:19:18.340689+0000","last_active":"2024-03-22T04:19:19.778909+0000","last_peered":"2024-03-22T04:19:19.778909+0000","last_clean":"2024-03-22T04:19:19.778909+0000","last_became_active":"2024-03-22T04:19:18.340033+0000","last_became_peered":"2024-03-22T04:19:18.340033+0000","last_unstale":"2024-03-22T04:19:19.778909+0000","last_undegraded":"2024-03-22T04:19:19.778909+0000","last_fullsized":"2024-03-22T04:19:19.778909+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:49:31.761279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322759+0000","last_change":"2024-03-22T04:19:17.325983+0000","last_active":"2024-03-22T04:19:19.322759+0000","last_peered":"2024-03-22T04:19:19.322759+0000","last_clean":"2024-03-22T04:19:19.322759+0000","last_became_active":"2024-03-22T04:19:17.325466+0000","last_became_peered":"2024-03-22T04:19:17.325466+0000","last_unstale":"2024-03-22T04:19:19.322759+0000","last_undegraded":"2024-03-22T04:19:19.322759+0000","last_fullsized":"2024-03-22T04:19:19.322759+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:46.581281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.339800+0000","last_change":"2024-03-22T04:19:18.337141+0000","last_active":"2024-03-22T04:19:18.339800+0000","last_peered":"2024-03-22T04:19:18.339800+0000","last_clean":"2024-03-22T04:19:18.339800+0000","last_became_active":"2024-03-22T04:19:18.336867+0000","last_became_peered":"2024-03-22T04:19:18.336867+0000","last_unstale":"2024-03-22T04:19:18.339800+0000","last_undegraded":"2024-03-22T04:19:18.339800+0000","last_fullsized":"2024-03-22T04:19:18.339800+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:13:07.271663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323482+0000","last_change":"2024-03-22T04:19:17.326510+0000","last_active":"2024-03-22T04:19:19.323482+0000","last_peered":"2024-03-22T04:19:19.323482+0000","last_clean":"2024-03-22T04:19:19.323482+0000","last_became_active":"2024-03-22T04:19:17.325991+0000","last_became_peered":"2024-03-22T04:19:17.325991+0000","last_unstale":"2024-03-22T04:19:19.323482+0000","last_undegraded":"2024-03-22T04:19:19.323482+0000","last_fullsized":"2024-03-22T04:19:19.323482+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:47:50.510343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778688+0000","last_change":"2024-03-22T04:19:18.339601+0000","last_active":"2024-03-22T04:19:19.778688+0000","last_peered":"2024-03-22T04:19:19.778688+0000","last_clean":"2024-03-22T04:19:19.778688+0000","last_became_active":"2024-03-22T04:19:18.339312+0000","last_became_peered":"2024-03-22T04:19:18.339312+0000","last_unstale":"2024-03-22T04:19:19.778688+0000","last_undegraded":"2024-03-22T04:19:19.778688+0000","last_fullsized":"2024-03-22T04:19:19.778688+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:28:35.933979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.324474+0000","last_change":"2024-03-22T04:19:17.323198+0000","last_active":"2024-03-22T04:19:19.324474+0000","last_peered":"2024-03-22T04:19:19.324474+0000","last_clean":"2024-03-22T04:19:19.324474+0000","last_became_active":"2024-03-22T04:19:17.322902+0000","last_became_peered":"2024-03-22T04:19:17.322902+0000","last_unstale":"2024-03-22T04:19:19.324474+0000","last_undegraded":"2024-03-22T04:19:19.324474+0000","last_fullsized":"2024-03-22T04:19:19.324474+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:35:17.288838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778772+0000","last_change":"2024-03-22T04:19:18.334033+0000","last_active":"2024-03-22T04:19:19.778772+0000","last_peered":"2024-03-22T04:19:19.778772+0000","last_clean":"2024-03-22T04:19:19.778772+0000","last_became_active":"2024-03-22T04:19:18.333788+0000","last_became_peered":"2024-03-22T04:19:18.333788+0000","last_unstale":"2024-03-22T04:19:19.778772+0000","last_undegraded":"2024-03-22T04:19:19.778772+0000","last_fullsized":"2024-03-22T04:19:19.778772+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:18:30.823672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"109'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.339694+0000","last_change":"2024-03-22T04:19:17.315344+0000","last_active":"2024-03-22T04:19:19.339694+0000","last_peered":"2024-03-22T04:19:19.339694+0000","last_clean":"2024-03-22T04:19:19.339694+0000","last_became_active":"2024-03-22T04:19:17.314614+0000","last_became_peered":"2024-03-22T04:19:17.314614+0000","last_unstale":"2024-03-22T04:19:19.339694+0000","last_undegraded":"2024-03-22T04:19:19.339694+0000","last_fullsized":"2024-03-22T04:19:19.339694+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:01:09.512640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323558+0000","last_change":"2024-03-22T04:19:18.339080+0000","last_active":"2024-03-22T04:19:19.323558+0000","last_peered":"2024-03-22T04:19:19.323558+0000","last_clean":"2024-03-22T04:19:19.323558+0000","last_became_active":"2024-03-22T04:19:18.338695+0000","last_became_peered":"2024-03-22T04:19:18.338695+0000","last_unstale":"2024-03-22T04:19:19.323558+0000","last_undegraded":"2024-03-22T04:19:19.323558+0000","last_fullsized":"2024-03-22T04:19:19.323558+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:47:49.253227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779212+0000","last_change":"2024-03-22T04:19:17.318521+0000","last_active":"2024-03-22T04:19:19.779212+0000","last_peered":"2024-03-22T04:19:19.779212+0000","last_clean":"2024-03-22T04:19:19.779212+0000","last_became_active":"2024-03-22T04:19:17.318203+0000","last_became_peered":"2024-03-22T04:19:17.318203+0000","last_unstale":"2024-03-22T04:19:19.779212+0000","last_undegraded":"2024-03-22T04:19:19.779212+0000","last_fullsized":"2024-03-22T04:19:19.779212+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:49:47.221788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742784+0000","last_change":"2024-03-22T04:19:18.333376+0000","last_active":"2024-03-22T04:19:19.742784+0000","last_peered":"2024-03-22T04:19:19.742784+0000","last_clean":"2024-03-22T04:19:19.742784+0000","last_became_active":"2024-03-22T04:19:18.333117+0000","last_became_peered":"2024-03-22T04:19:18.333117+0000","last_unstale":"2024-03-22T04:19:19.742784+0000","last_undegraded":"2024-03-22T04:19:19.742784+0000","last_fullsized":"2024-03-22T04:19:19.742784+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:54:28.206493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.316566+0000","last_change":"2024-03-22T04:19:17.318663+0000","last_active":"2024-03-22T04:19:18.316566+0000","last_peered":"2024-03-22T04:19:18.316566+0000","last_clean":"2024-03-22T04:19:18.316566+0000","last_became_active":"2024-03-22T04:19:17.318400+0000","last_became_peered":"2024-03-22T04:19:17.318400+0000","last_unstale":"2024-03-22T04:19:18.316566+0000","last_undegraded":"2024-03-22T04:19:18.316566+0000","last_fullsized":"2024-03-22T04:19:18.316566+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:33:50.944233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.339340+0000","last_change":"2024-03-22T04:19:18.335499+0000","last_active":"2024-03-22T04:19:18.339340+0000","last_peered":"2024-03-22T04:19:18.339340+0000","last_clean":"2024-03-22T04:19:18.339340+0000","last_became_active":"2024-03-22T04:19:18.333965+0000","last_became_peered":"2024-03-22T04:19:18.333965+0000","last_unstale":"2024-03-22T04:19:18.339340+0000","last_undegraded":"2024-03-22T04:19:18.339340+0000","last_fullsized":"2024-03-22T04:19:18.339340+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:20:47.281468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.778695+0000","last_change":"2024-03-22T04:19:17.322774+0000","last_active":"2024-03-22T04:19:19.778695+0000","last_peered":"2024-03-22T04:19:19.778695+0000","last_clean":"2024-03-22T04:19:19.778695+0000","last_became_active":"2024-03-22T04:19:17.322447+0000","last_became_peered":"2024-03-22T04:19:17.322447+0000","last_unstale":"2024-03-22T04:19:19.778695+0000","last_undegraded":"2024-03-22T04:19:19.778695+0000","last_fullsized":"2024-03-22T04:19:19.778695+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:46:41.660361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.338992+0000","last_change":"2024-03-22T04:19:18.335356+0000","last_active":"2024-03-22T04:19:18.338992+0000","last_peered":"2024-03-22T04:19:18.338992+0000","last_clean":"2024-03-22T04:19:18.338992+0000","last_became_active":"2024-03-22T04:19:18.334300+0000","last_became_peered":"2024-03-22T04:19:18.334300+0000","last_unstale":"2024-03-22T04:19:18.338992+0000","last_undegraded":"2024-03-22T04:19:18.338992+0000","last_fullsized":"2024-03-22T04:19:18.338992+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:49:36.153576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323773+0000","last_change":"2024-03-22T04:19:17.313969+0000","last_active":"2024-03-22T04:19:19.323773+0000","last_peered":"2024-03-22T04:19:19.323773+0000","last_clean":"2024-03-22T04:19:19.323773+0000","last_became_active":"2024-03-22T04:19:17.313399+0000","last_became_peered":"2024-03-22T04:19:17.313399+0000","last_unstale":"2024-03-22T04:19:19.323773+0000","last_undegraded":"2024-03-22T04:19:19.323773+0000","last_fullsized":"2024-03-22T04:19:19.323773+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:46.581281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323330+0000","last_change":"2024-03-22T04:19:18.328695+0000","last_active":"2024-03-22T04:19:19.323330+0000","last_peered":"2024-03-22T04:19:19.323330+0000","last_clean":"2024-03-22T04:19:19.323330+0000","last_became_active":"2024-03-22T04:19:18.328379+0000","last_became_peered":"2024-03-22T04:19:18.328379+0000","last_unstale":"2024-03-22T04:19:19.323330+0000","last_undegraded":"2024-03-22T04:19:19.323330+0000","last_fullsized":"2024-03-22T04:19:19.323330+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:15:35.873149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742590+0000","last_change":"2024-03-22T04:19:17.322375+0000","last_active":"2024-03-22T04:19:19.742590+0000","last_peered":"2024-03-22T04:19:19.742590+0000","last_clean":"2024-03-22T04:19:19.742590+0000","last_became_active":"2024-03-22T04:19:17.322125+0000","last_became_peered":"2024-03-22T04:19:17.322125+0000","last_unstale":"2024-03-22T04:19:19.742590+0000","last_undegraded":"2024-03-22T04:19:19.742590+0000","last_fullsized":"2024-03-22T04:19:19.742590+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:53:22.857897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.779286+0000","last_change":"2024-03-22T04:19:18.339401+0000","last_active":"2024-03-22T04:19:19.779286+0000","last_peered":"2024-03-22T04:19:19.779286+0000","last_clean":"2024-03-22T04:19:19.779286+0000","last_became_active":"2024-03-22T04:19:18.339105+0000","last_became_peered":"2024-03-22T04:19:18.339105+0000","last_unstale":"2024-03-22T04:19:19.779286+0000","last_undegraded":"2024-03-22T04:19:19.779286+0000","last_fullsized":"2024-03-22T04:19:19.779286+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:56.078667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.323271+0000","last_change":"2024-03-22T04:19:17.320664+0000","last_active":"2024-03-22T04:19:19.323271+0000","last_peered":"2024-03-22T04:19:19.323271+0000","last_clean":"2024-03-22T04:19:19.323271+0000","last_became_active":"2024-03-22T04:19:17.320369+0000","last_became_peered":"2024-03-22T04:19:17.320369+0000","last_unstale":"2024-03-22T04:19:19.323271+0000","last_undegraded":"2024-03-22T04:19:19.323271+0000","last_fullsized":"2024-03-22T04:19:19.323271+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:58:55.072203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.322802+0000","last_change":"2024-03-22T04:19:18.330608+0000","last_active":"2024-03-22T04:19:19.322802+0000","last_peered":"2024-03-22T04:19:19.322802+0000","last_clean":"2024-03-22T04:19:19.322802+0000","last_became_active":"2024-03-22T04:19:18.330220+0000","last_became_peered":"2024-03-22T04:19:18.330220+0000","last_unstale":"2024-03-22T04:19:19.322802+0000","last_undegraded":"2024-03-22T04:19:19.322802+0000","last_fullsized":"2024-03-22T04:19:19.322802+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:17.293257+0000","last_clean_scrub_stamp":"2024-03-22T04:19:17.293257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T04:20:51.390996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-03-22T04:19:18.351443+0000","last_change":"2024-03-22T04:19:17.323564+0000","last_active":"2024-03-22T04:19:18.351443+0000","last_peered":"2024-03-22T04:19:18.351443+0000","last_clean":"2024-03-22T04:19:18.351443+0000","last_became_active":"2024-03-22T04:19:17.323148+0000","last_became_peered":"2024-03-22T04:19:17.323148+0000","last_unstale":"2024-03-22T04:19:18.351443+0000","last_undegraded":"2024-03-22T04:19:18.351443+0000","last_fullsized":"2024-03-22T04:19:18.351443+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:19:16.286793+0000","last_clean_scrub_stamp":"2024-03-22T04:19:16.286793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:05:38.173133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":332,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-22T04:19:19.742255+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:19:19.742255+0000","last_peered":"2024-03-22T04:19:19.742255+0000","last_clean":"2024-03-22T04:19:19.742255+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:19:19.742255+0000","last_undegraded":"2024-03-22T04:19:19.742255+0000","last_fullsized":"2024-03-22T04:19:19.742255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509551,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":625146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":626106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1945600,"data_stored":1214986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509551,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":626106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":624606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509551,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1945600,"data_stored":1214954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":625076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":617836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:19:22.797 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T04:19:22.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-22T04:19:22.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0978600640 1 Processor -- start 2024-03-22T04:19:22.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0978600640 1 -- start start 2024-03-22T04:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0978600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f09700d20f0 0x7f09700d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0978600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f097015c790 0x7f09700701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0978600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09700d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0978600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f097015d7b0 con 0x7f097015c790 2024-03-22T04:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0978600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f09700d2a10 con 0x7f09700d20f0 2024-03-22T04:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0978600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f097015cc30 con 0x7f09700cfd30 2024-03-22T04:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0975200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f097015c790 0x7f09700701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0976600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09700d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0975c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f09700d20f0 0x7f09700d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0976600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09700d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:38918/0 (socket says 172.21.15.86:38918) 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0975200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f097015c790 0x7f09700701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40036/0 (socket says 172.21.15.86:40036) 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0976600640 1 -- 172.21.15.86:0/2663213766 learned_addr learned my addr 172.21.15.86:0/2663213766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0976600640 1 -- 172.21.15.86:0/2663213766 >> v2:172.21.15.196:3300/0 conn(0x7f09700d20f0 msgr2=0x7f09700d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0976600640 1 --2- 172.21.15.86:0/2663213766 >> v2:172.21.15.196:3300/0 conn(0x7f09700d20f0 0x7f09700d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0976600640 1 -- 172.21.15.86:0/2663213766 >> v2:172.21.15.86:3300/0 conn(0x7f097015c790 msgr2=0x7f09700701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0976600640 1 --2- 172.21.15.86:0/2663213766 >> v2:172.21.15.86:3300/0 conn(0x7f097015c790 0x7f09700701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.940+0000 7f0976600640 1 -- 172.21.15.86:0/2663213766 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097015cf10 con 0x7f09700cfd30 2024-03-22T04:19:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0976600640 1 --2- 172.21.15.86:0/2663213766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09700d0160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f09700d2cb0 tx=0x7f096801c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac37c155d00fa2b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f096fe00640 1 -- 172.21.15.86:0/2663213766 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09680a55e0 con 0x7f09700cfd30 2024-03-22T04:19:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f096fe00640 1 -- 172.21.15.86:0/2663213766 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09680a5780 con 0x7f09700cfd30 2024-03-22T04:19:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f096fe00640 1 -- 172.21.15.86:0/2663213766 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09680a5b50 con 0x7f09700cfd30 2024-03-22T04:19:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 -- 172.21.15.86:0/2663213766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 msgr2=0x7f09700d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 --2- 172.21.15.86:0/2663213766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09700d0160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f09700d2cb0 tx=0x7f096801c960 comp rx=0 tx=0).stop 2024-03-22T04:19:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 -- 172.21.15.86:0/2663213766 shutdown_connections 2024-03-22T04:19:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 --2- 172.21.15.86:0/2663213766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09700d0160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 --2- 172.21.15.86:0/2663213766 >> v2:172.21.15.86:3300/0 conn(0x7f097015c790 0x7f09700701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 --2- 172.21.15.86:0/2663213766 >> v2:172.21.15.196:3300/0 conn(0x7f09700d20f0 0x7f09700d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 -- 172.21.15.86:0/2663213766 >> 172.21.15.86:0/2663213766 conn(0x7f097006b660 msgr2=0x7f097006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 -- 172.21.15.86:0/2663213766 shutdown_connections 2024-03-22T04:19:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.944+0000 7f0978600640 1 -- 172.21.15.86:0/2663213766 wait complete. 2024-03-22T04:19:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 Processor -- start 2024-03-22T04:19:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 -- start start 2024-03-22T04:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09701660f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09700d20f0 0x7f0970164170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097015c790 0x7f0970153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0970156d00 con 0x7f09700d20f0 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0970156b80 con 0x7f097015c790 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0970070700 con 0x7f09700cfd30 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09700d20f0 0x7f0970164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09700d20f0 0x7f0970164170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40044/0 (socket says 172.21.15.86:40044) 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 -- 172.21.15.86:0/3994280150 learned_addr learned my addr 172.21.15.86:0/3994280150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975c00640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09701660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0976600640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097015c790 0x7f0970153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 -- 172.21.15.86:0/3994280150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 msgr2=0x7f09701660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09701660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 -- 172.21.15.86:0/3994280150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097015c790 msgr2=0x7f0970153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097015c790 0x7f0970153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 -- 172.21.15.86:0/3994280150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0970165540 con 0x7f09700d20f0 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0975200640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09700d20f0 0x7f0970164170 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f095c00e870 tx=0x7f095c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f096d600640 1 -- 172.21.15.86:0/3994280150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c017070 con 0x7f09700d20f0 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f096d600640 1 -- 172.21.15.86:0/3994280150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c00ee60 con 0x7f09700d20f0 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09701649c0 con 0x7f09700d20f0 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f096d600640 1 -- 172.21.15.86:0/3994280150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c00cb40 con 0x7f09700d20f0 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.948+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0970164f80 con 0x7f09700d20f0 2024-03-22T04:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.952+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0970074470 con 0x7f09700d20f0 2024-03-22T04:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.952+0000 7f096d600640 1 -- 172.21.15.86:0/3994280150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f095c01a070 con 0x7f09700d20f0 2024-03-22T04:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.952+0000 7f096d600640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0950067b20 0x7f0950069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.952+0000 7f096d600640 1 -- 172.21.15.86:0/3994280150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f095c085f00 con 0x7f09700d20f0 2024-03-22T04:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.952+0000 7f0975c00640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0950067b20 0x7f0950069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.952+0000 7f0975c00640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0950067b20 0x7f0950069fe0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0964009a40 tx=0x7f0964009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:22.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:22.956+0000 7f096d600640 1 -- 172.21.15.86:0/3994280150 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f095c0560a0 con 0x7f09700d20f0 2024-03-22T04:19:23.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.164+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f09700c9210 con 0x7f09700d20f0 2024-03-22T04:19:23.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.164+0000 7f096d600640 1 -- 172.21.15.86:0/3994280150 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f095c05a140 con 0x7f09700d20f0 2024-03-22T04:19:23.167 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T04:19:23.167 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:19:23.167 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-03-22T04:19:23.167 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-03-22T04:19:23.167 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-22T04:19:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0950067b20 msgr2=0x7f0950069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0950067b20 0x7f0950069fe0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0964009a40 tx=0x7f0964009190 comp rx=0 tx=0).stop 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09700d20f0 msgr2=0x7f0970164170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09700d20f0 0x7f0970164170 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f095c00e870 tx=0x7f095c00ed30 comp rx=0 tx=0).stop 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 shutdown_connections 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0950067b20 0x7f0950069fe0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f097015c790 0x7f0970153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09700d20f0 0x7f0970164170 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 --2- 172.21.15.86:0/3994280150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09700cfd30 0x7f09701660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 >> 172.21.15.86:0/3994280150 conn(0x7f097006b660 msgr2=0x7f097006dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:23.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 shutdown_connections 2024-03-22T04:19:23.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.168+0000 7f0978600640 1 -- 172.21.15.86:0/3994280150 wait complete. 2024-03-22T04:19:23.187 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:23.288 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-03-22T04:19:23.287+0000 7f485ca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:19:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5fe200640 1 Processor -- start 2024-03-22T04:19:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5fe200640 1 -- start start 2024-03-22T04:19:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5fe200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5f80d0200 0x7fa5f80d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5fe200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5f806c010 con 0x7fa5f80704a0 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5fe200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5f8162740 con 0x7fa5f80700c0 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5fe200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa5f8162a20 con 0x7fa5f80d0200 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5f7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5f80d0200 0x7fa5f80d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5f7400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa5f80700c0 0x7fa5f8072850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:36022/0 (socket says 172.21.15.86:36022) 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5f7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5f80d0200 0x7fa5f80d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40052/0 (socket says 172.21.15.86:40052) 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5f7400640 1 -- 172.21.15.86:0/3567928455 learned_addr learned my addr 172.21.15.86:0/3567928455 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5eea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa5f80704a0 0x7fa5f806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44382/0 (socket says 172.21.15.86:44382) 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5f7e00640 1 -- 172.21.15.86:0/3567928455 >> v1:172.21.15.196:6789/0 conn(0x7fa5f80700c0 legacy=0x7fa5f8072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5f7e00640 1 -- 172.21.15.86:0/3567928455 >> v1:172.21.15.86:6789/0 conn(0x7fa5f80704a0 legacy=0x7fa5f806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5f7e00640 1 -- 172.21.15.86:0/3567928455 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f8162d00 con 0x7fa5f80d0200 2024-03-22T04:19:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.324+0000 7fa5f6a00640 1 -- 172.21.15.86:0/3567928455 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024969667 0 0) 0x7fa5f806c010 con 0x7fa5f80704a0 2024-03-22T04:19:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5f7e00640 1 --2- 172.21.15.86:0/3567928455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f80d0690 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00097c0 tx=0x7fa5f001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73d4ce7b61670bf server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5f6a00640 1 -- 172.21.15.86:0/3567928455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00b0070 con 0x7fa5f80d0200 2024-03-22T04:19:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5f6a00640 1 -- 172.21.15.86:0/3567928455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f00ab600 con 0x7fa5f80d0200 2024-03-22T04:19:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5f6a00640 1 -- 172.21.15.86:0/3567928455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00ab9d0 con 0x7fa5f80d0200 2024-03-22T04:19:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5fe200640 1 -- 172.21.15.86:0/3567928455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 msgr2=0x7fa5f80d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5fe200640 1 --2- 172.21.15.86:0/3567928455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f80d0690 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00097c0 tx=0x7fa5f001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5fe200640 1 -- 172.21.15.86:0/3567928455 shutdown_connections 2024-03-22T04:19:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5fe200640 1 --2- 172.21.15.86:0/3567928455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f80d0690 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5fe200640 1 -- 172.21.15.86:0/3567928455 >> 172.21.15.86:0/3567928455 conn(0x7fa5f80b2d00 msgr2=0x7fa5f80b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5fe200640 1 -- 172.21.15.86:0/3567928455 shutdown_connections 2024-03-22T04:19:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.328+0000 7fa5fe200640 1 -- 172.21.15.86:0/3567928455 wait complete. 2024-03-22T04:19:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 Processor -- start 2024-03-22T04:19:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 -- start start 2024-03-22T04:19:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5f80700c0 0x7fa5f80b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5f80704a0 0x7fa5f81faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f81fb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f815fab0 con 0x7fa5f80d0200 2024-03-22T04:19:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8162500 con 0x7fa5f80704a0 2024-03-22T04:19:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f80aefe0 con 0x7fa5f80700c0 2024-03-22T04:19:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f81fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f81fb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40064/0 (socket says 172.21.15.86:40064) 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 -- 172.21.15.86:0/2993977723 learned_addr learned my addr 172.21.15.86:0/2993977723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7400640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5f80700c0 0x7fa5f80b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5eea00640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5f80704a0 0x7fa5f81faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 -- 172.21.15.86:0/2993977723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5f80700c0 msgr2=0x7fa5f80b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5f80700c0 0x7fa5f80b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 -- 172.21.15.86:0/2993977723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5f80704a0 msgr2=0x7fa5f81faf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5f80704a0 0x7fa5f81faf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 -- 172.21.15.86:0/2993977723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f80af290 con 0x7fa5f80d0200 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5f7e00640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f81fb450 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00ae040 tx=0x7fa5f0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5efe00640 1 -- 172.21.15.86:0/2993977723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00b0040 con 0x7fa5f80d0200 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5efe00640 1 -- 172.21.15.86:0/2993977723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f00b9070 con 0x7fa5f80d0200 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f80af570 con 0x7fa5f80d0200 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5efe00640 1 -- 172.21.15.86:0/2993977723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00b4630 con 0x7fa5f80d0200 2024-03-22T04:19:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f81fb990 con 0x7fa5f80d0200 2024-03-22T04:19:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.332+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f8068650 con 0x7fa5f80d0200 2024-03-22T04:19:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.336+0000 7fa5efe00640 1 -- 172.21.15.86:0/2993977723 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa5f00b47d0 con 0x7fa5f80d0200 2024-03-22T04:19:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.336+0000 7fa5efe00640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5d4067b20 0x7fa5d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.336+0000 7fa5efe00640 1 -- 172.21.15.86:0/2993977723 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fa5f011fcf0 con 0x7fa5f80d0200 2024-03-22T04:19:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.336+0000 7fa5f7400640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5d4067b20 0x7fa5d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.336+0000 7fa5f7400640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5d4067b20 0x7fa5d4069fe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005890 tx=0x7fa5e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.340+0000 7fa5efe00640 1 -- 172.21.15.86:0/2993977723 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa5f0124070 con 0x7fa5f80d0200 2024-03-22T04:19:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.528+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5f8070da0 con 0x7fa5f80d0200 2024-03-22T04:19:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.528+0000 7fa5efe00640 1 -- 172.21.15.86:0/2993977723 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5192 (secure 0 0 0) 0x7fa5f0016090 con 0x7fa5f80d0200 2024-03-22T04:19:23.529 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:23.529 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":77159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/269054026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":269054026},{"type":"v1","addr":"172.21.15.196:6834","nonce":269054026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:19:20.698020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77162":{"gid":77162,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2259098413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2259098413},{"type":"v1","addr":"172.21.15.196:6835","nonce":2259098413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77162,"qdb_cluster":[77162]},"id":7}]} 2024-03-22T04:19:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 84 2024-03-22T04:19:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5d4067b20 msgr2=0x7fa5d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5d4067b20 0x7fa5d4069fe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005890 tx=0x7fa5e8009040 comp rx=0 tx=0).stop 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 msgr2=0x7fa5f81fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f81fb450 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00ae040 tx=0x7fa5f0019240 comp rx=0 tx=0).stop 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 shutdown_connections 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa5d4067b20 0x7fa5d4069fe0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5f80704a0 0x7fa5f81faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f80d0200 0x7fa5f81fb450 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 --2- 172.21.15.86:0/2993977723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5f80700c0 0x7fa5f80b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 >> 172.21.15.86:0/2993977723 conn(0x7fa5f80b2d00 msgr2=0x7fa5f8160a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 shutdown_connections 2024-03-22T04:19:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:23.532+0000 7fa5fe200640 1 -- 172.21.15.86:0/2993977723 wait complete. 2024-03-22T04:19:23.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:19:17.684626+0000', 'modified': '2024-03-22T04:19:20.698020+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 77162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_77162': {'gid': 77162, 'name': 'b', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/2259098413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6833', 'nonce': 2259098413}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 2259098413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 77162, 'qdb_cluster': [77162]} 2024-03-22T04:19:23.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:19:23.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-22T04:19:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.088+0000 7fe575a00640 1 Processor -- start 2024-03-22T04:19:24.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.088+0000 7fe575a00640 1 -- start start 2024-03-22T04:19:24.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.088+0000 7fe575a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5700ad330 0x7fe57015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.088+0000 7fe575a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57015c7c0 con 0x7fe5700ad710 2024-03-22T04:19:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.088+0000 7fe575a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57006dea0 con 0x7fe57006ef40 2024-03-22T04:19:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.088+0000 7fe575a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe57006e180 con 0x7fe5700ad330 2024-03-22T04:19:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe5700ad710 0x7fe5701402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44398/0 (socket says 172.21.15.86:44398) 2024-03-22T04:19:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56f400640 1 -- 172.21.15.86:0/2958393463 learned_addr learned my addr 172.21.15.86:0/2958393463 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:24.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56ea00640 1 --2- 172.21.15.86:0/2958393463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5700ad330 0x7fe57015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56e000640 1 -- 172.21.15.86:0/2958393463 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2733170828 0 0) 0x7fe57015c7c0 con 0x7fe5700ad710 2024-03-22T04:19:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56e000640 1 -- 172.21.15.86:0/2958393463 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe554003530 con 0x7fe5700ad710 2024-03-22T04:19:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56e000640 1 -- 172.21.15.86:0/2958393463 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790630266 0 0) 0x7fe57006dea0 con 0x7fe57006ef40 2024-03-22T04:19:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56e000640 1 -- 172.21.15.86:0/2958393463 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe57015c7c0 con 0x7fe57006ef40 2024-03-22T04:19:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56ea00640 1 -- 172.21.15.86:0/2958393463 >> v1:172.21.15.196:6789/0 conn(0x7fe57006ef40 legacy=0x7fe57013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56ea00640 1 -- 172.21.15.86:0/2958393463 >> v1:172.21.15.86:6789/0 conn(0x7fe5700ad710 legacy=0x7fe5701402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56ea00640 1 -- 172.21.15.86:0/2958393463 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57006e460 con 0x7fe5700ad330 2024-03-22T04:19:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56ea00640 1 --2- 172.21.15.86:0/2958393463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5700ad330 0x7fe57015d4a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097f0 tx=0x7fe56001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b796b7cb31a10cd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56e000640 1 -- 172.21.15.86:0/2958393463 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600b1070 con 0x7fe5700ad330 2024-03-22T04:19:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56e000640 1 -- 172.21.15.86:0/2958393463 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5600ac620 con 0x7fe5700ad330 2024-03-22T04:19:24.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.092+0000 7fe56e000640 1 -- 172.21.15.86:0/2958393463 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600ac9f0 con 0x7fe5700ad330 2024-03-22T04:19:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- 172.21.15.86:0/2958393463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5700ad330 msgr2=0x7fe57015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 --2- 172.21.15.86:0/2958393463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5700ad330 0x7fe57015d4a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097f0 tx=0x7fe56001c4b0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- 172.21.15.86:0/2958393463 shutdown_connections 2024-03-22T04:19:24.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 --2- 172.21.15.86:0/2958393463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5700ad330 0x7fe57015d4a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- 172.21.15.86:0/2958393463 >> 172.21.15.86:0/2958393463 conn(0x7fe5700b1df0 msgr2=0x7fe5700b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- 172.21.15.86:0/2958393463 shutdown_connections 2024-03-22T04:19:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- 172.21.15.86:0/2958393463 wait complete. 2024-03-22T04:19:24.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 Processor -- start 2024-03-22T04:19:24.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- start start 2024-03-22T04:19:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe57006ef40 0x7fe570164870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5700ad330 0x7fe5700b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5700ad710 0x7fe570099170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe57016a780 con 0x7fe5700ad330 2024-03-22T04:19:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe57006db10 con 0x7fe5700ad710 2024-03-22T04:19:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe575a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe57016a900 con 0x7fe57006ef40 2024-03-22T04:19:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe566000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5700ad330 0x7fe5700b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe566000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5700ad330 0x7fe5700b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40072/0 (socket says 172.21.15.86:40072) 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe566000640 1 -- 172.21.15.86:0/3795406390 learned_addr learned my addr 172.21.15.86:0/3795406390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe56ea00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe57006ef40 0x7fe570164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe56f400640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5700ad710 0x7fe570099170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe566000640 1 -- 172.21.15.86:0/3795406390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe57006ef40 msgr2=0x7fe570164870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe566000640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe57006ef40 0x7fe570164870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe566000640 1 -- 172.21.15.86:0/3795406390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5700ad710 msgr2=0x7fe570099170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe566000640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5700ad710 0x7fe570099170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.096+0000 7fe566000640 1 -- 172.21.15.86:0/3795406390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57015d810 con 0x7fe5700ad330 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe566000640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5700ad330 0x7fe5700b4190 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c500 tx=0x7fe55800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe567400640 1 -- 172.21.15.86:0/3795406390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5580052d0 con 0x7fe5700ad330 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe567400640 1 -- 172.21.15.86:0/3795406390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe558005490 con 0x7fe5700ad330 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe575a00640 1 -- 172.21.15.86:0/3795406390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5700d0f80 con 0x7fe5700ad330 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe567400640 1 -- 172.21.15.86:0/3795406390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558012850 con 0x7fe5700ad330 2024-03-22T04:19:24.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe575a00640 1 -- 172.21.15.86:0/3795406390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5700b2740 con 0x7fe5700ad330 2024-03-22T04:19:24.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe567400640 1 -- 172.21.15.86:0/3795406390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe5580129f0 con 0x7fe5700ad330 2024-03-22T04:19:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe567400640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe54c067b20 0x7fe54c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe567400640 1 -- 172.21.15.86:0/3795406390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fe558085c80 con 0x7fe5700ad330 2024-03-22T04:19:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe56ea00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe54c067b20 0x7fe54c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.100+0000 7fe578eb6000 1 Processor -- start 2024-03-22T04:19:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- start start 2024-03-22T04:19:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe56ea00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe54c067b20 0x7fe54c069fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe5600ae000 tx=0x7fe5600a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da76250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d827140 con 0x55ac8da76790 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d87ab10 con 0x55ac8da29c20 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ac8d801c50 con 0x55ac8da75e70 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe56ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da76250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da296e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40084/0 (socket says 172.21.15.86:40084) 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 -- 172.21.15.86:0/2508606842 learned_addr learned my addr 172.21.15.86:0/2508606842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe56f400640 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 -- 172.21.15.86:0/2508606842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 msgr2=0x55ac8da76250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da76250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 -- 172.21.15.86:0/2508606842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 msgr2=0x55ac8da2a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 -- 172.21.15.86:0/2508606842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8d8bff20 con 0x55ac8da76790 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe56ea00640 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da76250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe566000640 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da296e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe558055b20 tx=0x7fe558055880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177dc40da5ae2fcc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe55f400640 1 -- 172.21.15.86:0/2508606842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558054120 con 0x55ac8da76790 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe55f400640 1 -- 172.21.15.86:0/2508606842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe558012ce0 con 0x55ac8da76790 2024-03-22T04:19:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe55f400640 1 -- 172.21.15.86:0/2508606842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558055270 con 0x55ac8da76790 2024-03-22T04:19:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- 172.21.15.86:0/2508606842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 msgr2=0x55ac8da296e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da296e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe558055b20 tx=0x7fe558055880 comp rx=0 tx=0).stop 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- 172.21.15.86:0/2508606842 shutdown_connections 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da296e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2508606842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da76250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- 172.21.15.86:0/2508606842 >> 172.21.15.86:0/2508606842 conn(0x55ac8da56c10 msgr2=0x55ac8da59000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- 172.21.15.86:0/2508606842 shutdown_connections 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- 172.21.15.86:0/2508606842 wait complete. 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 -- start start 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.104+0000 7fe578eb6000 1 Processor -- start 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da2ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da2f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d834ab0 con 0x55ac8da76790 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d876ac0 con 0x55ac8da29c20 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ac8d8baa60 con 0x55ac8da75e70 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 10 client.0 _ll_register_callbacks cb 0x55ac8da17380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da2f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe566000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da2ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:47476/0 (socket says 172.21.15.86:47476) 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56ea00640 1 -- 172.21.15.86:0/3433534661 learned_addr learned my addr 172.21.15.86:0/3433534661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56f400640 1 -- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 msgr2=0x55ac8da2ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56f400640 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da2ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56f400640 1 -- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 msgr2=0x55ac8da2e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56f400640 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56f400640 1 -- 172.21.15.86:0/3433534661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8d850570 con 0x55ac8da76790 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56f400640 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da2f290 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fe5680012f0 tx=0x7fe568012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be0bdef502a3d0b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe553e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe553e00640 1 -- 172.21.15.86:0/3433534661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568019070 con 0x55ac8da76790 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 -- 172.21.15.86:0/3433534661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe558012ce0 con 0x55ac8da76790 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe553e00640 1 -- 172.21.15.86:0/3433534661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fe568015440 con 0x55ac8da76790 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe553e00640 1 -- 172.21.15.86:0/3433534661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe568024050 con 0x55ac8da76790 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 10 client.8454 fetch_fsmap learned FSMap version 84 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 -- 172.21.15.86:0/3433534661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55ac8d846880 con 0x55ac8da76790 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe553e00640 1 -- 172.21.15.86:0/3433534661 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe558012ce0 con 0x55ac8da76790 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe553e00640 1 -- 172.21.15.86:0/3433534661 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7fe568027020 con 0x55ac8da76790 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 10 client.8454 fetch_fsmap finished waiting for FSMap version 84 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 10 client.8454 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 20 client.8454 populate_metadata read hostname 'smithi086' 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55ac8da2fca0 0x55ac8da87120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 4 client.8454 mds_command: new command op to 77162 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:19:24.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe578eb6000 1 -- 172.21.15.86:0/3433534661 --> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac8d815bc0 con 0x55ac8da2fca0 2024-03-22T04:19:24.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56ea00640 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55ac8da2fca0 0x55ac8da87120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe553e00640 10 client.8454 ms_handle_connect on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.108+0000 7fe56ea00640 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55ac8da2fca0 0x55ac8da87120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efd9136569b90c1 server_cookie=d52b1b142d54bc93 in_seq=0 out_seq=0 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe553e00640 1 -- 172.21.15.86:0/3433534661 <== mds.0 v2:172.21.15.196:6833/2259098413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ac8d815bc0 con 0x55ac8da2fca0 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe553e00640 10 client.8454 handle_command_reply: tid=0 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 client.8454 shutdown 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 -- 172.21.15.86:0/3433534661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 msgr2=0x55ac8da2f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da2f290 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fe5680012f0 tx=0x7fe568012690 comp rx=0 tx=0).stop 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 -- 172.21.15.86:0/3433534661 shutdown_connections 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55ac8da2fca0 0x55ac8da87120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da76790 0x55ac8da2f290 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da75e70 0x55ac8da2ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3433534661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da2e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe553e00640 0 client.8454 ms_handle_reset on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 -- 172.21.15.86:0/3433534661 >> 172.21.15.86:0/3433534661 conn(0x55ac8da56c10 msgr2=0x55ac8da59000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 -- 172.21.15.86:0/3433534661 shutdown_connections 2024-03-22T04:19:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 1 -- 172.21.15.86:0/3433534661 wait complete. 2024-03-22T04:19:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.112+0000 7fe578eb6000 20 client.8454 trim_cache size 0 max 16384 2024-03-22T04:19:24.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe578eb6000 1 Processor -- start 2024-03-22T04:19:24.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe578eb6000 1 -- start start 2024-03-22T04:19:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da29c20 0x55ac8da74f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da75e70 0x55ac8da75470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 0x55ac8da87120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe578eb6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe570204550 con 0x55ac8da29c20 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe578eb6000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5702035f0 con 0x55ac8da75e70 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe578eb6000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5701564f0 con 0x55ac8da76790 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe56ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da29c20 0x55ac8da74f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe56ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da29c20 0x55ac8da74f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40096/0 (socket says 172.21.15.86:40096) 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.128+0000 7fe56ea00640 1 -- 172.21.15.86:0/3228702875 learned_addr learned my addr 172.21.15.86:0/3228702875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da75e70 0x55ac8da75470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56f400640 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 0x55ac8da87120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56ea00640 1 -- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 msgr2=0x55ac8da87120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56ea00640 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 0x55ac8da87120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56ea00640 1 -- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da75e70 msgr2=0x55ac8da75470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56ea00640 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da75e70 0x55ac8da75470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56ea00640 1 -- 172.21.15.86:0/3228702875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8d802e80 con 0x55ac8da29c20 2024-03-22T04:19:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da75e70 0x55ac8da75470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:24.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56ea00640 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da29c20 0x55ac8da74f30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe560008010 tx=0x7fe560009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2759b253462eb0a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe55ea00640 1 -- 172.21.15.86:0/3228702875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600b1040 con 0x55ac8da29c20 2024-03-22T04:19:24.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe55ea00640 1 -- 172.21.15.86:0/3228702875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5600d3070 con 0x55ac8da29c20 2024-03-22T04:19:24.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe55ea00640 1 -- 172.21.15.86:0/3228702875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600acca0 con 0x55ac8da29c20 2024-03-22T04:19:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- 172.21.15.86:0/3228702875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da29c20 msgr2=0x55ac8da74f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da29c20 0x55ac8da74f30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe560008010 tx=0x7fe560009190 comp rx=0 tx=0).stop 2024-03-22T04:19:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- 172.21.15.86:0/3228702875 shutdown_connections 2024-03-22T04:19:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da75e70 0x55ac8da75470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 0x55ac8da87120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 --2- 172.21.15.86:0/3228702875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da29c20 0x55ac8da74f30 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- 172.21.15.86:0/3228702875 >> 172.21.15.86:0/3228702875 conn(0x55ac8da56c10 msgr2=0x55ac8da28d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- 172.21.15.86:0/3228702875 shutdown_connections 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- 172.21.15.86:0/3228702875 wait complete. 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- start start 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 Processor -- start 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da8a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da75e70 0x55ac8daa9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 0x55ac8daa97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d879e20 con 0x55ac8da75e70 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe570156b40 con 0x55ac8da29c20 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55ac8da32110 con 0x55ac8da76790 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe578eb6000 10 client.0 _ll_register_callbacks cb 0x7fe568001e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da75e70 0x55ac8daa9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da75e70 0x55ac8daa9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40106/0 (socket says 172.21.15.86:40106) 2024-03-22T04:19:24.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 -- 172.21.15.86:0/2295963299 learned_addr learned my addr 172.21.15.86:0/2295963299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56ea00640 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da8a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe56f400640 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 0x55ac8daa97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 -- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 msgr2=0x55ac8daa97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 0x55ac8daa97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 -- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 msgr2=0x55ac8da8a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da8a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.132+0000 7fe566000640 1 -- 172.21.15.86:0/2295963299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8d80e750 con 0x55ac8da75e70 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe566000640 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da75e70 0x55ac8daa9280 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe558052280 tx=0x7fe55801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a19a73d2c00750 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 1 -- 172.21.15.86:0/2295963299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5580520c0 con 0x55ac8da75e70 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 1 -- 172.21.15.86:0/2295963299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fe558052490 con 0x55ac8da75e70 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 -- 172.21.15.86:0/2295963299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5600d3070 con 0x55ac8da75e70 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 1 -- 172.21.15.86:0/2295963299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe558063050 con 0x55ac8da75e70 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 10 client.8460 fetch_fsmap learned FSMap version 84 2024-03-22T04:19:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 -- 172.21.15.86:0/2295963299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55ac8d821450 con 0x55ac8da75e70 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 1 -- 172.21.15.86:0/2295963299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5600d3070 con 0x55ac8da75e70 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 1 -- 172.21.15.86:0/2295963299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7fe55807d440 con 0x55ac8da75e70 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 10 client.8460 fetch_fsmap finished waiting for FSMap version 84 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 10 client.8460 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 20 client.8460 populate_metadata read hostname 'smithi086' 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55ac8daed860 0x55ac8daedc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 4 client.8460 mds_command: new command op to 77162 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 -- 172.21.15.86:0/2295963299 --> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] -- command(tid 0: {"prefix": "status"}) -- 0x55ac8d888050 con 0x55ac8daed860 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe56ea00640 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55ac8daed860 0x55ac8daedc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe56ea00640 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55ac8daed860 0x55ac8daedc40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1e90ad90e1fa77 server_cookie=780bd26a401915ab in_seq=0 out_seq=0 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 10 client.8460 ms_handle_connect on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 1 -- 172.21.15.86:0/2295963299 <== mds.0 v2:172.21.15.196:6833/2259098413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55ac8d888050 con 0x55ac8daed860 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 10 client.8460 handle_command_reply: tid=0 2024-03-22T04:19:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 client.8460 shutdown 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 -- 172.21.15.86:0/2295963299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da75e70 msgr2=0x55ac8daa9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da75e70 0x55ac8daa9280 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe558052280 tx=0x7fe55801d040 comp rx=0 tx=0).stop 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 -- 172.21.15.86:0/2295963299 shutdown_connections 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55ac8daed860 0x55ac8daedc40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ac8da76790 0x55ac8daa97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8da75e70 0x55ac8daa9280 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 --2- 172.21.15.86:0/2295963299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac8da29c20 0x55ac8da8a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe553e00640 0 client.8460 ms_handle_reset on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 -- 172.21.15.86:0/2295963299 >> 172.21.15.86:0/2295963299 conn(0x55ac8da56c10 msgr2=0x55ac8da73f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 -- 172.21.15.86:0/2295963299 shutdown_connections 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 1 -- 172.21.15.86:0/2295963299 wait complete. 2024-03-22T04:19:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe578eb6000 20 client.8460 trim_cache size 0 max 16384 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 77162, 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 5.8182739330000004, 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 84, 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 109, 2024-03-22T04:19:24.142 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 109, 2024-03-22T04:19:24.143 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 11.696114959999999 2024-03-22T04:19:24.143 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:19:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 -- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe54c067b20 msgr2=0x7fe54c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe54c067b20 0x7fe54c069fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe5600ae000 tx=0x7fe5600a7000 comp rx=0 tx=0).stop 2024-03-22T04:19:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 -- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5700ad330 msgr2=0x7fe5700b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5700ad330 0x7fe5700b4190 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c500 tx=0x7fe55800c9c0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 -- 172.21.15.86:0/3795406390 shutdown_connections 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe54c067b20 0x7fe54c069fe0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5700ad710 0x7fe570099170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5700ad330 0x7fe5700b4190 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 --2- 172.21.15.86:0/3795406390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe57006ef40 0x7fe570164870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 -- 172.21.15.86:0/3795406390 >> 172.21.15.86:0/3795406390 conn(0x7fe5700b1df0 msgr2=0x7fe5700cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 -- 172.21.15.86:0/3795406390 shutdown_connections 2024-03-22T04:19:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.136+0000 7fe575a00640 1 -- 172.21.15.86:0/3795406390 wait complete. 2024-03-22T04:19:24.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:19:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff118200640 1 Processor -- start 2024-03-22T04:19:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff118200640 1 -- start start 2024-03-22T04:19:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff118200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff11015c600 0x7ff11006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff118200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1100ce980 con 0x7ff1100b0680 2024-03-22T04:19:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff118200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11006ff00 con 0x7ff110160c10 2024-03-22T04:19:24.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff118200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff110161110 con 0x7ff11015c600 2024-03-22T04:19:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff115800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff11015c600 0x7ff11006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff115800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff11015c600 0x7ff11006b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40112/0 (socket says 172.21.15.86:40112) 2024-03-22T04:19:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff114e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff1100b0680 0x7ff1100cdb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44400/0 (socket says 172.21.15.86:44400) 2024-03-22T04:19:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff115800640 1 -- 172.21.15.86:0/3758439918 learned_addr learned my addr 172.21.15.86:0/3758439918 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff115800640 1 -- 172.21.15.86:0/3758439918 >> v1:172.21.15.196:6789/0 conn(0x7ff110160c10 legacy=0x7ff1100ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff115800640 1 -- 172.21.15.86:0/3758439918 >> v1:172.21.15.86:6789/0 conn(0x7ff1100b0680 legacy=0x7ff1100cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff115800640 1 -- 172.21.15.86:0/3758439918 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100706c0 con 0x7ff11015c600 2024-03-22T04:19:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.292+0000 7ff10fe00640 1 -- 172.21.15.86:0/3758439918 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4159904720 0 0) 0x7ff1100ce980 con 0x7ff1100b0680 2024-03-22T04:19:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff115800640 1 --2- 172.21.15.86:0/3758439918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff11015c600 0x7ff11006b660 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff104009870 tx=0x7ff10401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e811872e0d551c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff10fe00640 1 -- 172.21.15.86:0/3758439918 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040b1070 con 0x7ff11015c600 2024-03-22T04:19:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff10fe00640 1 -- 172.21.15.86:0/3758439918 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1040ac620 con 0x7ff11015c600 2024-03-22T04:19:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff10fe00640 1 -- 172.21.15.86:0/3758439918 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040ac9f0 con 0x7ff11015c600 2024-03-22T04:19:24.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff118200640 1 -- 172.21.15.86:0/3758439918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff11015c600 msgr2=0x7ff11006b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff118200640 1 --2- 172.21.15.86:0/3758439918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff11015c600 0x7ff11006b660 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff104009870 tx=0x7ff10401c4d0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff118200640 1 -- 172.21.15.86:0/3758439918 shutdown_connections 2024-03-22T04:19:24.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff118200640 1 --2- 172.21.15.86:0/3758439918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff11015c600 0x7ff11006b660 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff118200640 1 -- 172.21.15.86:0/3758439918 >> 172.21.15.86:0/3758439918 conn(0x7ff1100b2d00 msgr2=0x7ff1100b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:24.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff118200640 1 -- 172.21.15.86:0/3758439918 shutdown_connections 2024-03-22T04:19:24.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff118200640 1 -- 172.21.15.86:0/3758439918 wait complete. 2024-03-22T04:19:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.296+0000 7ff118200640 1 Processor -- start 2024-03-22T04:19:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 -- start start 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1100b0680 0x7ff1100c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015c600 0x7ff1100b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff110160c10 0x7ff1100b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100aeff0 con 0x7ff110160c10 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff11006bfb0 con 0x7ff11015c600 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff11015c9e0 con 0x7ff1100b0680 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff116200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff110160c10 0x7ff1100b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1100b0680 0x7ff1100c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1100b0680 0x7ff1100c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:39002/0 (socket says 172.21.15.86:39002) 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 -- 172.21.15.86:0/1888509409 learned_addr learned my addr 172.21.15.86:0/1888509409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:24.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff114e00640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015c600 0x7ff1100b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 -- 172.21.15.86:0/1888509409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015c600 msgr2=0x7ff1100b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015c600 0x7ff1100b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 -- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff110160c10 msgr2=0x7ff1100b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff110160c10 0x7ff1100b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 -- 172.21.15.86:0/1888509409 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100cfc50 con 0x7ff1100b0680 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff116200640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff110160c10 0x7ff1100b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff115800640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1100b0680 0x7ff1100c7430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff1040af040 tx=0x7ff10401ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff10d600640 1 -- 172.21.15.86:0/1888509409 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040bb020 con 0x7ff1100b0680 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff10d600640 1 -- 172.21.15.86:0/1888509409 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1040b7640 con 0x7ff1100b0680 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff10d600640 1 -- 172.21.15.86:0/1888509409 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040b1040 con 0x7ff1100b0680 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11015d7f0 con 0x7ff1100b0680 2024-03-22T04:19:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11015ddb0 con 0x7ff1100b0680 2024-03-22T04:19:24.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.300+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff110068650 con 0x7ff1100b0680 2024-03-22T04:19:24.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.304+0000 7ff10d600640 1 -- 172.21.15.86:0/1888509409 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff1040be020 con 0x7ff1100b0680 2024-03-22T04:19:24.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.304+0000 7ff10d600640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff0f0067b20 0x7ff0f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:24.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.304+0000 7ff10d600640 1 -- 172.21.15.86:0/1888509409 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7ff1041203e0 con 0x7ff1100b0680 2024-03-22T04:19:24.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.304+0000 7ff114e00640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff0f0067b20 0x7ff0f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:24.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.304+0000 7ff114e00640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff0f0067b20 0x7ff0f0069fe0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff11009ec80 tx=0x7ff0fc007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.308+0000 7ff10d600640 1 -- 172.21.15.86:0/1888509409 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff1040c5030 con 0x7ff1100b0680 2024-03-22T04:19:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.500+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff110072a50 con 0x7ff1100b0680 2024-03-22T04:19:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.500+0000 7ff10d600640 1 -- 172.21.15.86:0/1888509409 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7ff104016150 con 0x7ff1100b0680 2024-03-22T04:19:24.502 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff0f0067b20 msgr2=0x7ff0f0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff0f0067b20 0x7ff0f0069fe0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff11009ec80 tx=0x7ff0fc007240 comp rx=0 tx=0).stop 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1100b0680 msgr2=0x7ff1100c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1100b0680 0x7ff1100c7430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff1040af040 tx=0x7ff10401ad10 comp rx=0 tx=0).stop 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 shutdown_connections 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff0f0067b20 0x7ff0f0069fe0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015c600 0x7ff1100b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff110160c10 0x7ff1100b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 --2- 172.21.15.86:0/1888509409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1100b0680 0x7ff1100c7430 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 >> 172.21.15.86:0/1888509409 conn(0x7ff1100b2d00 msgr2=0x7ff110165d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 shutdown_connections 2024-03-22T04:19:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24.504+0000 7ff118200640 1 -- 172.21.15.86:0/1888509409 wait complete. 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:19:24.523 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:19:24.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:24.523 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:19:24.533 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:19:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:19:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:19:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:19:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:24.533 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41967sec preferred_lft 41967sec 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:19:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:24.535 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:19:24.536 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:19:24.536 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:19:24.536 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:24.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:24.536 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:19:24.591 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:19:24.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:24.593 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:19:24.645 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:19:24.646 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:19:24.646 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:19:24.647 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:19:24.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:19:24.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:19:24.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:19:24.647 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:19:24.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:19:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:19:24.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:24.779 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:19:24.788 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:19:24.788 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:19:24.788 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:19:24.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:24.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:19:24.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:19:24.850 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:24.898 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:19:24.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:19:24.899 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:24.953 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:19:24.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:24.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:19:25.042 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:19:25.042 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:19:25.042 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:19:25.058 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:19:25.058 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:19:25.058 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:19:25.058 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:19:25.058 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:19:25.065 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:19:25.065 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:19:25.066 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:19:25.066 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:19:25.066 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:19:25.066 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:19:25.066 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:19:25.066 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:19:25.069 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:19:25.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:25.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:25.090 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:19:25.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:19:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:19:25.202 INFO:teuthology.orchestra.run.smithi086.stdout:client8287 2024-03-22T04:19:25.202 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:25.202 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:25.211 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:25.211 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:25.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:25.263 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-22T04:19:25.429 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records in 2024-03-22T04:19:25.429 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records out 2024-03-22T04:19:25.429 INFO:teuthology.orchestra.run.smithi086.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.109191 s, 57.6 MB/s 2024-03-22T04:19:25.430 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:25.430 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:25.439 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:25.439 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:25.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd714c00640 1 Processor -- start 2024-03-22T04:19:25.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd714c00640 1 -- start start 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd714c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7100d0de0 0x7fd7100ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd714c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd7100d19a0 0x7fd7100cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd714c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7100d11c0 con 0x7fd71015fed0 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd714c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd7100cece0 con 0x7fd7100d0de0 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd714c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd7100cefc0 con 0x7fd7100d19a0 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd70de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7100d0de0 0x7fd7100ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd70de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7100d0de0 0x7fd7100ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40132/0 (socket says 172.21.15.86:40132) 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd70de00640 1 -- 172.21.15.86:0/2743871195 learned_addr learned my addr 172.21.15.86:0/2743871195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd705400640 1 --2- 172.21.15.86:0/2743871195 >> v2:172.21.15.196:3300/0 conn(0x7fd7100d19a0 0x7fd7100cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd70de00640 1 -- 172.21.15.86:0/2743871195 >> v2:172.21.15.196:3300/0 conn(0x7fd7100d19a0 msgr2=0x7fd7100cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd70de00640 1 --2- 172.21.15.86:0/2743871195 >> v2:172.21.15.196:3300/0 conn(0x7fd7100d19a0 0x7fd7100cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd70de00640 1 -- 172.21.15.86:0/2743871195 >> v1:172.21.15.196:6789/0 conn(0x7fd71015fed0 legacy=0x7fd7101602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.668+0000 7fd70de00640 1 -- 172.21.15.86:0/2743871195 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7100cf2a0 con 0x7fd7100d0de0 2024-03-22T04:19:25.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd70de00640 1 --2- 172.21.15.86:0/2743871195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d0de0 0x7fd7100ce6b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd7100d32d0 tx=0x7fd70001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2713d2e4f0a6d094 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd70d400640 1 -- 172.21.15.86:0/2743871195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000a57a0 con 0x7fd7100d0de0 2024-03-22T04:19:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd70d400640 1 -- 172.21.15.86:0/2743871195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7000a5940 con 0x7fd7100d0de0 2024-03-22T04:19:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd70d400640 1 -- 172.21.15.86:0/2743871195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000a5d10 con 0x7fd7100d0de0 2024-03-22T04:19:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd714c00640 1 -- 172.21.15.86:0/2743871195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d0de0 msgr2=0x7fd7100ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd714c00640 1 --2- 172.21.15.86:0/2743871195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d0de0 0x7fd7100ce6b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd7100d32d0 tx=0x7fd70001c9f0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd714c00640 1 -- 172.21.15.86:0/2743871195 shutdown_connections 2024-03-22T04:19:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd714c00640 1 --2- 172.21.15.86:0/2743871195 >> v2:172.21.15.196:3300/0 conn(0x7fd7100d19a0 0x7fd7100cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd714c00640 1 --2- 172.21.15.86:0/2743871195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d0de0 0x7fd7100ce6b0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd714c00640 1 -- 172.21.15.86:0/2743871195 >> 172.21.15.86:0/2743871195 conn(0x7fd71006f910 msgr2=0x7fd71006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd714c00640 1 -- 172.21.15.86:0/2743871195 shutdown_connections 2024-03-22T04:19:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.672+0000 7fd714c00640 1 -- 172.21.15.86:0/2743871195 wait complete. 2024-03-22T04:19:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 Processor -- start 2024-03-22T04:19:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 -- start start 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7100d0de0 0x7fd710155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d19a0 0x7fd710150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71015fed0 0x7fd7101653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100d1f30 con 0x7fd7100d19a0 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100d1db0 con 0x7fd7100d0de0 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd710071e30 con 0x7fd71015fed0 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d19a0 0x7fd710150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d19a0 0x7fd710150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40134/0 (socket says 172.21.15.86:40134) 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 -- 172.21.15.86:0/103051951 learned_addr learned my addr 172.21.15.86:0/103051951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd70e800640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71015fed0 0x7fd7101653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd70de00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7100d0de0 0x7fd710155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 -- 172.21.15.86:0/103051951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71015fed0 msgr2=0x7fd7101653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71015fed0 0x7fd7101653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 -- 172.21.15.86:0/103051951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7100d0de0 msgr2=0x7fd710155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7100d0de0 0x7fd710155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 -- 172.21.15.86:0/103051951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd710070440 con 0x7fd7100d19a0 2024-03-22T04:19:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd705400640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d19a0 0x7fd710150810 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800d800 tx=0x7fd6f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd706a00640 1 -- 172.21.15.86:0/103051951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800b6c0 con 0x7fd7100d19a0 2024-03-22T04:19:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd706a00640 1 -- 172.21.15.86:0/103051951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f8019070 con 0x7fd7100d19a0 2024-03-22T04:19:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd706a00640 1 -- 172.21.15.86:0/103051951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800ba60 con 0x7fd7100d19a0 2024-03-22T04:19:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd710151ee0 con 0x7fd7100d19a0 2024-03-22T04:19:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd710155210 con 0x7fd7100d19a0 2024-03-22T04:19:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.676+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd710154f00 con 0x7fd7100d19a0 2024-03-22T04:19:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.680+0000 7fd706a00640 1 -- 172.21.15.86:0/103051951 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd6f800bc00 con 0x7fd7100d19a0 2024-03-22T04:19:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.680+0000 7fd706a00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6ec067b20 0x7fd6ec069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.680+0000 7fd706a00640 1 -- 172.21.15.86:0/103051951 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fd6f8086460 con 0x7fd7100d19a0 2024-03-22T04:19:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.680+0000 7fd70de00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6ec067b20 0x7fd6ec069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.680+0000 7fd70de00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6ec067b20 0x7fd6ec069fe0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd7000ae000 tx=0x7fd7000a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.684+0000 7fd706a00640 1 -- 172.21.15.86:0/103051951 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd6f80565f0 con 0x7fd7100d19a0 2024-03-22T04:19:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.836+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd710157310 con 0x7fd7100d19a0 2024-03-22T04:19:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd706a00640 1 -- 172.21.15.86:0/103051951 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16900 (secure 0 0 0) 0x7fd6f805a690 con 0x7fd7100d19a0 2024-03-22T04:19:25.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:25.840 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":109,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:19:19.318781+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:19:15.365871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-22T04:19:16.647234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6ec067b20 msgr2=0x7fd6ec069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6ec067b20 0x7fd6ec069fe0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd7000ae000 tx=0x7fd7000a7070 comp rx=0 tx=0).stop 2024-03-22T04:19:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d19a0 msgr2=0x7fd710150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d19a0 0x7fd710150810 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800d800 tx=0x7fd6f800dcc0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 shutdown_connections 2024-03-22T04:19:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd6ec067b20 0x7fd6ec069fe0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd71015fed0 0x7fd7101653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7100d19a0 0x7fd710150810 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 --2- 172.21.15.86:0/103051951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7100d0de0 0x7fd710155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 >> 172.21.15.86:0/103051951 conn(0x7fd71006f910 msgr2=0x7fd71015a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 shutdown_connections 2024-03-22T04:19:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:25.840+0000 7fd714c00640 1 -- 172.21.15.86:0/103051951 wait complete. 2024-03-22T04:19:25.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:19:15.365871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:19:16.647234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:19:25.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:19:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.000+0000 7f19fe000640 1 Processor -- start 2024-03-22T04:19:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.000+0000 7f19fe000640 1 -- start start 2024-03-22T04:19:26.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.000+0000 7f19fe000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19f80b2360 0x7f19f80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.000+0000 7f19fe000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f815d710 con 0x7f19f80b3430 2024-03-22T04:19:26.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.000+0000 7f19fe000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f815fe40 con 0x7f19f815d0b0 2024-03-22T04:19:26.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.000+0000 7f19fe000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f19f80b2740 con 0x7f19f80b2360 2024-03-22T04:19:26.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19f80b2360 0x7f19f80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f19f80b3430 0x7f19f815c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33684/0 (socket says 172.21.15.86:33684) 2024-03-22T04:19:26.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19f80b2360 0x7f19f80b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52098/0 (socket says 172.21.15.86:52098) 2024-03-22T04:19:26.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f6a00640 1 -- 172.21.15.86:0/562963696 learned_addr learned my addr 172.21.15.86:0/562963696 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:26.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f7400640 1 -- 172.21.15.86:0/562963696 >> v1:172.21.15.196:6789/0 conn(0x7f19f815d0b0 legacy=0x7f19f80b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f7400640 1 -- 172.21.15.86:0/562963696 >> v1:172.21.15.86:6789/0 conn(0x7f19f80b3430 legacy=0x7f19f815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f7400640 1 -- 172.21.15.86:0/562963696 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f8160120 con 0x7f19f80b2360 2024-03-22T04:19:26.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f6000640 1 -- 172.21.15.86:0/562963696 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1196148952 0 0) 0x7f19f815d710 con 0x7f19f80b3430 2024-03-22T04:19:26.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f7400640 1 --2- 172.21.15.86:0/562963696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b2360 0x7f19f80b2ef0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097c0 tx=0x7f19e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d732a899a0eeb619 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f6000640 1 -- 172.21.15.86:0/562963696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00b0070 con 0x7f19f80b2360 2024-03-22T04:19:26.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f6000640 1 -- 172.21.15.86:0/562963696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e00ab600 con 0x7f19f80b2360 2024-03-22T04:19:26.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.004+0000 7f19f6000640 1 -- 172.21.15.86:0/562963696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00ab9d0 con 0x7f19f80b2360 2024-03-22T04:19:26.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- 172.21.15.86:0/562963696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b2360 msgr2=0x7f19f80b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 --2- 172.21.15.86:0/562963696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b2360 0x7f19f80b2ef0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097c0 tx=0x7f19e001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- 172.21.15.86:0/562963696 shutdown_connections 2024-03-22T04:19:26.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 --2- 172.21.15.86:0/562963696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b2360 0x7f19f80b2ef0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- 172.21.15.86:0/562963696 >> 172.21.15.86:0/562963696 conn(0x7f19f80cc940 msgr2=0x7f19f80ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:26.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- 172.21.15.86:0/562963696 shutdown_connections 2024-03-22T04:19:26.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- 172.21.15.86:0/562963696 wait complete. 2024-03-22T04:19:26.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 Processor -- start 2024-03-22T04:19:26.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- start start 2024-03-22T04:19:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19f80b2360 0x7f19f8151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b3430 0x7f19f81533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19f815d0b0 0x7f19f81538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19f816a370 con 0x7f19f80b3430 2024-03-22T04:19:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19f816a1f0 con 0x7f19f80b2360 2024-03-22T04:19:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19fe000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f19f816a4f0 con 0x7f19f815d0b0 2024-03-22T04:19:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b3430 0x7f19f81533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19f80b2360 0x7f19f8151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b3430 0x7f19f81533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52100/0 (socket says 172.21.15.86:52100) 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 -- 172.21.15.86:0/2886504758 learned_addr learned my addr 172.21.15.86:0/2886504758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f7e00640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19f815d0b0 0x7f19f81538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 -- 172.21.15.86:0/2886504758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19f815d0b0 msgr2=0x7f19f81538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19f815d0b0 0x7f19f81538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 -- 172.21.15.86:0/2886504758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19f80b2360 msgr2=0x7f19f8151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19f80b2360 0x7f19f8151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 -- 172.21.15.86:0/2886504758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f8156a30 con 0x7f19f80b3430 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f7400640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19f80b2360 0x7f19f8151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.008+0000 7f19f6a00640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b3430 0x7f19f81533b0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c4a0 tx=0x7f19e800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19ef400640 1 -- 172.21.15.86:0/2886504758 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8018070 con 0x7f19f80b3430 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19ef400640 1 -- 172.21.15.86:0/2886504758 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e80074e0 con 0x7f19f80b3430 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f8155b30 con 0x7f19f80b3430 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19ef400640 1 -- 172.21.15.86:0/2886504758 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8004a30 con 0x7f19f80b3430 2024-03-22T04:19:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f80a94d0 con 0x7f19f80b3430 2024-03-22T04:19:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f8074470 con 0x7f19f80b3430 2024-03-22T04:19:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19ef400640 1 -- 172.21.15.86:0/2886504758 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f19e801b070 con 0x7f19f80b3430 2024-03-22T04:19:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19ef400640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19d4067b20 0x7f19d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19ef400640 1 -- 172.21.15.86:0/2886504758 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f19e80853b0 con 0x7f19f80b3430 2024-03-22T04:19:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.012+0000 7f19f7400640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19d4067b20 0x7f19d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.016+0000 7f19f7400640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19d4067b20 0x7f19d4069fe0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f19e00ad000 tx=0x7f19e00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.016+0000 7f19ef400640 1 -- 172.21.15.86:0/2886504758 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f19e8055540 con 0x7f19f80b3430 2024-03-22T04:19:26.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.172+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19f80cc6d0 con 0x7f19f80b3430 2024-03-22T04:19:26.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.172+0000 7f19ef400640 1 -- 172.21.15.86:0/2886504758 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f19e80595e0 con 0x7f19f80b3430 2024-03-22T04:19:26.173 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19d4067b20 msgr2=0x7f19d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19d4067b20 0x7f19d4069fe0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f19e00ad000 tx=0x7f19e00a6000 comp rx=0 tx=0).stop 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b3430 msgr2=0x7f19f81533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b3430 0x7f19f81533b0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c4a0 tx=0x7f19e800c960 comp rx=0 tx=0).stop 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 shutdown_connections 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19d4067b20 0x7f19d4069fe0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19f815d0b0 0x7f19f81538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19f80b3430 0x7f19f81533b0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 --2- 172.21.15.86:0/2886504758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19f80b2360 0x7f19f8151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 >> 172.21.15.86:0/2886504758 conn(0x7f19f80cc940 msgr2=0x7f19f8152220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:26.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 shutdown_connections 2024-03-22T04:19:26.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.176+0000 7f19fe000640 1 -- 172.21.15.86:0/2886504758 wait complete. 2024-03-22T04:19:26.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:19:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4404400640 1 Processor -- start 2024-03-22T04:19:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4404400640 1 -- start start 2024-03-22T04:19:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4404400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43fc15e330 0x7f43fc0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4404400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc0737d0 0x7f43fc0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4404400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43fc0b5ad0 con 0x7f43fc15e710 2024-03-22T04:19:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4404400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f43fc0b4370 con 0x7f43fc15e330 2024-03-22T04:19:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4404400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f43fc072a10 con 0x7f43fc0737d0 2024-03-22T04:19:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4401a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43fc15e330 0x7f43fc0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4401a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43fc15e330 0x7f43fc0b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52102/0 (socket says 172.21.15.86:52102) 2024-03-22T04:19:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4401a00640 1 -- 172.21.15.86:0/2164335263 learned_addr learned my addr 172.21.15.86:0/2164335263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4401000640 1 --1- 172.21.15.86:0/2164335263 >> v1:172.21.15.86:6789/0 conn(0x7f43fc15e710 0x7f43fc0b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33696/0 (socket says 172.21.15.86:33696) 2024-03-22T04:19:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4402400640 1 --2- 172.21.15.86:0/2164335263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc0737d0 0x7f43fc0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4401a00640 1 -- 172.21.15.86:0/2164335263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc0737d0 msgr2=0x7f43fc0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4401a00640 1 --2- 172.21.15.86:0/2164335263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc0737d0 0x7f43fc0b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4401a00640 1 -- 172.21.15.86:0/2164335263 >> v1:172.21.15.86:6789/0 conn(0x7f43fc15e710 legacy=0x7f43fc0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4401a00640 1 -- 172.21.15.86:0/2164335263 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc072cf0 con 0x7f43fc15e330 2024-03-22T04:19:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.336+0000 7f4402400640 1 --2- 172.21.15.86:0/2164335263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc0737d0 0x7f43fc0b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4401a00640 1 --2- 172.21.15.86:0/2164335263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc0b4f60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009870 tx=0x7f43f001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f87ec482b49115 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f43fbe00640 1 -- 172.21.15.86:0/2164335263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f00a95f0 con 0x7f43fc15e330 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f43fbe00640 1 -- 172.21.15.86:0/2164335263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f00a9790 con 0x7f43fc15e330 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f43fbe00640 1 -- 172.21.15.86:0/2164335263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f00a9b60 con 0x7f43fc15e330 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4404400640 1 -- 172.21.15.86:0/2164335263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 msgr2=0x7f43fc0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4404400640 1 --2- 172.21.15.86:0/2164335263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc0b4f60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009870 tx=0x7f43f001c530 comp rx=0 tx=0).stop 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4404400640 1 -- 172.21.15.86:0/2164335263 shutdown_connections 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4404400640 1 --2- 172.21.15.86:0/2164335263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc0737d0 0x7f43fc0b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4404400640 1 --2- 172.21.15.86:0/2164335263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc0b4f60 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4404400640 1 -- 172.21.15.86:0/2164335263 >> 172.21.15.86:0/2164335263 conn(0x7f43fc0d0fe0 msgr2=0x7f43fc0d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4404400640 1 -- 172.21.15.86:0/2164335263 shutdown_connections 2024-03-22T04:19:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.340+0000 7f4404400640 1 -- 172.21.15.86:0/2164335263 wait complete. 2024-03-22T04:19:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 Processor -- start 2024-03-22T04:19:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 -- start start 2024-03-22T04:19:26.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc0737d0 0x7f43fc1fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc15e710 0x7f43fc15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc06efd0 con 0x7f43fc15e330 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc06ee50 con 0x7f43fc0737d0 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f43fc06f150 con 0x7f43fc15e710 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc15cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52108/0 (socket says 172.21.15.86:52108) 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 -- 172.21.15.86:0/961964717 learned_addr learned my addr 172.21.15.86:0/961964717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401a00640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc0737d0 0x7f43fc1fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4402400640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc15e710 0x7f43fc15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 -- 172.21.15.86:0/961964717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc15e710 msgr2=0x7f43fc15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc15e710 0x7f43fc15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 -- 172.21.15.86:0/961964717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc0737d0 msgr2=0x7f43fc1fa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc0737d0 0x7f43fc1fa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 -- 172.21.15.86:0/961964717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc155900 con 0x7f43fc15e330 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4401000640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc15cd70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c450 tx=0x7f43e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f43f9600640 1 -- 172.21.15.86:0/961964717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8018070 con 0x7f43fc15e330 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f43f9600640 1 -- 172.21.15.86:0/961964717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e8007480 con 0x7f43fc15e330 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f43f9600640 1 -- 172.21.15.86:0/961964717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e80047f0 con 0x7f43fc15e330 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc155be0 con 0x7f43fc15e330 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc1561a0 con 0x7f43fc15e330 2024-03-22T04:19:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.344+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43fc074470 con 0x7f43fc15e330 2024-03-22T04:19:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.348+0000 7f43f9600640 1 -- 172.21.15.86:0/961964717 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f43e8004990 con 0x7f43fc15e330 2024-03-22T04:19:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.348+0000 7f43f9600640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43dc067b50 0x7f43dc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.348+0000 7f43f9600640 1 -- 172.21.15.86:0/961964717 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f43e8085120 con 0x7f43fc15e330 2024-03-22T04:19:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.348+0000 7f4401a00640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43dc067b50 0x7f43dc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.348+0000 7f4401a00640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43dc067b50 0x7f43dc06a010 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f43f00ae000 tx=0x7f43f00b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.352+0000 7f43f9600640 1 -- 172.21.15.86:0/961964717 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f43e8055380 con 0x7f43fc15e330 2024-03-22T04:19:26.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.508+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f43fc06c4d0 con 0x7f43fc15e330 2024-03-22T04:19:26.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.508+0000 7f43f9600640 1 -- 172.21.15.86:0/961964717 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f43e8059420 con 0x7f43fc15e330 2024-03-22T04:19:26.509 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:19:26.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43dc067b50 msgr2=0x7f43dc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43dc067b50 0x7f43dc06a010 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f43f00ae000 tx=0x7f43f00b3040 comp rx=0 tx=0).stop 2024-03-22T04:19:26.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 msgr2=0x7f43fc15cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc15cd70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c450 tx=0x7f43e800c910 comp rx=0 tx=0).stop 2024-03-22T04:19:26.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 shutdown_connections 2024-03-22T04:19:26.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f43dc067b50 0x7f43dc06a010 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43fc15e710 0x7f43fc15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43fc15e330 0x7f43fc15cd70 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 --2- 172.21.15.86:0/961964717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43fc0737d0 0x7f43fc1fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 >> 172.21.15.86:0/961964717 conn(0x7f43fc0d0fe0 msgr2=0x7f43fc0b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:26.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 shutdown_connections 2024-03-22T04:19:26.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.512+0000 7f4404400640 1 -- 172.21.15.86:0/961964717 wait complete. 2024-03-22T04:19:26.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:19:26.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb91c200640 1 Processor -- start 2024-03-22T04:19:26.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb91c200640 1 -- start start 2024-03-22T04:19:26.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb91c200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9140b4eb0 0x7fb914162740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb91c200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91406b660 0x7fb91406baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb91c200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9140b5470 con 0x7fb91415e5b0 2024-03-22T04:19:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb91c200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb9141622b0 con 0x7fb9140b4eb0 2024-03-22T04:19:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb91c200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb9140b5750 con 0x7fb91406b660 2024-03-22T04:19:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb918e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9140b4eb0 0x7fb914162740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb919800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb91415e5b0 0x7fb9140b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:47524/0 (socket says 172.21.15.86:47524) 2024-03-22T04:19:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb918e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9140b4eb0 0x7fb914162740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52114/0 (socket says 172.21.15.86:52114) 2024-03-22T04:19:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb919800640 1 -- 172.21.15.86:0/3613561301 learned_addr learned my addr 172.21.15.86:0/3613561301 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb91a200640 1 --2- 172.21.15.86:0/3613561301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91406b660 0x7fb91406baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb913e00640 1 -- 172.21.15.86:0/3613561301 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228533647 0 0) 0x7fb9140b5470 con 0x7fb91415e5b0 2024-03-22T04:19:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb913e00640 1 -- 172.21.15.86:0/3613561301 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8fc003530 con 0x7fb91415e5b0 2024-03-22T04:19:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb918e00640 1 -- 172.21.15.86:0/3613561301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91406b660 msgr2=0x7fb91406baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb918e00640 1 --2- 172.21.15.86:0/3613561301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91406b660 0x7fb91406baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb918e00640 1 -- 172.21.15.86:0/3613561301 >> v1:172.21.15.196:6789/0 conn(0x7fb91415e5b0 legacy=0x7fb9140b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.668+0000 7fb918e00640 1 -- 172.21.15.86:0/3613561301 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9140b5a30 con 0x7fb9140b4eb0 2024-03-22T04:19:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb918e00640 1 --2- 172.21.15.86:0/3613561301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9140b4eb0 0x7fb914162740 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb9000097c0 tx=0x7fb90001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ce9eb2a0efed7c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb913e00640 1 -- 172.21.15.86:0/3613561301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9000b0070 con 0x7fb9140b4eb0 2024-03-22T04:19:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb913e00640 1 -- 172.21.15.86:0/3613561301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9000ab600 con 0x7fb9140b4eb0 2024-03-22T04:19:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb913e00640 1 -- 172.21.15.86:0/3613561301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9000ab9b0 con 0x7fb9140b4eb0 2024-03-22T04:19:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb91c200640 1 -- 172.21.15.86:0/3613561301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9140b4eb0 msgr2=0x7fb914162740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb91c200640 1 --2- 172.21.15.86:0/3613561301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9140b4eb0 0x7fb914162740 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb9000097c0 tx=0x7fb90001c410 comp rx=0 tx=0).stop 2024-03-22T04:19:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb91c200640 1 -- 172.21.15.86:0/3613561301 shutdown_connections 2024-03-22T04:19:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb91c200640 1 --2- 172.21.15.86:0/3613561301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91406b660 0x7fb91406baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb91c200640 1 --2- 172.21.15.86:0/3613561301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9140b4eb0 0x7fb914162740 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb91c200640 1 -- 172.21.15.86:0/3613561301 >> 172.21.15.86:0/3613561301 conn(0x7fb9140d0fa0 msgr2=0x7fb9140d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb91c200640 1 -- 172.21.15.86:0/3613561301 shutdown_connections 2024-03-22T04:19:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.672+0000 7fb91c200640 1 -- 172.21.15.86:0/3613561301 wait complete. 2024-03-22T04:19:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 Processor -- start 2024-03-22T04:19:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 -- start start 2024-03-22T04:19:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91406b660 0x7fb914149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9140b4eb0 0x7fb9140ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91415e5b0 0x7fb9140aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb91416b2e0 con 0x7fb91406b660 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb914072f00 con 0x7fb9140b4eb0 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb91416b460 con 0x7fb91415e5b0 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91406b660 0x7fb914149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91406b660 0x7fb914149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52122/0 (socket says 172.21.15.86:52122) 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 -- 172.21.15.86:0/1722174299 learned_addr learned my addr 172.21.15.86:0/1722174299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb918e00640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9140b4eb0 0x7fb9140ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91a200640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91415e5b0 0x7fb9140aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 -- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91415e5b0 msgr2=0x7fb9140aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91415e5b0 0x7fb9140aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 -- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9140b4eb0 msgr2=0x7fb9140ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9140b4eb0 0x7fb9140ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 -- 172.21.15.86:0/1722174299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91414cb10 con 0x7fb91406b660 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb918e00640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9140b4eb0 0x7fb9140ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb919800640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91406b660 0x7fb914149e00 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c3f0 tx=0x7fb90800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb911600640 1 -- 172.21.15.86:0/1722174299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908017070 con 0x7fb91406b660 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb911600640 1 -- 172.21.15.86:0/1722174299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb908005cf0 con 0x7fb91406b660 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91414cdf0 con 0x7fb91406b660 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb911600640 1 -- 172.21.15.86:0/1722174299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080072f0 con 0x7fb91406b660 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.676+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91414d3b0 con 0x7fb91406b660 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.680+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb914074470 con 0x7fb91406b660 2024-03-22T04:19:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.680+0000 7fb911600640 1 -- 172.21.15.86:0/1722174299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb90801a070 con 0x7fb91406b660 2024-03-22T04:19:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.680+0000 7fb911600640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8f4067b20 0x7fb8f4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.680+0000 7fb911600640 1 -- 172.21.15.86:0/1722174299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fb9080854a0 con 0x7fb91406b660 2024-03-22T04:19:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.680+0000 7fb918e00640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8f4067b20 0x7fb8f4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.680+0000 7fb918e00640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8f4067b20 0x7fb8f4069fe0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9000ad000 tx=0x7fb900008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:26.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.684+0000 7fb911600640 1 -- 172.21.15.86:0/1722174299 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb9080556d0 con 0x7fb91406b660 2024-03-22T04:19:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.836+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb91406c4a0 con 0x7fb91406b660 2024-03-22T04:19:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb911600640 1 -- 172.21.15.86:0/1722174299 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7fb908059770 con 0x7fb91406b660 2024-03-22T04:19:26.839 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8f4067b20 msgr2=0x7fb8f4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8f4067b20 0x7fb8f4069fe0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9000ad000 tx=0x7fb900008040 comp rx=0 tx=0).stop 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91406b660 msgr2=0x7fb914149e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91406b660 0x7fb914149e00 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c3f0 tx=0x7fb90800c8b0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 shutdown_connections 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb8f4067b20 0x7fb8f4069fe0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9140b4eb0 0x7fb9140ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb91415e5b0 0x7fb9140aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 --2- 172.21.15.86:0/1722174299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91406b660 0x7fb914149e00 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 >> 172.21.15.86:0/1722174299 conn(0x7fb9140d0fa0 msgr2=0x7fb91406ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 shutdown_connections 2024-03-22T04:19:26.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.840+0000 7fb91c200640 1 -- 172.21.15.86:0/1722174299 wait complete. 2024-03-22T04:19:26.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:19:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc898a00640 1 Processor -- start 2024-03-22T04:19:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc898a00640 1 -- start start 2024-03-22T04:19:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc898a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8940d0580 0x7fc894071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc898a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8941623d0 con 0x7fc89406e2f0 2024-03-22T04:19:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc898a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8941620f0 con 0x7fc89415e630 2024-03-22T04:19:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc898a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc89415ec10 con 0x7fc8940d0580 2024-03-22T04:19:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc891400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8940d0580 0x7fc894071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc891e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc89415e630 0x7fc8940cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:47536/0 (socket says 172.21.15.86:47536) 2024-03-22T04:19:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc891e00640 1 -- 172.21.15.86:0/1313792135 learned_addr learned my addr 172.21.15.86:0/1313792135 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc892800640 1 --1- 172.21.15.86:0/1313792135 >> v1:172.21.15.86:6789/0 conn(0x7fc89406e2f0 0x7fc894071640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33712/0 (socket says 172.21.15.86:33712) 2024-03-22T04:19:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc891400640 1 -- 172.21.15.86:0/1313792135 >> v1:172.21.15.196:6789/0 conn(0x7fc89415e630 legacy=0x7fc8940cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc891400640 1 -- 172.21.15.86:0/1313792135 >> v1:172.21.15.86:6789/0 conn(0x7fc89406e2f0 legacy=0x7fc894071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:26.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc891400640 1 -- 172.21.15.86:0/1313792135 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89415eef0 con 0x7fc8940d0580 2024-03-22T04:19:26.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:26.996+0000 7fc890a00640 1 -- 172.21.15.86:0/1313792135 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243382206 0 0) 0x7fc8941623d0 con 0x7fc89406e2f0 2024-03-22T04:19:27.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc891400640 1 --2- 172.21.15.86:0/1313792135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940d0580 0x7fc894071100 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc8800097c0 tx=0x7fc88001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbf3d247591ba37 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:27.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc890a00640 1 -- 172.21.15.86:0/1313792135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8800b0070 con 0x7fc8940d0580 2024-03-22T04:19:27.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc890a00640 1 -- 172.21.15.86:0/1313792135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8800ab600 con 0x7fc8940d0580 2024-03-22T04:19:27.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc890a00640 1 -- 172.21.15.86:0/1313792135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8800ab9d0 con 0x7fc8940d0580 2024-03-22T04:19:27.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc898a00640 1 -- 172.21.15.86:0/1313792135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940d0580 msgr2=0x7fc894071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc898a00640 1 --2- 172.21.15.86:0/1313792135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940d0580 0x7fc894071100 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc8800097c0 tx=0x7fc88001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc898a00640 1 -- 172.21.15.86:0/1313792135 shutdown_connections 2024-03-22T04:19:27.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc898a00640 1 --2- 172.21.15.86:0/1313792135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940d0580 0x7fc894071100 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc898a00640 1 -- 172.21.15.86:0/1313792135 >> 172.21.15.86:0/1313792135 conn(0x7fc89406f910 msgr2=0x7fc89406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:27.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc898a00640 1 -- 172.21.15.86:0/1313792135 shutdown_connections 2024-03-22T04:19:27.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.000+0000 7fc898a00640 1 -- 172.21.15.86:0/1313792135 wait complete. 2024-03-22T04:19:27.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 Processor -- start 2024-03-22T04:19:27.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 -- start start 2024-03-22T04:19:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc89406e2f0 0x7fc8940ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8940d0580 0x7fc8941563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89415e630 0x7fc8940cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8940cfaf0 con 0x7fc89415e630 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc89415f9d0 con 0x7fc89406e2f0 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8940cfc70 con 0x7fc8940d0580 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89415e630 0x7fc8940cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89415e630 0x7fc8940cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52134/0 (socket says 172.21.15.86:52134) 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 -- 172.21.15.86:0/1642864598 learned_addr learned my addr 172.21.15.86:0/1642864598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc891e00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc89406e2f0 0x7fc8940ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc891400640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8940d0580 0x7fc8941563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 -- 172.21.15.86:0/1642864598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8940d0580 msgr2=0x7fc8941563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8940d0580 0x7fc8941563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 -- 172.21.15.86:0/1642864598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc89406e2f0 msgr2=0x7fc8940ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc89406e2f0 0x7fc8940ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 -- 172.21.15.86:0/1642864598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8940ca8d0 con 0x7fc89415e630 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc892800640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89415e630 0x7fc8940cb970 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00c410 tx=0x7fc88c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc88a000640 1 -- 172.21.15.86:0/1642864598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0074e0 con 0x7fc89415e630 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc88a000640 1 -- 172.21.15.86:0/1642864598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88c0047f0 con 0x7fc89415e630 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8940cabb0 con 0x7fc89415e630 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc88a000640 1 -- 172.21.15.86:0/1642864598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c007680 con 0x7fc89415e630 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8940cb170 con 0x7fc89415e630 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.004+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc894068650 con 0x7fc89415e630 2024-03-22T04:19:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.008+0000 7fc88a000640 1 -- 172.21.15.86:0/1642864598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc88c011070 con 0x7fc89415e630 2024-03-22T04:19:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.008+0000 7fc88a000640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc874067b20 0x7fc874069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.008+0000 7fc88a000640 1 -- 172.21.15.86:0/1642864598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fc88c085ba0 con 0x7fc89415e630 2024-03-22T04:19:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.008+0000 7fc891e00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc874067b20 0x7fc874069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.008+0000 7fc891e00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc874067b20 0x7fc874069fe0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc884005c10 tx=0x7fc884009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:27.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.012+0000 7fc88a000640 1 -- 172.21.15.86:0/1642864598 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc88c055d30 con 0x7fc89415e630 2024-03-22T04:19:27.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.176+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc8940cecb0 con 0x7fc89415e630 2024-03-22T04:19:27.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.176+0000 7fc88a000640 1 -- 172.21.15.86:0/1642864598 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7fc88c059dd0 con 0x7fc89415e630 2024-03-22T04:19:27.179 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:19:27.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc874067b20 msgr2=0x7fc874069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc874067b20 0x7fc874069fe0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc884005c10 tx=0x7fc884009040 comp rx=0 tx=0).stop 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89415e630 msgr2=0x7fc8940cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89415e630 0x7fc8940cb970 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00c410 tx=0x7fc88c00c8d0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 shutdown_connections 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc874067b20 0x7fc874069fe0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89415e630 0x7fc8940cb970 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8940d0580 0x7fc8941563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 --2- 172.21.15.86:0/1642864598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc89406e2f0 0x7fc8940ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 >> 172.21.15.86:0/1642864598 conn(0x7fc89406f910 msgr2=0x7fc8940b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 shutdown_connections 2024-03-22T04:19:27.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.180+0000 7fc898a00640 1 -- 172.21.15.86:0/1642864598 wait complete. 2024-03-22T04:19:27.199 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:19:27.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:27.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:27.229 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:19:27.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:27.229 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:19:27.283 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.460+0000 7fb469000640 1 Processor -- start 2024-03-22T04:19:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.460+0000 7fb469000640 1 -- start start 2024-03-22T04:19:27.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb469000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4640ae540 0x7fb4640b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb469000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb464071920 con 0x7fb4640730d0 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb469000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb464071c00 con 0x7fb46415b3f0 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb469000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb4640716d0 con 0x7fb4640ae540 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb462800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4640ae540 0x7fb4640b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb461400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb4640730d0 0x7fb4640ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33720/0 (socket says 172.21.15.86:33720) 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb462800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4640ae540 0x7fb4640b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52138/0 (socket says 172.21.15.86:52138) 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb461400640 1 -- 172.21.15.86:0/3076081936 learned_addr learned my addr 172.21.15.86:0/3076081936 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb461e00640 1 --1- 172.21.15.86:0/3076081936 >> v1:172.21.15.196:6789/0 conn(0x7fb46415b3f0 0x7fb4640729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:47538/0 (socket says 172.21.15.86:47538) 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb462800640 1 -- 172.21.15.86:0/3076081936 >> v1:172.21.15.196:6789/0 conn(0x7fb46415b3f0 legacy=0x7fb4640729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb462800640 1 -- 172.21.15.86:0/3076081936 >> v1:172.21.15.86:6789/0 conn(0x7fb4640730d0 legacy=0x7fb4640ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.464+0000 7fb462800640 1 -- 172.21.15.86:0/3076081936 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb464071ee0 con 0x7fb4640ae540 2024-03-22T04:19:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb462800640 1 --2- 172.21.15.86:0/3076081936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4640ae540 0x7fb4640b4300 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00ba60 tx=0x7fb45c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b4f697fe81da97 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb460a00640 1 -- 172.21.15.86:0/3076081936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0b4070 con 0x7fb4640ae540 2024-03-22T04:19:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb460a00640 1 -- 172.21.15.86:0/3076081936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c0a7840 con 0x7fb4640ae540 2024-03-22T04:19:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb460a00640 1 -- 172.21.15.86:0/3076081936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0a7c10 con 0x7fb4640ae540 2024-03-22T04:19:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- 172.21.15.86:0/3076081936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4640ae540 msgr2=0x7fb4640b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 --2- 172.21.15.86:0/3076081936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4640ae540 0x7fb4640b4300 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00ba60 tx=0x7fb45c01eb70 comp rx=0 tx=0).stop 2024-03-22T04:19:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- 172.21.15.86:0/3076081936 shutdown_connections 2024-03-22T04:19:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 --2- 172.21.15.86:0/3076081936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4640ae540 0x7fb4640b4300 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- 172.21.15.86:0/3076081936 >> 172.21.15.86:0/3076081936 conn(0x7fb4640b1df0 msgr2=0x7fb4640b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- 172.21.15.86:0/3076081936 shutdown_connections 2024-03-22T04:19:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- 172.21.15.86:0/3076081936 wait complete. 2024-03-22T04:19:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 Processor -- start 2024-03-22T04:19:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- start start 2024-03-22T04:19:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4640730d0 0x7fb464094800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4640ae540 0x7fb464148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb46415b3f0 0x7fb464149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4640b4c50 con 0x7fb46415b3f0 2024-03-22T04:19:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4640b4ad0 con 0x7fb4640ae540 2024-03-22T04:19:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.468+0000 7fb469000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb46406e7a0 con 0x7fb4640730d0 2024-03-22T04:19:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb46415b3f0 0x7fb464149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb46415b3f0 0x7fb464149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52150/0 (socket says 172.21.15.86:52150) 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 -- 172.21.15.86:0/2801948630 learned_addr learned my addr 172.21.15.86:0/2801948630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb461e00640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4640730d0 0x7fb464094800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb461400640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4640ae540 0x7fb464148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 -- 172.21.15.86:0/2801948630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4640730d0 msgr2=0x7fb464094800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4640730d0 0x7fb464094800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 -- 172.21.15.86:0/2801948630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4640ae540 msgr2=0x7fb464148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4640ae540 0x7fb464148940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 -- 172.21.15.86:0/2801948630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46414c510 con 0x7fb46415b3f0 2024-03-22T04:19:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb462800640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb46415b3f0 0x7fb464149450 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0b2040 tx=0x7fb45c01b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb45a000640 1 -- 172.21.15.86:0/2801948630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0bf070 con 0x7fb46415b3f0 2024-03-22T04:19:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb45a000640 1 -- 172.21.15.86:0/2801948630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c0b5040 con 0x7fb46415b3f0 2024-03-22T04:19:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb45a000640 1 -- 172.21.15.86:0/2801948630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0b4070 con 0x7fb46415b3f0 2024-03-22T04:19:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb464154890 con 0x7fb46415b3f0 2024-03-22T04:19:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb464148e80 con 0x7fb46415b3f0 2024-03-22T04:19:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb464068650 con 0x7fb46415b3f0 2024-03-22T04:19:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb45a000640 1 -- 172.21.15.86:0/2801948630 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb45c0b9400 con 0x7fb46415b3f0 2024-03-22T04:19:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb45a000640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb444067b20 0x7fb444069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.472+0000 7fb45a000640 1 -- 172.21.15.86:0/2801948630 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fb45c121fc0 con 0x7fb46415b3f0 2024-03-22T04:19:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.476+0000 7fb461e00640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb444067b20 0x7fb444069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.476+0000 7fb461e00640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb444067b20 0x7fb444069fe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb4540012f0 tx=0x7fb454001660 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.476+0000 7fb45a000640 1 -- 172.21.15.86:0/2801948630 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb45c018760 con 0x7fb46415b3f0 2024-03-22T04:19:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.664+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb46415d000 con 0x7fb46415b3f0 2024-03-22T04:19:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.664+0000 7fb45a000640 1 -- 172.21.15.86:0/2801948630 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5192 (secure 0 0 0) 0x7fb45c0afe20 con 0x7fb46415b3f0 2024-03-22T04:19:27.666 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:27.666 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":77159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/269054026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":269054026},{"type":"v1","addr":"172.21.15.196:6834","nonce":269054026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:19:20.698020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77162":{"gid":77162,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2259098413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2259098413},{"type":"v1","addr":"172.21.15.196:6835","nonce":2259098413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77162,"qdb_cluster":[77162]},"id":7}]} 2024-03-22T04:19:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 84 2024-03-22T04:19:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb444067b20 msgr2=0x7fb444069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb444067b20 0x7fb444069fe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb4540012f0 tx=0x7fb454001660 comp rx=0 tx=0).stop 2024-03-22T04:19:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb46415b3f0 msgr2=0x7fb464149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb46415b3f0 0x7fb464149450 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0b2040 tx=0x7fb45c01b910 comp rx=0 tx=0).stop 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 shutdown_connections 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb444067b20 0x7fb444069fe0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4640ae540 0x7fb464148940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb46415b3f0 0x7fb464149450 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 --2- 172.21.15.86:0/2801948630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4640730d0 0x7fb464094800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 >> 172.21.15.86:0/2801948630 conn(0x7fb4640b1df0 msgr2=0x7fb46415e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 shutdown_connections 2024-03-22T04:19:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.668+0000 7fb469000640 1 -- 172.21.15.86:0/2801948630 wait complete. 2024-03-22T04:19:27.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-22T04:19:27.851 INFO:teuthology.orchestra.run.smithi196.stdout:{"message":"","return_code":0} 2024-03-22T04:19:27.852 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:19:27.852 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.988+0000 7f7501800640 1 Processor -- start 2024-03-22T04:19:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.988+0000 7f7501800640 1 -- start start 2024-03-22T04:19:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.988+0000 7f7501800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc147e10 0x7f74fc0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.988+0000 7f7501800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f74fc06c270 0x7f74fc162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.988+0000 7f7501800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74fc0b2bd0 con 0x7f74fc162540 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.988+0000 7f7501800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f74fc0b2610 con 0x7f74fc06c270 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.988+0000 7f7501800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f74fc0b28f0 con 0x7f74fc147e10 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74fb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f74fc162540 0x7f74fc15e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33734/0 (socket says 172.21.15.86:33734) 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74fb400640 1 -- 172.21.15.86:0/1079131426 learned_addr learned my addr 172.21.15.86:0/1079131426 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74f2000640 1 --2- 172.21.15.86:0/1079131426 >> v2:172.21.15.196:3300/0 conn(0x7f74fc06c270 0x7f74fc162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74faa00640 1 --2- 172.21.15.86:0/1079131426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc147e10 0x7f74fc0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74fa000640 1 -- 172.21.15.86:0/1079131426 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 57598782 0 0) 0x7f74fc0b2bd0 con 0x7f74fc162540 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74fa000640 1 -- 172.21.15.86:0/1079131426 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74e0003530 con 0x7f74fc162540 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74faa00640 1 -- 172.21.15.86:0/1079131426 >> v2:172.21.15.196:3300/0 conn(0x7f74fc06c270 msgr2=0x7f74fc162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74faa00640 1 --2- 172.21.15.86:0/1079131426 >> v2:172.21.15.196:3300/0 conn(0x7f74fc06c270 0x7f74fc162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74faa00640 1 -- 172.21.15.86:0/1079131426 >> v1:172.21.15.86:6789/0 conn(0x7f74fc162540 legacy=0x7f74fc15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74faa00640 1 -- 172.21.15.86:0/1079131426 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc0b5ce0 con 0x7f74fc147e10 2024-03-22T04:19:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74f2000640 1 --2- 172.21.15.86:0/1079131426 >> v2:172.21.15.196:3300/0 conn(0x7f74fc06c270 0x7f74fc162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:27.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74faa00640 1 --2- 172.21.15.86:0/1079131426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc147e10 0x7f74fc0b4100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0097c0 tx=0x7f74ec01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338ec144f42ae521 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.992+0000 7f74fa000640 1 -- 172.21.15.86:0/1079131426 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0b1070 con 0x7f74fc147e10 2024-03-22T04:19:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74fa000640 1 -- 172.21.15.86:0/1079131426 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ec0ac410 con 0x7f74fc147e10 2024-03-22T04:19:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74fa000640 1 -- 172.21.15.86:0/1079131426 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0ac7e0 con 0x7f74fc147e10 2024-03-22T04:19:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- 172.21.15.86:0/1079131426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc147e10 msgr2=0x7f74fc0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 --2- 172.21.15.86:0/1079131426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc147e10 0x7f74fc0b4100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0097c0 tx=0x7f74ec01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- 172.21.15.86:0/1079131426 shutdown_connections 2024-03-22T04:19:27.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 --2- 172.21.15.86:0/1079131426 >> v2:172.21.15.196:3300/0 conn(0x7f74fc06c270 0x7f74fc162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 --2- 172.21.15.86:0/1079131426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc147e10 0x7f74fc0b4100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:27.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- 172.21.15.86:0/1079131426 >> 172.21.15.86:0/1079131426 conn(0x7f74fc0b6620 msgr2=0x7f74fc0b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:27.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- 172.21.15.86:0/1079131426 shutdown_connections 2024-03-22T04:19:27.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- 172.21.15.86:0/1079131426 wait complete. 2024-03-22T04:19:27.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 Processor -- start 2024-03-22T04:19:27.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- start start 2024-03-22T04:19:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74fc06c270 0x7f74fc09c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74fc147e10 0x7f74fc09b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc162540 0x7f74fc09b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc162950 con 0x7f74fc06c270 2024-03-22T04:19:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc160d40 con 0x7f74fc147e10 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f7501800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f74fc16b150 con 0x7f74fc162540 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74fc06c270 0x7f74fc09c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74fc06c270 0x7f74fc09c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52164/0 (socket says 172.21.15.86:52164) 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74faa00640 1 -- 172.21.15.86:0/2791479041 learned_addr learned my addr 172.21.15.86:0/2791479041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74f2000640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74fc147e10 0x7f74fc09b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74fb400640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc162540 0x7f74fc09b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74faa00640 1 -- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc162540 msgr2=0x7f74fc09b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74faa00640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc162540 0x7f74fc09b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74faa00640 1 -- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74fc147e10 msgr2=0x7f74fc09b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74faa00640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74fc147e10 0x7f74fc09b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74faa00640 1 -- 172.21.15.86:0/2791479041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc09cea0 con 0x7f74fc06c270 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:27.996+0000 7f74fb400640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc162540 0x7f74fc09b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f74faa00640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74fc06c270 0x7f74fc09c530 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f74ec01a0d0 tx=0x7f74ec019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f74f3400640 1 -- 172.21.15.86:0/2791479041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0b9070 con 0x7f74fc06c270 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f74f3400640 1 -- 172.21.15.86:0/2791479041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ec0accc0 con 0x7f74fc06c270 2024-03-22T04:19:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74fc09d180 con 0x7f74fc06c270 2024-03-22T04:19:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f74f3400640 1 -- 172.21.15.86:0/2791479041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0b1040 con 0x7f74fc06c270 2024-03-22T04:19:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74fc09d740 con 0x7f74fc06c270 2024-03-22T04:19:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74fc068650 con 0x7f74fc06c270 2024-03-22T04:19:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f74f3400640 1 -- 172.21.15.86:0/2791479041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f74ec0bc070 con 0x7f74fc06c270 2024-03-22T04:19:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f74f3400640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f74d8067b20 0x7f74d8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f74f3400640 1 -- 172.21.15.86:0/2791479041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f74ec11f990 con 0x7f74fc06c270 2024-03-22T04:19:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.000+0000 7f74f2000640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f74d8067b20 0x7f74d8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.004+0000 7f74f2000640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f74d8067b20 0x7f74d8069fe0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009bd0 tx=0x7f74e400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.004+0000 7f74f3400640 1 -- 172.21.15.86:0/2791479041 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f74ec016090 con 0x7f74fc06c270 2024-03-22T04:19:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74fc15ed70 con 0x7f74fc06c270 2024-03-22T04:19:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f74f3400640 1 -- 172.21.15.86:0/2791479041 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5192 (secure 0 0 0) 0x7f74ec0efbb0 con 0x7f74fc06c270 2024-03-22T04:19:28.204 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:28.204 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":77159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/269054026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":269054026},{"type":"v1","addr":"172.21.15.196:6834","nonce":269054026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:19:20.698020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77162":{"gid":77162,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2259098413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2259098413},{"type":"v1","addr":"172.21.15.196:6835","nonce":2259098413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77162,"qdb_cluster":[77162]},"id":7}]} 2024-03-22T04:19:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 84 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f74d8067b20 msgr2=0x7f74d8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f74d8067b20 0x7f74d8069fe0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009bd0 tx=0x7f74e400b040 comp rx=0 tx=0).stop 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74fc06c270 msgr2=0x7f74fc09c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74fc06c270 0x7f74fc09c530 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f74ec01a0d0 tx=0x7f74ec019240 comp rx=0 tx=0).stop 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 shutdown_connections 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f74d8067b20 0x7f74d8069fe0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74fc147e10 0x7f74fc09b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74fc162540 0x7f74fc09b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 --2- 172.21.15.86:0/2791479041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74fc06c270 0x7f74fc09c530 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 >> 172.21.15.86:0/2791479041 conn(0x7f74fc0b6620 msgr2=0x7f74fc06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 shutdown_connections 2024-03-22T04:19:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.204+0000 7f7501800640 1 -- 172.21.15.86:0/2791479041 wait complete. 2024-03-22T04:19:28.223 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-03-22T04:19:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 Processor -- start 2024-03-22T04:19:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 -- start start 2024-03-22T04:19:28.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 0x5593ff147140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff109760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 0x5593ff14bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593ff136250 con 0x5593ff1da460 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5593ff0ab620 con 0x5593ff1da840 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5593ff1364f0 con 0x5593ff135cd0 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff109760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff109760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52172/0 (socket says 172.21.15.86:52172) 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 -- 172.21.15.86:0/3257650939 learned_addr learned my addr 172.21.15.86:0/3257650939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa772200640 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 0x5593ff147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa772c00640 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 0x5593ff14bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 -- 172.21.15.86:0/3257650939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 msgr2=0x5593ff147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 0x5593ff147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 -- 172.21.15.86:0/3257650939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 msgr2=0x5593ff14bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 0x5593ff14bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 -- 172.21.15.86:0/3257650939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ff1dd2a0 con 0x5593ff1da460 2024-03-22T04:19:28.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa771800640 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff109760 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa760009890 tx=0x7fa760005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadbf5f5d2756c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa76be00640 1 -- 172.21.15.86:0/3257650939 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760019070 con 0x5593ff1da460 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa76be00640 1 -- 172.21.15.86:0/3257650939 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa760008da0 con 0x5593ff1da460 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa76be00640 1 -- 172.21.15.86:0/3257650939 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600146c0 con 0x5593ff1da460 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.268+0000 7fa7750da080 1 -- 172.21.15.86:0/3257650939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 msgr2=0x5593ff109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff109760 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa760009890 tx=0x7fa760005ad0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- 172.21.15.86:0/3257650939 shutdown_connections 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 0x5593ff14bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff109760 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 --2- 172.21.15.86:0/3257650939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 0x5593ff147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- 172.21.15.86:0/3257650939 >> 172.21.15.86:0/3257650939 conn(0x5593ff131ec0 msgr2=0x5593ff108db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- 172.21.15.86:0/3257650939 shutdown_connections 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- 172.21.15.86:0/3257650939 wait complete. 2024-03-22T04:19:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 Processor -- start 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- start start 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 0x5593ff108d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 0x5593ff1d6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593ff146ca0 con 0x5593ff1da460 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5593ff1dd540 con 0x5593ff1da840 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5593ff150d60 con 0x5593ff135cd0 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff107fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52188/0 (socket says 172.21.15.86:52188) 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 -- 172.21.15.86:0/2303842544 learned_addr learned my addr 172.21.15.86:0/2303842544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa772c00640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 0x5593ff1d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa772200640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 0x5593ff108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 -- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 msgr2=0x5593ff108d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5593ff135cd0 0x5593ff108d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 -- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 msgr2=0x5593ff1d6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5593ff1da840 0x5593ff1d6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 -- 172.21.15.86:0/2303842544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ff1d7760 con 0x5593ff1da460 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa771800640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff107fd0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa760014900 tx=0x7fa760003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa76aa00640 1 -- 172.21.15.86:0/2303842544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760024020 con 0x5593ff1da460 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa76aa00640 1 -- 172.21.15.86:0/2303842544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76001a040 con 0x5593ff1da460 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5593ff1d7a00 con 0x5593ff1da460 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa76aa00640 1 -- 172.21.15.86:0/2303842544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760019070 con 0x5593ff1da460 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.272+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ff1d60b0 con 0x5593ff1da460 2024-03-22T04:19:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa76aa00640 1 -- 172.21.15.86:0/2303842544 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa760027020 con 0x5593ff1da460 2024-03-22T04:19:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa76aa00640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa754066530 0x7fa7540689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa76aa00640 1 -- 172.21.15.86:0/2303842544 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fa7600868f0 con 0x5593ff1da460 2024-03-22T04:19:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa772200640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa754066530 0x7fa7540689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5593ff108510 0x5593ff1d3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x5593ff275cb0 con 0x5593ff108510 2024-03-22T04:19:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa772c00640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5593ff108510 0x5593ff1d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa772200640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa754066530 0x7fa7540689f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x5593ff1502c0 tx=0x7fa7640a5000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa772c00640 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5593ff108510 0x5593ff1d3560 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa772c00640 1 -- 172.21.15.86:0/2303842544 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7fa76c006030 con 0x5593ff108510 2024-03-22T04:19:28.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5593ff108510 msgr2=0x5593ff1d3560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5593ff108510 0x5593ff1d3560 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa754066530 msgr2=0x7fa7540689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa754066530 0x7fa7540689f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x5593ff1502c0 tx=0x7fa7640a5000 comp rx=0 tx=0).stop 2024-03-22T04:19:28.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 msgr2=0x5593ff107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 --2- 172.21.15.86:0/2303842544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ff1da460 0x5593ff107fd0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa760014900 tx=0x7fa760003030 comp rx=0 tx=0).stop 2024-03-22T04:19:28.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa772c00640 1 -- 172.21.15.86:0/2303842544 reap_dead start 2024-03-22T04:19:28.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 shutdown_connections 2024-03-22T04:19:28.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 >> 172.21.15.86:0/2303842544 conn(0x5593ff131ec0 msgr2=0x5593ff1d8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:28.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 shutdown_connections 2024-03-22T04:19:28.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.276+0000 7fa7750da080 1 -- 172.21.15.86:0/2303842544 wait complete. 2024-03-22T04:19:28.283 INFO:teuthology.orchestra.run.smithi086.stdout:����� 2024-03-22T04:19:28.283 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:19:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18ce000640 1 Processor -- start 2024-03-22T04:19:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18ce000640 1 -- start start 2024-03-22T04:19:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18ce000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f18c815e000 0x7f18c8165960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18ce000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c815c550 con 0x7f18c806df90 2024-03-22T04:19:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18ce000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c80b4750 con 0x7f18c815ac10 2024-03-22T04:19:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18ce000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f18c815cb60 con 0x7f18c815e000 2024-03-22T04:19:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18c7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f18c815e000 0x7f18c8165960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18c7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f18c806df90 0x7f18c806e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33750/0 (socket says 172.21.15.86:33750) 2024-03-22T04:19:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18c7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f18c815e000 0x7f18c8165960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52192/0 (socket says 172.21.15.86:52192) 2024-03-22T04:19:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18c7400640 1 -- 172.21.15.86:0/240103256 learned_addr learned my addr 172.21.15.86:0/240103256 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:28.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18c7e00640 1 -- 172.21.15.86:0/240103256 >> v1:172.21.15.196:6789/0 conn(0x7f18c815ac10 legacy=0x7f18c80b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18c7e00640 1 -- 172.21.15.86:0/240103256 >> v1:172.21.15.86:6789/0 conn(0x7f18c806df90 legacy=0x7f18c806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.464+0000 7f18c7e00640 1 -- 172.21.15.86:0/240103256 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c80b4470 con 0x7f18c815e000 2024-03-22T04:19:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18c7e00640 1 --2- 172.21.15.86:0/240103256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815e000 0x7f18c8165960 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009d90 tx=0x7f18c001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3ec68e07c166ee server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18c6a00640 1 -- 172.21.15.86:0/240103256 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c00b2070 con 0x7f18c815e000 2024-03-22T04:19:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18c6a00640 1 -- 172.21.15.86:0/240103256 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c00a5910 con 0x7f18c815e000 2024-03-22T04:19:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18c6a00640 1 -- 172.21.15.86:0/240103256 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c00a5ce0 con 0x7f18c815e000 2024-03-22T04:19:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18ce000640 1 -- 172.21.15.86:0/240103256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815e000 msgr2=0x7f18c8165960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18ce000640 1 --2- 172.21.15.86:0/240103256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815e000 0x7f18c8165960 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009d90 tx=0x7f18c001c940 comp rx=0 tx=0).stop 2024-03-22T04:19:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18ce000640 1 -- 172.21.15.86:0/240103256 shutdown_connections 2024-03-22T04:19:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18ce000640 1 --2- 172.21.15.86:0/240103256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815e000 0x7f18c8165960 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18ce000640 1 -- 172.21.15.86:0/240103256 >> 172.21.15.86:0/240103256 conn(0x7f18c80b1df0 msgr2=0x7f18c80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18ce000640 1 -- 172.21.15.86:0/240103256 shutdown_connections 2024-03-22T04:19:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.468+0000 7f18ce000640 1 -- 172.21.15.86:0/240103256 wait complete. 2024-03-22T04:19:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 Processor -- start 2024-03-22T04:19:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 -- start start 2024-03-22T04:19:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c806df90 0x7f18c81545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815ac10 0x7f18c80ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c815e000 0x7f18c8150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18c806da90 con 0x7f18c815ac10 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18c806d910 con 0x7f18c815e000 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f18c806dc10 con 0x7f18c806df90 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815ac10 0x7f18c80ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815ac10 0x7f18c80ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52208/0 (socket says 172.21.15.86:52208) 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 -- 172.21.15.86:0/2763999065 learned_addr learned my addr 172.21.15.86:0/2763999065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18c7e00640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c815e000 0x7f18c8150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18c7400640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c806df90 0x7f18c81545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 -- 172.21.15.86:0/2763999065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c806df90 msgr2=0x7f18c81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c806df90 0x7f18c81545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 -- 172.21.15.86:0/2763999065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c815e000 msgr2=0x7f18c8150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c815e000 0x7f18c8150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 -- 172.21.15.86:0/2763999065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c80adea0 con 0x7f18c815ac10 2024-03-22T04:19:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bea00640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815ac10 0x7f18c80ad4b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f18b000c410 tx=0x7f18b000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bfe00640 1 -- 172.21.15.86:0/2763999065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0017070 con 0x7f18c815ac10 2024-03-22T04:19:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bfe00640 1 -- 172.21.15.86:0/2763999065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b00052d0 con 0x7f18c815ac10 2024-03-22T04:19:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18bfe00640 1 -- 172.21.15.86:0/2763999065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b00054c0 con 0x7f18c815ac10 2024-03-22T04:19:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c80ae160 con 0x7f18c815ac10 2024-03-22T04:19:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.472+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c80cd9c0 con 0x7f18c815ac10 2024-03-22T04:19:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.476+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c80cdc20 con 0x7f18c815ac10 2024-03-22T04:19:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.476+0000 7f18bfe00640 1 -- 172.21.15.86:0/2763999065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f18b001a070 con 0x7f18c815ac10 2024-03-22T04:19:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.476+0000 7f18bfe00640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18a4067b20 0x7f18a4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.476+0000 7f18bfe00640 1 -- 172.21.15.86:0/2763999065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f18b00854c0 con 0x7f18c815ac10 2024-03-22T04:19:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.476+0000 7f18c7400640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18a4067b20 0x7f18a4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:28.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.476+0000 7f18c7400640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18a4067b20 0x7f18a4069fe0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f18b8000f70 tx=0x7f18b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.480+0000 7f18bfe00640 1 -- 172.21.15.86:0/2763999065 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f18b0055650 con 0x7f18c815ac10 2024-03-22T04:19:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f18c8068650 con 0x7f18c815ac10 2024-03-22T04:19:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18bfe00640 1 -- 172.21.15.86:0/2763999065 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7f18b00596f0 con 0x7f18c815ac10 2024-03-22T04:19:28.672 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:19:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18a4067b20 msgr2=0x7f18a4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18a4067b20 0x7f18a4069fe0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f18b8000f70 tx=0x7f18b8009040 comp rx=0 tx=0).stop 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815ac10 msgr2=0x7f18c80ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815ac10 0x7f18c80ad4b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f18b000c410 tx=0x7f18b000c8d0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 shutdown_connections 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f18a4067b20 0x7f18a4069fe0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c815e000 0x7f18c8150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18c815ac10 0x7f18c80ad4b0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 --2- 172.21.15.86:0/2763999065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c806df90 0x7f18c81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 >> 172.21.15.86:0/2763999065 conn(0x7f18c80b1df0 msgr2=0x7f18c80d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 shutdown_connections 2024-03-22T04:19:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28.672+0000 7f18ce000640 1 -- 172.21.15.86:0/2763999065 wait complete. 2024-03-22T04:19:28.693 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:19:28.693 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:19:28.694 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:19:28.694 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:19:28.694 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:28.694 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:19:28.694 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:19:28.694 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:19:28.694 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:19:28.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:28.694 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:19:28.703 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:19:28.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41963sec preferred_lft 41963sec 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:19:28.705 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:19:28.706 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:19:28.706 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:28.706 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:19:28.706 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:19:28.706 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:19:28.706 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:19:28.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:28.706 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:19:28.763 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:19:28.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:28.765 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:19:28.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:19:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:19:28.819 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:19:28.819 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:19:28.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:19:28.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:19:28.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:19:28.819 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:19:28.860 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:19:28.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:19:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:19:28.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:28.953 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:19:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:19:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:19:29.003 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:19:29.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:29.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:19:29.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:19:29.070 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:29.118 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:19:29.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:19:29.119 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:29.170 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:19:29.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:29.170 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:19:29.258 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:19:29.258 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:19:29.258 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:19:29.273 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:19:29.273 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:19:29.274 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:19:29.274 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:19:29.274 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:19:29.282 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:19:29.282 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:19:29.283 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:19:29.283 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:19:29.283 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:19:29.283 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:19:29.283 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:19:29.283 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:19:29.287 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:19:29.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:29.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:29.353 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:19:29.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:19:29.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:19:29.451 INFO:teuthology.orchestra.run.smithi086.stdout:client77309 2024-03-22T04:19:29.451 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:29.451 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:29.460 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:29.460 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:29.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:29.510 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-22T04:19:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records in 2024-03-22T04:19:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records out 2024-03-22T04:19:29.656 INFO:teuthology.orchestra.run.smithi086.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.09075 s, 69.3 MB/s 2024-03-22T04:19:29.656 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:29.657 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:29.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:29.666 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-22T04:19:29.795 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records in 2024-03-22T04:19:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records out 2024-03-22T04:19:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0802479 s, 78.4 MB/s 2024-03-22T04:19:29.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:29.797 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-22T04:19:29.807 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-03-22T04:19:29.807 INFO:teuthology.orchestra.run.smithi086.stdout:./file2_sixmegs 2024-03-22T04:19:29.807 INFO:teuthology.orchestra.run.smithi086.stdout:./file3_sixmegs 2024-03-22T04:19:29.807 INFO:teuthology.orchestra.run.smithi086.stdout:./file1_sixmegs 2024-03-22T04:19:29.808 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:19:29.808 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:19:29.808 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:19:29.808 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:19:29.808 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-22T04:19:29.808 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:19:29.917 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:19:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:19:29.927 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:19:29.927 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:19:29.927 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:19:29.927 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:19:29.927 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-03-22T04:19:29.928 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:19:30.033 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-03-22T04:19:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:19:30.042 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:19:30.042 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:19:30.042 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:19:30.042 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:19:30.042 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-03-22T04:19:30.042 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:19:30.145 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-03-22T04:19:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:19:30.154 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:19:30.154 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:19:30.154 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:19:30.154 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:19:30.154 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-03-22T04:19:30.154 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:19:30.261 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-03-22T04:19:30.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:19:30.270 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:30.270 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:30.319 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:30.319 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:30.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:19:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f9a00640 1 Processor -- start 2024-03-22T04:19:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f9a00640 1 -- start start 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f9a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f34f415b3f0 0x7f34f40729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34f40cc3e0 0x7f34f4072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f9a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 0x7f34f40ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f9a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f34f40716d0 con 0x7f34f40cc3e0 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f9a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f34f406d990 con 0x7f34f415b3f0 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f9a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34f4071920 con 0x7f34f40b30f0 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34f40cc3e0 0x7f34f4072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34f40cc3e0 0x7f34f4072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52216/0 (socket says 172.21.15.86:52216) 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2000640 1 -- 172.21.15.86:0/2085812923 learned_addr learned my addr 172.21.15.86:0/2085812923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2a00640 1 --2- 172.21.15.86:0/2085812923 >> v2:172.21.15.196:3300/0 conn(0x7f34f415b3f0 0x7f34f40729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f3400640 1 --2- 172.21.15.86:0/2085812923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 0x7f34f40ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2000640 1 -- 172.21.15.86:0/2085812923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 msgr2=0x7f34f40ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:30.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2000640 1 --2- 172.21.15.86:0/2085812923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 0x7f34f40ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2000640 1 -- 172.21.15.86:0/2085812923 >> v2:172.21.15.196:3300/0 conn(0x7f34f415b3f0 msgr2=0x7f34f40729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:30.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2000640 1 --2- 172.21.15.86:0/2085812923 >> v2:172.21.15.196:3300/0 conn(0x7f34f415b3f0 0x7f34f40729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.548+0000 7f34f2000640 1 -- 172.21.15.86:0/2085812923 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f4071c00 con 0x7f34f40cc3e0 2024-03-22T04:19:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f2000640 1 --2- 172.21.15.86:0/2085812923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f40cc3e0 0x7f34f4072f00 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f34f406c510 tx=0x7f34e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb8440ec21a7b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f1600640 1 -- 172.21.15.86:0/2085812923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e00b0070 con 0x7f34f40cc3e0 2024-03-22T04:19:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f1600640 1 -- 172.21.15.86:0/2085812923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e00ab600 con 0x7f34f40cc3e0 2024-03-22T04:19:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f1600640 1 -- 172.21.15.86:0/2085812923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e00ab9d0 con 0x7f34f40cc3e0 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 -- 172.21.15.86:0/2085812923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f40cc3e0 msgr2=0x7f34f4072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 --2- 172.21.15.86:0/2085812923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f40cc3e0 0x7f34f4072f00 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f34f406c510 tx=0x7f34e001c490 comp rx=0 tx=0).stop 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 -- 172.21.15.86:0/2085812923 shutdown_connections 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 --2- 172.21.15.86:0/2085812923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 0x7f34f40ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 --2- 172.21.15.86:0/2085812923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f40cc3e0 0x7f34f4072f00 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 --2- 172.21.15.86:0/2085812923 >> v2:172.21.15.196:3300/0 conn(0x7f34f415b3f0 0x7f34f40729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 -- 172.21.15.86:0/2085812923 >> 172.21.15.86:0/2085812923 conn(0x7f34f40b1df0 msgr2=0x7f34f40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 -- 172.21.15.86:0/2085812923 shutdown_connections 2024-03-22T04:19:30.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 -- 172.21.15.86:0/2085812923 wait complete. 2024-03-22T04:19:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.552+0000 7f34f9a00640 1 Processor -- start 2024-03-22T04:19:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 -- start start 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 0x7f34f41530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f40cc3e0 0x7f34f40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f415b3f0 0x7f34f4153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34f415f1e0 con 0x7f34f415b3f0 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34f415f060 con 0x7f34f40cc3e0 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34f415f360 con 0x7f34f40b30f0 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f415b3f0 0x7f34f4153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f2a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 0x7f34f41530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f415b3f0 0x7f34f4153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52228/0 (socket says 172.21.15.86:52228) 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f40cc3e0 0x7f34f40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 -- 172.21.15.86:0/1900248216 learned_addr learned my addr 172.21.15.86:0/1900248216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 -- 172.21.15.86:0/1900248216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 msgr2=0x7f34f41530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:30.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 0x7f34f41530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 -- 172.21.15.86:0/1900248216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f40cc3e0 msgr2=0x7f34f40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f40cc3e0 0x7f34f40b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 -- 172.21.15.86:0/1900248216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f4151d60 con 0x7f34f415b3f0 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f3400640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f415b3f0 0x7f34f4153e20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00ef00 tx=0x7f34ec00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34eaa00640 1 -- 172.21.15.86:0/1900248216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec018020 con 0x7f34f415b3f0 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34eaa00640 1 -- 172.21.15.86:0/1900248216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec014400 con 0x7f34f415b3f0 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f4152040 con 0x7f34f415b3f0 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34eaa00640 1 -- 172.21.15.86:0/1900248216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0147f0 con 0x7f34f415b3f0 2024-03-22T04:19:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f41528e0 con 0x7f34f415b3f0 2024-03-22T04:19:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.556+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f4068650 con 0x7f34f415b3f0 2024-03-22T04:19:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.560+0000 7f34eaa00640 1 -- 172.21.15.86:0/1900248216 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f34ec01b020 con 0x7f34f415b3f0 2024-03-22T04:19:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.560+0000 7f34eaa00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f34d4067b20 0x7f34d4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.560+0000 7f34eaa00640 1 -- 172.21.15.86:0/1900248216 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f34ec086750 con 0x7f34f415b3f0 2024-03-22T04:19:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.560+0000 7f34f2a00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f34d4067b20 0x7f34d4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.560+0000 7f34f2a00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f34d4067b20 0x7f34d4069fe0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009ac0 tx=0x7f34e4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:30.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.564+0000 7f34eaa00640 1 -- 172.21.15.86:0/1900248216 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f34ec022070 con 0x7f34f415b3f0 2024-03-22T04:19:30.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.720+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34f4068e10 con 0x7f34f415b3f0 2024-03-22T04:19:30.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.720+0000 7f34eaa00640 1 -- 172.21.15.86:0/1900248216 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16900 (secure 0 0 0) 0x7f34ec056850 con 0x7f34f415b3f0 2024-03-22T04:19:30.722 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:30.723 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":109,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:19:19.318781+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:19:15.365871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-22T04:19:16.647234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f34d4067b20 msgr2=0x7f34d4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f34d4067b20 0x7f34d4069fe0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009ac0 tx=0x7f34e4009190 comp rx=0 tx=0).stop 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f415b3f0 msgr2=0x7f34f4153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f415b3f0 0x7f34f4153e20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00ef00 tx=0x7f34ec00b480 comp rx=0 tx=0).stop 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 shutdown_connections 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f34d4067b20 0x7f34d4069fe0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f40cc3e0 0x7f34f40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34f415b3f0 0x7f34f4153e20 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 --2- 172.21.15.86:0/1900248216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b30f0 0x7f34f41530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 >> 172.21.15.86:0/1900248216 conn(0x7f34f40b1df0 msgr2=0x7f34f415ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:30.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 shutdown_connections 2024-03-22T04:19:30.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.724+0000 7f34f9a00640 1 -- 172.21.15.86:0/1900248216 wait complete. 2024-03-22T04:19:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:19:15.365871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:19:16.647234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:19:30.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:19:30.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d4800640 1 Processor -- start 2024-03-22T04:19:30.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d4800640 1 -- start start 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d4800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb1cc15e970 0x7fb1cc0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d4800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc06d8f0 0x7fb1cc09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d4800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1cc09afb0 0x7fb1cc0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d4800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb1cc0b49a0 con 0x7fb1cc09afb0 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d4800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb1cc0ccda0 con 0x7fb1cc15e970 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d4800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb1cc06dd70 con 0x7fb1cc06d8f0 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb1cc15e970 0x7fb1cc0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1cc09afb0 0x7fb1cc0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb1cc15e970 0x7fb1cc0b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:39630/0 (socket says 172.21.15.86:39630) 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1e00640 1 -- 172.21.15.86:0/527459025 learned_addr learned my addr 172.21.15.86:0/527459025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1400640 1 --2- 172.21.15.86:0/527459025 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc06d8f0 0x7fb1cc09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1e00640 1 -- 172.21.15.86:0/527459025 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc06d8f0 msgr2=0x7fb1cc09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1e00640 1 --2- 172.21.15.86:0/527459025 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc06d8f0 0x7fb1cc09aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1e00640 1 -- 172.21.15.86:0/527459025 >> v2:172.21.15.86:3300/0 conn(0x7fb1cc09afb0 msgr2=0x7fb1cc0d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1e00640 1 --2- 172.21.15.86:0/527459025 >> v2:172.21.15.86:3300/0 conn(0x7fb1cc09afb0 0x7fb1cc0d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.884+0000 7fb1d1e00640 1 -- 172.21.15.86:0/527459025 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc06dff0 con 0x7fb1cc15e970 2024-03-22T04:19:30.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d1e00640 1 --2- 172.21.15.86:0/527459025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc15e970 0x7fb1cc0b12e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc069080 tx=0x7fb1c001c260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5e4c55ca5222a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d0a00640 1 -- 172.21.15.86:0/527459025 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00b0070 con 0x7fb1cc15e970 2024-03-22T04:19:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d0a00640 1 -- 172.21.15.86:0/527459025 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c00ab600 con 0x7fb1cc15e970 2024-03-22T04:19:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d0a00640 1 -- 172.21.15.86:0/527459025 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00ab9d0 con 0x7fb1cc15e970 2024-03-22T04:19:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 -- 172.21.15.86:0/527459025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc15e970 msgr2=0x7fb1cc0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 --2- 172.21.15.86:0/527459025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc15e970 0x7fb1cc0b12e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc069080 tx=0x7fb1c001c260 comp rx=0 tx=0).stop 2024-03-22T04:19:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 -- 172.21.15.86:0/527459025 shutdown_connections 2024-03-22T04:19:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 --2- 172.21.15.86:0/527459025 >> v2:172.21.15.86:3300/0 conn(0x7fb1cc09afb0 0x7fb1cc0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 --2- 172.21.15.86:0/527459025 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc06d8f0 0x7fb1cc09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 --2- 172.21.15.86:0/527459025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc15e970 0x7fb1cc0b12e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 -- 172.21.15.86:0/527459025 >> 172.21.15.86:0/527459025 conn(0x7fb1cc0b3bf0 msgr2=0x7fb1cc0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 -- 172.21.15.86:0/527459025 shutdown_connections 2024-03-22T04:19:30.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.888+0000 7fb1d4800640 1 -- 172.21.15.86:0/527459025 wait complete. 2024-03-22T04:19:30.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 Processor -- start 2024-03-22T04:19:30.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 -- start start 2024-03-22T04:19:30.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc06d8f0 0x7fb1cc1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc09afb0 0x7fb1cc0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15e970 0x7fb1cc150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0cd230 con 0x7fb1cc15e970 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc15ed50 con 0x7fb1cc06d8f0 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb1cc0cd3b0 con 0x7fb1cc09afb0 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15e970 0x7fb1cc150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15e970 0x7fb1cc150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52254/0 (socket says 172.21.15.86:52254) 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 -- 172.21.15.86:0/138725445 learned_addr learned my addr 172.21.15.86:0/138725445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d1e00640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc06d8f0 0x7fb1cc1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d1400640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc09afb0 0x7fb1cc0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 -- 172.21.15.86:0/138725445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc09afb0 msgr2=0x7fb1cc0b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc09afb0 0x7fb1cc0b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 -- 172.21.15.86:0/138725445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc06d8f0 msgr2=0x7fb1cc1576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc06d8f0 0x7fb1cc1576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 -- 172.21.15.86:0/138725445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc152b80 con 0x7fb1cc15e970 2024-03-22T04:19:30.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d2800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15e970 0x7fb1cc150ff0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009dd0 tx=0x7fb1c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:30.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1c6000640 1 -- 172.21.15.86:0/138725445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8018020 con 0x7fb1cc15e970 2024-03-22T04:19:30.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1c6000640 1 -- 172.21.15.86:0/138725445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c8014480 con 0x7fb1cc15e970 2024-03-22T04:19:30.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1cc0c9300 con 0x7fb1cc15e970 2024-03-22T04:19:30.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1c6000640 1 -- 172.21.15.86:0/138725445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8014870 con 0x7fb1cc15e970 2024-03-22T04:19:30.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1cc157c20 con 0x7fb1cc15e970 2024-03-22T04:19:30.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.892+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1cc0c9ad0 con 0x7fb1cc15e970 2024-03-22T04:19:30.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.896+0000 7fb1c6000640 1 -- 172.21.15.86:0/138725445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb1c801b020 con 0x7fb1cc15e970 2024-03-22T04:19:30.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.896+0000 7fb1c6000640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1b0067b20 0x7fb1b0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:30.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.896+0000 7fb1c6000640 1 -- 172.21.15.86:0/138725445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fb1c8086990 con 0x7fb1cc15e970 2024-03-22T04:19:30.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.896+0000 7fb1d1e00640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1b0067b20 0x7fb1b0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:30.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.896+0000 7fb1d1e00640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1b0067b20 0x7fb1b0069fe0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00ad000 tx=0x7fb1c00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:30.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:30.900+0000 7fb1c6000640 1 -- 172.21.15.86:0/138725445 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb1c8022070 con 0x7fb1cc15e970 2024-03-22T04:19:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.052+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1cc074470 con 0x7fb1cc15e970 2024-03-22T04:19:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.052+0000 7fb1c6000640 1 -- 172.21.15.86:0/138725445 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fb1c8010040 con 0x7fb1cc15e970 2024-03-22T04:19:31.053 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:19:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1b0067b20 msgr2=0x7fb1b0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1b0067b20 0x7fb1b0069fe0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00ad000 tx=0x7fb1c00a6000 comp rx=0 tx=0).stop 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15e970 msgr2=0x7fb1cc150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15e970 0x7fb1cc150ff0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009dd0 tx=0x7fb1c800c490 comp rx=0 tx=0).stop 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 shutdown_connections 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1b0067b20 0x7fb1b0069fe0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15e970 0x7fb1cc150ff0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc09afb0 0x7fb1cc0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 --2- 172.21.15.86:0/138725445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc06d8f0 0x7fb1cc1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 >> 172.21.15.86:0/138725445 conn(0x7fb1cc0b3bf0 msgr2=0x7fb1cc0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 shutdown_connections 2024-03-22T04:19:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.056+0000 7fb1d4800640 1 -- 172.21.15.86:0/138725445 wait complete. 2024-03-22T04:19:31.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:19:31.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f9ec00640 1 Processor -- start 2024-03-22T04:19:31.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f9ec00640 1 -- start start 2024-03-22T04:19:31.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f9ec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9f98072430 0x7f9f9815a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f9ec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f980af900 con 0x7f9f98071860 2024-03-22T04:19:31.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f9ec00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f980cfb00 con 0x7f9f980cf670 2024-03-22T04:19:31.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f9ec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9f980b2300 con 0x7f9f98072430 2024-03-22T04:19:31.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f97e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9f98071860 0x7f9f980b1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33756/0 (socket says 172.21.15.86:33756) 2024-03-22T04:19:31.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f97e00640 1 -- 172.21.15.86:0/3194794792 learned_addr learned my addr 172.21.15.86:0/3194794792 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:31.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.212+0000 7f9f9cc00640 1 --2- 172.21.15.86:0/3194794792 >> v2:172.21.15.196:3300/0 conn(0x7f9f98072430 0x7f9f9815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f96a00640 1 -- 172.21.15.86:0/3194794792 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228340138 0 0) 0x7f9f980af900 con 0x7f9f98071860 2024-03-22T04:19:31.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f96a00640 1 -- 172.21.15.86:0/3194794792 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f7c005740 con 0x7f9f98071860 2024-03-22T04:19:31.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f96a00640 1 -- 172.21.15.86:0/3194794792 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091103915 0 0) 0x7f9f980cfb00 con 0x7f9f980cf670 2024-03-22T04:19:31.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f96a00640 1 -- 172.21.15.86:0/3194794792 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f980af900 con 0x7f9f980cf670 2024-03-22T04:19:31.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9cc00640 1 -- 172.21.15.86:0/3194794792 >> v1:172.21.15.196:6789/0 conn(0x7f9f980cf670 legacy=0x7f9f98071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9cc00640 1 -- 172.21.15.86:0/3194794792 >> v1:172.21.15.86:6789/0 conn(0x7f9f98071860 legacy=0x7f9f980b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9cc00640 1 -- 172.21.15.86:0/3194794792 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f980ce5d0 con 0x7f9f98072430 2024-03-22T04:19:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9cc00640 1 --2- 172.21.15.86:0/3194794792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98072430 0x7f9f9815a830 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9f900097c0 tx=0x7f9f9001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69de30cb4d34f8d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f96a00640 1 -- 172.21.15.86:0/3194794792 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900b0070 con 0x7f9f98072430 2024-03-22T04:19:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f96a00640 1 -- 172.21.15.86:0/3194794792 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f900ab600 con 0x7f9f98072430 2024-03-22T04:19:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f96a00640 1 -- 172.21.15.86:0/3194794792 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900ab9d0 con 0x7f9f98072430 2024-03-22T04:19:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9ec00640 1 -- 172.21.15.86:0/3194794792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98072430 msgr2=0x7f9f9815a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9ec00640 1 --2- 172.21.15.86:0/3194794792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98072430 0x7f9f9815a830 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9f900097c0 tx=0x7f9f9001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9ec00640 1 -- 172.21.15.86:0/3194794792 shutdown_connections 2024-03-22T04:19:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9ec00640 1 --2- 172.21.15.86:0/3194794792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98072430 0x7f9f9815a830 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.216+0000 7f9f9ec00640 1 -- 172.21.15.86:0/3194794792 >> 172.21.15.86:0/3194794792 conn(0x7f9f980cc940 msgr2=0x7f9f980ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 -- 172.21.15.86:0/3194794792 shutdown_connections 2024-03-22T04:19:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 -- 172.21.15.86:0/3194794792 wait complete. 2024-03-22T04:19:31.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 Processor -- start 2024-03-22T04:19:31.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 -- start start 2024-03-22T04:19:31.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98071860 0x7f9f9814fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f98072430 0x7f9f98157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f980cf670 0x7f9f9815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9806dee0 con 0x7f9f98072430 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9815b180 con 0x7f9f98071860 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9806e060 con 0x7f9f980cf670 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f98072430 0x7f9f98157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98071860 0x7f9f9814fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98071860 0x7f9f9814fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:39650/0 (socket says 172.21.15.86:39650) 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97e00640 1 -- 172.21.15.86:0/142784061 learned_addr learned my addr 172.21.15.86:0/142784061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9cc00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f980cf670 0x7f9f9815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97400640 1 -- 172.21.15.86:0/142784061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f980cf670 msgr2=0x7f9f9815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97400640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f980cf670 0x7f9f9815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97400640 1 -- 172.21.15.86:0/142784061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98071860 msgr2=0x7f9f9814fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97400640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98071860 0x7f9f9814fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97400640 1 -- 172.21.15.86:0/142784061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f981503b0 con 0x7f9f98072430 2024-03-22T04:19:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97e00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98071860 0x7f9f9814fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f97400640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f98072430 0x7f9f98157c10 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c370 tx=0x7f9f84001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f8fe00640 1 -- 172.21.15.86:0/142784061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84017070 con 0x7f9f98072430 2024-03-22T04:19:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f8fe00640 1 -- 172.21.15.86:0/142784061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f840052f0 con 0x7f9f98072430 2024-03-22T04:19:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f8fe00640 1 -- 172.21.15.86:0/142784061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84003840 con 0x7f9f98072430 2024-03-22T04:19:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.220+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f98155720 con 0x7f9f98072430 2024-03-22T04:19:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.224+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f98157410 con 0x7f9f98072430 2024-03-22T04:19:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.224+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98074470 con 0x7f9f98072430 2024-03-22T04:19:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.224+0000 7f9f8fe00640 1 -- 172.21.15.86:0/142784061 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9f8401a070 con 0x7f9f98072430 2024-03-22T04:19:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.224+0000 7f9f8fe00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f78067b20 0x7f9f78069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.224+0000 7f9f8fe00640 1 -- 172.21.15.86:0/142784061 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f9f84085370 con 0x7f9f98072430 2024-03-22T04:19:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.224+0000 7f9f97e00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f78067b20 0x7f9f78069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.224+0000 7f9f97e00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f78067b20 0x7f9f78069fe0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9f88002240 tx=0x7f9f88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.228+0000 7f9f8fe00640 1 -- 172.21.15.86:0/142784061 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9f840555a0 con 0x7f9f98072430 2024-03-22T04:19:31.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f980ce0e0 con 0x7f9f98072430 2024-03-22T04:19:31.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f8fe00640 1 -- 172.21.15.86:0/142784061 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f9f84059640 con 0x7f9f98072430 2024-03-22T04:19:31.388 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:19:31.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f78067b20 msgr2=0x7f9f78069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f78067b20 0x7f9f78069fe0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9f88002240 tx=0x7f9f88009040 comp rx=0 tx=0).stop 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f98072430 msgr2=0x7f9f98157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f98072430 0x7f9f98157c10 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c370 tx=0x7f9f84001e80 comp rx=0 tx=0).stop 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 shutdown_connections 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f78067b20 0x7f9f78069fe0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f980cf670 0x7f9f9815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f98072430 0x7f9f98157c10 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 --2- 172.21.15.86:0/142784061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f98071860 0x7f9f9814fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 >> 172.21.15.86:0/142784061 conn(0x7f9f980cc940 msgr2=0x7f9f980b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 shutdown_connections 2024-03-22T04:19:31.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.388+0000 7f9f9ec00640 1 -- 172.21.15.86:0/142784061 wait complete. 2024-03-22T04:19:31.410 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:19:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.548+0000 7fb1d2800640 1 Processor -- start 2024-03-22T04:19:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.548+0000 7fb1d2800640 1 -- start start 2024-03-22T04:19:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1d2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1cc15d0b0 0x7fb1cc1671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1d2800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1cc15d710 con 0x7fb1cc0b2360 2024-03-22T04:19:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1d2800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1cc15fe40 con 0x7fb1cc0b3600 2024-03-22T04:19:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1d2800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb1cc0b2740 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1cc15d0b0 0x7fb1cc1671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1cb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb1cc0b2360 0x7fb1cc0b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33762/0 (socket says 172.21.15.86:33762) 2024-03-22T04:19:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1cb400640 1 -- 172.21.15.86:0/2079239855 learned_addr learned my addr 172.21.15.86:0/2079239855 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1cc15d0b0 0x7fb1cc1671d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52272/0 (socket says 172.21.15.86:52272) 2024-03-22T04:19:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1cbe00640 1 -- 172.21.15.86:0/2079239855 >> v1:172.21.15.196:6789/0 conn(0x7fb1cc0b3600 legacy=0x7fb1cc15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1cbe00640 1 -- 172.21.15.86:0/2079239855 >> v1:172.21.15.86:6789/0 conn(0x7fb1cc0b2360 legacy=0x7fb1cc0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1cbe00640 1 -- 172.21.15.86:0/2079239855 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc160120 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.552+0000 7fb1caa00640 1 -- 172.21.15.86:0/2079239855 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306331385 0 0) 0x7fb1cc15d710 con 0x7fb1cc0b2360 2024-03-22T04:19:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1cbe00640 1 --2- 172.21.15.86:0/2079239855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1671d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009d90 tx=0x7fb1c401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448452b804955e11 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1caa00640 1 -- 172.21.15.86:0/2079239855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40b2070 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1caa00640 1 -- 172.21.15.86:0/2079239855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c40a5920 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1caa00640 1 -- 172.21.15.86:0/2079239855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40a5cf0 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- 172.21.15.86:0/2079239855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 msgr2=0x7fb1cc1671d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 --2- 172.21.15.86:0/2079239855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1671d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009d90 tx=0x7fb1c401c970 comp rx=0 tx=0).stop 2024-03-22T04:19:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- 172.21.15.86:0/2079239855 shutdown_connections 2024-03-22T04:19:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 --2- 172.21.15.86:0/2079239855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1671d0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- 172.21.15.86:0/2079239855 >> 172.21.15.86:0/2079239855 conn(0x7fb1cc0cc940 msgr2=0x7fb1cc0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- 172.21.15.86:0/2079239855 shutdown_connections 2024-03-22T04:19:31.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- 172.21.15.86:0/2079239855 wait complete. 2024-03-22T04:19:31.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 Processor -- start 2024-03-22T04:19:31.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- start start 2024-03-22T04:19:31.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc0b2360 0x7fb1cc1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc0b3600 0x7fb1cc155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc160c00 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc169f30 con 0x7fb1cc0b2360 2024-03-22T04:19:31.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.556+0000 7fb1d2800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb1cc150930 con 0x7fb1cc0b3600 2024-03-22T04:19:31.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc0b2360 0x7fb1cc1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52286/0 (socket says 172.21.15.86:52286) 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 -- 172.21.15.86:0/577226244 learned_addr learned my addr 172.21.15.86:0/577226244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1c2a00640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc0b3600 0x7fb1cc155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 -- 172.21.15.86:0/577226244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc0b3600 msgr2=0x7fb1cc155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc0b3600 0x7fb1cc155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 -- 172.21.15.86:0/577226244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc0b2360 msgr2=0x7fb1cc1516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc0b2360 0x7fb1cc1516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 -- 172.21.15.86:0/577226244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc1543d0 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1cbe00640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1503f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40b0040 tx=0x7fb1c4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1c3e00640 1 -- 172.21.15.86:0/577226244 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40bd020 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1c3e00640 1 -- 172.21.15.86:0/577226244 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c40ad6c0 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1cc1546b0 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1c3e00640 1 -- 172.21.15.86:0/577226244 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40b2040 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1cc14bb00 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1cc074470 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1c3e00640 1 -- 172.21.15.86:0/577226244 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb1c40c0020 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1c3e00640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1a8067b50 0x7fb1a806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.560+0000 7fb1c3e00640 1 -- 172.21.15.86:0/577226244 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fb1c4120ff0 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.564+0000 7fb1cb400640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1a8067b50 0x7fb1a806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.564+0000 7fb1cb400640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1a8067b50 0x7fb1a806a010 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc007a90 tx=0x7fb1bc0079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.564+0000 7fb1c3e00640 1 -- 172.21.15.86:0/577226244 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb1c40c7070 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb1cc160d80 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1c3e00640 1 -- 172.21.15.86:0/577226244 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7fb1c4016560 con 0x7fb1cc15d0b0 2024-03-22T04:19:31.720 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:19:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1a8067b50 msgr2=0x7fb1a806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1a8067b50 0x7fb1a806a010 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc007a90 tx=0x7fb1bc0079e0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 msgr2=0x7fb1cc1503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1503f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40b0040 tx=0x7fb1c4019710 comp rx=0 tx=0).stop 2024-03-22T04:19:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 shutdown_connections 2024-03-22T04:19:31.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb1a8067b50 0x7fb1a806a010 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1cc15d0b0 0x7fb1cc1503f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1cc0b3600 0x7fb1cc155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 --2- 172.21.15.86:0/577226244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1cc0b2360 0x7fb1cc1516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 >> 172.21.15.86:0/577226244 conn(0x7fb1cc0cc940 msgr2=0x7fb1cc0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 shutdown_connections 2024-03-22T04:19:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.720+0000 7fb1d2800640 1 -- 172.21.15.86:0/577226244 wait complete. 2024-03-22T04:19:31.740 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:19:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.876+0000 7fe1a6800640 1 Processor -- start 2024-03-22T04:19:31.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.876+0000 7fe1a6800640 1 -- start start 2024-03-22T04:19:31.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe1a6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe1a015e630 0x7fe1a00cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe1a6800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe1a006e2f0 0x7fe1a00d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe1a6800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a006cc80 0x7fe1a006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe1a6800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a01623d0 con 0x7fe1a015e630 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe1a6800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a01620f0 con 0x7fe1a006e2f0 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe1a6800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a015ec10 con 0x7fe1a006cc80 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe19f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe1a015e630 0x7fe1a00cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe197e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe1a006e2f0 0x7fe1a00d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe19f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe1a015e630 0x7fe1a00cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52288/0 (socket says 172.21.15.86:52288) 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe19f400640 1 -- 172.21.15.86:0/3425146062 learned_addr learned my addr 172.21.15.86:0/3425146062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe19fe00640 1 --2- 172.21.15.86:0/3425146062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a006cc80 0x7fe1a006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe197e00640 1 -- 172.21.15.86:0/3425146062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a006cc80 msgr2=0x7fe1a006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe197e00640 1 --2- 172.21.15.86:0/3425146062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a006cc80 0x7fe1a006d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe197e00640 1 -- 172.21.15.86:0/3425146062 >> v2:172.21.15.86:3300/0 conn(0x7fe1a015e630 msgr2=0x7fe1a00cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe197e00640 1 --2- 172.21.15.86:0/3425146062 >> v2:172.21.15.86:3300/0 conn(0x7fe1a015e630 0x7fe1a00cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe197e00640 1 -- 172.21.15.86:0/3425146062 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a015eef0 con 0x7fe1a006e2f0 2024-03-22T04:19:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.880+0000 7fe19f400640 1 --2- 172.21.15.86:0/3425146062 >> v2:172.21.15.86:3300/0 conn(0x7fe1a015e630 0x7fe1a00cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:31.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe197e00640 1 --2- 172.21.15.86:0/3425146062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 0x7fe1a00d03b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00cc880 tx=0x7fe18801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363fe22d5f356499 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe19ea00640 1 -- 172.21.15.86:0/3425146062 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1880b0070 con 0x7fe1a006e2f0 2024-03-22T04:19:31.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe19ea00640 1 -- 172.21.15.86:0/3425146062 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1880ab600 con 0x7fe1a006e2f0 2024-03-22T04:19:31.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe19ea00640 1 -- 172.21.15.86:0/3425146062 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1880ab9d0 con 0x7fe1a006e2f0 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- 172.21.15.86:0/3425146062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 msgr2=0x7fe1a00d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 --2- 172.21.15.86:0/3425146062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 0x7fe1a00d03b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00cc880 tx=0x7fe18801c480 comp rx=0 tx=0).stop 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- 172.21.15.86:0/3425146062 shutdown_connections 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 --2- 172.21.15.86:0/3425146062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a006cc80 0x7fe1a006d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 --2- 172.21.15.86:0/3425146062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 0x7fe1a00d03b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 --2- 172.21.15.86:0/3425146062 >> v2:172.21.15.86:3300/0 conn(0x7fe1a015e630 0x7fe1a00cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- 172.21.15.86:0/3425146062 >> 172.21.15.86:0/3425146062 conn(0x7fe1a006f910 msgr2=0x7fe1a006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- 172.21.15.86:0/3425146062 shutdown_connections 2024-03-22T04:19:31.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- 172.21.15.86:0/3425146062 wait complete. 2024-03-22T04:19:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 Processor -- start 2024-03-22T04:19:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- start start 2024-03-22T04:19:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a006cc80 0x7fe1a0151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 0x7fe1a00b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a015e630 0x7fe1a0161170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a006d9f0 con 0x7fe1a006cc80 2024-03-22T04:19:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a015f9d0 con 0x7fe1a006e2f0 2024-03-22T04:19:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.884+0000 7fe1a6800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a00cfaf0 con 0x7fe1a015e630 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a006cc80 0x7fe1a0151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a006cc80 0x7fe1a0151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52294/0 (socket says 172.21.15.86:52294) 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 -- 172.21.15.86:0/2122373248 learned_addr learned my addr 172.21.15.86:0/2122373248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19fe00640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a015e630 0x7fe1a0161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe197e00640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 0x7fe1a00b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 -- 172.21.15.86:0/2122373248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a015e630 msgr2=0x7fe1a0161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a015e630 0x7fe1a0161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 -- 172.21.15.86:0/2122373248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 msgr2=0x7fe1a00b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 0x7fe1a00b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 -- 172.21.15.86:0/2122373248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a0157700 con 0x7fe1a006cc80 2024-03-22T04:19:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe19f400640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a006cc80 0x7fe1a0151560 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe19000e9d0 tx=0x7fe19000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe197400640 1 -- 172.21.15.86:0/2122373248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190017070 con 0x7fe1a006cc80 2024-03-22T04:19:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe197400640 1 -- 172.21.15.86:0/2122373248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19000ca60 con 0x7fe1a006cc80 2024-03-22T04:19:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a0156940 con 0x7fe1a006cc80 2024-03-22T04:19:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe197400640 1 -- 172.21.15.86:0/2122373248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19000ce10 con 0x7fe1a006cc80 2024-03-22T04:19:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a0156f00 con 0x7fe1a006cc80 2024-03-22T04:19:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a0068650 con 0x7fe1a006cc80 2024-03-22T04:19:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.888+0000 7fe197400640 1 -- 172.21.15.86:0/2122373248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe19001a070 con 0x7fe1a006cc80 2024-03-22T04:19:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.892+0000 7fe197400640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe17c067b20 0x7fe17c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.892+0000 7fe197400640 1 -- 172.21.15.86:0/2122373248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fe190085f70 con 0x7fe1a006cc80 2024-03-22T04:19:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.892+0000 7fe197e00640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe17c067b20 0x7fe17c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.892+0000 7fe197e00640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe17c067b20 0x7fe17c069fe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe1880ad000 tx=0x7fe188008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:31.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:31.892+0000 7fe197400640 1 -- 172.21.15.86:0/2122373248 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe1900561a0 con 0x7fe1a006cc80 2024-03-22T04:19:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.060+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe1a00c57c0 con 0x7fe1a006cc80 2024-03-22T04:19:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.060+0000 7fe197400640 1 -- 172.21.15.86:0/2122373248 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7fe19005a240 con 0x7fe1a006cc80 2024-03-22T04:19:32.062 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe17c067b20 msgr2=0x7fe17c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe17c067b20 0x7fe17c069fe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe1880ad000 tx=0x7fe188008040 comp rx=0 tx=0).stop 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a006cc80 msgr2=0x7fe1a0151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a006cc80 0x7fe1a0151560 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe19000e9d0 tx=0x7fe19000ee90 comp rx=0 tx=0).stop 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 shutdown_connections 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe17c067b20 0x7fe17c069fe0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a006e2f0 0x7fe1a00b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a015e630 0x7fe1a0161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 --2- 172.21.15.86:0/2122373248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a006cc80 0x7fe1a0151560 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 >> 172.21.15.86:0/2122373248 conn(0x7fe1a006f910 msgr2=0x7fe1a00b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 shutdown_connections 2024-03-22T04:19:32.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.064+0000 7fe1a6800640 1 -- 172.21.15.86:0/2122373248 wait complete. 2024-03-22T04:19:32.084 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:19:32.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:32.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:19:32.146 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:19:32.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:32.146 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:19:32.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:19:32.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.340+0000 7fbd7f800640 1 Processor -- start 2024-03-22T04:19:32.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.340+0000 7fbd7f800640 1 -- start start 2024-03-22T04:19:32.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.340+0000 7fbd7f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd7815b3f0 0x7fbd780729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.340+0000 7fbd7f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd78071920 con 0x7fbd780ae540 2024-03-22T04:19:32.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.340+0000 7fbd7f800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd78071c00 con 0x7fbd780cc3e0 2024-03-22T04:19:32.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.340+0000 7fbd7f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbd780716d0 con 0x7fbd7815b3f0 2024-03-22T04:19:32.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd7ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd7815b3f0 0x7fbd780729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd7d800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbd780ae540 0x7fbd780b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33766/0 (socket says 172.21.15.86:33766) 2024-03-22T04:19:32.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd7d800640 1 -- 172.21.15.86:0/2036216703 learned_addr learned my addr 172.21.15.86:0/2036216703 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:32.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd7ce00640 1 -- 172.21.15.86:0/2036216703 >> v1:172.21.15.196:6789/0 conn(0x7fbd780cc3e0 legacy=0x7fbd780ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd7ce00640 1 -- 172.21.15.86:0/2036216703 >> v1:172.21.15.86:6789/0 conn(0x7fbd780ae540 legacy=0x7fbd780b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd7ce00640 1 -- 172.21.15.86:0/2036216703 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd78071ee0 con 0x7fbd7815b3f0 2024-03-22T04:19:32.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd77400640 1 -- 172.21.15.86:0/2036216703 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076298496 0 0) 0x7fbd78071920 con 0x7fbd780ae540 2024-03-22T04:19:32.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd7ce00640 1 --2- 172.21.15.86:0/2036216703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780729c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00ba60 tx=0x7fbd6c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9979368b6752b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:32.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd77400640 1 -- 172.21.15.86:0/2036216703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0b4070 con 0x7fbd7815b3f0 2024-03-22T04:19:32.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd77400640 1 -- 172.21.15.86:0/2036216703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c0a7940 con 0x7fbd7815b3f0 2024-03-22T04:19:32.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.344+0000 7fbd77400640 1 -- 172.21.15.86:0/2036216703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0a7d10 con 0x7fbd7815b3f0 2024-03-22T04:19:32.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- 172.21.15.86:0/2036216703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 msgr2=0x7fbd780729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 --2- 172.21.15.86:0/2036216703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780729c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00ba60 tx=0x7fbd6c01eb70 comp rx=0 tx=0).stop 2024-03-22T04:19:32.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- 172.21.15.86:0/2036216703 shutdown_connections 2024-03-22T04:19:32.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 --2- 172.21.15.86:0/2036216703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780729c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- 172.21.15.86:0/2036216703 >> 172.21.15.86:0/2036216703 conn(0x7fbd780b1df0 msgr2=0x7fbd780b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:32.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- 172.21.15.86:0/2036216703 shutdown_connections 2024-03-22T04:19:32.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- 172.21.15.86:0/2036216703 wait complete. 2024-03-22T04:19:32.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 Processor -- start 2024-03-22T04:19:32.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- start start 2024-03-22T04:19:32.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd780ae540 0x7fbd7815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780cc3e0 0x7fbd7815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd780b30f0 con 0x7fbd7815b3f0 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd780b4c20 con 0x7fbd780cc3e0 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7f800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbd780b3270 con 0x7fbd780ae540 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52304/0 (socket says 172.21.15.86:52304) 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 -- 172.21.15.86:0/3425544472 learned_addr learned my addr 172.21.15.86:0/3425544472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd77e00640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780cc3e0 0x7fbd7815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7ce00640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd780ae540 0x7fbd7815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 -- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd780ae540 msgr2=0x7fbd7815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd780ae540 0x7fbd7815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 -- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780cc3e0 msgr2=0x7fbd7815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780cc3e0 0x7fbd7815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 -- 172.21.15.86:0/3425544472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7815cd10 con 0x7fbd7815b3f0 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd77e00640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780cc3e0 0x7fbd7815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.348+0000 7fbd7d800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780b2430 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000c780 tx=0x7fbd7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd74c00640 1 -- 172.21.15.86:0/3425544472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70018070 con 0x7fbd7815b3f0 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd74c00640 1 -- 172.21.15.86:0/3425544472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7000a640 con 0x7fbd7815b3f0 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd74c00640 1 -- 172.21.15.86:0/3425544472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000aab0 con 0x7fbd7815b3f0 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7815cff0 con 0x7fbd7815b3f0 2024-03-22T04:19:32.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7815d5b0 con 0x7fbd7815b3f0 2024-03-22T04:19:32.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd78068650 con 0x7fbd7815b3f0 2024-03-22T04:19:32.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd74c00640 1 -- 172.21.15.86:0/3425544472 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fbd7000ac50 con 0x7fbd7815b3f0 2024-03-22T04:19:32.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd74c00640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd58067b20 0x7fbd58069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd74c00640 1 -- 172.21.15.86:0/3425544472 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7fbd70085770 con 0x7fbd7815b3f0 2024-03-22T04:19:32.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd7ce00640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd58067b20 0x7fbd58069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.352+0000 7fbd7ce00640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd58067b20 0x7fbd58069fe0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0b1000 tx=0x7fbd6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:32.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.356+0000 7fbd74c00640 1 -- 172.21.15.86:0/3425544472 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbd70055f50 con 0x7fbd7815b3f0 2024-03-22T04:19:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd78156710 con 0x7fbd7815b3f0 2024-03-22T04:19:32.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd74c00640 1 -- 172.21.15.86:0/3425544472 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5192 (secure 0 0 0) 0x7fbd70055850 con 0x7fbd7815b3f0 2024-03-22T04:19:32.548 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:32.549 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":77159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/269054026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":269054026},{"type":"v1","addr":"172.21.15.196:6834","nonce":269054026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:19:20.698020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77162":{"gid":77162,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2259098413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2259098413},{"type":"v1","addr":"172.21.15.196:6835","nonce":2259098413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77162,"qdb_cluster":[77162]},"id":7}]} 2024-03-22T04:19:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 84 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd58067b20 msgr2=0x7fbd58069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd58067b20 0x7fbd58069fe0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0b1000 tx=0x7fbd6c008040 comp rx=0 tx=0).stop 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 msgr2=0x7fbd780b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780b2430 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000c780 tx=0x7fbd7000cc40 comp rx=0 tx=0).stop 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 shutdown_connections 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fbd58067b20 0x7fbd58069fe0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd780cc3e0 0x7fbd7815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd7815b3f0 0x7fbd780b2430 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 --2- 172.21.15.86:0/3425544472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd780ae540 0x7fbd7815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.548+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 >> 172.21.15.86:0/3425544472 conn(0x7fbd780b1df0 msgr2=0x7fbd781590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.552+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 shutdown_connections 2024-03-22T04:19:32.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.552+0000 7fbd7f800640 1 -- 172.21.15.86:0/3425544472 wait complete. 2024-03-22T04:19:32.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-22T04:19:32.735 INFO:teuthology.orchestra.run.smithi196.stdout:{"message":"","return_code":0} 2024-03-22T04:19:32.735 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-22T04:19:32.735 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:19:32.735 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:19:32.744 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:19:32.744 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:19:32.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:32.744 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:19:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-22T04:19:32.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:19:32.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-22T04:19:32.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:32.841 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-22T04:19:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.980+0000 7f9d6c000640 1 Processor -- start 2024-03-22T04:19:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.980+0000 7f9d6c000640 1 -- start start 2024-03-22T04:19:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.980+0000 7f9d6c000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9d640cfe90 0x7f9d640d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.980+0000 7f9d6c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d640ce8f0 con 0x7f9d640b0f90 2024-03-22T04:19:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.980+0000 7f9d6c000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d6415c890 con 0x7f9d6415c4b0 2024-03-22T04:19:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.980+0000 7f9d6c000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9d6409b620 con 0x7f9d640cfe90 2024-03-22T04:19:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d68c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9d640b0f90 0x7f9d640aecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33776/0 (socket says 172.21.15.86:33776) 2024-03-22T04:19:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d68c00640 1 -- 172.21.15.86:0/3635877656 learned_addr learned my addr 172.21.15.86:0/3635877656 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d6a000640 1 --2- 172.21.15.86:0/3635877656 >> v2:172.21.15.196:3300/0 conn(0x7f9d640cfe90 0x7f9d640d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d63e00640 1 -- 172.21.15.86:0/3635877656 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959052647 0 0) 0x7f9d640ce8f0 con 0x7f9d640b0f90 2024-03-22T04:19:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d63e00640 1 -- 172.21.15.86:0/3635877656 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d4c003530 con 0x7f9d640b0f90 2024-03-22T04:19:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d63e00640 1 -- 172.21.15.86:0/3635877656 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333308878 0 0) 0x7f9d6415c890 con 0x7f9d6415c4b0 2024-03-22T04:19:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d63e00640 1 -- 172.21.15.86:0/3635877656 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d640ce8f0 con 0x7f9d6415c4b0 2024-03-22T04:19:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d6a000640 1 -- 172.21.15.86:0/3635877656 >> v1:172.21.15.196:6789/0 conn(0x7f9d6415c4b0 legacy=0x7f9d640cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d6a000640 1 -- 172.21.15.86:0/3635877656 >> v1:172.21.15.86:6789/0 conn(0x7f9d640b0f90 legacy=0x7f9d640aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d6a000640 1 -- 172.21.15.86:0/3635877656 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6409b900 con 0x7f9d640cfe90 2024-03-22T04:19:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d6a000640 1 --2- 172.21.15.86:0/3635877656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d640cfe90 0x7f9d640d0690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009d90 tx=0x7f9d5c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f13e702eaf813de server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.984+0000 7f9d63e00640 1 -- 172.21.15.86:0/3635877656 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0b2070 con 0x7f9d640cfe90 2024-03-22T04:19:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d63e00640 1 -- 172.21.15.86:0/3635877656 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5c0a5910 con 0x7f9d640cfe90 2024-03-22T04:19:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d63e00640 1 -- 172.21.15.86:0/3635877656 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0a5ce0 con 0x7f9d640cfe90 2024-03-22T04:19:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- 172.21.15.86:0/3635877656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d640cfe90 msgr2=0x7f9d640d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 --2- 172.21.15.86:0/3635877656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d640cfe90 0x7f9d640d0690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009d90 tx=0x7f9d5c01c940 comp rx=0 tx=0).stop 2024-03-22T04:19:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- 172.21.15.86:0/3635877656 shutdown_connections 2024-03-22T04:19:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 --2- 172.21.15.86:0/3635877656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d640cfe90 0x7f9d640d0690 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- 172.21.15.86:0/3635877656 >> 172.21.15.86:0/3635877656 conn(0x7f9d640b3ba0 msgr2=0x7f9d640b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- 172.21.15.86:0/3635877656 shutdown_connections 2024-03-22T04:19:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- 172.21.15.86:0/3635877656 wait complete. 2024-03-22T04:19:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 Processor -- start 2024-03-22T04:19:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- start start 2024-03-22T04:19:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d640b0f90 0x7f9d640b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d640cfe90 0x7f9d641672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6415c4b0 0x7f9d641677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d640ccdb0 con 0x7f9d640b0f90 2024-03-22T04:19:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d640ccc30 con 0x7f9d6415c4b0 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6c000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d640af3d0 con 0x7f9d640cfe90 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d69600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d640b0f90 0x7f9d640b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d69600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d640b0f90 0x7f9d640b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52320/0 (socket says 172.21.15.86:52320) 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d69600640 1 -- 172.21.15.86:0/3653258166 learned_addr learned my addr 172.21.15.86:0/3653258166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d6a000640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6415c4b0 0x7f9d641677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.988+0000 7f9d68c00640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d640cfe90 0x7f9d641672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d69600640 1 -- 172.21.15.86:0/3653258166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d640cfe90 msgr2=0x7f9d641672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d69600640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d640cfe90 0x7f9d641672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d69600640 1 -- 172.21.15.86:0/3653258166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6415c4b0 msgr2=0x7f9d641677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d69600640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6415c4b0 0x7f9d641677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d69600640 1 -- 172.21.15.86:0/3653258166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d640b0a80 con 0x7f9d640b0f90 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d69600640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d640b0f90 0x7f9d640b2580 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c490 tx=0x7f9d5800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d61600640 1 -- 172.21.15.86:0/3653258166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580052d0 con 0x7f9d640b0f90 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d61600640 1 -- 172.21.15.86:0/3653258166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d58005470 con 0x7f9d640b0f90 2024-03-22T04:19:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d61600640 1 -- 172.21.15.86:0/3653258166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58012870 con 0x7f9d640b0f90 2024-03-22T04:19:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d6c000640 1 -- 172.21.15.86:0/3653258166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6406d380 con 0x7f9d640b0f90 2024-03-22T04:19:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d6c000640 1 -- 172.21.15.86:0/3653258166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d64166ab0 con 0x7f9d640b0f90 2024-03-22T04:19:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d61600640 1 -- 172.21.15.86:0/3653258166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9d58012a10 con 0x7f9d640b0f90 2024-03-22T04:19:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d61600640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d44067b20 0x7f9d44069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d61600640 1 -- 172.21.15.86:0/3653258166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f9d58085c00 con 0x7f9d640b0f90 2024-03-22T04:19:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.992+0000 7f9d68c00640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d44067b20 0x7f9d44069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 Processor -- start 2024-03-22T04:19:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- start start 2024-03-22T04:19:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d44067b20 0x7f9d44069fe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9d640b7af0 tx=0x7f9d50007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a0611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a017970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a018290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f119efe830 con 0x55f11a061730 2024-03-22T04:19:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f119e65e10 con 0x55f11a017eb0 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f119e02e50 con 0x55f11a060e10 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a017970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a017970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52328/0 (socket says 172.21.15.86:52328) 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 -- 172.21.15.86:0/149278197 learned_addr learned my addr 172.21.15.86:0/149278197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d69600640 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a0611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6a000640 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a018290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 -- 172.21.15.86:0/149278197 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 msgr2=0x55f11a0611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a0611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 -- 172.21.15.86:0/149278197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 msgr2=0x55f11a018290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a018290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 -- 172.21.15.86:0/149278197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f119deb510 con 0x55f11a061730 2024-03-22T04:19:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d68c00640 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a017970 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9d50007fd0 tx=0x7f9d50012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece78a0ff697ddc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d56000640 1 -- 172.21.15.86:0/149278197 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50026070 con 0x55f11a061730 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d56000640 1 -- 172.21.15.86:0/149278197 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5001b050 con 0x55f11a061730 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d56000640 1 -- 172.21.15.86:0/149278197 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50014bd0 con 0x55f11a061730 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/149278197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 msgr2=0x55f11a017970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a017970 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9d50007fd0 tx=0x7f9d50012000 comp rx=0 tx=0).stop 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/149278197 shutdown_connections 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a018290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a017970 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/149278197 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a0611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/149278197 >> 172.21.15.86:0/149278197 conn(0x55f11a013e60 msgr2=0x55f11a016250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/149278197 shutdown_connections 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:32.996+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/149278197 wait complete. 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 -- start start 2024-03-22T04:19:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 Processor -- start 2024-03-22T04:19:33.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a01d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a01d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a0221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f119e23640 con 0x55f11a061730 2024-03-22T04:19:33.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f119e61dc0 con 0x55f11a017eb0 2024-03-22T04:19:33.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f119e0e0b0 con 0x55f11a060e10 2024-03-22T04:19:33.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 10 client.0 _ll_register_callbacks cb 0x55f11a002d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:19:33.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a0221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a0221f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52332/0 (socket says 172.21.15.86:52332) 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 -- 172.21.15.86:0/1304052384 learned_addr learned my addr 172.21.15.86:0/1304052384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d69600640 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a01d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d68c00640 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a01d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 -- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 msgr2=0x55f11a01d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a01d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 -- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 msgr2=0x55f11a01d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a01d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 -- 172.21.15.86:0/1304052384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f119e23a40 con 0x55f11a061730 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d69600640 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a01d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6a000640 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a0221f0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c01a570 tx=0x7f9d5c01b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe10a1def3a7be34 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:19:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d4aa00640 1 -- 172.21.15.86:0/1304052384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0b2040 con 0x55f11a061730 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d4aa00640 1 -- 172.21.15.86:0/1304052384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f9d5c0a5a10 con 0x55f11a061730 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/1304052384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5001b050 con 0x55f11a061730 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d4aa00640 1 -- 172.21.15.86:0/1304052384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f9d5c0cc050 con 0x55f11a061730 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 10 client.8595 fetch_fsmap learned FSMap version 84 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/1304052384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55f119e1e370 con 0x55f11a061730 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d4aa00640 1 -- 172.21.15.86:0/1304052384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5001b050 con 0x55f11a061730 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d4aa00640 1 -- 172.21.15.86:0/1304052384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f9d5c0a5c10 con 0x55f11a061730 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 10 client.8595 fetch_fsmap finished waiting for FSMap version 84 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 10 client.8595 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 20 client.8595 populate_metadata read hostname 'smithi086' 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55f11a0700b0 0x55f11a0724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 4 client.8595 mds_command: new command op to 77162 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/1304052384 --> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f119e00ed0 con 0x55f11a0700b0 2024-03-22T04:19:33.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.000+0000 7f9d69600640 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55f11a0700b0 0x55f11a0724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d4aa00640 10 client.8595 ms_handle_connect on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d69600640 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55f11a0700b0 0x55f11a0724a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1844b9be2eea871d server_cookie=e43eaebe8d8dbad3 in_seq=0 out_seq=0 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d4aa00640 1 -- 172.21.15.86:0/1304052384 <== mds.0 v2:172.21.15.196:6833/2259098413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f119e00ed0 con 0x55f11a0700b0 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d4aa00640 10 client.8595 handle_command_reply: tid=0 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 client.8595 shutdown 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/1304052384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 msgr2=0x55f11a0221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a0221f0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c01a570 tx=0x7f9d5c01b180 comp rx=0 tx=0).stop 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/1304052384 shutdown_connections 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55f11a0700b0 0x55f11a0724a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a061730 0x55f11a0221f0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a01d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d4aa00640 0 client.8595 ms_handle_reset on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:33.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/1304052384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a017eb0 0x55f11a01d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/1304052384 >> 172.21.15.86:0/1304052384 conn(0x55f11a013e60 msgr2=0x55f11a016ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/1304052384 shutdown_connections 2024-03-22T04:19:33.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/1304052384 wait complete. 2024-03-22T04:19:33.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.004+0000 7f9d6f4e9000 20 client.8595 trim_cache size 0 max 16384 2024-03-22T04:19:33.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f5190600640 1 Processor -- start 2024-03-22T04:19:33.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f5190600640 1 -- start start 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f5190600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f518809acd0 0x7f518806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f5190600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 0x7f51880b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f5190600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518806ded0 con 0x7f518809b0b0 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f5190600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f51880ccb60 con 0x7f518809acd0 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f5190600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f518806e1b0 con 0x7f518815eb90 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f518dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f518809acd0 0x7f518806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f518d200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f518809b0b0 0x7f518815e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33790/0 (socket says 172.21.15.86:33790) 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f518d200640 1 -- 172.21.15.86:0/1224505358 learned_addr learned my addr 172.21.15.86:0/1224505358 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f518e600640 1 --2- 172.21.15.86:0/1224505358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 0x7f51880b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f518dc00640 1 -- 172.21.15.86:0/1224505358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 msgr2=0x7f51880b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f518dc00640 1 --2- 172.21.15.86:0/1224505358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 0x7f51880b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f518dc00640 1 -- 172.21.15.86:0/1224505358 >> v1:172.21.15.86:6789/0 conn(0x7f518809b0b0 legacy=0x7f518815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.008+0000 7f518dc00640 1 -- 172.21.15.86:0/1224505358 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518806e490 con 0x7f518809acd0 2024-03-22T04:19:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f518dc00640 1 --2- 172.21.15.86:0/1224505358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518809acd0 0x7f518806d8f0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f517c0097c0 tx=0x7f517c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ae2c3402fb3440 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5187e00640 1 -- 172.21.15.86:0/1224505358 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0b1070 con 0x7f518809acd0 2024-03-22T04:19:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5187e00640 1 -- 172.21.15.86:0/1224505358 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c0ac600 con 0x7f518809acd0 2024-03-22T04:19:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5187e00640 1 -- 172.21.15.86:0/1224505358 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0ac9d0 con 0x7f518809acd0 2024-03-22T04:19:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 -- 172.21.15.86:0/1224505358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518809acd0 msgr2=0x7f518806d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 --2- 172.21.15.86:0/1224505358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518809acd0 0x7f518806d8f0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f517c0097c0 tx=0x7f517c01c380 comp rx=0 tx=0).stop 2024-03-22T04:19:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 -- 172.21.15.86:0/1224505358 shutdown_connections 2024-03-22T04:19:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 --2- 172.21.15.86:0/1224505358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 0x7f51880b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 --2- 172.21.15.86:0/1224505358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518809acd0 0x7f518806d8f0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 -- 172.21.15.86:0/1224505358 >> 172.21.15.86:0/1224505358 conn(0x7f51880b3bf0 msgr2=0x7f51880b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 -- 172.21.15.86:0/1224505358 shutdown_connections 2024-03-22T04:19:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 -- 172.21.15.86:0/1224505358 wait complete. 2024-03-22T04:19:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.012+0000 7f5190600640 1 Processor -- start 2024-03-22T04:19:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 -- start start 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518809acd0 0x7f51881521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518809b0b0 0x7f51880afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 0x7f51880b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51880cd980 con 0x7f518809acd0 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51880cd800 con 0x7f518809b0b0 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f51880cdb00 con 0x7f518815eb90 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518809acd0 0x7f51881521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518809acd0 0x7f51881521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52348/0 (socket says 172.21.15.86:52348) 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 -- 172.21.15.86:0/487596215 learned_addr learned my addr 172.21.15.86:0/487596215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518e600640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 0x7f51880b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518d200640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518809b0b0 0x7f51880afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 -- 172.21.15.86:0/487596215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 msgr2=0x7f51880b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518815eb90 0x7f51880b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 -- 172.21.15.86:0/487596215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518809b0b0 msgr2=0x7f51880afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518809b0b0 0x7f51880afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 -- 172.21.15.86:0/487596215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51880ca510 con 0x7f518809acd0 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518d200640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518809b0b0 0x7f51880afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f518dc00640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518809acd0 0x7f51881521d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f517c01aa30 tx=0x7f517c01abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0bb020 con 0x7f518809acd0 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c0b7620 con 0x7f518809acd0 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 -- 172.21.15.86:0/487596215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51880ca7f0 con 0x7f518809acd0 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0b1040 con 0x7f518809acd0 2024-03-22T04:19:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 -- 172.21.15.86:0/487596215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5188158770 con 0x7f518809acd0 2024-03-22T04:19:33.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.016+0000 7f5190600640 1 -- 172.21.15.86:0/487596215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5188074470 con 0x7f518809acd0 2024-03-22T04:19:33.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.020+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f517c0be020 con 0x7f518809acd0 2024-03-22T04:19:33.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.020+0000 7f5185600640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5168067b20 0x7f5168069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.020+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f517c1209a0 con 0x7f518809acd0 2024-03-22T04:19:33.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.020+0000 7f518d200640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5168067b20 0x7f5168069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.020+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7f517c0c5030 con 0x7f518809acd0 2024-03-22T04:19:33.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.020+0000 7f5190600640 1 -- 172.21.15.86:0/487596215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f51880b23e0 con 0x7f518809acd0 2024-03-22T04:19:33.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.020+0000 7f518d200640 1 --2- 172.21.15.86:0/487596215 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5168067b20 0x7f5168069fe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5174000f70 tx=0x7f5174009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 Processor -- start 2024-03-22T04:19:33.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- start start 2024-03-22T04:19:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a017eb0 0x55f11a017260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a060e10 0x55f11a0177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 0x55f11a0724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d642040a0 con 0x55f11a060e10 2024-03-22T04:19:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d642030a0 con 0x55f11a061730 2024-03-22T04:19:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d64166200 con 0x55f11a017eb0 2024-03-22T04:19:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a060e10 0x55f11a0177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a060e10 0x55f11a0177a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52364/0 (socket says 172.21.15.86:52364) 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 -- 172.21.15.86:0/510863309 learned_addr learned my addr 172.21.15.86:0/510863309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d69600640 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a017eb0 0x55f11a017260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6a000640 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 0x55f11a0724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 -- 172.21.15.86:0/510863309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a017eb0 msgr2=0x55f11a017260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a017eb0 0x55f11a017260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 -- 172.21.15.86:0/510863309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 msgr2=0x55f11a0724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 0x55f11a0724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 -- 172.21.15.86:0/510863309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f119e030a0 con 0x55f11a060e10 2024-03-22T04:19:33.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d68c00640 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a060e10 0x55f11a0177a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9d50006000 tx=0x7f9d50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9aa35d144c1954a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d55600640 1 -- 172.21.15.86:0/510863309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50032070 con 0x55f11a060e10 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d55600640 1 -- 172.21.15.86:0/510863309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d50005860 con 0x55f11a060e10 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d55600640 1 -- 172.21.15.86:0/510863309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50026040 con 0x55f11a060e10 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/510863309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a060e10 msgr2=0x55f11a0177a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a060e10 0x55f11a0177a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9d50006000 tx=0x7f9d50014000 comp rx=0 tx=0).stop 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/510863309 shutdown_connections 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 0x55f11a0724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a060e10 0x55f11a0177a0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/510863309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a017eb0 0x55f11a017260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/510863309 >> 172.21.15.86:0/510863309 conn(0x55f11a013e60 msgr2=0x55f11a05fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/510863309 shutdown_connections 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.024+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/510863309 wait complete. 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 -- start start 2024-03-22T04:19:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 Processor -- start 2024-03-22T04:19:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a017eb0 0x55f11a0750c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 0x55f11a0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f11a013ce0 con 0x55f11a017eb0 2024-03-22T04:19:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6414ef40 con 0x55f11a061730 2024-03-22T04:19:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f119e65120 con 0x55f11a060e10 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 10 client.0 _ll_register_callbacks cb 0x7f9d4c0048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a017eb0 0x55f11a0750c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a017eb0 0x55f11a0750c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52366/0 (socket says 172.21.15.86:52366) 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 -- 172.21.15.86:0/2947266778 learned_addr learned my addr 172.21.15.86:0/2947266778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d68c00640 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6a000640 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 0x55f11a0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 -- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 msgr2=0x55f11a0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a0d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 -- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 msgr2=0x55f11a0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 0x55f11a0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 -- 172.21.15.86:0/2947266778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f119ee7840 con 0x55f11a017eb0 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d69600640 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a017eb0 0x55f11a0750c0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9d58055dc0 tx=0x7f9d58000ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dff633efbbd72c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:19:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d4aa00640 1 -- 172.21.15.86:0/2947266778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58055280 con 0x55f11a017eb0 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d4aa00640 1 -- 172.21.15.86:0/2947266778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f9d58055420 con 0x55f11a017eb0 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/2947266778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d50005860 con 0x55f11a017eb0 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d4aa00640 1 -- 172.21.15.86:0/2947266778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f9d58074390 con 0x55f11a017eb0 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 10 client.8610 fetch_fsmap learned FSMap version 84 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/2947266778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55f119b21470 con 0x55f11a017eb0 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d4aa00640 1 -- 172.21.15.86:0/2947266778 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d50005860 con 0x55f11a017eb0 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d4aa00640 1 -- 172.21.15.86:0/2947266778 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f9d5805c0d0 con 0x55f11a017eb0 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 10 client.8610 fetch_fsmap finished waiting for FSMap version 84 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 10 client.8610 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 20 client.8610 populate_metadata read hostname 'smithi086' 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55f11a0d1350 0x55f11a0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 4 client.8610 mds_command: new command op to 77162 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/2947266778 --> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55f119e3f6b0 con 0x55f11a0d1350 2024-03-22T04:19:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.028+0000 7f9d68c00640 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55f11a0d1350 0x55f11a0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d68c00640 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55f11a0d1350 0x55f11a0cd280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc87c07076ba0098 server_cookie=31205d0bc2f8bf29 in_seq=0 out_seq=0 2024-03-22T04:19:33.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d4aa00640 10 client.8610 ms_handle_connect on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:33.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d4aa00640 1 -- 172.21.15.86:0/2947266778 <== mds.0 v2:172.21.15.196:6833/2259098413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f119e3f6b0 con 0x55f11a0d1350 2024-03-22T04:19:33.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d4aa00640 10 client.8610 handle_command_reply: tid=0 2024-03-22T04:19:33.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 client.8610 shutdown 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/2947266778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a017eb0 msgr2=0x55f11a0750c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a017eb0 0x55f11a0750c0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9d58055dc0 tx=0x7f9d58000ff0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/2947266778 shutdown_connections 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55f11a0d1350 0x55f11a0cd280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f11a061730 0x55f11a0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f11a060e10 0x55f11a0d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 --2- 172.21.15.86:0/2947266778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f11a017eb0 0x55f11a0750c0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d4aa00640 0 client.8610 ms_handle_reset on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/2947266778 >> 172.21.15.86:0/2947266778 conn(0x55f11a013e60 msgr2=0x55f11a060110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/2947266778 shutdown_connections 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 1 -- 172.21.15.86:0/2947266778 wait complete. 2024-03-22T04:19:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6f4e9000 20 client.8610 trim_cache size 0 max 16384 2024-03-22T04:19:33.036 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:19:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:19:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "d312fd94-a9cc-4ffe-84bf-8d0fa4994c1c", 2024-03-22T04:19:33.036 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:19:33.036 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 -- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d44067b20 msgr2=0x7f9d44069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d44067b20 0x7f9d44069fe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9d640b7af0 tx=0x7f9d50007d40 comp rx=0 tx=0).stop 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 -- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d640b0f90 msgr2=0x7f9d640b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d640b0f90 0x7f9d640b2580 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c490 tx=0x7f9d5800c950 comp rx=0 tx=0).stop 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 -- 172.21.15.86:0/3653258166 shutdown_connections 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9d44067b20 0x7f9d44069fe0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6415c4b0 0x7f9d641677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d640cfe90 0x7f9d641672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 --2- 172.21.15.86:0/3653258166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d640b0f90 0x7f9d640b2580 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 -- 172.21.15.86:0/3653258166 >> 172.21.15.86:0/3653258166 conn(0x7f9d640b3ba0 msgr2=0x7f9d640aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 -- 172.21.15.86:0/3653258166 shutdown_connections 2024-03-22T04:19:33.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.032+0000 7f9d6c000640 1 -- 172.21.15.86:0/3653258166 wait complete. 2024-03-22T04:19:33.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:19:33.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-22T04:19:33.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ee00640 1 Processor -- start 2024-03-22T04:19:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ee00640 1 -- start start 2024-03-22T04:19:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ee00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f45180b2360 0x7f45180b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 0x7f451815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f451815cee0 0x7f451806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f45180b2740 con 0x7f451815cee0 2024-03-22T04:19:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ee00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f451815d710 con 0x7f45180b2360 2024-03-22T04:19:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f451815fe40 con 0x7f45180b3430 2024-03-22T04:19:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f451815cee0 0x7f451806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f451815cee0 0x7f451806d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52376/0 (socket says 172.21.15.86:52376) 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ce00640 1 -- 172.21.15.86:0/1059492402 learned_addr learned my addr 172.21.15.86:0/1059492402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f4517e00640 1 --2- 172.21.15.86:0/1059492402 >> v2:172.21.15.196:3300/0 conn(0x7f45180b2360 0x7f45180b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f450f400640 1 --2- 172.21.15.86:0/1059492402 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 0x7f451815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ce00640 1 -- 172.21.15.86:0/1059492402 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 msgr2=0x7f451815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ce00640 1 --2- 172.21.15.86:0/1059492402 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 0x7f451815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ce00640 1 -- 172.21.15.86:0/1059492402 >> v2:172.21.15.196:3300/0 conn(0x7f45180b2360 msgr2=0x7f45180b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ce00640 1 --2- 172.21.15.86:0/1059492402 >> v2:172.21.15.196:3300/0 conn(0x7f45180b2360 0x7f45180b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.188+0000 7f451ce00640 1 -- 172.21.15.86:0/1059492402 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4518160120 con 0x7f451815cee0 2024-03-22T04:19:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ce00640 1 --2- 172.21.15.86:0/1059492402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f451806d6d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f45180683e0 tx=0x7f451001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca262921f644488 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f4517400640 1 -- 172.21.15.86:0/1059492402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100a64b0 con 0x7f451815cee0 2024-03-22T04:19:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f4517400640 1 -- 172.21.15.86:0/1059492402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45100a6650 con 0x7f451815cee0 2024-03-22T04:19:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f4517400640 1 -- 172.21.15.86:0/1059492402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100a6a20 con 0x7f451815cee0 2024-03-22T04:19:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 -- 172.21.15.86:0/1059492402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 msgr2=0x7f451806d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 --2- 172.21.15.86:0/1059492402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f451806d6d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f45180683e0 tx=0x7f451001c730 comp rx=0 tx=0).stop 2024-03-22T04:19:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 -- 172.21.15.86:0/1059492402 shutdown_connections 2024-03-22T04:19:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 --2- 172.21.15.86:0/1059492402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f451806d6d0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 --2- 172.21.15.86:0/1059492402 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 0x7f451815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 --2- 172.21.15.86:0/1059492402 >> v2:172.21.15.196:3300/0 conn(0x7f45180b2360 0x7f45180b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 -- 172.21.15.86:0/1059492402 >> 172.21.15.86:0/1059492402 conn(0x7f45180cc940 msgr2=0x7f45180ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 -- 172.21.15.86:0/1059492402 shutdown_connections 2024-03-22T04:19:33.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.192+0000 7f451ee00640 1 -- 172.21.15.86:0/1059492402 wait complete. 2024-03-22T04:19:33.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 Processor -- start 2024-03-22T04:19:33.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 -- start start 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45180b2360 0x7f451814ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 0x7f45181fad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f45181373e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f451815d4d0 con 0x7f451815cee0 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f451815d350 con 0x7f45180b2360 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f45180b4d80 con 0x7f45180b3430 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f45181373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f4517e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45180b2360 0x7f451814ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f45181373e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52390/0 (socket says 172.21.15.86:52390) 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 -- 172.21.15.86:0/425041049 learned_addr learned my addr 172.21.15.86:0/425041049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f450f400640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 0x7f45181fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 -- 172.21.15.86:0/425041049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 msgr2=0x7f45181fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 0x7f45181fad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 -- 172.21.15.86:0/425041049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45180b2360 msgr2=0x7f451814ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45180b2360 0x7f451814ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 -- 172.21.15.86:0/425041049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45181fb280 con 0x7f451815cee0 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f4517e00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45180b2360 0x7f451814ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ce00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f45181373e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f451001ade0 tx=0x7f45100194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f4514c00640 1 -- 172.21.15.86:0/425041049 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100a6a00 con 0x7f451815cee0 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f4514c00640 1 -- 172.21.15.86:0/425041049 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45100a6ba0 con 0x7f451815cee0 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 -- 172.21.15.86:0/425041049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4518137920 con 0x7f451815cee0 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f4514c00640 1 -- 172.21.15.86:0/425041049 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100af480 con 0x7f451815cee0 2024-03-22T04:19:33.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.196+0000 7f451ee00640 1 -- 172.21.15.86:0/425041049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45181fb500 con 0x7f451815cee0 2024-03-22T04:19:33.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.200+0000 7f4514c00640 1 -- 172.21.15.86:0/425041049 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f45100af620 con 0x7f451815cee0 2024-03-22T04:19:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.200+0000 7f4514c00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f4067b20 0x7f44f4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.200+0000 7f4514c00640 1 -- 172.21.15.86:0/425041049 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f4510120390 con 0x7f451815cee0 2024-03-22T04:19:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.200+0000 7f4517e00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f4067b20 0x7f44f4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.200+0000 7f452232d000 1 Processor -- start 2024-03-22T04:19:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.200+0000 7f452232d000 1 -- start start 2024-03-22T04:19:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.200+0000 7f4517e00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f4067b20 0x7f44f4069fe0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4508009f50 tx=0x7f4508009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b414360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b3c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55586b218c40 con 0x55586b3c7f50 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55586b1b5d90 con 0x55586b413f80 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55586b214bf0 con 0x55586b4148a0 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b3c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b3c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52398/0 (socket says 172.21.15.86:52398) 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 -- 172.21.15.86:0/1921124262 learned_addr learned my addr 172.21.15.86:0/1921124262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f4517e00640 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b414360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f450f400640 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 -- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 msgr2=0x55586b3c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 -- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 msgr2=0x55586b414360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b414360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 -- 172.21.15.86:0/1921124262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55586b2c0130 con 0x55586b3c7f50 2024-03-22T04:19:33.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f4517e00640 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b414360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:33.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f451ce00640 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b3c8330 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f45100093e0 tx=0x7f45100efe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92986b0a5ed7b273 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f450cc00640 1 -- 172.21.15.86:0/1921124262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100f45c0 con 0x55586b3c7f50 2024-03-22T04:19:33.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f450cc00640 1 -- 172.21.15.86:0/1921124262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45100f8df0 con 0x55586b3c7f50 2024-03-22T04:19:33.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f450cc00640 1 -- 172.21.15.86:0/1921124262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45101200b0 con 0x55586b3c7f50 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- 172.21.15.86:0/1921124262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 msgr2=0x55586b3c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b3c8330 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f45100093e0 tx=0x7f45100efe80 comp rx=0 tx=0).stop 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- 172.21.15.86:0/1921124262 shutdown_connections 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b3c8330 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- 172.21.15.86:0/1921124262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b414360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- 172.21.15.86:0/1921124262 >> 172.21.15.86:0/1921124262 conn(0x55586b2b1990 msgr2=0x55586b3f7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- 172.21.15.86:0/1921124262 shutdown_connections 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- 172.21.15.86:0/1921124262 wait complete. 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- start start 2024-03-22T04:19:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 Processor -- start 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b3cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b3d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 0x55586b3f8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55586b1d4710 con 0x55586b413f80 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55586b1d6580 con 0x55586b4148a0 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55586b21d270 con 0x55586b3c7f50 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f452232d000 10 client.0 _ll_register_callbacks cb 0x55586b27a200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f450f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b3d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.204+0000 7f4517e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b3cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f451ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 0x55586b3f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f450f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b3d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52410/0 (socket says 172.21.15.86:52410) 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f450f400640 1 -- 172.21.15.86:0/2977981984 learned_addr learned my addr 172.21.15.86:0/2977981984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f450f400640 1 -- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 msgr2=0x55586b3cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f450f400640 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b3cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f450f400640 1 -- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 msgr2=0x55586b3f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f450f400640 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 0x55586b3f8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f450f400640 1 -- 172.21.15.86:0/2977981984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55586b1f1d30 con 0x55586b413f80 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f450f400640 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b3d0170 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4500014610 tx=0x7f4500014ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3fe9b4b514af1e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 1 -- 172.21.15.86:0/2977981984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500004300 con 0x55586b413f80 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 1 -- 172.21.15.86:0/2977981984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f4500003620 con 0x55586b413f80 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 -- 172.21.15.86:0/2977981984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45100f8df0 con 0x55586b413f80 2024-03-22T04:19:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 1 -- 172.21.15.86:0/2977981984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f4500027050 con 0x55586b413f80 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 10 client.8622 fetch_fsmap learned FSMap version 84 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 -- 172.21.15.86:0/2977981984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55586b25dfc0 con 0x55586b413f80 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 1 -- 172.21.15.86:0/2977981984 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45100f8df0 con 0x55586b413f80 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 1 -- 172.21.15.86:0/2977981984 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f450002a020 con 0x55586b413f80 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 10 client.8622 fetch_fsmap finished waiting for FSMap version 84 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 10 client.8622 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 20 client.8622 populate_metadata read hostname 'smithi086' 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55586b423220 0x55586b425610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 4 client.8622 mds_command: new command op to 77162 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 -- 172.21.15.86:0/2977981984 --> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55586b1b3dd0 con 0x55586b423220 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4517e00640 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55586b423220 0x55586b425610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4517e00640 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55586b423220 0x55586b425610 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7336311488c96bd9 server_cookie=b29cd1d560b7ac9e in_seq=0 out_seq=0 2024-03-22T04:19:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 10 client.8622 ms_handle_connect on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 1 -- 172.21.15.86:0/2977981984 <== mds.0 v2:172.21.15.196:6833/2259098413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55586b1b3dd0 con 0x55586b423220 2024-03-22T04:19:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 10 client.8622 handle_command_reply: tid=0 2024-03-22T04:19:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 client.8622 shutdown 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 -- 172.21.15.86:0/2977981984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 msgr2=0x55586b3d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b3d0170 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4500014610 tx=0x7f4500014ad0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 -- 172.21.15.86:0/2977981984 shutdown_connections 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55586b423220 0x55586b425610 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 0x55586b3f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f4505600640 0 client.8622 ms_handle_reset on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b3d0170 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 --2- 172.21.15.86:0/2977981984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b3cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 -- 172.21.15.86:0/2977981984 >> 172.21.15.86:0/2977981984 conn(0x55586b2b1990 msgr2=0x55586b3c7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 -- 172.21.15.86:0/2977981984 shutdown_connections 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 1 -- 172.21.15.86:0/2977981984 wait complete. 2024-03-22T04:19:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.208+0000 7f452232d000 20 client.8622 trim_cache size 0 max 16384 2024-03-22T04:19:33.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 Processor -- start 2024-03-22T04:19:33.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- start start 2024-03-22T04:19:33.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b428640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b3cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4518203db0 con 0x55586b3c7f50 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f451815e710 con 0x55586b413f80 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f451815da70 con 0x55586b4148a0 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b428640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b428640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52422/0 (socket says 172.21.15.86:52422) 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 -- 172.21.15.86:0/380629571 learned_addr learned my addr 172.21.15.86:0/380629571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f450f400640 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b3cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f451ce00640 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 -- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 msgr2=0x55586b3cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 -- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 msgr2=0x55586b3cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b3cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 -- 172.21.15.86:0/380629571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55586b1ee3a0 con 0x55586b3c7f50 2024-03-22T04:19:33.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f451ce00640 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:33.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4517e00640 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b428640 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4508004220 tx=0x7f4508018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294814bf517469da server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4507e00640 1 -- 172.21.15.86:0/380629571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508032070 con 0x55586b3c7f50 2024-03-22T04:19:33.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4507e00640 1 -- 172.21.15.86:0/380629571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450801e040 con 0x55586b3c7f50 2024-03-22T04:19:33.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f4507e00640 1 -- 172.21.15.86:0/380629571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450801d050 con 0x55586b3c7f50 2024-03-22T04:19:33.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- 172.21.15.86:0/380629571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 msgr2=0x55586b428640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b428640 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4508004220 tx=0x7f4508018000 comp rx=0 tx=0).stop 2024-03-22T04:19:33.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- 172.21.15.86:0/380629571 shutdown_connections 2024-03-22T04:19:33.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b413f80 0x55586b3cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b4148a0 0x55586b3cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 --2- 172.21.15.86:0/380629571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b3c7f50 0x55586b428640 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- 172.21.15.86:0/380629571 >> 172.21.15.86:0/380629571 conn(0x55586b2b1990 msgr2=0x55586b3c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- 172.21.15.86:0/380629571 shutdown_connections 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- 172.21.15.86:0/380629571 wait complete. 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 -- start start 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.228+0000 7f452232d000 1 Processor -- start 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b47fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b427930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 0x55586b427e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55586b2d4810 con 0x55586b413f80 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f451815e0c0 con 0x55586b4148a0 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55586b217f50 con 0x55586b3c7f50 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 10 client.0 _ll_register_callbacks cb 0x7f4500009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b427930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b427930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52438/0 (socket says 172.21.15.86:52438) 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 -- 172.21.15.86:0/2628829191 learned_addr learned my addr 172.21.15.86:0/2628829191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4517e00640 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b47fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f451ce00640 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 0x55586b427e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 -- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 msgr2=0x55586b47fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b47fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 -- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 msgr2=0x55586b427e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 0x55586b427e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 -- 172.21.15.86:0/2628829191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55586b19e380 con 0x55586b413f80 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4517e00640 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b47fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f450f400640 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b427930 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4500003850 tx=0x7f45000046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ff34e809d9506f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 1 -- 172.21.15.86:0/2628829191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450002f070 con 0x55586b413f80 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 1 -- 172.21.15.86:0/2628829191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f45000129e0 con 0x55586b413f80 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- 172.21.15.86:0/2628829191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f450801e040 con 0x55586b413f80 2024-03-22T04:19:33.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 1 -- 172.21.15.86:0/2628829191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f450003a050 con 0x55586b413f80 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 10 client.8628 fetch_fsmap learned FSMap version 84 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- 172.21.15.86:0/2628829191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55586b29a460 con 0x55586b413f80 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 1 -- 172.21.15.86:0/2628829191 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450801e040 con 0x55586b413f80 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 1 -- 172.21.15.86:0/2628829191 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f450003d020 con 0x55586b413f80 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 10 client.8628 fetch_fsmap finished waiting for FSMap version 84 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 10 client.8628 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 20 client.8628 populate_metadata read hostname 'smithi086' 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55586b4283b0 0x55586b482ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 4 client.8628 mds_command: new command op to 77162 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- 172.21.15.86:0/2628829191 --> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55586b1b8160 con 0x55586b4283b0 2024-03-22T04:19:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4517e00640 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55586b4283b0 0x55586b482ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:19:33.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4517e00640 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55586b4283b0 0x55586b482ea0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607be319c9075d15 server_cookie=1d5c6e264915ea66 in_seq=0 out_seq=0 2024-03-22T04:19:33.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 10 client.8628 ms_handle_connect on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:33.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 1 -- 172.21.15.86:0/2628829191 <== mds.0 v2:172.21.15.196:6833/2259098413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55586b1b8160 con 0x55586b4283b0 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 10 client.8628 handle_command_reply: tid=0 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 client.8628 shutdown 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- 172.21.15.86:0/2628829191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 msgr2=0x55586b427930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b427930 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4500003850 tx=0x7f45000046d0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- 172.21.15.86:0/2628829191 shutdown_connections 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55586b4283b0 0x55586b482ea0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55586b4148a0 0x55586b427e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55586b413f80 0x55586b427930 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 --2- 172.21.15.86:0/2628829191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55586b3c7f50 0x55586b47fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f4505600640 0 client.8628 ms_handle_reset on v2:172.21.15.196:6833/2259098413 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- 172.21.15.86:0/2628829191 >> 172.21.15.86:0/2628829191 conn(0x55586b2b1990 msgr2=0x55586b3ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- 172.21.15.86:0/2628829191 shutdown_connections 2024-03-22T04:19:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 1 -- 172.21.15.86:0/2628829191 wait complete. 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.232+0000 7f452232d000 20 client.8628 trim_cache size 0 max 16384 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "d312fd94-a9cc-4ffe-84bf-8d0fa4994c1c": { 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "d312fd94-a9cc-4ffe-84bf-8d0fa4994c1c", 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:19:33.239 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:19:33.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 -- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f4067b20 msgr2=0x7f44f4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f4067b20 0x7f44f4069fe0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4508009f50 tx=0x7f4508009190 comp rx=0 tx=0).stop 2024-03-22T04:19:33.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 -- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 msgr2=0x7f45181373e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:19:33.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f45181373e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f451001ade0 tx=0x7f45100194d0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 -- 172.21.15.86:0/425041049 shutdown_connections 2024-03-22T04:19:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f4067b20 0x7f44f4069fe0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451815cee0 0x7f45181373e0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45180b3430 0x7f45181fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 --2- 172.21.15.86:0/425041049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45180b2360 0x7f451814ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:19:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 -- 172.21.15.86:0/425041049 >> 172.21.15.86:0/425041049 conn(0x7f45180cc940 msgr2=0x7f451815b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:19:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 -- 172.21.15.86:0/425041049 shutdown_connections 2024-03-22T04:19:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:33.236+0000 7f451ee00640 1 -- 172.21.15.86:0/425041049 wait complete. 2024-03-22T04:19:33.253 INFO:tasks.cephfs.filesystem:scrub status for tag:d312fd94-a9cc-4ffe-84bf-8d0fa4994c1c - {'path': '/', 'tag': 'd312fd94-a9cc-4ffe-84bf-8d0fa4994c1c', 'options': 'recursive,repair'} 2024-03-22T04:19:34.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:34.396+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 7 ==== log(4 entries from seq 1 at 2024-03-22T04:19:33.034897+0000) ==== 695+0+0 (secure 0 0 0) 0x7f517c121000 con 0x7f518809acd0 2024-03-22T04:19:34.397 INFO:teuthology.orchestra.run.smithi086.stdout: cluster: 2024-03-22T04:19:34.397 INFO:teuthology.orchestra.run.smithi086.stdout: id: 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:19:34.397 INFO:teuthology.orchestra.run.smithi086.stdout: health: HEALTH_OK 2024-03-22T04:19:34.397 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: services: 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: mgr: y(active, since 9m), standbys: x 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: mds: 1/1 daemons up, 3 standby 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: data: 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: volumes: 1/1 healthy 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: pools: 3 pools, 97 pgs 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: objects: 28 objects, 13 MiB 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: usage: 281 MiB used, 720 GiB / 720 GiB avail 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: pgs: 97 active+clean 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:34.398 INFO:teuthology.orchestra.run.smithi086.stdout: io: 2024-03-22T04:19:34.399 INFO:teuthology.orchestra.run.smithi086.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 6 op/s wr 2024-03-22T04:19:34.399 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:34.399 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:19:34.399 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:19:33.034897+0000 mds.b [INF] scrub queued for path: / 2024-03-22T04:19:34.399 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:19:33.034902+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-03-22T04:19:34.399 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:19:33.034909+0000 mds.b [INF] scrub summary: active paths [/] 2024-03-22T04:19:34.399 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:19:33.036336+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-03-22T04:19:38.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:19:38.968+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 8 ==== log(1 entries from seq 5 at 2024-03-22T04:19:37.448130+0000) ==== 189+0+0 (secure 0 0 0) 0x7f517c0f70c0 con 0x7f518809acd0 2024-03-22T04:19:38.969 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:19:37.448130+0000 mds.b [INF] scrub summary: idle 2024-03-22T04:19:58.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:19:58.928 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:20:00.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:00.980+0000 7f5185600640 1 -- 172.21.15.86:0/487596215 <== mon.0 v2:172.21.15.86:3300/0 9 ==== log(1 entries from seq 1200 at 2024-03-22T04:20:00.000197+0000) ==== 187+0+0 (secure 0 0 0) 0x7f517c0ac580 con 0x7f518809acd0 2024-03-22T04:20:00.979 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:00.000197+0000 mon.a [INF] overall HEALTH_OK 2024-03-22T04:20:03.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:20:03.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-22T04:20:03.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2199400640 1 Processor -- start 2024-03-22T04:20:03.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2199400640 1 -- start start 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2199400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2194145b90 0x7f21940d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2199400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2194072460 0x7f21940728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2199400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2194145f70 con 0x7f21940d0aa0 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2199400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2194160850 con 0x7f2194145b90 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2199400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2194160b30 con 0x7f2194072460 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2191e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f21940d0aa0 0x7f219415fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51738/0 (socket says 172.21.15.86:51738) 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2191e00640 1 -- 172.21.15.86:0/1579320929 learned_addr learned my addr 172.21.15.86:0/1579320929 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2192800640 1 --2- 172.21.15.86:0/1579320929 >> v2:172.21.15.196:3300/0 conn(0x7f2194145b90 0x7f21940d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2193200640 1 --2- 172.21.15.86:0/1579320929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2194072460 0x7f21940728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2191400640 1 -- 172.21.15.86:0/1579320929 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483682792 0 0) 0x7f2194145f70 con 0x7f21940d0aa0 2024-03-22T04:20:03.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2191400640 1 -- 172.21.15.86:0/1579320929 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2178003530 con 0x7f21940d0aa0 2024-03-22T04:20:03.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2193200640 1 -- 172.21.15.86:0/1579320929 >> v2:172.21.15.196:3300/0 conn(0x7f2194145b90 msgr2=0x7f21940d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2193200640 1 --2- 172.21.15.86:0/1579320929 >> v2:172.21.15.196:3300/0 conn(0x7f2194145b90 0x7f21940d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2193200640 1 -- 172.21.15.86:0/1579320929 >> v1:172.21.15.86:6789/0 conn(0x7f21940d0aa0 legacy=0x7f219415fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2193200640 1 -- 172.21.15.86:0/1579320929 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21940cfd60 con 0x7f2194072460 2024-03-22T04:20:03.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.404+0000 7f2192800640 1 --2- 172.21.15.86:0/1579320929 >> v2:172.21.15.196:3300/0 conn(0x7f2194145b90 0x7f21940d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:03.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2193200640 1 --2- 172.21.15.86:0/1579320929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2194072460 0x7f21940728f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f218c0097c0 tx=0x7f218c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48a2847ca8c75297 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2191400640 1 -- 172.21.15.86:0/1579320929 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0b0070 con 0x7f2194072460 2024-03-22T04:20:03.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2191400640 1 -- 172.21.15.86:0/1579320929 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c0ab600 con 0x7f2194072460 2024-03-22T04:20:03.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2191400640 1 -- 172.21.15.86:0/1579320929 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0ab9d0 con 0x7f2194072460 2024-03-22T04:20:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2199400640 1 -- 172.21.15.86:0/1579320929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2194072460 msgr2=0x7f21940728f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2199400640 1 --2- 172.21.15.86:0/1579320929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2194072460 0x7f21940728f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f218c0097c0 tx=0x7f218c01c410 comp rx=0 tx=0).stop 2024-03-22T04:20:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2199400640 1 -- 172.21.15.86:0/1579320929 shutdown_connections 2024-03-22T04:20:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2199400640 1 --2- 172.21.15.86:0/1579320929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2194072460 0x7f21940728f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2199400640 1 --2- 172.21.15.86:0/1579320929 >> v2:172.21.15.196:3300/0 conn(0x7f2194145b90 0x7f21940d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2199400640 1 -- 172.21.15.86:0/1579320929 >> 172.21.15.86:0/1579320929 conn(0x7f21940d0fa0 msgr2=0x7f21940d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2199400640 1 -- 172.21.15.86:0/1579320929 shutdown_connections 2024-03-22T04:20:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.408+0000 7f2199400640 1 -- 172.21.15.86:0/1579320929 wait complete. 2024-03-22T04:20:03.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 Processor -- start 2024-03-22T04:20:03.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 -- start start 2024-03-22T04:20:03.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194072460 0x7f2194155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21940d0aa0 0x7f219415e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2194145b90 0x7f2194140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21941604a0 con 0x7f2194072460 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2194169130 con 0x7f2194145b90 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f219413ffb0 con 0x7f21940d0aa0 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194072460 0x7f2194155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194072460 0x7f2194155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57220/0 (socket says 172.21.15.86:57220) 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 -- 172.21.15.86:0/4069897166 learned_addr learned my addr 172.21.15.86:0/4069897166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2191e00640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21940d0aa0 0x7f219415e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2193200640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2194145b90 0x7f2194140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 -- 172.21.15.86:0/4069897166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21940d0aa0 msgr2=0x7f219415e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21940d0aa0 0x7f219415e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 -- 172.21.15.86:0/4069897166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2194145b90 msgr2=0x7f2194140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2194145b90 0x7f2194140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 -- 172.21.15.86:0/4069897166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2194140200 con 0x7f2194072460 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2192800640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194072460 0x7f2194155f20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f218400efc0 tx=0x7f218400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f218aa00640 1 -- 172.21.15.86:0/4069897166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184018020 con 0x7f2194072460 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f218aa00640 1 -- 172.21.15.86:0/4069897166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2184014680 con 0x7f2194072460 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 -- 172.21.15.86:0/4069897166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21941404e0 con 0x7f2194072460 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f218aa00640 1 -- 172.21.15.86:0/4069897166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184014a50 con 0x7f2194072460 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.412+0000 7f2199400640 1 -- 172.21.15.86:0/4069897166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2194156460 con 0x7f2194072460 2024-03-22T04:20:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f218aa00640 1 -- 172.21.15.86:0/4069897166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f218401b020 con 0x7f2194072460 2024-03-22T04:20:03.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f218aa00640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2174067b20 0x7f2174069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f218aa00640 1 -- 172.21.15.86:0/4069897166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f2184086e90 con 0x7f2194072460 2024-03-22T04:20:03.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f2191e00640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2174067b20 0x7f2174069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f219c913000 1 Processor -- start 2024-03-22T04:20:03.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f219c913000 1 -- start start 2024-03-22T04:20:03.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f2191e00640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2174067b20 0x7f2174069fe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f219415dd30 tx=0x7f2180009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f219c913000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a638a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f219c913000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a633da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f219c913000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 0x55a0a633e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f219c913000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a0a618ec40 con 0x55a0a638a8a0 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f219c913000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a0a612bd90 con 0x55a0a633df50 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.416+0000 7f219c913000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55a0a618abf0 con 0x55a0a6389f80 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a633da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2193200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 0x55a0a633e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2192800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a638a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a633da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57234/0 (socket says 172.21.15.86:57234) 2024-03-22T04:20:03.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 -- 172.21.15.86:0/187243732 learned_addr learned my addr 172.21.15.86:0/187243732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:03.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 -- 172.21.15.86:0/187243732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 msgr2=0x55a0a638a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 --2- 172.21.15.86:0/187243732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a638a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 -- 172.21.15.86:0/187243732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 msgr2=0x55a0a633e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 --2- 172.21.15.86:0/187243732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 0x55a0a633e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 -- 172.21.15.86:0/187243732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a6236130 con 0x55a0a638a8a0 2024-03-22T04:20:03.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2191e00640 1 --2- 172.21.15.86:0/187243732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a633da10 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f2180000f70 tx=0x7f2180000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1e9a56b5d48530 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f217f400640 1 -- 172.21.15.86:0/187243732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180025070 con 0x55a0a638a8a0 2024-03-22T04:20:03.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f217f400640 1 -- 172.21.15.86:0/187243732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180015bc0 con 0x55a0a638a8a0 2024-03-22T04:20:03.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f217f400640 1 -- 172.21.15.86:0/187243732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180020400 con 0x55a0a638a8a0 2024-03-22T04:20:03.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- 172.21.15.86:0/187243732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 msgr2=0x55a0a633da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 --2- 172.21.15.86:0/187243732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a633da10 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f2180000f70 tx=0x7f2180000ee0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- 172.21.15.86:0/187243732 shutdown_connections 2024-03-22T04:20:03.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 --2- 172.21.15.86:0/187243732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 0x55a0a633e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 --2- 172.21.15.86:0/187243732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a633da10 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 --2- 172.21.15.86:0/187243732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a638a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- 172.21.15.86:0/187243732 >> 172.21.15.86:0/187243732 conn(0x55a0a6227990 msgr2=0x55a0a636d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:03.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- 172.21.15.86:0/187243732 shutdown_connections 2024-03-22T04:20:03.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- 172.21.15.86:0/187243732 wait complete. 2024-03-22T04:20:03.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- start start 2024-03-22T04:20:03.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 Processor -- start 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a633df50 0x55a0a6345a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a636e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a636e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a0a614a710 con 0x55a0a633df50 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a0a614c580 con 0x55a0a638a8a0 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55a0a6193270 con 0x55a0a6389f80 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f219c913000 10 client.0 _ll_register_callbacks cb 0x55a0a61f0200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2192800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a633df50 0x55a0a6345a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2192800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a633df50 0x55a0a6345a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57248/0 (socket says 172.21.15.86:57248) 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.420+0000 7f2192800640 1 -- 172.21.15.86:0/1340011040 learned_addr learned my addr 172.21.15.86:0/1340011040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2191e00640 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a636e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2193200640 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a636e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2192800640 1 -- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 msgr2=0x55a0a636e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2192800640 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a636e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2192800640 1 -- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 msgr2=0x55a0a636e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2192800640 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a636e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2192800640 1 -- 172.21.15.86:0/1340011040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a6167d30 con 0x55a0a633df50 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2193200640 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a636e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2192800640 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a633df50 0x55a0a6345a40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f218400c990 tx=0x7f2184057020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b584364723ec81 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 1 -- 172.21.15.86:0/1340011040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184082020 con 0x55a0a633df50 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 -- 172.21.15.86:0/1340011040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2180015bc0 con 0x55a0a633df50 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 1 -- 172.21.15.86:0/1340011040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f2184056670 con 0x55a0a633df50 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 1 -- 172.21.15.86:0/1340011040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f218405f8f0 con 0x55a0a633df50 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 10 client.8640 fetch_fsmap learned FSMap version 84 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 -- 172.21.15.86:0/1340011040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55a0a61d3fc0 con 0x55a0a633df50 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 1 -- 172.21.15.86:0/1340011040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180015bc0 con 0x55a0a633df50 2024-03-22T04:20:03.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 1 -- 172.21.15.86:0/1340011040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f218405d2c0 con 0x55a0a633df50 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 10 client.8640 fetch_fsmap finished waiting for FSMap version 84 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 10 client.8640 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 20 client.8640 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55a0a6390630 0x55a0a6390a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 4 client.8640 mds_command: new command op to 77162 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 -- 172.21.15.86:0/1340011040 --> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0a6129dd0 con 0x55a0a6390630 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2191e00640 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55a0a6390630 0x55a0a6390a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2191e00640 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55a0a6390630 0x55a0a6390a10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2529e4c2ae2c5124 server_cookie=b16c0772910c81c7 in_seq=0 out_seq=0 2024-03-22T04:20:03.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 10 client.8640 ms_handle_connect on v2:172.21.15.196:6833/2259098413 2024-03-22T04:20:03.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 1 -- 172.21.15.86:0/1340011040 <== mds.0 v2:172.21.15.196:6833/2259098413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a0a6129dd0 con 0x55a0a6390630 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 10 client.8640 handle_command_reply: tid=0 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 client.8640 shutdown 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 -- 172.21.15.86:0/1340011040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a633df50 msgr2=0x55a0a6345a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a633df50 0x55a0a6345a40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f218400c990 tx=0x7f2184057020 comp rx=0 tx=0).stop 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 -- 172.21.15.86:0/1340011040 shutdown_connections 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55a0a6390630 0x55a0a6390a10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a636e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a636e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f2173e00640 0 client.8640 ms_handle_reset on v2:172.21.15.196:6833/2259098413 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 --2- 172.21.15.86:0/1340011040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a633df50 0x55a0a6345a40 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 -- 172.21.15.86:0/1340011040 >> 172.21.15.86:0/1340011040 conn(0x55a0a6227990 msgr2=0x55a0a633d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:03.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 -- 172.21.15.86:0/1340011040 shutdown_connections 2024-03-22T04:20:03.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 1 -- 172.21.15.86:0/1340011040 wait complete. 2024-03-22T04:20:03.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.424+0000 7f219c913000 20 client.8640 trim_cache size 0 max 16384 2024-03-22T04:20:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 Processor -- start 2024-03-22T04:20:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- start start 2024-03-22T04:20:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 0x55a0a638f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a6396a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a638eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2194203ff0 con 0x55a0a638a8a0 2024-03-22T04:20:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2194203020 con 0x55a0a633df50 2024-03-22T04:20:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f219415d270 con 0x55a0a6389f80 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a638eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a638eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57264/0 (socket says 172.21.15.86:57264) 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 -- 172.21.15.86:0/767264978 learned_addr learned my addr 172.21.15.86:0/767264978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2191e00640 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a6396a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2192800640 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 0x55a0a638f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 -- 172.21.15.86:0/767264978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 msgr2=0x55a0a6396a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a6396a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 -- 172.21.15.86:0/767264978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 msgr2=0x55a0a638f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 0x55a0a638f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 -- 172.21.15.86:0/767264978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a61643a0 con 0x55a0a638a8a0 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f2193200640 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a638eca0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f218c01aac0 tx=0x7f218c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b209329e2dd7db server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f217ea00640 1 -- 172.21.15.86:0/767264978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0b0040 con 0x55a0a638a8a0 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f217ea00640 1 -- 172.21.15.86:0/767264978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c0b6760 con 0x55a0a638a8a0 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f217ea00640 1 -- 172.21.15.86:0/767264978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0b6b30 con 0x55a0a638a8a0 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- 172.21.15.86:0/767264978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 msgr2=0x55a0a638eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a638eca0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f218c01aac0 tx=0x7f218c004740 comp rx=0 tx=0).stop 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- 172.21.15.86:0/767264978 shutdown_connections 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a638a8a0 0x55a0a638eca0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a6389f80 0x55a0a6396a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 --2- 172.21.15.86:0/767264978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a633df50 0x55a0a638f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- 172.21.15.86:0/767264978 >> 172.21.15.86:0/767264978 conn(0x55a0a6227990 msgr2=0x55a0a63460f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- 172.21.15.86:0/767264978 shutdown_connections 2024-03-22T04:20:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- 172.21.15.86:0/767264978 wait complete. 2024-03-22T04:20:03.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.444+0000 7f219c913000 1 -- start start 2024-03-22T04:20:03.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 Processor -- start 2024-03-22T04:20:03.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a633df50 0x55a0a6401fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a6389f80 0x55a0a6402500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a63ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a0a624a810 con 0x55a0a6389f80 2024-03-22T04:20:03.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21942029d0 con 0x55a0a638a8a0 2024-03-22T04:20:03.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55a0a618df50 con 0x55a0a633df50 2024-03-22T04:20:03.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 10 client.0 _ll_register_callbacks cb 0x7f2184053390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:03.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a6389f80 0x55a0a6402500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2192800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a633df50 0x55a0a6401fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a6389f80 0x55a0a6402500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57266/0 (socket says 172.21.15.86:57266) 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 -- 172.21.15.86:0/2860835991 learned_addr learned my addr 172.21.15.86:0/2860835991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2193200640 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a63ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 -- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a633df50 msgr2=0x55a0a6401fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a633df50 0x55a0a6401fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 -- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 msgr2=0x55a0a63ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a63ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 -- 172.21.15.86:0/2860835991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a6114380 con 0x55a0a6389f80 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2192800640 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a633df50 0x55a0a6401fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2191e00640 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a6389f80 0x55a0a6402500 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f218000fa80 tx=0x7f21800010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c8049e702eca56 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 1 -- 172.21.15.86:0/2860835991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180025040 con 0x55a0a6389f80 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 -- 172.21.15.86:0/2860835991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f218c0b6760 con 0x55a0a6389f80 2024-03-22T04:20:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 1 -- 172.21.15.86:0/2860835991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f21800047f0 con 0x55a0a6389f80 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 1 -- 172.21.15.86:0/2860835991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f218003c050 con 0x55a0a6389f80 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 10 client.8646 fetch_fsmap learned FSMap version 84 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 -- 172.21.15.86:0/2860835991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55a0a6210460 con 0x55a0a6389f80 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 1 -- 172.21.15.86:0/2860835991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c0b6760 con 0x55a0a6389f80 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 1 -- 172.21.15.86:0/2860835991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f218003f020 con 0x55a0a6389f80 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 10 client.8646 fetch_fsmap finished waiting for FSMap version 84 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 10 client.8646 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 20 client.8646 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55a0a6400490 0x55a0a6400870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 4 client.8646 mds_command: new command op to 77162 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 -- 172.21.15.86:0/2860835991 --> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0a612e160 con 0x55a0a6400490 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2192800640 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55a0a6400490 0x55a0a6400870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 10 client.8646 ms_handle_connect on v2:172.21.15.196:6833/2259098413 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2192800640 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55a0a6400490 0x55a0a6400870 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af18fb4c65ebe8 server_cookie=98664d57672889a9 in_seq=0 out_seq=0 2024-03-22T04:20:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 1 -- 172.21.15.86:0/2860835991 <== mds.0 v2:172.21.15.196:6833/2259098413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a0a612e160 con 0x55a0a6400490 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f2173e00640 10 client.8646 handle_command_reply: tid=0 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.448+0000 7f219c913000 1 client.8646 shutdown 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 -- 172.21.15.86:0/2860835991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a6389f80 msgr2=0x55a0a6402500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a6389f80 0x55a0a6402500 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f218000fa80 tx=0x7f21800010b0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 -- 172.21.15.86:0/2860835991 shutdown_connections 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:6833/2259098413,v1:172.21.15.196:6835/2259098413] conn(0x55a0a6400490 0x55a0a6400870 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0a638a8a0 0x55a0a63ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0a6389f80 0x55a0a6402500 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2173e00640 0 client.8646 ms_handle_reset on v2:172.21.15.196:6833/2259098413 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 --2- 172.21.15.86:0/2860835991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0a633df50 0x55a0a6401fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 -- 172.21.15.86:0/2860835991 >> 172.21.15.86:0/2860835991 conn(0x55a0a6227990 msgr2=0x55a0a63441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 -- 172.21.15.86:0/2860835991 shutdown_connections 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 1 -- 172.21.15.86:0/2860835991 wait complete. 2024-03-22T04:20:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f219c913000 20 client.8646 trim_cache size 0 max 16384 2024-03-22T04:20:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:20:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:20:03.455 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:20:03.455 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 -- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2174067b20 msgr2=0x7f2174069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2174067b20 0x7f2174069fe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f219415dd30 tx=0x7f2180009040 comp rx=0 tx=0).stop 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 -- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194072460 msgr2=0x7f2194155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194072460 0x7f2194155f20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f218400efc0 tx=0x7f218400c490 comp rx=0 tx=0).stop 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 -- 172.21.15.86:0/4069897166 shutdown_connections 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2174067b20 0x7f2174069fe0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2194145b90 0x7f2194140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21940d0aa0 0x7f219415e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 --2- 172.21.15.86:0/4069897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194072460 0x7f2194155f20 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 -- 172.21.15.86:0/4069897166 >> 172.21.15.86:0/4069897166 conn(0x7f21940d0fa0 msgr2=0x7f2194072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 -- 172.21.15.86:0/4069897166 shutdown_connections 2024-03-22T04:20:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.452+0000 7f2199400640 1 -- 172.21.15.86:0/4069897166 wait complete. 2024-03-22T04:20:03.470 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:20:03.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:20:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555fc00640 1 Processor -- start 2024-03-22T04:20:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555fc00640 1 -- start start 2024-03-22T04:20:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55580730d0 0x7f55580ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555fc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f55580cc3e0 0x7f55580ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5558071920 con 0x7f555815b3f0 2024-03-22T04:20:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f55580716d0 con 0x7f55580730d0 2024-03-22T04:20:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555fc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f555806d990 con 0x7f55580cc3e0 2024-03-22T04:20:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f554fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55580730d0 0x7f55580ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555d200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f555815b3f0 0x7f55580729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51748/0 (socket says 172.21.15.86:51748) 2024-03-22T04:20:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555d200640 1 -- 172.21.15.86:0/2296402957 learned_addr learned my addr 172.21.15.86:0/2296402957 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.708+0000 7f555dc00640 1 --2- 172.21.15.86:0/2296402957 >> v2:172.21.15.196:3300/0 conn(0x7f55580cc3e0 0x7f55580ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f554fe00640 1 -- 172.21.15.86:0/2296402957 >> v2:172.21.15.196:3300/0 conn(0x7f55580cc3e0 msgr2=0x7f55580ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f554fe00640 1 --2- 172.21.15.86:0/2296402957 >> v2:172.21.15.196:3300/0 conn(0x7f55580cc3e0 0x7f55580ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f554fe00640 1 -- 172.21.15.86:0/2296402957 >> v1:172.21.15.86:6789/0 conn(0x7f555815b3f0 legacy=0x7f55580729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f554fe00640 1 -- 172.21.15.86:0/2296402957 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5558071c00 con 0x7f55580730d0 2024-03-22T04:20:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f5557e00640 1 -- 172.21.15.86:0/2296402957 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668685135 0 0) 0x7f5558071920 con 0x7f555815b3f0 2024-03-22T04:20:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f554fe00640 1 --2- 172.21.15.86:0/2296402957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55580730d0 0x7f55580ade30 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5558073b20 tx=0x7f554001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26a8867969c2f36 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f5557e00640 1 -- 172.21.15.86:0/2296402957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55400b0070 con 0x7f55580730d0 2024-03-22T04:20:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f5557e00640 1 -- 172.21.15.86:0/2296402957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55400ab600 con 0x7f55580730d0 2024-03-22T04:20:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f5557e00640 1 -- 172.21.15.86:0/2296402957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55400ab9d0 con 0x7f55580730d0 2024-03-22T04:20:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f555fc00640 1 -- 172.21.15.86:0/2296402957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55580730d0 msgr2=0x7f55580ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f555fc00640 1 --2- 172.21.15.86:0/2296402957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55580730d0 0x7f55580ade30 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5558073b20 tx=0x7f554001c480 comp rx=0 tx=0).stop 2024-03-22T04:20:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f555fc00640 1 -- 172.21.15.86:0/2296402957 shutdown_connections 2024-03-22T04:20:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f555fc00640 1 --2- 172.21.15.86:0/2296402957 >> v2:172.21.15.196:3300/0 conn(0x7f55580cc3e0 0x7f55580ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f555fc00640 1 --2- 172.21.15.86:0/2296402957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55580730d0 0x7f55580ade30 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.712+0000 7f555fc00640 1 -- 172.21.15.86:0/2296402957 >> 172.21.15.86:0/2296402957 conn(0x7f55580b1df0 msgr2=0x7f55580b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 -- 172.21.15.86:0/2296402957 shutdown_connections 2024-03-22T04:20:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 -- 172.21.15.86:0/2296402957 wait complete. 2024-03-22T04:20:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 Processor -- start 2024-03-22T04:20:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 -- start start 2024-03-22T04:20:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55580730d0 0x7f555814bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580cc3e0 0x7f55580cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815b3f0 0x7f55580cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f555806e110 con 0x7f555815b3f0 2024-03-22T04:20:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f555806df90 con 0x7f55580730d0 2024-03-22T04:20:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f555806e290 con 0x7f55580cc3e0 2024-03-22T04:20:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815b3f0 0x7f55580cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815b3f0 0x7f55580cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57282/0 (socket says 172.21.15.86:57282) 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 -- 172.21.15.86:0/4084414027 learned_addr learned my addr 172.21.15.86:0/4084414027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555d200640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55580730d0 0x7f555814bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f554fe00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580cc3e0 0x7f55580cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 -- 172.21.15.86:0/4084414027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580cc3e0 msgr2=0x7f55580cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580cc3e0 0x7f55580cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 -- 172.21.15.86:0/4084414027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55580730d0 msgr2=0x7f555814bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55580730d0 0x7f555814bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 -- 172.21.15.86:0/4084414027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55580caeb0 con 0x7f555815b3f0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555dc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815b3f0 0x7f55580cb850 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f555000efc0 tx=0x7f555000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f5555600640 1 -- 172.21.15.86:0/4084414027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550018020 con 0x7f555815b3f0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f5555600640 1 -- 172.21.15.86:0/4084414027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5550014480 con 0x7f555815b3f0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f5555600640 1 -- 172.21.15.86:0/4084414027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550014870 con 0x7f555815b3f0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55580ca330 con 0x7f555815b3f0 2024-03-22T04:20:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.716+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55580ca8f0 con 0x7f555815b3f0 2024-03-22T04:20:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.720+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5558068650 con 0x7f555815b3f0 2024-03-22T04:20:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.720+0000 7f5555600640 1 -- 172.21.15.86:0/4084414027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f555001b020 con 0x7f555815b3f0 2024-03-22T04:20:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.720+0000 7f5555600640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5534067b20 0x7f5534069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.720+0000 7f5555600640 1 -- 172.21.15.86:0/4084414027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6101+0+0 (secure 0 0 0) 0x7f5550086af0 con 0x7f555815b3f0 2024-03-22T04:20:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.720+0000 7f555d200640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5534067b20 0x7f5534069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.720+0000 7f555d200640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5534067b20 0x7f5534069fe0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5548000f70 tx=0x7f5548007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.724+0000 7f5555600640 1 -- 172.21.15.86:0/4084414027 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5550022070 con 0x7f555815b3f0 2024-03-22T04:20:03.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.912+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f555815d5f0 con 0x7f555815b3f0 2024-03-22T04:20:03.914 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:20:03.912+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:20:03.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.916+0000 7f5555600640 1 -- 172.21.15.86:0/4084414027 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) ==== 161+0+0 (secure 0 0 0) 0x7f5550010040 con 0x7f555815b3f0 2024-03-22T04:20:03.915 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:20:03.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.916+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5534067b20 msgr2=0x7f5534069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.916+0000 7f555fc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5534067b20 0x7f5534069fe0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5548000f70 tx=0x7f5548007df0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.916+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815b3f0 msgr2=0x7f55580cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.916+0000 7f555fc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815b3f0 0x7f55580cb850 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f555000efc0 tx=0x7f555000c490 comp rx=0 tx=0).stop 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.920+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 shutdown_connections 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.920+0000 7f555fc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5534067b20 0x7f5534069fe0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.920+0000 7f555fc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f555815b3f0 0x7f55580cb850 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.920+0000 7f555fc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55580cc3e0 0x7f55580cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.920+0000 7f555fc00640 1 --2- 172.21.15.86:0/4084414027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55580730d0 0x7f555814bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.920+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 >> 172.21.15.86:0/4084414027 conn(0x7f55580b1df0 msgr2=0x7f555815c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.920+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 shutdown_connections 2024-03-22T04:20:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.920+0000 7f555fc00640 1 -- 172.21.15.86:0/4084414027 wait complete. 2024-03-22T04:20:03.931 INFO:tasks.ceph.mds.b.smithi196.stderr: -7105> 2024-03-22T04:19:23.287+0000 7f485ca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:20:03.938 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-03-22T04:20:03.980 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:20:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.988+0000 7f5b74a38400 1 -- start start 2024-03-22T04:20:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.988+0000 7f5b74a38400 1 Processor -- start 2024-03-22T04:20:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1ebbb800 0x563d1f988000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1ebbb400 0x563d1f988580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1ebbb000 0x563d1f988b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d1f914180 con 0x563d1ebbb000 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d1f914300 con 0x563d1ebbb400 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x563d1f914480 con 0x563d1ebbb800 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1ebbb000 0x563d1f988b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1ebbb000 0x563d1f988b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57286/0 (socket says 172.21.15.86:57286) 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 -- 172.21.15.86:0/3221281988 learned_addr learned my addr 172.21.15.86:0/3221281988 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b72000640 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1ebbb400 0x563d1f988580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b72a00640 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1ebbb800 0x563d1f988000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 -- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1ebbb800 msgr2=0x563d1f988000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1ebbb800 0x563d1f988000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 -- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1ebbb400 msgr2=0x563d1f988580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1ebbb400 0x563d1f988580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 -- 172.21.15.86:0/3221281988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d1ed2eb40 con 0x563d1ebbb000 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b72000640 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1ebbb400 0x563d1f988580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b73400640 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1ebbb000 0x563d1f988b00 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x563d1fa312f0 tx=0x563d1ec91560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93d013f83494a2d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b6f800640 1 -- 172.21.15.86:0/3221281988 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563d1ebf04e0 con 0x563d1ebbb000 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b6f800640 1 -- 172.21.15.86:0/3221281988 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563d1ec06e00 con 0x563d1ebbb000 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b6f800640 1 -- 172.21.15.86:0/3221281988 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563d1ebf0b60 con 0x563d1ebbb000 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- 172.21.15.86:0/3221281988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d1ed2ed20 con 0x563d1ebbb000 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- 172.21.15.86:0/3221281988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563d1ed2ef00 con 0x563d1ebbb000 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 4 waiting for MDS map... 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b6f800640 1 -- 172.21.15.86:0/3221281988 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x563d1fad4600 con 0x563d1ebbb000 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b6f800640 1 -- 172.21.15.86:0/3221281988 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 85) ==== 2885+0+0 (secure 0 0 0) 0x563d1ec11180 con 0x563d1ebbb000 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 4 Got MDS map 85 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 10 main: JournalTool::main 2024-03-22T04:20:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 4 main: JournalTool: connecting to RADOS... 2024-03-22T04:20:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 Processor -- start 2024-03-22T04:20:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- start start 2024-03-22T04:20:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7000 0x563d1f989080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7800 0x563d1f989b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d1f914600 con 0x563d1fad7400 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d1f914780 con 0x563d1fad7800 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b74a38400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x563d1f914900 con 0x563d1fad7000 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57302/0 (socket says 172.21.15.86:57302) 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b72a00640 1 -- 172.21.15.86:0/4107248273 learned_addr learned my addr 172.21.15.86:0/4107248273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.992+0000 7f5b72000640 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7800 0x563d1f989b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72000640 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7000 0x563d1f989080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 -- 172.21.15.86:0/4107248273 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7000 msgr2=0x563d1f989080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7000 0x563d1f989080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 -- 172.21.15.86:0/4107248273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7800 msgr2=0x563d1f989b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7800 0x563d1f989b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 -- 172.21.15.86:0/4107248273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d1ed2ed20 con 0x563d1fad7400 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x563d1fb246f0 tx=0x563d1fae5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13443c2d7cbd6dc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b6d000640 1 -- 172.21.15.86:0/4107248273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563d1ebf04e0 con 0x563d1fad7400 2024-03-22T04:20:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b6d000640 1 -- 172.21.15.86:0/4107248273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563d1ec06540 con 0x563d1fad7400 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b6d000640 1 -- 172.21.15.86:0/4107248273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563d1ebf0b60 con 0x563d1fad7400 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- 172.21.15.86:0/4107248273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 msgr2=0x563d1f989600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x563d1fb246f0 tx=0x563d1fae5800 comp rx=0 tx=0).stop 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- 172.21.15.86:0/4107248273 shutdown_connections 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7800 0x563d1f989b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 --2- 172.21.15.86:0/4107248273 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7000 0x563d1f989080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- 172.21.15.86:0/4107248273 >> 172.21.15.86:0/4107248273 conn(0x563d1ebbac00 msgr2=0x563d1ebe9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- 172.21.15.86:0/4107248273 shutdown_connections 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- 172.21.15.86:0/4107248273 wait complete. 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 Processor -- start 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- start start 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7000 0x563d1f989080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7800 0x563d1f989b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d1f915c80 con 0x563d1fad7400 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d1f915b00 con 0x563d1fad7000 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x563d1f915980 con 0x563d1fad7800 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57316/0 (socket says 172.21.15.86:57316) 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 -- 172.21.15.86:0/1262494740 learned_addr learned my addr 172.21.15.86:0/1262494740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72000640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7800 0x563d1f989b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72000640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7000 0x563d1f989080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 -- 172.21.15.86:0/1262494740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7800 msgr2=0x563d1f989b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1fad7800 0x563d1f989b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 -- 172.21.15.86:0/1262494740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7000 msgr2=0x563d1f989080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1fad7000 0x563d1f989080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 -- 172.21.15.86:0/1262494740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d1ed2fc20 con 0x563d1fad7400 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b72a00640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x563d1fb253b0 tx=0x563d1ec91500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b6b200640 1 -- 172.21.15.86:0/1262494740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563d1ebf0000 con 0x563d1fad7400 2024-03-22T04:20:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b6b200640 1 -- 172.21.15.86:0/1262494740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563d1ec06c40 con 0x563d1fad7400 2024-03-22T04:20:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b6b200640 1 -- 172.21.15.86:0/1262494740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563d1ebf04e0 con 0x563d1fad7400 2024-03-22T04:20:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563d1ed2fe00 con 0x563d1fad7400 2024-03-22T04:20:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d1ed2fc20 con 0x563d1fad7400 2024-03-22T04:20:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:03.996+0000 7f5b74a38400 4 main: JournalTool: resolving pool 15 2024-03-22T04:20:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b6b200640 1 -- 172.21.15.86:0/1262494740 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x563d1fb42c00 con 0x563d1fad7400 2024-03-22T04:20:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b6b200640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563d1ee9ec00 0x563d1f98a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b6b200640 1 -- 172.21.15.86:0/1262494740 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x563d1eeb4400 con 0x563d1fad7400 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 4 main: JournalTool: creating IoCtx.. 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 4 main: Executing for rank 0 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 4 Filtering by inode '1099511627777' 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563d1fad7c00 0x563d1f98a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x563d1ee9f000 con 0x563d1fad7c00 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b72000640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563d1ee9ec00 0x563d1f98a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b72000640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563d1fad7c00 0x563d1f98a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b72000640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563d1fad7c00 0x563d1f98a680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b72000640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563d1ee9ec00 0x563d1f98a100 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x563d1fa30900 tx=0x563d1f942900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b72000640 1 -- 172.21.15.86:0/1262494740 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x563d1f938000 con 0x563d1fad7c00 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 4 Pointer 400.00000000 is readable 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563d1ee9f400 0x563d1f98ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x563d1ee9f800 con 0x563d1ee9f400 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.000+0000 7f5b73400640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563d1ee9f400 0x563d1f98ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b73400640 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563d1ee9f400 0x563d1f98ac00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b73400640 1 -- 172.21.15.86:0/1262494740 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x563d1f938500 con 0x563d1ee9f400 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 10 Header 0x400000 0x40794b 0x407cc7 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 10 Starting journal scan from offset 0x40794b 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 31051~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x563d1ee9fc00 con 0x563d1ee9f400 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b73400640 1 -- 172.21.15.86:0/1262494740 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== osd_op_reply(3 200.00000001 [read 31051~892 out=892b] v0'0 uv29 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x563d1f938500 con 0x563d1ee9f400 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 10 Parsing data, 0x37c bytes available 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 10 Attempting decode at 0x40794b 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 10 Consumed 0x37c bytes 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 10 Valid entry at 0x40794b 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x563d1efaa000 con 0x563d1ee9f400 2024-03-22T04:20:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b73400640 1 -- 172.21.15.86:0/1262494740 <== osd.3 v2:172.21.15.86:6822/2837220459 3 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563d1f938500 con 0x563d1ee9f400 2024-03-22T04:20:04.007 INFO:teuthology.orchestra.run.smithi086.stdout:Events by type: 2024-03-22T04:20:04.007 INFO:teuthology.orchestra.run.smithi086.stdout:Errors: 0 2024-03-22T04:20:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 4 Reached end of journal objects 2024-03-22T04:20:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 4 Scanned objects, 0 missing, 1 valid 2024-03-22T04:20:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 4 Events scanned, 0 gaps 2024-03-22T04:20:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 4 Found 1 valid events 2024-03-22T04:20:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 4 Selected 0 events events for processing 2024-03-22T04:20:04.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 4 0 events 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563d1fad7c00 msgr2=0x563d1f98a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563d1fad7c00 0x563d1f98a680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563d1ee9f400 msgr2=0x563d1f98ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563d1ee9f400 0x563d1f98ac00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563d1ee9ec00 msgr2=0x563d1f98a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563d1ee9ec00 0x563d1f98a100 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x563d1fa30900 tx=0x563d1f942900 comp rx=0 tx=0).stop 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 msgr2=0x563d1f989600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 --2- 172.21.15.86:0/1262494740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1fad7400 0x563d1f989600 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x563d1fb253b0 tx=0x563d1ec91500 comp rx=0 tx=0).stop 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b72a00640 1 -- 172.21.15.86:0/1262494740 reap_dead start 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b72a00640 1 -- 172.21.15.86:0/1262494740 reap_dead start 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 shutdown_connections 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 >> 172.21.15.86:0/1262494740 conn(0x563d1ebbac00 msgr2=0x563d1ebe9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 shutdown_connections 2024-03-22T04:20:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/1262494740 wait complete. 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/3221281988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1ebbb000 msgr2=0x563d1f988b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1ebbb000 0x563d1f988b00 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x563d1fa312f0 tx=0x563d1ec91560 comp rx=0 tx=0).stop 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/3221281988 shutdown_connections 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d1ebbb400 0x563d1f988580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d1ebbb000 0x563d1f988b00 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 --2- 172.21.15.86:0/3221281988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563d1ebbb800 0x563d1f988000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/3221281988 >> 172.21.15.86:0/3221281988 conn(0x563d1ebbbc00 msgr2=0x563d1ebe9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/3221281988 shutdown_connections 2024-03-22T04:20:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.004+0000 7f5b74a38400 1 -- 172.21.15.86:0/3221281988 wait complete. 2024-03-22T04:20:04.011 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.072797, result: Events by type: Errors: 0 2024-03-22T04:20:04.011 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-03-22T04:20:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9dd78400 1 -- start start 2024-03-22T04:20:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9dd78400 1 Processor -- start 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559058623000 0x5590593f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559058623400 0x5590593f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559058623800 0x5590593f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559059404600 con 0x559058623000 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559059404480 con 0x559058623400 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559059404300 con 0x559058623800 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559058623000 0x5590593f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559058623000 0x5590593f0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57322/0 (socket says 172.21.15.86:57322) 2024-03-22T04:20:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9be00640 1 -- 172.21.15.86:0/3243639488 learned_addr learned my addr 172.21.15.86:0/3243639488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9b400640 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559058623400 0x5590593f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.056+0000 7f6b9c800640 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559058623800 0x5590593f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 -- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559058623800 msgr2=0x5590593f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559058623800 0x5590593f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 -- 172.21.15.86:0/3243639488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559058623000 msgr2=0x5590593f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559058623000 0x5590593f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 -- 172.21.15.86:0/3243639488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590593ea960 con 0x559058623400 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9be00640 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559058623000 0x5590593f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559058623400 0x5590593f0580 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55905948c990 tx=0x5590586f81e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa1b3ec91d3caac server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b98c00640 1 -- 172.21.15.86:0/3243639488 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5590586589c0 con 0x559058623400 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b98c00640 1 -- 172.21.15.86:0/3243639488 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55905866ec40 con 0x559058623400 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b98c00640 1 -- 172.21.15.86:0/3243639488 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5590586584e0 con 0x559058623400 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 -- 172.21.15.86:0/3243639488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590593ead20 con 0x559058623400 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 -- 172.21.15.86:0/3243639488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5590593eaf00 con 0x559058623400 2024-03-22T04:20:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 4 waiting for MDS map... 2024-03-22T04:20:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b98c00640 1 -- 172.21.15.86:0/3243639488 <== mon.1 v2:172.21.15.196:3300/0 4 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x5590593e5a00 con 0x559058623400 2024-03-22T04:20:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b98c00640 1 -- 172.21.15.86:0/3243639488 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 85) ==== 2885+0+0 (secure 0 0 0) 0x559058679180 con 0x559058623400 2024-03-22T04:20:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 4 Got MDS map 85 2024-03-22T04:20:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 10 main: JournalTool::main 2024-03-22T04:20:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 4 main: JournalTool: connecting to RADOS... 2024-03-22T04:20:04.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 Processor -- start 2024-03-22T04:20:04.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 -- start start 2024-03-22T04:20:04.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c000 0x5590593f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 0x5590593f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c800 0x5590593f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559059404180 con 0x55905959c800 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559058682480 con 0x55905959c000 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559058682900 con 0x55905959c400 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c800 0x5590593f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c800 0x5590593f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57328/0 (socket says 172.21.15.86:57328) 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 -- 172.21.15.86:0/1795936501 learned_addr learned my addr 172.21.15.86:0/1795936501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9c800640 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c000 0x5590593f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9be00640 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 0x5590593f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 -- 172.21.15.86:0/1795936501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 msgr2=0x5590593f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 0x5590593f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 -- 172.21.15.86:0/1795936501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c000 msgr2=0x5590593f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:04.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c000 0x5590593f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.060+0000 7f6b9b400640 1 -- 172.21.15.86:0/1795936501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590593ea1e0 con 0x55905959c800 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9b400640 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c800 0x5590593f1b80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x5590595a4120 tx=0x5590586f8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e86889baacdbf3f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b96400640 1 -- 172.21.15.86:0/1795936501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559058658b60 con 0x55905959c800 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b96400640 1 -- 172.21.15.86:0/1795936501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55905866ea80 con 0x55905959c800 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b96400640 1 -- 172.21.15.86:0/1795936501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559058658340 con 0x55905959c800 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- 172.21.15.86:0/1795936501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c800 msgr2=0x5590593f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c800 0x5590593f1b80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x5590595a4120 tx=0x5590586f8cc0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- 172.21.15.86:0/1795936501 shutdown_connections 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c800 0x5590593f1b80 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 0x5590593f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/1795936501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c000 0x5590593f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- 172.21.15.86:0/1795936501 >> 172.21.15.86:0/1795936501 conn(0x559059409c00 msgr2=0x559058650000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- 172.21.15.86:0/1795936501 shutdown_connections 2024-03-22T04:20:04.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- 172.21.15.86:0/1795936501 wait complete. 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 Processor -- start 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- start start 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c000 0x5590593f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 0x5590593f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c800 0x5590593f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559059405380 con 0x55905959c000 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559059405200 con 0x55905959c800 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559059405500 con 0x55905959c400 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c000 0x5590593f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c000 0x5590593f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57332/0 (socket says 172.21.15.86:57332) 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 -- 172.21.15.86:0/4227783217 learned_addr learned my addr 172.21.15.86:0/4227783217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9b400640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c800 0x5590593f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9be00640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 0x5590593f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 -- 172.21.15.86:0/4227783217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 msgr2=0x5590593f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55905959c400 0x5590593f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 -- 172.21.15.86:0/4227783217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c800 msgr2=0x5590593f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55905959c800 0x5590593f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 -- 172.21.15.86:0/4227783217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590593eba40 con 0x55905959c000 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9c800640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c000 0x5590593f1080 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x5590595a54a0 tx=0x5590593ff800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b94600640 1 -- 172.21.15.86:0/4227783217 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559058658b60 con 0x55905959c000 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b94600640 1 -- 172.21.15.86:0/4227783217 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55905866f340 con 0x55905959c000 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b94600640 1 -- 172.21.15.86:0/4227783217 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559058658340 con 0x55905959c000 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5590593ebc20 con 0x55905959c000 2024-03-22T04:20:04.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55905892a000 con 0x55905959c000 2024-03-22T04:20:04.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.064+0000 7f6b9dd78400 4 main: JournalTool: resolving pool 15 2024-03-22T04:20:04.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b94600640 1 -- 172.21.15.86:0/4227783217 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x55905959ac00 con 0x55905959c000 2024-03-22T04:20:04.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b94600640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55905891ac00 0x5590593f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b94600640 1 -- 172.21.15.86:0/4227783217 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x55905891c800 con 0x55905959c000 2024-03-22T04:20:04.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 4 main: JournalTool: creating IoCtx.. 2024-03-22T04:20:04.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 4 main: Executing for rank 0 2024-03-22T04:20:04.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 4 Filtering by inode '1099511627778' 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55905959cc00 0x5590593f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9be00640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55905891ac00 0x5590593f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x55905959d000 con 0x55905959cc00 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9b400640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55905959cc00 0x5590593f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9be00640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55905891ac00 0x5590593f2100 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55905948d3e0 tx=0x5590589a4de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9b400640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55905959cc00 0x5590593f2680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9b400640 1 -- 172.21.15.86:0/4227783217 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5590593a0000 con 0x55905959cc00 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 4 Pointer 400.00000000 is readable 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55905891b800 0x5590593f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x55905891bc00 con 0x55905891b800 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9c800640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55905891b800 0x5590593f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.068+0000 7f6b9c800640 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55905891b800 0x5590593f2c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9c800640 1 -- 172.21.15.86:0/4227783217 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5590593a0500 con 0x55905891b800 2024-03-22T04:20:04.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 10 Header 0x400000 0x40794b 0x407cc7 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 10 Starting journal scan from offset 0x40794b 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 31051~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x559058a16000 con 0x55905891b800 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9c800640 1 -- 172.21.15.86:0/4227783217 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== osd_op_reply(3 200.00000001 [read 31051~892 out=892b] v0'0 uv29 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5590593a0500 con 0x55905891b800 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 10 Parsing data, 0x37c bytes available 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 10 Attempting decode at 0x40794b 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 10 Consumed 0x37c bytes 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 10 Valid entry at 0x40794b 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x559058a16400 con 0x55905891b800 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9c800640 1 -- 172.21.15.86:0/4227783217 <== osd.3 v2:172.21.15.86:6822/2837220459 3 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5590593a0500 con 0x55905891b800 2024-03-22T04:20:04.076 INFO:teuthology.orchestra.run.smithi086.stdout:Events by type: 2024-03-22T04:20:04.077 INFO:teuthology.orchestra.run.smithi086.stdout:Errors: 0 2024-03-22T04:20:04.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 4 Reached end of journal objects 2024-03-22T04:20:04.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 4 Scanned objects, 0 missing, 1 valid 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 4 Events scanned, 0 gaps 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 4 Found 1 valid events 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 4 Selected 0 events events for processing 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 4 0 events 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55905959cc00 msgr2=0x5590593f2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x55905959cc00 0x5590593f2680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55905891b800 msgr2=0x5590593f2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x55905891b800 0x5590593f2c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55905891ac00 msgr2=0x5590593f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x55905891ac00 0x5590593f2100 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55905948d3e0 tx=0x5590589a4de0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c000 msgr2=0x5590593f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/4227783217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55905959c000 0x5590593f1080 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x5590595a54a0 tx=0x5590593ff800 comp rx=0 tx=0).stop 2024-03-22T04:20:04.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9be00640 1 -- 172.21.15.86:0/4227783217 reap_dead start 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9be00640 1 -- 172.21.15.86:0/4227783217 reap_dead start 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 shutdown_connections 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 >> 172.21.15.86:0/4227783217 conn(0x559059409c00 msgr2=0x559058650000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 shutdown_connections 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/4227783217 wait complete. 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559058623400 msgr2=0x5590593f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559058623400 0x5590593f0580 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55905948c990 tx=0x5590586f81e0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/3243639488 shutdown_connections 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559058623400 0x5590593f0580 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559058623800 0x5590593f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 --2- 172.21.15.86:0/3243639488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559058623000 0x5590593f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/3243639488 >> 172.21.15.86:0/3243639488 conn(0x559058622c00 msgr2=0x55905864f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/3243639488 shutdown_connections 2024-03-22T04:20:04.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.072+0000 7f6b9dd78400 1 -- 172.21.15.86:0/3243639488 wait complete. 2024-03-22T04:20:04.081 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.069759, result: Events by type: Errors: 0 2024-03-22T04:20:04.081 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-03-22T04:20:04.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a74146080 1 Processor -- start 2024-03-22T04:20:04.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a74146080 1 -- start start 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a74146080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e520 0x55aac3b97090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a74146080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3b7e900 0x55aac3c1f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a74146080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3c26b30 0x55aac3b52af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a74146080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aac3af5620 con 0x55aac3b7e520 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a74146080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aac3af9670 con 0x55aac3b7e900 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a74146080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55aac3b99de0 con 0x55aac3c26b30 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e520 0x55aac3b97090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e520 0x55aac3b97090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57362/0 (socket says 172.21.15.86:57362) 2024-03-22T04:20:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 -- 172.21.15.86:0/1445260213 learned_addr learned my addr 172.21.15.86:0/1445260213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a70a00640 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3b7e900 0x55aac3c1f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71e00640 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3c26b30 0x55aac3b52af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 -- 172.21.15.86:0/1445260213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3c26b30 msgr2=0x55aac3b52af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3c26b30 0x55aac3b52af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 -- 172.21.15.86:0/1445260213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3b7e900 msgr2=0x55aac3c1f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3b7e900 0x55aac3c1f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 -- 172.21.15.86:0/1445260213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aac3cba1c0 con 0x55aac3b7e520 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a71400640 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e520 0x55aac3b97090 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400b330 tx=0x7f2a6400b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33d85aacb31e97a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a6b400640 1 -- 172.21.15.86:0/1445260213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400d5d0 con 0x55aac3b7e520 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a6b400640 1 -- 172.21.15.86:0/1445260213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a64008070 con 0x55aac3b7e520 2024-03-22T04:20:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.168+0000 7f2a6b400640 1 -- 172.21.15.86:0/1445260213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400d9b0 con 0x55aac3b7e520 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- 172.21.15.86:0/1445260213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e520 msgr2=0x55aac3b97090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e520 0x55aac3b97090 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400b330 tx=0x7f2a6400b7f0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- 172.21.15.86:0/1445260213 shutdown_connections 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3b7e900 0x55aac3c1f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3c26b30 0x55aac3b52af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 --2- 172.21.15.86:0/1445260213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e520 0x55aac3b97090 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- 172.21.15.86:0/1445260213 >> 172.21.15.86:0/1445260213 conn(0x55aac3b7bec0 msgr2=0x55aac3c26390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- 172.21.15.86:0/1445260213 shutdown_connections 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- 172.21.15.86:0/1445260213 wait complete. 2024-03-22T04:20:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 Processor -- start 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- start start 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3b7e520 0x55aac3cb9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e900 0x55aac3c23820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3c26b30 0x55aac3c23d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aac3cba700 con 0x55aac3b7e900 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55aac3cba460 con 0x55aac3c26b30 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55aac3cba9a0 con 0x55aac3b7e520 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e900 0x55aac3c23820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e900 0x55aac3c23820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57372/0 (socket says 172.21.15.86:57372) 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 -- 172.21.15.86:0/2836939382 learned_addr learned my addr 172.21.15.86:0/2836939382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a71e00640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3c26b30 0x55aac3c23d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a71400640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3b7e520 0x55aac3cb9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 -- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3b7e520 msgr2=0x55aac3cb9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55aac3b7e520 0x55aac3cb9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 -- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3c26b30 msgr2=0x55aac3c23d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3c26b30 0x55aac3c23d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 -- 172.21.15.86:0/2836939382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aac3c2bd60 con 0x55aac3b7e900 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a71e00640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55aac3c26b30 0x55aac3c23d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a70a00640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e900 0x55aac3c23820 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ad040 tx=0x7f2a60018030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a6a000640 1 -- 172.21.15.86:0/2836939382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600bb070 con 0x55aac3b7e900 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a6a000640 1 -- 172.21.15.86:0/2836939382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a600b55f0 con 0x55aac3b7e900 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a6a000640 1 -- 172.21.15.86:0/2836939382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600af040 con 0x55aac3b7e900 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aac3c2c000 con 0x55aac3b7e900 2024-03-22T04:20:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.172+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aac3c2c540 con 0x55aac3b7e900 2024-03-22T04:20:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a6a000640 1 -- 172.21.15.86:0/2836939382 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2a600be070 con 0x55aac3b7e900 2024-03-22T04:20:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a6a000640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a54066500 0x7f2a540689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a6a000640 1 -- 172.21.15.86:0/2836939382 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7f2a6011fb00 con 0x55aac3b7e900 2024-03-22T04:20:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a71400640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a54066500 0x7f2a540689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a74146080 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55aac3cb9950 0x55aac3c2b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55aac3c2e400 con 0x55aac3cb9950 2024-03-22T04:20:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a71400640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a54066500 0x7f2a540689c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2a64000fc0 tx=0x7f2a64009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a71e00640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55aac3cb9950 0x55aac3c2b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.176+0000 7f2a71e00640 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55aac3cb9950 0x55aac3c2b560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a71e00640 1 -- 172.21.15.86:0/2836939382 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v110'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2a6c002bd0 con 0x55aac3cb9950 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55aac3cb9950 msgr2=0x55aac3c2b560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x55aac3cb9950 0x55aac3c2b560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a54066500 msgr2=0x7f2a540689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2a54066500 0x7f2a540689c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2a64000fc0 tx=0x7f2a64009000 comp rx=0 tx=0).stop 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e900 msgr2=0x55aac3c23820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 --2- 172.21.15.86:0/2836939382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac3b7e900 0x55aac3c23820 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ad040 tx=0x7f2a60018030 comp rx=0 tx=0).stop 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a71e00640 1 -- 172.21.15.86:0/2836939382 reap_dead start 2024-03-22T04:20:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 shutdown_connections 2024-03-22T04:20:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 >> 172.21.15.86:0/2836939382 conn(0x55aac3b7bec0 msgr2=0x55aac3b512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 shutdown_connections 2024-03-22T04:20:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.180+0000 7f2a74146080 1 -- 172.21.15.86:0/2836939382 wait complete. 2024-03-22T04:20:04.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-22T04:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a3e00640 1 Processor -- start 2024-03-22T04:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a3e00640 1 -- start start 2024-03-22T04:20:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff99c0b3600 0x7ff99c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a3e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99c15d710 con 0x7ff99c15cee0 2024-03-22T04:20:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99c15fe40 con 0x7ff99c0b2360 2024-03-22T04:20:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a3e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff99c0b2740 con 0x7ff99c0b3600 2024-03-22T04:20:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff99c0b3600 0x7ff99c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff99c15cee0 0x7ff99c0b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51788/0 (socket says 172.21.15.86:51788) 2024-03-22T04:20:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a1e00640 1 -- 172.21.15.86:0/2516694859 learned_addr learned my addr 172.21.15.86:0/2516694859 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:04.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a0a00640 1 -- 172.21.15.86:0/2516694859 >> v1:172.21.15.196:6789/0 conn(0x7ff99c0b2360 legacy=0x7ff99c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a0a00640 1 -- 172.21.15.86:0/2516694859 >> v1:172.21.15.86:6789/0 conn(0x7ff99c15cee0 legacy=0x7ff99c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff99be00640 1 -- 172.21.15.86:0/2516694859 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437484913 0 0) 0x7ff99c15d710 con 0x7ff99c15cee0 2024-03-22T04:20:04.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.364+0000 7ff9a0a00640 1 -- 172.21.15.86:0/2516694859 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c160120 con 0x7ff99c0b3600 2024-03-22T04:20:04.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff9a0a00640 1 --2- 172.21.15.86:0/2516694859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b3600 0x7ff99c15c9a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff9880097c0 tx=0x7ff98801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d823e574d8cea104 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff99be00640 1 -- 172.21.15.86:0/2516694859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a7070 con 0x7ff99c0b3600 2024-03-22T04:20:04.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff99be00640 1 -- 172.21.15.86:0/2516694859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9880b0070 con 0x7ff99c0b3600 2024-03-22T04:20:04.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff99be00640 1 -- 172.21.15.86:0/2516694859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880ab890 con 0x7ff99c0b3600 2024-03-22T04:20:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff9a3e00640 1 -- 172.21.15.86:0/2516694859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b3600 msgr2=0x7ff99c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff9a3e00640 1 --2- 172.21.15.86:0/2516694859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b3600 0x7ff99c15c9a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff9880097c0 tx=0x7ff98801c470 comp rx=0 tx=0).stop 2024-03-22T04:20:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff9a3e00640 1 -- 172.21.15.86:0/2516694859 shutdown_connections 2024-03-22T04:20:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff9a3e00640 1 --2- 172.21.15.86:0/2516694859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b3600 0x7ff99c15c9a0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff9a3e00640 1 -- 172.21.15.86:0/2516694859 >> 172.21.15.86:0/2516694859 conn(0x7ff99c0cc940 msgr2=0x7ff99c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff9a3e00640 1 -- 172.21.15.86:0/2516694859 shutdown_connections 2024-03-22T04:20:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.368+0000 7ff9a3e00640 1 -- 172.21.15.86:0/2516694859 wait complete. 2024-03-22T04:20:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 Processor -- start 2024-03-22T04:20:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 -- start start 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b2360 0x7ff99c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff99c0b3600 0x7ff99c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99c15cee0 0x7ff99c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c15d490 con 0x7ff99c0b2360 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c15d310 con 0x7ff99c15cee0 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff99c16a1f0 con 0x7ff99c0b3600 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b2360 0x7ff99c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a0a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff99c0b3600 0x7ff99c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b2360 0x7ff99c150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57394/0 (socket says 172.21.15.86:57394) 2024-03-22T04:20:04.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 -- 172.21.15.86:0/960634249 learned_addr learned my addr 172.21.15.86:0/960634249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1e00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99c15cee0 0x7ff99c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 -- 172.21.15.86:0/960634249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff99c0b3600 msgr2=0x7ff99c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff99c0b3600 0x7ff99c15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 -- 172.21.15.86:0/960634249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99c15cee0 msgr2=0x7ff99c1525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99c15cee0 0x7ff99c1525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 -- 172.21.15.86:0/960634249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c150080 con 0x7ff99c0b2360 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a0a00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff99c0b3600 0x7ff99c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a1400640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b2360 0x7ff99c150e40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c500 tx=0x7ff99000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff999600640 1 -- 172.21.15.86:0/960634249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990016020 con 0x7ff99c0b2360 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff999600640 1 -- 172.21.15.86:0/960634249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff990005220 con 0x7ff99c0b2360 2024-03-22T04:20:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff999600640 1 -- 172.21.15.86:0/960634249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990007030 con 0x7ff99c0b2360 2024-03-22T04:20:04.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c150360 con 0x7ff99c0b2360 2024-03-22T04:20:04.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c14f6a0 con 0x7ff99c0b2360 2024-03-22T04:20:04.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.372+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99c074470 con 0x7ff99c0b2360 2024-03-22T04:20:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.376+0000 7ff999600640 1 -- 172.21.15.86:0/960634249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff9900071f0 con 0x7ff99c0b2360 2024-03-22T04:20:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.376+0000 7ff999600640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff97c067b20 0x7ff97c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.376+0000 7ff999600640 1 -- 172.21.15.86:0/960634249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7ff990085ae0 con 0x7ff99c0b2360 2024-03-22T04:20:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.376+0000 7ff9a0a00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff97c067b20 0x7ff97c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.376+0000 7ff9a0a00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff97c067b20 0x7ff97c069fe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff9880af000 tx=0x7ff9880a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.380+0000 7ff999600640 1 -- 172.21.15.86:0/960634249 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff990020020 con 0x7ff99c0b2360 2024-03-22T04:20:04.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.568+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff99c06e9f0 con 0x7ff99c0b2360 2024-03-22T04:20:04.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.984+0000 7ff999600640 1 -- 172.21.15.86:0/960634249 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v86) ==== 162+0+0 (secure 0 0 0) 0x7ff990055c30 con 0x7ff99c0b2360 2024-03-22T04:20:04.985 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff97c067b20 msgr2=0x7ff97c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff97c067b20 0x7ff97c069fe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff9880af000 tx=0x7ff9880a7000 comp rx=0 tx=0).stop 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b2360 msgr2=0x7ff99c150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b2360 0x7ff99c150e40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c500 tx=0x7ff99000c9c0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 shutdown_connections 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff97c067b20 0x7ff97c069fe0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99c15cee0 0x7ff99c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff99c0b3600 0x7ff99c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 --2- 172.21.15.86:0/960634249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99c0b2360 0x7ff99c150e40 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 >> 172.21.15.86:0/960634249 conn(0x7ff99c0cc940 msgr2=0x7ff99c0b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:04.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 shutdown_connections 2024-03-22T04:20:04.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:04.988+0000 7ff9a3e00640 1 -- 172.21.15.86:0/960634249 wait complete. 2024-03-22T04:20:05.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:20:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4504200640 1 Processor -- start 2024-03-22T04:20:05.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4504200640 1 -- start start 2024-03-22T04:20:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4504200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f44fc0ce6b0 0x7f44fc0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:05.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4504200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0d11c0 con 0x7f44fc15fed0 2024-03-22T04:20:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4504200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0cece0 con 0x7f44fc0d0de0 2024-03-22T04:20:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4504200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f44fc0cefc0 con 0x7f44fc0ce6b0 2024-03-22T04:20:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4500e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f44fc0ce6b0 0x7f44fc0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4500e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f44fc0ce6b0 0x7f44fc0cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57396/0 (socket says 172.21.15.86:57396) 2024-03-22T04:20:05.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4500e00640 1 -- 172.21.15.86:0/1353332595 learned_addr learned my addr 172.21.15.86:0/1353332595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4502200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f44fc15fed0 0x7f44fc1602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51794/0 (socket says 172.21.15.86:51794) 2024-03-22T04:20:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4500e00640 1 -- 172.21.15.86:0/1353332595 >> v1:172.21.15.196:6789/0 conn(0x7f44fc0d0de0 legacy=0x7f44fc0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4500e00640 1 -- 172.21.15.86:0/1353332595 >> v1:172.21.15.86:6789/0 conn(0x7f44fc15fed0 legacy=0x7f44fc1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:05.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.148+0000 7f4500e00640 1 -- 172.21.15.86:0/1353332595 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc0cf2a0 con 0x7f44fc0ce6b0 2024-03-22T04:20:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f4500e00640 1 --2- 172.21.15.86:0/1353332595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0ce6b0 0x7f44fc0cfe70 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009870 tx=0x7f44e801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca8465ee4a75382 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:05.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f44fbe00640 1 -- 172.21.15.86:0/1353332595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80b2070 con 0x7f44fc0ce6b0 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f44fbe00640 1 -- 172.21.15.86:0/1353332595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e80a9070 con 0x7f44fc0ce6b0 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f44fbe00640 1 -- 172.21.15.86:0/1353332595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80ad880 con 0x7f44fc0ce6b0 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f4504200640 1 -- 172.21.15.86:0/1353332595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0ce6b0 msgr2=0x7f44fc0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f4504200640 1 --2- 172.21.15.86:0/1353332595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0ce6b0 0x7f44fc0cfe70 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009870 tx=0x7f44e801c560 comp rx=0 tx=0).stop 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f4504200640 1 -- 172.21.15.86:0/1353332595 shutdown_connections 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f4504200640 1 --2- 172.21.15.86:0/1353332595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0ce6b0 0x7f44fc0cfe70 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f4504200640 1 -- 172.21.15.86:0/1353332595 >> 172.21.15.86:0/1353332595 conn(0x7f44fc06f910 msgr2=0x7f44fc06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f4504200640 1 -- 172.21.15.86:0/1353332595 shutdown_connections 2024-03-22T04:20:05.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.152+0000 7f4504200640 1 -- 172.21.15.86:0/1353332595 wait complete. 2024-03-22T04:20:05.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 Processor -- start 2024-03-22T04:20:05.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 -- start start 2024-03-22T04:20:05.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc0ce6b0 0x7f44fc0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0d0de0 0x7f44fc167090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc15fed0 0x7f44fc153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc071fb0 con 0x7f44fc0d0de0 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc071e30 con 0x7f44fc0ce6b0 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc16a050 con 0x7f44fc15fed0 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0d0de0 0x7f44fc167090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0d0de0 0x7f44fc167090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57410/0 (socket says 172.21.15.86:57410) 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 -- 172.21.15.86:0/1463655100 learned_addr learned my addr 172.21.15.86:0/1463655100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:05.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4502200640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc15fed0 0x7f44fc153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4501800640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc0ce6b0 0x7f44fc0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 -- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc15fed0 msgr2=0x7f44fc153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc15fed0 0x7f44fc153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 -- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc0ce6b0 msgr2=0x7f44fc0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc0ce6b0 0x7f44fc0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 -- 172.21.15.86:0/1463655100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc1657b0 con 0x7f44fc0d0de0 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4502200640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc15fed0 0x7f44fc153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4501800640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc0ce6b0 0x7f44fc0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4500e00640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0d0de0 0x7f44fc167090 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f44e80b0040 tx=0x7f44e8019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f44f9600640 1 -- 172.21.15.86:0/1463655100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80bb070 con 0x7f44fc0d0de0 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f44f9600640 1 -- 172.21.15.86:0/1463655100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e80b6420 con 0x7f44fc0d0de0 2024-03-22T04:20:05.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f44f9600640 1 -- 172.21.15.86:0/1463655100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80b2040 con 0x7f44fc0d0de0 2024-03-22T04:20:05.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc165a90 con 0x7f44fc0d0de0 2024-03-22T04:20:05.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.156+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc06ca50 con 0x7f44fc0d0de0 2024-03-22T04:20:05.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.160+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc068650 con 0x7f44fc0d0de0 2024-03-22T04:20:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.160+0000 7f44f9600640 1 -- 172.21.15.86:0/1463655100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f44e80ab070 con 0x7f44fc0d0de0 2024-03-22T04:20:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.160+0000 7f44f9600640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.160+0000 7f44f9600640 1 -- 172.21.15.86:0/1463655100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7f44e811efe0 con 0x7f44fc0d0de0 2024-03-22T04:20:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.160+0000 7f4501800640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:05.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.160+0000 7f4501800640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f44f0001710 tx=0x7f44f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.164+0000 7f44f9600640 1 -- 172.21.15.86:0/1463655100 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f44e8016150 con 0x7f44fc0d0de0 2024-03-22T04:20:05.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.352+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44fc06e980 con 0x7f44fc0d0de0 2024-03-22T04:20:05.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.352+0000 7f44f9600640 1 -- 172.21.15.86:0/1463655100 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5178 (secure 0 0 0) 0x7f44e80ef7d0 con 0x7f44fc0d0de0 2024-03-22T04:20:05.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:05.354 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/70563317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":70563317},{"type":"v1","addr":"172.21.15.196:6835","nonce":70563317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:20:04.991627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77159":{"gid":77159,"name":"d","rank":0,"incarnation":87,"state":"up:replay","state_seq":1,"addr":"172.21.15.196:6834/269054026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":269054026},{"type":"v1","addr":"172.21.15.196:6834","nonce":269054026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-22T04:20:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 87 2024-03-22T04:20:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 msgr2=0x7f44dc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f44f0001710 tx=0x7f44f0009040 comp rx=0 tx=0).stop 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0d0de0 msgr2=0x7f44fc167090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0d0de0 0x7f44fc167090 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f44e80b0040 tx=0x7f44e8019300 comp rx=0 tx=0).stop 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 shutdown_connections 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44dc067b20 0x7f44dc069fe0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44fc15fed0 0x7f44fc153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc0d0de0 0x7f44fc167090 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 --2- 172.21.15.86:0/1463655100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc0ce6b0 0x7f44fc0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 >> 172.21.15.86:0/1463655100 conn(0x7f44fc06f910 msgr2=0x7f44fc15f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 shutdown_connections 2024-03-22T04:20:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:05.356+0000 7f4504200640 1 -- 172.21.15.86:0/1463655100 wait complete. 2024-03-22T04:20:05.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:19:17.684626+0000', 'modified': '2024-03-22T04:20:04.991627+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 77159}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_77159': {'gid': 77159, 'name': 'd', 'rank': 0, 'incarnation': 87, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.196:6834/269054026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 269054026}, {'type': 'v1', 'addr': '172.21.15.196:6834', 'nonce': 269054026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:20:05.778 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_77159:up:replay 2024-03-22T04:20:06.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:20:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.924+0000 7feca2200640 1 Processor -- start 2024-03-22T04:20:06.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.924+0000 7feca2200640 1 -- start start 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7feca2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fec9c0ad330 0x7fec9c15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7feca2200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 0x7fec9c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7feca2200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9c15c7c0 con 0x7fec9c0ad710 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7feca2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fec9c06dc40 con 0x7fec9c0ad330 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7feca2200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fec9c06dea0 con 0x7fec9c06ef40 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7fec9b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fec9c0ad330 0x7fec9c15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7fec9b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fec9c0ad330 0x7fec9c15d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33206/0 (socket says 172.21.15.86:33206) 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7fec9b400640 1 -- 172.21.15.86:0/2785330786 learned_addr learned my addr 172.21.15.86:0/2785330786 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7fec92a00640 1 --2- 172.21.15.86:0/2785330786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 0x7fec9c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7fec9b400640 1 -- 172.21.15.86:0/2785330786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 msgr2=0x7fec9c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:06.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7fec9b400640 1 --2- 172.21.15.86:0/2785330786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 0x7fec9c06d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:06.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7fec9b400640 1 -- 172.21.15.86:0/2785330786 >> v1:172.21.15.86:6789/0 conn(0x7fec9c0ad710 legacy=0x7fec9c13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:06.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.928+0000 7fec9b400640 1 -- 172.21.15.86:0/2785330786 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c06e180 con 0x7fec9c0ad330 2024-03-22T04:20:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7fec9b400640 1 --2- 172.21.15.86:0/2785330786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c15d4a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009870 tx=0x7fec8c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b81f46fd3a74c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7fec9aa00640 1 -- 172.21.15.86:0/2785330786 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0ab640 con 0x7fec9c0ad330 2024-03-22T04:20:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7fec9aa00640 1 -- 172.21.15.86:0/2785330786 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c0b0070 con 0x7fec9c0ad330 2024-03-22T04:20:06.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7fec9aa00640 1 -- 172.21.15.86:0/2785330786 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0aba00 con 0x7fec9c0ad330 2024-03-22T04:20:06.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- 172.21.15.86:0/2785330786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 msgr2=0x7fec9c15d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:06.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 --2- 172.21.15.86:0/2785330786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c15d4a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009870 tx=0x7fec8c01c530 comp rx=0 tx=0).stop 2024-03-22T04:20:06.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- 172.21.15.86:0/2785330786 shutdown_connections 2024-03-22T04:20:06.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 --2- 172.21.15.86:0/2785330786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 0x7fec9c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:06.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 --2- 172.21.15.86:0/2785330786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c15d4a0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:06.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- 172.21.15.86:0/2785330786 >> 172.21.15.86:0/2785330786 conn(0x7fec9c0b1df0 msgr2=0x7fec9c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:06.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- 172.21.15.86:0/2785330786 shutdown_connections 2024-03-22T04:20:06.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- 172.21.15.86:0/2785330786 wait complete. 2024-03-22T04:20:06.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 Processor -- start 2024-03-22T04:20:06.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- start start 2024-03-22T04:20:06.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 0x7fec9c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c166980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec9c0ad710 0x7fec9c0990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c1690e0 con 0x7fec9c0ad330 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c168f60 con 0x7fec9c0ad710 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.932+0000 7feca2200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fec9c169260 con 0x7fec9c06ef40 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c166980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33214/0 (socket says 172.21.15.86:33214) 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 -- 172.21.15.86:0/1949939380 learned_addr learned my addr 172.21.15.86:0/1949939380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec9b400640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 0x7fec9c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec9be00640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec9c0ad710 0x7fec9c0990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 -- 172.21.15.86:0/1949939380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 msgr2=0x7fec9c153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 0x7fec9c153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 -- 172.21.15.86:0/1949939380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec9c0ad710 msgr2=0x7fec9c0990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec9c0ad710 0x7fec9c0990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:06.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 -- 172.21.15.86:0/1949939380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c154530 con 0x7fec9c0ad330 2024-03-22T04:20:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec92a00640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c166980 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fec8400d5f0 tx=0x7fec8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec93e00640 1 -- 172.21.15.86:0/1949939380 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec840125f0 con 0x7fec9c0ad330 2024-03-22T04:20:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec93e00640 1 -- 172.21.15.86:0/1949939380 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec84019070 con 0x7fec9c0ad330 2024-03-22T04:20:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c154810 con 0x7fec9c0ad330 2024-03-22T04:20:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec93e00640 1 -- 172.21.15.86:0/1949939380 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec840129b0 con 0x7fec9c0ad330 2024-03-22T04:20:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c154dd0 con 0x7fec9c0ad330 2024-03-22T04:20:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9c068650 con 0x7fec9c0ad330 2024-03-22T04:20:06.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.936+0000 7fec93e00640 1 -- 172.21.15.86:0/1949939380 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fec84012b50 con 0x7fec9c0ad330 2024-03-22T04:20:06.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.940+0000 7fec93e00640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fec78067b20 0x7fec78069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:06.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.940+0000 7fec93e00640 1 -- 172.21.15.86:0/1949939380 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7fec84086460 con 0x7fec9c0ad330 2024-03-22T04:20:06.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.940+0000 7fec9b400640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fec78067b20 0x7fec78069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:06.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.940+0000 7fec9b400640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fec78067b20 0x7fec78069fe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0af000 tx=0x7fec8c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:06.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:06.944+0000 7fec93e00640 1 -- 172.21.15.86:0/1949939380 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fec84056c80 con 0x7fec9c0ad330 2024-03-22T04:20:07.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec9c156520 con 0x7fec9c0ad330 2024-03-22T04:20:07.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7fec93e00640 1 -- 172.21.15.86:0/1949939380 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7fec84056580 con 0x7fec9c0ad330 2024-03-22T04:20:07.132 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:07.133 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/70563317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":70563317},{"type":"v1","addr":"172.21.15.196:6835","nonce":70563317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:20:06.002718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77159":{"gid":77159,"name":"d","rank":0,"incarnation":87,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.196:6834/269054026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":269054026},{"type":"v1","addr":"172.21.15.196:6834","nonce":269054026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-22T04:20:07.133 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 89 2024-03-22T04:20:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fec78067b20 msgr2=0x7fec78069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7feca2200640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fec78067b20 0x7fec78069fe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0af000 tx=0x7fec8c0a7000 comp rx=0 tx=0).stop 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 msgr2=0x7fec9c166980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7feca2200640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c166980 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fec8400d5f0 tx=0x7fec8400dab0 comp rx=0 tx=0).stop 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 shutdown_connections 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7feca2200640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fec78067b20 0x7fec78069fe0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.132+0000 7feca2200640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec9c0ad710 0x7fec9c0990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.136+0000 7feca2200640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec9c0ad330 0x7fec9c166980 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.136+0000 7feca2200640 1 --2- 172.21.15.86:0/1949939380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec9c06ef40 0x7fec9c153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.136+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 >> 172.21.15.86:0/1949939380 conn(0x7fec9c0b1df0 msgr2=0x7fec9c15edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.136+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 shutdown_connections 2024-03-22T04:20:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:07.136+0000 7feca2200640 1 -- 172.21.15.86:0/1949939380 wait complete. 2024-03-22T04:20:07.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:19:17.684626+0000', 'modified': '2024-03-22T04:20:06.002718+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 77159}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_77159': {'gid': 77159, 'name': 'd', 'rank': 0, 'incarnation': 87, 'state': 'up:rejoin', 'state_seq': 16, 'addr': '172.21.15.196:6834/269054026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 269054026}, {'type': 'v1', 'addr': '172.21.15.196:6834', 'nonce': 269054026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:20:07.556 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_77159:up:rejoin 2024-03-22T04:20:08.288 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:20:08.290+0000 7fbb6a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:20:08.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:20:08.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5540c00640 1 Processor -- start 2024-03-22T04:20:08.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5540c00640 1 -- start start 2024-03-22T04:20:08.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5540c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f553c15b3f0 0x7f553c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:08.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5540c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c0cc3e0 0x7f553c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5540c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553c071920 con 0x7f553c0b30f0 2024-03-22T04:20:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5540c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f553c0716d0 con 0x7f553c15b3f0 2024-03-22T04:20:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5540c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f553c06d990 con 0x7f553c0cc3e0 2024-03-22T04:20:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f553a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f553c0b30f0 0x7f553c0ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44102/0 (socket says 172.21.15.86:44102) 2024-03-22T04:20:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f553a800640 1 -- 172.21.15.86:0/3004045096 learned_addr learned my addr 172.21.15.86:0/3004045096 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5539e00640 1 --2- 172.21.15.86:0/3004045096 >> v2:172.21.15.196:3300/0 conn(0x7f553c15b3f0 0x7f553c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5531400640 1 --2- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c0cc3e0 0x7f553c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5539400640 1 -- 172.21.15.86:0/3004045096 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1469816505 0 0) 0x7f553c071920 con 0x7f553c0b30f0 2024-03-22T04:20:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5539400640 1 -- 172.21.15.86:0/3004045096 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5520003530 con 0x7f553c0b30f0 2024-03-22T04:20:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5539e00640 1 -- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c0cc3e0 msgr2=0x7f553c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5539e00640 1 --2- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c0cc3e0 0x7f553c072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5539e00640 1 -- 172.21.15.86:0/3004045096 >> v1:172.21.15.86:6789/0 conn(0x7f553c0b30f0 legacy=0x7f553c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5539e00640 1 -- 172.21.15.86:0/3004045096 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c071c00 con 0x7f553c15b3f0 2024-03-22T04:20:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.704+0000 7f5531400640 1 --2- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c0cc3e0 0x7f553c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5539e00640 1 --2- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c15b3f0 0x7f553c0729c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f552c0097c0 tx=0x7f552c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef586ccbef3f9e9d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5539400640 1 -- 172.21.15.86:0/3004045096 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0b0070 con 0x7f553c15b3f0 2024-03-22T04:20:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5539400640 1 -- 172.21.15.86:0/3004045096 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c0ab600 con 0x7f553c15b3f0 2024-03-22T04:20:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5539400640 1 -- 172.21.15.86:0/3004045096 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0ab9d0 con 0x7f553c15b3f0 2024-03-22T04:20:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5540c00640 1 -- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c15b3f0 msgr2=0x7f553c0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5540c00640 1 --2- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c15b3f0 0x7f553c0729c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f552c0097c0 tx=0x7f552c01c410 comp rx=0 tx=0).stop 2024-03-22T04:20:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5540c00640 1 -- 172.21.15.86:0/3004045096 shutdown_connections 2024-03-22T04:20:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5540c00640 1 --2- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c0cc3e0 0x7f553c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5540c00640 1 --2- 172.21.15.86:0/3004045096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c15b3f0 0x7f553c0729c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5540c00640 1 -- 172.21.15.86:0/3004045096 >> 172.21.15.86:0/3004045096 conn(0x7f553c0b1df0 msgr2=0x7f553c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5540c00640 1 -- 172.21.15.86:0/3004045096 shutdown_connections 2024-03-22T04:20:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.708+0000 7f5540c00640 1 -- 172.21.15.86:0/3004045096 wait complete. 2024-03-22T04:20:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 Processor -- start 2024-03-22T04:20:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 -- start start 2024-03-22T04:20:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c0b30f0 0x7f553c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c0cc3e0 0x7f553c14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c15b3f0 0x7f553c152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f553c168580 con 0x7f553c0b30f0 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f553c168400 con 0x7f553c0cc3e0 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f553c168700 con 0x7f553c15b3f0 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c0b30f0 0x7f553c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c0b30f0 0x7f553c1542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33224/0 (socket says 172.21.15.86:33224) 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 -- 172.21.15.86:0/1254615047 learned_addr learned my addr 172.21.15.86:0/1254615047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5531400640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c0cc3e0 0x7f553c14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f553a800640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c15b3f0 0x7f553c152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 -- 172.21.15.86:0/1254615047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c15b3f0 msgr2=0x7f553c152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c15b3f0 0x7f553c152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 -- 172.21.15.86:0/1254615047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c0cc3e0 msgr2=0x7f553c14abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c0cc3e0 0x7f553c14abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 -- 172.21.15.86:0/1254615047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c153210 con 0x7f553c0b30f0 2024-03-22T04:20:08.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5539e00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c0b30f0 0x7f553c1542b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f552c0ae040 tx=0x7f552c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:08.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5532a00640 1 -- 172.21.15.86:0/1254615047 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0b0050 con 0x7f553c0b30f0 2024-03-22T04:20:08.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5532a00640 1 -- 172.21.15.86:0/1254615047 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c0bc070 con 0x7f553c0b30f0 2024-03-22T04:20:08.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5532a00640 1 -- 172.21.15.86:0/1254615047 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c006e20 con 0x7f553c0b30f0 2024-03-22T04:20:08.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c1534f0 con 0x7f553c0b30f0 2024-03-22T04:20:08.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c153ab0 con 0x7f553c0b30f0 2024-03-22T04:20:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.712+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c068650 con 0x7f553c0b30f0 2024-03-22T04:20:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.716+0000 7f5532a00640 1 -- 172.21.15.86:0/1254615047 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f552c0b5830 con 0x7f553c0b30f0 2024-03-22T04:20:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.716+0000 7f5532a00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5518067b20 0x7f5518069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.716+0000 7f5532a00640 1 -- 172.21.15.86:0/1254615047 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7f552c1207d0 con 0x7f553c0b30f0 2024-03-22T04:20:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.716+0000 7f5531400640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5518067b20 0x7f5518069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.716+0000 7f5531400640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5518067b20 0x7f5518069fe0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5524009580 tx=0x7f5524009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:08.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.720+0000 7f5532a00640 1 -- 172.21.15.86:0/1254615047 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f552c016090 con 0x7f553c0b30f0 2024-03-22T04:20:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f553c155040 con 0x7f553c0b30f0 2024-03-22T04:20:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5532a00640 1 -- 172.21.15.86:0/1254615047 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5188 (secure 0 0 0) 0x7f552c0f1080 con 0x7f553c0b30f0 2024-03-22T04:20:08.907 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:08.907 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/70563317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":70563317},{"type":"v1","addr":"172.21.15.196:6835","nonce":70563317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:20:08.006787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77159":{"gid":77159,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6834/269054026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":269054026},{"type":"v1","addr":"172.21.15.196:6834","nonce":269054026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77159,"qdb_cluster":[77159]},"id":7}]} 2024-03-22T04:20:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 90 2024-03-22T04:20:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5518067b20 msgr2=0x7f5518069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5518067b20 0x7f5518069fe0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5524009580 tx=0x7f5524009190 comp rx=0 tx=0).stop 2024-03-22T04:20:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c0b30f0 msgr2=0x7f553c1542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c0b30f0 0x7f553c1542b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f552c0ae040 tx=0x7f552c01ac50 comp rx=0 tx=0).stop 2024-03-22T04:20:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 shutdown_connections 2024-03-22T04:20:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5518067b20 0x7f5518069fe0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f553c0cc3e0 0x7f553c14abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f553c15b3f0 0x7f553c152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 --2- 172.21.15.86:0/1254615047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c0b30f0 0x7f553c1542b0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 >> 172.21.15.86:0/1254615047 conn(0x7f553c0b1df0 msgr2=0x7f553c0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 shutdown_connections 2024-03-22T04:20:08.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:08.908+0000 7f5540c00640 1 -- 172.21.15.86:0/1254615047 wait complete. 2024-03-22T04:20:09.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:19:17.684626+0000', 'modified': '2024-03-22T04:20:08.006787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 77159}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_77159': {'gid': 77159, 'name': 'd', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.196:6834/269054026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 269054026}, {'type': 'v1', 'addr': '172.21.15.196:6834', 'nonce': 269054026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 77159, 'qdb_cluster': [77159]} 2024-03-22T04:20:09.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:20:09.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-22T04:20:09.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f511ca00640 1 Processor -- start 2024-03-22T04:20:09.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f511ca00640 1 -- start start 2024-03-22T04:20:09.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f511ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f51180ad330 0x7f511815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f511ca00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f511806ef40 0x7f511806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f511ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f511815c7c0 con 0x7f51180ad710 2024-03-22T04:20:09.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f511ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f511806dc40 con 0x7f51180ad330 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f511ca00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f511806dea0 con 0x7f511806ef40 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f51180ad330 0x7f511815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f51180ad330 0x7f511815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33234/0 (socket says 172.21.15.86:33234) 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115e00640 1 -- 172.21.15.86:0/1878959677 learned_addr learned my addr 172.21.15.86:0/1878959677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5116800640 1 --1- 172.21.15.86:0/1878959677 >> v1:172.21.15.86:6789/0 conn(0x7f51180ad710 0x7f511813fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44118/0 (socket says 172.21.15.86:44118) 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115400640 1 --2- 172.21.15.86:0/1878959677 >> v2:172.21.15.196:3300/0 conn(0x7f511806ef40 0x7f511806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115e00640 1 -- 172.21.15.86:0/1878959677 >> v2:172.21.15.196:3300/0 conn(0x7f511806ef40 msgr2=0x7f511806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115e00640 1 --2- 172.21.15.86:0/1878959677 >> v2:172.21.15.196:3300/0 conn(0x7f511806ef40 0x7f511806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115e00640 1 -- 172.21.15.86:0/1878959677 >> v1:172.21.15.86:6789/0 conn(0x7f51180ad710 legacy=0x7f511813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115e00640 1 -- 172.21.15.86:0/1878959677 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511806e180 con 0x7f51180ad330 2024-03-22T04:20:09.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.472+0000 7f5115400640 1 --2- 172.21.15.86:0/1878959677 >> v2:172.21.15.196:3300/0 conn(0x7f511806ef40 0x7f511806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f5115e00640 1 --2- 172.21.15.86:0/1878959677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f511815d4a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f511806fbe0 tx=0x7f510801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f1a6c618323dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f5114a00640 1 -- 172.21.15.86:0/1878959677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080a5790 con 0x7f51180ad330 2024-03-22T04:20:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f5114a00640 1 -- 172.21.15.86:0/1878959677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51080a5930 con 0x7f51180ad330 2024-03-22T04:20:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f5114a00640 1 -- 172.21.15.86:0/1878959677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080a5d00 con 0x7f51180ad330 2024-03-22T04:20:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 -- 172.21.15.86:0/1878959677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 msgr2=0x7f511815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 --2- 172.21.15.86:0/1878959677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f511815d4a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f511806fbe0 tx=0x7f510801c9c0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 -- 172.21.15.86:0/1878959677 shutdown_connections 2024-03-22T04:20:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 --2- 172.21.15.86:0/1878959677 >> v2:172.21.15.196:3300/0 conn(0x7f511806ef40 0x7f511806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 --2- 172.21.15.86:0/1878959677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f511815d4a0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 -- 172.21.15.86:0/1878959677 >> 172.21.15.86:0/1878959677 conn(0x7f51180b1df0 msgr2=0x7f51180b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 -- 172.21.15.86:0/1878959677 shutdown_connections 2024-03-22T04:20:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 -- 172.21.15.86:0/1878959677 wait complete. 2024-03-22T04:20:09.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 Processor -- start 2024-03-22T04:20:09.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.476+0000 7f511ca00640 1 -- start start 2024-03-22T04:20:09.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f511ca00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511806ef40 0x7f5118159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f511ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f5118155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f511ca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51180ad710 0x7f51181fab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f511ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51181690e0 con 0x7f51180ad330 2024-03-22T04:20:09.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f511ca00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5118168f60 con 0x7f51180ad710 2024-03-22T04:20:09.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f511ca00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5118169260 con 0x7f511806ef40 2024-03-22T04:20:09.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f5118155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f5118155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33250/0 (socket says 172.21.15.86:33250) 2024-03-22T04:20:09.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511806ef40 0x7f5118159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 -- 172.21.15.86:0/1234818876 learned_addr learned my addr 172.21.15.86:0/1234818876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5116800640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51180ad710 0x7f51181fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 -- 172.21.15.86:0/1234818876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511806ef40 msgr2=0x7f5118159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511806ef40 0x7f5118159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 -- 172.21.15.86:0/1234818876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51180ad710 msgr2=0x7f51181fab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51180ad710 0x7f51181fab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 -- 172.21.15.86:0/1234818876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51181582b0 con 0x7f51180ad330 2024-03-22T04:20:09.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115e00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511806ef40 0x7f5118159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:09.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f5115400640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f5118155730 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f510400d830 tx=0x7f510400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f510e000640 1 -- 172.21.15.86:0/1234818876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104018020 con 0x7f51180ad330 2024-03-22T04:20:09.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f510e000640 1 -- 172.21.15.86:0/1234818876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51040096c0 con 0x7f51180ad330 2024-03-22T04:20:09.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f510e000640 1 -- 172.21.15.86:0/1234818876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104009ab0 con 0x7f51180ad330 2024-03-22T04:20:09.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f511ca00640 1 -- 172.21.15.86:0/1234818876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5118156280 con 0x7f51180ad330 2024-03-22T04:20:09.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.480+0000 7f511ca00640 1 -- 172.21.15.86:0/1234818876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5118157fd0 con 0x7f51180ad330 2024-03-22T04:20:09.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f510e000640 1 -- 172.21.15.86:0/1234818876 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f510401b020 con 0x7f51180ad330 2024-03-22T04:20:09.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f510e000640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50f8067b20 0x7f50f8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f510e000640 1 -- 172.21.15.86:0/1234818876 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7f51040866a0 con 0x7f51180ad330 2024-03-22T04:20:09.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50f8067b20 0x7f50f8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f511ff64000 1 Processor -- start 2024-03-22T04:20:09.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f511ff64000 1 -- start start 2024-03-22T04:20:09.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50f8067b20 0x7f50f8069fe0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f51080ae000 tx=0x7f51080a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f511ff64000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f511ff64000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a0ad790 0x55cb4a0606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f511ff64000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a060c20 0x55cb4a061000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f511ff64000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb49e5e140 con 0x55cb4a0ace70 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f511ff64000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb49eb1b10 con 0x55cb4a0ad790 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f511ff64000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cb49e38c50 con 0x55cb4a060c20 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33256/0 (socket says 172.21.15.86:33256) 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 -- 172.21.15.86:0/75277740 learned_addr learned my addr 172.21.15.86:0/75277740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115400640 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a0ad790 0x55cb4a0606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5116800640 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a060c20 0x55cb4a061000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 -- 172.21.15.86:0/75277740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a060c20 msgr2=0x55cb4a061000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a060c20 0x55cb4a061000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 -- 172.21.15.86:0/75277740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a0ad790 msgr2=0x55cb4a0606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a0ad790 0x55cb4a0606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 -- 172.21.15.86:0/75277740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb49ef6f20 con 0x55cb4a0ace70 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5115e00640 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0ad250 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f510801cf30 tx=0x7f5108004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c281104a462b25 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5102a00640 1 -- 172.21.15.86:0/75277740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080c6070 con 0x55cb4a0ace70 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5102a00640 1 -- 172.21.15.86:0/75277740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51080b6970 con 0x55cb4a0ace70 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.484+0000 7f5102a00640 1 -- 172.21.15.86:0/75277740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080b6d20 con 0x55cb4a0ace70 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- 172.21.15.86:0/75277740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 msgr2=0x55cb4a0ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0ad250 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f510801cf30 tx=0x7f5108004ac0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- 172.21.15.86:0/75277740 shutdown_connections 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a0ad790 0x55cb4a0606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a060c20 0x55cb4a061000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 --2- 172.21.15.86:0/75277740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0ad250 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- 172.21.15.86:0/75277740 >> 172.21.15.86:0/75277740 conn(0x55cb4a08dc10 msgr2=0x55cb4a090000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- 172.21.15.86:0/75277740 shutdown_connections 2024-03-22T04:20:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- 172.21.15.86:0/75277740 wait complete. 2024-03-22T04:20:09.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- start start 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 Processor -- start 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a068d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ad790 0x55cb4a065d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb49e6bab0 con 0x55cb4a0ace70 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cb49eadac0 con 0x55cb4a060c20 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cb49e57e80 con 0x55cb4a0ad790 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 10 client.0 _ll_register_callbacks cb 0x55cb4a04e380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33260/0 (socket says 172.21.15.86:33260) 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 -- 172.21.15.86:0/2547701679 learned_addr learned my addr 172.21.15.86:0/2547701679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115e00640 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a068d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5116800640 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ad790 0x55cb4a065d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 -- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ad790 msgr2=0x55cb4a065d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ad790 0x55cb4a065d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 -- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 msgr2=0x55cb4a068d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a068d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 -- 172.21.15.86:0/2547701679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb49e87570 con 0x55cb4a0ace70 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f5115400640 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0657e0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5104056570 tx=0x7f5104056870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232d7266ecae2c82 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f50f7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f50f7400640 1 -- 172.21.15.86:0/2547701679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510406e070 con 0x55cb4a0ace70 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f50f7400640 1 -- 172.21.15.86:0/2547701679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7f5104081c80 con 0x55cb4a0ace70 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- 172.21.15.86:0/2547701679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51080b6970 con 0x55cb4a0ace70 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f50f7400640 1 -- 172.21.15.86:0/2547701679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f510405f140 con 0x55cb4a0ace70 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 10 client.8745 fetch_fsmap learned FSMap version 90 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- 172.21.15.86:0/2547701679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55cb49e7d880 con 0x55cb4a0ace70 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f50f7400640 1 -- 172.21.15.86:0/2547701679 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51080b6970 con 0x55cb4a0ace70 2024-03-22T04:20:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f50f7400640 1 -- 172.21.15.86:0/2547701679 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7f5104013050 con 0x55cb4a0ace70 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 10 client.8745 fetch_fsmap finished waiting for FSMap version 90 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 10 client.8745 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 20 client.8745 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55cb4a066ac0 0x55cb4a0be120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 4 client.8745 mds_command: new command op to 77159 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.488+0000 7f511ff64000 1 -- 172.21.15.86:0/2547701679 --> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb49e4cbc0 con 0x55cb4a066ac0 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f5115e00640 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55cb4a066ac0 0x55cb4a0be120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f5115e00640 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55cb4a066ac0 0x55cb4a0be120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446fd32e0164e218 server_cookie=87341885bd9dbd51 in_seq=0 out_seq=0 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f50f7400640 10 client.8745 ms_handle_connect on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f50f7400640 1 -- 172.21.15.86:0/2547701679 <== mds.0 v2:172.21.15.196:6832/269054026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cb49e4cbc0 con 0x55cb4a066ac0 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f50f7400640 10 client.8745 handle_command_reply: tid=0 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 client.8745 shutdown 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 -- 172.21.15.86:0/2547701679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 msgr2=0x55cb4a0657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0657e0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5104056570 tx=0x7f5104056870 comp rx=0 tx=0).stop 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 -- 172.21.15.86:0/2547701679 shutdown_connections 2024-03-22T04:20:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55cb4a066ac0 0x55cb4a0be120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ad790 0x55cb4a065d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a0ace70 0x55cb4a0657e0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f50f7400640 0 client.8745 ms_handle_reset on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 --2- 172.21.15.86:0/2547701679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a068d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 -- 172.21.15.86:0/2547701679 >> 172.21.15.86:0/2547701679 conn(0x55cb4a08dc10 msgr2=0x55cb4a090000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 -- 172.21.15.86:0/2547701679 shutdown_connections 2024-03-22T04:20:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 1 -- 172.21.15.86:0/2547701679 wait complete. 2024-03-22T04:20:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.492+0000 7f511ff64000 20 client.8745 trim_cache size 0 max 16384 2024-03-22T04:20:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.508+0000 7f511ff64000 1 Processor -- start 2024-03-22T04:20:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.508+0000 7f511ff64000 1 -- start start 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5118204380 con 0x55cb4a08dc10 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51182033c0 con 0x55cb4a060c20 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f51180b3870 con 0x55cb4a0ace70 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33264/0 (socket says 172.21.15.86:33264) 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 -- 172.21.15.86:0/3711678240 learned_addr learned my addr 172.21.15.86:0/3711678240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115400640 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5116800640 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 -- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 msgr2=0x55cb4a0653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 -- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 msgr2=0x55cb4a0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a0ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 -- 172.21.15.86:0/3711678240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb49e39e80 con 0x55cb4a08dc10 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5116800640 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5115e00640 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0ac350 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f510801cea0 tx=0x7f51080b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334ffde8440e16a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5102000640 1 -- 172.21.15.86:0/3711678240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080dd070 con 0x55cb4a08dc10 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5102000640 1 -- 172.21.15.86:0/3711678240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51080b6b70 con 0x55cb4a08dc10 2024-03-22T04:20:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f5102000640 1 -- 172.21.15.86:0/3711678240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080c6050 con 0x55cb4a08dc10 2024-03-22T04:20:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- 172.21.15.86:0/3711678240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 msgr2=0x55cb4a0ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0ac350 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f510801cea0 tx=0x7f51080b7000 comp rx=0 tx=0).stop 2024-03-22T04:20:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- 172.21.15.86:0/3711678240 shutdown_connections 2024-03-22T04:20:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 --2- 172.21.15.86:0/3711678240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0ac350 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- 172.21.15.86:0/3711678240 >> 172.21.15.86:0/3711678240 conn(0x55cb4a067f40 msgr2=0x55cb4a05fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- 172.21.15.86:0/3711678240 shutdown_connections 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- 172.21.15.86:0/3711678240 wait complete. 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 -- start start 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.512+0000 7f511ff64000 1 Processor -- start 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a11ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0beee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0bf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb49eb0e20 con 0x55cb4a08dc10 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51180b3ec0 con 0x55cb4a060c20 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55cb49e76710 con 0x55cb4a0ace70 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 10 client.0 _ll_register_callbacks cb 0x7f5104056730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0beee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a11ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0beee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33278/0 (socket says 172.21.15.86:33278) 2024-03-22T04:20:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 -- 172.21.15.86:0/2635440888 learned_addr learned my addr 172.21.15.86:0/2635440888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5116800640 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 -- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 msgr2=0x55cb4a0bf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0bf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 -- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 msgr2=0x55cb4a11ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a11ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 -- 172.21.15.86:0/2635440888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb49e45750 con 0x55cb4a08dc10 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115e00640 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a11ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115400640 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0beee0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f5104052c80 tx=0x7f51040848b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cbfcfc3d6daff7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 1 -- 172.21.15.86:0/2635440888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510406e040 con 0x55cb4a08dc10 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 1 -- 172.21.15.86:0/2635440888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7f510407ac80 con 0x55cb4a08dc10 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- 172.21.15.86:0/2635440888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51080b6b70 con 0x55cb4a08dc10 2024-03-22T04:20:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 1 -- 172.21.15.86:0/2635440888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f51040561d0 con 0x55cb4a08dc10 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 10 client.8751 fetch_fsmap learned FSMap version 90 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- 172.21.15.86:0/2635440888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55cb49e58450 con 0x55cb4a08dc10 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 1 -- 172.21.15.86:0/2635440888 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51080b6b70 con 0x55cb4a08dc10 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 1 -- 172.21.15.86:0/2635440888 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7f51040525c0 con 0x55cb4a08dc10 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 10 client.8751 fetch_fsmap finished waiting for FSMap version 90 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 10 client.8751 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 20 client.8751 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55cb4a0ad790 0x55cb4a0bf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 4 client.8751 mds_command: new command op to 77159 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- 172.21.15.86:0/2635440888 --> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] -- command(tid 0: {"prefix": "status"}) -- 0x55cb49ebf050 con 0x55cb4a0ad790 2024-03-22T04:20:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115e00640 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55cb4a0ad790 0x55cb4a0bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f5115e00640 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55cb4a0ad790 0x55cb4a0bf960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0032deb0195083 server_cookie=43f956243d29a56 in_seq=0 out_seq=0 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 10 client.8751 ms_handle_connect on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 1 -- 172.21.15.86:0/2635440888 <== mds.0 v2:172.21.15.196:6832/269054026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55cb49ebf050 con 0x55cb4a0ad790 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 10 client.8751 handle_command_reply: tid=0 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 client.8751 shutdown 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- 172.21.15.86:0/2635440888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 msgr2=0x55cb4a0beee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0beee0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f5104052c80 tx=0x7f51040848b0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- 172.21.15.86:0/2635440888 shutdown_connections 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55cb4a0ad790 0x55cb4a0bf960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cb4a0ace70 0x55cb4a0bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4a08dc10 0x55cb4a0beee0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f50f7400640 0 client.8751 ms_handle_reset on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 --2- 172.21.15.86:0/2635440888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cb4a060c20 0x55cb4a11ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- 172.21.15.86:0/2635440888 >> 172.21.15.86:0/2635440888 conn(0x55cb4a067f40 msgr2=0x55cb4a0aaf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- 172.21.15.86:0/2635440888 shutdown_connections 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 1 -- 172.21.15.86:0/2635440888 wait complete. 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ff64000 20 client.8751 trim_cache size 0 max 16384 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 77159, 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.5271517289999998, 2024-03-22T04:20:09.522 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 90, 2024-03-22T04:20:09.523 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 110, 2024-03-22T04:20:09.523 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 110, 2024-03-22T04:20:09.523 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 57.077171327000002 2024-03-22T04:20:09.523 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:20:09.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.516+0000 7f511ca00640 1 -- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50f8067b20 msgr2=0x7f50f8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50f8067b20 0x7f50f8069fe0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f51080ae000 tx=0x7f51080a7070 comp rx=0 tx=0).stop 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 -- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 msgr2=0x7f5118155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f5118155730 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f510400d830 tx=0x7f510400dcf0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 -- 172.21.15.86:0/1234818876 shutdown_connections 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f50f8067b20 0x7f50f8069fe0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51180ad710 0x7f51181fab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180ad330 0x7f5118155730 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 --2- 172.21.15.86:0/1234818876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511806ef40 0x7f5118159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 -- 172.21.15.86:0/1234818876 >> 172.21.15.86:0/1234818876 conn(0x7f51180b1df0 msgr2=0x7f511815e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 -- 172.21.15.86:0/1234818876 shutdown_connections 2024-03-22T04:20:09.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.520+0000 7f511ca00640 1 -- 172.21.15.86:0/1234818876 wait complete. 2024-03-22T04:20:09.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-22T04:20:09.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:20:09.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-22T04:20:09.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.680+0000 7fc720c00640 1 Processor -- start 2024-03-22T04:20:09.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc720c00640 1 -- start start 2024-03-22T04:20:09.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc720c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c15d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc720c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71c15d670 con 0x7fc71c06e450 2024-03-22T04:20:09.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc720c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71c0ccac0 con 0x7fc71c06f720 2024-03-22T04:20:09.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc720c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c0cc870 con 0x7fc71c156600 2024-03-22T04:20:09.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc71be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc71c06e450 0x7fc71c071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44130/0 (socket says 172.21.15.86:44130) 2024-03-22T04:20:09.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc71be00640 1 -- 172.21.15.86:0/740538549 learned_addr learned my addr 172.21.15.86:0/740538549 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:09.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc71b400640 1 --2- 172.21.15.86:0/740538549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc71b400640 1 -- 172.21.15.86:0/740538549 >> v1:172.21.15.196:6789/0 conn(0x7fc71c06f720 legacy=0x7fc71c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc71b400640 1 -- 172.21.15.86:0/740538549 >> v1:172.21.15.86:6789/0 conn(0x7fc71c06e450 legacy=0x7fc71c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc71aa00640 1 -- 172.21.15.86:0/740538549 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3378080843 0 0) 0x7fc71c15d670 con 0x7fc71c06e450 2024-03-22T04:20:09.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.684+0000 7fc71b400640 1 -- 172.21.15.86:0/740538549 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c0ccda0 con 0x7fc71c156600 2024-03-22T04:20:09.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc71b400640 1 --2- 172.21.15.86:0/740538549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c15d130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009770 tx=0x7fc70c01c280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5dfcd01189b8d09 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc71aa00640 1 -- 172.21.15.86:0/740538549 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0b0070 con 0x7fc71c156600 2024-03-22T04:20:09.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc71aa00640 1 -- 172.21.15.86:0/740538549 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70c0ab600 con 0x7fc71c156600 2024-03-22T04:20:09.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc71aa00640 1 -- 172.21.15.86:0/740538549 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0ab9d0 con 0x7fc71c156600 2024-03-22T04:20:09.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 -- 172.21.15.86:0/740538549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 msgr2=0x7fc71c15d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 --2- 172.21.15.86:0/740538549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c15d130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009770 tx=0x7fc70c01c280 comp rx=0 tx=0).stop 2024-03-22T04:20:09.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 -- 172.21.15.86:0/740538549 shutdown_connections 2024-03-22T04:20:09.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 --2- 172.21.15.86:0/740538549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c15d130 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 -- 172.21.15.86:0/740538549 >> 172.21.15.86:0/740538549 conn(0x7fc71c06b660 msgr2=0x7fc71c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 -- 172.21.15.86:0/740538549 shutdown_connections 2024-03-22T04:20:09.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 -- 172.21.15.86:0/740538549 wait complete. 2024-03-22T04:20:09.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 Processor -- start 2024-03-22T04:20:09.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.688+0000 7fc720c00640 1 -- start start 2024-03-22T04:20:09.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc720c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c06e450 0x7fc71c145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc720c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc71c06f720 0x7fc71c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc720c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc720c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c072da0 con 0x7fc71c06e450 2024-03-22T04:20:09.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc720c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c072c20 con 0x7fc71c06f720 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc720c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c072f20 con 0x7fc71c156600 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c06e450 0x7fc71c145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c06e450 0x7fc71c145710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33286/0 (socket says 172.21.15.86:33286) 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 -- 172.21.15.86:0/342270410 learned_addr learned my addr 172.21.15.86:0/342270410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71be00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc713e00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc71c06f720 0x7fc71c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 -- 172.21.15.86:0/342270410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 msgr2=0x7fc71c142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 -- 172.21.15.86:0/342270410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc71c06f720 msgr2=0x7fc71c1530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc71c06f720 0x7fc71c1530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 -- 172.21.15.86:0/342270410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c153840 con 0x7fc71c06e450 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc71b400640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c06e450 0x7fc71c145710 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc70c01a930 tx=0x7fc70c01aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc713400640 1 -- 172.21.15.86:0/342270410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0b0050 con 0x7fc71c06e450 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc713400640 1 -- 172.21.15.86:0/342270410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70c0bc070 con 0x7fc71c06e450 2024-03-22T04:20:09.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc713400640 1 -- 172.21.15.86:0/342270410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0b5a50 con 0x7fc71c06e450 2024-03-22T04:20:09.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc720c00640 1 -- 172.21.15.86:0/342270410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c15da20 con 0x7fc71c06e450 2024-03-22T04:20:09.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.692+0000 7fc720c00640 1 -- 172.21.15.86:0/342270410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c142f40 con 0x7fc71c06e450 2024-03-22T04:20:09.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc713400640 1 -- 172.21.15.86:0/342270410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc70c0a4d10 con 0x7fc71c06e450 2024-03-22T04:20:09.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc713400640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6f8067b20 0x7fc6f8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc713400640 1 -- 172.21.15.86:0/342270410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7fc70c120550 con 0x7fc71c06e450 2024-03-22T04:20:09.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc713e00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6f8067b20 0x7fc6f8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc72574a000 1 Processor -- start 2024-03-22T04:20:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc72574a000 1 -- start start 2024-03-22T04:20:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc713e00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6f8067b20 0x7fc6f8069fe0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc71c15d770 tx=0x7fc704009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc72574a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc72574a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824f0730 0x5624824a6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc72574a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824a6eb0 0x5624824a7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc72574a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56248238d830 con 0x5624824f0730 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc72574a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5624822f4e10 con 0x5624824a6eb0 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc72574a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x562482291e50 con 0x5624824efe10 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc713e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824f0730 0x5624824a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc71be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824a6eb0 0x5624824a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc713e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824f0730 0x5624824a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33298/0 (socket says 172.21.15.86:33298) 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc713e00640 1 -- 172.21.15.86:0/4089971638 learned_addr learned my addr 172.21.15.86:0/4089971638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.696+0000 7fc71b400640 1 --2- 172.21.15.86:0/4089971638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71be00640 1 -- 172.21.15.86:0/4089971638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 msgr2=0x5624824f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71be00640 1 --2- 172.21.15.86:0/4089971638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71be00640 1 -- 172.21.15.86:0/4089971638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824f0730 msgr2=0x5624824a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71be00640 1 --2- 172.21.15.86:0/4089971638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824f0730 0x5624824a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71be00640 1 -- 172.21.15.86:0/4089971638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56248227a510 con 0x5624824a6eb0 2024-03-22T04:20:09.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71be00640 1 --2- 172.21.15.86:0/4089971638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824a6eb0 0x5624824a7290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc7140012f0 tx=0x7fc714001f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8beeaa7514397fe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc70be00640 1 -- 172.21.15.86:0/4089971638 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714007820 con 0x5624824a6eb0 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc70be00640 1 -- 172.21.15.86:0/4089971638 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc714018070 con 0x5624824a6eb0 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc70be00640 1 -- 172.21.15.86:0/4089971638 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140135f0 con 0x5624824a6eb0 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- 172.21.15.86:0/4089971638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824a6eb0 msgr2=0x5624824a7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 --2- 172.21.15.86:0/4089971638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824a6eb0 0x5624824a7290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc7140012f0 tx=0x7fc714001f40 comp rx=0 tx=0).stop 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- 172.21.15.86:0/4089971638 shutdown_connections 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 --2- 172.21.15.86:0/4089971638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824a6eb0 0x5624824a7290 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 --2- 172.21.15.86:0/4089971638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824f0730 0x5624824a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 --2- 172.21.15.86:0/4089971638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- 172.21.15.86:0/4089971638 >> 172.21.15.86:0/4089971638 conn(0x5624824a2e60 msgr2=0x5624824a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- 172.21.15.86:0/4089971638 shutdown_connections 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- 172.21.15.86:0/4089971638 wait complete. 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- start start 2024-03-22T04:20:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 Processor -- start 2024-03-22T04:20:09.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624824b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5624822b2640 con 0x5624824a6eb0 2024-03-22T04:20:09.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5624822f0dc0 con 0x5624824f0730 2024-03-22T04:20:09.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5624823b0f90 con 0x5624824efe10 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc72574a000 10 client.0 _ll_register_callbacks cb 0x562482491d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33300/0 (socket says 172.21.15.86:33300) 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71b400640 1 -- 172.21.15.86:0/22120261 learned_addr learned my addr 172.21.15.86:0/22120261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc71be00640 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624824b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.700+0000 7fc713e00640 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc71b400640 1 -- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 msgr2=0x5624824b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc71b400640 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc71b400640 1 -- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 msgr2=0x5624824b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc71b400640 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624824b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc71b400640 1 -- 172.21.15.86:0/22120261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624822b2a40 con 0x5624824a6eb0 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc71b400640 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824b11f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0ecc20 tx=0x7fc70c0abf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd3dbe594c7c4e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 1 -- 172.21.15.86:0/22120261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c108070 con 0x5624824a6eb0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 1 -- 172.21.15.86:0/22120261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7fc70c0f8c00 con 0x5624824a6eb0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 -- 172.21.15.86:0/22120261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc714018070 con 0x5624824a6eb0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 1 -- 172.21.15.86:0/22120261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc70c0f8e20 con 0x5624824a6eb0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 10 client.8763 fetch_fsmap learned FSMap version 90 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 -- 172.21.15.86:0/22120261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x5624822ad370 con 0x5624824a6eb0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 1 -- 172.21.15.86:0/22120261 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc714018070 con 0x5624824a6eb0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 1 -- 172.21.15.86:0/22120261 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fc70c0b5bf0 con 0x5624824a6eb0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 10 client.8763 fetch_fsmap finished waiting for FSMap version 90 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 10 client.8763 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 20 client.8763 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x5624824ff0b0 0x5624825014a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 4 client.8763 mds_command: new command op to 77159 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 -- 172.21.15.86:0/22120261 --> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56248228fed0 con 0x5624824ff0b0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc713e00640 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x5624824ff0b0 0x5624825014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc713e00640 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x5624824ff0b0 0x5624825014a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfe48c8d6c19c1f server_cookie=cee31726028087b0 in_seq=0 out_seq=0 2024-03-22T04:20:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 10 client.8763 ms_handle_connect on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 1 -- 172.21.15.86:0/22120261 <== mds.0 v2:172.21.15.196:6832/269054026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56248228fed0 con 0x5624824ff0b0 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 10 client.8763 handle_command_reply: tid=0 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 client.8763 shutdown 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 -- 172.21.15.86:0/22120261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 msgr2=0x5624824b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824b11f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0ecc20 tx=0x7fc70c0abf90 comp rx=0 tx=0).stop 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 -- 172.21.15.86:0/22120261 shutdown_connections 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x5624824ff0b0 0x5624825014a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624824b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 --2- 172.21.15.86:0/22120261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824b11f0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc708c00640 0 client.8763 ms_handle_reset on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 -- 172.21.15.86:0/22120261 >> 172.21.15.86:0/22120261 conn(0x5624824a2e60 msgr2=0x5624824a5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 -- 172.21.15.86:0/22120261 shutdown_connections 2024-03-22T04:20:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 1 -- 172.21.15.86:0/22120261 wait complete. 2024-03-22T04:20:09.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.704+0000 7fc72574a000 20 client.8763 trim_cache size 0 max 16384 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f4516a00640 1 Processor -- start 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f4516a00640 1 -- start start 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f4516a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f451009b180 0x7f45100b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f4516a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451006dff0 con 0x7f451006d8f0 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f4516a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451006e2d0 con 0x7f451015e970 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f4516a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f45100b49a0 con 0x7f451009b180 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f450f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f451009b180 0x7f45100b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f4514a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f451006d8f0 0x7f45100cd230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44142/0 (socket says 172.21.15.86:44142) 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f450f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f451009b180 0x7f45100b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33308/0 (socket says 172.21.15.86:33308) 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f4514a00640 1 -- 172.21.15.86:0/3803814854 learned_addr learned my addr 172.21.15.86:0/3803814854 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f450f400640 1 -- 172.21.15.86:0/3803814854 >> v1:172.21.15.196:6789/0 conn(0x7f451015e970 legacy=0x7f451009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f450f400640 1 -- 172.21.15.86:0/3803814854 >> v1:172.21.15.86:6789/0 conn(0x7f451006d8f0 legacy=0x7f45100cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.708+0000 7f450f400640 1 -- 172.21.15.86:0/3803814854 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45100cc750 con 0x7f451009b180 2024-03-22T04:20:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f450f400640 1 --2- 172.21.15.86:0/3803814854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451009b180 0x7f45100b12e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097c0 tx=0x7f44fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae22128dc56e443 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f450ea00640 1 -- 172.21.15.86:0/3803814854 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0b1070 con 0x7f451009b180 2024-03-22T04:20:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f450ea00640 1 -- 172.21.15.86:0/3803814854 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44fc0ac5f0 con 0x7f451009b180 2024-03-22T04:20:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f450ea00640 1 -- 172.21.15.86:0/3803814854 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0ac9c0 con 0x7f451009b180 2024-03-22T04:20:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 -- 172.21.15.86:0/3803814854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451009b180 msgr2=0x7f45100b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 --2- 172.21.15.86:0/3803814854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451009b180 0x7f45100b12e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097c0 tx=0x7f44fc01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 -- 172.21.15.86:0/3803814854 shutdown_connections 2024-03-22T04:20:09.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 --2- 172.21.15.86:0/3803814854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451009b180 0x7f45100b12e0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 -- 172.21.15.86:0/3803814854 >> 172.21.15.86:0/3803814854 conn(0x7f45100b3bf0 msgr2=0x7f45100b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 -- 172.21.15.86:0/3803814854 shutdown_connections 2024-03-22T04:20:09.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 -- 172.21.15.86:0/3803814854 wait complete. 2024-03-22T04:20:09.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 Processor -- start 2024-03-22T04:20:09.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.712+0000 7f4516a00640 1 -- start start 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f451006d8f0 0x7f4510150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f451009b180 0x7f4510156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451015e970 0x7f45101565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45100ce150 con 0x7f451015e970 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f451015ed50 con 0x7f451009b180 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f45100ce2d0 con 0x7f451006d8f0 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4514a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451015e970 0x7f45101565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f450fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f451006d8f0 0x7f4510150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f450fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f451006d8f0 0x7f4510150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:47874/0 (socket says 172.21.15.86:47874) 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f450fe00640 1 -- 172.21.15.86:0/605763694 learned_addr learned my addr 172.21.15.86:0/605763694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f450f400640 1 --2- 172.21.15.86:0/605763694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f451009b180 0x7f4510156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4514a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451015e970 0x7f45101565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33310/0 (socket says 172.21.15.86:33310) 2024-03-22T04:20:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4514a00640 1 -- 172.21.15.86:0/605763694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f451006d8f0 msgr2=0x7f4510150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4514a00640 1 --2- 172.21.15.86:0/605763694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f451006d8f0 0x7f4510150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4514a00640 1 -- 172.21.15.86:0/605763694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f451009b180 msgr2=0x7f4510156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4514a00640 1 --2- 172.21.15.86:0/605763694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f451009b180 0x7f4510156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4514a00640 1 -- 172.21.15.86:0/605763694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45100b4e80 con 0x7f451015e970 2024-03-22T04:20:09.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4514a00640 1 --2- 172.21.15.86:0/605763694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f451015e970 0x7f45101565a0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f450800caa0 tx=0x7f450800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508017070 con 0x7f451015e970 2024-03-22T04:20:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450800ab40 con 0x7f451015e970 2024-03-22T04:20:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508012400 con 0x7f451015e970 2024-03-22T04:20:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 -- 172.21.15.86:0/605763694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45101549a0 con 0x7f451015e970 2024-03-22T04:20:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 -- 172.21.15.86:0/605763694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4510154f60 con 0x7f451015e970 2024-03-22T04:20:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4516a00640 1 -- 172.21.15.86:0/605763694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4510074470 con 0x7f451015e970 2024-03-22T04:20:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.716+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f450801a070 con 0x7f451015e970 2024-03-22T04:20:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.720+0000 7f4507e00640 1 --2- 172.21.15.86:0/605763694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f0067b20 0x7f44f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.720+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7f4508085c10 con 0x7f451015e970 2024-03-22T04:20:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.720+0000 7f450fe00640 1 --2- 172.21.15.86:0/605763694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f0067b20 0x7f44f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.720+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+500 (secure 0 0 0) 0x7f450808a220 con 0x7f451015e970 2024-03-22T04:20:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.720+0000 7f4516a00640 1 -- 172.21.15.86:0/605763694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f45100b7c20 con 0x7f451015e970 2024-03-22T04:20:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.720+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 7 ==== log(2 entries from seq 1222 at 2024-03-22T04:20:08.006846+0000) ==== 389+0+0 (secure 0 0 0) 0x7f450808c070 con 0x7f451015e970 2024-03-22T04:20:09.720 INFO:teuthology.orchestra.run.smithi086.stdout: cluster: 2024-03-22T04:20:09.720 INFO:teuthology.orchestra.run.smithi086.stdout: id: 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:20:09.720 INFO:teuthology.orchestra.run.smithi086.stdout: health: HEALTH_OK 2024-03-22T04:20:09.720 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:09.720 INFO:teuthology.orchestra.run.smithi086.stdout: services: 2024-03-22T04:20:09.720 INFO:teuthology.orchestra.run.smithi086.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: mgr: y(active, since 9m), standbys: x 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: mds: 1/1 daemons up, 3 standby 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: data: 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: volumes: 1/1 healthy 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: pools: 3 pools, 97 pgs 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: objects: 30 objects, 19 MiB 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: pgs: 97 active+clean 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: io: 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: client: 1.7 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:09.721 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:08.006846+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T04:20:09.722 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:08.006871+0000 mon.a [INF] Cluster is now healthy 2024-03-22T04:20:09.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.720+0000 7f450fe00640 1 --2- 172.21.15.86:0/605763694 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f44f0067b20 0x7f44f0069fe0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f45000018b0 tx=0x7f4500009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc72574a000 1 Processor -- start 2024-03-22T04:20:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc72574a000 1 -- start start 2024-03-22T04:20:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc72574a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc72574a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc72574a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624825014a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc72574a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c203af0 con 0x5624824a6eb0 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc72574a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c202b30 con 0x5624824f0730 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc72574a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c15f050 con 0x5624824efe10 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33324/0 (socket says 172.21.15.86:33324) 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 -- 172.21.15.86:0/204483472 learned_addr learned my addr 172.21.15.86:0/204483472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc713e00640 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71be00640 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624825014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 -- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 msgr2=0x5624824ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 -- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 msgr2=0x5624825014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624825014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 -- 172.21.15.86:0/204483472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624822920a0 con 0x5624824a6eb0 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71be00640 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624825014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.724+0000 7fc71b400640 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824ac180 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc70c01c0b0 tx=0x7fc70c105000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3ec959f73107a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc70b400640 1 -- 172.21.15.86:0/204483472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c108070 con 0x5624824a6eb0 2024-03-22T04:20:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc70b400640 1 -- 172.21.15.86:0/204483472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70c0b1040 con 0x5624824a6eb0 2024-03-22T04:20:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc70b400640 1 -- 172.21.15.86:0/204483472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0f0300 con 0x5624824a6eb0 2024-03-22T04:20:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- 172.21.15.86:0/204483472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 msgr2=0x5624824ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824ac180 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc70c01c0b0 tx=0x7fc70c105000 comp rx=0 tx=0).stop 2024-03-22T04:20:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- 172.21.15.86:0/204483472 shutdown_connections 2024-03-22T04:20:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x5624825014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824efe10 0x5624824ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 --2- 172.21.15.86:0/204483472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824a6eb0 0x5624824ac180 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- 172.21.15.86:0/204483472 >> 172.21.15.86:0/204483472 conn(0x5624824a2e60 msgr2=0x5624824a6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- 172.21.15.86:0/204483472 shutdown_connections 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- 172.21.15.86:0/204483472 wait complete. 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- start start 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 Processor -- start 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824a6eb0 0x562482504520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824efe10 0x56248255bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x56248255c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56248227c620 con 0x5624824efe10 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c15f6a0 con 0x5624824f0730 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5624822ae8b0 con 0x5624824a6eb0 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 10 client.0 _ll_register_callbacks cb 0x7fc70c0f0590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824efe10 0x56248255bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824efe10 0x56248255bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33336/0 (socket says 172.21.15.86:33336) 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 -- 172.21.15.86:0/1458106017 learned_addr learned my addr 172.21.15.86:0/1458106017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc71b400640 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824a6eb0 0x562482504520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc71be00640 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x56248255c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 -- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824a6eb0 msgr2=0x562482504520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824a6eb0 0x562482504520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 -- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 msgr2=0x56248255c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x56248255c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 -- 172.21.15.86:0/1458106017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562482376840 con 0x5624824efe10 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc71b400640 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824a6eb0 0x562482504520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc713e00640 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824efe10 0x56248255bb50 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc70400fbf0 tx=0x7fc704001240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b270ef445f17fb87 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc708c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc708c00640 1 -- 172.21.15.86:0/1458106017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70402e070 con 0x5624824efe10 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc708c00640 1 -- 172.21.15.86:0/1458106017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7fc70402ac40 con 0x5624824efe10 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.728+0000 7fc72574a000 1 -- 172.21.15.86:0/1458106017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c0b1040 con 0x5624824efe10 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc708c00640 1 -- 172.21.15.86:0/1458106017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc704039050 con 0x5624824efe10 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 10 client.8778 fetch_fsmap learned FSMap version 90 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 -- 172.21.15.86:0/1458106017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x562481fb0470 con 0x5624824efe10 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc708c00640 1 -- 172.21.15.86:0/1458106017 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70c0b1040 con 0x5624824efe10 2024-03-22T04:20:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc708c00640 1 -- 172.21.15.86:0/1458106017 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fc70403c020 con 0x5624824efe10 2024-03-22T04:20:09.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 10 client.8778 fetch_fsmap finished waiting for FSMap version 90 2024-03-22T04:20:09.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 10 client.8778 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-03-22T04:20:09.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 20 client.8778 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:09.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x56248255c5d0 0x56248255c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 4 client.8778 mds_command: new command op to 77159 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-22T04:20:09.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 -- 172.21.15.86:0/1458106017 --> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5624822ce6b0 con 0x56248255c5d0 2024-03-22T04:20:09.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc71b400640 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x56248255c5d0 0x56248255c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.735 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:20:09.734+0000 7fbb65800640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-22T04:20:09.735 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:20:09.734+0000 7fbb65800640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-22T04:20:09.735 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:20:09.734+0000 7fbb65800640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-22T04:20:09.735 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-03-22T04:20:09.734+0000 7fbb65800640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-22T04:20:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc71b400640 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x56248255c5d0 0x56248255c9e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451b776e880f5eed server_cookie=be1fb8d9b0c5cd3a in_seq=0 out_seq=0 2024-03-22T04:20:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc708c00640 10 client.8778 ms_handle_connect on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc708c00640 1 -- 172.21.15.86:0/1458106017 <== mds.0 v2:172.21.15.196:6832/269054026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5624822ce6b0 con 0x56248255c5d0 2024-03-22T04:20:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc708c00640 10 client.8778 handle_command_reply: tid=0 2024-03-22T04:20:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 client.8778 shutdown 2024-03-22T04:20:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 -- 172.21.15.86:0/1458106017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824efe10 msgr2=0x56248255bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824efe10 0x56248255bb50 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc70400fbf0 tx=0x7fc704001240 comp rx=0 tx=0).stop 2024-03-22T04:20:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 -- 172.21.15.86:0/1458106017 shutdown_connections 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x56248255c5d0 0x56248255c9e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624824f0730 0x56248255c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624824efe10 0x56248255bb50 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc708c00640 0 client.8778 ms_handle_reset on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 --2- 172.21.15.86:0/1458106017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5624824a6eb0 0x562482504520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 -- 172.21.15.86:0/1458106017 >> 172.21.15.86:0/1458106017 conn(0x5624824a2e60 msgr2=0x5624824ab170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 -- 172.21.15.86:0/1458106017 shutdown_connections 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 1 -- 172.21.15.86:0/1458106017 wait complete. 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc72574a000 20 client.8778 trim_cache size 0 max 16384 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:20:09.737 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-22T04:20:09.738 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "75385786-1ee5-421c-b8a6-92eb0b337f70", 2024-03-22T04:20:09.738 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-22T04:20:09.738 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 -- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6f8067b20 msgr2=0x7fc6f8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6f8067b20 0x7fc6f8069fe0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc71c15d770 tx=0x7fc704009040 comp rx=0 tx=0).stop 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 -- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c06e450 msgr2=0x7fc71c145710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c06e450 0x7fc71c145710 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc70c01a930 tx=0x7fc70c01aac0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 -- 172.21.15.86:0/342270410 shutdown_connections 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc6f8067b20 0x7fc6f8069fe0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc71c06f720 0x7fc71c1530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc71c156600 0x7fc71c142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 --2- 172.21.15.86:0/342270410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c06e450 0x7fc71c145710 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 -- 172.21.15.86:0/342270410 >> 172.21.15.86:0/342270410 conn(0x7fc71c06b660 msgr2=0x7fc71c06ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 -- 172.21.15.86:0/342270410 shutdown_connections 2024-03-22T04:20:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.732+0000 7fc720c00640 1 -- 172.21.15.86:0/342270410 wait complete. 2024-03-22T04:20:09.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:20:09.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-22T04:20:09.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe091e00640 1 Processor -- start 2024-03-22T04:20:09.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe091e00640 1 -- start start 2024-03-22T04:20:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe091e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe08c15c230 0x7fe08c0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe091e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe08c0cfca0 con 0x7fe08c0d2b30 2024-03-22T04:20:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe091e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe08c0cff80 con 0x7fe08c0ce900 2024-03-22T04:20:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe091e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe08c0d05d0 con 0x7fe08c15c230 2024-03-22T04:20:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe08be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe08c0d2b30 0x7fe08c0cd650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44156/0 (socket says 172.21.15.86:44156) 2024-03-22T04:20:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe08be00640 1 -- 172.21.15.86:0/3723217864 learned_addr learned my addr 172.21.15.86:0/3723217864 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe090a00640 1 --2- 172.21.15.86:0/3723217864 >> v2:172.21.15.196:3300/0 conn(0x7fe08c15c230 0x7fe08c0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe08b400640 1 -- 172.21.15.86:0/3723217864 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330452056 0 0) 0x7fe08c0cfca0 con 0x7fe08c0d2b30 2024-03-22T04:20:09.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe08b400640 1 -- 172.21.15.86:0/3723217864 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe074003530 con 0x7fe08c0d2b30 2024-03-22T04:20:09.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe090a00640 1 -- 172.21.15.86:0/3723217864 >> v1:172.21.15.196:6789/0 conn(0x7fe08c0ce900 legacy=0x7fe08c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe08b400640 1 -- 172.21.15.86:0/3723217864 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3596376726 0 0) 0x7fe08c0cff80 con 0x7fe08c0ce900 2024-03-22T04:20:09.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe090a00640 1 -- 172.21.15.86:0/3723217864 >> v1:172.21.15.86:6789/0 conn(0x7fe08c0d2b30 legacy=0x7fe08c0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.892+0000 7fe090a00640 1 -- 172.21.15.86:0/3723217864 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c0d0260 con 0x7fe08c15c230 2024-03-22T04:20:09.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe090a00640 1 --2- 172.21.15.86:0/3723217864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c15c230 0x7fe08c0d0a60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe0800097a0 tx=0x7fe08001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd740c60a5e40690 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe08b400640 1 -- 172.21.15.86:0/3723217864 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800b1070 con 0x7fe08c15c230 2024-03-22T04:20:09.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe08b400640 1 -- 172.21.15.86:0/3723217864 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0800ac5f0 con 0x7fe08c15c230 2024-03-22T04:20:09.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe08b400640 1 -- 172.21.15.86:0/3723217864 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800ac9c0 con 0x7fe08c15c230 2024-03-22T04:20:09.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 -- 172.21.15.86:0/3723217864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c15c230 msgr2=0x7fe08c0d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 --2- 172.21.15.86:0/3723217864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c15c230 0x7fe08c0d0a60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe0800097a0 tx=0x7fe08001c480 comp rx=0 tx=0).stop 2024-03-22T04:20:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 -- 172.21.15.86:0/3723217864 shutdown_connections 2024-03-22T04:20:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 --2- 172.21.15.86:0/3723217864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c15c230 0x7fe08c0d0a60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 -- 172.21.15.86:0/3723217864 >> 172.21.15.86:0/3723217864 conn(0x7fe08c0d1f30 msgr2=0x7fe08c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 -- 172.21.15.86:0/3723217864 shutdown_connections 2024-03-22T04:20:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 -- 172.21.15.86:0/3723217864 wait complete. 2024-03-22T04:20:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 Processor -- start 2024-03-22T04:20:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.896+0000 7fe091e00640 1 -- start start 2024-03-22T04:20:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c0ce900 0x7fe08c14eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe08c0d2b30 0x7fe08c1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe08c15c230 0x7fe08c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c0b5920 con 0x7fe08c0d2b30 2024-03-22T04:20:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c0b57a0 con 0x7fe08c0ce900 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe08c0cdd60 con 0x7fe08c15c230 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe08c0d2b30 0x7fe08c1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe090a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c0ce900 0x7fe08c14eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe08c0d2b30 0x7fe08c1544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33342/0 (socket says 172.21.15.86:33342) 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 -- 172.21.15.86:0/1471414629 learned_addr learned my addr 172.21.15.86:0/1471414629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091400640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe08c15c230 0x7fe08c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 -- 172.21.15.86:0/1471414629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe08c15c230 msgr2=0x7fe08c154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe08c15c230 0x7fe08c154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 -- 172.21.15.86:0/1471414629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c0ce900 msgr2=0x7fe08c14eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c0ce900 0x7fe08c14eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 -- 172.21.15.86:0/1471414629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c15f320 con 0x7fe08c0d2b30 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe090a00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c0ce900 0x7fe08c14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe08be00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe08c0d2b30 0x7fe08c1544e0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fe07800c3c0 tx=0x7fe07800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe088c00640 1 -- 172.21.15.86:0/1471414629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078017070 con 0x7fe08c0d2b30 2024-03-22T04:20:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe088c00640 1 -- 172.21.15.86:0/1471414629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078005cf0 con 0x7fe08c0d2b30 2024-03-22T04:20:09.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe088c00640 1 -- 172.21.15.86:0/1471414629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780072f0 con 0x7fe08c0d2b30 2024-03-22T04:20:09.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091e00640 1 -- 172.21.15.86:0/1471414629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c15f600 con 0x7fe08c0d2b30 2024-03-22T04:20:09.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.900+0000 7fe091e00640 1 -- 172.21.15.86:0/1471414629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c15f010 con 0x7fe08c0d2b30 2024-03-22T04:20:09.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe088c00640 1 -- 172.21.15.86:0/1471414629 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe07801a070 con 0x7fe08c0d2b30 2024-03-22T04:20:09.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe088c00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe06c067b20 0x7fe06c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe088c00640 1 -- 172.21.15.86:0/1471414629 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7fe0780854a0 con 0x7fe08c0d2b30 2024-03-22T04:20:09.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe06c067b20 0x7fe06c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe0967f8000 1 Processor -- start 2024-03-22T04:20:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe0967f8000 1 -- start start 2024-03-22T04:20:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe06c067b20 0x7fe06c069fe0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe0800ae000 tx=0x7fe0800a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351b5c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 0x55d351b0fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351b10330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe0967f8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d351960c40 con 0x55d351b0ff50 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe0967f8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d3518fdd90 con 0x55d351b5bf80 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe0967f8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d35195cbf0 con 0x55d351b5c8a0 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351b5c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe091400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351b10330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351b5c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:34636/0 (socket says 172.21.15.86:34636) 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 -- 172.21.15.86:0/3572407099 learned_addr learned my addr 172.21.15.86:0/3572407099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe08be00640 1 --2- 172.21.15.86:0/3572407099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 0x55d351b0fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 -- 172.21.15.86:0/3572407099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 msgr2=0x55d351b0fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 --2- 172.21.15.86:0/3572407099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 0x55d351b0fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 -- 172.21.15.86:0/3572407099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 msgr2=0x55d351b10330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 --2- 172.21.15.86:0/3572407099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351b10330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 -- 172.21.15.86:0/3572407099 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d351a08130 con 0x55d351b5bf80 2024-03-22T04:20:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe090a00640 1 --2- 172.21.15.86:0/3572407099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351b5c360 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe08001c4c0 tx=0x7fe080004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5636b20468e921d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.904+0000 7fe07d600640 1 -- 172.21.15.86:0/3572407099 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800bb040 con 0x55d351b5bf80 2024-03-22T04:20:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe07d600640 1 -- 172.21.15.86:0/3572407099 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0800b2040 con 0x55d351b5bf80 2024-03-22T04:20:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe07d600640 1 -- 172.21.15.86:0/3572407099 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800b1050 con 0x55d351b5bf80 2024-03-22T04:20:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- 172.21.15.86:0/3572407099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 msgr2=0x55d351b5c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3572407099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351b5c360 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe08001c4c0 tx=0x7fe080004740 comp rx=0 tx=0).stop 2024-03-22T04:20:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- 172.21.15.86:0/3572407099 shutdown_connections 2024-03-22T04:20:09.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3572407099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351b10330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3572407099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 0x55d351b0fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3572407099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351b5c360 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- 172.21.15.86:0/3572407099 >> 172.21.15.86:0/3572407099 conn(0x55d3519f9990 msgr2=0x55d351b3f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- 172.21.15.86:0/3572407099 shutdown_connections 2024-03-22T04:20:09.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- 172.21.15.86:0/3572407099 wait complete. 2024-03-22T04:20:09.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- start start 2024-03-22T04:20:09.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 Processor -- start 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b0ff50 0x55d351b17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5bf80 0x55d351b40380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5c8a0 0x55d351b408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d35191c710 con 0x55d351b5bf80 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d35191e580 con 0x55d351b5c8a0 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d351965270 con 0x55d351b0ff50 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 10 client.0 _ll_register_callbacks cb 0x55d3519c2200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5bf80 0x55d351b40380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5bf80 0x55d351b40380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33362/0 (socket says 172.21.15.86:33362) 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 -- 172.21.15.86:0/3549027834 learned_addr learned my addr 172.21.15.86:0/3549027834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe091400640 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5c8a0 0x55d351b408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe090a00640 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b0ff50 0x55d351b17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 -- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b0ff50 msgr2=0x55d351b17a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b0ff50 0x55d351b17a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 -- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5c8a0 msgr2=0x55d351b408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5c8a0 0x55d351b408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 -- 172.21.15.86:0/3549027834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d351939d30 con 0x55d351b5bf80 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe08be00640 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5bf80 0x55d351b40380 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fe078051a40 tx=0x7fe078055720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25ea17bdc009a88 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe072000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe072000640 1 -- 172.21.15.86:0/3549027834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078083060 con 0x55d351b5bf80 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe072000640 1 -- 172.21.15.86:0/3549027834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7fe078083400 con 0x55d351b5bf80 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe0967f8000 1 -- 172.21.15.86:0/3549027834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0800b2040 con 0x55d351b5bf80 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.908+0000 7fe072000640 1 -- 172.21.15.86:0/3549027834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe07805da20 con 0x55d351b5bf80 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 10 client.8790 fetch_fsmap learned FSMap version 90 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 -- 172.21.15.86:0/3549027834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55d3519a5fc0 con 0x55d351b5bf80 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe072000640 1 -- 172.21.15.86:0/3549027834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0800b2040 con 0x55d351b5bf80 2024-03-22T04:20:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe072000640 1 -- 172.21.15.86:0/3549027834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fe078007490 con 0x55d351b5bf80 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 10 client.8790 fetch_fsmap finished waiting for FSMap version 90 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 10 client.8790 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 20 client.8790 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55d351b625c0 0x55d351b629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 4 client.8790 mds_command: new command op to 77159 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 -- 172.21.15.86:0/3549027834 --> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3518fbdd0 con 0x55d351b625c0 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe090a00640 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55d351b625c0 0x55d351b629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe090a00640 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55d351b625c0 0x55d351b629a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a35c32ece6b8729e server_cookie=ef7378e54e422a98 in_seq=0 out_seq=0 2024-03-22T04:20:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe072000640 10 client.8790 ms_handle_connect on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe072000640 1 -- 172.21.15.86:0/3549027834 <== mds.0 v2:172.21.15.196:6832/269054026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d3518fbdd0 con 0x55d351b625c0 2024-03-22T04:20:09.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe072000640 10 client.8790 handle_command_reply: tid=0 2024-03-22T04:20:09.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 client.8790 shutdown 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 -- 172.21.15.86:0/3549027834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5bf80 msgr2=0x55d351b40380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5bf80 0x55d351b40380 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fe078051a40 tx=0x7fe078055720 comp rx=0 tx=0).stop 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 -- 172.21.15.86:0/3549027834 shutdown_connections 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55d351b625c0 0x55d351b629a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5c8a0 0x55d351b408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5bf80 0x55d351b40380 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3549027834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b0ff50 0x55d351b17a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe072000640 0 client.8790 ms_handle_reset on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 -- 172.21.15.86:0/3549027834 >> 172.21.15.86:0/3549027834 conn(0x55d3519f9990 msgr2=0x55d351b0f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 -- 172.21.15.86:0/3549027834 shutdown_connections 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 1 -- 172.21.15.86:0/3549027834 wait complete. 2024-03-22T04:20:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.912+0000 7fe0967f8000 20 client.8790 trim_cache size 0 max 16384 2024-03-22T04:20:09.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.928+0000 7fe0967f8000 1 Processor -- start 2024-03-22T04:20:09.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.928+0000 7fe0967f8000 1 -- start start 2024-03-22T04:20:09.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b0ff50 0x55d351b16f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5bf80 0x55d351b72680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5c8a0 0x55d351b161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c2036e0 con 0x55d351b5c8a0 2024-03-22T04:20:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c2027b0 con 0x55d351b0ff50 2024-03-22T04:20:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe08c15d110 con 0x55d351b5bf80 2024-03-22T04:20:09.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5c8a0 0x55d351b161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5c8a0 0x55d351b161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33364/0 (socket says 172.21.15.86:33364) 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 -- 172.21.15.86:0/3553293166 learned_addr learned my addr 172.21.15.86:0/3553293166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe090a00640 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b0ff50 0x55d351b16f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe08be00640 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5bf80 0x55d351b72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 -- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5bf80 msgr2=0x55d351b72680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5bf80 0x55d351b72680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 -- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b0ff50 msgr2=0x55d351b16f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b0ff50 0x55d351b16f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 -- 172.21.15.86:0/3553293166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3519363a0 con 0x55d351b5c8a0 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe08be00640 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5bf80 0x55d351b72680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe090a00640 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b0ff50 0x55d351b16f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe091400640 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5c8a0 0x55d351b161c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fe084001f50 tx=0x7fe084015780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fe4f1075f8b0ca server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe07cc00640 1 -- 172.21.15.86:0/3553293166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084003980 con 0x55d351b5c8a0 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe07cc00640 1 -- 172.21.15.86:0/3553293166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe08401a030 con 0x55d351b5c8a0 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe07cc00640 1 -- 172.21.15.86:0/3553293166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084003d80 con 0x55d351b5c8a0 2024-03-22T04:20:09.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- 172.21.15.86:0/3553293166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5c8a0 msgr2=0x55d351b161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5c8a0 0x55d351b161c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fe084001f50 tx=0x7fe084015780 comp rx=0 tx=0).stop 2024-03-22T04:20:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- 172.21.15.86:0/3553293166 shutdown_connections 2024-03-22T04:20:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b5c8a0 0x55d351b161c0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5bf80 0x55d351b72680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3553293166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b0ff50 0x55d351b16f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- 172.21.15.86:0/3553293166 >> 172.21.15.86:0/3553293166 conn(0x55d3519f9990 msgr2=0x55d351b180f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- 172.21.15.86:0/3553293166 shutdown_connections 2024-03-22T04:20:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- 172.21.15.86:0/3553293166 wait complete. 2024-03-22T04:20:09.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 -- start start 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.932+0000 7fe0967f8000 1 Processor -- start 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351bcbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351bcc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 0x55d351b649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d351a1c810 con 0x55d351b0ff50 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c15d760 con 0x55d351b5bf80 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d351907350 con 0x55d351b5c8a0 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 10 client.0 _ll_register_callbacks cb 0x7fe078054f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351bcbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351bcbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33378/0 (socket says 172.21.15.86:33378) 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 -- 172.21.15.86:0/3229809198 learned_addr learned my addr 172.21.15.86:0/3229809198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe08be00640 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351bcc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091400640 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 0x55d351b649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 -- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 msgr2=0x55d351b649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 0x55d351b649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 -- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 msgr2=0x55d351bcc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351bcc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 -- 172.21.15.86:0/3229809198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3518e6380 con 0x55d351b0ff50 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe08be00640 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351bcc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe090a00640 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351bcbbd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fe08001c960 tx=0x7fe0800b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bd4c01a89f0e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 1 -- 172.21.15.86:0/3229809198 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800bb040 con 0x55d351b0ff50 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 1 -- 172.21.15.86:0/3229809198 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6187+0+0 (secure 0 0 0) 0x7fe080003960 con 0x55d351b0ff50 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- 172.21.15.86:0/3229809198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe08401a030 con 0x55d351b0ff50 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 1 -- 172.21.15.86:0/3229809198 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe0800b1040 con 0x55d351b0ff50 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 10 client.8796 fetch_fsmap learned FSMap version 90 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- 172.21.15.86:0/3229809198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55d3519e2460 con 0x55d351b0ff50 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 1 -- 172.21.15.86:0/3229809198 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe08401a030 con 0x55d351b0ff50 2024-03-22T04:20:09.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 1 -- 172.21.15.86:0/3229809198 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fe0800ab070 con 0x55d351b0ff50 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 10 client.8796 fetch_fsmap finished waiting for FSMap version 90 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 10 client.8796 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 20 client.8796 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55d351b64ef0 0x55d351b65300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 4 client.8796 mds_command: new command op to 77159 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- 172.21.15.86:0/3229809198 --> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d351900160 con 0x55d351b64ef0 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe08be00640 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55d351b64ef0 0x55d351b65300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 10 client.8796 ms_handle_connect on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe08be00640 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55d351b64ef0 0x55d351b65300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9236a2c9b91acdea server_cookie=b4a0be0cf7ba5d3a in_seq=0 out_seq=0 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 1 -- 172.21.15.86:0/3229809198 <== mds.0 v2:172.21.15.196:6832/269054026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55d351900160 con 0x55d351b64ef0 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 10 client.8796 handle_command_reply: tid=0 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 client.8796 shutdown 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- 172.21.15.86:0/3229809198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 msgr2=0x55d351bcbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351bcbbd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fe08001c960 tx=0x7fe0800b7000 comp rx=0 tx=0).stop 2024-03-22T04:20:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- 172.21.15.86:0/3229809198 shutdown_connections 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55d351b64ef0 0x55d351b65300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d351b5bf80 0x55d351bcc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d351b5c8a0 0x55d351b649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe072000640 0 client.8796 ms_handle_reset on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 --2- 172.21.15.86:0/3229809198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d351b0ff50 0x55d351bcbbd0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- 172.21.15.86:0/3229809198 >> 172.21.15.86:0/3229809198 conn(0x55d3519f9990 msgr2=0x55d351b60ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- 172.21.15.86:0/3229809198 shutdown_connections 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 1 -- 172.21.15.86:0/3229809198 wait complete. 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe0967f8000 20 client.8796 trim_cache size 0 max 16384 2024-03-22T04:20:09.941 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: "75385786-1ee5-421c-b8a6-92eb0b337f70": { 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "75385786-1ee5-421c-b8a6-92eb0b337f70", 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:09.942 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 -- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe06c067b20 msgr2=0x7fe06c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe06c067b20 0x7fe06c069fe0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe0800ae000 tx=0x7fe0800a7000 comp rx=0 tx=0).stop 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 -- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe08c0d2b30 msgr2=0x7fe08c1544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe08c0d2b30 0x7fe08c1544e0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fe07800c3c0 tx=0x7fe07800c880 comp rx=0 tx=0).stop 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 -- 172.21.15.86:0/1471414629 shutdown_connections 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe06c067b20 0x7fe06c069fe0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe08c15c230 0x7fe08c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe08c0d2b30 0x7fe08c1544e0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 --2- 172.21.15.86:0/1471414629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe08c0ce900 0x7fe08c14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.936+0000 7fe091e00640 1 -- 172.21.15.86:0/1471414629 >> 172.21.15.86:0/1471414629 conn(0x7fe08c0d1f30 msgr2=0x7fe08c0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.940+0000 7fe091e00640 1 -- 172.21.15.86:0/1471414629 shutdown_connections 2024-03-22T04:20:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:09.940+0000 7fe091e00640 1 -- 172.21.15.86:0/1471414629 wait complete. 2024-03-22T04:20:09.957 INFO:tasks.cephfs.filesystem:scrub status for tag:75385786-1ee5-421c-b8a6-92eb0b337f70 - {'path': '/', 'tag': '75385786-1ee5-421c-b8a6-92eb0b337f70', 'options': 'recursive,repair'} 2024-03-22T04:20:11.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:11.012+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 8 ==== log(8 entries from seq 1 at 2024-03-22T04:20:09.736295+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f4508053e80 con 0x7f451015e970 2024-03-22T04:20:11.014 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:09.736295+0000 mds.d [INF] scrub queued for path: / 2024-03-22T04:20:11.015 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:09.736300+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-03-22T04:20:11.015 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:09.736306+0000 mds.d [INF] scrub summary: active paths [/] 2024-03-22T04:20:11.015 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:09.737369+0000 mds.d [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-03-22T04:20:11.015 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:09.737381+0000 mds.d [ERR] inode table repaired for inode: 0x10000000001 2024-03-22T04:20:11.015 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:09.737475+0000 mds.d [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-03-22T04:20:11.015 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:09.737480+0000 mds.d [ERR] inode table repaired for inode: 0x10000000002 2024-03-22T04:20:11.015 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:09.738053+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-03-22T04:20:13.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:13.016+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 9 ==== log(1 entries from seq 9 at 2024-03-22T04:20:12.448726+0000) ==== 189+0+0 (secure 0 0 0) 0x7f45080540e0 con 0x7f451015e970 2024-03-22T04:20:13.018 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:20:12.448726+0000 mds.d [INF] scrub summary: idle 2024-03-22T04:20:16.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:16.023+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 10 ==== log(4 entries from seq 1228 at 2024-03-22T04:20:15.999410+0000) ==== 1172+0+0 (secure 0 0 0) 0x7f4508083fd0 con 0x7f451015e970 2024-03-22T04:20:17.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:17.043+0000 7f4507e00640 1 -- 172.21.15.86:0/605763694 <== mon.0 v2:172.21.15.86:3300/0 11 ==== log(4 entries from seq 1232 at 2024-03-22T04:20:16.035363+0000) ==== 1180+0+0 (secure 0 0 0) 0x7f4508052450 con 0x7f451015e970 2024-03-22T04:20:28.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:20:28.999 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:20:39.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:20:39.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-22T04:20:40.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.107+0000 7ff2f0c00640 1 Processor -- start 2024-03-22T04:20:40.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.107+0000 7ff2f0c00640 1 -- start start 2024-03-22T04:20:40.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.107+0000 7ff2f0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2ec0b3600 0x7ff2ec15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.107+0000 7ff2f0c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec15cee0 0x7ff2ec0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.107+0000 7ff2f0c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2ec15d710 con 0x7ff2ec0b2360 2024-03-22T04:20:40.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.107+0000 7ff2f0c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff2ec0b2740 con 0x7ff2ec0b3600 2024-03-22T04:20:40.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.107+0000 7ff2f0c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ec15fe40 con 0x7ff2ec15cee0 2024-03-22T04:20:40.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.107+0000 7ff2e1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2ec0b3600 0x7ff2ec15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff2ec0b2360 0x7ff2ec0b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51634/0 (socket says 172.21.15.86:51634) 2024-03-22T04:20:40.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e9e00640 1 -- 172.21.15.86:0/878635330 learned_addr learned my addr 172.21.15.86:0/878635330 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:40.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2ea800640 1 --2- 172.21.15.86:0/878635330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec15cee0 0x7ff2ec0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e9400640 1 -- 172.21.15.86:0/878635330 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250372703 0 0) 0x7ff2ec15d710 con 0x7ff2ec0b2360 2024-03-22T04:20:40.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e1400640 1 -- 172.21.15.86:0/878635330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec15cee0 msgr2=0x7ff2ec0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e1400640 1 --2- 172.21.15.86:0/878635330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec15cee0 0x7ff2ec0b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e1400640 1 -- 172.21.15.86:0/878635330 >> v1:172.21.15.86:6789/0 conn(0x7ff2ec0b2360 legacy=0x7ff2ec0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e1400640 1 -- 172.21.15.86:0/878635330 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec160120 con 0x7ff2ec0b3600 2024-03-22T04:20:40.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e1400640 1 --2- 172.21.15.86:0/878635330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec0b3600 0x7ff2ec15c9a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097c0 tx=0x7ff2d401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7d153d8a09fc78 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e9400640 1 -- 172.21.15.86:0/878635330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40a7070 con 0x7ff2ec0b3600 2024-03-22T04:20:40.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e9400640 1 -- 172.21.15.86:0/878635330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d40b0070 con 0x7ff2ec0b3600 2024-03-22T04:20:40.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.111+0000 7ff2e9400640 1 -- 172.21.15.86:0/878635330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40ab890 con 0x7ff2ec0b3600 2024-03-22T04:20:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- 172.21.15.86:0/878635330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec0b3600 msgr2=0x7ff2ec15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/878635330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec0b3600 0x7ff2ec15c9a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097c0 tx=0x7ff2d401c470 comp rx=0 tx=0).stop 2024-03-22T04:20:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- 172.21.15.86:0/878635330 shutdown_connections 2024-03-22T04:20:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/878635330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec15cee0 0x7ff2ec0b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/878635330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec0b3600 0x7ff2ec15c9a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- 172.21.15.86:0/878635330 >> 172.21.15.86:0/878635330 conn(0x7ff2ec0cc940 msgr2=0x7ff2ec0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- 172.21.15.86:0/878635330 shutdown_connections 2024-03-22T04:20:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- 172.21.15.86:0/878635330 wait complete. 2024-03-22T04:20:40.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 Processor -- start 2024-03-22T04:20:40.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- start start 2024-03-22T04:20:40.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2ec0b2360 0x7ff2ec1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec0b3600 0x7ff2ec145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec15cee0 0x7ff2ec15b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec15d470 con 0x7ff2ec15cee0 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec15d2f0 con 0x7ff2ec0b2360 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2f0c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ec0b5ad0 con 0x7ff2ec0b3600 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec15cee0 0x7ff2ec15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec15cee0 0x7ff2ec15b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54512/0 (socket says 172.21.15.86:54512) 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2ea800640 1 -- 172.21.15.86:0/1421472072 learned_addr learned my addr 172.21.15.86:0/1421472072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2ec0b2360 0x7ff2ec1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2e1400640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec0b3600 0x7ff2ec145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2ea800640 1 -- 172.21.15.86:0/1421472072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec0b3600 msgr2=0x7ff2ec145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2ea800640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec0b3600 0x7ff2ec145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2ea800640 1 -- 172.21.15.86:0/1421472072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2ec0b2360 msgr2=0x7ff2ec1ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2ea800640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2ec0b2360 0x7ff2ec1ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.115+0000 7ff2ea800640 1 -- 172.21.15.86:0/1421472072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec146030 con 0x7ff2ec15cee0 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2ea800640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec15cee0 0x7ff2ec15b600 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400efc0 tx=0x7ff2e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2e2a00640 1 -- 172.21.15.86:0/1421472072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4018020 con 0x7ff2ec15cee0 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2e2a00640 1 -- 172.21.15.86:0/1421472072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e4014480 con 0x7ff2ec15cee0 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2f0c00640 1 -- 172.21.15.86:0/1421472072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec1462b0 con 0x7ff2ec15cee0 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2e2a00640 1 -- 172.21.15.86:0/1421472072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4014870 con 0x7ff2ec15cee0 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2f0c00640 1 -- 172.21.15.86:0/1421472072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec146870 con 0x7ff2ec15cee0 2024-03-22T04:20:40.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2e2a00640 1 -- 172.21.15.86:0/1421472072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff2e401b020 con 0x7ff2ec15cee0 2024-03-22T04:20:40.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2e2a00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2c8067b20 0x7ff2c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2e2a00640 1 -- 172.21.15.86:0/1421472072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6303+0+0 (secure 0 0 0) 0x7ff2e4086a30 con 0x7ff2ec15cee0 2024-03-22T04:20:40.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2c8067b20 0x7ff2c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2f4006000 1 Processor -- start 2024-03-22T04:20:40.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.119+0000 7ff2f4006000 1 -- start start 2024-03-22T04:20:40.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2c8067b20 0x7ff2c8069fe0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc007fd0 tx=0x7ff2dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e51360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e04a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e05330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9c55c40 con 0x55b0a9e518a0 2024-03-22T04:20:40.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9bf2d90 con 0x55b0a9e50f80 2024-03-22T04:20:40.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b0a9c51bf0 con 0x55b0a9e04f50 2024-03-22T04:20:40.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e04a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e04a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54528/0 (socket says 172.21.15.86:54528) 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 -- 172.21.15.86:0/44225349 learned_addr learned my addr 172.21.15.86:0/44225349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e51360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2ea800640 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e05330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 -- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 msgr2=0x55b0a9e05330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e05330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 -- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 msgr2=0x55b0a9e51360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e51360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 -- 172.21.15.86:0/44225349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a9cfd130 con 0x55b0a9e518a0 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2ea800640 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e05330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2e1400640 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e04a10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff2d401c4b0 tx=0x7ff2d4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2519f2b500f0fc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2daa00640 1 -- 172.21.15.86:0/44225349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4003620 con 0x55b0a9e518a0 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2daa00640 1 -- 172.21.15.86:0/44225349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d40b0040 con 0x55b0a9e518a0 2024-03-22T04:20:40.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2daa00640 1 -- 172.21.15.86:0/44225349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40039c0 con 0x55b0a9e518a0 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- 172.21.15.86:0/44225349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 msgr2=0x55b0a9e04a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e04a10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff2d401c4b0 tx=0x7ff2d4004740 comp rx=0 tx=0).stop 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- 172.21.15.86:0/44225349 shutdown_connections 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e05330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e04a10 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 --2- 172.21.15.86:0/44225349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e51360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- 172.21.15.86:0/44225349 >> 172.21.15.86:0/44225349 conn(0x55b0a9cee990 msgr2=0x55b0a9e34150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- 172.21.15.86:0/44225349 shutdown_connections 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- 172.21.15.86:0/44225349 wait complete. 2024-03-22T04:20:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 -- start start 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.123+0000 7ff2f4006000 1 Processor -- start 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e0c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e35380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 0x55b0a9e358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9c11710 con 0x55b0a9e04f50 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9c13580 con 0x55b0a9e50f80 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b0a9c5a270 con 0x55b0a9e518a0 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 10 client.0 _ll_register_callbacks cb 0x55b0a9cb7200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e0c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e0c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54532/0 (socket says 172.21.15.86:54532) 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 -- 172.21.15.86:0/3225216102 learned_addr learned my addr 172.21.15.86:0/3225216102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e1400640 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e35380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2ea800640 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 0x55b0a9e358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 -- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 msgr2=0x55b0a9e358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 0x55b0a9e358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 -- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 msgr2=0x55b0a9e35380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e35380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 -- 172.21.15.86:0/3225216102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a9c2ed30 con 0x55b0a9e04f50 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e1400640 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e35380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e0c9c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0146d0 tx=0x7ff2dc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a69b1cb78c2b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2cf400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2cf400640 1 -- 172.21.15.86:0/3225216102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc025020 con 0x55b0a9e04f50 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2cf400640 1 -- 172.21.15.86:0/3225216102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6303+0+0 (secure 0 0 0) 0x7ff2dc013780 con 0x55b0a9e04f50 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 -- 172.21.15.86:0/3225216102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d40b0040 con 0x55b0a9e04f50 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2cf400640 1 -- 172.21.15.86:0/3225216102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff2dc02f050 con 0x55b0a9e04f50 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 10 client.8811 fetch_fsmap learned FSMap version 90 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 -- 172.21.15.86:0/3225216102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55b0a9c9afc0 con 0x55b0a9e04f50 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2cf400640 1 -- 172.21.15.86:0/3225216102 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d40b0040 con 0x55b0a9e04f50 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2cf400640 1 -- 172.21.15.86:0/3225216102 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7ff2dc032020 con 0x55b0a9e04f50 2024-03-22T04:20:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 10 client.8811 fetch_fsmap finished waiting for FSMap version 90 2024-03-22T04:20:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 10 client.8811 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-03-22T04:20:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 20 client.8811 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55b0a9e575c0 0x55b0a9e579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 4 client.8811 mds_command: new command op to 77159 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:20:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2f4006000 1 -- 172.21.15.86:0/3225216102 --> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0a9bf0dd0 con 0x55b0a9e575c0 2024-03-22T04:20:40.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e1400640 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55b0a9e575c0 0x55b0a9e579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2e1400640 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55b0a9e575c0 0x55b0a9e579d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d88ed171ae9fc6e server_cookie=229748afe9ed7802 in_seq=0 out_seq=0 2024-03-22T04:20:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.127+0000 7ff2cf400640 10 client.8811 ms_handle_connect on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2cf400640 1 -- 172.21.15.86:0/3225216102 <== mds.0 v2:172.21.15.196:6832/269054026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b0a9bf0dd0 con 0x55b0a9e575c0 2024-03-22T04:20:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2cf400640 10 client.8811 handle_command_reply: tid=0 2024-03-22T04:20:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 client.8811 shutdown 2024-03-22T04:20:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 -- 172.21.15.86:0/3225216102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 msgr2=0x55b0a9e0c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e0c9c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0146d0 tx=0x7ff2dc005800 comp rx=0 tx=0).stop 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 -- 172.21.15.86:0/3225216102 shutdown_connections 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55b0a9e575c0 0x55b0a9e579d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e35380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 0x55b0a9e358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2cf400640 0 client.8811 ms_handle_reset on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 --2- 172.21.15.86:0/3225216102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e0c9c0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 -- 172.21.15.86:0/3225216102 >> 172.21.15.86:0/3225216102 conn(0x55b0a9cee990 msgr2=0x55b0a9e04110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 -- 172.21.15.86:0/3225216102 shutdown_connections 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 1 -- 172.21.15.86:0/3225216102 wait complete. 2024-03-22T04:20:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.131+0000 7ff2f4006000 20 client.8811 trim_cache size 0 max 16384 2024-03-22T04:20:40.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2f4006000 1 Processor -- start 2024-03-22T04:20:40.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2f4006000 1 -- start start 2024-03-22T04:20:40.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e67680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e0b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e0bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2f4006000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec2080c0 con 0x55b0a9e518a0 2024-03-22T04:20:40.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2f4006000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec2070e0 con 0x55b0a9e50f80 2024-03-22T04:20:40.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2f4006000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ec147c90 con 0x55b0a9e04f50 2024-03-22T04:20:40.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e0bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e0bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54546/0 (socket says 172.21.15.86:54546) 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 -- 172.21.15.86:0/2934938462 learned_addr learned my addr 172.21.15.86:0/2934938462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2e1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e0b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/2934938462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e67680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 -- 172.21.15.86:0/2934938462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 msgr2=0x55b0a9e67680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 --2- 172.21.15.86:0/2934938462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e67680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 -- 172.21.15.86:0/2934938462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 msgr2=0x55b0a9e0b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 --2- 172.21.15.86:0/2934938462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e0b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 -- 172.21.15.86:0/2934938462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a9c2b3a0 con 0x55b0a9e518a0 2024-03-22T04:20:40.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2ea800640 1 --2- 172.21.15.86:0/2934938462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e0bde0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400ef90 tx=0x7ff2e4056c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fd1e93ebd46244 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2da000640 1 -- 172.21.15.86:0/2934938462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4054e90 con 0x55b0a9e518a0 2024-03-22T04:20:40.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2da000640 1 -- 172.21.15.86:0/2934938462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e405d0b0 con 0x55b0a9e518a0 2024-03-22T04:20:40.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.147+0000 7ff2da000640 1 -- 172.21.15.86:0/2934938462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4055fe0 con 0x55b0a9e518a0 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- 172.21.15.86:0/2934938462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 msgr2=0x55b0a9e0bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2934938462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e0bde0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400ef90 tx=0x7ff2e4056c00 comp rx=0 tx=0).stop 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- 172.21.15.86:0/2934938462 shutdown_connections 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2934938462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9e0b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2934938462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e518a0 0x55b0a9e0bde0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2934938462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e04f50 0x55b0a9e67680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- 172.21.15.86:0/2934938462 >> 172.21.15.86:0/2934938462 conn(0x55b0a9cee990 msgr2=0x55b0a9e0b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- 172.21.15.86:0/2934938462 shutdown_connections 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- 172.21.15.86:0/2934938462 wait complete. 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- start start 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 Processor -- start 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9ebc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 0x55b0a9ebcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9d11810 con 0x55b0a9e04f50 2024-03-22T04:20:40.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1482e0 con 0x55b0a9e50f80 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b0a9c54f50 con 0x55b0a9e518a0 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 10 client.0 _ll_register_callbacks cb 0x7ff2dc001560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54550/0 (socket says 172.21.15.86:54550) 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 -- 172.21.15.86:0/2388904041 learned_addr learned my addr 172.21.15.86:0/2388904041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e1400640 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9ebc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2ea800640 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 0x55b0a9ebcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 -- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 msgr2=0x55b0a9ebcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 0x55b0a9ebcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 -- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 msgr2=0x55b0a9ebc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9ebc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 -- 172.21.15.86:0/2388904041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a9bdb380 con 0x55b0a9e04f50 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2e9e00640 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e673d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0068a0 tx=0x7ff2dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dda3ac3cd3f40a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2cf400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2cf400640 1 -- 172.21.15.86:0/2388904041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc025040 con 0x55b0a9e04f50 2024-03-22T04:20:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2cf400640 1 -- 172.21.15.86:0/2388904041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6303+0+0 (secure 0 0 0) 0x7ff2dc013b10 con 0x55b0a9e04f50 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- 172.21.15.86:0/2388904041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e405d0b0 con 0x55b0a9e04f50 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2cf400640 1 -- 172.21.15.86:0/2388904041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff2dc043050 con 0x55b0a9e04f50 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 10 client.8817 fetch_fsmap learned FSMap version 90 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 -- 172.21.15.86:0/2388904041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55b0a9cd7460 con 0x55b0a9e04f50 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2cf400640 1 -- 172.21.15.86:0/2388904041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e405d0b0 con 0x55b0a9e04f50 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2cf400640 1 -- 172.21.15.86:0/2388904041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7ff2dc046020 con 0x55b0a9e04f50 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 10 client.8817 fetch_fsmap finished waiting for FSMap version 90 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 10 client.8817 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 20 client.8817 populate_metadata read hostname 'smithi086' 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.151+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55b0a9ebd260 0x55b0a9ebd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 4 client.8817 mds_command: new command op to 77159 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 -- 172.21.15.86:0/2388904041 --> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0a9bf5160 con 0x55b0a9ebd260 2024-03-22T04:20:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2e1400640 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55b0a9ebd260 0x55b0a9ebd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2e1400640 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55b0a9ebd260 0x55b0a9ebd670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd10a22169cf5b6 server_cookie=f761cd198e3d25c2 in_seq=0 out_seq=0 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2cf400640 10 client.8817 ms_handle_connect on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2cf400640 1 -- 172.21.15.86:0/2388904041 <== mds.0 v2:172.21.15.196:6832/269054026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0a9bf5160 con 0x55b0a9ebd260 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2cf400640 10 client.8817 handle_command_reply: tid=0 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 client.8817 shutdown 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 -- 172.21.15.86:0/2388904041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 msgr2=0x55b0a9e673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e673d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0068a0 tx=0x7ff2dc015000 comp rx=0 tx=0).stop 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 -- 172.21.15.86:0/2388904041 shutdown_connections 2024-03-22T04:20:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:6832/269054026,v1:172.21.15.196:6834/269054026] conn(0x55b0a9ebd260 0x55b0a9ebd670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b0a9e50f80 0x55b0a9ebc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b0a9e518a0 0x55b0a9ebcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 --2- 172.21.15.86:0/2388904041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a9e04f50 0x55b0a9e673d0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2cf400640 0 client.8817 ms_handle_reset on v2:172.21.15.196:6832/269054026 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 -- 172.21.15.86:0/2388904041 >> 172.21.15.86:0/2388904041 conn(0x55b0a9cee990 msgr2=0x55b0a9e34550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 -- 172.21.15.86:0/2388904041 shutdown_connections 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 1 -- 172.21.15.86:0/2388904041 wait complete. 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f4006000 20 client.8817 trim_cache size 0 max 16384 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:20:40.160 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-22T04:20:40.161 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-22T04:20:40.161 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:20:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 -- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2c8067b20 msgr2=0x7ff2c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2c8067b20 0x7ff2c8069fe0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc007fd0 tx=0x7ff2dc009040 comp rx=0 tx=0).stop 2024-03-22T04:20:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 -- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec15cee0 msgr2=0x7ff2ec15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec15cee0 0x7ff2ec15b600 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400efc0 tx=0x7ff2e400c490 comp rx=0 tx=0).stop 2024-03-22T04:20:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 -- 172.21.15.86:0/1421472072 shutdown_connections 2024-03-22T04:20:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff2c8067b20 0x7ff2c8069fe0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ec15cee0 0x7ff2ec15b600 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2ec0b3600 0x7ff2ec145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 --2- 172.21.15.86:0/1421472072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2ec0b2360 0x7ff2ec1ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 -- 172.21.15.86:0/1421472072 >> 172.21.15.86:0/1421472072 conn(0x7ff2ec0cc940 msgr2=0x7ff2ec0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 -- 172.21.15.86:0/1421472072 shutdown_connections 2024-03-22T04:20:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.155+0000 7ff2f0c00640 1 -- 172.21.15.86:0/1421472072 wait complete. 2024-03-22T04:20:40.175 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-22T04:20:40.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:20:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91c00640 1 Processor -- start 2024-03-22T04:20:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91c00640 1 -- start start 2024-03-22T04:20:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9c8c09b180 0x7f9c8c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8c0b33b0 con 0x7f9c8c161cd0 2024-03-22T04:20:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8c0d1cc0 con 0x7f9c8c0b4a90 2024-03-22T04:20:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8c0d0dd0 con 0x7f9c8c09b180 2024-03-22T04:20:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c8be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9c8c161cd0 0x7f9c8c09aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51648/0 (socket says 172.21.15.86:51648) 2024-03-22T04:20:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c8be00640 1 -- 172.21.15.86:0/3393781564 learned_addr learned my addr 172.21.15.86:0/3393781564 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91200640 1 --2- 172.21.15.86:0/3393781564 >> v2:172.21.15.196:3300/0 conn(0x7f9c8c09b180 0x7f9c8c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c8b400640 1 -- 172.21.15.86:0/3393781564 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934028050 0 0) 0x7f9c8c0b33b0 con 0x7f9c8c161cd0 2024-03-22T04:20:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c8b400640 1 -- 172.21.15.86:0/3393781564 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c70003530 con 0x7f9c8c161cd0 2024-03-22T04:20:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c8b400640 1 -- 172.21.15.86:0/3393781564 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515013994 0 0) 0x7f9c8c0d1cc0 con 0x7f9c8c0b4a90 2024-03-22T04:20:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c8b400640 1 -- 172.21.15.86:0/3393781564 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c8c0b33b0 con 0x7f9c8c0b4a90 2024-03-22T04:20:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91200640 1 -- 172.21.15.86:0/3393781564 >> v1:172.21.15.196:6789/0 conn(0x7f9c8c0b4a90 legacy=0x7f9c8c0b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91200640 1 -- 172.21.15.86:0/3393781564 >> v1:172.21.15.86:6789/0 conn(0x7f9c8c161cd0 legacy=0x7f9c8c09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.415+0000 7f9c91200640 1 -- 172.21.15.86:0/3393781564 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c0d1fa0 con 0x7f9c8c09b180 2024-03-22T04:20:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c91200640 1 --2- 172.21.15.86:0/3393781564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 0x7f9c8c0b3690 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009d90 tx=0x7f9c8401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1fb2d05daa88e06 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c8b400640 1 -- 172.21.15.86:0/3393781564 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840b2070 con 0x7f9c8c09b180 2024-03-22T04:20:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c8b400640 1 -- 172.21.15.86:0/3393781564 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840a5830 con 0x7f9c8c09b180 2024-03-22T04:20:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c8b400640 1 -- 172.21.15.86:0/3393781564 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840a5c00 con 0x7f9c8c09b180 2024-03-22T04:20:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c91c00640 1 -- 172.21.15.86:0/3393781564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 msgr2=0x7f9c8c0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c91c00640 1 --2- 172.21.15.86:0/3393781564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 0x7f9c8c0b3690 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009d90 tx=0x7f9c8401c970 comp rx=0 tx=0).stop 2024-03-22T04:20:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c91c00640 1 -- 172.21.15.86:0/3393781564 shutdown_connections 2024-03-22T04:20:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c91c00640 1 --2- 172.21.15.86:0/3393781564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 0x7f9c8c0b3690 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c91c00640 1 -- 172.21.15.86:0/3393781564 >> 172.21.15.86:0/3393781564 conn(0x7f9c8c0b3d60 msgr2=0x7f9c8c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c91c00640 1 -- 172.21.15.86:0/3393781564 shutdown_connections 2024-03-22T04:20:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.419+0000 7f9c91c00640 1 -- 172.21.15.86:0/3393781564 wait complete. 2024-03-22T04:20:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 Processor -- start 2024-03-22T04:20:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 -- start start 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 0x7f9c8c143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c8c0b4a90 0x7f9c8c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c8c161cd0 0x7f9c8c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c16ce10 con 0x7f9c8c161cd0 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c16cc90 con 0x7f9c8c09b180 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8c16cf90 con 0x7f9c8c0b4a90 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c8c161cd0 0x7f9c8c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c8be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 0x7f9c8c143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c8c161cd0 0x7f9c8c0b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54562/0 (socket says 172.21.15.86:54562) 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c8be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 0x7f9c8c143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:55942/0 (socket says 172.21.15.86:55942) 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c83400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c8c0b4a90 0x7f9c8c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 -- 172.21.15.86:0/3467269414 learned_addr learned my addr 172.21.15.86:0/3467269414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 -- 172.21.15.86:0/3467269414 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c8c0b4a90 msgr2=0x7f9c8c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c8c0b4a90 0x7f9c8c0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 -- 172.21.15.86:0/3467269414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 msgr2=0x7f9c8c143e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 0x7f9c8c143e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 -- 172.21.15.86:0/3467269414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c0b6d20 con 0x7f9c8c161cd0 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91200640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c8c161cd0 0x7f9c8c0b6110 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9c8401a5a0 tx=0x7f9c84019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c88c00640 1 -- 172.21.15.86:0/3467269414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840b2050 con 0x7f9c8c161cd0 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c88c00640 1 -- 172.21.15.86:0/3467269414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840be070 con 0x7f9c8c161cd0 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c143320 con 0x7f9c8c161cd0 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c88c00640 1 -- 172.21.15.86:0/3467269414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84009450 con 0x7f9c8c161cd0 2024-03-22T04:20:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.423+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c0cf490 con 0x7f9c8c161cd0 2024-03-22T04:20:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.427+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c074470 con 0x7f9c8c161cd0 2024-03-22T04:20:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.427+0000 7f9c88c00640 1 -- 172.21.15.86:0/3467269414 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9c840adb80 con 0x7f9c8c161cd0 2024-03-22T04:20:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.427+0000 7f9c88c00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c68067b20 0x7f9c68069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.427+0000 7f9c88c00640 1 -- 172.21.15.86:0/3467269414 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6303+0+0 (secure 0 0 0) 0x7f9c84121330 con 0x7f9c8c161cd0 2024-03-22T04:20:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.427+0000 7f9c8be00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c68067b20 0x7f9c68069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.427+0000 7f9c8be00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c68067b20 0x7f9c68069fe0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c007fd0 tx=0x7f9c7c007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.431+0000 7f9c88c00640 1 -- 172.21.15.86:0/3467269414 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9c84016560 con 0x7f9c8c161cd0 2024-03-22T04:20:40.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.619+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9c8c09ad50 con 0x7f9c8c161cd0 2024-03-22T04:20:40.622 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:20:40.623+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:20:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.623+0000 7f9c88c00640 1 -- 172.21.15.86:0/3467269414 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7f9c840f1520 con 0x7f9c8c161cd0 2024-03-22T04:20:40.624 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:20:40.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c68067b20 msgr2=0x7f9c68069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c68067b20 0x7f9c68069fe0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c007fd0 tx=0x7f9c7c007d80 comp rx=0 tx=0).stop 2024-03-22T04:20:40.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c8c161cd0 msgr2=0x7f9c8c0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c8c161cd0 0x7f9c8c0b6110 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9c8401a5a0 tx=0x7f9c84019710 comp rx=0 tx=0).stop 2024-03-22T04:20:40.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 shutdown_connections 2024-03-22T04:20:40.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c68067b20 0x7f9c68069fe0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c8c161cd0 0x7f9c8c0b6110 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c8c0b4a90 0x7f9c8c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 --2- 172.21.15.86:0/3467269414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c8c09b180 0x7f9c8c143e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 >> 172.21.15.86:0/3467269414 conn(0x7f9c8c0b3d60 msgr2=0x7f9c8c0cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 shutdown_connections 2024-03-22T04:20:40.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.627+0000 7f9c91c00640 1 -- 172.21.15.86:0/3467269414 wait complete. 2024-03-22T04:20:40.635 INFO:tasks.ceph.mds.d.smithi196.stderr: -1360> 2024-03-22T04:20:08.290+0000 7fbb6a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:20:40.636 INFO:tasks.ceph.mds.d.smithi196.stderr: -1152> 2024-03-22T04:20:09.734+0000 7fbb65800640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-22T04:20:40.636 INFO:tasks.ceph.mds.d.smithi196.stderr: -1149> 2024-03-22T04:20:09.734+0000 7fbb65800640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-22T04:20:40.636 INFO:tasks.ceph.mds.d.smithi196.stderr: -1120> 2024-03-22T04:20:09.734+0000 7fbb65800640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-22T04:20:40.636 INFO:tasks.ceph.mds.d.smithi196.stderr: -1117> 2024-03-22T04:20:09.734+0000 7fbb65800640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-22T04:20:40.647 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-03-22T04:20:40.677 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:20:40.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 -- start start 2024-03-22T04:20:40.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 Processor -- start 2024-03-22T04:20:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565410e4d800 0x565411c0a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565410e4d400 0x565411c0a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565410e4d000 0x565411c0ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565411c34180 con 0x565410e4d000 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565410eac900 con 0x565410e4d800 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x565410eac480 con 0x565410e4d400 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565410e4d000 0x565411c0ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565410e4d000 0x565411c0ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54578/0 (socket says 172.21.15.86:54578) 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 -- 172.21.15.86:0/3627405537 learned_addr learned my addr 172.21.15.86:0/3627405537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2000640 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565410e4d800 0x565411c0a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d1600640 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565410e4d400 0x565411c0a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 -- 172.21.15.86:0/3627405537 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565410e4d400 msgr2=0x565411c0a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565410e4d400 0x565411c0a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 -- 172.21.15.86:0/3627405537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565410e4d800 msgr2=0x565411c0a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565410e4d800 0x565411c0a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 -- 172.21.15.86:0/3627405537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565411c08b40 con 0x565410e4d000 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d2a00640 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565410e4d000 0x565411c0ab00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x565411ccccc0 tx=0x565410f20180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cfbad3f8a6d862 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48cee00640 1 -- 172.21.15.86:0/3627405537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565410e829c0 con 0x565410e4d000 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48cee00640 1 -- 172.21.15.86:0/3627405537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565410e981c0 con 0x565410e4d000 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48cee00640 1 -- 172.21.15.86:0/3627405537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565410e824e0 con 0x565410e4d000 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3627405537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565411c08d20 con 0x565410e4d000 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3627405537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x565411c08f00 con 0x565410e4d000 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 4 waiting for MDS map... 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48cee00640 1 -- 172.21.15.86:0/3627405537 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x565411d68200 con 0x565410e4d000 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48cee00640 1 -- 172.21.15.86:0/3627405537 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 91) ==== 2885+0+0 (secure 0 0 0) 0x565410ea3180 con 0x565410e4d000 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 4 Got MDS map 91 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 10 main: main 2024-03-22T04:20:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.691+0000 7f48d3fa7400 4 main: connecting to RADOS... 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 Processor -- start 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- start start 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 0x565411c0b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dba000 0x565411c0b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dba400 0x565411c0bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565410eac180 con 0x565411dba400 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565411c34d80 con 0x565411dba000 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x565411c34f00 con 0x565411c19c00 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d1600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 0x565411c0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dba000 0x565411c0b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dba000 0x565411c0b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:55988/0 (socket says 172.21.15.86:55988) 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 -- 172.21.15.86:0/3161289090 learned_addr learned my addr 172.21.15.86:0/3161289090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d1600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 0x565411c0b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:42162/0 (socket says 172.21.15.86:42162) 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d1600640 1 --2- 172.21.15.86:0/3161289090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dba400 0x565411c0bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 -- 172.21.15.86:0/3161289090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 msgr2=0x565411c0b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 --2- 172.21.15.86:0/3161289090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 0x565411c0b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 -- 172.21.15.86:0/3161289090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dba400 msgr2=0x565411c0bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 --2- 172.21.15.86:0/3161289090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dba400 0x565411c0bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 -- 172.21.15.86:0/3161289090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565411c081e0 con 0x565411dba000 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d2000640 1 --2- 172.21.15.86:0/3161289090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dba000 0x565411c0b600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x565411dce0c0 tx=0x565411d6b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd8cf5946c97df0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48cc600640 1 -- 172.21.15.86:0/3161289090 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565410e829c0 con 0x565411dba000 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48cc600640 1 -- 172.21.15.86:0/3161289090 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565410e98c40 con 0x565411dba000 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48cc600640 1 -- 172.21.15.86:0/3161289090 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565410e824e0 con 0x565411dba000 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3161289090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dba000 msgr2=0x565411c0b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3161289090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dba000 0x565411c0b600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x565411dce0c0 tx=0x565411d6b920 comp rx=0 tx=0).stop 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3161289090 shutdown_connections 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3161289090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dba000 0x565411c0b600 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3161289090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dba400 0x565411c0bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3161289090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 0x565411c0b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3161289090 >> 172.21.15.86:0/3161289090 conn(0x565410e4cc00 msgr2=0x565410e7b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3161289090 shutdown_connections 2024-03-22T04:20:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3161289090 wait complete. 2024-03-22T04:20:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 Processor -- start 2024-03-22T04:20:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.695+0000 7f48d3fa7400 1 -- start start 2024-03-22T04:20:40.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 0x565411c0b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dbb800 0x565411c0b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dbb400 0x565411c0bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565411c35200 con 0x565411dbb400 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565411c35080 con 0x565411dbb800 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x565411c35380 con 0x565411c19c00 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dbb400 0x565411c0bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dbb400 0x565411c0bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54602/0 (socket says 172.21.15.86:54602) 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 -- 172.21.15.86:0/3030442443 learned_addr learned my addr 172.21.15.86:0/3030442443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d2000640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dbb800 0x565411c0b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 0x565411c0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 -- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 msgr2=0x565411c0b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565411c19c00 0x565411c0b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 -- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dbb800 msgr2=0x565411c0b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dbb800 0x565411c0b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 -- 172.21.15.86:0/3030442443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565411c09a40 con 0x565411dbb400 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d2000640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565411dbb800 0x565411c0b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d1600640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dbb400 0x565411c0bb80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x565411dcf3e0 tx=0x565410f1e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48ca800640 1 -- 172.21.15.86:0/3030442443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565410e829c0 con 0x565411dbb400 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48ca800640 1 -- 172.21.15.86:0/3030442443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565410e98540 con 0x565411dbb400 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48ca800640 1 -- 172.21.15.86:0/3030442443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565410e824e0 con 0x565411dbb400 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565411c09c20 con 0x565411dbb400 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565411c09c20 con 0x565411dbb400 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 4 main: resolving pool 15 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48ca800640 1 -- 172.21.15.86:0/3030442443 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x565411db8c00 con 0x565411dbb400 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48ca800640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565411dd5400 0x565411c0c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48ca800640 1 -- 172.21.15.86:0/3030442443 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x565411154600 con 0x565411dbb400 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 4 main: creating IoCtx.. 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x565411dbb000 0x565411c0c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d2000640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565411dd5400 0x565411c0c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.699+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x565411dbac00 con 0x565411dbb000 2024-03-22T04:20:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d1600640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x565411dbb000 0x565411c0c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:40.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d2000640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565411dd5400 0x565411c0c100 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x565411dce0c0 tx=0x565410f1e900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d1600640 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x565411dbb000 0x565411c0c680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:40.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d1600640 1 -- 172.21.15.86:0/3030442443 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x565411bca000 con 0x565411dbb000 2024-03-22T04:20:40.705 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:20:40.705 INFO:teuthology.orchestra.run.smithi086.stdout: "0": { 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "data": { 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 5, 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "inotable": { 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "projected_free": [ 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "start": 1099511627779, 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "len": 1099511627773 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "free": [ 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "start": 1099511627779, 2024-03-22T04:20:40.706 INFO:teuthology.orchestra.run.smithi086.stdout: "len": 1099511627773 2024-03-22T04:20:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T04:20:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: "result": 0 2024-03-22T04:20:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:40.707 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:20:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:40.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x565411dbb000 msgr2=0x565411c0c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x565411dbb000 0x565411c0c680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565411dd5400 msgr2=0x565411c0c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x565411dd5400 0x565411c0c100 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x565411dce0c0 tx=0x565410f1e900 comp rx=0 tx=0).stop 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dbb400 msgr2=0x565411c0bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3030442443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565411dbb400 0x565411c0bb80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x565411dcf3e0 tx=0x565410f1e840 comp rx=0 tx=0).stop 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d2000640 1 -- 172.21.15.86:0/3030442443 reap_dead start 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 shutdown_connections 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 >> 172.21.15.86:0/3030442443 conn(0x565410e4cc00 msgr2=0x565410e7b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 shutdown_connections 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3030442443 wait complete. 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3627405537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565410e4d000 msgr2=0x565411c0ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565410e4d000 0x565411c0ab00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x565411ccccc0 tx=0x565410f20180 comp rx=0 tx=0).stop 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3627405537 shutdown_connections 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565410e4d000 0x565411c0ab00 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x565410e4d400 0x565411c0a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 --2- 172.21.15.86:0/3627405537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565410e4d800 0x565411c0a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3627405537 >> 172.21.15.86:0/3627405537 conn(0x565410e4dc00 msgr2=0x565410e7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3627405537 shutdown_connections 2024-03-22T04:20:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:40.703+0000 7f48d3fa7400 1 -- 172.21.15.86:0/3627405537 wait complete. 2024-03-22T04:20:40.711 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.063518, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-03-22T04:20:40.711 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:20:40.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:20:40.833 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:20:40.925 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:20:40.925 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:20:40.925 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:20:40.925 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:20:40.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:20:40.934 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:20:40.934 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:20:40.934 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:20:40.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:20:40.943 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:20:40.943 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:20:40.943 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:20:40.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:20:40.986 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:20:40.986 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:20:40.986 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:20:40.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:20:40.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:20:41.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.163+0000 7fb228200640 1 Processor -- start 2024-03-22T04:20:41.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.163+0000 7fb228200640 1 -- start start 2024-03-22T04:20:41.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb228200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2200aff50 0x7fb22006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb228200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200ae5a0 con 0x7fb22006cbb0 2024-03-22T04:20:41.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb228200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb220144b80 con 0x7fb2201444f0 2024-03-22T04:20:41.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb228200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb22006d010 con 0x7fb2200aff50 2024-03-22T04:20:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb225800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2200aff50 0x7fb22006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb225800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2200aff50 0x7fb22006c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54608/0 (socket says 172.21.15.86:54608) 2024-03-22T04:20:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb225800640 1 -- 172.21.15.86:0/1787849779 learned_addr learned my addr 172.21.15.86:0/1787849779 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb224e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb22006cbb0 0x7fb220143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51660/0 (socket says 172.21.15.86:51660) 2024-03-22T04:20:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb225800640 1 -- 172.21.15.86:0/1787849779 >> v1:172.21.15.196:6789/0 conn(0x7fb2201444f0 legacy=0x7fb22015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb225800640 1 -- 172.21.15.86:0/1787849779 >> v1:172.21.15.86:6789/0 conn(0x7fb22006cbb0 legacy=0x7fb220143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb225800640 1 -- 172.21.15.86:0/1787849779 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb220162170 con 0x7fb2200aff50 2024-03-22T04:20:41.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.167+0000 7fb21fe00640 1 -- 172.21.15.86:0/1787849779 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404203784 0 0) 0x7fb20c001120 con 0x7fb22006cbb0 2024-03-22T04:20:41.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb225800640 1 --2- 172.21.15.86:0/1787849779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2200aff50 0x7fb22006c670 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fb214009870 tx=0x7fb21401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f138e3a3d6f282fa server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb21fe00640 1 -- 172.21.15.86:0/1787849779 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2140b0070 con 0x7fb2200aff50 2024-03-22T04:20:41.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb21fe00640 1 -- 172.21.15.86:0/1787849779 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2140ab600 con 0x7fb2200aff50 2024-03-22T04:20:41.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb21fe00640 1 -- 172.21.15.86:0/1787849779 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2140ab9d0 con 0x7fb2200aff50 2024-03-22T04:20:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 -- 172.21.15.86:0/1787849779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2200aff50 msgr2=0x7fb22006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 --2- 172.21.15.86:0/1787849779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2200aff50 0x7fb22006c670 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fb214009870 tx=0x7fb21401c4d0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 -- 172.21.15.86:0/1787849779 shutdown_connections 2024-03-22T04:20:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 --2- 172.21.15.86:0/1787849779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2200aff50 0x7fb22006c670 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 -- 172.21.15.86:0/1787849779 >> 172.21.15.86:0/1787849779 conn(0x7fb220071fe0 msgr2=0x7fb2200723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 -- 172.21.15.86:0/1787849779 shutdown_connections 2024-03-22T04:20:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 -- 172.21.15.86:0/1787849779 wait complete. 2024-03-22T04:20:41.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 Processor -- start 2024-03-22T04:20:41.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.171+0000 7fb228200640 1 -- start start 2024-03-22T04:20:41.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cbb0 0x7fb2201476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200aff50 0x7fb220128800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2201444f0 0x7fb220145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200b04b0 con 0x7fb22006cbb0 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200b0330 con 0x7fb2200aff50 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb22016c310 con 0x7fb2201444f0 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cbb0 0x7fb2201476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cbb0 0x7fb2201476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54622/0 (socket says 172.21.15.86:54622) 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 -- 172.21.15.86:0/1688788254 learned_addr learned my addr 172.21.15.86:0/1688788254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb224e00640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200aff50 0x7fb220128800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb226200640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2201444f0 0x7fb220145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 -- 172.21.15.86:0/1688788254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2201444f0 msgr2=0x7fb220145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2201444f0 0x7fb220145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 -- 172.21.15.86:0/1688788254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200aff50 msgr2=0x7fb220128800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200aff50 0x7fb220128800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 -- 172.21.15.86:0/1688788254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb220146230 con 0x7fb22006cbb0 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb225800640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cbb0 0x7fb2201476a0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb2140ae040 tx=0x7fb21401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb21d600640 1 -- 172.21.15.86:0/1688788254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2140b0040 con 0x7fb22006cbb0 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb21d600640 1 -- 172.21.15.86:0/1688788254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2140bc070 con 0x7fb22006cbb0 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22013fad0 con 0x7fb22006cbb0 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb21d600640 1 -- 172.21.15.86:0/1688788254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2140b6880 con 0x7fb22006cbb0 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb220140090 con 0x7fb22006cbb0 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb220068650 con 0x7fb22006cbb0 2024-03-22T04:20:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.175+0000 7fb21d600640 1 -- 172.21.15.86:0/1688788254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb2140b6a20 con 0x7fb22006cbb0 2024-03-22T04:20:41.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.179+0000 7fb21d600640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.179+0000 7fb21d600640 1 -- 172.21.15.86:0/1688788254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7fb214120580 con 0x7fb22006cbb0 2024-03-22T04:20:41.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.179+0000 7fb224e00640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.179+0000 7fb224e00640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb22012a7f0 tx=0x7fb20c007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.183+0000 7fb21d600640 1 -- 172.21.15.86:0/1688788254 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb214016150 con 0x7fb22006cbb0 2024-03-22T04:20:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.371+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb22006d1a0 con 0x7fb22006cbb0 2024-03-22T04:20:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.371+0000 7fb21d600640 1 -- 172.21.15.86:0/1688788254 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+4342 (secure 0 0 0) 0x7fb2140f0560 con 0x7fb22006cbb0 2024-03-22T04:20:41.374 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:41.374 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/70563317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":70563317},{"type":"v1","addr":"172.21.15.196:6835","nonce":70563317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:20:40.626899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-22T04:20:41.374 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 91 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 msgr2=0x7fb200069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb22012a7f0 tx=0x7fb20c007b60 comp rx=0 tx=0).stop 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cbb0 msgr2=0x7fb2201476a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cbb0 0x7fb2201476a0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb2140ae040 tx=0x7fb21401ad10 comp rx=0 tx=0).stop 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 shutdown_connections 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200aff50 0x7fb220128800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2201444f0 0x7fb220145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 --2- 172.21.15.86:0/1688788254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb22006cbb0 0x7fb2201476a0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 >> 172.21.15.86:0/1688788254 conn(0x7fb220071fe0 msgr2=0x7fb220158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 shutdown_connections 2024-03-22T04:20:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.375+0000 7fb228200640 1 -- 172.21.15.86:0/1688788254 wait complete. 2024-03-22T04:20:41.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:20:41.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f661ba00640 1 Processor -- start 2024-03-22T04:20:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f661ba00640 1 -- start start 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f661ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f661ba00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f661406e2f0 0x7f66140d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f661ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66141623d0 con 0x7f661406cc80 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f661ba00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f66141620f0 con 0x7f661406e2f0 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f661ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f661415ec10 con 0x7f661415e630 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6619a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f661406cc80 0x7f661406d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51674/0 (socket says 172.21.15.86:51674) 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6619a00640 1 -- 172.21.15.86:0/3301111318 learned_addr learned my addr 172.21.15.86:0/3301111318 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6619000640 1 --2- 172.21.15.86:0/3301111318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f660be00640 1 --2- 172.21.15.86:0/3301111318 >> v2:172.21.15.196:3300/0 conn(0x7f661406e2f0 0x7f66140d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6613e00640 1 -- 172.21.15.86:0/3301111318 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960503060 0 0) 0x7f66141623d0 con 0x7f661406cc80 2024-03-22T04:20:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6613e00640 1 -- 172.21.15.86:0/3301111318 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65f8003530 con 0x7f661406cc80 2024-03-22T04:20:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6619000640 1 -- 172.21.15.86:0/3301111318 >> v2:172.21.15.196:3300/0 conn(0x7f661406e2f0 msgr2=0x7f66140d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6619000640 1 --2- 172.21.15.86:0/3301111318 >> v2:172.21.15.196:3300/0 conn(0x7f661406e2f0 0x7f66140d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6619000640 1 -- 172.21.15.86:0/3301111318 >> v1:172.21.15.86:6789/0 conn(0x7f661406cc80 legacy=0x7f661406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f6619000640 1 -- 172.21.15.86:0/3301111318 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661415eef0 con 0x7f661415e630 2024-03-22T04:20:41.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.531+0000 7f660be00640 1 --2- 172.21.15.86:0/3301111318 >> v2:172.21.15.196:3300/0 conn(0x7f661406e2f0 0x7f66140d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f6619000640 1 --2- 172.21.15.86:0/3301111318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140cfe70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f66040097c0 tx=0x7f660401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2aef5e7ae6b4fe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f6613e00640 1 -- 172.21.15.86:0/3301111318 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66040b0070 con 0x7f661415e630 2024-03-22T04:20:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f6613e00640 1 -- 172.21.15.86:0/3301111318 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66040ab600 con 0x7f661415e630 2024-03-22T04:20:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f6613e00640 1 -- 172.21.15.86:0/3301111318 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66040ab9d0 con 0x7f661415e630 2024-03-22T04:20:41.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f661ba00640 1 -- 172.21.15.86:0/3301111318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 msgr2=0x7f66140cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f661ba00640 1 --2- 172.21.15.86:0/3301111318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140cfe70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f66040097c0 tx=0x7f660401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f661ba00640 1 -- 172.21.15.86:0/3301111318 shutdown_connections 2024-03-22T04:20:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f661ba00640 1 --2- 172.21.15.86:0/3301111318 >> v2:172.21.15.196:3300/0 conn(0x7f661406e2f0 0x7f66140d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f661ba00640 1 --2- 172.21.15.86:0/3301111318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140cfe70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f661ba00640 1 -- 172.21.15.86:0/3301111318 >> 172.21.15.86:0/3301111318 conn(0x7f661406f910 msgr2=0x7f661406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f661ba00640 1 -- 172.21.15.86:0/3301111318 shutdown_connections 2024-03-22T04:20:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.535+0000 7f661ba00640 1 -- 172.21.15.86:0/3301111318 wait complete. 2024-03-22T04:20:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 Processor -- start 2024-03-22T04:20:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 -- start start 2024-03-22T04:20:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f661406cc80 0x7f661415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661406e2f0 0x7f66140c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66140cfaf0 con 0x7f661406cc80 2024-03-22T04:20:41.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f661415f9d0 con 0x7f661406e2f0 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f66140cfc70 con 0x7f661415e630 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f661406cc80 0x7f661415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f661406cc80 0x7f661415d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54628/0 (socket says 172.21.15.86:54628) 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 -- 172.21.15.86:0/2135067367 learned_addr learned my addr 172.21.15.86:0/2135067367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f660be00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661406e2f0 0x7f66140c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619a00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 -- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 msgr2=0x7f66140c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 -- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661406e2f0 msgr2=0x7f66140c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661406e2f0 0x7f66140c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 -- 172.21.15.86:0/2135067367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66140b6130 con 0x7f661406cc80 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619a00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6619000640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f661406cc80 0x7f661415d390 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f660401a0d0 tx=0x7f6604019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6611600640 1 -- 172.21.15.86:0/2135067367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66040b0040 con 0x7f661406cc80 2024-03-22T04:20:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6611600640 1 -- 172.21.15.86:0/2135067367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66040bd070 con 0x7f661406cc80 2024-03-22T04:20:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66140b6410 con 0x7f661406cc80 2024-03-22T04:20:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f6611600640 1 -- 172.21.15.86:0/2135067367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604006e20 con 0x7f661406cc80 2024-03-22T04:20:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.539+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66140b69d0 con 0x7f661406cc80 2024-03-22T04:20:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.543+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6614068650 con 0x7f661406cc80 2024-03-22T04:20:41.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.543+0000 7f6611600640 1 -- 172.21.15.86:0/2135067367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f66040b3070 con 0x7f661406cc80 2024-03-22T04:20:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.543+0000 7f6611600640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f65f0067b20 0x7f65f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.543+0000 7f6611600640 1 -- 172.21.15.86:0/2135067367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7f6604120220 con 0x7f661406cc80 2024-03-22T04:20:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.543+0000 7f660be00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f65f0067b20 0x7f65f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.543+0000 7f660be00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f65f0067b20 0x7f65f0069fe0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f66140c8ad0 tx=0x7f65fc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.547+0000 7f6611600640 1 -- 172.21.15.86:0/2135067367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6604016090 con 0x7f661406cc80 2024-03-22T04:20:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.703+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66140cf6e0 con 0x7f661406cc80 2024-03-22T04:20:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f6611600640 1 -- 172.21.15.86:0/2135067367 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17349 (secure 0 0 0) 0x7f66040f08e0 con 0x7f661406cc80 2024-03-22T04:20:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:41.707 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":113,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:40.626642+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:19:15.365871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-22T04:19:16.647234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.3","mappings":[{"from":2,"to":5}]},{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"16.3f","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6832/269054026":"2024-03-23T04:20:40.626610+0000","172.21.15.196:6835/2259098413":"2024-03-23T04:20:03.917951+0000","172.21.15.196:6834/269054026":"2024-03-23T04:20:40.626610+0000","172.21.15.196:6833/2259098413":"2024-03-23T04:20:03.917951+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:20:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f65f0067b20 msgr2=0x7f65f0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f65f0067b20 0x7f65f0069fe0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f66140c8ad0 tx=0x7f65fc009190 comp rx=0 tx=0).stop 2024-03-22T04:20:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f661406cc80 msgr2=0x7f661415d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f661406cc80 0x7f661415d390 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f660401a0d0 tx=0x7f6604019240 comp rx=0 tx=0).stop 2024-03-22T04:20:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 shutdown_connections 2024-03-22T04:20:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f65f0067b20 0x7f65f0069fe0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f661406e2f0 0x7f66140c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661415e630 0x7f66140c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 --2- 172.21.15.86:0/2135067367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f661406cc80 0x7f661415d390 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 >> 172.21.15.86:0/2135067367 conn(0x7f661406f910 msgr2=0x7f66140b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 shutdown_connections 2024-03-22T04:20:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.707+0000 7f661ba00640 1 -- 172.21.15.86:0/2135067367 wait complete. 2024-03-22T04:20:41.728 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:19:15.365871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:19:16.647234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:20:41.728 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:20:41.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0f400640 1 Processor -- start 2024-03-22T04:20:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0f400640 1 -- start start 2024-03-22T04:20:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca080b3600 0x7fca0815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca0815cee0 0x7fca080b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0f400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca0815d710 con 0x7fca080b2360 2024-03-22T04:20:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0f400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fca080b2740 con 0x7fca080b3600 2024-03-22T04:20:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0f400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fca0815fe40 con 0x7fca0815cee0 2024-03-22T04:20:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0ca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fca080b2360 0x7fca080b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51680/0 (socket says 172.21.15.86:51680) 2024-03-22T04:20:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0ca00640 1 -- 172.21.15.86:0/1568719839 learned_addr learned my addr 172.21.15.86:0/1568719839 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca07e00640 1 --2- 172.21.15.86:0/1568719839 >> v2:172.21.15.86:3300/0 conn(0x7fca080b3600 0x7fca0815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca0d400640 1 --2- 172.21.15.86:0/1568719839 >> v2:172.21.15.196:3300/0 conn(0x7fca0815cee0 0x7fca080b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca07400640 1 -- 172.21.15.86:0/1568719839 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3230330716 0 0) 0x7fca0815d710 con 0x7fca080b2360 2024-03-22T04:20:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca07400640 1 -- 172.21.15.86:0/1568719839 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9ec003530 con 0x7fca080b2360 2024-03-22T04:20:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca07e00640 1 -- 172.21.15.86:0/1568719839 >> v2:172.21.15.196:3300/0 conn(0x7fca0815cee0 msgr2=0x7fca080b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca07e00640 1 --2- 172.21.15.86:0/1568719839 >> v2:172.21.15.196:3300/0 conn(0x7fca0815cee0 0x7fca080b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca07e00640 1 -- 172.21.15.86:0/1568719839 >> v1:172.21.15.86:6789/0 conn(0x7fca080b2360 legacy=0x7fca080b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.867+0000 7fca07e00640 1 -- 172.21.15.86:0/1568719839 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca08160120 con 0x7fca080b3600 2024-03-22T04:20:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca07e00640 1 --2- 172.21.15.86:0/1568719839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0815c9a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fca0806e970 tx=0x7fc9f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a915067ea3c4d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca07400640 1 -- 172.21.15.86:0/1568719839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f40ab650 con 0x7fca080b3600 2024-03-22T04:20:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca07400640 1 -- 172.21.15.86:0/1568719839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f40b0070 con 0x7fca080b3600 2024-03-22T04:20:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca07400640 1 -- 172.21.15.86:0/1568719839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f40aba10 con 0x7fca080b3600 2024-03-22T04:20:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 -- 172.21.15.86:0/1568719839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 msgr2=0x7fca0815c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 --2- 172.21.15.86:0/1568719839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0815c9a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fca0806e970 tx=0x7fc9f401c450 comp rx=0 tx=0).stop 2024-03-22T04:20:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 -- 172.21.15.86:0/1568719839 shutdown_connections 2024-03-22T04:20:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 --2- 172.21.15.86:0/1568719839 >> v2:172.21.15.196:3300/0 conn(0x7fca0815cee0 0x7fca080b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 --2- 172.21.15.86:0/1568719839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0815c9a0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 -- 172.21.15.86:0/1568719839 >> 172.21.15.86:0/1568719839 conn(0x7fca080cc940 msgr2=0x7fca080ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 -- 172.21.15.86:0/1568719839 shutdown_connections 2024-03-22T04:20:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 -- 172.21.15.86:0/1568719839 wait complete. 2024-03-22T04:20:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.871+0000 7fca0f400640 1 Processor -- start 2024-03-22T04:20:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 -- start start 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca080b2360 0x7fca081548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0814c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca0815cee0 0x7fca08155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca0815d470 con 0x7fca080b3600 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca0815d2f0 con 0x7fca080b2360 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca080b5ad0 con 0x7fca0815cee0 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0814c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0814c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54650/0 (socket says 172.21.15.86:54650) 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 -- 172.21.15.86:0/3203052980 learned_addr learned my addr 172.21.15.86:0/3203052980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0ca00640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca080b2360 0x7fca081548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0d400640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca0815cee0 0x7fca08155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 -- 172.21.15.86:0/3203052980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca0815cee0 msgr2=0x7fca08155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca0815cee0 0x7fca08155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 -- 172.21.15.86:0/3203052980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca080b2360 msgr2=0x7fca081548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca080b2360 0x7fca081548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 -- 172.21.15.86:0/3203052980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0814c690 con 0x7fca080b3600 2024-03-22T04:20:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca07e00640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0814c150 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40ad040 tx=0x7fc9f401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca04c00640 1 -- 172.21.15.86:0/3203052980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f40bf610 con 0x7fca080b3600 2024-03-22T04:20:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca04c00640 1 -- 172.21.15.86:0/3203052980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f40bf7d0 con 0x7fca080b3600 2024-03-22T04:20:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca04c00640 1 -- 172.21.15.86:0/3203052980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f40bfbc0 con 0x7fca080b3600 2024-03-22T04:20:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca08155940 con 0x7fca080b3600 2024-03-22T04:20:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca081540e0 con 0x7fca080b3600 2024-03-22T04:20:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.875+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca08074470 con 0x7fca080b3600 2024-03-22T04:20:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.879+0000 7fca04c00640 1 -- 172.21.15.86:0/3203052980 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc9f40bd020 con 0x7fca080b3600 2024-03-22T04:20:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.879+0000 7fca04c00640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9e8067b20 0x7fc9e8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.879+0000 7fca04c00640 1 -- 172.21.15.86:0/3203052980 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7fc9f4120980 con 0x7fca080b3600 2024-03-22T04:20:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.879+0000 7fca0ca00640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9e8067b20 0x7fc9e8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:41.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.879+0000 7fca0ca00640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9e8067b20 0x7fc9e8069fe0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8005c10 tx=0x7fc9f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:41.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:41.883+0000 7fca04c00640 1 -- 172.21.15.86:0/3203052980 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc9f4016070 con 0x7fca080b3600 2024-03-22T04:20:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca0806e9f0 con 0x7fca080b3600 2024-03-22T04:20:42.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca04c00640 1 -- 172.21.15.86:0/3203052980 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fc9f40abe20 con 0x7fca080b3600 2024-03-22T04:20:42.040 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9e8067b20 msgr2=0x7fc9e8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9e8067b20 0x7fc9e8069fe0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8005c10 tx=0x7fc9f8009040 comp rx=0 tx=0).stop 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 msgr2=0x7fca0814c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0814c150 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40ad040 tx=0x7fc9f401ac90 comp rx=0 tx=0).stop 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 shutdown_connections 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc9e8067b20 0x7fc9e8069fe0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca0815cee0 0x7fca08155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca080b3600 0x7fca0814c150 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 --2- 172.21.15.86:0/3203052980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca080b2360 0x7fca081548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 >> 172.21.15.86:0/3203052980 conn(0x7fca080cc940 msgr2=0x7fca080cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:42.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 shutdown_connections 2024-03-22T04:20:42.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.039+0000 7fca0f400640 1 -- 172.21.15.86:0/3203052980 wait complete. 2024-03-22T04:20:42.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:20:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.199+0000 7ff596400640 1 Processor -- start 2024-03-22T04:20:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff596400640 1 -- start start 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff596400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 0x7ff5900b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff596400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5900b3430 0x7ff59015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff596400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff59015d710 con 0x7ff59015cee0 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff596400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5900b3890 con 0x7ff5900b3430 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff596400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff5900b2740 con 0x7ff5900b2360 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff586a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5900b3430 0x7ff59015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff586a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5900b3430 0x7ff59015c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54666/0 (socket says 172.21.15.86:54666) 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff586a00640 1 -- 172.21.15.86:0/1769603929 learned_addr learned my addr 172.21.15.86:0/1769603929 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff58f400640 1 --2- 172.21.15.86:0/1769603929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 0x7ff5900b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff586a00640 1 -- 172.21.15.86:0/1769603929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 msgr2=0x7ff5900b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff586a00640 1 --2- 172.21.15.86:0/1769603929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 0x7ff5900b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff586a00640 1 -- 172.21.15.86:0/1769603929 >> v1:172.21.15.196:6789/0 conn(0x7ff59015cee0 legacy=0x7ff59006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.203+0000 7ff586a00640 1 -- 172.21.15.86:0/1769603929 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59015fe40 con 0x7ff5900b3430 2024-03-22T04:20:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff586a00640 1 --2- 172.21.15.86:0/1769603929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5900b3430 0x7ff59015c9a0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff578009870 tx=0x7ff57801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbef55d84c1f680c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff58ea00640 1 -- 172.21.15.86:0/1769603929 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780ab650 con 0x7ff5900b3430 2024-03-22T04:20:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff58ea00640 1 -- 172.21.15.86:0/1769603929 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5780b0070 con 0x7ff5900b3430 2024-03-22T04:20:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff58ea00640 1 -- 172.21.15.86:0/1769603929 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780aba10 con 0x7ff5900b3430 2024-03-22T04:20:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 -- 172.21.15.86:0/1769603929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5900b3430 msgr2=0x7ff59015c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 --2- 172.21.15.86:0/1769603929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5900b3430 0x7ff59015c9a0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff578009870 tx=0x7ff57801c530 comp rx=0 tx=0).stop 2024-03-22T04:20:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 -- 172.21.15.86:0/1769603929 shutdown_connections 2024-03-22T04:20:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 --2- 172.21.15.86:0/1769603929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5900b3430 0x7ff59015c9a0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 --2- 172.21.15.86:0/1769603929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 0x7ff5900b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 -- 172.21.15.86:0/1769603929 >> 172.21.15.86:0/1769603929 conn(0x7ff5900cc940 msgr2=0x7ff5900ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 -- 172.21.15.86:0/1769603929 shutdown_connections 2024-03-22T04:20:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 -- 172.21.15.86:0/1769603929 wait complete. 2024-03-22T04:20:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 Processor -- start 2024-03-22T04:20:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.207+0000 7ff596400640 1 -- start start 2024-03-22T04:20:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 0x7ff59014b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5900b3430 0x7ff590155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff59015cee0 0x7ff59014d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff590160c00 con 0x7ff59015cee0 2024-03-22T04:20:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5900b5d90 con 0x7ff5900b3430 2024-03-22T04:20:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff5901571c0 con 0x7ff5900b2360 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff59015cee0 0x7ff59014d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff59015cee0 0x7ff59014d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54676/0 (socket says 172.21.15.86:54676) 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 -- 172.21.15.86:0/2376775883 learned_addr learned my addr 172.21.15.86:0/2376775883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58f400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 0x7ff59014b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff586a00640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5900b3430 0x7ff590155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 -- 172.21.15.86:0/2376775883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 msgr2=0x7ff59014b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 0x7ff59014b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 -- 172.21.15.86:0/2376775883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5900b3430 msgr2=0x7ff590155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5900b3430 0x7ff590155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 -- 172.21.15.86:0/2376775883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5901549f0 con 0x7ff59015cee0 2024-03-22T04:20:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff58fe00640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff59015cee0 0x7ff59014d2d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff58800c340 tx=0x7ff58800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff587e00640 1 -- 172.21.15.86:0/2376775883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588005220 con 0x7ff59015cee0 2024-03-22T04:20:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff587e00640 1 -- 172.21.15.86:0/2376775883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5880053c0 con 0x7ff59015cee0 2024-03-22T04:20:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff587e00640 1 -- 172.21.15.86:0/2376775883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588007270 con 0x7ff59015cee0 2024-03-22T04:20:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff590154cd0 con 0x7ff59015cee0 2024-03-22T04:20:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff590155290 con 0x7ff59015cee0 2024-03-22T04:20:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff590074470 con 0x7ff59015cee0 2024-03-22T04:20:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.211+0000 7ff587e00640 1 -- 172.21.15.86:0/2376775883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff588019020 con 0x7ff59015cee0 2024-03-22T04:20:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.215+0000 7ff587e00640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff56c067b20 0x7ff56c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.215+0000 7ff587e00640 1 -- 172.21.15.86:0/2376775883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7ff588085360 con 0x7ff59015cee0 2024-03-22T04:20:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.215+0000 7ff58f400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff56c067b20 0x7ff56c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.215+0000 7ff58f400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff56c067b20 0x7ff56c069fe0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff5800113e0 tx=0x7ff58000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.219+0000 7ff587e00640 1 -- 172.21.15.86:0/2376775883 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff588055460 con 0x7ff59015cee0 2024-03-22T04:20:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff59006e9f0 con 0x7ff59015cee0 2024-03-22T04:20:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff587e00640 1 -- 172.21.15.86:0/2376775883 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7ff588059500 con 0x7ff59015cee0 2024-03-22T04:20:42.375 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:20:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff56c067b20 msgr2=0x7ff56c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff56c067b20 0x7ff56c069fe0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff5800113e0 tx=0x7ff58000b040 comp rx=0 tx=0).stop 2024-03-22T04:20:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff59015cee0 msgr2=0x7ff59014d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff59015cee0 0x7ff59014d2d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff58800c340 tx=0x7ff58800c800 comp rx=0 tx=0).stop 2024-03-22T04:20:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 shutdown_connections 2024-03-22T04:20:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff56c067b20 0x7ff56c069fe0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5900b3430 0x7ff590155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff59015cee0 0x7ff59014d2d0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 --2- 172.21.15.86:0/2376775883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5900b2360 0x7ff59014b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 >> 172.21.15.86:0/2376775883 conn(0x7ff5900cc940 msgr2=0x7ff59015e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 shutdown_connections 2024-03-22T04:20:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.375+0000 7ff596400640 1 -- 172.21.15.86:0/2376775883 wait complete. 2024-03-22T04:20:42.396 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:20:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.531+0000 7f6d87600640 1 Processor -- start 2024-03-22T04:20:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.531+0000 7f6d87600640 1 -- start start 2024-03-22T04:20:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d87600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d800b2360 0x7f6d800b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d87600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6d8015d0b0 0x7f6d800b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d87600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d8015d710 con 0x7f6d800b3430 2024-03-22T04:20:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d87600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6d800b2740 con 0x7f6d800b2360 2024-03-22T04:20:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d87600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8015d4b0 con 0x7f6d8015d0b0 2024-03-22T04:20:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d84c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d800b2360 0x7f6d800b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d77e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6d800b3430 0x7f6d8015c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51686/0 (socket says 172.21.15.86:51686) 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d84c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d800b2360 0x7f6d800b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54682/0 (socket says 172.21.15.86:54682) 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d77e00640 1 -- 172.21.15.86:0/612968052 learned_addr learned my addr 172.21.15.86:0/612968052 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d85600640 1 --2- 172.21.15.86:0/612968052 >> v2:172.21.15.196:3300/0 conn(0x7f6d8015d0b0 0x7f6d800b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d84c00640 1 -- 172.21.15.86:0/612968052 >> v2:172.21.15.196:3300/0 conn(0x7f6d8015d0b0 msgr2=0x7f6d800b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d84c00640 1 --2- 172.21.15.86:0/612968052 >> v2:172.21.15.196:3300/0 conn(0x7f6d8015d0b0 0x7f6d800b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d84c00640 1 -- 172.21.15.86:0/612968052 >> v1:172.21.15.86:6789/0 conn(0x7f6d800b3430 legacy=0x7f6d8015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d84c00640 1 -- 172.21.15.86:0/612968052 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8015fe40 con 0x7f6d800b2360 2024-03-22T04:20:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.535+0000 7f6d7fe00640 1 -- 172.21.15.86:0/612968052 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3303192882 0 0) 0x7f6d8015d710 con 0x7f6d800b3430 2024-03-22T04:20:42.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d84c00640 1 --2- 172.21.15.86:0/612968052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800b2360 0x7f6d800b2ef0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6d8006e940 tx=0x7f6d7001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e309fe99be35843 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d7fe00640 1 -- 172.21.15.86:0/612968052 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d700b0070 con 0x7f6d800b2360 2024-03-22T04:20:42.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d7fe00640 1 -- 172.21.15.86:0/612968052 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d700ab600 con 0x7f6d800b2360 2024-03-22T04:20:42.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d7fe00640 1 -- 172.21.15.86:0/612968052 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d700ab9d0 con 0x7f6d800b2360 2024-03-22T04:20:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 -- 172.21.15.86:0/612968052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800b2360 msgr2=0x7f6d800b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 --2- 172.21.15.86:0/612968052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800b2360 0x7f6d800b2ef0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6d8006e940 tx=0x7f6d7001c480 comp rx=0 tx=0).stop 2024-03-22T04:20:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 -- 172.21.15.86:0/612968052 shutdown_connections 2024-03-22T04:20:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 --2- 172.21.15.86:0/612968052 >> v2:172.21.15.196:3300/0 conn(0x7f6d8015d0b0 0x7f6d800b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 --2- 172.21.15.86:0/612968052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800b2360 0x7f6d800b2ef0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 -- 172.21.15.86:0/612968052 >> 172.21.15.86:0/612968052 conn(0x7f6d800cc940 msgr2=0x7f6d800ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 -- 172.21.15.86:0/612968052 shutdown_connections 2024-03-22T04:20:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 -- 172.21.15.86:0/612968052 wait complete. 2024-03-22T04:20:42.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 Processor -- start 2024-03-22T04:20:42.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.539+0000 7f6d87600640 1 -- start start 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d800b2360 0x7f6d80147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d800b3430 0x7f6d8015de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d8015d0b0 0x7f6d8015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d80160c00 con 0x7f6d8015d0b0 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d800b5ad0 con 0x7f6d800b3430 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6d80150290 con 0x7f6d800b2360 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d8015d0b0 0x7f6d8015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d8015d0b0 0x7f6d8015e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54690/0 (socket says 172.21.15.86:54690) 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 -- 172.21.15.86:0/1966045423 learned_addr learned my addr 172.21.15.86:0/1966045423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d77e00640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d800b3430 0x7f6d8015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d84c00640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d800b2360 0x7f6d80147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 -- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d800b2360 msgr2=0x7f6d80147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d800b2360 0x7f6d80147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 -- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d800b3430 msgr2=0x7f6d8015de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d800b3430 0x7f6d8015de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 -- 172.21.15.86:0/1966045423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8014ce90 con 0x7f6d8015d0b0 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d84c00640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d800b2360 0x7f6d80147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d85600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d8015d0b0 0x7f6d8015e380 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6d78007e60 tx=0x7f6d7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d7d600640 1 -- 172.21.15.86:0/1966045423 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78014420 con 0x7f6d8015d0b0 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d7d600640 1 -- 172.21.15.86:0/1966045423 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d78019070 con 0x7f6d8015d0b0 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d7d600640 1 -- 172.21.15.86:0/1966045423 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78014820 con 0x7f6d8015d0b0 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8014d170 con 0x7f6d8015d0b0 2024-03-22T04:20:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8014d730 con 0x7f6d8015d0b0 2024-03-22T04:20:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d80074470 con 0x7f6d8015d0b0 2024-03-22T04:20:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.543+0000 7f6d7d600640 1 -- 172.21.15.86:0/1966045423 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6d780149c0 con 0x7f6d8015d0b0 2024-03-22T04:20:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.547+0000 7f6d7d600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d5c067b50 0x7f6d5c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.547+0000 7f6d7d600640 1 -- 172.21.15.86:0/1966045423 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7f6d780869a0 con 0x7f6d8015d0b0 2024-03-22T04:20:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.547+0000 7f6d84c00640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d5c067b50 0x7f6d5c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.547+0000 7f6d84c00640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d5c067b50 0x7f6d5c06a010 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6d700ad000 tx=0x7f6d70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.551+0000 7f6d7d600640 1 -- 172.21.15.86:0/1966045423 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6d78057060 con 0x7f6d8015d0b0 2024-03-22T04:20:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.703+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d8006e9f0 con 0x7f6d8015d0b0 2024-03-22T04:20:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.703+0000 7f6d7d600640 1 -- 172.21.15.86:0/1966045423 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7f6d78056960 con 0x7f6d8015d0b0 2024-03-22T04:20:42.704 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:20:42.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d5c067b50 msgr2=0x7f6d5c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d5c067b50 0x7f6d5c06a010 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6d700ad000 tx=0x7f6d70008040 comp rx=0 tx=0).stop 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d8015d0b0 msgr2=0x7f6d8015e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d8015d0b0 0x7f6d8015e380 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6d78007e60 tx=0x7f6d7800e4b0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 shutdown_connections 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6d5c067b50 0x7f6d5c06a010 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d800b3430 0x7f6d8015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d8015d0b0 0x7f6d8015e380 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 --2- 172.21.15.86:0/1966045423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d800b2360 0x7f6d80147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 >> 172.21.15.86:0/1966045423 conn(0x7f6d800cc940 msgr2=0x7f6d800b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 shutdown_connections 2024-03-22T04:20:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.707+0000 7f6d87600640 1 -- 172.21.15.86:0/1966045423 wait complete. 2024-03-22T04:20:42.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:20:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.859+0000 7f5e20c00640 1 Processor -- start 2024-03-22T04:20:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.859+0000 7f5e20c00640 1 -- start start 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e20c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e1c15b3f0 0x7f5e1c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e20c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5e1c0cc3e0 0x7f5e1c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e20c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e1c071920 con 0x7f5e1c0b30f0 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e20c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e1c0716d0 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e20c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5e1c06d990 con 0x7f5e1c0cc3e0 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e1c15b3f0 0x7f5e1c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e1c15b3f0 0x7f5e1c0729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54698/0 (socket says 172.21.15.86:54698) 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19e00640 1 -- 172.21.15.86:0/1472069191 learned_addr learned my addr 172.21.15.86:0/1472069191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19400640 1 --2- 172.21.15.86:0/1472069191 >> v2:172.21.15.196:3300/0 conn(0x7f5e1c0cc3e0 0x7f5e1c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19e00640 1 -- 172.21.15.86:0/1472069191 >> v2:172.21.15.196:3300/0 conn(0x7f5e1c0cc3e0 msgr2=0x7f5e1c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19e00640 1 --2- 172.21.15.86:0/1472069191 >> v2:172.21.15.196:3300/0 conn(0x7f5e1c0cc3e0 0x7f5e1c072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19e00640 1 -- 172.21.15.86:0/1472069191 >> v1:172.21.15.196:6789/0 conn(0x7f5e1c0b30f0 legacy=0x7f5e1c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19e00640 1 -- 172.21.15.86:0/1472069191 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c071c00 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.863+0000 7f5e19e00640 1 --2- 172.21.15.86:0/1472069191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c0729c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c070ed0 tx=0x7f5e0401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8492cf82f2a25866 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e18a00640 1 -- 172.21.15.86:0/1472069191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e040b0070 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e18a00640 1 -- 172.21.15.86:0/1472069191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e040ab600 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e18a00640 1 -- 172.21.15.86:0/1472069191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e040ab9d0 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- 172.21.15.86:0/1472069191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 msgr2=0x7f5e1c0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 --2- 172.21.15.86:0/1472069191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c0729c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c070ed0 tx=0x7f5e0401c450 comp rx=0 tx=0).stop 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- 172.21.15.86:0/1472069191 shutdown_connections 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 --2- 172.21.15.86:0/1472069191 >> v2:172.21.15.196:3300/0 conn(0x7f5e1c0cc3e0 0x7f5e1c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 --2- 172.21.15.86:0/1472069191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c0729c0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- 172.21.15.86:0/1472069191 >> 172.21.15.86:0/1472069191 conn(0x7f5e1c0b1df0 msgr2=0x7f5e1c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- 172.21.15.86:0/1472069191 shutdown_connections 2024-03-22T04:20:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- 172.21.15.86:0/1472069191 wait complete. 2024-03-22T04:20:42.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 Processor -- start 2024-03-22T04:20:42.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- start start 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e1c0b30f0 0x7f5e1c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e1c0cc3e0 0x7f5e1c14a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c168580 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c168400 con 0x7f5e1c0b30f0 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e20c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c168700 con 0x7f5e1c0cc3e0 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e1a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e1a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c14f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54708/0 (socket says 172.21.15.86:54708) 2024-03-22T04:20:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e1a800640 1 -- 172.21.15.86:0/3394979022 learned_addr learned my addr 172.21.15.86:0/3394979022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e19e00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e1c0b30f0 0x7f5e1c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e19400640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e1c0cc3e0 0x7f5e1c14a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e1a800640 1 -- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e1c0cc3e0 msgr2=0x7f5e1c14a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e1a800640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e1c0cc3e0 0x7f5e1c14a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e1a800640 1 -- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e1c0b30f0 msgr2=0x7f5e1c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.867+0000 7f5e1a800640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e1c0b30f0 0x7f5e1c14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e1a800640 1 -- 172.21.15.86:0/3394979022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c150850 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e19400640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e1c0cc3e0 0x7f5e1c14a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e1a800640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c14f3a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b970 tx=0x7f5e1400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e12000640 1 -- 172.21.15.86:0/3394979022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14019070 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e12000640 1 -- 172.21.15.86:0/3394979022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1400cae0 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c150b30 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e12000640 1 -- 172.21.15.86:0/3394979022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14014440 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c1510f0 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1c068650 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e12000640 1 -- 172.21.15.86:0/3394979022 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5e1401c070 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e12000640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5df8067b20 0x7f5df8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e12000640 1 -- 172.21.15.86:0/3394979022 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7f5e14085d80 con 0x7f5e1c15b3f0 2024-03-22T04:20:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.871+0000 7f5e19e00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5df8067b20 0x7f5df8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.875+0000 7f5e19e00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5df8067b20 0x7f5df8069fe0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c06f270 tx=0x7f5e04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:42.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:42.875+0000 7f5e12000640 1 -- 172.21.15.86:0/3394979022 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5e14055df0 con 0x7f5e1c15b3f0 2024-03-22T04:20:43.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.063+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e1c09ddf0 con 0x7f5e1c15b3f0 2024-03-22T04:20:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.063+0000 7f5e12000640 1 -- 172.21.15.86:0/3394979022 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5167 (secure 0 0 0) 0x7f5e14059e90 con 0x7f5e1c15b3f0 2024-03-22T04:20:43.066 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:43.066 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/70563317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":70563317},{"type":"v1","addr":"172.21.15.196:6835","nonce":70563317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8605,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/193924359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":193924359},{"type":"v1","addr":"172.21.15.196:6834","nonce":193924359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:20:40.626899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-22T04:20:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 92 2024-03-22T04:20:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5df8067b20 msgr2=0x7f5df8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5df8067b20 0x7f5df8069fe0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c06f270 tx=0x7f5e04009040 comp rx=0 tx=0).stop 2024-03-22T04:20:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 msgr2=0x7f5e1c14f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c14f3a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b970 tx=0x7f5e1400be30 comp rx=0 tx=0).stop 2024-03-22T04:20:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 shutdown_connections 2024-03-22T04:20:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5df8067b20 0x7f5df8069fe0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1c15b3f0 0x7f5e1c14f3a0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e1c0cc3e0 0x7f5e1c14a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 --2- 172.21.15.86:0/3394979022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e1c0b30f0 0x7f5e1c14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 >> 172.21.15.86:0/3394979022 conn(0x7f5e1c0b1df0 msgr2=0x7f5e1c158580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 shutdown_connections 2024-03-22T04:20:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.067+0000 7f5e20c00640 1 -- 172.21.15.86:0/3394979022 wait complete. 2024-03-22T04:20:43.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:20:43.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.219+0000 7f64d4600640 1 Processor -- start 2024-03-22T04:20:43.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d4600640 1 -- start start 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d4600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0700c0 0x7f64cc072850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64cc0704a0 0x7f64cc06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d4600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64cc162740 con 0x7f64cc06be40 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f64cc162a20 con 0x7f64cc0704a0 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d4600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64cc162d00 con 0x7f64cc0700c0 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d1200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64cc0704a0 0x7f64cc06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d2600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f64cc06be40 0x7f64cc0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51692/0 (socket says 172.21.15.86:51692) 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d2600640 1 -- 172.21.15.86:0/3109466877 learned_addr learned my addr 172.21.15.86:0/3109466877 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:43.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d1c00640 1 --2- 172.21.15.86:0/3109466877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0700c0 0x7f64cc072850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d1200640 1 -- 172.21.15.86:0/3109466877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0700c0 msgr2=0x7f64cc072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:43.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d1200640 1 --2- 172.21.15.86:0/3109466877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0700c0 0x7f64cc072850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d1200640 1 -- 172.21.15.86:0/3109466877 >> v1:172.21.15.86:6789/0 conn(0x7f64cc06be40 legacy=0x7f64cc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64d1200640 1 -- 172.21.15.86:0/3109466877 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64cc162fe0 con 0x7f64cc0704a0 2024-03-22T04:20:43.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.223+0000 7f64cbe00640 1 -- 172.21.15.86:0/3109466877 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262937604 0 0) 0x7f64cc162740 con 0x7f64cc06be40 2024-03-22T04:20:43.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d1200640 1 --2- 172.21.15.86:0/3109466877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0704a0 0x7f64cc06b900 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f64b80097c0 tx=0x7f64b801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b449287d0ce07140 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64cbe00640 1 -- 172.21.15.86:0/3109466877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80b1070 con 0x7f64cc0704a0 2024-03-22T04:20:43.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64cbe00640 1 -- 172.21.15.86:0/3109466877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b80ac410 con 0x7f64cc0704a0 2024-03-22T04:20:43.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64cbe00640 1 -- 172.21.15.86:0/3109466877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc162500 con 0x7f64cc0704a0 2024-03-22T04:20:43.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 -- 172.21.15.86:0/3109466877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0704a0 msgr2=0x7f64cc06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 --2- 172.21.15.86:0/3109466877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0704a0 0x7f64cc06b900 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f64b80097c0 tx=0x7f64b801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 -- 172.21.15.86:0/3109466877 shutdown_connections 2024-03-22T04:20:43.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 --2- 172.21.15.86:0/3109466877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0704a0 0x7f64cc06b900 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 --2- 172.21.15.86:0/3109466877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc0700c0 0x7f64cc072850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 -- 172.21.15.86:0/3109466877 >> 172.21.15.86:0/3109466877 conn(0x7f64cc0b2d00 msgr2=0x7f64cc0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:43.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 -- 172.21.15.86:0/3109466877 shutdown_connections 2024-03-22T04:20:43.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 -- 172.21.15.86:0/3109466877 wait complete. 2024-03-22T04:20:43.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 Processor -- start 2024-03-22T04:20:43.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.227+0000 7f64d4600640 1 -- start start 2024-03-22T04:20:43.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc06be40 0x7f64cc143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0700c0 0x7f64cc0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc0704a0 0x7f64cc0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc15f820 con 0x7f64cc0700c0 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc0d0b20 con 0x7f64cc0704a0 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64cc15f9a0 con 0x7f64cc06be40 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0700c0 0x7f64cc0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0700c0 0x7f64cc0c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54718/0 (socket says 172.21.15.86:54718) 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 -- 172.21.15.86:0/3992215468 learned_addr learned my addr 172.21.15.86:0/3992215468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1c00640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc06be40 0x7f64cc143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d2600640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc0704a0 0x7f64cc0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 -- 172.21.15.86:0/3992215468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc06be40 msgr2=0x7f64cc143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc06be40 0x7f64cc143f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 -- 172.21.15.86:0/3992215468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc0704a0 msgr2=0x7f64cc0c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc0704a0 0x7f64cc0c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 -- 172.21.15.86:0/3992215468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64cc144450 con 0x7f64cc0700c0 2024-03-22T04:20:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d1200640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0700c0 0x7f64cc0c8240 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f64b801a0d0 tx=0x7f64b8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64c9600640 1 -- 172.21.15.86:0/3992215468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80bb020 con 0x7f64cc0700c0 2024-03-22T04:20:43.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64c9600640 1 -- 172.21.15.86:0/3992215468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b80ace20 con 0x7f64cc0700c0 2024-03-22T04:20:43.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64cc0aedb0 con 0x7f64cc0700c0 2024-03-22T04:20:43.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64c9600640 1 -- 172.21.15.86:0/3992215468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b80b1040 con 0x7f64cc0700c0 2024-03-22T04:20:43.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64cc0af370 con 0x7f64cc0700c0 2024-03-22T04:20:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64cc068650 con 0x7f64cc0700c0 2024-03-22T04:20:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.231+0000 7f64c9600640 1 -- 172.21.15.86:0/3992215468 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f64b80be020 con 0x7f64cc0700c0 2024-03-22T04:20:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.235+0000 7f64c9600640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64ac067b20 0x7f64ac069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.235+0000 7f64c9600640 1 -- 172.21.15.86:0/3992215468 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7f64b81203c0 con 0x7f64cc0700c0 2024-03-22T04:20:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.235+0000 7f64d1c00640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64ac067b20 0x7f64ac069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.235+0000 7f64d1c00640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64ac067b20 0x7f64ac069fe0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f64c000b9d0 tx=0x7f64c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.239+0000 7f64c9600640 1 -- 172.21.15.86:0/3992215468 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f64b80c5030 con 0x7f64cc0700c0 2024-03-22T04:20:43.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.395+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64cc0ce050 con 0x7f64cc0700c0 2024-03-22T04:20:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64c9600640 1 -- 172.21.15.86:0/3992215468 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17349 (secure 0 0 0) 0x7f64b8016090 con 0x7f64cc0700c0 2024-03-22T04:20:43.399 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:43.399 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":113,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:40.626642+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:19:15.365871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-22T04:19:16.647234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.3","mappings":[{"from":2,"to":5}]},{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"16.3f","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6832/269054026":"2024-03-23T04:20:40.626610+0000","172.21.15.196:6835/2259098413":"2024-03-23T04:20:03.917951+0000","172.21.15.196:6834/269054026":"2024-03-23T04:20:40.626610+0000","172.21.15.196:6833/2259098413":"2024-03-23T04:20:03.917951+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64ac067b20 msgr2=0x7f64ac069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64ac067b20 0x7f64ac069fe0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f64c000b9d0 tx=0x7f64c0008040 comp rx=0 tx=0).stop 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0700c0 msgr2=0x7f64cc0c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0700c0 0x7f64cc0c8240 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f64b801a0d0 tx=0x7f64b8019240 comp rx=0 tx=0).stop 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 shutdown_connections 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f64ac067b20 0x7f64ac069fe0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64cc0704a0 0x7f64cc0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64cc0700c0 0x7f64cc0c8240 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 --2- 172.21.15.86:0/3992215468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64cc06be40 0x7f64cc143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 >> 172.21.15.86:0/3992215468 conn(0x7f64cc0b2d00 msgr2=0x7f64cc0b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 shutdown_connections 2024-03-22T04:20:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.399+0000 7f64d4600640 1 -- 172.21.15.86:0/3992215468 wait complete. 2024-03-22T04:20:43.420 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:20:43.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:20:43.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef26600640 1 Processor -- start 2024-03-22T04:20:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef26600640 1 -- start start 2024-03-22T04:20:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef26600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fef200d0de0 0x7fef200ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef26600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef200d11c0 con 0x7fef200d19a0 2024-03-22T04:20:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef26600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef200cece0 con 0x7fef2015fed0 2024-03-22T04:20:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef26600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fef200d0580 con 0x7fef200d0de0 2024-03-22T04:20:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fef200d19a0 0x7fef200cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51704/0 (socket says 172.21.15.86:51704) 2024-03-22T04:20:43.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1ea00640 1 -- 172.21.15.86:0/3645749103 learned_addr learned my addr 172.21.15.86:0/3645749103 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1f400640 1 --2- 172.21.15.86:0/3645749103 >> v2:172.21.15.196:3300/0 conn(0x7fef200d0de0 0x7fef200ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110013067 0 0) 0x7fef200d11c0 con 0x7fef200d19a0 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef04003530 con 0x7fef200d19a0 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3434557682 0 0) 0x7fef04003530 con 0x7fef200d19a0 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 >> v2:172.21.15.196:3300/0 conn(0x7fef200d0de0 msgr2=0x7fef200ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 --2- 172.21.15.86:0/3645749103 >> v2:172.21.15.196:3300/0 conn(0x7fef200d0de0 0x7fef200ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 >> v1:172.21.15.196:6789/0 conn(0x7fef2015fed0 legacy=0x7fef201602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef200cefc0 con 0x7fef200d19a0 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fef0c001680 con 0x7fef200d19a0 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 >> v1:172.21.15.86:6789/0 conn(0x7fef200d19a0 legacy=0x7fef200cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040039c0 0x7fef04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef04004810 0x7fef04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef040051d0 0x7fef0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef200d0580 con 0x7fef040051d0 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400c070 con 0x7fef040039c0 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef0400c270 con 0x7fef04004810 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1fe00640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef040051d0 0x7fef0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1f400640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040039c0 0x7fef04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.555+0000 7fef1ea00640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef04004810 0x7fef04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1fe00640 1 -- 172.21.15.86:0/3645749103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef04004810 msgr2=0x7fef04004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1fe00640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef04004810 0x7fef04004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1fe00640 1 -- 172.21.15.86:0/3645749103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040039c0 msgr2=0x7fef04003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1fe00640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040039c0 0x7fef04003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1fe00640 1 -- 172.21.15.86:0/3645749103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef04003530 con 0x7fef040051d0 2024-03-22T04:20:43.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1fe00640 1 -- 172.21.15.86:0/3645749103 reap_dead start 2024-03-22T04:20:43.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1fe00640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef040051d0 0x7fef0400b9d0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fef18001e70 tx=0x7fef1801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13edbadae9566a66 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18001a90 con 0x7fef040051d0 2024-03-22T04:20:43.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef18003ba0 con 0x7fef040051d0 2024-03-22T04:20:43.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef1e000640 1 -- 172.21.15.86:0/3645749103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18005020 con 0x7fef040051d0 2024-03-22T04:20:43.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef26600640 1 -- 172.21.15.86:0/3645749103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef040051d0 msgr2=0x7fef0400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef26600640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef040051d0 0x7fef0400b9d0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fef18001e70 tx=0x7fef1801c780 comp rx=0 tx=0).stop 2024-03-22T04:20:43.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef26600640 1 -- 172.21.15.86:0/3645749103 shutdown_connections 2024-03-22T04:20:43.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef26600640 1 --2- 172.21.15.86:0/3645749103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef040051d0 0x7fef0400b9d0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef26600640 1 -- 172.21.15.86:0/3645749103 >> 172.21.15.86:0/3645749103 conn(0x7fef2006f910 msgr2=0x7fef2006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:43.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef26600640 1 -- 172.21.15.86:0/3645749103 shutdown_connections 2024-03-22T04:20:43.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.559+0000 7fef26600640 1 -- 172.21.15.86:0/3645749103 wait complete. 2024-03-22T04:20:43.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 Processor -- start 2024-03-22T04:20:43.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 -- start start 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040051d0 0x7fef2014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef200c9450 0x7fef200b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b4db0 0x7fef201f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef20071100 con 0x7fef200c9450 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef200cf580 con 0x7fef040051d0 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef200d1df0 con 0x7fef200b4db0 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef200c9450 0x7fef200b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef200c9450 0x7fef200b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54744/0 (socket says 172.21.15.86:54744) 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 -- 172.21.15.86:0/1517413134 learned_addr learned my addr 172.21.15.86:0/1517413134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1f400640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040051d0 0x7fef2014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1fe00640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b4db0 0x7fef201f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 -- 172.21.15.86:0/1517413134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b4db0 msgr2=0x7fef201f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b4db0 0x7fef201f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 -- 172.21.15.86:0/1517413134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040051d0 msgr2=0x7fef2014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040051d0 0x7fef2014aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 -- 172.21.15.86:0/1517413134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef200caf40 con 0x7fef200c9450 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef1ea00640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef200c9450 0x7fef200b7400 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fef0c007290 tx=0x7fef0c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef17400640 1 -- 172.21.15.86:0/1517413134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00f9a0 con 0x7fef200c9450 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef17400640 1 -- 172.21.15.86:0/1517413134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0c018070 con 0x7fef200c9450 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef200cb220 con 0x7fef200c9450 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef17400640 1 -- 172.21.15.86:0/1517413134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00fdc0 con 0x7fef200c9450 2024-03-22T04:20:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.563+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef200cb7e0 con 0x7fef200c9450 2024-03-22T04:20:43.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.567+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef20068650 con 0x7fef200c9450 2024-03-22T04:20:43.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.567+0000 7fef17400640 1 -- 172.21.15.86:0/1517413134 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fef0c01d460 con 0x7fef200c9450 2024-03-22T04:20:43.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.567+0000 7fef17400640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef00067b20 0x7fef00069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.567+0000 7fef17400640 1 -- 172.21.15.86:0/1517413134 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7fef0c086240 con 0x7fef200c9450 2024-03-22T04:20:43.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.567+0000 7fef1f400640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef00067b20 0x7fef00069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.567+0000 7fef1f400640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef00067b20 0x7fef00069fe0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fef10009aa0 tx=0x7fef10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.571+0000 7fef17400640 1 -- 172.21.15.86:0/1517413134 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fef0c056990 con 0x7fef200c9450 2024-03-22T04:20:43.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.759+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef200ce740 con 0x7fef200c9450 2024-03-22T04:20:43.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.759+0000 7fef17400640 1 -- 172.21.15.86:0/1517413134 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5167 (secure 0 0 0) 0x7fef0c056290 con 0x7fef200c9450 2024-03-22T04:20:43.762 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:43.762 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/70563317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":70563317},{"type":"v1","addr":"172.21.15.196:6835","nonce":70563317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8605,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/193924359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":193924359},{"type":"v1","addr":"172.21.15.196:6834","nonce":193924359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:20:40.626899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-22T04:20:43.762 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 92 2024-03-22T04:20:43.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef00067b20 msgr2=0x7fef00069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef00067b20 0x7fef00069fe0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fef10009aa0 tx=0x7fef10008040 comp rx=0 tx=0).stop 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef200c9450 msgr2=0x7fef200b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef200c9450 0x7fef200b7400 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fef0c007290 tx=0x7fef0c007750 comp rx=0 tx=0).stop 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 shutdown_connections 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef00067b20 0x7fef00069fe0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b4db0 0x7fef201f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef200c9450 0x7fef200b7400 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 --2- 172.21.15.86:0/1517413134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef040051d0 0x7fef2014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 >> 172.21.15.86:0/1517413134 conn(0x7fef2006f910 msgr2=0x7fef200cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 shutdown_connections 2024-03-22T04:20:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.763+0000 7fef26600640 1 -- 172.21.15.86:0/1517413134 wait complete. 2024-03-22T04:20:43.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:20:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53e00640 1 Processor -- start 2024-03-22T04:20:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53e00640 1 -- start start 2024-03-22T04:20:43.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5415f860 0x7fdf540ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf5409b180 0x7fdf540b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf54073cf0 con 0x7fdf540cdb00 2024-03-22T04:20:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdf540ce280 con 0x7fdf5409b180 2024-03-22T04:20:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf540ce560 con 0x7fdf5415f860 2024-03-22T04:20:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf5409b180 0x7fdf540b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf52000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdf540cdb00 0x7fdf5409aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51718/0 (socket says 172.21.15.86:51718) 2024-03-22T04:20:43.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf52000640 1 -- 172.21.15.86:0/662625450 learned_addr learned my addr 172.21.15.86:0/662625450 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf52a00640 1 --2- 172.21.15.86:0/662625450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5415f860 0x7fdf540ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53400640 1 -- 172.21.15.86:0/662625450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5415f860 msgr2=0x7fdf540ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53400640 1 --2- 172.21.15.86:0/662625450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5415f860 0x7fdf540ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53400640 1 -- 172.21.15.86:0/662625450 >> v1:172.21.15.86:6789/0 conn(0x7fdf540cdb00 legacy=0x7fdf5409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf53400640 1 -- 172.21.15.86:0/662625450 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf540ce840 con 0x7fdf5409b180 2024-03-22T04:20:43.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.919+0000 7fdf51600640 1 -- 172.21.15.86:0/662625450 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370914163 0 0) 0x7fdf54073cf0 con 0x7fdf540cdb00 2024-03-22T04:20:43.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53400640 1 --2- 172.21.15.86:0/662625450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5409b180 0x7fdf540b21d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0097c0 tx=0x7fdf4c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1005b0178ac13f61 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf51600640 1 -- 172.21.15.86:0/662625450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0a7070 con 0x7fdf5409b180 2024-03-22T04:20:43.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf51600640 1 -- 172.21.15.86:0/662625450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0b0070 con 0x7fdf5409b180 2024-03-22T04:20:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf51600640 1 -- 172.21.15.86:0/662625450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0ab890 con 0x7fdf5409b180 2024-03-22T04:20:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53e00640 1 -- 172.21.15.86:0/662625450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5409b180 msgr2=0x7fdf540b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53e00640 1 --2- 172.21.15.86:0/662625450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5409b180 0x7fdf540b21d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0097c0 tx=0x7fdf4c01c470 comp rx=0 tx=0).stop 2024-03-22T04:20:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53e00640 1 -- 172.21.15.86:0/662625450 shutdown_connections 2024-03-22T04:20:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53e00640 1 --2- 172.21.15.86:0/662625450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5409b180 0x7fdf540b21d0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53e00640 1 --2- 172.21.15.86:0/662625450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf5415f860 0x7fdf540ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53e00640 1 -- 172.21.15.86:0/662625450 >> 172.21.15.86:0/662625450 conn(0x7fdf540b3d60 msgr2=0x7fdf540b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:43.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53e00640 1 -- 172.21.15.86:0/662625450 shutdown_connections 2024-03-22T04:20:43.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.923+0000 7fdf53e00640 1 -- 172.21.15.86:0/662625450 wait complete. 2024-03-22T04:20:43.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 Processor -- start 2024-03-22T04:20:43.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 -- start start 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5409b180 0x7fdf541596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf540cdb00 0x7fdf5409e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5415f860 0x7fdf540ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5415fdc0 con 0x7fdf5415f860 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5415fc40 con 0x7fdf5409b180 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf540b2f20 con 0x7fdf540cdb00 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5415f860 0x7fdf540ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5415f860 0x7fdf540ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54762/0 (socket says 172.21.15.86:54762) 2024-03-22T04:20:43.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 -- 172.21.15.86:0/1657125687 learned_addr learned my addr 172.21.15.86:0/1657125687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf52a00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5409b180 0x7fdf541596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf52000640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf540cdb00 0x7fdf5409e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 -- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf540cdb00 msgr2=0x7fdf5409e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf540cdb00 0x7fdf5409e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 -- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5409b180 msgr2=0x7fdf541596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5409b180 0x7fdf541596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 -- 172.21.15.86:0/1657125687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf540cc160 con 0x7fdf5415f860 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf52a00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5409b180 0x7fdf541596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf52000640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf540cdb00 0x7fdf5409e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53400640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5415f860 0x7fdf540ca930 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0ad040 tx=0x7fdf4c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf4aa00640 1 -- 172.21.15.86:0/1657125687 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0bf610 con 0x7fdf5415f860 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf4aa00640 1 -- 172.21.15.86:0/1657125687 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0bf7b0 con 0x7fdf5415f860 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf4aa00640 1 -- 172.21.15.86:0/1657125687 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0bfba0 con 0x7fdf5415f860 2024-03-22T04:20:43.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5409ec80 con 0x7fdf5415f860 2024-03-22T04:20:43.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5409f240 con 0x7fdf5415f860 2024-03-22T04:20:43.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.927+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf54074470 con 0x7fdf5415f860 2024-03-22T04:20:43.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.931+0000 7fdf4aa00640 1 -- 172.21.15.86:0/1657125687 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdf4c0bd020 con 0x7fdf5415f860 2024-03-22T04:20:43.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.931+0000 7fdf4aa00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf34067b20 0x7fdf34069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:43.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.931+0000 7fdf4aa00640 1 -- 172.21.15.86:0/1657125687 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7fdf4c120430 con 0x7fdf5415f860 2024-03-22T04:20:43.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.931+0000 7fdf52a00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf34067b20 0x7fdf34069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:43.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.931+0000 7fdf52a00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf34067b20 0x7fdf34069fe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdf44003fd0 tx=0x7fdf44003a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:43.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:43.935+0000 7fdf4aa00640 1 -- 172.21.15.86:0/1657125687 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdf4c0c6070 con 0x7fdf5415f860 2024-03-22T04:20:44.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.123+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf540b08b0 con 0x7fdf5415f860 2024-03-22T04:20:44.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.123+0000 7fdf4aa00640 1 -- 172.21.15.86:0/1657125687 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5167 (secure 0 0 0) 0x7fdf4c016090 con 0x7fdf5415f860 2024-03-22T04:20:44.126 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:44.126 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/70563317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":70563317},{"type":"v1","addr":"172.21.15.196:6835","nonce":70563317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8605,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/193924359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":193924359},{"type":"v1","addr":"172.21.15.196:6834","nonce":193924359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:19:17.684626+0000","modified":"2024-03-22T04:20:40.626899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-22T04:20:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 92 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf34067b20 msgr2=0x7fdf34069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf34067b20 0x7fdf34069fe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdf44003fd0 tx=0x7fdf44003a00 comp rx=0 tx=0).stop 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5415f860 msgr2=0x7fdf540ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5415f860 0x7fdf540ca930 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0ad040 tx=0x7fdf4c01acb0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 shutdown_connections 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf34067b20 0x7fdf34069fe0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5415f860 0x7fdf540ca930 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf540cdb00 0x7fdf5409e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 --2- 172.21.15.86:0/1657125687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf5409b180 0x7fdf541596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 >> 172.21.15.86:0/1657125687 conn(0x7fdf540b3d60 msgr2=0x7fdf5409aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 shutdown_connections 2024-03-22T04:20:44.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.127+0000 7fdf53e00640 1 -- 172.21.15.86:0/1657125687 wait complete. 2024-03-22T04:20:44.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:20:44.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.279+0000 7fab57800640 1 Processor -- start 2024-03-22T04:20:44.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.279+0000 7fab57800640 1 -- start start 2024-03-22T04:20:44.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab57800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab5006df90 0x7fab5006e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab57800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5015c550 con 0x7fab5015bc80 2024-03-22T04:20:44.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab57800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5006e8b0 con 0x7fab5015ac10 2024-03-22T04:20:44.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab57800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fab5015cb60 con 0x7fab5006df90 2024-03-22T04:20:44.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab54e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab5006df90 0x7fab5006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab55800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fab5015bc80 0x7fab5015cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51732/0 (socket says 172.21.15.86:51732) 2024-03-22T04:20:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab55800640 1 -- 172.21.15.86:0/3564275590 learned_addr learned my addr 172.21.15.86:0/3564275590 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab4f400640 1 -- 172.21.15.86:0/3564275590 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969468270 0 0) 0x7fab5015c550 con 0x7fab5015bc80 2024-03-22T04:20:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab4f400640 1 -- 172.21.15.86:0/3564275590 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab34003450 con 0x7fab5015bc80 2024-03-22T04:20:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab54e00640 1 -- 172.21.15.86:0/3564275590 >> v1:172.21.15.196:6789/0 conn(0x7fab5015ac10 legacy=0x7fab500b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab54e00640 1 -- 172.21.15.86:0/3564275590 >> v1:172.21.15.86:6789/0 conn(0x7fab5015bc80 legacy=0x7fab5015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.283+0000 7fab54e00640 1 -- 172.21.15.86:0/3564275590 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab500b4750 con 0x7fab5006df90 2024-03-22T04:20:44.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab54e00640 1 --2- 172.21.15.86:0/3564275590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab5006e370 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fab400097c0 tx=0x7fab4001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257b09a34b7fdbdc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab4f400640 1 -- 172.21.15.86:0/3564275590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400b1070 con 0x7fab5006df90 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab4f400640 1 -- 172.21.15.86:0/3564275590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab400ac410 con 0x7fab5006df90 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab4f400640 1 -- 172.21.15.86:0/3564275590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400ac7e0 con 0x7fab5006df90 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 -- 172.21.15.86:0/3564275590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 msgr2=0x7fab5006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 --2- 172.21.15.86:0/3564275590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab5006e370 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fab400097c0 tx=0x7fab4001c430 comp rx=0 tx=0).stop 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 -- 172.21.15.86:0/3564275590 shutdown_connections 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 --2- 172.21.15.86:0/3564275590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab5006e370 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 -- 172.21.15.86:0/3564275590 >> 172.21.15.86:0/3564275590 conn(0x7fab500b1df0 msgr2=0x7fab500b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 -- 172.21.15.86:0/3564275590 shutdown_connections 2024-03-22T04:20:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 -- 172.21.15.86:0/3564275590 wait complete. 2024-03-22T04:20:44.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 Processor -- start 2024-03-22T04:20:44.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 -- start start 2024-03-22T04:20:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.287+0000 7fab57800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab501518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab57800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab5015ac10 0x7fab500ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab57800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab5015bc80 0x7fab500aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab57800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab5015da90 con 0x7fab5006df90 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab57800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab5015d910 con 0x7fab5015ac10 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab57800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab5015dc10 con 0x7fab5015bc80 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab501518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab501518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54774/0 (socket says 172.21.15.86:54774) 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 -- 172.21.15.86:0/3187202881 learned_addr learned my addr 172.21.15.86:0/3187202881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab4fe00640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab5015ac10 0x7fab500ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab55800640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab5015bc80 0x7fab500aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 -- 172.21.15.86:0/3187202881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab5015bc80 msgr2=0x7fab500aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab5015bc80 0x7fab500aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 -- 172.21.15.86:0/3187202881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab5015ac10 msgr2=0x7fab500ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab5015ac10 0x7fab500ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 -- 172.21.15.86:0/3187202881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab50151e30 con 0x7fab5006df90 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab54e00640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab501518f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fab400af040 tx=0x7fab400191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab4cc00640 1 -- 172.21.15.86:0/3187202881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400b9070 con 0x7fab5006df90 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab4cc00640 1 -- 172.21.15.86:0/3187202881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab400acd90 con 0x7fab5006df90 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab4cc00640 1 -- 172.21.15.86:0/3187202881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400b1040 con 0x7fab5006df90 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab500ccb40 con 0x7fab5006df90 2024-03-22T04:20:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab500cd100 con 0x7fab5006df90 2024-03-22T04:20:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab50068650 con 0x7fab5006df90 2024-03-22T04:20:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab4cc00640 1 -- 172.21.15.86:0/3187202881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fab400bc070 con 0x7fab5006df90 2024-03-22T04:20:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.291+0000 7fab4cc00640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab30067b50 0x7fab3006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.295+0000 7fab4cc00640 1 -- 172.21.15.86:0/3187202881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7fab4011fdf0 con 0x7fab5006df90 2024-03-22T04:20:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.295+0000 7fab4fe00640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab30067b50 0x7fab3006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.295+0000 7fab4fe00640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab30067b50 0x7fab3006a010 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fab50072f60 tx=0x7fab3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:44.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.295+0000 7fab4cc00640 1 -- 172.21.15.86:0/3187202881 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fab40016050 con 0x7fab5006df90 2024-03-22T04:20:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.451+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab5006ed60 con 0x7fab5006df90 2024-03-22T04:20:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab4cc00640 1 -- 172.21.15.86:0/3187202881 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17349 (secure 0 0 0) 0x7fab400f0540 con 0x7fab5006df90 2024-03-22T04:20:44.455 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:44.455 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":113,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:40.626642+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:19:15.365871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-22T04:19:16.647234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.3","mappings":[{"from":2,"to":5}]},{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"16.3f","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6832/269054026":"2024-03-23T04:20:40.626610+0000","172.21.15.196:6835/2259098413":"2024-03-23T04:20:03.917951+0000","172.21.15.196:6834/269054026":"2024-03-23T04:20:40.626610+0000","172.21.15.196:6833/2259098413":"2024-03-23T04:20:03.917951+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab30067b50 msgr2=0x7fab3006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab30067b50 0x7fab3006a010 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fab50072f60 tx=0x7fab3c009040 comp rx=0 tx=0).stop 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 msgr2=0x7fab501518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab501518f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fab400af040 tx=0x7fab400191d0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 shutdown_connections 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab30067b50 0x7fab3006a010 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab5015ac10 0x7fab500ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab5015bc80 0x7fab500aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 --2- 172.21.15.86:0/3187202881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab5006df90 0x7fab501518f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 >> 172.21.15.86:0/3187202881 conn(0x7fab500b1df0 msgr2=0x7fab50159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 shutdown_connections 2024-03-22T04:20:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.455+0000 7fab57800640 1 -- 172.21.15.86:0/3187202881 wait complete. 2024-03-22T04:20:44.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:20:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca9800640 1 Processor -- start 2024-03-22T04:20:44.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca9800640 1 -- start start 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca9800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 0x7f9ca415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ca40ad710 0x7f9ca4142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca9800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca415c7c0 con 0x7f9ca406ef40 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca9800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca406dea0 con 0x7f9ca40ad710 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca9800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca406e180 con 0x7f9ca40ad330 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ca40ad710 0x7f9ca4142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ca40ad710 0x7f9ca4142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54780/0 (socket says 172.21.15.86:54780) 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca3400640 1 -- 172.21.15.86:0/3787869848 learned_addr learned my addr 172.21.15.86:0/3787869848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca2a00640 1 --2- 172.21.15.86:0/3787869848 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 0x7f9ca415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca3400640 1 -- 172.21.15.86:0/3787869848 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 msgr2=0x7f9ca415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:44.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca3400640 1 --2- 172.21.15.86:0/3787869848 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 0x7f9ca415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca3400640 1 -- 172.21.15.86:0/3787869848 >> v1:172.21.15.196:6789/0 conn(0x7f9ca406ef40 legacy=0x7f9ca413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:44.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.611+0000 7f9ca3400640 1 -- 172.21.15.86:0/3787869848 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca406e460 con 0x7f9ca40ad710 2024-03-22T04:20:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca3400640 1 --2- 172.21.15.86:0/3787869848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca40ad710 0x7f9ca4142690 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009e50 tx=0x7f9c9c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8dcbba063d6fae server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca2000640 1 -- 172.21.15.86:0/3787869848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0a5840 con 0x7f9ca40ad710 2024-03-22T04:20:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca2000640 1 -- 172.21.15.86:0/3787869848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c0a59e0 con 0x7f9ca40ad710 2024-03-22T04:20:44.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca2000640 1 -- 172.21.15.86:0/3787869848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0a5db0 con 0x7f9ca40ad710 2024-03-22T04:20:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca9800640 1 -- 172.21.15.86:0/3787869848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca40ad710 msgr2=0x7f9ca4142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca9800640 1 --2- 172.21.15.86:0/3787869848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca40ad710 0x7f9ca4142690 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009e50 tx=0x7f9c9c01c9c0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca9800640 1 -- 172.21.15.86:0/3787869848 shutdown_connections 2024-03-22T04:20:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca9800640 1 --2- 172.21.15.86:0/3787869848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca40ad710 0x7f9ca4142690 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca9800640 1 --2- 172.21.15.86:0/3787869848 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 0x7f9ca415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca9800640 1 -- 172.21.15.86:0/3787869848 >> 172.21.15.86:0/3787869848 conn(0x7f9ca40b1df0 msgr2=0x7f9ca40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca9800640 1 -- 172.21.15.86:0/3787869848 shutdown_connections 2024-03-22T04:20:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.615+0000 7f9ca9800640 1 -- 172.21.15.86:0/3787869848 wait complete. 2024-03-22T04:20:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 Processor -- start 2024-03-22T04:20:44.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 -- start start 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca406ef40 0x7f9ca4152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 0x7f9ca4099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca40ad710 0x7f9ca415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41690e0 con 0x7f9ca406ef40 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca4168f60 con 0x7f9ca40ad710 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca4169260 con 0x7f9ca40ad330 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca406ef40 0x7f9ca4152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca406ef40 0x7f9ca4152f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54796/0 (socket says 172.21.15.86:54796) 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 -- 172.21.15.86:0/1950925012 learned_addr learned my addr 172.21.15.86:0/1950925012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9c9a000640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 0x7f9ca4099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca3400640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca40ad710 0x7f9ca415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 -- 172.21.15.86:0/1950925012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 msgr2=0x7f9ca4099940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 0x7f9ca4099940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 -- 172.21.15.86:0/1950925012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca40ad710 msgr2=0x7f9ca415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca40ad710 0x7f9ca415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 -- 172.21.15.86:0/1950925012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca4144c80 con 0x7f9ca406ef40 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca2a00640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca406ef40 0x7f9ca4152f00 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400d5f0 tx=0x7f9c9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9c9b400640 1 -- 172.21.15.86:0/1950925012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940125f0 con 0x7f9ca406ef40 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9c9b400640 1 -- 172.21.15.86:0/1950925012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c94019070 con 0x7f9ca406ef40 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca4144f60 con 0x7f9ca406ef40 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9c9b400640 1 -- 172.21.15.86:0/1950925012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940129b0 con 0x7f9ca406ef40 2024-03-22T04:20:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.619+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca4145520 con 0x7f9ca406ef40 2024-03-22T04:20:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.623+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca4068650 con 0x7f9ca406ef40 2024-03-22T04:20:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.623+0000 7f9c9b400640 1 -- 172.21.15.86:0/1950925012 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9c94012b50 con 0x7f9ca406ef40 2024-03-22T04:20:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.623+0000 7f9c9b400640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c80067b20 0x7f9c80069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.623+0000 7f9c9b400640 1 -- 172.21.15.86:0/1950925012 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7f9c94086570 con 0x7f9ca406ef40 2024-03-22T04:20:44.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.623+0000 7f9c9a000640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c80067b20 0x7f9c80069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:44.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.623+0000 7f9c9a000640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c80067b20 0x7f9c80069fe0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4153440 tx=0x7f9c8c009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:44.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.627+0000 7f9c9b400640 1 -- 172.21.15.86:0/1950925012 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9c94056550 con 0x7f9ca406ef40 2024-03-22T04:20:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:44.815+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9ca4159b00 con 0x7f9ca406ef40 2024-03-22T04:20:45.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9c9b400640 1 -- 172.21.15.86:0/1950925012 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f9c9405a5f0 con 0x7f9ca406ef40 2024-03-22T04:20:45.103 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c80067b20 msgr2=0x7f9c80069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c80067b20 0x7f9c80069fe0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4153440 tx=0x7f9c8c009060 comp rx=0 tx=0).stop 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca406ef40 msgr2=0x7f9ca4152f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca406ef40 0x7f9ca4152f00 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400d5f0 tx=0x7f9c9400dab0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 shutdown_connections 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c80067b20 0x7f9c80069fe0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca40ad710 0x7f9ca415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca40ad330 0x7f9ca4099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 --2- 172.21.15.86:0/1950925012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ca406ef40 0x7f9ca4152f00 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 >> 172.21.15.86:0/1950925012 conn(0x7f9ca40b1df0 msgr2=0x7f9ca40b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 shutdown_connections 2024-03-22T04:20:45.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.103+0000 7f9ca9800640 1 -- 172.21.15.86:0/1950925012 wait complete. 2024-03-22T04:20:45.124 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T04:20:45.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.259+0000 7f4eb7e00640 1 Processor -- start 2024-03-22T04:20:45.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.259+0000 7f4eb7e00640 1 -- start start 2024-03-22T04:20:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4eb006e7e0 0x7f4eb006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072b00 0x7f4eb00b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb7e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb015c770 con 0x7f4eb0072650 2024-03-22T04:20:45.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb7e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb0071350 con 0x7f4eb006e7e0 2024-03-22T04:20:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb7e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb006f100 con 0x7f4eb0072b00 2024-03-22T04:20:45.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4eb006e7e0 0x7f4eb006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:45.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb4a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f4eb0072650 0x7f4eb00b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:50816/0 (socket says 172.21.15.86:50816) 2024-03-22T04:20:45.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4eb006e7e0 0x7f4eb006ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:56146/0 (socket says 172.21.15.86:56146) 2024-03-22T04:20:45.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb4a00640 1 -- 172.21.15.86:0/496845236 learned_addr learned my addr 172.21.15.86:0/496845236 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:45.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb5e00640 1 --2- 172.21.15.86:0/496845236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072b00 0x7f4eb00b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:45.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb5400640 1 -- 172.21.15.86:0/496845236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072b00 msgr2=0x7f4eb00b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:45.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb5400640 1 --2- 172.21.15.86:0/496845236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072b00 0x7f4eb00b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb5400640 1 -- 172.21.15.86:0/496845236 >> v1:172.21.15.196:6789/0 conn(0x7f4eb0072650 legacy=0x7f4eb00b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:45.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.263+0000 7f4eb5400640 1 -- 172.21.15.86:0/496845236 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb0073150 con 0x7f4eb006e7e0 2024-03-22T04:20:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb5400640 1 --2- 172.21.15.86:0/496845236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb006e7e0 0x7f4eb006ebc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009d90 tx=0x7f4ea401c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2f63f87bde703d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eafe00640 1 -- 172.21.15.86:0/496845236 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40b2070 con 0x7f4eb006e7e0 2024-03-22T04:20:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eafe00640 1 -- 172.21.15.86:0/496845236 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea40a5770 con 0x7f4eb006e7e0 2024-03-22T04:20:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eafe00640 1 -- 172.21.15.86:0/496845236 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40a5b40 con 0x7f4eb006e7e0 2024-03-22T04:20:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- 172.21.15.86:0/496845236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb006e7e0 msgr2=0x7f4eb006ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/496845236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb006e7e0 0x7f4eb006ebc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009d90 tx=0x7f4ea401c770 comp rx=0 tx=0).stop 2024-03-22T04:20:45.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- 172.21.15.86:0/496845236 shutdown_connections 2024-03-22T04:20:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/496845236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072b00 0x7f4eb00b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/496845236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb006e7e0 0x7f4eb006ebc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- 172.21.15.86:0/496845236 >> 172.21.15.86:0/496845236 conn(0x7f4eb00cc940 msgr2=0x7f4eb00ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- 172.21.15.86:0/496845236 shutdown_connections 2024-03-22T04:20:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- 172.21.15.86:0/496845236 wait complete. 2024-03-22T04:20:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 Processor -- start 2024-03-22T04:20:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- start start 2024-03-22T04:20:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb006e7e0 0x7f4eb01429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072650 0x7f4eb014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb0072b00 0x7f4eb014edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0072080 con 0x7f4eb006e7e0 2024-03-22T04:20:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb00ce930 con 0x7f4eb0072b00 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.267+0000 7f4eb7e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb014c3c0 con 0x7f4eb0072650 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb006e7e0 0x7f4eb01429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb006e7e0 0x7f4eb01429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54810/0 (socket says 172.21.15.86:54810) 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 -- 172.21.15.86:0/1778974219 learned_addr learned my addr 172.21.15.86:0/1778974219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072650 0x7f4eb014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5e00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb0072b00 0x7f4eb014edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 -- 172.21.15.86:0/1778974219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072650 msgr2=0x7f4eb014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072650 0x7f4eb014be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 -- 172.21.15.86:0/1778974219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb0072b00 msgr2=0x7f4eb014edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb0072b00 0x7f4eb014edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 -- 172.21.15.86:0/1778974219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb014dd30 con 0x7f4eb006e7e0 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb5400640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb006e7e0 0x7f4eb01429e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f4ea401ae20 tx=0x7f4ea4019510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4ead600640 1 -- 172.21.15.86:0/1778974219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40bd020 con 0x7f4eb006e7e0 2024-03-22T04:20:45.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4ead600640 1 -- 172.21.15.86:0/1778974219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea40b8470 con 0x7f4eb006e7e0 2024-03-22T04:20:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb014e010 con 0x7f4eb006e7e0 2024-03-22T04:20:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4ead600640 1 -- 172.21.15.86:0/1778974219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40b2040 con 0x7f4eb006e7e0 2024-03-22T04:20:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb014e5d0 con 0x7f4eb006e7e0 2024-03-22T04:20:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb0074470 con 0x7f4eb006e7e0 2024-03-22T04:20:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4ead600640 1 -- 172.21.15.86:0/1778974219 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4ea40c0020 con 0x7f4eb006e7e0 2024-03-22T04:20:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.271+0000 7f4ead600640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e90067b20 0x7f4e90069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.275+0000 7f4ead600640 1 -- 172.21.15.86:0/1778974219 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7f4ea4120d60 con 0x7f4eb006e7e0 2024-03-22T04:20:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.275+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e90067b20 0x7f4e90069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:45.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.275+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e90067b20 0x7f4e90069fe0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4eb015eb20 tx=0x7f4e9c007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:45.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.275+0000 7f4ead600640 1 -- 172.21.15.86:0/1778974219 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4ea40c7030 con 0x7f4eb006e7e0 2024-03-22T04:20:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:45.467+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4eb0072200 con 0x7f4eb006e7e0 2024-03-22T04:20:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.107+0000 7f4ead600640 1 -- 172.21.15.86:0/1778974219 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) ==== 104+0+0 (secure 0 0 0) 0x7f4ea4016480 con 0x7f4eb006e7e0 2024-03-22T04:20:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e90067b20 msgr2=0x7f4e90069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e90067b20 0x7f4e90069fe0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4eb015eb20 tx=0x7f4e9c007e30 comp rx=0 tx=0).stop 2024-03-22T04:20:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb006e7e0 msgr2=0x7f4eb01429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb006e7e0 0x7f4eb01429e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f4ea401ae20 tx=0x7f4ea4019510 comp rx=0 tx=0).stop 2024-03-22T04:20:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 shutdown_connections 2024-03-22T04:20:46.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4e90067b20 0x7f4e90069fe0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4eb0072b00 0x7f4eb014edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4eb0072650 0x7f4eb014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 --2- 172.21.15.86:0/1778974219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb006e7e0 0x7f4eb01429e0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 >> 172.21.15.86:0/1778974219 conn(0x7f4eb00cc940 msgr2=0x7f4eb01593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 shutdown_connections 2024-03-22T04:20:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.111+0000 7f4eb7e00640 1 -- 172.21.15.86:0/1778974219 wait complete. 2024-03-22T04:20:46.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T04:20:46.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f03a0e00640 1 Processor -- start 2024-03-22T04:20:46.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f03a0e00640 1 -- start start 2024-03-22T04:20:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f03a0e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f039c15e5a0 0x7f039c0d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f03a0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f039c0cf5e0 0x7f039c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f03a0e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c15e980 con 0x7f039c0d4600 2024-03-22T04:20:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f03a0e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f039c0c7cf0 con 0x7f039c0cf5e0 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f03a0e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f039c0d4ab0 con 0x7f039c15e5a0 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f039be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f039c0cf5e0 0x7f039c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f039be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f039c0cf5e0 0x7f039c0c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39440/0 (socket says 172.21.15.86:39440) 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f039be00640 1 -- 172.21.15.86:0/1511595182 learned_addr learned my addr 172.21.15.86:0/1511595182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.279+0000 7f0392a00640 1 --1- 172.21.15.86:0/1511595182 >> v1:172.21.15.86:6789/0 conn(0x7f039c0d4600 0x7f039c0ceed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37688/0 (socket says 172.21.15.86:37688) 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039b400640 1 --2- 172.21.15.86:0/1511595182 >> v2:172.21.15.196:3300/0 conn(0x7f039c15e5a0 0x7f039c0d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039be00640 1 -- 172.21.15.86:0/1511595182 >> v2:172.21.15.196:3300/0 conn(0x7f039c15e5a0 msgr2=0x7f039c0d40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039be00640 1 --2- 172.21.15.86:0/1511595182 >> v2:172.21.15.196:3300/0 conn(0x7f039c15e5a0 0x7f039c0d40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039be00640 1 -- 172.21.15.86:0/1511595182 >> v1:172.21.15.86:6789/0 conn(0x7f039c0d4600 legacy=0x7f039c0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039be00640 1 -- 172.21.15.86:0/1511595182 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0cf9c0 con 0x7f039c0cf5e0 2024-03-22T04:20:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039be00640 1 --2- 172.21.15.86:0/1511595182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c0cf5e0 0x7f039c0c8740 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f039c0d1c00 tx=0x7f039401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a455e27d73d753a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039aa00640 1 -- 172.21.15.86:0/1511595182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940a57f0 con 0x7f039c0cf5e0 2024-03-22T04:20:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039aa00640 1 -- 172.21.15.86:0/1511595182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03940a5990 con 0x7f039c0cf5e0 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f039aa00640 1 -- 172.21.15.86:0/1511595182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940a5d60 con 0x7f039c0cf5e0 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f03a0e00640 1 -- 172.21.15.86:0/1511595182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c0cf5e0 msgr2=0x7f039c0c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f03a0e00640 1 --2- 172.21.15.86:0/1511595182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c0cf5e0 0x7f039c0c8740 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f039c0d1c00 tx=0x7f039401c930 comp rx=0 tx=0).stop 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f03a0e00640 1 -- 172.21.15.86:0/1511595182 shutdown_connections 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f03a0e00640 1 --2- 172.21.15.86:0/1511595182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c0cf5e0 0x7f039c0c8740 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f03a0e00640 1 --2- 172.21.15.86:0/1511595182 >> v2:172.21.15.196:3300/0 conn(0x7f039c15e5a0 0x7f039c0d40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f03a0e00640 1 -- 172.21.15.86:0/1511595182 >> 172.21.15.86:0/1511595182 conn(0x7f039c08c000 msgr2=0x7f039c09d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.283+0000 7f03a0e00640 1 -- 172.21.15.86:0/1511595182 shutdown_connections 2024-03-22T04:20:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 -- 172.21.15.86:0/1511595182 wait complete. 2024-03-22T04:20:46.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 Processor -- start 2024-03-22T04:20:46.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 -- start start 2024-03-22T04:20:46.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f039c0cf5e0 0x7f039c0b8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f039c0d4600 0x7f039c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15e5a0 0x7f039c0b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0c9490 con 0x7f039c15e5a0 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0c8220 con 0x7f039c0cf5e0 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f03a0e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f039c0c9610 con 0x7f039c0d4600 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15e5a0 0x7f039c0b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15e5a0 0x7f039c0b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39442/0 (socket says 172.21.15.86:39442) 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f039c0cf5e0 0x7f039c0b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 -- 172.21.15.86:0/503225513 learned_addr learned my addr 172.21.15.86:0/503225513 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f0392a00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f039c0d4600 0x7f039c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 -- 172.21.15.86:0/503225513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f039c0d4600 msgr2=0x7f039c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f039c0d4600 0x7f039c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 -- 172.21.15.86:0/503225513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f039c0cf5e0 msgr2=0x7f039c0b8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f039c0cf5e0 0x7f039c0b8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 -- 172.21.15.86:0/503225513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0cbc50 con 0x7f039c15e5a0 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f039be00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15e5a0 0x7f039c0b6dc0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f039401a560 tx=0x7f03940196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.287+0000 7f0393e00640 1 -- 172.21.15.86:0/503225513 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940bc020 con 0x7f039c15e5a0 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f0393e00640 1 -- 172.21.15.86:0/503225513 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03940b8670 con 0x7f039c15e5a0 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c0cbf30 con 0x7f039c15e5a0 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f0393e00640 1 -- 172.21.15.86:0/503225513 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940b8a60 con 0x7f039c15e5a0 2024-03-22T04:20:46.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c0cc4f0 con 0x7f039c15e5a0 2024-03-22T04:20:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c074470 con 0x7f039c15e5a0 2024-03-22T04:20:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f0393e00640 1 -- 172.21.15.86:0/503225513 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f03940bf020 con 0x7f039c15e5a0 2024-03-22T04:20:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f0393e00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0378067b20 0x7f0378069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f0393e00640 1 -- 172.21.15.86:0/503225513 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6389+0+0 (secure 0 0 0) 0x7f03941211d0 con 0x7f039c15e5a0 2024-03-22T04:20:46.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f039b400640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0378067b20 0x7f0378069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.291+0000 7f039b400640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0378067b20 0x7f0378069fe0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f038c0079c0 tx=0x7f038c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:46.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.295+0000 7f0393e00640 1 -- 172.21.15.86:0/503225513 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f03940c6030 con 0x7f039c15e5a0 2024-03-22T04:20:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:46.459+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f039c06e190 con 0x7f039c15e5a0 2024-03-22T04:20:47.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.115+0000 7f0393e00640 1 -- 172.21.15.86:0/503225513 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) ==== 181+0+0 (secure 0 0 0) 0x7f03940165e0 con 0x7f039c15e5a0 2024-03-22T04:20:47.117 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T04:20:47.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0378067b20 msgr2=0x7f0378069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:47.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0378067b20 0x7f0378069fe0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f038c0079c0 tx=0x7f038c008040 comp rx=0 tx=0).stop 2024-03-22T04:20:47.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15e5a0 msgr2=0x7f039c0b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:47.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15e5a0 0x7f039c0b6dc0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f039401a560 tx=0x7f03940196d0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 shutdown_connections 2024-03-22T04:20:47.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0378067b20 0x7f0378069fe0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15e5a0 0x7f039c0b6dc0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f039c0d4600 0x7f039c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 --2- 172.21.15.86:0/503225513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f039c0cf5e0 0x7f039c0b8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 >> 172.21.15.86:0/503225513 conn(0x7f039c08c000 msgr2=0x7f039c06d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:47.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 shutdown_connections 2024-03-22T04:20:47.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.119+0000 7f03a0e00640 1 -- 172.21.15.86:0/503225513 wait complete. 2024-03-22T04:20:47.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T04:20:47.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6ed200640 1 Processor -- start 2024-03-22T04:20:47.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6ed200640 1 -- start start 2024-03-22T04:20:47.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6ed200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806b660 0x7ff6e80ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:47.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6ed200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff6e815c8e0 0x7ff6e80d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:47.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6ed200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6e815da70 con 0x7ff6e806ba40 2024-03-22T04:20:47.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6ed200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e815d1d0 con 0x7ff6e815c8e0 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6ed200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e815d4b0 con 0x7ff6e806b660 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff6e806ba40 0x7ff6e806fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37692/0 (socket says 172.21.15.86:37692) 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e5400640 1 -- 172.21.15.86:0/3643381239 learned_addr learned my addr 172.21.15.86:0/3643381239 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e6800640 1 --2- 172.21.15.86:0/3643381239 >> v2:172.21.15.86:3300/0 conn(0x7ff6e815c8e0 0x7ff6e80d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e5e00640 1 --2- 172.21.15.86:0/3643381239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806b660 0x7ff6e80ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e4a00640 1 -- 172.21.15.86:0/3643381239 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573673601 0 0) 0x7ff6e815da70 con 0x7ff6e806ba40 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e4a00640 1 -- 172.21.15.86:0/3643381239 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6cc003530 con 0x7ff6e806ba40 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e6800640 1 -- 172.21.15.86:0/3643381239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806b660 msgr2=0x7ff6e80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e6800640 1 --2- 172.21.15.86:0/3643381239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806b660 0x7ff6e80ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e6800640 1 -- 172.21.15.86:0/3643381239 >> v1:172.21.15.86:6789/0 conn(0x7ff6e806ba40 legacy=0x7ff6e806fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.287+0000 7ff6e6800640 1 -- 172.21.15.86:0/3643381239 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e815d790 con 0x7ff6e815c8e0 2024-03-22T04:20:47.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6e6800640 1 --2- 172.21.15.86:0/3643381239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e815c8e0 0x7ff6e80d0380 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00097c0 tx=0x7ff6e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64c4add3c5fc4b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:47.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6e4a00640 1 -- 172.21.15.86:0/3643381239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00b0070 con 0x7ff6e815c8e0 2024-03-22T04:20:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6e4a00640 1 -- 172.21.15.86:0/3643381239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e00ab600 con 0x7ff6e815c8e0 2024-03-22T04:20:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6e4a00640 1 -- 172.21.15.86:0/3643381239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00a4e60 con 0x7ff6e815c8e0 2024-03-22T04:20:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6ed200640 1 -- 172.21.15.86:0/3643381239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e815c8e0 msgr2=0x7ff6e80d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3643381239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e815c8e0 0x7ff6e80d0380 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00097c0 tx=0x7ff6e001c410 comp rx=0 tx=0).stop 2024-03-22T04:20:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6ed200640 1 -- 172.21.15.86:0/3643381239 shutdown_connections 2024-03-22T04:20:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3643381239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e815c8e0 0x7ff6e80d0380 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3643381239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806b660 0x7ff6e80ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6ed200640 1 -- 172.21.15.86:0/3643381239 >> 172.21.15.86:0/3643381239 conn(0x7ff6e80d1120 msgr2=0x7ff6e80d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6ed200640 1 -- 172.21.15.86:0/3643381239 shutdown_connections 2024-03-22T04:20:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.291+0000 7ff6ed200640 1 -- 172.21.15.86:0/3643381239 wait complete. 2024-03-22T04:20:47.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 Processor -- start 2024-03-22T04:20:47.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 -- start start 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806b660 0x7ff6e80b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806ba40 0x7ff6e815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e815c8e0 0x7ff6e8144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e8168ed0 con 0x7ff6e806b660 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e815cf80 con 0x7ff6e815c8e0 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e8169050 con 0x7ff6e806ba40 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806b660 0x7ff6e80b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806b660 0x7ff6e80b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39458/0 (socket says 172.21.15.86:39458) 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 -- 172.21.15.86:0/3028568927 learned_addr learned my addr 172.21.15.86:0/3028568927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5400640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806ba40 0x7ff6e815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e6800640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e815c8e0 0x7ff6e8144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 -- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806ba40 msgr2=0x7ff6e815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806ba40 0x7ff6e815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 -- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e815c8e0 msgr2=0x7ff6e8144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e815c8e0 0x7ff6e8144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 -- 172.21.15.86:0/3028568927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e81faef0 con 0x7ff6e806b660 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5400640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806ba40 0x7ff6e815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6e5e00640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806b660 0x7ff6e80b5c80 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800efc0 tx=0x7ff6d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6de000640 1 -- 172.21.15.86:0/3028568927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8018020 con 0x7ff6e806b660 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6de000640 1 -- 172.21.15.86:0/3028568927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d8014480 con 0x7ff6e806b660 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6de000640 1 -- 172.21.15.86:0/3028568927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8014850 con 0x7ff6e806b660 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e81fa170 con 0x7ff6e806b660 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e81fa9d0 con 0x7ff6e806b660 2024-03-22T04:20:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.295+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e8074580 con 0x7ff6e806b660 2024-03-22T04:20:47.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.299+0000 7ff6de000640 1 -- 172.21.15.86:0/3028568927 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff6d801b020 con 0x7ff6e806b660 2024-03-22T04:20:47.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.299+0000 7ff6de000640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6c8067b20 0x7ff6c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:47.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.299+0000 7ff6de000640 1 -- 172.21.15.86:0/3028568927 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5951+0+0 (secure 0 0 0) 0x7ff6d8086a00 con 0x7ff6e806b660 2024-03-22T04:20:47.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.299+0000 7ff6e5400640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6c8067b20 0x7ff6c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.299+0000 7ff6e5400640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6c8067b20 0x7ff6c8069fe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff6e815c3c0 tx=0x7ff6d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.303+0000 7ff6de000640 1 -- 172.21.15.86:0/3028568927 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff6d8022070 con 0x7ff6e806b660 2024-03-22T04:20:47.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:47.455+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6e80cc980 con 0x7ff6e806b660 2024-03-22T04:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.127+0000 7ff6de000640 1 -- 172.21.15.86:0/3028568927 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) ==== 169+0+0 (secure 0 0 0) 0x7ff6d8010040 con 0x7ff6e806b660 2024-03-22T04:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T04:20:48.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6c8067b20 msgr2=0x7ff6c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6c8067b20 0x7ff6c8069fe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff6e815c3c0 tx=0x7ff6d4009040 comp rx=0 tx=0).stop 2024-03-22T04:20:48.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806b660 msgr2=0x7ff6e80b5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806b660 0x7ff6e80b5c80 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800efc0 tx=0x7ff6d800c490 comp rx=0 tx=0).stop 2024-03-22T04:20:48.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 shutdown_connections 2024-03-22T04:20:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff6c8067b20 0x7ff6c8069fe0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e815c8e0 0x7ff6e8144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e806ba40 0x7ff6e815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 --2- 172.21.15.86:0/3028568927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e806b660 0x7ff6e80b5c80 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.131+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 >> 172.21.15.86:0/3028568927 conn(0x7ff6e80d1120 msgr2=0x7ff6e8069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.135+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 shutdown_connections 2024-03-22T04:20:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.135+0000 7ff6ed200640 1 -- 172.21.15.86:0/3028568927 wait complete. 2024-03-22T04:20:48.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-22T04:20:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7dfac00640 1 Processor -- start 2024-03-22T04:20:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7dfac00640 1 -- start start 2024-03-22T04:20:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7dfac00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7df415e970 0x7f7df40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7dfac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7dfac00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df406dd70 con 0x7f7df409afb0 2024-03-22T04:20:48.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7dfac00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7df40b49a0 con 0x7f7df415e970 2024-03-22T04:20:48.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7dfac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7df40ccda0 con 0x7f7df406d8f0 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7df415e970 0x7f7df40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7df415e970 0x7f7df40b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:59052/0 (socket says 172.21.15.86:59052) 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df8c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7df409afb0 0x7f7df40d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:59344/0 (socket says 172.21.15.86:59344) 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3e00640 1 -- 172.21.15.86:0/671786087 learned_addr learned my addr 172.21.15.86:0/671786087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df2a00640 1 -- 172.21.15.86:0/671786087 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2137795379 0 0) 0x7f7dec0014a0 con 0x7f7df409afb0 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df2a00640 1 -- 172.21.15.86:0/671786087 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dd8003530 con 0x7f7df409afb0 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3400640 1 -- 172.21.15.86:0/671786087 >> v2:172.21.15.196:3300/0 conn(0x7f7df415e970 msgr2=0x7f7df40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3400640 1 --2- 172.21.15.86:0/671786087 >> v2:172.21.15.196:3300/0 conn(0x7f7df415e970 0x7f7df40b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3400640 1 -- 172.21.15.86:0/671786087 >> v1:172.21.15.196:6789/0 conn(0x7f7df409afb0 legacy=0x7f7df40d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3400640 1 -- 172.21.15.86:0/671786087 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df406dff0 con 0x7f7df406d8f0 2024-03-22T04:20:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.295+0000 7f7df3e00640 1 --2- 172.21.15.86:0/671786087 >> v2:172.21.15.196:3300/0 conn(0x7f7df415e970 0x7f7df40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:48.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7df3400640 1 --2- 172.21.15.86:0/671786087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df409aa70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7de00097c0 tx=0x7f7de001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ee30ae03c582a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:48.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7df2a00640 1 -- 172.21.15.86:0/671786087 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00b2070 con 0x7f7df406d8f0 2024-03-22T04:20:48.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7df2a00640 1 -- 172.21.15.86:0/671786087 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de00ad640 con 0x7f7df406d8f0 2024-03-22T04:20:48.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7df2a00640 1 -- 172.21.15.86:0/671786087 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00ada10 con 0x7f7df406d8f0 2024-03-22T04:20:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 -- 172.21.15.86:0/671786087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 msgr2=0x7f7df409aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 --2- 172.21.15.86:0/671786087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df409aa70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7de00097c0 tx=0x7f7de001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 -- 172.21.15.86:0/671786087 shutdown_connections 2024-03-22T04:20:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 --2- 172.21.15.86:0/671786087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df409aa70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 --2- 172.21.15.86:0/671786087 >> v2:172.21.15.196:3300/0 conn(0x7f7df415e970 0x7f7df40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 -- 172.21.15.86:0/671786087 >> 172.21.15.86:0/671786087 conn(0x7f7df40b3bf0 msgr2=0x7f7df40b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 -- 172.21.15.86:0/671786087 shutdown_connections 2024-03-22T04:20:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 -- 172.21.15.86:0/671786087 wait complete. 2024-03-22T04:20:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.299+0000 7f7dfac00640 1 Processor -- start 2024-03-22T04:20:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 -- start start 2024-03-22T04:20:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df4151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7df409afb0 0x7f7df4156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df415e970 0x7f7df4153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7df416b550 con 0x7f7df409afb0 2024-03-22T04:20:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7df415ed50 con 0x7f7df415e970 2024-03-22T04:20:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7df416b6d0 con 0x7f7df406d8f0 2024-03-22T04:20:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7df409afb0 0x7f7df4156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7df409afb0 0x7f7df4156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39468/0 (socket says 172.21.15.86:39468) 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 -- 172.21.15.86:0/435004993 learned_addr learned my addr 172.21.15.86:0/435004993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3e00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df4151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df8c00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df415e970 0x7f7df4153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 -- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 msgr2=0x7f7df4151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df4151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 -- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df415e970 msgr2=0x7f7df4153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df415e970 0x7f7df4153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 -- 172.21.15.86:0/435004993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df4157920 con 0x7f7df409afb0 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df8c00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df415e970 0x7f7df4153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7df3400640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7df409afb0 0x7f7df4156f40 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7de00b0040 tx=0x7f7de0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7debe00640 1 -- 172.21.15.86:0/435004993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00b2040 con 0x7f7df409afb0 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7debe00640 1 -- 172.21.15.86:0/435004993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de00bc070 con 0x7f7df409afb0 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df4157c00 con 0x7f7df409afb0 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7debe00640 1 -- 172.21.15.86:0/435004993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0006e20 con 0x7f7df409afb0 2024-03-22T04:20:48.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df40c94b0 con 0x7f7df409afb0 2024-03-22T04:20:48.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df4074470 con 0x7f7df409afb0 2024-03-22T04:20:48.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.303+0000 7f7debe00640 1 -- 172.21.15.86:0/435004993 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7de00b6670 con 0x7f7df409afb0 2024-03-22T04:20:48.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.307+0000 7f7debe00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7dd4067b50 0x7f7dd406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.307+0000 7f7debe00640 1 -- 172.21.15.86:0/435004993 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5421+0+0 (secure 0 0 0) 0x7f7de011fdd0 con 0x7f7df409afb0 2024-03-22T04:20:48.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.307+0000 7f7df3e00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7dd4067b50 0x7f7dd406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.307+0000 7f7df3e00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7dd4067b50 0x7f7dd406a010 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7df4162120 tx=0x7f7de4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.311+0000 7f7debe00640 1 -- 172.21.15.86:0/435004993 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7de0016090 con 0x7f7df409afb0 2024-03-22T04:20:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.467+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f7df41590a0 con 0x7f7df409afb0 2024-03-22T04:20:48.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7debe00640 1 -- 172.21.15.86:0/435004993 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f7de00f07e0 con 0x7f7df409afb0 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7dd4067b50 msgr2=0x7f7dd406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7dd4067b50 0x7f7dd406a010 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7df4162120 tx=0x7f7de4008040 comp rx=0 tx=0).stop 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7df409afb0 msgr2=0x7f7df4156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7df409afb0 0x7f7df4156f40 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7de00b0040 tx=0x7f7de0019240 comp rx=0 tx=0).stop 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 shutdown_connections 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7dd4067b50 0x7f7dd406a010 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df415e970 0x7f7df4153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7df409afb0 0x7f7df4156f40 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 --2- 172.21.15.86:0/435004993 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df406d8f0 0x7f7df4151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 >> 172.21.15.86:0/435004993 conn(0x7f7df40b3bf0 msgr2=0x7f7df406f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 shutdown_connections 2024-03-22T04:20:48.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.471+0000 7f7dfac00640 1 -- 172.21.15.86:0/435004993 wait complete. 2024-03-22T04:20:48.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-22T04:20:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f835aa00640 1 Processor -- start 2024-03-22T04:20:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f835aa00640 1 -- start start 2024-03-22T04:20:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f835aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f835415b3f0 0x7f83540729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f835aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83540cc3e0 0x7f8354072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f835aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8354071920 con 0x7f83540b30f0 2024-03-22T04:20:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f835aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f83540716d0 con 0x7f835415b3f0 2024-03-22T04:20:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f835aa00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f835406d990 con 0x7f83540cc3e0 2024-03-22T04:20:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8353e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f835415b3f0 0x7f83540729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8353e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f835415b3f0 0x7f83540729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39482/0 (socket says 172.21.15.86:39482) 2024-03-22T04:20:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8353e00640 1 -- 172.21.15.86:0/2697484653 learned_addr learned my addr 172.21.15.86:0/2697484653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8353400640 1 --2- 172.21.15.86:0/2697484653 >> v2:172.21.15.196:3300/0 conn(0x7f83540cc3e0 0x7f8354072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8353e00640 1 -- 172.21.15.86:0/2697484653 >> v2:172.21.15.196:3300/0 conn(0x7f83540cc3e0 msgr2=0x7f8354072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8353e00640 1 --2- 172.21.15.86:0/2697484653 >> v2:172.21.15.196:3300/0 conn(0x7f83540cc3e0 0x7f8354072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8353e00640 1 -- 172.21.15.86:0/2697484653 >> v1:172.21.15.86:6789/0 conn(0x7f83540b30f0 legacy=0x7f83540ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8352a00640 1 -- 172.21.15.86:0/2697484653 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 500189251 0 0) 0x7f8354071920 con 0x7f83540b30f0 2024-03-22T04:20:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.627+0000 7f8353e00640 1 -- 172.21.15.86:0/2697484653 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8354071c00 con 0x7f835415b3f0 2024-03-22T04:20:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f8353e00640 1 --2- 172.21.15.86:0/2697484653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835415b3f0 0x7f83540729c0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8354070ed0 tx=0x7f833c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42fc069a546f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f8352a00640 1 -- 172.21.15.86:0/2697484653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c0b2070 con 0x7f835415b3f0 2024-03-22T04:20:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f8352a00640 1 -- 172.21.15.86:0/2697484653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c0ad680 con 0x7f835415b3f0 2024-03-22T04:20:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f8352a00640 1 -- 172.21.15.86:0/2697484653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c0ada50 con 0x7f835415b3f0 2024-03-22T04:20:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f835aa00640 1 -- 172.21.15.86:0/2697484653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835415b3f0 msgr2=0x7f83540729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f835aa00640 1 --2- 172.21.15.86:0/2697484653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835415b3f0 0x7f83540729c0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8354070ed0 tx=0x7f833c01c490 comp rx=0 tx=0).stop 2024-03-22T04:20:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f835aa00640 1 -- 172.21.15.86:0/2697484653 shutdown_connections 2024-03-22T04:20:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f835aa00640 1 --2- 172.21.15.86:0/2697484653 >> v2:172.21.15.196:3300/0 conn(0x7f83540cc3e0 0x7f8354072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f835aa00640 1 --2- 172.21.15.86:0/2697484653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835415b3f0 0x7f83540729c0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f835aa00640 1 -- 172.21.15.86:0/2697484653 >> 172.21.15.86:0/2697484653 conn(0x7f83540b1df0 msgr2=0x7f83540b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f835aa00640 1 -- 172.21.15.86:0/2697484653 shutdown_connections 2024-03-22T04:20:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.631+0000 7f835aa00640 1 -- 172.21.15.86:0/2697484653 wait complete. 2024-03-22T04:20:48.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 Processor -- start 2024-03-22T04:20:48.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 -- start start 2024-03-22T04:20:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83540b30f0 0x7f835414a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83540cc3e0 0x7f835414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835415b3f0 0x7f835414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8354168580 con 0x7f83540b30f0 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8354168400 con 0x7f835415b3f0 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8354168700 con 0x7f83540cc3e0 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83540b30f0 0x7f835414a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83540b30f0 0x7f835414a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39494/0 (socket says 172.21.15.86:39494) 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 -- 172.21.15.86:0/2269858323 learned_addr learned my addr 172.21.15.86:0/2269858323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 -- 172.21.15.86:0/2269858323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83540cc3e0 msgr2=0x7f835414f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83540cc3e0 0x7f835414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 -- 172.21.15.86:0/2269858323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835415b3f0 msgr2=0x7f835414f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8358a00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835415b3f0 0x7f835414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835415b3f0 0x7f835414f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 -- 172.21.15.86:0/2269858323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835414f0a0 con 0x7f83540b30f0 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8358a00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835415b3f0 0x7f835414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:20:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f8353e00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83540b30f0 0x7f835414a5c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f833c0190a0 tx=0x7f833c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f834be00640 1 -- 172.21.15.86:0/2269858323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c0b2040 con 0x7f83540b30f0 2024-03-22T04:20:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f834be00640 1 -- 172.21.15.86:0/2269858323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c0adc70 con 0x7f83540b30f0 2024-03-22T04:20:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835414e000 con 0x7f83540b30f0 2024-03-22T04:20:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f834be00640 1 -- 172.21.15.86:0/2269858323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c004230 con 0x7f83540b30f0 2024-03-22T04:20:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835414e5c0 con 0x7f83540b30f0 2024-03-22T04:20:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.635+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8354068650 con 0x7f83540b30f0 2024-03-22T04:20:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.639+0000 7f834be00640 1 -- 172.21.15.86:0/2269858323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f833c003620 con 0x7f83540b30f0 2024-03-22T04:20:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.639+0000 7f834be00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8330067b20 0x7f8330069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.639+0000 7f834be00640 1 -- 172.21.15.86:0/2269858323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5421+0+0 (secure 0 0 0) 0x7f833c120090 con 0x7f83540b30f0 2024-03-22T04:20:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.639+0000 7f8353400640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8330067b20 0x7f8330069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.639+0000 7f8353400640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8330067b20 0x7f8330069fe0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f835414b840 tx=0x7f8344008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:48.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.643+0000 7f834be00640 1 -- 172.21.15.86:0/2269858323 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f833c016110 con 0x7f83540b30f0 2024-03-22T04:20:48.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:48.851+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f8354155240 con 0x7f83540b30f0 2024-03-22T04:20:49.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f834be00640 1 -- 172.21.15.86:0/2269858323 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v498) ==== 146+0+0 (secure 0 0 0) 0x7f833c0ade20 con 0x7f83540b30f0 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8330067b20 msgr2=0x7f8330069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8330067b20 0x7f8330069fe0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f835414b840 tx=0x7f8344008040 comp rx=0 tx=0).stop 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83540b30f0 msgr2=0x7f835414a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83540b30f0 0x7f835414a5c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f833c0190a0 tx=0x7f833c01acd0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 shutdown_connections 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8330067b20 0x7f8330069fe0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835415b3f0 0x7f835414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83540cc3e0 0x7f835414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 --2- 172.21.15.86:0/2269858323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83540b30f0 0x7f835414a5c0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 >> 172.21.15.86:0/2269858323 conn(0x7f83540b1df0 msgr2=0x7f835406e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 shutdown_connections 2024-03-22T04:20:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.131+0000 7f835aa00640 1 -- 172.21.15.86:0/2269858323 wait complete. 2024-03-22T04:20:49.155 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-22T04:20:49.155 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-22T04:20:49.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:20:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f4782a00640 1 Processor -- start 2024-03-22T04:20:49.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f4782a00640 1 -- start start 2024-03-22T04:20:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f4782a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 0x7f477c071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f4782a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f477c1623d0 con 0x7f477c15e630 2024-03-22T04:20:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f4782a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f477c1620f0 con 0x7f477c06e2f0 2024-03-22T04:20:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f4782a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f477c15ec10 con 0x7f477c0d0580 2024-03-22T04:20:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f477c15e630 0x7f477c0cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37714/0 (socket says 172.21.15.86:37714) 2024-03-22T04:20:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477be00640 1 -- 172.21.15.86:0/3188383036 learned_addr learned my addr 172.21.15.86:0/3188383036 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477b400640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 0x7f477c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 683189981 0 0) 0x7f477c1623d0 con 0x7f477c15e630 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4760003530 con 0x7f477c15e630 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2443595681 0 0) 0x7f4760003530 con 0x7f477c15e630 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 msgr2=0x7f477c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 0x7f477c071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 >> v1:172.21.15.196:6789/0 conn(0x7f477c06e2f0 legacy=0x7f477c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c15eef0 con 0x7f477c15e630 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f476c001680 con 0x7f477c15e630 2024-03-22T04:20:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 >> v1:172.21.15.86:6789/0 conn(0x7f477c15e630 legacy=0x7f477c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47600039f0 0x7f4760003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4760004810 0x7f4760004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 0x7f47600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f477c15ec10 con 0x7f47600039f0 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f476000bab0 con 0x7f4760004810 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f476000bcb0 con 0x7f477c0d0580 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477be00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47600039f0 0x7f4760003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f4780a00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 0x7f47600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477b400640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4760004810 0x7f4760004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477be00640 1 -- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 msgr2=0x7f47600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477be00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 0x7f47600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477be00640 1 -- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4760004810 msgr2=0x7f4760004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477be00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4760004810 0x7f4760004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477be00640 1 -- 172.21.15.86:0/3188383036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4760003530 con 0x7f47600039f0 2024-03-22T04:20:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f4780a00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0d0580 0x7f47600051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.291+0000 7f477b400640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4760004810 0x7f4760004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:49.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f477be00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47600039f0 0x7f4760003e10 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c210 tx=0x7f476c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d38f50997dc591d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c007d40 con 0x7f47600039f0 2024-03-22T04:20:49.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476c0a9070 con 0x7f47600039f0 2024-03-22T04:20:49.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f477aa00640 1 -- 172.21.15.86:0/3188383036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c005d40 con 0x7f47600039f0 2024-03-22T04:20:49.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f4782a00640 1 -- 172.21.15.86:0/3188383036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47600039f0 msgr2=0x7f4760003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f4782a00640 1 --2- 172.21.15.86:0/3188383036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47600039f0 0x7f4760003e10 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c210 tx=0x7f476c01cca0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f4780a00640 1 -- 172.21.15.86:0/3188383036 reap_dead start 2024-03-22T04:20:49.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f4782a00640 1 -- 172.21.15.86:0/3188383036 shutdown_connections 2024-03-22T04:20:49.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f4782a00640 1 -- 172.21.15.86:0/3188383036 >> 172.21.15.86:0/3188383036 conn(0x7f477c06f910 msgr2=0x7f477c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:49.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f4782a00640 1 -- 172.21.15.86:0/3188383036 shutdown_connections 2024-03-22T04:20:49.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.295+0000 7f4782a00640 1 -- 172.21.15.86:0/3188383036 wait complete. 2024-03-22T04:20:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 Processor -- start 2024-03-22T04:20:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 -- start start 2024-03-22T04:20:49.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47600039f0 0x7f477c147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c148260 0x7f477c1682c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f477c15be30 0x7f477c168800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f477c0cfaf0 con 0x7f477c15be30 2024-03-22T04:20:49.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f477c15f9d0 con 0x7f477c148260 2024-03-22T04:20:49.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f477c0cfc70 con 0x7f47600039f0 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f477c15be30 0x7f477c168800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f477c15be30 0x7f477c168800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39512/0 (socket says 172.21.15.86:39512) 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 -- 172.21.15.86:0/1903314073 learned_addr learned my addr 172.21.15.86:0/1903314073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f477b400640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c148260 0x7f477c1682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f477be00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47600039f0 0x7f477c147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 -- 172.21.15.86:0/1903314073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47600039f0 msgr2=0x7f477c147d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47600039f0 0x7f477c147d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 -- 172.21.15.86:0/1903314073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c148260 msgr2=0x7f477c1682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c148260 0x7f477c1682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 -- 172.21.15.86:0/1903314073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c0c8900 con 0x7f477c15be30 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4780a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f477c15be30 0x7f477c168800 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f4774000f90 tx=0x7f477400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4773e00640 1 -- 172.21.15.86:0/1903314073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774006b60 con 0x7f477c15be30 2024-03-22T04:20:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4773e00640 1 -- 172.21.15.86:0/1903314073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4774006d20 con 0x7f477c15be30 2024-03-22T04:20:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477c0b5000 con 0x7f477c15be30 2024-03-22T04:20:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4773e00640 1 -- 172.21.15.86:0/1903314073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774003980 con 0x7f477c15be30 2024-03-22T04:20:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477c0b52e0 con 0x7f477c15be30 2024-03-22T04:20:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.299+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477c068650 con 0x7f477c15be30 2024-03-22T04:20:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.303+0000 7f4773e00640 1 -- 172.21.15.86:0/1903314073 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f477401a030 con 0x7f477c15be30 2024-03-22T04:20:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.303+0000 7f4773e00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f475c067b50 0x7f475c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.303+0000 7f4773e00640 1 -- 172.21.15.86:0/1903314073 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5421+0+0 (secure 0 0 0) 0x7f4774085230 con 0x7f477c15be30 2024-03-22T04:20:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.303+0000 7f477be00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f475c067b50 0x7f475c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.303+0000 7f477be00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f475c067b50 0x7f475c06a010 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f476c0ab000 tx=0x7f476c0b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.311+0000 7f4773e00640 1 -- 172.21.15.86:0/1903314073 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4774021020 con 0x7f477c15be30 2024-03-22T04:20:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.471+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f477c0b29d0 con 0x7f477c15be30 2024-03-22T04:20:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4773e00640 1 -- 172.21.15.86:0/1903314073 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+13435 (secure 0 0 0) 0x7f4774006f10 con 0x7f477c15be30 2024-03-22T04:20:49.475 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:49.475 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":115,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:48.129107+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6832/269054026":"2024-03-23T04:20:40.626610+0000","172.21.15.196:6835/2259098413":"2024-03-23T04:20:03.917951+0000","172.21.15.196:6834/269054026":"2024-03-23T04:20:40.626610+0000","172.21.15.196:6833/2259098413":"2024-03-23T04:20:03.917951+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:20:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f475c067b50 msgr2=0x7f475c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f475c067b50 0x7f475c06a010 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f476c0ab000 tx=0x7f476c0b1040 comp rx=0 tx=0).stop 2024-03-22T04:20:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f477c15be30 msgr2=0x7f477c168800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f477c15be30 0x7f477c168800 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f4774000f90 tx=0x7f477400eb90 comp rx=0 tx=0).stop 2024-03-22T04:20:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 shutdown_connections 2024-03-22T04:20:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f475c067b50 0x7f475c06a010 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c148260 0x7f477c1682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f477c15be30 0x7f477c168800 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 --2- 172.21.15.86:0/1903314073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47600039f0 0x7f477c147d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 >> 172.21.15.86:0/1903314073 conn(0x7f477c06f910 msgr2=0x7f477c162600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 shutdown_connections 2024-03-22T04:20:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.475+0000 7f4782a00640 1 -- 172.21.15.86:0/1903314073 wait complete. 2024-03-22T04:20:49.496 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:20:49.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:20:49.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f1066800640 1 Processor -- start 2024-03-22T04:20:49.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f1066800640 1 -- start start 2024-03-22T04:20:49.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f1066800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 0x7f1060072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f1066800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1060073150 con 0x7f106015b760 2024-03-22T04:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f1066800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10600715a0 con 0x7f106006e7e0 2024-03-22T04:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f1066800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1060071350 con 0x7f1060072650 2024-03-22T04:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f106015b760 0x7f106015bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37718/0 (socket says 172.21.15.86:37718) 2024-03-22T04:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105fe00640 1 -- 172.21.15.86:0/654817816 learned_addr learned my addr 172.21.15.86:0/654817816 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105ea00640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 0x7f1060072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3342744370 0 0) 0x7f1060073150 con 0x7f106015b760 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1044005740 con 0x7f106015b760 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1708702348 0 0) 0x7f10600715a0 con 0x7f106006e7e0 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1060073150 con 0x7f106006e7e0 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2098588736 0 0) 0x7f1044005740 con 0x7f106015b760 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 msgr2=0x7f1060072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 0x7f1060072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 >> v1:172.21.15.196:6789/0 conn(0x7f106006e7e0 legacy=0x7f106006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1060071880 con 0x7f106015b760 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f1058001a00 con 0x7f106015b760 2024-03-22T04:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105ea00640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 0x7f1060072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 >> v1:172.21.15.86:6789/0 conn(0x7f106015b760 legacy=0x7f106015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044005d90 0x7f1044006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 0x7f1044006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10440073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1060071350 con 0x7f1044006f60 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f104400dcf0 con 0x7f1044005d90 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f104400def0 con 0x7f1060072650 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105fe00640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10440073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105f400640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044005d90 0x7f1044006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105ea00640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 0x7f1044006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105fe00640 1 -- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 msgr2=0x7f1044006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105fe00640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060072650 0x7f1044006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105fe00640 1 -- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044005d90 msgr2=0x7f1044006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105fe00640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1044005d90 0x7f1044006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.635+0000 7f105fe00640 1 -- 172.21.15.86:0/654817816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1044005740 con 0x7f1044006f60 2024-03-22T04:20:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f105fe00640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10440073e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f105800a720 tx=0x7f105801d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b8c6cdbd26a6a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105800a370 con 0x7f1044006f60 2024-03-22T04:20:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10580ad650 con 0x7f1044006f60 2024-03-22T04:20:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f105e000640 1 -- 172.21.15.86:0/654817816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10580ada40 con 0x7f1044006f60 2024-03-22T04:20:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f1066800640 1 -- 172.21.15.86:0/654817816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 msgr2=0x7f10440073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f1066800640 1 --2- 172.21.15.86:0/654817816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10440073e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f105800a720 tx=0x7f105801d240 comp rx=0 tx=0).stop 2024-03-22T04:20:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f105fe00640 1 -- 172.21.15.86:0/654817816 reap_dead start 2024-03-22T04:20:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f1066800640 1 -- 172.21.15.86:0/654817816 shutdown_connections 2024-03-22T04:20:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f1066800640 1 -- 172.21.15.86:0/654817816 >> 172.21.15.86:0/654817816 conn(0x7f10600cc940 msgr2=0x7f10600ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f1066800640 1 -- 172.21.15.86:0/654817816 shutdown_connections 2024-03-22T04:20:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.639+0000 7f1066800640 1 -- 172.21.15.86:0/654817816 wait complete. 2024-03-22T04:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 Processor -- start 2024-03-22T04:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 -- start start 2024-03-22T04:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10600c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060144280 0x7f10600cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10600ca040 0x7f10600ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1060072080 con 0x7f1044006f60 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10600ce930 con 0x7f10600ca040 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f10600c7c20 con 0x7f1060144280 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10600c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10600c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39534/0 (socket says 172.21.15.86:39534) 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 -- 172.21.15.86:0/2704226248 learned_addr learned my addr 172.21.15.86:0/2704226248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105ea00640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060144280 0x7f10600cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105fe00640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10600ca040 0x7f10600ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 -- 172.21.15.86:0/2704226248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060144280 msgr2=0x7f10600cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060144280 0x7f10600cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 -- 172.21.15.86:0/2704226248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10600ca040 msgr2=0x7f10600ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10600ca040 0x7f10600ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 -- 172.21.15.86:0/2704226248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10600cc5a0 con 0x7f1044006f60 2024-03-22T04:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f105f400640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10600c76e0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f1050012830 tx=0x7f1050012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1057400640 1 -- 172.21.15.86:0/2704226248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050010bd0 con 0x7f1044006f60 2024-03-22T04:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1057400640 1 -- 172.21.15.86:0/2704226248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105001b070 con 0x7f1044006f60 2024-03-22T04:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1057400640 1 -- 172.21.15.86:0/2704226248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050012e80 con 0x7f1044006f60 2024-03-22T04:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10600c5ac0 con 0x7f1044006f60 2024-03-22T04:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.643+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10600c6080 con 0x7f1044006f60 2024-03-22T04:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.647+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1060074470 con 0x7f1044006f60 2024-03-22T04:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.647+0000 7f1057400640 1 -- 172.21.15.86:0/2704226248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1050016480 con 0x7f1044006f60 2024-03-22T04:20:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.647+0000 7f1057400640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1040067b50 0x7f104006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.647+0000 7f1057400640 1 -- 172.21.15.86:0/2704226248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5421+0+0 (secure 0 0 0) 0x7f1050089580 con 0x7f1044006f60 2024-03-22T04:20:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.647+0000 7f105ea00640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1040067b50 0x7f104006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.647+0000 7f105ea00640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1040067b50 0x7f104006a010 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f10600c6ee0 tx=0x7f104c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.651+0000 7f1057400640 1 -- 172.21.15.86:0/2704226248 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f105005a0b0 con 0x7f1044006f60 2024-03-22T04:20:49.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1060072200 con 0x7f1044006f60 2024-03-22T04:20:49.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1057400640 1 -- 172.21.15.86:0/2704226248 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f10500599b0 con 0x7f1044006f60 2024-03-22T04:20:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:20:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1040067b50 msgr2=0x7f104006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1040067b50 0x7f104006a010 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f10600c6ee0 tx=0x7f104c008040 comp rx=0 tx=0).stop 2024-03-22T04:20:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 msgr2=0x7f10600c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10600c76e0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f1050012830 tx=0x7f1050012c30 comp rx=0 tx=0).stop 2024-03-22T04:20:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 shutdown_connections 2024-03-22T04:20:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1040067b50 0x7f104006a010 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10600ca040 0x7f10600ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1060144280 0x7f10600cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 --2- 172.21.15.86:0/2704226248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1044006f60 0x7f10600c76e0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 >> 172.21.15.86:0/2704226248 conn(0x7f10600cc940 msgr2=0x7f106015a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 shutdown_connections 2024-03-22T04:20:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.807+0000 7f1066800640 1 -- 172.21.15.86:0/2704226248 wait complete. 2024-03-22T04:20:49.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-22T04:20:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2217a00640 1 Processor -- start 2024-03-22T04:20:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2217a00640 1 -- start start 2024-03-22T04:20:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2217a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22100d0de0 0x7f22100ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2217a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d19a0 0x7f22100cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2217a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22100d11c0 con 0x7f22100d1dc0 2024-03-22T04:20:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2217a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f22100cece0 con 0x7f22100d0de0 2024-03-22T04:20:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2217a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f22100cefc0 con 0x7f22100d19a0 2024-03-22T04:20:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2215000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22100d0de0 0x7f22100ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2215000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f22100d0de0 0x7f22100ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39540/0 (socket says 172.21.15.86:39540) 2024-03-22T04:20:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2215000640 1 -- 172.21.15.86:0/539893731 learned_addr learned my addr 172.21.15.86:0/539893731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2215000640 1 -- 172.21.15.86:0/539893731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d19a0 msgr2=0x7f22100cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2215000640 1 --2- 172.21.15.86:0/539893731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d19a0 0x7f22100cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2215000640 1 -- 172.21.15.86:0/539893731 >> v1:172.21.15.196:6789/0 conn(0x7f22100d1dc0 legacy=0x7f221015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.963+0000 7f2215000640 1 -- 172.21.15.86:0/539893731 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22100cf2a0 con 0x7f22100d0de0 2024-03-22T04:20:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2215000640 1 --2- 172.21.15.86:0/539893731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d0de0 0x7f22100ce6b0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2200009e50 tx=0x7f220001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e847cbb9e22b07d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f220f400640 1 -- 172.21.15.86:0/539893731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22000a5760 con 0x7f22100d0de0 2024-03-22T04:20:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f220f400640 1 -- 172.21.15.86:0/539893731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22000a5900 con 0x7f22100d0de0 2024-03-22T04:20:49.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f220f400640 1 -- 172.21.15.86:0/539893731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22000a5cd0 con 0x7f22100d0de0 2024-03-22T04:20:49.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2217a00640 1 -- 172.21.15.86:0/539893731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d0de0 msgr2=0x7f22100ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2217a00640 1 --2- 172.21.15.86:0/539893731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d0de0 0x7f22100ce6b0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2200009e50 tx=0x7f220001c9f0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2217a00640 1 -- 172.21.15.86:0/539893731 shutdown_connections 2024-03-22T04:20:49.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2217a00640 1 --2- 172.21.15.86:0/539893731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d19a0 0x7f22100cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2217a00640 1 --2- 172.21.15.86:0/539893731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d0de0 0x7f22100ce6b0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2217a00640 1 -- 172.21.15.86:0/539893731 >> 172.21.15.86:0/539893731 conn(0x7f221006f910 msgr2=0x7f221006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:49.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2217a00640 1 -- 172.21.15.86:0/539893731 shutdown_connections 2024-03-22T04:20:49.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.967+0000 7f2217a00640 1 -- 172.21.15.86:0/539893731 wait complete. 2024-03-22T04:20:49.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 Processor -- start 2024-03-22T04:20:49.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 -- start start 2024-03-22T04:20:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d0de0 0x7f22100b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22100d19a0 0x7f2210151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d1dc0 0x7f22100c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2210160760 con 0x7f22100d1dc0 2024-03-22T04:20:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22101605e0 con 0x7f22100d19a0 2024-03-22T04:20:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2210071e30 con 0x7f22100d0de0 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d1dc0 0x7f22100c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d1dc0 0x7f22100c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39548/0 (socket says 172.21.15.86:39548) 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 -- 172.21.15.86:0/2114968797 learned_addr learned my addr 172.21.15.86:0/2114968797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f220fe00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22100d19a0 0x7f2210151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215000640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d0de0 0x7f22100b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 -- 172.21.15.86:0/2114968797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d0de0 msgr2=0x7f22100b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d0de0 0x7f22100b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 -- 172.21.15.86:0/2114968797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22100d19a0 msgr2=0x7f2210151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22100d19a0 0x7f2210151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 -- 172.21.15.86:0/2114968797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22100b5b80 con 0x7f22100d1dc0 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2215a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d1dc0 0x7f22100c8770 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f220800c3c0 tx=0x7f220800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f220cc00640 1 -- 172.21.15.86:0/2114968797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208011790 con 0x7f22100d1dc0 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f220cc00640 1 -- 172.21.15.86:0/2114968797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208017070 con 0x7f22100d1dc0 2024-03-22T04:20:49.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f220cc00640 1 -- 172.21.15.86:0/2114968797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208011b70 con 0x7f22100d1dc0 2024-03-22T04:20:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22100b5e60 con 0x7f22100d1dc0 2024-03-22T04:20:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22100b6420 con 0x7f22100d1dc0 2024-03-22T04:20:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.971+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2210068650 con 0x7f22100d1dc0 2024-03-22T04:20:49.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.975+0000 7f220cc00640 1 -- 172.21.15.86:0/2114968797 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2208011d10 con 0x7f22100d1dc0 2024-03-22T04:20:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.975+0000 7f220cc00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0067b20 0x7f21f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.975+0000 7f220cc00640 1 -- 172.21.15.86:0/2114968797 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5421+0+0 (secure 0 0 0) 0x7f22080858a0 con 0x7f22100d1dc0 2024-03-22T04:20:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.975+0000 7f2215000640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0067b20 0x7f21f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:49.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.975+0000 7f2215000640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0067b20 0x7f21f0069fe0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f22000ae000 tx=0x7f22000a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:49.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:49.979+0000 7f220cc00640 1 -- 172.21.15.86:0/2114968797 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2208021020 con 0x7f22100d1dc0 2024-03-22T04:20:50.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:50.191+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f22100ce290 con 0x7f22100d1dc0 2024-03-22T04:20:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f220cc00640 1 -- 172.21.15.86:0/2114968797 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v500) ==== 147+0+0 (secure 0 0 0) 0x7f22080563c0 con 0x7f22100d1dc0 2024-03-22T04:20:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0067b20 msgr2=0x7f21f0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0067b20 0x7f21f0069fe0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f22000ae000 tx=0x7f22000a7070 comp rx=0 tx=0).stop 2024-03-22T04:20:51.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d1dc0 msgr2=0x7f22100c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d1dc0 0x7f22100c8770 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f220800c3c0 tx=0x7f220800c880 comp rx=0 tx=0).stop 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 shutdown_connections 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f21f0067b20 0x7f21f0069fe0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22100d19a0 0x7f2210151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22100d1dc0 0x7f22100c8770 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 --2- 172.21.15.86:0/2114968797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22100d0de0 0x7f22100b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 >> 172.21.15.86:0/2114968797 conn(0x7f221006f910 msgr2=0x7f221006fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 shutdown_connections 2024-03-22T04:20:51.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.143+0000 7f2217a00640 1 -- 172.21.15.86:0/2114968797 wait complete. 2024-03-22T04:20:51.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-22T04:20:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.307+0000 7f88b2200640 1 Processor -- start 2024-03-22T04:20:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88b2200640 1 -- start start 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88b2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88ac0c6f20 0x7f88ac072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88b2200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f88ac162740 0x7f88ac0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88b2200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88ac0cc7c0 con 0x7f88ac0cb1d0 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88b2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f88ac0cb720 con 0x7f88ac0c6f20 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88b2200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f88ac0cba00 con 0x7f88ac162740 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88ab400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88ac0c6f20 0x7f88ac072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88ab400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88ac0c6f20 0x7f88ac072840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39554/0 (socket says 172.21.15.86:39554) 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88ab400640 1 -- 172.21.15.86:0/3386079009 learned_addr learned my addr 172.21.15.86:0/3386079009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88abe00640 1 --2- 172.21.15.86:0/3386079009 >> v2:172.21.15.196:3300/0 conn(0x7f88ac162740 0x7f88ac0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88ab400640 1 -- 172.21.15.86:0/3386079009 >> v2:172.21.15.196:3300/0 conn(0x7f88ac162740 msgr2=0x7f88ac0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88ab400640 1 --2- 172.21.15.86:0/3386079009 >> v2:172.21.15.196:3300/0 conn(0x7f88ac162740 0x7f88ac0c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88ab400640 1 -- 172.21.15.86:0/3386079009 >> v1:172.21.15.196:6789/0 conn(0x7f88ac0cb1d0 legacy=0x7f88ac162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.311+0000 7f88ab400640 1 -- 172.21.15.86:0/3386079009 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac0cbce0 con 0x7f88ac0c6f20 2024-03-22T04:20:51.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88ab400640 1 --2- 172.21.15.86:0/3386079009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac072840 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0cad30 tx=0x7f889c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4701abb665049f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:51.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88aaa00640 1 -- 172.21.15.86:0/3386079009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c0aa650 con 0x7f88ac0c6f20 2024-03-22T04:20:51.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88aaa00640 1 -- 172.21.15.86:0/3386079009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c0af070 con 0x7f88ac0c6f20 2024-03-22T04:20:51.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88aaa00640 1 -- 172.21.15.86:0/3386079009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c0aaa10 con 0x7f88ac0c6f20 2024-03-22T04:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 -- 172.21.15.86:0/3386079009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 msgr2=0x7f88ac072840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 --2- 172.21.15.86:0/3386079009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac072840 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0cad30 tx=0x7f889c01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 -- 172.21.15.86:0/3386079009 shutdown_connections 2024-03-22T04:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 --2- 172.21.15.86:0/3386079009 >> v2:172.21.15.196:3300/0 conn(0x7f88ac162740 0x7f88ac0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 --2- 172.21.15.86:0/3386079009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac072840 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 -- 172.21.15.86:0/3386079009 >> 172.21.15.86:0/3386079009 conn(0x7f88ac0d0730 msgr2=0x7f88ac0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 -- 172.21.15.86:0/3386079009 shutdown_connections 2024-03-22T04:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 -- 172.21.15.86:0/3386079009 wait complete. 2024-03-22T04:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 Processor -- start 2024-03-22T04:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.315+0000 7f88b2200640 1 -- start start 2024-03-22T04:20:51.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac1535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac0cb1d0 0x7f88ac1fb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88ac162740 0x7f88ac0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac0c64c0 con 0x7f88ac0c6f20 2024-03-22T04:20:51.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac0c6340 con 0x7f88ac0cb1d0 2024-03-22T04:20:51.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f88ac0c6640 con 0x7f88ac162740 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac1535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac1535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39560/0 (socket says 172.21.15.86:39560) 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 -- 172.21.15.86:0/1191627986 learned_addr learned my addr 172.21.15.86:0/1191627986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88a2a00640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac0cb1d0 0x7f88ac1fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88abe00640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88ac162740 0x7f88ac0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 -- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88ac162740 msgr2=0x7f88ac0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88ac162740 0x7f88ac0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 -- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac0cb1d0 msgr2=0x7f88ac1fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac0cb1d0 0x7f88ac1fb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 -- 172.21.15.86:0/1191627986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac06f040 con 0x7f88ac0c6f20 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88abe00640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88ac162740 0x7f88ac0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:51.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88ab400640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac1535e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f889c01a120 tx=0x7f889c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:51.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88a3e00640 1 -- 172.21.15.86:0/1191627986 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c0bd5f0 con 0x7f88ac0c6f20 2024-03-22T04:20:51.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88a3e00640 1 -- 172.21.15.86:0/1191627986 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c0a6040 con 0x7f88ac0c6f20 2024-03-22T04:20:51.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88a3e00640 1 -- 172.21.15.86:0/1191627986 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c0bd9f0 con 0x7f88ac0c6f20 2024-03-22T04:20:51.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ac06f320 con 0x7f88ac0c6f20 2024-03-22T04:20:51.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ac06f8e0 con 0x7f88ac0c6f20 2024-03-22T04:20:51.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ac074470 con 0x7f88ac0c6f20 2024-03-22T04:20:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.319+0000 7f88a3e00640 1 -- 172.21.15.86:0/1191627986 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f889c0b33f0 con 0x7f88ac0c6f20 2024-03-22T04:20:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.323+0000 7f88a3e00640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8888067b50 0x7f888806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.323+0000 7f88a3e00640 1 -- 172.21.15.86:0/1191627986 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5421+0+0 (secure 0 0 0) 0x7f889c11fdf0 con 0x7f88ac0c6f20 2024-03-22T04:20:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.323+0000 7f88a2a00640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8888067b50 0x7f888806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.323+0000 7f88a2a00640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8888067b50 0x7f888806a010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0c9c10 tx=0x7f8894009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:51.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.327+0000 7f88a3e00640 1 -- 172.21.15.86:0/1191627986 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f889c0c3060 con 0x7f88ac0c6f20 2024-03-22T04:20:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.483+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f88ac0cec70 con 0x7f88ac0c6f20 2024-03-22T04:20:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.487+0000 7f88a3e00640 1 -- 172.21.15.86:0/1191627986 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f889c016110 con 0x7f88ac0c6f20 2024-03-22T04:20:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8888067b50 msgr2=0x7f888806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8888067b50 0x7f888806a010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0c9c10 tx=0x7f8894009040 comp rx=0 tx=0).stop 2024-03-22T04:20:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 msgr2=0x7f88ac1535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac1535e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f889c01a120 tx=0x7f889c01ad30 comp rx=0 tx=0).stop 2024-03-22T04:20:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 shutdown_connections 2024-03-22T04:20:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8888067b50 0x7f888806a010 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac0cb1d0 0x7f88ac1fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88ac162740 0x7f88ac0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 --2- 172.21.15.86:0/1191627986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ac0c6f20 0x7f88ac1535e0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 >> 172.21.15.86:0/1191627986 conn(0x7f88ac0d0730 msgr2=0x7f88ac0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 shutdown_connections 2024-03-22T04:20:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.491+0000 7f88b2200640 1 -- 172.21.15.86:0/1191627986 wait complete. 2024-03-22T04:20:51.510 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:20:51.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:20:51.569 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:20:51.618 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:20:51.618 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:20:51.618 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:20:51.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:20:51.628 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:20:51.628 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:20:51.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:20:51.637 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:20:51.637 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:20:51.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:20:51.678 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:20:51.678 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:20:51.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:20:51.687 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-22T04:20:51.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:20:51.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0328000640 1 Processor -- start 2024-03-22T04:20:51.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0328000640 1 -- start start 2024-03-22T04:20:51.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0328000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f032015b3f0 0x7f03200729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0328000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f03200cc3e0 0x7f0320072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0328000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0320071920 con 0x7f03200b30f0 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0328000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f03200716d0 con 0x7f03200cc3e0 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0328000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f032006d990 con 0x7f032015b3f0 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0324c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f03200cc3e0 0x7f0320072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0324c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f03200cc3e0 0x7f0320072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39570/0 (socket says 172.21.15.86:39570) 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0324c00640 1 -- 172.21.15.86:0/4285952553 learned_addr learned my addr 172.21.15.86:0/4285952553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.855+0000 7f0325600640 1 --2- 172.21.15.86:0/4285952553 >> v2:172.21.15.196:3300/0 conn(0x7f032015b3f0 0x7f03200729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0324c00640 1 -- 172.21.15.86:0/4285952553 >> v2:172.21.15.196:3300/0 conn(0x7f032015b3f0 msgr2=0x7f03200729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0324c00640 1 --2- 172.21.15.86:0/4285952553 >> v2:172.21.15.196:3300/0 conn(0x7f032015b3f0 0x7f03200729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0324c00640 1 -- 172.21.15.86:0/4285952553 >> v1:172.21.15.196:6789/0 conn(0x7f03200b30f0 legacy=0x7f03200ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0324c00640 1 -- 172.21.15.86:0/4285952553 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0320071c00 con 0x7f03200cc3e0 2024-03-22T04:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0325600640 1 --2- 172.21.15.86:0/4285952553 >> v2:172.21.15.196:3300/0 conn(0x7f032015b3f0 0x7f03200729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:51.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0324c00640 1 --2- 172.21.15.86:0/4285952553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03200cc3e0 0x7f0320072f00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f032006c510 tx=0x7f030c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9b6ea5ac8d500f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:51.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f031fe00640 1 -- 172.21.15.86:0/4285952553 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0b2070 con 0x7f03200cc3e0 2024-03-22T04:20:51.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f031fe00640 1 -- 172.21.15.86:0/4285952553 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f030c0ad680 con 0x7f03200cc3e0 2024-03-22T04:20:51.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f031fe00640 1 -- 172.21.15.86:0/4285952553 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0ada50 con 0x7f03200cc3e0 2024-03-22T04:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0328000640 1 -- 172.21.15.86:0/4285952553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03200cc3e0 msgr2=0x7f0320072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0328000640 1 --2- 172.21.15.86:0/4285952553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03200cc3e0 0x7f0320072f00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f032006c510 tx=0x7f030c01c490 comp rx=0 tx=0).stop 2024-03-22T04:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0328000640 1 -- 172.21.15.86:0/4285952553 shutdown_connections 2024-03-22T04:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0328000640 1 --2- 172.21.15.86:0/4285952553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03200cc3e0 0x7f0320072f00 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0328000640 1 --2- 172.21.15.86:0/4285952553 >> v2:172.21.15.196:3300/0 conn(0x7f032015b3f0 0x7f03200729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.859+0000 7f0328000640 1 -- 172.21.15.86:0/4285952553 >> 172.21.15.86:0/4285952553 conn(0x7f03200b1df0 msgr2=0x7f03200b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 -- 172.21.15.86:0/4285952553 shutdown_connections 2024-03-22T04:20:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 -- 172.21.15.86:0/4285952553 wait complete. 2024-03-22T04:20:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 Processor -- start 2024-03-22T04:20:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 -- start start 2024-03-22T04:20:51.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03200b30f0 0x7f032009c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03200cc3e0 0x7f0320154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032015b3f0 0x7f03201594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0320168580 con 0x7f032015b3f0 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0320168400 con 0x7f03200cc3e0 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0320168700 con 0x7f03200b30f0 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032015b3f0 0x7f03201594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032015b3f0 0x7f03201594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39574/0 (socket says 172.21.15.86:39574) 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 -- 172.21.15.86:0/977908773 learned_addr learned my addr 172.21.15.86:0/977908773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0325600640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03200b30f0 0x7f032009c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0324c00640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03200cc3e0 0x7f0320154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 -- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03200b30f0 msgr2=0x7f032009c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03200b30f0 0x7f032009c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 -- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03200cc3e0 msgr2=0x7f0320154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03200cc3e0 0x7f0320154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 -- 172.21.15.86:0/977908773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032009a550 con 0x7f032015b3f0 2024-03-22T04:20:51.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0325600640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03200b30f0 0x7f032009c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:51.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0326000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032015b3f0 0x7f03201594c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f031800c760 tx=0x7f031800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:51.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f031d600640 1 -- 172.21.15.86:0/977908773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318003b80 con 0x7f032015b3f0 2024-03-22T04:20:51.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f031d600640 1 -- 172.21.15.86:0/977908773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0318003d20 con 0x7f032015b3f0 2024-03-22T04:20:51.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f031d600640 1 -- 172.21.15.86:0/977908773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800aa30 con 0x7f032015b3f0 2024-03-22T04:20:51.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032009a830 con 0x7f032015b3f0 2024-03-22T04:20:51.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.863+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032009adf0 con 0x7f032015b3f0 2024-03-22T04:20:51.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.867+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0320068650 con 0x7f032015b3f0 2024-03-22T04:20:51.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.867+0000 7f031d600640 1 -- 172.21.15.86:0/977908773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f031800abd0 con 0x7f032015b3f0 2024-03-22T04:20:51.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.867+0000 7f031d600640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0300067b20 0x7f0300069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:51.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.867+0000 7f031d600640 1 -- 172.21.15.86:0/977908773 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5421+0+0 (secure 0 0 0) 0x7f0318085b80 con 0x7f032015b3f0 2024-03-22T04:20:51.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.867+0000 7f0325600640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0300067b20 0x7f0300069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:51.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.867+0000 7f0325600640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0300067b20 0x7f0300069fe0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0314009970 tx=0x7f0314008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:51.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:51.871+0000 7f031d600640 1 -- 172.21.15.86:0/977908773 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0318056740 con 0x7f032015b3f0 2024-03-22T04:20:52.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f032015dfa0 con 0x7f032015b3f0 2024-03-22T04:20:52.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f031d600640 1 -- 172.21.15.86:0/977908773 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+3866 (secure 0 0 0) 0x7f0318056040 con 0x7f032015b3f0 2024-03-22T04:20:52.064 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:52.064 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8322,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2454912174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2454912174},{"type":"v1","addr":"172.21.15.86:6836","nonce":2454912174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8331,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3288438732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3288438732},{"type":"v1","addr":"172.21.15.86:6837","nonce":3288438732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/70563317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":70563317},{"type":"v1","addr":"172.21.15.196:6835","nonce":70563317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8605,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/193924359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":193924359},{"type":"v1","addr":"172.21.15.196:6834","nonce":193924359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[]} 2024-03-22T04:20:52.064 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0300067b20 msgr2=0x7f0300069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0300067b20 0x7f0300069fe0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0314009970 tx=0x7f0314008040 comp rx=0 tx=0).stop 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032015b3f0 msgr2=0x7f03201594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032015b3f0 0x7f03201594c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f031800c760 tx=0x7f031800cc20 comp rx=0 tx=0).stop 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 shutdown_connections 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f0300067b20 0x7f0300069fe0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03200cc3e0 0x7f0320154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032015b3f0 0x7f03201594c0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 --2- 172.21.15.86:0/977908773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03200b30f0 0x7f032009c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 >> 172.21.15.86:0/977908773 conn(0x7f03200b1df0 msgr2=0x7f03200b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 shutdown_connections 2024-03-22T04:20:52.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.063+0000 7f0328000640 1 -- 172.21.15.86:0/977908773 wait complete. 2024-03-22T04:20:52.085 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-22T04:20:52.085 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-22T04:20:52.086 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-22T04:20:52.086 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:20:52.086 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-22T04:20:52.086 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-22T04:20:52.086 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-22T04:20:52.086 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:20:52.086 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-22T04:20:52.087 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-22T04:20:52.087 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-22T04:20:52.087 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:20:52.087 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-22T04:20:52.087 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-22T04:20:52.087 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-22T04:20:52.087 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:20:52.098 INFO:tasks.ceph.mds.a:Stopped 2024-03-22T04:20:52.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-22T04:20:52.100 INFO:tasks.ceph.mds.c:Stopped 2024-03-22T04:20:52.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-22T04:20:52.102 INFO:tasks.ceph.mds.b:Stopped 2024-03-22T04:20:52.103 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-22T04:20:52.105 INFO:tasks.ceph.mds.d:Stopped 2024-03-22T04:20:52.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-22T04:20:52.145 INFO:tasks.ceph.mds.a:Started 2024-03-22T04:20:52.145 DEBUG:teuthology.parallel:result is None 2024-03-22T04:20:52.146 INFO:tasks.ceph.mds.c:Started 2024-03-22T04:20:52.146 DEBUG:teuthology.parallel:result is None 2024-03-22T04:20:52.152 INFO:tasks.ceph.mds.b:Started 2024-03-22T04:20:52.152 DEBUG:teuthology.parallel:result is None 2024-03-22T04:20:52.154 INFO:tasks.ceph.mds.d:Started 2024-03-22T04:20:52.154 DEBUG:teuthology.parallel:result is None 2024-03-22T04:20:52.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-22T04:20:52.234 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-03-22T04:20:52.234 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:20:52.245 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-03-22T04:20:52.245 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-03-22T04:20:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.299+0000 7f39e3c00640 1 Processor -- start 2024-03-22T04:20:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.299+0000 7f39e3c00640 1 -- start start 2024-03-22T04:20:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e3c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc0b2850 0x7f39dc15c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e3c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39dc06c200 0x7f39dc06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e3c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39dc0b2c30 con 0x7f39dc06bba0 2024-03-22T04:20:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e3c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f39dc15de60 con 0x7f39dc06c200 2024-03-22T04:20:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e3c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc15d200 con 0x7f39dc0b2850 2024-03-22T04:20:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e1c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f39dc06bba0 0x7f39dc15e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37750/0 (socket says 172.21.15.86:37750) 2024-03-22T04:20:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e1c00640 1 -- 172.21.15.86:0/785156931 learned_addr learned my addr 172.21.15.86:0/785156931 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e1200640 1 --2- 172.21.15.86:0/785156931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc0b2850 0x7f39dc15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39d3e00640 1 --2- 172.21.15.86:0/785156931 >> v2:172.21.15.196:3300/0 conn(0x7f39dc06c200 0x7f39dc06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:52.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39dbe00640 1 -- 172.21.15.86:0/785156931 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980143573 0 0) 0x7f39dc0b2c30 con 0x7f39dc06bba0 2024-03-22T04:20:52.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39dbe00640 1 -- 172.21.15.86:0/785156931 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39c0003530 con 0x7f39dc06bba0 2024-03-22T04:20:52.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e1200640 1 -- 172.21.15.86:0/785156931 >> v2:172.21.15.196:3300/0 conn(0x7f39dc06c200 msgr2=0x7f39dc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:52.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e1200640 1 --2- 172.21.15.86:0/785156931 >> v2:172.21.15.196:3300/0 conn(0x7f39dc06c200 0x7f39dc06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e1200640 1 -- 172.21.15.86:0/785156931 >> v1:172.21.15.86:6789/0 conn(0x7f39dc06bba0 legacy=0x7f39dc15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:52.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.303+0000 7f39e1200640 1 -- 172.21.15.86:0/785156931 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc15d4e0 con 0x7f39dc0b2850 2024-03-22T04:20:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e1200640 1 --2- 172.21.15.86:0/785156931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc0b2850 0x7f39dc15c890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0097c0 tx=0x7f39cc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f29dc4d8dd60a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39dbe00640 1 -- 172.21.15.86:0/785156931 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0b2070 con 0x7f39dc0b2850 2024-03-22T04:20:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39dbe00640 1 -- 172.21.15.86:0/785156931 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc0ad640 con 0x7f39dc0b2850 2024-03-22T04:20:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39dbe00640 1 -- 172.21.15.86:0/785156931 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0ada10 con 0x7f39dc0b2850 2024-03-22T04:20:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- 172.21.15.86:0/785156931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc0b2850 msgr2=0x7f39dc15c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 --2- 172.21.15.86:0/785156931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc0b2850 0x7f39dc15c890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0097c0 tx=0x7f39cc01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- 172.21.15.86:0/785156931 shutdown_connections 2024-03-22T04:20:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 --2- 172.21.15.86:0/785156931 >> v2:172.21.15.196:3300/0 conn(0x7f39dc06c200 0x7f39dc06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 --2- 172.21.15.86:0/785156931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc0b2850 0x7f39dc15c890 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- 172.21.15.86:0/785156931 >> 172.21.15.86:0/785156931 conn(0x7f39dc0b6620 msgr2=0x7f39dc0b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- 172.21.15.86:0/785156931 shutdown_connections 2024-03-22T04:20:52.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- 172.21.15.86:0/785156931 wait complete. 2024-03-22T04:20:52.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 Processor -- start 2024-03-22T04:20:52.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- start start 2024-03-22T04:20:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39dc06bba0 0x7f39dc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc06c200 0x7f39dc153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0b2850 0x7f39dc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc15f210 con 0x7f39dc06bba0 2024-03-22T04:20:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc15cfe0 con 0x7f39dc0b2850 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e3c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc15f390 con 0x7f39dc06c200 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.307+0000 7f39e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39dc06bba0 0x7f39dc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39dc06bba0 0x7f39dc15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39610/0 (socket says 172.21.15.86:39610) 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1200640 1 -- 172.21.15.86:0/2991336356 learned_addr learned my addr 172.21.15.86:0/2991336356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc06c200 0x7f39dc153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1c00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0b2850 0x7f39dc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1200640 1 -- 172.21.15.86:0/2991336356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc06c200 msgr2=0x7f39dc153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1200640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc06c200 0x7f39dc153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1200640 1 -- 172.21.15.86:0/2991336356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0b2850 msgr2=0x7f39dc155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1200640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0b2850 0x7f39dc155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1200640 1 -- 172.21.15.86:0/2991336356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc15bcf0 con 0x7f39dc06bba0 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e1200640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39dc06bba0 0x7f39dc15b6b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f39cc01ab50 tx=0x7f39cc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39d9600640 1 -- 172.21.15.86:0/2991336356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0b2040 con 0x7f39dc06bba0 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39d9600640 1 -- 172.21.15.86:0/2991336356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc0bf070 con 0x7f39dc06bba0 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc152760 con 0x7f39dc06bba0 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39d9600640 1 -- 172.21.15.86:0/2991336356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc006e20 con 0x7f39dc06bba0 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc152d20 con 0x7f39dc06bba0 2024-03-22T04:20:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc152550 con 0x7f39dc06bba0 2024-03-22T04:20:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39d9600640 1 -- 172.21.15.86:0/2991336356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f39cc0b5070 con 0x7f39dc06bba0 2024-03-22T04:20:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39d9600640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f39b8067b20 0x7f39b8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39d9600640 1 -- 172.21.15.86:0/2991336356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5421+0+0 (secure 0 0 0) 0x7f39cc11ffe0 con 0x7f39dc06bba0 2024-03-22T04:20:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.311+0000 7f39d3e00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f39b8067b20 0x7f39b8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:52.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.315+0000 7f39d3e00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f39b8067b20 0x7f39b8069fe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f39dc156c90 tx=0x7f39c4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.315+0000 7f39d9600640 1 -- 172.21.15.86:0/2991336356 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f39cc016090 con 0x7f39dc06bba0 2024-03-22T04:20:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:52.483+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f39dc068650 con 0x7f39dc06bba0 2024-03-22T04:20:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.519+0000 7f39d9600640 1 -- 172.21.15.86:0/2991336356 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) ==== 97+0+0 (secure 0 0 0) 0x7f39cc0f0440 con 0x7f39dc06bba0 2024-03-22T04:20:53.519 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f39b8067b20 msgr2=0x7f39b8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f39b8067b20 0x7f39b8069fe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f39dc156c90 tx=0x7f39c4009190 comp rx=0 tx=0).stop 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39dc06bba0 msgr2=0x7f39dc15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39dc06bba0 0x7f39dc15b6b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f39cc01ab50 tx=0x7f39cc019240 comp rx=0 tx=0).stop 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 shutdown_connections 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f39b8067b20 0x7f39b8069fe0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0b2850 0x7f39dc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc06c200 0x7f39dc153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 --2- 172.21.15.86:0/2991336356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39dc06bba0 0x7f39dc15b6b0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 >> 172.21.15.86:0/2991336356 conn(0x7f39dc0b6620 msgr2=0x7f39dc06fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 shutdown_connections 2024-03-22T04:20:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.523+0000 7f39e3c00640 1 -- 172.21.15.86:0/2991336356 wait complete. 2024-03-22T04:20:53.546 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-22T04:20:53.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c62c00640 1 Processor -- start 2024-03-22T04:20:53.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c62c00640 1 -- start start 2024-03-22T04:20:53.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c62c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c5c147880 0x7f9c5c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:53.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c62c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9c5c06cec0 0x7f9c5c0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:53.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c62c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c15ea90 con 0x7f9c5c0b4890 2024-03-22T04:20:53.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c62c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5c0b6320 con 0x7f9c5c147880 2024-03-22T04:20:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c62c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5c0b4ce0 con 0x7f9c5c06cec0 2024-03-22T04:20:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c60c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9c5c0b4890 0x7f9c5c162080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37762/0 (socket says 172.21.15.86:37762) 2024-03-22T04:20:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c5be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c5c147880 0x7f9c5c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c60c00640 1 -- 172.21.15.86:0/652993531 learned_addr learned my addr 172.21.15.86:0/652993531 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c53400640 1 --2- 172.21.15.86:0/652993531 >> v2:172.21.15.196:3300/0 conn(0x7f9c5c06cec0 0x7f9c5c0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:53.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c5be00640 1 -- 172.21.15.86:0/652993531 >> v2:172.21.15.196:3300/0 conn(0x7f9c5c06cec0 msgr2=0x7f9c5c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:53.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c5be00640 1 --2- 172.21.15.86:0/652993531 >> v2:172.21.15.196:3300/0 conn(0x7f9c5c06cec0 0x7f9c5c0b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c5be00640 1 -- 172.21.15.86:0/652993531 >> v1:172.21.15.86:6789/0 conn(0x7f9c5c0b4890 legacy=0x7f9c5c162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:53.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c5be00640 1 -- 172.21.15.86:0/652993531 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c06d4f0 con 0x7f9c5c147880 2024-03-22T04:20:53.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.687+0000 7f9c5b400640 1 -- 172.21.15.86:0/652993531 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031656360 0 0) 0x7f9c5c15ea90 con 0x7f9c5c0b4890 2024-03-22T04:20:53.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c5be00640 1 --2- 172.21.15.86:0/652993531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c147880 0x7f9c5c06c980 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0097c0 tx=0x7f9c4c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8a057ffeb6db79 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:53.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c5b400640 1 -- 172.21.15.86:0/652993531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b0070 con 0x7f9c5c147880 2024-03-22T04:20:53.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c5b400640 1 -- 172.21.15.86:0/652993531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0ab600 con 0x7f9c5c147880 2024-03-22T04:20:53.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c5b400640 1 -- 172.21.15.86:0/652993531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0ab9d0 con 0x7f9c5c147880 2024-03-22T04:20:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c62c00640 1 -- 172.21.15.86:0/652993531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c147880 msgr2=0x7f9c5c06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c62c00640 1 --2- 172.21.15.86:0/652993531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c147880 0x7f9c5c06c980 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0097c0 tx=0x7f9c4c01c460 comp rx=0 tx=0).stop 2024-03-22T04:20:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c62c00640 1 -- 172.21.15.86:0/652993531 shutdown_connections 2024-03-22T04:20:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c62c00640 1 --2- 172.21.15.86:0/652993531 >> v2:172.21.15.196:3300/0 conn(0x7f9c5c06cec0 0x7f9c5c0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c62c00640 1 --2- 172.21.15.86:0/652993531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c147880 0x7f9c5c06c980 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c62c00640 1 -- 172.21.15.86:0/652993531 >> 172.21.15.86:0/652993531 conn(0x7f9c5c06ba60 msgr2=0x7f9c5c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c62c00640 1 -- 172.21.15.86:0/652993531 shutdown_connections 2024-03-22T04:20:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.691+0000 7f9c62c00640 1 -- 172.21.15.86:0/652993531 wait complete. 2024-03-22T04:20:53.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 Processor -- start 2024-03-22T04:20:53.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 -- start start 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c06cec0 0x7f9c5c13f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5c0b4890 0x7f9c5c157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5c147880 0x7f9c5c145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c16b420 con 0x7f9c5c06cec0 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1480a0 con 0x7f9c5c0b4890 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c16b5a0 con 0x7f9c5c147880 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c06cec0 0x7f9c5c13f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c06cec0 0x7f9c5c13f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39640/0 (socket says 172.21.15.86:39640) 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 -- 172.21.15.86:0/1201856902 learned_addr learned my addr 172.21.15.86:0/1201856902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c53400640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5c0b4890 0x7f9c5c157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c60c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5c147880 0x7f9c5c145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 -- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5c147880 msgr2=0x7f9c5c145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5c147880 0x7f9c5c145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 -- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5c0b4890 msgr2=0x7f9c5c157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5c0b4890 0x7f9c5c157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 -- 172.21.15.86:0/1201856902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c160e70 con 0x7f9c5c06cec0 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c60c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5c147880 0x7f9c5c145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c5be00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c06cec0 0x7f9c5c13f3b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0ae040 tx=0x7f9c4c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c58c00640 1 -- 172.21.15.86:0/1201856902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b0050 con 0x7f9c5c06cec0 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c58c00640 1 -- 172.21.15.86:0/1201856902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0bc070 con 0x7f9c5c06cec0 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c161110 con 0x7f9c5c06cec0 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c58c00640 1 -- 172.21.15.86:0/1201856902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b5850 con 0x7f9c5c06cec0 2024-03-22T04:20:53.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c1616d0 con 0x7f9c5c06cec0 2024-03-22T04:20:53.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.695+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c161b00 con 0x7f9c5c06cec0 2024-03-22T04:20:53.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.699+0000 7f9c58c00640 1 -- 172.21.15.86:0/1201856902 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9c4c0b59f0 con 0x7f9c5c06cec0 2024-03-22T04:20:53.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.699+0000 7f9c58c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c38067b20 0x7f9c38069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:53.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.699+0000 7f9c58c00640 1 -- 172.21.15.86:0/1201856902 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5249+0+0 (secure 0 0 0) 0x7f9c4c120540 con 0x7f9c5c06cec0 2024-03-22T04:20:53.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.699+0000 7f9c53400640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c38067b20 0x7f9c38069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:53.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.699+0000 7f9c53400640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c38067b20 0x7f9c38069fe0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c140050 tx=0x7f9c44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:53.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.703+0000 7f9c58c00640 1 -- 172.21.15.86:0/1201856902 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9c4c016050 con 0x7f9c5c06cec0 2024-03-22T04:20:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.911+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9c5c072220 con 0x7f9c5c06cec0 2024-03-22T04:20:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.911+0000 7f9c58c00640 1 -- 172.21.15.86:0/1201856902 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f9c4c0f1190 con 0x7f9c5c06cec0 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lugRjGBAAdauJ09BlLhlX2JWLMgZpMA==", 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lqSs2JBAAluj9GeBpU8LH/kQNUwxLkw==", 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l+mJoHxAAfBBgAdzic4gJYZgiilzC4Q==", 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:20:53.916 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lVXRZKxAAL6CcsiT9j7bqcdbYMVvtDg==", 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.917 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB9BP1lQARFCRAAZqQgDH/zIwluK5NBFl+4zA==", 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCABP1l6EoyEBAAlMuOYlrC5w8iS3LteQK5zA==", 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:20:53.918 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCDBP1laDGVFRAAgtv92haG1WJFyynItZeAOg==", 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCGBP1l4BxpGxAApwcqq8ECE2EuBkRzbD3FqA==", 2024-03-22T04:20:53.919 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCMBP1lrQg5JBAA960ur3/tgi3Aq4+PGdJChQ==", 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.920 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCPBP1lOIl+KRAA4WW1KnodRaclilsYq2Jnow==", 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-03-22T04:20:53.921 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCSBP1lADO6LhAAYHZpWnR6dFis9d1hcnrTPg==", 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCVBP1lN+cTNBAAIuoV60bkT8gtChWYHMg0rg==", 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA==", 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.923 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lW0ZtOBAAyw1cxIuB3zoIvSbN918HPg==", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lHn7WNBAARVd5bTvoHVxJFQY1OGRG4w==", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:20:53.924 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB6BP1l9dnPAhAAth6pPdig8PcsvmFutE+61w==", 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-03-22T04:20:53.925 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB4BP1lvLoQORAADwVEr78V9aUc1YblO47Gaw==", 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1lG/xiFBAAdwZWCv88DAmOurV7jcuy9g==", 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.926 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQB5BP1llQy/EBAAf9f6Q0meHICAHUekbmxwRQ==", 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-22T04:20:53.927 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:20:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c38067b20 msgr2=0x7f9c38069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c38067b20 0x7f9c38069fe0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c140050 tx=0x7f9c44008040 comp rx=0 tx=0).stop 2024-03-22T04:20:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c06cec0 msgr2=0x7f9c5c13f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c06cec0 0x7f9c5c13f3b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0ae040 tx=0x7f9c4c01aca0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 shutdown_connections 2024-03-22T04:20:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9c38067b20 0x7f9c38069fe0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c5c0b4890 0x7f9c5c157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c5c147880 0x7f9c5c145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 --2- 172.21.15.86:0/1201856902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c06cec0 0x7f9c5c13f3b0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 >> 172.21.15.86:0/1201856902 conn(0x7f9c5c06ba60 msgr2=0x7f9c5c15c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 shutdown_connections 2024-03-22T04:20:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:53.915+0000 7f9c62c00640 1 -- 172.21.15.86:0/1201856902 wait complete. 2024-03-22T04:20:53.936 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:20:54.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc14a800640 1 Processor -- start 2024-03-22T04:20:54.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc14a800640 1 -- start start 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc14a800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc14415b3f0 0x7fc1440729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc14a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc1440ae540 0x7fc144072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc14a800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc144071920 con 0x7fc1440cc3e0 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc14a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc1440716d0 con 0x7fc1440ae540 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc14a800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc14406d990 con 0x7fc14415b3f0 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc143e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc1440ae540 0x7fc144072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc143e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc1440ae540 0x7fc144072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39644/0 (socket says 172.21.15.86:39644) 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc143e00640 1 -- 172.21.15.86:0/2449068452 learned_addr learned my addr 172.21.15.86:0/2449068452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:54.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc143400640 1 --2- 172.21.15.86:0/2449068452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc14415b3f0 0x7fc1440729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc143e00640 1 -- 172.21.15.86:0/2449068452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc14415b3f0 msgr2=0x7fc1440729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:54.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc143e00640 1 --2- 172.21.15.86:0/2449068452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc14415b3f0 0x7fc1440729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc143e00640 1 -- 172.21.15.86:0/2449068452 >> v1:172.21.15.196:6789/0 conn(0x7fc1440cc3e0 legacy=0x7fc1440ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.071+0000 7fc143e00640 1 -- 172.21.15.86:0/2449068452 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc144071c00 con 0x7fc1440ae540 2024-03-22T04:20:54.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc143e00640 1 --2- 172.21.15.86:0/2449068452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1440ae540 0x7fc144072f00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009e50 tx=0x7fc13c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cae86ff59b4a17 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc142000640 1 -- 172.21.15.86:0/2449068452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0a5750 con 0x7fc1440ae540 2024-03-22T04:20:54.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc142000640 1 -- 172.21.15.86:0/2449068452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13c0a58f0 con 0x7fc1440ae540 2024-03-22T04:20:54.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc142000640 1 -- 172.21.15.86:0/2449068452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0a5cc0 con 0x7fc1440ae540 2024-03-22T04:20:54.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc14a800640 1 -- 172.21.15.86:0/2449068452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1440ae540 msgr2=0x7fc144072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc14a800640 1 --2- 172.21.15.86:0/2449068452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1440ae540 0x7fc144072f00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009e50 tx=0x7fc13c01c9a0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc14a800640 1 -- 172.21.15.86:0/2449068452 shutdown_connections 2024-03-22T04:20:54.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc14a800640 1 --2- 172.21.15.86:0/2449068452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1440ae540 0x7fc144072f00 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc14a800640 1 --2- 172.21.15.86:0/2449068452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc14415b3f0 0x7fc1440729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc14a800640 1 -- 172.21.15.86:0/2449068452 >> 172.21.15.86:0/2449068452 conn(0x7fc1440b1df0 msgr2=0x7fc1440b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:54.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc14a800640 1 -- 172.21.15.86:0/2449068452 shutdown_connections 2024-03-22T04:20:54.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.075+0000 7fc14a800640 1 -- 172.21.15.86:0/2449068452 wait complete. 2024-03-22T04:20:54.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 Processor -- start 2024-03-22T04:20:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 -- start start 2024-03-22T04:20:54.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1440ae540 0x7fc144099540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1440cc3e0 0x7fc144151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14415b3f0 0x7fc144155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc144168580 con 0x7fc14415b3f0 2024-03-22T04:20:54.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc144168400 con 0x7fc1440cc3e0 2024-03-22T04:20:54.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc144168700 con 0x7fc1440ae540 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14415b3f0 0x7fc144155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14415b3f0 0x7fc144155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39648/0 (socket says 172.21.15.86:39648) 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 -- 172.21.15.86:0/3043137748 learned_addr learned my addr 172.21.15.86:0/3043137748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143400640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1440ae540 0x7fc144099540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc142a00640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1440cc3e0 0x7fc144151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 -- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1440ae540 msgr2=0x7fc144099540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1440ae540 0x7fc144099540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 -- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1440cc3e0 msgr2=0x7fc144151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1440cc3e0 0x7fc144151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 -- 172.21.15.86:0/3043137748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc144153720 con 0x7fc14415b3f0 2024-03-22T04:20:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc142a00640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1440cc3e0 0x7fc144151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc143e00640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14415b3f0 0x7fc144155d50 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0af040 tx=0x7fc13c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc13b400640 1 -- 172.21.15.86:0/3043137748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0bc020 con 0x7fc14415b3f0 2024-03-22T04:20:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc13b400640 1 -- 172.21.15.86:0/3043137748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13c0b8670 con 0x7fc14415b3f0 2024-03-22T04:20:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc144153a00 con 0x7fc14415b3f0 2024-03-22T04:20:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc13b400640 1 -- 172.21.15.86:0/3043137748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c009510 con 0x7fc14415b3f0 2024-03-22T04:20:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc144153fc0 con 0x7fc14415b3f0 2024-03-22T04:20:54.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.079+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc144068650 con 0x7fc14415b3f0 2024-03-22T04:20:54.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.083+0000 7fc13b400640 1 -- 172.21.15.86:0/3043137748 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc13c0bf020 con 0x7fc14415b3f0 2024-03-22T04:20:54.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.083+0000 7fc13b400640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc120067b20 0x7fc120069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.083+0000 7fc13b400640 1 -- 172.21.15.86:0/3043137748 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5249+0+0 (secure 0 0 0) 0x7fc13c120c60 con 0x7fc14415b3f0 2024-03-22T04:20:54.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.083+0000 7fc143400640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc120067b20 0x7fc120069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.083+0000 7fc143400640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc120067b20 0x7fc120069fe0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc134009ac0 tx=0x7fc134009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.087+0000 7fc13b400640 1 -- 172.21.15.86:0/3043137748 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc13c0c6030 con 0x7fc14415b3f0 2024-03-22T04:20:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.247+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1440b0d90 con 0x7fc14415b3f0 2024-03-22T04:20:54.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.247+0000 7fc13b400640 1 -- 172.21.15.86:0/3043137748 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+13174 (secure 0 0 0) 0x7fc13c016620 con 0x7fc14415b3f0 2024-03-22T04:20:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:54.249 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":120,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:53.521100+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:20:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc120067b20 msgr2=0x7fc120069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc120067b20 0x7fc120069fe0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc134009ac0 tx=0x7fc134009190 comp rx=0 tx=0).stop 2024-03-22T04:20:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14415b3f0 msgr2=0x7fc144155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14415b3f0 0x7fc144155d50 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0af040 tx=0x7fc13c019740 comp rx=0 tx=0).stop 2024-03-22T04:20:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 shutdown_connections 2024-03-22T04:20:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc120067b20 0x7fc120069fe0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1440cc3e0 0x7fc144151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14415b3f0 0x7fc144155d50 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 --2- 172.21.15.86:0/3043137748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1440ae540 0x7fc144099540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 >> 172.21.15.86:0/3043137748 conn(0x7fc1440b1df0 msgr2=0x7fc1440b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 shutdown_connections 2024-03-22T04:20:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.251+0000 7fc14a800640 1 -- 172.21.15.86:0/3043137748 wait complete. 2024-03-22T04:20:54.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:20:54.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:20:54.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.407+0000 7f5d51200640 1 Processor -- start 2024-03-22T04:20:54.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d51200640 1 -- start start 2024-03-22T04:20:54.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d51200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5d4c0d0dc0 0x7f5d4c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d51200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c0d1250 con 0x7f5d4c15e460 2024-03-22T04:20:54.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d51200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c0d1530 con 0x7f5d4c0707b0 2024-03-22T04:20:54.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d51200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4c0ccae0 con 0x7f5d4c0d0dc0 2024-03-22T04:20:54.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5d4c15e460 0x7f5d4c0ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37772/0 (socket says 172.21.15.86:37772) 2024-03-22T04:20:54.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4b400640 1 -- 172.21.15.86:0/1406741812 learned_addr learned my addr 172.21.15.86:0/1406741812 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:54.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d42a00640 1 --2- 172.21.15.86:0/1406741812 >> v2:172.21.15.196:3300/0 conn(0x7f5d4c0d0dc0 0x7f5d4c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2752689204 0 0) 0x7f5d4c0d1250 con 0x7f5d4c15e460 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d30003530 con 0x7f5d4c15e460 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3321215386 0 0) 0x7f5d30003530 con 0x7f5d4c15e460 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 >> v2:172.21.15.196:3300/0 conn(0x7f5d4c0d0dc0 msgr2=0x7f5d4c0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 --2- 172.21.15.86:0/1406741812 >> v2:172.21.15.196:3300/0 conn(0x7f5d4c0d0dc0 0x7f5d4c0cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 >> v1:172.21.15.196:6789/0 conn(0x7f5d4c0707b0 legacy=0x7f5d4c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c0cc770 con 0x7f5d4c15e460 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f5d3c001680 con 0x7f5d4c15e460 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d42a00640 1 --2- 172.21.15.86:0/1406741812 >> v2:172.21.15.196:3300/0 conn(0x7f5d4c0d0dc0 0x7f5d4c0cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 >> v1:172.21.15.86:6789/0 conn(0x7f5d4c15e460 legacy=0x7f5d4c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d30003a10 0x7f5d30003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d300051d0 0x7f5d3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0ccae0 con 0x7f5d30004810 2024-03-22T04:20:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3000c0d0 con 0x7f5d30003a10 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3000c2d0 con 0x7f5d300051d0 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d42a00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4b400640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d30003a10 0x7f5d30003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4be00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d300051d0 0x7f5d3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d42a00640 1 -- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d300051d0 msgr2=0x7f5d3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d42a00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d300051d0 0x7f5d3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d42a00640 1 -- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d30003a10 msgr2=0x7f5d30003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d42a00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d30003a10 0x7f5d30003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d42a00640 1 -- 172.21.15.86:0/1406741812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d30003530 con 0x7f5d30004810 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4be00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d300051d0 0x7f5d3000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.411+0000 7f5d4be00640 1 -- 172.21.15.86:0/1406741812 reap_dead start 2024-03-22T04:20:54.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d42a00640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d30004c90 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009cb0 tx=0x7f5d3401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b530009f703493a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340b2070 con 0x7f5d30004810 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d340a57c0 con 0x7f5d30004810 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d4aa00640 1 -- 172.21.15.86:0/1406741812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34009880 con 0x7f5d30004810 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d51200640 1 -- 172.21.15.86:0/1406741812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 msgr2=0x7f5d30004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d51200640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d30004c90 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009cb0 tx=0x7f5d3401cc10 comp rx=0 tx=0).stop 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d51200640 1 -- 172.21.15.86:0/1406741812 shutdown_connections 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d51200640 1 --2- 172.21.15.86:0/1406741812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d30004c90 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d51200640 1 -- 172.21.15.86:0/1406741812 >> 172.21.15.86:0/1406741812 conn(0x7f5d4c0d1f80 msgr2=0x7f5d4c0d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d51200640 1 -- 172.21.15.86:0/1406741812 shutdown_connections 2024-03-22T04:20:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.415+0000 7f5d51200640 1 -- 172.21.15.86:0/1406741812 wait complete. 2024-03-22T04:20:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 Processor -- start 2024-03-22T04:20:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 -- start start 2024-03-22T04:20:54.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d4c153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4c1576b0 0x7f5d4c160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4c1611b0 0x7f5d4c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0ce980 con 0x7f5d30004810 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0ce800 con 0x7f5d4c1576b0 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c0ceb00 con 0x7f5d4c1611b0 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d4c153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d4c153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39674/0 (socket says 172.21.15.86:39674) 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 -- 172.21.15.86:0/2299932467 learned_addr learned my addr 172.21.15.86:0/2299932467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 -- 172.21.15.86:0/2299932467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4c1611b0 msgr2=0x7f5d4c1566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4c1611b0 0x7f5d4c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 -- 172.21.15.86:0/2299932467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4c1576b0 msgr2=0x7f5d4c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4c1576b0 0x7f5d4c160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 -- 172.21.15.86:0/2299932467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c0c6970 con 0x7f5d30004810 2024-03-22T04:20:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d4b400640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d4c153640 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c001c10 tx=0x7f5d3c012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d43e00640 1 -- 172.21.15.86:0/2299932467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c01c070 con 0x7f5d30004810 2024-03-22T04:20:54.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d43e00640 1 -- 172.21.15.86:0/2299932467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3c014810 con 0x7f5d30004810 2024-03-22T04:20:54.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d43e00640 1 -- 172.21.15.86:0/2299932467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c014c00 con 0x7f5d30004810 2024-03-22T04:20:54.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c0c6c50 con 0x7f5d30004810 2024-03-22T04:20:54.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c0c7210 con 0x7f5d30004810 2024-03-22T04:20:54.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.419+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c04f900 con 0x7f5d30004810 2024-03-22T04:20:54.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.423+0000 7f5d43e00640 1 -- 172.21.15.86:0/2299932467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5d3c0153e0 con 0x7f5d30004810 2024-03-22T04:20:54.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.423+0000 7f5d43e00640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d28067b20 0x7f5d28069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.423+0000 7f5d43e00640 1 -- 172.21.15.86:0/2299932467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5249+0+0 (secure 0 0 0) 0x7f5d3c0888f0 con 0x7f5d30004810 2024-03-22T04:20:54.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.423+0000 7f5d42a00640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d28067b20 0x7f5d28069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.423+0000 7f5d42a00640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d28067b20 0x7f5d28069fe0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5d340af000 tx=0x7f5d34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.427+0000 7f5d43e00640 1 -- 172.21.15.86:0/2299932467 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5d3c0594b0 con 0x7f5d30004810 2024-03-22T04:20:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.583+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d4c161c90 con 0x7f5d30004810 2024-03-22T04:20:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.583+0000 7f5d43e00640 1 -- 172.21.15.86:0/2299932467 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) ==== 93+0+10 (secure 0 0 0) 0x7f5d3c058db0 con 0x7f5d30004810 2024-03-22T04:20:54.586 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d28067b20 msgr2=0x7f5d28069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d28067b20 0x7f5d28069fe0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5d340af000 tx=0x7f5d34008040 comp rx=0 tx=0).stop 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 msgr2=0x7f5d4c153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d4c153640 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c001c10 tx=0x7f5d3c012920 comp rx=0 tx=0).stop 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 shutdown_connections 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5d28067b20 0x7f5d28069fe0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d4c1576b0 0x7f5d4c160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d4c1611b0 0x7f5d4c1566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 --2- 172.21.15.86:0/2299932467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d30004810 0x7f5d4c153640 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 >> 172.21.15.86:0/2299932467 conn(0x7f5d4c0d1f80 msgr2=0x7f5d4c0afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:54.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 shutdown_connections 2024-03-22T04:20:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.587+0000 7f5d51200640 1 -- 172.21.15.86:0/2299932467 wait complete. 2024-03-22T04:20:54.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-22T04:20:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e6c00640 1 Processor -- start 2024-03-22T04:20:54.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e6c00640 1 -- start start 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e6c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99e00ad330 0x7f99e015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e6c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e015c7c0 con 0x7f99e006ef40 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e6c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e006dea0 con 0x7f99e00ad710 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e6c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f99e006e180 con 0x7f99e00ad330 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99d7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f99e006ef40 0x7f99e013fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37788/0 (socket says 172.21.15.86:37788) 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99d7400640 1 -- 172.21.15.86:0/2142133450 learned_addr learned my addr 172.21.15.86:0/2142133450 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99dfe00640 1 --2- 172.21.15.86:0/2142133450 >> v2:172.21.15.196:3300/0 conn(0x7f99e00ad330 0x7f99e015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266813719 0 0) 0x7f99e015c7c0 con 0x7f99e006ef40 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99c4003530 con 0x7f99e006ef40 2024-03-22T04:20:54.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1948940034 0 0) 0x7f99c4003530 con 0x7f99e006ef40 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 >> v2:172.21.15.196:3300/0 conn(0x7f99e00ad330 msgr2=0x7f99e015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 --2- 172.21.15.86:0/2142133450 >> v2:172.21.15.196:3300/0 conn(0x7f99e00ad330 0x7f99e015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 >> v1:172.21.15.196:6789/0 conn(0x7f99e00ad710 legacy=0x7f99e01402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e006e460 con 0x7f99e006ef40 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f99c80016a0 con 0x7f99e006ef40 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 >> v1:172.21.15.86:6789/0 conn(0x7f99e006ef40 legacy=0x7f99e013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99c4003a10 0x7f99c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c4004810 0x7f99c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99e006e180 con 0x7f99c40051d0 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99c400c0d0 con 0x7f99c4004810 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99c400c2d0 con 0x7f99c4003a10 2024-03-22T04:20:54.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e4c00640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99d7400640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c4004810 0x7f99c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99dfe00640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99c4003a10 0x7f99c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e4c00640 1 -- 172.21.15.86:0/2142133450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99c4003a10 msgr2=0x7f99c4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e4c00640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99c4003a10 0x7f99c4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e4c00640 1 -- 172.21.15.86:0/2142133450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c4004810 msgr2=0x7f99c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e4c00640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99c4004810 0x7f99c4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.747+0000 7f99e4c00640 1 -- 172.21.15.86:0/2142133450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c4003530 con 0x7f99c40051d0 2024-03-22T04:20:54.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e4c00640 1 -- 172.21.15.86:0/2142133450 reap_dead start 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e4c00640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99c400b9d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f99d8001e70 tx=0x7f99d801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561d6083ece2d116 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8001a90 con 0x7f99c40051d0 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d8003ba0 con 0x7f99c40051d0 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99df400640 1 -- 172.21.15.86:0/2142133450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8005020 con 0x7f99c40051d0 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e6c00640 1 -- 172.21.15.86:0/2142133450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 msgr2=0x7f99c400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e6c00640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99c400b9d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f99d8001e70 tx=0x7f99d801c780 comp rx=0 tx=0).stop 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e6c00640 1 -- 172.21.15.86:0/2142133450 shutdown_connections 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e6c00640 1 --2- 172.21.15.86:0/2142133450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99c400b9d0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e6c00640 1 -- 172.21.15.86:0/2142133450 >> 172.21.15.86:0/2142133450 conn(0x7f99e00b1df0 msgr2=0x7f99e00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e6c00640 1 -- 172.21.15.86:0/2142133450 shutdown_connections 2024-03-22T04:20:54.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.751+0000 7f99e6c00640 1 -- 172.21.15.86:0/2142133450 wait complete. 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 Processor -- start 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 -- start start 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99e0158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e00c7360 0x7f99e00c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e00c8020 0x7f99e0099590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99e015d4a0 con 0x7f99c40051d0 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99e006ea20 con 0x7f99e00c8020 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99e015d620 con 0x7f99e00c7360 2024-03-22T04:20:54.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99e0158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99e0158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39688/0 (socket says 172.21.15.86:39688) 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 -- 172.21.15.86:0/2863004948 learned_addr learned my addr 172.21.15.86:0/2863004948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 -- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e00c7360 msgr2=0x7f99e00c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e4c00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e00c8020 0x7f99e0099590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99d7400640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e00c7360 0x7f99e00c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e00c7360 0x7f99e00c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 -- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e00c8020 msgr2=0x7f99e0099590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e00c8020 0x7f99e0099590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 -- 172.21.15.86:0/2863004948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e00c62c0 con 0x7f99c40051d0 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e4c00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e00c8020 0x7f99e0099590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:20:54.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99d7400640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e00c7360 0x7f99e00c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:20:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dfe00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99e0158a80 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f99d0011560 tx=0x7f99d0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dcc00640 1 -- 172.21.15.86:0/2863004948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0015930 con 0x7f99c40051d0 2024-03-22T04:20:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dcc00640 1 -- 172.21.15.86:0/2863004948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d0015ad0 con 0x7f99c40051d0 2024-03-22T04:20:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99dcc00640 1 -- 172.21.15.86:0/2863004948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d000f460 con 0x7f99c40051d0 2024-03-22T04:20:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e00c65a0 con 0x7f99c40051d0 2024-03-22T04:20:54.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.755+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e00c6b60 con 0x7f99c40051d0 2024-03-22T04:20:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.759+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e00c7740 con 0x7f99c40051d0 2024-03-22T04:20:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.759+0000 7f99dcc00640 1 -- 172.21.15.86:0/2863004948 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f99d000f600 con 0x7f99c40051d0 2024-03-22T04:20:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.759+0000 7f99dcc00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99bc067b50 0x7f99bc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.759+0000 7f99dcc00640 1 -- 172.21.15.86:0/2863004948 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5249+0+0 (secure 0 0 0) 0x7f99d0086690 con 0x7f99c40051d0 2024-03-22T04:20:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.759+0000 7f99d7400640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99bc067b50 0x7f99bc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.759+0000 7f99d7400640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99bc067b50 0x7f99bc06a010 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f99e00b3e90 tx=0x7f99c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:54.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.763+0000 7f99dcc00640 1 -- 172.21.15.86:0/2863004948 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f99e00c7740 con 0x7f99c40051d0 2024-03-22T04:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.919+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f99e0068650 con 0x7f99c40051d0 2024-03-22T04:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99dcc00640 1 -- 172.21.15.86:0/2863004948 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) ==== 84+0+65 (secure 0 0 0) 0x7f99d0019030 con 0x7f99c40051d0 2024-03-22T04:20:54.922 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:54.923 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-22T04:20:54.923 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-22T04:20:54.923 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-22T04:20:54.923 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-22T04:20:54.923 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-22T04:20:54.923 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-22T04:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99bc067b50 msgr2=0x7f99bc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99bc067b50 0x7f99bc06a010 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f99e00b3e90 tx=0x7f99c8009040 comp rx=0 tx=0).stop 2024-03-22T04:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 msgr2=0x7f99e0158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99e0158a80 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f99d0011560 tx=0x7f99d0011a20 comp rx=0 tx=0).stop 2024-03-22T04:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 shutdown_connections 2024-03-22T04:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f99bc067b50 0x7f99bc06a010 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e00c8020 0x7f99e0099590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99e00c7360 0x7f99e00c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 --2- 172.21.15.86:0/2863004948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c40051d0 0x7f99e0158a80 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 >> 172.21.15.86:0/2863004948 conn(0x7f99e00b1df0 msgr2=0x7f99e015ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 shutdown_connections 2024-03-22T04:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:54.923+0000 7f99e6c00640 1 -- 172.21.15.86:0/2863004948 wait complete. 2024-03-22T04:20:54.943 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-22T04:20:54.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-22T04:20:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee9d200640 1 Processor -- start 2024-03-22T04:20:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee9d200640 1 -- start start 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee9d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fee9806df30 0x7fee980b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee9d200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fee980ad8e0 0x7fee980b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee9d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee980adea0 con 0x7fee98148030 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee9d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fee980ae180 con 0x7fee9806df30 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee9d200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fee980ae460 con 0x7fee980ad8e0 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee95400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fee9806df30 0x7fee980b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee95e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee98148030 0x7fee9815eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37800/0 (socket says 172.21.15.86:37800) 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee95e00640 1 -- 172.21.15.86:0/3799935703 learned_addr learned my addr 172.21.15.86:0/3799935703 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:55.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee96800640 1 --2- 172.21.15.86:0/3799935703 >> v2:172.21.15.196:3300/0 conn(0x7fee980ad8e0 0x7fee980b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:55.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee95400640 1 -- 172.21.15.86:0/3799935703 >> v2:172.21.15.196:3300/0 conn(0x7fee980ad8e0 msgr2=0x7fee980b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:55.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee95400640 1 --2- 172.21.15.86:0/3799935703 >> v2:172.21.15.196:3300/0 conn(0x7fee980ad8e0 0x7fee980b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:55.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee95400640 1 -- 172.21.15.86:0/3799935703 >> v1:172.21.15.86:6789/0 conn(0x7fee98148030 legacy=0x7fee9815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:55.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee95400640 1 -- 172.21.15.86:0/3799935703 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee980ae740 con 0x7fee9806df30 2024-03-22T04:20:55.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee94a00640 1 -- 172.21.15.86:0/3799935703 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542611026 0 0) 0x7fee980adea0 con 0x7fee98148030 2024-03-22T04:20:55.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.083+0000 7fee96800640 1 --2- 172.21.15.86:0/3799935703 >> v2:172.21.15.196:3300/0 conn(0x7fee980ad8e0 0x7fee980b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:55.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee95400640 1 --2- 172.21.15.86:0/3799935703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee980b3840 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fee9806f0d0 tx=0x7fee8401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1261b7685119d237 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:55.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee94a00640 1 -- 172.21.15.86:0/3799935703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840b0070 con 0x7fee9806df30 2024-03-22T04:20:55.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee94a00640 1 -- 172.21.15.86:0/3799935703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee840ab600 con 0x7fee9806df30 2024-03-22T04:20:55.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee94a00640 1 -- 172.21.15.86:0/3799935703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840ab9d0 con 0x7fee9806df30 2024-03-22T04:20:55.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 -- 172.21.15.86:0/3799935703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 msgr2=0x7fee980b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:55.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 --2- 172.21.15.86:0/3799935703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee980b3840 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fee9806f0d0 tx=0x7fee8401c480 comp rx=0 tx=0).stop 2024-03-22T04:20:55.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 -- 172.21.15.86:0/3799935703 shutdown_connections 2024-03-22T04:20:55.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 --2- 172.21.15.86:0/3799935703 >> v2:172.21.15.196:3300/0 conn(0x7fee980ad8e0 0x7fee980b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:55.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 --2- 172.21.15.86:0/3799935703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee980b3840 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:55.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 -- 172.21.15.86:0/3799935703 >> 172.21.15.86:0/3799935703 conn(0x7fee9806b660 msgr2=0x7fee9806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:55.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 -- 172.21.15.86:0/3799935703 shutdown_connections 2024-03-22T04:20:55.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 -- 172.21.15.86:0/3799935703 wait complete. 2024-03-22T04:20:55.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.087+0000 7fee9d200640 1 Processor -- start 2024-03-22T04:20:55.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 -- start start 2024-03-22T04:20:55.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee9815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee980ad8e0 0x7fee9815d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee98148030 0x7fee981603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee9816b070 con 0x7fee9806df30 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee980af960 con 0x7fee980ad8e0 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fee9816b1f0 con 0x7fee98148030 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee9815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee9815bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39698/0 (socket says 172.21.15.86:39698) 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 -- 172.21.15.86:0/1286729415 learned_addr learned my addr 172.21.15.86:0/1286729415 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee96800640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee98148030 0x7fee981603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95400640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee980ad8e0 0x7fee9815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 -- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee98148030 msgr2=0x7fee981603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee98148030 0x7fee981603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:55.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 -- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee980ad8e0 msgr2=0x7fee9815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee980ad8e0 0x7fee9815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 -- 172.21.15.86:0/1286729415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee98144330 con 0x7fee9806df30 2024-03-22T04:20:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee96800640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee98148030 0x7fee981603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:20:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95400640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee980ad8e0 0x7fee9815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:20:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee95e00640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee9815bc40 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c480 tx=0x7fee8800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee8e000640 1 -- 172.21.15.86:0/1286729415 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88018070 con 0x7fee9806df30 2024-03-22T04:20:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee8e000640 1 -- 172.21.15.86:0/1286729415 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88007500 con 0x7fee9806df30 2024-03-22T04:20:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee8e000640 1 -- 172.21.15.86:0/1286729415 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88004a30 con 0x7fee9806df30 2024-03-22T04:20:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee98144610 con 0x7fee9806df30 2024-03-22T04:20:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981450f0 con 0x7fee9806df30 2024-03-22T04:20:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.091+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee98074580 con 0x7fee9806df30 2024-03-22T04:20:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.095+0000 7fee8e000640 1 -- 172.21.15.86:0/1286729415 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fee88004bd0 con 0x7fee9806df30 2024-03-22T04:20:55.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.095+0000 7fee8e000640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee78067b20 0x7fee78069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:55.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.095+0000 7fee8e000640 1 -- 172.21.15.86:0/1286729415 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5249+0+0 (secure 0 0 0) 0x7fee88084ff0 con 0x7fee9806df30 2024-03-22T04:20:55.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.095+0000 7fee95400640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee78067b20 0x7fee78069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:55.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.095+0000 7fee95400640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee78067b20 0x7fee78069fe0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fee840ad000 tx=0x7fee84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:55.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.099+0000 7fee8e000640 1 -- 172.21.15.86:0/1286729415 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fee88055c10 con 0x7fee9806df30 2024-03-22T04:20:55.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:55.255+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fee9815f520 con 0x7fee9806df30 2024-03-22T04:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.167+0000 7fee8e000640 1 -- 172.21.15.86:0/1286729415 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) ==== 138+0+0 (secure 0 0 0) 0x7fee88055510 con 0x7fee9806df30 2024-03-22T04:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-22T04:20:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee78067b20 msgr2=0x7fee78069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee78067b20 0x7fee78069fe0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fee840ad000 tx=0x7fee84008040 comp rx=0 tx=0).stop 2024-03-22T04:20:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 msgr2=0x7fee9815bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee9815bc40 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c480 tx=0x7fee8800c940 comp rx=0 tx=0).stop 2024-03-22T04:20:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 shutdown_connections 2024-03-22T04:20:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee78067b20 0x7fee78069fe0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee980ad8e0 0x7fee9815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee98148030 0x7fee981603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 --2- 172.21.15.86:0/1286729415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9806df30 0x7fee9815bc40 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 >> 172.21.15.86:0/1286729415 conn(0x7fee9806b660 msgr2=0x7fee98071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 shutdown_connections 2024-03-22T04:20:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.171+0000 7fee9d200640 1 -- 172.21.15.86:0/1286729415 wait complete. 2024-03-22T04:20:56.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-22T04:20:56.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feebb200640 1 Processor -- start 2024-03-22T04:20:56.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feebb200640 1 -- start start 2024-03-22T04:20:56.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feebb200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feebb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feebc0d07d0 0x7feebc06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feebb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feebc0cee60 con 0x7feebc0ce940 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feebb200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feebc0cf140 con 0x7feebc0d07d0 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feebb200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feebc0cf420 con 0x7feebc15e260 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeba800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feebc0d07d0 0x7feebc06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeb9400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feebc0ce940 0x7feebc0d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49798/0 (socket says 172.21.15.86:49798) 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeb9400640 1 -- 172.21.15.86:0/570254959 learned_addr learned my addr 172.21.15.86:0/570254959 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeb9e00640 1 --2- 172.21.15.86:0/570254959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeba800640 1 -- 172.21.15.86:0/570254959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 msgr2=0x7feebc15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeba800640 1 --2- 172.21.15.86:0/570254959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeba800640 1 -- 172.21.15.86:0/570254959 >> v1:172.21.15.86:6789/0 conn(0x7feebc0ce940 legacy=0x7feebc0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeba800640 1 -- 172.21.15.86:0/570254959 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc0cf700 con 0x7feebc0d07d0 2024-03-22T04:20:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.347+0000 7feeb9e00640 1 --2- 172.21.15.86:0/570254959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feeba800640 1 --2- 172.21.15.86:0/570254959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0d07d0 0x7feebc06d160 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb401c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dbc877171d8cd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feeb8a00640 1 -- 172.21.15.86:0/570254959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40b2070 con 0x7feebc0d07d0 2024-03-22T04:20:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feeb8a00640 1 -- 172.21.15.86:0/570254959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb40a5820 con 0x7feebc0d07d0 2024-03-22T04:20:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feeb8a00640 1 -- 172.21.15.86:0/570254959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40a5bf0 con 0x7feebc0d07d0 2024-03-22T04:20:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 -- 172.21.15.86:0/570254959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0d07d0 msgr2=0x7feebc06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 --2- 172.21.15.86:0/570254959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0d07d0 0x7feebc06d160 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb401c800 comp rx=0 tx=0).stop 2024-03-22T04:20:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 -- 172.21.15.86:0/570254959 shutdown_connections 2024-03-22T04:20:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 --2- 172.21.15.86:0/570254959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0d07d0 0x7feebc06d160 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 --2- 172.21.15.86:0/570254959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 -- 172.21.15.86:0/570254959 >> 172.21.15.86:0/570254959 conn(0x7feebc0d1fb0 msgr2=0x7feebc0d23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 -- 172.21.15.86:0/570254959 shutdown_connections 2024-03-22T04:20:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 -- 172.21.15.86:0/570254959 wait complete. 2024-03-22T04:20:56.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.351+0000 7feebb200640 1 Processor -- start 2024-03-22T04:20:56.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 -- start start 2024-03-22T04:20:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0ce940 0x7feebc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feebc0d07d0 0x7feebc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0cc3a0 con 0x7feebc0ce940 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0cc220 con 0x7feebc0d07d0 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feebc15eb80 con 0x7feebc15e260 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0ce940 0x7feebc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0ce940 0x7feebc0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33850/0 (socket says 172.21.15.86:33850) 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 -- 172.21.15.86:0/3142162947 learned_addr learned my addr 172.21.15.86:0/3142162947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9400640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feebc0d07d0 0x7feebc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeba800640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 -- 172.21.15.86:0/3142162947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 msgr2=0x7feebc1533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc1533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 -- 172.21.15.86:0/3142162947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feebc0d07d0 msgr2=0x7feebc0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feebc0d07d0 0x7feebc0b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 -- 172.21.15.86:0/3142162947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc0af530 con 0x7feebc0ce940 2024-03-22T04:20:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb9e00640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0ce940 0x7feebc0aece0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7feeac00e810 tx=0x7feeac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb2000640 1 -- 172.21.15.86:0/3142162947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac017070 con 0x7feebc0ce940 2024-03-22T04:20:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb2000640 1 -- 172.21.15.86:0/3142162947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeac00c8d0 con 0x7feebc0ce940 2024-03-22T04:20:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feeb2000640 1 -- 172.21.15.86:0/3142162947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac00f040 con 0x7feebc0ce940 2024-03-22T04:20:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc0af810 con 0x7feebc0ce940 2024-03-22T04:20:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc0afdd0 con 0x7feebc0ce940 2024-03-22T04:20:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.355+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc074470 con 0x7feebc0ce940 2024-03-22T04:20:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.359+0000 7feeb2000640 1 -- 172.21.15.86:0/3142162947 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7feeac01a070 con 0x7feebc0ce940 2024-03-22T04:20:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.359+0000 7feeb2000640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee9c067b20 0x7fee9c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.359+0000 7feeb2000640 1 -- 172.21.15.86:0/3142162947 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5619+0+0 (secure 0 0 0) 0x7feeac086070 con 0x7feebc0ce940 2024-03-22T04:20:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.359+0000 7feeb9400640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee9c067b20 0x7fee9c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.359+0000 7feeb9400640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee9c067b20 0x7fee9c069fe0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feebc161aa0 tx=0x7feea800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.363+0000 7feeb2000640 1 -- 172.21.15.86:0/3142162947 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7feeac056a20 con 0x7feebc0ce940 2024-03-22T04:20:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:56.515+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7feebc0cdd60 con 0x7feebc0ce940 2024-03-22T04:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.171+0000 7feeb2000640 1 -- 172.21.15.86:0/3142162947 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) ==== 170+0+0 (secure 0 0 0) 0x7feeac056320 con 0x7feebc0ce940 2024-03-22T04:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-22T04:20:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee9c067b20 msgr2=0x7fee9c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee9c067b20 0x7fee9c069fe0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feebc161aa0 tx=0x7feea800b040 comp rx=0 tx=0).stop 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0ce940 msgr2=0x7feebc0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0ce940 0x7feebc0aece0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7feeac00e810 tx=0x7feeac00ecd0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 shutdown_connections 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fee9c067b20 0x7fee9c069fe0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feebc0d07d0 0x7feebc0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feebc15e260 0x7feebc1533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 --2- 172.21.15.86:0/3142162947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feebc0ce940 0x7feebc0aece0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 >> 172.21.15.86:0/3142162947 conn(0x7feebc0d1fb0 msgr2=0x7feebc15d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 shutdown_connections 2024-03-22T04:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.175+0000 7feebb200640 1 -- 172.21.15.86:0/3142162947 wait complete. 2024-03-22T04:20:57.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-22T04:20:57.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.351+0000 7f8471c00640 1 Processor -- start 2024-03-22T04:20:57.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.351+0000 7f8471c00640 1 -- start start 2024-03-22T04:20:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f8471c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c06e7e0 0x7f846c06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f8471c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846c15c770 con 0x7f846c072650 2024-03-22T04:20:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f8471c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846c0b29a0 con 0x7f846c072ad0 2024-03-22T04:20:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f8471c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f846c071350 con 0x7f846c06e7e0 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f8462000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f846c072650 0x7f846c0b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49812/0 (socket says 172.21.15.86:49812) 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f8462000640 1 -- 172.21.15.86:0/4047952647 learned_addr learned my addr 172.21.15.86:0/4047952647 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846aa00640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c06e7e0 0x7f846c06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3846153235 0 0) 0x7f8454001120 con 0x7f846c072650 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8450005740 con 0x7f846c072650 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093474219 0 0) 0x7f846c0b29a0 con 0x7f846c072ad0 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8454001120 con 0x7f846c072ad0 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 127285795 0 0) 0x7f8450005740 con 0x7f846c072650 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c06e7e0 msgr2=0x7f846c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c06e7e0 0x7f846c06ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 >> v1:172.21.15.196:6789/0 conn(0x7f846c072ad0 legacy=0x7f846c0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c06f100 con 0x7f846c072650 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f8454001850 con 0x7f846c072650 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 >> v1:172.21.15.86:6789/0 conn(0x7f846c072650 legacy=0x7f846c0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f8450006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c06e7e0 0x7f8450006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8450006f60 0x7f84500073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f846c071350 con 0x7f8450005d90 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f845000dd70 con 0x7f8450006f60 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f845000df40 con 0x7f846c06e7e0 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846aa00640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f8450006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f8462000640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c06e7e0 0x7f8450006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846b400640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8450006f60 0x7f84500073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846aa00640 1 -- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c06e7e0 msgr2=0x7f8450006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846aa00640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c06e7e0 0x7f8450006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846aa00640 1 -- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8450006f60 msgr2=0x7f84500073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:57.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846aa00640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8450006f60 0x7f84500073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.355+0000 7f846aa00640 1 -- 172.21.15.86:0/4047952647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8450005740 con 0x7f8450005d90 2024-03-22T04:20:57.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f846aa00640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f8450006190 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f845c009b10 tx=0x7f845c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a84638c4937c418 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c0096b0 con 0x7f8450005d90 2024-03-22T04:20:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c0a9450 con 0x7f8450005d90 2024-03-22T04:20:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f846a000640 1 -- 172.21.15.86:0/4047952647 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c0a9840 con 0x7f8450005d90 2024-03-22T04:20:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f8471c00640 1 -- 172.21.15.86:0/4047952647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 msgr2=0x7f8450006190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f8471c00640 1 --2- 172.21.15.86:0/4047952647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f8450006190 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f845c009b10 tx=0x7f845c01ca70 comp rx=0 tx=0).stop 2024-03-22T04:20:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f846b400640 1 -- 172.21.15.86:0/4047952647 reap_dead start 2024-03-22T04:20:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f8471c00640 1 -- 172.21.15.86:0/4047952647 shutdown_connections 2024-03-22T04:20:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f8471c00640 1 -- 172.21.15.86:0/4047952647 >> 172.21.15.86:0/4047952647 conn(0x7f846c0cc940 msgr2=0x7f846c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f8471c00640 1 -- 172.21.15.86:0/4047952647 shutdown_connections 2024-03-22T04:20:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.359+0000 7f8471c00640 1 -- 172.21.15.86:0/4047952647 wait complete. 2024-03-22T04:20:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 Processor -- start 2024-03-22T04:20:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 -- start start 2024-03-22T04:20:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f846c142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c0c9270 0x7f846c1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c15a8c0 0x7f846c15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f846c0ce930 con 0x7f8450005d90 2024-03-22T04:20:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f846c071b60 con 0x7f846c15a8c0 2024-03-22T04:20:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f846c072080 con 0x7f846c0c9270 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f846c142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f846c142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33874/0 (socket says 172.21.15.86:33874) 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 -- 172.21.15.86:0/1048397429 learned_addr learned my addr 172.21.15.86:0/1048397429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846b400640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c15a8c0 0x7f846c15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8462000640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c0c9270 0x7f846c1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 -- 172.21.15.86:0/1048397429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c0c9270 msgr2=0x7f846c1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c0c9270 0x7f846c1584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 -- 172.21.15.86:0/1048397429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c15a8c0 msgr2=0x7f846c15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c15a8c0 0x7f846c15ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 -- 172.21.15.86:0/1048397429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c143140 con 0x7f8450005d90 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f846aa00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f846c142c00 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f845c009b10 tx=0x7f8450007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8463400640 1 -- 172.21.15.86:0/1048397429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c0096b0 con 0x7f8450005d90 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8463400640 1 -- 172.21.15.86:0/1048397429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c0a9c60 con 0x7f8450005d90 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c0c97e0 con 0x7f8450005d90 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8463400640 1 -- 172.21.15.86:0/1048397429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c0ba630 con 0x7f8450005d90 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c0c9da0 con 0x7f8450005d90 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.363+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c074470 con 0x7f8450005d90 2024-03-22T04:20:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.367+0000 7f8463400640 1 -- 172.21.15.86:0/1048397429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f845c0ba7d0 con 0x7f8450005d90 2024-03-22T04:20:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.367+0000 7f8463400640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8448067b20 0x7f8448069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.367+0000 7f8463400640 1 -- 172.21.15.86:0/1048397429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6001+0+0 (secure 0 0 0) 0x7f845c120d50 con 0x7f8450005d90 2024-03-22T04:20:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.367+0000 7f8462000640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8448067b20 0x7f8448069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.367+0000 7f8462000640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8448067b20 0x7f8448069fe0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f846c1468b0 tx=0x7f8454009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:57.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.371+0000 7f8463400640 1 -- 172.21.15.86:0/1048397429 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f845c0c8060 con 0x7f8450005d90 2024-03-22T04:20:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:57.555+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f846c15f710 con 0x7f8450005d90 2024-03-22T04:20:58.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.199+0000 7f8463400640 1 -- 172.21.15.86:0/1048397429 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v96) ==== 174+0+0 (secure 0 0 0) 0x7f845c0ad020 con 0x7f8450005d90 2024-03-22T04:20:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 17 and data pool 18 2024-03-22T04:20:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8448067b20 msgr2=0x7f8448069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8448067b20 0x7f8448069fe0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f846c1468b0 tx=0x7f8454009040 comp rx=0 tx=0).stop 2024-03-22T04:20:58.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 msgr2=0x7f846c142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f846c142c00 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f845c009b10 tx=0x7f8450007a50 comp rx=0 tx=0).stop 2024-03-22T04:20:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 shutdown_connections 2024-03-22T04:20:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8448067b20 0x7f8448069fe0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f846c15a8c0 0x7f846c15ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f846c0c9270 0x7f846c1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 --2- 172.21.15.86:0/1048397429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8450005d90 0x7f846c142c00 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 >> 172.21.15.86:0/1048397429 conn(0x7f846c0cc940 msgr2=0x7f846c159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 shutdown_connections 2024-03-22T04:20:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.207+0000 7f8471c00640 1 -- 172.21.15.86:0/1048397429 wait complete. 2024-03-22T04:20:58.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:20:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7c5600640 1 Processor -- start 2024-03-22T04:20:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7c5600640 1 -- start start 2024-03-22T04:20:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7c5600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7c00aff50 0x7fa7c006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7c5600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c01444f0 0x7fa7c015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7c5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c00ae5a0 con 0x7fa7c006cbb0 2024-03-22T04:20:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7c5600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c006d010 con 0x7fa7c00aff50 2024-03-22T04:20:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7c5600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c0144940 con 0x7fa7c01444f0 2024-03-22T04:20:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7bea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7c00aff50 0x7fa7c006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7b6000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7c006cbb0 0x7fa7c0143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49816/0 (socket says 172.21.15.86:49816) 2024-03-22T04:20:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.379+0000 7fa7b6000640 1 -- 172.21.15.86:0/2515530231 learned_addr learned my addr 172.21.15.86:0/2515530231 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7bf400640 1 --2- 172.21.15.86:0/2515530231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c01444f0 0x7fa7c015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7bea00640 1 -- 172.21.15.86:0/2515530231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c01444f0 msgr2=0x7fa7c015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7bea00640 1 --2- 172.21.15.86:0/2515530231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c01444f0 0x7fa7c015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7bea00640 1 -- 172.21.15.86:0/2515530231 >> v1:172.21.15.86:6789/0 conn(0x7fa7c006cbb0 legacy=0x7fa7c0143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7be000640 1 -- 172.21.15.86:0/2515530231 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347438215 0 0) 0x7fa7a8001120 con 0x7fa7c006cbb0 2024-03-22T04:20:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7bea00640 1 -- 172.21.15.86:0/2515530231 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c0144b80 con 0x7fa7c00aff50 2024-03-22T04:20:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7bf400640 1 --2- 172.21.15.86:0/2515530231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c01444f0 0x7fa7c015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:20:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7bea00640 1 --2- 172.21.15.86:0/2515530231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c00aff50 0x7fa7c006c670 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00097c0 tx=0x7fa7b001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54166e89490dad19 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7be000640 1 -- 172.21.15.86:0/2515530231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00b0070 con 0x7fa7c00aff50 2024-03-22T04:20:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7be000640 1 -- 172.21.15.86:0/2515530231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b00ab600 con 0x7fa7c00aff50 2024-03-22T04:20:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.383+0000 7fa7be000640 1 -- 172.21.15.86:0/2515530231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00ab9d0 con 0x7fa7c00aff50 2024-03-22T04:20:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- 172.21.15.86:0/2515530231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c00aff50 msgr2=0x7fa7c006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 --2- 172.21.15.86:0/2515530231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c00aff50 0x7fa7c006c670 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00097c0 tx=0x7fa7b001c410 comp rx=0 tx=0).stop 2024-03-22T04:20:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- 172.21.15.86:0/2515530231 shutdown_connections 2024-03-22T04:20:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 --2- 172.21.15.86:0/2515530231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c01444f0 0x7fa7c015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 --2- 172.21.15.86:0/2515530231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c00aff50 0x7fa7c006c670 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- 172.21.15.86:0/2515530231 >> 172.21.15.86:0/2515530231 conn(0x7fa7c0071fe0 msgr2=0x7fa7c00723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- 172.21.15.86:0/2515530231 shutdown_connections 2024-03-22T04:20:58.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- 172.21.15.86:0/2515530231 wait complete. 2024-03-22T04:20:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 Processor -- start 2024-03-22T04:20:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- start start 2024-03-22T04:20:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c006cbb0 0x7fa7c0141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c00aff50 0x7fa7c013d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c01444f0 0x7fa7c00b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0162f30 con 0x7fa7c01444f0 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00b0330 con 0x7fa7c006cbb0 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7c5600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c0141580 con 0x7fa7c00aff50 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c01444f0 0x7fa7c00b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c01444f0 0x7fa7c00b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33892/0 (socket says 172.21.15.86:33892) 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 -- 172.21.15.86:0/900408530 learned_addr learned my addr 172.21.15.86:0/900408530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bea00640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c006cbb0 0x7fa7c0141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7b6000640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c00aff50 0x7fa7c013d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 -- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c00aff50 msgr2=0x7fa7c013d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c00aff50 0x7fa7c013d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 -- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c006cbb0 msgr2=0x7fa7c0141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c006cbb0 0x7fa7c0141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 -- 172.21.15.86:0/900408530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c00b4370 con 0x7fa7c01444f0 2024-03-22T04:20:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7b6000640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c00aff50 0x7fa7c013d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:20:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7bf400640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c01444f0 0x7fa7c00b54d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800cd50 tx=0x7fa7b800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.387+0000 7fa7b7400640 1 -- 172.21.15.86:0/900408530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800f680 con 0x7fa7c01444f0 2024-03-22T04:20:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7b7400640 1 -- 172.21.15.86:0/900408530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b800f840 con 0x7fa7c01444f0 2024-03-22T04:20:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c00b4650 con 0x7fa7c01444f0 2024-03-22T04:20:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7b7400640 1 -- 172.21.15.86:0/900408530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800fc30 con 0x7fa7c01444f0 2024-03-22T04:20:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c013da10 con 0x7fa7c01444f0 2024-03-22T04:20:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c0068650 con 0x7fa7c01444f0 2024-03-22T04:20:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7b7400640 1 -- 172.21.15.86:0/900408530 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa7b80213f0 con 0x7fa7c01444f0 2024-03-22T04:20:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7b7400640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa79c067b20 0x7fa79c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7b7400640 1 -- 172.21.15.86:0/900408530 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6101+0+0 (secure 0 0 0) 0x7fa7b8086de0 con 0x7fa7c01444f0 2024-03-22T04:20:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.391+0000 7fa7bea00640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa79c067b20 0x7fa79c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.395+0000 7fa7bea00640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa79c067b20 0x7fa79c069fe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00ad000 tx=0x7fa7b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.395+0000 7fa7b7400640 1 -- 172.21.15.86:0/900408530 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa7b8014030 con 0x7fa7c01444f0 2024-03-22T04:20:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.551+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7c006f110 con 0x7fa7c01444f0 2024-03-22T04:20:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7b7400640 1 -- 172.21.15.86:0/900408530 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+16969 (secure 0 0 0) 0x7fa7b8057660 con 0x7fa7c01444f0 2024-03-22T04:20:58.555 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:58.555 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":123,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:57.561497+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa79c067b20 msgr2=0x7fa79c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa79c067b20 0x7fa79c069fe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00ad000 tx=0x7fa7b0008040 comp rx=0 tx=0).stop 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c01444f0 msgr2=0x7fa7c00b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c01444f0 0x7fa7c00b54d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800cd50 tx=0x7fa7b800a650 comp rx=0 tx=0).stop 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 shutdown_connections 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa79c067b20 0x7fa79c069fe0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c01444f0 0x7fa7c00b54d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7c00aff50 0x7fa7c013d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 --2- 172.21.15.86:0/900408530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7c006cbb0 0x7fa7c0141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 >> 172.21.15.86:0/900408530 conn(0x7fa7c0071fe0 msgr2=0x7fa7c006e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 shutdown_connections 2024-03-22T04:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.555+0000 7fa7c5600640 1 -- 172.21.15.86:0/900408530 wait complete. 2024-03-22T04:20:58.576 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:20:58.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff027200640 1 Processor -- start 2024-03-22T04:20:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff027200640 1 -- start start 2024-03-22T04:20:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff027200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff0200d0580 0x7ff020071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff027200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff02006e2f0 0x7ff020071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff027200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0201623d0 con 0x7ff02015e630 2024-03-22T04:20:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff027200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff0201620f0 con 0x7ff0200d0580 2024-03-22T04:20:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff027200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff02015ec10 con 0x7ff02006e2f0 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff017e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff0200d0580 0x7ff020071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff01fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff02015e630 0x7ff0200cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49824/0 (socket says 172.21.15.86:49824) 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff01fe00640 1 -- 172.21.15.86:0/1178082706 learned_addr learned my addr 172.21.15.86:0/1178082706 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff025200640 1 --2- 172.21.15.86:0/1178082706 >> v2:172.21.15.196:3300/0 conn(0x7ff02006e2f0 0x7ff020071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff017e00640 1 -- 172.21.15.86:0/1178082706 >> v2:172.21.15.196:3300/0 conn(0x7ff02006e2f0 msgr2=0x7ff020071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff017e00640 1 --2- 172.21.15.86:0/1178082706 >> v2:172.21.15.196:3300/0 conn(0x7ff02006e2f0 0x7ff020071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff017e00640 1 -- 172.21.15.86:0/1178082706 >> v1:172.21.15.86:6789/0 conn(0x7ff02015e630 legacy=0x7ff0200cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff017e00640 1 -- 172.21.15.86:0/1178082706 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02015eef0 con 0x7ff0200d0580 2024-03-22T04:20:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.715+0000 7ff01f400640 1 -- 172.21.15.86:0/1178082706 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3144232980 0 0) 0x7ff0201623d0 con 0x7ff02015e630 2024-03-22T04:20:58.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff017e00640 1 --2- 172.21.15.86:0/1178082706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0200d0580 0x7ff020071100 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff0200d2200 tx=0x7ff00801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a30e69098203c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:58.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff01f400640 1 -- 172.21.15.86:0/1178082706 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080b0070 con 0x7ff0200d0580 2024-03-22T04:20:58.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff01f400640 1 -- 172.21.15.86:0/1178082706 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0080ab600 con 0x7ff0200d0580 2024-03-22T04:20:58.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff01f400640 1 -- 172.21.15.86:0/1178082706 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080ab9d0 con 0x7ff0200d0580 2024-03-22T04:20:58.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff027200640 1 -- 172.21.15.86:0/1178082706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0200d0580 msgr2=0x7ff020071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff027200640 1 --2- 172.21.15.86:0/1178082706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0200d0580 0x7ff020071100 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff0200d2200 tx=0x7ff00801c480 comp rx=0 tx=0).stop 2024-03-22T04:20:58.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff027200640 1 -- 172.21.15.86:0/1178082706 shutdown_connections 2024-03-22T04:20:58.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff027200640 1 --2- 172.21.15.86:0/1178082706 >> v2:172.21.15.196:3300/0 conn(0x7ff02006e2f0 0x7ff020071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff027200640 1 --2- 172.21.15.86:0/1178082706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0200d0580 0x7ff020071100 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff027200640 1 -- 172.21.15.86:0/1178082706 >> 172.21.15.86:0/1178082706 conn(0x7ff02006f910 msgr2=0x7ff02006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:58.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff027200640 1 -- 172.21.15.86:0/1178082706 shutdown_connections 2024-03-22T04:20:58.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.719+0000 7ff027200640 1 -- 172.21.15.86:0/1178082706 wait complete. 2024-03-22T04:20:58.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff027200640 1 Processor -- start 2024-03-22T04:20:58.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff027200640 1 -- start start 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff027200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02006e2f0 0x7ff0200c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff027200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0200d0580 0x7ff020156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff027200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02015e630 0x7ff02015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff027200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff020071f90 con 0x7ff02006e2f0 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff027200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff02015f9d0 con 0x7ff02015e630 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff027200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff0200cfaf0 con 0x7ff0200d0580 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff01fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02006e2f0 0x7ff0200c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff01fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02006e2f0 0x7ff0200c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33910/0 (socket says 172.21.15.86:33910) 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.723+0000 7ff01fe00640 1 -- 172.21.15.86:0/752240655 learned_addr learned my addr 172.21.15.86:0/752240655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff025200640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02015e630 0x7ff02015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01fe00640 1 -- 172.21.15.86:0/752240655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0200d0580 msgr2=0x7ff020156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01fe00640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0200d0580 0x7ff020156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01fe00640 1 -- 172.21.15.86:0/752240655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02015e630 msgr2=0x7ff02015fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01fe00640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02015e630 0x7ff02015fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01fe00640 1 -- 172.21.15.86:0/752240655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff020160610 con 0x7ff02006e2f0 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff025200640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02015e630 0x7ff02015fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01fe00640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02006e2f0 0x7ff0200c8010 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c480 tx=0x7ff01000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01cc00640 1 -- 172.21.15.86:0/752240655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010018070 con 0x7ff02006e2f0 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01cc00640 1 -- 172.21.15.86:0/752240655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff010007500 con 0x7ff02006e2f0 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0200c7530 con 0x7ff02006e2f0 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01cc00640 1 -- 172.21.15.86:0/752240655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010004a30 con 0x7ff02006e2f0 2024-03-22T04:20:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0200c7af0 con 0x7ff02006e2f0 2024-03-22T04:20:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff020068650 con 0x7ff02006e2f0 2024-03-22T04:20:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01cc00640 1 -- 172.21.15.86:0/752240655 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff01001b070 con 0x7ff02006e2f0 2024-03-22T04:20:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01cc00640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feffc067b20 0x7feffc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.727+0000 7ff01cc00640 1 -- 172.21.15.86:0/752240655 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6101+0+0 (secure 0 0 0) 0x7ff010085430 con 0x7ff02006e2f0 2024-03-22T04:20:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.731+0000 7ff017e00640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feffc067b20 0x7feffc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.731+0000 7ff017e00640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feffc067b20 0x7feffc069fe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ad000 tx=0x7ff008008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.735+0000 7ff01cc00640 1 -- 172.21.15.86:0/752240655 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff010055ca0 con 0x7ff02006e2f0 2024-03-22T04:20:58.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.915+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0200d16f0 con 0x7ff02006e2f0 2024-03-22T04:20:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.915+0000 7ff01cc00640 1 -- 172.21.15.86:0/752240655 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+16969 (secure 0 0 0) 0x7ff0100555a0 con 0x7ff02006e2f0 2024-03-22T04:20:58.917 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:20:58.917 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":123,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:57.561497+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:20:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feffc067b20 msgr2=0x7feffc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feffc067b20 0x7feffc069fe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ad000 tx=0x7ff008008040 comp rx=0 tx=0).stop 2024-03-22T04:20:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02006e2f0 msgr2=0x7ff0200c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02006e2f0 0x7ff0200c8010 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c480 tx=0x7ff01000c940 comp rx=0 tx=0).stop 2024-03-22T04:20:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 shutdown_connections 2024-03-22T04:20:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feffc067b20 0x7feffc069fe0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff02015e630 0x7ff02015fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0200d0580 0x7ff020156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 --2- 172.21.15.86:0/752240655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02006e2f0 0x7ff0200c8010 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 >> 172.21.15.86:0/752240655 conn(0x7ff02006f910 msgr2=0x7ff0201610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 shutdown_connections 2024-03-22T04:20:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:58.919+0000 7ff027200640 1 -- 172.21.15.86:0/752240655 wait complete. 2024-03-22T04:20:58.940 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-22T04:20:59.057 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:20:59.061 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:20:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.079+0000 7f2c46800640 1 Processor -- start 2024-03-22T04:20:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.079+0000 7f2c46800640 1 -- start start 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.079+0000 7f2c46800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2c400b2360 0x7f2c400b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c46800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c4015d0b0 0x7f2c400b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c46800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4015d710 con 0x7f2c400b3430 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c46800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2c400b2740 con 0x7f2c400b2360 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c46800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4015d4b0 con 0x7f2c4015d0b0 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2c400b3430 0x7f2c4015c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49830/0 (socket says 172.21.15.86:49830) 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3ea00640 1 -- 172.21.15.86:0/3574022325 learned_addr learned my addr 172.21.15.86:0/3574022325 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3f400640 1 --2- 172.21.15.86:0/3574022325 >> v2:172.21.15.196:3300/0 conn(0x7f2c400b2360 0x7f2c400b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3fe00640 1 --2- 172.21.15.86:0/3574022325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c4015d0b0 0x7f2c400b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3e000640 1 -- 172.21.15.86:0/3574022325 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420361597 0 0) 0x7f2c4015d710 con 0x7f2c400b3430 2024-03-22T04:20:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3e000640 1 -- 172.21.15.86:0/3574022325 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c24003530 con 0x7f2c400b3430 2024-03-22T04:20:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3f400640 1 -- 172.21.15.86:0/3574022325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c4015d0b0 msgr2=0x7f2c400b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:20:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3f400640 1 --2- 172.21.15.86:0/3574022325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c4015d0b0 0x7f2c400b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3f400640 1 -- 172.21.15.86:0/3574022325 >> v1:172.21.15.86:6789/0 conn(0x7f2c400b3430 legacy=0x7f2c4015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3f400640 1 -- 172.21.15.86:0/3574022325 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4015fe40 con 0x7f2c400b2360 2024-03-22T04:20:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.083+0000 7f2c3f400640 1 --2- 172.21.15.86:0/3574022325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c400b2360 0x7f2c400b2ef0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c300097f0 tx=0x7f2c3001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb9d5c65c1f2ca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c3e000640 1 -- 172.21.15.86:0/3574022325 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300b2070 con 0x7f2c400b2360 2024-03-22T04:20:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c3e000640 1 -- 172.21.15.86:0/3574022325 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c300ad640 con 0x7f2c400b2360 2024-03-22T04:20:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c3e000640 1 -- 172.21.15.86:0/3574022325 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300ada10 con 0x7f2c400b2360 2024-03-22T04:20:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 -- 172.21.15.86:0/3574022325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c400b2360 msgr2=0x7f2c400b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 --2- 172.21.15.86:0/3574022325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c400b2360 0x7f2c400b2ef0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c300097f0 tx=0x7f2c3001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:20:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 -- 172.21.15.86:0/3574022325 shutdown_connections 2024-03-22T04:20:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 --2- 172.21.15.86:0/3574022325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c4015d0b0 0x7f2c400b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 --2- 172.21.15.86:0/3574022325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c400b2360 0x7f2c400b2ef0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 -- 172.21.15.86:0/3574022325 >> 172.21.15.86:0/3574022325 conn(0x7f2c400cc940 msgr2=0x7f2c400ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:20:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 -- 172.21.15.86:0/3574022325 shutdown_connections 2024-03-22T04:20:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 -- 172.21.15.86:0/3574022325 wait complete. 2024-03-22T04:20:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 Processor -- start 2024-03-22T04:20:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.087+0000 7f2c46800640 1 -- start start 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c46800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c400b2360 0x7f2c40147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c46800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c400b3430 0x7f2c40148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c46800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4015d0b0 0x7f2c401556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c46800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40160c00 con 0x7f2c400b3430 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c46800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c400b5ad0 con 0x7f2c4015d0b0 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c46800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4014b9e0 con 0x7f2c400b2360 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c400b3430 0x7f2c40148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c400b3430 0x7f2c40148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33914/0 (socket says 172.21.15.86:33914) 2024-03-22T04:20:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 -- 172.21.15.86:0/2463272824 learned_addr learned my addr 172.21.15.86:0/2463272824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:20:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3f400640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c400b2360 0x7f2c40147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3fe00640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4015d0b0 0x7f2c401556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 -- 172.21.15.86:0/2463272824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c400b2360 msgr2=0x7f2c40147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c400b2360 0x7f2c40147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 -- 172.21.15.86:0/2463272824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4015d0b0 msgr2=0x7f2c401556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:20:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4015d0b0 0x7f2c401556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:20:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 -- 172.21.15.86:0/2463272824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4014bcc0 con 0x7f2c400b3430 2024-03-22T04:20:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c3ea00640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c400b3430 0x7f2c40148160 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0010f0 tx=0x7f2c2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c37400640 1 -- 172.21.15.86:0/2463272824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c017070 con 0x7f2c400b3430 2024-03-22T04:20:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c37400640 1 -- 172.21.15.86:0/2463272824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c005d10 con 0x7f2c400b3430 2024-03-22T04:20:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c37400640 1 -- 172.21.15.86:0/2463272824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c007310 con 0x7f2c400b3430 2024-03-22T04:20:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4014bfa0 con 0x7f2c400b3430 2024-03-22T04:20:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.091+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4014c560 con 0x7f2c400b3430 2024-03-22T04:20:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.095+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c40074470 con 0x7f2c400b3430 2024-03-22T04:20:59.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.095+0000 7f2c37400640 1 -- 172.21.15.86:0/2463272824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f2c2c01a070 con 0x7f2c400b3430 2024-03-22T04:20:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.095+0000 7f2c37400640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c20067b20 0x7f2c20069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:20:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.095+0000 7f2c37400640 1 -- 172.21.15.86:0/2463272824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6101+0+0 (secure 0 0 0) 0x7f2c2c085580 con 0x7f2c400b3430 2024-03-22T04:20:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.095+0000 7f2c3f400640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c20067b20 0x7f2c20069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:20:59.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.095+0000 7f2c3f400640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c20067b20 0x7f2c20069fe0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2c300af000 tx=0x7f2c30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:20:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.099+0000 7f2c37400640 1 -- 172.21.15.86:0/2463272824 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2c2c055680 con 0x7f2c400b3430 2024-03-22T04:20:59.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:20:59.295+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2c4006e9f0 con 0x7f2c400b3430 2024-03-22T04:21:00.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.203+0000 7f2c37400640 1 -- 172.21.15.86:0/2463272824 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) ==== 118+0+0 (secure 0 0 0) 0x7f2c2c059720 con 0x7f2c400b3430 2024-03-22T04:21:00.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c20067b20 msgr2=0x7f2c20069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:00.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c20067b20 0x7f2c20069fe0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2c300af000 tx=0x7f2c30008040 comp rx=0 tx=0).stop 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c400b3430 msgr2=0x7f2c40148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c400b3430 0x7f2c40148160 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0010f0 tx=0x7f2c2c00c8f0 comp rx=0 tx=0).stop 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 shutdown_connections 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2c20067b20 0x7f2c20069fe0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4015d0b0 0x7f2c401556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c400b3430 0x7f2c40148160 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 --2- 172.21.15.86:0/2463272824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c400b2360 0x7f2c40147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 >> 172.21.15.86:0/2463272824 conn(0x7f2c400cc940 msgr2=0x7f2c4015de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 shutdown_connections 2024-03-22T04:21:00.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.207+0000 7f2c46800640 1 -- 172.21.15.86:0/2463272824 wait complete. 2024-03-22T04:21:00.236 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-22T04:21:00.237 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-22T04:21:00.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.407+0000 7f54b4c00640 1 Processor -- start 2024-03-22T04:21:00.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.407+0000 7f54b4c00640 1 -- start start 2024-03-22T04:21:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54b4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f54b015c730 0x7f54b009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54b4c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b00cda00 con 0x7f54b00b3610 2024-03-22T04:21:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54b4c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b00cdce0 con 0x7f54b00b11c0 2024-03-22T04:21:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54b4c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f54b009b910 con 0x7f54b015c730 2024-03-22T04:21:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54ade00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f54b015c730 0x7f54b009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54ad400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f54b00b3610 0x7f54b009afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49840/0 (socket says 172.21.15.86:49840) 2024-03-22T04:21:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54ad400640 1 -- 172.21.15.86:0/1258416282 learned_addr learned my addr 172.21.15.86:0/1258416282 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54ade00640 1 -- 172.21.15.86:0/1258416282 >> v1:172.21.15.196:6789/0 conn(0x7f54b00b11c0 legacy=0x7f54b00b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54ade00640 1 -- 172.21.15.86:0/1258416282 >> v1:172.21.15.86:6789/0 conn(0x7f54b00b3610 legacy=0x7f54b009afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54ade00640 1 -- 172.21.15.86:0/1258416282 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b00cdfc0 con 0x7f54b015c730 2024-03-22T04:21:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.411+0000 7f54aca00640 1 -- 172.21.15.86:0/1258416282 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1109735724 0 0) 0x7f54b00cda00 con 0x7f54b00b3610 2024-03-22T04:21:00.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54ade00640 1 --2- 172.21.15.86:0/1258416282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b015c730 0x7f54b009aa70 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f54a00097f0 tx=0x7f54a001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d679305006a7d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:00.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54aca00640 1 -- 172.21.15.86:0/1258416282 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a00b0070 con 0x7f54b015c730 2024-03-22T04:21:00.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54aca00640 1 -- 172.21.15.86:0/1258416282 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a00ab600 con 0x7f54b015c730 2024-03-22T04:21:00.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54aca00640 1 -- 172.21.15.86:0/1258416282 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a00ab9d0 con 0x7f54b015c730 2024-03-22T04:21:00.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 -- 172.21.15.86:0/1258416282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b015c730 msgr2=0x7f54b009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:00.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 --2- 172.21.15.86:0/1258416282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b015c730 0x7f54b009aa70 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f54a00097f0 tx=0x7f54a001c410 comp rx=0 tx=0).stop 2024-03-22T04:21:00.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 -- 172.21.15.86:0/1258416282 shutdown_connections 2024-03-22T04:21:00.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 --2- 172.21.15.86:0/1258416282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b015c730 0x7f54b009aa70 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:00.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 -- 172.21.15.86:0/1258416282 >> 172.21.15.86:0/1258416282 conn(0x7f54b00b3bf0 msgr2=0x7f54b00b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:00.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 -- 172.21.15.86:0/1258416282 shutdown_connections 2024-03-22T04:21:00.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 -- 172.21.15.86:0/1258416282 wait complete. 2024-03-22T04:21:00.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 Processor -- start 2024-03-22T04:21:00.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.415+0000 7f54b4c00640 1 -- start start 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b00b11c0 0x7f54b01556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54b00b3610 0x7f54b00c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b015c730 0x7f54b00c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54b00b2e50 con 0x7f54b00b11c0 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54b00b2cd0 con 0x7f54b015c730 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f54b009b6c0 con 0x7f54b00b3610 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b00b11c0 0x7f54b01556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b00b11c0 0x7f54b01556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33932/0 (socket says 172.21.15.86:33932) 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 -- 172.21.15.86:0/4137875728 learned_addr learned my addr 172.21.15.86:0/4137875728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ad400640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54b00b3610 0x7f54b00c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ae800640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b015c730 0x7f54b00c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 -- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54b00b3610 msgr2=0x7f54b00c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54b00b3610 0x7f54b00c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 -- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b015c730 msgr2=0x7f54b00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b015c730 0x7f54b00c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 -- 172.21.15.86:0/4137875728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b00ca830 con 0x7f54b00b11c0 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ae800640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b015c730 0x7f54b00c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54ade00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b00b11c0 0x7f54b01556a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f54a00ae040 tx=0x7f54a001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54a6000640 1 -- 172.21.15.86:0/4137875728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a00b0050 con 0x7f54b00b11c0 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54a6000640 1 -- 172.21.15.86:0/4137875728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a00bc070 con 0x7f54b00b11c0 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b00cab10 con 0x7f54b00b11c0 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54a6000640 1 -- 172.21.15.86:0/4137875728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a00b5af0 con 0x7f54b00b11c0 2024-03-22T04:21:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b00cb0d0 con 0x7f54b00b11c0 2024-03-22T04:21:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b0074470 con 0x7f54b00b11c0 2024-03-22T04:21:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.419+0000 7f54a6000640 1 -- 172.21.15.86:0/4137875728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f54a00b5c90 con 0x7f54b00b11c0 2024-03-22T04:21:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.423+0000 7f54a6000640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5490067b50 0x7f549006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.423+0000 7f54a6000640 1 -- 172.21.15.86:0/4137875728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f54a01212b0 con 0x7f54b00b11c0 2024-03-22T04:21:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.423+0000 7f54ad400640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5490067b50 0x7f549006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.423+0000 7f54ad400640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5490067b50 0x7f549006a010 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f54b00cda80 tx=0x7f549c007b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.423+0000 7f54a6000640 1 -- 172.21.15.86:0/4137875728 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f54a0016090 con 0x7f54b00b11c0 2024-03-22T04:21:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:00.615+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f54b0156a40 con 0x7f54b00b11c0 2024-03-22T04:21:01.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.207+0000 7f54a6000640 1 -- 172.21.15.86:0/4137875728 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) ==== 122+0+0 (secure 0 0 0) 0x7f54a00f1ca0 con 0x7f54b00b11c0 2024-03-22T04:21:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5490067b50 msgr2=0x7f549006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5490067b50 0x7f549006a010 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f54b00cda80 tx=0x7f549c007b10 comp rx=0 tx=0).stop 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b00b11c0 msgr2=0x7f54b01556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b00b11c0 0x7f54b01556a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f54a00ae040 tx=0x7f54a001ac50 comp rx=0 tx=0).stop 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 shutdown_connections 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5490067b50 0x7f549006a010 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54b015c730 0x7f54b00c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54b00b3610 0x7f54b00c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 --2- 172.21.15.86:0/4137875728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b00b11c0 0x7f54b01556a0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 >> 172.21.15.86:0/4137875728 conn(0x7f54b00b3bf0 msgr2=0x7f54b006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 shutdown_connections 2024-03-22T04:21:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.211+0000 7f54b4c00640 1 -- 172.21.15.86:0/4137875728 wait complete. 2024-03-22T04:21:01.230 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-22T04:21:01.231 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:01.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f27000640 1 Processor -- start 2024-03-22T04:21:01.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f27000640 1 -- start start 2024-03-22T04:21:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f27000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f200d0de0 0x7f9f200ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f27000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f200d11c0 con 0x7f9f200d19a0 2024-03-22T04:21:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f27000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f200cece0 con 0x7f9f2015fed0 2024-03-22T04:21:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f27000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9f200d0580 con 0x7f9f200d0de0 2024-03-22T04:21:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f1f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9f200d19a0 0x7f9f200cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49852/0 (socket says 172.21.15.86:49852) 2024-03-22T04:21:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f1f400640 1 -- 172.21.15.86:0/586347700 learned_addr learned my addr 172.21.15.86:0/586347700 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f1fe00640 1 --2- 172.21.15.86:0/586347700 >> v2:172.21.15.86:3300/0 conn(0x7f9f200d0de0 0x7f9f200ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f1ea00640 1 -- 172.21.15.86:0/586347700 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942309716 0 0) 0x7f9f200d11c0 con 0x7f9f200d19a0 2024-03-22T04:21:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f1ea00640 1 -- 172.21.15.86:0/586347700 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f04003530 con 0x7f9f200d19a0 2024-03-22T04:21:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f1fe00640 1 -- 172.21.15.86:0/586347700 >> v1:172.21.15.196:6789/0 conn(0x7f9f2015fed0 legacy=0x7f9f201602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f1fe00640 1 -- 172.21.15.86:0/586347700 >> v1:172.21.15.86:6789/0 conn(0x7f9f200d19a0 legacy=0x7f9f200cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.379+0000 7f9f1fe00640 1 -- 172.21.15.86:0/586347700 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f200cefc0 con 0x7f9f200d0de0 2024-03-22T04:21:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f1fe00640 1 --2- 172.21.15.86:0/586347700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f200d0de0 0x7f9f200ce6b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097c0 tx=0x7f9f1001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a402f998f03a26 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f1ea00640 1 -- 172.21.15.86:0/586347700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100b1070 con 0x7f9f200d0de0 2024-03-22T04:21:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f1ea00640 1 -- 172.21.15.86:0/586347700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f100ac5f0 con 0x7f9f200d0de0 2024-03-22T04:21:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f1ea00640 1 -- 172.21.15.86:0/586347700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100ac9c0 con 0x7f9f200d0de0 2024-03-22T04:21:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f27000640 1 -- 172.21.15.86:0/586347700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f200d0de0 msgr2=0x7f9f200ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f27000640 1 --2- 172.21.15.86:0/586347700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f200d0de0 0x7f9f200ce6b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097c0 tx=0x7f9f1001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f27000640 1 -- 172.21.15.86:0/586347700 shutdown_connections 2024-03-22T04:21:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f27000640 1 --2- 172.21.15.86:0/586347700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f200d0de0 0x7f9f200ce6b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f27000640 1 -- 172.21.15.86:0/586347700 >> 172.21.15.86:0/586347700 conn(0x7f9f2006f910 msgr2=0x7f9f2006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f27000640 1 -- 172.21.15.86:0/586347700 shutdown_connections 2024-03-22T04:21:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.383+0000 7f9f27000640 1 -- 172.21.15.86:0/586347700 wait complete. 2024-03-22T04:21:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 Processor -- start 2024-03-22T04:21:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 -- start start 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f200d0de0 0x7f9f200b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f200d19a0 0x7f9f200c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f2015fed0 0x7f9f200b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f200d1f30 con 0x7f9f2015fed0 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f200d1db0 con 0x7f9f200d19a0 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9f20071e30 con 0x7f9f200d0de0 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f2015fed0 0x7f9f200b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f2015fed0 0x7f9f200b7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33950/0 (socket says 172.21.15.86:33950) 2024-03-22T04:21:01.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 -- 172.21.15.86:0/2573420224 learned_addr learned my addr 172.21.15.86:0/2573420224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f1fe00640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f200d0de0 0x7f9f200b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f1f400640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f200d19a0 0x7f9f200c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 -- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f200d0de0 msgr2=0x7f9f200b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f200d0de0 0x7f9f200b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 -- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f200d19a0 msgr2=0x7f9f200c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f200d19a0 0x7f9f200c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 -- 172.21.15.86:0/2573420224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f200ca690 con 0x7f9f2015fed0 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f1f400640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f200d19a0 0x7f9f200c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f25000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f2015fed0 0x7f9f200b7440 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c7a0 tx=0x7f9f1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f17e00640 1 -- 172.21.15.86:0/2573420224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18003b80 con 0x7f9f2015fed0 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f17e00640 1 -- 172.21.15.86:0/2573420224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f18003d20 con 0x7f9f2015fed0 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f17e00640 1 -- 172.21.15.86:0/2573420224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800aa30 con 0x7f9f2015fed0 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f200b2aa0 con 0x7f9f2015fed0 2024-03-22T04:21:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.387+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f200b3060 con 0x7f9f2015fed0 2024-03-22T04:21:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.391+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f20068650 con 0x7f9f2015fed0 2024-03-22T04:21:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.391+0000 7f9f17e00640 1 -- 172.21.15.86:0/2573420224 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9f1800abd0 con 0x7f9f2015fed0 2024-03-22T04:21:01.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.391+0000 7f9f17e00640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f00067b20 0x7f9f00069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.391+0000 7f9f17e00640 1 -- 172.21.15.86:0/2573420224 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f9f18085e90 con 0x7f9f2015fed0 2024-03-22T04:21:01.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.391+0000 7f9f1fe00640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f00067b20 0x7f9f00069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.391+0000 7f9f1fe00640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f00067b20 0x7f9f00069fe0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9f100ae000 tx=0x7f9f10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:01.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.395+0000 7f9f17e00640 1 -- 172.21.15.86:0/2573420224 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9f18056700 con 0x7f9f2015fed0 2024-03-22T04:21:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.583+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f20070560 con 0x7f9f2015fed0 2024-03-22T04:21:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.583+0000 7f9f17e00640 1 -- 172.21.15.86:0/2573420224 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5194 (secure 0 0 0) 0x7f9f18087070 con 0x7f9f2015fed0 2024-03-22T04:21:01.584 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:01.584 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:00.620053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77636":{"gid":77636,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3165266972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3165266972},{"type":"v1","addr":"172.21.15.86:6837","nonce":3165266972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77636,"qdb_cluster":[77636]},"id":8}]} 2024-03-22T04:21:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 99 2024-03-22T04:21:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f00067b20 msgr2=0x7f9f00069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f00067b20 0x7f9f00069fe0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9f100ae000 tx=0x7f9f10008040 comp rx=0 tx=0).stop 2024-03-22T04:21:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f2015fed0 msgr2=0x7f9f200b7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f2015fed0 0x7f9f200b7440 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c7a0 tx=0x7f9f1800cc60 comp rx=0 tx=0).stop 2024-03-22T04:21:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 shutdown_connections 2024-03-22T04:21:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9f00067b20 0x7f9f00069fe0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f200d19a0 0x7f9f200c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f2015fed0 0x7f9f200b7440 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 --2- 172.21.15.86:0/2573420224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f200d0de0 0x7f9f200b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 >> 172.21.15.86:0/2573420224 conn(0x7f9f2006f910 msgr2=0x7f9f2015efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:01.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 shutdown_connections 2024-03-22T04:21:01.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.587+0000 7f9f27000640 1 -- 172.21.15.86:0/2573420224 wait complete. 2024-03-22T04:21:01.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.739+0000 7fe668600640 1 Processor -- start 2024-03-22T04:21:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.739+0000 7fe668600640 1 -- start start 2024-03-22T04:21:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe668600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe66015e900 0x7fe660162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe668600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 0x7fe6600d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe668600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6600d4bb0 con 0x7fe66015e520 2024-03-22T04:21:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe668600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe6600cf760 con 0x7fe66015e900 2024-03-22T04:21:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe668600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600d3390 con 0x7fe6600d41c0 2024-03-22T04:21:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe665c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe66015e520 0x7fe6600cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49868/0 (socket says 172.21.15.86:49868) 2024-03-22T04:21:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe665c00640 1 -- 172.21.15.86:0/2964671321 learned_addr learned my addr 172.21.15.86:0/2964671321 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe665200640 1 --2- 172.21.15.86:0/2964671321 >> v2:172.21.15.196:3300/0 conn(0x7fe66015e900 0x7fe660162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe666600640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 0x7fe6600d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578749699 0 0) 0x7fe654001120 con 0x7fe66015e520 2024-03-22T04:21:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe648003530 con 0x7fe66015e520 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1259713543 0 0) 0x7fe648003530 con 0x7fe66015e520 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 msgr2=0x7fe6600d4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 0x7fe6600d4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 >> v2:172.21.15.196:3300/0 conn(0x7fe66015e900 msgr2=0x7fe660162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 --2- 172.21.15.86:0/2964671321 >> v2:172.21.15.196:3300/0 conn(0x7fe66015e900 0x7fe660162d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66006cd70 con 0x7fe66015e520 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fe654001850 con 0x7fe66015e520 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe666600640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 0x7fe6600d4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe665200640 1 --2- 172.21.15.86:0/2964671321 >> v2:172.21.15.196:3300/0 conn(0x7fe66015e900 0x7fe660162d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 >> v1:172.21.15.86:6789/0 conn(0x7fe66015e520 legacy=0x7fe6600cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6480039c0 0x7fe648003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 0x7fe648004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe6480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600cf760 con 0x7fe648004d50 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600d3390 con 0x7fe6480039c0 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe64800b9d0 con 0x7fe6600d41c0 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe666600640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe6480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe665200640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 0x7fe648004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe665c00640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6480039c0 0x7fe648003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe666600640 1 -- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 msgr2=0x7fe648004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe666600640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600d41c0 0x7fe648004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe666600640 1 -- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6480039c0 msgr2=0x7fe648003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe666600640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6480039c0 0x7fe648003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.743+0000 7fe666600640 1 -- 172.21.15.86:0/2964671321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe648003530 con 0x7fe648004d50 2024-03-22T04:21:01.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe666600640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe6480051d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e350 tx=0x7fe65801ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3adb599345944c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580b2070 con 0x7fe648004d50 2024-03-22T04:21:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580a5e10 con 0x7fe648004d50 2024-03-22T04:21:01.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe65fe00640 1 -- 172.21.15.86:0/2964671321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658009de0 con 0x7fe648004d50 2024-03-22T04:21:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe668600640 1 -- 172.21.15.86:0/2964671321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 msgr2=0x7fe6480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe668600640 1 --2- 172.21.15.86:0/2964671321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe6480051d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e350 tx=0x7fe65801ce00 comp rx=0 tx=0).stop 2024-03-22T04:21:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe666600640 1 -- 172.21.15.86:0/2964671321 reap_dead start 2024-03-22T04:21:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe668600640 1 -- 172.21.15.86:0/2964671321 shutdown_connections 2024-03-22T04:21:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe668600640 1 -- 172.21.15.86:0/2964671321 >> 172.21.15.86:0/2964671321 conn(0x7fe66008c000 msgr2=0x7fe66009d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe668600640 1 -- 172.21.15.86:0/2964671321 shutdown_connections 2024-03-22T04:21:01.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.747+0000 7fe668600640 1 -- 172.21.15.86:0/2964671321 wait complete. 2024-03-22T04:21:01.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe668600640 1 Processor -- start 2024-03-22T04:21:01.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe668600640 1 -- start start 2024-03-22T04:21:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe668600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe66015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe668600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600c9290 0x7fe66015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe668600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66015d8c0 0x7fe6601e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe668600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600cff40 con 0x7fe648004d50 2024-03-22T04:21:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe668600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600cfc60 con 0x7fe66015d8c0 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe668600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600d0220 con 0x7fe6600c9290 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe665c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe66015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe665c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe66015cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33970/0 (socket says 172.21.15.86:33970) 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe665c00640 1 -- 172.21.15.86:0/371402676 learned_addr learned my addr 172.21.15.86:0/371402676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe665200640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600c9290 0x7fe66015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.751+0000 7fe666600640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66015d8c0 0x7fe6601e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe665c00640 1 -- 172.21.15.86:0/371402676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600c9290 msgr2=0x7fe66015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe665c00640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600c9290 0x7fe66015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe665c00640 1 -- 172.21.15.86:0/371402676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66015d8c0 msgr2=0x7fe6601e9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe665c00640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66015d8c0 0x7fe6601e9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe665c00640 1 -- 172.21.15.86:0/371402676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6601eaaf0 con 0x7fe648004d50 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe665c00640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe66015cb50 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe654007310 tx=0x7fe6540077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe65d600640 1 -- 172.21.15.86:0/371402676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65400fa80 con 0x7fe648004d50 2024-03-22T04:21:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe65d600640 1 -- 172.21.15.86:0/371402676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe654018070 con 0x7fe648004d50 2024-03-22T04:21:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe65d600640 1 -- 172.21.15.86:0/371402676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65400fe80 con 0x7fe648004d50 2024-03-22T04:21:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6601eadd0 con 0x7fe648004d50 2024-03-22T04:21:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6600c8a90 con 0x7fe648004d50 2024-03-22T04:21:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe660068650 con 0x7fe648004d50 2024-03-22T04:21:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe65d600640 1 -- 172.21.15.86:0/371402676 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe65401d460 con 0x7fe648004d50 2024-03-22T04:21:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe65d600640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe65d600640 1 -- 172.21.15.86:0/371402676 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fe6540860b0 con 0x7fe648004d50 2024-03-22T04:21:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.755+0000 7fe665200640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.759+0000 7fe665200640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe6600b67b0 tx=0x7fe64c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.763+0000 7fe65d600640 1 -- 172.21.15.86:0/371402676 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe6600d4bb0 con 0x7fe648004d50 2024-03-22T04:21:01.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.923+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6600ba3c0 con 0x7fe648004d50 2024-03-22T04:21:01.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.923+0000 7fe65d600640 1 -- 172.21.15.86:0/371402676 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+16956 (secure 0 0 0) 0x7fe6540562e0 con 0x7fe648004d50 2024-03-22T04:21:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:01.926 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:59.189511+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 msgr2=0x7fe640069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe6600b67b0 tx=0x7fe64c009190 comp rx=0 tx=0).stop 2024-03-22T04:21:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 msgr2=0x7fe66015cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe66015cb50 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe654007310 tx=0x7fe6540077d0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 shutdown_connections 2024-03-22T04:21:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe640067b20 0x7fe640069fe0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66015d8c0 0x7fe6601e9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6600c9290 0x7fe66015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 --2- 172.21.15.86:0/371402676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe648004d50 0x7fe66015cb50 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 >> 172.21.15.86:0/371402676 conn(0x7fe66008c000 msgr2=0x7fe6600cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 shutdown_connections 2024-03-22T04:21:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:01.927+0000 7fe668600640 1 -- 172.21.15.86:0/371402676 wait complete. 2024-03-22T04:21:01.947 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-22T04:21:01.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:21:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6135000640 1 Processor -- start 2024-03-22T04:21:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6135000640 1 -- start start 2024-03-22T04:21:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6135000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f613015e520 0x7f6130162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6135000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f61300d41c0 0x7f61300d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6135000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61300d4b90 con 0x7f613015e900 2024-03-22T04:21:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6135000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f61300cf760 con 0x7f61300d41c0 2024-03-22T04:21:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6135000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f61300d3390 con 0x7f613015e520 2024-03-22T04:21:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6127e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f613015e900 0x7f61300cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49882/0 (socket says 172.21.15.86:49882) 2024-03-22T04:21:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f6127e00640 1 -- 172.21.15.86:0/2704568698 learned_addr learned my addr 172.21.15.86:0/2704568698 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f612e800640 1 --2- 172.21.15.86:0/2704568698 >> v2:172.21.15.86:3300/0 conn(0x7f61300d41c0 0x7f61300d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f612de00640 1 --2- 172.21.15.86:0/2704568698 >> v2:172.21.15.196:3300/0 conn(0x7f613015e520 0x7f6130162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f612d400640 1 -- 172.21.15.86:0/2704568698 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342224395 0 0) 0x7f61300d4b90 con 0x7f613015e900 2024-03-22T04:21:02.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f612d400640 1 -- 172.21.15.86:0/2704568698 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6114003530 con 0x7f613015e900 2024-03-22T04:21:02.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f612e800640 1 -- 172.21.15.86:0/2704568698 >> v2:172.21.15.196:3300/0 conn(0x7f613015e520 msgr2=0x7f6130162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f612e800640 1 --2- 172.21.15.86:0/2704568698 >> v2:172.21.15.196:3300/0 conn(0x7f613015e520 0x7f6130162d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f612e800640 1 -- 172.21.15.86:0/2704568698 >> v1:172.21.15.86:6789/0 conn(0x7f613015e900 legacy=0x7f61300cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.083+0000 7f612e800640 1 -- 172.21.15.86:0/2704568698 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613006cd70 con 0x7f61300d41c0 2024-03-22T04:21:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f612e800640 1 --2- 172.21.15.86:0/2704568698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f61300d4650 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f613008e980 tx=0x7f612801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37f8f10691835b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f612d400640 1 -- 172.21.15.86:0/2704568698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280b2070 con 0x7f61300d41c0 2024-03-22T04:21:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f612d400640 1 -- 172.21.15.86:0/2704568698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61280a59d0 con 0x7f61300d41c0 2024-03-22T04:21:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f612d400640 1 -- 172.21.15.86:0/2704568698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280a5da0 con 0x7f61300d41c0 2024-03-22T04:21:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f6135000640 1 -- 172.21.15.86:0/2704568698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 msgr2=0x7f61300d4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f6135000640 1 --2- 172.21.15.86:0/2704568698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f61300d4650 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f613008e980 tx=0x7f612801c910 comp rx=0 tx=0).stop 2024-03-22T04:21:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f6135000640 1 -- 172.21.15.86:0/2704568698 shutdown_connections 2024-03-22T04:21:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f6135000640 1 --2- 172.21.15.86:0/2704568698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f61300d4650 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f6135000640 1 --2- 172.21.15.86:0/2704568698 >> v2:172.21.15.196:3300/0 conn(0x7f613015e520 0x7f6130162d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f6135000640 1 -- 172.21.15.86:0/2704568698 >> 172.21.15.86:0/2704568698 conn(0x7f613008c000 msgr2=0x7f613009d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f6135000640 1 -- 172.21.15.86:0/2704568698 shutdown_connections 2024-03-22T04:21:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.087+0000 7f6135000640 1 -- 172.21.15.86:0/2704568698 wait complete. 2024-03-22T04:21:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 Processor -- start 2024-03-22T04:21:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 -- start start 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f6130155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f613015e520 0x7f61300caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f613015e900 0x7f61300c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61300d2b70 con 0x7f61300d41c0 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61300d0220 con 0x7f613015e900 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f61300d2cf0 con 0x7f613015e520 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f6130155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f6130155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33986/0 (socket says 172.21.15.86:33986) 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 -- 172.21.15.86:0/299000539 learned_addr learned my addr 172.21.15.86:0/299000539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6127e00640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f613015e520 0x7f61300caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612e800640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f613015e900 0x7f61300c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 -- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f613015e520 msgr2=0x7f61300caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f613015e520 0x7f61300caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 -- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f613015e900 msgr2=0x7f61300c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f613015e900 0x7f61300c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 -- 172.21.15.86:0/299000539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61300b7210 con 0x7f61300d41c0 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612e800640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f613015e900 0x7f61300c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f612de00640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f6130155ee0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f612000e960 tx=0x7f612000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6126000640 1 -- 172.21.15.86:0/299000539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120017070 con 0x7f61300d41c0 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6126000640 1 -- 172.21.15.86:0/299000539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612000ca00 con 0x7f61300d41c0 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61300b74f0 con 0x7f61300d41c0 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6126000640 1 -- 172.21.15.86:0/299000539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612000cdb0 con 0x7f61300d41c0 2024-03-22T04:21:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.091+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61300c8d20 con 0x7f61300d41c0 2024-03-22T04:21:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.095+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6130068650 con 0x7f61300d41c0 2024-03-22T04:21:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.095+0000 7f6126000640 1 -- 172.21.15.86:0/299000539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f612001a070 con 0x7f61300d41c0 2024-03-22T04:21:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.095+0000 7f6126000640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f610c067b20 0x7f610c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.095+0000 7f6126000640 1 -- 172.21.15.86:0/299000539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f6120086410 con 0x7f61300d41c0 2024-03-22T04:21:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.095+0000 7f6127e00640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f610c067b20 0x7f610c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.099+0000 7f6127e00640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f610c067b20 0x7f610c069fe0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f61301f9ff0 tx=0x7f6118009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.099+0000 7f6126000640 1 -- 172.21.15.86:0/299000539 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6120056630 con 0x7f61300d41c0 2024-03-22T04:21:02.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.267+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61300b6fc0 con 0x7f610c067b20 2024-03-22T04:21:02.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.275+0000 7f6126000640 1 -- 172.21.15.86:0/299000539 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252473 (secure 0 0 0) 0x7f61300b6fc0 con 0x7f610c067b20 2024-03-22T04:21:02.278 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:02.280 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f610c067b20 msgr2=0x7f610c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f610c067b20 0x7f610c069fe0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f61301f9ff0 tx=0x7f6118009040 comp rx=0 tx=0).stop 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 msgr2=0x7f6130155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f6130155ee0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f612000e960 tx=0x7f612000ee20 comp rx=0 tx=0).stop 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 shutdown_connections 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f610c067b20 0x7f610c069fe0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f613015e900 0x7f61300c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f613015e520 0x7f61300caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 --2- 172.21.15.86:0/299000539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61300d41c0 0x7f6130155ee0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 >> 172.21.15.86:0/299000539 conn(0x7f613008c000 msgr2=0x7f61300d06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 shutdown_connections 2024-03-22T04:21:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.279+0000 7f6135000640 1 -- 172.21.15.86:0/299000539 wait complete. 2024-03-22T04:21:02.299 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-03-22T04:21:01.934137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262684,"kb_used_data":13188,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712036,"statfs":{"total":773094113280,"available":772825124864,"internally_reserved":0,"allocated":13504512,"data_stored":6911848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759967"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798884+0000","last_change":"2024-03-22T04:20:58.222971+0000","last_active":"2024-03-22T04:21:00.798884+0000","last_peered":"2024-03-22T04:21:00.798884+0000","last_clean":"2024-03-22T04:21:00.798884+0000","last_became_active":"2024-03-22T04:20:58.222689+0000","last_became_peered":"2024-03-22T04:20:58.222689+0000","last_unstale":"2024-03-22T04:21:00.798884+0000","last_undegraded":"2024-03-22T04:21:00.798884+0000","last_fullsized":"2024-03-22T04:21:00.798884+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:13:51.733623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196209+0000","last_change":"2024-03-22T04:20:58.215246+0000","last_active":"2024-03-22T04:20:59.196209+0000","last_peered":"2024-03-22T04:20:59.196209+0000","last_clean":"2024-03-22T04:20:59.196209+0000","last_became_active":"2024-03-22T04:20:58.214293+0000","last_became_peered":"2024-03-22T04:20:58.214293+0000","last_unstale":"2024-03-22T04:20:59.196209+0000","last_undegraded":"2024-03-22T04:20:59.196209+0000","last_fullsized":"2024-03-22T04:20:59.196209+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:11:12.810457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411310+0000","last_change":"2024-03-22T04:20:58.220620+0000","last_active":"2024-03-22T04:20:59.411310+0000","last_peered":"2024-03-22T04:20:59.411310+0000","last_clean":"2024-03-22T04:20:59.411310+0000","last_became_active":"2024-03-22T04:20:58.220351+0000","last_became_peered":"2024-03-22T04:20:58.220351+0000","last_unstale":"2024-03-22T04:20:59.411310+0000","last_undegraded":"2024-03-22T04:20:59.411310+0000","last_fullsized":"2024-03-22T04:20:59.411310+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:25:06.930467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.211068+0000","last_change":"2024-03-22T04:20:58.199863+0000","last_active":"2024-03-22T04:20:58.211068+0000","last_peered":"2024-03-22T04:20:58.211068+0000","last_clean":"2024-03-22T04:20:58.211068+0000","last_became_active":"2024-03-22T04:20:58.199613+0000","last_became_peered":"2024-03-22T04:20:58.199613+0000","last_unstale":"2024-03-22T04:20:58.211068+0000","last_undegraded":"2024-03-22T04:20:58.211068+0000","last_fullsized":"2024-03-22T04:20:58.211068+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:09:54.936549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223829+0000","last_change":"2024-03-22T04:20:58.208517+0000","last_active":"2024-03-22T04:20:59.223829+0000","last_peered":"2024-03-22T04:20:59.223829+0000","last_clean":"2024-03-22T04:20:59.223829+0000","last_became_active":"2024-03-22T04:20:58.208262+0000","last_became_peered":"2024-03-22T04:20:58.208262+0000","last_unstale":"2024-03-22T04:20:59.223829+0000","last_undegraded":"2024-03-22T04:20:59.223829+0000","last_fullsized":"2024-03-22T04:20:59.223829+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:10.211901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196045+0000","last_change":"2024-03-22T04:20:58.200287+0000","last_active":"2024-03-22T04:20:59.196045+0000","last_peered":"2024-03-22T04:20:59.196045+0000","last_clean":"2024-03-22T04:20:59.196045+0000","last_became_active":"2024-03-22T04:20:58.199223+0000","last_became_peered":"2024-03-22T04:20:58.199223+0000","last_unstale":"2024-03-22T04:20:59.196045+0000","last_undegraded":"2024-03-22T04:20:59.196045+0000","last_fullsized":"2024-03-22T04:20:59.196045+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:46:09.806256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195640+0000","last_change":"2024-03-22T04:20:58.208682+0000","last_active":"2024-03-22T04:20:59.195640+0000","last_peered":"2024-03-22T04:20:59.195640+0000","last_clean":"2024-03-22T04:20:59.195640+0000","last_became_active":"2024-03-22T04:20:58.208370+0000","last_became_peered":"2024-03-22T04:20:58.208370+0000","last_unstale":"2024-03-22T04:20:59.195640+0000","last_undegraded":"2024-03-22T04:20:59.195640+0000","last_fullsized":"2024-03-22T04:20:59.195640+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:46:56.275840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.211762+0000","last_change":"2024-03-22T04:20:58.210386+0000","last_active":"2024-03-22T04:20:58.211762+0000","last_peered":"2024-03-22T04:20:58.211762+0000","last_clean":"2024-03-22T04:20:58.211762+0000","last_became_active":"2024-03-22T04:20:58.210125+0000","last_became_peered":"2024-03-22T04:20:58.210125+0000","last_unstale":"2024-03-22T04:20:58.211762+0000","last_undegraded":"2024-03-22T04:20:58.211762+0000","last_fullsized":"2024-03-22T04:20:58.211762+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:23.404065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.215112+0000","last_change":"2024-03-22T04:20:58.209966+0000","last_active":"2024-03-22T04:20:58.215112+0000","last_peered":"2024-03-22T04:20:58.215112+0000","last_clean":"2024-03-22T04:20:58.215112+0000","last_became_active":"2024-03-22T04:20:58.209691+0000","last_became_peered":"2024-03-22T04:20:58.209691+0000","last_unstale":"2024-03-22T04:20:58.215112+0000","last_undegraded":"2024-03-22T04:20:58.215112+0000","last_fullsized":"2024-03-22T04:20:58.215112+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:00.087275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195857+0000","last_change":"2024-03-22T04:20:58.214938+0000","last_active":"2024-03-22T04:20:59.195857+0000","last_peered":"2024-03-22T04:20:59.195857+0000","last_clean":"2024-03-22T04:20:59.195857+0000","last_became_active":"2024-03-22T04:20:58.213844+0000","last_became_peered":"2024-03-22T04:20:58.213844+0000","last_unstale":"2024-03-22T04:20:59.195857+0000","last_undegraded":"2024-03-22T04:20:59.195857+0000","last_fullsized":"2024-03-22T04:20:59.195857+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:57:59.364439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798781+0000","last_change":"2024-03-22T04:20:58.214717+0000","last_active":"2024-03-22T04:21:00.798781+0000","last_peered":"2024-03-22T04:21:00.798781+0000","last_clean":"2024-03-22T04:21:00.798781+0000","last_became_active":"2024-03-22T04:20:58.214404+0000","last_became_peered":"2024-03-22T04:20:58.214404+0000","last_unstale":"2024-03-22T04:21:00.798781+0000","last_undegraded":"2024-03-22T04:21:00.798781+0000","last_fullsized":"2024-03-22T04:21:00.798781+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:58:59.669806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226682+0000","last_change":"2024-03-22T04:20:58.211901+0000","last_active":"2024-03-22T04:20:59.226682+0000","last_peered":"2024-03-22T04:20:59.226682+0000","last_clean":"2024-03-22T04:20:59.226682+0000","last_became_active":"2024-03-22T04:20:58.211274+0000","last_became_peered":"2024-03-22T04:20:58.211274+0000","last_unstale":"2024-03-22T04:20:59.226682+0000","last_undegraded":"2024-03-22T04:20:59.226682+0000","last_fullsized":"2024-03-22T04:20:59.226682+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:18:01.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195745+0000","last_change":"2024-03-22T04:20:58.208749+0000","last_active":"2024-03-22T04:20:59.195745+0000","last_peered":"2024-03-22T04:20:59.195745+0000","last_clean":"2024-03-22T04:20:59.195745+0000","last_became_active":"2024-03-22T04:20:58.208466+0000","last_became_peered":"2024-03-22T04:20:58.208466+0000","last_unstale":"2024-03-22T04:20:59.195745+0000","last_undegraded":"2024-03-22T04:20:59.195745+0000","last_fullsized":"2024-03-22T04:20:59.195745+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:05:47.974526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226533+0000","last_change":"2024-03-22T04:20:58.215495+0000","last_active":"2024-03-22T04:20:59.226533+0000","last_peered":"2024-03-22T04:20:59.226533+0000","last_clean":"2024-03-22T04:20:59.226533+0000","last_became_active":"2024-03-22T04:20:58.215196+0000","last_became_peered":"2024-03-22T04:20:58.215196+0000","last_unstale":"2024-03-22T04:20:59.226533+0000","last_undegraded":"2024-03-22T04:20:59.226533+0000","last_fullsized":"2024-03-22T04:20:59.226533+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:26:41.212828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193605+0000","last_change":"2024-03-22T04:20:58.217151+0000","last_active":"2024-03-22T04:20:59.193605+0000","last_peered":"2024-03-22T04:20:59.193605+0000","last_clean":"2024-03-22T04:20:59.193605+0000","last_became_active":"2024-03-22T04:20:58.216896+0000","last_became_peered":"2024-03-22T04:20:58.216896+0000","last_unstale":"2024-03-22T04:20:59.193605+0000","last_undegraded":"2024-03-22T04:20:59.193605+0000","last_fullsized":"2024-03-22T04:20:59.193605+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:00.087275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226289+0000","last_change":"2024-03-22T04:20:58.210999+0000","last_active":"2024-03-22T04:20:59.226289+0000","last_peered":"2024-03-22T04:20:59.226289+0000","last_clean":"2024-03-22T04:20:59.226289+0000","last_became_active":"2024-03-22T04:20:58.210694+0000","last_became_peered":"2024-03-22T04:20:58.210694+0000","last_unstale":"2024-03-22T04:20:59.226289+0000","last_undegraded":"2024-03-22T04:20:59.226289+0000","last_fullsized":"2024-03-22T04:20:59.226289+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:09:57.731456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226948+0000","last_change":"2024-03-22T04:20:58.216415+0000","last_active":"2024-03-22T04:20:59.226948+0000","last_peered":"2024-03-22T04:20:59.226948+0000","last_clean":"2024-03-22T04:20:59.226948+0000","last_became_active":"2024-03-22T04:20:58.216146+0000","last_became_peered":"2024-03-22T04:20:58.216146+0000","last_unstale":"2024-03-22T04:20:59.226948+0000","last_undegraded":"2024-03-22T04:20:59.226948+0000","last_fullsized":"2024-03-22T04:20:59.226948+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:25:15.119439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226844+0000","last_change":"2024-03-22T04:20:58.218473+0000","last_active":"2024-03-22T04:20:59.226844+0000","last_peered":"2024-03-22T04:20:59.226844+0000","last_clean":"2024-03-22T04:20:59.226844+0000","last_became_active":"2024-03-22T04:20:58.218215+0000","last_became_peered":"2024-03-22T04:20:58.218215+0000","last_unstale":"2024-03-22T04:20:59.226844+0000","last_undegraded":"2024-03-22T04:20:59.226844+0000","last_fullsized":"2024-03-22T04:20:59.226844+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:11:11.489439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224082+0000","last_change":"2024-03-22T04:20:58.213328+0000","last_active":"2024-03-22T04:20:59.224082+0000","last_peered":"2024-03-22T04:20:59.224082+0000","last_clean":"2024-03-22T04:20:59.224082+0000","last_became_active":"2024-03-22T04:20:58.212990+0000","last_became_peered":"2024-03-22T04:20:58.212990+0000","last_unstale":"2024-03-22T04:20:59.224082+0000","last_undegraded":"2024-03-22T04:20:59.224082+0000","last_fullsized":"2024-03-22T04:20:59.224082+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:22:01.029228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196131+0000","last_change":"2024-03-22T04:20:58.213193+0000","last_active":"2024-03-22T04:20:59.196131+0000","last_peered":"2024-03-22T04:20:59.196131+0000","last_clean":"2024-03-22T04:20:59.196131+0000","last_became_active":"2024-03-22T04:20:58.212934+0000","last_became_peered":"2024-03-22T04:20:58.212934+0000","last_unstale":"2024-03-22T04:20:59.196131+0000","last_undegraded":"2024-03-22T04:20:59.196131+0000","last_fullsized":"2024-03-22T04:20:59.196131+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:09:57.731456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223979+0000","last_change":"2024-03-22T04:20:58.203933+0000","last_active":"2024-03-22T04:20:59.223979+0000","last_peered":"2024-03-22T04:20:59.223979+0000","last_clean":"2024-03-22T04:20:59.223979+0000","last_became_active":"2024-03-22T04:20:58.203699+0000","last_became_peered":"2024-03-22T04:20:58.203699+0000","last_unstale":"2024-03-22T04:20:59.223979+0000","last_undegraded":"2024-03-22T04:20:59.223979+0000","last_fullsized":"2024-03-22T04:20:59.223979+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:42:45.630898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193903+0000","last_change":"2024-03-22T04:20:58.222182+0000","last_active":"2024-03-22T04:20:59.193903+0000","last_peered":"2024-03-22T04:20:59.193903+0000","last_clean":"2024-03-22T04:20:59.193903+0000","last_became_active":"2024-03-22T04:20:58.221950+0000","last_became_peered":"2024-03-22T04:20:58.221950+0000","last_unstale":"2024-03-22T04:20:59.193903+0000","last_undegraded":"2024-03-22T04:20:59.193903+0000","last_fullsized":"2024-03-22T04:20:59.193903+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:35:05.550551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193575+0000","last_change":"2024-03-22T04:20:58.216302+0000","last_active":"2024-03-22T04:20:59.193575+0000","last_peered":"2024-03-22T04:20:59.193575+0000","last_clean":"2024-03-22T04:20:59.193575+0000","last_became_active":"2024-03-22T04:20:58.215959+0000","last_became_peered":"2024-03-22T04:20:58.215959+0000","last_unstale":"2024-03-22T04:20:59.193575+0000","last_undegraded":"2024-03-22T04:20:59.193575+0000","last_fullsized":"2024-03-22T04:20:59.193575+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:05:26.957988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193490+0000","last_change":"2024-03-22T04:20:58.216552+0000","last_active":"2024-03-22T04:20:59.193490+0000","last_peered":"2024-03-22T04:20:59.193490+0000","last_clean":"2024-03-22T04:20:59.193490+0000","last_became_active":"2024-03-22T04:20:58.216268+0000","last_became_peered":"2024-03-22T04:20:58.216268+0000","last_unstale":"2024-03-22T04:20:59.193490+0000","last_undegraded":"2024-03-22T04:20:59.193490+0000","last_fullsized":"2024-03-22T04:20:59.193490+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:46:49.894553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798870+0000","last_change":"2024-03-22T04:20:58.216558+0000","last_active":"2024-03-22T04:21:00.798870+0000","last_peered":"2024-03-22T04:21:00.798870+0000","last_clean":"2024-03-22T04:21:00.798870+0000","last_became_active":"2024-03-22T04:20:58.216295+0000","last_became_peered":"2024-03-22T04:20:58.216295+0000","last_unstale":"2024-03-22T04:21:00.798870+0000","last_undegraded":"2024-03-22T04:21:00.798870+0000","last_fullsized":"2024-03-22T04:21:00.798870+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:12.302545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.200858+0000","last_change":"2024-03-22T04:20:58.199755+0000","last_active":"2024-03-22T04:20:58.200858+0000","last_peered":"2024-03-22T04:20:58.200858+0000","last_clean":"2024-03-22T04:20:58.200858+0000","last_became_active":"2024-03-22T04:20:58.199386+0000","last_became_peered":"2024-03-22T04:20:58.199386+0000","last_unstale":"2024-03-22T04:20:58.200858+0000","last_undegraded":"2024-03-22T04:20:58.200858+0000","last_fullsized":"2024-03-22T04:20:58.200858+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:39.078591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224181+0000","last_change":"2024-03-22T04:20:58.210557+0000","last_active":"2024-03-22T04:20:59.224181+0000","last_peered":"2024-03-22T04:20:59.224181+0000","last_clean":"2024-03-22T04:20:59.224181+0000","last_became_active":"2024-03-22T04:20:58.208480+0000","last_became_peered":"2024-03-22T04:20:58.208480+0000","last_unstale":"2024-03-22T04:20:59.224181+0000","last_undegraded":"2024-03-22T04:20:59.224181+0000","last_fullsized":"2024-03-22T04:20:59.224181+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:46:56.275840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226719+0000","last_change":"2024-03-22T04:20:58.211640+0000","last_active":"2024-03-22T04:20:59.226719+0000","last_peered":"2024-03-22T04:20:59.226719+0000","last_clean":"2024-03-22T04:20:59.226719+0000","last_became_active":"2024-03-22T04:20:58.210929+0000","last_became_peered":"2024-03-22T04:20:58.210929+0000","last_unstale":"2024-03-22T04:20:59.226719+0000","last_undegraded":"2024-03-22T04:20:59.226719+0000","last_fullsized":"2024-03-22T04:20:59.226719+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:41:42.186149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411220+0000","last_change":"2024-03-22T04:20:58.220302+0000","last_active":"2024-03-22T04:20:59.411220+0000","last_peered":"2024-03-22T04:20:59.411220+0000","last_clean":"2024-03-22T04:20:59.411220+0000","last_became_active":"2024-03-22T04:20:58.219851+0000","last_became_peered":"2024-03-22T04:20:58.219851+0000","last_unstale":"2024-03-22T04:20:59.411220+0000","last_undegraded":"2024-03-22T04:20:59.411220+0000","last_fullsized":"2024-03-22T04:20:59.411220+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:55:12.522706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411139+0000","last_change":"2024-03-22T04:20:58.213080+0000","last_active":"2024-03-22T04:20:59.411139+0000","last_peered":"2024-03-22T04:20:59.411139+0000","last_clean":"2024-03-22T04:20:59.411139+0000","last_became_active":"2024-03-22T04:20:58.212820+0000","last_became_peered":"2024-03-22T04:20:58.212820+0000","last_unstale":"2024-03-22T04:20:59.411139+0000","last_undegraded":"2024-03-22T04:20:59.411139+0000","last_fullsized":"2024-03-22T04:20:59.411139+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:04:44.118815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798445+0000","last_change":"2024-03-22T04:20:58.223008+0000","last_active":"2024-03-22T04:21:00.798445+0000","last_peered":"2024-03-22T04:21:00.798445+0000","last_clean":"2024-03-22T04:21:00.798445+0000","last_became_active":"2024-03-22T04:20:58.222522+0000","last_became_peered":"2024-03-22T04:20:58.222522+0000","last_unstale":"2024-03-22T04:21:00.798445+0000","last_undegraded":"2024-03-22T04:21:00.798445+0000","last_fullsized":"2024-03-22T04:21:00.798445+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:56.301080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798559+0000","last_change":"2024-03-22T04:20:58.212697+0000","last_active":"2024-03-22T04:21:00.798559+0000","last_peered":"2024-03-22T04:21:00.798559+0000","last_clean":"2024-03-22T04:21:00.798559+0000","last_became_active":"2024-03-22T04:20:58.212423+0000","last_became_peered":"2024-03-22T04:20:58.212423+0000","last_unstale":"2024-03-22T04:21:00.798559+0000","last_undegraded":"2024-03-22T04:21:00.798559+0000","last_fullsized":"2024-03-22T04:21:00.798559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:23:43.142833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.214890+0000","last_change":"2024-03-22T04:20:58.209494+0000","last_active":"2024-03-22T04:20:58.214890+0000","last_peered":"2024-03-22T04:20:58.214890+0000","last_clean":"2024-03-22T04:20:58.214890+0000","last_became_active":"2024-03-22T04:20:58.209211+0000","last_became_peered":"2024-03-22T04:20:58.209211+0000","last_unstale":"2024-03-22T04:20:58.214890+0000","last_undegraded":"2024-03-22T04:20:58.214890+0000","last_fullsized":"2024-03-22T04:20:58.214890+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:58:37.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"123'4","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224225+0000","last_change":"2024-03-22T04:20:57.203371+0000","last_active":"2024-03-22T04:20:59.224225+0000","last_peered":"2024-03-22T04:20:59.224225+0000","last_clean":"2024-03-22T04:20:59.224225+0000","last_became_active":"2024-03-22T04:20:57.203114+0000","last_became_peered":"2024-03-22T04:20:57.203114+0000","last_unstale":"2024-03-22T04:20:59.224225+0000","last_undegraded":"2024-03-22T04:20:59.224225+0000","last_fullsized":"2024-03-22T04:20:59.224225+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:35:57.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195594+0000","last_change":"2024-03-22T04:20:58.218672+0000","last_active":"2024-03-22T04:20:59.195594+0000","last_peered":"2024-03-22T04:20:59.195594+0000","last_clean":"2024-03-22T04:20:59.195594+0000","last_became_active":"2024-03-22T04:20:58.218386+0000","last_became_peered":"2024-03-22T04:20:58.218386+0000","last_unstale":"2024-03-22T04:20:59.195594+0000","last_undegraded":"2024-03-22T04:20:59.195594+0000","last_fullsized":"2024-03-22T04:20:59.195594+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:44.015270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226816+0000","last_change":"2024-03-22T04:20:57.199617+0000","last_active":"2024-03-22T04:20:59.226816+0000","last_peered":"2024-03-22T04:20:59.226816+0000","last_clean":"2024-03-22T04:20:59.226816+0000","last_became_active":"2024-03-22T04:20:57.198363+0000","last_became_peered":"2024-03-22T04:20:57.198363+0000","last_unstale":"2024-03-22T04:20:59.226816+0000","last_undegraded":"2024-03-22T04:20:59.226816+0000","last_fullsized":"2024-03-22T04:20:59.226816+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:35:04.542956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798800+0000","last_change":"2024-03-22T04:20:58.224091+0000","last_active":"2024-03-22T04:21:00.798800+0000","last_peered":"2024-03-22T04:21:00.798800+0000","last_clean":"2024-03-22T04:21:00.798800+0000","last_became_active":"2024-03-22T04:20:58.222720+0000","last_became_peered":"2024-03-22T04:20:58.222720+0000","last_unstale":"2024-03-22T04:21:00.798800+0000","last_undegraded":"2024-03-22T04:21:00.798800+0000","last_fullsized":"2024-03-22T04:21:00.798800+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:24:51.175665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193948+0000","last_change":"2024-03-22T04:20:57.201503+0000","last_active":"2024-03-22T04:20:59.193948+0000","last_peered":"2024-03-22T04:20:59.193948+0000","last_clean":"2024-03-22T04:20:59.193948+0000","last_became_active":"2024-03-22T04:20:57.200242+0000","last_became_peered":"2024-03-22T04:20:57.200242+0000","last_unstale":"2024-03-22T04:20:59.193948+0000","last_undegraded":"2024-03-22T04:20:59.193948+0000","last_fullsized":"2024-03-22T04:20:59.193948+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:35:57.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.211164+0000","last_change":"2024-03-22T04:20:58.201807+0000","last_active":"2024-03-22T04:20:58.211164+0000","last_peered":"2024-03-22T04:20:58.211164+0000","last_clean":"2024-03-22T04:20:58.211164+0000","last_became_active":"2024-03-22T04:20:58.201456+0000","last_became_peered":"2024-03-22T04:20:58.201456+0000","last_unstale":"2024-03-22T04:20:58.211164+0000","last_undegraded":"2024-03-22T04:20:58.211164+0000","last_fullsized":"2024-03-22T04:20:58.211164+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:17:05.954187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196127+0000","last_change":"2024-03-22T04:20:57.190089+0000","last_active":"2024-03-22T04:20:59.196127+0000","last_peered":"2024-03-22T04:20:59.196127+0000","last_clean":"2024-03-22T04:20:59.196127+0000","last_became_active":"2024-03-22T04:20:57.189816+0000","last_became_peered":"2024-03-22T04:20:57.189816+0000","last_unstale":"2024-03-22T04:20:59.196127+0000","last_undegraded":"2024-03-22T04:20:59.196127+0000","last_fullsized":"2024-03-22T04:20:59.196127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:40.563070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.211262+0000","last_change":"2024-03-22T04:20:58.206926+0000","last_active":"2024-03-22T04:20:58.211262+0000","last_peered":"2024-03-22T04:20:58.211262+0000","last_clean":"2024-03-22T04:20:58.211262+0000","last_became_active":"2024-03-22T04:20:58.206610+0000","last_became_peered":"2024-03-22T04:20:58.206610+0000","last_unstale":"2024-03-22T04:20:58.211262+0000","last_undegraded":"2024-03-22T04:20:58.211262+0000","last_fullsized":"2024-03-22T04:20:58.211262+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:44.479262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798590+0000","last_change":"2024-03-22T04:20:57.203428+0000","last_active":"2024-03-22T04:21:00.798590+0000","last_peered":"2024-03-22T04:21:00.798590+0000","last_clean":"2024-03-22T04:21:00.798590+0000","last_became_active":"2024-03-22T04:20:57.203140+0000","last_became_peered":"2024-03-22T04:20:57.203140+0000","last_unstale":"2024-03-22T04:21:00.798590+0000","last_undegraded":"2024-03-22T04:21:00.798590+0000","last_fullsized":"2024-03-22T04:21:00.798590+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:11:29.959142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798466+0000","last_change":"2024-03-22T04:20:58.211986+0000","last_active":"2024-03-22T04:21:00.798466+0000","last_peered":"2024-03-22T04:21:00.798466+0000","last_clean":"2024-03-22T04:21:00.798466+0000","last_became_active":"2024-03-22T04:20:58.211715+0000","last_became_peered":"2024-03-22T04:20:58.211715+0000","last_unstale":"2024-03-22T04:21:00.798466+0000","last_undegraded":"2024-03-22T04:21:00.798466+0000","last_fullsized":"2024-03-22T04:21:00.798466+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:00.567483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.235010+0000","last_change":"2024-03-22T04:20:57.197586+0000","last_active":"2024-03-22T04:20:58.235010+0000","last_peered":"2024-03-22T04:20:58.235010+0000","last_clean":"2024-03-22T04:20:58.235010+0000","last_became_active":"2024-03-22T04:20:57.197287+0000","last_became_peered":"2024-03-22T04:20:57.197287+0000","last_unstale":"2024-03-22T04:20:58.235010+0000","last_undegraded":"2024-03-22T04:20:58.235010+0000","last_fullsized":"2024-03-22T04:20:58.235010+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:09:51.909709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411113+0000","last_change":"2024-03-22T04:20:58.215650+0000","last_active":"2024-03-22T04:20:59.411113+0000","last_peered":"2024-03-22T04:20:59.411113+0000","last_clean":"2024-03-22T04:20:59.411113+0000","last_became_active":"2024-03-22T04:20:58.215236+0000","last_became_peered":"2024-03-22T04:20:58.215236+0000","last_unstale":"2024-03-22T04:20:59.411113+0000","last_undegraded":"2024-03-22T04:20:59.411113+0000","last_fullsized":"2024-03-22T04:20:59.411113+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:44:59.790696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798543+0000","last_change":"2024-03-22T04:20:57.198172+0000","last_active":"2024-03-22T04:21:00.798543+0000","last_peered":"2024-03-22T04:21:00.798543+0000","last_clean":"2024-03-22T04:21:00.798543+0000","last_became_active":"2024-03-22T04:20:57.197912+0000","last_became_peered":"2024-03-22T04:20:57.197912+0000","last_unstale":"2024-03-22T04:21:00.798543+0000","last_undegraded":"2024-03-22T04:21:00.798543+0000","last_fullsized":"2024-03-22T04:21:00.798543+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:25:14.111844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.219026+0000","last_change":"2024-03-22T04:20:58.214509+0000","last_active":"2024-03-22T04:20:58.219026+0000","last_peered":"2024-03-22T04:20:58.219026+0000","last_clean":"2024-03-22T04:20:58.219026+0000","last_became_active":"2024-03-22T04:20:58.214238+0000","last_became_peered":"2024-03-22T04:20:58.214238+0000","last_unstale":"2024-03-22T04:20:58.219026+0000","last_undegraded":"2024-03-22T04:20:58.219026+0000","last_fullsized":"2024-03-22T04:20:58.219026+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:56:57.403406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411040+0000","last_change":"2024-03-22T04:20:57.205148+0000","last_active":"2024-03-22T04:20:59.411040+0000","last_peered":"2024-03-22T04:20:59.411040+0000","last_clean":"2024-03-22T04:20:59.411040+0000","last_became_active":"2024-03-22T04:20:57.204836+0000","last_became_peered":"2024-03-22T04:20:57.204836+0000","last_unstale":"2024-03-22T04:20:59.411040+0000","last_undegraded":"2024-03-22T04:20:59.411040+0000","last_fullsized":"2024-03-22T04:20:59.411040+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:28.187381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193798+0000","last_change":"2024-03-22T04:20:58.217853+0000","last_active":"2024-03-22T04:20:59.193798+0000","last_peered":"2024-03-22T04:20:59.193798+0000","last_clean":"2024-03-22T04:20:59.193798+0000","last_became_active":"2024-03-22T04:20:58.217567+0000","last_became_peered":"2024-03-22T04:20:58.217567+0000","last_unstale":"2024-03-22T04:20:59.193798+0000","last_undegraded":"2024-03-22T04:20:59.193798+0000","last_fullsized":"2024-03-22T04:20:59.193798+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:59:50.535061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.194941+0000","last_change":"2024-03-22T04:20:57.201973+0000","last_active":"2024-03-22T04:20:58.194941+0000","last_peered":"2024-03-22T04:20:58.194941+0000","last_clean":"2024-03-22T04:20:58.194941+0000","last_became_active":"2024-03-22T04:20:57.201696+0000","last_became_peered":"2024-03-22T04:20:57.201696+0000","last_unstale":"2024-03-22T04:20:58.194941+0000","last_undegraded":"2024-03-22T04:20:58.194941+0000","last_fullsized":"2024-03-22T04:20:58.194941+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:21:15.643220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.210548+0000","last_change":"2024-03-22T04:20:58.202110+0000","last_active":"2024-03-22T04:20:58.210548+0000","last_peered":"2024-03-22T04:20:58.210548+0000","last_clean":"2024-03-22T04:20:58.210548+0000","last_became_active":"2024-03-22T04:20:58.201843+0000","last_became_peered":"2024-03-22T04:20:58.201843+0000","last_unstale":"2024-03-22T04:20:58.210548+0000","last_undegraded":"2024-03-22T04:20:58.210548+0000","last_fullsized":"2024-03-22T04:20:58.210548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:46:49.894553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.194484+0000","last_change":"2024-03-22T04:20:57.202784+0000","last_active":"2024-03-22T04:20:58.194484+0000","last_peered":"2024-03-22T04:20:58.194484+0000","last_clean":"2024-03-22T04:20:58.194484+0000","last_became_active":"2024-03-22T04:20:57.202347+0000","last_became_peered":"2024-03-22T04:20:57.202347+0000","last_unstale":"2024-03-22T04:20:58.194484+0000","last_undegraded":"2024-03-22T04:20:58.194484+0000","last_fullsized":"2024-03-22T04:20:58.194484+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:04:29.099232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411464+0000","last_change":"2024-03-22T04:20:58.213824+0000","last_active":"2024-03-22T04:20:59.411464+0000","last_peered":"2024-03-22T04:20:59.411464+0000","last_clean":"2024-03-22T04:20:59.411464+0000","last_became_active":"2024-03-22T04:20:58.213507+0000","last_became_peered":"2024-03-22T04:20:58.213507+0000","last_unstale":"2024-03-22T04:20:59.411464+0000","last_undegraded":"2024-03-22T04:20:59.411464+0000","last_fullsized":"2024-03-22T04:20:59.411464+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:43:23.051623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"123'3","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798884+0000","last_change":"2024-03-22T04:20:57.197449+0000","last_active":"2024-03-22T04:21:00.798884+0000","last_peered":"2024-03-22T04:21:00.798884+0000","last_clean":"2024-03-22T04:21:00.798884+0000","last_became_active":"2024-03-22T04:20:57.197182+0000","last_became_peered":"2024-03-22T04:20:57.197182+0000","last_unstale":"2024-03-22T04:21:00.798884+0000","last_undegraded":"2024-03-22T04:21:00.798884+0000","last_fullsized":"2024-03-22T04:21:00.798884+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:14:11.874875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224063+0000","last_change":"2024-03-22T04:20:58.209311+0000","last_active":"2024-03-22T04:20:59.224063+0000","last_peered":"2024-03-22T04:20:59.224063+0000","last_clean":"2024-03-22T04:20:59.224063+0000","last_became_active":"2024-03-22T04:20:58.209060+0000","last_became_peered":"2024-03-22T04:20:58.209060+0000","last_unstale":"2024-03-22T04:20:59.224063+0000","last_undegraded":"2024-03-22T04:20:59.224063+0000","last_fullsized":"2024-03-22T04:20:59.224063+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:22:53.658661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"123'6","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196236+0000","last_change":"2024-03-22T04:20:57.197731+0000","last_active":"2024-03-22T04:20:59.196236+0000","last_peered":"2024-03-22T04:20:59.196236+0000","last_clean":"2024-03-22T04:20:59.196236+0000","last_became_active":"2024-03-22T04:20:57.197075+0000","last_became_peered":"2024-03-22T04:20:57.197075+0000","last_unstale":"2024-03-22T04:20:59.196236+0000","last_undegraded":"2024-03-22T04:20:59.196236+0000","last_fullsized":"2024-03-22T04:20:59.196236+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:04:29.099232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223598+0000","last_change":"2024-03-22T04:20:58.205356+0000","last_active":"2024-03-22T04:20:59.223598+0000","last_peered":"2024-03-22T04:20:59.223598+0000","last_clean":"2024-03-22T04:20:59.223598+0000","last_became_active":"2024-03-22T04:20:58.205084+0000","last_became_peered":"2024-03-22T04:20:58.205084+0000","last_unstale":"2024-03-22T04:20:59.223598+0000","last_undegraded":"2024-03-22T04:20:59.223598+0000","last_fullsized":"2024-03-22T04:20:59.223598+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:47:25.460863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.234948+0000","last_change":"2024-03-22T04:20:57.201231+0000","last_active":"2024-03-22T04:20:58.234948+0000","last_peered":"2024-03-22T04:20:58.234948+0000","last_clean":"2024-03-22T04:20:58.234948+0000","last_became_active":"2024-03-22T04:20:57.200959+0000","last_became_peered":"2024-03-22T04:20:57.200959+0000","last_unstale":"2024-03-22T04:20:58.234948+0000","last_undegraded":"2024-03-22T04:20:58.234948+0000","last_fullsized":"2024-03-22T04:20:58.234948+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:50.014269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195357+0000","last_change":"2024-03-22T04:20:58.204421+0000","last_active":"2024-03-22T04:20:59.195357+0000","last_peered":"2024-03-22T04:20:59.195357+0000","last_clean":"2024-03-22T04:20:59.195357+0000","last_became_active":"2024-03-22T04:20:58.204126+0000","last_became_peered":"2024-03-22T04:20:58.204126+0000","last_unstale":"2024-03-22T04:20:59.195357+0000","last_undegraded":"2024-03-22T04:20:59.195357+0000","last_fullsized":"2024-03-22T04:20:59.195357+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:22:01.029228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411381+0000","last_change":"2024-03-22T04:20:57.206482+0000","last_active":"2024-03-22T04:20:59.411381+0000","last_peered":"2024-03-22T04:20:59.411381+0000","last_clean":"2024-03-22T04:20:59.411381+0000","last_became_active":"2024-03-22T04:20:57.206022+0000","last_became_peered":"2024-03-22T04:20:57.206022+0000","last_unstale":"2024-03-22T04:20:59.411381+0000","last_undegraded":"2024-03-22T04:20:59.411381+0000","last_fullsized":"2024-03-22T04:20:59.411381+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:23:50.299030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226598+0000","last_change":"2024-03-22T04:20:58.213422+0000","last_active":"2024-03-22T04:20:59.226598+0000","last_peered":"2024-03-22T04:20:59.226598+0000","last_clean":"2024-03-22T04:20:59.226598+0000","last_became_active":"2024-03-22T04:20:58.212246+0000","last_became_peered":"2024-03-22T04:20:58.212246+0000","last_unstale":"2024-03-22T04:20:59.226598+0000","last_undegraded":"2024-03-22T04:20:59.226598+0000","last_fullsized":"2024-03-22T04:20:59.226598+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:26:00.354215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.195727+0000","last_change":"2024-03-22T04:20:57.199212+0000","last_active":"2024-03-22T04:20:58.195727+0000","last_peered":"2024-03-22T04:20:58.195727+0000","last_clean":"2024-03-22T04:20:58.195727+0000","last_became_active":"2024-03-22T04:20:57.198951+0000","last_became_peered":"2024-03-22T04:20:57.198951+0000","last_unstale":"2024-03-22T04:20:58.195727+0000","last_undegraded":"2024-03-22T04:20:58.195727+0000","last_fullsized":"2024-03-22T04:20:58.195727+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:18:34.331983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.212353+0000","last_change":"2024-03-22T04:20:58.206196+0000","last_active":"2024-03-22T04:20:58.212353+0000","last_peered":"2024-03-22T04:20:58.212353+0000","last_clean":"2024-03-22T04:20:58.212353+0000","last_became_active":"2024-03-22T04:20:58.205879+0000","last_became_peered":"2024-03-22T04:20:58.205879+0000","last_unstale":"2024-03-22T04:20:58.212353+0000","last_undegraded":"2024-03-22T04:20:58.212353+0000","last_fullsized":"2024-03-22T04:20:58.212353+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:05:26.957988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226995+0000","last_change":"2024-03-22T04:20:57.202964+0000","last_active":"2024-03-22T04:20:59.226995+0000","last_peered":"2024-03-22T04:20:59.226995+0000","last_clean":"2024-03-22T04:20:59.226995+0000","last_became_active":"2024-03-22T04:20:57.201717+0000","last_became_peered":"2024-03-22T04:20:57.201717+0000","last_unstale":"2024-03-22T04:20:59.226995+0000","last_undegraded":"2024-03-22T04:20:59.226995+0000","last_fullsized":"2024-03-22T04:20:59.226995+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:59:49.527465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798846+0000","last_change":"2024-03-22T04:20:58.206088+0000","last_active":"2024-03-22T04:21:00.798846+0000","last_peered":"2024-03-22T04:21:00.798846+0000","last_clean":"2024-03-22T04:21:00.798846+0000","last_became_active":"2024-03-22T04:20:58.205833+0000","last_became_peered":"2024-03-22T04:20:58.205833+0000","last_unstale":"2024-03-22T04:21:00.798846+0000","last_undegraded":"2024-03-22T04:21:00.798846+0000","last_fullsized":"2024-03-22T04:21:00.798846+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:00:45.333754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226958+0000","last_change":"2024-03-22T04:20:57.197246+0000","last_active":"2024-03-22T04:20:59.226958+0000","last_peered":"2024-03-22T04:20:59.226958+0000","last_clean":"2024-03-22T04:20:59.226958+0000","last_became_active":"2024-03-22T04:20:57.196926+0000","last_became_peered":"2024-03-22T04:20:57.196926+0000","last_unstale":"2024-03-22T04:20:59.226958+0000","last_undegraded":"2024-03-22T04:20:59.226958+0000","last_fullsized":"2024-03-22T04:20:59.226958+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:47:24.453267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196346+0000","last_change":"2024-03-22T04:20:58.215460+0000","last_active":"2024-03-22T04:20:59.196346+0000","last_peered":"2024-03-22T04:20:59.196346+0000","last_clean":"2024-03-22T04:20:59.196346+0000","last_became_active":"2024-03-22T04:20:58.214641+0000","last_became_peered":"2024-03-22T04:20:58.214641+0000","last_unstale":"2024-03-22T04:20:59.196346+0000","last_undegraded":"2024-03-22T04:20:59.196346+0000","last_fullsized":"2024-03-22T04:20:59.196346+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:08:35.684362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224233+0000","last_change":"2024-03-22T04:20:57.195249+0000","last_active":"2024-03-22T04:20:59.224233+0000","last_peered":"2024-03-22T04:20:59.224233+0000","last_clean":"2024-03-22T04:20:59.224233+0000","last_became_active":"2024-03-22T04:20:57.194917+0000","last_became_peered":"2024-03-22T04:20:57.194917+0000","last_unstale":"2024-03-22T04:20:59.224233+0000","last_undegraded":"2024-03-22T04:20:59.224233+0000","last_fullsized":"2024-03-22T04:20:59.224233+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:40.563070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224138+0000","last_change":"2024-03-22T04:20:58.215991+0000","last_active":"2024-03-22T04:20:59.224138+0000","last_peered":"2024-03-22T04:20:59.224138+0000","last_clean":"2024-03-22T04:20:59.224138+0000","last_became_active":"2024-03-22T04:20:58.213718+0000","last_became_peered":"2024-03-22T04:20:58.213718+0000","last_unstale":"2024-03-22T04:20:59.224138+0000","last_undegraded":"2024-03-22T04:20:59.224138+0000","last_fullsized":"2024-03-22T04:20:59.224138+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:44.015270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798671+0000","last_change":"2024-03-22T04:20:57.199018+0000","last_active":"2024-03-22T04:21:00.798671+0000","last_peered":"2024-03-22T04:21:00.798671+0000","last_clean":"2024-03-22T04:21:00.798671+0000","last_became_active":"2024-03-22T04:20:57.198781+0000","last_became_peered":"2024-03-22T04:20:57.198781+0000","last_unstale":"2024-03-22T04:21:00.798671+0000","last_undegraded":"2024-03-22T04:21:00.798671+0000","last_fullsized":"2024-03-22T04:21:00.798671+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:44:56.457026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.194931+0000","last_change":"2024-03-22T04:20:58.209629+0000","last_active":"2024-03-22T04:20:59.194931+0000","last_peered":"2024-03-22T04:20:59.194931+0000","last_clean":"2024-03-22T04:20:59.194931+0000","last_became_active":"2024-03-22T04:20:58.209253+0000","last_became_peered":"2024-03-22T04:20:58.209253+0000","last_unstale":"2024-03-22T04:20:59.194931+0000","last_undegraded":"2024-03-22T04:20:59.194931+0000","last_fullsized":"2024-03-22T04:20:59.194931+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:42:45.630898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226477+0000","last_change":"2024-03-22T04:20:57.199769+0000","last_active":"2024-03-22T04:20:59.226477+0000","last_peered":"2024-03-22T04:20:59.226477+0000","last_clean":"2024-03-22T04:20:59.226477+0000","last_became_active":"2024-03-22T04:20:57.199002+0000","last_became_peered":"2024-03-22T04:20:57.199002+0000","last_unstale":"2024-03-22T04:20:59.226477+0000","last_undegraded":"2024-03-22T04:20:59.226477+0000","last_fullsized":"2024-03-22T04:20:59.226477+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:05:25.950393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226474+0000","last_change":"2024-03-22T04:20:58.215712+0000","last_active":"2024-03-22T04:20:59.226474+0000","last_peered":"2024-03-22T04:20:59.226474+0000","last_clean":"2024-03-22T04:20:59.226474+0000","last_became_active":"2024-03-22T04:20:58.215441+0000","last_became_peered":"2024-03-22T04:20:58.215441+0000","last_unstale":"2024-03-22T04:20:59.226474+0000","last_undegraded":"2024-03-22T04:20:59.226474+0000","last_fullsized":"2024-03-22T04:20:59.226474+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:08:35.684362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193706+0000","last_change":"2024-03-22T04:20:57.199824+0000","last_active":"2024-03-22T04:20:59.193706+0000","last_peered":"2024-03-22T04:20:59.193706+0000","last_clean":"2024-03-22T04:20:59.193706+0000","last_became_active":"2024-03-22T04:20:57.199517+0000","last_became_peered":"2024-03-22T04:20:57.199517+0000","last_unstale":"2024-03-22T04:20:59.193706+0000","last_undegraded":"2024-03-22T04:20:59.193706+0000","last_fullsized":"2024-03-22T04:20:59.193706+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:06:07.301146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411273+0000","last_change":"2024-03-22T04:20:58.215453+0000","last_active":"2024-03-22T04:20:59.411273+0000","last_peered":"2024-03-22T04:20:59.411273+0000","last_clean":"2024-03-22T04:20:59.411273+0000","last_became_active":"2024-03-22T04:20:58.214882+0000","last_became_peered":"2024-03-22T04:20:58.214882+0000","last_unstale":"2024-03-22T04:20:59.411273+0000","last_undegraded":"2024-03-22T04:20:59.411273+0000","last_fullsized":"2024-03-22T04:20:59.411273+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:10.703932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411036+0000","last_change":"2024-03-22T04:20:57.205967+0000","last_active":"2024-03-22T04:20:59.411036+0000","last_peered":"2024-03-22T04:20:59.411036+0000","last_clean":"2024-03-22T04:20:59.411036+0000","last_became_active":"2024-03-22T04:20:57.204791+0000","last_became_peered":"2024-03-22T04:20:57.204791+0000","last_unstale":"2024-03-22T04:20:59.411036+0000","last_undegraded":"2024-03-22T04:20:59.411036+0000","last_fullsized":"2024-03-22T04:20:59.411036+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:56.806429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798539+0000","last_change":"2024-03-22T04:20:58.207851+0000","last_active":"2024-03-22T04:21:00.798539+0000","last_peered":"2024-03-22T04:21:00.798539+0000","last_clean":"2024-03-22T04:21:00.798539+0000","last_became_active":"2024-03-22T04:20:58.207593+0000","last_became_peered":"2024-03-22T04:20:58.207593+0000","last_unstale":"2024-03-22T04:21:00.798539+0000","last_undegraded":"2024-03-22T04:21:00.798539+0000","last_fullsized":"2024-03-22T04:21:00.798539+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:40:00.374676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223523+0000","last_change":"2024-03-22T04:20:57.192045+0000","last_active":"2024-03-22T04:20:59.223523+0000","last_peered":"2024-03-22T04:20:59.223523+0000","last_clean":"2024-03-22T04:20:59.223523+0000","last_became_active":"2024-03-22T04:20:57.191382+0000","last_became_peered":"2024-03-22T04:20:57.191382+0000","last_unstale":"2024-03-22T04:20:59.223523+0000","last_undegraded":"2024-03-22T04:20:59.223523+0000","last_fullsized":"2024-03-22T04:20:59.223523+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:43:22.044028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226852+0000","last_change":"2024-03-22T04:20:58.213414+0000","last_active":"2024-03-22T04:20:59.226852+0000","last_peered":"2024-03-22T04:20:59.226852+0000","last_clean":"2024-03-22T04:20:59.226852+0000","last_became_active":"2024-03-22T04:20:58.213152+0000","last_became_peered":"2024-03-22T04:20:58.213152+0000","last_unstale":"2024-03-22T04:20:59.226852+0000","last_undegraded":"2024-03-22T04:20:59.226852+0000","last_fullsized":"2024-03-22T04:20:59.226852+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:23:45.898765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195942+0000","last_change":"2024-03-22T04:20:57.203732+0000","last_active":"2024-03-22T04:20:59.195942+0000","last_peered":"2024-03-22T04:20:59.195942+0000","last_clean":"2024-03-22T04:20:59.195942+0000","last_became_active":"2024-03-22T04:20:57.203286+0000","last_became_peered":"2024-03-22T04:20:57.203286+0000","last_unstale":"2024-03-22T04:20:59.195942+0000","last_undegraded":"2024-03-22T04:20:59.195942+0000","last_fullsized":"2024-03-22T04:20:59.195942+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:21:15.643220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411411+0000","last_change":"2024-03-22T04:20:58.224052+0000","last_active":"2024-03-22T04:20:59.411411+0000","last_peered":"2024-03-22T04:20:59.411411+0000","last_clean":"2024-03-22T04:20:59.411411+0000","last_became_active":"2024-03-22T04:20:58.223821+0000","last_became_peered":"2024-03-22T04:20:58.223821+0000","last_unstale":"2024-03-22T04:20:59.411411+0000","last_undegraded":"2024-03-22T04:20:59.411411+0000","last_fullsized":"2024-03-22T04:20:59.411411+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:34:10.619536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"124'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.225639+0000","last_change":"2024-03-22T04:20:57.191274+0000","last_active":"2024-03-22T04:20:59.225639+0000","last_peered":"2024-03-22T04:20:59.225639+0000","last_clean":"2024-03-22T04:20:59.225639+0000","last_became_active":"2024-03-22T04:20:57.188254+0000","last_became_peered":"2024-03-22T04:20:57.188254+0000","last_unstale":"2024-03-22T04:20:59.225639+0000","last_undegraded":"2024-03-22T04:20:59.225639+0000","last_fullsized":"2024-03-22T04:20:59.225639+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:35:43.709680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.210700+0000","last_change":"2024-03-22T04:20:58.201237+0000","last_active":"2024-03-22T04:20:58.210700+0000","last_peered":"2024-03-22T04:20:58.210700+0000","last_clean":"2024-03-22T04:20:58.210700+0000","last_became_active":"2024-03-22T04:20:58.200966+0000","last_became_peered":"2024-03-22T04:20:58.200966+0000","last_unstale":"2024-03-22T04:20:58.210700+0000","last_undegraded":"2024-03-22T04:20:58.210700+0000","last_fullsized":"2024-03-22T04:20:58.210700+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:42:08.739393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224269+0000","last_change":"2024-03-22T04:20:57.201573+0000","last_active":"2024-03-22T04:20:59.224269+0000","last_peered":"2024-03-22T04:20:59.224269+0000","last_clean":"2024-03-22T04:20:59.224269+0000","last_became_active":"2024-03-22T04:20:57.201167+0000","last_became_peered":"2024-03-22T04:20:57.201167+0000","last_unstale":"2024-03-22T04:20:59.224269+0000","last_undegraded":"2024-03-22T04:20:59.224269+0000","last_fullsized":"2024-03-22T04:20:59.224269+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:06:07.301146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411495+0000","last_change":"2024-03-22T04:20:58.216652+0000","last_active":"2024-03-22T04:20:59.411495+0000","last_peered":"2024-03-22T04:20:59.411495+0000","last_clean":"2024-03-22T04:20:59.411495+0000","last_became_active":"2024-03-22T04:20:58.216394+0000","last_became_peered":"2024-03-22T04:20:58.216394+0000","last_unstale":"2024-03-22T04:20:59.411495+0000","last_undegraded":"2024-03-22T04:20:59.411495+0000","last_fullsized":"2024-03-22T04:20:59.411495+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:40.654890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226873+0000","last_change":"2024-03-22T04:20:57.201875+0000","last_active":"2024-03-22T04:20:59.226873+0000","last_peered":"2024-03-22T04:20:59.226873+0000","last_clean":"2024-03-22T04:20:59.226873+0000","last_became_active":"2024-03-22T04:20:57.201390+0000","last_became_peered":"2024-03-22T04:20:57.201390+0000","last_unstale":"2024-03-22T04:20:59.226873+0000","last_undegraded":"2024-03-22T04:20:59.226873+0000","last_fullsized":"2024-03-22T04:20:59.226873+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:19:59.079679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226409+0000","last_change":"2024-03-22T04:20:58.212721+0000","last_active":"2024-03-22T04:20:59.226409+0000","last_peered":"2024-03-22T04:20:59.226409+0000","last_clean":"2024-03-22T04:20:59.226409+0000","last_became_active":"2024-03-22T04:20:58.212353+0000","last_became_peered":"2024-03-22T04:20:58.212353+0000","last_unstale":"2024-03-22T04:20:59.226409+0000","last_undegraded":"2024-03-22T04:20:59.226409+0000","last_fullsized":"2024-03-22T04:20:59.226409+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:11:12.810457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223985+0000","last_change":"2024-03-22T04:20:57.203310+0000","last_active":"2024-03-22T04:20:59.223985+0000","last_peered":"2024-03-22T04:20:59.223985+0000","last_clean":"2024-03-22T04:20:59.223985+0000","last_became_active":"2024-03-22T04:20:57.203005+0000","last_became_peered":"2024-03-22T04:20:57.203005+0000","last_unstale":"2024-03-22T04:20:59.223985+0000","last_undegraded":"2024-03-22T04:20:59.223985+0000","last_fullsized":"2024-03-22T04:20:59.223985+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:30:21.983198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195192+0000","last_change":"2024-03-22T04:20:58.204089+0000","last_active":"2024-03-22T04:20:59.195192+0000","last_peered":"2024-03-22T04:20:59.195192+0000","last_clean":"2024-03-22T04:20:59.195192+0000","last_became_active":"2024-03-22T04:20:58.203780+0000","last_became_peered":"2024-03-22T04:20:58.203780+0000","last_unstale":"2024-03-22T04:20:59.195192+0000","last_undegraded":"2024-03-22T04:20:59.195192+0000","last_fullsized":"2024-03-22T04:20:59.195192+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:22:53.658661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193894+0000","last_change":"2024-03-22T04:20:57.201032+0000","last_active":"2024-03-22T04:20:59.193894+0000","last_peered":"2024-03-22T04:20:59.193894+0000","last_clean":"2024-03-22T04:20:59.193894+0000","last_became_active":"2024-03-22T04:20:57.200787+0000","last_became_peered":"2024-03-22T04:20:57.200787+0000","last_unstale":"2024-03-22T04:20:59.193894+0000","last_undegraded":"2024-03-22T04:20:59.193894+0000","last_fullsized":"2024-03-22T04:20:59.193894+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:40.563070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223812+0000","last_change":"2024-03-22T04:20:58.212663+0000","last_active":"2024-03-22T04:20:59.223812+0000","last_peered":"2024-03-22T04:20:59.223812+0000","last_clean":"2024-03-22T04:20:59.223812+0000","last_became_active":"2024-03-22T04:20:58.210906+0000","last_became_peered":"2024-03-22T04:20:58.210906+0000","last_unstale":"2024-03-22T04:20:59.223812+0000","last_undegraded":"2024-03-22T04:20:59.223812+0000","last_fullsized":"2024-03-22T04:20:59.223812+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:30:21.470056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195468+0000","last_change":"2024-03-22T04:20:57.197383+0000","last_active":"2024-03-22T04:20:59.195468+0000","last_peered":"2024-03-22T04:20:59.195468+0000","last_clean":"2024-03-22T04:20:59.195468+0000","last_became_active":"2024-03-22T04:20:57.196699+0000","last_became_peered":"2024-03-22T04:20:57.196699+0000","last_unstale":"2024-03-22T04:20:59.195468+0000","last_undegraded":"2024-03-22T04:20:59.195468+0000","last_fullsized":"2024-03-22T04:20:59.195468+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:42:07.731797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.206243+0000","last_change":"2024-03-22T04:20:58.203871+0000","last_active":"2024-03-22T04:20:58.206243+0000","last_peered":"2024-03-22T04:20:58.206243+0000","last_clean":"2024-03-22T04:20:58.206243+0000","last_became_active":"2024-03-22T04:20:58.203566+0000","last_became_peered":"2024-03-22T04:20:58.203566+0000","last_unstale":"2024-03-22T04:20:58.206243+0000","last_undegraded":"2024-03-22T04:20:58.206243+0000","last_fullsized":"2024-03-22T04:20:58.206243+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:39:15.432447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223578+0000","last_change":"2024-03-22T04:20:57.192242+0000","last_active":"2024-03-22T04:20:59.223578+0000","last_peered":"2024-03-22T04:20:59.223578+0000","last_clean":"2024-03-22T04:20:59.223578+0000","last_became_active":"2024-03-22T04:20:57.191762+0000","last_became_peered":"2024-03-22T04:20:57.191762+0000","last_unstale":"2024-03-22T04:20:59.223578+0000","last_undegraded":"2024-03-22T04:20:59.223578+0000","last_fullsized":"2024-03-22T04:20:59.223578+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:39.647294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223902+0000","last_change":"2024-03-22T04:20:58.214907+0000","last_active":"2024-03-22T04:20:59.223902+0000","last_peered":"2024-03-22T04:20:59.223902+0000","last_clean":"2024-03-22T04:20:59.223902+0000","last_became_active":"2024-03-22T04:20:58.214651+0000","last_became_peered":"2024-03-22T04:20:58.214651+0000","last_unstale":"2024-03-22T04:20:59.223902+0000","last_undegraded":"2024-03-22T04:20:59.223902+0000","last_fullsized":"2024-03-22T04:20:59.223902+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:26:42.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"123'8","reported_seq":20,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.237747+0000","last_change":"2024-03-22T04:20:57.196319+0000","last_active":"2024-03-22T04:20:58.237747+0000","last_peered":"2024-03-22T04:20:58.237747+0000","last_clean":"2024-03-22T04:20:58.237747+0000","last_became_active":"2024-03-22T04:20:57.195923+0000","last_became_peered":"2024-03-22T04:20:57.195923+0000","last_unstale":"2024-03-22T04:20:58.237747+0000","last_undegraded":"2024-03-22T04:20:58.237747+0000","last_fullsized":"2024-03-22T04:20:58.237747+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:51:23.862475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":361,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193372+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:20:59.193372+0000","last_peered":"2024-03-22T04:20:59.193372+0000","last_clean":"2024-03-22T04:20:59.193372+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:20:59.193372+0000","last_undegraded":"2024-03-22T04:20:59.193372+0000","last_fullsized":"2024-03-22T04:20:59.193372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509571,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1536000,"data_stored":718296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509570,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1540096,"data_stored":718318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509571,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":1310276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509571,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":711694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509571,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":713798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509571,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2138112,"data_stored":1308824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509571,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1548288,"data_stored":718982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1531904,"data_stored":711660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:21:02.308 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:21:02.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7be0400640 1 Processor -- start 2024-03-22T04:21:02.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7be0400640 1 -- start start 2024-03-22T04:21:02.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7be0400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7bd80d0de0 0x7f7bd80ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7be0400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d19a0 0x7f7bd80cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7be0400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bd80d1dc0 0x7f7bd815fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7be0400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd80d11c0 con 0x7f7bd80d1dc0 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7be0400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd80cece0 con 0x7f7bd80d0de0 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7be0400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd80cefc0 con 0x7f7bd80d19a0 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7bde400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bd80d1dc0 0x7f7bd815fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7bde400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bd80d1dc0 0x7f7bd815fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33988/0 (socket says 172.21.15.86:33988) 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.479+0000 7f7bde400640 1 -- 172.21.15.86:0/4001728774 learned_addr learned my addr 172.21.15.86:0/4001728774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bdda00640 1 --2- 172.21.15.86:0/4001728774 >> v2:172.21.15.196:3300/0 conn(0x7f7bd80d0de0 0x7f7bd80ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bde400640 1 -- 172.21.15.86:0/4001728774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d19a0 msgr2=0x7f7bd80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bdd000640 1 --2- 172.21.15.86:0/4001728774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d19a0 0x7f7bd80cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bde400640 1 --2- 172.21.15.86:0/4001728774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d19a0 0x7f7bd80cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bde400640 1 -- 172.21.15.86:0/4001728774 >> v2:172.21.15.196:3300/0 conn(0x7f7bd80d0de0 msgr2=0x7f7bd80ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bde400640 1 --2- 172.21.15.86:0/4001728774 >> v2:172.21.15.196:3300/0 conn(0x7f7bd80d0de0 0x7f7bd80ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bde400640 1 -- 172.21.15.86:0/4001728774 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd80cf2a0 con 0x7f7bd80d1dc0 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bdda00640 1 --2- 172.21.15.86:0/4001728774 >> v2:172.21.15.196:3300/0 conn(0x7f7bd80d0de0 0x7f7bd80ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:02.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bdd000640 1 --2- 172.21.15.86:0/4001728774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d19a0 0x7f7bd80cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:02.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bde400640 1 --2- 172.21.15.86:0/4001728774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d1dc0 0x7f7bd815fed0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f7bd806f4c0 tx=0x7f7bd001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1cddeda5eab85a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bd7e00640 1 -- 172.21.15.86:0/4001728774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00a5670 con 0x7f7bd80d1dc0 2024-03-22T04:21:02.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bd7e00640 1 -- 172.21.15.86:0/4001728774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd00a5810 con 0x7f7bd80d1dc0 2024-03-22T04:21:02.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7bd7e00640 1 -- 172.21.15.86:0/4001728774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00a5be0 con 0x7f7bd80d1dc0 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7be0400640 1 -- 172.21.15.86:0/4001728774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d1dc0 msgr2=0x7f7bd815fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.483+0000 7f7be0400640 1 --2- 172.21.15.86:0/4001728774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d1dc0 0x7f7bd815fed0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f7bd806f4c0 tx=0x7f7bd001c9f0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 -- 172.21.15.86:0/4001728774 shutdown_connections 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 --2- 172.21.15.86:0/4001728774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d1dc0 0x7f7bd815fed0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 --2- 172.21.15.86:0/4001728774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d19a0 0x7f7bd80cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 --2- 172.21.15.86:0/4001728774 >> v2:172.21.15.196:3300/0 conn(0x7f7bd80d0de0 0x7f7bd80ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 -- 172.21.15.86:0/4001728774 >> 172.21.15.86:0/4001728774 conn(0x7f7bd806f910 msgr2=0x7f7bd806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 -- 172.21.15.86:0/4001728774 shutdown_connections 2024-03-22T04:21:02.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 -- 172.21.15.86:0/4001728774 wait complete. 2024-03-22T04:21:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 Processor -- start 2024-03-22T04:21:02.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 -- start start 2024-03-22T04:21:02.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd80d0de0 0x7f7bd80b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d19a0 0x7f7bd8143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d1dc0 0x7f7bd81653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8071fb0 con 0x7f7bd80d19a0 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8071e30 con 0x7f7bd80d0de0 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7be0400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd815c8f0 con 0x7f7bd80d1dc0 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d19a0 0x7f7bd8143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d19a0 0x7f7bd8143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33990/0 (socket says 172.21.15.86:33990) 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 -- 172.21.15.86:0/1273423492 learned_addr learned my addr 172.21.15.86:0/1273423492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdda00640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd80d0de0 0x7f7bd80b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bde400640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d1dc0 0x7f7bd81653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 -- 172.21.15.86:0/1273423492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d1dc0 msgr2=0x7f7bd81653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d1dc0 0x7f7bd81653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 -- 172.21.15.86:0/1273423492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd80d0de0 msgr2=0x7f7bd80b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd80d0de0 0x7f7bd80b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 -- 172.21.15.86:0/1273423492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd8156580 con 0x7f7bd80d19a0 2024-03-22T04:21:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.487+0000 7f7bdd000640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d19a0 0x7f7bd8143480 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c7c0 tx=0x7f7bc400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7bd5600640 1 -- 172.21.15.86:0/1273423492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4007a30 con 0x7f7bd80d19a0 2024-03-22T04:21:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7bd5600640 1 -- 172.21.15.86:0/1273423492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc4007bd0 con 0x7f7bd80d19a0 2024-03-22T04:21:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7bd5600640 1 -- 172.21.15.86:0/1273423492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40113f0 con 0x7f7bd80d19a0 2024-03-22T04:21:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd80b4db0 con 0x7f7bd80d19a0 2024-03-22T04:21:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd80b5370 con 0x7f7bd80d19a0 2024-03-22T04:21:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd8068650 con 0x7f7bd80d19a0 2024-03-22T04:21:02.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7bd5600640 1 -- 172.21.15.86:0/1273423492 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7bc4011590 con 0x7f7bd80d19a0 2024-03-22T04:21:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7bd5600640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bb8067b20 0x7f7bb8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7bd5600640 1 -- 172.21.15.86:0/1273423492 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f7bc4086290 con 0x7f7bd80d19a0 2024-03-22T04:21:02.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7bdda00640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bb8067b20 0x7f7bb8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.491+0000 7f7bdda00640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bb8067b20 0x7f7bb8069fe0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00aaf0 tx=0x7f7bcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.495+0000 7f7bd5600640 1 -- 172.21.15.86:0/1273423492 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7bc4015020 con 0x7f7bd80d19a0 2024-03-22T04:21:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.747+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bd80ce670 con 0x7f7bb8067b20 2024-03-22T04:21:02.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.755+0000 7f7bd5600640 1 -- 172.21.15.86:0/1273423492 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252473 (secure 0 0 0) 0x7f7bd80ce670 con 0x7f7bb8067b20 2024-03-22T04:21:02.758 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:02.759 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:21:02.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bb8067b20 msgr2=0x7f7bb8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bb8067b20 0x7f7bb8069fe0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00aaf0 tx=0x7f7bcc008040 comp rx=0 tx=0).stop 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d19a0 msgr2=0x7f7bd8143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d19a0 0x7f7bd8143480 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c7c0 tx=0x7f7bc400cc80 comp rx=0 tx=0).stop 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 shutdown_connections 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7bb8067b20 0x7f7bb8069fe0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bd80d1dc0 0x7f7bd81653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bd80d19a0 0x7f7bd8143480 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 --2- 172.21.15.86:0/1273423492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bd80d0de0 0x7f7bd80b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 >> 172.21.15.86:0/1273423492 conn(0x7f7bd806f910 msgr2=0x7f7bd806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:02.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 shutdown_connections 2024-03-22T04:21:02.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.759+0000 7f7be0400640 1 -- 172.21.15.86:0/1273423492 wait complete. 2024-03-22T04:21:02.786 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-03-22T04:21:01.934137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262684,"kb_used_data":13188,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712036,"statfs":{"total":773094113280,"available":772825124864,"internally_reserved":0,"allocated":13504512,"data_stored":6911848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759967"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798884+0000","last_change":"2024-03-22T04:20:58.222971+0000","last_active":"2024-03-22T04:21:00.798884+0000","last_peered":"2024-03-22T04:21:00.798884+0000","last_clean":"2024-03-22T04:21:00.798884+0000","last_became_active":"2024-03-22T04:20:58.222689+0000","last_became_peered":"2024-03-22T04:20:58.222689+0000","last_unstale":"2024-03-22T04:21:00.798884+0000","last_undegraded":"2024-03-22T04:21:00.798884+0000","last_fullsized":"2024-03-22T04:21:00.798884+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:13:51.733623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196209+0000","last_change":"2024-03-22T04:20:58.215246+0000","last_active":"2024-03-22T04:20:59.196209+0000","last_peered":"2024-03-22T04:20:59.196209+0000","last_clean":"2024-03-22T04:20:59.196209+0000","last_became_active":"2024-03-22T04:20:58.214293+0000","last_became_peered":"2024-03-22T04:20:58.214293+0000","last_unstale":"2024-03-22T04:20:59.196209+0000","last_undegraded":"2024-03-22T04:20:59.196209+0000","last_fullsized":"2024-03-22T04:20:59.196209+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:11:12.810457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411310+0000","last_change":"2024-03-22T04:20:58.220620+0000","last_active":"2024-03-22T04:20:59.411310+0000","last_peered":"2024-03-22T04:20:59.411310+0000","last_clean":"2024-03-22T04:20:59.411310+0000","last_became_active":"2024-03-22T04:20:58.220351+0000","last_became_peered":"2024-03-22T04:20:58.220351+0000","last_unstale":"2024-03-22T04:20:59.411310+0000","last_undegraded":"2024-03-22T04:20:59.411310+0000","last_fullsized":"2024-03-22T04:20:59.411310+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:25:06.930467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.211068+0000","last_change":"2024-03-22T04:20:58.199863+0000","last_active":"2024-03-22T04:20:58.211068+0000","last_peered":"2024-03-22T04:20:58.211068+0000","last_clean":"2024-03-22T04:20:58.211068+0000","last_became_active":"2024-03-22T04:20:58.199613+0000","last_became_peered":"2024-03-22T04:20:58.199613+0000","last_unstale":"2024-03-22T04:20:58.211068+0000","last_undegraded":"2024-03-22T04:20:58.211068+0000","last_fullsized":"2024-03-22T04:20:58.211068+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:09:54.936549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223829+0000","last_change":"2024-03-22T04:20:58.208517+0000","last_active":"2024-03-22T04:20:59.223829+0000","last_peered":"2024-03-22T04:20:59.223829+0000","last_clean":"2024-03-22T04:20:59.223829+0000","last_became_active":"2024-03-22T04:20:58.208262+0000","last_became_peered":"2024-03-22T04:20:58.208262+0000","last_unstale":"2024-03-22T04:20:59.223829+0000","last_undegraded":"2024-03-22T04:20:59.223829+0000","last_fullsized":"2024-03-22T04:20:59.223829+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:39:10.211901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196045+0000","last_change":"2024-03-22T04:20:58.200287+0000","last_active":"2024-03-22T04:20:59.196045+0000","last_peered":"2024-03-22T04:20:59.196045+0000","last_clean":"2024-03-22T04:20:59.196045+0000","last_became_active":"2024-03-22T04:20:58.199223+0000","last_became_peered":"2024-03-22T04:20:58.199223+0000","last_unstale":"2024-03-22T04:20:59.196045+0000","last_undegraded":"2024-03-22T04:20:59.196045+0000","last_fullsized":"2024-03-22T04:20:59.196045+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:46:09.806256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195640+0000","last_change":"2024-03-22T04:20:58.208682+0000","last_active":"2024-03-22T04:20:59.195640+0000","last_peered":"2024-03-22T04:20:59.195640+0000","last_clean":"2024-03-22T04:20:59.195640+0000","last_became_active":"2024-03-22T04:20:58.208370+0000","last_became_peered":"2024-03-22T04:20:58.208370+0000","last_unstale":"2024-03-22T04:20:59.195640+0000","last_undegraded":"2024-03-22T04:20:59.195640+0000","last_fullsized":"2024-03-22T04:20:59.195640+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:46:56.275840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.211762+0000","last_change":"2024-03-22T04:20:58.210386+0000","last_active":"2024-03-22T04:20:58.211762+0000","last_peered":"2024-03-22T04:20:58.211762+0000","last_clean":"2024-03-22T04:20:58.211762+0000","last_became_active":"2024-03-22T04:20:58.210125+0000","last_became_peered":"2024-03-22T04:20:58.210125+0000","last_unstale":"2024-03-22T04:20:58.211762+0000","last_undegraded":"2024-03-22T04:20:58.211762+0000","last_fullsized":"2024-03-22T04:20:58.211762+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:02:23.404065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.215112+0000","last_change":"2024-03-22T04:20:58.209966+0000","last_active":"2024-03-22T04:20:58.215112+0000","last_peered":"2024-03-22T04:20:58.215112+0000","last_clean":"2024-03-22T04:20:58.215112+0000","last_became_active":"2024-03-22T04:20:58.209691+0000","last_became_peered":"2024-03-22T04:20:58.209691+0000","last_unstale":"2024-03-22T04:20:58.215112+0000","last_undegraded":"2024-03-22T04:20:58.215112+0000","last_fullsized":"2024-03-22T04:20:58.215112+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:00.087275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195857+0000","last_change":"2024-03-22T04:20:58.214938+0000","last_active":"2024-03-22T04:20:59.195857+0000","last_peered":"2024-03-22T04:20:59.195857+0000","last_clean":"2024-03-22T04:20:59.195857+0000","last_became_active":"2024-03-22T04:20:58.213844+0000","last_became_peered":"2024-03-22T04:20:58.213844+0000","last_unstale":"2024-03-22T04:20:59.195857+0000","last_undegraded":"2024-03-22T04:20:59.195857+0000","last_fullsized":"2024-03-22T04:20:59.195857+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:57:59.364439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798781+0000","last_change":"2024-03-22T04:20:58.214717+0000","last_active":"2024-03-22T04:21:00.798781+0000","last_peered":"2024-03-22T04:21:00.798781+0000","last_clean":"2024-03-22T04:21:00.798781+0000","last_became_active":"2024-03-22T04:20:58.214404+0000","last_became_peered":"2024-03-22T04:20:58.214404+0000","last_unstale":"2024-03-22T04:21:00.798781+0000","last_undegraded":"2024-03-22T04:21:00.798781+0000","last_fullsized":"2024-03-22T04:21:00.798781+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:58:59.669806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226682+0000","last_change":"2024-03-22T04:20:58.211901+0000","last_active":"2024-03-22T04:20:59.226682+0000","last_peered":"2024-03-22T04:20:59.226682+0000","last_clean":"2024-03-22T04:20:59.226682+0000","last_became_active":"2024-03-22T04:20:58.211274+0000","last_became_peered":"2024-03-22T04:20:58.211274+0000","last_unstale":"2024-03-22T04:20:59.226682+0000","last_undegraded":"2024-03-22T04:20:59.226682+0000","last_fullsized":"2024-03-22T04:20:59.226682+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:18:01.501617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195745+0000","last_change":"2024-03-22T04:20:58.208749+0000","last_active":"2024-03-22T04:20:59.195745+0000","last_peered":"2024-03-22T04:20:59.195745+0000","last_clean":"2024-03-22T04:20:59.195745+0000","last_became_active":"2024-03-22T04:20:58.208466+0000","last_became_peered":"2024-03-22T04:20:58.208466+0000","last_unstale":"2024-03-22T04:20:59.195745+0000","last_undegraded":"2024-03-22T04:20:59.195745+0000","last_fullsized":"2024-03-22T04:20:59.195745+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:05:47.974526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226533+0000","last_change":"2024-03-22T04:20:58.215495+0000","last_active":"2024-03-22T04:20:59.226533+0000","last_peered":"2024-03-22T04:20:59.226533+0000","last_clean":"2024-03-22T04:20:59.226533+0000","last_became_active":"2024-03-22T04:20:58.215196+0000","last_became_peered":"2024-03-22T04:20:58.215196+0000","last_unstale":"2024-03-22T04:20:59.226533+0000","last_undegraded":"2024-03-22T04:20:59.226533+0000","last_fullsized":"2024-03-22T04:20:59.226533+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:26:41.212828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193605+0000","last_change":"2024-03-22T04:20:58.217151+0000","last_active":"2024-03-22T04:20:59.193605+0000","last_peered":"2024-03-22T04:20:59.193605+0000","last_clean":"2024-03-22T04:20:59.193605+0000","last_became_active":"2024-03-22T04:20:58.216896+0000","last_became_peered":"2024-03-22T04:20:58.216896+0000","last_unstale":"2024-03-22T04:20:59.193605+0000","last_undegraded":"2024-03-22T04:20:59.193605+0000","last_fullsized":"2024-03-22T04:20:59.193605+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:20:00.087275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226289+0000","last_change":"2024-03-22T04:20:58.210999+0000","last_active":"2024-03-22T04:20:59.226289+0000","last_peered":"2024-03-22T04:20:59.226289+0000","last_clean":"2024-03-22T04:20:59.226289+0000","last_became_active":"2024-03-22T04:20:58.210694+0000","last_became_peered":"2024-03-22T04:20:58.210694+0000","last_unstale":"2024-03-22T04:20:59.226289+0000","last_undegraded":"2024-03-22T04:20:59.226289+0000","last_fullsized":"2024-03-22T04:20:59.226289+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:09:57.731456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226948+0000","last_change":"2024-03-22T04:20:58.216415+0000","last_active":"2024-03-22T04:20:59.226948+0000","last_peered":"2024-03-22T04:20:59.226948+0000","last_clean":"2024-03-22T04:20:59.226948+0000","last_became_active":"2024-03-22T04:20:58.216146+0000","last_became_peered":"2024-03-22T04:20:58.216146+0000","last_unstale":"2024-03-22T04:20:59.226948+0000","last_undegraded":"2024-03-22T04:20:59.226948+0000","last_fullsized":"2024-03-22T04:20:59.226948+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:25:15.119439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226844+0000","last_change":"2024-03-22T04:20:58.218473+0000","last_active":"2024-03-22T04:20:59.226844+0000","last_peered":"2024-03-22T04:20:59.226844+0000","last_clean":"2024-03-22T04:20:59.226844+0000","last_became_active":"2024-03-22T04:20:58.218215+0000","last_became_peered":"2024-03-22T04:20:58.218215+0000","last_unstale":"2024-03-22T04:20:59.226844+0000","last_undegraded":"2024-03-22T04:20:59.226844+0000","last_fullsized":"2024-03-22T04:20:59.226844+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:11:11.489439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224082+0000","last_change":"2024-03-22T04:20:58.213328+0000","last_active":"2024-03-22T04:20:59.224082+0000","last_peered":"2024-03-22T04:20:59.224082+0000","last_clean":"2024-03-22T04:20:59.224082+0000","last_became_active":"2024-03-22T04:20:58.212990+0000","last_became_peered":"2024-03-22T04:20:58.212990+0000","last_unstale":"2024-03-22T04:20:59.224082+0000","last_undegraded":"2024-03-22T04:20:59.224082+0000","last_fullsized":"2024-03-22T04:20:59.224082+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:22:01.029228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196131+0000","last_change":"2024-03-22T04:20:58.213193+0000","last_active":"2024-03-22T04:20:59.196131+0000","last_peered":"2024-03-22T04:20:59.196131+0000","last_clean":"2024-03-22T04:20:59.196131+0000","last_became_active":"2024-03-22T04:20:58.212934+0000","last_became_peered":"2024-03-22T04:20:58.212934+0000","last_unstale":"2024-03-22T04:20:59.196131+0000","last_undegraded":"2024-03-22T04:20:59.196131+0000","last_fullsized":"2024-03-22T04:20:59.196131+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:09:57.731456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223979+0000","last_change":"2024-03-22T04:20:58.203933+0000","last_active":"2024-03-22T04:20:59.223979+0000","last_peered":"2024-03-22T04:20:59.223979+0000","last_clean":"2024-03-22T04:20:59.223979+0000","last_became_active":"2024-03-22T04:20:58.203699+0000","last_became_peered":"2024-03-22T04:20:58.203699+0000","last_unstale":"2024-03-22T04:20:59.223979+0000","last_undegraded":"2024-03-22T04:20:59.223979+0000","last_fullsized":"2024-03-22T04:20:59.223979+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:42:45.630898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193903+0000","last_change":"2024-03-22T04:20:58.222182+0000","last_active":"2024-03-22T04:20:59.193903+0000","last_peered":"2024-03-22T04:20:59.193903+0000","last_clean":"2024-03-22T04:20:59.193903+0000","last_became_active":"2024-03-22T04:20:58.221950+0000","last_became_peered":"2024-03-22T04:20:58.221950+0000","last_unstale":"2024-03-22T04:20:59.193903+0000","last_undegraded":"2024-03-22T04:20:59.193903+0000","last_fullsized":"2024-03-22T04:20:59.193903+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:35:05.550551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193575+0000","last_change":"2024-03-22T04:20:58.216302+0000","last_active":"2024-03-22T04:20:59.193575+0000","last_peered":"2024-03-22T04:20:59.193575+0000","last_clean":"2024-03-22T04:20:59.193575+0000","last_became_active":"2024-03-22T04:20:58.215959+0000","last_became_peered":"2024-03-22T04:20:58.215959+0000","last_unstale":"2024-03-22T04:20:59.193575+0000","last_undegraded":"2024-03-22T04:20:59.193575+0000","last_fullsized":"2024-03-22T04:20:59.193575+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:05:26.957988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193490+0000","last_change":"2024-03-22T04:20:58.216552+0000","last_active":"2024-03-22T04:20:59.193490+0000","last_peered":"2024-03-22T04:20:59.193490+0000","last_clean":"2024-03-22T04:20:59.193490+0000","last_became_active":"2024-03-22T04:20:58.216268+0000","last_became_peered":"2024-03-22T04:20:58.216268+0000","last_unstale":"2024-03-22T04:20:59.193490+0000","last_undegraded":"2024-03-22T04:20:59.193490+0000","last_fullsized":"2024-03-22T04:20:59.193490+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:46:49.894553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798870+0000","last_change":"2024-03-22T04:20:58.216558+0000","last_active":"2024-03-22T04:21:00.798870+0000","last_peered":"2024-03-22T04:21:00.798870+0000","last_clean":"2024-03-22T04:21:00.798870+0000","last_became_active":"2024-03-22T04:20:58.216295+0000","last_became_peered":"2024-03-22T04:20:58.216295+0000","last_unstale":"2024-03-22T04:21:00.798870+0000","last_undegraded":"2024-03-22T04:21:00.798870+0000","last_fullsized":"2024-03-22T04:21:00.798870+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:24:12.302545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.200858+0000","last_change":"2024-03-22T04:20:58.199755+0000","last_active":"2024-03-22T04:20:58.200858+0000","last_peered":"2024-03-22T04:20:58.200858+0000","last_clean":"2024-03-22T04:20:58.200858+0000","last_became_active":"2024-03-22T04:20:58.199386+0000","last_became_peered":"2024-03-22T04:20:58.199386+0000","last_unstale":"2024-03-22T04:20:58.200858+0000","last_undegraded":"2024-03-22T04:20:58.200858+0000","last_fullsized":"2024-03-22T04:20:58.200858+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:36:39.078591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224181+0000","last_change":"2024-03-22T04:20:58.210557+0000","last_active":"2024-03-22T04:20:59.224181+0000","last_peered":"2024-03-22T04:20:59.224181+0000","last_clean":"2024-03-22T04:20:59.224181+0000","last_became_active":"2024-03-22T04:20:58.208480+0000","last_became_peered":"2024-03-22T04:20:58.208480+0000","last_unstale":"2024-03-22T04:20:59.224181+0000","last_undegraded":"2024-03-22T04:20:59.224181+0000","last_fullsized":"2024-03-22T04:20:59.224181+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:46:56.275840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226719+0000","last_change":"2024-03-22T04:20:58.211640+0000","last_active":"2024-03-22T04:20:59.226719+0000","last_peered":"2024-03-22T04:20:59.226719+0000","last_clean":"2024-03-22T04:20:59.226719+0000","last_became_active":"2024-03-22T04:20:58.210929+0000","last_became_peered":"2024-03-22T04:20:58.210929+0000","last_unstale":"2024-03-22T04:20:59.226719+0000","last_undegraded":"2024-03-22T04:20:59.226719+0000","last_fullsized":"2024-03-22T04:20:59.226719+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:41:42.186149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411220+0000","last_change":"2024-03-22T04:20:58.220302+0000","last_active":"2024-03-22T04:20:59.411220+0000","last_peered":"2024-03-22T04:20:59.411220+0000","last_clean":"2024-03-22T04:20:59.411220+0000","last_became_active":"2024-03-22T04:20:58.219851+0000","last_became_peered":"2024-03-22T04:20:58.219851+0000","last_unstale":"2024-03-22T04:20:59.411220+0000","last_undegraded":"2024-03-22T04:20:59.411220+0000","last_fullsized":"2024-03-22T04:20:59.411220+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:55:12.522706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411139+0000","last_change":"2024-03-22T04:20:58.213080+0000","last_active":"2024-03-22T04:20:59.411139+0000","last_peered":"2024-03-22T04:20:59.411139+0000","last_clean":"2024-03-22T04:20:59.411139+0000","last_became_active":"2024-03-22T04:20:58.212820+0000","last_became_peered":"2024-03-22T04:20:58.212820+0000","last_unstale":"2024-03-22T04:20:59.411139+0000","last_undegraded":"2024-03-22T04:20:59.411139+0000","last_fullsized":"2024-03-22T04:20:59.411139+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:04:44.118815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798445+0000","last_change":"2024-03-22T04:20:58.223008+0000","last_active":"2024-03-22T04:21:00.798445+0000","last_peered":"2024-03-22T04:21:00.798445+0000","last_clean":"2024-03-22T04:21:00.798445+0000","last_became_active":"2024-03-22T04:20:58.222522+0000","last_became_peered":"2024-03-22T04:20:58.222522+0000","last_unstale":"2024-03-22T04:21:00.798445+0000","last_undegraded":"2024-03-22T04:21:00.798445+0000","last_fullsized":"2024-03-22T04:21:00.798445+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:20:56.301080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798559+0000","last_change":"2024-03-22T04:20:58.212697+0000","last_active":"2024-03-22T04:21:00.798559+0000","last_peered":"2024-03-22T04:21:00.798559+0000","last_clean":"2024-03-22T04:21:00.798559+0000","last_became_active":"2024-03-22T04:20:58.212423+0000","last_became_peered":"2024-03-22T04:20:58.212423+0000","last_unstale":"2024-03-22T04:21:00.798559+0000","last_undegraded":"2024-03-22T04:21:00.798559+0000","last_fullsized":"2024-03-22T04:21:00.798559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:23:43.142833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.214890+0000","last_change":"2024-03-22T04:20:58.209494+0000","last_active":"2024-03-22T04:20:58.214890+0000","last_peered":"2024-03-22T04:20:58.214890+0000","last_clean":"2024-03-22T04:20:58.214890+0000","last_became_active":"2024-03-22T04:20:58.209211+0000","last_became_peered":"2024-03-22T04:20:58.209211+0000","last_unstale":"2024-03-22T04:20:58.214890+0000","last_undegraded":"2024-03-22T04:20:58.214890+0000","last_fullsized":"2024-03-22T04:20:58.214890+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:58:37.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"123'4","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224225+0000","last_change":"2024-03-22T04:20:57.203371+0000","last_active":"2024-03-22T04:20:59.224225+0000","last_peered":"2024-03-22T04:20:59.224225+0000","last_clean":"2024-03-22T04:20:59.224225+0000","last_became_active":"2024-03-22T04:20:57.203114+0000","last_became_peered":"2024-03-22T04:20:57.203114+0000","last_unstale":"2024-03-22T04:20:59.224225+0000","last_undegraded":"2024-03-22T04:20:59.224225+0000","last_fullsized":"2024-03-22T04:20:59.224225+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:35:57.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195594+0000","last_change":"2024-03-22T04:20:58.218672+0000","last_active":"2024-03-22T04:20:59.195594+0000","last_peered":"2024-03-22T04:20:59.195594+0000","last_clean":"2024-03-22T04:20:59.195594+0000","last_became_active":"2024-03-22T04:20:58.218386+0000","last_became_peered":"2024-03-22T04:20:58.218386+0000","last_unstale":"2024-03-22T04:20:59.195594+0000","last_undegraded":"2024-03-22T04:20:59.195594+0000","last_fullsized":"2024-03-22T04:20:59.195594+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:44.015270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226816+0000","last_change":"2024-03-22T04:20:57.199617+0000","last_active":"2024-03-22T04:20:59.226816+0000","last_peered":"2024-03-22T04:20:59.226816+0000","last_clean":"2024-03-22T04:20:59.226816+0000","last_became_active":"2024-03-22T04:20:57.198363+0000","last_became_peered":"2024-03-22T04:20:57.198363+0000","last_unstale":"2024-03-22T04:20:59.226816+0000","last_undegraded":"2024-03-22T04:20:59.226816+0000","last_fullsized":"2024-03-22T04:20:59.226816+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:35:04.542956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798800+0000","last_change":"2024-03-22T04:20:58.224091+0000","last_active":"2024-03-22T04:21:00.798800+0000","last_peered":"2024-03-22T04:21:00.798800+0000","last_clean":"2024-03-22T04:21:00.798800+0000","last_became_active":"2024-03-22T04:20:58.222720+0000","last_became_peered":"2024-03-22T04:20:58.222720+0000","last_unstale":"2024-03-22T04:21:00.798800+0000","last_undegraded":"2024-03-22T04:21:00.798800+0000","last_fullsized":"2024-03-22T04:21:00.798800+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:24:51.175665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193948+0000","last_change":"2024-03-22T04:20:57.201503+0000","last_active":"2024-03-22T04:20:59.193948+0000","last_peered":"2024-03-22T04:20:59.193948+0000","last_clean":"2024-03-22T04:20:59.193948+0000","last_became_active":"2024-03-22T04:20:57.200242+0000","last_became_peered":"2024-03-22T04:20:57.200242+0000","last_unstale":"2024-03-22T04:20:59.193948+0000","last_undegraded":"2024-03-22T04:20:59.193948+0000","last_fullsized":"2024-03-22T04:20:59.193948+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:35:57.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.211164+0000","last_change":"2024-03-22T04:20:58.201807+0000","last_active":"2024-03-22T04:20:58.211164+0000","last_peered":"2024-03-22T04:20:58.211164+0000","last_clean":"2024-03-22T04:20:58.211164+0000","last_became_active":"2024-03-22T04:20:58.201456+0000","last_became_peered":"2024-03-22T04:20:58.201456+0000","last_unstale":"2024-03-22T04:20:58.211164+0000","last_undegraded":"2024-03-22T04:20:58.211164+0000","last_fullsized":"2024-03-22T04:20:58.211164+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:17:05.954187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196127+0000","last_change":"2024-03-22T04:20:57.190089+0000","last_active":"2024-03-22T04:20:59.196127+0000","last_peered":"2024-03-22T04:20:59.196127+0000","last_clean":"2024-03-22T04:20:59.196127+0000","last_became_active":"2024-03-22T04:20:57.189816+0000","last_became_peered":"2024-03-22T04:20:57.189816+0000","last_unstale":"2024-03-22T04:20:59.196127+0000","last_undegraded":"2024-03-22T04:20:59.196127+0000","last_fullsized":"2024-03-22T04:20:59.196127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:40.563070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.211262+0000","last_change":"2024-03-22T04:20:58.206926+0000","last_active":"2024-03-22T04:20:58.211262+0000","last_peered":"2024-03-22T04:20:58.211262+0000","last_clean":"2024-03-22T04:20:58.211262+0000","last_became_active":"2024-03-22T04:20:58.206610+0000","last_became_peered":"2024-03-22T04:20:58.206610+0000","last_unstale":"2024-03-22T04:20:58.211262+0000","last_undegraded":"2024-03-22T04:20:58.211262+0000","last_fullsized":"2024-03-22T04:20:58.211262+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:20:44.479262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798590+0000","last_change":"2024-03-22T04:20:57.203428+0000","last_active":"2024-03-22T04:21:00.798590+0000","last_peered":"2024-03-22T04:21:00.798590+0000","last_clean":"2024-03-22T04:21:00.798590+0000","last_became_active":"2024-03-22T04:20:57.203140+0000","last_became_peered":"2024-03-22T04:20:57.203140+0000","last_unstale":"2024-03-22T04:21:00.798590+0000","last_undegraded":"2024-03-22T04:21:00.798590+0000","last_fullsized":"2024-03-22T04:21:00.798590+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:11:29.959142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798466+0000","last_change":"2024-03-22T04:20:58.211986+0000","last_active":"2024-03-22T04:21:00.798466+0000","last_peered":"2024-03-22T04:21:00.798466+0000","last_clean":"2024-03-22T04:21:00.798466+0000","last_became_active":"2024-03-22T04:20:58.211715+0000","last_became_peered":"2024-03-22T04:20:58.211715+0000","last_unstale":"2024-03-22T04:21:00.798466+0000","last_undegraded":"2024-03-22T04:21:00.798466+0000","last_fullsized":"2024-03-22T04:21:00.798466+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:00.567483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"123'1","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.235010+0000","last_change":"2024-03-22T04:20:57.197586+0000","last_active":"2024-03-22T04:20:58.235010+0000","last_peered":"2024-03-22T04:20:58.235010+0000","last_clean":"2024-03-22T04:20:58.235010+0000","last_became_active":"2024-03-22T04:20:57.197287+0000","last_became_peered":"2024-03-22T04:20:57.197287+0000","last_unstale":"2024-03-22T04:20:58.235010+0000","last_undegraded":"2024-03-22T04:20:58.235010+0000","last_fullsized":"2024-03-22T04:20:58.235010+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:09:51.909709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411113+0000","last_change":"2024-03-22T04:20:58.215650+0000","last_active":"2024-03-22T04:20:59.411113+0000","last_peered":"2024-03-22T04:20:59.411113+0000","last_clean":"2024-03-22T04:20:59.411113+0000","last_became_active":"2024-03-22T04:20:58.215236+0000","last_became_peered":"2024-03-22T04:20:58.215236+0000","last_unstale":"2024-03-22T04:20:59.411113+0000","last_undegraded":"2024-03-22T04:20:59.411113+0000","last_fullsized":"2024-03-22T04:20:59.411113+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:44:59.790696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798543+0000","last_change":"2024-03-22T04:20:57.198172+0000","last_active":"2024-03-22T04:21:00.798543+0000","last_peered":"2024-03-22T04:21:00.798543+0000","last_clean":"2024-03-22T04:21:00.798543+0000","last_became_active":"2024-03-22T04:20:57.197912+0000","last_became_peered":"2024-03-22T04:20:57.197912+0000","last_unstale":"2024-03-22T04:21:00.798543+0000","last_undegraded":"2024-03-22T04:21:00.798543+0000","last_fullsized":"2024-03-22T04:21:00.798543+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:25:14.111844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.219026+0000","last_change":"2024-03-22T04:20:58.214509+0000","last_active":"2024-03-22T04:20:58.219026+0000","last_peered":"2024-03-22T04:20:58.219026+0000","last_clean":"2024-03-22T04:20:58.219026+0000","last_became_active":"2024-03-22T04:20:58.214238+0000","last_became_peered":"2024-03-22T04:20:58.214238+0000","last_unstale":"2024-03-22T04:20:58.219026+0000","last_undegraded":"2024-03-22T04:20:58.219026+0000","last_fullsized":"2024-03-22T04:20:58.219026+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:56:57.403406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411040+0000","last_change":"2024-03-22T04:20:57.205148+0000","last_active":"2024-03-22T04:20:59.411040+0000","last_peered":"2024-03-22T04:20:59.411040+0000","last_clean":"2024-03-22T04:20:59.411040+0000","last_became_active":"2024-03-22T04:20:57.204836+0000","last_became_peered":"2024-03-22T04:20:57.204836+0000","last_unstale":"2024-03-22T04:20:59.411040+0000","last_undegraded":"2024-03-22T04:20:59.411040+0000","last_fullsized":"2024-03-22T04:20:59.411040+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:16:28.187381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193798+0000","last_change":"2024-03-22T04:20:58.217853+0000","last_active":"2024-03-22T04:20:59.193798+0000","last_peered":"2024-03-22T04:20:59.193798+0000","last_clean":"2024-03-22T04:20:59.193798+0000","last_became_active":"2024-03-22T04:20:58.217567+0000","last_became_peered":"2024-03-22T04:20:58.217567+0000","last_unstale":"2024-03-22T04:20:59.193798+0000","last_undegraded":"2024-03-22T04:20:59.193798+0000","last_fullsized":"2024-03-22T04:20:59.193798+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:59:50.535061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.194941+0000","last_change":"2024-03-22T04:20:57.201973+0000","last_active":"2024-03-22T04:20:58.194941+0000","last_peered":"2024-03-22T04:20:58.194941+0000","last_clean":"2024-03-22T04:20:58.194941+0000","last_became_active":"2024-03-22T04:20:57.201696+0000","last_became_peered":"2024-03-22T04:20:57.201696+0000","last_unstale":"2024-03-22T04:20:58.194941+0000","last_undegraded":"2024-03-22T04:20:58.194941+0000","last_fullsized":"2024-03-22T04:20:58.194941+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:21:15.643220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.210548+0000","last_change":"2024-03-22T04:20:58.202110+0000","last_active":"2024-03-22T04:20:58.210548+0000","last_peered":"2024-03-22T04:20:58.210548+0000","last_clean":"2024-03-22T04:20:58.210548+0000","last_became_active":"2024-03-22T04:20:58.201843+0000","last_became_peered":"2024-03-22T04:20:58.201843+0000","last_unstale":"2024-03-22T04:20:58.210548+0000","last_undegraded":"2024-03-22T04:20:58.210548+0000","last_fullsized":"2024-03-22T04:20:58.210548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:46:49.894553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.194484+0000","last_change":"2024-03-22T04:20:57.202784+0000","last_active":"2024-03-22T04:20:58.194484+0000","last_peered":"2024-03-22T04:20:58.194484+0000","last_clean":"2024-03-22T04:20:58.194484+0000","last_became_active":"2024-03-22T04:20:57.202347+0000","last_became_peered":"2024-03-22T04:20:57.202347+0000","last_unstale":"2024-03-22T04:20:58.194484+0000","last_undegraded":"2024-03-22T04:20:58.194484+0000","last_fullsized":"2024-03-22T04:20:58.194484+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:04:29.099232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411464+0000","last_change":"2024-03-22T04:20:58.213824+0000","last_active":"2024-03-22T04:20:59.411464+0000","last_peered":"2024-03-22T04:20:59.411464+0000","last_clean":"2024-03-22T04:20:59.411464+0000","last_became_active":"2024-03-22T04:20:58.213507+0000","last_became_peered":"2024-03-22T04:20:58.213507+0000","last_unstale":"2024-03-22T04:20:59.411464+0000","last_undegraded":"2024-03-22T04:20:59.411464+0000","last_fullsized":"2024-03-22T04:20:59.411464+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:43:23.051623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"123'3","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798884+0000","last_change":"2024-03-22T04:20:57.197449+0000","last_active":"2024-03-22T04:21:00.798884+0000","last_peered":"2024-03-22T04:21:00.798884+0000","last_clean":"2024-03-22T04:21:00.798884+0000","last_became_active":"2024-03-22T04:20:57.197182+0000","last_became_peered":"2024-03-22T04:20:57.197182+0000","last_unstale":"2024-03-22T04:21:00.798884+0000","last_undegraded":"2024-03-22T04:21:00.798884+0000","last_fullsized":"2024-03-22T04:21:00.798884+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:14:11.874875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224063+0000","last_change":"2024-03-22T04:20:58.209311+0000","last_active":"2024-03-22T04:20:59.224063+0000","last_peered":"2024-03-22T04:20:59.224063+0000","last_clean":"2024-03-22T04:20:59.224063+0000","last_became_active":"2024-03-22T04:20:58.209060+0000","last_became_peered":"2024-03-22T04:20:58.209060+0000","last_unstale":"2024-03-22T04:20:59.224063+0000","last_undegraded":"2024-03-22T04:20:59.224063+0000","last_fullsized":"2024-03-22T04:20:59.224063+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:22:53.658661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"123'6","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196236+0000","last_change":"2024-03-22T04:20:57.197731+0000","last_active":"2024-03-22T04:20:59.196236+0000","last_peered":"2024-03-22T04:20:59.196236+0000","last_clean":"2024-03-22T04:20:59.196236+0000","last_became_active":"2024-03-22T04:20:57.197075+0000","last_became_peered":"2024-03-22T04:20:57.197075+0000","last_unstale":"2024-03-22T04:20:59.196236+0000","last_undegraded":"2024-03-22T04:20:59.196236+0000","last_fullsized":"2024-03-22T04:20:59.196236+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:04:29.099232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223598+0000","last_change":"2024-03-22T04:20:58.205356+0000","last_active":"2024-03-22T04:20:59.223598+0000","last_peered":"2024-03-22T04:20:59.223598+0000","last_clean":"2024-03-22T04:20:59.223598+0000","last_became_active":"2024-03-22T04:20:58.205084+0000","last_became_peered":"2024-03-22T04:20:58.205084+0000","last_unstale":"2024-03-22T04:20:59.223598+0000","last_undegraded":"2024-03-22T04:20:59.223598+0000","last_fullsized":"2024-03-22T04:20:59.223598+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:47:25.460863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"123'2","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.234948+0000","last_change":"2024-03-22T04:20:57.201231+0000","last_active":"2024-03-22T04:20:58.234948+0000","last_peered":"2024-03-22T04:20:58.234948+0000","last_clean":"2024-03-22T04:20:58.234948+0000","last_became_active":"2024-03-22T04:20:57.200959+0000","last_became_peered":"2024-03-22T04:20:57.200959+0000","last_unstale":"2024-03-22T04:20:58.234948+0000","last_undegraded":"2024-03-22T04:20:58.234948+0000","last_fullsized":"2024-03-22T04:20:58.234948+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:59:50.014269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195357+0000","last_change":"2024-03-22T04:20:58.204421+0000","last_active":"2024-03-22T04:20:59.195357+0000","last_peered":"2024-03-22T04:20:59.195357+0000","last_clean":"2024-03-22T04:20:59.195357+0000","last_became_active":"2024-03-22T04:20:58.204126+0000","last_became_peered":"2024-03-22T04:20:58.204126+0000","last_unstale":"2024-03-22T04:20:59.195357+0000","last_undegraded":"2024-03-22T04:20:59.195357+0000","last_fullsized":"2024-03-22T04:20:59.195357+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:22:01.029228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411381+0000","last_change":"2024-03-22T04:20:57.206482+0000","last_active":"2024-03-22T04:20:59.411381+0000","last_peered":"2024-03-22T04:20:59.411381+0000","last_clean":"2024-03-22T04:20:59.411381+0000","last_became_active":"2024-03-22T04:20:57.206022+0000","last_became_peered":"2024-03-22T04:20:57.206022+0000","last_unstale":"2024-03-22T04:20:59.411381+0000","last_undegraded":"2024-03-22T04:20:59.411381+0000","last_fullsized":"2024-03-22T04:20:59.411381+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:23:50.299030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226598+0000","last_change":"2024-03-22T04:20:58.213422+0000","last_active":"2024-03-22T04:20:59.226598+0000","last_peered":"2024-03-22T04:20:59.226598+0000","last_clean":"2024-03-22T04:20:59.226598+0000","last_became_active":"2024-03-22T04:20:58.212246+0000","last_became_peered":"2024-03-22T04:20:58.212246+0000","last_unstale":"2024-03-22T04:20:59.226598+0000","last_undegraded":"2024-03-22T04:20:59.226598+0000","last_fullsized":"2024-03-22T04:20:59.226598+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:26:00.354215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.195727+0000","last_change":"2024-03-22T04:20:57.199212+0000","last_active":"2024-03-22T04:20:58.195727+0000","last_peered":"2024-03-22T04:20:58.195727+0000","last_clean":"2024-03-22T04:20:58.195727+0000","last_became_active":"2024-03-22T04:20:57.198951+0000","last_became_peered":"2024-03-22T04:20:57.198951+0000","last_unstale":"2024-03-22T04:20:58.195727+0000","last_undegraded":"2024-03-22T04:20:58.195727+0000","last_fullsized":"2024-03-22T04:20:58.195727+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:18:34.331983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.212353+0000","last_change":"2024-03-22T04:20:58.206196+0000","last_active":"2024-03-22T04:20:58.212353+0000","last_peered":"2024-03-22T04:20:58.212353+0000","last_clean":"2024-03-22T04:20:58.212353+0000","last_became_active":"2024-03-22T04:20:58.205879+0000","last_became_peered":"2024-03-22T04:20:58.205879+0000","last_unstale":"2024-03-22T04:20:58.212353+0000","last_undegraded":"2024-03-22T04:20:58.212353+0000","last_fullsized":"2024-03-22T04:20:58.212353+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:05:26.957988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226995+0000","last_change":"2024-03-22T04:20:57.202964+0000","last_active":"2024-03-22T04:20:59.226995+0000","last_peered":"2024-03-22T04:20:59.226995+0000","last_clean":"2024-03-22T04:20:59.226995+0000","last_became_active":"2024-03-22T04:20:57.201717+0000","last_became_peered":"2024-03-22T04:20:57.201717+0000","last_unstale":"2024-03-22T04:20:59.226995+0000","last_undegraded":"2024-03-22T04:20:59.226995+0000","last_fullsized":"2024-03-22T04:20:59.226995+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:59:49.527465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798846+0000","last_change":"2024-03-22T04:20:58.206088+0000","last_active":"2024-03-22T04:21:00.798846+0000","last_peered":"2024-03-22T04:21:00.798846+0000","last_clean":"2024-03-22T04:21:00.798846+0000","last_became_active":"2024-03-22T04:20:58.205833+0000","last_became_peered":"2024-03-22T04:20:58.205833+0000","last_unstale":"2024-03-22T04:21:00.798846+0000","last_undegraded":"2024-03-22T04:21:00.798846+0000","last_fullsized":"2024-03-22T04:21:00.798846+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T08:00:45.333754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226958+0000","last_change":"2024-03-22T04:20:57.197246+0000","last_active":"2024-03-22T04:20:59.226958+0000","last_peered":"2024-03-22T04:20:59.226958+0000","last_clean":"2024-03-22T04:20:59.226958+0000","last_became_active":"2024-03-22T04:20:57.196926+0000","last_became_peered":"2024-03-22T04:20:57.196926+0000","last_unstale":"2024-03-22T04:20:59.226958+0000","last_undegraded":"2024-03-22T04:20:59.226958+0000","last_fullsized":"2024-03-22T04:20:59.226958+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:47:24.453267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.196346+0000","last_change":"2024-03-22T04:20:58.215460+0000","last_active":"2024-03-22T04:20:59.196346+0000","last_peered":"2024-03-22T04:20:59.196346+0000","last_clean":"2024-03-22T04:20:59.196346+0000","last_became_active":"2024-03-22T04:20:58.214641+0000","last_became_peered":"2024-03-22T04:20:58.214641+0000","last_unstale":"2024-03-22T04:20:59.196346+0000","last_undegraded":"2024-03-22T04:20:59.196346+0000","last_fullsized":"2024-03-22T04:20:59.196346+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:08:35.684362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224233+0000","last_change":"2024-03-22T04:20:57.195249+0000","last_active":"2024-03-22T04:20:59.224233+0000","last_peered":"2024-03-22T04:20:59.224233+0000","last_clean":"2024-03-22T04:20:59.224233+0000","last_became_active":"2024-03-22T04:20:57.194917+0000","last_became_peered":"2024-03-22T04:20:57.194917+0000","last_unstale":"2024-03-22T04:20:59.224233+0000","last_undegraded":"2024-03-22T04:20:59.224233+0000","last_fullsized":"2024-03-22T04:20:59.224233+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:40.563070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224138+0000","last_change":"2024-03-22T04:20:58.215991+0000","last_active":"2024-03-22T04:20:59.224138+0000","last_peered":"2024-03-22T04:20:59.224138+0000","last_clean":"2024-03-22T04:20:59.224138+0000","last_became_active":"2024-03-22T04:20:58.213718+0000","last_became_peered":"2024-03-22T04:20:58.213718+0000","last_unstale":"2024-03-22T04:20:59.224138+0000","last_undegraded":"2024-03-22T04:20:59.224138+0000","last_fullsized":"2024-03-22T04:20:59.224138+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:23:44.015270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798671+0000","last_change":"2024-03-22T04:20:57.199018+0000","last_active":"2024-03-22T04:21:00.798671+0000","last_peered":"2024-03-22T04:21:00.798671+0000","last_clean":"2024-03-22T04:21:00.798671+0000","last_became_active":"2024-03-22T04:20:57.198781+0000","last_became_peered":"2024-03-22T04:20:57.198781+0000","last_unstale":"2024-03-22T04:21:00.798671+0000","last_undegraded":"2024-03-22T04:21:00.798671+0000","last_fullsized":"2024-03-22T04:21:00.798671+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:44:56.457026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.194931+0000","last_change":"2024-03-22T04:20:58.209629+0000","last_active":"2024-03-22T04:20:59.194931+0000","last_peered":"2024-03-22T04:20:59.194931+0000","last_clean":"2024-03-22T04:20:59.194931+0000","last_became_active":"2024-03-22T04:20:58.209253+0000","last_became_peered":"2024-03-22T04:20:58.209253+0000","last_unstale":"2024-03-22T04:20:59.194931+0000","last_undegraded":"2024-03-22T04:20:59.194931+0000","last_fullsized":"2024-03-22T04:20:59.194931+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:42:45.630898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226477+0000","last_change":"2024-03-22T04:20:57.199769+0000","last_active":"2024-03-22T04:20:59.226477+0000","last_peered":"2024-03-22T04:20:59.226477+0000","last_clean":"2024-03-22T04:20:59.226477+0000","last_became_active":"2024-03-22T04:20:57.199002+0000","last_became_peered":"2024-03-22T04:20:57.199002+0000","last_unstale":"2024-03-22T04:20:59.226477+0000","last_undegraded":"2024-03-22T04:20:59.226477+0000","last_fullsized":"2024-03-22T04:20:59.226477+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:05:25.950393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226474+0000","last_change":"2024-03-22T04:20:58.215712+0000","last_active":"2024-03-22T04:20:59.226474+0000","last_peered":"2024-03-22T04:20:59.226474+0000","last_clean":"2024-03-22T04:20:59.226474+0000","last_became_active":"2024-03-22T04:20:58.215441+0000","last_became_peered":"2024-03-22T04:20:58.215441+0000","last_unstale":"2024-03-22T04:20:59.226474+0000","last_undegraded":"2024-03-22T04:20:59.226474+0000","last_fullsized":"2024-03-22T04:20:59.226474+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T14:08:35.684362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193706+0000","last_change":"2024-03-22T04:20:57.199824+0000","last_active":"2024-03-22T04:20:59.193706+0000","last_peered":"2024-03-22T04:20:59.193706+0000","last_clean":"2024-03-22T04:20:59.193706+0000","last_became_active":"2024-03-22T04:20:57.199517+0000","last_became_peered":"2024-03-22T04:20:57.199517+0000","last_unstale":"2024-03-22T04:20:59.193706+0000","last_undegraded":"2024-03-22T04:20:59.193706+0000","last_fullsized":"2024-03-22T04:20:59.193706+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:06:07.301146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411273+0000","last_change":"2024-03-22T04:20:58.215453+0000","last_active":"2024-03-22T04:20:59.411273+0000","last_peered":"2024-03-22T04:20:59.411273+0000","last_clean":"2024-03-22T04:20:59.411273+0000","last_became_active":"2024-03-22T04:20:58.214882+0000","last_became_peered":"2024-03-22T04:20:58.214882+0000","last_unstale":"2024-03-22T04:20:59.411273+0000","last_undegraded":"2024-03-22T04:20:59.411273+0000","last_fullsized":"2024-03-22T04:20:59.411273+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:20:10.703932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411036+0000","last_change":"2024-03-22T04:20:57.205967+0000","last_active":"2024-03-22T04:20:59.411036+0000","last_peered":"2024-03-22T04:20:59.411036+0000","last_clean":"2024-03-22T04:20:59.411036+0000","last_became_active":"2024-03-22T04:20:57.204791+0000","last_became_peered":"2024-03-22T04:20:57.204791+0000","last_unstale":"2024-03-22T04:20:59.411036+0000","last_undegraded":"2024-03-22T04:20:59.411036+0000","last_fullsized":"2024-03-22T04:20:59.411036+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:33:56.806429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:21:00.798539+0000","last_change":"2024-03-22T04:20:58.207851+0000","last_active":"2024-03-22T04:21:00.798539+0000","last_peered":"2024-03-22T04:21:00.798539+0000","last_clean":"2024-03-22T04:21:00.798539+0000","last_became_active":"2024-03-22T04:20:58.207593+0000","last_became_peered":"2024-03-22T04:20:58.207593+0000","last_unstale":"2024-03-22T04:21:00.798539+0000","last_undegraded":"2024-03-22T04:21:00.798539+0000","last_fullsized":"2024-03-22T04:21:00.798539+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T06:40:00.374676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223523+0000","last_change":"2024-03-22T04:20:57.192045+0000","last_active":"2024-03-22T04:20:59.223523+0000","last_peered":"2024-03-22T04:20:59.223523+0000","last_clean":"2024-03-22T04:20:59.223523+0000","last_became_active":"2024-03-22T04:20:57.191382+0000","last_became_peered":"2024-03-22T04:20:57.191382+0000","last_unstale":"2024-03-22T04:20:59.223523+0000","last_undegraded":"2024-03-22T04:20:59.223523+0000","last_fullsized":"2024-03-22T04:20:59.223523+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:43:22.044028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226852+0000","last_change":"2024-03-22T04:20:58.213414+0000","last_active":"2024-03-22T04:20:59.226852+0000","last_peered":"2024-03-22T04:20:59.226852+0000","last_clean":"2024-03-22T04:20:59.226852+0000","last_became_active":"2024-03-22T04:20:58.213152+0000","last_became_peered":"2024-03-22T04:20:58.213152+0000","last_unstale":"2024-03-22T04:20:59.226852+0000","last_undegraded":"2024-03-22T04:20:59.226852+0000","last_fullsized":"2024-03-22T04:20:59.226852+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:23:45.898765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195942+0000","last_change":"2024-03-22T04:20:57.203732+0000","last_active":"2024-03-22T04:20:59.195942+0000","last_peered":"2024-03-22T04:20:59.195942+0000","last_clean":"2024-03-22T04:20:59.195942+0000","last_became_active":"2024-03-22T04:20:57.203286+0000","last_became_peered":"2024-03-22T04:20:57.203286+0000","last_unstale":"2024-03-22T04:20:59.195942+0000","last_undegraded":"2024-03-22T04:20:59.195942+0000","last_fullsized":"2024-03-22T04:20:59.195942+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:21:15.643220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411411+0000","last_change":"2024-03-22T04:20:58.224052+0000","last_active":"2024-03-22T04:20:59.411411+0000","last_peered":"2024-03-22T04:20:59.411411+0000","last_clean":"2024-03-22T04:20:59.411411+0000","last_became_active":"2024-03-22T04:20:58.223821+0000","last_became_peered":"2024-03-22T04:20:58.223821+0000","last_unstale":"2024-03-22T04:20:59.411411+0000","last_undegraded":"2024-03-22T04:20:59.411411+0000","last_fullsized":"2024-03-22T04:20:59.411411+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:34:10.619536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"124'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.225639+0000","last_change":"2024-03-22T04:20:57.191274+0000","last_active":"2024-03-22T04:20:59.225639+0000","last_peered":"2024-03-22T04:20:59.225639+0000","last_clean":"2024-03-22T04:20:59.225639+0000","last_became_active":"2024-03-22T04:20:57.188254+0000","last_became_peered":"2024-03-22T04:20:57.188254+0000","last_unstale":"2024-03-22T04:20:59.225639+0000","last_undegraded":"2024-03-22T04:20:59.225639+0000","last_fullsized":"2024-03-22T04:20:59.225639+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:35:43.709680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.210700+0000","last_change":"2024-03-22T04:20:58.201237+0000","last_active":"2024-03-22T04:20:58.210700+0000","last_peered":"2024-03-22T04:20:58.210700+0000","last_clean":"2024-03-22T04:20:58.210700+0000","last_became_active":"2024-03-22T04:20:58.200966+0000","last_became_peered":"2024-03-22T04:20:58.200966+0000","last_unstale":"2024-03-22T04:20:58.210700+0000","last_undegraded":"2024-03-22T04:20:58.210700+0000","last_fullsized":"2024-03-22T04:20:58.210700+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:42:08.739393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.224269+0000","last_change":"2024-03-22T04:20:57.201573+0000","last_active":"2024-03-22T04:20:59.224269+0000","last_peered":"2024-03-22T04:20:59.224269+0000","last_clean":"2024-03-22T04:20:59.224269+0000","last_became_active":"2024-03-22T04:20:57.201167+0000","last_became_peered":"2024-03-22T04:20:57.201167+0000","last_unstale":"2024-03-22T04:20:59.224269+0000","last_undegraded":"2024-03-22T04:20:59.224269+0000","last_fullsized":"2024-03-22T04:20:59.224269+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:06:07.301146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.411495+0000","last_change":"2024-03-22T04:20:58.216652+0000","last_active":"2024-03-22T04:20:59.411495+0000","last_peered":"2024-03-22T04:20:59.411495+0000","last_clean":"2024-03-22T04:20:59.411495+0000","last_became_active":"2024-03-22T04:20:58.216394+0000","last_became_peered":"2024-03-22T04:20:58.216394+0000","last_unstale":"2024-03-22T04:20:59.411495+0000","last_undegraded":"2024-03-22T04:20:59.411495+0000","last_fullsized":"2024-03-22T04:20:59.411495+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:40.654890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226873+0000","last_change":"2024-03-22T04:20:57.201875+0000","last_active":"2024-03-22T04:20:59.226873+0000","last_peered":"2024-03-22T04:20:59.226873+0000","last_clean":"2024-03-22T04:20:59.226873+0000","last_became_active":"2024-03-22T04:20:57.201390+0000","last_became_peered":"2024-03-22T04:20:57.201390+0000","last_unstale":"2024-03-22T04:20:59.226873+0000","last_undegraded":"2024-03-22T04:20:59.226873+0000","last_fullsized":"2024-03-22T04:20:59.226873+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:19:59.079679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.226409+0000","last_change":"2024-03-22T04:20:58.212721+0000","last_active":"2024-03-22T04:20:59.226409+0000","last_peered":"2024-03-22T04:20:59.226409+0000","last_clean":"2024-03-22T04:20:59.226409+0000","last_became_active":"2024-03-22T04:20:58.212353+0000","last_became_peered":"2024-03-22T04:20:58.212353+0000","last_unstale":"2024-03-22T04:20:59.226409+0000","last_undegraded":"2024-03-22T04:20:59.226409+0000","last_fullsized":"2024-03-22T04:20:59.226409+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T13:11:12.810457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223985+0000","last_change":"2024-03-22T04:20:57.203310+0000","last_active":"2024-03-22T04:20:59.223985+0000","last_peered":"2024-03-22T04:20:59.223985+0000","last_clean":"2024-03-22T04:20:59.223985+0000","last_became_active":"2024-03-22T04:20:57.203005+0000","last_became_peered":"2024-03-22T04:20:57.203005+0000","last_unstale":"2024-03-22T04:20:59.223985+0000","last_undegraded":"2024-03-22T04:20:59.223985+0000","last_fullsized":"2024-03-22T04:20:59.223985+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T12:30:21.983198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195192+0000","last_change":"2024-03-22T04:20:58.204089+0000","last_active":"2024-03-22T04:20:59.195192+0000","last_peered":"2024-03-22T04:20:59.195192+0000","last_clean":"2024-03-22T04:20:59.195192+0000","last_became_active":"2024-03-22T04:20:58.203780+0000","last_became_peered":"2024-03-22T04:20:58.203780+0000","last_unstale":"2024-03-22T04:20:59.195192+0000","last_undegraded":"2024-03-22T04:20:59.195192+0000","last_fullsized":"2024-03-22T04:20:59.195192+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:22:53.658661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193894+0000","last_change":"2024-03-22T04:20:57.201032+0000","last_active":"2024-03-22T04:20:59.193894+0000","last_peered":"2024-03-22T04:20:59.193894+0000","last_clean":"2024-03-22T04:20:59.193894+0000","last_became_active":"2024-03-22T04:20:57.200787+0000","last_became_peered":"2024-03-22T04:20:57.200787+0000","last_unstale":"2024-03-22T04:20:59.193894+0000","last_undegraded":"2024-03-22T04:20:59.193894+0000","last_fullsized":"2024-03-22T04:20:59.193894+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:26:40.563070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223812+0000","last_change":"2024-03-22T04:20:58.212663+0000","last_active":"2024-03-22T04:20:59.223812+0000","last_peered":"2024-03-22T04:20:59.223812+0000","last_clean":"2024-03-22T04:20:59.223812+0000","last_became_active":"2024-03-22T04:20:58.210906+0000","last_became_peered":"2024-03-22T04:20:58.210906+0000","last_unstale":"2024-03-22T04:20:59.223812+0000","last_undegraded":"2024-03-22T04:20:59.223812+0000","last_fullsized":"2024-03-22T04:20:59.223812+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T09:30:21.470056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.195468+0000","last_change":"2024-03-22T04:20:57.197383+0000","last_active":"2024-03-22T04:20:59.195468+0000","last_peered":"2024-03-22T04:20:59.195468+0000","last_clean":"2024-03-22T04:20:59.195468+0000","last_became_active":"2024-03-22T04:20:57.196699+0000","last_became_peered":"2024-03-22T04:20:57.196699+0000","last_unstale":"2024-03-22T04:20:59.195468+0000","last_undegraded":"2024-03-22T04:20:59.195468+0000","last_fullsized":"2024-03-22T04:20:59.195468+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:42:07.731797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.206243+0000","last_change":"2024-03-22T04:20:58.203871+0000","last_active":"2024-03-22T04:20:58.206243+0000","last_peered":"2024-03-22T04:20:58.206243+0000","last_clean":"2024-03-22T04:20:58.206243+0000","last_became_active":"2024-03-22T04:20:58.203566+0000","last_became_peered":"2024-03-22T04:20:58.203566+0000","last_unstale":"2024-03-22T04:20:58.206243+0000","last_undegraded":"2024-03-22T04:20:58.206243+0000","last_fullsized":"2024-03-22T04:20:58.206243+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:39:15.432447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223578+0000","last_change":"2024-03-22T04:20:57.192242+0000","last_active":"2024-03-22T04:20:59.223578+0000","last_peered":"2024-03-22T04:20:59.223578+0000","last_clean":"2024-03-22T04:20:59.223578+0000","last_became_active":"2024-03-22T04:20:57.191762+0000","last_became_peered":"2024-03-22T04:20:57.191762+0000","last_unstale":"2024-03-22T04:20:59.223578+0000","last_undegraded":"2024-03-22T04:20:59.223578+0000","last_fullsized":"2024-03-22T04:20:59.223578+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:37:39.647294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.223902+0000","last_change":"2024-03-22T04:20:58.214907+0000","last_active":"2024-03-22T04:20:59.223902+0000","last_peered":"2024-03-22T04:20:59.223902+0000","last_clean":"2024-03-22T04:20:59.223902+0000","last_became_active":"2024-03-22T04:20:58.214651+0000","last_became_peered":"2024-03-22T04:20:58.214651+0000","last_unstale":"2024-03-22T04:20:59.223902+0000","last_undegraded":"2024-03-22T04:20:59.223902+0000","last_fullsized":"2024-03-22T04:20:59.223902+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:57.173517+0000","last_clean_scrub_stamp":"2024-03-22T04:20:57.173517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T10:26:42.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"123'8","reported_seq":20,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-03-22T04:20:58.237747+0000","last_change":"2024-03-22T04:20:57.196319+0000","last_active":"2024-03-22T04:20:58.237747+0000","last_peered":"2024-03-22T04:20:58.237747+0000","last_clean":"2024-03-22T04:20:58.237747+0000","last_became_active":"2024-03-22T04:20:57.195923+0000","last_became_peered":"2024-03-22T04:20:57.195923+0000","last_unstale":"2024-03-22T04:20:58.237747+0000","last_undegraded":"2024-03-22T04:20:58.237747+0000","last_fullsized":"2024-03-22T04:20:58.237747+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:20:56.165922+0000","last_clean_scrub_stamp":"2024-03-22T04:20:56.165922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T05:51:23.862475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":361,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-03-22T04:20:59.193372+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:20:59.193372+0000","last_peered":"2024-03-22T04:20:59.193372+0000","last_clean":"2024-03-22T04:20:59.193372+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:20:59.193372+0000","last_undegraded":"2024-03-22T04:20:59.193372+0000","last_fullsized":"2024-03-22T04:20:59.193372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509571,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1536000,"data_stored":718296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509570,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1540096,"data_stored":718318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509571,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":1310276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509571,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":711694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509571,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":713798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509571,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2138112,"data_stored":1308824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509571,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1548288,"data_stored":718982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1531904,"data_stored":711660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:21:02.790 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-22T04:21:02.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-22T04:21:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.971+0000 7f9670c00640 1 Processor -- start 2024-03-22T04:21:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.971+0000 7f9670c00640 1 -- start start 2024-03-22T04:21:02.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9670c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f966c0cf0d0 0x7f966c072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9670c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f966c072970 0x7f966c1625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9670c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f966c0d0550 con 0x7f966c0d0170 2024-03-22T04:21:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9670c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f966c1620a0 con 0x7f966c0cf0d0 2024-03-22T04:21:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9670c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f966c15f110 con 0x7f966c072970 2024-03-22T04:21:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9663e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f966c0cf0d0 0x7f966c072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9663e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f966c0cf0d0 0x7f966c072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33994/0 (socket says 172.21.15.86:33994) 2024-03-22T04:21:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9663e00640 1 -- 172.21.15.86:0/1694891134 learned_addr learned my addr 172.21.15.86:0/1694891134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f966be00640 1 --2- 172.21.15.86:0/1694891134 >> v2:172.21.15.196:3300/0 conn(0x7f966c072970 0x7f966c1625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9663e00640 1 -- 172.21.15.86:0/1694891134 >> v2:172.21.15.196:3300/0 conn(0x7f966c072970 msgr2=0x7f966c1625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:02.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9663e00640 1 --2- 172.21.15.86:0/1694891134 >> v2:172.21.15.196:3300/0 conn(0x7f966c072970 0x7f966c1625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9663e00640 1 -- 172.21.15.86:0/1694891134 >> v1:172.21.15.196:6789/0 conn(0x7f966c0d0170 legacy=0x7f966c15e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:02.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.975+0000 7f9663e00640 1 -- 172.21.15.86:0/1694891134 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c15f3f0 con 0x7f966c0cf0d0 2024-03-22T04:21:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9663e00640 1 --2- 172.21.15.86:0/1694891134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c0cf0d0 0x7f966c072430 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f966c0cd5d0 tx=0x7f965401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e9b855d0e60f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f966aa00640 1 -- 172.21.15.86:0/1694891134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540ab5f0 con 0x7f966c0cf0d0 2024-03-22T04:21:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f966aa00640 1 -- 172.21.15.86:0/1694891134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96540b0070 con 0x7f966c0cf0d0 2024-03-22T04:21:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f966aa00640 1 -- 172.21.15.86:0/1694891134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540ab9b0 con 0x7f966c0cf0d0 2024-03-22T04:21:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- 172.21.15.86:0/1694891134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c0cf0d0 msgr2=0x7f966c072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 --2- 172.21.15.86:0/1694891134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c0cf0d0 0x7f966c072430 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f966c0cd5d0 tx=0x7f965401c4e0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- 172.21.15.86:0/1694891134 shutdown_connections 2024-03-22T04:21:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 --2- 172.21.15.86:0/1694891134 >> v2:172.21.15.196:3300/0 conn(0x7f966c072970 0x7f966c1625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 --2- 172.21.15.86:0/1694891134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c0cf0d0 0x7f966c072430 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- 172.21.15.86:0/1694891134 >> 172.21.15.86:0/1694891134 conn(0x7f966c06c940 msgr2=0x7f966c06cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- 172.21.15.86:0/1694891134 shutdown_connections 2024-03-22T04:21:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- 172.21.15.86:0/1694891134 wait complete. 2024-03-22T04:21:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 Processor -- start 2024-03-22T04:21:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- start start 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c072970 0x7f966c143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966c0cf0d0 0x7f966c154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966c0d0170 0x7f966c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f966c16b650 con 0x7f966c072970 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f966c16b4d0 con 0x7f966c0cf0d0 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f9670c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f966c16b7d0 con 0x7f966c0d0170 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f966b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c072970 0x7f966c143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.979+0000 7f966b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c072970 0x7f966c143950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34002/0 (socket says 172.21.15.86:34002) 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f966b400640 1 -- 172.21.15.86:0/2270954086 learned_addr learned my addr 172.21.15.86:0/2270954086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9663e00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966c0cf0d0 0x7f966c154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f966be00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966c0d0170 0x7f966c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f966b400640 1 -- 172.21.15.86:0/2270954086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966c0d0170 msgr2=0x7f966c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f966b400640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966c0d0170 0x7f966c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f966b400640 1 -- 172.21.15.86:0/2270954086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966c0cf0d0 msgr2=0x7f966c154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f966b400640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966c0cf0d0 0x7f966c154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f966b400640 1 -- 172.21.15.86:0/2270954086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c15d170 con 0x7f966c072970 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f966b400640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c072970 0x7f966c143950 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c450 tx=0x7f965c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9663400640 1 -- 172.21.15.86:0/2270954086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c012830 con 0x7f966c072970 2024-03-22T04:21:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9663400640 1 -- 172.21.15.86:0/2270954086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965c017070 con 0x7f966c072970 2024-03-22T04:21:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9663400640 1 -- 172.21.15.86:0/2270954086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c012c30 con 0x7f966c072970 2024-03-22T04:21:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966c15d450 con 0x7f966c072970 2024-03-22T04:21:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966c15da10 con 0x7f966c072970 2024-03-22T04:21:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f966c074470 con 0x7f966c072970 2024-03-22T04:21:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9663400640 1 -- 172.21.15.86:0/2270954086 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f965c01c460 con 0x7f966c072970 2024-03-22T04:21:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9663400640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9648067b20 0x7f9648069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.983+0000 7f9663400640 1 -- 172.21.15.86:0/2270954086 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f965c085d00 con 0x7f966c072970 2024-03-22T04:21:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.987+0000 7f9663e00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9648067b20 0x7f9648069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.987+0000 7f9663e00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9648067b20 0x7f9648069fe0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f96540af000 tx=0x7f96540a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:02.987+0000 7f9663400640 1 -- 172.21.15.86:0/2270954086 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f965c055f20 con 0x7f966c072970 2024-03-22T04:21:03.167 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-03-22T04:21:03.167+0000 7fc300600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:21:03.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.199+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f966c0ce490 con 0x7f966c072970 2024-03-22T04:21:03.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9663400640 1 -- 172.21.15.86:0/2270954086 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f965c059fc0 con 0x7f966c072970 2024-03-22T04:21:03.202 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-22T04:21:03.202 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB5BP1l3EwWLxAAYGQTyWO58dHk1XGjGNJfxA== 2024-03-22T04:21:03.202 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-03-22T04:21:03.203 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-03-22T04:21:03.203 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-22T04:21:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-03-22T04:21:03.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9648067b20 msgr2=0x7f9648069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:03.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9648067b20 0x7f9648069fe0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f96540af000 tx=0x7f96540a7000 comp rx=0 tx=0).stop 2024-03-22T04:21:03.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c072970 msgr2=0x7f966c143950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:03.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c072970 0x7f966c143950 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c450 tx=0x7f965c00c910 comp rx=0 tx=0).stop 2024-03-22T04:21:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 shutdown_connections 2024-03-22T04:21:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9648067b20 0x7f9648069fe0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966c0cf0d0 0x7f966c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966c0d0170 0x7f966c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 --2- 172.21.15.86:0/2270954086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f966c072970 0x7f966c143950 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 >> 172.21.15.86:0/2270954086 conn(0x7f966c06c940 msgr2=0x7f966c159000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 shutdown_connections 2024-03-22T04:21:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.203+0000 7f9670c00640 1 -- 172.21.15.86:0/2270954086 wait complete. 2024-03-22T04:21:03.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:03.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.359+0000 7f34c1a00640 1 Processor -- start 2024-03-22T04:21:03.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.359+0000 7f34c1a00640 1 -- start start 2024-03-22T04:21:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34c1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34bc15b1e0 0x7f34bc070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34c1a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f34bc0d01b0 0x7f34bc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34c1a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34bc072780 con 0x7f34bc073540 2024-03-22T04:21:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34c1a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f34bc0d06c0 con 0x7f34bc15b1e0 2024-03-22T04:21:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34c1a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f34bc0730b0 con 0x7f34bc0d01b0 2024-03-22T04:21:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34bc15b1e0 0x7f34bc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:03.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34bc15b1e0 0x7f34bc070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34004/0 (socket says 172.21.15.86:34004) 2024-03-22T04:21:03.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34bbe00640 1 -- 172.21.15.86:0/234537119 learned_addr learned my addr 172.21.15.86:0/234537119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:03.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34bb400640 1 --2- 172.21.15.86:0/234537119 >> v2:172.21.15.196:3300/0 conn(0x7f34bc0d01b0 0x7f34bc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:03.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34bbe00640 1 -- 172.21.15.86:0/234537119 >> v2:172.21.15.196:3300/0 conn(0x7f34bc0d01b0 msgr2=0x7f34bc0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:03.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34bbe00640 1 --2- 172.21.15.86:0/234537119 >> v2:172.21.15.196:3300/0 conn(0x7f34bc0d01b0 0x7f34bc0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34bbe00640 1 -- 172.21.15.86:0/234537119 >> v1:172.21.15.196:6789/0 conn(0x7f34bc073540 legacy=0x7f34bc0b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:03.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.363+0000 7f34bbe00640 1 -- 172.21.15.86:0/234537119 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc072a60 con 0x7f34bc15b1e0 2024-03-22T04:21:03.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34bbe00640 1 --2- 172.21.15.86:0/234537119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc15b1e0 0x7f34bc070470 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0d0840 tx=0x7f34ac01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a27617a76449bdc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:03.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34baa00640 1 -- 172.21.15.86:0/234537119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0b0070 con 0x7f34bc15b1e0 2024-03-22T04:21:03.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34baa00640 1 -- 172.21.15.86:0/234537119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ac0a4e50 con 0x7f34bc15b1e0 2024-03-22T04:21:03.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34baa00640 1 -- 172.21.15.86:0/234537119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0ab820 con 0x7f34bc15b1e0 2024-03-22T04:21:03.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- 172.21.15.86:0/234537119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc15b1e0 msgr2=0x7f34bc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:03.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 --2- 172.21.15.86:0/234537119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc15b1e0 0x7f34bc070470 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0d0840 tx=0x7f34ac01c3c0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- 172.21.15.86:0/234537119 shutdown_connections 2024-03-22T04:21:03.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 --2- 172.21.15.86:0/234537119 >> v2:172.21.15.196:3300/0 conn(0x7f34bc0d01b0 0x7f34bc0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 --2- 172.21.15.86:0/234537119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc15b1e0 0x7f34bc070470 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- 172.21.15.86:0/234537119 >> 172.21.15.86:0/234537119 conn(0x7f34bc0d1000 msgr2=0x7f34bc0d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- 172.21.15.86:0/234537119 shutdown_connections 2024-03-22T04:21:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- 172.21.15.86:0/234537119 wait complete. 2024-03-22T04:21:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 Processor -- start 2024-03-22T04:21:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- start start 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34bc073540 0x7f34bc1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc0d01b0 0x7f34bc15d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc15b1e0 0x7f34bc06e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc165f30 con 0x7f34bc0d01b0 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc165db0 con 0x7f34bc15b1e0 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc1660b0 con 0x7f34bc073540 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc0d01b0 0x7f34bc15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc0d01b0 0x7f34bc15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34012/0 (socket says 172.21.15.86:34012) 2024-03-22T04:21:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34bb400640 1 -- 172.21.15.86:0/1220290295 learned_addr learned my addr 172.21.15.86:0/1220290295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34bbe00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34bc073540 0x7f34bc1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.367+0000 7f34c1000640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc15b1e0 0x7f34bc06e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34bb400640 1 -- 172.21.15.86:0/1220290295 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34bc073540 msgr2=0x7f34bc1521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34bb400640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34bc073540 0x7f34bc1521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34bb400640 1 -- 172.21.15.86:0/1220290295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc15b1e0 msgr2=0x7f34bc06e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34bb400640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc15b1e0 0x7f34bc06e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34bb400640 1 -- 172.21.15.86:0/1220290295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc1528b0 con 0x7f34bc0d01b0 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34bb400640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc0d01b0 0x7f34bc15d600 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f34a800d7d0 tx=0x7f34a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34b3e00640 1 -- 172.21.15.86:0/1220290295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a800b6c0 con 0x7f34bc0d01b0 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34b3e00640 1 -- 172.21.15.86:0/1220290295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a8019070 con 0x7f34bc0d01b0 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc152b90 con 0x7f34bc0d01b0 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34b3e00640 1 -- 172.21.15.86:0/1220290295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a800baa0 con 0x7f34bc0d01b0 2024-03-22T04:21:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc14a850 con 0x7f34bc0d01b0 2024-03-22T04:21:03.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc074470 con 0x7f34bc0d01b0 2024-03-22T04:21:03.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34b3e00640 1 -- 172.21.15.86:0/1220290295 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f34a800bc40 con 0x7f34bc0d01b0 2024-03-22T04:21:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34b3e00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f349c067b20 0x7f349c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34b3e00640 1 -- 172.21.15.86:0/1220290295 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f34a8086450 con 0x7f34bc0d01b0 2024-03-22T04:21:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.371+0000 7f34bbe00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f349c067b20 0x7f349c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.375+0000 7f34bbe00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f349c067b20 0x7f349c069fe0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0ad000 tx=0x7f34ac0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:03.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.383+0000 7f34b3e00640 1 -- 172.21.15.86:0/1220290295 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f34a80565e0 con 0x7f34bc0d01b0 2024-03-22T04:21:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.687+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34bc0cd430 con 0x7f34bc0d01b0 2024-03-22T04:21:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34b3e00640 1 -- 172.21.15.86:0/1220290295 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5194 (secure 0 0 0) 0x7f34a805a680 con 0x7f34bc0d01b0 2024-03-22T04:21:03.690 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:03.691 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:00.620053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77636":{"gid":77636,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3165266972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3165266972},{"type":"v1","addr":"172.21.15.86:6837","nonce":3165266972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77636,"qdb_cluster":[77636]},"id":8}]} 2024-03-22T04:21:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 99 2024-03-22T04:21:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f349c067b20 msgr2=0x7f349c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f349c067b20 0x7f349c069fe0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0ad000 tx=0x7f34ac0a6000 comp rx=0 tx=0).stop 2024-03-22T04:21:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc0d01b0 msgr2=0x7f34bc15d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc0d01b0 0x7f34bc15d600 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f34a800d7d0 tx=0x7f34a800dc90 comp rx=0 tx=0).stop 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 shutdown_connections 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f349c067b20 0x7f349c069fe0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34bc15b1e0 0x7f34bc06e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34bc0d01b0 0x7f34bc15d600 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 --2- 172.21.15.86:0/1220290295 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34bc073540 0x7f34bc1521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 >> 172.21.15.86:0/1220290295 conn(0x7f34bc0d1000 msgr2=0x7f34bc1534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 shutdown_connections 2024-03-22T04:21:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:03.691+0000 7f34c1a00640 1 -- 172.21.15.86:0/1220290295 wait complete. 2024-03-22T04:21:04.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:20:57.561818+0000', 'modified': '2024-03-22T04:21:00.620053+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 77636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_77636': {'gid': 77636, 'name': 'a', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3165266972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 3165266972}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3165266972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 77636, 'qdb_cluster': [77636]} 2024-03-22T04:21:04.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:21:04.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-22T04:21:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a8400640 1 Processor -- start 2024-03-22T04:21:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a8400640 1 -- start start 2024-03-22T04:21:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a8400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd9a006e2f0 0x7fd9a0071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a8400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a01623d0 con 0x7fd9a015e630 2024-03-22T04:21:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a8400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a01620f0 con 0x7fd9a00d0580 2024-03-22T04:21:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a8400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a015ec10 con 0x7fd9a006e2f0 2024-03-22T04:21:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a5a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd9a015e630 0x7fd9a00cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49886/0 (socket says 172.21.15.86:49886) 2024-03-22T04:21:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a5a00640 1 -- 172.21.15.86:0/2583638751 learned_addr learned my addr 172.21.15.86:0/2583638751 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a6400640 1 --2- 172.21.15.86:0/2583638751 >> v2:172.21.15.196:3300/0 conn(0x7fd9a006e2f0 0x7fd9a0071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd99fe00640 1 -- 172.21.15.86:0/2583638751 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1378217735 0 0) 0x7fd9a01623d0 con 0x7fd9a015e630 2024-03-22T04:21:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd99fe00640 1 -- 172.21.15.86:0/2583638751 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd988003530 con 0x7fd9a015e630 2024-03-22T04:21:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a6400640 1 -- 172.21.15.86:0/2583638751 >> v1:172.21.15.196:6789/0 conn(0x7fd9a00d0580 legacy=0x7fd9a0071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd99fe00640 1 -- 172.21.15.86:0/2583638751 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479798056 0 0) 0x7fd9a01620f0 con 0x7fd9a00d0580 2024-03-22T04:21:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a6400640 1 -- 172.21.15.86:0/2583638751 >> v1:172.21.15.86:6789/0 conn(0x7fd9a015e630 legacy=0x7fd9a00cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.255+0000 7fd9a6400640 1 -- 172.21.15.86:0/2583638751 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a015eef0 con 0x7fd9a006e2f0 2024-03-22T04:21:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd9a6400640 1 --2- 172.21.15.86:0/2583638751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a006e2f0 0x7fd9a0071810 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd998009d90 tx=0x7fd99801c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=224e73488fac12f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd99fe00640 1 -- 172.21.15.86:0/2583638751 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994001600 con 0x7fd9a006e2f0 2024-03-22T04:21:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd99fe00640 1 -- 172.21.15.86:0/2583638751 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9980a6440 con 0x7fd9a006e2f0 2024-03-22T04:21:04.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd99fe00640 1 -- 172.21.15.86:0/2583638751 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980a6810 con 0x7fd9a006e2f0 2024-03-22T04:21:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd9a8400640 1 -- 172.21.15.86:0/2583638751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a006e2f0 msgr2=0x7fd9a0071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd9a8400640 1 --2- 172.21.15.86:0/2583638751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a006e2f0 0x7fd9a0071810 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd998009d90 tx=0x7fd99801c7b0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd9a8400640 1 -- 172.21.15.86:0/2583638751 shutdown_connections 2024-03-22T04:21:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd9a8400640 1 --2- 172.21.15.86:0/2583638751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a006e2f0 0x7fd9a0071810 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd9a8400640 1 -- 172.21.15.86:0/2583638751 >> 172.21.15.86:0/2583638751 conn(0x7fd9a006f910 msgr2=0x7fd9a006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:04.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd9a8400640 1 -- 172.21.15.86:0/2583638751 shutdown_connections 2024-03-22T04:21:04.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.259+0000 7fd9a8400640 1 -- 172.21.15.86:0/2583638751 wait complete. 2024-03-22T04:21:04.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 Processor -- start 2024-03-22T04:21:04.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 -- start start 2024-03-22T04:21:04.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a006e2f0 0x7fd9a00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a00d0580 0x7fd9a015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015e630 0x7fd9a01faa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00cfaf0 con 0x7fd9a015e630 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a015f9d0 con 0x7fd9a00d0580 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a00cfc70 con 0x7fd9a006e2f0 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015e630 0x7fd9a01faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015e630 0x7fd9a01faa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34020/0 (socket says 172.21.15.86:34020) 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 -- 172.21.15.86:0/2950036047 learned_addr learned my addr 172.21.15.86:0/2950036047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a006e2f0 0x7fd9a00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a5000640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a00d0580 0x7fd9a015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 -- 172.21.15.86:0/2950036047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a006e2f0 msgr2=0x7fd9a00c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a006e2f0 0x7fd9a00c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 -- 172.21.15.86:0/2950036047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a00d0580 msgr2=0x7fd9a015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a00d0580 0x7fd9a015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 -- 172.21.15.86:0/2950036047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a0156330 con 0x7fd9a015e630 2024-03-22T04:21:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a6400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015e630 0x7fd9a01faa20 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fd99801a3e0 tx=0x7fd998019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd99d600640 1 -- 172.21.15.86:0/2950036047 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980a6a30 con 0x7fd9a015e630 2024-03-22T04:21:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd99d600640 1 -- 172.21.15.86:0/2950036047 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9980a6bd0 con 0x7fd9a015e630 2024-03-22T04:21:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd99d600640 1 -- 172.21.15.86:0/2950036047 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980039a0 con 0x7fd9a015e630 2024-03-22T04:21:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 -- 172.21.15.86:0/2950036047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a0156610 con 0x7fd9a015e630 2024-03-22T04:21:04.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.263+0000 7fd9a8400640 1 -- 172.21.15.86:0/2950036047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a015c5e0 con 0x7fd9a015e630 2024-03-22T04:21:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.267+0000 7fd99d600640 1 -- 172.21.15.86:0/2950036047 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd998003b40 con 0x7fd9a015e630 2024-03-22T04:21:04.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.267+0000 7fd99d600640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd980067b20 0x7fd980069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.267+0000 7fd99d600640 1 -- 172.21.15.86:0/2950036047 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fd9981201b0 con 0x7fd9a015e630 2024-03-22T04:21:04.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.267+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd980067b20 0x7fd980069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.267+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd980067b20 0x7fd980069fe0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd994000f70 tx=0x7fd994001900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.267+0000 7fd9ab95d000 1 Processor -- start 2024-03-22T04:21:04.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.267+0000 7fd9ab95d000 1 -- start start 2024-03-22T04:21:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.267+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c5e70 0x55b8994c6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b89947a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b899277140 con 0x55b8994c5e70 2024-03-22T04:21:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b8992cab10 con 0x55b8994c6790 2024-03-22T04:21:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b899251c50 con 0x55b899479c20 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c5e70 0x55b8994c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c5e70 0x55b8994c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34036/0 (socket says 172.21.15.86:34036) 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 -- 172.21.15.86:0/1051931485 learned_addr learned my addr 172.21.15.86:0/1051931485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5000640 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a6400640 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b89947a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 -- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 msgr2=0x55b89947a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b89947a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 -- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 msgr2=0x55b8994796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 -- 172.21.15.86:0/1051931485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b89930ff20 con 0x55b8994c5e70 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5000640 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c5e70 0x55b8994c6250 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fd99400eb60 tx=0x7fd994011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6b9b03b16f51fe server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd992000640 1 -- 172.21.15.86:0/1051931485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994004e30 con 0x55b8994c5e70 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd992000640 1 -- 172.21.15.86:0/1051931485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd994005260 con 0x55b8994c5e70 2024-03-22T04:21:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd992000640 1 -- 172.21.15.86:0/1051931485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994005610 con 0x55b8994c5e70 2024-03-22T04:21:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- 172.21.15.86:0/1051931485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c5e70 msgr2=0x55b8994c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c5e70 0x55b8994c6250 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fd99400eb60 tx=0x7fd994011000 comp rx=0 tx=0).stop 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- 172.21.15.86:0/1051931485 shutdown_connections 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b89947a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/1051931485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c5e70 0x55b8994c6250 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- 172.21.15.86:0/1051931485 >> 172.21.15.86:0/1051931485 conn(0x55b8994a6c10 msgr2=0x55b8994a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- 172.21.15.86:0/1051931485 shutdown_connections 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- 172.21.15.86:0/1051931485 wait complete. 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 -- start start 2024-03-22T04:21:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.271+0000 7fd9ab95d000 1 Processor -- start 2024-03-22T04:21:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b89947e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b89947ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b89947f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b899284ab0 con 0x55b8994c6790 2024-03-22T04:21:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b8992c6ac0 con 0x55b8994c5e70 2024-03-22T04:21:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b89930aa60 con 0x55b899479c20 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 10 client.0 _ll_register_callbacks cb 0x55b899467380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b89947f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b89947f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34040/0 (socket says 172.21.15.86:34040) 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 -- 172.21.15.86:0/154726746 learned_addr learned my addr 172.21.15.86:0/154726746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b89947e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a5000640 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b89947ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 -- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 msgr2=0x55b89947e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b89947e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 -- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 msgr2=0x55b89947ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b89947ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 -- 172.21.15.86:0/154726746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8992a0570 con 0x55b8994c6790 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a5000640 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b89947ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a6400640 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b89947f290 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fd99801c7f0 tx=0x7fd9980ae440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa012e205ef5c979 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd986a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd986a00640 1 -- 172.21.15.86:0/154726746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980f43e0 con 0x55b8994c6790 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd986a00640 1 -- 172.21.15.86:0/154726746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fd9980ef6e0 con 0x55b8994c6790 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 -- 172.21.15.86:0/154726746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd994005260 con 0x55b8994c6790 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd986a00640 1 -- 172.21.15.86:0/154726746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fd9980f8c10 con 0x55b8994c6790 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 10 client.9156 fetch_fsmap learned FSMap version 99 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 -- 172.21.15.86:0/154726746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55b899296880 con 0x55b8994c6790 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd986a00640 1 -- 172.21.15.86:0/154726746 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd994005260 con 0x55b8994c6790 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd986a00640 1 -- 172.21.15.86:0/154726746 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7fd9981080b0 con 0x55b8994c6790 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 10 client.9156 fetch_fsmap finished waiting for FSMap version 99 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 10 client.9156 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 20 client.9156 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x55b89947fca0 0x55b8994d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 4 client.9156 mds_command: new command op to 77636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:21:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9ab95d000 1 -- 172.21.15.86:0/154726746 --> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b899265bc0 con 0x55b89947fca0 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.275+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x55b89947fca0 0x55b8994d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x55b89947fca0 0x55b8994d7120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3dd1d23e3d74237 server_cookie=75455d72e4e09cc7 in_seq=0 out_seq=0 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd986a00640 10 client.9156 ms_handle_connect on v2:172.21.15.86:6835/3165266972 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd986a00640 1 -- 172.21.15.86:0/154726746 <== mds.0 v2:172.21.15.86:6835/3165266972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fd99402c9c0 con 0x55b89947fca0 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd986a00640 10 client.9156 handle_command_reply: tid=0 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 client.9156 shutdown 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 -- 172.21.15.86:0/154726746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 msgr2=0x55b89947f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b89947f290 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fd99801c7f0 tx=0x7fd9980ae440 comp rx=0 tx=0).stop 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 -- 172.21.15.86:0/154726746 shutdown_connections 2024-03-22T04:21:04.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x55b89947fca0 0x55b8994d7120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b89947ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b89947f290 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/154726746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b89947e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 -- 172.21.15.86:0/154726746 >> 172.21.15.86:0/154726746 conn(0x55b8994a6c10 msgr2=0x55b8994a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd986a00640 0 client.9156 ms_handle_reset on v2:172.21.15.86:6835/3165266972 2024-03-22T04:21:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 -- 172.21.15.86:0/154726746 shutdown_connections 2024-03-22T04:21:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 1 -- 172.21.15.86:0/154726746 wait complete. 2024-03-22T04:21:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.279+0000 7fd9ab95d000 20 client.9156 trim_cache size 0 max 16384 2024-03-22T04:21:04.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.295+0000 7fd9ab95d000 1 Processor -- start 2024-03-22T04:21:04.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.295+0000 7fd9ab95d000 1 -- start start 2024-03-22T04:21:04.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b8994c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b8994c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b8994d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0203080 con 0x55b8994c6790 2024-03-22T04:21:04.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0158ae0 con 0x55b8994c5e70 2024-03-22T04:21:04.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a0159130 con 0x55b899479c20 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b8994d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b8994d7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34052/0 (socket says 172.21.15.86:34052) 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 -- 172.21.15.86:0/4215663245 learned_addr learned my addr 172.21.15.86:0/4215663245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a5000640 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b8994c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b8994c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 -- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 msgr2=0x55b8994c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b8994c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 -- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 msgr2=0x55b8994c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b8994c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 -- 172.21.15.86:0/4215663245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b899252e80 con 0x55b8994c6790 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a5000640 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b8994c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9a6400640 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b8994d7120 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fd9980981d0 tx=0x7fd9980f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8144510d5695b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd991600640 1 -- 172.21.15.86:0/4215663245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980f43e0 con 0x55b8994c6790 2024-03-22T04:21:04.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd991600640 1 -- 172.21.15.86:0/4215663245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9980f40f0 con 0x55b8994c6790 2024-03-22T04:21:04.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd991600640 1 -- 172.21.15.86:0/4215663245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980ef6e0 con 0x55b8994c6790 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- 172.21.15.86:0/4215663245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 msgr2=0x55b8994d7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b8994d7120 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fd9980981d0 tx=0x7fd9980f2000 comp rx=0 tx=0).stop 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- 172.21.15.86:0/4215663245 shutdown_connections 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c5e70 0x55b8994c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8994c6790 0x55b8994d7120 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/4215663245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b899479c20 0x55b8994c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- 172.21.15.86:0/4215663245 >> 172.21.15.86:0/4215663245 conn(0x55b8994a6c10 msgr2=0x55b899478d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- 172.21.15.86:0/4215663245 shutdown_connections 2024-03-22T04:21:04.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- 172.21.15.86:0/4215663245 wait complete. 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- start start 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 Processor -- start 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899479c20 0x55b899534390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b8994c5e70 0x55b8994f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b899386b30 con 0x55b899479c20 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b899270e80 con 0x55b8994c6790 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b8992c99f0 con 0x55b8994c5e70 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.299+0000 7fd9ab95d000 10 client.0 _ll_register_callbacks cb 0x7fd998009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899479c20 0x55b899534390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899479c20 0x55b899534390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34064/0 (socket says 172.21.15.86:34064) 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 -- 172.21.15.86:0/3085333642 learned_addr learned my addr 172.21.15.86:0/3085333642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5000640 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b8994c5e70 0x55b8994f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a6400640 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 -- 172.21.15.86:0/3085333642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b8994c5e70 msgr2=0x55b8994f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b8994c5e70 0x55b8994f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 -- 172.21.15.86:0/3085333642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 msgr2=0x55b8994f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 -- 172.21.15.86:0/3085333642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b89925e750 con 0x55b899479c20 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5a00640 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899479c20 0x55b899534390 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd994001770 tx=0x7fd994001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ed68e3514e2e7d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 1 -- 172.21.15.86:0/3085333642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994032060 con 0x55b899479c20 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 1 -- 172.21.15.86:0/3085333642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fd9940055d0 con 0x55b899479c20 2024-03-22T04:21:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 -- 172.21.15.86:0/3085333642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9980f40f0 con 0x55b899479c20 2024-03-22T04:21:04.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 1 -- 172.21.15.86:0/3085333642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fd994021450 con 0x55b899479c20 2024-03-22T04:21:04.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 10 client.9162 fetch_fsmap learned FSMap version 99 2024-03-22T04:21:04.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 -- 172.21.15.86:0/3085333642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55b899271450 con 0x55b899479c20 2024-03-22T04:21:04.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 1 -- 172.21.15.86:0/3085333642 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9980f40f0 con 0x55b899479c20 2024-03-22T04:21:04.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 1 -- 172.21.15.86:0/3085333642 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7fd9940215f0 con 0x55b899479c20 2024-03-22T04:21:04.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 10 client.9162 fetch_fsmap finished waiting for FSMap version 99 2024-03-22T04:21:04.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 10 client.9162 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 20 client.9162 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x55b8994f14f0 0x55b8994f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 4 client.9162 mds_command: new command op to 77636 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 -- 172.21.15.86:0/3085333642 --> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] -- command(tid 0: {"prefix": "status"}) -- 0x55b8992d8050 con 0x55b8994f14f0 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5000640 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x55b8994f14f0 0x55b8994f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 10 client.9162 ms_handle_connect on v2:172.21.15.86:6835/3165266972 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9a5000640 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x55b8994f14f0 0x55b8994f18d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeaaf131f6d90ecf server_cookie=ba31f834647768db in_seq=0 out_seq=0 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 1 -- 172.21.15.86:0/3085333642 <== mds.0 v2:172.21.15.86:6835/3165266972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55b8992d8050 con 0x55b8994f14f0 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 10 client.9162 handle_command_reply: tid=0 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 client.9162 shutdown 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 -- 172.21.15.86:0/3085333642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899479c20 msgr2=0x55b899534390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899479c20 0x55b899534390 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd994001770 tx=0x7fd994001700 comp rx=0 tx=0).stop 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 -- 172.21.15.86:0/3085333642 shutdown_connections 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x55b8994f14f0 0x55b8994f18d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8994c6790 0x55b8994f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b8994c5e70 0x55b8994f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd986a00640 0 client.9162 ms_handle_reset on v2:172.21.15.86:6835/3165266972 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 --2- 172.21.15.86:0/3085333642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899479c20 0x55b899534390 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 -- 172.21.15.86:0/3085333642 >> 172.21.15.86:0/3085333642 conn(0x55b8994a6c10 msgr2=0x55b8994c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 -- 172.21.15.86:0/3085333642 shutdown_connections 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 1 -- 172.21.15.86:0/3085333642 wait complete. 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.303+0000 7fd9ab95d000 20 client.9162 trim_cache size 0 max 16384 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 77636, 2024-03-22T04:21:04.309 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:21:04.310 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:21:04.310 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:21:04.310 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.0958837629999998, 2024-03-22T04:21:04.310 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 99, 2024-03-22T04:21:04.310 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 124, 2024-03-22T04:21:04.310 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 124, 2024-03-22T04:21:04.310 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 12.066043000000001 2024-03-22T04:21:04.310 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 -- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd980067b20 msgr2=0x7fd980069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd980067b20 0x7fd980069fe0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd994000f70 tx=0x7fd994001900 comp rx=0 tx=0).stop 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 -- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015e630 msgr2=0x7fd9a01faa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015e630 0x7fd9a01faa20 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fd99801a3e0 tx=0x7fd998019550 comp rx=0 tx=0).stop 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 -- 172.21.15.86:0/2950036047 shutdown_connections 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd980067b20 0x7fd980069fe0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a00d0580 0x7fd9a015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015e630 0x7fd9a01faa20 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 --2- 172.21.15.86:0/2950036047 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a006e2f0 0x7fd9a00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 -- 172.21.15.86:0/2950036047 >> 172.21.15.86:0/2950036047 conn(0x7fd9a006f910 msgr2=0x7fd9a0071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:04.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 -- 172.21.15.86:0/2950036047 shutdown_connections 2024-03-22T04:21:04.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.307+0000 7fd9a8400640 1 -- 172.21.15.86:0/2950036047 wait complete. 2024-03-22T04:21:04.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:21:04.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f8946e00640 1 Processor -- start 2024-03-22T04:21:04.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f8946e00640 1 -- start start 2024-03-22T04:21:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f8946e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d0de0 0x7f89400ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f8946e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f89400d1dc0 0x7f894015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f8946e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89400d11c0 con 0x7f89400d19a0 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f8946e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f89400d0580 con 0x7f89400d1dc0 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f8946e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f89400cece0 con 0x7f89400d0de0 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f893f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f89400d19a0 0x7f89400cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49894/0 (socket says 172.21.15.86:49894) 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f893f400640 1 -- 172.21.15.86:0/1484552726 learned_addr learned my addr 172.21.15.86:0/1484552726 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f8944e00640 1 --2- 172.21.15.86:0/1484552726 >> v2:172.21.15.196:3300/0 conn(0x7f89400d1dc0 0x7f894015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.459+0000 7f893fe00640 1 --2- 172.21.15.86:0/1484552726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d0de0 0x7f89400ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.463+0000 7f893ea00640 1 -- 172.21.15.86:0/1484552726 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1080189869 0 0) 0x7f89400d11c0 con 0x7f89400d19a0 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.463+0000 7f893ea00640 1 -- 172.21.15.86:0/1484552726 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8924003530 con 0x7f89400d19a0 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.463+0000 7f8944e00640 1 -- 172.21.15.86:0/1484552726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d0de0 msgr2=0x7f89400ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.463+0000 7f8944e00640 1 --2- 172.21.15.86:0/1484552726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d0de0 0x7f89400ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.463+0000 7f8944e00640 1 -- 172.21.15.86:0/1484552726 >> v1:172.21.15.86:6789/0 conn(0x7f89400d19a0 legacy=0x7f89400cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.463+0000 7f8944e00640 1 -- 172.21.15.86:0/1484552726 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89400cefc0 con 0x7f89400d1dc0 2024-03-22T04:21:04.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8944e00640 1 --2- 172.21.15.86:0/1484552726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d1dc0 0x7f894015fed0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8938009d90 tx=0x7f893801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed7596e9503ae1b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f893ea00640 1 -- 172.21.15.86:0/1484552726 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380b2070 con 0x7f89400d1dc0 2024-03-22T04:21:04.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f893ea00640 1 -- 172.21.15.86:0/1484552726 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89380a5830 con 0x7f89400d1dc0 2024-03-22T04:21:04.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f893ea00640 1 -- 172.21.15.86:0/1484552726 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380a5c00 con 0x7f89400d1dc0 2024-03-22T04:21:04.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8946e00640 1 -- 172.21.15.86:0/1484552726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d1dc0 msgr2=0x7f894015fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8946e00640 1 --2- 172.21.15.86:0/1484552726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d1dc0 0x7f894015fed0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8938009d90 tx=0x7f893801c970 comp rx=0 tx=0).stop 2024-03-22T04:21:04.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8946e00640 1 -- 172.21.15.86:0/1484552726 shutdown_connections 2024-03-22T04:21:04.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8946e00640 1 --2- 172.21.15.86:0/1484552726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d1dc0 0x7f894015fed0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8946e00640 1 --2- 172.21.15.86:0/1484552726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d0de0 0x7f89400ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8946e00640 1 -- 172.21.15.86:0/1484552726 >> 172.21.15.86:0/1484552726 conn(0x7f894006f910 msgr2=0x7f894006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:04.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8946e00640 1 -- 172.21.15.86:0/1484552726 shutdown_connections 2024-03-22T04:21:04.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.467+0000 7f8946e00640 1 -- 172.21.15.86:0/1484552726 wait complete. 2024-03-22T04:21:04.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 Processor -- start 2024-03-22T04:21:04.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 -- start start 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d0de0 0x7f89400b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d19a0 0x7f8940154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89400d1dc0 0x7f89400b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8940071fb0 con 0x7f89400d1dc0 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8940071e30 con 0x7f89400d0de0 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8940169040 con 0x7f89400d19a0 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89400d1dc0 0x7f89400b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89400d1dc0 0x7f89400b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34074/0 (socket says 172.21.15.86:34074) 2024-03-22T04:21:04.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 -- 172.21.15.86:0/2863742969 learned_addr learned my addr 172.21.15.86:0/2863742969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:04.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f893fe00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d0de0 0x7f89400b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f893f400640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d19a0 0x7f8940154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 -- 172.21.15.86:0/2863742969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d19a0 msgr2=0x7f8940154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d19a0 0x7f8940154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 -- 172.21.15.86:0/2863742969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d0de0 msgr2=0x7f89400b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d0de0 0x7f89400b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 -- 172.21.15.86:0/2863742969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89400b2240 con 0x7f89400d1dc0 2024-03-22T04:21:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8944e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89400d1dc0 0x7f89400b32c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f89380b0040 tx=0x7f8938019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8937e00640 1 -- 172.21.15.86:0/2863742969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380b2050 con 0x7f89400d1dc0 2024-03-22T04:21:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8937e00640 1 -- 172.21.15.86:0/2863742969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89380be070 con 0x7f89400d1dc0 2024-03-22T04:21:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8937e00640 1 -- 172.21.15.86:0/2863742969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380ad890 con 0x7f89400d1dc0 2024-03-22T04:21:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89400b2520 con 0x7f89400d1dc0 2024-03-22T04:21:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89400b2ae0 con 0x7f89400d1dc0 2024-03-22T04:21:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.471+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8940068650 con 0x7f89400d1dc0 2024-03-22T04:21:04.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.475+0000 7f8937e00640 1 -- 172.21.15.86:0/2863742969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f89380ada30 con 0x7f89400d1dc0 2024-03-22T04:21:04.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.475+0000 7f8937e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8920067b20 0x7f8920069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:04.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.475+0000 7f8937e00640 1 -- 172.21.15.86:0/2863742969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f8938121180 con 0x7f89400d1dc0 2024-03-22T04:21:04.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.475+0000 7f893fe00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8920067b20 0x7f8920069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:04.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.475+0000 7f893fe00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8920067b20 0x7f8920069fe0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8930009a50 tx=0x7f8930008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:04.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.479+0000 7f8937e00640 1 -- 172.21.15.86:0/2863742969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8938016560 con 0x7f89400d1dc0 2024-03-22T04:21:04.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.743+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8940153d40 con 0x7f89400d1dc0 2024-03-22T04:21:04.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.743+0000 7f8937e00640 1 -- 172.21.15.86:0/2863742969 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f89380f1b40 con 0x7f89400d1dc0 2024-03-22T04:21:04.746 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:21:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8920067b20 msgr2=0x7f8920069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8920067b20 0x7f8920069fe0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8930009a50 tx=0x7f8930008040 comp rx=0 tx=0).stop 2024-03-22T04:21:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89400d1dc0 msgr2=0x7f89400b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89400d1dc0 0x7f89400b32c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f89380b0040 tx=0x7f8938019710 comp rx=0 tx=0).stop 2024-03-22T04:21:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 shutdown_connections 2024-03-22T04:21:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8920067b20 0x7f8920069fe0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89400d1dc0 0x7f89400b32c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89400d19a0 0x7f8940154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 --2- 172.21.15.86:0/2863742969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89400d0de0 0x7f89400b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:04.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 >> 172.21.15.86:0/2863742969 conn(0x7f894006f910 msgr2=0x7f894015c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:04.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 shutdown_connections 2024-03-22T04:21:04.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04.747+0000 7f8946e00640 1 -- 172.21.15.86:0/2863742969 wait complete. 2024-03-22T04:21:04.775 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:21:04.775 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:21:04.776 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:21:04.776 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:21:04.776 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:04.776 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:21:04.776 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:21:04.776 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:21:04.776 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:04.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:04.777 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41867sec preferred_lft 41867sec 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:04.790 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:21:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:04.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:04.792 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:21:04.847 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:21:04.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:04.850 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:21:04.901 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:21:04.901 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:04.902 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:21:04.902 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:21:04.903 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:21:04.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:21:04.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:21:04.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:21:04.903 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:21:04.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:21:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:05.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:05.037 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:21:05.046 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:21:05.046 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:21:05.047 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:05.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:05.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:21:05.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:21:05.109 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:05.157 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:05.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:21:05.158 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:05.205 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:21:05.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:05.205 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:05.294 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:05.294 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:05.294 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:05.309 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:05.309 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:05.309 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:05.309 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:21:05.309 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:21:05.317 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:05.318 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:05.318 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:05.318 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:21:05.318 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:21:05.318 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:21:05.318 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:21:05.318 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:21:05.322 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:21:05.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:05.322 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:05.341 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:21:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:21:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:05.454 INFO:teuthology.orchestra.run.smithi086.stdout:client9171 2024-03-22T04:21:05.455 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:05.455 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> parent = os.path.join(mount_point, "parent") 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> os.mkdir(parent) 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> flushed = os.path.join(parent, "flushed") 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> os.mkdir(flushed) 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> for f in ["alpha", "bravo", "charlie"]: 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-03-22T04:21:05.464 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:05.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:05.589 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-22T04:21:05.643 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-03-22T04:21:05.644 INFO:teuthology.orchestra.run.smithi086.stdout:./parent 2024-03-22T04:21:05.644 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed 2024-03-22T04:21:05.644 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/charlie 2024-03-22T04:21:05.644 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/alpha 2024-03-22T04:21:05.644 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/bravo 2024-03-22T04:21:05.645 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:05.645 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:05.645 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:05.645 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:05.645 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-22T04:21:05.645 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:05.761 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:21:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:05.772 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:05.772 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:05.772 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:05.773 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:05.773 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-22T04:21:05.773 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:05.887 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-03-22T04:21:05.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:05.896 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:05.896 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:05.896 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:05.896 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:05.896 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-22T04:21:05.897 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:06.001 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-03-22T04:21:06.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:06.011 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:06.011 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:06.011 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:06.011 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:06.011 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-22T04:21:06.011 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:06.117 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628280 2024-03-22T04:21:06.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:06.126 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:06.126 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:06.126 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:06.126 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:06.126 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-22T04:21:06.126 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:06.238 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-03-22T04:21:06.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:06.247 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:06.247 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:06.247 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:06.247 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:06.248 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-22T04:21:06.248 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:06.354 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628279 2024-03-22T04:21:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:06.363 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:06.363 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:06.416 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:06.416 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:06.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:06.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.643+0000 7fa2eb800640 1 Processor -- start 2024-03-22T04:21:06.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.643+0000 7fa2eb800640 1 -- start start 2024-03-22T04:21:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2eb800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2e40ce6b0 0x7fa2e40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2eb800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e40d11c0 con 0x7fa2e415fed0 2024-03-22T04:21:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2eb800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e40cece0 con 0x7fa2e40d0de0 2024-03-22T04:21:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2eb800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e40cefc0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2e9800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa2e415fed0 0x7fa2e41602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54520/0 (socket says 172.21.15.86:54520) 2024-03-22T04:21:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2e9800640 1 -- 172.21.15.86:0/2500453671 learned_addr learned my addr 172.21.15.86:0/2500453671 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2e3e00640 1 --2- 172.21.15.86:0/2500453671 >> v2:172.21.15.196:3300/0 conn(0x7fa2e40ce6b0 0x7fa2e40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2e3400640 1 -- 172.21.15.86:0/2500453671 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997695382 0 0) 0x7fa2e40d11c0 con 0x7fa2e415fed0 2024-03-22T04:21:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2e3400640 1 -- 172.21.15.86:0/2500453671 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2cc003530 con 0x7fa2e415fed0 2024-03-22T04:21:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2e3e00640 1 -- 172.21.15.86:0/2500453671 >> v1:172.21.15.196:6789/0 conn(0x7fa2e40d0de0 legacy=0x7fa2e40d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2e3e00640 1 -- 172.21.15.86:0/2500453671 >> v1:172.21.15.86:6789/0 conn(0x7fa2e415fed0 legacy=0x7fa2e41602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.647+0000 7fa2e3e00640 1 -- 172.21.15.86:0/2500453671 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e40cf2a0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2e3e00640 1 --2- 172.21.15.86:0/2500453671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e40cfe70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00097c0 tx=0x7fa2d001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac05ee59c2b17003 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2e3400640 1 -- 172.21.15.86:0/2500453671 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00b0070 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2e3400640 1 -- 172.21.15.86:0/2500453671 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d00ab600 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2e3400640 1 -- 172.21.15.86:0/2500453671 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00ab9b0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 -- 172.21.15.86:0/2500453671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40ce6b0 msgr2=0x7fa2e40cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 --2- 172.21.15.86:0/2500453671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e40cfe70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00097c0 tx=0x7fa2d001c4a0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 -- 172.21.15.86:0/2500453671 shutdown_connections 2024-03-22T04:21:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 --2- 172.21.15.86:0/2500453671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e40cfe70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 -- 172.21.15.86:0/2500453671 >> 172.21.15.86:0/2500453671 conn(0x7fa2e406f910 msgr2=0x7fa2e406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 -- 172.21.15.86:0/2500453671 shutdown_connections 2024-03-22T04:21:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 -- 172.21.15.86:0/2500453671 wait complete. 2024-03-22T04:21:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 Processor -- start 2024-03-22T04:21:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 -- start start 2024-03-22T04:21:06.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e4141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40d0de0 0x7fa2e41653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.651+0000 7fa2eb800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2e415fed0 0x7fa2e415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2eb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4071fb0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2eb800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4071e30 con 0x7fa2e40d0de0 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2eb800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e416a050 con 0x7fa2e415fed0 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e4141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e4141a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37112/0 (socket says 172.21.15.86:37112) 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 -- 172.21.15.86:0/806656672 learned_addr learned my addr 172.21.15.86:0/806656672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e3e00640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40d0de0 0x7fa2e41653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e9800640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2e415fed0 0x7fa2e415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 -- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2e415fed0 msgr2=0x7fa2e415ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2e415fed0 0x7fa2e415ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 -- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40d0de0 msgr2=0x7fa2e41653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40d0de0 0x7fa2e41653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 -- 172.21.15.86:0/806656672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e41423d0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e9800640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2e415fed0 0x7fa2e415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e8e00640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e4141a60 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c530 tx=0x7fa2d800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e0c00640 1 -- 172.21.15.86:0/806656672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8016020 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e0c00640 1 -- 172.21.15.86:0/806656672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d8005220 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e0c00640 1 -- 172.21.15.86:0/806656672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8007030 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e41426b0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e4154930 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e4068650 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.655+0000 7fa2e0c00640 1 -- 172.21.15.86:0/806656672 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa2d80071d0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.659+0000 7fa2e0c00640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2c4067b20 0x7fa2c4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.659+0000 7fa2e0c00640 1 -- 172.21.15.86:0/806656672 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fa2d8085a10 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.659+0000 7fa2e3e00640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2c4067b20 0x7fa2c4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.659+0000 7fa2e3e00640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2c4067b20 0x7fa2c4069fe0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00ad000 tx=0x7fa2d00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.663+0000 7fa2e0c00640 1 -- 172.21.15.86:0/806656672 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa2d8020020 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.819+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2e415b4f0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.819+0000 7fa2e0c00640 1 -- 172.21.15.86:0/806656672 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+16956 (secure 0 0 0) 0x7fa2d80074c0 con 0x7fa2e40ce6b0 2024-03-22T04:21:06.821 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:06.821 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:59.189511+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2c4067b20 msgr2=0x7fa2c4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2c4067b20 0x7fa2c4069fe0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00ad000 tx=0x7fa2d00a6000 comp rx=0 tx=0).stop 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e40ce6b0 msgr2=0x7fa2e4141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e4141a60 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c530 tx=0x7fa2d800c9f0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 shutdown_connections 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa2c4067b20 0x7fa2c4069fe0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e40d0de0 0x7fa2e41653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2e415fed0 0x7fa2e415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 --2- 172.21.15.86:0/806656672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e40ce6b0 0x7fa2e4141a60 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 >> 172.21.15.86:0/806656672 conn(0x7fa2e406f910 msgr2=0x7fa2e41677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 shutdown_connections 2024-03-22T04:21:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.823+0000 7fa2eb800640 1 -- 172.21.15.86:0/806656672 wait complete. 2024-03-22T04:21:06.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:20:55.270181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:20:56.532178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:21:06.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:21:06.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f500d400640 1 Processor -- start 2024-03-22T04:21:06.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f500d400640 1 -- start start 2024-03-22T04:21:06.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f500d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f500815c730 0x7f500809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f500d400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b3610 0x7f500809afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f500d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50080cda00 con 0x7f50080b11c0 2024-03-22T04:21:06.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f500d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f500809b910 con 0x7f500815c730 2024-03-22T04:21:06.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f500d400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f50080b1710 con 0x7f50080b3610 2024-03-22T04:21:06.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5006800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f500815c730 0x7f500809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5006800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f500815c730 0x7f500809aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37114/0 (socket says 172.21.15.86:37114) 2024-03-22T04:21:06.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5007200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f50080b11c0 0x7f50080d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54522/0 (socket says 172.21.15.86:54522) 2024-03-22T04:21:06.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5006800640 1 -- 172.21.15.86:0/1363551571 learned_addr learned my addr 172.21.15.86:0/1363551571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:06.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f4fffe00640 1 --2- 172.21.15.86:0/1363551571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b3610 0x7f500809afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5006800640 1 -- 172.21.15.86:0/1363551571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b3610 msgr2=0x7f500809afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:06.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5006800640 1 --2- 172.21.15.86:0/1363551571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b3610 0x7f500809afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5006800640 1 -- 172.21.15.86:0/1363551571 >> v1:172.21.15.86:6789/0 conn(0x7f50080b11c0 legacy=0x7f50080d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5006800640 1 -- 172.21.15.86:0/1363551571 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50080cdce0 con 0x7f500815c730 2024-03-22T04:21:06.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f5005e00640 1 -- 172.21.15.86:0/1363551571 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2035813428 0 0) 0x7f50080cda00 con 0x7f50080b11c0 2024-03-22T04:21:06.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.983+0000 7f4fffe00640 1 --2- 172.21.15.86:0/1363551571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b3610 0x7f500809afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f5006800640 1 --2- 172.21.15.86:0/1363551571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500809aa70 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009870 tx=0x7f4ff801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8ae478bf115973 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f5005e00640 1 -- 172.21.15.86:0/1363551571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80b1070 con 0x7f500815c730 2024-03-22T04:21:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f5005e00640 1 -- 172.21.15.86:0/1363551571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff80ac410 con 0x7f500815c730 2024-03-22T04:21:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f5005e00640 1 -- 172.21.15.86:0/1363551571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80ac7e0 con 0x7f500815c730 2024-03-22T04:21:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f500d400640 1 -- 172.21.15.86:0/1363551571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 msgr2=0x7f500809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f500d400640 1 --2- 172.21.15.86:0/1363551571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500809aa70 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009870 tx=0x7f4ff801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f500d400640 1 -- 172.21.15.86:0/1363551571 shutdown_connections 2024-03-22T04:21:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f500d400640 1 --2- 172.21.15.86:0/1363551571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b3610 0x7f500809afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f500d400640 1 --2- 172.21.15.86:0/1363551571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500809aa70 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f500d400640 1 -- 172.21.15.86:0/1363551571 >> 172.21.15.86:0/1363551571 conn(0x7f50080b3bf0 msgr2=0x7f50080b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f500d400640 1 -- 172.21.15.86:0/1363551571 shutdown_connections 2024-03-22T04:21:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.987+0000 7f500d400640 1 -- 172.21.15.86:0/1363551571 wait complete. 2024-03-22T04:21:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 Processor -- start 2024-03-22T04:21:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 -- start start 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b11c0 0x7f50080cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50080b3610 0x7f50081594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5008169510 con 0x7f500815c730 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5008169390 con 0x7f50080b3610 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5008169690 con 0x7f50080b11c0 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37120/0 (socket says 172.21.15.86:37120) 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 -- 172.21.15.86:0/4028482765 learned_addr learned my addr 172.21.15.86:0/4028482765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5006800640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b11c0 0x7f50080cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f4fffe00640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50080b3610 0x7f50081594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 -- 172.21.15.86:0/4028482765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b11c0 msgr2=0x7f50080cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b11c0 0x7f50080cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 -- 172.21.15.86:0/4028482765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50080b3610 msgr2=0x7f50081594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50080b3610 0x7f50081594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 -- 172.21.15.86:0/4028482765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5008159d00 con 0x7f500815c730 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f5007200640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500806d3d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f500000c730 tx=0x7f500000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f4ffea00640 1 -- 172.21.15.86:0/4028482765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000018070 con 0x7f500815c730 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f4ffea00640 1 -- 172.21.15.86:0/4028482765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500000ce40 con 0x7f500815c730 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f4ffea00640 1 -- 172.21.15.86:0/4028482765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500000a860 con 0x7f500815c730 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5008159fe0 con 0x7f500815c730 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500815a5a0 con 0x7f500815c730 2024-03-22T04:21:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.991+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5008074470 con 0x7f500815c730 2024-03-22T04:21:06.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.995+0000 7f4ffea00640 1 -- 172.21.15.86:0/4028482765 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f500001b070 con 0x7f500815c730 2024-03-22T04:21:06.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.995+0000 7f4ffea00640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fe4067b20 0x7f4fe4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:06.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.995+0000 7f4ffea00640 1 -- 172.21.15.86:0/4028482765 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f5000085570 con 0x7f500815c730 2024-03-22T04:21:06.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.995+0000 7f5006800640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fe4067b20 0x7f4fe4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:06.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.995+0000 7f5006800640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fe4067b20 0x7f4fe4069fe0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80ae000 tx=0x7f4ff8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:06.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:06.999+0000 7f4ffea00640 1 -- 172.21.15.86:0/4028482765 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5000055790 con 0x7f500815c730 2024-03-22T04:21:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.151+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5008166c40 con 0x7f500815c730 2024-03-22T04:21:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f4ffea00640 1 -- 172.21.15.86:0/4028482765 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) ==== 93+0+10 (secure 0 0 0) 0x7f5000059830 con 0x7f500815c730 2024-03-22T04:21:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:21:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fe4067b20 msgr2=0x7f4fe4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fe4067b20 0x7f4fe4069fe0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80ae000 tx=0x7f4ff8008040 comp rx=0 tx=0).stop 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 msgr2=0x7f500806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500806d3d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f500000c730 tx=0x7f500000cbf0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 shutdown_connections 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4fe4067b20 0x7f4fe4069fe0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50080b3610 0x7f50081594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500815c730 0x7f500806d3d0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 --2- 172.21.15.86:0/4028482765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f50080b11c0 0x7f50080cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 >> 172.21.15.86:0/4028482765 conn(0x7f50080b3bf0 msgr2=0x7f5008070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 shutdown_connections 2024-03-22T04:21:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.155+0000 7f500d400640 1 -- 172.21.15.86:0/4028482765 wait complete. 2024-03-22T04:21:07.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:21:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.315+0000 7f41c2000640 1 Processor -- start 2024-03-22T04:21:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.315+0000 7f41c2000640 1 -- start start 2024-03-22T04:21:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41c2000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b3610 0x7f41bc09b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41c2000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc0cda00 con 0x7f41bc15c730 2024-03-22T04:21:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41c2000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc0cdce0 con 0x7f41bc0b11c0 2024-03-22T04:21:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41c2000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc09b910 con 0x7f41bc0b3610 2024-03-22T04:21:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41bb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f41bc15c730 0x7f41bc09aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54534/0 (socket says 172.21.15.86:54534) 2024-03-22T04:21:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41bb400640 1 -- 172.21.15.86:0/1600948578 learned_addr learned my addr 172.21.15.86:0/1600948578 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41b3e00640 1 --2- 172.21.15.86:0/1600948578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b3610 0x7f41bc09b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41baa00640 1 -- 172.21.15.86:0/1600948578 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1454091476 0 0) 0x7f41bc0cda00 con 0x7f41bc15c730 2024-03-22T04:21:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41baa00640 1 -- 172.21.15.86:0/1600948578 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41a0003530 con 0x7f41bc15c730 2024-03-22T04:21:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41b3e00640 1 -- 172.21.15.86:0/1600948578 >> v1:172.21.15.196:6789/0 conn(0x7f41bc0b11c0 legacy=0x7f41bc0b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41baa00640 1 -- 172.21.15.86:0/1600948578 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2527113663 0 0) 0x7f41a0003530 con 0x7f41bc15c730 2024-03-22T04:21:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41b3e00640 1 -- 172.21.15.86:0/1600948578 >> v1:172.21.15.86:6789/0 conn(0x7f41bc15c730 legacy=0x7f41bc09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.319+0000 7f41b3e00640 1 -- 172.21.15.86:0/1600948578 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc0cdfc0 con 0x7f41bc0b3610 2024-03-22T04:21:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41b3e00640 1 --2- 172.21.15.86:0/1600948578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b3610 0x7f41bc09b180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009770 tx=0x7f41a401c1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f706ffb7fec639 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41baa00640 1 -- 172.21.15.86:0/1600948578 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40b1070 con 0x7f41bc0b3610 2024-03-22T04:21:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41baa00640 1 -- 172.21.15.86:0/1600948578 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a40a4cd0 con 0x7f41bc0b3610 2024-03-22T04:21:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41baa00640 1 -- 172.21.15.86:0/1600948578 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac001680 con 0x7f41bc0b3610 2024-03-22T04:21:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- 172.21.15.86:0/1600948578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b3610 msgr2=0x7f41bc09b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 --2- 172.21.15.86:0/1600948578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b3610 0x7f41bc09b180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009770 tx=0x7f41a401c1a0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- 172.21.15.86:0/1600948578 shutdown_connections 2024-03-22T04:21:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 --2- 172.21.15.86:0/1600948578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b3610 0x7f41bc09b180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- 172.21.15.86:0/1600948578 >> 172.21.15.86:0/1600948578 conn(0x7f41bc0b3bf0 msgr2=0x7f41bc0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- 172.21.15.86:0/1600948578 shutdown_connections 2024-03-22T04:21:07.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- 172.21.15.86:0/1600948578 wait complete. 2024-03-22T04:21:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 Processor -- start 2024-03-22T04:21:07.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- start start 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b11c0 0x7f41bc0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc0b3610 0x7f41bc0c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc15c730 0x7f41bc070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc0b2e50 con 0x7f41bc15c730 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc0b2cd0 con 0x7f41bc0b3610 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.323+0000 7f41c2000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc09b6c0 con 0x7f41bc0b11c0 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc15c730 0x7f41bc070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc15c730 0x7f41bc070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37124/0 (socket says 172.21.15.86:37124) 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 -- 172.21.15.86:0/1639153767 learned_addr learned my addr 172.21.15.86:0/1639153767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41b3e00640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc0b3610 0x7f41bc0c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bb400640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b11c0 0x7f41bc0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 -- 172.21.15.86:0/1639153767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b11c0 msgr2=0x7f41bc0cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b11c0 0x7f41bc0cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 -- 172.21.15.86:0/1639153767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc0b3610 msgr2=0x7f41bc0c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc0b3610 0x7f41bc0c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 -- 172.21.15.86:0/1639153767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc0c68a0 con 0x7f41bc15c730 2024-03-22T04:21:07.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41bbe00640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc15c730 0x7f41bc070140 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c8c0 tx=0x7f41b400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41b3400640 1 -- 172.21.15.86:0/1639153767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4017070 con 0x7f41bc15c730 2024-03-22T04:21:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41b3400640 1 -- 172.21.15.86:0/1639153767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b4003db0 con 0x7f41bc15c730 2024-03-22T04:21:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41b3400640 1 -- 172.21.15.86:0/1639153767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b40052b0 con 0x7f41bc15c730 2024-03-22T04:21:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc0c6b80 con 0x7f41bc15c730 2024-03-22T04:21:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc0c7140 con 0x7f41bc15c730 2024-03-22T04:21:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc074470 con 0x7f41bc15c730 2024-03-22T04:21:07.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.327+0000 7f41b3400640 1 -- 172.21.15.86:0/1639153767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f41b401a070 con 0x7f41bc15c730 2024-03-22T04:21:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.331+0000 7f41b3400640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4198067b20 0x7f4198069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.331+0000 7f41b3400640 1 -- 172.21.15.86:0/1639153767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f41b40857d0 con 0x7f41bc15c730 2024-03-22T04:21:07.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.331+0000 7f41bb400640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4198067b20 0x7f4198069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.331+0000 7f41bb400640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4198067b20 0x7f4198069fe0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0cfca0 tx=0x7f41ac001e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:07.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.335+0000 7f41b3400640 1 -- 172.21.15.86:0/1639153767 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f41b40559f0 con 0x7f41bc15c730 2024-03-22T04:21:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.487+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41bc0b3fd0 con 0x7f41bc15c730 2024-03-22T04:21:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.487+0000 7f41b3400640 1 -- 172.21.15.86:0/1639153767 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) ==== 104+0+11 (secure 0 0 0) 0x7f41b4059a90 con 0x7f41bc15c730 2024-03-22T04:21:07.490 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:21:07.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4198067b20 msgr2=0x7f4198069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4198067b20 0x7f4198069fe0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0cfca0 tx=0x7f41ac001e10 comp rx=0 tx=0).stop 2024-03-22T04:21:07.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc15c730 msgr2=0x7f41bc070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc15c730 0x7f41bc070140 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c8c0 tx=0x7f41b400cd80 comp rx=0 tx=0).stop 2024-03-22T04:21:07.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 shutdown_connections 2024-03-22T04:21:07.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4198067b20 0x7f4198069fe0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41bc0b3610 0x7f41bc0c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc15c730 0x7f41bc070140 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 --2- 172.21.15.86:0/1639153767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41bc0b11c0 0x7f41bc0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 >> 172.21.15.86:0/1639153767 conn(0x7f41bc0b3bf0 msgr2=0x7f41bc06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 shutdown_connections 2024-03-22T04:21:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.491+0000 7f41c2000640 1 -- 172.21.15.86:0/1639153767 wait complete. 2024-03-22T04:21:07.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:21:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.647+0000 7fe2a7400640 1 Processor -- start 2024-03-22T04:21:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.647+0000 7fe2a7400640 1 -- start start 2024-03-22T04:21:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe2a7400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe2a015e970 0x7fe2a00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe2a7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a006d8f0 0x7fe2a009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe2a7400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a006dd70 con 0x7fe2a009afb0 2024-03-22T04:21:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe2a7400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a00b49a0 con 0x7fe2a006d8f0 2024-03-22T04:21:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe2a7400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a00ccda0 con 0x7fe2a015e970 2024-03-22T04:21:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe29fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a006d8f0 0x7fe2a009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe29fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a006d8f0 0x7fe2a009aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37126/0 (socket says 172.21.15.86:37126) 2024-03-22T04:21:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe2a5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2a009afb0 0x7fe2a00d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54550/0 (socket says 172.21.15.86:54550) 2024-03-22T04:21:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe29fe00640 1 -- 172.21.15.86:0/3130934312 learned_addr learned my addr 172.21.15.86:0/3130934312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/3130934312 >> v2:172.21.15.196:3300/0 conn(0x7fe2a015e970 0x7fe2a00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe29fe00640 1 -- 172.21.15.86:0/3130934312 >> v2:172.21.15.196:3300/0 conn(0x7fe2a015e970 msgr2=0x7fe2a00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe29fe00640 1 --2- 172.21.15.86:0/3130934312 >> v2:172.21.15.196:3300/0 conn(0x7fe2a015e970 0x7fe2a00b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe29fe00640 1 -- 172.21.15.86:0/3130934312 >> v1:172.21.15.86:6789/0 conn(0x7fe2a009afb0 legacy=0x7fe2a00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe29fe00640 1 -- 172.21.15.86:0/3130934312 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a006dff0 con 0x7fe2a006d8f0 2024-03-22T04:21:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.651+0000 7fe29f400640 1 -- 172.21.15.86:0/3130934312 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4158024790 0 0) 0x7fe2980014a0 con 0x7fe2a009afb0 2024-03-22T04:21:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe29fe00640 1 --2- 172.21.15.86:0/3130934312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a006d8f0 0x7fe2a009aa70 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0069080 tx=0x7fe28c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d8368a9c6b4e88 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe29f400640 1 -- 172.21.15.86:0/3130934312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0ab610 con 0x7fe2a006d8f0 2024-03-22T04:21:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe29f400640 1 -- 172.21.15.86:0/3130934312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28c0b0070 con 0x7fe2a006d8f0 2024-03-22T04:21:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe29f400640 1 -- 172.21.15.86:0/3130934312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0ab9d0 con 0x7fe2a006d8f0 2024-03-22T04:21:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 -- 172.21.15.86:0/3130934312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a006d8f0 msgr2=0x7fe2a009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 --2- 172.21.15.86:0/3130934312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a006d8f0 0x7fe2a009aa70 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0069080 tx=0x7fe28c01c520 comp rx=0 tx=0).stop 2024-03-22T04:21:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 -- 172.21.15.86:0/3130934312 shutdown_connections 2024-03-22T04:21:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 --2- 172.21.15.86:0/3130934312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a006d8f0 0x7fe2a009aa70 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 --2- 172.21.15.86:0/3130934312 >> v2:172.21.15.196:3300/0 conn(0x7fe2a015e970 0x7fe2a00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 -- 172.21.15.86:0/3130934312 >> 172.21.15.86:0/3130934312 conn(0x7fe2a00b3bf0 msgr2=0x7fe2a00b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 -- 172.21.15.86:0/3130934312 shutdown_connections 2024-03-22T04:21:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 -- 172.21.15.86:0/3130934312 wait complete. 2024-03-22T04:21:07.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 Processor -- start 2024-03-22T04:21:07.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.655+0000 7fe2a7400640 1 -- start start 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a006d8f0 0x7fe2a01566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a009afb0 0x7fe2a00c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e970 0x7fe2a00c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a016b550 con 0x7fe2a009afb0 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a015ed50 con 0x7fe2a015e970 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a016b6d0 con 0x7fe2a006d8f0 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a009afb0 0x7fe2a00c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a009afb0 0x7fe2a00c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37138/0 (socket says 172.21.15.86:37138) 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 -- 172.21.15.86:0/2793892199 learned_addr learned my addr 172.21.15.86:0/2793892199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:07.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a5400640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e970 0x7fe2a00c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a006d8f0 0x7fe2a01566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 -- 172.21.15.86:0/2793892199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a006d8f0 msgr2=0x7fe2a01566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a006d8f0 0x7fe2a01566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 -- 172.21.15.86:0/2793892199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e970 msgr2=0x7fe2a00c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e970 0x7fe2a00c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 -- 172.21.15.86:0/2793892199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a00c8460 con 0x7fe2a009afb0 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29fe00640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a009afb0 0x7fe2a00c91d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fe28c019130 tx=0x7fe28c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29cc00640 1 -- 172.21.15.86:0/2793892199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0be5f0 con 0x7fe2a009afb0 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29cc00640 1 -- 172.21.15.86:0/2793892199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28c0be790 con 0x7fe2a009afb0 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29cc00640 1 -- 172.21.15.86:0/2793892199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0beb60 con 0x7fe2a009afb0 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a00c8740 con 0x7fe2a009afb0 2024-03-22T04:21:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a0157ae0 con 0x7fe2a009afb0 2024-03-22T04:21:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a0074470 con 0x7fe2a009afb0 2024-03-22T04:21:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.659+0000 7fe29cc00640 1 -- 172.21.15.86:0/2793892199 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe28c0b43f0 con 0x7fe2a009afb0 2024-03-22T04:21:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.663+0000 7fe29cc00640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.663+0000 7fe29cc00640 1 -- 172.21.15.86:0/2793892199 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fe28c120850 con 0x7fe2a009afb0 2024-03-22T04:21:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.663+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.663+0000 7fe2a4a00640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b930 tx=0x7fe294008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:07.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.667+0000 7fe29cc00640 1 -- 172.21.15.86:0/2793892199 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe28c0a8050 con 0x7fe2a009afb0 2024-03-22T04:21:07.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.823+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2a00d1110 con 0x7fe2a009afb0 2024-03-22T04:21:07.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.823+0000 7fe29cc00640 1 -- 172.21.15.86:0/2793892199 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) ==== 100+0+11 (secure 0 0 0) 0x7fe28c016110 con 0x7fe2a009afb0 2024-03-22T04:21:07.824 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:21:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.823+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 msgr2=0x7fe280069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.823+0000 7fe2a7400640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b930 tx=0x7fe294008040 comp rx=0 tx=0).stop 2024-03-22T04:21:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.823+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a009afb0 msgr2=0x7fe2a00c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.823+0000 7fe2a7400640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a009afb0 0x7fe2a00c91d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fe28c019130 tx=0x7fe28c01ad60 comp rx=0 tx=0).stop 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.827+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 shutdown_connections 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.827+0000 7fe2a7400640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe280067b20 0x7fe280069fe0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.827+0000 7fe2a7400640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a015e970 0x7fe2a00c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.827+0000 7fe2a7400640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a009afb0 0x7fe2a00c91d0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.827+0000 7fe2a7400640 1 --2- 172.21.15.86:0/2793892199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2a006d8f0 0x7fe2a01566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.827+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 >> 172.21.15.86:0/2793892199 conn(0x7fe2a00b3bf0 msgr2=0x7fe2a0070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.827+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 shutdown_connections 2024-03-22T04:21:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.827+0000 7fe2a7400640 1 -- 172.21.15.86:0/2793892199 wait complete. 2024-03-22T04:21:07.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:21:07.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.983+0000 7feef5c00640 1 Processor -- start 2024-03-22T04:21:07.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.983+0000 7feef5c00640 1 -- start start 2024-03-22T04:21:07.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.983+0000 7feef5c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d0de0 0x7feef00ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.983+0000 7feef5c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef00d11c0 con 0x7feef00d1dc0 2024-03-22T04:21:07.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.983+0000 7feef5c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef00cece0 con 0x7feef00d19a0 2024-03-22T04:21:07.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.983+0000 7feef5c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feef00d0580 con 0x7feef00d0de0 2024-03-22T04:21:07.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.983+0000 7feeef400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feef00d1dc0 0x7feef015fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54562/0 (socket says 172.21.15.86:54562) 2024-03-22T04:21:07.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.983+0000 7feeef400640 1 -- 172.21.15.86:0/875646036 learned_addr learned my addr 172.21.15.86:0/875646036 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:07.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeeea00640 1 --2- 172.21.15.86:0/875646036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d0de0 0x7feef00ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeed600640 1 -- 172.21.15.86:0/875646036 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3077514091 0 0) 0x7feef00d11c0 con 0x7feef00d1dc0 2024-03-22T04:21:07.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeed600640 1 -- 172.21.15.86:0/875646036 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feed4003530 con 0x7feef00d1dc0 2024-03-22T04:21:07.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeed600640 1 -- 172.21.15.86:0/875646036 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 113584334 0 0) 0x7feef00cece0 con 0x7feef00d19a0 2024-03-22T04:21:07.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeed600640 1 -- 172.21.15.86:0/875646036 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feef00d11c0 con 0x7feef00d19a0 2024-03-22T04:21:07.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeeea00640 1 -- 172.21.15.86:0/875646036 >> v1:172.21.15.196:6789/0 conn(0x7feef00d19a0 legacy=0x7feef00cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeeea00640 1 -- 172.21.15.86:0/875646036 >> v1:172.21.15.86:6789/0 conn(0x7feef00d1dc0 legacy=0x7feef015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeeea00640 1 -- 172.21.15.86:0/875646036 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef00cefc0 con 0x7feef00d0de0 2024-03-22T04:21:07.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeeea00640 1 --2- 172.21.15.86:0/875646036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d0de0 0x7feef00ce6b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feee00097f0 tx=0x7feee001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d18f18a2ccfbef90 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeed600640 1 -- 172.21.15.86:0/875646036 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00b1070 con 0x7feef00d0de0 2024-03-22T04:21:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeed600640 1 -- 172.21.15.86:0/875646036 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee00ac630 con 0x7feef00d0de0 2024-03-22T04:21:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.987+0000 7feeed600640 1 -- 172.21.15.86:0/875646036 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00a8050 con 0x7feef00d0de0 2024-03-22T04:21:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- 172.21.15.86:0/875646036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d0de0 msgr2=0x7feef00ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 --2- 172.21.15.86:0/875646036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d0de0 0x7feef00ce6b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feee00097f0 tx=0x7feee001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- 172.21.15.86:0/875646036 shutdown_connections 2024-03-22T04:21:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 --2- 172.21.15.86:0/875646036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d0de0 0x7feef00ce6b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- 172.21.15.86:0/875646036 >> 172.21.15.86:0/875646036 conn(0x7feef006f910 msgr2=0x7feef006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- 172.21.15.86:0/875646036 shutdown_connections 2024-03-22T04:21:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- 172.21.15.86:0/875646036 wait complete. 2024-03-22T04:21:07.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 Processor -- start 2024-03-22T04:21:07.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- start start 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef00d0de0 0x7feef0152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef00d19a0 0x7feef00c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d1dc0 0x7feef00b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feef0160760 con 0x7feef00d0de0 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feef01605e0 con 0x7feef00d19a0 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feef5c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feef0071e30 con 0x7feef00d1dc0 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeeea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef00d0de0 0x7feef0152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeeea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef00d0de0 0x7feef0152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37148/0 (socket says 172.21.15.86:37148) 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeeea00640 1 -- 172.21.15.86:0/3493852239 learned_addr learned my addr 172.21.15.86:0/3493852239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeee000640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef00d19a0 0x7feef00c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeef400640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d1dc0 0x7feef00b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeeea00640 1 -- 172.21.15.86:0/3493852239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d1dc0 msgr2=0x7feef00b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeeea00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d1dc0 0x7feef00b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeeea00640 1 -- 172.21.15.86:0/3493852239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef00d19a0 msgr2=0x7feef00c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeeea00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef00d19a0 0x7feef00c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.991+0000 7feeeea00640 1 -- 172.21.15.86:0/3493852239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef00c8470 con 0x7feef00d0de0 2024-03-22T04:21:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.995+0000 7feeeea00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef00d0de0 0x7feef0152b50 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7feee00190f0 tx=0x7feee0019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.995+0000 7feee6a00640 1 -- 172.21.15.86:0/3493852239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00b9020 con 0x7feef00d0de0 2024-03-22T04:21:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.995+0000 7feee6a00640 1 -- 172.21.15.86:0/3493852239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee00b5410 con 0x7feef00d0de0 2024-03-22T04:21:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.995+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef006e4a0 con 0x7feef00d0de0 2024-03-22T04:21:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.995+0000 7feee6a00640 1 -- 172.21.15.86:0/3493852239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00b1040 con 0x7feef00d0de0 2024-03-22T04:21:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.995+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef00b2330 con 0x7feef00d0de0 2024-03-22T04:21:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.995+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef0068650 con 0x7feef00d0de0 2024-03-22T04:21:07.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.995+0000 7feee6a00640 1 -- 172.21.15.86:0/3493852239 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7feee00bc020 con 0x7feef00d0de0 2024-03-22T04:21:07.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.999+0000 7feee6a00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feed0067b50 0x7feed006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:07.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.999+0000 7feee6a00640 1 -- 172.21.15.86:0/3493852239 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7feee011f5b0 con 0x7feef00d0de0 2024-03-22T04:21:07.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.999+0000 7feeee000640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feed0067b50 0x7feed006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:07.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:07.999+0000 7feeee000640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feed0067b50 0x7feed006a010 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feedc007fd0 tx=0x7feedc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.003+0000 7feee6a00640 1 -- 172.21.15.86:0/3493852239 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7feee00160d0 con 0x7feef00d0de0 2024-03-22T04:21:08.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.167+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feef015d4d0 con 0x7feef00d0de0 2024-03-22T04:21:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feee6a00640 1 -- 172.21.15.86:0/3493852239 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) ==== 80+0+0 (secure 0 0 0) 0x7feee00efec0 con 0x7feef00d0de0 2024-03-22T04:21:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:21:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feed0067b50 msgr2=0x7feed006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feed0067b50 0x7feed006a010 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feedc007fd0 tx=0x7feedc009040 comp rx=0 tx=0).stop 2024-03-22T04:21:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef00d0de0 msgr2=0x7feef0152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef00d0de0 0x7feef0152b50 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7feee00190f0 tx=0x7feee0019280 comp rx=0 tx=0).stop 2024-03-22T04:21:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 shutdown_connections 2024-03-22T04:21:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feed0067b50 0x7feed006a010 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feef00d19a0 0x7feef00c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feef00d1dc0 0x7feef00b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 --2- 172.21.15.86:0/3493852239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef00d0de0 0x7feef0152b50 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 >> 172.21.15.86:0/3493852239 conn(0x7feef006f910 msgr2=0x7feef015c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 shutdown_connections 2024-03-22T04:21:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.171+0000 7feef5c00640 1 -- 172.21.15.86:0/3493852239 wait complete. 2024-03-22T04:21:08.196 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:21:08.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:08.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:08.249 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:21:08.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:08.249 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:21:08.269 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-22T04:21:08.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5cec00640 1 Processor -- start 2024-03-22T04:21:08.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5cec00640 1 -- start start 2024-03-22T04:21:08.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5cec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c815c730 0x7fe5c809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5cec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe5c80b11c0 0x7fe5c80b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5cec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c80cda00 con 0x7fe5c80b3610 2024-03-22T04:21:08.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5cec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c809b910 con 0x7fe5c80b11c0 2024-03-22T04:21:08.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5cec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c80b1710 con 0x7fe5c815c730 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5ccc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe5c80b11c0 0x7fe5c80b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5bfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe5c80b3610 0x7fe5c809afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54578/0 (socket says 172.21.15.86:54578) 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5bfe00640 1 -- 172.21.15.86:0/385553013 learned_addr learned my addr 172.21.15.86:0/385553013 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/385553013 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c815c730 0x7fe5c809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5ccc00640 1 -- 172.21.15.86:0/385553013 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c815c730 msgr2=0x7fe5c809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/385553013 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c815c730 0x7fe5c809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5ccc00640 1 -- 172.21.15.86:0/385553013 >> v1:172.21.15.86:6789/0 conn(0x7fe5c80b3610 legacy=0x7fe5c809afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5ccc00640 1 -- 172.21.15.86:0/385553013 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c80cdce0 con 0x7fe5c80b11c0 2024-03-22T04:21:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.447+0000 7fe5c7400640 1 -- 172.21.15.86:0/385553013 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686708845 0 0) 0x7fe5c80cda00 con 0x7fe5c80b3610 2024-03-22T04:21:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/385553013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b11c0 0x7fe5c80b1ba0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009d90 tx=0x7fe5c001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f25921d3aa2bd39 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5c7400640 1 -- 172.21.15.86:0/385553013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00b2070 con 0x7fe5c80b11c0 2024-03-22T04:21:08.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5c7400640 1 -- 172.21.15.86:0/385553013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c00a58b0 con 0x7fe5c80b11c0 2024-03-22T04:21:08.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5c7400640 1 -- 172.21.15.86:0/385553013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00a5c80 con 0x7fe5c80b11c0 2024-03-22T04:21:08.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 -- 172.21.15.86:0/385553013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b11c0 msgr2=0x7fe5c80b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 --2- 172.21.15.86:0/385553013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b11c0 0x7fe5c80b1ba0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009d90 tx=0x7fe5c001c8e0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 -- 172.21.15.86:0/385553013 shutdown_connections 2024-03-22T04:21:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 --2- 172.21.15.86:0/385553013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b11c0 0x7fe5c80b1ba0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 --2- 172.21.15.86:0/385553013 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c815c730 0x7fe5c809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 -- 172.21.15.86:0/385553013 >> 172.21.15.86:0/385553013 conn(0x7fe5c80b3bf0 msgr2=0x7fe5c80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 -- 172.21.15.86:0/385553013 shutdown_connections 2024-03-22T04:21:08.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 -- 172.21.15.86:0/385553013 wait complete. 2024-03-22T04:21:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 Processor -- start 2024-03-22T04:21:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.455+0000 7fe5cec00640 1 -- start start 2024-03-22T04:21:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5cec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c80b11c0 0x7fe5c81573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5cec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b3610 0x7fe5c80c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5cec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c815c730 0x7fe5c806d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5cec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80b2cc0 con 0x7fe5c80b3610 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5cec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80b2b40 con 0x7fe5c815c730 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5cec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c80b2e40 con 0x7fe5c80b11c0 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b3610 0x7fe5c80c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b3610 0x7fe5c80c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37168/0 (socket says 172.21.15.86:37168) 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 -- 172.21.15.86:0/214576975 learned_addr learned my addr 172.21.15.86:0/214576975 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c80b11c0 0x7fe5c81573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c815c730 0x7fe5c806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 -- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c80b11c0 msgr2=0x7fe5c81573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c80b11c0 0x7fe5c81573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 -- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c815c730 msgr2=0x7fe5c806d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c815c730 0x7fe5c806d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 -- 172.21.15.86:0/214576975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c8157eb0 con 0x7fe5c80b3610 2024-03-22T04:21:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c815c730 0x7fe5c806d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b3610 0x7fe5c80c83f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000c480 tx=0x7fe5b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5c4c00640 1 -- 172.21.15.86:0/214576975 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0018070 con 0x7fe5c80b3610 2024-03-22T04:21:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5c4c00640 1 -- 172.21.15.86:0/214576975 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b0007500 con 0x7fe5c80b3610 2024-03-22T04:21:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5cec00640 1 -- 172.21.15.86:0/214576975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c80b5950 con 0x7fe5c80b3610 2024-03-22T04:21:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5c4c00640 1 -- 172.21.15.86:0/214576975 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0004a30 con 0x7fe5c80b3610 2024-03-22T04:21:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.459+0000 7fe5cec00640 1 -- 172.21.15.86:0/214576975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c80c7910 con 0x7fe5c80b3610 2024-03-22T04:21:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c4c00640 1 -- 172.21.15.86:0/214576975 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe5b001b070 con 0x7fe5c80b3610 2024-03-22T04:21:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 Processor -- start 2024-03-22T04:21:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- start start 2024-03-22T04:21:08.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c4c00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5a4067b50 0x7fe5a406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c4c00640 1 -- 172.21.15.86:0/214576975 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fe5b0085350 con 0x7fe5c80b3610 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5a4067b50 0x7fe5a406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bef80 0x558b857bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b85772a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 0x558b85773330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b855c3c40 con 0x558b857bef80 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b85560d90 con 0x558b85772f50 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x558b855bfbf0 con 0x558b857bf8a0 2024-03-22T04:21:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bef80 0x558b857bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bef80 0x558b857bf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37174/0 (socket says 172.21.15.86:37174) 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 -- 172.21.15.86:0/44047833 learned_addr learned my addr 172.21.15.86:0/44047833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b85772a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 0x558b85773330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 -- 172.21.15.86:0/44047833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 msgr2=0x558b85772a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b85772a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5a4067b50 0x7fe5a406a010 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00af000 tx=0x7fe5c0004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 -- 172.21.15.86:0/44047833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 msgr2=0x558b85773330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 0x558b85773330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 -- 172.21.15.86:0/44047833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8566b130 con 0x558b857bef80 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bef80 0x558b857bf360 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800b880 tx=0x7fe5b800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e62fb736f5267 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5bcc00640 1 -- 172.21.15.86:0/44047833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8018020 con 0x558b857bef80 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5bcc00640 1 -- 172.21.15.86:0/44047833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b800f9c0 con 0x558b857bef80 2024-03-22T04:21:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5bcc00640 1 -- 172.21.15.86:0/44047833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800fd90 con 0x558b857bef80 2024-03-22T04:21:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- 172.21.15.86:0/44047833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bef80 msgr2=0x558b857bf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bef80 0x558b857bf360 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800b880 tx=0x7fe5b800bc80 comp rx=0 tx=0).stop 2024-03-22T04:21:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- 172.21.15.86:0/44047833 shutdown_connections 2024-03-22T04:21:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 0x558b85773330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b85772a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 --2- 172.21.15.86:0/44047833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bef80 0x558b857bf360 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- 172.21.15.86:0/44047833 >> 172.21.15.86:0/44047833 conn(0x558b8565c990 msgr2=0x558b857a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- 172.21.15.86:0/44047833 shutdown_connections 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.463+0000 7fe5d2103000 1 -- 172.21.15.86:0/44047833 wait complete. 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 -- start start 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 Processor -- start 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b857be260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 0x558b857be7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b857a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b8557f710 con 0x558b85772f50 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558b85581580 con 0x558b857bef80 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x558b855c8270 con 0x558b857bf8a0 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 10 client.0 _ll_register_callbacks cb 0x558b85625200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b857be260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b857be260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37176/0 (socket says 172.21.15.86:37176) 2024-03-22T04:21:08.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 -- 172.21.15.86:0/1052062545 learned_addr learned my addr 172.21.15.86:0/1052062545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 0x558b857be7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b857a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 -- 172.21.15.86:0/1052062545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 msgr2=0x558b857a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b857a3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 -- 172.21.15.86:0/1052062545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 msgr2=0x558b857be7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 0x558b857be7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 -- 172.21.15.86:0/1052062545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8559cd30 con 0x558b85772f50 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b857be260 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x558b855cb0c0 tx=0x7fe5b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649b4fe10ec6e97b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5b5600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5b5600640 1 -- 172.21.15.86:0/1052062545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8003620 con 0x558b85772f50 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5b5600640 1 -- 172.21.15.86:0/1052062545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fe5b8029c80 con 0x558b85772f50 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 -- 172.21.15.86:0/1052062545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b800f9c0 con 0x558b85772f50 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5b5600640 1 -- 172.21.15.86:0/1052062545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fe5b8030050 con 0x558b85772f50 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 10 client.9222 fetch_fsmap learned FSMap version 99 2024-03-22T04:21:08.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 -- 172.21.15.86:0/1052062545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x558b85608fc0 con 0x558b85772f50 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5b5600640 1 -- 172.21.15.86:0/1052062545 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b800f9c0 con 0x558b85772f50 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5b5600640 1 -- 172.21.15.86:0/1052062545 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7fe5b8033020 con 0x558b85772f50 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 10 client.9222 fetch_fsmap finished waiting for FSMap version 99 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 10 client.9222 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 20 client.9222 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x558b857a2b80 0x558b857a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 4 client.9222 mds_command: new command op to 77636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5d2103000 1 -- 172.21.15.86:0/1052062545 --> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b8555edd0 con 0x558b857a2b80 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.467+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x558b857a2b80 0x558b857a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x558b857a2b80 0x558b857a3d90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4826949e993561ec server_cookie=2dd3ce2021b4a78e in_seq=0 out_seq=0 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5b5600640 10 client.9222 ms_handle_connect on v2:172.21.15.86:6835/3165266972 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5b5600640 1 -- 172.21.15.86:0/1052062545 <== mds.0 v2:172.21.15.86:6835/3165266972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558b8555edd0 con 0x558b857a2b80 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5b5600640 10 client.9222 handle_command_reply: tid=0 2024-03-22T04:21:08.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 client.9222 shutdown 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 -- 172.21.15.86:0/1052062545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 msgr2=0x558b857be260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b857be260 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x558b855cb0c0 tx=0x7fe5b8004740 comp rx=0 tx=0).stop 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 -- 172.21.15.86:0/1052062545 shutdown_connections 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x558b857a2b80 0x558b857a3d90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 0x558b857be7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b857a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1052062545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b857be260 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5b5600640 0 client.9222 ms_handle_reset on v2:172.21.15.86:6835/3165266972 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 -- 172.21.15.86:0/1052062545 >> 172.21.15.86:0/1052062545 conn(0x558b8565c990 msgr2=0x558b85772110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 -- 172.21.15.86:0/1052062545 shutdown_connections 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 1 -- 172.21.15.86:0/1052062545 wait complete. 2024-03-22T04:21:08.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.471+0000 7fe5d2103000 20 client.9222 trim_cache size 0 max 16384 2024-03-22T04:21:08.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.483+0000 7fe5d2103000 1 Processor -- start 2024-03-22T04:21:08.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.483+0000 7fe5d2103000 1 -- start start 2024-03-22T04:21:08.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b85772180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 0x558b857726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b857d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8204030 con 0x558b85772f50 2024-03-22T04:21:08.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8203030 con 0x558b857bef80 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c80c6730 con 0x558b857bf8a0 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b85772180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b85772180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37184/0 (socket says 172.21.15.86:37184) 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 -- 172.21.15.86:0/3186014788 learned_addr learned my addr 172.21.15.86:0/3186014788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b857d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 0x558b857726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 -- 172.21.15.86:0/3186014788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 msgr2=0x558b857d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b857d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 -- 172.21.15.86:0/3186014788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 msgr2=0x558b857726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 0x558b857726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 -- 172.21.15.86:0/3186014788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b857a36f0 con 0x558b85772f50 2024-03-22T04:21:08.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b85772180 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800bcc0 tx=0x7fe5b8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3de40d9599e454 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5b7e00640 1 -- 172.21.15.86:0/3186014788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8039070 con 0x558b85772f50 2024-03-22T04:21:08.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5b7e00640 1 -- 172.21.15.86:0/3186014788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b8000e50 con 0x558b85772f50 2024-03-22T04:21:08.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5b7e00640 1 -- 172.21.15.86:0/3186014788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8018040 con 0x558b85772f50 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- 172.21.15.86:0/3186014788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 msgr2=0x558b85772180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b85772180 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800bcc0 tx=0x7fe5b8021040 comp rx=0 tx=0).stop 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- 172.21.15.86:0/3186014788 shutdown_connections 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b857bef80 0x558b857726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bf8a0 0x558b857d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 --2- 172.21.15.86:0/3186014788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b85772f50 0x558b85772180 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- 172.21.15.86:0/3186014788 >> 172.21.15.86:0/3186014788 conn(0x558b8565c990 msgr2=0x558b857cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- 172.21.15.86:0/3186014788 shutdown_connections 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- 172.21.15.86:0/3186014788 wait complete. 2024-03-22T04:21:08.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 -- start start 2024-03-22T04:21:08.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.487+0000 7fe5d2103000 1 Processor -- start 2024-03-22T04:21:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 0x558b8582e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bef80 0x558b8582ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bf8a0 0x558b857d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b8567f810 con 0x558b857bf8a0 2024-03-22T04:21:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8202a10 con 0x558b85772f50 2024-03-22T04:21:08.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x558b8557f710 con 0x558b857bef80 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 10 client.0 _ll_register_callbacks cb 0x7fe5c00b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bf8a0 0x558b857d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bf8a0 0x558b857d37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37192/0 (socket says 172.21.15.86:37192) 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 -- 172.21.15.86:0/1822842671 learned_addr learned my addr 172.21.15.86:0/1822842671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 0x558b8582e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5bfe00640 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bef80 0x558b8582ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 -- 172.21.15.86:0/1822842671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bef80 msgr2=0x558b8582ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bef80 0x558b8582ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 -- 172.21.15.86:0/1822842671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 msgr2=0x558b8582e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 0x558b8582e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 -- 172.21.15.86:0/1822842671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b855993a0 con 0x558b857bf8a0 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5ccc00640 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bf8a0 0x558b857d37b0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe5c001c920 tx=0x7fe5c001b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed1f415d89831a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5b5600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5b5600640 1 -- 172.21.15.86:0/1822842671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00c7070 con 0x558b857bf8a0 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5b5600640 1 -- 172.21.15.86:0/1822842671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fe5c00ad700 con 0x558b857bf8a0 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 -- 172.21.15.86:0/1822842671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b8000e50 con 0x558b857bf8a0 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5b5600640 1 -- 172.21.15.86:0/1822842671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fe5c00d1050 con 0x558b857bf8a0 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 10 client.9228 fetch_fsmap learned FSMap version 99 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 -- 172.21.15.86:0/1822842671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x558b85549380 con 0x558b857bf8a0 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5b5600640 1 -- 172.21.15.86:0/1822842671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b8000e50 con 0x558b857bf8a0 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5b5600640 1 -- 172.21.15.86:0/1822842671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7fe5c00d4020 con 0x558b857bf8a0 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 10 client.9228 fetch_fsmap finished waiting for FSMap version 99 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 10 client.9228 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 20 client.9228 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x558b857d44b0 0x558b857d48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 4 client.9228 mds_command: new command op to 77636 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-22T04:21:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5d2103000 1 -- 172.21.15.86:0/1822842671 --> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b85563160 con 0x558b857d44b0 2024-03-22T04:21:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.491+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x558b857d44b0 0x558b857d48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.495+0000 7fe5b5600640 10 client.9228 ms_handle_connect on v2:172.21.15.86:6835/3165266972 2024-03-22T04:21:08.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.495+0000 7fe5c7e00640 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x558b857d44b0 0x558b857d48c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe352092ab0a7a62 server_cookie=33f8344f0884448 in_seq=0 out_seq=0 2024-03-22T04:21:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5b5600640 1 -- 172.21.15.86:0/1822842671 <== mds.0 v2:172.21.15.86:6835/3165266972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b85563160 con 0x558b857d44b0 2024-03-22T04:21:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5b5600640 10 client.9228 handle_command_reply: tid=0 2024-03-22T04:21:08.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 client.9228 shutdown 2024-03-22T04:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 -- 172.21.15.86:0/1822842671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bf8a0 msgr2=0x558b857d37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bf8a0 0x558b857d37b0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe5c001c920 tx=0x7fe5c001b120 comp rx=0 tx=0).stop 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 -- 172.21.15.86:0/1822842671 shutdown_connections 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.86:6835/3165266972,v1:172.21.15.86:6837/3165266972] conn(0x558b857d44b0 0x558b857d48c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b857bf8a0 0x558b857d37b0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558b857bef80 0x558b8582ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5b5600640 0 client.9228 ms_handle_reset on v2:172.21.15.86:6835/3165266972 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 --2- 172.21.15.86:0/1822842671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558b85772f50 0x558b8582e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.503+0000 7fe5d2103000 1 -- 172.21.15.86:0/1822842671 >> 172.21.15.86:0/1822842671 conn(0x558b8565c990 msgr2=0x558b857cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5d2103000 1 -- 172.21.15.86:0/1822842671 shutdown_connections 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5d2103000 1 -- 172.21.15.86:0/1822842671 wait complete. 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5d2103000 20 client.9228 trim_cache size 0 max 16384 2024-03-22T04:21:08.507 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "message": "", 2024-03-22T04:21:08.508 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0 2024-03-22T04:21:08.508 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 -- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5a4067b50 msgr2=0x7fe5a406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5a4067b50 0x7fe5a406a010 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00af000 tx=0x7fe5c0004ac0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 -- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b3610 msgr2=0x7fe5c80c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b3610 0x7fe5c80c83f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000c480 tx=0x7fe5b000c940 comp rx=0 tx=0).stop 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 -- 172.21.15.86:0/214576975 shutdown_connections 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe5a4067b50 0x7fe5a406a010 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c815c730 0x7fe5c806d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c80b3610 0x7fe5c80c83f0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 --2- 172.21.15.86:0/214576975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c80b11c0 0x7fe5c81573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 -- 172.21.15.86:0/214576975 >> 172.21.15.86:0/214576975 conn(0x7fe5c80b3bf0 msgr2=0x7fe5c80cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 -- 172.21.15.86:0/214576975 shutdown_connections 2024-03-22T04:21:08.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.507+0000 7fe5cec00640 1 -- 172.21.15.86:0/214576975 wait complete. 2024-03-22T04:21:08.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:21:08.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f307fa00640 1 Processor -- start 2024-03-22T04:21:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f307fa00640 1 -- start start 2024-03-22T04:21:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f307fa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f307815e630 0x7f30780cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f307fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f307806e2f0 0x7f30780d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f307fa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 0x7f307806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f307fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f30781623d0 con 0x7f307806e2f0 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f307fa00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f30781620f0 con 0x7f307815e630 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f307fa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f307815ec10 con 0x7f307806cc80 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f3077e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f307806e2f0 0x7f30780d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f3077e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f307806e2f0 0x7f30780d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37206/0 (socket says 172.21.15.86:37206) 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.659+0000 7f3077e00640 1 -- 172.21.15.86:0/3975331147 learned_addr learned my addr 172.21.15.86:0/3975331147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f307da00640 1 --2- 172.21.15.86:0/3975331147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 0x7f307806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f307d000640 1 --2- 172.21.15.86:0/3975331147 >> v2:172.21.15.196:3300/0 conn(0x7f307815e630 0x7f30780cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077e00640 1 -- 172.21.15.86:0/3975331147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 msgr2=0x7f307806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077e00640 1 --2- 172.21.15.86:0/3975331147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 0x7f307806d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077e00640 1 -- 172.21.15.86:0/3975331147 >> v2:172.21.15.196:3300/0 conn(0x7f307815e630 msgr2=0x7f30780cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077e00640 1 --2- 172.21.15.86:0/3975331147 >> v2:172.21.15.196:3300/0 conn(0x7f307815e630 0x7f30780cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077e00640 1 -- 172.21.15.86:0/3975331147 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307815eef0 con 0x7f307806e2f0 2024-03-22T04:21:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077e00640 1 --2- 172.21.15.86:0/3975331147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780d03b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f3078072430 tx=0x7f306401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82fc2588e4430ca server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077400640 1 -- 172.21.15.86:0/3975331147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640b0070 con 0x7f307806e2f0 2024-03-22T04:21:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077400640 1 -- 172.21.15.86:0/3975331147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30640ab600 con 0x7f307806e2f0 2024-03-22T04:21:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f3077400640 1 -- 172.21.15.86:0/3975331147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640ab9d0 con 0x7f307806e2f0 2024-03-22T04:21:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f307fa00640 1 -- 172.21.15.86:0/3975331147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 msgr2=0x7f30780d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.663+0000 7f307fa00640 1 --2- 172.21.15.86:0/3975331147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780d03b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f3078072430 tx=0x7f306401c520 comp rx=0 tx=0).stop 2024-03-22T04:21:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 -- 172.21.15.86:0/3975331147 shutdown_connections 2024-03-22T04:21:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 --2- 172.21.15.86:0/3975331147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 0x7f307806d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 --2- 172.21.15.86:0/3975331147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780d03b0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 --2- 172.21.15.86:0/3975331147 >> v2:172.21.15.196:3300/0 conn(0x7f307815e630 0x7f30780cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 -- 172.21.15.86:0/3975331147 >> 172.21.15.86:0/3975331147 conn(0x7f307806f910 msgr2=0x7f307806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 -- 172.21.15.86:0/3975331147 shutdown_connections 2024-03-22T04:21:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 -- 172.21.15.86:0/3975331147 wait complete. 2024-03-22T04:21:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 Processor -- start 2024-03-22T04:21:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 -- start start 2024-03-22T04:21:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 0x7f3078152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307815e630 0x7f30780cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f307806d9f0 con 0x7f307806e2f0 2024-03-22T04:21:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f307815f9d0 con 0x7f307815e630 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307fa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30780cfaf0 con 0x7f307806cc80 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37210/0 (socket says 172.21.15.86:37210) 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 -- 172.21.15.86:0/4067484431 learned_addr learned my addr 172.21.15.86:0/4067484431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307d000640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 0x7f3078152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f307da00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307815e630 0x7f30780cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 -- 172.21.15.86:0/4067484431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 msgr2=0x7f3078152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 0x7f3078152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 -- 172.21.15.86:0/4067484431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307815e630 msgr2=0x7f30780cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307815e630 0x7f30780cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 -- 172.21.15.86:0/4067484431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3078161390 con 0x7f307806e2f0 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.667+0000 7f3077e00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780cccb0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f306401abd0 tx=0x7f30640192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f3074c00640 1 -- 172.21.15.86:0/4067484431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640ba070 con 0x7f307806e2f0 2024-03-22T04:21:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f3074c00640 1 -- 172.21.15.86:0/4067484431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30640b4620 con 0x7f307806e2f0 2024-03-22T04:21:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30780c8a60 con 0x7f307806e2f0 2024-03-22T04:21:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f3074c00640 1 -- 172.21.15.86:0/4067484431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640b0040 con 0x7f307806e2f0 2024-03-22T04:21:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781f79d0 con 0x7f307806e2f0 2024-03-22T04:21:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3078068650 con 0x7f307806e2f0 2024-03-22T04:21:08.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f3074c00640 1 -- 172.21.15.86:0/4067484431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f30640bd070 con 0x7f307806e2f0 2024-03-22T04:21:08.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f3074c00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3058067b20 0x7f3058069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:08.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.671+0000 7f3074c00640 1 -- 172.21.15.86:0/4067484431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f306411f190 con 0x7f307806e2f0 2024-03-22T04:21:08.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.675+0000 7f307d000640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3058067b20 0x7f3058069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:08.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.675+0000 7f307d000640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3058067b20 0x7f3058069fe0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f30780cdb70 tx=0x7f306c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:08.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.679+0000 7f3074c00640 1 -- 172.21.15.86:0/4067484431 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3064016110 con 0x7f307806e2f0 2024-03-22T04:21:08.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f30781f74a0 con 0x7f307806e2f0 2024-03-22T04:21:08.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f3074c00640 1 -- 172.21.15.86:0/4067484431 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f30640ef320 con 0x7f307806e2f0 2024-03-22T04:21:08.872 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:21:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3058067b20 msgr2=0x7f3058069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3058067b20 0x7f3058069fe0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f30780cdb70 tx=0x7f306c008040 comp rx=0 tx=0).stop 2024-03-22T04:21:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 msgr2=0x7f30780cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780cccb0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f306401abd0 tx=0x7f30640192c0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 shutdown_connections 2024-03-22T04:21:08.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3058067b20 0x7f3058069fe0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307815e630 0x7f30780cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307806e2f0 0x7f30780cccb0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 --2- 172.21.15.86:0/4067484431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307806cc80 0x7f3078152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:08.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.871+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 >> 172.21.15.86:0/4067484431 conn(0x7f307806f910 msgr2=0x7f3078162550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:08.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.875+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 shutdown_connections 2024-03-22T04:21:08.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:08.875+0000 7f307fa00640 1 -- 172.21.15.86:0/4067484431 wait complete. 2024-03-22T04:21:08.892 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:21:08.893 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:21:08.893 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:21:08.893 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:21:08.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:08.893 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:21:08.893 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:21:08.893 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:21:08.893 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:08.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:08.893 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:21:08.903 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41863sec preferred_lft 41863sec 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:08.904 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:08.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:08.905 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:21:08.967 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:21:08.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:08.969 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:21:09.021 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:21:09.021 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:21:09.021 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:21:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:21:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:21:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:21:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:21:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:21:09.022 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:21:09.022 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:21:09.022 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:21:09.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:21:09.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:21:09.023 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:21:09.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:21:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:09.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:09.156 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:21:09.207 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:21:09.207 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:21:09.208 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:09.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:09.208 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:21:09.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:21:09.274 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:09.323 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:09.323 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:21:09.323 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:09.374 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:21:09.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:09.374 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:09.465 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:09.465 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:09.465 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:09.480 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:09.480 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:09.481 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:09.481 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:21:09.481 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:21:09.490 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:09.490 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:09.490 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:09.490 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:21:09.490 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:21:09.490 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:21:09.491 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:21:09.491 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:21:09.494 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:21:09.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:09.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:09.514 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:21:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:21:09.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:09.613 INFO:teuthology.orchestra.run.smithi086.stdout:client8923 2024-03-22T04:21:09.613 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:09.613 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:09.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:09.622 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-03-22T04:21:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:09.679 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-03-22T04:21:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:8+0 records in 2024-03-22T04:21:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:8+0 records out 2024-03-22T04:21:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0478877 s, 175 MB/s 2024-03-22T04:21:09.780 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:09.780 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:09.780 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:09.780 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:09.780 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-22T04:21:09.780 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:09.844 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627779 2024-03-22T04:21:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:09.853 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:09.853 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:09.853 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:09.853 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:09.853 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-22T04:21:09.853 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:09.957 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627780 2024-03-22T04:21:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:09.966 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:09.966 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:10.018 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:10.019 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:10.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:10.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.247+0000 7ff72bc00640 1 Processor -- start 2024-03-22T04:21:10.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.247+0000 7ff72bc00640 1 -- start start 2024-03-22T04:21:10.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff72bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff72415e630 0x7ff7240cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff72bc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff7240d03b0 0x7ff72406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff72bc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7241623d0 con 0x7ff72406e2f0 2024-03-22T04:21:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff72bc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff7240d0800 con 0x7ff7240d03b0 2024-03-22T04:21:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff72bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff7241620f0 con 0x7ff72415e630 2024-03-22T04:21:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff723e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff72406e2f0 0x7ff72406cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54586/0 (socket says 172.21.15.86:54586) 2024-03-22T04:21:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff723e00640 1 -- 172.21.15.86:0/698638899 learned_addr learned my addr 172.21.15.86:0/698638899 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff729200640 1 --2- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff72415e630 0x7ff7240cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff729c00640 1 --2- 172.21.15.86:0/698638899 >> v2:172.21.15.196:3300/0 conn(0x7ff7240d03b0 0x7ff72406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff723400640 1 -- 172.21.15.86:0/698638899 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2167736761 0 0) 0x7ff7241623d0 con 0x7ff72406e2f0 2024-03-22T04:21:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff723400640 1 -- 172.21.15.86:0/698638899 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff70c003530 con 0x7ff72406e2f0 2024-03-22T04:21:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff729c00640 1 -- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff72415e630 msgr2=0x7ff7240cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff729c00640 1 --2- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff72415e630 0x7ff7240cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff729c00640 1 -- 172.21.15.86:0/698638899 >> v1:172.21.15.86:6789/0 conn(0x7ff72406e2f0 legacy=0x7ff72406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff729c00640 1 -- 172.21.15.86:0/698638899 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72415ec10 con 0x7ff7240d03b0 2024-03-22T04:21:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.251+0000 7ff729200640 1 --2- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff72415e630 0x7ff7240cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:10.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff729c00640 1 --2- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7240d03b0 0x7ff72406d390 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009d90 tx=0x7ff71c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3802171147be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff723400640 1 -- 172.21.15.86:0/698638899 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0b2070 con 0x7ff7240d03b0 2024-03-22T04:21:10.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff723400640 1 -- 172.21.15.86:0/698638899 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff71c0a5830 con 0x7ff7240d03b0 2024-03-22T04:21:10.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff723400640 1 -- 172.21.15.86:0/698638899 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0a5be0 con 0x7ff7240d03b0 2024-03-22T04:21:10.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff72bc00640 1 -- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7240d03b0 msgr2=0x7ff72406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff72bc00640 1 --2- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7240d03b0 0x7ff72406d390 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009d90 tx=0x7ff71c01c970 comp rx=0 tx=0).stop 2024-03-22T04:21:10.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff72bc00640 1 -- 172.21.15.86:0/698638899 shutdown_connections 2024-03-22T04:21:10.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff72bc00640 1 --2- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff7240d03b0 0x7ff72406d390 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff72bc00640 1 --2- 172.21.15.86:0/698638899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff72415e630 0x7ff7240cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff72bc00640 1 -- 172.21.15.86:0/698638899 >> 172.21.15.86:0/698638899 conn(0x7ff72406f910 msgr2=0x7ff72406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.255+0000 7ff72bc00640 1 -- 172.21.15.86:0/698638899 shutdown_connections 2024-03-22T04:21:10.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 -- 172.21.15.86:0/698638899 wait complete. 2024-03-22T04:21:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 Processor -- start 2024-03-22T04:21:10.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 -- start start 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff72406e2f0 0x7ff7241546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7240d03b0 0x7ff7240c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff72415e630 0x7ff724154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff72415f9d0 con 0x7ff72415e630 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7240d1f60 con 0x7ff72406e2f0 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff72bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff7240cfaf0 con 0x7ff7240d03b0 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff72415e630 0x7ff724154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff72415e630 0x7ff724154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37220/0 (socket says 172.21.15.86:37220) 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 -- 172.21.15.86:0/3862042683 learned_addr learned my addr 172.21.15.86:0/3862042683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729200640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff72406e2f0 0x7ff7241546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff723e00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7240d03b0 0x7ff7240c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 -- 172.21.15.86:0/3862042683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7240d03b0 msgr2=0x7ff7240c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7240d03b0 0x7ff7240c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 -- 172.21.15.86:0/3862042683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff72406e2f0 msgr2=0x7ff7241546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff72406e2f0 0x7ff7241546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 -- 172.21.15.86:0/3862042683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7241561c0 con 0x7ff72415e630 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.259+0000 7ff729c00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff72415e630 0x7ff724154e70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0b0040 tx=0x7ff71c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff720c00640 1 -- 172.21.15.86:0/3862042683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0b2050 con 0x7ff72415e630 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff720c00640 1 -- 172.21.15.86:0/3862042683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff71c0be070 con 0x7ff72415e630 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff720c00640 1 -- 172.21.15.86:0/3862042683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c003db0 con 0x7ff72415e630 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7241564a0 con 0x7ff72415e630 2024-03-22T04:21:10.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff724156a60 con 0x7ff72415e630 2024-03-22T04:21:10.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff724068650 con 0x7ff72415e630 2024-03-22T04:21:10.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff720c00640 1 -- 172.21.15.86:0/3862042683 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff71c004040 con 0x7ff72415e630 2024-03-22T04:21:10.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff720c00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff704067b20 0x7ff704069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff720c00640 1 -- 172.21.15.86:0/3862042683 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7ff71c120e50 con 0x7ff72415e630 2024-03-22T04:21:10.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.263+0000 7ff729200640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff704067b20 0x7ff704069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.267+0000 7ff729200640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff704067b20 0x7ff704069fe0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff718009580 tx=0x7ff718009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.271+0000 7ff720c00640 1 -- 172.21.15.86:0/3862042683 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff71c0c8020 con 0x7ff72415e630 2024-03-22T04:21:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff724160760 con 0x7ff72415e630 2024-03-22T04:21:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff720c00640 1 -- 172.21.15.86:0/3862042683 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+16956 (secure 0 0 0) 0x7ff71c016560 con 0x7ff72415e630 2024-03-22T04:21:10.428 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:10.428 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:20:59.189511+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff704067b20 msgr2=0x7ff704069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff704067b20 0x7ff704069fe0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff718009580 tx=0x7ff718009190 comp rx=0 tx=0).stop 2024-03-22T04:21:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff72415e630 msgr2=0x7ff724154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff72415e630 0x7ff724154e70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0b0040 tx=0x7ff71c019710 comp rx=0 tx=0).stop 2024-03-22T04:21:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 shutdown_connections 2024-03-22T04:21:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff704067b20 0x7ff704069fe0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff72415e630 0x7ff724154e70 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7240d03b0 0x7ff7240c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 --2- 172.21.15.86:0/3862042683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff72406e2f0 0x7ff7241546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.427+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 >> 172.21.15.86:0/3862042683 conn(0x7ff72406f910 msgr2=0x7ff7240d2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.431+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 shutdown_connections 2024-03-22T04:21:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.431+0000 7ff72bc00640 1 -- 172.21.15.86:0/3862042683 wait complete. 2024-03-22T04:21:10.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:20:55.270181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:20:56.532178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:21:10.449 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:21:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc602c00640 1 Processor -- start 2024-03-22T04:21:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc602c00640 1 -- start start 2024-03-22T04:21:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc602c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc161cf0 0x7fc5fc0d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc602c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc5fc0d11f0 0x7fc5fc0cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc602c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5fc0d51a0 con 0x7fc5fc0b32b0 2024-03-22T04:21:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc602c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc5fc0d0670 con 0x7fc5fc0d11f0 2024-03-22T04:21:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc602c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc0d0420 con 0x7fc5fc161cf0 2024-03-22T04:21:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc5f3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc5fc0b32b0 0x7fc5fc0b4240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54590/0 (socket says 172.21.15.86:54590) 2024-03-22T04:21:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc600c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc5fc0d11f0 0x7fc5fc0cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc5f3e00640 1 -- 172.21.15.86:0/3171340552 learned_addr learned my addr 172.21.15.86:0/3171340552 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc5fbe00640 1 --2- 172.21.15.86:0/3171340552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc161cf0 0x7fc5fc0d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc600c00640 1 -- 172.21.15.86:0/3171340552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc161cf0 msgr2=0x7fc5fc0d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.587+0000 7fc600c00640 1 --2- 172.21.15.86:0/3171340552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc161cf0 0x7fc5fc0d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc600c00640 1 -- 172.21.15.86:0/3171340552 >> v1:172.21.15.86:6789/0 conn(0x7fc5fc0b32b0 legacy=0x7fc5fc0b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc600c00640 1 -- 172.21.15.86:0/3171340552 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc0d4c20 con 0x7fc5fc0d11f0 2024-03-22T04:21:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc5fb400640 1 -- 172.21.15.86:0/3171340552 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2372296153 0 0) 0x7fc5fc0d51a0 con 0x7fc5fc0b32b0 2024-03-22T04:21:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc600c00640 1 --2- 172.21.15.86:0/3171340552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc0d11f0 0x7fc5fc0cfee0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009d90 tx=0x7fc5f401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e044b2777b323393 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc5fb400640 1 -- 172.21.15.86:0/3171340552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40b4070 con 0x7fc5fc0d11f0 2024-03-22T04:21:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc5fb400640 1 -- 172.21.15.86:0/3171340552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f40af600 con 0x7fc5fc0d11f0 2024-03-22T04:21:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc5fb400640 1 -- 172.21.15.86:0/3171340552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40af9d0 con 0x7fc5fc0d11f0 2024-03-22T04:21:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc602c00640 1 -- 172.21.15.86:0/3171340552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc0d11f0 msgr2=0x7fc5fc0cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.591+0000 7fc602c00640 1 --2- 172.21.15.86:0/3171340552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc0d11f0 0x7fc5fc0cfee0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009d90 tx=0x7fc5f401f150 comp rx=0 tx=0).stop 2024-03-22T04:21:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.595+0000 7fc602c00640 1 -- 172.21.15.86:0/3171340552 shutdown_connections 2024-03-22T04:21:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.595+0000 7fc602c00640 1 --2- 172.21.15.86:0/3171340552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc0d11f0 0x7fc5fc0cfee0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.595+0000 7fc602c00640 1 --2- 172.21.15.86:0/3171340552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc161cf0 0x7fc5fc0d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.595+0000 7fc602c00640 1 -- 172.21.15.86:0/3171340552 >> 172.21.15.86:0/3171340552 conn(0x7fc5fc09d630 msgr2=0x7fc5fc0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.595+0000 7fc602c00640 1 -- 172.21.15.86:0/3171340552 shutdown_connections 2024-03-22T04:21:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.595+0000 7fc602c00640 1 -- 172.21.15.86:0/3171340552 wait complete. 2024-03-22T04:21:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.595+0000 7fc602c00640 1 Processor -- start 2024-03-22T04:21:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.595+0000 7fc602c00640 1 -- start start 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc0b32b0 0x7fc5fc15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5fc0d11f0 0x7fc5fc0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc161cf0 0x7fc5fc0ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc0d3480 con 0x7fc5fc161cf0 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc0d3300 con 0x7fc5fc0d11f0 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc1620d0 con 0x7fc5fc0b32b0 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc161cf0 0x7fc5fc0ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc161cf0 0x7fc5fc0ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37228/0 (socket says 172.21.15.86:37228) 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 -- 172.21.15.86:0/1979516450 learned_addr learned my addr 172.21.15.86:0/1979516450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc5fbe00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc0b32b0 0x7fc5fc15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc5f3e00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5fc0d11f0 0x7fc5fc0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 -- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc0b32b0 msgr2=0x7fc5fc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc0b32b0 0x7fc5fc15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 -- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5fc0d11f0 msgr2=0x7fc5fc0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5fc0d11f0 0x7fc5fc0ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 -- 172.21.15.86:0/1979516450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc15cea0 con 0x7fc5fc161cf0 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc5fbe00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc0b32b0 0x7fc5fc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc600c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc161cf0 0x7fc5fc0ca630 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009d90 tx=0x7fc5f401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc5f8c00640 1 -- 172.21.15.86:0/1979516450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40b4040 con 0x7fc5fc161cf0 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc5f8c00640 1 -- 172.21.15.86:0/1979516450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f40039a0 con 0x7fc5fc161cf0 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc5f8c00640 1 -- 172.21.15.86:0/1979516450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4009450 con 0x7fc5fc161cf0 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc0b90e0 con 0x7fc5fc161cf0 2024-03-22T04:21:10.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc0c82b0 con 0x7fc5fc161cf0 2024-03-22T04:21:10.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc074470 con 0x7fc5fc161cf0 2024-03-22T04:21:10.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.599+0000 7fc5f8c00640 1 -- 172.21.15.86:0/1979516450 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc5f40c5060 con 0x7fc5fc161cf0 2024-03-22T04:21:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.603+0000 7fc5f8c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5d8067b20 0x7fc5d8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.603+0000 7fc5f8c00640 1 -- 172.21.15.86:0/1979516450 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fc5f41237a0 con 0x7fc5fc161cf0 2024-03-22T04:21:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.603+0000 7fc5fbe00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5d8067b20 0x7fc5d8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.603+0000 7fc5fbe00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5d8067b20 0x7fc5d8069fe0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0c77e0 tx=0x7fc5ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.607+0000 7fc5f8c00640 1 -- 172.21.15.86:0/1979516450 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc5f4018cb0 con 0x7fc5fc161cf0 2024-03-22T04:21:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5fc075020 con 0x7fc5fc161cf0 2024-03-22T04:21:10.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc5f8c00640 1 -- 172.21.15.86:0/1979516450 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) ==== 93+0+10 (secure 0 0 0) 0x7fc5f40c5350 con 0x7fc5fc161cf0 2024-03-22T04:21:10.764 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:21:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5d8067b20 msgr2=0x7fc5d8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5d8067b20 0x7fc5d8069fe0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0c77e0 tx=0x7fc5ec008040 comp rx=0 tx=0).stop 2024-03-22T04:21:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc161cf0 msgr2=0x7fc5fc0ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc161cf0 0x7fc5fc0ca630 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009d90 tx=0x7fc5f401bef0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 shutdown_connections 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc5d8067b20 0x7fc5d8069fe0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5fc0d11f0 0x7fc5fc0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5fc161cf0 0x7fc5fc0ca630 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 --2- 172.21.15.86:0/1979516450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5fc0b32b0 0x7fc5fc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.763+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 >> 172.21.15.86:0/1979516450 conn(0x7fc5fc09d630 msgr2=0x7fc5fc06c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.767+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 shutdown_connections 2024-03-22T04:21:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.767+0000 7fc602c00640 1 -- 172.21.15.86:0/1979516450 wait complete. 2024-03-22T04:21:10.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:21:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.927+0000 7f1179600640 1 Processor -- start 2024-03-22T04:21:10.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.927+0000 7f1179600640 1 -- start start 2024-03-22T04:21:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.927+0000 7f1179600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f117415c730 0x7f117409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1179600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11740cda00 con 0x7f11740b3610 2024-03-22T04:21:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1179600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11740cdce0 con 0x7f11740b11c0 2024-03-22T04:21:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1179600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f117409b910 con 0x7f117415c730 2024-03-22T04:21:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f116a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f11740b3610 0x7f117409afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54596/0 (socket says 172.21.15.86:54596) 2024-03-22T04:21:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f116a000640 1 -- 172.21.15.86:0/3856248255 learned_addr learned my addr 172.21.15.86:0/3856248255 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1172a00640 1 --2- 172.21.15.86:0/3856248255 >> v2:172.21.15.196:3300/0 conn(0x7f117415c730 0x7f117409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1172000640 1 -- 172.21.15.86:0/3856248255 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870324264 0 0) 0x7f11740cda00 con 0x7f11740b3610 2024-03-22T04:21:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1172000640 1 -- 172.21.15.86:0/3856248255 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1158003530 con 0x7f11740b3610 2024-03-22T04:21:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1172a00640 1 -- 172.21.15.86:0/3856248255 >> v1:172.21.15.196:6789/0 conn(0x7f11740b11c0 legacy=0x7f11740b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1172a00640 1 -- 172.21.15.86:0/3856248255 >> v1:172.21.15.86:6789/0 conn(0x7f11740b3610 legacy=0x7f117409afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1172a00640 1 -- 172.21.15.86:0/3856248255 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11740cdfc0 con 0x7f117415c730 2024-03-22T04:21:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.931+0000 7f1172a00640 1 --2- 172.21.15.86:0/3856248255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117415c730 0x7f117409aa70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f11640097f0 tx=0x7f116401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77081aee8e2196c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1172000640 1 -- 172.21.15.86:0/3856248255 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11640b0070 con 0x7f117415c730 2024-03-22T04:21:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1172000640 1 -- 172.21.15.86:0/3856248255 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11640ab600 con 0x7f117415c730 2024-03-22T04:21:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1172000640 1 -- 172.21.15.86:0/3856248255 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11640ab9d0 con 0x7f117415c730 2024-03-22T04:21:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 -- 172.21.15.86:0/3856248255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117415c730 msgr2=0x7f117409aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 --2- 172.21.15.86:0/3856248255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117415c730 0x7f117409aa70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f11640097f0 tx=0x7f116401c410 comp rx=0 tx=0).stop 2024-03-22T04:21:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 -- 172.21.15.86:0/3856248255 shutdown_connections 2024-03-22T04:21:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 --2- 172.21.15.86:0/3856248255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f117415c730 0x7f117409aa70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 -- 172.21.15.86:0/3856248255 >> 172.21.15.86:0/3856248255 conn(0x7f11740b3bf0 msgr2=0x7f11740b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 -- 172.21.15.86:0/3856248255 shutdown_connections 2024-03-22T04:21:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 -- 172.21.15.86:0/3856248255 wait complete. 2024-03-22T04:21:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 Processor -- start 2024-03-22T04:21:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.935+0000 7f1179600640 1 -- start start 2024-03-22T04:21:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1179600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11740b11c0 0x7f11740b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1179600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11740b3610 0x7f1174148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1179600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117415c730 0x7f11740cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1179600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11740b2e90 con 0x7f117415c730 2024-03-22T04:21:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1179600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11740b2d10 con 0x7f11740b3610 2024-03-22T04:21:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1179600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f117409b6c0 con 0x7f11740b11c0 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117415c730 0x7f11740cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117415c730 0x7f11740cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37238/0 (socket says 172.21.15.86:37238) 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 -- 172.21.15.86:0/2396690081 learned_addr learned my addr 172.21.15.86:0/2396690081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1172a00640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11740b11c0 0x7f11740b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f116a000640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11740b3610 0x7f1174148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 -- 172.21.15.86:0/2396690081 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11740b11c0 msgr2=0x7f11740b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11740b11c0 0x7f11740b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 -- 172.21.15.86:0/2396690081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11740b3610 msgr2=0x7f1174148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11740b3610 0x7f1174148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 -- 172.21.15.86:0/2396690081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11740b7c00 con 0x7f117415c730 2024-03-22T04:21:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1173400640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117415c730 0x7f11740cb3a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f116c00c760 tx=0x7f116c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f116b400640 1 -- 172.21.15.86:0/2396690081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c003b80 con 0x7f117415c730 2024-03-22T04:21:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f116b400640 1 -- 172.21.15.86:0/2396690081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116c003d20 con 0x7f117415c730 2024-03-22T04:21:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117415a920 con 0x7f117415c730 2024-03-22T04:21:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f116b400640 1 -- 172.21.15.86:0/2396690081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c00aa30 con 0x7f117415c730 2024-03-22T04:21:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.939+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117415aee0 con 0x7f117415c730 2024-03-22T04:21:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.943+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1174074470 con 0x7f117415c730 2024-03-22T04:21:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.943+0000 7f116b400640 1 -- 172.21.15.86:0/2396690081 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f116c00abd0 con 0x7f117415c730 2024-03-22T04:21:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.943+0000 7f116b400640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1150067b50 0x7f115006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.943+0000 7f116b400640 1 -- 172.21.15.86:0/2396690081 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f116c085f60 con 0x7f117415c730 2024-03-22T04:21:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.943+0000 7f1172a00640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1150067b50 0x7f115006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:10.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.943+0000 7f1172a00640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1150067b50 0x7f115006a010 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f11640ad000 tx=0x7f11640a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:10.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:10.947+0000 7f116b400640 1 -- 172.21.15.86:0/2396690081 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f116c056740 con 0x7f117415c730 2024-03-22T04:21:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.115+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f11740c5730 con 0x7f117415c730 2024-03-22T04:21:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.115+0000 7f116b400640 1 -- 172.21.15.86:0/2396690081 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) ==== 104+0+11 (secure 0 0 0) 0x7f116c056040 con 0x7f117415c730 2024-03-22T04:21:11.116 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:21:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1150067b50 msgr2=0x7f115006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1150067b50 0x7f115006a010 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f11640ad000 tx=0x7f11640a6000 comp rx=0 tx=0).stop 2024-03-22T04:21:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117415c730 msgr2=0x7f11740cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117415c730 0x7f11740cb3a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f116c00c760 tx=0x7f116c00cc20 comp rx=0 tx=0).stop 2024-03-22T04:21:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 shutdown_connections 2024-03-22T04:21:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1150067b50 0x7f115006a010 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11740b3610 0x7f1174148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117415c730 0x7f11740cb3a0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 --2- 172.21.15.86:0/2396690081 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11740b11c0 0x7f11740b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 >> 172.21.15.86:0/2396690081 conn(0x7f11740b3bf0 msgr2=0x7f1174070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 shutdown_connections 2024-03-22T04:21:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.119+0000 7f1179600640 1 -- 172.21.15.86:0/2396690081 wait complete. 2024-03-22T04:21:11.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:21:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.275+0000 7f6eb2800640 1 Processor -- start 2024-03-22T04:21:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.275+0000 7f6eb2800640 1 -- start start 2024-03-22T04:21:11.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.275+0000 7f6eb2800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6eac0b11c0 0x7f6eac09b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.275+0000 7f6eb2800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eac0cda00 con 0x7f6eac0b3610 2024-03-22T04:21:11.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.275+0000 7f6eb2800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eac0cdce0 con 0x7f6eac15c730 2024-03-22T04:21:11.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.275+0000 7f6eb2800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6eac09b910 con 0x7f6eac0b11c0 2024-03-22T04:21:11.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.275+0000 7f6ea3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6eac0b3610 0x7f6eac0b1ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54602/0 (socket says 172.21.15.86:54602) 2024-03-22T04:21:11.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.275+0000 7f6ea3e00640 1 -- 172.21.15.86:0/3965464818 learned_addr learned my addr 172.21.15.86:0/3965464818 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:11.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eabe00640 1 --2- 172.21.15.86:0/3965464818 >> v2:172.21.15.196:3300/0 conn(0x7f6eac0b11c0 0x7f6eac09b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123391237 0 0) 0x7f6eac0cda00 con 0x7f6eac0b3610 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e90003530 con 0x7f6eac0b3610 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346675092 0 0) 0x7f6eac0cdce0 con 0x7f6eac15c730 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6eac0cda00 con 0x7f6eac15c730 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 135921559 0 0) 0x7f6e90003530 con 0x7f6eac0b3610 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 >> v2:172.21.15.196:3300/0 conn(0x7f6eac0b11c0 msgr2=0x7f6eac09b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/3965464818 >> v2:172.21.15.196:3300/0 conn(0x7f6eac0b11c0 0x7f6eac09b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 >> v1:172.21.15.196:6789/0 conn(0x7f6eac15c730 legacy=0x7f6eac09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac0cdfc0 con 0x7f6eac0b3610 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f6e94001680 con 0x7f6eac0b3610 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 >> v1:172.21.15.86:6789/0 conn(0x7f6eac0b3610 legacy=0x7f6eac0b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e90003bd0 0x7f6e90003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e90004810 0x7f6e90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6e9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac09b910 con 0x7f6e90005220 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9000c170 con 0x7f6e90003bd0 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9000c370 con 0x7f6e90004810 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eabe00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6e9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eab400640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e90003bd0 0x7f6e90003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6ea3e00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e90004810 0x7f6e90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eabe00640 1 -- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e90004810 msgr2=0x7f6e90004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eabe00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e90004810 0x7f6e90004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eabe00640 1 -- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e90003bd0 msgr2=0x7f6e90003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eabe00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e90003bd0 0x7f6e90003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6eabe00640 1 -- 172.21.15.86:0/3965464818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e90003530 con 0x7f6e90005220 2024-03-22T04:21:11.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.279+0000 7f6ea3e00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e90004810 0x7f6e90004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eabe00640 1 -- 172.21.15.86:0/3965464818 reap_dead start 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eabe00640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6e9000b9d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009fc0 tx=0x7f6ea401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58304cd1b5470bc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4009c10 con 0x7f6e90005220 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea40a5d10 con 0x7f6e90005220 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eaaa00640 1 -- 172.21.15.86:0/3965464818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4003be0 con 0x7f6e90005220 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 -- 172.21.15.86:0/3965464818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 msgr2=0x7f6e9000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6e9000b9d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009fc0 tx=0x7f6ea401ccc0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 -- 172.21.15.86:0/3965464818 shutdown_connections 2024-03-22T04:21:11.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 --2- 172.21.15.86:0/3965464818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6e9000b9d0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 -- 172.21.15.86:0/3965464818 >> 172.21.15.86:0/3965464818 conn(0x7f6eac0b3bf0 msgr2=0x7f6eac0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:11.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 -- 172.21.15.86:0/3965464818 shutdown_connections 2024-03-22T04:21:11.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 -- 172.21.15.86:0/3965464818 wait complete. 2024-03-22T04:21:11.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 Processor -- start 2024-03-22T04:21:11.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.283+0000 7f6eb2800640 1 -- start start 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6eac1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6eac0c5c30 0x7f6eac15f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eac0c6090 0x7f6eac0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac09aa70 con 0x7f6e90005220 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac0b1710 con 0x7f6eac0c6090 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6eac09abf0 con 0x7f6eac0c5c30 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6eac1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6eac1535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37258/0 (socket says 172.21.15.86:37258) 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 -- 172.21.15.86:0/3031473284 learned_addr learned my addr 172.21.15.86:0/3031473284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6ea3e00640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6eac0c5c30 0x7f6eac15f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eabe00640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eac0c6090 0x7f6eac0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 -- 172.21.15.86:0/3031473284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6eac0c5c30 msgr2=0x7f6eac15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6eac0c5c30 0x7f6eac15f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 -- 172.21.15.86:0/3031473284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eac0c6090 msgr2=0x7f6eac0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eac0c6090 0x7f6eac0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 -- 172.21.15.86:0/3031473284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac153ae0 con 0x7f6e90005220 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eab400640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6eac1535a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c012830 tx=0x7f6e9c012cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6ea3400640 1 -- 172.21.15.86:0/3031473284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c010bd0 con 0x7f6e90005220 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6ea3400640 1 -- 172.21.15.86:0/3031473284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c01b070 con 0x7f6e90005220 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6ea3400640 1 -- 172.21.15.86:0/3031473284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c016480 con 0x7f6e90005220 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac0c9370 con 0x7f6e90005220 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac0c9930 con 0x7f6e90005220 2024-03-22T04:21:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac074470 con 0x7f6e90005220 2024-03-22T04:21:11.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.287+0000 7f6ea3400640 1 -- 172.21.15.86:0/3031473284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6e9c016620 con 0x7f6e90005220 2024-03-22T04:21:11.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.291+0000 7f6ea3400640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e88067b50 0x7f6e8806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.291+0000 7f6ea3400640 1 -- 172.21.15.86:0/3031473284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f6e9c08a190 con 0x7f6e90005220 2024-03-22T04:21:11.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.291+0000 7f6ea3e00640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e88067b50 0x7f6e8806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.291+0000 7f6ea3e00640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e88067b50 0x7f6e8806a010 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6e94001ec0 tx=0x7f6e94007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:11.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.291+0000 7f6ea3400640 1 -- 172.21.15.86:0/3031473284 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6e9c05a3a0 con 0x7f6e90005220 2024-03-22T04:21:11.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.447+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6eac0b1b40 con 0x7f6e90005220 2024-03-22T04:21:11.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.447+0000 7f6ea3400640 1 -- 172.21.15.86:0/3031473284 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) ==== 100+0+11 (secure 0 0 0) 0x7f6e9c05e440 con 0x7f6e90005220 2024-03-22T04:21:11.450 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e88067b50 msgr2=0x7f6e8806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e88067b50 0x7f6e8806a010 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6e94001ec0 tx=0x7f6e94007040 comp rx=0 tx=0).stop 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 msgr2=0x7f6eac1535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6eac1535a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c012830 tx=0x7f6e9c012cf0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 shutdown_connections 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6e88067b50 0x7f6e8806a010 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eac0c6090 0x7f6eac0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6eac0c5c30 0x7f6eac15f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 --2- 172.21.15.86:0/3031473284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e90005220 0x7f6eac1535a0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 >> 172.21.15.86:0/3031473284 conn(0x7f6eac0b3bf0 msgr2=0x7f6eac06f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:11.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 shutdown_connections 2024-03-22T04:21:11.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.451+0000 7f6eb2800640 1 -- 172.21.15.86:0/3031473284 wait complete. 2024-03-22T04:21:11.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:21:11.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.607+0000 7f8a39a00640 1 Processor -- start 2024-03-22T04:21:11.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.607+0000 7f8a39a00640 1 -- start start 2024-03-22T04:21:11.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.607+0000 7f8a39a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8a3415fed0 0x7f8a341602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.607+0000 7f8a39a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a340d11c0 con 0x7f8a340d0de0 2024-03-22T04:21:11.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.607+0000 7f8a39a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a34160820 con 0x7f8a340ce6b0 2024-03-22T04:21:11.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.607+0000 7f8a39a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8a340d0580 con 0x7f8a3415fed0 2024-03-22T04:21:11.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a32a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8a340d0de0 0x7f8a340d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54610/0 (socket says 172.21.15.86:54610) 2024-03-22T04:21:11.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a33400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8a3415fed0 0x7f8a341602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a32a00640 1 -- 172.21.15.86:0/3411752829 learned_addr learned my addr 172.21.15.86:0/3411752829 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:11.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a33400640 1 -- 172.21.15.86:0/3411752829 >> v1:172.21.15.196:6789/0 conn(0x7f8a340ce6b0 legacy=0x7f8a340cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a33400640 1 -- 172.21.15.86:0/3411752829 >> v1:172.21.15.86:6789/0 conn(0x7f8a340d0de0 legacy=0x7f8a340d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a31600640 1 -- 172.21.15.86:0/3411752829 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2282607811 0 0) 0x7f8a340d11c0 con 0x7f8a340d0de0 2024-03-22T04:21:11.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a33400640 1 -- 172.21.15.86:0/3411752829 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a340cece0 con 0x7f8a3415fed0 2024-03-22T04:21:11.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a33400640 1 --2- 172.21.15.86:0/3411752829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415fed0 0x7f8a341602e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009d90 tx=0x7f8a2c01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633da10b393adc38 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:11.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a31600640 1 -- 172.21.15.86:0/3411752829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0a6670 con 0x7f8a3415fed0 2024-03-22T04:21:11.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a31600640 1 -- 172.21.15.86:0/3411752829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2c0a6810 con 0x7f8a3415fed0 2024-03-22T04:21:11.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.611+0000 7f8a31600640 1 -- 172.21.15.86:0/3411752829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0a6be0 con 0x7f8a3415fed0 2024-03-22T04:21:11.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- 172.21.15.86:0/3411752829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415fed0 msgr2=0x7f8a341602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 --2- 172.21.15.86:0/3411752829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415fed0 0x7f8a341602e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009d90 tx=0x7f8a2c01c820 comp rx=0 tx=0).stop 2024-03-22T04:21:11.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- 172.21.15.86:0/3411752829 shutdown_connections 2024-03-22T04:21:11.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 --2- 172.21.15.86:0/3411752829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415fed0 0x7f8a341602e0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- 172.21.15.86:0/3411752829 >> 172.21.15.86:0/3411752829 conn(0x7f8a3406f910 msgr2=0x7f8a3406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:11.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- 172.21.15.86:0/3411752829 shutdown_connections 2024-03-22T04:21:11.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- 172.21.15.86:0/3411752829 wait complete. 2024-03-22T04:21:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 Processor -- start 2024-03-22T04:21:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- start start 2024-03-22T04:21:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a340ce6b0 0x7f8a340c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340d0de0 0x7f8a34155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a3415fed0 0x7f8a34158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34071e30 con 0x7f8a340d0de0 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34071100 con 0x7f8a340ce6b0 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a39a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8a34071fb0 con 0x7f8a3415fed0 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340d0de0 0x7f8a34155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340d0de0 0x7f8a34155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37286/0 (socket says 172.21.15.86:37286) 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32000640 1 -- 172.21.15.86:0/2207369649 learned_addr learned my addr 172.21.15.86:0/2207369649 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a340ce6b0 0x7f8a340c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a33400640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a3415fed0 0x7f8a34158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32000640 1 -- 172.21.15.86:0/2207369649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a3415fed0 msgr2=0x7f8a34158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32000640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a3415fed0 0x7f8a34158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32000640 1 -- 172.21.15.86:0/2207369649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a340ce6b0 msgr2=0x7f8a340c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32000640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a340ce6b0 0x7f8a340c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.615+0000 7f8a32000640 1 -- 172.21.15.86:0/2207369649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3415de80 con 0x7f8a340d0de0 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a32000640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340d0de0 0x7f8a34155110 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c7f0 tx=0x7f8a2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a2aa00640 1 -- 172.21.15.86:0/2207369649 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20017070 con 0x7f8a340d0de0 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a2aa00640 1 -- 172.21.15.86:0/2207369649 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20005480 con 0x7f8a340d0de0 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3415e160 con 0x7f8a340d0de0 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a2aa00640 1 -- 172.21.15.86:0/2207369649 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2000a880 con 0x7f8a340d0de0 2024-03-22T04:21:11.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3415e720 con 0x7f8a340d0de0 2024-03-22T04:21:11.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a34068650 con 0x7f8a340d0de0 2024-03-22T04:21:11.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a2aa00640 1 -- 172.21.15.86:0/2207369649 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8a2001a070 con 0x7f8a340d0de0 2024-03-22T04:21:11.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a2aa00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a14067b20 0x7f8a14069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:11.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a2aa00640 1 -- 172.21.15.86:0/2207369649 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f8a200856f0 con 0x7f8a340d0de0 2024-03-22T04:21:11.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.619+0000 7f8a32a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a14067b20 0x7f8a14069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:11.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.623+0000 7f8a32a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a14067b20 0x7f8a14069fe0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8a24007bc0 tx=0x7f8a24007b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.623+0000 7f8a2aa00640 1 -- 172.21.15.86:0/2207369649 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8a34160820 con 0x7f8a340d0de0 2024-03-22T04:21:11.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.791+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a340d0190 con 0x7f8a340d0de0 2024-03-22T04:21:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.791+0000 7f8a2aa00640 1 -- 172.21.15.86:0/2207369649 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) ==== 80+0+0 (secure 0 0 0) 0x7f8a20055880 con 0x7f8a340d0de0 2024-03-22T04:21:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-03-22T04:21:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a14067b20 msgr2=0x7f8a14069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a14067b20 0x7f8a14069fe0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8a24007bc0 tx=0x7f8a24007b10 comp rx=0 tx=0).stop 2024-03-22T04:21:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340d0de0 msgr2=0x7f8a34155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340d0de0 0x7f8a34155110 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c7f0 tx=0x7f8a2000ccb0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 shutdown_connections 2024-03-22T04:21:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a14067b20 0x7f8a14069fe0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a3415fed0 0x7f8a34158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340d0de0 0x7f8a34155110 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 --2- 172.21.15.86:0/2207369649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a340ce6b0 0x7f8a340c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 >> 172.21.15.86:0/2207369649 conn(0x7f8a3406f910 msgr2=0x7f8a3415c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 shutdown_connections 2024-03-22T04:21:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:11.795+0000 7f8a39a00640 1 -- 172.21.15.86:0/2207369649 wait complete. 2024-03-22T04:21:11.814 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:21:11.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:11.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:11.947 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:21:11.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:11.948 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:21:11.975 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:11.975 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:12.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:21:12.026 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:21:12.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:12.026 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:21:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-22T04:21:12.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:21:12.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:21:12.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f102bc00640 1 Processor -- start 2024-03-22T04:21:12.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f102bc00640 1 -- start start 2024-03-22T04:21:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f102bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10240d0de0 0x7f10240ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f102bc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f10240d1dc0 0x7f102415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f102bc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10240d11c0 con 0x7f10240d19a0 2024-03-22T04:21:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f102bc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f10240d0580 con 0x7f10240d1dc0 2024-03-22T04:21:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f102bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f10240cece0 con 0x7f10240d0de0 2024-03-22T04:21:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f10240d19a0 0x7f10240cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54626/0 (socket says 172.21.15.86:54626) 2024-03-22T04:21:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023e00640 1 -- 172.21.15.86:0/1880885765 learned_addr learned my addr 172.21.15.86:0/1880885765 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1029200640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10240d0de0 0x7f10240ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1029c00640 1 --2- 172.21.15.86:0/1880885765 >> v2:172.21.15.196:3300/0 conn(0x7f10240d1dc0 0x7f102415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 798114151 0 0) 0x7f10240d11c0 con 0x7f10240d19a0 2024-03-22T04:21:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f100c003530 con 0x7f10240d19a0 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3286551576 0 0) 0x7f100c003530 con 0x7f10240d19a0 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10240d0de0 msgr2=0x7f10240ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023400640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10240d0de0 0x7f10240ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 >> v2:172.21.15.196:3300/0 conn(0x7f10240d1dc0 msgr2=0x7f102415fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.259+0000 7f1023400640 1 --2- 172.21.15.86:0/1880885765 >> v2:172.21.15.196:3300/0 conn(0x7f10240d1dc0 0x7f102415fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10240cefc0 con 0x7f10240d19a0 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f1010001680 con 0x7f10240d19a0 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 >> v1:172.21.15.86:6789/0 conn(0x7f10240d19a0 legacy=0x7f10240cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f100c0039c0 0x7f100c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10240d0de0 0x7f100c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c004d50 0x7f100c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10240d0580 con 0x7f100c004d50 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10240cece0 con 0x7f100c0039c0 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f100c00b9d0 con 0x7f10240d0de0 2024-03-22T04:21:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029c00640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c004d50 0x7f100c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029200640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f100c0039c0 0x7f100c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1023e00640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10240d0de0 0x7f100c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029200640 1 -- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10240d0de0 msgr2=0x7f100c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029200640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10240d0de0 0x7f100c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029200640 1 -- 172.21.15.86:0/1880885765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c004d50 msgr2=0x7f100c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029200640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c004d50 0x7f100c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029200640 1 -- 172.21.15.86:0/1880885765 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c003530 con 0x7f100c0039c0 2024-03-22T04:21:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029c00640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c004d50 0x7f100c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:12.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.263+0000 7f1029200640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f100c0039c0 0x7f100c003de0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1018009ae0 tx=0x7f101801ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d434a6a44e190 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:12.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10180b2070 con 0x7f100c0039c0 2024-03-22T04:21:12.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10180a5660 con 0x7f100c0039c0 2024-03-22T04:21:12.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f1023400640 1 -- 172.21.15.86:0/1880885765 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10180096b0 con 0x7f100c0039c0 2024-03-22T04:21:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 -- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f100c0039c0 msgr2=0x7f100c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 --2- 172.21.15.86:0/1880885765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f100c0039c0 0x7f100c003de0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1018009ae0 tx=0x7f101801ca70 comp rx=0 tx=0).stop 2024-03-22T04:21:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f1029c00640 1 -- 172.21.15.86:0/1880885765 reap_dead start 2024-03-22T04:21:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 -- 172.21.15.86:0/1880885765 shutdown_connections 2024-03-22T04:21:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 -- 172.21.15.86:0/1880885765 >> 172.21.15.86:0/1880885765 conn(0x7f102406f910 msgr2=0x7f102406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 -- 172.21.15.86:0/1880885765 shutdown_connections 2024-03-22T04:21:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 -- 172.21.15.86:0/1880885765 wait complete. 2024-03-22T04:21:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 Processor -- start 2024-03-22T04:21:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 -- start start 2024-03-22T04:21:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c0039c0 0x7f10240b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.267+0000 7f102bc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024159810 0x7f10241f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f102bc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10241f7a90 0x7f10240b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f102bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1024071100 con 0x7f100c0039c0 2024-03-22T04:21:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f102bc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10240cf580 con 0x7f1024159810 2024-03-22T04:21:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f102bc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f10240ce6b0 con 0x7f10241f7a90 2024-03-22T04:21:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c0039c0 0x7f10240b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c0039c0 0x7f10240b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37294/0 (socket says 172.21.15.86:37294) 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 -- 172.21.15.86:0/350508013 learned_addr learned my addr 172.21.15.86:0/350508013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1023e00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024159810 0x7f10241f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029c00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10241f7a90 0x7f10240b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 -- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10241f7a90 msgr2=0x7f10240b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10241f7a90 0x7f10240b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 -- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024159810 msgr2=0x7f10241f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024159810 0x7f10241f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 -- 172.21.15.86:0/350508013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10240b4db0 con 0x7f100c0039c0 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1023e00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024159810 0x7f10241f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:12.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1029200640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c0039c0 0x7f10240b5850 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f10180b0040 tx=0x7f100c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1020c00640 1 -- 172.21.15.86:0/350508013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10180bc030 con 0x7f100c0039c0 2024-03-22T04:21:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1020c00640 1 -- 172.21.15.86:0/350508013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101800f000 con 0x7f100c0039c0 2024-03-22T04:21:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1020c00640 1 -- 172.21.15.86:0/350508013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10180b2040 con 0x7f100c0039c0 2024-03-22T04:21:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10241f8010 con 0x7f100c0039c0 2024-03-22T04:21:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10240b5050 con 0x7f100c0039c0 2024-03-22T04:21:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1024068650 con 0x7f100c0039c0 2024-03-22T04:21:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.271+0000 7f1020c00640 1 -- 172.21.15.86:0/350508013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f10180bf070 con 0x7f100c0039c0 2024-03-22T04:21:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.275+0000 7f1020c00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1004067b20 0x7f1004069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.275+0000 7f1020c00640 1 -- 172.21.15.86:0/350508013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f10181201f0 con 0x7f100c0039c0 2024-03-22T04:21:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.275+0000 7f1023e00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1004067b20 0x7f1004069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.275+0000 7f1023e00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1004067b20 0x7f1004069fe0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f10240b09f0 tx=0x7f1010009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.283+0000 7f1020c00640 1 -- 172.21.15.86:0/350508013 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f10180c7020 con 0x7f100c0039c0 2024-03-22T04:21:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.559+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f102415f750 con 0x7f100c0039c0 2024-03-22T04:21:12.559 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:21:12.559+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:21:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.559+0000 7f1020c00640 1 -- 172.21.15.86:0/350508013 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) ==== 161+0+0 (secure 0 0 0) 0x7f1018016780 con 0x7f100c0039c0 2024-03-22T04:21:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:21:12.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1004067b20 msgr2=0x7f1004069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1004067b20 0x7f1004069fe0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f10240b09f0 tx=0x7f1010009040 comp rx=0 tx=0).stop 2024-03-22T04:21:12.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c0039c0 msgr2=0x7f10240b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c0039c0 0x7f10240b5850 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f10180b0040 tx=0x7f100c005840 comp rx=0 tx=0).stop 2024-03-22T04:21:12.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 shutdown_connections 2024-03-22T04:21:12.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1004067b20 0x7f1004069fe0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024159810 0x7f10241f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10241f7a90 0x7f10240b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 --2- 172.21.15.86:0/350508013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f100c0039c0 0x7f10240b5850 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 >> 172.21.15.86:0/350508013 conn(0x7f102406f910 msgr2=0x7f102415bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:12.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 shutdown_connections 2024-03-22T04:21:12.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.563+0000 7f102bc00640 1 -- 172.21.15.86:0/350508013 wait complete. 2024-03-22T04:21:12.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:21:12.588 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:21:12.589 INFO:tasks.ceph.mds.a.smithi086.stderr: -8427> 2024-03-22T04:21:03.167+0000 7fc300600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-03-22T04:21:12.629 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-03-22T04:21:12.630 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.631 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-03-22T04:21:12.632 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.633 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.634 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:12.635 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:21:12.636 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-03-22T04:21:12.637 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-03-22T04:21:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = False 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:21:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-03-22T04:21:12.640 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-03-22T04:21:12.641 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.642 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on skipped message = False 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-03-22T04:21:12.643 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-03-22T04:21:12.644 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-03-22T04:21:12.645 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-03-22T04:21:12.646 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-03-22T04:21:12.647 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-03-22T04:21:12.648 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: mon down mkfs grace = 300 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:21:12.649 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.650 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-03-22T04:21:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-03-22T04:21:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-03-22T04:21:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = False 2024-03-22T04:21:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:21:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:21:12.651 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:21:12.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:21:12.654 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:21:12.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-03-22T04:21:12.697 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-03-22T04:21:12.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-03-22T04:21:12.699 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-03-22T04:21:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-03-22T04:21:12.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-03-22T04:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-03-22T04:21:12.703 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:21:12.704 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.705 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.706 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:12.707 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-03-22T04:21:12.708 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = False 2024-03-22T04:21:12.708 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-03-22T04:21:12.708 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = False 2024-03-22T04:21:12.708 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:21:12.708 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:21:12.708 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:21:12.735 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-03-22T04:21:12.735 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-03-22T04:21:12.735 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:21:12.736 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:21:12.737 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on skipped message = False 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:21:12.738 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-03-22T04:21:12.739 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-03-22T04:21:12.740 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-03-22T04:21:12.741 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-03-22T04:21:12.742 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-03-22T04:21:12.743 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: mon down mkfs grace = 300 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:21:12.744 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-03-22T04:21:12.745 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = False 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = False 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:21:12.746 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:21:12.750 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-03-22T04:21:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 Processor -- start 2024-03-22T04:21:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- start start 2024-03-22T04:21:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d79ba000 0x5654d7a8bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 0x5654d7a8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d7a92090 0x5654d7a93960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654d79edf60 con 0x5654d7a92090 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654d7960620 con 0x5654d7a91770 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5654d79ee0e0 con 0x5654d79ba000 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d7a92090 0x5654d7a93960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdf200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 0x5654d7a8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdfc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d79ba000 0x5654d7a8bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d7a92090 0x5654d7a93960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37328/0 (socket says 172.21.15.86:37328) 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be0600640 1 -- 172.21.15.86:0/3677649860 learned_addr learned my addr 172.21.15.86:0/3677649860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdf200640 1 -- 172.21.15.86:0/3677649860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d79ba000 msgr2=0x5654d7a8bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdf200640 1 --2- 172.21.15.86:0/3677649860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d79ba000 0x5654d7a8bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdf200640 1 -- 172.21.15.86:0/3677649860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d7a92090 msgr2=0x5654d7a93960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdf200640 1 --2- 172.21.15.86:0/3677649860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d7a92090 0x5654d7a93960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdf200640 1 -- 172.21.15.86:0/3677649860 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d7a92d60 con 0x5654d7a91770 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be0600640 1 --2- 172.21.15.86:0/3677649860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d7a92090 0x5654d7a93960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdf200640 1 --2- 172.21.15.86:0/3677649860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 0x5654d7a8c000 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x5654d79c2400 tx=0x7f5bcc003030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98bc1de869b106eb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdde00640 1 -- 172.21.15.86:0/3677649860 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0b1040 con 0x5654d7a91770 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdde00640 1 -- 172.21.15.86:0/3677649860 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0bb070 con 0x5654d7a91770 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5bdde00640 1 -- 172.21.15.86:0/3677649860 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0acae0 con 0x5654d7a91770 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- 172.21.15.86:0/3677649860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 msgr2=0x5654d7a8c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 --2- 172.21.15.86:0/3677649860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 0x5654d7a8c000 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x5654d79c2400 tx=0x7f5bcc003030 comp rx=0 tx=0).stop 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- 172.21.15.86:0/3677649860 shutdown_connections 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 --2- 172.21.15.86:0/3677649860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 0x5654d7a8c000 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 --2- 172.21.15.86:0/3677649860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d7a92090 0x5654d7a93960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 --2- 172.21.15.86:0/3677649860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d79ba000 0x5654d7a8bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- 172.21.15.86:0/3677649860 >> 172.21.15.86:0/3677649860 conn(0x5654d79ba8d0 msgr2=0x5654d7a924d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- 172.21.15.86:0/3677649860 shutdown_connections 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.799+0000 7f5be2968080 1 -- 172.21.15.86:0/3677649860 wait complete. 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 Processor -- start 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 -- start start 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d79ba000 0x5654d7a8e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 0x5654d7a97d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d7a92090 0x5654d7a886c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654d7a8c540 con 0x5654d79ba000 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654d7b231e0 con 0x5654d7a91770 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5654d7a93000 con 0x5654d7a92090 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d79ba000 0x5654d7a8e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d79ba000 0x5654d7a8e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37338/0 (socket says 172.21.15.86:37338) 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 -- 172.21.15.86:0/1781908104 learned_addr learned my addr 172.21.15.86:0/1781908104 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be0600640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d7a92090 0x5654d7a886c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdf200640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 0x5654d7a97d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 -- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d7a92090 msgr2=0x5654d7a886c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5654d7a92090 0x5654d7a886c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 -- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 msgr2=0x5654d7a97d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654d7a91770 0x5654d7a97d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 -- 172.21.15.86:0/1781908104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d7a8e9c0 con 0x5654d79ba000 2024-03-22T04:21:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdfc00640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d79ba000 0x5654d7a8e120 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0008720 tx=0x7f5bd0008be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdca00640 1 -- 172.21.15.86:0/1781908104 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0019070 con 0x5654d79ba000 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdca00640 1 -- 172.21.15.86:0/1781908104 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd0010ba0 con 0x5654d79ba000 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5bdca00640 1 -- 172.21.15.86:0/1781908104 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0008e30 con 0x5654d79ba000 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5654d7a8ec60 con 0x5654d79ba000 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.803+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d7a8f1a0 con 0x5654d79ba000 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5bdca00640 1 -- 172.21.15.86:0/1781908104 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5bd001c070 con 0x5654d79ba000 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5bdca00640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5bc0066530 0x7f5bc00689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5bdca00640 1 -- 172.21.15.86:0/1781908104 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f5bd0085840 con 0x5654d79ba000 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5bdf200640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5bc0066530 0x7f5bc00689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be2968080 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5654d7a98240 0x5654d7a96450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5654d7a97680 con 0x5654d7a98240 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be0600640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5654d7a98240 0x5654d7a96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5bdf200640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5bc0066530 0x7f5bc00689f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0ba000 tx=0x7f5bcc0ae040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be0600640 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5654d7a98240 0x5654d7a96450 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be0600640 1 -- 172.21.15.86:0/1781908104 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v125'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5bd8002810 con 0x5654d7a98240 2024-03-22T04:21:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5654d7a98240 msgr2=0x5654d7a96450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be2968080 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5654d7a98240 0x5654d7a96450 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5bc0066530 msgr2=0x7f5bc00689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be2968080 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5bc0066530 0x7f5bc00689f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0ba000 tx=0x7f5bcc0ae040 comp rx=0 tx=0).stop 2024-03-22T04:21:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d79ba000 msgr2=0x5654d7a8e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be2968080 1 --2- 172.21.15.86:0/1781908104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654d79ba000 0x5654d7a8e120 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0008720 tx=0x7f5bd0008be0 comp rx=0 tx=0).stop 2024-03-22T04:21:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.807+0000 7f5be0600640 1 -- 172.21.15.86:0/1781908104 reap_dead start 2024-03-22T04:21:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.811+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 shutdown_connections 2024-03-22T04:21:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.811+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 >> 172.21.15.86:0/1781908104 conn(0x5654d79ba8d0 msgr2=0x5654d7a93960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.811+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 shutdown_connections 2024-03-22T04:21:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.811+0000 7f5be2968080 1 -- 172.21.15.86:0/1781908104 wait complete. 2024-03-22T04:21:12.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-22T04:21:12.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26e2600640 1 Processor -- start 2024-03-22T04:21:12.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26e2600640 1 -- start start 2024-03-22T04:21:13.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26e2600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f26dc0b2370 0x7f26dc0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26e2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26dc15b3c0 0x7f26dc0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26e2600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26dc15f180 con 0x7f26dc15afe0 2024-03-22T04:21:13.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26e2600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f26dc0b3f20 con 0x7f26dc15b3c0 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26e2600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f26dc15f460 con 0x7f26dc0b2370 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26dc15b3c0 0x7f26dc0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26db400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f26dc15afe0 0x7f26dc0b1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54632/0 (socket says 172.21.15.86:54632) 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26db400640 1 -- 172.21.15.86:0/978024639 learned_addr learned my addr 172.21.15.86:0/978024639 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26daa00640 1 --2- 172.21.15.86:0/978024639 >> v2:172.21.15.196:3300/0 conn(0x7f26dc0b2370 0x7f26dc0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26da000640 1 -- 172.21.15.86:0/978024639 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2484480554 0 0) 0x7f26dc15f180 con 0x7f26dc15afe0 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26da000640 1 -- 172.21.15.86:0/978024639 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26c0003530 con 0x7f26dc15afe0 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26dbe00640 1 -- 172.21.15.86:0/978024639 >> v2:172.21.15.196:3300/0 conn(0x7f26dc0b2370 msgr2=0x7f26dc0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26dbe00640 1 --2- 172.21.15.86:0/978024639 >> v2:172.21.15.196:3300/0 conn(0x7f26dc0b2370 0x7f26dc0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26dbe00640 1 -- 172.21.15.86:0/978024639 >> v1:172.21.15.86:6789/0 conn(0x7f26dc15afe0 legacy=0x7f26dc0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:12.999+0000 7f26dbe00640 1 -- 172.21.15.86:0/978024639 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc15f740 con 0x7f26dc15b3c0 2024-03-22T04:21:13.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26dbe00640 1 --2- 172.21.15.86:0/978024639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc0b39e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0cd620 tx=0x7f26d401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64708aac1f7fc8fd server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:13.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26da000640 1 -- 172.21.15.86:0/978024639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d40b2070 con 0x7f26dc15b3c0 2024-03-22T04:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26da000640 1 -- 172.21.15.86:0/978024639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d40a59e0 con 0x7f26dc15b3c0 2024-03-22T04:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26da000640 1 -- 172.21.15.86:0/978024639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d40a5db0 con 0x7f26dc15b3c0 2024-03-22T04:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 -- 172.21.15.86:0/978024639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 msgr2=0x7f26dc0b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 --2- 172.21.15.86:0/978024639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc0b39e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0cd620 tx=0x7f26d401d360 comp rx=0 tx=0).stop 2024-03-22T04:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 -- 172.21.15.86:0/978024639 shutdown_connections 2024-03-22T04:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 --2- 172.21.15.86:0/978024639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc0b39e0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 --2- 172.21.15.86:0/978024639 >> v2:172.21.15.196:3300/0 conn(0x7f26dc0b2370 0x7f26dc0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 -- 172.21.15.86:0/978024639 >> 172.21.15.86:0/978024639 conn(0x7f26dc06b660 msgr2=0x7f26dc06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 -- 172.21.15.86:0/978024639 shutdown_connections 2024-03-22T04:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 -- 172.21.15.86:0/978024639 wait complete. 2024-03-22T04:21:13.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 Processor -- start 2024-03-22T04:21:13.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.003+0000 7f26e2600640 1 -- start start 2024-03-22T04:21:13.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26dc0b2370 0x7f26dc152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26dc15afe0 0x7f26dc164f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc1654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc0cfaa0 con 0x7f26dc15b3c0 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc0cf920 con 0x7f26dc15afe0 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f26dc071ff0 con 0x7f26dc0b2370 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc1654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc1654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37354/0 (socket says 172.21.15.86:37354) 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 -- 172.21.15.86:0/2394637564 learned_addr learned my addr 172.21.15.86:0/2394637564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26db400640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26dc0b2370 0x7f26dc152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26daa00640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26dc15afe0 0x7f26dc164f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 -- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26dc0b2370 msgr2=0x7f26dc152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26dc0b2370 0x7f26dc152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 -- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26dc15afe0 msgr2=0x7f26dc164f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26dc15afe0 0x7f26dc164f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 -- 172.21.15.86:0/2394637564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc15a620 con 0x7f26dc15b3c0 2024-03-22T04:21:13.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26db400640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26dc0b2370 0x7f26dc152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:13.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26daa00640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26dc15afe0 0x7f26dc164f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:13.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26dbe00640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc1654c0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f26d40b0040 tx=0x7f26d40196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:13.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26d3400640 1 -- 172.21.15.86:0/2394637564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d40bc030 con 0x7f26dc15b3c0 2024-03-22T04:21:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26d3400640 1 -- 172.21.15.86:0/2394637564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d40ad440 con 0x7f26dc15b3c0 2024-03-22T04:21:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26d3400640 1 -- 172.21.15.86:0/2394637564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d40b2040 con 0x7f26dc15b3c0 2024-03-22T04:21:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc159aa0 con 0x7f26dc15b3c0 2024-03-22T04:21:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc15a060 con 0x7f26dc15b3c0 2024-03-22T04:21:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.007+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26dc075220 con 0x7f26dc15b3c0 2024-03-22T04:21:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.011+0000 7f26d3400640 1 -- 172.21.15.86:0/2394637564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f26d40bf070 con 0x7f26dc15b3c0 2024-03-22T04:21:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.011+0000 7f26d3400640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f26bc067b20 0x7f26bc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.011+0000 7f26d3400640 1 -- 172.21.15.86:0/2394637564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f26d41204c0 con 0x7f26dc15b3c0 2024-03-22T04:21:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.011+0000 7f26db400640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f26bc067b20 0x7f26bc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.011+0000 7f26db400640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f26bc067b20 0x7f26bc069fe0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005890 tx=0x7f26cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:13.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.015+0000 7f26d3400640 1 -- 172.21.15.86:0/2394637564 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f26d40c7020 con 0x7f26dc15b3c0 2024-03-22T04:21:13.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.203+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f26dc072ef0 con 0x7f26dc15b3c0 2024-03-22T04:21:13.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.563+0000 7f26d3400640 1 -- 172.21.15.86:0/2394637564 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) ==== 162+0+0 (secure 0 0 0) 0x7f26d4016520 con 0x7f26dc15b3c0 2024-03-22T04:21:13.563 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f26bc067b20 msgr2=0x7f26bc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f26bc067b20 0x7f26bc069fe0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005890 tx=0x7f26cc009040 comp rx=0 tx=0).stop 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 msgr2=0x7f26dc1654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc1654c0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f26d40b0040 tx=0x7f26d40196a0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 shutdown_connections 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f26bc067b20 0x7f26bc069fe0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26dc15afe0 0x7f26dc164f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26dc15b3c0 0x7f26dc1654c0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 --2- 172.21.15.86:0/2394637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26dc0b2370 0x7f26dc152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 >> 172.21.15.86:0/2394637564 conn(0x7f26dc06b660 msgr2=0x7f26dc0b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 shutdown_connections 2024-03-22T04:21:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.567+0000 7f26e2600640 1 -- 172.21.15.86:0/2394637564 wait complete. 2024-03-22T04:21:13.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbdc600640 1 Processor -- start 2024-03-22T04:21:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbdc600640 1 -- start start 2024-03-22T04:21:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbdc600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbd4165e30 0x7fcbd40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbdc600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cff30 0x7fcbd40d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbdc600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd4071650 con 0x7fcbd40cdea0 2024-03-22T04:21:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbdc600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd40d08d0 con 0x7fcbd4165e30 2024-03-22T04:21:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbdc600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd40cea40 con 0x7fcbd40cff30 2024-03-22T04:21:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbd9c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbd4165e30 0x7fcbd40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd9c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbd4165e30 0x7fcbd40723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:46144/0 (socket says 172.21.15.86:46144) 2024-03-22T04:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd9c00640 1 -- 172.21.15.86:0/829446426 learned_addr learned my addr 172.21.15.86:0/829446426 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.727+0000 7fcbd9200640 1 --2- 172.21.15.86:0/829446426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cff30 0x7fcbd40d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbda600640 1 --1- 172.21.15.86:0/829446426 >> v1:172.21.15.196:6789/0 conn(0x7fcbd40cdea0 0x7fcbd40ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:42914/0 (socket says 172.21.15.86:42914) 2024-03-22T04:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd9c00640 1 -- 172.21.15.86:0/829446426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cff30 msgr2=0x7fcbd40d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd9c00640 1 --2- 172.21.15.86:0/829446426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cff30 0x7fcbd40d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd9c00640 1 -- 172.21.15.86:0/829446426 >> v1:172.21.15.196:6789/0 conn(0x7fcbd40cdea0 legacy=0x7fcbd40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd9c00640 1 -- 172.21.15.86:0/829446426 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd40718c0 con 0x7fcbd4165e30 2024-03-22T04:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd3e00640 1 -- 172.21.15.86:0/829446426 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029293002 0 0) 0x7fcbd4071650 con 0x7fcbd40cdea0 2024-03-22T04:21:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd9c00640 1 --2- 172.21.15.86:0/829446426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 0x7fcbd40723a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009870 tx=0x7fcbc801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10906835af11d29b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd3e00640 1 -- 172.21.15.86:0/829446426 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80a95f0 con 0x7fcbd4165e30 2024-03-22T04:21:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd3e00640 1 -- 172.21.15.86:0/829446426 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc80a9790 con 0x7fcbd4165e30 2024-03-22T04:21:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.731+0000 7fcbd3e00640 1 -- 172.21.15.86:0/829446426 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80a9b60 con 0x7fcbd4165e30 2024-03-22T04:21:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- 172.21.15.86:0/829446426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 msgr2=0x7fcbd40723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 --2- 172.21.15.86:0/829446426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 0x7fcbd40723a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009870 tx=0x7fcbc801c530 comp rx=0 tx=0).stop 2024-03-22T04:21:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- 172.21.15.86:0/829446426 shutdown_connections 2024-03-22T04:21:13.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 --2- 172.21.15.86:0/829446426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cff30 0x7fcbd40d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 --2- 172.21.15.86:0/829446426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 0x7fcbd40723a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- 172.21.15.86:0/829446426 >> 172.21.15.86:0/829446426 conn(0x7fcbd406b660 msgr2=0x7fcbd406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:13.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- 172.21.15.86:0/829446426 shutdown_connections 2024-03-22T04:21:13.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- 172.21.15.86:0/829446426 wait complete. 2024-03-22T04:21:13.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 Processor -- start 2024-03-22T04:21:13.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- start start 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cdea0 0x7fcbd415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd40cff30 0x7fcbd4156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 0x7fcbd414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd4166210 con 0x7fcbd40cff30 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40b5220 con 0x7fcbd4165e30 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbdc600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd4166390 con 0x7fcbd40cdea0 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd40cff30 0x7fcbd4156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd40cff30 0x7fcbd4156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37360/0 (socket says 172.21.15.86:37360) 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9200640 1 -- 172.21.15.86:0/966449557 learned_addr learned my addr 172.21.15.86:0/966449557 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9c00640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cdea0 0x7fcbd415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbda600640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 0x7fcbd414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9200640 1 -- 172.21.15.86:0/966449557 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cdea0 msgr2=0x7fcbd415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9200640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cdea0 0x7fcbd415f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9200640 1 -- 172.21.15.86:0/966449557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 msgr2=0x7fcbd414de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9200640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 0x7fcbd414de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.735+0000 7fcbd9200640 1 -- 172.21.15.86:0/966449557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd415aad0 con 0x7fcbd40cff30 2024-03-22T04:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd9200640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd40cff30 0x7fcbd4156960 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000e9e0 tx=0x7fcbc000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd1600640 1 -- 172.21.15.86:0/966449557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0017070 con 0x7fcbd40cff30 2024-03-22T04:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd1600640 1 -- 172.21.15.86:0/966449557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc000c9d0 con 0x7fcbd40cff30 2024-03-22T04:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd1600640 1 -- 172.21.15.86:0/966449557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000cd80 con 0x7fcbd40cff30 2024-03-22T04:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd414eb80 con 0x7fcbd40cff30 2024-03-22T04:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd414ee20 con 0x7fcbd40cff30 2024-03-22T04:21:13.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd4075220 con 0x7fcbd40cff30 2024-03-22T04:21:13.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd1600640 1 -- 172.21.15.86:0/966449557 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcbc001a070 con 0x7fcbd40cff30 2024-03-22T04:21:13.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd1600640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcbb4067b20 0x7fcbb4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:13.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd1600640 1 -- 172.21.15.86:0/966449557 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7fcbc00864a0 con 0x7fcbd40cff30 2024-03-22T04:21:13.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd9c00640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcbb4067b20 0x7fcbb4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:13.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.739+0000 7fcbd9c00640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcbb4067b20 0x7fcbb4069fe0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80ae000 tx=0x7fcbc80b4040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.743+0000 7fcbd1600640 1 -- 172.21.15.86:0/966449557 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcbc0056670 con 0x7fcbd40cff30 2024-03-22T04:21:13.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbd415f220 con 0x7fcbd40cff30 2024-03-22T04:21:13.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbd1600640 1 -- 172.21.15.86:0/966449557 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5190 (secure 0 0 0) 0x7fcbc005a710 con 0x7fcbd40cff30 2024-03-22T04:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:13.570354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77801":{"gid":77801,"name":"a","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.86:6837/4220897872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4220897872},{"type":"v1","addr":"172.21.15.86:6837","nonce":4220897872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-22T04:21:13.936 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 102 2024-03-22T04:21:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcbb4067b20 msgr2=0x7fcbb4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcbb4067b20 0x7fcbb4069fe0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80ae000 tx=0x7fcbc80b4040 comp rx=0 tx=0).stop 2024-03-22T04:21:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd40cff30 msgr2=0x7fcbd4156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd40cff30 0x7fcbd4156960 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000e9e0 tx=0x7fcbc000eea0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 shutdown_connections 2024-03-22T04:21:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcbb4067b20 0x7fcbb4069fe0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd4165e30 0x7fcbd414de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd40cff30 0x7fcbd4156960 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 --2- 172.21.15.86:0/966449557 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd40cdea0 0x7fcbd415f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.935+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 >> 172.21.15.86:0/966449557 conn(0x7fcbd406b660 msgr2=0x7fcbd40af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.939+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 shutdown_connections 2024-03-22T04:21:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:13.939+0000 7fcbdc600640 1 -- 172.21.15.86:0/966449557 wait complete. 2024-03-22T04:21:14.359 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:20:57.561818+0000', 'modified': '2024-03-22T04:21:13.570354+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 77801}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_77801': {'gid': 77801, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/4220897872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 4220897872}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 4220897872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:21:14.359 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_77801:up:replay 2024-03-22T04:21:15.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea91800640 1 Processor -- start 2024-03-22T04:21:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea91800640 1 -- start start 2024-03-22T04:21:15.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea91800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fea8c15c350 0x7fea8c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:15.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea91800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8c0d0b30 con 0x7fea8c15c730 2024-03-22T04:21:15.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea91800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8c0d0dd0 con 0x7fea8c0b63f0 2024-03-22T04:21:15.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea91800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fea8c0d10b0 con 0x7fea8c15c350 2024-03-22T04:21:15.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fea8c15c730 0x7fea8c0d1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54636/0 (socket says 172.21.15.86:54636) 2024-03-22T04:21:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8b400640 1 -- 172.21.15.86:0/1122245904 learned_addr learned my addr 172.21.15.86:0/1122245904 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8aa00640 1 --2- 172.21.15.86:0/1122245904 >> v2:172.21.15.196:3300/0 conn(0x7fea8c15c350 0x7fea8c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8a000640 1 -- 172.21.15.86:0/1122245904 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177055226 0 0) 0x7fea8c0d0b30 con 0x7fea8c15c730 2024-03-22T04:21:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8a000640 1 -- 172.21.15.86:0/1122245904 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea70003530 con 0x7fea8c15c730 2024-03-22T04:21:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8aa00640 1 -- 172.21.15.86:0/1122245904 >> v1:172.21.15.196:6789/0 conn(0x7fea8c0b63f0 legacy=0x7fea8c070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8aa00640 1 -- 172.21.15.86:0/1122245904 >> v1:172.21.15.86:6789/0 conn(0x7fea8c15c730 legacy=0x7fea8c0d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8a000640 1 -- 172.21.15.86:0/1122245904 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4003243420 0 0) 0x7fea70003530 con 0x7fea8c15c730 2024-03-22T04:21:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.506+0000 7fea8aa00640 1 -- 172.21.15.86:0/1122245904 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c0d1390 con 0x7fea8c15c350 2024-03-22T04:21:15.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea8aa00640 1 --2- 172.21.15.86:0/1122245904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 0x7fea8c0b5eb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0097c0 tx=0x7fea7c01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aae734c7b6dd6a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:15.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea8a000640 1 -- 172.21.15.86:0/1122245904 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0b0070 con 0x7fea8c15c350 2024-03-22T04:21:15.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea8a000640 1 -- 172.21.15.86:0/1122245904 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7c0ab400 con 0x7fea8c15c350 2024-03-22T04:21:15.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea8a000640 1 -- 172.21.15.86:0/1122245904 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84001a00 con 0x7fea8c15c350 2024-03-22T04:21:15.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea91800640 1 -- 172.21.15.86:0/1122245904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 msgr2=0x7fea8c0b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:15.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea91800640 1 --2- 172.21.15.86:0/1122245904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 0x7fea8c0b5eb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0097c0 tx=0x7fea7c01c390 comp rx=0 tx=0).stop 2024-03-22T04:21:15.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea91800640 1 -- 172.21.15.86:0/1122245904 shutdown_connections 2024-03-22T04:21:15.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea91800640 1 --2- 172.21.15.86:0/1122245904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 0x7fea8c0b5eb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:15.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea91800640 1 -- 172.21.15.86:0/1122245904 >> 172.21.15.86:0/1122245904 conn(0x7fea8c0b3cf0 msgr2=0x7fea8c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:15.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea91800640 1 -- 172.21.15.86:0/1122245904 shutdown_connections 2024-03-22T04:21:15.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.510+0000 7fea91800640 1 -- 172.21.15.86:0/1122245904 wait complete. 2024-03-22T04:21:15.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 Processor -- start 2024-03-22T04:21:15.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 -- start start 2024-03-22T04:21:15.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea8c0b63f0 0x7fea8c15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:15.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 0x7fea8c1476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:15.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea8c15c730 0x7fea8c145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c0d2ab0 con 0x7fea8c0b63f0 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c0718c0 con 0x7fea8c15c350 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fea8c16a0d0 con 0x7fea8c15c730 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea8c0b63f0 0x7fea8c15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea8c0b63f0 0x7fea8c15cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37364/0 (socket says 172.21.15.86:37364) 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 -- 172.21.15.86:0/2830218330 learned_addr learned my addr 172.21.15.86:0/2830218330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8b400640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea8c15c730 0x7fea8c145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea82000640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 0x7fea8c1476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 -- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea8c15c730 msgr2=0x7fea8c145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea8c15c730 0x7fea8c145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 -- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 msgr2=0x7fea8c1476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 0x7fea8c1476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 -- 172.21.15.86:0/2830218330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c15d120 con 0x7fea8c0b63f0 2024-03-22T04:21:15.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea82000640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 0x7fea8c1476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:15.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea8aa00640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea8c0b63f0 0x7fea8c15cb40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0ae040 tx=0x7fea7c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:15.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea83400640 1 -- 172.21.15.86:0/2830218330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0b0050 con 0x7fea8c0b63f0 2024-03-22T04:21:15.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea83400640 1 -- 172.21.15.86:0/2830218330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7c0bc070 con 0x7fea8c0b63f0 2024-03-22T04:21:15.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea83400640 1 -- 172.21.15.86:0/2830218330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0b78b0 con 0x7fea8c0b63f0 2024-03-22T04:21:15.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8c15d400 con 0x7fea8c0b63f0 2024-03-22T04:21:15.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.514+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8c14ae90 con 0x7fea8c0b63f0 2024-03-22T04:21:15.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.518+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8c075220 con 0x7fea8c0b63f0 2024-03-22T04:21:15.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.518+0000 7fea83400640 1 -- 172.21.15.86:0/2830218330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fea7c0b7a50 con 0x7fea8c0b63f0 2024-03-22T04:21:15.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.518+0000 7fea83400640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea68067b50 0x7fea6806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:15.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.518+0000 7fea83400640 1 -- 172.21.15.86:0/2830218330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7fea7c120810 con 0x7fea8c0b63f0 2024-03-22T04:21:15.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.518+0000 7fea82000640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea68067b50 0x7fea6806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:15.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.518+0000 7fea82000640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea68067b50 0x7fea6806a010 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0b2530 tx=0x7fea74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:15.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.522+0000 7fea83400640 1 -- 172.21.15.86:0/2830218330 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fea7c015fb0 con 0x7fea8c0b63f0 2024-03-22T04:21:15.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea8c0d2ce0 con 0x7fea8c0b63f0 2024-03-22T04:21:15.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea83400640 1 -- 172.21.15.86:0/2830218330 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5190 (secure 0 0 0) 0x7fea7c0f1190 con 0x7fea8c0b63f0 2024-03-22T04:21:15.716 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:15.716 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:14.587303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77801":{"gid":77801,"name":"a","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.86:6837/4220897872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4220897872},{"type":"v1","addr":"172.21.15.86:6837","nonce":4220897872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-22T04:21:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 104 2024-03-22T04:21:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea68067b50 msgr2=0x7fea6806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea68067b50 0x7fea6806a010 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0b2530 tx=0x7fea74009040 comp rx=0 tx=0).stop 2024-03-22T04:21:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea8c0b63f0 msgr2=0x7fea8c15cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea8c0b63f0 0x7fea8c15cb40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0ae040 tx=0x7fea7c01abd0 comp rx=0 tx=0).stop 2024-03-22T04:21:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 shutdown_connections 2024-03-22T04:21:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fea68067b50 0x7fea6806a010 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea8c15c350 0x7fea8c1476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea8c15c730 0x7fea8c145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 --2- 172.21.15.86:0/2830218330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea8c0b63f0 0x7fea8c15cb40 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 >> 172.21.15.86:0/2830218330 conn(0x7fea8c0b3cf0 msgr2=0x7fea8c070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 shutdown_connections 2024-03-22T04:21:15.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:15.714+0000 7fea91800640 1 -- 172.21.15.86:0/2830218330 wait complete. 2024-03-22T04:21:16.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:20:57.561818+0000', 'modified': '2024-03-22T04:21:14.587303+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 77801}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_77801': {'gid': 77801, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.86:6837/4220897872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 4220897872}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 4220897872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:21:16.138 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_77801:up:rejoin 2024-03-22T04:21:17.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:17.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f363e400640 1 Processor -- start 2024-03-22T04:21:17.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f363e400640 1 -- start start 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f363e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f363815c350 0x7f36380b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f363e400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f36380b63f0 0x7f3638070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f363e400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36380714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f363e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f36380d0b30 con 0x7f363815c350 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f363e400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f36380d0dd0 con 0x7f36380b63f0 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f363e400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f36380d10b0 con 0x7f363815c730 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f363815c350 0x7f36380b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f363815c350 0x7f36380b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58130/0 (socket says 172.21.15.86:58130) 2024-03-22T04:21:17.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637400640 1 -- 172.21.15.86:0/2256686437 learned_addr learned my addr 172.21.15.86:0/2256686437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:17.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637400640 1 -- 172.21.15.86:0/2256686437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 msgr2=0x7f36380714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:17.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f362fe00640 1 --2- 172.21.15.86:0/2256686437 >> v2:172.21.15.196:3300/0 conn(0x7f36380b63f0 0x7f3638070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:17.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637e00640 1 --2- 172.21.15.86:0/2256686437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36380714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:17.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637400640 1 --2- 172.21.15.86:0/2256686437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36380714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637400640 1 -- 172.21.15.86:0/2256686437 >> v2:172.21.15.196:3300/0 conn(0x7f36380b63f0 msgr2=0x7f3638070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:17.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637400640 1 --2- 172.21.15.86:0/2256686437 >> v2:172.21.15.196:3300/0 conn(0x7f36380b63f0 0x7f3638070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637400640 1 -- 172.21.15.86:0/2256686437 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36380d1390 con 0x7f363815c350 2024-03-22T04:21:17.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f362fe00640 1 --2- 172.21.15.86:0/2256686437 >> v2:172.21.15.196:3300/0 conn(0x7f36380b63f0 0x7f3638070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:17.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.274+0000 7f3637e00640 1 --2- 172.21.15.86:0/2256686437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36380714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:17.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f3637400640 1 --2- 172.21.15.86:0/2256686437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f36380b5eb0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f36380b1c10 tx=0x7f362801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb99557cb1158159 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:17.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f3636a00640 1 -- 172.21.15.86:0/2256686437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280aa430 con 0x7f363815c350 2024-03-22T04:21:17.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f3636a00640 1 -- 172.21.15.86:0/2256686437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36280af070 con 0x7f363815c350 2024-03-22T04:21:17.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f3636a00640 1 -- 172.21.15.86:0/2256686437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280aa7f0 con 0x7f363815c350 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 -- 172.21.15.86:0/2256686437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 msgr2=0x7f36380b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 --2- 172.21.15.86:0/2256686437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f36380b5eb0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f36380b1c10 tx=0x7f362801c4f0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 -- 172.21.15.86:0/2256686437 shutdown_connections 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 --2- 172.21.15.86:0/2256686437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36380714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 --2- 172.21.15.86:0/2256686437 >> v2:172.21.15.196:3300/0 conn(0x7f36380b63f0 0x7f3638070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 --2- 172.21.15.86:0/2256686437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f36380b5eb0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 -- 172.21.15.86:0/2256686437 >> 172.21.15.86:0/2256686437 conn(0x7f36380b3cf0 msgr2=0x7f36380b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 -- 172.21.15.86:0/2256686437 shutdown_connections 2024-03-22T04:21:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.278+0000 7f363e400640 1 -- 172.21.15.86:0/2256686437 wait complete. 2024-03-22T04:21:17.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 Processor -- start 2024-03-22T04:21:17.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 -- start start 2024-03-22T04:21:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380b63f0 0x7f36381436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f3638157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36381577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36381608c0 con 0x7f363815c350 2024-03-22T04:21:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3638160740 con 0x7f36380b63f0 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3638160a40 con 0x7f363815c730 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f3638157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f3638157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58132/0 (socket says 172.21.15.86:58132) 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 -- 172.21.15.86:0/3841664898 learned_addr learned my addr 172.21.15.86:0/3841664898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f3637400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380b63f0 0x7f36381436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f3637e00640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36381577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 -- 172.21.15.86:0/3841664898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 msgr2=0x7f36381577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36381577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 -- 172.21.15.86:0/3841664898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380b63f0 msgr2=0x7f36381436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380b63f0 0x7f36381436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 -- 172.21.15.86:0/3841664898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3638148bd0 con 0x7f363815c350 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362fe00640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f3638157260 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f362000c820 tx=0x7f362000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362f400640 1 -- 172.21.15.86:0/3841664898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620015070 con 0x7f363815c350 2024-03-22T04:21:17.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362f400640 1 -- 172.21.15.86:0/3841664898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3620007a30 con 0x7f363815c350 2024-03-22T04:21:17.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3638148e70 con 0x7f363815c350 2024-03-22T04:21:17.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f362f400640 1 -- 172.21.15.86:0/3841664898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620007e20 con 0x7f363815c350 2024-03-22T04:21:17.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3638149430 con 0x7f363815c350 2024-03-22T04:21:17.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.282+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3638075220 con 0x7f363815c350 2024-03-22T04:21:17.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.286+0000 7f362f400640 1 -- 172.21.15.86:0/3841664898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3620018070 con 0x7f363815c350 2024-03-22T04:21:17.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.286+0000 7f362f400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3614067b20 0x7f3614069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:17.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.286+0000 7f362f400640 1 -- 172.21.15.86:0/3841664898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f3620085ee0 con 0x7f363815c350 2024-03-22T04:21:17.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.286+0000 7f3637400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3614067b20 0x7f3614069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:17.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.286+0000 7f3637400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3614067b20 0x7f3614069fe0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f36280ae000 tx=0x7f36280a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.290+0000 7f362f400640 1 -- 172.21.15.86:0/3841664898 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f36200560b0 con 0x7f363815c350 2024-03-22T04:21:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.474+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36380b44e0 con 0x7f363815c350 2024-03-22T04:21:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f362f400640 1 -- 172.21.15.86:0/3841664898 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5199 (secure 0 0 0) 0x7f362005a150 con 0x7f363815c350 2024-03-22T04:21:17.479 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:17.479 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:16.586211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77801":{"gid":77801,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/4220897872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4220897872},{"type":"v1","addr":"172.21.15.86:6837","nonce":4220897872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77801,"qdb_cluster":[77801]},"id":8}]} 2024-03-22T04:21:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 105 2024-03-22T04:21:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3614067b20 msgr2=0x7f3614069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3614067b20 0x7f3614069fe0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f36280ae000 tx=0x7f36280a7000 comp rx=0 tx=0).stop 2024-03-22T04:21:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 msgr2=0x7f3638157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f3638157260 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f362000c820 tx=0x7f362000cce0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 shutdown_connections 2024-03-22T04:21:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3614067b20 0x7f3614069fe0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f363815c730 0x7f36381577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f363815c350 0x7f3638157260 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 --2- 172.21.15.86:0/3841664898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36380b63f0 0x7f36381436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:17.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 >> 172.21.15.86:0/3841664898 conn(0x7f36380b3cf0 msgr2=0x7f36380cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:17.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 shutdown_connections 2024-03-22T04:21:17.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:17.478+0000 7f363e400640 1 -- 172.21.15.86:0/3841664898 wait complete. 2024-03-22T04:21:17.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:20:57.561818+0000', 'modified': '2024-03-22T04:21:16.586211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 77801}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_77801': {'gid': 77801, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6837/4220897872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 4220897872}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 4220897872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 77801, 'qdb_cluster': [77801]} 2024-03-22T04:21:17.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:21:17.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-22T04:21:18.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.038+0000 7f7ef3200640 1 Processor -- start 2024-03-22T04:21:18.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef3200640 1 -- start start 2024-03-22T04:21:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef3200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7ef4161720 0x7f7ef4070280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ef40afd40 0x7f7ef406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef3200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 0x7f7ef40c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef40ca2d0 con 0x7f7ef40afd40 2024-03-22T04:21:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef3200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef40cce00 con 0x7f7ef4161720 2024-03-22T04:21:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef3200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef40cc280 con 0x7f7ef40c9920 2024-03-22T04:21:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ef40afd40 0x7f7ef406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ef40afd40 0x7f7ef406b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58140/0 (socket says 172.21.15.86:58140) 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1400640 1 -- 172.21.15.86:0/2925610001 learned_addr learned my addr 172.21.15.86:0/2925610001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/2925610001 >> v2:172.21.15.196:3300/0 conn(0x7f7ef4161720 0x7f7ef4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef2800640 1 --2- 172.21.15.86:0/2925610001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 0x7f7ef40c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1400640 1 -- 172.21.15.86:0/2925610001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 msgr2=0x7f7ef40c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1400640 1 --2- 172.21.15.86:0/2925610001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 0x7f7ef40c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1400640 1 -- 172.21.15.86:0/2925610001 >> v2:172.21.15.196:3300/0 conn(0x7f7ef4161720 msgr2=0x7f7ef4070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1400640 1 --2- 172.21.15.86:0/2925610001 >> v2:172.21.15.196:3300/0 conn(0x7f7ef4161720 0x7f7ef4070280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1400640 1 -- 172.21.15.86:0/2925610001 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef40cc560 con 0x7f7ef40afd40 2024-03-22T04:21:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.042+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/2925610001 >> v2:172.21.15.196:3300/0 conn(0x7f7ef4161720 0x7f7ef4070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:18.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef1400640 1 --2- 172.21.15.86:0/2925610001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef40afd40 0x7f7ef406b660 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4072d80 tx=0x7f7ee001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bed269dad87a54 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef0a00640 1 -- 172.21.15.86:0/2925610001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00ab620 con 0x7f7ef40afd40 2024-03-22T04:21:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef0a00640 1 -- 172.21.15.86:0/2925610001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee00b0070 con 0x7f7ef40afd40 2024-03-22T04:21:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef0a00640 1 -- 172.21.15.86:0/2925610001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00ab9e0 con 0x7f7ef40afd40 2024-03-22T04:21:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 -- 172.21.15.86:0/2925610001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef40afd40 msgr2=0x7f7ef406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 --2- 172.21.15.86:0/2925610001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef40afd40 0x7f7ef406b660 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4072d80 tx=0x7f7ee001c460 comp rx=0 tx=0).stop 2024-03-22T04:21:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 -- 172.21.15.86:0/2925610001 shutdown_connections 2024-03-22T04:21:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 --2- 172.21.15.86:0/2925610001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 0x7f7ef40c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 --2- 172.21.15.86:0/2925610001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef40afd40 0x7f7ef406b660 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 --2- 172.21.15.86:0/2925610001 >> v2:172.21.15.196:3300/0 conn(0x7f7ef4161720 0x7f7ef4070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 -- 172.21.15.86:0/2925610001 >> 172.21.15.86:0/2925610001 conn(0x7f7ef40cd050 msgr2=0x7f7ef40cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 -- 172.21.15.86:0/2925610001 shutdown_connections 2024-03-22T04:21:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.046+0000 7f7ef3200640 1 -- 172.21.15.86:0/2925610001 wait complete. 2024-03-22T04:21:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 Processor -- start 2024-03-22T04:21:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 -- start start 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40afd40 0x7f7ef415ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 0x7f7ef4153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef4161720 0x7f7ef4156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef40cc000 con 0x7f7ef4161720 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef40cbe80 con 0x7f7ef40afd40 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef40cec90 con 0x7f7ef40c9920 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef4161720 0x7f7ef4156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef4161720 0x7f7ef4156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58142/0 (socket says 172.21.15.86:58142) 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 -- 172.21.15.86:0/859412634 learned_addr learned my addr 172.21.15.86:0/859412634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40afd40 0x7f7ef415ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef1400640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 0x7f7ef4153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 -- 172.21.15.86:0/859412634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 msgr2=0x7f7ef4153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 0x7f7ef4153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 -- 172.21.15.86:0/859412634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40afd40 msgr2=0x7f7ef415ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40afd40 0x7f7ef415ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 -- 172.21.15.86:0/859412634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef415f370 con 0x7f7ef4161720 2024-03-22T04:21:18.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef2800640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef4161720 0x7f7ef4156a30 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f7eec007e60 tx=0x7f7eec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7eea000640 1 -- 172.21.15.86:0/859412634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec014620 con 0x7f7ef4161720 2024-03-22T04:21:18.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7eea000640 1 -- 172.21.15.86:0/859412634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec019070 con 0x7f7ef4161720 2024-03-22T04:21:18.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7eea000640 1 -- 172.21.15.86:0/859412634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec014a20 con 0x7f7ef4161720 2024-03-22T04:21:18.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 -- 172.21.15.86:0/859412634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef415f650 con 0x7f7ef4161720 2024-03-22T04:21:18.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.050+0000 7f7ef3200640 1 -- 172.21.15.86:0/859412634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef415fc10 con 0x7f7ef4161720 2024-03-22T04:21:18.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7eea000640 1 -- 172.21.15.86:0/859412634 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7eec014bc0 con 0x7f7ef4161720 2024-03-22T04:21:18.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7eea000640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7ed4067b20 0x7f7ed4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7eea000640 1 -- 172.21.15.86:0/859412634 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f7eec0868e0 con 0x7f7ef4161720 2024-03-22T04:21:18.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7ed4067b20 0x7f7ed4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7ed4067b20 0x7f7ed4069fe0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7ef406ccb0 tx=0x7f7ee4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7efc500000 1 Processor -- start 2024-03-22T04:21:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7efc500000 1 -- start start 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7efc500000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c989e70 0x55e55c98a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7efc500000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c98a790 0x55e55c93d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7efc500000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c93e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7efc500000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e55c73b140 con 0x55e55c98a790 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7efc500000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e55c78eb10 con 0x55e55c93dc20 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.054+0000 7f7efc500000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e55c715c50 con 0x55e55c989e70 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c98a790 0x55e55c93d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c98a790 0x55e55c93d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58144/0 (socket says 172.21.15.86:58144) 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 -- 172.21.15.86:0/975925544 learned_addr learned my addr 172.21.15.86:0/975925544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c989e70 0x55e55c98a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef2800640 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c93e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 -- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c989e70 msgr2=0x55e55c98a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c989e70 0x55e55c98a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 -- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 msgr2=0x55e55c93e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c93e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 -- 172.21.15.86:0/975925544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55c7d3f20 con 0x55e55c98a790 2024-03-22T04:21:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef2800640 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c93e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7ef1400640 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c98a790 0x55e55c93d6e0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c10 tx=0x7f7ee0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc6f3de804fb5c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7edea00640 1 -- 172.21.15.86:0/975925544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00aba40 con 0x55e55c98a790 2024-03-22T04:21:18.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7edea00640 1 -- 172.21.15.86:0/975925544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee00b0040 con 0x55e55c98a790 2024-03-22T04:21:18.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7edea00640 1 -- 172.21.15.86:0/975925544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00abe00 con 0x55e55c98a790 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 -- 172.21.15.86:0/975925544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c98a790 msgr2=0x55e55c93d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c98a790 0x55e55c93d6e0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c10 tx=0x7f7ee0004740 comp rx=0 tx=0).stop 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 -- 172.21.15.86:0/975925544 shutdown_connections 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c93e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c98a790 0x55e55c93d6e0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 --2- 172.21.15.86:0/975925544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c989e70 0x55e55c98a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 -- 172.21.15.86:0/975925544 >> 172.21.15.86:0/975925544 conn(0x55e55c96ac10 msgr2=0x55e55c96d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 -- 172.21.15.86:0/975925544 shutdown_connections 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 -- 172.21.15.86:0/975925544 wait complete. 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 -- start start 2024-03-22T04:21:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.058+0000 7f7efc500000 1 Processor -- start 2024-03-22T04:21:18.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c942810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c942d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c98a790 0x55e55c943290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e55c748ab0 con 0x55e55c989e70 2024-03-22T04:21:18.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e55c78aac0 con 0x55e55c93dc20 2024-03-22T04:21:18.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e55c734e80 con 0x55e55c98a790 2024-03-22T04:21:18.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 10 client.0 _ll_register_callbacks cb 0x55e55c92b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:18.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c942d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c942810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c942d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58148/0 (socket says 172.21.15.86:58148) 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 -- 172.21.15.86:0/1243073369 learned_addr learned my addr 172.21.15.86:0/1243073369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef2800640 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c98a790 0x55e55c943290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 -- 172.21.15.86:0/1243073369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c98a790 msgr2=0x55e55c943290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c98a790 0x55e55c943290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 -- 172.21.15.86:0/1243073369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 msgr2=0x55e55c942810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c942810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 -- 172.21.15.86:0/1243073369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55c764570 con 0x55e55c989e70 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1400640 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c942d50 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7ee001a090 tx=0x7f7ee001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2043d54409f0dc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ed3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ed3400640 1 -- 172.21.15.86:0/1243073369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00ce060 con 0x55e55c989e70 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ed3400640 1 -- 172.21.15.86:0/1243073369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f7ee00042c0 con 0x55e55c989e70 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 -- 172.21.15.86:0/1243073369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee00b0040 con 0x55e55c989e70 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ed3400640 1 -- 172.21.15.86:0/1243073369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f7ee00d4050 con 0x55e55c989e70 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 10 client.9327 fetch_fsmap learned FSMap version 105 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 -- 172.21.15.86:0/1243073369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55e55c75a880 con 0x55e55c989e70 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ed3400640 1 -- 172.21.15.86:0/1243073369 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee00b0040 con 0x55e55c989e70 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ed3400640 1 -- 172.21.15.86:0/1243073369 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f7ee00d7020 con 0x55e55c989e70 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 10 client.9327 fetch_fsmap finished waiting for FSMap version 105 2024-03-22T04:21:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 10 client.9327 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:21:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 20 client.9327 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x55e55c943ca0 0x55e55c99b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 4 client.9327 mds_command: new command op to 77801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:21:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7efc500000 1 -- 172.21.15.86:0/1243073369 --> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e55c729bc0 con 0x55e55c943ca0 2024-03-22T04:21:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x55e55c943ca0 0x55e55c99b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x55e55c943ca0 0x55e55c99b120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708ccc7295536c4a server_cookie=60aee184eff5a937 in_seq=0 out_seq=0 2024-03-22T04:21:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.062+0000 7f7ed3400640 10 client.9327 ms_handle_connect on v2:172.21.15.86:6835/4220897872 2024-03-22T04:21:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7ed3400640 1 -- 172.21.15.86:0/1243073369 <== mds.0 v2:172.21.15.86:6835/4220897872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e55c729bc0 con 0x55e55c943ca0 2024-03-22T04:21:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7ed3400640 10 client.9327 handle_command_reply: tid=0 2024-03-22T04:21:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 client.9327 shutdown 2024-03-22T04:21:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 -- 172.21.15.86:0/1243073369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 msgr2=0x55e55c942d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c942d50 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7ee001a090 tx=0x7f7ee001aca0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 -- 172.21.15.86:0/1243073369 shutdown_connections 2024-03-22T04:21:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x55e55c943ca0 0x55e55c99b120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c98a790 0x55e55c943290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c942d50 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7ed3400640 0 client.9327 ms_handle_reset on v2:172.21.15.86:6835/4220897872 2024-03-22T04:21:18.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 --2- 172.21.15.86:0/1243073369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c942810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 -- 172.21.15.86:0/1243073369 >> 172.21.15.86:0/1243073369 conn(0x55e55c96ac10 msgr2=0x55e55c96d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:18.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 -- 172.21.15.86:0/1243073369 shutdown_connections 2024-03-22T04:21:18.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 1 -- 172.21.15.86:0/1243073369 wait complete. 2024-03-22T04:21:18.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.066+0000 7f7efc500000 20 client.9327 trim_cache size 0 max 16384 2024-03-22T04:21:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7efc500000 1 Processor -- start 2024-03-22T04:21:18.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7efc500000 1 -- start start 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7efc500000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c96ac10 0x55e55c988f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7efc500000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c989470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7efc500000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c99b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7efc500000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4202a10 con 0x55e55c989e70 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7efc500000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4201da0 con 0x55e55c93dc20 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7efc500000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef42023c0 con 0x55e55c96ac10 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7ef2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c99b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7ef2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c99b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58154/0 (socket says 172.21.15.86:58154) 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.094+0000 7f7ef2800640 1 -- 172.21.15.86:0/3687838923 learned_addr learned my addr 172.21.15.86:0/3687838923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c96ac10 0x55e55c988f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef1400640 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c989470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 -- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c96ac10 msgr2=0x55e55c988f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c96ac10 0x55e55c988f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 -- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 msgr2=0x55e55c989470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c989470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 -- 172.21.15.86:0/3687838923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55c716e80 con 0x55e55c989e70 2024-03-22T04:21:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c96ac10 0x55e55c988f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c99b120 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7eec056720 tx=0x7f7eec05d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98089c32da6da73c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ede000640 1 -- 172.21.15.86:0/3687838923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec05aac0 con 0x55e55c989e70 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ede000640 1 -- 172.21.15.86:0/3687838923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec05f2f0 con 0x55e55c989e70 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ede000640 1 -- 172.21.15.86:0/3687838923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec053090 con 0x55e55c989e70 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- 172.21.15.86:0/3687838923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 msgr2=0x55e55c99b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c99b120 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7eec056720 tx=0x7f7eec05d040 comp rx=0 tx=0).stop 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- 172.21.15.86:0/3687838923 shutdown_connections 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c93dc20 0x55e55c989470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c99b120 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 --2- 172.21.15.86:0/3687838923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c96ac10 0x55e55c988f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- 172.21.15.86:0/3687838923 >> 172.21.15.86:0/3687838923 conn(0x55e55c945490 msgr2=0x55e55c93cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- 172.21.15.86:0/3687838923 shutdown_connections 2024-03-22T04:21:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- 172.21.15.86:0/3687838923 wait complete. 2024-03-22T04:21:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- start start 2024-03-22T04:21:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 Processor -- start 2024-03-22T04:21:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c93dc20 0x55e55c99e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c96ac10 0x55e55c9f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c9f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e55c734e80 con 0x55e55c989e70 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e55c7cea60 con 0x55e55c96ac10 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e55c748ab0 con 0x55e55c93dc20 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7efc500000 10 client.0 _ll_register_callbacks cb 0x7f7ee4007650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c9f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c9f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58162/0 (socket says 172.21.15.86:58162) 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 -- 172.21.15.86:0/640827990 learned_addr learned my addr 172.21.15.86:0/640827990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c93dc20 0x55e55c99e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef1400640 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c96ac10 0x55e55c9f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 -- 172.21.15.86:0/640827990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c93dc20 msgr2=0x55e55c99e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c93dc20 0x55e55c99e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 -- 172.21.15.86:0/640827990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c96ac10 msgr2=0x55e55c9f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c96ac10 0x55e55c9f90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.098+0000 7f7ef2800640 1 -- 172.21.15.86:0/640827990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55c722750 con 0x55e55c989e70 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef2800640 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c9f9600 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0569b0 tx=0x7f7eec020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c517a00e0fd2c95 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 1 -- 172.21.15.86:0/640827990 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec072070 con 0x55e55c989e70 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 1 -- 172.21.15.86:0/640827990 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f7eec05f060 con 0x55e55c989e70 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 -- 172.21.15.86:0/640827990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eec05f2f0 con 0x55e55c989e70 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 1 -- 172.21.15.86:0/640827990 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f7eec084050 con 0x55e55c989e70 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 10 client.9333 fetch_fsmap learned FSMap version 105 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 -- 172.21.15.86:0/640827990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55e55c735450 con 0x55e55c989e70 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 1 -- 172.21.15.86:0/640827990 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec05f2f0 con 0x55e55c989e70 2024-03-22T04:21:18.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 1 -- 172.21.15.86:0/640827990 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f7eec055e70 con 0x55e55c989e70 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 10 client.9333 fetch_fsmap finished waiting for FSMap version 105 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 10 client.9333 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 20 client.9333 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x55e55c98a790 0x55e55c9f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 4 client.9333 mds_command: new command op to 77801 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 -- 172.21.15.86:0/640827990 --> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] -- command(tid 0: {"prefix": "status"}) -- 0x55e55c79c050 con 0x55e55c98a790 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x55e55c98a790 0x55e55c9f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 10 client.9333 ms_handle_connect on v2:172.21.15.86:6835/4220897872 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef1e00640 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x55e55c98a790 0x55e55c9f5b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e311a959b0665 server_cookie=d452c5233ad1910b in_seq=0 out_seq=0 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 1 -- 172.21.15.86:0/640827990 <== mds.0 v2:172.21.15.86:6835/4220897872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55e55c79c050 con 0x55e55c98a790 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 10 client.9333 handle_command_reply: tid=0 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 client.9333 shutdown 2024-03-22T04:21:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 -- 172.21.15.86:0/640827990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 msgr2=0x55e55c9f9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c9f9600 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0569b0 tx=0x7f7eec020040 comp rx=0 tx=0).stop 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 -- 172.21.15.86:0/640827990 shutdown_connections 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x55e55c98a790 0x55e55c9f5b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e55c96ac10 0x55e55c9f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e55c989e70 0x55e55c9f9600 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 --2- 172.21.15.86:0/640827990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e55c93dc20 0x55e55c99e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ed3400640 0 client.9333 ms_handle_reset on v2:172.21.15.86:6835/4220897872 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 -- 172.21.15.86:0/640827990 >> 172.21.15.86:0/640827990 conn(0x55e55c945490 msgr2=0x55e55c987f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 -- 172.21.15.86:0/640827990 shutdown_connections 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 1 -- 172.21.15.86:0/640827990 wait complete. 2024-03-22T04:21:18.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7efc500000 20 client.9333 trim_cache size 0 max 16384 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 77801, 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.5293558420000002, 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 105, 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 125, 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 125, 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 5.4463870830000003 2024-03-22T04:21:18.111 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 -- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7ed4067b20 msgr2=0x7f7ed4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7ed4067b20 0x7f7ed4069fe0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7ef406ccb0 tx=0x7f7ee4008040 comp rx=0 tx=0).stop 2024-03-22T04:21:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 -- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef4161720 msgr2=0x7f7ef4156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef4161720 0x7f7ef4156a30 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f7eec007e60 tx=0x7f7eec00e4b0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 -- 172.21.15.86:0/859412634 shutdown_connections 2024-03-22T04:21:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7ed4067b20 0x7f7ed4069fe0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef4161720 0x7f7ef4156a30 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40c9920 0x7f7ef4153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 --2- 172.21.15.86:0/859412634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40afd40 0x7f7ef415ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 -- 172.21.15.86:0/859412634 >> 172.21.15.86:0/859412634 conn(0x7f7ef40cd050 msgr2=0x7f7ef40cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:18.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 -- 172.21.15.86:0/859412634 shutdown_connections 2024-03-22T04:21:18.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.102+0000 7f7ef3200640 1 -- 172.21.15.86:0/859412634 wait complete. 2024-03-22T04:21:18.124 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:21:18.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f2974e00640 1 Processor -- start 2024-03-22T04:21:18.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f2974e00640 1 -- start start 2024-03-22T04:21:18.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f2974e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2970147c70 0x7f297006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f2974e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2970161fc0 0x7f29700b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f2974e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f297006e0a0 con 0x7f29700b5530 2024-03-22T04:21:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f2974e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2970148050 con 0x7f2970161fc0 2024-03-22T04:21:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f2974e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f297006d490 con 0x7f2970147c70 2024-03-22T04:21:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f296e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f29700b5530 0x7f2970070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60646/0 (socket says 172.21.15.86:60646) 2024-03-22T04:21:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f296e800640 1 -- 172.21.15.86:0/1858657147 learned_addr learned my addr 172.21.15.86:0/1858657147 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f296de00640 1 --2- 172.21.15.86:0/1858657147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2970147c70 0x7f297006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.270+0000 7f2967e00640 1 --2- 172.21.15.86:0/1858657147 >> v2:172.21.15.196:3300/0 conn(0x7f2970161fc0 0x7f29700b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296d400640 1 -- 172.21.15.86:0/1858657147 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979170780 0 0) 0x7f297006e0a0 con 0x7f29700b5530 2024-03-22T04:21:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296d400640 1 -- 172.21.15.86:0/1858657147 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2954003530 con 0x7f29700b5530 2024-03-22T04:21:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296de00640 1 -- 172.21.15.86:0/1858657147 >> v2:172.21.15.196:3300/0 conn(0x7f2970161fc0 msgr2=0x7f29700b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296de00640 1 --2- 172.21.15.86:0/1858657147 >> v2:172.21.15.196:3300/0 conn(0x7f2970161fc0 0x7f29700b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296de00640 1 -- 172.21.15.86:0/1858657147 >> v1:172.21.15.86:6789/0 conn(0x7f29700b5530 legacy=0x7f2970070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296d400640 1 -- 172.21.15.86:0/1858657147 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 435444436 0 0) 0x7f2954003530 con 0x7f29700b5530 2024-03-22T04:21:18.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296de00640 1 -- 172.21.15.86:0/1858657147 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297006d770 con 0x7f2970147c70 2024-03-22T04:21:18.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296de00640 1 --2- 172.21.15.86:0/1858657147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2970147c70 0x7f297006cc40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29600097c0 tx=0x7f296001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c790e352d45e803 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296d400640 1 -- 172.21.15.86:0/1858657147 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29600b0070 con 0x7f2970147c70 2024-03-22T04:21:18.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296d400640 1 -- 172.21.15.86:0/1858657147 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29600ab400 con 0x7f2970147c70 2024-03-22T04:21:18.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.274+0000 7f296d400640 1 -- 172.21.15.86:0/1858657147 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968001a00 con 0x7f2970147c70 2024-03-22T04:21:18.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- 172.21.15.86:0/1858657147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2970147c70 msgr2=0x7f297006cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 --2- 172.21.15.86:0/1858657147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2970147c70 0x7f297006cc40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29600097c0 tx=0x7f296001c3c0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- 172.21.15.86:0/1858657147 shutdown_connections 2024-03-22T04:21:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 --2- 172.21.15.86:0/1858657147 >> v2:172.21.15.196:3300/0 conn(0x7f2970161fc0 0x7f29700b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 --2- 172.21.15.86:0/1858657147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2970147c70 0x7f297006cc40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- 172.21.15.86:0/1858657147 >> 172.21.15.86:0/1858657147 conn(0x7f29700b3cf0 msgr2=0x7f29700b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- 172.21.15.86:0/1858657147 shutdown_connections 2024-03-22T04:21:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- 172.21.15.86:0/1858657147 wait complete. 2024-03-22T04:21:18.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 Processor -- start 2024-03-22T04:21:18.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- start start 2024-03-22T04:21:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29700b5530 0x7f29701630d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2970147c70 0x7f297015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2970161fc0 0x7f297015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f297016b4a0 con 0x7f2970147c70 2024-03-22T04:21:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f297016b320 con 0x7f2970161fc0 2024-03-22T04:21:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2974e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f297016b620 con 0x7f29700b5530 2024-03-22T04:21:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2970147c70 0x7f297015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2970147c70 0x7f297015d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58172/0 (socket says 172.21.15.86:58172) 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 -- 172.21.15.86:0/4075549425 learned_addr learned my addr 172.21.15.86:0/4075549425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f296de00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29700b5530 0x7f29701630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f296e800640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2970161fc0 0x7f297015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 -- 172.21.15.86:0/4075549425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29700b5530 msgr2=0x7f29701630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29700b5530 0x7f29701630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 -- 172.21.15.86:0/4075549425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2970161fc0 msgr2=0x7f297015c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2970161fc0 0x7f297015c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 -- 172.21.15.86:0/4075549425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29701419f0 con 0x7f2970147c70 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.278+0000 7f2967e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2970147c70 0x7f297015d030 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f295800e870 tx=0x7f295800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2966000640 1 -- 172.21.15.86:0/4075549425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958017070 con 0x7f2970147c70 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2966000640 1 -- 172.21.15.86:0/4075549425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f295800c940 con 0x7f2970147c70 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2966000640 1 -- 172.21.15.86:0/4075549425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295800ccf0 con 0x7f2970147c70 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2970141cd0 con 0x7f2970147c70 2024-03-22T04:21:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297015a8f0 con 0x7f2970147c70 2024-03-22T04:21:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2970075220 con 0x7f2970147c70 2024-03-22T04:21:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2966000640 1 -- 172.21.15.86:0/4075549425 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f295801a070 con 0x7f2970147c70 2024-03-22T04:21:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2966000640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f294c067b20 0x7f294c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f2966000640 1 -- 172.21.15.86:0/4075549425 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f29580862d0 con 0x7f2970147c70 2024-03-22T04:21:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f296de00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f294c067b20 0x7f294c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.282+0000 7f296de00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f294c067b20 0x7f294c069fe0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f29600ad000 tx=0x7f2960008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.286+0000 7f2966000640 1 -- 172.21.15.86:0/4075549425 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2958056b90 con 0x7f2970147c70 2024-03-22T04:21:18.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.486+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f29700d2ce0 con 0x7f2970147c70 2024-03-22T04:21:18.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.486+0000 7f2966000640 1 -- 172.21.15.86:0/4075549425 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7f2958056490 con 0x7f2970147c70 2024-03-22T04:21:18.489 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:21:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f294c067b20 msgr2=0x7f294c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f294c067b20 0x7f294c069fe0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f29600ad000 tx=0x7f2960008040 comp rx=0 tx=0).stop 2024-03-22T04:21:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2970147c70 msgr2=0x7f297015d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2970147c70 0x7f297015d030 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f295800e870 tx=0x7f295800ec70 comp rx=0 tx=0).stop 2024-03-22T04:21:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 shutdown_connections 2024-03-22T04:21:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f294c067b20 0x7f294c069fe0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2970161fc0 0x7f297015c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2970147c70 0x7f297015d030 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 --2- 172.21.15.86:0/4075549425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29700b5530 0x7f29701630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 >> 172.21.15.86:0/4075549425 conn(0x7f29700b3cf0 msgr2=0x7f29701623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 shutdown_connections 2024-03-22T04:21:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18.490+0000 7f2974e00640 1 -- 172.21.15.86:0/4075549425 wait complete. 2024-03-22T04:21:18.510 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:21:18.510 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:21:18.510 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:21:18.510 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:21:18.510 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:18.510 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:21:18.510 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:21:18.511 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:21:18.511 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:18.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:18.511 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:21:18.520 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:21:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:18.521 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41853sec preferred_lft 41853sec 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:21:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:18.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:18.524 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:21:18.577 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-03-22T04:21:18.574+0000 7f3151400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:21:18.579 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:21:18.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:18.582 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:21:18.637 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:18.638 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:18.638 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:18.638 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:21:18.638 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:21:18.638 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:21:18.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:21:18.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:21:18.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:21:18.638 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:21:18.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:21:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:18.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:18.771 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:21:18.780 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:21:18.780 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:21:18.781 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:18.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:18.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:21:18.841 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:21:18.841 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:18.890 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:18.890 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:21:18.890 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:18.941 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:21:18.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:18.941 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:19.029 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:19.029 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:19.029 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:19.044 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:19.044 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:19.044 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:19.044 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:21:19.045 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:21:19.054 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:19.054 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:19.054 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:19.054 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:21:19.054 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:21:19.054 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:21:19.054 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:21:19.054 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:21:19.059 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:21:19.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:19.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:19.078 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:21:19.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:21:19.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:19.182 INFO:teuthology.orchestra.run.smithi086.stdout:client77849 2024-03-22T04:21:19.182 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:19.182 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:19.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:19.191 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-22T04:21:19.246 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-03-22T04:21:19.247 INFO:teuthology.orchestra.run.smithi086.stdout:./parent 2024-03-22T04:21:19.247 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed 2024-03-22T04:21:19.247 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/charlie 2024-03-22T04:21:19.247 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/alpha 2024-03-22T04:21:19.247 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/unflushed 2024-03-22T04:21:19.247 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/unflushed/jfile 2024-03-22T04:21:19.248 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:19.248 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:19.248 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:19.248 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:19.248 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-22T04:21:19.248 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:19.353 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:21:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:19.362 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:19.362 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:19.362 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:19.362 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:19.362 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-22T04:21:19.362 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:19.469 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-03-22T04:21:19.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:19.478 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:19.478 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:19.478 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:19.478 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:19.478 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-22T04:21:19.478 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:19.586 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-03-22T04:21:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:19.595 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:19.595 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:19.595 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:19.595 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:19.596 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-22T04:21:19.596 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:19.701 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628280 2024-03-22T04:21:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:19.710 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:19.710 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:19.710 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:19.710 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:19.711 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-22T04:21:19.711 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:19.817 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-03-22T04:21:19.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:19.826 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:19.826 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:19.826 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:19.826 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:19.826 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-22T04:21:19.826 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:19.934 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627779 2024-03-22T04:21:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:19.943 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:19.943 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:19.943 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:19.943 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:19.943 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-22T04:21:19.943 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:20.049 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627780 2024-03-22T04:21:20.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:20.058 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:20.058 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:20.111 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:20.111 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:20.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.338+0000 7f8bc2800640 1 Processor -- start 2024-03-22T04:21:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.338+0000 7f8bc2800640 1 -- start start 2024-03-22T04:21:20.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bc2800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8bbc06bd70 0x7f8bbc06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bc2800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bbc15cd10 con 0x7f8bbc145cf0 2024-03-22T04:21:20.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bc2800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bbc073190 con 0x7f8bbc15c890 2024-03-22T04:21:20.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bc2800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8bbc0cdbb0 con 0x7f8bbc06bd70 2024-03-22T04:21:20.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8bbc145cf0 0x7f8bbc072a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60660/0 (socket says 172.21.15.86:60660) 2024-03-22T04:21:20.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbb400640 1 -- 172.21.15.86:0/2222559371 learned_addr learned my addr 172.21.15.86:0/2222559371 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:20.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbbe00640 1 --2- 172.21.15.86:0/2222559371 >> v2:172.21.15.196:3300/0 conn(0x7f8bbc06bd70 0x7f8bbc06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2483696304 0 0) 0x7f8bbc15cd10 con 0x7f8bbc145cf0 2024-03-22T04:21:20.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ba0003530 con 0x7f8bbc145cf0 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 661861380 0 0) 0x7f8ba0003530 con 0x7f8bbc145cf0 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 >> v2:172.21.15.196:3300/0 conn(0x7f8bbc06bd70 msgr2=0x7f8bbc06e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 --2- 172.21.15.86:0/2222559371 >> v2:172.21.15.196:3300/0 conn(0x7f8bbc06bd70 0x7f8bbc06e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 >> v1:172.21.15.196:6789/0 conn(0x7f8bbc15c890 legacy=0x7f8bbc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc0cde00 con 0x7f8bbc145cf0 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f8bac001680 con 0x7f8bbc145cf0 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 >> v1:172.21.15.86:6789/0 conn(0x7f8bbc145cf0 legacy=0x7f8bbc072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ba00039c0 0x7f8ba0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba0004810 0x7f8ba0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8ba000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc0cdbb0 con 0x7f8ba0005220 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba000c150 con 0x7f8ba0004810 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba000c350 con 0x7f8ba00039c0 2024-03-22T04:21:20.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbbe00640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8ba000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbb400640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ba00039c0 0x7f8ba0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bb3e00640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba0004810 0x7f8ba0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbbe00640 1 -- 172.21.15.86:0/2222559371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ba00039c0 msgr2=0x7f8ba0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:20.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbbe00640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ba00039c0 0x7f8ba0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbbe00640 1 -- 172.21.15.86:0/2222559371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba0004810 msgr2=0x7f8ba0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:20.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbbe00640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba0004810 0x7f8ba0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.342+0000 7f8bbbe00640 1 -- 172.21.15.86:0/2222559371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba0003530 con 0x7f8ba0005220 2024-03-22T04:21:20.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bbbe00640 1 -- 172.21.15.86:0/2222559371 reap_dead start 2024-03-22T04:21:20.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bbbe00640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8ba000b9d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0b6520 tx=0x7f8bb401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c337946bdf12493 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4009c10 con 0x7f8ba0005220 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb40a5c70 con 0x7f8ba0005220 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bbaa00640 1 -- 172.21.15.86:0/2222559371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4003be0 con 0x7f8ba0005220 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bc2800640 1 -- 172.21.15.86:0/2222559371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 msgr2=0x7f8ba000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bc2800640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8ba000b9d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0b6520 tx=0x7f8bb401cc60 comp rx=0 tx=0).stop 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bc2800640 1 -- 172.21.15.86:0/2222559371 shutdown_connections 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bc2800640 1 --2- 172.21.15.86:0/2222559371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8ba000b9d0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bc2800640 1 -- 172.21.15.86:0/2222559371 >> 172.21.15.86:0/2222559371 conn(0x7f8bbc06c770 msgr2=0x7f8bbc06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bc2800640 1 -- 172.21.15.86:0/2222559371 shutdown_connections 2024-03-22T04:21:20.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.346+0000 7f8bc2800640 1 -- 172.21.15.86:0/2222559371 wait complete. 2024-03-22T04:21:20.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 Processor -- start 2024-03-22T04:21:20.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 -- start start 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8bbc0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc0b2170 0x7f8bbc15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15dd50 0x7f8bbc15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc0ce6a0 con 0x7f8ba0005220 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc0ce3c0 con 0x7f8bbc0b2170 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8bbc0ce980 con 0x7f8bbc15dd50 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8bbc0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8bbc0b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58190/0 (socket says 172.21.15.86:58190) 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 -- 172.21.15.86:0/618401328 learned_addr learned my addr 172.21.15.86:0/618401328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbbe00640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15dd50 0x7f8bbc15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bb3e00640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc0b2170 0x7f8bbc15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 -- 172.21.15.86:0/618401328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15dd50 msgr2=0x7f8bbc15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15dd50 0x7f8bbc15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 -- 172.21.15.86:0/618401328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc0b2170 msgr2=0x7f8bbc15bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc0b2170 0x7f8bbc15bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 -- 172.21.15.86:0/618401328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc0cfb70 con 0x7f8ba0005220 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bbb400640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8bbc0b1510 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8bac012490 tx=0x7f8bac012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bb3400640 1 -- 172.21.15.86:0/618401328 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac01c070 con 0x7f8ba0005220 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bb3400640 1 -- 172.21.15.86:0/618401328 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bac014750 con 0x7f8ba0005220 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bbc1335b0 con 0x7f8ba0005220 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bb3400640 1 -- 172.21.15.86:0/618401328 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac014b40 con 0x7f8ba0005220 2024-03-22T04:21:20.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.350+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bbc0b29c0 con 0x7f8ba0005220 2024-03-22T04:21:20.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.354+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bbc075220 con 0x7f8ba0005220 2024-03-22T04:21:20.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.354+0000 7f8bb3400640 1 -- 172.21.15.86:0/618401328 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8bac01f070 con 0x7f8ba0005220 2024-03-22T04:21:20.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.354+0000 7f8bb3400640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b98067b20 0x7f8b98069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.354+0000 7f8bb3400640 1 -- 172.21.15.86:0/618401328 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f8bac088ad0 con 0x7f8ba0005220 2024-03-22T04:21:20.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.354+0000 7f8bb3e00640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b98067b20 0x7f8b98069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.354+0000 7f8bb3e00640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b98067b20 0x7f8b98069fe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0b6430 tx=0x7f8ba4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:20.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.358+0000 7f8bb3400640 1 -- 172.21.15.86:0/618401328 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8bac0592f0 con 0x7f8ba0005220 2024-03-22T04:21:20.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.526+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bbc06c460 con 0x7f8ba0005220 2024-03-22T04:21:20.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.526+0000 7f8bb3400640 1 -- 172.21.15.86:0/618401328 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+17085 (secure 0 0 0) 0x7f8bac058bf0 con 0x7f8ba0005220 2024-03-22T04:21:20.529 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:20.529 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:21:12.563915+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/3165266972":"2024-03-23T04:21:12.563892+0000","172.21.15.86:6835/3165266972":"2024-03-23T04:21:12.563892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b98067b20 msgr2=0x7f8b98069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b98067b20 0x7f8b98069fe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0b6430 tx=0x7f8ba4009040 comp rx=0 tx=0).stop 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 msgr2=0x7f8bbc0b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8bbc0b1510 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8bac012490 tx=0x7f8bac012950 comp rx=0 tx=0).stop 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 shutdown_connections 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8b98067b20 0x7f8b98069fe0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc0b2170 0x7f8bbc15bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15dd50 0x7f8bbc15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 --2- 172.21.15.86:0/618401328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ba0005220 0x7f8bbc0b1510 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 >> 172.21.15.86:0/618401328 conn(0x7f8bbc06c770 msgr2=0x7f8bbc1602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 shutdown_connections 2024-03-22T04:21:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.530+0000 7f8bc2800640 1 -- 172.21.15.86:0/618401328 wait complete. 2024-03-22T04:21:20.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:20:55.270181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:20:56.532178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:21:20.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:21:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e53800640 1 Processor -- start 2024-03-22T04:21:20.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e53800640 1 -- start start 2024-03-22T04:21:20.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e53800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e4c0b2370 0x7f8e4c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e53800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4c15f180 con 0x7f8e4c15b3c0 2024-03-22T04:21:20.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e53800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4c15f460 con 0x7f8e4c15afe0 2024-03-22T04:21:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e53800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c15f740 con 0x7f8e4c0b2370 2024-03-22T04:21:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e4be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e4c0b2370 0x7f8e4c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e51800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8e4c15b3c0 0x7f8e4c0717e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60670/0 (socket says 172.21.15.86:60670) 2024-03-22T04:21:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e51800640 1 -- 172.21.15.86:0/91575813 learned_addr learned my addr 172.21.15.86:0/91575813 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e4be00640 1 -- 172.21.15.86:0/91575813 >> v1:172.21.15.196:6789/0 conn(0x7f8e4c15afe0 legacy=0x7f8e4c0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e4be00640 1 -- 172.21.15.86:0/91575813 >> v1:172.21.15.86:6789/0 conn(0x7f8e4c15b3c0 legacy=0x7f8e4c0717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e4b400640 1 -- 172.21.15.86:0/91575813 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292487717 0 0) 0x7f8e4c15f180 con 0x7f8e4c15b3c0 2024-03-22T04:21:20.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.690+0000 7f8e4be00640 1 -- 172.21.15.86:0/91575813 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c15fa20 con 0x7f8e4c0b2370 2024-03-22T04:21:20.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e4be00640 1 --2- 172.21.15.86:0/91575813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c0b2370 0x7f8e4c0cdae0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8e380097c0 tx=0x7f8e4c072700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe82923862950508 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:20.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e4b400640 1 -- 172.21.15.86:0/91575813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380ab680 con 0x7f8e4c0b2370 2024-03-22T04:21:20.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e4b400640 1 -- 172.21.15.86:0/91575813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e380b0070 con 0x7f8e4c0b2370 2024-03-22T04:21:20.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e4b400640 1 -- 172.21.15.86:0/91575813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380aba40 con 0x7f8e4c0b2370 2024-03-22T04:21:20.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e53800640 1 -- 172.21.15.86:0/91575813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c0b2370 msgr2=0x7f8e4c0cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e53800640 1 --2- 172.21.15.86:0/91575813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c0b2370 0x7f8e4c0cdae0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8e380097c0 tx=0x7f8e4c072700 comp rx=0 tx=0).stop 2024-03-22T04:21:20.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e53800640 1 -- 172.21.15.86:0/91575813 shutdown_connections 2024-03-22T04:21:20.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e53800640 1 --2- 172.21.15.86:0/91575813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c0b2370 0x7f8e4c0cdae0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e53800640 1 -- 172.21.15.86:0/91575813 >> 172.21.15.86:0/91575813 conn(0x7f8e4c06b660 msgr2=0x7f8e4c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:20.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e53800640 1 -- 172.21.15.86:0/91575813 shutdown_connections 2024-03-22T04:21:20.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.694+0000 7f8e53800640 1 -- 172.21.15.86:0/91575813 wait complete. 2024-03-22T04:21:20.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 Processor -- start 2024-03-22T04:21:20.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 -- start start 2024-03-22T04:21:20.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c0b2370 0x7f8e4c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c15afe0 0x7f8e4c0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4c15b3c0 0x7f8e4c1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c0cfaa0 con 0x7f8e4c15afe0 2024-03-22T04:21:20.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c0cf920 con 0x7f8e4c0b2370 2024-03-22T04:21:20.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c1691e0 con 0x7f8e4c15b3c0 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c15afe0 0x7f8e4c0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c15afe0 0x7f8e4c0c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58216/0 (socket says 172.21.15.86:58216) 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 -- 172.21.15.86:0/1952299945 learned_addr learned my addr 172.21.15.86:0/1952299945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 -- 172.21.15.86:0/1952299945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4c15b3c0 msgr2=0x7f8e4c1416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4c15b3c0 0x7f8e4c1416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e50e00640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c0b2370 0x7f8e4c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 -- 172.21.15.86:0/1952299945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c0b2370 msgr2=0x7f8e4c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c0b2370 0x7f8e4c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 -- 172.21.15.86:0/1952299945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c14f400 con 0x7f8e4c15afe0 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e50e00640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c0b2370 0x7f8e4c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:20.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e4be00640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c15afe0 0x7f8e4c0c9ec0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8e380ad040 tx=0x7f8e38019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:20.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e48c00640 1 -- 172.21.15.86:0/1952299945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38004230 con 0x7f8e4c15afe0 2024-03-22T04:21:20.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e48c00640 1 -- 172.21.15.86:0/1952299945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e380a7070 con 0x7f8e4c15afe0 2024-03-22T04:21:20.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e48c00640 1 -- 172.21.15.86:0/1952299945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003620 con 0x7f8e4c15afe0 2024-03-22T04:21:20.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c15d460 con 0x7f8e4c15afe0 2024-03-22T04:21:20.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c15da20 con 0x7f8e4c15afe0 2024-03-22T04:21:20.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.698+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c075220 con 0x7f8e4c15afe0 2024-03-22T04:21:20.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.702+0000 7f8e48c00640 1 -- 172.21.15.86:0/1952299945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8e380037c0 con 0x7f8e4c15afe0 2024-03-22T04:21:20.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.702+0000 7f8e48c00640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e2c067b50 0x7f8e2c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:20.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.702+0000 7f8e48c00640 1 -- 172.21.15.86:0/1952299945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f8e38120790 con 0x7f8e4c15afe0 2024-03-22T04:21:20.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.702+0000 7f8e50e00640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e2c067b50 0x7f8e2c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:20.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.702+0000 7f8e50e00640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e2c067b50 0x7f8e2c06a010 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8e40003e40 tx=0x7f8e40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:20.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.710+0000 7f8e48c00640 1 -- 172.21.15.86:0/1952299945 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8e380c6070 con 0x7f8e4c15afe0 2024-03-22T04:21:20.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.874+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e4c072ef0 con 0x7f8e4c15afe0 2024-03-22T04:21:20.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.874+0000 7f8e48c00640 1 -- 172.21.15.86:0/1952299945 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) ==== 93+0+10 (secure 0 0 0) 0x7f8e38016050 con 0x7f8e4c15afe0 2024-03-22T04:21:20.877 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e2c067b50 msgr2=0x7f8e2c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e2c067b50 0x7f8e2c06a010 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8e40003e40 tx=0x7f8e40009040 comp rx=0 tx=0).stop 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c15afe0 msgr2=0x7f8e4c0c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c15afe0 0x7f8e4c0c9ec0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8e380ad040 tx=0x7f8e38019200 comp rx=0 tx=0).stop 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 shutdown_connections 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8e2c067b50 0x7f8e2c06a010 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4c15b3c0 0x7f8e4c1416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e4c15afe0 0x7f8e4c0c9ec0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 --2- 172.21.15.86:0/1952299945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e4c0b2370 0x7f8e4c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 >> 172.21.15.86:0/1952299945 conn(0x7f8e4c06b660 msgr2=0x7f8e4c0b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 shutdown_connections 2024-03-22T04:21:20.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:20.878+0000 7f8e53800640 1 -- 172.21.15.86:0/1952299945 wait complete. 2024-03-22T04:21:20.904 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:21:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac8a00640 1 Processor -- start 2024-03-22T04:21:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac8a00640 1 -- start start 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac8a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4ac4147aa0 0x7f4ac40d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac8a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac4073350 con 0x7f4ac4147510 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac8a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac40cd950 con 0x7f4ac406bd70 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac8a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac41478f0 con 0x7f4ac4147aa0 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac2800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4ac4147510 0x7f4ac406b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60684/0 (socket says 172.21.15.86:60684) 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac2800640 1 -- 172.21.15.86:0/3987855809 learned_addr learned my addr 172.21.15.86:0/3987855809 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac3200640 1 --2- 172.21.15.86:0/3987855809 >> v2:172.21.15.196:3300/0 conn(0x7f4ac4147aa0 0x7f4ac40d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac1400640 1 -- 172.21.15.86:0/3987855809 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3797264995 0 0) 0x7f4ac4073350 con 0x7f4ac4147510 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac1400640 1 -- 172.21.15.86:0/3987855809 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4aa8003530 con 0x7f4ac4147510 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac3200640 1 -- 172.21.15.86:0/3987855809 >> v1:172.21.15.196:6789/0 conn(0x7f4ac406bd70 legacy=0x7f4ac406fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac1400640 1 -- 172.21.15.86:0/3987855809 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1466773855 0 0) 0x7f4aa8003530 con 0x7f4ac4147510 2024-03-22T04:21:21.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.366+0000 7f4ac3200640 1 -- 172.21.15.86:0/3987855809 >> v1:172.21.15.86:6789/0 conn(0x7f4ac4147510 legacy=0x7f4ac406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac3200640 1 -- 172.21.15.86:0/3987855809 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac40d1290 con 0x7f4ac4147aa0 2024-03-22T04:21:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac3200640 1 --2- 172.21.15.86:0/3987855809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 0x7f4ac40d0c00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9c75e51f8ee4de server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac1400640 1 -- 172.21.15.86:0/3987855809 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0b2070 con 0x7f4ac4147aa0 2024-03-22T04:21:21.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac1400640 1 -- 172.21.15.86:0/3987855809 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4abc0a55c0 con 0x7f4ac4147aa0 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac1400640 1 -- 172.21.15.86:0/3987855809 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4001680 con 0x7f4ac4147aa0 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac8a00640 1 -- 172.21.15.86:0/3987855809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 msgr2=0x7f4ac40d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac8a00640 1 --2- 172.21.15.86:0/3987855809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 0x7f4ac40d0c00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc01c890 comp rx=0 tx=0).stop 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac8a00640 1 -- 172.21.15.86:0/3987855809 shutdown_connections 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac8a00640 1 --2- 172.21.15.86:0/3987855809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 0x7f4ac40d0c00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac8a00640 1 -- 172.21.15.86:0/3987855809 >> 172.21.15.86:0/3987855809 conn(0x7f4ac406e7e0 msgr2=0x7f4ac406ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac8a00640 1 -- 172.21.15.86:0/3987855809 shutdown_connections 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.370+0000 7f4ac8a00640 1 -- 172.21.15.86:0/3987855809 wait complete. 2024-03-22T04:21:21.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac8a00640 1 Processor -- start 2024-03-22T04:21:21.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac8a00640 1 -- start start 2024-03-22T04:21:21.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac8a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac406bd70 0x7f4ac415cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac4147510 0x7f4ac4160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac8a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 0x7f4ac4142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac8a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac416c3b0 con 0x7f4ac4147510 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac8a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac416c230 con 0x7f4ac4147aa0 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac8a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac416c530 con 0x7f4ac406bd70 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac4147510 0x7f4ac4160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac4147510 0x7f4ac4160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58224/0 (socket says 172.21.15.86:58224) 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 -- 172.21.15.86:0/295370564 learned_addr learned my addr 172.21.15.86:0/295370564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac2800640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac406bd70 0x7f4ac415cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac3200640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 0x7f4ac4142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 -- 172.21.15.86:0/295370564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac406bd70 msgr2=0x7f4ac415cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac406bd70 0x7f4ac415cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 -- 172.21.15.86:0/295370564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 msgr2=0x7f4ac4142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 0x7f4ac4142640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 -- 172.21.15.86:0/295370564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac415e3c0 con 0x7f4ac4147510 2024-03-22T04:21:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4ac1e00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac4147510 0x7f4ac4160c60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000c3f0 tx=0x7f4ab000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.374+0000 7f4abaa00640 1 -- 172.21.15.86:0/295370564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0017070 con 0x7f4ac4147510 2024-03-22T04:21:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4abaa00640 1 -- 172.21.15.86:0/295370564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab00052d0 con 0x7f4ac4147510 2024-03-22T04:21:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac415e6a0 con 0x7f4ac4147510 2024-03-22T04:21:21.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4abaa00640 1 -- 172.21.15.86:0/295370564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0003840 con 0x7f4ac4147510 2024-03-22T04:21:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac415ec60 con 0x7f4ac4147510 2024-03-22T04:21:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac4075220 con 0x7f4ac4147510 2024-03-22T04:21:21.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4abaa00640 1 -- 172.21.15.86:0/295370564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4ab001a070 con 0x7f4ac4147510 2024-03-22T04:21:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4abaa00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4aa4067b50 0x7f4aa406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4abaa00640 1 -- 172.21.15.86:0/295370564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f4ab0085400 con 0x7f4ac4147510 2024-03-22T04:21:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4ac2800640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4aa4067b50 0x7f4aa406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.378+0000 7f4ac2800640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4aa4067b50 0x7f4aa406a010 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40011f0 tx=0x7f4ab4001700 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:21.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.382+0000 7f4abaa00640 1 -- 172.21.15.86:0/295370564 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4ab0005480 con 0x7f4ac4147510 2024-03-22T04:21:21.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.538+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4ac4072b40 con 0x7f4ac4147510 2024-03-22T04:21:21.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.538+0000 7f4abaa00640 1 -- 172.21.15.86:0/295370564 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) ==== 104+0+11 (secure 0 0 0) 0x7f4ab0055550 con 0x7f4ac4147510 2024-03-22T04:21:21.541 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:21:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4aa4067b50 msgr2=0x7f4aa406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4aa4067b50 0x7f4aa406a010 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40011f0 tx=0x7f4ab4001700 comp rx=0 tx=0).stop 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac4147510 msgr2=0x7f4ac4160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac4147510 0x7f4ac4160c60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000c3f0 tx=0x7f4ab000c8b0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 shutdown_connections 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4aa4067b50 0x7f4aa406a010 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4147aa0 0x7f4ac4142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac4147510 0x7f4ac4160c60 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 --2- 172.21.15.86:0/295370564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac406bd70 0x7f4ac415cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 >> 172.21.15.86:0/295370564 conn(0x7f4ac406e7e0 msgr2=0x7f4ac40cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 shutdown_connections 2024-03-22T04:21:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.542+0000 7f4ac8a00640 1 -- 172.21.15.86:0/295370564 wait complete. 2024-03-22T04:21:21.562 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:21:21.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6c05e00640 1 Processor -- start 2024-03-22T04:21:21.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6c05e00640 1 -- start start 2024-03-22T04:21:21.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6c05e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c000d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6c05e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c00141d30 con 0x7f6c0013f620 2024-03-22T04:21:21.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6c05e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c00141fd0 con 0x7f6c0013f240 2024-03-22T04:21:21.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6c05e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6c001422b0 con 0x7f6c000cf250 2024-03-22T04:21:21.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6bf7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6c0013f620 0x7f6c000cfd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60700/0 (socket says 172.21.15.86:60700) 2024-03-22T04:21:21.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6bf7e00640 1 -- 172.21.15.86:0/158863893 learned_addr learned my addr 172.21.15.86:0/158863893 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:21.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6bff400640 1 --2- 172.21.15.86:0/158863893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c000d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.714+0000 7f6bfe000640 1 -- 172.21.15.86:0/158863893 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070386098 0 0) 0x7f6c00141d30 con 0x7f6c0013f620 2024-03-22T04:21:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bfe000640 1 -- 172.21.15.86:0/158863893 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6be4003530 con 0x7f6c0013f620 2024-03-22T04:21:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bfe000640 1 -- 172.21.15.86:0/158863893 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228339874 0 0) 0x7f6c00141fd0 con 0x7f6c0013f240 2024-03-22T04:21:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bfe000640 1 -- 172.21.15.86:0/158863893 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c00141d30 con 0x7f6c0013f240 2024-03-22T04:21:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bfe000640 1 -- 172.21.15.86:0/158863893 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 947739481 0 0) 0x7f6be4003530 con 0x7f6c0013f620 2024-03-22T04:21:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bff400640 1 -- 172.21.15.86:0/158863893 >> v1:172.21.15.196:6789/0 conn(0x7f6c0013f240 legacy=0x7f6c000ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bff400640 1 -- 172.21.15.86:0/158863893 >> v1:172.21.15.86:6789/0 conn(0x7f6c0013f620 legacy=0x7f6c000cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bff400640 1 -- 172.21.15.86:0/158863893 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c00142590 con 0x7f6c000cf250 2024-03-22T04:21:21.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bff400640 1 --2- 172.21.15.86:0/158863893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c000d0410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009dc0 tx=0x7f6bf801c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95bbbf1b5b12b595 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:21.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bfe000640 1 -- 172.21.15.86:0/158863893 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80b2070 con 0x7f6c000cf250 2024-03-22T04:21:21.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bfe000640 1 -- 172.21.15.86:0/158863893 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf80a5650 con 0x7f6c000cf250 2024-03-22T04:21:21.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.718+0000 7f6bfe000640 1 -- 172.21.15.86:0/158863893 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be80016a0 con 0x7f6c000cf250 2024-03-22T04:21:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 -- 172.21.15.86:0/158863893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 msgr2=0x7f6c000d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 --2- 172.21.15.86:0/158863893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c000d0410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009dc0 tx=0x7f6bf801c7f0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 -- 172.21.15.86:0/158863893 shutdown_connections 2024-03-22T04:21:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 --2- 172.21.15.86:0/158863893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c000d0410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 -- 172.21.15.86:0/158863893 >> 172.21.15.86:0/158863893 conn(0x7f6c000b3d60 msgr2=0x7f6c000b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 -- 172.21.15.86:0/158863893 shutdown_connections 2024-03-22T04:21:21.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 -- 172.21.15.86:0/158863893 wait complete. 2024-03-22T04:21:21.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 Processor -- start 2024-03-22T04:21:21.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 -- start start 2024-03-22T04:21:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.722+0000 7f6c05e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c00160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6c05e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c0013f240 0x7f6c000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6c05e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0013f620 0x7f6c001569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6c05e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00169e20 con 0x7f6c0013f620 2024-03-22T04:21:21.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6c05e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00169ca0 con 0x7f6c0013f240 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6c05e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6c00169fa0 con 0x7f6c000cf250 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0013f620 0x7f6c001569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0013f620 0x7f6c001569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58232/0 (socket says 172.21.15.86:58232) 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 -- 172.21.15.86:0/2646887619 learned_addr learned my addr 172.21.15.86:0/2646887619 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bf7e00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c0013f240 0x7f6c000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bfea00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c00160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 -- 172.21.15.86:0/2646887619 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 msgr2=0x7f6c00160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c00160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 -- 172.21.15.86:0/2646887619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c0013f240 msgr2=0x7f6c000b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c0013f240 0x7f6c000b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 -- 172.21.15.86:0/2646887619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c000b5390 con 0x7f6c0013f620 2024-03-22T04:21:21.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bff400640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0013f620 0x7f6c001569d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f6bf801ccd0 tx=0x7f6bf8019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bf7400640 1 -- 172.21.15.86:0/2646887619 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80b2050 con 0x7f6c0013f620 2024-03-22T04:21:21.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bf7400640 1 -- 172.21.15.86:0/2646887619 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf80be070 con 0x7f6c0013f620 2024-03-22T04:21:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c000b5630 con 0x7f6c0013f620 2024-03-22T04:21:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bf7400640 1 -- 172.21.15.86:0/2646887619 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80ada50 con 0x7f6c0013f620 2024-03-22T04:21:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c000b5bf0 con 0x7f6c0013f620 2024-03-22T04:21:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c00075220 con 0x7f6c0013f620 2024-03-22T04:21:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.726+0000 7f6bf7400640 1 -- 172.21.15.86:0/2646887619 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6bf80adbf0 con 0x7f6c0013f620 2024-03-22T04:21:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.730+0000 7f6bf7400640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6bdc067b20 0x7f6bdc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.730+0000 7f6bf7400640 1 -- 172.21.15.86:0/2646887619 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f6bf8121450 con 0x7f6c0013f620 2024-03-22T04:21:21.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.730+0000 7f6bfea00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6bdc067b20 0x7f6bdc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:21.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.730+0000 7f6bfea00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6bdc067b20 0x7f6bdc069fe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00011f0 tx=0x7f6bf0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:21.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.734+0000 7f6bf7400640 1 -- 172.21.15.86:0/2646887619 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6bf80164a0 con 0x7f6c0013f620 2024-03-22T04:21:21.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c000b1890 con 0x7f6c0013f620 2024-03-22T04:21:21.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6bf7400640 1 -- 172.21.15.86:0/2646887619 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) ==== 100+0+11 (secure 0 0 0) 0x7f6bf80ab340 con 0x7f6c0013f620 2024-03-22T04:21:21.891 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:21:21.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6bdc067b20 msgr2=0x7f6bdc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6bdc067b20 0x7f6bdc069fe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00011f0 tx=0x7f6bf0009040 comp rx=0 tx=0).stop 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0013f620 msgr2=0x7f6c001569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0013f620 0x7f6c001569d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f6bf801ccd0 tx=0x7f6bf8019590 comp rx=0 tx=0).stop 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 shutdown_connections 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6bdc067b20 0x7f6bdc069fe0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c0013f240 0x7f6c000b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0013f620 0x7f6c001569d0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 --2- 172.21.15.86:0/2646887619 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c000cf250 0x7f6c00160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 >> 172.21.15.86:0/2646887619 conn(0x7f6c000b3d60 msgr2=0x7f6c000d25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 shutdown_connections 2024-03-22T04:21:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:21.890+0000 7f6c05e00640 1 -- 172.21.15.86:0/2646887619 wait complete. 2024-03-22T04:21:21.915 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:21:22.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.050+0000 7f982ee00640 1 Processor -- start 2024-03-22T04:21:22.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.050+0000 7f982ee00640 1 -- start start 2024-03-22T04:21:22.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f982ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98280d2400 0x7f98280cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f982ee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98280d29e0 con 0x7f9828145d50 2024-03-22T04:21:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f982ee00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98280d2cc0 con 0x7f98280ceb80 2024-03-22T04:21:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f982ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98280cf520 con 0x7f98280d2400 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f9827e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9828145d50 0x7f98280b5140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60702/0 (socket says 172.21.15.86:60702) 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f9827e00640 1 -- 172.21.15.86:0/4068925732 learned_addr learned my addr 172.21.15.86:0/4068925732 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f982ce00640 1 --2- 172.21.15.86:0/4068925732 >> v2:172.21.15.86:3300/0 conn(0x7f98280d2400 0x7f98280cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f9827400640 1 -- 172.21.15.86:0/4068925732 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437474286 0 0) 0x7f98280d29e0 con 0x7f9828145d50 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f9827400640 1 -- 172.21.15.86:0/4068925732 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f980c003530 con 0x7f9828145d50 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f9827400640 1 -- 172.21.15.86:0/4068925732 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 376407408 0 0) 0x7f98280d2cc0 con 0x7f98280ceb80 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f9827400640 1 -- 172.21.15.86:0/4068925732 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98280d29e0 con 0x7f98280ceb80 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f982ce00640 1 -- 172.21.15.86:0/4068925732 >> v1:172.21.15.196:6789/0 conn(0x7f98280ceb80 legacy=0x7f98280d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f982ce00640 1 -- 172.21.15.86:0/4068925732 >> v1:172.21.15.86:6789/0 conn(0x7f9828145d50 legacy=0x7f98280b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.054+0000 7f982ce00640 1 -- 172.21.15.86:0/4068925732 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98280cff90 con 0x7f98280d2400 2024-03-22T04:21:22.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ce00640 1 --2- 172.21.15.86:0/4068925732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280d2400 0x7f98280cefe0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9820009d90 tx=0x7f982001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86b8e765901a2b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:22.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f9827400640 1 -- 172.21.15.86:0/4068925732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200b2070 con 0x7f98280d2400 2024-03-22T04:21:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f9827400640 1 -- 172.21.15.86:0/4068925732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98200a57f0 con 0x7f98280d2400 2024-03-22T04:21:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f9827400640 1 -- 172.21.15.86:0/4068925732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200a5bc0 con 0x7f98280d2400 2024-03-22T04:21:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 -- 172.21.15.86:0/4068925732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280d2400 msgr2=0x7f98280cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 --2- 172.21.15.86:0/4068925732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280d2400 0x7f98280cefe0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9820009d90 tx=0x7f982001c930 comp rx=0 tx=0).stop 2024-03-22T04:21:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 -- 172.21.15.86:0/4068925732 shutdown_connections 2024-03-22T04:21:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 --2- 172.21.15.86:0/4068925732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280d2400 0x7f98280cefe0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 -- 172.21.15.86:0/4068925732 >> 172.21.15.86:0/4068925732 conn(0x7f98280b3e40 msgr2=0x7f98280b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 -- 172.21.15.86:0/4068925732 shutdown_connections 2024-03-22T04:21:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 -- 172.21.15.86:0/4068925732 wait complete. 2024-03-22T04:21:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 Processor -- start 2024-03-22T04:21:22.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 -- start start 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280ceb80 0x7f98280b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.058+0000 7f982ee00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98280d2400 0x7f98281407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f982ee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9828145d50 0x7f98281620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f982ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98280d1370 con 0x7f98280ceb80 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f982ee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98280d11f0 con 0x7f9828145d50 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f982ee00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f982816a320 con 0x7f98280d2400 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280ceb80 0x7f98280b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280ceb80 0x7f98280b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58242/0 (socket says 172.21.15.86:58242) 2024-03-22T04:21:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 -- 172.21.15.86:0/3587755510 learned_addr learned my addr 172.21.15.86:0/3587755510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f981fe00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98280d2400 0x7f98281407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f982ce00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9828145d50 0x7f98281620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 -- 172.21.15.86:0/3587755510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98280d2400 msgr2=0x7f98281407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98280d2400 0x7f98281407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 -- 172.21.15.86:0/3587755510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9828145d50 msgr2=0x7f98281620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9828145d50 0x7f98281620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 -- 172.21.15.86:0/3587755510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9828144f40 con 0x7f98280ceb80 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9827e00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280ceb80 0x7f98280b4cf0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f981800c500 tx=0x7f981800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9824c00640 1 -- 172.21.15.86:0/3587755510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818017070 con 0x7f98280ceb80 2024-03-22T04:21:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9824c00640 1 -- 172.21.15.86:0/3587755510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9818005cf0 con 0x7f98280ceb80 2024-03-22T04:21:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9828145220 con 0x7f98280ceb80 2024-03-22T04:21:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9824c00640 1 -- 172.21.15.86:0/3587755510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98180072f0 con 0x7f98280ceb80 2024-03-22T04:21:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9828145500 con 0x7f98280ceb80 2024-03-22T04:21:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9828075220 con 0x7f98280ceb80 2024-03-22T04:21:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.062+0000 7f9824c00640 1 -- 172.21.15.86:0/3587755510 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f981801a070 con 0x7f98280ceb80 2024-03-22T04:21:22.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.066+0000 7f9824c00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9804067b20 0x7f9804069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.066+0000 7f9824c00640 1 -- 172.21.15.86:0/3587755510 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f9818085490 con 0x7f98280ceb80 2024-03-22T04:21:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.066+0000 7f981fe00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9804067b20 0x7f9804069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.066+0000 7f981fe00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9804067b20 0x7f9804069fe0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f98281682e0 tx=0x7f9810009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.070+0000 7f9824c00640 1 -- 172.21.15.86:0/3587755510 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9818055660 con 0x7f98280ceb80 2024-03-22T04:21:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9828068800 con 0x7f98280ceb80 2024-03-22T04:21:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f9824c00640 1 -- 172.21.15.86:0/3587755510 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v125) ==== 80+0+122 (secure 0 0 0) 0x7f9818059700 con 0x7f98280ceb80 2024-03-22T04:21:22.264 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/3165266972 2024-03-23T04:21:12.563892+0000 2024-03-22T04:21:22.264 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/3165266972 2024-03-23T04:21:12.563892+0000 2024-03-22T04:21:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-03-22T04:21:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9804067b20 msgr2=0x7f9804069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9804067b20 0x7f9804069fe0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f98281682e0 tx=0x7f9810009040 comp rx=0 tx=0).stop 2024-03-22T04:21:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280ceb80 msgr2=0x7f98280b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280ceb80 0x7f98280b4cf0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f981800c500 tx=0x7f981800c9c0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 shutdown_connections 2024-03-22T04:21:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9804067b20 0x7f9804069fe0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9828145d50 0x7f98281620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98280d2400 0x7f98281407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 --2- 172.21.15.86:0/3587755510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98280ceb80 0x7f98280b4cf0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 >> 172.21.15.86:0/3587755510 conn(0x7f98280b3e40 msgr2=0x7f98280b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:22.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 shutdown_connections 2024-03-22T04:21:22.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.262+0000 7f982ee00640 1 -- 172.21.15.86:0/3587755510 wait complete. 2024-03-22T04:21:22.292 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:21:22.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:22.293 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:22.323 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:21:22.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:22.324 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:21:22.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f2657200640 1 Processor -- start 2024-03-22T04:21:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f2657200640 1 -- start start 2024-03-22T04:21:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f2657200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f26500b5700 0x7f2650169310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f2657200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265006e0a0 con 0x7f2650147c70 2024-03-22T04:21:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f2657200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265006d490 con 0x7f2650161fc0 2024-03-22T04:21:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f2657200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2650148050 con 0x7f26500b5700 2024-03-22T04:21:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f264fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2650147c70 0x7f265006cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60706/0 (socket says 172.21.15.86:60706) 2024-03-22T04:21:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f264fe00640 1 -- 172.21.15.86:0/249838397 learned_addr learned my addr 172.21.15.86:0/249838397 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340382113 0 0) 0x7f265006e0a0 con 0x7f2650147c70 2024-03-22T04:21:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f2655200640 1 --2- 172.21.15.86:0/249838397 >> v2:172.21.15.196:3300/0 conn(0x7f26500b5700 0x7f2650169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.562+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2634003530 con 0x7f2650147c70 2024-03-22T04:21:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3566181713 0 0) 0x7f2634003530 con 0x7f2650147c70 2024-03-22T04:21:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 >> v2:172.21.15.196:3300/0 conn(0x7f26500b5700 msgr2=0x7f2650169310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 --2- 172.21.15.86:0/249838397 >> v2:172.21.15.196:3300/0 conn(0x7f26500b5700 0x7f2650169310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 >> v1:172.21.15.196:6789/0 conn(0x7f2650161fc0 legacy=0x7f26500b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265006d770 con 0x7f2650147c70 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f2640001680 con 0x7f2650147c70 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 >> v1:172.21.15.86:6789/0 conn(0x7f2650147c70 legacy=0x7f265006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26340039c0 0x7f2634003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004810 0x7f2634004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26340051d0 0x7f263400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2650148050 con 0x7f2634004810 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f263400c070 con 0x7f26340039c0 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f263400c270 con 0x7f26340051d0 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2647e00640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004810 0x7f2634004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f264fe00640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26340039c0 0x7f2634003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2655200640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26340051d0 0x7f263400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2647e00640 1 -- 172.21.15.86:0/249838397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26340051d0 msgr2=0x7f263400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:22.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2647e00640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26340051d0 0x7f263400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2647e00640 1 -- 172.21.15.86:0/249838397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26340039c0 msgr2=0x7f2634003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:22.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2647e00640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26340039c0 0x7f2634003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2647e00640 1 -- 172.21.15.86:0/249838397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2634003530 con 0x7f2634004810 2024-03-22T04:21:22.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2655200640 1 -- 172.21.15.86:0/249838397 reap_dead start 2024-03-22T04:21:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.566+0000 7f2647e00640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004810 0x7f2634004c90 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f2638001780 tx=0x7f263801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30e1d057af0f8f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26380013a0 con 0x7f2634004810 2024-03-22T04:21:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26380052f0 con 0x7f2634004810 2024-03-22T04:21:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f264f400640 1 -- 172.21.15.86:0/249838397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638003840 con 0x7f2634004810 2024-03-22T04:21:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- 172.21.15.86:0/249838397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004810 msgr2=0x7f2634004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004810 0x7f2634004c90 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f2638001780 tx=0x7f263801c400 comp rx=0 tx=0).stop 2024-03-22T04:21:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- 172.21.15.86:0/249838397 shutdown_connections 2024-03-22T04:21:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 --2- 172.21.15.86:0/249838397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004810 0x7f2634004c90 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- 172.21.15.86:0/249838397 >> 172.21.15.86:0/249838397 conn(0x7f26500b3cf0 msgr2=0x7f26500b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- 172.21.15.86:0/249838397 shutdown_connections 2024-03-22T04:21:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- 172.21.15.86:0/249838397 wait complete. 2024-03-22T04:21:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 Processor -- start 2024-03-22T04:21:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- start start 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2634004810 0x7f265013e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265014fc90 0x7f265013be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26501442f0 0x7f2650144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26500b4ff0 con 0x7f265014fc90 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f265006dd30 con 0x7f26501442f0 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2657200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f26500b5170 con 0x7f2634004810 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2647e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265014fc90 0x7f265013be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2647e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265014fc90 0x7f265013be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58266/0 (socket says 172.21.15.86:58266) 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.570+0000 7f2647e00640 1 -- 172.21.15.86:0/1993388146 learned_addr learned my addr 172.21.15.86:0/1993388146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2647e00640 1 -- 172.21.15.86:0/1993388146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2634004810 msgr2=0x7f265013e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2647e00640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2634004810 0x7f265013e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2647e00640 1 -- 172.21.15.86:0/1993388146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26501442f0 msgr2=0x7f2650144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2655200640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26501442f0 0x7f2650144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2647e00640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26501442f0 0x7f2650144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2647e00640 1 -- 172.21.15.86:0/1993388146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265013e820 con 0x7f265014fc90 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2655200640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26501442f0 0x7f2650144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2647e00640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265014fc90 0x7f265013be20 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f2638005040 tx=0x7f2634005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f264cc00640 1 -- 172.21.15.86:0/1993388146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638007a40 con 0x7f265014fc90 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f264cc00640 1 -- 172.21.15.86:0/1993388146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26380ba070 con 0x7f265014fc90 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265013eb00 con 0x7f265014fc90 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f264cc00640 1 -- 172.21.15.86:0/1993388146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638007250 con 0x7f265014fc90 2024-03-22T04:21:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265013f0c0 con 0x7f265014fc90 2024-03-22T04:21:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2650144cb0 con 0x7f265014fc90 2024-03-22T04:21:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f264cc00640 1 -- 172.21.15.86:0/1993388146 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f26380052f0 con 0x7f265014fc90 2024-03-22T04:21:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f264cc00640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f262c067b20 0x7f262c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f264cc00640 1 -- 172.21.15.86:0/1993388146 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f26381204e0 con 0x7f265014fc90 2024-03-22T04:21:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.574+0000 7f264fe00640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f262c067b20 0x7f262c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.578+0000 7f264fe00640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f262c067b20 0x7f262c069fe0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2640001060 tx=0x7f2640001ae0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:22.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.578+0000 7f264cc00640 1 -- 172.21.15.86:0/1993388146 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2638016110 con 0x7f265014fc90 2024-03-22T04:21:22.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.770+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2650075220 con 0x7f265014fc90 2024-03-22T04:21:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f264cc00640 1 -- 172.21.15.86:0/1993388146 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5199 (secure 0 0 0) 0x7f26380f0d00 con 0x7f265014fc90 2024-03-22T04:21:22.775 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:22.775 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:16.586211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":77801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_77801":{"gid":77801,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/4220897872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4220897872},{"type":"v1","addr":"172.21.15.86:6837","nonce":4220897872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":77801,"qdb_cluster":[77801]},"id":8}]} 2024-03-22T04:21:22.775 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 105 2024-03-22T04:21:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f262c067b20 msgr2=0x7f262c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f262c067b20 0x7f262c069fe0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2640001060 tx=0x7f2640001ae0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265014fc90 msgr2=0x7f265013be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265014fc90 0x7f265013be20 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f2638005040 tx=0x7f2634005840 comp rx=0 tx=0).stop 2024-03-22T04:21:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 shutdown_connections 2024-03-22T04:21:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f262c067b20 0x7f262c069fe0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26501442f0 0x7f2650144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265014fc90 0x7f265013be20 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 --2- 172.21.15.86:0/1993388146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2634004810 0x7f265013e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:22.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 >> 172.21.15.86:0/1993388146 conn(0x7f26500b3cf0 msgr2=0x7f265006cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:22.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 shutdown_connections 2024-03-22T04:21:22.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:22.774+0000 7f2657200640 1 -- 172.21.15.86:0/1993388146 wait complete. 2024-03-22T04:21:22.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-03-22T04:21:22.982 INFO:teuthology.orchestra.run.smithi086.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-03-22T04:21:22.983 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-03-22T04:21:22.983 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-22T04:21:23.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e92519080 1 Processor -- start 2024-03-22T04:21:23.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e92519080 1 -- start start 2024-03-22T04:21:23.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e92519080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1a481d0 0x55d1b1a475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e92519080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b1f770 0x55d1b1a47b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e92519080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1b20090 0x55d1b1b1de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e92519080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1b1a7bf60 con 0x55d1b1b1f770 2024-03-22T04:21:23.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e92519080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d1b19ee620 con 0x55d1b1a481d0 2024-03-22T04:21:23.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e92519080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d1b1a7c0e0 con 0x55d1b1b20090 2024-03-22T04:21:23.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e8ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b1f770 0x55d1b1a47b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e8ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b1f770 0x55d1b1a47b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58282/0 (socket says 172.21.15.86:58282) 2024-03-22T04:21:23.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e8ee00640 1 -- 172.21.15.86:0/622111826 learned_addr learned my addr 172.21.15.86:0/622111826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e8f800640 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1a481d0 0x55d1b1a475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.026+0000 7f7e90200640 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1b20090 0x55d1b1b1de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 -- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1b20090 msgr2=0x55d1b1b1de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1b20090 0x55d1b1b1de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 -- 172.21.15.86:0/622111826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b1f770 msgr2=0x55d1b1a47b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b1f770 0x55d1b1a47b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 -- 172.21.15.86:0/622111826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b1b1aa40 con 0x55d1b1a481d0 2024-03-22T04:21:23.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8ee00640 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b1f770 0x55d1b1a47b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:23.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1a481d0 0x55d1b1a475e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000a7b0 tx=0x7f7e8000ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3768293dd137620e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8da00640 1 -- 172.21.15.86:0/622111826 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8001a070 con 0x55d1b1a481d0 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8da00640 1 -- 172.21.15.86:0/622111826 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8000ada0 con 0x55d1b1a481d0 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8da00640 1 -- 172.21.15.86:0/622111826 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80008c30 con 0x55d1b1a481d0 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1a481d0 msgr2=0x55d1b1a475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1a481d0 0x55d1b1a475e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000a7b0 tx=0x7f7e8000ac70 comp rx=0 tx=0).stop 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- 172.21.15.86:0/622111826 shutdown_connections 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1b20090 0x55d1b1b1de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b1f770 0x55d1b1a47b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 --2- 172.21.15.86:0/622111826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1a481d0 0x55d1b1a475e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- 172.21.15.86:0/622111826 >> 172.21.15.86:0/622111826 conn(0x55d1b1a488d0 msgr2=0x55d1b1b21b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- 172.21.15.86:0/622111826 shutdown_connections 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- 172.21.15.86:0/622111826 wait complete. 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 Processor -- start 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- start start 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1a481d0 0x55d1b1b1ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1b1f770 0x55d1b1b28e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b20090 0x55d1b1b29350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1b1b1af80 con 0x55d1b1b20090 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d1b1b1ace0 con 0x55d1b1b1f770 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e92519080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d1b1b1b220 con 0x55d1b1a481d0 2024-03-22T04:21:23.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e90200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b20090 0x55d1b1b29350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1a481d0 0x55d1b1b1ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8ee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1b1f770 0x55d1b1b28e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1a481d0 0x55d1b1b1ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:47128/0 (socket says 172.21.15.86:47128) 2024-03-22T04:21:23.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.030+0000 7f7e8f800640 1 -- 172.21.15.86:0/2608891574 learned_addr learned my addr 172.21.15.86:0/2608891574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e8f800640 1 -- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1b1f770 msgr2=0x55d1b1b28e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e8f800640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1b1f770 0x55d1b1b28e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e8f800640 1 -- 172.21.15.86:0/2608891574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b20090 msgr2=0x55d1b1b29350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e8f800640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b20090 0x55d1b1b29350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e8f800640 1 -- 172.21.15.86:0/2608891574 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b1b29a70 con 0x55d1b1a481d0 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e8ee00640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1b1b1f770 0x55d1b1b28e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e90200640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b1b20090 0x55d1b1b29350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e8f800640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1a481d0 0x55d1b1b1ea80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e80014000 tx=0x7f7e80003030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e87e00640 1 -- 172.21.15.86:0/2608891574 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8001a040 con 0x55d1b1a481d0 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e87e00640 1 -- 172.21.15.86:0/2608891574 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80020430 con 0x55d1b1a481d0 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1b1b29d10 con 0x55d1b1a481d0 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e87e00640 1 -- 172.21.15.86:0/2608891574 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80020800 con 0x55d1b1a481d0 2024-03-22T04:21:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b1b23a50 con 0x55d1b1a481d0 2024-03-22T04:21:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e87e00640 1 -- 172.21.15.86:0/2608891574 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f7e800209a0 con 0x55d1b1a481d0 2024-03-22T04:21:23.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:21:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.034+0000 7f7e87e00640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e70066530 0x7f7e700689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e87e00640 1 -- 172.21.15.86:0/2608891574 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f7e800863f0 con 0x55d1b1a481d0 2024-03-22T04:21:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e8ee00640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e70066530 0x7f7e700689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55d1b1b1efc0 0x55d1b1b23ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) -- 0x55d1b1b24430 con 0x55d1b1b1efc0 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e8ee00640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e70066530 0x7f7e700689f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00baa0 tx=0x7f7e7c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e90200640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55d1b1b1efc0 0x55d1b1b23ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e90200640 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55d1b1b1efc0 0x55d1b1b23ef0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e90200640 1 -- 172.21.15.86:0/2608891574 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f7e880aa4f0 con 0x55d1b1b1efc0 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55d1b1b1efc0 msgr2=0x55d1b1b23ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x55d1b1b1efc0 0x55d1b1b23ef0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e70066530 msgr2=0x7f7e700689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7e70066530 0x7f7e700689f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00baa0 tx=0x7f7e7c00d040 comp rx=0 tx=0).stop 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1a481d0 msgr2=0x55d1b1b1ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 --2- 172.21.15.86:0/2608891574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1b1a481d0 0x55d1b1b1ea80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e80014000 tx=0x7f7e80003030 comp rx=0 tx=0).stop 2024-03-22T04:21:23.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e90200640 1 -- 172.21.15.86:0/2608891574 reap_dead start 2024-03-22T04:21:23.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 shutdown_connections 2024-03-22T04:21:23.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 >> 172.21.15.86:0/2608891574 conn(0x55d1b1a488d0 msgr2=0x55d1b1b20ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 shutdown_connections 2024-03-22T04:21:23.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.038+0000 7f7e92519080 1 -- 172.21.15.86:0/2608891574 wait complete. 2024-03-22T04:21:23.049 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-22T04:21:23.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.138+0000 7ff16e21d080 1 Processor -- start 2024-03-22T04:21:23.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.138+0000 7ff16e21d080 1 -- start start 2024-03-22T04:21:23.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b9497f0 0x55603ba24e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603b98f930 0x55603ba1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603ba1b6c0 0x55603b947320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55603ba1d030 con 0x55603b9497f0 2024-03-22T04:21:23.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55603b8ec620 con 0x55603b98f930 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55603ba1d1b0 con 0x55603ba1b6c0 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b9497f0 0x55603ba24e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b9497f0 0x55603ba24e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58312/0 (socket says 172.21.15.86:58312) 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 -- 172.21.15.86:0/3727799327 learned_addr learned my addr 172.21.15.86:0/3727799327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16be00640 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603ba1b6c0 0x55603b947320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16aa00640 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603b98f930 0x55603ba1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 -- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603ba1b6c0 msgr2=0x55603b947320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603ba1b6c0 0x55603b947320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 -- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603b98f930 msgr2=0x55603ba1ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603b98f930 0x55603ba1ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 -- 172.21.15.86:0/3727799327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55603ba1fb60 con 0x55603b9497f0 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16aa00640 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603b98f930 0x55603ba1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16b400640 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b9497f0 0x55603ba24e90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff15c018f50 tx=0x7ff15c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf96b8d224a2b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff169600640 1 -- 172.21.15.86:0/3727799327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0af040 con 0x55603b9497f0 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff169600640 1 -- 172.21.15.86:0/3727799327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15c0b9070 con 0x55603b9497f0 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff169600640 1 -- 172.21.15.86:0/3727799327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0aae50 con 0x55603b9497f0 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- 172.21.15.86:0/3727799327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b9497f0 msgr2=0x55603ba24e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b9497f0 0x55603ba24e90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff15c018f50 tx=0x7ff15c003030 comp rx=0 tx=0).stop 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- 172.21.15.86:0/3727799327 shutdown_connections 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603b98f930 0x55603ba1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603ba1b6c0 0x55603b947320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 --2- 172.21.15.86:0/3727799327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b9497f0 0x55603ba24e90 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- 172.21.15.86:0/3727799327 >> 172.21.15.86:0/3727799327 conn(0x55603b972ed0 msgr2=0x55603b977410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- 172.21.15.86:0/3727799327 shutdown_connections 2024-03-22T04:21:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- 172.21.15.86:0/3727799327 wait complete. 2024-03-22T04:21:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 Processor -- start 2024-03-22T04:21:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.142+0000 7ff16e21d080 1 -- start start 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16e21d080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603b9497f0 0x55603b98dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16e21d080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b98f930 0x55603ba19ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16e21d080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603ba1b6c0 0x55603ba1a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16e21d080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55603ba200a0 con 0x55603b98f930 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16e21d080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55603ba1fe00 con 0x55603ba1b6c0 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16e21d080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55603ba20340 con 0x55603b9497f0 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b98f930 0x55603ba19ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603b9497f0 0x55603b98dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603b9497f0 0x55603b98dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:47156/0 (socket says 172.21.15.86:47156) 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b98f930 0x55603ba19ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58314/0 (socket says 172.21.15.86:58314) 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 -- 172.21.15.86:0/3293900061 learned_addr learned my addr 172.21.15.86:0/3293900061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16be00640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603ba1b6c0 0x55603ba1a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 -- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603ba1b6c0 msgr2=0x55603ba1a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55603ba1b6c0 0x55603ba1a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 -- 172.21.15.86:0/3293900061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b98f930 msgr2=0x55603ba19ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b98f930 0x55603ba19ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 -- 172.21.15.86:0/3293900061 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55603bab8b10 con 0x55603b9497f0 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16aa00640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603b98f930 0x55603ba19ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16b400640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603b9497f0 0x55603b98dc40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0b6040 tx=0x7ff15c0180c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff163e00640 1 -- 172.21.15.86:0/3293900061 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0af040 con 0x55603b9497f0 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff163e00640 1 -- 172.21.15.86:0/3293900061 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15c0b44d0 con 0x55603b9497f0 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff163e00640 1 -- 172.21.15.86:0/3293900061 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0b48c0 con 0x55603b9497f0 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55603bab8db0 con 0x55603b9497f0 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.146+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55603bab92f0 con 0x55603b9497f0 2024-03-22T04:21:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff163e00640 1 -- 172.21.15.86:0/3293900061 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff15c0c7030 con 0x55603b9497f0 2024-03-22T04:21:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff163e00640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff14c066500 0x7ff14c0689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff163e00640 1 -- 172.21.15.86:0/3293900061 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7ff15c120040 con 0x55603b9497f0 2024-03-22T04:21:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16aa00640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff14c066500 0x7ff14c0689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55603ba16860 0x55603ba16c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) -- 0x55603ba17180 con 0x55603ba16860 2024-03-22T04:21:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16be00640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55603ba16860 0x55603ba16c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16aa00640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff14c066500 0x7ff14c0689c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c7b0 tx=0x7ff158001850 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16be00640 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55603ba16860 0x55603ba16c40 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16be00640 1 -- 172.21.15.86:0/3293900061 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7ff16400a900 con 0x55603ba16860 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55603ba16860 msgr2=0x55603ba16c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x55603ba16860 0x55603ba16c40 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff14c066500 msgr2=0x7ff14c0689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff14c066500 0x7ff14c0689c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c7b0 tx=0x7ff158001850 comp rx=0 tx=0).stop 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603b9497f0 msgr2=0x55603b98dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 --2- 172.21.15.86:0/3293900061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55603b9497f0 0x55603b98dc40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0b6040 tx=0x7ff15c0180c0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16be00640 1 -- 172.21.15.86:0/3293900061 reap_dead start 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 shutdown_connections 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 >> 172.21.15.86:0/3293900061 conn(0x55603b972ed0 msgr2=0x55603b947320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 shutdown_connections 2024-03-22T04:21:23.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.150+0000 7ff16e21d080 1 -- 172.21.15.86:0/3293900061 wait complete. 2024-03-22T04:21:23.162 INFO:teuthology.orchestra.run.smithi086.stdout:mytag123 2024-03-22T04:21:23.162 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-22T04:21:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c55f91080 1 Processor -- start 2024-03-22T04:21:23.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c55f91080 1 -- start start 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c55f91080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a67fc00 0x56534a71b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c55f91080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a697c50 0x56534a651ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c55f91080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a694e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c55f91080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56534a5f5620 con 0x56534a7244c0 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c55f91080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56534a5f9670 con 0x56534a697c50 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c55f91080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56534a698c60 con 0x56534a67fc00 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a694e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a694e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58330/0 (socket says 172.21.15.86:58330) 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53c00640 1 -- 172.21.15.86:0/1392340310 learned_addr learned my addr 172.21.15.86:0/1392340310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c52800640 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a697c50 0x56534a651ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53200640 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a67fc00 0x56534a71b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53c00640 1 -- 172.21.15.86:0/1392340310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a67fc00 msgr2=0x56534a71b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53c00640 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a67fc00 0x56534a71b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53c00640 1 -- 172.21.15.86:0/1392340310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a697c50 msgr2=0x56534a651ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53c00640 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a697c50 0x56534a651ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.254+0000 7f5c53c00640 1 -- 172.21.15.86:0/1392340310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a6907f0 con 0x56534a7244c0 2024-03-22T04:21:23.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a694e70 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c001710 tx=0x7f5c4c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29c34458fea7795 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c51400640 1 -- 172.21.15.86:0/1392340310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c017070 con 0x56534a7244c0 2024-03-22T04:21:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c51400640 1 -- 172.21.15.86:0/1392340310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c00f710 con 0x56534a7244c0 2024-03-22T04:21:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c51400640 1 -- 172.21.15.86:0/1392340310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c00fb00 con 0x56534a7244c0 2024-03-22T04:21:23.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- 172.21.15.86:0/1392340310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 msgr2=0x56534a694e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a694e70 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c001710 tx=0x7f5c4c006f90 comp rx=0 tx=0).stop 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- 172.21.15.86:0/1392340310 shutdown_connections 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a697c50 0x56534a651ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a694e70 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 --2- 172.21.15.86:0/1392340310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a67fc00 0x56534a71b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- 172.21.15.86:0/1392340310 >> 172.21.15.86:0/1392340310 conn(0x56534a67a030 msgr2=0x56534a64df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- 172.21.15.86:0/1392340310 shutdown_connections 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- 172.21.15.86:0/1392340310 wait complete. 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 Processor -- start 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- start start 2024-03-22T04:21:23.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a67fc00 0x56534a71cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a697c50 0x56534a67e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a7c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56534a690550 con 0x56534a7244c0 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56534a721430 con 0x56534a67fc00 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c55f91080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x56534a721690 con 0x56534a697c50 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a7c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a7c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58334/0 (socket says 172.21.15.86:58334) 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 -- 172.21.15.86:0/362132137 learned_addr learned my addr 172.21.15.86:0/362132137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53200640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a67fc00 0x56534a71cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c52800640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a697c50 0x56534a67e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 -- 172.21.15.86:0/362132137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a697c50 msgr2=0x56534a67e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56534a697c50 0x56534a67e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 -- 172.21.15.86:0/362132137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a67fc00 msgr2=0x56534a71cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56534a67fc00 0x56534a71cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 -- 172.21.15.86:0/362132137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a71d230 con 0x56534a7244c0 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.258+0000 7f5c53c00640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a7c1fe0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c015040 tx=0x7f5c4c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c4be00640 1 -- 172.21.15.86:0/362132137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c017030 con 0x56534a7244c0 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c4be00640 1 -- 172.21.15.86:0/362132137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c00f510 con 0x56534a7244c0 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c4be00640 1 -- 172.21.15.86:0/362132137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c011c40 con 0x56534a7244c0 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56534a71d4d0 con 0x56534a7244c0 2024-03-22T04:21:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534a71da10 con 0x56534a7244c0 2024-03-22T04:21:23.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c4be00640 1 -- 172.21.15.86:0/362132137 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5c4c01e430 con 0x56534a7244c0 2024-03-22T04:21:23.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c4be00640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5c34066530 0x7f5c340689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c4be00640 1 -- 172.21.15.86:0/362132137 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f5c4c085d40 con 0x56534a7244c0 2024-03-22T04:21:23.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.262+0000 7f5c53200640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5c34066530 0x7f5c340689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x56534a7bf860 0x56534a7bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) -- 0x56534a7c0180 con 0x56534a7bf860 2024-03-22T04:21:23.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c52800640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x56534a7bf860 0x56534a7bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c53200640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5c34066530 0x7f5c340689f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000c10 tx=0x7f5c44010040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c52800640 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x56534a7bf860 0x56534a7bfc40 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c52800640 1 -- 172.21.15.86:0/362132137 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f5c400b2040 con 0x56534a7bf860 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x56534a7bf860 msgr2=0x56534a7bfc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x56534a7bf860 0x56534a7bfc40 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5c34066530 msgr2=0x7f5c340689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5c34066530 0x7f5c340689f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000c10 tx=0x7f5c44010040 comp rx=0 tx=0).stop 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 msgr2=0x56534a7c1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 --2- 172.21.15.86:0/362132137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56534a7244c0 0x56534a7c1fe0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c015040 tx=0x7f5c4c001260 comp rx=0 tx=0).stop 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c53c00640 1 -- 172.21.15.86:0/362132137 reap_dead start 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 shutdown_connections 2024-03-22T04:21:23.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 >> 172.21.15.86:0/362132137 conn(0x56534a67a030 msgr2=0x56534a64df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 shutdown_connections 2024-03-22T04:21:23.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.266+0000 7f5c55f91080 1 -- 172.21.15.86:0/362132137 wait complete. 2024-03-22T04:21:23.274 INFO:teuthology.orchestra.run.smithi086.stdout:mytag123 2024-03-22T04:21:23.275 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-03-22T04:21:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 Processor -- start 2024-03-22T04:21:23.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- start start 2024-03-22T04:21:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573561f30c0 0x557356208840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299640 0x5573561edf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557356299a20 0x55735632e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557356168620 con 0x557356299640 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5573561f0360 con 0x5573561f30c0 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x557356295c90 con 0x557356299a20 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299640 0x5573561edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299640 0x5573561edf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58342/0 (socket says 172.21.15.86:58342) 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 -- 172.21.15.86:0/145580752 learned_addr learned my addr 172.21.15.86:0/145580752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8ea00640 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557356299a20 0x55735632e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8e000640 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573561f30c0 0x557356208840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 -- 172.21.15.86:0/145580752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557356299a20 msgr2=0x55735632e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557356299a20 0x55735632e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 -- 172.21.15.86:0/145580752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573561f30c0 msgr2=0x557356208840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573561f30c0 0x557356208840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 -- 172.21.15.86:0/145580752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735629c790 con 0x557356299640 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b8d600640 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299640 0x5573561edf10 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009880 tx=0x7f6b7c005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2faa767c02204f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b87e00640 1 -- 172.21.15.86:0/145580752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c019070 con 0x557356299640 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b87e00640 1 -- 172.21.15.86:0/145580752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7c008da0 con 0x557356299640 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b87e00640 1 -- 172.21.15.86:0/145580752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0146a0 con 0x557356299640 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- 172.21.15.86:0/145580752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299640 msgr2=0x5573561edf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299640 0x5573561edf10 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009880 tx=0x7f6b7c005a10 comp rx=0 tx=0).stop 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- 172.21.15.86:0/145580752 shutdown_connections 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557356299a20 0x55735632e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299640 0x5573561edf10 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 --2- 172.21.15.86:0/145580752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5573561f30c0 0x557356208840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- 172.21.15.86:0/145580752 >> 172.21.15.86:0/145580752 conn(0x5573561ed030 msgr2=0x5573561c4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- 172.21.15.86:0/145580752 shutdown_connections 2024-03-22T04:21:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.322+0000 7f6b90d85080 1 -- 172.21.15.86:0/145580752 wait complete. 2024-03-22T04:21:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 Processor -- start 2024-03-22T04:21:23.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 -- start start 2024-03-22T04:21:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5573561f30c0 0x557356293ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557356299640 0x55735629ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299a20 0x557356296080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55735629ccd0 con 0x557356299a20 2024-03-22T04:21:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55735629ca30 con 0x557356299640 2024-03-22T04:21:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55735629c4c0 con 0x5573561f30c0 2024-03-22T04:21:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299a20 0x557356296080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299a20 0x557356296080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58354/0 (socket says 172.21.15.86:58354) 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557356299640 0x55735629ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 -- 172.21.15.86:0/3096415269 learned_addr learned my addr 172.21.15.86:0/3096415269 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8e000640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5573561f30c0 0x557356293ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 -- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5573561f30c0 msgr2=0x557356293ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5573561f30c0 0x557356293ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 -- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557356299640 msgr2=0x55735629ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557356299640 0x55735629ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 -- 172.21.15.86:0/3096415269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735629b750 con 0x557356299a20 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8d600640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557356299640 0x55735629ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b8ea00640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299a20 0x557356296080 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f6b88002210 tx=0x7f6b8800d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b86a00640 1 -- 172.21.15.86:0/3096415269 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88018070 con 0x557356299a20 2024-03-22T04:21:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b86a00640 1 -- 172.21.15.86:0/3096415269 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b880060d0 con 0x557356299a20 2024-03-22T04:21:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55735629b9f0 con 0x557356299a20 2024-03-22T04:21:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b86a00640 1 -- 172.21.15.86:0/3096415269 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88004510 con 0x557356299a20 2024-03-22T04:21:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.326+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557356291ce0 con 0x557356299a20 2024-03-22T04:21:23.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b86a00640 1 -- 172.21.15.86:0/3096415269 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6b8801b070 con 0x557356299a20 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b86a00640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b70066500 0x7f6b700689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b86a00640 1 -- 172.21.15.86:0/3096415269 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f6b88084370 con 0x557356299a20 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b8e000640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b70066500 0x7f6b700689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55735629b2a0 0x557356292690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) -- 0x557356292bd0 con 0x55735629b2a0 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b8d600640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55735629b2a0 0x557356292690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b8e000640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b70066500 0x7f6b700689c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000a500 tx=0x7f6b800a5000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b8d600640 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55735629b2a0 0x557356292690 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b8d600640 1 -- 172.21.15.86:0/3096415269 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f6b7c014480 con 0x55735629b2a0 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55735629b2a0 msgr2=0x557356292690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x55735629b2a0 0x557356292690 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b70066500 msgr2=0x7f6b700689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b70066500 0x7f6b700689c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000a500 tx=0x7f6b800a5000 comp rx=0 tx=0).stop 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299a20 msgr2=0x557356296080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 --2- 172.21.15.86:0/3096415269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557356299a20 0x557356296080 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f6b88002210 tx=0x7f6b8800d5f0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b8ea00640 1 -- 172.21.15.86:0/3096415269 reap_dead start 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 shutdown_connections 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.330+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 >> 172.21.15.86:0/3096415269 conn(0x5573561ed030 msgr2=0x5573561edf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.334+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 shutdown_connections 2024-03-22T04:21:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.334+0000 7f6b90d85080 1 -- 172.21.15.86:0/3096415269 wait complete. 2024-03-22T04:21:23.338 INFO:teuthology.orchestra.run.smithi086.stdout:mytag123 2024-03-22T04:21:23.338 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-22T04:21:23.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35f5400640 1 Processor -- start 2024-03-22T04:21:23.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35f5400640 1 -- start start 2024-03-22T04:21:23.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35f5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35f013f620 0x7f35f00cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35f5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f00cf250 0x7f35f00d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35f5400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f00d0780 con 0x7f35f013f240 2024-03-22T04:21:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35f5400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35f0141d30 con 0x7f35f013f620 2024-03-22T04:21:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35f5400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f35f0141fd0 con 0x7f35f00cf250 2024-03-22T04:21:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ede00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35f013f620 0x7f35f00cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ee800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f35f013f240 0x7f35f00ceb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60726/0 (socket says 172.21.15.86:60726) 2024-03-22T04:21:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ee800640 1 -- 172.21.15.86:0/2828588422 learned_addr learned my addr 172.21.15.86:0/2828588422 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:23.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ef200640 1 --2- 172.21.15.86:0/2828588422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f00cf250 0x7f35f00d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ede00640 1 -- 172.21.15.86:0/2828588422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f00cf250 msgr2=0x7f35f00d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ede00640 1 --2- 172.21.15.86:0/2828588422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f00cf250 0x7f35f00d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ede00640 1 -- 172.21.15.86:0/2828588422 >> v1:172.21.15.86:6789/0 conn(0x7f35f013f240 legacy=0x7f35f00ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ede00640 1 -- 172.21.15.86:0/2828588422 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f01422b0 con 0x7f35f013f620 2024-03-22T04:21:23.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.478+0000 7f35ed400640 1 -- 172.21.15.86:0/2828588422 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330071493 0 0) 0x7f35f00d0780 con 0x7f35f013f240 2024-03-22T04:21:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35ede00640 1 --2- 172.21.15.86:0/2828588422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f013f620 0x7f35f00cfd00 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0097c0 tx=0x7f35dc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f77166fc8e9975e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35ed400640 1 -- 172.21.15.86:0/2828588422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0b0070 con 0x7f35f013f620 2024-03-22T04:21:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35ed400640 1 -- 172.21.15.86:0/2828588422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35dc0ab600 con 0x7f35f013f620 2024-03-22T04:21:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35ed400640 1 -- 172.21.15.86:0/2828588422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0ab9d0 con 0x7f35f013f620 2024-03-22T04:21:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35f5400640 1 -- 172.21.15.86:0/2828588422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f013f620 msgr2=0x7f35f00cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35f5400640 1 --2- 172.21.15.86:0/2828588422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f013f620 0x7f35f00cfd00 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0097c0 tx=0x7f35dc01c3e0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35f5400640 1 -- 172.21.15.86:0/2828588422 shutdown_connections 2024-03-22T04:21:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35f5400640 1 --2- 172.21.15.86:0/2828588422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f00cf250 0x7f35f00d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35f5400640 1 --2- 172.21.15.86:0/2828588422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f013f620 0x7f35f00cfd00 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35f5400640 1 -- 172.21.15.86:0/2828588422 >> 172.21.15.86:0/2828588422 conn(0x7f35f00b3d60 msgr2=0x7f35f00b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35f5400640 1 -- 172.21.15.86:0/2828588422 shutdown_connections 2024-03-22T04:21:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.482+0000 7f35f5400640 1 -- 172.21.15.86:0/2828588422 wait complete. 2024-03-22T04:21:23.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35f5400640 1 Processor -- start 2024-03-22T04:21:23.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35f5400640 1 -- start start 2024-03-22T04:21:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35f5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f00cf250 0x7f35f01f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35f5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f013f240 0x7f35f0158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35f5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f013f620 0x7f35f0158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35f5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0071c80 con 0x7f35f00cf250 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35f5400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0071b00 con 0x7f35f013f240 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35f5400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f35f0071e00 con 0x7f35f013f620 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f00cf250 0x7f35f01f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f00cf250 0x7f35f01f8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58368/0 (socket says 172.21.15.86:58368) 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 -- 172.21.15.86:0/824582410 learned_addr learned my addr 172.21.15.86:0/824582410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 -- 172.21.15.86:0/824582410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f013f620 msgr2=0x7f35f0158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f013f620 0x7f35f0158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ede00640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f013f240 0x7f35f0158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 -- 172.21.15.86:0/824582410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f013f240 msgr2=0x7f35f0158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f013f240 0x7f35f0158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 -- 172.21.15.86:0/824582410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f013d620 con 0x7f35f00cf250 2024-03-22T04:21:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ede00640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f013f240 0x7f35f0158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.486+0000 7f35ee800640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f00cf250 0x7f35f01f8b30 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f35e00011f0 tx=0x7f35e000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35e6a00640 1 -- 172.21.15.86:0/824582410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0018070 con 0x7f35f00cf250 2024-03-22T04:21:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35e6a00640 1 -- 172.21.15.86:0/824582410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e00074e0 con 0x7f35f00cf250 2024-03-22T04:21:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35e6a00640 1 -- 172.21.15.86:0/824582410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0004a10 con 0x7f35f00cf250 2024-03-22T04:21:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35f5400640 1 -- 172.21.15.86:0/824582410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f013d900 con 0x7f35f00cf250 2024-03-22T04:21:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35f5400640 1 -- 172.21.15.86:0/824582410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f013dec0 con 0x7f35f00cf250 2024-03-22T04:21:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35e6a00640 1 -- 172.21.15.86:0/824582410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f35e0004bb0 con 0x7f35f00cf250 2024-03-22T04:21:23.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35e6a00640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35d0067b20 0x7f35d0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35e6a00640 1 -- 172.21.15.86:0/824582410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f35e0085510 con 0x7f35f00cf250 2024-03-22T04:21:23.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35ede00640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35d0067b20 0x7f35d0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.490+0000 7f35ede00640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35d0067b20 0x7f35d0069fe0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0ad000 tx=0x7f35dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 Processor -- start 2024-03-22T04:21:23.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- start start 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d31360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11d318a0 0x557a11ce4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11ce4f50 0x557a11ce5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a11b35c40 con 0x557a11d30f80 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a11ad2d90 con 0x557a11d318a0 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x557a11b31bf0 con 0x557a11ce4f50 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d31360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d31360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58374/0 (socket says 172.21.15.86:58374) 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 -- 172.21.15.86:0/131881333 learned_addr learned my addr 172.21.15.86:0/131881333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ede00640 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11d318a0 0x557a11ce4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ef200640 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11ce4f50 0x557a11ce5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 -- 172.21.15.86:0/131881333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11ce4f50 msgr2=0x557a11ce5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11ce4f50 0x557a11ce5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 -- 172.21.15.86:0/131881333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11d318a0 msgr2=0x557a11ce4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11d318a0 0x557a11ce4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 -- 172.21.15.86:0/131881333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a11bdd130 con 0x557a11d30f80 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35ee800640 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d31360 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f35e0007fd0 tx=0x7f35e0055630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9766d79b644e54a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35db400640 1 -- 172.21.15.86:0/131881333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e005dfb0 con 0x557a11d30f80 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35db400640 1 -- 172.21.15.86:0/131881333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0053930 con 0x557a11d30f80 2024-03-22T04:21:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35db400640 1 -- 172.21.15.86:0/131881333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00850c0 con 0x557a11d30f80 2024-03-22T04:21:23.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- 172.21.15.86:0/131881333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 msgr2=0x557a11d31360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d31360 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f35e0007fd0 tx=0x7f35e0055630 comp rx=0 tx=0).stop 2024-03-22T04:21:23.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- 172.21.15.86:0/131881333 shutdown_connections 2024-03-22T04:21:23.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11d318a0 0x557a11ce4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11ce4f50 0x557a11ce5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 --2- 172.21.15.86:0/131881333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d31360 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- 172.21.15.86:0/131881333 >> 172.21.15.86:0/131881333 conn(0x557a11bce990 msgr2=0x557a11d14150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- 172.21.15.86:0/131881333 shutdown_connections 2024-03-22T04:21:23.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- 172.21.15.86:0/131881333 wait complete. 2024-03-22T04:21:23.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 -- start start 2024-03-22T04:21:23.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.494+0000 7f35f87c5000 1 Processor -- start 2024-03-22T04:21:23.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11ceca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d15380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d318a0 0x557a11d158c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a11af1710 con 0x557a11d30f80 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a11af3580 con 0x557a11ce4f50 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x557a11b3a270 con 0x557a11d318a0 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 10 client.0 _ll_register_callbacks cb 0x557a11b97200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d15380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d15380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58386/0 (socket says 172.21.15.86:58386) 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 -- 172.21.15.86:0/2914930205 learned_addr learned my addr 172.21.15.86:0/2914930205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ee800640 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11ceca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ef200640 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d318a0 0x557a11d158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 -- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d318a0 msgr2=0x557a11d158c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d318a0 0x557a11d158c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 -- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 msgr2=0x557a11ceca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11ceca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 -- 172.21.15.86:0/2914930205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a11b0ed30 con 0x557a11d30f80 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ef200640 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d318a0 0x557a11d158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ede00640 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d15380 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f35dc008010 tx=0x7f35dc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6e4318383969e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35cfe00640 1 -- 172.21.15.86:0/2914930205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0c8060 con 0x557a11d30f80 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35cfe00640 1 -- 172.21.15.86:0/2914930205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f35dc004230 con 0x557a11d30f80 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 -- 172.21.15.86:0/2914930205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e0053930 con 0x557a11d30f80 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35cfe00640 1 -- 172.21.15.86:0/2914930205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f35dc0ce050 con 0x557a11d30f80 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 10 client.9435 fetch_fsmap learned FSMap version 105 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 -- 172.21.15.86:0/2914930205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x557a11b7afc0 con 0x557a11d30f80 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35cfe00640 1 -- 172.21.15.86:0/2914930205 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0053930 con 0x557a11d30f80 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35cfe00640 1 -- 172.21.15.86:0/2914930205 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f35dc0d1020 con 0x557a11d30f80 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 10 client.9435 fetch_fsmap finished waiting for FSMap version 105 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 10 client.9435 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-22T04:21:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 20 client.9435 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:23.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x557a11d375c0 0x557a11d379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 4 client.9435 mds_command: new command op to 77801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:21:23.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35f87c5000 1 -- 172.21.15.86:0/2914930205 --> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a11ad0dd0 con 0x557a11d375c0 2024-03-22T04:21:23.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.498+0000 7f35ee800640 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x557a11d375c0 0x557a11d379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35ee800640 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x557a11d375c0 0x557a11d379d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f760627954447381 server_cookie=84175f3e24a0a4 in_seq=0 out_seq=0 2024-03-22T04:21:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35cfe00640 10 client.9435 ms_handle_connect on v2:172.21.15.86:6835/4220897872 2024-03-22T04:21:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35cfe00640 1 -- 172.21.15.86:0/2914930205 <== mds.0 v2:172.21.15.86:6835/4220897872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557a11ad0dd0 con 0x557a11d375c0 2024-03-22T04:21:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35cfe00640 10 client.9435 handle_command_reply: tid=0 2024-03-22T04:21:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 client.9435 shutdown 2024-03-22T04:21:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 -- 172.21.15.86:0/2914930205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 msgr2=0x557a11d15380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d15380 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f35dc008010 tx=0x7f35dc01ac20 comp rx=0 tx=0).stop 2024-03-22T04:21:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 -- 172.21.15.86:0/2914930205 shutdown_connections 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x557a11d375c0 0x557a11d379d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d318a0 0x557a11d158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d15380 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 --2- 172.21.15.86:0/2914930205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11ceca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35cfe00640 0 client.9435 ms_handle_reset on v2:172.21.15.86:6835/4220897872 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 -- 172.21.15.86:0/2914930205 >> 172.21.15.86:0/2914930205 conn(0x557a11bce990 msgr2=0x557a11ce4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 -- 172.21.15.86:0/2914930205 shutdown_connections 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 1 -- 172.21.15.86:0/2914930205 wait complete. 2024-03-22T04:21:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.502+0000 7f35f87c5000 20 client.9435 trim_cache size 0 max 16384 2024-03-22T04:21:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.522+0000 7f35f87c5000 1 Processor -- start 2024-03-22T04:21:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.522+0000 7f35f87c5000 1 -- start start 2024-03-22T04:21:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.522+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11bce990 0x557a11d47680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.522+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11d363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0202050 con 0x557a11d30f80 2024-03-22T04:21:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35f02010d0 con 0x557a11ce4f50 2024-03-22T04:21:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f35f00b5cc0 con 0x557a11bce990 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d368f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58400/0 (socket says 172.21.15.86:58400) 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 -- 172.21.15.86:0/232780300 learned_addr learned my addr 172.21.15.86:0/232780300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ee800640 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11bce990 0x557a11d47680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ede00640 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11d363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 -- 172.21.15.86:0/232780300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11bce990 msgr2=0x557a11d47680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11bce990 0x557a11d47680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 -- 172.21.15.86:0/232780300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 msgr2=0x557a11d363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11d363b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 -- 172.21.15.86:0/232780300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a11b0b3a0 con 0x557a11d30f80 2024-03-22T04:21:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ef200640 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d368f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f35e80098e0 tx=0x7f35e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df4db94f5261f0b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35daa00640 1 -- 172.21.15.86:0/232780300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8019020 con 0x557a11d30f80 2024-03-22T04:21:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35daa00640 1 -- 172.21.15.86:0/232780300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e80153f0 con 0x557a11d30f80 2024-03-22T04:21:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35daa00640 1 -- 172.21.15.86:0/232780300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e80157e0 con 0x557a11d30f80 2024-03-22T04:21:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- 172.21.15.86:0/232780300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 msgr2=0x557a11d368f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d368f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f35e80098e0 tx=0x7f35e8009da0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- 172.21.15.86:0/232780300 shutdown_connections 2024-03-22T04:21:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11d363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11d30f80 0x557a11d368f0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 --2- 172.21.15.86:0/232780300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11bce990 0x557a11d47680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- 172.21.15.86:0/232780300 >> 172.21.15.86:0/232780300 conn(0x557a11d14400 msgr2=0x557a11d35ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- 172.21.15.86:0/232780300 shutdown_connections 2024-03-22T04:21:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- 172.21.15.86:0/232780300 wait complete. 2024-03-22T04:21:23.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- start start 2024-03-22T04:21:23.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 Processor -- start 2024-03-22T04:21:23.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11bce990 0x557a11d3abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11d3b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d30f80 0x557a11d3b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a11bf1810 con 0x557a11bce990 2024-03-22T04:21:23.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35f00b6310 con 0x557a11ce4f50 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x557a11b34f50 con 0x557a11d30f80 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35f87c5000 10 client.0 _ll_register_callbacks cb 0x7f35dc0ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11bce990 0x557a11d3abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.526+0000 7f35ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11bce990 0x557a11d3abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58416/0 (socket says 172.21.15.86:58416) 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ee800640 1 -- 172.21.15.86:0/3874339430 learned_addr learned my addr 172.21.15.86:0/3874339430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ef200640 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d30f80 0x557a11d3b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ede00640 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11d3b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ee800640 1 -- 172.21.15.86:0/3874339430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d30f80 msgr2=0x557a11d3b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ee800640 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d30f80 0x557a11d3b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ee800640 1 -- 172.21.15.86:0/3874339430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 msgr2=0x557a11d3b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ee800640 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11d3b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ee800640 1 -- 172.21.15.86:0/3874339430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a11abb380 con 0x557a11bce990 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ee800640 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11bce990 0x557a11d3abe0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f35e000ceb0 tx=0x7f35e0085050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d6bd629d0c35 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35cfe00640 1 -- 172.21.15.86:0/3874339430 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0051ca0 con 0x557a11bce990 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35cfe00640 1 -- 172.21.15.86:0/3874339430 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7f35e001dc80 con 0x557a11bce990 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 1 -- 172.21.15.86:0/3874339430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e80153f0 con 0x557a11bce990 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35cfe00640 1 -- 172.21.15.86:0/3874339430 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f35e0071030 con 0x557a11bce990 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 10 client.9441 fetch_fsmap learned FSMap version 105 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 1 -- 172.21.15.86:0/3874339430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x557a11bb7460 con 0x557a11bce990 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35cfe00640 1 -- 172.21.15.86:0/3874339430 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e80153f0 con 0x557a11bce990 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35cfe00640 1 -- 172.21.15.86:0/3874339430 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f35e00850c0 con 0x557a11bce990 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 10 client.9441 fetch_fsmap finished waiting for FSMap version 105 2024-03-22T04:21:23.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 10 client.9441 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-22T04:21:23.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 20 client.9441 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:23.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x557a11d318a0 0x557a11d3bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 4 client.9441 mds_command: new command op to 77801 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-22T04:21:23.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35f87c5000 1 -- 172.21.15.86:0/3874339430 --> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a11ad5160 con 0x557a11d318a0 2024-03-22T04:21:23.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ede00640 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x557a11d318a0 0x557a11d3bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35cfe00640 10 client.9441 ms_handle_connect on v2:172.21.15.86:6835/4220897872 2024-03-22T04:21:23.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.530+0000 7f35ede00640 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x557a11d318a0 0x557a11d3bba0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0be4b6e57b7b06 server_cookie=f55448ff8bbba623 in_seq=0 out_seq=0 2024-03-22T04:21:23.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35cfe00640 1 -- 172.21.15.86:0/3874339430 <== mds.0 v2:172.21.15.86:6835/4220897872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a11ad5160 con 0x557a11d318a0 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35cfe00640 10 client.9441 handle_command_reply: tid=0 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 client.9441 shutdown 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 -- 172.21.15.86:0/3874339430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11bce990 msgr2=0x557a11d3abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11bce990 0x557a11d3abe0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f35e000ceb0 tx=0x7f35e0085050 comp rx=0 tx=0).stop 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 -- 172.21.15.86:0/3874339430 shutdown_connections 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.86:6835/4220897872,v1:172.21.15.86:6837/4220897872] conn(0x557a11d318a0 0x557a11d3bba0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a11ce4f50 0x557a11d3b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x557a11d30f80 0x557a11d3b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35cfe00640 0 client.9441 ms_handle_reset on v2:172.21.15.86:6835/4220897872 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 --2- 172.21.15.86:0/3874339430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a11bce990 0x557a11d3abe0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 -- 172.21.15.86:0/3874339430 >> 172.21.15.86:0/3874339430 conn(0x557a11d14400 msgr2=0x557a11d36530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 -- 172.21.15.86:0/3874339430 shutdown_connections 2024-03-22T04:21:23.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 1 -- 172.21.15.86:0/3874339430 wait complete. 2024-03-22T04:21:23.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f87c5000 20 client.9441 trim_cache size 0 max 16384 2024-03-22T04:21:23.545 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:23.545 INFO:teuthology.orchestra.run.smithi086.stdout: "message": "", 2024-03-22T04:21:23.545 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0 2024-03-22T04:21:23.545 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 -- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35d0067b20 msgr2=0x7f35d0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35d0067b20 0x7f35d0069fe0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0ad000 tx=0x7f35dc008040 comp rx=0 tx=0).stop 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 -- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f00cf250 msgr2=0x7f35f01f8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f00cf250 0x7f35f01f8b30 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f35e00011f0 tx=0x7f35e000c9b0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 -- 172.21.15.86:0/824582410 shutdown_connections 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f35d0067b20 0x7f35d0069fe0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f013f240 0x7f35f0158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f013f620 0x7f35f0158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 --2- 172.21.15.86:0/824582410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f00cf250 0x7f35f01f8b30 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 -- 172.21.15.86:0/824582410 >> 172.21.15.86:0/824582410 conn(0x7f35f00b3d60 msgr2=0x7f35f00b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 -- 172.21.15.86:0/824582410 shutdown_connections 2024-03-22T04:21:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.542+0000 7f35f5400640 1 -- 172.21.15.86:0/824582410 wait complete. 2024-03-22T04:21:23.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:21:23.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.702+0000 7fa17b600640 1 Processor -- start 2024-03-22T04:21:23.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.702+0000 7fa17b600640 1 -- start start 2024-03-22T04:21:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa17b600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1740d0040 0x7fa1740d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa17b600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa17406f9b0 0x7fa17406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa17b600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa174144260 con 0x7fa174143cb0 2024-03-22T04:21:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa17b600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa174071210 con 0x7fa17406f9b0 2024-03-22T04:21:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa17b600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa174144540 con 0x7fa1740d0040 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa178c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1740d0040 0x7fa1740d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa173e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa174143cb0 0x7fa174141c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:35544/0 (socket says 172.21.15.86:35544) 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa179600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa17406f9b0 0x7fa17406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa173e00640 1 -- 172.21.15.86:0/3663967196 learned_addr learned my addr 172.21.15.86:0/3663967196 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa178c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1740d0040 0x7fa1740d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:47282/0 (socket says 172.21.15.86:47282) 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa173400640 1 -- 172.21.15.86:0/3663967196 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2394073027 0 0) 0x7fa174144260 con 0x7fa174143cb0 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa178c00640 1 -- 172.21.15.86:0/3663967196 >> v2:172.21.15.196:3300/0 conn(0x7fa17406f9b0 msgr2=0x7fa17406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa178c00640 1 --2- 172.21.15.86:0/3663967196 >> v2:172.21.15.196:3300/0 conn(0x7fa17406f9b0 0x7fa17406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa178c00640 1 -- 172.21.15.86:0/3663967196 >> v1:172.21.15.196:6789/0 conn(0x7fa174143cb0 legacy=0x7fa174141c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa178c00640 1 -- 172.21.15.86:0/3663967196 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa174144820 con 0x7fa1740d0040 2024-03-22T04:21:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.706+0000 7fa179600640 1 --2- 172.21.15.86:0/3663967196 >> v2:172.21.15.196:3300/0 conn(0x7fa17406f9b0 0x7fa17406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa178c00640 1 --2- 172.21.15.86:0/3663967196 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1740d0040 0x7fa1740d1a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1680097c0 tx=0x7fa16801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f6ec90e2720c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa173400640 1 -- 172.21.15.86:0/3663967196 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1680b0070 con 0x7fa1740d0040 2024-03-22T04:21:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa173400640 1 -- 172.21.15.86:0/3663967196 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1680ab600 con 0x7fa1740d0040 2024-03-22T04:21:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa173400640 1 -- 172.21.15.86:0/3663967196 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1680ab9d0 con 0x7fa1740d0040 2024-03-22T04:21:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 -- 172.21.15.86:0/3663967196 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1740d0040 msgr2=0x7fa1740d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 --2- 172.21.15.86:0/3663967196 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1740d0040 0x7fa1740d1a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1680097c0 tx=0x7fa16801c410 comp rx=0 tx=0).stop 2024-03-22T04:21:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 -- 172.21.15.86:0/3663967196 shutdown_connections 2024-03-22T04:21:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 --2- 172.21.15.86:0/3663967196 >> v2:172.21.15.196:3300/0 conn(0x7fa17406f9b0 0x7fa17406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 --2- 172.21.15.86:0/3663967196 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1740d0040 0x7fa1740d1a60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 -- 172.21.15.86:0/3663967196 >> 172.21.15.86:0/3663967196 conn(0x7fa1740d20b0 msgr2=0x7fa1740d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 -- 172.21.15.86:0/3663967196 shutdown_connections 2024-03-22T04:21:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 -- 172.21.15.86:0/3663967196 wait complete. 2024-03-22T04:21:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 Processor -- start 2024-03-22T04:21:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.710+0000 7fa17b600640 1 -- start start 2024-03-22T04:21:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa17406f9b0 0x7fa17415cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1740d0040 0x7fa1740b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174143cb0 0x7fa1740c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa174071eb0 con 0x7fa1740d0040 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740d0630 con 0x7fa174143cb0 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa174072030 con 0x7fa17406f9b0 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1740d0040 0x7fa1740b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1740d0040 0x7fa1740b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58432/0 (socket says 172.21.15.86:58432) 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 -- 172.21.15.86:0/717160575 learned_addr learned my addr 172.21.15.86:0/717160575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa178c00640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa17406f9b0 0x7fa17415cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa179600640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174143cb0 0x7fa1740c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 -- 172.21.15.86:0/717160575 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa17406f9b0 msgr2=0x7fa17415cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa17406f9b0 0x7fa17415cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 -- 172.21.15.86:0/717160575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174143cb0 msgr2=0x7fa1740c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174143cb0 0x7fa1740c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 -- 172.21.15.86:0/717160575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17415ddd0 con 0x7fa1740d0040 2024-03-22T04:21:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa173e00640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1740d0040 0x7fa1740b26b0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fa16000c520 tx=0x7fa16000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa170c00640 1 -- 172.21.15.86:0/717160575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160018070 con 0x7fa1740d0040 2024-03-22T04:21:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17415daf0 con 0x7fa1740d0040 2024-03-22T04:21:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa170c00640 1 -- 172.21.15.86:0/717160575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1600074e0 con 0x7fa1740d0040 2024-03-22T04:21:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa170c00640 1 -- 172.21.15.86:0/717160575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1600049f0 con 0x7fa1740d0040 2024-03-22T04:21:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa174157240 con 0x7fa1740d0040 2024-03-22T04:21:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa174075220 con 0x7fa1740d0040 2024-03-22T04:21:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.714+0000 7fa170c00640 1 -- 172.21.15.86:0/717160575 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa160004b90 con 0x7fa1740d0040 2024-03-22T04:21:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.718+0000 7fa170c00640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa154067b20 0x7fa154069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.718+0000 7fa170c00640 1 -- 172.21.15.86:0/717160575 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6187+0+0 (secure 0 0 0) 0x7fa160085320 con 0x7fa1740d0040 2024-03-22T04:21:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.718+0000 7fa178c00640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa154067b20 0x7fa154069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.718+0000 7fa178c00640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa154067b20 0x7fa154069fe0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa1680ad000 tx=0x7fa1680a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:23.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.726+0000 7fa170c00640 1 -- 172.21.15.86:0/717160575 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa160055470 con 0x7fa1740d0040 2024-03-22T04:21:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.918+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa1740b4e00 con 0x7fa1740d0040 2024-03-22T04:21:23.921 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:21:23.918+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:21:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.922+0000 7fa170c00640 1 -- 172.21.15.86:0/717160575 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7fa1600850e0 con 0x7fa1740d0040 2024-03-22T04:21:23.923 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:21:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa154067b20 msgr2=0x7fa154069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa154067b20 0x7fa154069fe0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa1680ad000 tx=0x7fa1680a6000 comp rx=0 tx=0).stop 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1740d0040 msgr2=0x7fa1740b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1740d0040 0x7fa1740b26b0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fa16000c520 tx=0x7fa16000c920 comp rx=0 tx=0).stop 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 shutdown_connections 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa154067b20 0x7fa154069fe0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174143cb0 0x7fa1740c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1740d0040 0x7fa1740b26b0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 --2- 172.21.15.86:0/717160575 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa17406f9b0 0x7fa17415cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 >> 172.21.15.86:0/717160575 conn(0x7fa1740d20b0 msgr2=0x7fa17406fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 shutdown_connections 2024-03-22T04:21:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.926+0000 7fa17b600640 1 -- 172.21.15.86:0/717160575 wait complete. 2024-03-22T04:21:23.937 INFO:tasks.ceph.mds.a.smithi086.stderr: -1858> 2024-03-22T04:21:18.574+0000 7f3151400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:21:23.949 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-03-22T04:21:23.981 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:21:23.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.994+0000 7f64ad83a400 1 -- start start 2024-03-22T04:21:23.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.994+0000 7f64ad83a400 1 Processor -- start 2024-03-22T04:21:23.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.994+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba5e45000 0x556ba6c0e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.994+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5e45400 0x556ba6c0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.994+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba5e45800 0x556ba6c0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.994+0000 7f64ad83a400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ba6d4c180 con 0x556ba5e45000 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.994+0000 7f64ad83a400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556ba6d4c300 con 0x556ba5e45400 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.994+0000 7f64ad83a400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556ba6d4c480 con 0x556ba5e45800 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba5e45000 0x556ba6c0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba5e45000 0x556ba6c0e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58462/0 (socket says 172.21.15.86:58462) 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 -- 172.21.15.86:0/1690796849 learned_addr learned my addr 172.21.15.86:0/1690796849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba5e45800 0x556ba6c0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64aae00640 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5e45400 0x556ba6c0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 -- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba5e45800 msgr2=0x556ba6c0eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba5e45800 0x556ba6c0eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 -- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5e45400 msgr2=0x556ba6c0e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5e45400 0x556ba6c0e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 -- 172.21.15.86:0/1690796849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba6c0cb40 con 0x556ba5e45000 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64aae00640 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5e45400 0x556ba6c0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba5e45000 0x556ba6c0e000 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x556ba6caea80 tx=0x556ba5f1a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64989d5e2ca6469a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64a8600640 1 -- 172.21.15.86:0/1690796849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ba5e7a9c0 con 0x556ba5e45000 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64a8600640 1 -- 172.21.15.86:0/1690796849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ba5e90e00 con 0x556ba5e45000 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64a8600640 1 -- 172.21.15.86:0/1690796849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ba5e7a4e0 con 0x556ba5e45000 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 -- 172.21.15.86:0/1690796849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba6c0cd20 con 0x556ba5e45000 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 -- 172.21.15.86:0/1690796849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556ba6c0cf00 con 0x556ba5e45000 2024-03-22T04:21:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 4 waiting for MDS map... 2024-03-22T04:21:24.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64a8600640 1 -- 172.21.15.86:0/1690796849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x556ba6c07a00 con 0x556ba5e45000 2024-03-22T04:21:24.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64a8600640 1 -- 172.21.15.86:0/1690796849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x556ba5e9b180 con 0x556ba5e45000 2024-03-22T04:21:24.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 4 Got MDS map 106 2024-03-22T04:21:24.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 10 main: JournalTool::main 2024-03-22T04:21:24.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 4 main: JournalTool: connecting to RADOS... 2024-03-22T04:21:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 Processor -- start 2024-03-22T04:21:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 -- start start 2024-03-22T04:21:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19c00 0x556ba6c0f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbc000 0x556ba6c0f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 0x556ba6c0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ba6d4c600 con 0x556ba6c19c00 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556ba6d4c780 con 0x556ba6dbc400 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ad83a400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556ba6d4c900 con 0x556ba6dbc000 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19c00 0x556ba6c0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19c00 0x556ba6c0f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58468/0 (socket says 172.21.15.86:58468) 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 -- 172.21.15.86:0/213431155 learned_addr learned my addr 172.21.15.86:0/213431155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ab800640 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbc000 0x556ba6c0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64aae00640 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 0x556ba6c0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 -- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbc000 msgr2=0x556ba6c0f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbc000 0x556ba6c0f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 -- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 msgr2=0x556ba6c0fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 0x556ba6c0fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64ac200640 1 -- 172.21.15.86:0/213431155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba6c0c1e0 con 0x556ba6c19c00 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:23.998+0000 7f64aae00640 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 0x556ba6c0fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19c00 0x556ba6c0f080 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x556ba6c264b0 tx=0x556ba6d6ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575adc7d3f1954fb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64a5e00640 1 -- 172.21.15.86:0/213431155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ba5e7a9c0 con 0x556ba6c19c00 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64a5e00640 1 -- 172.21.15.86:0/213431155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ba5e90a80 con 0x556ba6c19c00 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64a5e00640 1 -- 172.21.15.86:0/213431155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ba5e7a4e0 con 0x556ba6c19c00 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- 172.21.15.86:0/213431155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19c00 msgr2=0x556ba6c0f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19c00 0x556ba6c0f080 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x556ba6c264b0 tx=0x556ba6d6ac00 comp rx=0 tx=0).stop 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- 172.21.15.86:0/213431155 shutdown_connections 2024-03-22T04:21:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 0x556ba6c0fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbc000 0x556ba6c0f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 --2- 172.21.15.86:0/213431155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19c00 0x556ba6c0f080 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- 172.21.15.86:0/213431155 >> 172.21.15.86:0/213431155 conn(0x556ba6c19800 msgr2=0x556ba5e72000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- 172.21.15.86:0/213431155 shutdown_connections 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- 172.21.15.86:0/213431155 wait complete. 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 Processor -- start 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- start start 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19800 0x556ba6c0f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbd000 0x556ba6c0f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbcc00 0x556ba6c0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ba6d4dc80 con 0x556ba6c19800 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556ba6d4db00 con 0x556ba6dbcc00 2024-03-22T04:21:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556ba6d4d980 con 0x556ba6dbd000 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19800 0x556ba6c0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19800 0x556ba6c0f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58476/0 (socket says 172.21.15.86:58476) 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 -- 172.21.15.86:0/3219864214 learned_addr learned my addr 172.21.15.86:0/3219864214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64aae00640 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbcc00 0x556ba6c0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ab800640 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbd000 0x556ba6c0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 -- 172.21.15.86:0/3219864214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbd000 msgr2=0x556ba6c0f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbd000 0x556ba6c0f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 -- 172.21.15.86:0/3219864214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbcc00 msgr2=0x556ba6c0fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbcc00 0x556ba6c0fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 -- 172.21.15.86:0/3219864214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba6c0da40 con 0x556ba6c19800 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ac200640 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19800 0x556ba6c0f080 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x556ba6c27770 tx=0x556ba5f1a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64a4000640 1 -- 172.21.15.86:0/3219864214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ba5e7a000 con 0x556ba6c19800 2024-03-22T04:21:24.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64a4000640 1 -- 172.21.15.86:0/3219864214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ba5e91340 con 0x556ba6c19800 2024-03-22T04:21:24.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- 172.21.15.86:0/3219864214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ba6c0dc20 con 0x556ba6c19800 2024-03-22T04:21:24.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64a4000640 1 -- 172.21.15.86:0/3219864214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ba5e7a9c0 con 0x556ba6c19800 2024-03-22T04:21:24.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 1 -- 172.21.15.86:0/3219864214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba6c0da40 con 0x556ba6c19800 2024-03-22T04:21:24.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.002+0000 7f64ad83a400 4 main: JournalTool: resolving pool 17 2024-03-22T04:21:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64a4000640 1 -- 172.21.15.86:0/3219864214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x556ba6dbac00 con 0x556ba6c19800 2024-03-22T04:21:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64a4000640 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x556ba622a000 0x556ba6c10100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64a4000640 1 -- 172.21.15.86:0/3219864214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x556ba6140400 con 0x556ba6c19800 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64ad83a400 4 main: JournalTool: creating IoCtx.. 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64ad83a400 4 main: Executing for rank 0 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64ab800640 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x556ba622a000 0x556ba6c10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64ab800640 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x556ba622a000 0x556ba6c10100 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x556ba6cafb00 tx=0x556ba5f1ad80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64ad83a400 1 -- start start 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.006+0000 7f64ad83a400 1 Processor -- start 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 0x556ba6c10680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6dbc000 0x556ba6c10c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba622b400 0x556ba6c11180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ba6d4d680 con 0x556ba6dbc000 2024-03-22T04:21:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556ba6d4c600 con 0x556ba6dbc400 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556ba6d4c780 con 0x556ba622b400 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6dbc000 0x556ba6c10c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6dbc000 0x556ba6c10c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58490/0 (socket says 172.21.15.86:58490) 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 -- 172.21.15.86:0/3911225134 learned_addr learned my addr 172.21.15.86:0/3911225134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ac200640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 0x556ba6c10680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64aae00640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba622b400 0x556ba6c11180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 -- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba622b400 msgr2=0x556ba6c11180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba622b400 0x556ba6c11180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 -- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 msgr2=0x556ba6c10680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbc400 0x556ba6c10680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba627e780 con 0x556ba6dbc000 2024-03-22T04:21:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ab800640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6dbc000 0x556ba6c10c00 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x556ba6cafce0 tx=0x556ba5f1a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78ccf293cb93a07 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64a0400640 1 -- 172.21.15.86:0/3911225134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ba5e7ab60 con 0x556ba6dbc000 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64a0400640 1 -- 172.21.15.86:0/3911225134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ba5e90c40 con 0x556ba6dbc000 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba627e960 con 0x556ba6dbc000 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556ba627eb40 con 0x556ba6dbc000 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 4 waiting for MDS map... 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64a0400640 1 -- 172.21.15.86:0/3911225134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ba5e7a340 con 0x556ba6dbc000 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64a0400640 1 -- 172.21.15.86:0/3911225134 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x556ba6dc2800 con 0x556ba6dbc000 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64a0400640 1 -- 172.21.15.86:0/3911225134 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x556ba5e9b500 con 0x556ba6dbc000 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 4 Got MDS map 106 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x556ba622b800 0x556ba6c11700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.010+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba622bc00 con 0x556ba622b800 2024-03-22T04:21:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.014+0000 7f64ac200640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x556ba622b800 0x556ba6c11700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.014+0000 7f64ac200640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x556ba622b800 0x556ba6c11700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=e8a0a0bcf0b2ee63 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.014+0000 7f64ac200640 1 -- 172.21.15.86:0/3911225134 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v126'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2000 con 0x556ba622b800 2024-03-22T04:21:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.014+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x556ba6285c00 0x556ba62a6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.014+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba62aa000 con 0x556ba6285c00 2024-03-22T04:21:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.014+0000 7f64aae00640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x556ba6285c00 0x556ba62a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.014+0000 7f64aae00640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x556ba6285c00 0x556ba62a6000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=e824c8ad6d568ae9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64aae00640 1 -- 172.21.15.86:0/3911225134 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v126'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2500 con 0x556ba6285c00 2024-03-22T04:21:24.019 INFO:teuthology.orchestra.run.smithi086.stdout:writing EResetJournal entry 2024-03-22T04:21:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba62aa400 con 0x556ba6285c00 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x556ba62aa800 0x556ba62a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba62aac00 con 0x556ba62aa800 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba62ab000 con 0x556ba622b800 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x556ba62ab400 0x556ba62a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba62ab800 con 0x556ba62ab400 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x556ba62abc00 0x556ba62a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba62aa000 con 0x556ba62abc00 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba62aa400 con 0x556ba622b800 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ac200640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x556ba62aa800 0x556ba62a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ab800640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x556ba62ab400 0x556ba62a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64aae00640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x556ba62abc00 0x556ba62a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ac200640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x556ba62aa800 0x556ba62a6580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=66fabb369248b321 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ab800640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x556ba62ab400 0x556ba62a6b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=f2387a21ead67e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64aae00640 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x556ba62abc00 0x556ba62a7080 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=e016956454fd12e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64aae00640 1 -- 172.21.15.86:0/3911225134 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== osd_op_reply(3 200.00000001 [delete] v126'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2500 con 0x556ba6285c00 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ac200640 1 -- 172.21.15.86:0/3911225134 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== osd_op_reply(8 200.00000006 [delete] v126'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2000 con 0x556ba622b800 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ac200640 1 -- 172.21.15.86:0/3911225134 <== osd.6 v2:172.21.15.196:6816/1648471396 3 ==== osd_op_reply(5 200.00000003 [delete] v126'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2000 con 0x556ba622b800 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ab800640 1 -- 172.21.15.86:0/3911225134 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(6 200.00000004 [delete] v126'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2780 con 0x556ba62ab400 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64ac200640 1 -- 172.21.15.86:0/3911225134 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(4 200.00000002 [delete] v126'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2000 con 0x556ba62aa800 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f649e600640 1 -- 172.21.15.86:0/3911225134 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556ba622bc00 con 0x556ba6285c00 2024-03-22T04:21:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.018+0000 7f64aae00640 1 -- 172.21.15.86:0/3911225134 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(7 200.00000005 [delete] v126'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2500 con 0x556ba62abc00 2024-03-22T04:21:24.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64aae00640 1 -- 172.21.15.86:0/3911225134 <== osd.5 v2:172.21.15.196:6808/2743139076 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v126'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x556ba6bc2500 con 0x556ba6285c00 2024-03-22T04:21:24.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-03-22T04:21:24.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x556ba62ab400 msgr2=0x556ba62a6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x556ba62ab400 0x556ba62a6b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x556ba62aa800 msgr2=0x556ba62a6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x556ba62aa800 0x556ba62a6580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x556ba62abc00 msgr2=0x556ba62a7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x556ba62abc00 0x556ba62a7080 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x556ba6285c00 msgr2=0x556ba62a6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x556ba6285c00 0x556ba62a6000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64aae00640 1 -- 172.21.15.86:0/3911225134 reap_dead start 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x556ba622b800 msgr2=0x556ba6c11700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x556ba622b800 0x556ba6c11700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6dbc000 msgr2=0x556ba6c10c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6dbc000 0x556ba6c10c00 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x556ba6cafce0 tx=0x556ba5f1a000 comp rx=0 tx=0).stop 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 shutdown_connections 2024-03-22T04:21:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x556ba6285c00 0x556ba62a6000 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6dbc000 0x556ba6c10c00 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3911225134 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x556ba622b800 0x556ba6c11700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 >> 172.21.15.86:0/3911225134 conn(0x556ba6dbc800 msgr2=0x556ba6278000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 shutdown_connections 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3911225134 wait complete. 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x556ba622a000 msgr2=0x556ba6c10100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x556ba622a000 0x556ba6c10100 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x556ba6cafb00 tx=0x556ba5f1ad80 comp rx=0 tx=0).stop 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19800 msgr2=0x556ba6c0f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19800 0x556ba6c0f080 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x556ba6c27770 tx=0x556ba5f1a120 comp rx=0 tx=0).stop 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3219864214 shutdown_connections 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x556ba622a000 0x556ba6c10100 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba6dbcc00 0x556ba6c0fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba6dbd000 0x556ba6c0f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/3219864214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba6c19800 0x556ba6c0f080 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3219864214 >> 172.21.15.86:0/3219864214 conn(0x556ba6c19c00 msgr2=0x556ba5e72000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3219864214 shutdown_connections 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/3219864214 wait complete. 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/1690796849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba5e45000 msgr2=0x556ba6c0e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba5e45000 0x556ba6c0e000 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x556ba6caea80 tx=0x556ba5f1a1e0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/1690796849 shutdown_connections 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556ba5e45400 0x556ba6c0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ba5e45800 0x556ba6c0eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 --2- 172.21.15.86:0/1690796849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ba5e45000 0x556ba6c0e000 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/1690796849 >> 172.21.15.86:0/1690796849 conn(0x556ba5e44c00 msgr2=0x556ba5e71800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/1690796849 shutdown_connections 2024-03-22T04:21:24.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.022+0000 7f64ad83a400 1 -- 172.21.15.86:0/1690796849 wait complete. 2024-03-22T04:21:24.031 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.081522, result: writing EResetJournal entry 2024-03-22T04:21:24.031 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-22T04:21:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 -- start start 2024-03-22T04:21:24.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 Processor -- start 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f157b800 0x5622f2344000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f157b400 0x5622f2344580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f157b000 0x5622f2344b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622f22e8600 con 0x5622f157b800 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622f22e8480 con 0x5622f157b400 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5622f22e8300 con 0x5622f157b000 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f157b800 0x5622f2344000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f157b800 0x5622f2344000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58518/0 (socket says 172.21.15.86:58518) 2024-03-22T04:21:24.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 -- 172.21.15.86:0/1657268723 learned_addr learned my addr 172.21.15.86:0/1657268723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 -- 172.21.15.86:0/1657268723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f157b000 msgr2=0x5622f2344b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1657268723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f157b000 0x5622f2344b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 -- 172.21.15.86:0/1657268723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f157b400 msgr2=0x5622f2344580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1657268723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f157b400 0x5622f2344580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 -- 172.21.15.86:0/1657268723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f15a01e0 con 0x5622f157b800 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1657268723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f157b800 0x5622f2344000 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x5622f23cf8f0 tx=0x5622f1655440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f671755dcbded619 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9ac00640 1 -- 172.21.15.86:0/1657268723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5622f15b04e0 con 0x5622f157b800 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9ac00640 1 -- 172.21.15.86:0/1657268723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5622f15c6c40 con 0x5622f157b800 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9ac00640 1 -- 172.21.15.86:0/1657268723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5622f15b0b60 con 0x5622f157b800 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1657268723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f15a12c0 con 0x5622f157b800 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1657268723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5622f15a1a40 con 0x5622f157b800 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9fda68c0 4 waiting for MDS map... 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9ac00640 1 -- 172.21.15.86:0/1657268723 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x5622f2496600 con 0x5622f157b800 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.114+0000 7f9a9ac00640 1 -- 172.21.15.86:0/1657268723 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x5622f15d1180 con 0x5622f157b800 2024-03-22T04:21:24.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 4 Got MDS map 106 2024-03-22T04:21:24.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 4 datascan.main: Using metadata pool output 2024-03-22T04:21:24.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 4 datascan.main: connecting to RADOS... 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 Processor -- start 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- start start 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 0x5622f2345080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 0x5622f2345b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622f22e8180 con 0x5622f249b400 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622f15da480 con 0x5622f249b800 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5622f15da900 con 0x5622f249b000 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58524/0 (socket says 172.21.15.86:58524) 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 -- 172.21.15.86:0/202953805 learned_addr learned my addr 172.21.15.86:0/202953805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9d400640 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 0x5622f2345b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9e800640 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 0x5622f2345080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 -- 172.21.15.86:0/202953805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 msgr2=0x5622f2345080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 0x5622f2345080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 -- 172.21.15.86:0/202953805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 msgr2=0x5622f2345b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 0x5622f2345b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 -- 172.21.15.86:0/202953805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f15a1c20 con 0x5622f249b400 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9de00640 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x5622f24b2bd0 tx=0x5622f1655380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee574ee192db00a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a98400640 1 -- 172.21.15.86:0/202953805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5622f15b0340 con 0x5622f249b400 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a98400640 1 -- 172.21.15.86:0/202953805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5622f15c6a80 con 0x5622f249b400 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a98400640 1 -- 172.21.15.86:0/202953805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5622f15b04e0 con 0x5622f249b400 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/202953805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 msgr2=0x5622f2345600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x5622f24b2bd0 tx=0x5622f1655380 comp rx=0 tx=0).stop 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/202953805 shutdown_connections 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 0x5622f2345b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/202953805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 0x5622f2345080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/202953805 >> 172.21.15.86:0/202953805 conn(0x5622f157ac00 msgr2=0x5622f15a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/202953805 shutdown_connections 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/202953805 wait complete. 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 Processor -- start 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.118+0000 7f9a9fda68c0 1 -- start start 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 0x5622f2345080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 0x5622f2345b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622f22e9380 con 0x5622f249b400 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622f22e9500 con 0x5622f249b800 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5622f22e8180 con 0x5622f249b000 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58528/0 (socket says 172.21.15.86:58528) 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 learned_addr learned my addr 172.21.15.86:0/1517245091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9d400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 0x5622f2345b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 0x5622f2345080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 msgr2=0x5622f2345080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f249b000 0x5622f2345080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 msgr2=0x5622f2345b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 0x5622f2345b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f15a01e0 con 0x5622f249b400 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9d400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f249b800 0x5622f2345b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x5622f24b2e40 tx=0x5622f24bad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a96600640 1 -- 172.21.15.86:0/1517245091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5622f15b0b60 con 0x5622f249b400 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a96600640 1 -- 172.21.15.86:0/1517245091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5622f15c7340 con 0x5622f249b400 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a96600640 1 -- 172.21.15.86:0/1517245091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5622f15b0000 con 0x5622f249b400 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5622f15a12c0 con 0x5622f249b400 2024-03-22T04:21:24.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f15a1a40 con 0x5622f249b400 2024-03-22T04:21:24.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 4 datascan.init: resolving metadata pool 17 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a96600640 1 -- 172.21.15.86:0/1517245091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x5622f2494c00 con 0x5622f249b400 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a96600640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5622f186d000 0x5622f2346100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a96600640 1 -- 172.21.15.86:0/1517245091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x5622f187e600 con 0x5622f249b400 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.122+0000 7f9a9fda68c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x5622f249bc00 0x5622f2346680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5622f186d000 0x5622f2346100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f186d400 con 0x5622f249bc00 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9d400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x5622f249bc00 0x5622f2346680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5622f186d000 0x5622f2346100 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x5622f22fee40 tx=0x5622f18fcf00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9d400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x5622f249bc00 0x5622f2346680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(1 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a97a00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5622f1677400 0x5622f197a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f1677000 con 0x5622f1677400 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5622f1677400 0x5622f197a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5622f1677400 0x5622f197a000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(2 [pgnls start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a98400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5622f1676c00 0x5622f197a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196ec00 con 0x5622f1676c00 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5622f1676c00 0x5622f197a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5622f1676c00 0x5622f197a580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(3 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.126+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f1676800 con 0x5622f249bc00 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== osd_op_reply(4 [pgnls start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a98400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5622f186d400 0x5622f197ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196f000 con 0x5622f186d400 2024-03-22T04:21:24.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9d400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5622f186d400 0x5622f197ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9d400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5622f186d400 0x5622f197ab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(5 [pgnls start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a97a00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5622f196fc00 0x5622f197b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a000 con 0x5622f196fc00 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5622f196fc00 0x5622f197b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5622f196fc00 0x5622f197b600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== osd_op_reply(6 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f196fc00 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a98400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5622f196f400 0x5622f197b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196f800 con 0x5622f196f400 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5622f196f400 0x5622f197b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9de00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5622f196f400 0x5622f197b080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(7 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f196f400 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.130+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a400 con 0x5622f196f400 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.4 v2:172.21.15.196:6800/655314492 2 ==== osd_op_reply(8 [pgnls start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f196f400 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198b000 con 0x5622f249bc00 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 3 ==== osd_op_reply(9 [pgnls start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a800 con 0x5622f249bc00 2024-03-22T04:21:24.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 4 ==== osd_op_reply(10 [pgnls start_epoch 126 out=49b] v124'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198b400 con 0x5622f196fc00 2024-03-22T04:21:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== osd_op_reply(11 [pgnls start_epoch 126 out=49b] v124'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f196fc00 2024-03-22T04:21:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198ac00 con 0x5622f249bc00 2024-03-22T04:21:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 5 ==== osd_op_reply(12 [pgnls start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a98400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5622f198b800 0x5622f197c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198bc00 con 0x5622f198b800 2024-03-22T04:21:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9d400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5622f198b800 0x5622f197c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9d400640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5622f198b800 0x5622f197c100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(13 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f198b800 2024-03-22T04:21:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198b000 con 0x5622f186d400 2024-03-22T04:21:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== osd_op_reply(14 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a800 con 0x5622f196f400 2024-03-22T04:21:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.4 v2:172.21.15.196:6800/655314492 3 ==== osd_op_reply(15 [pgnls start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f196f400 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f1676800 con 0x5622f198b800 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== osd_op_reply(16 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f198b800 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198ac00 con 0x5622f1676c00 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 2 ==== osd_op_reply(17 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a97a00640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5622f1677000 0x5622f197bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.134+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196f000 con 0x5622f1677000 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5622f1677000 0x5622f197bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9e800640 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5622f1677000 0x5622f197bb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(18 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198bc00 con 0x5622f249bc00 2024-03-22T04:21:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 6 ==== osd_op_reply(19 [pgnls start_epoch 126 out=81b] v125'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x5622f186d800 con 0x5622f249bc00 2024-03-22T04:21:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 7 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9fda68c0 10 datascan.operator(): handling object 1099511627778.0 2024-03-22T04:21:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5622f186dc00 con 0x5622f249bc00 2024-03-22T04:21:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 8 ==== osd_op_reply(21 10000000002.00000000 [call] v126'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a400 con 0x5622f249bc00 2024-03-22T04:21:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.138+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 9 ==== osd_op_reply(22 [pgnls start_epoch 126 out=49b] v124'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a000 con 0x5622f1677000 2024-03-22T04:21:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 2 ==== osd_op_reply(23 [pgnls start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196e800 con 0x5622f196fc00 2024-03-22T04:21:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.1 v2:172.21.15.86:6800/3761623609 3 ==== osd_op_reply(24 [pgnls start_epoch 126 out=49b] v124'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f196fc00 2024-03-22T04:21:24.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198b400 con 0x5622f1677000 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 3 ==== osd_op_reply(25 [pgnls start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199c000 con 0x5622f186d400 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 3 ==== osd_op_reply(26 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198b000 con 0x5622f198b800 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.2 v2:172.21.15.86:6816/4132808680 3 ==== osd_op_reply(27 [pgnls start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f198b800 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199c400 con 0x5622f196f400 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.4 v2:172.21.15.196:6800/655314492 4 ==== osd_op_reply(28 [pgnls start_epoch 126 out=49b] v124'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f196f400 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198bc00 con 0x5622f1677000 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 4 ==== osd_op_reply(29 [pgnls start_epoch 126 out=49b] v124'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199c800 con 0x5622f186d400 2024-03-22T04:21:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 4 ==== osd_op_reply(30 [pgnls start_epoch 126 out=81b] v124'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a800 con 0x5622f186d400 2024-03-22T04:21:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 5 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9fda68c0 10 datascan.operator(): handling object 1099511628279.0 2024-03-22T04:21:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.142+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5622f196ec00 con 0x5622f186d400 2024-03-22T04:21:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 6 ==== osd_op_reply(32 100000001f7.00000000 [call] v126'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196f800 con 0x5622f186d400 2024-03-22T04:21:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 7 ==== osd_op_reply(33 [pgnls start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199cc00 con 0x5622f249bc00 2024-03-22T04:21:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 10 ==== osd_op_reply(34 [pgnls start_epoch 126 out=49b] v124'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a400 con 0x5622f1677400 2024-03-22T04:21:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== osd_op_reply(35 [pgnls start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199d000 con 0x5622f186d400 2024-03-22T04:21:24.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 8 ==== osd_op_reply(36 [pgnls start_epoch 126 out=49b] v124'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f186d800 con 0x5622f186d400 2024-03-22T04:21:24.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 9 ==== osd_op_reply(37 [pgnls start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199d400 con 0x5622f1677400 2024-03-22T04:21:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 3 ==== osd_op_reply(38 [pgnls start_epoch 126 out=49b] v124'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f186dc00 con 0x5622f186d400 2024-03-22T04:21:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 10 ==== osd_op_reply(39 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.146+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198b000 con 0x5622f196fc00 2024-03-22T04:21:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.1 v2:172.21.15.86:6800/3761623609 4 ==== osd_op_reply(40 [pgnls start_epoch 126 out=81b] v124'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5622f16ec500 con 0x5622f196fc00 2024-03-22T04:21:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x5622f186cc00 con 0x5622f196fc00 2024-03-22T04:21:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.1 v2:172.21.15.86:6800/3761623609 5 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5622f16ec500 con 0x5622f196fc00 2024-03-22T04:21:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9fda68c0 10 datascan.operator(): handling object 1099511627780.1 2024-03-22T04:21:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5622f186c800 con 0x5622f1677000 2024-03-22T04:21:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 5 ==== osd_op_reply(42 10000000004.00000000 [call] v126'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199c800 con 0x5622f249bc00 2024-03-22T04:21:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 11 ==== osd_op_reply(43 [pgnls start_epoch 126 out=49b] v124'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a800 con 0x5622f196fc00 2024-03-22T04:21:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.1 v2:172.21.15.86:6800/3761623609 6 ==== osd_op_reply(44 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f196fc00 2024-03-22T04:21:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f1676800 con 0x5622f1677400 2024-03-22T04:21:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 4 ==== osd_op_reply(45 [pgnls start_epoch 126 out=81b] v125'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x5622f199cc00 con 0x5622f1677400 2024-03-22T04:21:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9fda68c0 10 datascan.operator(): handling object 1099511628280.0 2024-03-22T04:21:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.150+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5622f199d000 con 0x5622f1677400 2024-03-22T04:21:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v126'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199c000 con 0x5622f186d400 2024-03-22T04:21:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 11 ==== osd_op_reply(48 [pgnls start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a400 con 0x5622f1677000 2024-03-22T04:21:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 6 ==== osd_op_reply(49 [pgnls start_epoch 126 out=81b] v126'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x5622f199d800 con 0x5622f1677000 2024-03-22T04:21:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 7 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9fda68c0 10 datascan.operator(): handling object 1099511627780.0 2024-03-22T04:21:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5622f199dc00 con 0x5622f1677000 2024-03-22T04:21:24.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 8 ==== osd_op_reply(51 10000000004.00000000 [call] v126'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a000 con 0x5622f1676c00 2024-03-22T04:21:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 3 ==== osd_op_reply(52 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.154+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198b000 con 0x5622f198b800 2024-03-22T04:21:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.2 v2:172.21.15.86:6816/4132808680 4 ==== osd_op_reply(53 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f198b800 2024-03-22T04:21:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196ec00 con 0x5622f1677400 2024-03-22T04:21:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 7 ==== osd_op_reply(54 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a800 con 0x5622f1677400 2024-03-22T04:21:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 8 ==== osd_op_reply(55 [pgnls start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196f800 con 0x5622f1676c00 2024-03-22T04:21:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 4 ==== osd_op_reply(56 [pgnls start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f186cc00 con 0x5622f186d400 2024-03-22T04:21:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 12 ==== osd_op_reply(57 [pgnls start_epoch 126 out=49b] v124'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196f000 con 0x5622f1677000 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 9 ==== osd_op_reply(58 [pgnls start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f186c800 con 0x5622f196f400 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.4 v2:172.21.15.196:6800/655314492 5 ==== osd_op_reply(59 [pgnls start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f196f400 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196e800 con 0x5622f1677000 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 10 ==== osd_op_reply(60 [pgnls start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199d400 con 0x5622f1676c00 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 5 ==== osd_op_reply(61 [pgnls start_epoch 126 out=49b] v124'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f186d800 con 0x5622f186d400 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.5 v2:172.21.15.196:6808/2743139076 13 ==== osd_op_reply(62 [pgnls start_epoch 126 out=49b] v124'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f186d400 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f196ec00 con 0x5622f1677400 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 9 ==== osd_op_reply(63 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f19a8000 con 0x5622f1676c00 2024-03-22T04:21:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 6 ==== osd_op_reply(64 [pgnls start_epoch 126 out=49b] v124'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.158+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199cc00 con 0x5622f196f400 2024-03-22T04:21:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.4 v2:172.21.15.196:6800/655314492 6 ==== osd_op_reply(65 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f196f400 2024-03-22T04:21:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f19a8400 con 0x5622f1676c00 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 7 ==== osd_op_reply(66 [pgnls start_epoch 126 out=49b] v124'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199d000 con 0x5622f1677000 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.7 v2:172.21.15.196:6817/2847924285 11 ==== osd_op_reply(67 [pgnls start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677000 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f19a8800 con 0x5622f1677400 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 10 ==== osd_op_reply(68 [pgnls start_epoch 126 out=49b] v124'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199d800 con 0x5622f1676c00 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 8 ==== osd_op_reply(69 [pgnls start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f19a8c00 con 0x5622f198b800 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.2 v2:172.21.15.86:6816/4132808680 5 ==== osd_op_reply(70 [pgnls start_epoch 126 out=49b] v124'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f198b800 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f199dc00 con 0x5622f1676c00 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9de00640 1 -- 172.21.15.86:0/1517245091 <== osd.0 v2:172.21.15.86:6808/2750248043 9 ==== osd_op_reply(71 [pgnls start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec780 con 0x5622f1676c00 2024-03-22T04:21:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f19a9000 con 0x5622f249bc00 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.6 v2:172.21.15.196:6816/1648471396 12 ==== osd_op_reply(72 [pgnls start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f249bc00 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a98400640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f198a400 con 0x5622f198b800 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 <== osd.2 v2:172.21.15.86:6816/4132808680 6 ==== osd_op_reply(73 [pgnls start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec000 con 0x5622f198b800 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a97a00640 1 -- 172.21.15.86:0/1517245091 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x5622f19a9400 con 0x5622f1677400 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9e800640 1 -- 172.21.15.86:0/1517245091 <== osd.3 v2:172.21.15.86:6822/2837220459 11 ==== osd_op_reply(74 [pgnls start_epoch 126 out=49b] v124'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5622f16ec500 con 0x5622f1677400 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5622f1676c00 msgr2=0x5622f197a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x5622f1676c00 0x5622f197a580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5622f196fc00 msgr2=0x5622f197b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x5622f196fc00 0x5622f197b600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5622f198b800 msgr2=0x5622f197c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x5622f198b800 0x5622f197c100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5622f1677400 msgr2=0x5622f197a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x5622f1677400 0x5622f197a000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 reap_dead start 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5622f196f400 msgr2=0x5622f197b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x5622f196f400 0x5622f197b080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5622f186d400 msgr2=0x5622f197ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x5622f186d400 0x5622f197ab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x5622f249bc00 msgr2=0x5622f2346680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x5622f249bc00 0x5622f2346680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5622f1677000 msgr2=0x5622f197bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.162+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x5622f1677000 0x5622f197bb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9d400640 1 -- 172.21.15.86:0/1517245091 reap_dead start 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5622f186d000 msgr2=0x5622f2346100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5622f186d000 0x5622f2346100 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x5622f22fee40 tx=0x5622f18fcf00 comp rx=0 tx=0).stop 2024-03-22T04:21:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 msgr2=0x5622f2345600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x5622f24b2e40 tx=0x5622f24bad20 comp rx=0 tx=0).stop 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 shutdown_connections 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x5622f186d000 0x5622f2346100 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1517245091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f249b400 0x5622f2345600 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 >> 172.21.15.86:0/1517245091 conn(0x5622f157ac00 msgr2=0x5622f15a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 shutdown_connections 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1517245091 wait complete. 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1657268723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f157b800 msgr2=0x5622f2344000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1657268723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f157b800 0x5622f2344000 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x5622f23cf8f0 tx=0x5622f1655440 comp rx=0 tx=0).stop 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1657268723 shutdown_connections 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1657268723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622f157b400 0x5622f2344580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1657268723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622f157b000 0x5622f2344b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 --2- 172.21.15.86:0/1657268723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622f157b800 0x5622f2344000 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1657268723 >> 172.21.15.86:0/1657268723 conn(0x5622f157bc00 msgr2=0x5622f15a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1657268723 shutdown_connections 2024-03-22T04:21:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.166+0000 7f9a9fda68c0 1 -- 172.21.15.86:0/1657268723 wait complete. 2024-03-22T04:21:24.172 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.141347, result: 2024-03-22T04:21:24.172 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-03-22T04:21:24.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 -- start start 2024-03-22T04:21:24.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 Processor -- start 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e165b1800 0x563e1737e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e165b1400 0x563e1737e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e165b1000 0x563e1737eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e166ac180 con 0x563e165b1400 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e16610900 con 0x563e165b1800 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x563e16610480 con 0x563e165b1000 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e165b1400 0x563e1737e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e165b1400 0x563e1737e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58534/0 (socket says 172.21.15.86:58534) 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 -- 172.21.15.86:0/2525042389 learned_addr learned my addr 172.21.15.86:0/2525042389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467d600640 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e165b1000 0x563e1737eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467cc00640 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e165b1800 0x563e1737e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 -- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e165b1000 msgr2=0x563e1737eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e165b1000 0x563e1737eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 -- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e165b1800 msgr2=0x563e1737e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e165b1800 0x563e1737e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 -- 172.21.15.86:0/2525042389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e165d61e0 con 0x563e165b1400 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467cc00640 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e165b1800 0x563e1737e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467c200640 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e165b1400 0x563e1737e580 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x563e1741b020 tx=0x563e1668a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31df1bd5962148 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f4679a00640 1 -- 172.21.15.86:0/2525042389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563e165e69c0 con 0x563e165b1400 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f4679a00640 1 -- 172.21.15.86:0/2525042389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563e165fc1c0 con 0x563e165b1400 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f4679a00640 1 -- 172.21.15.86:0/2525042389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563e165e64e0 con 0x563e165b1400 2024-03-22T04:21:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 -- 172.21.15.86:0/2525042389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e165d74a0 con 0x563e165b1400 2024-03-22T04:21:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 -- 172.21.15.86:0/2525042389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563e165d63c0 con 0x563e165b1400 2024-03-22T04:21:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 4 waiting for MDS map... 2024-03-22T04:21:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f4679a00640 1 -- 172.21.15.86:0/2525042389 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x563e17388000 con 0x563e165b1400 2024-03-22T04:21:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f4679a00640 1 -- 172.21.15.86:0/2525042389 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x563e16607180 con 0x563e165b1400 2024-03-22T04:21:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 4 Got MDS map 106 2024-03-22T04:21:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-03-22T04:21:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 4 datascan.main: Using metadata pool output 2024-03-22T04:21:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 4 datascan.main: connecting to RADOS... 2024-03-22T04:21:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 Processor -- start 2024-03-22T04:21:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.262+0000 7f467ec168c0 1 -- start start 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752c000 0x563e1737f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752c400 0x563e1737fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e16610180 con 0x563e1738bc00 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e166acd80 con 0x563e1752c400 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x563e166acf00 con 0x563e1752c000 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58540/0 (socket says 172.21.15.86:58540) 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 -- 172.21.15.86:0/1304003978 learned_addr learned my addr 172.21.15.86:0/1304003978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467c200640 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752c400 0x563e1737fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467cc00640 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752c000 0x563e1737f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 -- 172.21.15.86:0/1304003978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752c000 msgr2=0x563e1737f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752c000 0x563e1737f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 -- 172.21.15.86:0/1304003978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752c400 msgr2=0x563e1737fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752c400 0x563e1737fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 -- 172.21.15.86:0/1304003978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e165d61e0 con 0x563e1738bc00 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x563e174d0f30 tx=0x563e174dcc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a297661c791c169 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f4677200640 1 -- 172.21.15.86:0/1304003978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563e165e69c0 con 0x563e1738bc00 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f4677200640 1 -- 172.21.15.86:0/1304003978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563e165fcc40 con 0x563e1738bc00 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f4677200640 1 -- 172.21.15.86:0/1304003978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563e165e64e0 con 0x563e1738bc00 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- 172.21.15.86:0/1304003978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 msgr2=0x563e1737f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x563e174d0f30 tx=0x563e174dcc00 comp rx=0 tx=0).stop 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- 172.21.15.86:0/1304003978 shutdown_connections 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752c400 0x563e1737fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752c000 0x563e1737f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- 172.21.15.86:0/1304003978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- 172.21.15.86:0/1304003978 >> 172.21.15.86:0/1304003978 conn(0x563e165b0c00 msgr2=0x563e165df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- 172.21.15.86:0/1304003978 shutdown_connections 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- 172.21.15.86:0/1304003978 wait complete. 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 Processor -- start 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- start start 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752d800 0x563e1737f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752d400 0x563e1737fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e166ad200 con 0x563e1738bc00 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e166ad080 con 0x563e1752d800 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467ec168c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x563e166ad380 con 0x563e1752d400 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58550/0 (socket says 172.21.15.86:58550) 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 learned_addr learned my addr 172.21.15.86:0/873302335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467c200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752d400 0x563e1737fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.266+0000 7f467cc00640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752d800 0x563e1737f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752d400 msgr2=0x563e1737fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e1752d400 0x563e1737fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752d800 msgr2=0x563e1737f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e1752d800 0x563e1737f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e165d72c0 con 0x563e1738bc00 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x563e174d1bf0 tx=0x563e17339f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f4675400640 1 -- 172.21.15.86:0/873302335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563e165e6000 con 0x563e1738bc00 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f4675400640 1 -- 172.21.15.86:0/873302335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563e165fc540 con 0x563e1738bc00 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563e165d7a40 con 0x563e1738bc00 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f4675400640 1 -- 172.21.15.86:0/873302335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563e165e69c0 con 0x563e1738bc00 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e165d7c20 con 0x563e1738bc00 2024-03-22T04:21:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467ec168c0 4 datascan.init: resolving metadata pool 17 2024-03-22T04:21:24.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f4675400640 1 -- 172.21.15.86:0/873302335 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x563e168a6c00 con 0x563e1738bc00 2024-03-22T04:21:24.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f4675400640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563e1689c800 0x563e17380100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f4675400640 1 -- 172.21.15.86:0/873302335 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x563e168a2a00 con 0x563e1738bc00 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467ec168c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467ec168c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467ec168c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563e1752d000 0x563e17380680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467cc00640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563e1689c800 0x563e17380100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x563e1752cc00 con 0x563e1752d000 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467c200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563e1752d000 0x563e17380680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.270+0000 7f467cc00640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563e1689c800 0x563e17380100 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x563e17334cf0 tx=0x563e1692ade0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467c200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563e1752d000 0x563e17380680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563e1752c800 0x563e17380c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x563e1752c400 con 0x563e1752c800 2024-03-22T04:21:24.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563e1752c800 0x563e17380c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563e1752c800 0x563e17380c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563e1752c000 0x563e17381180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689d800 con 0x563e1752c000 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467cc00640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563e1752c000 0x563e17381180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467cc00640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563e1752c000 0x563e17381180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f4676800640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563e169aa000 0x563e169b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aa400 con 0x563e169aa000 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467c200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563e169aa000 0x563e169b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.274+0000 7f467c200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563e169aa000 0x563e169b0000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aac00 con 0x563e1752c800 2024-03-22T04:21:24.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aa800 con 0x563e1752c000 2024-03-22T04:21:24.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169ab000 con 0x563e1752d000 2024-03-22T04:21:24.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f4676800640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563e169ab800 0x563e169b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169abc00 con 0x563e169ab800 2024-03-22T04:21:24.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563e169ab800 0x563e169b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563e169ab800 0x563e169b0580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169ab800 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f4677200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x563e169ab400 0x563e169b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be000 con 0x563e169ab400 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467cc00640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x563e169ab400 0x563e169b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467cc00640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x563e169ab400 0x563e169b1080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.278+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be000 con 0x563e169ab400 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be400 con 0x563e1752c000 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aa800 con 0x563e1752c000 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 126 out=49b] v124'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be800 con 0x563e169ab800 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 126 out=49b] v124'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169ab800 2024-03-22T04:21:24.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689d400 con 0x563e1752c000 2024-03-22T04:21:24.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4677200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563e169be800 0x563e169b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aac00 con 0x563e169be800 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467c200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563e169be800 0x563e169b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467c200640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563e169be800 0x563e169b1600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169be800 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689d800 con 0x563e1752d000 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169abc00 con 0x563e169ab400 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aa800 con 0x563e169be800 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169be800 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.282+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1752c400 con 0x563e1752c800 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4676800640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x563e169abc00 0x563e169b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aa400 con 0x563e169abc00 2024-03-22T04:21:24.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x563e169abc00 0x563e169b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467d600640 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x563e169abc00 0x563e169b0b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 126 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be000 con 0x563e1752c000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 6 ==== osd_op_reply(21 [pgnls-filter start_epoch 126 out=81b] v126'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689dc00 con 0x563e1752c000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be400 con 0x563e169aa000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169ab000 con 0x563e1752c800 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689d400 con 0x563e1752c000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1752cc00 con 0x563e1752d000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689d800 con 0x563e169ab800 2024-03-22T04:21:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.1 v2:172.21.15.86:6800/3761623609 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169ab800 2024-03-22T04:21:24.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.286+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bec00 con 0x563e169ab400 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aac00 con 0x563e169ab400 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 5 ==== osd_op_reply(29 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf000 con 0x563e1752c000 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be000 con 0x563e1752c000 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 10 ==== osd_op_reply(31 [pgnls-filter start_epoch 126 out=49b] v124'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf400 con 0x563e169ab800 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.1 v2:172.21.15.86:6800/3761623609 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 126 out=49b] v124'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169ab800 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bec00 con 0x563e1752c000 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 11 ==== osd_op_reply(33 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf800 con 0x563e169be800 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.2 v2:172.21.15.86:6816/4132808680 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169be800 2024-03-22T04:21:24.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf000 con 0x563e1752d000 2024-03-22T04:21:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bfc00 con 0x563e169ab400 2024-03-22T04:21:24.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 6 ==== osd_op_reply(36 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aac00 con 0x563e169be800 2024-03-22T04:21:24.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.2 v2:172.21.15.86:6816/4132808680 4 ==== osd_op_reply(37 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169be800 2024-03-22T04:21:24.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.290+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689d800 con 0x563e1752c800 2024-03-22T04:21:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689dc00 con 0x563e169abc00 2024-03-22T04:21:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 2 ==== osd_op_reply(39 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aa800 con 0x563e1752c000 2024-03-22T04:21:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 12 ==== osd_op_reply(40 [pgnls-filter start_epoch 126 out=49b] v126'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689d400 con 0x563e1752c000 2024-03-22T04:21:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 13 ==== osd_op_reply(41 [pgnls-filter start_epoch 126 out=49b] v124'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aac00 con 0x563e169abc00 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be400 con 0x563e169ab800 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.1 v2:172.21.15.86:6800/3761623609 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 126 out=49b] v124'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169ab800 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169aa400 con 0x563e169abc00 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 4 ==== osd_op_reply(44 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf800 con 0x563e1752d000 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169ab000 con 0x563e169be800 2024-03-22T04:21:24.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.2 v2:172.21.15.86:6816/4132808680 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169be800 2024-03-22T04:21:24.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf000 con 0x563e169ab400 2024-03-22T04:21:24.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 7 ==== osd_op_reply(47 [pgnls-filter start_epoch 126 out=49b] v124'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.294+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1752cc00 con 0x563e169abc00 2024-03-22T04:21:24.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 5 ==== osd_op_reply(48 [pgnls-filter start_epoch 126 out=49b] v124'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf400 con 0x563e1752d000 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 7 ==== osd_op_reply(49 [pgnls-filter start_epoch 126 out=81b] v126'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 10 datascan.operator(): handling object 100000001f7.0 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x563e169aac00 con 0x563e1752d000 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 8 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x563e169aa400 con 0x563e1752d000 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 9 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x563e169be400 con 0x563e169ab400 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 8 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.get_frag_of: frag is * 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x563e1689d800 con 0x563e1752d000 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 10 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x563e1689dc00 con 0x563e1752d000 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 11 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 20 datascan.read_dentry: bravo_head not found in result 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-03-22T04:21:24.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.298+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf800 con 0x563e1752d000 2024-03-22T04:21:24.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 12 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v126'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467ec168c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-22T04:21:24.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf400 con 0x563e1752d000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 13 ==== osd_op_reply(56 [pgnls-filter start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf000 con 0x563e1752c000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 14 ==== osd_op_reply(57 [pgnls-filter start_epoch 126 out=49b] v124'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1752c400 con 0x563e169aa000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 3 ==== osd_op_reply(58 [pgnls-filter start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be400 con 0x563e1752d000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 14 ==== osd_op_reply(59 [pgnls-filter start_epoch 126 out=49b] v124'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d6000 con 0x563e1752d000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 15 ==== osd_op_reply(60 [pgnls-filter start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d8000 con 0x563e169aa000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 4 ==== osd_op_reply(61 [pgnls-filter start_epoch 126 out=49b] v124'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d6400 con 0x563e1752d000 2024-03-22T04:21:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d8400 con 0x563e169ab800 2024-03-22T04:21:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.1 v2:172.21.15.86:6800/3761623609 6 ==== osd_op_reply(63 [pgnls-filter start_epoch 126 out=49b] v124'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169ab800 2024-03-22T04:21:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d6800 con 0x563e1752c000 2024-03-22T04:21:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 15 ==== osd_op_reply(64 [pgnls-filter start_epoch 126 out=49b] v124'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.302+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d8800 con 0x563e169ab800 2024-03-22T04:21:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.1 v2:172.21.15.86:6800/3761623609 7 ==== osd_op_reply(65 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169ab800 2024-03-22T04:21:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d6c00 con 0x563e169aa000 2024-03-22T04:21:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 126 out=49b] v126'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d8c00 con 0x563e1752d000 2024-03-22T04:21:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 17 ==== osd_op_reply(67 [pgnls-filter start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d7000 con 0x563e169abc00 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 6 ==== osd_op_reply(68 [pgnls-filter start_epoch 126 out=49b] v126'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d9000 con 0x563e1752c800 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d7400 con 0x563e169be800 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.2 v2:172.21.15.86:6816/4132808680 6 ==== osd_op_reply(70 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169be800 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d9400 con 0x563e169aa000 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 6 ==== osd_op_reply(71 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d7800 con 0x563e169aa000 2024-03-22T04:21:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 7 ==== osd_op_reply(72 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d9800 con 0x563e1752c800 2024-03-22T04:21:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d7c00 con 0x563e1752d000 2024-03-22T04:21:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 126 out=49b] v124'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.306+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d9c00 con 0x563e169abc00 2024-03-22T04:21:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 7 ==== osd_op_reply(75 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d6000 con 0x563e169ab400 2024-03-22T04:21:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169dc000 con 0x563e169abc00 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 8 ==== osd_op_reply(77 [pgnls-filter start_epoch 126 out=49b] v124'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d6400 con 0x563e1752c800 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 8 ==== osd_op_reply(78 [pgnls-filter start_epoch 126 out=49b] v124'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169dc400 con 0x563e1752d000 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.5 v2:172.21.15.196:6808/2743139076 19 ==== osd_op_reply(79 [pgnls-filter start_epoch 126 out=49b] v124'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e1752d000 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169d8000 con 0x563e169aa000 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169dc800 con 0x563e1752c800 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 9 ==== osd_op_reply(81 [pgnls-filter start_epoch 126 out=49b] v124'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf800 con 0x563e169ab400 2024-03-22T04:21:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.4 v2:172.21.15.196:6800/655314492 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e169ab400 2024-03-22T04:21:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169dcc00 con 0x563e1752c800 2024-03-22T04:21:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 10 ==== osd_op_reply(83 [pgnls-filter start_epoch 126 out=49b] v124'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169bf400 con 0x563e169abc00 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.7 v2:172.21.15.196:6817/2847924285 9 ==== osd_op_reply(84 [pgnls-filter start_epoch 126 out=49b] v124'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e169abc00 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.310+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169dd000 con 0x563e169aa000 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 9 ==== osd_op_reply(85 [pgnls-filter start_epoch 126 out=49b] v124'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169be400 con 0x563e1752c800 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 11 ==== osd_op_reply(86 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169dd400 con 0x563e169be800 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.2 v2:172.21.15.86:6816/4132808680 7 ==== osd_op_reply(87 [pgnls-filter start_epoch 126 out=49b] v124'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169be800 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1752c400 con 0x563e1752c800 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467d600640 1 -- 172.21.15.86:0/873302335 <== osd.0 v2:172.21.15.86:6808/2750248043 12 ==== osd_op_reply(88 [pgnls-filter start_epoch 126 out=49b] v124'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722500 con 0x563e1752c800 2024-03-22T04:21:24.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169dd800 con 0x563e1752c000 2024-03-22T04:21:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 <== osd.6 v2:172.21.15.196:6816/1648471396 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 126 out=49b] v124'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722780 con 0x563e1752c000 2024-03-22T04:21:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f4676800640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e1689dc00 con 0x563e169be800 2024-03-22T04:21:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.2 v2:172.21.15.86:6816/4132808680 8 ==== osd_op_reply(90 [pgnls-filter start_epoch 126 out=49b] v124'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169be800 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f4677200640 1 -- 172.21.15.86:0/873302335 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 126 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x563e169ddc00 con 0x563e169aa000 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467c200640 1 -- 172.21.15.86:0/873302335 <== osd.3 v2:172.21.15.86:6822/2837220459 10 ==== osd_op_reply(91 [pgnls-filter start_epoch 126 out=49b] v124'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563e16722000 con 0x563e169aa000 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563e1752c800 msgr2=0x563e17380c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x563e1752c800 0x563e17380c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563e169ab800 msgr2=0x563e169b0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x563e169ab800 0x563e169b0580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563e169be800 msgr2=0x563e169b1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x563e169be800 0x563e169b1600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563e169aa000 msgr2=0x563e169b0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x563e169aa000 0x563e169b0000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 reap_dead start 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x563e169ab400 msgr2=0x563e169b1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x563e169ab400 0x563e169b1080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 reap_dead start 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563e1752d000 msgr2=0x563e17380680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563e1752d000 0x563e17380680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563e1752c000 msgr2=0x563e17381180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563e1752c000 0x563e17381180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x563e169abc00 msgr2=0x563e169b0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x563e169abc00 0x563e169b0b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563e1689c800 msgr2=0x563e17380100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563e1689c800 0x563e17380100 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x563e17334cf0 tx=0x563e1692ade0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 msgr2=0x563e1737f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x563e174d1bf0 tx=0x563e17339f80 comp rx=0 tx=0).stop 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 shutdown_connections 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467cc00640 1 -- 172.21.15.86:0/873302335 reap_dead start 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x563e169abc00 0x563e169b0b00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x563e1752d000 0x563e17380680 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x563e1689c800 0x563e17380100 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x563e1752c000 0x563e17381180 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/873302335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e1738bc00 0x563e1737f080 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 >> 172.21.15.86:0/873302335 conn(0x563e165b0c00 msgr2=0x563e165df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 shutdown_connections 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/873302335 wait complete. 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/2525042389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e165b1400 msgr2=0x563e1737e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e165b1400 0x563e1737e580 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x563e1741b020 tx=0x563e1668a180 comp rx=0 tx=0).stop 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/2525042389 shutdown_connections 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563e165b1000 0x563e1737eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e165b1400 0x563e1737e580 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 --2- 172.21.15.86:0/2525042389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e165b1800 0x563e1737e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/2525042389 >> 172.21.15.86:0/2525042389 conn(0x563e165b1c00 msgr2=0x563e165df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/2525042389 shutdown_connections 2024-03-22T04:21:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.314+0000 7f467ec168c0 1 -- 172.21.15.86:0/2525042389 wait complete. 2024-03-22T04:21:24.322 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.149832, result: 2024-03-22T04:21:24.323 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-22T04:21:24.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 -- start start 2024-03-22T04:21:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 Processor -- start 2024-03-22T04:21:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561351c4b000 0x561352a16000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561351c4b400 0x561352a16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561351c4b800 0x561352a16b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561352a24180 con 0x561351c4b000 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561352a24300 con 0x561351c4b400 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561352a24480 con 0x561351c4b800 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561351c4b000 0x561352a16000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561351c4b000 0x561352a16000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58564/0 (socket says 172.21.15.86:58564) 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 -- 172.21.15.86:0/2555384429 learned_addr learned my addr 172.21.15.86:0/2555384429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f9419e00640 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561351c4b400 0x561352a16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 -- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561351c4b800 msgr2=0x561352a16b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941b200640 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561351c4b800 0x561352a16b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561351c4b800 0x561352a16b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 -- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561351c4b400 msgr2=0x561352a16580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561351c4b400 0x561352a16580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 -- 172.21.15.86:0/2555384429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561351c701e0 con 0x561351c4b000 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f9419e00640 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561351c4b400 0x561352a16580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941b200640 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561351c4b800 0x561352a16b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941a800640 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561351c4b000 0x561352a16000 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x561352aaeff0 tx=0x561351d241e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b5e9e067453e68 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f9417600640 1 -- 172.21.15.86:0/2555384429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561351c809c0 con 0x561351c4b000 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f9417600640 1 -- 172.21.15.86:0/2555384429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561351c96e00 con 0x561351c4b000 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f9417600640 1 -- 172.21.15.86:0/2555384429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561351c804e0 con 0x561351c4b000 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 -- 172.21.15.86:0/2555384429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561351c714a0 con 0x561351c4b000 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 1 -- 172.21.15.86:0/2555384429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561351c703c0 con 0x561351c4b000 2024-03-22T04:21:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f941c72e8c0 4 waiting for MDS map... 2024-03-22T04:21:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.406+0000 7f9417600640 1 -- 172.21.15.86:0/2555384429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x561352a1aa00 con 0x561351c4b000 2024-03-22T04:21:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9417600640 1 -- 172.21.15.86:0/2555384429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x561351ca1180 con 0x561351c4b000 2024-03-22T04:21:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 4 Got MDS map 106 2024-03-22T04:21:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 4 datascan.main: Using metadata pool output 2024-03-22T04:21:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 4 datascan.main: connecting to RADOS... 2024-03-22T04:21:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 Processor -- start 2024-03-22T04:21:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- start start 2024-03-22T04:21:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39c00 0x561352a17080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bce000 0x561352a17600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bce400 0x561352a17b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561352a24600 con 0x561352bce400 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561352a24780 con 0x561352a39c00 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561352a24900 con 0x561352bce000 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bce400 0x561352a17b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bce400 0x561352a17b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58578/0 (socket says 172.21.15.86:58578) 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 -- 172.21.15.86:0/1063127818 learned_addr learned my addr 172.21.15.86:0/1063127818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941b200640 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39c00 0x561352a17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941a800640 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bce000 0x561352a17600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 -- 172.21.15.86:0/1063127818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bce000 msgr2=0x561352a17600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bce000 0x561352a17600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 -- 172.21.15.86:0/1063127818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39c00 msgr2=0x561352a17080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39c00 0x561352a17080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 -- 172.21.15.86:0/1063127818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561351c701e0 con 0x561352bce400 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9419e00640 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bce400 0x561352a17b80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x561352b74a80 tx=0x561352b7ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c14e463eeeae804 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9414e00640 1 -- 172.21.15.86:0/1063127818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561351c809c0 con 0x561352bce400 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9414e00640 1 -- 172.21.15.86:0/1063127818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561351c96a80 con 0x561352bce400 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f9414e00640 1 -- 172.21.15.86:0/1063127818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561351c804e0 con 0x561352bce400 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- 172.21.15.86:0/1063127818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bce400 msgr2=0x561352a17b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bce400 0x561352a17b80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x561352b74a80 tx=0x561352b7ccc0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- 172.21.15.86:0/1063127818 shutdown_connections 2024-03-22T04:21:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bce400 0x561352a17b80 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bce000 0x561352a17600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/1063127818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39c00 0x561352a17080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- 172.21.15.86:0/1063127818 >> 172.21.15.86:0/1063127818 conn(0x561352a39800 msgr2=0x561351c78000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- 172.21.15.86:0/1063127818 shutdown_connections 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 -- 172.21.15.86:0/1063127818 wait complete. 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.410+0000 7f941c72e8c0 1 Processor -- start 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 -- start start 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39800 0x561352a17080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bcf000 0x561352a17600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bcec00 0x561352a17b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561352a25c80 con 0x561352bcec00 2024-03-22T04:21:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561352a25b00 con 0x561352a39800 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x561352a25980 con 0x561352bcf000 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bcec00 0x561352a17b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bcec00 0x561352a17b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58584/0 (socket says 172.21.15.86:58584) 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 learned_addr learned my addr 172.21.15.86:0/4124802222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bcf000 0x561352a17600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39800 0x561352a17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bcf000 msgr2=0x561352a17600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561352bcf000 0x561352a17600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39800 msgr2=0x561352a17080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561352a39800 0x561352a17080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561351c712c0 con 0x561352bcec00 2024-03-22T04:21:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9419e00640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bcec00 0x561352a17b80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x561352b75d40 tx=0x561351d24060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9413000640 1 -- 172.21.15.86:0/4124802222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561351c80000 con 0x561352bcec00 2024-03-22T04:21:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9413000640 1 -- 172.21.15.86:0/4124802222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561351c97340 con 0x561352bcec00 2024-03-22T04:21:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561351c71a40 con 0x561352bcec00 2024-03-22T04:21:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9413000640 1 -- 172.21.15.86:0/4124802222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561351c809c0 con 0x561352bcec00 2024-03-22T04:21:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561351c71c20 con 0x561352bcec00 2024-03-22T04:21:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f941c72e8c0 4 datascan.init: resolving metadata pool 17 2024-03-22T04:21:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.414+0000 7f9413000640 1 -- 172.21.15.86:0/4124802222 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x561352b5cc00 con 0x561352bcec00 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f9413000640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x561352036000 0x561352a18100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f9413000640 1 -- 172.21.15.86:0/4124802222 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x561352b5aa00 con 0x561352bcec00 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941c72e8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941c72e8c0 4 datascan.main: resolving metadata pool 17 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x561352036000 0x561352a18100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561352bce800 0x561352a18680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352bce400 con 0x561352bce800 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561352bce800 0x561352a18680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x561352036000 0x561352a18100 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x5613529cee40 tx=0x56135203e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561352bce800 0x561352a18680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== osd_op_reply(1 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x561352bce400 0x561352044000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037000 con 0x561352bce400 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x561352bce400 0x561352044000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x561352bce400 0x561352044000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== osd_op_reply(2 [pgnls start_epoch 126 out=177b] v125'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037000 con 0x561352bce800 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 2 ==== osd_op_reply(3 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x561352037000 0x561352044580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037400 con 0x561352037000 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.418+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x561352037000 0x561352044580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x561352037000 0x561352044580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== osd_op_reply(4 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352037000 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x561352037400 0x561352044b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037800 con 0x561352037400 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f9419e00640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x561352037400 0x561352044b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f9419e00640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x561352037400 0x561352044b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== osd_op_reply(5 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f9419e00640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561352037800 0x561352045080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054000 con 0x561352037800 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561352037800 0x561352045080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561352037800 0x561352045080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== osd_op_reply(6 [pgnls start_epoch 126 out=149b] v126'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561352054000 0x561352045600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352036c00 con 0x561352054000 2024-03-22T04:21:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561352054000 0x561352045600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561352054000 0x561352045600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== osd_op_reply(7 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.422+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352036c00 con 0x561352037800 2024-03-22T04:21:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== osd_op_reply(8 [pgnls start_epoch 126 out=73b] v125'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x561352036c00 0x561352045b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054c00 con 0x561352036c00 2024-03-22T04:21:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f9419e00640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x561352036c00 0x561352045b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f9419e00640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x561352036c00 0x561352045b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== osd_op_reply(9 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20780 con 0x561352036c00 2024-03-22T04:21:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054c00 con 0x561352037000 2024-03-22T04:21:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.7 v2:172.21.15.196:6817/2847924285 2 ==== osd_op_reply(10 [pgnls start_epoch 126 out=81b] v125'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x561351d20000 con 0x561352037000 2024-03-22T04:21:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054c00 con 0x561352036c00 2024-03-22T04:21:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== osd_op_reply(11 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20780 con 0x561352036c00 2024-03-22T04:21:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054c00 con 0x561352bce400 2024-03-22T04:21:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 2 ==== osd_op_reply(12 [pgnls start_epoch 126 out=49b] v126'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054c00 con 0x561352bce800 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 3 ==== osd_op_reply(13 [pgnls start_epoch 126 out=77b] v125'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941a800640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x561352054c00 0x561352046680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352054c00 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x561352054c00 0x561352046680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x561352054c00 0x561352046680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== osd_op_reply(14 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352054c00 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.426+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352037400 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== osd_op_reply(15 [pgnls start_epoch 126 out=73b] v125'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352037400 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 3 ==== osd_op_reply(16 [pgnls start_epoch 126 out=49b] v126'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352037800 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 3 ==== osd_op_reply(17 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352bce400 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 3 ==== osd_op_reply(18 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352037000 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.7 v2:172.21.15.196:6817/2847924285 3 ==== osd_op_reply(19 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352037000 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352054000 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== osd_op_reply(20 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352bce800 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 4 ==== osd_op_reply(21 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352054000 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 3 ==== osd_op_reply(22 [pgnls start_epoch 126 out=97b] v125'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352bce800 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 5 ==== osd_op_reply(23 [pgnls start_epoch 126 out=74b] v123'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352036c00 2024-03-22T04:21:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.1 v2:172.21.15.86:6800/3761623609 3 ==== osd_op_reply(24 [pgnls start_epoch 126 out=73b] v125'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x561351d20780 con 0x561352036c00 2024-03-22T04:21:24.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.430+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352bce800 2024-03-22T04:21:24.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 6 ==== osd_op_reply(25 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352bce400 2024-03-22T04:21:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 4 ==== osd_op_reply(26 [pgnls start_epoch 126 out=98b] v125'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352037400 2024-03-22T04:21:24.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 4 ==== osd_op_reply(27 [pgnls start_epoch 126 out=76b] v126'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352054000 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 4 ==== osd_op_reply(28 [pgnls start_epoch 126 out=49b] v126'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352037800 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 4 ==== osd_op_reply(29 [pgnls start_epoch 126 out=105b] v126'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352054c00 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== osd_op_reply(30 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352054c00 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352037400 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 5 ==== osd_op_reply(31 [pgnls start_epoch 126 out=73b] v126'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352bce800 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 7 ==== osd_op_reply(32 [pgnls start_epoch 126 out=101b] v126'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352bce000 con 0x561352bce400 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352055800 con 0x561352bce400 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.434+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352055c00 con 0x561352bce400 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352055400 con 0x561352bce400 2024-03-22T04:21:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352055000 con 0x561352bce400 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352055800 con 0x561352037800 2024-03-22T04:21:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 5 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352055c00 con 0x561352037800 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352bce000 con 0x561352037000 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.7 v2:172.21.15.196:6817/2847924285 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=531b] v0'0 uv3 ondisk = 0) ==== 164+0+531 (crc 0 0 0) 0x561351d20000 con 0x561352037000 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206e000 con 0x561352037400 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 6 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-22T04:21:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206e400 con 0x561352054000 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 5 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206e800 con 0x561352036c00 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.1 v2:172.21.15.86:6800/3761623609 4 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20780 con 0x561352036c00 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-22T04:21:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-22T04:21:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.438+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206ec00 con 0x561352bce400 2024-03-22T04:21:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 10 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-22T04:21:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-22T04:21:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-22T04:21:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206f000 con 0x561352037800 2024-03-22T04:21:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-03-22T04:21:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206f400 con 0x561352037800 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1561b] v0'0 uv9 ondisk = 0) ==== 164+0+1561 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 20 datascan.scan_links: injected ino detected 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206f800 con 0x561352bce800 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 8 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x56135206fc00 con 0x561352bce800 2024-03-22T04:21:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 9 ==== osd_op_reply(48 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x56135206fc00 con 0x561352bce400 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 11 ==== osd_op_reply(49 [pgnls start_epoch 126 out=177b] v125'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x56135206fc00 con 0x561352bce800 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 10 ==== osd_op_reply(50 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x56135206fc00 con 0x561352037000 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.7 v2:172.21.15.196:6817/2847924285 5 ==== osd_op_reply(51 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352037000 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x56135206fc00 con 0x561352037400 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 7 ==== osd_op_reply(52 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.442+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037800 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 9 ==== osd_op_reply(53 [pgnls start_epoch 126 out=149b] v126'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352054000 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 6 ==== osd_op_reply(54 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037800 2024-03-22T04:21:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 10 ==== osd_op_reply(55 [pgnls start_epoch 126 out=73b] v125'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352036c00 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.1 v2:172.21.15.86:6800/3761623609 5 ==== osd_op_reply(56 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20780 con 0x561352036c00 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037000 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.7 v2:172.21.15.196:6817/2847924285 6 ==== osd_op_reply(57 [pgnls start_epoch 126 out=81b] v125'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x561351d20000 con 0x561352037000 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352036c00 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.1 v2:172.21.15.86:6800/3761623609 6 ==== osd_op_reply(58 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20780 con 0x561352036c00 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352bce400 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 12 ==== osd_op_reply(59 [pgnls start_epoch 126 out=49b] v126'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352bce800 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 11 ==== osd_op_reply(60 [pgnls start_epoch 126 out=77b] v125'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352054c00 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.2 v2:172.21.15.86:6816/4132808680 3 ==== osd_op_reply(61 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352054c00 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037400 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 8 ==== osd_op_reply(62 [pgnls start_epoch 126 out=73b] v125'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037400 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 9 ==== osd_op_reply(63 [pgnls start_epoch 126 out=49b] v126'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.446+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037800 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 11 ==== osd_op_reply(64 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352bce400 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 13 ==== osd_op_reply(65 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037000 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.7 v2:172.21.15.196:6817/2847924285 7 ==== osd_op_reply(66 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352037000 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352054000 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 7 ==== osd_op_reply(67 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352bce800 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 12 ==== osd_op_reply(68 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352054000 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 8 ==== osd_op_reply(69 [pgnls start_epoch 126 out=97b] v125'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352bce800 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 13 ==== osd_op_reply(70 [pgnls start_epoch 126 out=74b] v123'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352036c00 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.1 v2:172.21.15.86:6800/3761623609 7 ==== osd_op_reply(71 [pgnls start_epoch 126 out=73b] v125'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x561351d20780 con 0x561352036c00 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352bce800 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 14 ==== osd_op_reply(72 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352bce400 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 14 ==== osd_op_reply(73 [pgnls start_epoch 126 out=98b] v125'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037400 2024-03-22T04:21:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 10 ==== osd_op_reply(74 [pgnls start_epoch 126 out=76b] v126'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.450+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352054000 2024-03-22T04:21:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 9 ==== osd_op_reply(75 [pgnls start_epoch 126 out=49b] v126'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037800 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 12 ==== osd_op_reply(76 [pgnls start_epoch 126 out=105b] v126'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352054c00 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.2 v2:172.21.15.86:6816/4132808680 4 ==== osd_op_reply(77 [pgnls start_epoch 126 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561351d20500 con 0x561352054c00 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037400 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 11 ==== osd_op_reply(78 [pgnls start_epoch 126 out=73b] v126'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 126 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352bce800 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 15 ==== osd_op_reply(79 [pgnls start_epoch 126 out=101b] v126'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206e000 con 0x561352bce400 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 15 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206e800 con 0x561352bce400 2024-03-22T04:21:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 16 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206fc00 con 0x561352bce400 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 17 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206ec00 con 0x561352bce400 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 18 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206f000 con 0x561352bce400 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 19 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-22T04:21:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.454+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352055c00 con 0x561352037800 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 13 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352055c00 con 0x561352037800 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 14 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206f400 con 0x561352037000 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.7 v2:172.21.15.196:6817/2847924285 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=531b] v0'0 uv3 ondisk = 0) ==== 164+0+531 (crc 0 0 0) 0x561351d20000 con 0x561352037000 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206e000 con 0x561352037400 2024-03-22T04:21:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.6 v2:172.21.15.196:6816/1648471396 12 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20780 con 0x561352037400 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206e800 con 0x561352054000 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.3 v2:172.21.15.86:6822/2837220459 10 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20000 con 0x561352054000 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206fc00 con 0x561352036c00 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 <== osd.1 v2:172.21.15.86:6800/3761623609 8 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20780 con 0x561352036c00 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206ec00 con 0x561352bce400 2024-03-22T04:21:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 20 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206f000 con 0x561352037800 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206e400 con 0x561352037800 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1561b] v0'0 uv9 ondisk = 0) ==== 164+0+1561 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 20 datascan.scan_links: injected ino detected 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-22T04:21:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.458+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206f800 con 0x561352bce800 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 16 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 datascan.scan_links: processing 1 injected_inos 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x56135206f400 con 0x561352037800 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 20 datascan.scan_links: first is now 2 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561352054800 con 0x561352037800 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.5 v2:172.21.15.196:6808/2743139076 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v126'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x561351d20500 con 0x561352037800 2024-03-22T04:21:24.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-22T04:21:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 datascan.scan_links: updating inotable 2024-03-22T04:21:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352bce000 con 0x561352bce400 2024-03-22T04:21:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-03-22T04:21:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-03-22T04:21:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352bce400 2024-03-22T04:21:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941b200640 1 -- 172.21.15.86:0/4124802222 <== osd.4 v2:172.21.15.196:6800/655314492 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v126'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x561351d20500 con 0x561352bce400 2024-03-22T04:21:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 10 datascan.scan_links: updating snaptable 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.462+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) -- 0x561352037c00 con 0x561352bce800 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941a800640 1 -- 172.21.15.86:0/4124802222 <== osd.0 v2:172.21.15.86:6808/2750248043 17 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x561351d20000 con 0x561352bce800 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561352bce800 msgr2=0x561352a18680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x561352bce800 0x561352a18680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x561352036c00 msgr2=0x561352045b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x561352036c00 0x561352045b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x561352054c00 msgr2=0x561352046680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x561352054c00 0x561352046680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561352054000 msgr2=0x561352045600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x561352054000 0x561352045600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x561352bce400 msgr2=0x561352044000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x561352bce400 0x561352044000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561352037800 msgr2=0x561352045080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x561352037800 0x561352045080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x561352037400 msgr2=0x561352044b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x561352037400 0x561352044b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x561352037000 msgr2=0x561352044580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x561352037000 0x561352044580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 reap_dead start 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 reap_dead start 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 reap_dead start 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f9419e00640 1 -- 172.21.15.86:0/4124802222 reap_dead start 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x561352036000 msgr2=0x561352a18100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x561352036000 0x561352a18100 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x5613529cee40 tx=0x56135203e000 comp rx=0 tx=0).stop 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bcec00 msgr2=0x561352a17b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bcec00 0x561352a17b80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x561352b75d40 tx=0x561351d24060 comp rx=0 tx=0).stop 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 shutdown_connections 2024-03-22T04:21:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x561352036000 0x561352a18100 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/4124802222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561352bcec00 0x561352a17b80 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 >> 172.21.15.86:0/4124802222 conn(0x561352a39c00 msgr2=0x561351c78000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 shutdown_connections 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/4124802222 wait complete. 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/2555384429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561351c4b000 msgr2=0x561352a16000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561351c4b000 0x561352a16000 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x561352aaeff0 tx=0x561351d241e0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/2555384429 shutdown_connections 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561351c4b400 0x561352a16580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561351c4b800 0x561352a16b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 --2- 172.21.15.86:0/2555384429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561351c4b000 0x561352a16000 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/2555384429 >> 172.21.15.86:0/2555384429 conn(0x561351c4ac00 msgr2=0x561351c77800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/2555384429 shutdown_connections 2024-03-22T04:21:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.466+0000 7f941c72e8c0 1 -- 172.21.15.86:0/2555384429 wait complete. 2024-03-22T04:21:24.477 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.154058, result: 2024-03-22T04:21:24.477 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:21:24.521 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:21:24.559 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-03-22T04:21:24.559 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-03-22T04:21:24.559 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:21:24.559 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:21:24.560 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-03-22T04:21:24.561 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-03-22T04:21:24.562 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.563 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.564 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-03-22T04:21:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:24.566 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.567 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-03-22T04:21:24.568 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-03-22T04:21:24.569 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = False 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:21:24.570 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-03-22T04:21:24.571 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-03-22T04:21:24.572 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on skipped message = False 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.573 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-03-22T04:21:24.574 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-03-22T04:21:24.575 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:21:24.576 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-03-22T04:21:24.577 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: mon down mkfs grace = 300 2024-03-22T04:21:24.578 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:21:24.579 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = False 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:21:24.580 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:21:24.581 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:21:24.583 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:21:24.628 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.667 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.668 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.669 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:21:24.670 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-03-22T04:21:24.671 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-03-22T04:21:24.672 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.673 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-03-22T04:21:24.674 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:21:24.675 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-03-22T04:21:24.676 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-03-22T04:21:24.677 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-22T04:21:24.678 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-03-22T04:21:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-22T04:21:24.680 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.86,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on skipped message = False 2024-03-22T04:21:24.681 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 1365483d-2fb4-4579-a50b-2ce299f625b2 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.682 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-03-22T04:21:24.683 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.684 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-03-22T04:21:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.686 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: mon down mkfs grace = 300 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-03-22T04:21:24.687 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-03-22T04:21:24.688 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-03-22T04:21:24.690 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-22T04:21:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac2e200640 1 Processor -- start 2024-03-22T04:21:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac2e200640 1 -- start start 2024-03-22T04:21:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac2e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac2815e7e0 0x7fac280d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac2e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac280b4710 0x7fac2806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac2e200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac280d15f0 con 0x7fac28070390 2024-03-22T04:21:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac2e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fac280b4b70 con 0x7fac280b4710 2024-03-22T04:21:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac2e200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fac2806f970 con 0x7fac2815e7e0 2024-03-22T04:21:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac1ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac280b4710 0x7fac2806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac1ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac280b4710 0x7fac2806fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58586/0 (socket says 172.21.15.86:58586) 2024-03-22T04:21:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac1ea00640 1 -- 172.21.15.86:0/3181554766 learned_addr learned my addr 172.21.15.86:0/3181554766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac27400640 1 --2- 172.21.15.86:0/3181554766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac2815e7e0 0x7fac280d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac1ea00640 1 -- 172.21.15.86:0/3181554766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac2815e7e0 msgr2=0x7fac280d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:24.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac1ea00640 1 --2- 172.21.15.86:0/3181554766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac2815e7e0 0x7fac280d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac1ea00640 1 -- 172.21.15.86:0/3181554766 >> v1:172.21.15.196:6789/0 conn(0x7fac28070390 legacy=0x7fac2809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.870+0000 7fac1ea00640 1 -- 172.21.15.86:0/3181554766 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac28070820 con 0x7fac280b4710 2024-03-22T04:21:24.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac1ea00640 1 --2- 172.21.15.86:0/3181554766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b4710 0x7fac2806fe50 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fac10009e50 tx=0x7fac1001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1580f91538bb1f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac26a00640 1 -- 172.21.15.86:0/3181554766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac100a56c0 con 0x7fac280b4710 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac26a00640 1 -- 172.21.15.86:0/3181554766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac100a5860 con 0x7fac280b4710 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac26a00640 1 -- 172.21.15.86:0/3181554766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac100a5c30 con 0x7fac280b4710 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac2e200640 1 -- 172.21.15.86:0/3181554766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b4710 msgr2=0x7fac2806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac2e200640 1 --2- 172.21.15.86:0/3181554766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b4710 0x7fac2806fe50 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fac10009e50 tx=0x7fac1001c8e0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac2e200640 1 -- 172.21.15.86:0/3181554766 shutdown_connections 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac2e200640 1 --2- 172.21.15.86:0/3181554766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b4710 0x7fac2806fe50 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac2e200640 1 --2- 172.21.15.86:0/3181554766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac2815e7e0 0x7fac280d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac2e200640 1 -- 172.21.15.86:0/3181554766 >> 172.21.15.86:0/3181554766 conn(0x7fac280b3bf0 msgr2=0x7fac280b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac2e200640 1 -- 172.21.15.86:0/3181554766 shutdown_connections 2024-03-22T04:21:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.874+0000 7fac2e200640 1 -- 172.21.15.86:0/3181554766 wait complete. 2024-03-22T04:21:24.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 Processor -- start 2024-03-22T04:21:24.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 -- start start 2024-03-22T04:21:24.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28070390 0x7fac28152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac280b4710 0x7fac280b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac2815e7e0 0x7fac280b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac280ce370 con 0x7fac28070390 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac280cce40 con 0x7fac2815e7e0 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fac280ce4f0 con 0x7fac280b4710 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28070390 0x7fac28152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28070390 0x7fac28152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58600/0 (socket says 172.21.15.86:58600) 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 -- 172.21.15.86:0/1000940066 learned_addr learned my addr 172.21.15.86:0/1000940066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac1ea00640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac280b4710 0x7fac280b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27e00640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac2815e7e0 0x7fac280b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 -- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac280b4710 msgr2=0x7fac280b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac280b4710 0x7fac280b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 -- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac2815e7e0 msgr2=0x7fac280b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac2815e7e0 0x7fac280b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 -- 172.21.15.86:0/1000940066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac28153770 con 0x7fac28070390 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27e00640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac2815e7e0 0x7fac280b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac27400640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28070390 0x7fac28152f20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fac1800e930 tx=0x7fac1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac1fe00640 1 -- 172.21.15.86:0/1000940066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18017070 con 0x7fac28070390 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac1fe00640 1 -- 172.21.15.86:0/1000940066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1800ca00 con 0x7fac28070390 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac28153a50 con 0x7fac28070390 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac1fe00640 1 -- 172.21.15.86:0/1000940066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1800cdb0 con 0x7fac28070390 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac28154010 con 0x7fac28070390 2024-03-22T04:21:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.878+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac28074470 con 0x7fac28070390 2024-03-22T04:21:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.882+0000 7fac1fe00640 1 -- 172.21.15.86:0/1000940066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fac1801a070 con 0x7fac28070390 2024-03-22T04:21:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.882+0000 7fac1fe00640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac04067b20 0x7fac04069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.882+0000 7fac1fe00640 1 -- 172.21.15.86:0/1000940066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7fac18086510 con 0x7fac28070390 2024-03-22T04:21:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.882+0000 7fac1ea00640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac04067b20 0x7fac04069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.882+0000 7fac1ea00640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac04067b20 0x7fac04069fe0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac100ae000 tx=0x7fac100a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:24.886+0000 7fac1fe00640 1 -- 172.21.15.86:0/1000940066 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fac180566b0 con 0x7fac28070390 2024-03-22T04:21:25.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.078+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fac2806f750 con 0x7fac28070390 2024-03-22T04:21:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.930+0000 7fac1fe00640 1 -- 172.21.15.86:0/1000940066 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) ==== 162+0+0 (secure 0 0 0) 0x7fac1805a750 con 0x7fac28070390 2024-03-22T04:21:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac04067b20 msgr2=0x7fac04069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac04067b20 0x7fac04069fe0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac100ae000 tx=0x7fac100a7070 comp rx=0 tx=0).stop 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28070390 msgr2=0x7fac28152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28070390 0x7fac28152f20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fac1800e930 tx=0x7fac1800edf0 comp rx=0 tx=0).stop 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 shutdown_connections 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fac04067b20 0x7fac04069fe0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac2815e7e0 0x7fac280b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac280b4710 0x7fac280b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 --2- 172.21.15.86:0/1000940066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28070390 0x7fac28152f20 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 >> 172.21.15.86:0/1000940066 conn(0x7fac280b3bf0 msgr2=0x7fac2809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 shutdown_connections 2024-03-22T04:21:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:25.934+0000 7fac2e200640 1 -- 172.21.15.86:0/1000940066 wait complete. 2024-03-22T04:21:25.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:26.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.098+0000 7f4f6dc00640 1 Processor -- start 2024-03-22T04:21:26.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.098+0000 7f4f6dc00640 1 -- start start 2024-03-22T04:21:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.098+0000 7f4f6dc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4f68161880 0x7f4f680b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.098+0000 7f4f6dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f6806e350 con 0x7f4f6806de10 2024-03-22T04:21:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.098+0000 7f4f6dc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f6806e630 con 0x7f4f68161c60 2024-03-22T04:21:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.098+0000 7f4f6dc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4f6815ee80 con 0x7f4f68161880 2024-03-22T04:21:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f67400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4f6806de10 0x7f4f6815fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39370/0 (socket says 172.21.15.86:39370) 2024-03-22T04:21:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f67400640 1 -- 172.21.15.86:0/3454052915 learned_addr learned my addr 172.21.15.86:0/3454052915 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3621233168 0 0) 0x7f4f6806e350 con 0x7f4f6806de10 2024-03-22T04:21:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f4c003530 con 0x7f4f6806de10 2024-03-22T04:21:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66a00640 1 --2- 172.21.15.86:0/3454052915 >> v2:172.21.15.196:3300/0 conn(0x7f4f68161880 0x7f4f680b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2158694217 0 0) 0x7f4f4c003530 con 0x7f4f6806de10 2024-03-22T04:21:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 >> v2:172.21.15.196:3300/0 conn(0x7f4f68161880 msgr2=0x7f4f680b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:26.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 --2- 172.21.15.86:0/3454052915 >> v2:172.21.15.196:3300/0 conn(0x7f4f68161880 0x7f4f680b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 >> v1:172.21.15.196:6789/0 conn(0x7f4f68161c60 legacy=0x7f4f6806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6806e910 con 0x7f4f6806de10 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f4f60001680 con 0x7f4f6806de10 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 >> v1:172.21.15.86:6789/0 conn(0x7f4f6806de10 legacy=0x7f4f6815fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c0039c0 0x7f4f4c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f4c004810 0x7f4f4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f4c0051d0 0x7f4f4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6815ee80 con 0x7f4f4c004810 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c00c170 con 0x7f4f4c0039c0 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c00c370 con 0x7f4f4c0051d0 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f5e000640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f4c004810 0x7f4f4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66a00640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c0039c0 0x7f4f4c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f67400640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f4c0051d0 0x7f4f4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f5e000640 1 -- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f4c0051d0 msgr2=0x7f4f4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:26.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f5e000640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f4c0051d0 0x7f4f4c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f5e000640 1 -- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c0039c0 msgr2=0x7f4f4c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f5e000640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c0039c0 0x7f4f4c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f5e000640 1 -- 172.21.15.86:0/3454052915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c003530 con 0x7f4f4c004810 2024-03-22T04:21:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f66a00640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c0039c0 0x7f4f4c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.102+0000 7f4f67400640 1 -- 172.21.15.86:0/3454052915 reap_dead start 2024-03-22T04:21:26.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f5e000640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f4c004810 0x7f4f4c004c90 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4f50001e70 tx=0x7f4f5001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638b3132806abe73 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50001a90 con 0x7f4f4c004810 2024-03-22T04:21:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50003b80 con 0x7f4f4c004810 2024-03-22T04:21:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f66000640 1 -- 172.21.15.86:0/3454052915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50005080 con 0x7f4f4c004810 2024-03-22T04:21:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 -- 172.21.15.86:0/3454052915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f4c004810 msgr2=0x7f4f4c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f4c004810 0x7f4f4c004c90 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4f50001e70 tx=0x7f4f5001c730 comp rx=0 tx=0).stop 2024-03-22T04:21:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 -- 172.21.15.86:0/3454052915 shutdown_connections 2024-03-22T04:21:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 --2- 172.21.15.86:0/3454052915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f4c004810 0x7f4f4c004c90 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 -- 172.21.15.86:0/3454052915 >> 172.21.15.86:0/3454052915 conn(0x7f4f680b1df0 msgr2=0x7f4f680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 -- 172.21.15.86:0/3454052915 shutdown_connections 2024-03-22T04:21:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 -- 172.21.15.86:0/3454052915 wait complete. 2024-03-22T04:21:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 Processor -- start 2024-03-22T04:21:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.106+0000 7f4f6dc00640 1 -- start start 2024-03-22T04:21:26.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c004810 0x7f4f68159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:26.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f681560f0 0x7f4f681f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68155950 0x7f4f680ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f68162250 con 0x7f4f681560f0 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681620d0 con 0x7f4f4c004810 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f681623d0 con 0x7f4f68155950 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f681560f0 0x7f4f681f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f681560f0 0x7f4f681f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45022/0 (socket says 172.21.15.86:45022) 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 -- 172.21.15.86:0/919190574 learned_addr learned my addr 172.21.15.86:0/919190574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f66a00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c004810 0x7f4f68159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f67400640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68155950 0x7f4f680ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 -- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68155950 msgr2=0x7f4f680ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68155950 0x7f4f680ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 -- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c004810 msgr2=0x7f4f68159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c004810 0x7f4f68159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 -- 172.21.15.86:0/919190574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f681f7990 con 0x7f4f681560f0 2024-03-22T04:21:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f66a00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c004810 0x7f4f68159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5e000640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f681560f0 0x7f4f681f8450 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4f5001cc10 tx=0x7f4f4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5f400640 1 -- 172.21.15.86:0/919190574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50001a90 con 0x7f4f681560f0 2024-03-22T04:21:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5f400640 1 -- 172.21.15.86:0/919190574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50003d30 con 0x7f4f681560f0 2024-03-22T04:21:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5f400640 1 -- 172.21.15.86:0/919190574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50006b60 con 0x7f4f681560f0 2024-03-22T04:21:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6809a310 con 0x7f4f681560f0 2024-03-22T04:21:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681f7c50 con 0x7f4f681560f0 2024-03-22T04:21:26.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f68068650 con 0x7f4f681560f0 2024-03-22T04:21:26.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.110+0000 7f4f5f400640 1 -- 172.21.15.86:0/919190574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4f50006d00 con 0x7f4f681560f0 2024-03-22T04:21:26.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.114+0000 7f4f5f400640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b50 0x7f4f4406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:26.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.114+0000 7f4f5f400640 1 -- 172.21.15.86:0/919190574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f4f50120070 con 0x7f4f681560f0 2024-03-22T04:21:26.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.114+0000 7f4f66a00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b50 0x7f4f4406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.114+0000 7f4f66a00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b50 0x7f4f4406a010 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009bf0 tx=0x7f4f58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.118+0000 7f4f5f400640 1 -- 172.21.15.86:0/919190574 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4f500c1070 con 0x7f4f681560f0 2024-03-22T04:21:26.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f680cfdb0 con 0x7f4f681560f0 2024-03-22T04:21:26.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f5f400640 1 -- 172.21.15.86:0/919190574 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5187 (secure 0 0 0) 0x7f4f500b6030 con 0x7f4f681560f0 2024-03-22T04:21:26.308 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:26.308 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:25.938369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.86:6837/3342090879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3342090879},{"type":"v1","addr":"172.21.15.86:6837","nonce":3342090879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-22T04:21:26.308 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 109 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b50 msgr2=0x7f4f4406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b50 0x7f4f4406a010 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009bf0 tx=0x7f4f58008040 comp rx=0 tx=0).stop 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f681560f0 msgr2=0x7f4f681f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f681560f0 0x7f4f681f8450 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4f5001cc10 tx=0x7f4f4c005840 comp rx=0 tx=0).stop 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 shutdown_connections 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f44067b50 0x7f4f4406a010 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f68155950 0x7f4f680ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f681560f0 0x7f4f681f8450 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 --2- 172.21.15.86:0/919190574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f4c004810 0x7f4f68159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:26.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 >> 172.21.15.86:0/919190574 conn(0x7f4f680b1df0 msgr2=0x7f4f68072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 shutdown_connections 2024-03-22T04:21:26.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:26.306+0000 7f4f6dc00640 1 -- 172.21.15.86:0/919190574 wait complete. 2024-03-22T04:21:26.731 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:20:57.561818+0000', 'modified': '2024-03-22T04:21:25.938369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9124}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9124': {'gid': 9124, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/3342090879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 3342090879}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3342090879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:21:26.732 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9124:up:replay 2024-03-22T04:21:27.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:27.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa394e00640 1 Processor -- start 2024-03-22T04:21:27.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa394e00640 1 -- start start 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa394e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa3900ce6b0 0x7fa3900cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa394e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa39015fed0 0x7fa3901602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa394e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3900d11c0 con 0x7fa3900d0de0 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa394e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa390160820 con 0x7fa3900ce6b0 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa394e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa3900cece0 con 0x7fa39015fed0 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa38d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa3900ce6b0 0x7fa3900cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa38d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa3900ce6b0 0x7fa3900cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45032/0 (socket says 172.21.15.86:45032) 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa38d400640 1 -- 172.21.15.86:0/1748685317 learned_addr learned my addr 172.21.15.86:0/1748685317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:27.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa38e800640 1 --2- 172.21.15.86:0/1748685317 >> v2:172.21.15.196:3300/0 conn(0x7fa39015fed0 0x7fa3901602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:27.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa38d400640 1 -- 172.21.15.86:0/1748685317 >> v2:172.21.15.196:3300/0 conn(0x7fa39015fed0 msgr2=0x7fa3901602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:27.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa38d400640 1 --2- 172.21.15.86:0/1748685317 >> v2:172.21.15.196:3300/0 conn(0x7fa39015fed0 0x7fa3901602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:27.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa38d400640 1 -- 172.21.15.86:0/1748685317 >> v1:172.21.15.196:6789/0 conn(0x7fa3900d0de0 legacy=0x7fa3900d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:27.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.878+0000 7fa38d400640 1 -- 172.21.15.86:0/1748685317 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3900cefc0 con 0x7fa3900ce6b0 2024-03-22T04:21:27.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa38d400640 1 --2- 172.21.15.86:0/1748685317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900ce6b0 0x7fa3900cfe70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fa3900cc880 tx=0x7fa37c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7139587612d9b07e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:27.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa38ca00640 1 -- 172.21.15.86:0/1748685317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0b0070 con 0x7fa3900ce6b0 2024-03-22T04:21:27.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa38ca00640 1 -- 172.21.15.86:0/1748685317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37c0ab600 con 0x7fa3900ce6b0 2024-03-22T04:21:27.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa38ca00640 1 -- 172.21.15.86:0/1748685317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0ab9d0 con 0x7fa3900ce6b0 2024-03-22T04:21:27.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 -- 172.21.15.86:0/1748685317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900ce6b0 msgr2=0x7fa3900cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:27.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 --2- 172.21.15.86:0/1748685317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900ce6b0 0x7fa3900cfe70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fa3900cc880 tx=0x7fa37c01c520 comp rx=0 tx=0).stop 2024-03-22T04:21:27.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 -- 172.21.15.86:0/1748685317 shutdown_connections 2024-03-22T04:21:27.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 --2- 172.21.15.86:0/1748685317 >> v2:172.21.15.196:3300/0 conn(0x7fa39015fed0 0x7fa3901602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:27.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 --2- 172.21.15.86:0/1748685317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900ce6b0 0x7fa3900cfe70 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:27.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 -- 172.21.15.86:0/1748685317 >> 172.21.15.86:0/1748685317 conn(0x7fa39006f910 msgr2=0x7fa39006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:27.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 -- 172.21.15.86:0/1748685317 shutdown_connections 2024-03-22T04:21:27.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 -- 172.21.15.86:0/1748685317 wait complete. 2024-03-22T04:21:27.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 Processor -- start 2024-03-22T04:21:27.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.882+0000 7fa394e00640 1 -- start start 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3900ce6b0 0x7fa390155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900d0de0 0x7fa390151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa39015fed0 0x7fa390152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3900d0530 con 0x7fa3900d0de0 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3900d03b0 con 0x7fa3900ce6b0 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa390071e30 con 0x7fa39015fed0 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900d0de0 0x7fa390151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900d0de0 0x7fa390151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45038/0 (socket says 172.21.15.86:45038) 2024-03-22T04:21:27.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 -- 172.21.15.86:0/3846216891 learned_addr learned my addr 172.21.15.86:0/3846216891 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:27.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 -- 172.21.15.86:0/3846216891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa39015fed0 msgr2=0x7fa390152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:27.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa39015fed0 0x7fa390152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:27.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 -- 172.21.15.86:0/3846216891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3900ce6b0 msgr2=0x7fa390155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:27.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3900ce6b0 0x7fa390155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:27.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 -- 172.21.15.86:0/3846216891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa390152d60 con 0x7fa3900d0de0 2024-03-22T04:21:27.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa38d400640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900d0de0 0x7fa390151790 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fa37c01ca00 tx=0x7fa37c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:27.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa386000640 1 -- 172.21.15.86:0/3846216891 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0ba070 con 0x7fa3900d0de0 2024-03-22T04:21:27.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa386000640 1 -- 172.21.15.86:0/3846216891 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37c0b4430 con 0x7fa3900d0de0 2024-03-22T04:21:27.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa390154740 con 0x7fa3900d0de0 2024-03-22T04:21:27.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa386000640 1 -- 172.21.15.86:0/3846216891 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0b0040 con 0x7fa3900d0de0 2024-03-22T04:21:27.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa390154d00 con 0x7fa3900d0de0 2024-03-22T04:21:27.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa390154530 con 0x7fa3900d0de0 2024-03-22T04:21:27.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.886+0000 7fa386000640 1 -- 172.21.15.86:0/3846216891 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa37c0bd070 con 0x7fa3900d0de0 2024-03-22T04:21:27.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.890+0000 7fa386000640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa370067b50 0x7fa37006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:27.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.890+0000 7fa386000640 1 -- 172.21.15.86:0/3846216891 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7fa37c11f0a0 con 0x7fa3900d0de0 2024-03-22T04:21:27.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.890+0000 7fa38de00640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa370067b50 0x7fa37006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:27.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.890+0000 7fa38de00640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa370067b50 0x7fa37006a010 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa380000f70 tx=0x7fa3800079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:27.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:27.894+0000 7fa386000640 1 -- 172.21.15.86:0/3846216891 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa37c016110 con 0x7fa3900d0de0 2024-03-22T04:21:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa390068650 con 0x7fa3900d0de0 2024-03-22T04:21:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa386000640 1 -- 172.21.15.86:0/3846216891 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5187 (secure 0 0 0) 0x7fa37c0ef180 con 0x7fa3900d0de0 2024-03-22T04:21:28.088 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:28.088 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:26.954823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.86:6837/3342090879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3342090879},{"type":"v1","addr":"172.21.15.86:6837","nonce":3342090879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-22T04:21:28.088 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 111 2024-03-22T04:21:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa370067b50 msgr2=0x7fa37006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa370067b50 0x7fa37006a010 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa380000f70 tx=0x7fa3800079e0 comp rx=0 tx=0).stop 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900d0de0 msgr2=0x7fa390151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900d0de0 0x7fa390151790 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fa37c01ca00 tx=0x7fa37c0192c0 comp rx=0 tx=0).stop 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 shutdown_connections 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa370067b50 0x7fa37006a010 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa39015fed0 0x7fa390152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3900d0de0 0x7fa390151790 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 --2- 172.21.15.86:0/3846216891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3900ce6b0 0x7fa390155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 >> 172.21.15.86:0/3846216891 conn(0x7fa39006f910 msgr2=0x7fa39015b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 shutdown_connections 2024-03-22T04:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:28.086+0000 7fa394e00640 1 -- 172.21.15.86:0/3846216891 wait complete. 2024-03-22T04:21:28.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:20:57.561818+0000', 'modified': '2024-03-22T04:21:26.954823+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9124}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9124': {'gid': 9124, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.86:6837/3342090879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 3342090879}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3342090879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-22T04:21:28.512 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9124:up:rejoin 2024-03-22T04:21:29.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:21:29.132 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:21:29.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:29.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f982aa00640 1 Processor -- start 2024-03-22T04:21:29.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f982aa00640 1 -- start start 2024-03-22T04:21:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f982aa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f982415e900 0x7f9824162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f982aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98240d41c0 0x7f98240d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f982aa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98240d4bb0 con 0x7f982415e520 2024-03-22T04:21:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f982aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98240cf760 con 0x7f98240d41c0 2024-03-22T04:21:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f982aa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f98240d3390 con 0x7f982415e900 2024-03-22T04:21:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f9828a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98240d41c0 0x7f98240d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f9828a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98240d41c0 0x7f98240d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45050/0 (socket says 172.21.15.86:45050) 2024-03-22T04:21:29.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.646+0000 7f9828a00640 1 -- 172.21.15.86:0/988421345 learned_addr learned my addr 172.21.15.86:0/988421345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:29.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f981b400640 1 --2- 172.21.15.86:0/988421345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f982415e900 0x7f9824162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:29.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f9828a00640 1 -- 172.21.15.86:0/988421345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f982415e900 msgr2=0x7f9824162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:29.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f9828a00640 1 --2- 172.21.15.86:0/988421345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f982415e900 0x7f9824162d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f9828a00640 1 -- 172.21.15.86:0/988421345 >> v1:172.21.15.196:6789/0 conn(0x7f982415e520 legacy=0x7f98240cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:29.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f9828a00640 1 -- 172.21.15.86:0/988421345 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982406cd70 con 0x7f98240d41c0 2024-03-22T04:21:29.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f9828a00640 1 --2- 172.21.15.86:0/988421345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98240d41c0 0x7f98240d4670 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f981c009e50 tx=0x7f981c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccf8ecb0a128d7a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:29.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f9823400640 1 -- 172.21.15.86:0/988421345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0a58e0 con 0x7f98240d41c0 2024-03-22T04:21:29.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f9823400640 1 -- 172.21.15.86:0/988421345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c0a5a80 con 0x7f98240d41c0 2024-03-22T04:21:29.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f9823400640 1 -- 172.21.15.86:0/988421345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0a5e50 con 0x7f98240d41c0 2024-03-22T04:21:29.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f982aa00640 1 -- 172.21.15.86:0/988421345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98240d41c0 msgr2=0x7f98240d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:29.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.650+0000 7f982aa00640 1 --2- 172.21.15.86:0/988421345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98240d41c0 0x7f98240d4670 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f981c009e50 tx=0x7f981c01ca10 comp rx=0 tx=0).stop 2024-03-22T04:21:29.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 -- 172.21.15.86:0/988421345 shutdown_connections 2024-03-22T04:21:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 --2- 172.21.15.86:0/988421345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f982415e900 0x7f9824162d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 --2- 172.21.15.86:0/988421345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98240d41c0 0x7f98240d4670 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 -- 172.21.15.86:0/988421345 >> 172.21.15.86:0/988421345 conn(0x7f982408c000 msgr2=0x7f982409d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 -- 172.21.15.86:0/988421345 shutdown_connections 2024-03-22T04:21:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 -- 172.21.15.86:0/988421345 wait complete. 2024-03-22T04:21:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 Processor -- start 2024-03-22T04:21:29.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 -- start start 2024-03-22T04:21:29.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98240d41c0 0x7f9824154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:29.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f982415e520 0x7f98240c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:29.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982415e900 0x7f98241555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:29.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98240d2b70 con 0x7f982415e900 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98240d0220 con 0x7f982415e520 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f982aa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f98240d2cf0 con 0x7f98240d41c0 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982415e900 0x7f98241555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982415e900 0x7f98241555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45056/0 (socket says 172.21.15.86:45056) 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 -- 172.21.15.86:0/638060069 learned_addr learned my addr 172.21.15.86:0/638060069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9823e00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98240d41c0 0x7f9824154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f981b400640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f982415e520 0x7f98240c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 -- 172.21.15.86:0/638060069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98240d41c0 msgr2=0x7f9824154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98240d41c0 0x7f9824154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 -- 172.21.15.86:0/638060069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f982415e520 msgr2=0x7f98240c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f982415e520 0x7f98240c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 -- 172.21.15.86:0/638060069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98240c7f70 con 0x7f982415e900 2024-03-22T04:21:29.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.654+0000 7f9828a00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982415e900 0x7f98241555e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f981c0af040 tx=0x7f981c0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:29.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f9820c00640 1 -- 172.21.15.86:0/638060069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0bc020 con 0x7f982415e900 2024-03-22T04:21:29.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f9820c00640 1 -- 172.21.15.86:0/638060069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c007dd0 con 0x7f982415e900 2024-03-22T04:21:29.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f9820c00640 1 -- 172.21.15.86:0/638060069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0a58e0 con 0x7f982415e900 2024-03-22T04:21:29.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98240c8250 con 0x7f982415e900 2024-03-22T04:21:29.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98240c8810 con 0x7f982415e900 2024-03-22T04:21:29.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9824068650 con 0x7f982415e900 2024-03-22T04:21:29.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f9820c00640 1 -- 172.21.15.86:0/638060069 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f981c0bf020 con 0x7f982415e900 2024-03-22T04:21:29.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f9820c00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9800067b20 0x7f9800069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:29.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f9820c00640 1 -- 172.21.15.86:0/638060069 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f981c121500 con 0x7f982415e900 2024-03-22T04:21:29.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f9823e00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9800067b20 0x7f9800069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:29.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.658+0000 7f9823e00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9800067b20 0x7f9800069fe0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9814001810 tx=0x7f9814009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:29.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.662+0000 7f9820c00640 1 -- 172.21.15.86:0/638060069 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f981c0c6030 con 0x7f982415e900 2024-03-22T04:21:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.854+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98240cb2d0 con 0x7f982415e900 2024-03-22T04:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.854+0000 7f9820c00640 1 -- 172.21.15.86:0/638060069 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5194 (secure 0 0 0) 0x7f981c016600 con 0x7f982415e900 2024-03-22T04:21:29.855 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:29.856 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:28.964789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3342090879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3342090879},{"type":"v1","addr":"172.21.15.86:6837","nonce":3342090879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9124,"qdb_cluster":[9124]},"id":8}]} 2024-03-22T04:21:29.856 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-03-22T04:21:29.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9800067b20 msgr2=0x7f9800069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9800067b20 0x7f9800069fe0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9814001810 tx=0x7f9814009040 comp rx=0 tx=0).stop 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982415e900 msgr2=0x7f98241555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982415e900 0x7f98241555e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f981c0af040 tx=0x7f981c0197b0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 shutdown_connections 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9800067b20 0x7f9800069fe0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f982415e520 0x7f98240c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982415e900 0x7f98241555e0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 --2- 172.21.15.86:0/638060069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98240d41c0 0x7f9824154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 >> 172.21.15.86:0/638060069 conn(0x7f982408c000 msgr2=0x7f982406cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 shutdown_connections 2024-03-22T04:21:29.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:29.858+0000 7f982aa00640 1 -- 172.21.15.86:0/638060069 wait complete. 2024-03-22T04:21:29.935 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-03-22T04:21:29.934+0000 7f8705400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:21:30.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-22T04:20:57.561818+0000', 'modified': '2024-03-22T04:21:28.964789+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9124}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9124': {'gid': 9124, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6837/3342090879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 3342090879}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3342090879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9124, 'qdb_cluster': [9124]} 2024-03-22T04:21:30.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-22T04:21:30.290 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-22T04:21:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 Processor -- start 2024-03-22T04:21:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 -- start start 2024-03-22T04:21:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feec815f860 0x7feec80ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feec80cdb00 0x7feec809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec809afb0 0x7feec80d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feec8073cf0 con 0x7feec80cdb00 2024-03-22T04:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7feec809b390 con 0x7feec815f860 2024-03-22T04:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feec80ce280 con 0x7feec809afb0 2024-03-22T04:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feec80cdb00 0x7feec809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feec80cdb00 0x7feec809aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45058/0 (socket says 172.21.15.86:45058) 2024-03-22T04:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 -- 172.21.15.86:0/2799582225 learned_addr learned my addr 172.21.15.86:0/2799582225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec6800640 1 --2- 172.21.15.86:0/2799582225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec809afb0 0x7feec80d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5e00640 1 --2- 172.21.15.86:0/2799582225 >> v2:172.21.15.196:3300/0 conn(0x7feec815f860 0x7feec80ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 -- 172.21.15.86:0/2799582225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec809afb0 msgr2=0x7feec80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 --2- 172.21.15.86:0/2799582225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec809afb0 0x7feec80d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 -- 172.21.15.86:0/2799582225 >> v2:172.21.15.196:3300/0 conn(0x7feec815f860 msgr2=0x7feec80ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 --2- 172.21.15.86:0/2799582225 >> v2:172.21.15.196:3300/0 conn(0x7feec815f860 0x7feec80ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 -- 172.21.15.86:0/2799582225 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec80ce560 con 0x7feec80cdb00 2024-03-22T04:21:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec5400640 1 --2- 172.21.15.86:0/2799582225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec80cdb00 0x7feec809aa70 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7feec80733d0 tx=0x7feeb401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2505184a4ee78ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec4a00640 1 -- 172.21.15.86:0/2799582225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40b0070 con 0x7feec80cdb00 2024-03-22T04:21:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec4a00640 1 -- 172.21.15.86:0/2799582225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb40ab600 con 0x7feec80cdb00 2024-03-22T04:21:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec4a00640 1 -- 172.21.15.86:0/2799582225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40ab9d0 con 0x7feec80cdb00 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 -- 172.21.15.86:0/2799582225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec80cdb00 msgr2=0x7feec809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 --2- 172.21.15.86:0/2799582225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec80cdb00 0x7feec809aa70 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7feec80733d0 tx=0x7feeb401c230 comp rx=0 tx=0).stop 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 -- 172.21.15.86:0/2799582225 shutdown_connections 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 --2- 172.21.15.86:0/2799582225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec809afb0 0x7feec80d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 --2- 172.21.15.86:0/2799582225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec80cdb00 0x7feec809aa70 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 --2- 172.21.15.86:0/2799582225 >> v2:172.21.15.196:3300/0 conn(0x7feec815f860 0x7feec80ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.430+0000 7feec7200640 1 -- 172.21.15.86:0/2799582225 >> 172.21.15.86:0/2799582225 conn(0x7feec80b3d60 msgr2=0x7feec80b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 -- 172.21.15.86:0/2799582225 shutdown_connections 2024-03-22T04:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 -- 172.21.15.86:0/2799582225 wait complete. 2024-03-22T04:21:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 Processor -- start 2024-03-22T04:21:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 -- start start 2024-03-22T04:21:30.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec809afb0 0x7feec81551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec80cdb00 0x7feec81663b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec815f860 0x7feec806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feec815fc40 con 0x7feec815f860 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feec80cdf70 con 0x7feec809afb0 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec7200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feec815fdc0 con 0x7feec80cdb00 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec815f860 0x7feec806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec80cdb00 0x7feec81663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec809afb0 0x7feec81551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec815f860 0x7feec806c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45068/0 (socket says 172.21.15.86:45068) 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec5400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec80cdb00 0x7feec81663b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:48694/0 (socket says 172.21.15.86:48694) 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec809afb0 0x7feec81551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:41064/0 (socket says 172.21.15.86:41064) 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 -- 172.21.15.86:0/1238152718 learned_addr learned my addr 172.21.15.86:0/1238152718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 -- 172.21.15.86:0/1238152718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec80cdb00 msgr2=0x7feec81663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec80cdb00 0x7feec81663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 -- 172.21.15.86:0/1238152718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec809afb0 msgr2=0x7feec81551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec809afb0 0x7feec81551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 -- 172.21.15.86:0/1238152718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec81658f0 con 0x7feec815f860 2024-03-22T04:21:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.434+0000 7feec6800640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec815f860 0x7feec806c770 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7feec0007d50 tx=0x7feec000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feebe000640 1 -- 172.21.15.86:0/1238152718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0014420 con 0x7feec815f860 2024-03-22T04:21:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feebe000640 1 -- 172.21.15.86:0/1238152718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0019070 con 0x7feec815f860 2024-03-22T04:21:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feebe000640 1 -- 172.21.15.86:0/1238152718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0014820 con 0x7feec815f860 2024-03-22T04:21:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feec7200640 1 -- 172.21.15.86:0/1238152718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec806d1a0 con 0x7feec815f860 2024-03-22T04:21:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feec7200640 1 -- 172.21.15.86:0/1238152718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec8165bb0 con 0x7feec815f860 2024-03-22T04:21:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feebe000640 1 -- 172.21.15.86:0/1238152718 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7feec00149c0 con 0x7feec815f860 2024-03-22T04:21:30.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feebe000640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feea8067b70 0x7feea806a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feebe000640 1 -- 172.21.15.86:0/1238152718 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7feec0086740 con 0x7feec815f860 2024-03-22T04:21:30.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feec5e00640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feea8067b70 0x7feea806a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.438+0000 7feed0751000 1 Processor -- start 2024-03-22T04:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feea8067b70 0x7feea806a030 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7feec8155d80 tx=0x7feeb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- start start 2024-03-22T04:21:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f3de250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3916e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f391c20 0x560a6f392000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a6f18f140 con 0x560a6f3dde70 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a6f1e2b10 con 0x560a6f3de790 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x560a6f169c50 con 0x560a6f391c20 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f3de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f3de250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45070/0 (socket says 172.21.15.86:45070) 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 -- 172.21.15.86:0/166997708 learned_addr learned my addr 172.21.15.86:0/166997708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec6800640 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f391c20 0x560a6f392000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5400640 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 -- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f391c20 msgr2=0x560a6f392000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f391c20 0x560a6f392000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 -- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 msgr2=0x560a6f3916e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3916e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 -- 172.21.15.86:0/166997708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6f227f20 con 0x560a6f3dde70 2024-03-22T04:21:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5400640 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feec5e00640 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f3de250 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7feeb8007fd0 tx=0x7feeb8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6deca58c5f20791d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feeb2a00640 1 -- 172.21.15.86:0/166997708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb801c050 con 0x560a6f3dde70 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feeb2a00640 1 -- 172.21.15.86:0/166997708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8026070 con 0x560a6f3dde70 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feeb2a00640 1 -- 172.21.15.86:0/166997708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8021440 con 0x560a6f3dde70 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- 172.21.15.86:0/166997708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 msgr2=0x560a6f3de250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f3de250 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7feeb8007fd0 tx=0x7feeb8013000 comp rx=0 tx=0).stop 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- 172.21.15.86:0/166997708 shutdown_connections 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f391c20 0x560a6f392000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 --2- 172.21.15.86:0/166997708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f3de250 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- 172.21.15.86:0/166997708 >> 172.21.15.86:0/166997708 conn(0x560a6f3bec10 msgr2=0x560a6f3c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- 172.21.15.86:0/166997708 shutdown_connections 2024-03-22T04:21:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- 172.21.15.86:0/166997708 wait complete. 2024-03-22T04:21:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.442+0000 7feed0751000 1 -- start start 2024-03-22T04:21:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 Processor -- start 2024-03-22T04:21:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 0x560a6f396810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f396d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3de790 0x560a6f397290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a6f19cab0 con 0x560a6f3dde70 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a6f1deac0 con 0x560a6f391c20 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x560a6f29eb30 con 0x560a6f3de790 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 10 client.0 _ll_register_callbacks cb 0x560a6f37f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f396d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f396d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45074/0 (socket says 172.21.15.86:45074) 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 -- 172.21.15.86:0/2614887195 learned_addr learned my addr 172.21.15.86:0/2614887195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5e00640 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 0x560a6f396810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec6800640 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3de790 0x560a6f397290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 -- 172.21.15.86:0/2614887195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3de790 msgr2=0x560a6f397290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3de790 0x560a6f397290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 -- 172.21.15.86:0/2614887195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 msgr2=0x560a6f396810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 0x560a6f396810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 -- 172.21.15.86:0/2614887195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6f1b8570 con 0x560a6f3dde70 2024-03-22T04:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5400640 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f396d50 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7feeb4000c10 tx=0x7feeb401aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa21e2432497c4b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feea7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feea7400640 1 -- 172.21.15.86:0/2614887195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40bb070 con 0x560a6f3dde70 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 -- 172.21.15.86:0/2614887195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb8026070 con 0x560a6f3dde70 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feea7400640 1 -- 172.21.15.86:0/2614887195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7feeb40abd00 con 0x560a6f3dde70 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feea7400640 1 -- 172.21.15.86:0/2614887195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7feeb40c6050 con 0x560a6f3dde70 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 10 client.9537 fetch_fsmap learned FSMap version 112 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 -- 172.21.15.86:0/2614887195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x560a6f1ae880 con 0x560a6f3dde70 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feea7400640 1 -- 172.21.15.86:0/2614887195 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8026070 con 0x560a6f3dde70 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feea7400640 1 -- 172.21.15.86:0/2614887195 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 112) ==== 3494+0+0 (secure 0 0 0) 0x7feeb40c9020 con 0x560a6f3dde70 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 10 client.9537 fetch_fsmap finished waiting for FSMap version 112 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 10 client.9537 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 20 client.9537 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] conn(0x560a6f397ca0 0x560a6f3ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 4 client.9537 mds_command: new command op to 9124 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-22T04:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feed0751000 1 -- 172.21.15.86:0/2614887195 --> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a6f17dbc0 con 0x560a6f397ca0 2024-03-22T04:21:30.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5e00640 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] conn(0x560a6f397ca0 0x560a6f3ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feec5e00640 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] conn(0x560a6f397ca0 0x560a6f3ef120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9028fc0f24d6e8c0 server_cookie=e5f28fb397aa2264 in_seq=0 out_seq=0 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.446+0000 7feea7400640 10 client.9537 ms_handle_connect on v2:172.21.15.86:6835/3342090879 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feea7400640 1 -- 172.21.15.86:0/2614887195 <== mds.0 v2:172.21.15.86:6835/3342090879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560a6f17dbc0 con 0x560a6f397ca0 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feea7400640 10 client.9537 handle_command_reply: tid=0 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 client.9537 shutdown 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 -- 172.21.15.86:0/2614887195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 msgr2=0x560a6f396d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f396d50 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7feeb4000c10 tx=0x7feeb401aa70 comp rx=0 tx=0).stop 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 -- 172.21.15.86:0/2614887195 shutdown_connections 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] conn(0x560a6f397ca0 0x560a6f3ef120 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3de790 0x560a6f397290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3dde70 0x560a6f396d50 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feea7400640 0 client.9537 ms_handle_reset on v2:172.21.15.86:6835/3342090879 2024-03-22T04:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 --2- 172.21.15.86:0/2614887195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 0x560a6f396810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 -- 172.21.15.86:0/2614887195 >> 172.21.15.86:0/2614887195 conn(0x560a6f3bec10 msgr2=0x560a6f3c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 -- 172.21.15.86:0/2614887195 shutdown_connections 2024-03-22T04:21:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 1 -- 172.21.15.86:0/2614887195 wait complete. 2024-03-22T04:21:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.454+0000 7feed0751000 20 client.9537 trim_cache size 0 max 16384 2024-03-22T04:21:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feed0751000 1 Processor -- start 2024-03-22T04:21:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feed0751000 1 -- start start 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feed0751000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f391c20 0x560a6f3dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feed0751000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 0x560a6f3dd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feed0751000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feed0751000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feec8202fd0 con 0x560a6f391c20 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feed0751000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feec8164eb0 con 0x560a6f3de790 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feed0751000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feec8202980 con 0x560a6f3dde70 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.466+0000 7feec5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f391c20 0x560a6f3dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f391c20 0x560a6f3dcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45084/0 (socket says 172.21.15.86:45084) 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 -- 172.21.15.86:0/4231468430 learned_addr learned my addr 172.21.15.86:0/4231468430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5400640 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 0x560a6f3dd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec6800640 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 -- 172.21.15.86:0/4231468430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 msgr2=0x560a6f3dd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 0x560a6f3dd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 -- 172.21.15.86:0/4231468430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 msgr2=0x560a6f3ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 -- 172.21.15.86:0/4231468430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6f16ae80 con 0x560a6f391c20 2024-03-22T04:21:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f391c20 0x560a6f3dcf30 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7feeb8010ab0 tx=0x7feeb8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0becb83399a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feeb2000640 1 -- 172.21.15.86:0/4231468430 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb801c040 con 0x560a6f391c20 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feeb2000640 1 -- 172.21.15.86:0/4231468430 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8026070 con 0x560a6f391c20 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feeb2000640 1 -- 172.21.15.86:0/4231468430 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8035670 con 0x560a6f391c20 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- 172.21.15.86:0/4231468430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f391c20 msgr2=0x560a6f3dcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f391c20 0x560a6f3dcf30 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7feeb8010ab0 tx=0x7feeb8015000 comp rx=0 tx=0).stop 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- 172.21.15.86:0/4231468430 shutdown_connections 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f3de790 0x560a6f3ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 0x560a6f3dd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 --2- 172.21.15.86:0/4231468430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f391c20 0x560a6f3dcf30 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- 172.21.15.86:0/4231468430 >> 172.21.15.86:0/4231468430 conn(0x560a6f3bec10 msgr2=0x560a6f398f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- 172.21.15.86:0/4231468430 shutdown_connections 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- 172.21.15.86:0/4231468430 wait complete. 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- start start 2024-03-22T04:21:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 Processor -- start 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 0x560a6f3f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 0x560a6f44d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3de790 0x560a6f44d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a6f188e80 con 0x560a6f3de790 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a6f222a60 con 0x560a6f391c20 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x560a6f197220 con 0x560a6f3dde70 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feed0751000 10 client.0 _ll_register_callbacks cb 0x7feeb40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3de790 0x560a6f44d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 0x560a6f3f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3de790 0x560a6f44d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45098/0 (socket says 172.21.15.86:45098) 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec6800640 1 -- 172.21.15.86:0/255602421 learned_addr learned my addr 172.21.15.86:0/255602421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.470+0000 7feec5400640 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 0x560a6f44d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec6800640 1 -- 172.21.15.86:0/255602421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 msgr2=0x560a6f44d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec6800640 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 0x560a6f44d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec6800640 1 -- 172.21.15.86:0/255602421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 msgr2=0x560a6f3f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec6800640 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 0x560a6f3f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec6800640 1 -- 172.21.15.86:0/255602421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6f176750 con 0x560a6f3de790 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec6800640 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3de790 0x560a6f44d810 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7feec000e9b0 tx=0x7feec0056a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5703f7c430815d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 1 -- 172.21.15.86:0/255602421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec006e020 con 0x560a6f3de790 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 1 -- 172.21.15.86:0/255602421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7feec005ef70 con 0x560a6f3de790 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 -- 172.21.15.86:0/255602421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb8026070 con 0x560a6f3de790 2024-03-22T04:21:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 1 -- 172.21.15.86:0/255602421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7feec0054cf0 con 0x560a6f3de790 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 10 client.9543 fetch_fsmap learned FSMap version 112 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 -- 172.21.15.86:0/255602421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x560a6f189450 con 0x560a6f3de790 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 1 -- 172.21.15.86:0/255602421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8026070 con 0x560a6f3de790 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 1 -- 172.21.15.86:0/255602421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 112) ==== 3494+0+0 (secure 0 0 0) 0x7feec0057bb0 con 0x560a6f3de790 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 10 client.9543 fetch_fsmap finished waiting for FSMap version 112 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 10 client.9543 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 20 client.9543 populate_metadata read hostname 'smithi086' 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] conn(0x560a6f449d60 0x560a6f44a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 4 client.9543 mds_command: new command op to 9124 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 -- 172.21.15.86:0/255602421 --> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] -- command(tid 0: {"prefix": "status"}) -- 0x560a6f1f0050 con 0x560a6f449d60 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec5e00640 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] conn(0x560a6f449d60 0x560a6f44a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec5e00640 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] conn(0x560a6f449d60 0x560a6f44a140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9c3de764db2e9e server_cookie=2e2b02e8daecc502 in_seq=0 out_seq=0 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 10 client.9543 ms_handle_connect on v2:172.21.15.86:6835/3342090879 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 1 -- 172.21.15.86:0/255602421 <== mds.0 v2:172.21.15.86:6835/3342090879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x560a6f1f0050 con 0x560a6f449d60 2024-03-22T04:21:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 10 client.9543 handle_command_reply: tid=0 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 client.9543 shutdown 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 -- 172.21.15.86:0/255602421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3de790 msgr2=0x560a6f44d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3de790 0x560a6f44d810 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7feec000e9b0 tx=0x7feec0056a70 comp rx=0 tx=0).stop 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 -- 172.21.15.86:0/255602421 shutdown_connections 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.86:6835/3342090879,v1:172.21.15.86:6837/3342090879] conn(0x560a6f449d60 0x560a6f44a140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6f3de790 0x560a6f44d810 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560a6f3dde70 0x560a6f44d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 --2- 172.21.15.86:0/255602421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a6f391c20 0x560a6f3f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 -- 172.21.15.86:0/255602421 >> 172.21.15.86:0/255602421 conn(0x560a6f3bec10 msgr2=0x560a6f3dbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feea7400640 0 client.9543 ms_handle_reset on v2:172.21.15.86:6835/3342090879 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 -- 172.21.15.86:0/255602421 shutdown_connections 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 1 -- 172.21.15.86:0/255602421 wait complete. 2024-03-22T04:21:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feed0751000 20 client.9543 trim_cache size 0 max 16384 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1365483d-2fb4-4579-a50b-2ce299f625b2", 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9124, 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.5348137580000003, 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 112, 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 126, 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 126, 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 6.4921819049999998 2024-03-22T04:21:30.481 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 -- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feea8067b70 msgr2=0x7feea806a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feea8067b70 0x7feea806a030 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7feec8155d80 tx=0x7feeb8008040 comp rx=0 tx=0).stop 2024-03-22T04:21:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 -- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec815f860 msgr2=0x7feec806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec815f860 0x7feec806c770 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7feec0007d50 tx=0x7feec000e4b0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 -- 172.21.15.86:0/1238152718 shutdown_connections 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7feea8067b70 0x7feea806a030 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec815f860 0x7feec806c770 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feec80cdb00 0x7feec81663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 --2- 172.21.15.86:0/1238152718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feec809afb0 0x7feec81551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.474+0000 7feec7200640 1 -- 172.21.15.86:0/1238152718 >> 172.21.15.86:0/1238152718 conn(0x7feec80b3d60 msgr2=0x7feec80b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.478+0000 7feec7200640 1 -- 172.21.15.86:0/1238152718 shutdown_connections 2024-03-22T04:21:30.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.478+0000 7feec7200640 1 -- 172.21.15.86:0/1238152718 wait complete. 2024-03-22T04:21:30.502 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-22T04:21:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b3e00640 1 Processor -- start 2024-03-22T04:21:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b3e00640 1 -- start start 2024-03-22T04:21:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc6ac161c60 0x7fc6ac06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b3e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac06e350 con 0x7fc6ac0b3800 2024-03-22T04:21:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac06e630 con 0x7fc6ac161880 2024-03-22T04:21:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b3e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ac15ee80 con 0x7fc6ac161c60 2024-03-22T04:21:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc6ac0b3800 0x7fc6ac15fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39384/0 (socket says 172.21.15.86:39384) 2024-03-22T04:21:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b1e00640 1 -- 172.21.15.86:0/1444504270 learned_addr learned my addr 172.21.15.86:0/1444504270 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b0a00640 1 --2- 172.21.15.86:0/1444504270 >> v2:172.21.15.196:3300/0 conn(0x7fc6ac161c60 0x7fc6ac06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6abe00640 1 -- 172.21.15.86:0/1444504270 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917853975 0 0) 0x7fc6ac06e350 con 0x7fc6ac0b3800 2024-03-22T04:21:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6abe00640 1 -- 172.21.15.86:0/1444504270 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc694003530 con 0x7fc6ac0b3800 2024-03-22T04:21:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b0a00640 1 -- 172.21.15.86:0/1444504270 >> v1:172.21.15.196:6789/0 conn(0x7fc6ac161880 legacy=0x7fc6ac0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b0a00640 1 -- 172.21.15.86:0/1444504270 >> v1:172.21.15.86:6789/0 conn(0x7fc6ac0b3800 legacy=0x7fc6ac15fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.642+0000 7fc6b0a00640 1 -- 172.21.15.86:0/1444504270 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac06e910 con 0x7fc6ac161c60 2024-03-22T04:21:30.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b0a00640 1 --2- 172.21.15.86:0/1444504270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161c60 0x7fc6ac06d700 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc6980097c0 tx=0x7fc69801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16591bd40effdfdb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6abe00640 1 -- 172.21.15.86:0/1444504270 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6980b1070 con 0x7fc6ac161c60 2024-03-22T04:21:30.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6abe00640 1 -- 172.21.15.86:0/1444504270 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6980a8050 con 0x7fc6ac161c60 2024-03-22T04:21:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6abe00640 1 -- 172.21.15.86:0/1444504270 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6980ac860 con 0x7fc6ac161c60 2024-03-22T04:21:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 -- 172.21.15.86:0/1444504270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161c60 msgr2=0x7fc6ac06d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 --2- 172.21.15.86:0/1444504270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161c60 0x7fc6ac06d700 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc6980097c0 tx=0x7fc69801c3e0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 -- 172.21.15.86:0/1444504270 shutdown_connections 2024-03-22T04:21:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 --2- 172.21.15.86:0/1444504270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161c60 0x7fc6ac06d700 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 -- 172.21.15.86:0/1444504270 >> 172.21.15.86:0/1444504270 conn(0x7fc6ac0b1df0 msgr2=0x7fc6ac0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 -- 172.21.15.86:0/1444504270 shutdown_connections 2024-03-22T04:21:30.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 -- 172.21.15.86:0/1444504270 wait complete. 2024-03-22T04:21:30.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 Processor -- start 2024-03-22T04:21:30.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.650+0000 7fc6b3e00640 1 -- start start 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6ac0b3800 0x7fc6ac0af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161880 0x7fc6ac14c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac161c60 0x7fc6ac159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac160dd0 con 0x7fc6ac161c60 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac160c50 con 0x7fc6ac161880 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac160f50 con 0x7fc6ac0b3800 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac161c60 0x7fc6ac159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac161c60 0x7fc6ac159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45110/0 (socket says 172.21.15.86:45110) 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 -- 172.21.15.86:0/3890852408 learned_addr learned my addr 172.21.15.86:0/3890852408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b0a00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161880 0x7fc6ac14c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1400640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6ac0b3800 0x7fc6ac0af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 -- 172.21.15.86:0/3890852408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6ac0b3800 msgr2=0x7fc6ac0af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6ac0b3800 0x7fc6ac0af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 -- 172.21.15.86:0/3890852408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161880 msgr2=0x7fc6ac14c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161880 0x7fc6ac14c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 -- 172.21.15.86:0/3890852408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac15af20 con 0x7fc6ac161c60 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b1e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac161c60 0x7fc6ac159f60 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400ca70 tx=0x7fc6a400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6a9600640 1 -- 172.21.15.86:0/3890852408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4017070 con 0x7fc6ac161c60 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6a9600640 1 -- 172.21.15.86:0/3890852408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a400abd0 con 0x7fc6ac161c60 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6a9600640 1 -- 172.21.15.86:0/3890852408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4012400 con 0x7fc6ac161c60 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac0ad720 con 0x7fc6ac161c60 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac151260 con 0x7fc6ac161c60 2024-03-22T04:21:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac068650 con 0x7fc6ac161c60 2024-03-22T04:21:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.654+0000 7fc6a9600640 1 -- 172.21.15.86:0/3890852408 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fc6a401a070 con 0x7fc6ac161c60 2024-03-22T04:21:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.658+0000 7fc6a9600640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc68c067b20 0x7fc68c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.658+0000 7fc6a9600640 1 -- 172.21.15.86:0/3890852408 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7fc6a4085df0 con 0x7fc6ac161c60 2024-03-22T04:21:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.658+0000 7fc6b1400640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc68c067b20 0x7fc68c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.658+0000 7fc6b1400640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc68c067b20 0x7fc68c069fe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0b1450 tx=0x7fc6a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.662+0000 7fc6a9600640 1 -- 172.21.15.86:0/3890852408 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc6ac15af20 con 0x7fc6ac161c60 2024-03-22T04:21:30.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc6ac0cfdb0 con 0x7fc6ac161c60 2024-03-22T04:21:30.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6a9600640 1 -- 172.21.15.86:0/3890852408 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) ==== 53+0+73 (secure 0 0 0) 0x7fc6a4055ee0 con 0x7fc6ac161c60 2024-03-22T04:21:30.847 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-22T04:21:30.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc68c067b20 msgr2=0x7fc68c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc68c067b20 0x7fc68c069fe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0b1450 tx=0x7fc6a0009040 comp rx=0 tx=0).stop 2024-03-22T04:21:30.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac161c60 msgr2=0x7fc6ac159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:30.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac161c60 0x7fc6ac159f60 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400ca70 tx=0x7fc6a400cf30 comp rx=0 tx=0).stop 2024-03-22T04:21:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 shutdown_connections 2024-03-22T04:21:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fc68c067b20 0x7fc68c069fe0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ac161880 0x7fc6ac14c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac161c60 0x7fc6ac159f60 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 --2- 172.21.15.86:0/3890852408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6ac0b3800 0x7fc6ac0af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.850+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 >> 172.21.15.86:0/3890852408 conn(0x7fc6ac0b1df0 msgr2=0x7fc6ac0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.854+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 shutdown_connections 2024-03-22T04:21:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:30.854+0000 7fc6b3e00640 1 -- 172.21.15.86:0/3890852408 wait complete. 2024-03-22T04:21:30.869 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-22T04:21:30.870 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T04:21:30.870 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-22T04:21:30.870 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T04:21:30.870 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:30.870 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-22T04:21:30.870 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-22T04:21:30.870 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T04:21:30.870 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:30.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:30.870 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-22T04:21:30.880 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T04:21:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T04:21:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T04:21:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-22T04:21:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:30.880 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:30.880 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41841sec preferred_lft 41841sec 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:34:15:4e:c9 brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-22T04:21:30.881 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-22T04:21:30.882 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-22T04:21:30.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:30.883 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T04:21:30.939 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:21:30.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:30.941 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-22T04:21:30.993 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-22T04:21:30.993 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T04:21:30.993 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-22T04:21:30.994 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T04:21:30.994 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-22T04:21:30.994 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-22T04:21:30.994 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T04:21:30.994 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:30.994 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:30.994 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-22T04:21:30.994 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T04:21:30.994 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-22T04:21:30.994 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-22T04:21:30.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T04:21:30.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T04:21:30.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T04:21:30.995 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-22T04:21:31.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-22T04:21:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:31.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:31.128 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:21:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:21:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:21:31.137 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:31.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:31.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-22T04:21:31.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:21:31.197 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:31.245 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T04:21:31.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T04:21:31.245 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:31.292 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T04:21:31.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:31.293 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:31.381 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:31.381 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:31.381 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:31.396 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:31.396 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:31.396 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:31.396 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:21:31.396 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:21:31.406 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T04:21:31.406 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T04:21:31.406 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-22T04:21:31.406 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.196:3300,172.21.15.196:3301" 2024-03-22T04:21:31.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T04:21:31.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790" 2024-03-22T04:21:31.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.196:6789,172.21.15.196:6790:/ 2024-03-22T04:21:31.407 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1365483d-2fb4-4579-a50b-2ce299f625b2" will pass to kernel 2024-03-22T04:21:31.411 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T04:21:31.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:31.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:31.430 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T04:21:31.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-22T04:21:31.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:31.529 INFO:teuthology.orchestra.run.smithi086.stdout:client9226 2024-03-22T04:21:31.529 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:31.529 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:31.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:31.538 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-22T04:21:31.597 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-03-22T04:21:31.597 INFO:teuthology.orchestra.run.smithi086.stdout:./parent 2024-03-22T04:21:31.597 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed 2024-03-22T04:21:31.597 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/charlie 2024-03-22T04:21:31.597 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/alpha 2024-03-22T04:21:31.597 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/bravo 2024-03-22T04:21:31.597 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/unflushed 2024-03-22T04:21:31.597 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/unflushed/jfile 2024-03-22T04:21:31.598 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:31.599 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:31.599 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:31.599 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:31.599 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-22T04:21:31.599 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:31.705 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-22T04:21:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:31.714 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:31.714 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:31.714 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:31.714 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:31.714 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-22T04:21:31.714 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:31.825 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-03-22T04:21:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:31.834 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:31.835 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:31.835 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:31.835 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:31.835 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-22T04:21:31.835 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:31.946 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-03-22T04:21:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:31.955 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:31.956 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:31.956 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:31.956 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:31.956 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-22T04:21:31.956 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:32.060 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628280 2024-03-22T04:21:32.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:32.069 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:32.070 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:32.070 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:32.070 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:32.070 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-22T04:21:32.070 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:32.177 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-03-22T04:21:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:32.185 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:32.186 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:32.186 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:32.186 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:32.186 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-22T04:21:32.186 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:32.300 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628279 2024-03-22T04:21:32.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:32.310 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:32.310 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:32.310 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:32.310 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:32.310 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-22T04:21:32.310 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:32.416 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627779 2024-03-22T04:21:32.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:32.425 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-03-22T04:21:32.426 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-03-22T04:21:32.426 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-03-22T04:21:32.426 DEBUG:teuthology.orchestra.run.smithi086:> 2024-03-22T04:21:32.426 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-22T04:21:32.426 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-03-22T04:21:32.532 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627780 2024-03-22T04:21:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-22T04:21:32.541 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-22T04:21:32.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:21:32.648 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-22T04:21:32.705 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-22T04:21:32.705 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:21:32.705 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:32.705 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:32.713 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:32.714 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:32.766 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:32.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.946+0000 7fa799e00640 1 Processor -- start 2024-03-22T04:21:32.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.946+0000 7fa799e00640 1 -- start start 2024-03-22T04:21:32.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa799e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7940d0de0 0x7fa7940ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:32.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa799e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7940d11c0 con 0x7fa79415fed0 2024-03-22T04:21:32.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa799e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7940cece0 con 0x7fa7940d19a0 2024-03-22T04:21:32.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa799e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7940d0580 con 0x7fa7940d0de0 2024-03-22T04:21:32.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa792a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7940d0de0 0x7fa7940ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:32.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa793400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa79415fed0 0x7fa7941602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39392/0 (socket says 172.21.15.86:39392) 2024-03-22T04:21:32.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa793400640 1 -- 172.21.15.86:0/98933319 learned_addr learned my addr 172.21.15.86:0/98933319 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:32.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa792a00640 1 -- 172.21.15.86:0/98933319 >> v1:172.21.15.196:6789/0 conn(0x7fa7940d19a0 legacy=0x7fa7940cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:32.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa792a00640 1 -- 172.21.15.86:0/98933319 >> v1:172.21.15.86:6789/0 conn(0x7fa79415fed0 legacy=0x7fa7941602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:32.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa792a00640 1 -- 172.21.15.86:0/98933319 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7940cefc0 con 0x7fa7940d0de0 2024-03-22T04:21:32.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.950+0000 7fa792000640 1 -- 172.21.15.86:0/98933319 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 914850042 0 0) 0x7fa7940d11c0 con 0x7fa79415fed0 2024-03-22T04:21:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa792a00640 1 --2- 172.21.15.86:0/98933319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7940d0de0 0x7fa7940ce6b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa7840097c0 tx=0x7fa78401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb30848dd01249d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa792000640 1 -- 172.21.15.86:0/98933319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840b1070 con 0x7fa7940d0de0 2024-03-22T04:21:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa792000640 1 -- 172.21.15.86:0/98933319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7840ac5f0 con 0x7fa7940d0de0 2024-03-22T04:21:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa792000640 1 -- 172.21.15.86:0/98933319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840ac9c0 con 0x7fa7940d0de0 2024-03-22T04:21:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa799e00640 1 -- 172.21.15.86:0/98933319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7940d0de0 msgr2=0x7fa7940ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa799e00640 1 --2- 172.21.15.86:0/98933319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7940d0de0 0x7fa7940ce6b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa7840097c0 tx=0x7fa78401c4a0 comp rx=0 tx=0).stop 2024-03-22T04:21:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa799e00640 1 -- 172.21.15.86:0/98933319 shutdown_connections 2024-03-22T04:21:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa799e00640 1 --2- 172.21.15.86:0/98933319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7940d0de0 0x7fa7940ce6b0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa799e00640 1 -- 172.21.15.86:0/98933319 >> 172.21.15.86:0/98933319 conn(0x7fa79406f910 msgr2=0x7fa79406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa799e00640 1 -- 172.21.15.86:0/98933319 shutdown_connections 2024-03-22T04:21:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.954+0000 7fa799e00640 1 -- 172.21.15.86:0/98933319 wait complete. 2024-03-22T04:21:32.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 Processor -- start 2024-03-22T04:21:32.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 -- start start 2024-03-22T04:21:32.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7940d0de0 0x7fa794154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:32.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7940d19a0 0x7fa794166a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79415fed0 0x7fa79406c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7940d1f30 con 0x7fa79415fed0 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7940d1db0 con 0x7fa7940d19a0 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa794071e30 con 0x7fa7940d0de0 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79415fed0 0x7fa79406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa792a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7940d0de0 0x7fa794154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79415fed0 0x7fa79406c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45122/0 (socket says 172.21.15.86:45122) 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 -- 172.21.15.86:0/3702207058 learned_addr learned my addr 172.21.15.86:0/3702207058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa78be00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7940d19a0 0x7fa794166a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 -- 172.21.15.86:0/3702207058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7940d0de0 msgr2=0x7fa794154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7940d0de0 0x7fa794154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 -- 172.21.15.86:0/3702207058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7940d19a0 msgr2=0x7fa794166a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7940d19a0 0x7fa794166a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 -- 172.21.15.86:0/3702207058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7940b74b0 con 0x7fa79415fed0 2024-03-22T04:21:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa793400640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79415fed0 0x7fa79406c770 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c7d0 tx=0x7fa78c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa78b400640 1 -- 172.21.15.86:0/3702207058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c018070 con 0x7fa79415fed0 2024-03-22T04:21:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa78b400640 1 -- 172.21.15.86:0/3702207058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78c00a640 con 0x7fa79415fed0 2024-03-22T04:21:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7940b7790 con 0x7fa79415fed0 2024-03-22T04:21:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa78b400640 1 -- 172.21.15.86:0/3702207058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00aa70 con 0x7fa79415fed0 2024-03-22T04:21:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.958+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7940ca850 con 0x7fa79415fed0 2024-03-22T04:21:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.962+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa794068650 con 0x7fa79415fed0 2024-03-22T04:21:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.962+0000 7fa78b400640 1 -- 172.21.15.86:0/3702207058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa78c01b070 con 0x7fa79415fed0 2024-03-22T04:21:32.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.962+0000 7fa78b400640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa770067b20 0x7fa770069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:32.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.962+0000 7fa78b400640 1 -- 172.21.15.86:0/3702207058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7fa78c0856b0 con 0x7fa79415fed0 2024-03-22T04:21:32.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.962+0000 7fa792a00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa770067b20 0x7fa770069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:32.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.966+0000 7fa792a00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa770067b20 0x7fa770069fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa7840ae000 tx=0x7fa7840a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:32.966+0000 7fa78b400640 1 -- 172.21.15.86:0/3702207058 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa78c055830 con 0x7fa79415fed0 2024-03-22T04:21:33.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.122+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7941551b0 con 0x7fa79415fed0 2024-03-22T04:21:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa78b400640 1 -- 172.21.15.86:0/3702207058 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+17215 (secure 0 0 0) 0x7fa78c0598d0 con 0x7fa79415fed0 2024-03-22T04:21:33.123 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:33.123 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":126,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:21:23.925937+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6835/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6837/3165266972":"2024-03-23T04:21:12.563892+0000","172.21.15.86:6835/3165266972":"2024-03-23T04:21:12.563892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa770067b20 msgr2=0x7fa770069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa770067b20 0x7fa770069fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa7840ae000 tx=0x7fa7840a7000 comp rx=0 tx=0).stop 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79415fed0 msgr2=0x7fa79406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79415fed0 0x7fa79406c770 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c7d0 tx=0x7fa78c00cc90 comp rx=0 tx=0).stop 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 shutdown_connections 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa770067b20 0x7fa770069fe0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7940d19a0 0x7fa794166a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79415fed0 0x7fa79406c770 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 --2- 172.21.15.86:0/3702207058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7940d0de0 0x7fa794154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 >> 172.21.15.86:0/3702207058 conn(0x7fa79406f910 msgr2=0x7fa794155990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 shutdown_connections 2024-03-22T04:21:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.126+0000 7fa799e00640 1 -- 172.21.15.86:0/3702207058 wait complete. 2024-03-22T04:21:33.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:20:55.270181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:20:56.532178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:21:33.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:21:33.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.286+0000 7fd3fec00640 1 Processor -- start 2024-03-22T04:21:33.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.286+0000 7fd3fec00640 1 -- start start 2024-03-22T04:21:33.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3fec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd3f815e970 0x7fd3f80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3fec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd3f809b180 0x7fd3f80cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3fec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3f806dd70 con 0x7fd3f806d8f0 2024-03-22T04:21:33.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3fec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd3f80b49a0 con 0x7fd3f815e970 2024-03-22T04:21:33.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3fec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd3f80ccda0 con 0x7fd3f809b180 2024-03-22T04:21:33.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3f7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd3f815e970 0x7fd3f80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3f7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd3f806d8f0 0x7fd3f809aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39398/0 (socket says 172.21.15.86:39398) 2024-03-22T04:21:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3f7400640 1 -- 172.21.15.86:0/915100115 learned_addr learned my addr 172.21.15.86:0/915100115 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3fcc00640 1 --2- 172.21.15.86:0/915100115 >> v2:172.21.15.196:3300/0 conn(0x7fd3f809b180 0x7fd3f80cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3f7e00640 1 -- 172.21.15.86:0/915100115 >> v2:172.21.15.196:3300/0 conn(0x7fd3f809b180 msgr2=0x7fd3f80cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3f7e00640 1 --2- 172.21.15.86:0/915100115 >> v2:172.21.15.196:3300/0 conn(0x7fd3f809b180 0x7fd3f80cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3f7e00640 1 -- 172.21.15.86:0/915100115 >> v1:172.21.15.86:6789/0 conn(0x7fd3f806d8f0 legacy=0x7fd3f809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3f7e00640 1 -- 172.21.15.86:0/915100115 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f806dff0 con 0x7fd3f815e970 2024-03-22T04:21:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3fcc00640 1 --2- 172.21.15.86:0/915100115 >> v2:172.21.15.196:3300/0 conn(0x7fd3f809b180 0x7fd3f80cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.290+0000 7fd3f6a00640 1 -- 172.21.15.86:0/915100115 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3378385555 0 0) 0x7fd3e4001120 con 0x7fd3f806d8f0 2024-03-22T04:21:33.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3f7e00640 1 --2- 172.21.15.86:0/915100115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f80b12e0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8073160 tx=0x7fd3e801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692de844e5222714 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3f6a00640 1 -- 172.21.15.86:0/915100115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80ab650 con 0x7fd3f815e970 2024-03-22T04:21:33.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3f6a00640 1 -- 172.21.15.86:0/915100115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e80b0070 con 0x7fd3f815e970 2024-03-22T04:21:33.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3f6a00640 1 -- 172.21.15.86:0/915100115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80aba10 con 0x7fd3f815e970 2024-03-22T04:21:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- 172.21.15.86:0/915100115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 msgr2=0x7fd3f80b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 --2- 172.21.15.86:0/915100115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f80b12e0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8073160 tx=0x7fd3e801c450 comp rx=0 tx=0).stop 2024-03-22T04:21:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- 172.21.15.86:0/915100115 shutdown_connections 2024-03-22T04:21:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 --2- 172.21.15.86:0/915100115 >> v2:172.21.15.196:3300/0 conn(0x7fd3f809b180 0x7fd3f80cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 --2- 172.21.15.86:0/915100115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f80b12e0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- 172.21.15.86:0/915100115 >> 172.21.15.86:0/915100115 conn(0x7fd3f80b3bf0 msgr2=0x7fd3f80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- 172.21.15.86:0/915100115 shutdown_connections 2024-03-22T04:21:33.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- 172.21.15.86:0/915100115 wait complete. 2024-03-22T04:21:33.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 Processor -- start 2024-03-22T04:21:33.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- start start 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3f806d8f0 0x7fd3f81f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3f809b180 0x7fd3f814e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f8156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f80cdf80 con 0x7fd3f815e970 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f815ed50 con 0x7fd3f809b180 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f80ce100 con 0x7fd3f806d8f0 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f8156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f8156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45150/0 (socket says 172.21.15.86:45150) 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.294+0000 7fd3fcc00640 1 -- 172.21.15.86:0/3350658979 learned_addr learned my addr 172.21.15.86:0/3350658979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3f7e00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3f806d8f0 0x7fd3f81f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3f7400640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3f809b180 0x7fd3f814e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fcc00640 1 -- 172.21.15.86:0/3350658979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3f806d8f0 msgr2=0x7fd3f81f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fcc00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3f806d8f0 0x7fd3f81f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fcc00640 1 -- 172.21.15.86:0/3350658979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3f809b180 msgr2=0x7fd3f814e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fcc00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3f809b180 0x7fd3f814e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fcc00640 1 -- 172.21.15.86:0/3350658979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f8156ec0 con 0x7fd3f815e970 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fcc00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f8156730 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000ce80 tx=0x7fd3f000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3efe00640 1 -- 172.21.15.86:0/3350658979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f001a070 con 0x7fd3f815e970 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3efe00640 1 -- 172.21.15.86:0/3350658979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f000d880 con 0x7fd3f815e970 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3efe00640 1 -- 172.21.15.86:0/3350658979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000dc70 con 0x7fd3f815e970 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f81571a0 con 0x7fd3f815e970 2024-03-22T04:21:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f8157760 con 0x7fd3f815e970 2024-03-22T04:21:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f8074470 con 0x7fd3f815e970 2024-03-22T04:21:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3efe00640 1 -- 172.21.15.86:0/3350658979 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd3f001d070 con 0x7fd3f815e970 2024-03-22T04:21:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3efe00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3d8067b50 0x7fd3d806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.298+0000 7fd3efe00640 1 -- 172.21.15.86:0/3350658979 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7fd3f0086810 con 0x7fd3f815e970 2024-03-22T04:21:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.302+0000 7fd3f7e00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3d8067b50 0x7fd3d806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.302+0000 7fd3f7e00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3d8067b50 0x7fd3d806a010 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80af000 tx=0x7fd3e80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.302+0000 7fd3efe00640 1 -- 172.21.15.86:0/3350658979 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd3f0056980 con 0x7fd3f815e970 2024-03-22T04:21:33.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.458+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3f80d0d60 con 0x7fd3f815e970 2024-03-22T04:21:33.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3efe00640 1 -- 172.21.15.86:0/3350658979 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7fd3f005aa20 con 0x7fd3f815e970 2024-03-22T04:21:33.458 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:21:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3d8067b50 msgr2=0x7fd3d806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3d8067b50 0x7fd3d806a010 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80af000 tx=0x7fd3e80a7000 comp rx=0 tx=0).stop 2024-03-22T04:21:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 msgr2=0x7fd3f8156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f8156730 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000ce80 tx=0x7fd3f000a710 comp rx=0 tx=0).stop 2024-03-22T04:21:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 shutdown_connections 2024-03-22T04:21:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd3d8067b50 0x7fd3d806a010 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3f809b180 0x7fd3f814e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3f815e970 0x7fd3f8156730 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 --2- 172.21.15.86:0/3350658979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3f806d8f0 0x7fd3f81f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 >> 172.21.15.86:0/3350658979 conn(0x7fd3f80b3bf0 msgr2=0x7fd3f80b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 shutdown_connections 2024-03-22T04:21:33.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.462+0000 7fd3fec00640 1 -- 172.21.15.86:0/3350658979 wait complete. 2024-03-22T04:21:33.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:21:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab69400640 1 Processor -- start 2024-03-22T04:21:33.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab69400640 1 -- start start 2024-03-22T04:21:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab69400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab640b3610 0x7fab6409b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab69400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fab640b11c0 0x7fab640b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab69400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab640cda00 con 0x7fab6415c730 2024-03-22T04:21:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab69400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fab6409b910 con 0x7fab640b11c0 2024-03-22T04:21:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab69400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab640b1710 con 0x7fab640b3610 2024-03-22T04:21:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab61e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab640b3610 0x7fab6409b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab61e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab640b3610 0x7fab6409b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:48776/0 (socket says 172.21.15.86:48776) 2024-03-22T04:21:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab61e00640 1 -- 172.21.15.86:0/4269758268 learned_addr learned my addr 172.21.15.86:0/4269758268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.622+0000 7fab63200640 1 --2- 172.21.15.86:0/4269758268 >> v2:172.21.15.196:3300/0 conn(0x7fab640b11c0 0x7fab640b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab61e00640 1 -- 172.21.15.86:0/4269758268 >> v2:172.21.15.196:3300/0 conn(0x7fab640b11c0 msgr2=0x7fab640b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab61e00640 1 --2- 172.21.15.86:0/4269758268 >> v2:172.21.15.196:3300/0 conn(0x7fab640b11c0 0x7fab640b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab61e00640 1 -- 172.21.15.86:0/4269758268 >> v1:172.21.15.196:6789/0 conn(0x7fab6415c730 legacy=0x7fab6409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab61e00640 1 -- 172.21.15.86:0/4269758268 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab640cdce0 con 0x7fab640b3610 2024-03-22T04:21:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab63200640 1 --2- 172.21.15.86:0/4269758268 >> v2:172.21.15.196:3300/0 conn(0x7fab640b11c0 0x7fab640b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab61e00640 1 --2- 172.21.15.86:0/4269758268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab640b3610 0x7fab6409b180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab50009870 tx=0x7fab5001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da6a8e03a97c067 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab61400640 1 -- 172.21.15.86:0/4269758268 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab500b0070 con 0x7fab640b3610 2024-03-22T04:21:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab61400640 1 -- 172.21.15.86:0/4269758268 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab500ab600 con 0x7fab640b3610 2024-03-22T04:21:33.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab61400640 1 -- 172.21.15.86:0/4269758268 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab500ab9d0 con 0x7fab640b3610 2024-03-22T04:21:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab69400640 1 -- 172.21.15.86:0/4269758268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab640b3610 msgr2=0x7fab6409b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab69400640 1 --2- 172.21.15.86:0/4269758268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab640b3610 0x7fab6409b180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab50009870 tx=0x7fab5001c4d0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab69400640 1 -- 172.21.15.86:0/4269758268 shutdown_connections 2024-03-22T04:21:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab69400640 1 --2- 172.21.15.86:0/4269758268 >> v2:172.21.15.196:3300/0 conn(0x7fab640b11c0 0x7fab640b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab69400640 1 --2- 172.21.15.86:0/4269758268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab640b3610 0x7fab6409b180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.626+0000 7fab69400640 1 -- 172.21.15.86:0/4269758268 >> 172.21.15.86:0/4269758268 conn(0x7fab640b3bf0 msgr2=0x7fab640b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 -- 172.21.15.86:0/4269758268 shutdown_connections 2024-03-22T04:21:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 -- 172.21.15.86:0/4269758268 wait complete. 2024-03-22T04:21:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 Processor -- start 2024-03-22T04:21:33.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 -- start start 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640b11c0 0x7fab641546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab640b3610 0x7fab640cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab6415c730 0x7fab640c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab640b2c80 con 0x7fab640b11c0 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab640b2b00 con 0x7fab640b3610 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab69400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab640b2e00 con 0x7fab6415c730 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640b11c0 0x7fab641546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640b11c0 0x7fab641546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45160/0 (socket says 172.21.15.86:45160) 2024-03-22T04:21:33.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 -- 172.21.15.86:0/1497573238 learned_addr learned my addr 172.21.15.86:0/1497573238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab61e00640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab640b3610 0x7fab640cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab63200640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab6415c730 0x7fab640c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 -- 172.21.15.86:0/1497573238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab6415c730 msgr2=0x7fab640c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab6415c730 0x7fab640c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 -- 172.21.15.86:0/1497573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab640b3610 msgr2=0x7fab640cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab640b3610 0x7fab640cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 -- 172.21.15.86:0/1497573238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab64154bf0 con 0x7fab640b11c0 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.630+0000 7fab62800640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640b11c0 0x7fab641546b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c370 tx=0x7fab5400c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab5aa00640 1 -- 172.21.15.86:0/1497573238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540052d0 con 0x7fab640b11c0 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab5aa00640 1 -- 172.21.15.86:0/1497573238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54005470 con 0x7fab640b11c0 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab5aa00640 1 -- 172.21.15.86:0/1497573238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54007260 con 0x7fab640b11c0 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab640ca600 con 0x7fab640b11c0 2024-03-22T04:21:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab640cabc0 con 0x7fab640b11c0 2024-03-22T04:21:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab64074470 con 0x7fab640b11c0 2024-03-22T04:21:33.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab5aa00640 1 -- 172.21.15.86:0/1497573238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fab54007400 con 0x7fab640b11c0 2024-03-22T04:21:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab5aa00640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab44067b20 0x7fab44069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab5aa00640 1 -- 172.21.15.86:0/1497573238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7fab54085510 con 0x7fab640b11c0 2024-03-22T04:21:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab61e00640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab44067b20 0x7fab44069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.634+0000 7fab61e00640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab44067b20 0x7fab44069fe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fab500ad000 tx=0x7fab50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.638+0000 7fab5aa00640 1 -- 172.21.15.86:0/1497573238 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fab540555f0 con 0x7fab640b11c0 2024-03-22T04:21:33.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fab641594c0 con 0x7fab640b11c0 2024-03-22T04:21:33.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab5aa00640 1 -- 172.21.15.86:0/1497573238 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) ==== 104+0+11 (secure 0 0 0) 0x7fab54059690 con 0x7fab640b11c0 2024-03-22T04:21:33.812 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab44067b20 msgr2=0x7fab44069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab44067b20 0x7fab44069fe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fab500ad000 tx=0x7fab50008040 comp rx=0 tx=0).stop 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640b11c0 msgr2=0x7fab641546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640b11c0 0x7fab641546b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c370 tx=0x7fab5400c770 comp rx=0 tx=0).stop 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 shutdown_connections 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fab44067b20 0x7fab44069fe0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab640b3610 0x7fab640cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab6415c730 0x7fab640c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 --2- 172.21.15.86:0/1497573238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640b11c0 0x7fab641546b0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.814+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 >> 172.21.15.86:0/1497573238 conn(0x7fab640b3bf0 msgr2=0x7fab6406fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.818+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 shutdown_connections 2024-03-22T04:21:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.818+0000 7fab69400640 1 -- 172.21.15.86:0/1497573238 wait complete. 2024-03-22T04:21:33.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:21:33.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f9cc00640 1 Processor -- start 2024-03-22T04:21:33.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f9cc00640 1 -- start start 2024-03-22T04:21:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f9cc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f9815c730 0x7f4f9809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f9cc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4f980b11c0 0x7f4f980b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f9cc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f980cda00 con 0x7f4f980b3610 2024-03-22T04:21:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f9cc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4f9809b910 con 0x7f4f980b11c0 2024-03-22T04:21:33.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f9cc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f980b1710 con 0x7f4f9815c730 2024-03-22T04:21:33.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f95e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f9815c730 0x7f4f9809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f96800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4f980b11c0 0x7f4f980b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f95e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f9815c730 0x7f4f9809aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:48792/0 (socket says 172.21.15.86:48792) 2024-03-22T04:21:33.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f95e00640 1 -- 172.21.15.86:0/2284792864 learned_addr learned my addr 172.21.15.86:0/2284792864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f95e00640 1 -- 172.21.15.86:0/2284792864 >> v2:172.21.15.196:3300/0 conn(0x7f4f980b11c0 msgr2=0x7f4f980b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f95e00640 1 --2- 172.21.15.86:0/2284792864 >> v2:172.21.15.196:3300/0 conn(0x7f4f980b11c0 0x7f4f980b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f95e00640 1 -- 172.21.15.86:0/2284792864 >> v1:172.21.15.196:6789/0 conn(0x7f4f980b3610 legacy=0x7f4f9809afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.978+0000 7f4f95e00640 1 -- 172.21.15.86:0/2284792864 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f980cdce0 con 0x7f4f9815c730 2024-03-22T04:21:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.982+0000 7f4f95e00640 1 --2- 172.21.15.86:0/2284792864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f9815c730 0x7f4f9809aa70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009870 tx=0x7f4f8801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d7ecef2ea911bb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f95400640 1 -- 172.21.15.86:0/2284792864 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880b0070 con 0x7f4f9815c730 2024-03-22T04:21:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f95400640 1 -- 172.21.15.86:0/2284792864 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f880ab600 con 0x7f4f9815c730 2024-03-22T04:21:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f95400640 1 -- 172.21.15.86:0/2284792864 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880ab9d0 con 0x7f4f9815c730 2024-03-22T04:21:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 -- 172.21.15.86:0/2284792864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f9815c730 msgr2=0x7f4f9809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/2284792864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f9815c730 0x7f4f9809aa70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009870 tx=0x7f4f8801c4d0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 -- 172.21.15.86:0/2284792864 shutdown_connections 2024-03-22T04:21:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/2284792864 >> v2:172.21.15.196:3300/0 conn(0x7f4f980b11c0 0x7f4f980b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/2284792864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f9815c730 0x7f4f9809aa70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 -- 172.21.15.86:0/2284792864 >> 172.21.15.86:0/2284792864 conn(0x7f4f980b3bf0 msgr2=0x7f4f980b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 -- 172.21.15.86:0/2284792864 shutdown_connections 2024-03-22T04:21:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 -- 172.21.15.86:0/2284792864 wait complete. 2024-03-22T04:21:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 Processor -- start 2024-03-22T04:21:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.986+0000 7f4f9cc00640 1 -- start start 2024-03-22T04:21:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f980b11c0 0x7f4f9806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f980b3610 0x7f4f9806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9815c730 0x7f4f9806dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f980b2c80 con 0x7f4f980b11c0 2024-03-22T04:21:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f980b2b00 con 0x7f4f9815c730 2024-03-22T04:21:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f980b2e00 con 0x7f4f980b3610 2024-03-22T04:21:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f980b11c0 0x7f4f9806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f980b11c0 0x7f4f9806cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45176/0 (socket says 172.21.15.86:45176) 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 -- 172.21.15.86:0/1900129553 learned_addr learned my addr 172.21.15.86:0/1900129553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f8fe00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f980b3610 0x7f4f9806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f96800640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9815c730 0x7f4f9806dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 -- 172.21.15.86:0/1900129553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f980b3610 msgr2=0x7f4f9806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f980b3610 0x7f4f9806d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 -- 172.21.15.86:0/1900129553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9815c730 msgr2=0x7f4f9806dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9815c730 0x7f4f9806dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 -- 172.21.15.86:0/1900129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f980c6c80 con 0x7f4f980b11c0 2024-03-22T04:21:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f95e00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f980b11c0 0x7f4f9806cd40 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4f880ae040 tx=0x7f4f88019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f8e000640 1 -- 172.21.15.86:0/1900129553 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880b0050 con 0x7f4f980b11c0 2024-03-22T04:21:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f8e000640 1 -- 172.21.15.86:0/1900129553 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f880b1040 con 0x7f4f980b11c0 2024-03-22T04:21:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9806d280 con 0x7f4f980b11c0 2024-03-22T04:21:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f8e000640 1 -- 172.21.15.86:0/1900129553 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880b5610 con 0x7f4f980b11c0 2024-03-22T04:21:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9806e680 con 0x7f4f980b11c0 2024-03-22T04:21:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.990+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f98074470 con 0x7f4f980b11c0 2024-03-22T04:21:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.994+0000 7f4f8e000640 1 -- 172.21.15.86:0/1900129553 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4f880be020 con 0x7f4f980b11c0 2024-03-22T04:21:33.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.994+0000 7f4f8e000640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f74067b20 0x7f4f74069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:33.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.994+0000 7f4f8e000640 1 -- 172.21.15.86:0/1900129553 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f4f8811fcb0 con 0x7f4f980b11c0 2024-03-22T04:21:33.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.994+0000 7f4f8fe00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f74067b20 0x7f4f74069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:33.994+0000 7f4f8fe00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f74067b20 0x7f4f74069fe0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4f9806ca90 tx=0x7f4f80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:33.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.002+0000 7f4f8e000640 1 -- 172.21.15.86:0/1900129553 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4f880c5020 con 0x7f4f980b11c0 2024-03-22T04:21:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.158+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f9815fb30 con 0x7f4f980b11c0 2024-03-22T04:21:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.158+0000 7f4f8e000640 1 -- 172.21.15.86:0/1900129553 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) ==== 100+0+11 (secure 0 0 0) 0x7f4f88016150 con 0x7f4f980b11c0 2024-03-22T04:21:34.157 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:21:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f74067b20 msgr2=0x7f4f74069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f74067b20 0x7f4f74069fe0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4f9806ca90 tx=0x7f4f80009040 comp rx=0 tx=0).stop 2024-03-22T04:21:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f980b11c0 msgr2=0x7f4f9806cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f980b11c0 0x7f4f9806cd40 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4f880ae040 tx=0x7f4f88019270 comp rx=0 tx=0).stop 2024-03-22T04:21:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 shutdown_connections 2024-03-22T04:21:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4f74067b20 0x7f4f74069fe0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f9815c730 0x7f4f9806dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f980b3610 0x7f4f9806d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 --2- 172.21.15.86:0/1900129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f980b11c0 0x7f4f9806cd40 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 >> 172.21.15.86:0/1900129553 conn(0x7f4f980b3bf0 msgr2=0x7f4f9815b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 shutdown_connections 2024-03-22T04:21:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.162+0000 7f4f9cc00640 1 -- 172.21.15.86:0/1900129553 wait complete. 2024-03-22T04:21:34.178 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T04:21:34.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe657c00640 1 Processor -- start 2024-03-22T04:21:34.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe657c00640 1 -- start start 2024-03-22T04:21:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe657c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6500ad330 0x7fe65015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe657c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe65006ef40 0x7fe65006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe657c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe65015c7c0 con 0x7fe6500ad710 2024-03-22T04:21:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe657c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe65006dc40 con 0x7fe6500ad330 2024-03-22T04:21:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe657c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe65006dea0 con 0x7fe65006ef40 2024-03-22T04:21:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe655200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6500ad330 0x7fe65015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe655c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe6500ad710 0x7fe65013fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39410/0 (socket says 172.21.15.86:39410) 2024-03-22T04:21:34.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe655c00640 1 -- 172.21.15.86:0/2410339024 learned_addr learned my addr 172.21.15.86:0/2410339024 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe64fe00640 1 --2- 172.21.15.86:0/2410339024 >> v2:172.21.15.196:3300/0 conn(0x7fe65006ef40 0x7fe65006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe64f400640 1 -- 172.21.15.86:0/2410339024 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4071613688 0 0) 0x7fe65015c7c0 con 0x7fe6500ad710 2024-03-22T04:21:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe64f400640 1 -- 172.21.15.86:0/2410339024 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe638003530 con 0x7fe6500ad710 2024-03-22T04:21:34.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.318+0000 7fe655200640 1 -- 172.21.15.86:0/2410339024 >> v2:172.21.15.196:3300/0 conn(0x7fe65006ef40 msgr2=0x7fe65006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe655200640 1 --2- 172.21.15.86:0/2410339024 >> v2:172.21.15.196:3300/0 conn(0x7fe65006ef40 0x7fe65006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe655200640 1 -- 172.21.15.86:0/2410339024 >> v1:172.21.15.86:6789/0 conn(0x7fe6500ad710 legacy=0x7fe65013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe655200640 1 -- 172.21.15.86:0/2410339024 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65006e180 con 0x7fe6500ad330 2024-03-22T04:21:34.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe64fe00640 1 --2- 172.21.15.86:0/2410339024 >> v2:172.21.15.196:3300/0 conn(0x7fe65006ef40 0x7fe65006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe655200640 1 --2- 172.21.15.86:0/2410339024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 0x7fe65015d4a0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe65006fbe0 tx=0x7fe64401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b054f929f4134634 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe64f400640 1 -- 172.21.15.86:0/2410339024 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440b2070 con 0x7fe6500ad330 2024-03-22T04:21:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe64f400640 1 -- 172.21.15.86:0/2410339024 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6440a9070 con 0x7fe6500ad330 2024-03-22T04:21:34.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe64f400640 1 -- 172.21.15.86:0/2410339024 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440ad880 con 0x7fe6500ad330 2024-03-22T04:21:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe657c00640 1 -- 172.21.15.86:0/2410339024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 msgr2=0x7fe65015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe657c00640 1 --2- 172.21.15.86:0/2410339024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 0x7fe65015d4a0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe65006fbe0 tx=0x7fe64401c450 comp rx=0 tx=0).stop 2024-03-22T04:21:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe657c00640 1 -- 172.21.15.86:0/2410339024 shutdown_connections 2024-03-22T04:21:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe657c00640 1 --2- 172.21.15.86:0/2410339024 >> v2:172.21.15.196:3300/0 conn(0x7fe65006ef40 0x7fe65006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe657c00640 1 --2- 172.21.15.86:0/2410339024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 0x7fe65015d4a0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe657c00640 1 -- 172.21.15.86:0/2410339024 >> 172.21.15.86:0/2410339024 conn(0x7fe6500b1df0 msgr2=0x7fe6500b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe657c00640 1 -- 172.21.15.86:0/2410339024 shutdown_connections 2024-03-22T04:21:34.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.322+0000 7fe657c00640 1 -- 172.21.15.86:0/2410339024 wait complete. 2024-03-22T04:21:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe657c00640 1 Processor -- start 2024-03-22T04:21:34.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe657c00640 1 -- start start 2024-03-22T04:21:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe657c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65006ef40 0x7fe6501667c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe657c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 0x7fe65015b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe657c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6500ad710 0x7fe6500b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe657c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501690e0 con 0x7fe6500ad330 2024-03-22T04:21:34.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe657c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe650168f60 con 0x7fe6500ad710 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe657c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe650169260 con 0x7fe65006ef40 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe64fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 0x7fe65015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe655200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65006ef40 0x7fe6501667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe655200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65006ef40 0x7fe6501667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:48810/0 (socket says 172.21.15.86:48810) 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe655200640 1 -- 172.21.15.86:0/681979145 learned_addr learned my addr 172.21.15.86:0/681979145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe655c00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6500ad710 0x7fe6500b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe64fe00640 1 -- 172.21.15.86:0/681979145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65006ef40 msgr2=0x7fe6501667c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe64fe00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65006ef40 0x7fe6501667c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe64fe00640 1 -- 172.21.15.86:0/681979145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6500ad710 msgr2=0x7fe6500b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe64fe00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6500ad710 0x7fe6500b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe64fe00640 1 -- 172.21.15.86:0/681979145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe650166d00 con 0x7fe6500ad330 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe655200640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65006ef40 0x7fe6501667c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe64fe00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 0x7fe65015b080 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00d5d0 tx=0x7fe63c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:34.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.326+0000 7fe64cc00640 1 -- 172.21.15.86:0/681979145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c018020 con 0x7fe6500ad330 2024-03-22T04:21:34.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe64cc00640 1 -- 172.21.15.86:0/681979145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63c014620 con 0x7fe6500ad330 2024-03-22T04:21:34.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65009a5c0 con 0x7fe6500ad330 2024-03-22T04:21:34.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe64cc00640 1 -- 172.21.15.86:0/681979145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c014a10 con 0x7fe6500ad330 2024-03-22T04:21:34.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65009ab80 con 0x7fe6500ad330 2024-03-22T04:21:34.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe650068650 con 0x7fe6500ad330 2024-03-22T04:21:34.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe64cc00640 1 -- 172.21.15.86:0/681979145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fe63c01b020 con 0x7fe6500ad330 2024-03-22T04:21:34.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe64cc00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe630067b20 0x7fe630069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe64cc00640 1 -- 172.21.15.86:0/681979145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7fe63c086540 con 0x7fe6500ad330 2024-03-22T04:21:34.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.330+0000 7fe655200640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe630067b20 0x7fe630069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.334+0000 7fe655200640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe630067b20 0x7fe630069fe0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe64401c490 tx=0x7fe644008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:34.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.334+0000 7fe64cc00640 1 -- 172.21.15.86:0/681979145 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe63c022020 con 0x7fe6500ad330 2024-03-22T04:21:34.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe650140080 con 0x7fe6500ad330 2024-03-22T04:21:34.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe64cc00640 1 -- 172.21.15.86:0/681979145 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v126) ==== 80+0+244 (secure 0 0 0) 0x7fe63c056620 con 0x7fe6500ad330 2024-03-22T04:21:34.507 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/4220897872 2024-03-23T04:21:23.925914+0000 2024-03-22T04:21:34.507 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/4220897872 2024-03-23T04:21:23.925914+0000 2024-03-22T04:21:34.507 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/3165266972 2024-03-23T04:21:12.563892+0000 2024-03-22T04:21:34.507 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/3165266972 2024-03-23T04:21:12.563892+0000 2024-03-22T04:21:34.508 INFO:teuthology.orchestra.run.smithi086.stderr:listed 4 entries 2024-03-22T04:21:34.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe630067b20 msgr2=0x7fe630069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe630067b20 0x7fe630069fe0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe64401c490 tx=0x7fe644008040 comp rx=0 tx=0).stop 2024-03-22T04:21:34.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 msgr2=0x7fe65015b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 0x7fe65015b080 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00d5d0 tx=0x7fe63c00da90 comp rx=0 tx=0).stop 2024-03-22T04:21:34.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 shutdown_connections 2024-03-22T04:21:34.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fe630067b20 0x7fe630069fe0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6500ad710 0x7fe6500b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6500ad330 0x7fe65015b080 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 --2- 172.21.15.86:0/681979145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe65006ef40 0x7fe6501667c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 >> 172.21.15.86:0/681979145 conn(0x7fe6500b1df0 msgr2=0x7fe650071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:34.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 shutdown_connections 2024-03-22T04:21:34.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.510+0000 7fe657c00640 1 -- 172.21.15.86:0/681979145 wait complete. 2024-03-22T04:21:34.528 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T04:21:34.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:34.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T04:21:34.554 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:21:34.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:34.555 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T04:21:34.606 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:21:34.606 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:21:34.606 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:34.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:21:34.617 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:21:34.617 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:34.617 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:34.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:21:34.659 INFO:tasks.cephfs.mount:Terminating background process 2024-03-22T04:21:34.659 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:21:34.659 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:34.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:21:34.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.838+0000 7f4adb000640 1 Processor -- start 2024-03-22T04:21:34.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.838+0000 7f4adb000640 1 -- start start 2024-03-22T04:21:34.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4adb000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ad415e3a0 0x7f4ad40affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4adb000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4ad40b0680 0x7f4ad406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4adb000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad406bfb0 con 0x7f4ad406bba0 2024-03-22T04:21:34.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4adb000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad40b04e0 con 0x7f4ad415e3a0 2024-03-22T04:21:34.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4adb000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad415ebb0 con 0x7f4ad40b0680 2024-03-22T04:21:34.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4ad9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ad415e3a0 0x7f4ad40affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4ad9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ad415e3a0 0x7f4ad40affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45206/0 (socket says 172.21.15.86:45206) 2024-03-22T04:21:34.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4ad9c00640 1 -- 172.21.15.86:0/3026402680 learned_addr learned my addr 172.21.15.86:0/3026402680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:34.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4ad9200640 1 --2- 172.21.15.86:0/3026402680 >> v2:172.21.15.196:3300/0 conn(0x7f4ad40b0680 0x7f4ad406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4ad9c00640 1 -- 172.21.15.86:0/3026402680 >> v2:172.21.15.196:3300/0 conn(0x7f4ad40b0680 msgr2=0x7f4ad406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4ad9c00640 1 --2- 172.21.15.86:0/3026402680 >> v2:172.21.15.196:3300/0 conn(0x7f4ad40b0680 0x7f4ad406b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4ad9c00640 1 -- 172.21.15.86:0/3026402680 >> v1:172.21.15.196:6789/0 conn(0x7f4ad406bba0 legacy=0x7f4ad4161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:34.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.842+0000 7f4ad9c00640 1 -- 172.21.15.86:0/3026402680 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad415ee90 con 0x7f4ad415e3a0 2024-03-22T04:21:34.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4ad9c00640 1 --2- 172.21.15.86:0/3026402680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad40affa0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4068a80 tx=0x7f4ac801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d31811f656429db server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4ad3e00640 1 -- 172.21.15.86:0/3026402680 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80b0070 con 0x7f4ad415e3a0 2024-03-22T04:21:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4ad3e00640 1 -- 172.21.15.86:0/3026402680 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac80ab600 con 0x7f4ad415e3a0 2024-03-22T04:21:34.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4ad3e00640 1 -- 172.21.15.86:0/3026402680 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80ab9d0 con 0x7f4ad415e3a0 2024-03-22T04:21:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4adb000640 1 -- 172.21.15.86:0/3026402680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 msgr2=0x7f4ad40affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4adb000640 1 --2- 172.21.15.86:0/3026402680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad40affa0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4068a80 tx=0x7f4ac801c490 comp rx=0 tx=0).stop 2024-03-22T04:21:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4adb000640 1 -- 172.21.15.86:0/3026402680 shutdown_connections 2024-03-22T04:21:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4adb000640 1 --2- 172.21.15.86:0/3026402680 >> v2:172.21.15.196:3300/0 conn(0x7f4ad40b0680 0x7f4ad406b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4adb000640 1 --2- 172.21.15.86:0/3026402680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad40affa0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4adb000640 1 -- 172.21.15.86:0/3026402680 >> 172.21.15.86:0/3026402680 conn(0x7f4ad40d1000 msgr2=0x7f4ad40d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4adb000640 1 -- 172.21.15.86:0/3026402680 shutdown_connections 2024-03-22T04:21:34.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.846+0000 7f4adb000640 1 -- 172.21.15.86:0/3026402680 wait complete. 2024-03-22T04:21:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 Processor -- start 2024-03-22T04:21:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 -- start start 2024-03-22T04:21:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad406bba0 0x7f4ad4150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ad40b0680 0x7f4ad4152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4070b50 con 0x7f4ad415e3a0 2024-03-22T04:21:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad415e990 con 0x7f4ad406bba0 2024-03-22T04:21:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad4070cd0 con 0x7f4ad40b0680 2024-03-22T04:21:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad415faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45216/0 (socket says 172.21.15.86:45216) 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 -- 172.21.15.86:0/3917170781 learned_addr learned my addr 172.21.15.86:0/3917170781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ad9c00640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad406bba0 0x7f4ad4150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ad9200640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ad40b0680 0x7f4ad4152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 -- 172.21.15.86:0/3917170781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ad40b0680 msgr2=0x7f4ad4152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ad40b0680 0x7f4ad4152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 -- 172.21.15.86:0/3917170781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad406bba0 msgr2=0x7f4ad4150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad406bba0 0x7f4ad4150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 -- 172.21.15.86:0/3917170781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad41513c0 con 0x7f4ad415e3a0 2024-03-22T04:21:34.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ada600640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad415faa0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00c760 tx=0x7f4acc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:34.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ad1600640 1 -- 172.21.15.86:0/3917170781 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc003b80 con 0x7f4ad415e3a0 2024-03-22T04:21:34.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ad1600640 1 -- 172.21.15.86:0/3917170781 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc003d20 con 0x7f4ad415e3a0 2024-03-22T04:21:34.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad4157810 con 0x7f4ad415e3a0 2024-03-22T04:21:34.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.850+0000 7f4ad1600640 1 -- 172.21.15.86:0/3917170781 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc00aa30 con 0x7f4ad415e3a0 2024-03-22T04:21:34.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.854+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad4157dd0 con 0x7f4ad415e3a0 2024-03-22T04:21:34.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.854+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad4074470 con 0x7f4ad415e3a0 2024-03-22T04:21:34.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.854+0000 7f4ad1600640 1 -- 172.21.15.86:0/3917170781 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f4acc00abd0 con 0x7f4ad415e3a0 2024-03-22T04:21:34.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.854+0000 7f4ad1600640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ab4067b20 0x7f4ab4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:34.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.854+0000 7f4ad1600640 1 -- 172.21.15.86:0/3917170781 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f4acc085fc0 con 0x7f4ad415e3a0 2024-03-22T04:21:34.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.854+0000 7f4ad9c00640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ab4067b20 0x7f4ab4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:34.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.854+0000 7f4ad9c00640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ab4067b20 0x7f4ab4069fe0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80ad000 tx=0x7f4ac8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:34.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:34.858+0000 7f4ad1600640 1 -- 172.21.15.86:0/3917170781 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4ad406bfb0 con 0x7f4ad415e3a0 2024-03-22T04:21:35.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.050+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ad4069970 con 0x7f4ad415e3a0 2024-03-22T04:21:35.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.050+0000 7f4ad1600640 1 -- 172.21.15.86:0/3917170781 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5194 (secure 0 0 0) 0x7f4acc0566f0 con 0x7f4ad415e3a0 2024-03-22T04:21:35.048 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:35.048 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:28.964789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3342090879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3342090879},{"type":"v1","addr":"172.21.15.86:6837","nonce":3342090879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9124,"qdb_cluster":[9124]},"id":8}]} 2024-03-22T04:21:35.049 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-03-22T04:21:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.050+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ab4067b20 msgr2=0x7f4ab4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.050+0000 7f4adb000640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ab4067b20 0x7f4ab4069fe0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80ad000 tx=0x7f4ac8008040 comp rx=0 tx=0).stop 2024-03-22T04:21:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.050+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 msgr2=0x7f4ad415faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.050+0000 7f4adb000640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad415faa0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00c760 tx=0x7f4acc00cc20 comp rx=0 tx=0).stop 2024-03-22T04:21:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.054+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 shutdown_connections 2024-03-22T04:21:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.054+0000 7f4adb000640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4ab4067b20 0x7f4ab4069fe0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.054+0000 7f4adb000640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad415e3a0 0x7f4ad415faa0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.054+0000 7f4adb000640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ad40b0680 0x7f4ad4152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.054+0000 7f4adb000640 1 --2- 172.21.15.86:0/3917170781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ad406bba0 0x7f4ad4150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.054+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 >> 172.21.15.86:0/3917170781 conn(0x7f4ad40d1000 msgr2=0x7f4ad40b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.054+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 shutdown_connections 2024-03-22T04:21:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.054+0000 7f4adb000640 1 -- 172.21.15.86:0/3917170781 wait complete. 2024-03-22T04:21:35.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:35.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f2775000640 1 Processor -- start 2024-03-22T04:21:35.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f2775000640 1 -- start start 2024-03-22T04:21:35.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f2775000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27700ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f2775000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f27700d19a0 0x7f27700cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f2775000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27700d11c0 con 0x7f27700d1dc0 2024-03-22T04:21:35.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f2775000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f27700cece0 con 0x7f27700d19a0 2024-03-22T04:21:35.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f2775000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f27700cefc0 con 0x7f27700d0de0 2024-03-22T04:21:35.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f276de00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27700ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f276d400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f27700d19a0 0x7f27700cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f276de00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27700ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:48826/0 (socket says 172.21.15.86:48826) 2024-03-22T04:21:35.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.206+0000 7f276de00640 1 -- 172.21.15.86:0/2585556422 learned_addr learned my addr 172.21.15.86:0/2585556422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:35.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276d400640 1 -- 172.21.15.86:0/2585556422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 msgr2=0x7f27700ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:35.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276d400640 1 --2- 172.21.15.86:0/2585556422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27700ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276d400640 1 -- 172.21.15.86:0/2585556422 >> v1:172.21.15.196:6789/0 conn(0x7f27700d1dc0 legacy=0x7f277015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276d400640 1 -- 172.21.15.86:0/2585556422 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27700cf2a0 con 0x7f27700d19a0 2024-03-22T04:21:35.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276de00640 1 --2- 172.21.15.86:0/2585556422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27700ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:35.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276d400640 1 --2- 172.21.15.86:0/2585556422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d19a0 0x7f27700cfe70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f275c0097c0 tx=0x7f275c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9cf3351fd37077 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276ca00640 1 -- 172.21.15.86:0/2585556422 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0b2070 con 0x7f27700d19a0 2024-03-22T04:21:35.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276ca00640 1 -- 172.21.15.86:0/2585556422 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c0ad640 con 0x7f27700d19a0 2024-03-22T04:21:35.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.210+0000 7f276ca00640 1 -- 172.21.15.86:0/2585556422 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0ada10 con 0x7f27700d19a0 2024-03-22T04:21:35.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 -- 172.21.15.86:0/2585556422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d19a0 msgr2=0x7f27700cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 --2- 172.21.15.86:0/2585556422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d19a0 0x7f27700cfe70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f275c0097c0 tx=0x7f275c01c4a0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 -- 172.21.15.86:0/2585556422 shutdown_connections 2024-03-22T04:21:35.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 --2- 172.21.15.86:0/2585556422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d19a0 0x7f27700cfe70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 --2- 172.21.15.86:0/2585556422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27700ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 -- 172.21.15.86:0/2585556422 >> 172.21.15.86:0/2585556422 conn(0x7f277006f910 msgr2=0x7f277006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:35.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 -- 172.21.15.86:0/2585556422 shutdown_connections 2024-03-22T04:21:35.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 -- 172.21.15.86:0/2585556422 wait complete. 2024-03-22T04:21:35.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 Processor -- start 2024-03-22T04:21:35.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.214+0000 7f2775000640 1 -- start start 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27701506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27700d19a0 0x7f27701537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d1dc0 0x7f27700c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2770160760 con 0x7f27700d19a0 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27701605e0 con 0x7f27700d1dc0 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2770071e30 con 0x7f27700d0de0 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27700d19a0 0x7f27701537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27700d19a0 0x7f27701537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45232/0 (socket says 172.21.15.86:45232) 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 -- 172.21.15.86:0/2119756970 learned_addr learned my addr 172.21.15.86:0/2119756970 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:35.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276de00640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27701506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276e800640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d1dc0 0x7f27700c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 -- 172.21.15.86:0/2119756970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 msgr2=0x7f27701506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27701506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 -- 172.21.15.86:0/2119756970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d1dc0 msgr2=0x7f27700c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d1dc0 0x7f27700c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 -- 172.21.15.86:0/2119756970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2770150ca0 con 0x7f27700d19a0 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f276d400640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27700d19a0 0x7f27701537b0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f275c0b0040 tx=0x7f275c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2766000640 1 -- 172.21.15.86:0/2119756970 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0ba020 con 0x7f27700d19a0 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2766000640 1 -- 172.21.15.86:0/2119756970 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c0b6410 con 0x7f27700d19a0 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2766000640 1 -- 172.21.15.86:0/2119756970 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0b2040 con 0x7f27700d19a0 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2770150f80 con 0x7f27700d19a0 2024-03-22T04:21:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2770151540 con 0x7f27700d19a0 2024-03-22T04:21:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2770068650 con 0x7f27700d19a0 2024-03-22T04:21:35.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.218+0000 7f2766000640 1 -- 172.21.15.86:0/2119756970 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f275c0bd020 con 0x7f27700d19a0 2024-03-22T04:21:35.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.222+0000 7f2766000640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2750067b20 0x7f2750069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.222+0000 7f2766000640 1 -- 172.21.15.86:0/2119756970 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f275c11ffc0 con 0x7f27700d19a0 2024-03-22T04:21:35.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.222+0000 7f276de00640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2750067b20 0x7f2750069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.222+0000 7f276de00640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2750067b20 0x7f2750069fe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2760006cb0 tx=0x7f2760009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.226+0000 7f2766000640 1 -- 172.21.15.86:0/2119756970 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f275c0c4030 con 0x7f27700d19a0 2024-03-22T04:21:35.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.382+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27700cc640 con 0x7f27700d19a0 2024-03-22T04:21:35.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2766000640 1 -- 172.21.15.86:0/2119756970 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+17215 (secure 0 0 0) 0x7f275c016090 con 0x7f27700d19a0 2024-03-22T04:21:35.383 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:35.383 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":126,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:21:23.925937+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6835/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6837/3165266972":"2024-03-23T04:21:12.563892+0000","172.21.15.86:6835/3165266972":"2024-03-23T04:21:12.563892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2750067b20 msgr2=0x7f2750069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2750067b20 0x7f2750069fe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2760006cb0 tx=0x7f2760009240 comp rx=0 tx=0).stop 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27700d19a0 msgr2=0x7f27701537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27700d19a0 0x7f27701537b0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f275c0b0040 tx=0x7f275c019240 comp rx=0 tx=0).stop 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 shutdown_connections 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2750067b20 0x7f2750069fe0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27700d1dc0 0x7f27700c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27700d19a0 0x7f27701537b0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 --2- 172.21.15.86:0/2119756970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27700d0de0 0x7f27701506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 >> 172.21.15.86:0/2119756970 conn(0x7f277006f910 msgr2=0x7f2770069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 shutdown_connections 2024-03-22T04:21:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.386+0000 7f2775000640 1 -- 172.21.15.86:0/2119756970 wait complete. 2024-03-22T04:21:35.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T04:10:21.815722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-22T04:20:55.270181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-22T04:20:56.532178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T04:21:35.404 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T04:21:35.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.546+0000 7f74aec00640 1 Processor -- start 2024-03-22T04:21:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.546+0000 7f74aec00640 1 -- start start 2024-03-22T04:21:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.546+0000 7f74aec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f74a80cd830 0x7f74a80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.546+0000 7f74aec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a80b14c0 con 0x7f74a80cfb90 2024-03-22T04:21:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.546+0000 7f74aec00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a80ccad0 con 0x7f74a80d05a0 2024-03-22T04:21:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.546+0000 7f74aec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f74a80d0090 con 0x7f74a80cd830 2024-03-22T04:21:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ace00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f74a80cfb90 0x7f74a815d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39420/0 (socket says 172.21.15.86:39420) 2024-03-22T04:21:35.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ace00640 1 -- 172.21.15.86:0/3945847060 learned_addr learned my addr 172.21.15.86:0/3945847060 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ae200640 1 --2- 172.21.15.86:0/3945847060 >> v2:172.21.15.196:3300/0 conn(0x7f74a80cd830 0x7f74a80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2304668080 0 0) 0x7f7494001120 con 0x7f74a80cfb90 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7490003530 con 0x7f74a80cfb90 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3738421526 0 0) 0x7f7490003530 con 0x7f74a80cfb90 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 >> v2:172.21.15.196:3300/0 conn(0x7f74a80cd830 msgr2=0x7f74a80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 --2- 172.21.15.86:0/3945847060 >> v2:172.21.15.196:3300/0 conn(0x7f74a80cd830 0x7f74a80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 >> v1:172.21.15.196:6789/0 conn(0x7f74a80d05a0 legacy=0x7f74a80d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a80ccd50 con 0x7f74a80cfb90 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f7494001850 con 0x7f74a80cfb90 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ae200640 1 --2- 172.21.15.86:0/3945847060 >> v2:172.21.15.196:3300/0 conn(0x7f74a80cd830 0x7f74a80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 >> v1:172.21.15.86:6789/0 conn(0x7f74a80cfb90 legacy=0x7f74a815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f7490003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7490004810 0x7f7490004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74900051d0 0x7f749000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74a80d0090 con 0x7f7490003a10 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f749000c0d0 con 0x7f74900051d0 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f749000c2d0 con 0x7f7490004810 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ad800640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f7490003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ae200640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74900051d0 0x7f749000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ace00640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7490004810 0x7f7490004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ad800640 1 -- 172.21.15.86:0/3945847060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7490004810 msgr2=0x7f7490004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ad800640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7490004810 0x7f7490004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ad800640 1 -- 172.21.15.86:0/3945847060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74900051d0 msgr2=0x7f749000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ad800640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74900051d0 0x7f749000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ad800640 1 -- 172.21.15.86:0/3945847060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7490003530 con 0x7f7490003a10 2024-03-22T04:21:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.550+0000 7f74ae200640 1 -- 172.21.15.86:0/3945847060 reap_dead start 2024-03-22T04:21:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74ad800640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f7490003e30 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f749c001780 tx=0x7f749c01c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e656806adad1c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c0013a0 con 0x7f7490003a10 2024-03-22T04:21:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749c0052f0 con 0x7f7490003a10 2024-03-22T04:21:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74a7e00640 1 -- 172.21.15.86:0/3945847060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c003840 con 0x7f7490003a10 2024-03-22T04:21:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74aec00640 1 -- 172.21.15.86:0/3945847060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 msgr2=0x7f7490003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74aec00640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f7490003e30 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f749c001780 tx=0x7f749c01c220 comp rx=0 tx=0).stop 2024-03-22T04:21:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74aec00640 1 -- 172.21.15.86:0/3945847060 shutdown_connections 2024-03-22T04:21:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74aec00640 1 --2- 172.21.15.86:0/3945847060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f7490003e30 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74aec00640 1 -- 172.21.15.86:0/3945847060 >> 172.21.15.86:0/3945847060 conn(0x7f74a809aa70 msgr2=0x7f74a809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74aec00640 1 -- 172.21.15.86:0/3945847060 shutdown_connections 2024-03-22T04:21:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.554+0000 7f74aec00640 1 -- 172.21.15.86:0/3945847060 wait complete. 2024-03-22T04:21:35.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 Processor -- start 2024-03-22T04:21:35.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 -- start start 2024-03-22T04:21:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f74a81597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74a80c7960 0x7f74a80c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74a809cdf0 0x7f74a809d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74a80d0f10 con 0x7f7490003a10 2024-03-22T04:21:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74a80d0d90 con 0x7f74a809cdf0 2024-03-22T04:21:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f74a80cee80 con 0x7f74a80c7960 2024-03-22T04:21:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f74a81597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f74a81597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45248/0 (socket says 172.21.15.86:45248) 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 -- 172.21.15.86:0/2225567748 learned_addr learned my addr 172.21.15.86:0/2225567748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ace00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74a80c7960 0x7f74a80c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ae200640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74a809cdf0 0x7f74a809d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 -- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74a80c7960 msgr2=0x7f74a80c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74a80c7960 0x7f74a80c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 -- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74a809cdf0 msgr2=0x7f74a809d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74a809cdf0 0x7f74a809d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 -- 172.21.15.86:0/2225567748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a809db70 con 0x7f7490003a10 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ace00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74a80c7960 0x7f74a80c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74ad800640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f74a81597c0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f749c001780 tx=0x7f7490005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74a5600640 1 -- 172.21.15.86:0/2225567748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c007a40 con 0x7f7490003a10 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74a5600640 1 -- 172.21.15.86:0/2225567748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749c0bc070 con 0x7f7490003a10 2024-03-22T04:21:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74a5600640 1 -- 172.21.15.86:0/2225567748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c0013a0 con 0x7f7490003a10 2024-03-22T04:21:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a809de50 con 0x7f7490003a10 2024-03-22T04:21:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.558+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a809e410 con 0x7f7490003a10 2024-03-22T04:21:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.562+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a804f900 con 0x7f7490003a10 2024-03-22T04:21:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.562+0000 7f74a5600640 1 -- 172.21.15.86:0/2225567748 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f749c0b4040 con 0x7f7490003a10 2024-03-22T04:21:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.562+0000 7f74a5600640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7488067b50 0x7f748806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.562+0000 7f74a5600640 1 -- 172.21.15.86:0/2225567748 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f749c11ffc0 con 0x7f7490003a10 2024-03-22T04:21:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.562+0000 7f74ace00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7488067b50 0x7f748806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.562+0000 7f74ace00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7488067b50 0x7f748806a010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7494001060 tx=0x7f7494009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.566+0000 7f74a5600640 1 -- 172.21.15.86:0/2225567748 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f74a80ccad0 con 0x7f7490003a10 2024-03-22T04:21:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.722+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74a80b7f40 con 0x7f7490003a10 2024-03-22T04:21:35.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.722+0000 7f74a5600640 1 -- 172.21.15.86:0/2225567748 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f749c0f06f0 con 0x7f7490003a10 2024-03-22T04:21:35.721 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-22T04:21:35.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7488067b50 msgr2=0x7f748806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7488067b50 0x7f748806a010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7494001060 tx=0x7f7494009040 comp rx=0 tx=0).stop 2024-03-22T04:21:35.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 msgr2=0x7f74a81597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f74a81597c0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f749c001780 tx=0x7f7490005840 comp rx=0 tx=0).stop 2024-03-22T04:21:35.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 shutdown_connections 2024-03-22T04:21:35.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f7488067b50 0x7f748806a010 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74a809cdf0 0x7f74a809d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74a80c7960 0x7f74a80c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 --2- 172.21.15.86:0/2225567748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7490003a10 0x7f74a81597c0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 >> 172.21.15.86:0/2225567748 conn(0x7f74a809aa70 msgr2=0x7f74a809ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:35.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 shutdown_connections 2024-03-22T04:21:35.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.726+0000 7f74aec00640 1 -- 172.21.15.86:0/2225567748 wait complete. 2024-03-22T04:21:35.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-22T04:21:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f13800640 1 Processor -- start 2024-03-22T04:21:35.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f13800640 1 -- start start 2024-03-22T04:21:35.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f13800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5f0c0afb90 0x7f5f0c0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f13800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f0c074de0 con 0x7f5f0c15af10 2024-03-22T04:21:35.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f13800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f0c0c6630 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f13800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5f0c0c7240 con 0x7f5f0c0afb90 2024-03-22T04:21:35.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f10e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5f0c15af10 0x7f5f0c0cc250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40172/0 (socket says 172.21.15.86:40172) 2024-03-22T04:21:35.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f10e00640 1 -- 172.21.15.86:0/477069082 learned_addr learned my addr 172.21.15.86:0/477069082 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:35.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f11800640 1 --2- 172.21.15.86:0/477069082 >> v2:172.21.15.196:3300/0 conn(0x7f5f0c0afb90 0x7f5f0c0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f0be00640 1 -- 172.21.15.86:0/477069082 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929669937 0 0) 0x7f5f0c074de0 con 0x7f5f0c15af10 2024-03-22T04:21:35.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f0be00640 1 -- 172.21.15.86:0/477069082 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ef0003530 con 0x7f5f0c15af10 2024-03-22T04:21:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f11800640 1 -- 172.21.15.86:0/477069082 >> v1:172.21.15.196:6789/0 conn(0x7f5f0c15b2f0 legacy=0x7f5f0c0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f11800640 1 -- 172.21.15.86:0/477069082 >> v1:172.21.15.86:6789/0 conn(0x7f5f0c15af10 legacy=0x7f5f0c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.918+0000 7f5f11800640 1 -- 172.21.15.86:0/477069082 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c0c6910 con 0x7f5f0c0afb90 2024-03-22T04:21:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f11800640 1 --2- 172.21.15.86:0/477069082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c0afb90 0x7f5f0c0c6000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009d90 tx=0x7f5f0401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20eb5d5274ca94c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f0be00640 1 -- 172.21.15.86:0/477069082 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040b2070 con 0x7f5f0c0afb90 2024-03-22T04:21:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f0be00640 1 -- 172.21.15.86:0/477069082 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f040a57a0 con 0x7f5f0c0afb90 2024-03-22T04:21:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f0be00640 1 -- 172.21.15.86:0/477069082 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040a5b70 con 0x7f5f0c0afb90 2024-03-22T04:21:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f13800640 1 -- 172.21.15.86:0/477069082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c0afb90 msgr2=0x7f5f0c0c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f13800640 1 --2- 172.21.15.86:0/477069082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c0afb90 0x7f5f0c0c6000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009d90 tx=0x7f5f0401c8e0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f13800640 1 -- 172.21.15.86:0/477069082 shutdown_connections 2024-03-22T04:21:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f13800640 1 --2- 172.21.15.86:0/477069082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c0afb90 0x7f5f0c0c6000 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f13800640 1 -- 172.21.15.86:0/477069082 >> 172.21.15.86:0/477069082 conn(0x7f5f0c0d0730 msgr2=0x7f5f0c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f13800640 1 -- 172.21.15.86:0/477069082 shutdown_connections 2024-03-22T04:21:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.922+0000 7f5f13800640 1 -- 172.21.15.86:0/477069082 wait complete. 2024-03-22T04:21:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 Processor -- start 2024-03-22T04:21:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 -- start start 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f0c0afb90 0x7f5f0c0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c15af10 0x7f5f0c0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c15b2f0 0x7f5f0c150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c071ce0 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c0cf470 con 0x7f5f0c15af10 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c071e60 con 0x7f5f0c0afb90 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c15b2f0 0x7f5f0c150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c15b2f0 0x7f5f0c150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36882/0 (socket says 172.21.15.86:36882) 2024-03-22T04:21:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 -- 172.21.15.86:0/3513008632 learned_addr learned my addr 172.21.15.86:0/3513008632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:35.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f10e00640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f0c0afb90 0x7f5f0c0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f03e00640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c15af10 0x7f5f0c0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 -- 172.21.15.86:0/3513008632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f0c0afb90 msgr2=0x7f5f0c0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f0c0afb90 0x7f5f0c0af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 -- 172.21.15.86:0/3513008632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c15af10 msgr2=0x7f5f0c0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c15af10 0x7f5f0c0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 -- 172.21.15.86:0/3513008632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c0c9f70 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f11800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c15b2f0 0x7f5f0c150f60 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5f0401a510 tx=0x7f5f04019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f09600640 1 -- 172.21.15.86:0/3513008632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040b2050 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f09600640 1 -- 172.21.15.86:0/3513008632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f040be070 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0c0ca250 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f09600640 1 -- 172.21.15.86:0/3513008632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040ad850 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.926+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0c0c7df0 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.930+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0c1f7450 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.930+0000 7f5f09600640 1 -- 172.21.15.86:0/3513008632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f5f040ad9f0 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.930+0000 7f5f09600640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5ee8067b20 0x7f5ee8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:35.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.930+0000 7f5f09600640 1 -- 172.21.15.86:0/3513008632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f5f04121160 con 0x7f5f0c15b2f0 2024-03-22T04:21:35.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.930+0000 7f5f10e00640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5ee8067b20 0x7f5ee8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.930+0000 7f5f10e00640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5ee8067b20 0x7f5ee8069fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0011f0 tx=0x7f5efc007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:35.934+0000 7f5f09600640 1 -- 172.21.15.86:0/3513008632 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5f0c1f7450 con 0x7f5f0c15b2f0 2024-03-22T04:21:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5f0c074470 con 0x7f5f0c15b2f0 2024-03-22T04:21:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f09600640 1 -- 172.21.15.86:0/3513008632 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) ==== 104+0+11 (secure 0 0 0) 0x7f5f04016560 con 0x7f5f0c15b2f0 2024-03-22T04:21:36.091 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-22T04:21:36.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5ee8067b20 msgr2=0x7f5ee8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5ee8067b20 0x7f5ee8069fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0011f0 tx=0x7f5efc007d40 comp rx=0 tx=0).stop 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c15b2f0 msgr2=0x7f5f0c150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c15b2f0 0x7f5f0c150f60 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5f0401a510 tx=0x7f5f04019680 comp rx=0 tx=0).stop 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 shutdown_connections 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f5ee8067b20 0x7f5ee8069fe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0c15af10 0x7f5f0c0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c15b2f0 0x7f5f0c150f60 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 --2- 172.21.15.86:0/3513008632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f0c0afb90 0x7f5f0c0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 >> 172.21.15.86:0/3513008632 conn(0x7f5f0c0d0730 msgr2=0x7f5f0c0b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 shutdown_connections 2024-03-22T04:21:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.094+0000 7f5f13800640 1 -- 172.21.15.86:0/3513008632 wait complete. 2024-03-22T04:21:36.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-22T04:21:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.258+0000 7f9916e00640 1 Processor -- start 2024-03-22T04:21:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.258+0000 7f9916e00640 1 -- start start 2024-03-22T04:21:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9916e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f991015c730 0x7f991009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9916e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99100b11c0 0x7f99100b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9916e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99100cda00 con 0x7f99100b3610 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9916e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f991009b910 con 0x7f99100b11c0 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9916e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99100b1710 con 0x7f991015c730 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9914e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99100b11c0 0x7f99100b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f990fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f991015c730 0x7f991009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9907e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f99100b3610 0x7f991009afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:40124/0 (socket says 172.21.15.86:40124) 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f990fe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f991015c730 0x7f991009aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:60886/0 (socket says 172.21.15.86:60886) 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9914e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99100b11c0 0x7f99100b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:58476/0 (socket says 172.21.15.86:58476) 2024-03-22T04:21:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9907e00640 1 -- 172.21.15.86:0/2009287727 learned_addr learned my addr 172.21.15.86:0/2009287727 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f990fe00640 1 -- 172.21.15.86:0/2009287727 >> v2:172.21.15.196:3300/0 conn(0x7f99100b11c0 msgr2=0x7f99100b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f990fe00640 1 --2- 172.21.15.86:0/2009287727 >> v2:172.21.15.196:3300/0 conn(0x7f99100b11c0 0x7f99100b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f990fe00640 1 -- 172.21.15.86:0/2009287727 >> v1:172.21.15.196:6789/0 conn(0x7f99100b3610 legacy=0x7f991009afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f990fe00640 1 -- 172.21.15.86:0/2009287727 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99100cdce0 con 0x7f991015c730 2024-03-22T04:21:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.262+0000 7f9914e00640 1 --2- 172.21.15.86:0/2009287727 >> v2:172.21.15.196:3300/0 conn(0x7f99100b11c0 0x7f99100b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:36.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f990fe00640 1 --2- 172.21.15.86:0/2009287727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f991015c730 0x7f991009aa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f99000097c0 tx=0x7f990001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7bdd56281514a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f990f400640 1 -- 172.21.15.86:0/2009287727 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000b0070 con 0x7f991015c730 2024-03-22T04:21:36.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f990f400640 1 -- 172.21.15.86:0/2009287727 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99000ab600 con 0x7f991015c730 2024-03-22T04:21:36.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f990f400640 1 -- 172.21.15.86:0/2009287727 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000ab9d0 con 0x7f991015c730 2024-03-22T04:21:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 -- 172.21.15.86:0/2009287727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f991015c730 msgr2=0x7f991009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 --2- 172.21.15.86:0/2009287727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f991015c730 0x7f991009aa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f99000097c0 tx=0x7f990001c410 comp rx=0 tx=0).stop 2024-03-22T04:21:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 -- 172.21.15.86:0/2009287727 shutdown_connections 2024-03-22T04:21:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 --2- 172.21.15.86:0/2009287727 >> v2:172.21.15.196:3300/0 conn(0x7f99100b11c0 0x7f99100b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 --2- 172.21.15.86:0/2009287727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f991015c730 0x7f991009aa70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 -- 172.21.15.86:0/2009287727 >> 172.21.15.86:0/2009287727 conn(0x7f99100b3bf0 msgr2=0x7f99100b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 -- 172.21.15.86:0/2009287727 shutdown_connections 2024-03-22T04:21:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 -- 172.21.15.86:0/2009287727 wait complete. 2024-03-22T04:21:36.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 Processor -- start 2024-03-22T04:21:36.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.266+0000 7f9916e00640 1 -- start start 2024-03-22T04:21:36.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99100b11c0 0x7f991015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99100b3610 0x7f99100b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f991015c730 0x7f99100b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99100b2c80 con 0x7f99100b11c0 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99100b2b00 con 0x7f991015c730 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99100b2e00 con 0x7f99100b3610 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99100b11c0 0x7f991015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99100b11c0 0x7f991015afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36898/0 (socket says 172.21.15.86:36898) 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 -- 172.21.15.86:0/1640433671 learned_addr learned my addr 172.21.15.86:0/1640433671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9914e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f991015c730 0x7f99100b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9907e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99100b3610 0x7f99100b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 -- 172.21.15.86:0/1640433671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99100b3610 msgr2=0x7f99100b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99100b3610 0x7f99100b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 -- 172.21.15.86:0/1640433671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f991015c730 msgr2=0x7f99100b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f991015c730 0x7f99100b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 -- 172.21.15.86:0/1640433671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991006d5a0 con 0x7f99100b11c0 2024-03-22T04:21:36.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990fe00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99100b11c0 0x7f991015afa0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f990001aac0 tx=0x7f99000191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990cc00640 1 -- 172.21.15.86:0/1640433671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000b0040 con 0x7f99100b11c0 2024-03-22T04:21:36.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990cc00640 1 -- 172.21.15.86:0/1640433671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99000be070 con 0x7f99100b11c0 2024-03-22T04:21:36.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990cc00640 1 -- 172.21.15.86:0/1640433671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000b7680 con 0x7f99100b11c0 2024-03-22T04:21:36.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991006d880 con 0x7f99100b11c0 2024-03-22T04:21:36.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991006de40 con 0x7f99100b11c0 2024-03-22T04:21:36.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9910074470 con 0x7f99100b11c0 2024-03-22T04:21:36.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.270+0000 7f990cc00640 1 -- 172.21.15.86:0/1640433671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f99000b3040 con 0x7f99100b11c0 2024-03-22T04:21:36.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.274+0000 7f990cc00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98ec067b20 0x7f98ec069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.274+0000 7f990cc00640 1 -- 172.21.15.86:0/1640433671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f99001202e0 con 0x7f99100b11c0 2024-03-22T04:21:36.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.274+0000 7f9907e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98ec067b20 0x7f98ec069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.274+0000 7f9907e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98ec067b20 0x7f98ec069fe0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f99101529a0 tx=0x7f98f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.278+0000 7f990cc00640 1 -- 172.21.15.86:0/1640433671 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9900016090 con 0x7f99100b11c0 2024-03-22T04:21:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.438+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f99100c5770 con 0x7f99100b11c0 2024-03-22T04:21:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.438+0000 7f990cc00640 1 -- 172.21.15.86:0/1640433671 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) ==== 100+0+11 (secure 0 0 0) 0x7f99000f0980 con 0x7f99100b11c0 2024-03-22T04:21:36.437 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98ec067b20 msgr2=0x7f98ec069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98ec067b20 0x7f98ec069fe0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f99101529a0 tx=0x7f98f8009040 comp rx=0 tx=0).stop 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99100b11c0 msgr2=0x7f991015afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99100b11c0 0x7f991015afa0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f990001aac0 tx=0x7f99000191b0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 shutdown_connections 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98ec067b20 0x7f98ec069fe0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f991015c730 0x7f99100b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99100b3610 0x7f99100b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 --2- 172.21.15.86:0/1640433671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99100b11c0 0x7f991015afa0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 >> 172.21.15.86:0/1640433671 conn(0x7f99100b3bf0 msgr2=0x7f99100aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 shutdown_connections 2024-03-22T04:21:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.442+0000 7f9916e00640 1 -- 172.21.15.86:0/1640433671 wait complete. 2024-03-22T04:21:36.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46af200640 1 Processor -- start 2024-03-22T04:21:36.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46af200640 1 -- start start 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46af200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b015aee0 0x7f46b006de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46af200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f46b00b4780 0x7f46b006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46af200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f46b0071da0 0x7f46b00721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46af200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f46b00b43d0 con 0x7f46b00b4780 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46af200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f46b0072720 con 0x7f46b0071da0 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46af200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46b006e500 con 0x7f46b015aee0 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ad400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f46b00b4780 0x7f46b006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ad400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f46b00b4780 0x7f46b006f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36914/0 (socket says 172.21.15.86:36914) 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ad400640 1 -- 172.21.15.86:0/401073570 learned_addr learned my addr 172.21.15.86:0/401073570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ade00640 1 --2- 172.21.15.86:0/401073570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b015aee0 0x7f46b006de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ae800640 1 --2- 172.21.15.86:0/401073570 >> v2:172.21.15.196:3300/0 conn(0x7f46b0071da0 0x7f46b00721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ad400640 1 -- 172.21.15.86:0/401073570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b015aee0 msgr2=0x7f46b006de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ad400640 1 --2- 172.21.15.86:0/401073570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b015aee0 0x7f46b006de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ad400640 1 -- 172.21.15.86:0/401073570 >> v2:172.21.15.196:3300/0 conn(0x7f46b0071da0 msgr2=0x7f46b00721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ad400640 1 --2- 172.21.15.86:0/401073570 >> v2:172.21.15.196:3300/0 conn(0x7f46b0071da0 0x7f46b00721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ad400640 1 -- 172.21.15.86:0/401073570 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b006e7e0 con 0x7f46b00b4780 2024-03-22T04:21:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.598+0000 7f46ade00640 1 --2- 172.21.15.86:0/401073570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b015aee0 0x7f46b006de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:36.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46ad400640 1 --2- 172.21.15.86:0/401073570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b006f5a0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f46b0072fc0 tx=0x7f469c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a96b2a02cf9dbf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46aca00640 1 -- 172.21.15.86:0/401073570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0aa660 con 0x7f46b00b4780 2024-03-22T04:21:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46aca00640 1 -- 172.21.15.86:0/401073570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f469c0af070 con 0x7f46b00b4780 2024-03-22T04:21:36.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46aca00640 1 -- 172.21.15.86:0/401073570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0aaa20 con 0x7f46b00b4780 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 -- 172.21.15.86:0/401073570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 msgr2=0x7f46b006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 --2- 172.21.15.86:0/401073570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b006f5a0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f46b0072fc0 tx=0x7f469c01c520 comp rx=0 tx=0).stop 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 -- 172.21.15.86:0/401073570 shutdown_connections 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 --2- 172.21.15.86:0/401073570 >> v2:172.21.15.196:3300/0 conn(0x7f46b0071da0 0x7f46b00721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 --2- 172.21.15.86:0/401073570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b006f5a0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 --2- 172.21.15.86:0/401073570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b015aee0 0x7f46b006de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 -- 172.21.15.86:0/401073570 >> 172.21.15.86:0/401073570 conn(0x7f46b006b660 msgr2=0x7f46b006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 -- 172.21.15.86:0/401073570 shutdown_connections 2024-03-22T04:21:36.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.602+0000 7f46af200640 1 -- 172.21.15.86:0/401073570 wait complete. 2024-03-22T04:21:36.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46af200640 1 Processor -- start 2024-03-22T04:21:36.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46af200640 1 -- start start 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46af200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b0071da0 0x7f46b0150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46af200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46af200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b015aee0 0x7f46b015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46af200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46b015b440 con 0x7f46b00b4780 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46af200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46b015b2c0 con 0x7f46b015aee0 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46af200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46b006fae0 con 0x7f46b0071da0 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46ad400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46ade00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b0071da0 0x7f46b0150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46ad400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36918/0 (socket says 172.21.15.86:36918) 2024-03-22T04:21:36.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46ad400640 1 -- 172.21.15.86:0/3377028789 learned_addr learned my addr 172.21.15.86:0/3377028789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.606+0000 7f46ae800640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b015aee0 0x7f46b015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46ad400640 1 -- 172.21.15.86:0/3377028789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b0071da0 msgr2=0x7f46b0150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46ad400640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b0071da0 0x7f46b0150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46ad400640 1 -- 172.21.15.86:0/3377028789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b015aee0 msgr2=0x7f46b015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46ad400640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b015aee0 0x7f46b015bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46ad400640 1 -- 172.21.15.86:0/3377028789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b015d4b0 con 0x7f46b00b4780 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46ad400640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b015b6a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f469c01abd0 tx=0x7f469c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46a6000640 1 -- 172.21.15.86:0/3377028789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0bd5f0 con 0x7f46b00b4780 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46a6000640 1 -- 172.21.15.86:0/3377028789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f469c0bd790 con 0x7f46b00b4780 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b015c410 con 0x7f46b00b4780 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46a6000640 1 -- 172.21.15.86:0/3377028789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0bdb80 con 0x7f46b00b4780 2024-03-22T04:21:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b015c9d0 con 0x7f46b00b4780 2024-03-22T04:21:36.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b0074470 con 0x7f46b00b4780 2024-03-22T04:21:36.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.610+0000 7f46a6000640 1 -- 172.21.15.86:0/3377028789 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f469c0b33f0 con 0x7f46b00b4780 2024-03-22T04:21:36.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.614+0000 7f46a6000640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4690067b20 0x7f4690069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.614+0000 7f46a6000640 1 -- 172.21.15.86:0/3377028789 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f469c120ed0 con 0x7f46b00b4780 2024-03-22T04:21:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.614+0000 7f46ade00640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4690067b20 0x7f4690069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.614+0000 7f46ade00640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4690067b20 0x7f4690069fe0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009950 tx=0x7f46a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.618+0000 7f46a6000640 1 -- 172.21.15.86:0/3377028789 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f469c0c3060 con 0x7f46b00b4780 2024-03-22T04:21:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.802+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46b00b0cb0 con 0x7f46b00b4780 2024-03-22T04:21:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46a6000640 1 -- 172.21.15.86:0/3377028789 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5194 (secure 0 0 0) 0x7f469c016110 con 0x7f46b00b4780 2024-03-22T04:21:36.802 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:36.803 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:28.964789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3342090879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3342090879},{"type":"v1","addr":"172.21.15.86:6837","nonce":3342090879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9124,"qdb_cluster":[9124]},"id":8}]} 2024-03-22T04:21:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-03-22T04:21:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4690067b20 msgr2=0x7f4690069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4690067b20 0x7f4690069fe0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009950 tx=0x7f46a0008040 comp rx=0 tx=0).stop 2024-03-22T04:21:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 msgr2=0x7f46b015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b015b6a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f469c01abd0 tx=0x7f469c01ad60 comp rx=0 tx=0).stop 2024-03-22T04:21:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 shutdown_connections 2024-03-22T04:21:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4690067b20 0x7f4690069fe0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b015aee0 0x7f46b015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b00b4780 0x7f46b015b6a0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 --2- 172.21.15.86:0/3377028789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46b0071da0 0x7f46b0150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 >> 172.21.15.86:0/3377028789 conn(0x7f46b006b660 msgr2=0x7f46b00b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 shutdown_connections 2024-03-22T04:21:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.806+0000 7f46af200640 1 -- 172.21.15.86:0/3377028789 wait complete. 2024-03-22T04:21:36.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77e1a00640 1 Processor -- start 2024-03-22T04:21:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77e1a00640 1 -- start start 2024-03-22T04:21:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77e1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77dc0d0de0 0x7f77dc0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77e1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d19a0 0x7f77dc0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77e1a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f77dc0d1dc0 0x7f77dc15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77e1a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f77dc0d11c0 con 0x7f77dc0d0de0 2024-03-22T04:21:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77e1a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f77dc0cece0 con 0x7f77dc0d1dc0 2024-03-22T04:21:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77e1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f77dc0cefc0 con 0x7f77dc0d19a0 2024-03-22T04:21:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77daa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77dc0d0de0 0x7f77dc0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77daa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77dc0d0de0 0x7f77dc0ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36920/0 (socket says 172.21.15.86:36920) 2024-03-22T04:21:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.962+0000 7f77daa00640 1 -- 172.21.15.86:0/2997366460 learned_addr learned my addr 172.21.15.86:0/2997366460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77db400640 1 --2- 172.21.15.86:0/2997366460 >> v2:172.21.15.196:3300/0 conn(0x7f77dc0d1dc0 0x7f77dc15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77da000640 1 --2- 172.21.15.86:0/2997366460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d19a0 0x7f77dc0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77daa00640 1 -- 172.21.15.86:0/2997366460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d19a0 msgr2=0x7f77dc0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77daa00640 1 --2- 172.21.15.86:0/2997366460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d19a0 0x7f77dc0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77daa00640 1 -- 172.21.15.86:0/2997366460 >> v2:172.21.15.196:3300/0 conn(0x7f77dc0d1dc0 msgr2=0x7f77dc15fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77daa00640 1 --2- 172.21.15.86:0/2997366460 >> v2:172.21.15.196:3300/0 conn(0x7f77dc0d1dc0 0x7f77dc15fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77daa00640 1 -- 172.21.15.86:0/2997366460 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc0cf2a0 con 0x7f77dc0d0de0 2024-03-22T04:21:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77da000640 1 --2- 172.21.15.86:0/2997366460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d19a0 0x7f77dc0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:36.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77daa00640 1 --2- 172.21.15.86:0/2997366460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc0ce6b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0b3e00 tx=0x7f77cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbb689334b3280b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77d9600640 1 -- 172.21.15.86:0/2997366460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0b0070 con 0x7f77dc0d0de0 2024-03-22T04:21:36.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77d9600640 1 -- 172.21.15.86:0/2997366460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc0ab600 con 0x7f77dc0d0de0 2024-03-22T04:21:36.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77d9600640 1 -- 172.21.15.86:0/2997366460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0ab9d0 con 0x7f77dc0d0de0 2024-03-22T04:21:36.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77e1a00640 1 -- 172.21.15.86:0/2997366460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 msgr2=0x7f77dc0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2997366460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc0ce6b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0b3e00 tx=0x7f77cc01c520 comp rx=0 tx=0).stop 2024-03-22T04:21:36.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77e1a00640 1 -- 172.21.15.86:0/2997366460 shutdown_connections 2024-03-22T04:21:36.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2997366460 >> v2:172.21.15.196:3300/0 conn(0x7f77dc0d1dc0 0x7f77dc15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2997366460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d19a0 0x7f77dc0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2997366460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc0ce6b0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.966+0000 7f77e1a00640 1 -- 172.21.15.86:0/2997366460 >> 172.21.15.86:0/2997366460 conn(0x7f77dc06f910 msgr2=0x7f77dc06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 -- 172.21.15.86:0/2997366460 shutdown_connections 2024-03-22T04:21:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 -- 172.21.15.86:0/2997366460 wait complete. 2024-03-22T04:21:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 Processor -- start 2024-03-22T04:21:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 -- start start 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77dc0d19a0 0x7f77dc155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d1dc0 0x7f77dc06df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc071fb0 con 0x7f77dc0d0de0 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc071e30 con 0x7f77dc0d19a0 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f77dc15c8f0 con 0x7f77dc0d1dc0 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc06c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36932/0 (socket says 172.21.15.86:36932) 2024-03-22T04:21:36.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 -- 172.21.15.86:0/2649298027 learned_addr learned my addr 172.21.15.86:0/2649298027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77da000640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77dc0d19a0 0x7f77dc155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77db400640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d1dc0 0x7f77dc06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 -- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d1dc0 msgr2=0x7f77dc06df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d1dc0 0x7f77dc06df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 -- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77dc0d19a0 msgr2=0x7f77dc155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77dc0d19a0 0x7f77dc155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 -- 172.21.15.86:0/2649298027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc154a60 con 0x7f77dc0d0de0 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77da000640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77dc0d19a0 0x7f77dc155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77daa00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc06c870 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0ae040 tx=0x7f77cc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77d2a00640 1 -- 172.21.15.86:0/2649298027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0ba070 con 0x7f77dc0d0de0 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77d2a00640 1 -- 172.21.15.86:0/2649298027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc0b4620 con 0x7f77dc0d0de0 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77d2a00640 1 -- 172.21.15.86:0/2649298027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0b0040 con 0x7f77dc0d0de0 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.970+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc06ce60 con 0x7f77dc0d0de0 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.974+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc06d420 con 0x7f77dc0d0de0 2024-03-22T04:21:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.974+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc068650 con 0x7f77dc0d0de0 2024-03-22T04:21:36.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.974+0000 7f77d2a00640 1 -- 172.21.15.86:0/2649298027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f77cc0ab4c0 con 0x7f77dc0d0de0 2024-03-22T04:21:36.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.974+0000 7f77d2a00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f77bc067b20 0x7f77bc069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:36.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.974+0000 7f77d2a00640 1 -- 172.21.15.86:0/2649298027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f77cc11f470 con 0x7f77dc0d0de0 2024-03-22T04:21:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.974+0000 7f77da000640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f77bc067b20 0x7f77bc069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.974+0000 7f77da000640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f77bc067b20 0x7f77bc069fe0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f77c8006fd0 tx=0x7f77c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:36.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:36.978+0000 7f77d2a00640 1 -- 172.21.15.86:0/2649298027 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f77cc016110 con 0x7f77dc0d0de0 2024-03-22T04:21:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.138+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77dc0d1430 con 0x7f77dc0d0de0 2024-03-22T04:21:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.138+0000 7f77d2a00640 1 -- 172.21.15.86:0/2649298027 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+17215 (secure 0 0 0) 0x7f77cc0efba0 con 0x7f77dc0d0de0 2024-03-22T04:21:37.137 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:37.137 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":126,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:21:23.925937+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6835/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6837/3165266972":"2024-03-23T04:21:12.563892+0000","172.21.15.86:6835/3165266972":"2024-03-23T04:21:12.563892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f77bc067b20 msgr2=0x7f77bc069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f77bc067b20 0x7f77bc069fe0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f77c8006fd0 tx=0x7f77c8008040 comp rx=0 tx=0).stop 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 msgr2=0x7f77dc06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc06c870 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0ae040 tx=0x7f77cc0192c0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 shutdown_connections 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f77bc067b20 0x7f77bc069fe0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77dc0d19a0 0x7f77dc155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77dc0d1dc0 0x7f77dc06df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 --2- 172.21.15.86:0/2649298027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77dc0d0de0 0x7f77dc06c870 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 >> 172.21.15.86:0/2649298027 conn(0x7f77dc06f910 msgr2=0x7f77dc0d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 shutdown_connections 2024-03-22T04:21:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.142+0000 7f77e1a00640 1 -- 172.21.15.86:0/2649298027 wait complete. 2024-03-22T04:21:37.158 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:21:37.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:37.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4717e00640 1 Processor -- start 2024-03-22T04:21:37.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4717e00640 1 -- start start 2024-03-22T04:21:37.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4717e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471015e630 0x7f47100cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4717e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f471006e2f0 0x7f47100d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4717e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47101623d0 con 0x7f471006cc80 2024-03-22T04:21:37.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4717e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f47101620f0 con 0x7f471006e2f0 2024-03-22T04:21:37.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4717e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f471015ec10 con 0x7f471015e630 2024-03-22T04:21:37.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4715e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f471006cc80 0x7f471006d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40188/0 (socket says 172.21.15.86:40188) 2024-03-22T04:21:37.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.298+0000 7f4715e00640 1 -- 172.21.15.86:0/996933916 learned_addr learned my addr 172.21.15.86:0/996933916 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715400640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471015e630 0x7f47100cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4714a00640 1 --2- 172.21.15.86:0/996933916 >> v2:172.21.15.196:3300/0 conn(0x7f471006e2f0 0x7f47100d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3779147650 0 0) 0x7f47101623d0 con 0x7f471006cc80 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46f8003530 con 0x7f471006cc80 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2221206369 0 0) 0x7f46f8003530 con 0x7f471006cc80 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471015e630 msgr2=0x7f47100cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471015e630 0x7f47100cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 >> v2:172.21.15.196:3300/0 conn(0x7f471006e2f0 msgr2=0x7f47100d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 --2- 172.21.15.86:0/996933916 >> v2:172.21.15.196:3300/0 conn(0x7f471006e2f0 0x7f47100d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471015eef0 con 0x7f471006cc80 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f4708001a00 con 0x7f471006cc80 2024-03-22T04:21:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 >> v1:172.21.15.86:6789/0 conn(0x7f471006cc80 legacy=0x7f471006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f46f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471015e630 0x7f46f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f8004d50 0x7f46f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47101620f0 con 0x7f46f80039c0 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f471015ec10 con 0x7f46f8004d50 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46f800b9d0 con 0x7f471015e630 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715400640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f46f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715e00640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f8004d50 0x7f46f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4714a00640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471015e630 0x7f46f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715400640 1 -- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471015e630 msgr2=0x7f46f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715400640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f471015e630 0x7f46f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715400640 1 -- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f8004d50 msgr2=0x7f46f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715400640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f8004d50 0x7f46f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715400640 1 -- 172.21.15.86:0/996933916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f8003530 con 0x7f46f80039c0 2024-03-22T04:21:37.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.302+0000 7f4715e00640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46f8004d50 0x7f46f80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:37.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4715400640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f46f8003de0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f4704009ae0 tx=0x7f470401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9c6b1e700b5728 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:37.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040096b0 con 0x7f46f80039c0 2024-03-22T04:21:37.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47040a6450 con 0x7f46f80039c0 2024-03-22T04:21:37.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f470fe00640 1 -- 172.21.15.86:0/996933916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040a6840 con 0x7f46f80039c0 2024-03-22T04:21:37.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- 172.21.15.86:0/996933916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 msgr2=0x7f46f8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 --2- 172.21.15.86:0/996933916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f46f8003de0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f4704009ae0 tx=0x7f470401ca70 comp rx=0 tx=0).stop 2024-03-22T04:21:37.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4715e00640 1 -- 172.21.15.86:0/996933916 reap_dead start 2024-03-22T04:21:37.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- 172.21.15.86:0/996933916 shutdown_connections 2024-03-22T04:21:37.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- 172.21.15.86:0/996933916 >> 172.21.15.86:0/996933916 conn(0x7f471006f910 msgr2=0x7f471006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:37.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- 172.21.15.86:0/996933916 shutdown_connections 2024-03-22T04:21:37.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- 172.21.15.86:0/996933916 wait complete. 2024-03-22T04:21:37.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 Processor -- start 2024-03-22T04:21:37.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- start start 2024-03-22T04:21:37.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f47101560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47100c8020 0x7f47100cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47100cc8a0 0x7f47101fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47100cfaf0 con 0x7f46f80039c0 2024-03-22T04:21:37.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f471015f9d0 con 0x7f47100c8020 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.306+0000 7f4717e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f47100cfc70 con 0x7f47100cc8a0 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f47101560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f47101560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36944/0 (socket says 172.21.15.86:36944) 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 -- 172.21.15.86:0/537478584 learned_addr learned my addr 172.21.15.86:0/537478584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715e00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47100cc8a0 0x7f47101fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4714a00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47100c8020 0x7f47100cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 -- 172.21.15.86:0/537478584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47100cc8a0 msgr2=0x7f47101fb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47100cc8a0 0x7f47101fb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 -- 172.21.15.86:0/537478584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47100c8020 msgr2=0x7f47100cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47100c8020 0x7f47100cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 -- 172.21.15.86:0/537478584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47101f81c0 con 0x7f46f80039c0 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4715400640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f47101560f0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4704009ae0 tx=0x7f46f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f470d600640 1 -- 172.21.15.86:0/537478584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040be070 con 0x7f46f80039c0 2024-03-22T04:21:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f470d600640 1 -- 172.21.15.86:0/537478584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470400f000 con 0x7f46f80039c0 2024-03-22T04:21:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47100b60d0 con 0x7f46f80039c0 2024-03-22T04:21:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f470d600640 1 -- 172.21.15.86:0/537478584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040a7430 con 0x7f46f80039c0 2024-03-22T04:21:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47100c8520 con 0x7f46f80039c0 2024-03-22T04:21:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4710068650 con 0x7f46f80039c0 2024-03-22T04:21:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f470d600640 1 -- 172.21.15.86:0/537478584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f47040c1070 con 0x7f46f80039c0 2024-03-22T04:21:37.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.310+0000 7f470d600640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f0067b20 0x7f46f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.314+0000 7f470d600640 1 -- 172.21.15.86:0/537478584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f470411faf0 con 0x7f46f80039c0 2024-03-22T04:21:37.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.314+0000 7f4714a00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f0067b20 0x7f46f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.314+0000 7f4714a00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f0067b20 0x7f46f0069fe0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f471015db90 tx=0x7f46fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:37.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.322+0000 7f470d600640 1 -- 172.21.15.86:0/537478584 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f47101f81c0 con 0x7f46f80039c0 2024-03-22T04:21:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.506+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47100cf440 con 0x7f46f80039c0 2024-03-22T04:21:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.506+0000 7f470d600640 1 -- 172.21.15.86:0/537478584 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5194 (secure 0 0 0) 0x7f47040ac020 con 0x7f46f80039c0 2024-03-22T04:21:37.506 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:37.506 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:28.964789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3342090879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3342090879},{"type":"v1","addr":"172.21.15.86:6837","nonce":3342090879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9124,"qdb_cluster":[9124]},"id":8}]} 2024-03-22T04:21:37.506 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f0067b20 msgr2=0x7f46f0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f0067b20 0x7f46f0069fe0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f471015db90 tx=0x7f46fc008040 comp rx=0 tx=0).stop 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 msgr2=0x7f47101560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f47101560f0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4704009ae0 tx=0x7f46f8005840 comp rx=0 tx=0).stop 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 shutdown_connections 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f46f0067b20 0x7f46f0069fe0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47100c8020 0x7f47100cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47100cc8a0 0x7f47101fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 --2- 172.21.15.86:0/537478584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f80039c0 0x7f47101560f0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 >> 172.21.15.86:0/537478584 conn(0x7f471006f910 msgr2=0x7f4710162600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:37.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 shutdown_connections 2024-03-22T04:21:37.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.510+0000 7f4717e00640 1 -- 172.21.15.86:0/537478584 wait complete. 2024-03-22T04:21:37.527 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:21:37.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa90be00640 1 Processor -- start 2024-03-22T04:21:37.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa90be00640 1 -- start start 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa90be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad330 0x7fa90415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa90be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa90406ef40 0x7fa90406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa90be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9040ad710 0x7fa90413fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa90be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa90406dc40 con 0x7fa9040ad710 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa90be00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa90415c7c0 con 0x7fa90406ef40 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa90be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa90406dea0 con 0x7fa9040ad330 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9040ad710 0x7fa90413fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9040ad710 0x7fa90413fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36946/0 (socket says 172.21.15.86:36946) 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909e00640 1 -- 172.21.15.86:0/2551349891 learned_addr learned my addr 172.21.15.86:0/2551349891 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa908a00640 1 --2- 172.21.15.86:0/2551349891 >> v2:172.21.15.196:3300/0 conn(0x7fa90406ef40 0x7fa90406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909400640 1 --2- 172.21.15.86:0/2551349891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad330 0x7fa90415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909e00640 1 -- 172.21.15.86:0/2551349891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad330 msgr2=0x7fa90415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909e00640 1 --2- 172.21.15.86:0/2551349891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad330 0x7fa90415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909e00640 1 -- 172.21.15.86:0/2551349891 >> v2:172.21.15.196:3300/0 conn(0x7fa90406ef40 msgr2=0x7fa90406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909e00640 1 --2- 172.21.15.86:0/2551349891 >> v2:172.21.15.196:3300/0 conn(0x7fa90406ef40 0x7fa90406d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.666+0000 7fa909e00640 1 -- 172.21.15.86:0/2551349891 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90406e180 con 0x7fa9040ad710 2024-03-22T04:21:37.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa909e00640 1 --2- 172.21.15.86:0/2551349891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad710 0x7fa90413fbb0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fa90406c590 tx=0x7fa8fc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5306224923529fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa903e00640 1 -- 172.21.15.86:0/2551349891 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0a5840 con 0x7fa9040ad710 2024-03-22T04:21:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa903e00640 1 -- 172.21.15.86:0/2551349891 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8fc0a59e0 con 0x7fa9040ad710 2024-03-22T04:21:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa903e00640 1 -- 172.21.15.86:0/2551349891 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0a5db0 con 0x7fa9040ad710 2024-03-22T04:21:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 -- 172.21.15.86:0/2551349891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad710 msgr2=0x7fa90413fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 --2- 172.21.15.86:0/2551349891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad710 0x7fa90413fbb0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fa90406c590 tx=0x7fa8fc01c9c0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 -- 172.21.15.86:0/2551349891 shutdown_connections 2024-03-22T04:21:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 --2- 172.21.15.86:0/2551349891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad710 0x7fa90413fbb0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 --2- 172.21.15.86:0/2551349891 >> v2:172.21.15.196:3300/0 conn(0x7fa90406ef40 0x7fa90406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 --2- 172.21.15.86:0/2551349891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad330 0x7fa90415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 -- 172.21.15.86:0/2551349891 >> 172.21.15.86:0/2551349891 conn(0x7fa9040b1df0 msgr2=0x7fa9040b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 -- 172.21.15.86:0/2551349891 shutdown_connections 2024-03-22T04:21:37.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.670+0000 7fa90be00640 1 -- 172.21.15.86:0/2551349891 wait complete. 2024-03-22T04:21:37.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 Processor -- start 2024-03-22T04:21:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 -- start start 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90406ef40 0x7fa904099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad330 0x7fa90409a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad710 0x7fa904165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa904141800 con 0x7fa9040ad330 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa904141680 con 0x7fa90406ef40 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa904141980 con 0x7fa9040ad710 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad330 0x7fa90409a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad330 0x7fa90409a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36952/0 (socket says 172.21.15.86:36952) 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 -- 172.21.15.86:0/3004107609 learned_addr learned my addr 172.21.15.86:0/3004107609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa909400640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90406ef40 0x7fa904099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa909e00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad710 0x7fa904165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 -- 172.21.15.86:0/3004107609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad710 msgr2=0x7fa904165300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad710 0x7fa904165300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 -- 172.21.15.86:0/3004107609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90406ef40 msgr2=0x7fa904099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90406ef40 0x7fa904099a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 -- 172.21.15.86:0/3004107609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9041660d0 con 0x7fa9040ad330 2024-03-22T04:21:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa908a00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad330 0x7fa90409a1e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000d830 tx=0x7fa8f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa901600640 1 -- 172.21.15.86:0/3004107609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0018020 con 0x7fa9040ad330 2024-03-22T04:21:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa901600640 1 -- 172.21.15.86:0/3004107609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f00096a0 con 0x7fa9040ad330 2024-03-22T04:21:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90409d870 con 0x7fa9040ad330 2024-03-22T04:21:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa901600640 1 -- 172.21.15.86:0/3004107609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0009a50 con 0x7fa9040ad330 2024-03-22T04:21:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.674+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90409de30 con 0x7fa9040ad330 2024-03-22T04:21:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.678+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa904068650 con 0x7fa9040ad330 2024-03-22T04:21:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.678+0000 7fa901600640 1 -- 172.21.15.86:0/3004107609 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fa8f001b020 con 0x7fa9040ad330 2024-03-22T04:21:37.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.678+0000 7fa901600640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8e4067b20 0x7fa8e4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:37.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.678+0000 7fa901600640 1 -- 172.21.15.86:0/3004107609 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7fa8f0086840 con 0x7fa9040ad330 2024-03-22T04:21:37.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.678+0000 7fa909400640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8e4067b20 0x7fa8e4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:37.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.682+0000 7fa909400640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8e4067b20 0x7fa8e4069fe0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8007a60 tx=0x7fa8f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:37.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.682+0000 7fa901600640 1 -- 172.21.15.86:0/3004107609 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa8f0022020 con 0x7fa9040ad330 2024-03-22T04:21:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.870+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa904072770 con 0x7fa9040ad330 2024-03-22T04:21:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.870+0000 7fa901600640 1 -- 172.21.15.86:0/3004107609 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5194 (secure 0 0 0) 0x7fa8f0056930 con 0x7fa9040ad330 2024-03-22T04:21:37.869 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:37.869 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T04:20:57.561818+0000","modified":"2024-03-22T04:21:28.964789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9124":{"gid":9124,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3342090879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3342090879},{"type":"v1","addr":"172.21.15.86:6837","nonce":3342090879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9124,"qdb_cluster":[9124]},"id":8}]} 2024-03-22T04:21:37.869 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8e4067b20 msgr2=0x7fa8e4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8e4067b20 0x7fa8e4069fe0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8007a60 tx=0x7fa8f8008040 comp rx=0 tx=0).stop 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad330 msgr2=0x7fa90409a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad330 0x7fa90409a1e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000d830 tx=0x7fa8f000dcf0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 shutdown_connections 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fa8e4067b20 0x7fa8e4069fe0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9040ad710 0x7fa904165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9040ad330 0x7fa90409a1e0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 --2- 172.21.15.86:0/3004107609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90406ef40 0x7fa904099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 >> 172.21.15.86:0/3004107609 conn(0x7fa9040b1df0 msgr2=0x7fa9040b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:37.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 shutdown_connections 2024-03-22T04:21:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:37.874+0000 7fa90be00640 1 -- 172.21.15.86:0/3004107609 wait complete. 2024-03-22T04:21:37.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.030+0000 7ff80be00640 1 Processor -- start 2024-03-22T04:21:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.030+0000 7ff80be00640 1 -- start start 2024-03-22T04:21:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff80be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff8040cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff80be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8040d11c0 con 0x7ff8040d0de0 2024-03-22T04:21:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff80be00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8040cece0 con 0x7ff80415fed0 2024-03-22T04:21:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff80be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff8040cefc0 con 0x7ff8040ce6b0 2024-03-22T04:21:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff809400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff8040d0de0 0x7ff8040d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40192/0 (socket says 172.21.15.86:40192) 2024-03-22T04:21:38.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff809400640 1 -- 172.21.15.86:0/1213904905 learned_addr learned my addr 172.21.15.86:0/1213904905 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff808a00640 1 --2- 172.21.15.86:0/1213904905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff8040cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff803e00640 1 -- 172.21.15.86:0/1213904905 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2669643031 0 0) 0x7ff8040d11c0 con 0x7ff8040d0de0 2024-03-22T04:21:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff803e00640 1 -- 172.21.15.86:0/1213904905 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7ec003530 con 0x7ff8040d0de0 2024-03-22T04:21:38.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff803e00640 1 -- 172.21.15.86:0/1213904905 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1293163928 0 0) 0x7ff8040cece0 con 0x7ff80415fed0 2024-03-22T04:21:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff803e00640 1 -- 172.21.15.86:0/1213904905 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8040d11c0 con 0x7ff80415fed0 2024-03-22T04:21:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff808a00640 1 -- 172.21.15.86:0/1213904905 >> v1:172.21.15.196:6789/0 conn(0x7ff80415fed0 legacy=0x7ff804071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff808a00640 1 -- 172.21.15.86:0/1213904905 >> v1:172.21.15.86:6789/0 conn(0x7ff8040d0de0 legacy=0x7ff8040d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.034+0000 7ff808a00640 1 -- 172.21.15.86:0/1213904905 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8040cf2a0 con 0x7ff8040ce6b0 2024-03-22T04:21:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff808a00640 1 --2- 172.21.15.86:0/1213904905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff8040cfe70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00097f0 tx=0x7ff7f001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e72753aa71322e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff803e00640 1 -- 172.21.15.86:0/1213904905 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00ab5f0 con 0x7ff8040ce6b0 2024-03-22T04:21:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff803e00640 1 -- 172.21.15.86:0/1213904905 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f00b0070 con 0x7ff8040ce6b0 2024-03-22T04:21:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff803e00640 1 -- 172.21.15.86:0/1213904905 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00ab9b0 con 0x7ff8040ce6b0 2024-03-22T04:21:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 -- 172.21.15.86:0/1213904905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 msgr2=0x7ff8040cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 --2- 172.21.15.86:0/1213904905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff8040cfe70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00097f0 tx=0x7ff7f001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 -- 172.21.15.86:0/1213904905 shutdown_connections 2024-03-22T04:21:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 --2- 172.21.15.86:0/1213904905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff8040cfe70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 -- 172.21.15.86:0/1213904905 >> 172.21.15.86:0/1213904905 conn(0x7ff80406f910 msgr2=0x7ff80406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 -- 172.21.15.86:0/1213904905 shutdown_connections 2024-03-22T04:21:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 -- 172.21.15.86:0/1213904905 wait complete. 2024-03-22T04:21:38.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 Processor -- start 2024-03-22T04:21:38.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.038+0000 7ff80be00640 1 -- start start 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff804156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8040d0de0 0x7ff804150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80415fed0 0x7ff8040b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8040d0530 con 0x7ff80415fed0 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8040d03b0 con 0x7ff8040d0de0 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff80416a860 con 0x7ff8040ce6b0 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80415fed0 0x7ff8040b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff804156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80415fed0 0x7ff8040b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36964/0 (socket says 172.21.15.86:36964) 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff804156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:60994/0 (socket says 172.21.15.86:60994) 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff808a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8040d0de0 0x7ff804150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 -- 172.21.15.86:0/1218522921 learned_addr learned my addr 172.21.15.86:0/1218522921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 -- 172.21.15.86:0/1218522921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 msgr2=0x7ff804156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff804156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 -- 172.21.15.86:0/1218522921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8040d0de0 msgr2=0x7ff804150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8040d0de0 0x7ff804150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 -- 172.21.15.86:0/1218522921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8041f90e0 con 0x7ff80415fed0 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff809e00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80415fed0 0x7ff8040b15f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c790 tx=0x7ff7fc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff801600640 1 -- 172.21.15.86:0/1218522921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc003b80 con 0x7ff80415fed0 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff801600640 1 -- 172.21.15.86:0/1218522921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7fc003d20 con 0x7ff80415fed0 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8041f93c0 con 0x7ff80415fed0 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff801600640 1 -- 172.21.15.86:0/1218522921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00aa50 con 0x7ff80415fed0 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8041f9980 con 0x7ff80415fed0 2024-03-22T04:21:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.042+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff804068650 con 0x7ff80415fed0 2024-03-22T04:21:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.046+0000 7ff801600640 1 -- 172.21.15.86:0/1218522921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff7fc00abf0 con 0x7ff80415fed0 2024-03-22T04:21:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.046+0000 7ff801600640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff7e4067b50 0x7ff7e406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.046+0000 7ff801600640 1 -- 172.21.15.86:0/1218522921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7ff7fc086070 con 0x7ff80415fed0 2024-03-22T04:21:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.046+0000 7ff809400640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff7e4067b50 0x7ff7e406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.046+0000 7ff809400640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff7e4067b50 0x7ff7e406a010 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8007fd0 tx=0x7ff7f8007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.050+0000 7ff801600640 1 -- 172.21.15.86:0/1218522921 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff7fc0560d0 con 0x7ff80415fed0 2024-03-22T04:21:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.210+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff804153320 con 0x7ff80415fed0 2024-03-22T04:21:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff801600640 1 -- 172.21.15.86:0/1218522921 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+17215 (secure 0 0 0) 0x7ff7fc05a170 con 0x7ff80415fed0 2024-03-22T04:21:38.211 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:38.211 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":126,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:21:23.925937+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-22T04:20:55.270181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-22T04:20:56.532178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6835/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6837/3165266972":"2024-03-23T04:21:12.563892+0000","172.21.15.86:6835/3165266972":"2024-03-23T04:21:12.563892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff7e4067b50 msgr2=0x7ff7e406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff7e4067b50 0x7ff7e406a010 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8007fd0 tx=0x7ff7f8007d80 comp rx=0 tx=0).stop 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80415fed0 msgr2=0x7ff8040b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80415fed0 0x7ff8040b15f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c790 tx=0x7ff7fc00cc50 comp rx=0 tx=0).stop 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 shutdown_connections 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff7e4067b50 0x7ff7e406a010 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8040d0de0 0x7ff804150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80415fed0 0x7ff8040b15f0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 --2- 172.21.15.86:0/1218522921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8040ce6b0 0x7ff804156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 >> 172.21.15.86:0/1218522921 conn(0x7ff80406f910 msgr2=0x7ff8040d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.214+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 shutdown_connections 2024-03-22T04:21:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.218+0000 7ff80be00640 1 -- 172.21.15.86:0/1218522921 wait complete. 2024-03-22T04:21:38.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-22T04:21:38.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9f200640 1 Processor -- start 2024-03-22T04:21:38.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9f200640 1 -- start start 2024-03-22T04:21:38.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9f200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8aa006bd70 0x7f8aa00affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa00b0680 0x7f8aa006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa015ebb0 con 0x7f8aa015e3a0 2024-03-22T04:21:38.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9f200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa00b04e0 con 0x7f8aa006bd70 2024-03-22T04:21:38.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa015ee90 con 0x7f8aa00b0680 2024-03-22T04:21:38.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a95400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8aa006bd70 0x7f8aa00affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9e800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa00b0680 0x7f8aa006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9de00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f8aa015e3a0 0x7f8aa006b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:40136/0 (socket says 172.21.15.86:40136) 2024-03-22T04:21:38.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a9de00640 1 -- 172.21.15.86:0/1812769735 learned_addr learned my addr 172.21.15.86:0/1812769735 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:38.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a95400640 1 -- 172.21.15.86:0/1812769735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa00b0680 msgr2=0x7f8aa006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:38.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a95400640 1 --2- 172.21.15.86:0/1812769735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa00b0680 0x7f8aa006fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a95400640 1 -- 172.21.15.86:0/1812769735 >> v1:172.21.15.196:6789/0 conn(0x7f8aa015e3a0 legacy=0x7f8aa006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.374+0000 7f8a95400640 1 -- 172.21.15.86:0/1812769735 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa015f170 con 0x7f8aa006bd70 2024-03-22T04:21:38.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a95400640 1 --2- 172.21.15.86:0/1812769735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa006bd70 0x7f8aa00affa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009d90 tx=0x7f8a8801c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc79ca046d981508 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9d400640 1 -- 172.21.15.86:0/1812769735 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880b2070 con 0x7f8aa006bd70 2024-03-22T04:21:38.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9d400640 1 -- 172.21.15.86:0/1812769735 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a880a5710 con 0x7f8aa006bd70 2024-03-22T04:21:38.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9d400640 1 -- 172.21.15.86:0/1812769735 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880a5ae0 con 0x7f8aa006bd70 2024-03-22T04:21:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9f200640 1 -- 172.21.15.86:0/1812769735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa006bd70 msgr2=0x7f8aa00affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9f200640 1 --2- 172.21.15.86:0/1812769735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa006bd70 0x7f8aa00affa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009d90 tx=0x7f8a8801c850 comp rx=0 tx=0).stop 2024-03-22T04:21:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9f200640 1 -- 172.21.15.86:0/1812769735 shutdown_connections 2024-03-22T04:21:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9f200640 1 --2- 172.21.15.86:0/1812769735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa00b0680 0x7f8aa006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9f200640 1 --2- 172.21.15.86:0/1812769735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa006bd70 0x7f8aa00affa0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9f200640 1 -- 172.21.15.86:0/1812769735 >> 172.21.15.86:0/1812769735 conn(0x7f8aa00d1000 msgr2=0x7f8aa00d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9f200640 1 -- 172.21.15.86:0/1812769735 shutdown_connections 2024-03-22T04:21:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.378+0000 7f8a9f200640 1 -- 172.21.15.86:0/1812769735 wait complete. 2024-03-22T04:21:38.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 Processor -- start 2024-03-22T04:21:38.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 -- start start 2024-03-22T04:21:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa006bd70 0x7f8aa01512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa00b0680 0x7f8aa0152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa015e3a0 0x7f8aa014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0070aa0 con 0x7f8aa015e3a0 2024-03-22T04:21:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa015e990 con 0x7f8aa00b0680 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa0070c20 con 0x7f8aa006bd70 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa015e3a0 0x7f8aa014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa015e3a0 0x7f8aa014fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36968/0 (socket says 172.21.15.86:36968) 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 -- 172.21.15.86:0/3034961411 learned_addr learned my addr 172.21.15.86:0/3034961411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9de00640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa006bd70 0x7f8aa01512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a95400640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa00b0680 0x7f8aa0152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 -- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa006bd70 msgr2=0x7f8aa01512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa006bd70 0x7f8aa01512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 -- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa00b0680 msgr2=0x7f8aa0152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa00b0680 0x7f8aa0152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 -- 172.21.15.86:0/3034961411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa0151bd0 con 0x7f8aa015e3a0 2024-03-22T04:21:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a95400640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa00b0680 0x7f8aa0152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9e800640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa015e3a0 0x7f8aa014fca0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800e7f0 tx=0x7f8a9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a96a00640 1 -- 172.21.15.86:0/3034961411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98017070 con 0x7f8aa015e3a0 2024-03-22T04:21:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a96a00640 1 -- 172.21.15.86:0/3034961411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9800c940 con 0x7f8aa015e3a0 2024-03-22T04:21:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a96a00640 1 -- 172.21.15.86:0/3034961411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800eea0 con 0x7f8aa015e3a0 2024-03-22T04:21:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa01518f0 con 0x7f8aa015e3a0 2024-03-22T04:21:38.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.382+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa015bd20 con 0x7f8aa015e3a0 2024-03-22T04:21:38.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.386+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa0074470 con 0x7f8aa015e3a0 2024-03-22T04:21:38.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.386+0000 7f8a96a00640 1 -- 172.21.15.86:0/3034961411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f8a9801a070 con 0x7f8aa015e3a0 2024-03-22T04:21:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.386+0000 7f8a96a00640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a7c067b20 0x7f8a7c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.386+0000 7f8a96a00640 1 -- 172.21.15.86:0/3034961411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6273+0+0 (secure 0 0 0) 0x7f8a98085ff0 con 0x7f8aa015e3a0 2024-03-22T04:21:38.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.386+0000 7f8a9de00640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a7c067b20 0x7f8a7c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.386+0000 7f8a9de00640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a7c067b20 0x7f8a7c069fe0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8a90000f70 tx=0x7f8a90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.390+0000 7f8a96a00640 1 -- 172.21.15.86:0/3034961411 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8a98056810 con 0x7f8aa015e3a0 2024-03-22T04:21:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.630+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8aa015d9b0 con 0x7f8aa015e3a0 2024-03-22T04:21:38.630 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-03-22T04:21:38.634+0000 7f3dcd200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T04:21:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.634+0000 7f8a96a00640 1 -- 172.21.15.86:0/3034961411 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7f8a98056110 con 0x7f8aa015e3a0 2024-03-22T04:21:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-22T04:21:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a7c067b20 msgr2=0x7f8a7c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a7c067b20 0x7f8a7c069fe0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8a90000f70 tx=0x7f8a90009040 comp rx=0 tx=0).stop 2024-03-22T04:21:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa015e3a0 msgr2=0x7f8aa014fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa015e3a0 0x7f8aa014fca0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800e7f0 tx=0x7f8a9800ecb0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 shutdown_connections 2024-03-22T04:21:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8a7c067b20 0x7f8a7c069fe0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa00b0680 0x7f8aa0152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa015e3a0 0x7f8aa014fca0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 --2- 172.21.15.86:0/3034961411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8aa006bd70 0x7f8aa01512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 >> 172.21.15.86:0/3034961411 conn(0x7f8aa00d1000 msgr2=0x7f8aa00d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 shutdown_connections 2024-03-22T04:21:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.638+0000 7f8a9f200640 1 -- 172.21.15.86:0/3034961411 wait complete. 2024-03-22T04:21:38.641 INFO:tasks.ceph.mds.a.smithi086.stderr: -1204> 2024-03-22T04:21:29.934+0000 7f8705400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-22T04:21:38.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-22T04:21:38.684 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-03-22T04:21:38.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f854fc00640 1 Processor -- start 2024-03-22T04:21:38.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f854fc00640 1 -- start start 2024-03-22T04:21:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f854fc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480b2660 0x7f854806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f854fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8548162fe0 con 0x7f85480b4780 2024-03-22T04:21:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f854fc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f854806ccb0 con 0x7f85480c50d0 2024-03-22T04:21:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f854fc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f85480b39c0 con 0x7f85480b2660 2024-03-22T04:21:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f853fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f85480b4780 0x7f85480d0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40202/0 (socket says 172.21.15.86:40202) 2024-03-22T04:21:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f853fe00640 1 -- 172.21.15.86:0/1656718758 learned_addr learned my addr 172.21.15.86:0/1656718758 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.798+0000 7f854d200640 1 --2- 172.21.15.86:0/1656718758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480b2660 0x7f854806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f8547e00640 1 -- 172.21.15.86:0/1656718758 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136047905 0 0) 0x7f8548162fe0 con 0x7f85480b4780 2024-03-22T04:21:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f8547e00640 1 -- 172.21.15.86:0/1656718758 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f852c003530 con 0x7f85480b4780 2024-03-22T04:21:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f854d200640 1 -- 172.21.15.86:0/1656718758 >> v1:172.21.15.196:6789/0 conn(0x7f85480c50d0 legacy=0x7f85480cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f8547e00640 1 -- 172.21.15.86:0/1656718758 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172722073 0 0) 0x7f854806ccb0 con 0x7f85480c50d0 2024-03-22T04:21:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f854d200640 1 -- 172.21.15.86:0/1656718758 >> v1:172.21.15.86:6789/0 conn(0x7f85480b4780 legacy=0x7f85480d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f854d200640 1 -- 172.21.15.86:0/1656718758 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85480b3400 con 0x7f85480b2660 2024-03-22T04:21:38.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f854d200640 1 --2- 172.21.15.86:0/1656718758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480b2660 0x7f854806c770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f85380097f0 tx=0x7f853801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5068d5c863b4be89 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f8547e00640 1 -- 172.21.15.86:0/1656718758 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380b0070 con 0x7f85480b2660 2024-03-22T04:21:38.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f8547e00640 1 -- 172.21.15.86:0/1656718758 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85380ab600 con 0x7f85480b2660 2024-03-22T04:21:38.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.802+0000 7f8547e00640 1 -- 172.21.15.86:0/1656718758 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380ab9d0 con 0x7f85480b2660 2024-03-22T04:21:38.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- 172.21.15.86:0/1656718758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480b2660 msgr2=0x7f854806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 --2- 172.21.15.86:0/1656718758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480b2660 0x7f854806c770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f85380097f0 tx=0x7f853801c4e0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- 172.21.15.86:0/1656718758 shutdown_connections 2024-03-22T04:21:38.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 --2- 172.21.15.86:0/1656718758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480b2660 0x7f854806c770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- 172.21.15.86:0/1656718758 >> 172.21.15.86:0/1656718758 conn(0x7f85480b3bf0 msgr2=0x7f85480b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:38.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- 172.21.15.86:0/1656718758 shutdown_connections 2024-03-22T04:21:38.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- 172.21.15.86:0/1656718758 wait complete. 2024-03-22T04:21:38.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 Processor -- start 2024-03-22T04:21:38.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- start start 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85480b2660 0x7f854815c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85480b4780 0x7f8548157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480c50d0 0x7f85481653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8548170f40 con 0x7f85480b4780 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85480c5920 con 0x7f85480b2660 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854fc00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f85481710c0 con 0x7f85480c50d0 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85480b4780 0x7f8548157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85480b4780 0x7f8548157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36992/0 (socket says 172.21.15.86:36992) 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 -- 172.21.15.86:0/1413751449 learned_addr learned my addr 172.21.15.86:0/1413751449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854d200640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85480b2660 0x7f854815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f854dc00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480c50d0 0x7f85481653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 -- 172.21.15.86:0/1413751449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480c50d0 msgr2=0x7f85481653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480c50d0 0x7f85481653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 -- 172.21.15.86:0/1413751449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85480b2660 msgr2=0x7f854815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85480b2660 0x7f854815c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 -- 172.21.15.86:0/1413751449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85480cbf20 con 0x7f85480b4780 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f853fe00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85480b4780 0x7f8548157280 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8530001ec0 tx=0x7f853000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.806+0000 7f8545600640 1 -- 172.21.15.86:0/1413751449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530017070 con 0x7f85480b4780 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f8545600640 1 -- 172.21.15.86:0/1413751449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8530005d50 con 0x7f85480b4780 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8548157c80 con 0x7f85480b4780 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f8545600640 1 -- 172.21.15.86:0/1413751449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530007270 con 0x7f85480b4780 2024-03-22T04:21:38.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8548158520 con 0x7f85480b4780 2024-03-22T04:21:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8548157e50 con 0x7f85480b4780 2024-03-22T04:21:38.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f8545600640 1 -- 172.21.15.86:0/1413751449 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f853001a070 con 0x7f85480b4780 2024-03-22T04:21:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f8545600640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8524067b50 0x7f852406a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f8545600640 1 -- 172.21.15.86:0/1413751449 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6359+0+0 (secure 0 0 0) 0x7f8530085470 con 0x7f85480b4780 2024-03-22T04:21:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f854d200640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8524067b50 0x7f852406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:38.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.810+0000 7f854d200640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8524067b50 0x7f852406a010 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f85380ad000 tx=0x7f85380a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:38.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:38.814+0000 7f8545600640 1 -- 172.21.15.86:0/1413751449 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8530055d40 con 0x7f85480b4780 2024-03-22T04:21:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.006+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8548074470 con 0x7f85480b4780 2024-03-22T04:21:39.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f8545600640 1 -- 172.21.15.86:0/1413751449 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) ==== 104+0+0 (secure 0 0 0) 0x7f8530055640 con 0x7f85480b4780 2024-03-22T04:21:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8524067b50 msgr2=0x7f852406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8524067b50 0x7f852406a010 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f85380ad000 tx=0x7f85380a6000 comp rx=0 tx=0).stop 2024-03-22T04:21:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85480b4780 msgr2=0x7f8548157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85480b4780 0x7f8548157280 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8530001ec0 tx=0x7f853000c960 comp rx=0 tx=0).stop 2024-03-22T04:21:39.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 shutdown_connections 2024-03-22T04:21:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f8524067b50 0x7f852406a010 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85480c50d0 0x7f85481653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85480b4780 0x7f8548157280 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 --2- 172.21.15.86:0/1413751449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85480b2660 0x7f854815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 >> 172.21.15.86:0/1413751449 conn(0x7f85480b3bf0 msgr2=0x7f8548070110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 shutdown_connections 2024-03-22T04:21:39.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.638+0000 7f854fc00640 1 -- 172.21.15.86:0/1413751449 wait complete. 2024-03-22T04:21:39.661 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-22T04:21:39.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ef800640 1 Processor -- start 2024-03-22T04:21:39.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ef800640 1 -- start start 2024-03-22T04:21:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ef800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f41e815e6d0 0x7f41e806f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ef800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41e8071df0 0x7f41e8072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ef800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e80cef30 con 0x7f41e80afef0 2024-03-22T04:21:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ef800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41e8072780 con 0x7f41e8071df0 2024-03-22T04:21:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ef800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f41e80cf210 con 0x7f41e815e6d0 2024-03-22T04:21:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ed800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41e8071df0 0x7f41e8072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ed800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41e8071df0 0x7f41e8072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37008/0 (socket says 172.21.15.86:37008) 2024-03-22T04:21:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ed800640 1 -- 172.21.15.86:0/788532720 learned_addr learned my addr 172.21.15.86:0/788532720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.850+0000 7f41ece00640 1 --2- 172.21.15.86:0/788532720 >> v2:172.21.15.196:3300/0 conn(0x7f41e815e6d0 0x7f41e806f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:39.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ed800640 1 -- 172.21.15.86:0/788532720 >> v2:172.21.15.196:3300/0 conn(0x7f41e815e6d0 msgr2=0x7f41e806f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:39.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ed800640 1 --2- 172.21.15.86:0/788532720 >> v2:172.21.15.196:3300/0 conn(0x7f41e815e6d0 0x7f41e806f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ed800640 1 -- 172.21.15.86:0/788532720 >> v1:172.21.15.196:6789/0 conn(0x7f41e80afef0 legacy=0x7f41e80ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:39.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ed800640 1 -- 172.21.15.86:0/788532720 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e806f080 con 0x7f41e8071df0 2024-03-22T04:21:39.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ed800640 1 --2- 172.21.15.86:0/788532720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e8071df0 0x7f41e8072240 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f41e8075b50 tx=0x7f41e001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de95f6fb94bfcda7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41e7400640 1 -- 172.21.15.86:0/788532720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e00b0070 con 0x7f41e8071df0 2024-03-22T04:21:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41e7400640 1 -- 172.21.15.86:0/788532720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e00ab600 con 0x7f41e8071df0 2024-03-22T04:21:39.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41e7400640 1 -- 172.21.15.86:0/788532720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e00ab9d0 con 0x7f41e8071df0 2024-03-22T04:21:39.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ef800640 1 -- 172.21.15.86:0/788532720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e8071df0 msgr2=0x7f41e8072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:39.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ef800640 1 --2- 172.21.15.86:0/788532720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e8071df0 0x7f41e8072240 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f41e8075b50 tx=0x7f41e001c450 comp rx=0 tx=0).stop 2024-03-22T04:21:39.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ef800640 1 -- 172.21.15.86:0/788532720 shutdown_connections 2024-03-22T04:21:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ef800640 1 --2- 172.21.15.86:0/788532720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e8071df0 0x7f41e8072240 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ef800640 1 --2- 172.21.15.86:0/788532720 >> v2:172.21.15.196:3300/0 conn(0x7f41e815e6d0 0x7f41e806f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ef800640 1 -- 172.21.15.86:0/788532720 >> 172.21.15.86:0/788532720 conn(0x7f41e80d07e0 msgr2=0x7f41e80d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ef800640 1 -- 172.21.15.86:0/788532720 shutdown_connections 2024-03-22T04:21:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.854+0000 7f41ef800640 1 -- 172.21.15.86:0/788532720 wait complete. 2024-03-22T04:21:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 Processor -- start 2024-03-22T04:21:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 -- start start 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e8071df0 0x7f41e8151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41e80afef0 0x7f41e8151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e815e6d0 0x7f41e81f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41e806c200 con 0x7f41e815e6d0 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41e806c080 con 0x7f41e8071df0 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f41e8150960 con 0x7f41e80afef0 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e815e6d0 0x7f41e81f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e815e6d0 0x7f41e81f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37016/0 (socket says 172.21.15.86:37016) 2024-03-22T04:21:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 -- 172.21.15.86:0/432332696 learned_addr learned my addr 172.21.15.86:0/432332696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ece00640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e8071df0 0x7f41e8151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41e7e00640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41e80afef0 0x7f41e8151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 -- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41e80afef0 msgr2=0x7f41e8151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41e80afef0 0x7f41e8151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 -- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e8071df0 msgr2=0x7f41e8151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e8071df0 0x7f41e8151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 -- 172.21.15.86:0/432332696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e81f8470 con 0x7f41e815e6d0 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41e7e00640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41e80afef0 0x7f41e8151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:39.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ed800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e815e6d0 0x7f41e81f7940 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f41e00ae040 tx=0x7f41e00191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:39.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41e4c00640 1 -- 172.21.15.86:0/432332696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e00b8020 con 0x7f41e815e6d0 2024-03-22T04:21:39.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41e4c00640 1 -- 172.21.15.86:0/432332696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e00b4630 con 0x7f41e815e6d0 2024-03-22T04:21:39.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41e4c00640 1 -- 172.21.15.86:0/432332696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e00b0040 con 0x7f41e815e6d0 2024-03-22T04:21:39.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e81f8750 con 0x7f41e815e6d0 2024-03-22T04:21:39.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.858+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e8150e70 con 0x7f41e815e6d0 2024-03-22T04:21:39.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.862+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e8074580 con 0x7f41e815e6d0 2024-03-22T04:21:39.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.862+0000 7f41e4c00640 1 -- 172.21.15.86:0/432332696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f41e00bb020 con 0x7f41e815e6d0 2024-03-22T04:21:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.862+0000 7f41e4c00640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41c8067b20 0x7f41c8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.862+0000 7f41e4c00640 1 -- 172.21.15.86:0/432332696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6359+0+0 (secure 0 0 0) 0x7f41e011fa20 con 0x7f41e815e6d0 2024-03-22T04:21:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.862+0000 7f41ece00640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41c8067b20 0x7f41c8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.862+0000 7f41ece00640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41c8067b20 0x7f41c8069fe0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f41e8153460 tx=0x7f41d80098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:39.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:39.866+0000 7f41e4c00640 1 -- 172.21.15.86:0/432332696 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f41e00c2070 con 0x7f41e815e6d0 2024-03-22T04:21:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:40.030+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f41e80d16c0 con 0x7f41e815e6d0 2024-03-22T04:21:41.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.014+0000 7f41e4c00640 1 -- 172.21.15.86:0/432332696 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) ==== 181+0+0 (secure 0 0 0) 0x7f41e00160d0 con 0x7f41e815e6d0 2024-03-22T04:21:41.014 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-22T04:21:41.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41c8067b20 msgr2=0x7f41c8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:41.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41c8067b20 0x7f41c8069fe0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f41e8153460 tx=0x7f41d80098d0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e815e6d0 msgr2=0x7f41e81f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:41.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e815e6d0 0x7f41e81f7940 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f41e00ae040 tx=0x7f41e00191f0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 shutdown_connections 2024-03-22T04:21:41.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f41c8067b20 0x7f41c8069fe0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e815e6d0 0x7f41e81f7940 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41e80afef0 0x7f41e8151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 --2- 172.21.15.86:0/432332696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41e8071df0 0x7f41e8151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 >> 172.21.15.86:0/432332696 conn(0x7f41e80d07e0 msgr2=0x7f41e80b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:41.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 shutdown_connections 2024-03-22T04:21:41.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.018+0000 7f41ef800640 1 -- 172.21.15.86:0/432332696 wait complete. 2024-03-22T04:21:41.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-22T04:21:41.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.190+0000 7fdf8de00640 1 Processor -- start 2024-03-22T04:21:41.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.190+0000 7fdf8de00640 1 -- start start 2024-03-22T04:21:41.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf8de00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880b1cd0 0x7fdf880b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:41.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf8de00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdf8806de70 0x7fdf8813f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:41.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf8de00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf880b25f0 con 0x7fdf880cfa70 2024-03-22T04:21:41.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf8de00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdf880cea80 con 0x7fdf8806de70 2024-03-22T04:21:41.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf8de00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8806e4d0 con 0x7fdf880b1cd0 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf86a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880b1cd0 0x7fdf880b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf87400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdf8806de70 0x7fdf8813f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf86000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fdf880cfa70 0x7fdf8806d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:40178/0 (socket says 172.21.15.86:40178) 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf86a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880b1cd0 0x7fdf880b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.86:32842/0 (socket says 172.21.15.86:32842) 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf86000640 1 -- 172.21.15.86:0/5175833 learned_addr learned my addr 172.21.15.86:0/5175833 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf87400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdf8806de70 0x7fdf8813f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:58660/0 (socket says 172.21.15.86:58660) 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf87400640 1 -- 172.21.15.86:0/5175833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880b1cd0 msgr2=0x7fdf880b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf87400640 1 --2- 172.21.15.86:0/5175833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880b1cd0 0x7fdf880b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf87400640 1 -- 172.21.15.86:0/5175833 >> v1:172.21.15.196:6789/0 conn(0x7fdf880cfa70 legacy=0x7fdf8806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf87400640 1 -- 172.21.15.86:0/5175833 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf880cecb0 con 0x7fdf8806de70 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf85600640 1 -- 172.21.15.86:0/5175833 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2181748097 0 0) 0x7fdf880b25f0 con 0x7fdf880cfa70 2024-03-22T04:21:41.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.194+0000 7fdf86a00640 1 --2- 172.21.15.86:0/5175833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880b1cd0 0x7fdf880b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:41.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf87400640 1 --2- 172.21.15.86:0/5175833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf8806de70 0x7fdf8813f8d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009d90 tx=0x7fdf8001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeaa3d359a8bd8e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:41.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf85600640 1 -- 172.21.15.86:0/5175833 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf800b2070 con 0x7fdf8806de70 2024-03-22T04:21:41.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf85600640 1 -- 172.21.15.86:0/5175833 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf800a5920 con 0x7fdf8806de70 2024-03-22T04:21:41.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf85600640 1 -- 172.21.15.86:0/5175833 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf800a5cf0 con 0x7fdf8806de70 2024-03-22T04:21:41.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 -- 172.21.15.86:0/5175833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf8806de70 msgr2=0x7fdf8813f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:41.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 --2- 172.21.15.86:0/5175833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf8806de70 0x7fdf8813f8d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009d90 tx=0x7fdf8001c970 comp rx=0 tx=0).stop 2024-03-22T04:21:41.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 -- 172.21.15.86:0/5175833 shutdown_connections 2024-03-22T04:21:41.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 --2- 172.21.15.86:0/5175833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf8806de70 0x7fdf8813f8d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 --2- 172.21.15.86:0/5175833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880b1cd0 0x7fdf880b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 -- 172.21.15.86:0/5175833 >> 172.21.15.86:0/5175833 conn(0x7fdf8806b660 msgr2=0x7fdf8806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:41.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 -- 172.21.15.86:0/5175833 shutdown_connections 2024-03-22T04:21:41.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 -- 172.21.15.86:0/5175833 wait complete. 2024-03-22T04:21:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 Processor -- start 2024-03-22T04:21:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.198+0000 7fdf8de00640 1 -- start start 2024-03-22T04:21:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf8806de70 0x7fdf880b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf880b1cd0 0x7fdf88156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880cfa70 0x7fdf88138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8813f5e0 con 0x7fdf8806de70 2024-03-22T04:21:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8813f460 con 0x7fdf880b1cd0 2024-03-22T04:21:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf88168d80 con 0x7fdf880cfa70 2024-03-22T04:21:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf8806de70 0x7fdf880b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf8806de70 0x7fdf880b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37032/0 (socket says 172.21.15.86:37032) 2024-03-22T04:21:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 -- 172.21.15.86:0/3854319924 learned_addr learned my addr 172.21.15.86:0/3854319924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 -- 172.21.15.86:0/3854319924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880cfa70 msgr2=0x7fdf88138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880cfa70 0x7fdf88138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 -- 172.21.15.86:0/3854319924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf880b1cd0 msgr2=0x7fdf88156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86000640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf880b1cd0 0x7fdf88156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf880b1cd0 0x7fdf88156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 -- 172.21.15.86:0/3854319924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf88157320 con 0x7fdf8806de70 2024-03-22T04:21:41.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86000640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf880b1cd0 0x7fdf88156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:41.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf86a00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf8806de70 0x7fdf880b4b80 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800ebe0 tx=0x7fdf7800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:41.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf7f400640 1 -- 172.21.15.86:0/3854319924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78017070 con 0x7fdf8806de70 2024-03-22T04:21:41.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf7f400640 1 -- 172.21.15.86:0/3854319924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7800cc30 con 0x7fdf8806de70 2024-03-22T04:21:41.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf880b2fd0 con 0x7fdf8806de70 2024-03-22T04:21:41.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf7f400640 1 -- 172.21.15.86:0/3854319924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78013470 con 0x7fdf8806de70 2024-03-22T04:21:41.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf880b3590 con 0x7fdf8806de70 2024-03-22T04:21:41.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf88074580 con 0x7fdf8806de70 2024-03-22T04:21:41.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.202+0000 7fdf7f400640 1 -- 172.21.15.86:0/3854319924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fdf7801a070 con 0x7fdf8806de70 2024-03-22T04:21:41.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.206+0000 7fdf7f400640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf68067b20 0x7fdf68069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:41.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.206+0000 7fdf7f400640 1 -- 172.21.15.86:0/3854319924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5921+0+0 (secure 0 0 0) 0x7fdf78085fe0 con 0x7fdf8806de70 2024-03-22T04:21:41.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.206+0000 7fdf86000640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf68067b20 0x7fdf68069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:41.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.206+0000 7fdf86000640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf68067b20 0x7fdf68069fe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdf74007fd0 tx=0x7fdf74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:41.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.210+0000 7fdf7f400640 1 -- 172.21.15.86:0/3854319924 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdf88157320 con 0x7fdf8806de70 2024-03-22T04:21:41.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:41.362+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf8806c670 con 0x7fdf8806de70 2024-03-22T04:21:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.026+0000 7fdf7f400640 1 -- 172.21.15.86:0/3854319924 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v129) ==== 169+0+0 (secure 0 0 0) 0x7fdf78056890 con 0x7fdf8806de70 2024-03-22T04:21:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-22T04:21:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.026+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf68067b20 msgr2=0x7fdf68069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.026+0000 7fdf8de00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf68067b20 0x7fdf68069fe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdf74007fd0 tx=0x7fdf74009040 comp rx=0 tx=0).stop 2024-03-22T04:21:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf8806de70 msgr2=0x7fdf880b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf8806de70 0x7fdf880b4b80 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800ebe0 tx=0x7fdf7800c520 comp rx=0 tx=0).stop 2024-03-22T04:21:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 shutdown_connections 2024-03-22T04:21:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fdf68067b20 0x7fdf68069fe0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf880b1cd0 0x7fdf88156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf880cfa70 0x7fdf88138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 --2- 172.21.15.86:0/3854319924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf8806de70 0x7fdf880b4b80 secure :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800ebe0 tx=0x7fdf7800c520 comp rx=0 tx=0).stop 2024-03-22T04:21:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 >> 172.21.15.86:0/3854319924 conn(0x7fdf8806b660 msgr2=0x7fdf8813e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 shutdown_connections 2024-03-22T04:21:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.030+0000 7fdf8de00640 1 -- 172.21.15.86:0/3854319924 wait complete. 2024-03-22T04:21:42.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-22T04:21:42.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.190+0000 7f4687800640 1 Processor -- start 2024-03-22T04:21:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.190+0000 7f4687800640 1 -- start start 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.190+0000 7f4687800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f468006e7e0 0x7f468006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.190+0000 7f4687800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4680072650 0x7f4680072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.190+0000 7f4687800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46800b1b80 0x7f46800b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.190+0000 7f4687800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4680071350 con 0x7f468006e7e0 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4687800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f468006f100 con 0x7f4680072650 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4687800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4680073150 con 0x7f46800b1b80 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f468006e7e0 0x7f468006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f468006e7e0 0x7f468006ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37042/0 (socket says 172.21.15.86:37042) 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 -- 172.21.15.86:0/2081667630 learned_addr learned my addr 172.21.15.86:0/2081667630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4677e00640 1 --2- 172.21.15.86:0/2081667630 >> v2:172.21.15.196:3300/0 conn(0x7f4680072650 0x7f4680072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4685800640 1 --2- 172.21.15.86:0/2081667630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46800b1b80 0x7f46800b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 -- 172.21.15.86:0/2081667630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46800b1b80 msgr2=0x7f46800b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 --2- 172.21.15.86:0/2081667630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46800b1b80 0x7f46800b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 -- 172.21.15.86:0/2081667630 >> v2:172.21.15.196:3300/0 conn(0x7f4680072650 msgr2=0x7f4680072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 --2- 172.21.15.86:0/2081667630 >> v2:172.21.15.196:3300/0 conn(0x7f4680072650 0x7f4680072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 -- 172.21.15.86:0/2081667630 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46800715a0 con 0x7f468006e7e0 2024-03-22T04:21:42.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f4684e00640 1 --2- 172.21.15.86:0/2081667630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468006ebc0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f4680073a00 tx=0x7f467001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cb95b57c7778ac server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:42.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f467fe00640 1 -- 172.21.15.86:0/2081667630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700b4070 con 0x7f468006e7e0 2024-03-22T04:21:42.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f467fe00640 1 -- 172.21.15.86:0/2081667630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46700a7ac0 con 0x7f468006e7e0 2024-03-22T04:21:42.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.194+0000 7f467fe00640 1 -- 172.21.15.86:0/2081667630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700af440 con 0x7f468006e7e0 2024-03-22T04:21:42.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- 172.21.15.86:0/2081667630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 msgr2=0x7f468006ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 --2- 172.21.15.86:0/2081667630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468006ebc0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f4680073a00 tx=0x7f467001eb30 comp rx=0 tx=0).stop 2024-03-22T04:21:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- 172.21.15.86:0/2081667630 shutdown_connections 2024-03-22T04:21:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 --2- 172.21.15.86:0/2081667630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46800b1b80 0x7f46800b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 --2- 172.21.15.86:0/2081667630 >> v2:172.21.15.196:3300/0 conn(0x7f4680072650 0x7f4680072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 --2- 172.21.15.86:0/2081667630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468006ebc0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- 172.21.15.86:0/2081667630 >> 172.21.15.86:0/2081667630 conn(0x7f46800cc940 msgr2=0x7f46800ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- 172.21.15.86:0/2081667630 shutdown_connections 2024-03-22T04:21:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- 172.21.15.86:0/2081667630 wait complete. 2024-03-22T04:21:42.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 Processor -- start 2024-03-22T04:21:42.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- start start 2024-03-22T04:21:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4680072650 0x7f468014e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46800b1b80 0x7f468014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4680072080 con 0x7f468006e7e0 2024-03-22T04:21:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46800ce930 con 0x7f46800b1b80 2024-03-22T04:21:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4687800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f468014ebc0 con 0x7f4680072650 2024-03-22T04:21:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4684e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4684e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468014efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37050/0 (socket says 172.21.15.86:37050) 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4684e00640 1 -- 172.21.15.86:0/3685225776 learned_addr learned my addr 172.21.15.86:0/3685225776 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4677e00640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4680072650 0x7f468014e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4685800640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46800b1b80 0x7f468014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4684e00640 1 -- 172.21.15.86:0/3685225776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4680072650 msgr2=0x7f468014e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4684e00640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4680072650 0x7f468014e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4684e00640 1 -- 172.21.15.86:0/3685225776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46800b1b80 msgr2=0x7f468014c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4684e00640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46800b1b80 0x7f468014c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.198+0000 7f4684e00640 1 -- 172.21.15.86:0/3685225776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4680159620 con 0x7f468006e7e0 2024-03-22T04:21:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f4684e00640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468014efd0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f467000b2c0 tx=0x7f467001b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f467d600640 1 -- 172.21.15.86:0/3685225776 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700bf070 con 0x7f468006e7e0 2024-03-22T04:21:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f467d600640 1 -- 172.21.15.86:0/3685225776 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46700b6050 con 0x7f468006e7e0 2024-03-22T04:21:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f467d600640 1 -- 172.21.15.86:0/3685225776 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700b4030 con 0x7f468006e7e0 2024-03-22T04:21:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4680159900 con 0x7f468006e7e0 2024-03-22T04:21:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4680159ec0 con 0x7f468006e7e0 2024-03-22T04:21:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4680074470 con 0x7f468006e7e0 2024-03-22T04:21:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f467d600640 1 -- 172.21.15.86:0/3685225776 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f46700c2070 con 0x7f468006e7e0 2024-03-22T04:21:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f467d600640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f465c067b20 0x7f465c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f467d600640 1 -- 172.21.15.86:0/3685225776 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f46701200a0 con 0x7f468006e7e0 2024-03-22T04:21:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.202+0000 7f4677e00640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f465c067b20 0x7f465c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.206+0000 7f4677e00640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f465c067b20 0x7f465c069fe0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f468015e740 tx=0x7f4668008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.206+0000 7f467d600640 1 -- 172.21.15.86:0/3685225776 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f46700187e0 con 0x7f468006e7e0 2024-03-22T04:21:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.362+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f46800cc720 con 0x7f468006e7e0 2024-03-22T04:21:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f467d600640 1 -- 172.21.15.86:0/3685225776 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f46700f0ad0 con 0x7f468006e7e0 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f465c067b20 msgr2=0x7f465c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f465c067b20 0x7f465c069fe0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f468015e740 tx=0x7f4668008040 comp rx=0 tx=0).stop 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 msgr2=0x7f468014efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468014efd0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f467000b2c0 tx=0x7f467001b8d0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 shutdown_connections 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f465c067b20 0x7f465c069fe0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46800b1b80 0x7f468014c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4680072650 0x7f468014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 --2- 172.21.15.86:0/3685225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f468006e7e0 0x7f468014efd0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 >> 172.21.15.86:0/3685225776 conn(0x7f46800cc940 msgr2=0x7f46800cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 shutdown_connections 2024-03-22T04:21:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.366+0000 7f4687800640 1 -- 172.21.15.86:0/3685225776 wait complete. 2024-03-22T04:21:42.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-22T04:21:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f4224a00640 1 Processor -- start 2024-03-22T04:21:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f4224a00640 1 -- start start 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f4224a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f422015b3f0 0x7f42200729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f4224a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f42200cc3e0 0x7f4220072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f4224a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f42200b30f0 0x7f42200ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f4224a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f42200716d0 con 0x7f42200cc3e0 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f4224a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f422006d990 con 0x7f42200b30f0 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f4224a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4220071920 con 0x7f422015b3f0 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f42200cc3e0 0x7f4220072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f42200cc3e0 0x7f4220072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37054/0 (socket says 172.21.15.86:37054) 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421d400640 1 -- 172.21.15.86:0/60793118 learned_addr learned my addr 172.21.15.86:0/60793118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421de00640 1 --2- 172.21.15.86:0/60793118 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f422015b3f0 0x7f42200729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421e800640 1 --2- 172.21.15.86:0/60793118 >> v2:172.21.15.196:3300/0 conn(0x7f42200b30f0 0x7f42200ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421d400640 1 -- 172.21.15.86:0/60793118 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f422015b3f0 msgr2=0x7f42200729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421d400640 1 --2- 172.21.15.86:0/60793118 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f422015b3f0 0x7f42200729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421d400640 1 -- 172.21.15.86:0/60793118 >> v2:172.21.15.196:3300/0 conn(0x7f42200b30f0 msgr2=0x7f42200ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421d400640 1 --2- 172.21.15.86:0/60793118 >> v2:172.21.15.196:3300/0 conn(0x7f42200b30f0 0x7f42200ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421d400640 1 -- 172.21.15.86:0/60793118 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4220071c00 con 0x7f42200cc3e0 2024-03-22T04:21:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.522+0000 7f421e800640 1 --2- 172.21.15.86:0/60793118 >> v2:172.21.15.196:3300/0 conn(0x7f42200b30f0 0x7f42200ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f421d400640 1 --2- 172.21.15.86:0/60793118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f4220072f00 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f422006f130 tx=0x7f420c01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7016a89dff0d6ec server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f421ca00640 1 -- 172.21.15.86:0/60793118 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0b4070 con 0x7f42200cc3e0 2024-03-22T04:21:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f421ca00640 1 -- 172.21.15.86:0/60793118 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c0a79f0 con 0x7f42200cc3e0 2024-03-22T04:21:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f421ca00640 1 -- 172.21.15.86:0/60793118 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0a7dc0 con 0x7f42200cc3e0 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 -- 172.21.15.86:0/60793118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 msgr2=0x7f4220072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 --2- 172.21.15.86:0/60793118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f4220072f00 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f422006f130 tx=0x7f420c01eb60 comp rx=0 tx=0).stop 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 -- 172.21.15.86:0/60793118 shutdown_connections 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 --2- 172.21.15.86:0/60793118 >> v2:172.21.15.196:3300/0 conn(0x7f42200b30f0 0x7f42200ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 --2- 172.21.15.86:0/60793118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f4220072f00 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 --2- 172.21.15.86:0/60793118 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f422015b3f0 0x7f42200729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 -- 172.21.15.86:0/60793118 >> 172.21.15.86:0/60793118 conn(0x7f42200b1df0 msgr2=0x7f42200b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 -- 172.21.15.86:0/60793118 shutdown_connections 2024-03-22T04:21:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.526+0000 7f4224a00640 1 -- 172.21.15.86:0/60793118 wait complete. 2024-03-22T04:21:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 Processor -- start 2024-03-22T04:21:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 -- start start 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42200b30f0 0x7f42200999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f42201566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f422015b3f0 0x7f4220156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f422015f1e0 con 0x7f42200cc3e0 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f422015f060 con 0x7f422015b3f0 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f422015f360 con 0x7f42200b30f0 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f42201566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f42201566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37062/0 (socket says 172.21.15.86:37062) 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 -- 172.21.15.86:0/1298278285 learned_addr learned my addr 172.21.15.86:0/1298278285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421de00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42200b30f0 0x7f42200999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421e800640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f422015b3f0 0x7f4220156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 -- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42200b30f0 msgr2=0x7f42200999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42200b30f0 0x7f42200999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 -- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f422015b3f0 msgr2=0x7f4220156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f422015b3f0 0x7f4220156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 -- 172.21.15.86:0/1298278285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4220099ef0 con 0x7f42200cc3e0 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421de00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42200b30f0 0x7f42200999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f421d400640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f42201566a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f420c00b2c0 tx=0x7f420c01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4216000640 1 -- 172.21.15.86:0/1298278285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0bf070 con 0x7f42200cc3e0 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4216000640 1 -- 172.21.15.86:0/1298278285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c0afa50 con 0x7f42200cc3e0 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422014f580 con 0x7f42200cc3e0 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4216000640 1 -- 172.21.15.86:0/1298278285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0b4070 con 0x7f42200cc3e0 2024-03-22T04:21:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.530+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422014fb40 con 0x7f42200cc3e0 2024-03-22T04:21:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.534+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4220068650 con 0x7f42200cc3e0 2024-03-22T04:21:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.534+0000 7f4216000640 1 -- 172.21.15.86:0/1298278285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f420c0c2070 con 0x7f42200cc3e0 2024-03-22T04:21:42.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.534+0000 7f4216000640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4200067b20 0x7f4200069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:42.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.534+0000 7f4216000640 1 -- 172.21.15.86:0/1298278285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f420c1201a0 con 0x7f42200cc3e0 2024-03-22T04:21:42.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.534+0000 7f421de00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4200067b20 0x7f4200069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:42.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.534+0000 7f421de00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4200067b20 0x7f4200069fe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4210009b10 tx=0x7f4210008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.538+0000 7f4216000640 1 -- 172.21.15.86:0/1298278285 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f420c0187e0 con 0x7f42200cc3e0 2024-03-22T04:21:42.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:42.762+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f4220072c50 con 0x7f42200cc3e0 2024-03-22T04:21:43.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.026+0000 7f4216000640 1 -- 172.21.15.86:0/1298278285 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v548) ==== 144+0+0 (secure 0 0 0) 0x7f420c0f0bd0 con 0x7f42200cc3e0 2024-03-22T04:21:43.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.026+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4200067b20 msgr2=0x7f4200069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:43.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.026+0000 7f4224a00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4200067b20 0x7f4200069fe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4210009b10 tx=0x7f4210008040 comp rx=0 tx=0).stop 2024-03-22T04:21:43.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.026+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 msgr2=0x7f42201566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:43.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.026+0000 7f4224a00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f42201566a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f420c00b2c0 tx=0x7f420c01d3a0 comp rx=0 tx=0).stop 2024-03-22T04:21:43.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.026+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 shutdown_connections 2024-03-22T04:21:43.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.030+0000 7f4224a00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f4200067b20 0x7f4200069fe0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:43.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.030+0000 7f4224a00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f422015b3f0 0x7f4220156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:43.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.030+0000 7f4224a00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42200cc3e0 0x7f42201566a0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:43.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.030+0000 7f4224a00640 1 --2- 172.21.15.86:0/1298278285 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42200b30f0 0x7f42200999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:43.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.030+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 >> 172.21.15.86:0/1298278285 conn(0x7f42200b1df0 msgr2=0x7f42200b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:43.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.030+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 shutdown_connections 2024-03-22T04:21:43.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:43.030+0000 7f4224a00640 1 -- 172.21.15.86:0/1298278285 wait complete. 2024-03-22T04:21:43.047 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-22T04:21:43.048 INFO:tasks.cephfs_test_runner: 2024-03-22T04:21:43.048 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-22T04:21:43.048 INFO:tasks.cephfs_test_runner:Ran 7 tests in 618.416s 2024-03-22T04:21:43.048 INFO:tasks.cephfs_test_runner: 2024-03-22T04:21:43.049 INFO:tasks.cephfs_test_runner:OK 2024-03-22T04:21:43.049 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-22T04:21:43.059 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-22T04:21:43.067 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-22T04:21:43.067 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:43.067 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:43.076 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:21:43.077 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-22T04:21:43.077 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:43.125 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354616k,nr_inodes=4088654,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8583 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-22T04:21:43.125 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:21:43.125 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:43.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:21:43.136 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-22T04:21:43.136 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T04:21:43.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354576k,nr_inodes=4088644,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9413 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-22T04:21:43.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.188 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:21:43.197 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T04:21:43.197 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-22T04:21:43.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.197 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T04:21:43.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.258 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-22T04:21:43.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-22T04:21:43.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.504 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-22T04:21:43.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-22T04:21:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-03-22T04:21:43.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:21:43.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.587 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-03-22T04:21:43.597 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-22T04:21:43.597 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-22T04:21:43.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T04:21:43.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.659 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-22T04:21:43.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.754 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-03-22T04:21:43.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.893 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-03-22T04:21:43.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T04:21:43.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-03-22T04:21:43.971 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-03-22T04:21:43.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:21:43.973 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-22T04:21:43.984 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-22T04:21:43.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:21:44.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.126+0000 7f9409a00640 1 Processor -- start 2024-03-22T04:21:44.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.126+0000 7f9409a00640 1 -- start start 2024-03-22T04:21:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.126+0000 7f9409a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f940415b3f0 0x7f94040729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.126+0000 7f9409a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f94040cc3e0 0x7f9404072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.126+0000 7f9409a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94040b30f0 0x7f94040ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.126+0000 7f9409a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f94040716d0 con 0x7f94040b30f0 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.126+0000 7f9409a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f940406d990 con 0x7f94040cc3e0 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.126+0000 7f9409a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9404071920 con 0x7f940415b3f0 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94040b30f0 0x7f94040ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94040b30f0 0x7f94040ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37072/0 (socket says 172.21.15.86:37072) 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 -- 172.21.15.86:0/2751105232 learned_addr learned my addr 172.21.15.86:0/2751105232 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9402a00640 1 --2- 172.21.15.86:0/2751105232 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f940415b3f0 0x7f94040729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9402000640 1 --2- 172.21.15.86:0/2751105232 >> v2:172.21.15.196:3300/0 conn(0x7f94040cc3e0 0x7f9404072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 -- 172.21.15.86:0/2751105232 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f940415b3f0 msgr2=0x7f94040729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 --2- 172.21.15.86:0/2751105232 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f940415b3f0 0x7f94040729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 -- 172.21.15.86:0/2751105232 >> v2:172.21.15.196:3300/0 conn(0x7f94040cc3e0 msgr2=0x7f9404072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 --2- 172.21.15.86:0/2751105232 >> v2:172.21.15.196:3300/0 conn(0x7f94040cc3e0 0x7f9404072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 -- 172.21.15.86:0/2751105232 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9404071c00 con 0x7f94040b30f0 2024-03-22T04:21:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9402000640 1 --2- 172.21.15.86:0/2751105232 >> v2:172.21.15.196:3300/0 conn(0x7f94040cc3e0 0x7f9404072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:44.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9403400640 1 --2- 172.21.15.86:0/2751105232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040b30f0 0x7f94040ade30 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f940406d080 tx=0x7f93fc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5766cf00e305e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9401600640 1 -- 172.21.15.86:0/2751105232 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0a55e0 con 0x7f94040b30f0 2024-03-22T04:21:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9401600640 1 -- 172.21.15.86:0/2751105232 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc0a5780 con 0x7f94040b30f0 2024-03-22T04:21:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.130+0000 7f9401600640 1 -- 172.21.15.86:0/2751105232 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0a5b50 con 0x7f94040b30f0 2024-03-22T04:21:44.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- 172.21.15.86:0/2751105232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040b30f0 msgr2=0x7f94040ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 --2- 172.21.15.86:0/2751105232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040b30f0 0x7f94040ade30 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f940406d080 tx=0x7f93fc01c960 comp rx=0 tx=0).stop 2024-03-22T04:21:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- 172.21.15.86:0/2751105232 shutdown_connections 2024-03-22T04:21:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 --2- 172.21.15.86:0/2751105232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040b30f0 0x7f94040ade30 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 --2- 172.21.15.86:0/2751105232 >> v2:172.21.15.196:3300/0 conn(0x7f94040cc3e0 0x7f9404072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 --2- 172.21.15.86:0/2751105232 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f940415b3f0 0x7f94040729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- 172.21.15.86:0/2751105232 >> 172.21.15.86:0/2751105232 conn(0x7f94040b1df0 msgr2=0x7f94040b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- 172.21.15.86:0/2751105232 shutdown_connections 2024-03-22T04:21:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- 172.21.15.86:0/2751105232 wait complete. 2024-03-22T04:21:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 Processor -- start 2024-03-22T04:21:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- start start 2024-03-22T04:21:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94040b30f0 0x7f9404152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040cc3e0 0x7f94040b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f940415b3f0 0x7f94040b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f940415f1e0 con 0x7f94040cc3e0 2024-03-22T04:21:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f940415f060 con 0x7f940415b3f0 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9409a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f940415f360 con 0x7f94040b30f0 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040cc3e0 0x7f94040b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040cc3e0 0x7f94040b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37078/0 (socket says 172.21.15.86:37078) 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402000640 1 -- 172.21.15.86:0/1372944560 learned_addr learned my addr 172.21.15.86:0/1372944560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94040b30f0 0x7f9404152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9403400640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f940415b3f0 0x7f94040b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402000640 1 -- 172.21.15.86:0/1372944560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94040b30f0 msgr2=0x7f9404152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402000640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94040b30f0 0x7f9404152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402000640 1 -- 172.21.15.86:0/1372944560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f940415b3f0 msgr2=0x7f94040b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402000640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f940415b3f0 0x7f94040b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.134+0000 7f9402000640 1 -- 172.21.15.86:0/1372944560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940409b010 con 0x7f94040cc3e0 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f9402000640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040cc3e0 0x7f94040b08f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e7e0 tx=0x7f93ec00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f93faa00640 1 -- 172.21.15.86:0/1372944560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00f040 con 0x7f94040cc3e0 2024-03-22T04:21:44.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f93faa00640 1 -- 172.21.15.86:0/1372944560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ec004030 con 0x7f94040cc3e0 2024-03-22T04:21:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940409b2f0 con 0x7f94040cc3e0 2024-03-22T04:21:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f93faa00640 1 -- 172.21.15.86:0/1372944560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0043e0 con 0x7f94040cc3e0 2024-03-22T04:21:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94040cdce0 con 0x7f94040cc3e0 2024-03-22T04:21:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9404068650 con 0x7f94040cc3e0 2024-03-22T04:21:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f93faa00640 1 -- 172.21.15.86:0/1372944560 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f93ec00c6d0 con 0x7f94040cc3e0 2024-03-22T04:21:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f93faa00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93e0067b50 0x7f93e006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f93faa00640 1 -- 172.21.15.86:0/1372944560 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f93ec085b70 con 0x7f94040cc3e0 2024-03-22T04:21:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.138+0000 7f9402a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93e0067b50 0x7f93e006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.142+0000 7f9402a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93e0067b50 0x7f93e006a010 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9404154f90 tx=0x7f93f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.142+0000 7f93faa00640 1 -- 172.21.15.86:0/1372944560 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f93ec017020 con 0x7f94040cc3e0 2024-03-22T04:21:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.294+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94040cdb20 con 0x7f93e0067b50 2024-03-22T04:21:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f93faa00640 1 -- 172.21.15.86:0/1372944560 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7f93f40118e0 con 0x7f93e0067b50 2024-03-22T04:21:44.295 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:21:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93e0067b50 msgr2=0x7f93e006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93e0067b50 0x7f93e006a010 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9404154f90 tx=0x7f93f4008040 comp rx=0 tx=0).stop 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040cc3e0 msgr2=0x7f94040b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040cc3e0 0x7f94040b08f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e7e0 tx=0x7f93ec00eca0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 shutdown_connections 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f93e0067b50 0x7f93e006a010 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f940415b3f0 0x7f94040b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94040cc3e0 0x7f94040b08f0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 --2- 172.21.15.86:0/1372944560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94040b30f0 0x7f9404152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 >> 172.21.15.86:0/1372944560 conn(0x7f94040b1df0 msgr2=0x7f9404155d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 shutdown_connections 2024-03-22T04:21:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.298+0000 7f9409a00640 1 -- 172.21.15.86:0/1372944560 wait complete. 2024-03-22T04:21:44.315 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-03-22T04:21:43.945874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":279388,"kb_used_data":29944,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695332,"statfs":{"total":773094113280,"available":772808019968,"internally_reserved":0,"allocated":30662656,"data_stored":23881538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392189,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-176,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":367,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-22T04:21:38.640157+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:21:38.640157+0000","last_peered":"2024-03-22T04:21:38.640157+0000","last_clean":"2024-03-22T04:21:38.640157+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:21:38.640157+0000","last_undegraded":"2024-03-22T04:21:38.640157+0000","last_fullsized":"2024-03-22T04:21:38.640157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509579,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36812,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335028,"statfs":{"total":96636764160,"available":96599068672,"internally_reserved":0,"allocated":5767168,"data_stored":4932277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509578,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":738005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2183168,"data_stored":1331112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509579,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":738070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":5656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":5791744,"data_stored":4935637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":6369280,"data_stored":5522805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5779456,"data_stored":4932963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32752,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:21:44.315 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:21:44.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bfe00640 1 Processor -- start 2024-03-22T04:21:44.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bfe00640 1 -- start start 2024-03-22T04:21:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bfe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f98c015b1e0 0x7f98c0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c00d01b0 0x7f98c00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bfe00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c0072780 con 0x7f98c0073540 2024-03-22T04:21:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bfe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f98c00d06c0 con 0x7f98c015b1e0 2024-03-22T04:21:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bfe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f98c00730b0 con 0x7f98c00d01b0 2024-03-22T04:21:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f98c015b1e0 0x7f98c0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98be000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c00d01b0 0x7f98c00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bf400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f98c0073540 0x7f98c00b4070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:40194/0 (socket says 172.21.15.86:40194) 2024-03-22T04:21:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bf400640 1 -- 172.21.15.86:0/3663229808 learned_addr learned my addr 172.21.15.86:0/3663229808 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bd600640 1 -- 172.21.15.86:0/3663229808 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325657277 0 0) 0x7f98c0072780 con 0x7f98c0073540 2024-03-22T04:21:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bd600640 1 -- 172.21.15.86:0/3663229808 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98a4005740 con 0x7f98c0073540 2024-03-22T04:21:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98be000640 1 -- 172.21.15.86:0/3663229808 >> v2:172.21.15.196:3300/0 conn(0x7f98c015b1e0 msgr2=0x7f98c0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98be000640 1 --2- 172.21.15.86:0/3663229808 >> v2:172.21.15.196:3300/0 conn(0x7f98c015b1e0 0x7f98c0070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98be000640 1 -- 172.21.15.86:0/3663229808 >> v1:172.21.15.196:6789/0 conn(0x7f98c0073540 legacy=0x7f98c00b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98be000640 1 -- 172.21.15.86:0/3663229808 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c0072a60 con 0x7f98c00d01b0 2024-03-22T04:21:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.454+0000 7f98bea00640 1 --2- 172.21.15.86:0/3663229808 >> v2:172.21.15.196:3300/0 conn(0x7f98c015b1e0 0x7f98c0070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:44.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98be000640 1 --2- 172.21.15.86:0/3663229808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c00d01b0 0x7f98c00b5390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0097c0 tx=0x7f98ac01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2f58842bfd7b71 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bd600640 1 -- 172.21.15.86:0/3663229808 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0b0070 con 0x7f98c00d01b0 2024-03-22T04:21:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bd600640 1 -- 172.21.15.86:0/3663229808 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac0ab600 con 0x7f98c00d01b0 2024-03-22T04:21:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bd600640 1 -- 172.21.15.86:0/3663229808 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0ab9d0 con 0x7f98c00d01b0 2024-03-22T04:21:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bfe00640 1 -- 172.21.15.86:0/3663229808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c00d01b0 msgr2=0x7f98c00b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bfe00640 1 --2- 172.21.15.86:0/3663229808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c00d01b0 0x7f98c00b5390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0097c0 tx=0x7f98ac01c380 comp rx=0 tx=0).stop 2024-03-22T04:21:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bfe00640 1 -- 172.21.15.86:0/3663229808 shutdown_connections 2024-03-22T04:21:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bfe00640 1 --2- 172.21.15.86:0/3663229808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c00d01b0 0x7f98c00b5390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bfe00640 1 --2- 172.21.15.86:0/3663229808 >> v2:172.21.15.196:3300/0 conn(0x7f98c015b1e0 0x7f98c0070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bfe00640 1 -- 172.21.15.86:0/3663229808 >> 172.21.15.86:0/3663229808 conn(0x7f98c00d1000 msgr2=0x7f98c00d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bfe00640 1 -- 172.21.15.86:0/3663229808 shutdown_connections 2024-03-22T04:21:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.458+0000 7f98bfe00640 1 -- 172.21.15.86:0/3663229808 wait complete. 2024-03-22T04:21:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 Processor -- start 2024-03-22T04:21:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 -- start start 2024-03-22T04:21:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c0073540 0x7f98c0157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c00d01b0 0x7f98c0155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c015b1e0 0x7f98c0155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c006b7e0 con 0x7f98c0073540 2024-03-22T04:21:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98c006b660 con 0x7f98c00d01b0 2024-03-22T04:21:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f98c006b960 con 0x7f98c015b1e0 2024-03-22T04:21:44.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c0073540 0x7f98c0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c0073540 0x7f98c0157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37090/0 (socket says 172.21.15.86:37090) 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 -- 172.21.15.86:0/2177999123 learned_addr learned my addr 172.21.15.86:0/2177999123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bf400640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c015b1e0 0x7f98c0155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98be000640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c00d01b0 0x7f98c0155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 -- 172.21.15.86:0/2177999123 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c015b1e0 msgr2=0x7f98c0155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c015b1e0 0x7f98c0155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 -- 172.21.15.86:0/2177999123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c00d01b0 msgr2=0x7f98c0155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c00d01b0 0x7f98c0155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 -- 172.21.15.86:0/2177999123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c01565a0 con 0x7f98c0073540 2024-03-22T04:21:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bea00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c0073540 0x7f98c0157640 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f98b000e980 tx=0x7f98b000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98b6a00640 1 -- 172.21.15.86:0/2177999123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0017070 con 0x7f98c0073540 2024-03-22T04:21:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98b6a00640 1 -- 172.21.15.86:0/2177999123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b000c9a0 con 0x7f98c0073540 2024-03-22T04:21:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c0156880 con 0x7f98c0073540 2024-03-22T04:21:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98b6a00640 1 -- 172.21.15.86:0/2177999123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b000cd70 con 0x7f98c0073540 2024-03-22T04:21:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c0156e40 con 0x7f98c0073540 2024-03-22T04:21:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.462+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c0074470 con 0x7f98c0073540 2024-03-22T04:21:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.466+0000 7f98b6a00640 1 -- 172.21.15.86:0/2177999123 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f98b001a070 con 0x7f98c0073540 2024-03-22T04:21:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.466+0000 7f98b6a00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98a0067b20 0x7f98a0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.466+0000 7f98b6a00640 1 -- 172.21.15.86:0/2177999123 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f98b00860c0 con 0x7f98c0073540 2024-03-22T04:21:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.466+0000 7f98be000640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98a0067b20 0x7f98a0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.466+0000 7f98be000640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98a0067b20 0x7f98a0069fe0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0ad000 tx=0x7f98ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.470+0000 7f98b6a00640 1 -- 172.21.15.86:0/2177999123 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f98b0056b90 con 0x7f98c0073540 2024-03-22T04:21:44.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.622+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98c00b5390 con 0x7f98a0067b20 2024-03-22T04:21:44.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.622+0000 7f98b6a00640 1 -- 172.21.15.86:0/2177999123 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7f98c00b5390 con 0x7f98a0067b20 2024-03-22T04:21:44.621 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:44.622 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:21:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98a0067b20 msgr2=0x7f98a0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98a0067b20 0x7f98a0069fe0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0ad000 tx=0x7f98ac008040 comp rx=0 tx=0).stop 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c0073540 msgr2=0x7f98c0157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c0073540 0x7f98c0157640 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f98b000e980 tx=0x7f98b000ee40 comp rx=0 tx=0).stop 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 shutdown_connections 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f98a0067b20 0x7f98a0069fe0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98c00d01b0 0x7f98c0155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98c015b1e0 0x7f98c0155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2177999123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c0073540 0x7f98c0157640 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 >> 172.21.15.86:0/2177999123 conn(0x7f98c00d1000 msgr2=0x7f98c00b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 shutdown_connections 2024-03-22T04:21:44.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.626+0000 7f98bfe00640 1 -- 172.21.15.86:0/2177999123 wait complete. 2024-03-22T04:21:44.642 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-03-22T04:21:43.945874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":279388,"kb_used_data":29944,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695332,"statfs":{"total":773094113280,"available":772808019968,"internally_reserved":0,"allocated":30662656,"data_stored":23881538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392189,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-176,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":367,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-22T04:21:38.640157+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:21:38.640157+0000","last_peered":"2024-03-22T04:21:38.640157+0000","last_clean":"2024-03-22T04:21:38.640157+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:21:38.640157+0000","last_undegraded":"2024-03-22T04:21:38.640157+0000","last_fullsized":"2024-03-22T04:21:38.640157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509579,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36812,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335028,"statfs":{"total":96636764160,"available":96599068672,"internally_reserved":0,"allocated":5767168,"data_stored":4932277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509578,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":738005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2183168,"data_stored":1331112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509579,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":738070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":5656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":5791744,"data_stored":4935637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":6369280,"data_stored":5522805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5779456,"data_stored":4932963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32752,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:21:44.642 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-22T04:21:44.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:21:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.778+0000 7fb3ae600640 1 Processor -- start 2024-03-22T04:21:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.778+0000 7fb3ae600640 1 -- start start 2024-03-22T04:21:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3ae600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3a815b3f0 0x7fb3a80729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3ae600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb3a80cc3e0 0x7fb3a8072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3ae600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 0x7fb3a80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3ae600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a80716d0 con 0x7fb3a815b3f0 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3ae600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a806d990 con 0x7fb3a80cc3e0 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3ae600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a8071920 con 0x7fb3a80b30f0 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3a7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3a815b3f0 0x7fb3a80729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3a7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3a815b3f0 0x7fb3a80729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37094/0 (socket says 172.21.15.86:37094) 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3a7400640 1 -- 172.21.15.86:0/1033748388 learned_addr learned my addr 172.21.15.86:0/1033748388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3a7400640 1 -- 172.21.15.86:0/1033748388 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 msgr2=0x7fb3a80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3a7400640 1 --2- 172.21.15.86:0/1033748388 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 0x7fb3a80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3a7400640 1 -- 172.21.15.86:0/1033748388 >> v2:172.21.15.196:3300/0 conn(0x7fb3a80cc3e0 msgr2=0x7fb3a8072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb39ea00640 1 --2- 172.21.15.86:0/1033748388 >> v2:172.21.15.196:3300/0 conn(0x7fb3a80cc3e0 0x7fb3a8072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3a7400640 1 --2- 172.21.15.86:0/1033748388 >> v2:172.21.15.196:3300/0 conn(0x7fb3a80cc3e0 0x7fb3a8072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb3a7400640 1 -- 172.21.15.86:0/1033748388 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a8071c00 con 0x7fb3a815b3f0 2024-03-22T04:21:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.782+0000 7fb39ea00640 1 --2- 172.21.15.86:0/1033748388 >> v2:172.21.15.196:3300/0 conn(0x7fb3a80cc3e0 0x7fb3a8072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:44.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3a7400640 1 --2- 172.21.15.86:0/1033748388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a80729c0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8070ed0 tx=0x7fb39801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f41393c8952db server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3a6a00640 1 -- 172.21.15.86:0/1033748388 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980b0070 con 0x7fb3a815b3f0 2024-03-22T04:21:44.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3a6a00640 1 -- 172.21.15.86:0/1033748388 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3980ab600 con 0x7fb3a815b3f0 2024-03-22T04:21:44.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3a6a00640 1 -- 172.21.15.86:0/1033748388 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980ab9d0 con 0x7fb3a815b3f0 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 -- 172.21.15.86:0/1033748388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 msgr2=0x7fb3a80729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1033748388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a80729c0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8070ed0 tx=0x7fb39801c490 comp rx=0 tx=0).stop 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 -- 172.21.15.86:0/1033748388 shutdown_connections 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1033748388 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 0x7fb3a80ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1033748388 >> v2:172.21.15.196:3300/0 conn(0x7fb3a80cc3e0 0x7fb3a8072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1033748388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a80729c0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 -- 172.21.15.86:0/1033748388 >> 172.21.15.86:0/1033748388 conn(0x7fb3a80b1df0 msgr2=0x7fb3a80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 -- 172.21.15.86:0/1033748388 shutdown_connections 2024-03-22T04:21:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 -- 172.21.15.86:0/1033748388 wait complete. 2024-03-22T04:21:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 Processor -- start 2024-03-22T04:21:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.786+0000 7fb3ae600640 1 -- start start 2024-03-22T04:21:44.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 0x7fb3a809af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3a80cc3e0 0x7fb3a81520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a81587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a815f1e0 con 0x7fb3a815b3f0 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a815f060 con 0x7fb3a80cc3e0 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a815f360 con 0x7fb3a80b30f0 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a81587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a81587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37102/0 (socket says 172.21.15.86:37102) 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 -- 172.21.15.86:0/1109218812 learned_addr learned my addr 172.21.15.86:0/1109218812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb39ea00640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3a80cc3e0 0x7fb3a81520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7400640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 0x7fb3a809af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 -- 172.21.15.86:0/1109218812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 msgr2=0x7fb3a809af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 0x7fb3a809af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 -- 172.21.15.86:0/1109218812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3a80cc3e0 msgr2=0x7fb3a81520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3a80cc3e0 0x7fb3a81520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 -- 172.21.15.86:0/1109218812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a809ba70 con 0x7fb3a815b3f0 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3a7e00640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a81587e0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000b9a0 tx=0x7fb3a000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb39fe00640 1 -- 172.21.15.86:0/1109218812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0019070 con 0x7fb3a815b3f0 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb39fe00640 1 -- 172.21.15.86:0/1109218812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a000cbc0 con 0x7fb3a815b3f0 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb39fe00640 1 -- 172.21.15.86:0/1109218812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0014440 con 0x7fb3a815b3f0 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a809b790 con 0x7fb3a815b3f0 2024-03-22T04:21:44.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a815d490 con 0x7fb3a815b3f0 2024-03-22T04:21:44.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a8068650 con 0x7fb3a815b3f0 2024-03-22T04:21:44.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.790+0000 7fb39fe00640 1 -- 172.21.15.86:0/1109218812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb3a001c070 con 0x7fb3a815b3f0 2024-03-22T04:21:44.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.794+0000 7fb39fe00640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb384067b20 0x7fb384069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:44.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.794+0000 7fb39fe00640 1 -- 172.21.15.86:0/1109218812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7fb3a0085ad0 con 0x7fb3a815b3f0 2024-03-22T04:21:44.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.794+0000 7fb3a7400640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb384067b20 0x7fb384069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:44.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.794+0000 7fb3a7400640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb384067b20 0x7fb384069fe0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb3980ad000 tx=0x7fb3980a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:44.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.798+0000 7fb39fe00640 1 -- 172.21.15.86:0/1109218812 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb3a0055f50 con 0x7fb3a815b3f0 2024-03-22T04:21:44.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3a809b480 con 0x7fb384067b20 2024-03-22T04:21:44.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb39fe00640 1 -- 172.21.15.86:0/1109218812 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7fb3a809b480 con 0x7fb384067b20 2024-03-22T04:21:44.947 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:44.948 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:21:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb384067b20 msgr2=0x7fb384069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb384067b20 0x7fb384069fe0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb3980ad000 tx=0x7fb3980a6000 comp rx=0 tx=0).stop 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 msgr2=0x7fb3a81587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a81587e0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000b9a0 tx=0x7fb3a000be60 comp rx=0 tx=0).stop 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 shutdown_connections 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb384067b20 0x7fb384069fe0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3a80cc3e0 0x7fb3a81520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3a815b3f0 0x7fb3a81587e0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 --2- 172.21.15.86:0/1109218812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3a80b30f0 0x7fb3a809af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 >> 172.21.15.86:0/1109218812 conn(0x7fb3a80b1df0 msgr2=0x7fb3a80b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 shutdown_connections 2024-03-22T04:21:44.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:44.950+0000 7fb3ae600640 1 -- 172.21.15.86:0/1109218812 wait complete. 2024-03-22T04:21:44.968 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-03-22T04:21:43.945874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":279388,"kb_used_data":29944,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695332,"statfs":{"total":773094113280,"available":772808019968,"internally_reserved":0,"allocated":30662656,"data_stored":23881538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392189,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-176,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":367,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-22T04:21:38.640157+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:21:38.640157+0000","last_peered":"2024-03-22T04:21:38.640157+0000","last_clean":"2024-03-22T04:21:38.640157+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:21:38.640157+0000","last_undegraded":"2024-03-22T04:21:38.640157+0000","last_fullsized":"2024-03-22T04:21:38.640157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509579,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36812,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335028,"statfs":{"total":96636764160,"available":96599068672,"internally_reserved":0,"allocated":5767168,"data_stored":4932277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509578,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":738005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2183168,"data_stored":1331112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509579,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":738070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":5656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":5791744,"data_stored":4935637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":6369280,"data_stored":5522805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5779456,"data_stored":4932963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32752,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:21:44.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T04:21:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd319e00640 1 Processor -- start 2024-03-22T04:21:45.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd319e00640 1 -- start start 2024-03-22T04:21:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd319e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd31415e630 0x7fd3140cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd319e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31406daa0 con 0x7fd3140d03b0 2024-03-22T04:21:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd319e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3141623d0 con 0x7fd31406e2f0 2024-03-22T04:21:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd319e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd3140d0800 con 0x7fd31415e630 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd313400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd3140d03b0 0x7fd31406d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40212/0 (socket says 172.21.15.86:40212) 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd313400640 1 -- 172.21.15.86:0/3643868245 learned_addr learned my addr 172.21.15.86:0/3643868245 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312a00640 1 --2- 172.21.15.86:0/3643868245 >> v2:172.21.15.196:3300/0 conn(0x7fd31415e630 0x7fd3140cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312000640 1 -- 172.21.15.86:0/3643868245 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 863960764 0 0) 0x7fd31406daa0 con 0x7fd3140d03b0 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312000640 1 -- 172.21.15.86:0/3643868245 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2f8003530 con 0x7fd3140d03b0 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312000640 1 -- 172.21.15.86:0/3643868245 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 402602445 0 0) 0x7fd3141623d0 con 0x7fd31406e2f0 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312000640 1 -- 172.21.15.86:0/3643868245 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd31406daa0 con 0x7fd31406e2f0 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312000640 1 -- 172.21.15.86:0/3643868245 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1597761845 0 0) 0x7fd2f8003530 con 0x7fd3140d03b0 2024-03-22T04:21:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312a00640 1 -- 172.21.15.86:0/3643868245 >> v1:172.21.15.196:6789/0 conn(0x7fd31406e2f0 legacy=0x7fd31406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312a00640 1 -- 172.21.15.86:0/3643868245 >> v1:172.21.15.86:6789/0 conn(0x7fd3140d03b0 legacy=0x7fd31406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.110+0000 7fd312a00640 1 -- 172.21.15.86:0/3643868245 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3141620f0 con 0x7fd31415e630 2024-03-22T04:21:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd312a00640 1 --2- 172.21.15.86:0/3643868245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 0x7fd3140cfe70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd304009d90 tx=0x7fd30401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf7201efee289b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd312000640 1 -- 172.21.15.86:0/3643868245 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c001680 con 0x7fd31415e630 2024-03-22T04:21:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd312000640 1 -- 172.21.15.86:0/3643868245 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3040a64e0 con 0x7fd31415e630 2024-03-22T04:21:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd312000640 1 -- 172.21.15.86:0/3643868245 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3040a68b0 con 0x7fd31415e630 2024-03-22T04:21:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd319e00640 1 -- 172.21.15.86:0/3643868245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 msgr2=0x7fd3140cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd319e00640 1 --2- 172.21.15.86:0/3643868245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 0x7fd3140cfe70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd304009d90 tx=0x7fd30401c890 comp rx=0 tx=0).stop 2024-03-22T04:21:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd319e00640 1 -- 172.21.15.86:0/3643868245 shutdown_connections 2024-03-22T04:21:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd319e00640 1 --2- 172.21.15.86:0/3643868245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 0x7fd3140cfe70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd319e00640 1 -- 172.21.15.86:0/3643868245 >> 172.21.15.86:0/3643868245 conn(0x7fd31406f910 msgr2=0x7fd31406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd319e00640 1 -- 172.21.15.86:0/3643868245 shutdown_connections 2024-03-22T04:21:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.114+0000 7fd319e00640 1 -- 172.21.15.86:0/3643868245 wait complete. 2024-03-22T04:21:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 Processor -- start 2024-03-22T04:21:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 -- start start 2024-03-22T04:21:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31406e2f0 0x7fd31415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3140d03b0 0x7fd314153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 0x7fd314159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140d1f60 con 0x7fd31406e2f0 2024-03-22T04:21:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd31415f4b0 con 0x7fd31415e630 2024-03-22T04:21:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd31415f9d0 con 0x7fd3140d03b0 2024-03-22T04:21:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31406e2f0 0x7fd31415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:45.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31406e2f0 0x7fd31415d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37116/0 (socket says 172.21.15.86:37116) 2024-03-22T04:21:45.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 -- 172.21.15.86:0/450520850 learned_addr learned my addr 172.21.15.86:0/450520850 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 -- 172.21.15.86:0/450520850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3140d03b0 msgr2=0x7fd314153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3140d03b0 0x7fd314153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 -- 172.21.15.86:0/450520850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 msgr2=0x7fd314159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd313400640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 0x7fd314159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 0x7fd314159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 -- 172.21.15.86:0/450520850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd314158cc0 con 0x7fd31406e2f0 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd313400640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 0x7fd314159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd312a00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31406e2f0 0x7fd31415d940 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd3040b0040 tx=0x7fd304019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd30be00640 1 -- 172.21.15.86:0/450520850 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3040c2070 con 0x7fd31406e2f0 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd30be00640 1 -- 172.21.15.86:0/450520850 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd304003ba0 con 0x7fd31406e2f0 2024-03-22T04:21:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd314158fa0 con 0x7fd31406e2f0 2024-03-22T04:21:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd30be00640 1 -- 172.21.15.86:0/450520850 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304009450 con 0x7fd31406e2f0 2024-03-22T04:21:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.118+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31415d140 con 0x7fd31406e2f0 2024-03-22T04:21:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.122+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd314068650 con 0x7fd31406e2f0 2024-03-22T04:21:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.122+0000 7fd30be00640 1 -- 172.21.15.86:0/450520850 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd3040c5070 con 0x7fd31406e2f0 2024-03-22T04:21:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.122+0000 7fd30be00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2f0067b20 0x7fd2f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.122+0000 7fd30be00640 1 -- 172.21.15.86:0/450520850 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7fd304120280 con 0x7fd31406e2f0 2024-03-22T04:21:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.122+0000 7fd30a000640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2f0067b20 0x7fd2f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:45.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.122+0000 7fd30a000640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2f0067b20 0x7fd2f0069fe0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc007fd0 tx=0x7fd2fc007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:45.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.126+0000 7fd30be00640 1 -- 172.21.15.86:0/450520850 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd304016480 con 0x7fd31406e2f0 2024-03-22T04:21:45.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.282+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3140cfaf0 con 0x7fd31406e2f0 2024-03-22T04:21:45.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.282+0000 7fd30be00640 1 -- 172.21.15.86:0/450520850 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+13563 (secure 0 0 0) 0x7fd3040f0d50 con 0x7fd31406e2f0 2024-03-22T04:21:45.281 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:45.282 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":129,"fsid":"1365483d-2fb4-4579-a50b-2ce299f625b2","created":"2024-03-22T04:10:08.786532+0000","modified":"2024-03-22T04:21:42.018687+0000","last_up_change":"2024-03-22T04:10:20.563707+0000","last_in_change":"2024-03-22T04:10:11.288822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T04:10:21.815722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8e92443d-5bf1-45e4-81bd-0e22f9ecfae4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6809","nonce":2750248043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6811","nonce":2750248043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6815","nonce":2750248043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2750248043},{"type":"v1","addr":"172.21.15.86:6813","nonce":2750248043}]},"public_addr":"172.21.15.86:6809/2750248043","cluster_addr":"172.21.15.86:6811/2750248043","heartbeat_back_addr":"172.21.15.86:6815/2750248043","heartbeat_front_addr":"172.21.15.86:6813/2750248043","state":["exists","up"]},{"osd":1,"uuid":"3c186e62-fd3b-40fa-93b4-72f33353c135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6801","nonce":3761623609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6803","nonce":3761623609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6807","nonce":3761623609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3761623609},{"type":"v1","addr":"172.21.15.86:6805","nonce":3761623609}]},"public_addr":"172.21.15.86:6801/3761623609","cluster_addr":"172.21.15.86:6803/3761623609","heartbeat_back_addr":"172.21.15.86:6807/3761623609","heartbeat_front_addr":"172.21.15.86:6805/3761623609","state":["exists","up"]},{"osd":2,"uuid":"5d711bf0-fab3-4e06-8190-04a685ccc537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6817","nonce":4132808680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6819","nonce":4132808680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6826","nonce":4132808680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4132808680},{"type":"v1","addr":"172.21.15.86:6821","nonce":4132808680}]},"public_addr":"172.21.15.86:6817/4132808680","cluster_addr":"172.21.15.86:6819/4132808680","heartbeat_back_addr":"172.21.15.86:6826/4132808680","heartbeat_front_addr":"172.21.15.86:6821/4132808680","state":["exists","up"]},{"osd":3,"uuid":"d846ee42-3859-4fac-9cd0-2476e0b5d4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6823","nonce":2837220459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6827","nonce":2837220459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6831","nonce":2837220459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2837220459},{"type":"v1","addr":"172.21.15.86:6829","nonce":2837220459}]},"public_addr":"172.21.15.86:6823/2837220459","cluster_addr":"172.21.15.86:6827/2837220459","heartbeat_back_addr":"172.21.15.86:6831/2837220459","heartbeat_front_addr":"172.21.15.86:6829/2837220459","state":["exists","up"]},{"osd":4,"uuid":"879fbacc-c9f4-4a38-a7db-fa4179109db9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6801","nonce":655314492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6803","nonce":655314492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6807","nonce":655314492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":655314492},{"type":"v1","addr":"172.21.15.196:6805","nonce":655314492}]},"public_addr":"172.21.15.196:6801/655314492","cluster_addr":"172.21.15.196:6803/655314492","heartbeat_back_addr":"172.21.15.196:6807/655314492","heartbeat_front_addr":"172.21.15.196:6805/655314492","state":["exists","up"]},{"osd":5,"uuid":"7afce081-80c5-4cfd-9bb7-3becb644f0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6809","nonce":2743139076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6811","nonce":2743139076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6815","nonce":2743139076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2743139076},{"type":"v1","addr":"172.21.15.196:6813","nonce":2743139076}]},"public_addr":"172.21.15.196:6809/2743139076","cluster_addr":"172.21.15.196:6811/2743139076","heartbeat_back_addr":"172.21.15.196:6815/2743139076","heartbeat_front_addr":"172.21.15.196:6813/2743139076","state":["exists","up"]},{"osd":6,"uuid":"d7750fe3-0854-4f19-b8a4-2ba996534d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6818","nonce":1648471396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6821","nonce":1648471396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6828","nonce":1648471396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":1648471396},{"type":"v1","addr":"172.21.15.196:6824","nonce":1648471396}]},"public_addr":"172.21.15.196:6818/1648471396","cluster_addr":"172.21.15.196:6821/1648471396","heartbeat_back_addr":"172.21.15.196:6828/1648471396","heartbeat_front_addr":"172.21.15.196:6824/1648471396","state":["exists","up"]},{"osd":7,"uuid":"41569ccb-8394-4d80-8731-da57174ad08d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6819","nonce":2847924285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6825","nonce":2847924285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6831","nonce":2847924285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6827","nonce":2847924285},{"type":"v1","addr":"172.21.15.196:6829","nonce":2847924285}]},"public_addr":"172.21.15.196:6819/2847924285","cluster_addr":"172.21.15.196:6825/2847924285","heartbeat_back_addr":"172.21.15.196:6831/2847924285","heartbeat_front_addr":"172.21.15.196:6829/2847924285","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.895479+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.914510+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:17.042281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.967741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.749597+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.858648+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.900036+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T04:10:16.911431+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/3342090879":"2024-03-23T04:21:38.634941+0000","172.21.15.86:6837/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6835/4220897872":"2024-03-23T04:21:23.925914+0000","172.21.15.86:6835/3342090879":"2024-03-23T04:21:38.634941+0000","172.21.15.86:6837/3165266972":"2024-03-23T04:21:12.563892+0000","172.21.15.86:6835/3165266972":"2024-03-23T04:21:12.563892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T04:21:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2f0067b20 msgr2=0x7fd2f0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2f0067b20 0x7fd2f0069fe0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc007fd0 tx=0x7fd2fc007960 comp rx=0 tx=0).stop 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31406e2f0 msgr2=0x7fd31415d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31406e2f0 0x7fd31415d940 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd3040b0040 tx=0x7fd304019630 comp rx=0 tx=0).stop 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 shutdown_connections 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd2f0067b20 0x7fd2f0069fe0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd31415e630 0x7fd314159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3140d03b0 0x7fd314153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 --2- 172.21.15.86:0/450520850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd31406e2f0 0x7fd31415d940 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 >> 172.21.15.86:0/450520850 conn(0x7fd31406f910 msgr2=0x7fd3140b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 shutdown_connections 2024-03-22T04:21:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:45.286+0000 7fd319e00640 1 -- 172.21.15.86:0/450520850 wait complete. 2024-03-22T04:21:46.304 INFO:tasks.ceph:Scrubbing osd.0 2024-03-22T04:21:46.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-22T04:21:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.454+0000 7f6b61000640 1 Processor -- start 2024-03-22T04:21:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.454+0000 7f6b61000640 1 -- start start 2024-03-22T04:21:46.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b61000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6b5c154e40 0x7f6b5c0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b61000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b5c072310 con 0x7f6b5c071e00 2024-03-22T04:21:46.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b61000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b5c0725f0 con 0x7f6b5c06dc40 2024-03-22T04:21:46.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b61000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6b5c0b3200 con 0x7f6b5c154e40 2024-03-22T04:21:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6b5c071e00 0x7f6b5c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45916/0 (socket says 172.21.15.86:45916) 2024-03-22T04:21:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59400640 1 -- 172.21.15.86:0/3554787886 learned_addr learned my addr 172.21.15.86:0/3554787886 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3664397999 0 0) 0x7f6b5c072310 con 0x7f6b5c071e00 2024-03-22T04:21:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59e00640 1 --2- 172.21.15.86:0/3554787886 >> v2:172.21.15.196:3300/0 conn(0x7f6b5c154e40 0x7f6b5c0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b40003530 con 0x7f6b5c071e00 2024-03-22T04:21:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1970291726 0 0) 0x7f6b40003530 con 0x7f6b5c071e00 2024-03-22T04:21:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 >> v2:172.21.15.196:3300/0 conn(0x7f6b5c154e40 msgr2=0x7f6b5c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 --2- 172.21.15.86:0/3554787886 >> v2:172.21.15.196:3300/0 conn(0x7f6b5c154e40 0x7f6b5c0718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 >> v1:172.21.15.196:6789/0 conn(0x7f6b5c06dc40 legacy=0x7f6b5c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c0728d0 con 0x7f6b5c071e00 2024-03-22T04:21:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f6b48001680 con 0x7f6b5c071e00 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 >> v1:172.21.15.86:6789/0 conn(0x7f6b5c071e00 legacy=0x7f6b5c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b400039c0 0x7f6b40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b40004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b400051d0 0x7f6b4000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c0b3200 con 0x7f6b40004810 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4000c170 con 0x7f6b400039c0 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4000c370 con 0x7f6b400051d0 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59400640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b40004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59e00640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b400039c0 0x7f6b40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b5a800640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b400051d0 0x7f6b4000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59400640 1 -- 172.21.15.86:0/3554787886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b400051d0 msgr2=0x7f6b4000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59400640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b400051d0 0x7f6b4000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59400640 1 -- 172.21.15.86:0/3554787886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b400039c0 msgr2=0x7f6b40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59400640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b400039c0 0x7f6b40003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b59400640 1 -- 172.21.15.86:0/3554787886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b40003530 con 0x7f6b40004810 2024-03-22T04:21:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.458+0000 7f6b5a800640 1 -- 172.21.15.86:0/3554787886 reap_dead start 2024-03-22T04:21:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b59400640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b40004c90 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c210 tx=0x7f6b4801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1985123479068b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48007d20 con 0x7f6b40004810 2024-03-22T04:21:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b480a9070 con 0x7f6b40004810 2024-03-22T04:21:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b58a00640 1 -- 172.21.15.86:0/3554787886 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48005da0 con 0x7f6b40004810 2024-03-22T04:21:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b61000640 1 -- 172.21.15.86:0/3554787886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 msgr2=0x7f6b40004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b61000640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b40004c90 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c210 tx=0x7f6b4801cc10 comp rx=0 tx=0).stop 2024-03-22T04:21:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b61000640 1 -- 172.21.15.86:0/3554787886 shutdown_connections 2024-03-22T04:21:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b61000640 1 --2- 172.21.15.86:0/3554787886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b40004c90 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b61000640 1 -- 172.21.15.86:0/3554787886 >> 172.21.15.86:0/3554787886 conn(0x7f6b5c06b660 msgr2=0x7f6b5c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b61000640 1 -- 172.21.15.86:0/3554787886 shutdown_connections 2024-03-22T04:21:46.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.462+0000 7f6b61000640 1 -- 172.21.15.86:0/3554787886 wait complete. 2024-03-22T04:21:46.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 Processor -- start 2024-03-22T04:21:46.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 -- start start 2024-03-22T04:21:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b5c158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5c0b4990 0x7f6b5c143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5c0b3590 0x7f6b5c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c0b2900 con 0x7f6b40004810 2024-03-22T04:21:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c0b2780 con 0x7f6b5c0b3590 2024-03-22T04:21:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6b5c15e4b0 con 0x7f6b5c0b4990 2024-03-22T04:21:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b5c158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b5c158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51236/0 (socket says 172.21.15.86:51236) 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 -- 172.21.15.86:0/1340829746 learned_addr learned my addr 172.21.15.86:0/1340829746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59400640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5c0b4990 0x7f6b5c143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b5a800640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5c0b3590 0x7f6b5c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 -- 172.21.15.86:0/1340829746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5c0b4990 msgr2=0x7f6b5c143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5c0b4990 0x7f6b5c143bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 -- 172.21.15.86:0/1340829746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5c0b3590 msgr2=0x7f6b5c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5c0b3590 0x7f6b5c0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 -- 172.21.15.86:0/1340829746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c1594b0 con 0x7f6b40004810 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b59e00640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b5c158f70 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c011e30 tx=0x7f6b4c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c015630 con 0x7f6b40004810 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4c0157d0 con 0x7f6b40004810 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c159750 con 0x7f6b40004810 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c015ba0 con 0x7f6b40004810 2024-03-22T04:21:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c145760 con 0x7f6b40004810 2024-03-22T04:21:46.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.466+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b5c145b40 con 0x7f6b40004810 2024-03-22T04:21:46.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f6b4c01d400 con 0x7f6b40004810 2024-03-22T04:21:46.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b52000640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b3c067b50 0x7f6b3c06a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f6b4c087920 con 0x7f6b40004810 2024-03-22T04:21:46.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b59400640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b3c067b50 0x7f6b3c06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b52000640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f6b3c06e130 0x7f6b3c070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b4c027e50 con 0x7f6b3c06e130 2024-03-22T04:21:46.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7f6b4c087d10 con 0x7f6b40004810 2024-03-22T04:21:46.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b5a800640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f6b3c06e130 0x7f6b3c070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b59400640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b3c067b50 0x7f6b3c06a010 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6b480ab000 tx=0x7f6b480b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b5a800640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f6b3c06e130 0x7f6b3c070570 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.470+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 <== osd.0 v2:172.21.15.86:6808/2750248043 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b4c027e50 con 0x7f6b3c06e130 2024-03-22T04:21:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.482+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 --> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6b5c074580 con 0x7f6b3c06e130 2024-03-22T04:21:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b52000640 1 -- 172.21.15.86:0/1340829746 <== osd.0 v2:172.21.15.86:6808/2750248043 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6b5c074580 con 0x7f6b3c06e130 2024-03-22T04:21:46.482 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:46.482 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-22T04:21:46.482 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f6b3c06e130 msgr2=0x7f6b3c070570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6808/2750248043,v1:172.21.15.86:6809/2750248043] conn(0x7f6b3c06e130 0x7f6b3c070570 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b3c067b50 msgr2=0x7f6b3c06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6b3c067b50 0x7f6b3c06a010 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6b480ab000 tx=0x7f6b480b0040 comp rx=0 tx=0).stop 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 msgr2=0x7f6b5c158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 --2- 172.21.15.86:0/1340829746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b40004810 0x7f6b5c158f70 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c011e30 tx=0x7f6b4c00f580 comp rx=0 tx=0).stop 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b5a800640 1 -- 172.21.15.86:0/1340829746 reap_dead start 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 shutdown_connections 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 >> 172.21.15.86:0/1340829746 conn(0x7f6b5c06b660 msgr2=0x7f6b5c15c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 shutdown_connections 2024-03-22T04:21:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.486+0000 7f6b61000640 1 -- 172.21.15.86:0/1340829746 wait complete. 2024-03-22T04:21:46.501 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-22T04:21:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.638+0000 7fef74800640 1 Processor -- start 2024-03-22T04:21:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.638+0000 7fef74800640 1 -- start start 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.638+0000 7fef74800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fef6c0ce6b0 0x7fef6c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.638+0000 7fef74800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fef6c15fed0 0x7fef6c1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.638+0000 7fef74800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef6c0d11c0 con 0x7fef6c0d0de0 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.638+0000 7fef74800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fef6c160820 con 0x7fef6c0ce6b0 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.638+0000 7fef74800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fef6c0cece0 con 0x7fef6c15fed0 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.638+0000 7fef71400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fef6c0ce6b0 0x7fef6c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef71e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fef6c0d0de0 0x7fef6c0d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45926/0 (socket says 172.21.15.86:45926) 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef71e00640 1 -- 172.21.15.86:0/1959602342 learned_addr learned my addr 172.21.15.86:0/1959602342 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef72800640 1 --2- 172.21.15.86:0/1959602342 >> v2:172.21.15.196:3300/0 conn(0x7fef6c15fed0 0x7fef6c1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef71400640 1 -- 172.21.15.86:0/1959602342 >> v2:172.21.15.196:3300/0 conn(0x7fef6c15fed0 msgr2=0x7fef6c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef71400640 1 --2- 172.21.15.86:0/1959602342 >> v2:172.21.15.196:3300/0 conn(0x7fef6c15fed0 0x7fef6c1602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef71400640 1 -- 172.21.15.86:0/1959602342 >> v1:172.21.15.86:6789/0 conn(0x7fef6c0d0de0 legacy=0x7fef6c0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef71400640 1 -- 172.21.15.86:0/1959602342 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c0cefc0 con 0x7fef6c0ce6b0 2024-03-22T04:21:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef70a00640 1 -- 172.21.15.86:0/1959602342 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1174146987 0 0) 0x7fef6c0d11c0 con 0x7fef6c0d0de0 2024-03-22T04:21:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef72800640 1 --2- 172.21.15.86:0/1959602342 >> v2:172.21.15.196:3300/0 conn(0x7fef6c15fed0 0x7fef6c1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef71400640 1 --2- 172.21.15.86:0/1959602342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0ce6b0 0x7fef6c0cfe70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0cc880 tx=0x7fef5c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f47a63414d00c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef70a00640 1 -- 172.21.15.86:0/1959602342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0b0070 con 0x7fef6c0ce6b0 2024-03-22T04:21:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef70a00640 1 -- 172.21.15.86:0/1959602342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c0ab600 con 0x7fef6c0ce6b0 2024-03-22T04:21:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.642+0000 7fef70a00640 1 -- 172.21.15.86:0/1959602342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0ab9d0 con 0x7fef6c0ce6b0 2024-03-22T04:21:46.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- 172.21.15.86:0/1959602342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0ce6b0 msgr2=0x7fef6c0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 --2- 172.21.15.86:0/1959602342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0ce6b0 0x7fef6c0cfe70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0cc880 tx=0x7fef5c01c480 comp rx=0 tx=0).stop 2024-03-22T04:21:46.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- 172.21.15.86:0/1959602342 shutdown_connections 2024-03-22T04:21:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 --2- 172.21.15.86:0/1959602342 >> v2:172.21.15.196:3300/0 conn(0x7fef6c15fed0 0x7fef6c1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 --2- 172.21.15.86:0/1959602342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0ce6b0 0x7fef6c0cfe70 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- 172.21.15.86:0/1959602342 >> 172.21.15.86:0/1959602342 conn(0x7fef6c06f910 msgr2=0x7fef6c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- 172.21.15.86:0/1959602342 shutdown_connections 2024-03-22T04:21:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- 172.21.15.86:0/1959602342 wait complete. 2024-03-22T04:21:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 Processor -- start 2024-03-22T04:21:46.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- start start 2024-03-22T04:21:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef6c0ce6b0 0x7fef6c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0d0de0 0x7fef6c0cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef6c15fed0 0x7fef6c14c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c0d0530 con 0x7fef6c0d0de0 2024-03-22T04:21:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c0d03b0 con 0x7fef6c0ce6b0 2024-03-22T04:21:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef74800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c071e30 con 0x7fef6c15fed0 2024-03-22T04:21:46.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0d0de0 0x7fef6c0cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0d0de0 0x7fef6c0cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51242/0 (socket says 172.21.15.86:51242) 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 -- 172.21.15.86:0/3627944163 learned_addr learned my addr 172.21.15.86:0/3627944163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71e00640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef6c0ce6b0 0x7fef6c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef72800640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef6c15fed0 0x7fef6c14c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 -- 172.21.15.86:0/3627944163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef6c15fed0 msgr2=0x7fef6c14c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef6c15fed0 0x7fef6c14c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 -- 172.21.15.86:0/3627944163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef6c0ce6b0 msgr2=0x7fef6c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef6c0ce6b0 0x7fef6c0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 -- 172.21.15.86:0/3627944163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c0cc180 con 0x7fef6c0d0de0 2024-03-22T04:21:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.646+0000 7fef71400640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0d0de0 0x7fef6c0cbc40 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fef5c01a0b0 tx=0x7fef5c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef66000640 1 -- 172.21.15.86:0/3627944163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0b0050 con 0x7fef6c0d0de0 2024-03-22T04:21:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef66000640 1 -- 172.21.15.86:0/3627944163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c0b1040 con 0x7fef6c0d0de0 2024-03-22T04:21:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef66000640 1 -- 172.21.15.86:0/3627944163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004220 con 0x7fef6c0d0de0 2024-03-22T04:21:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c0ca2f0 con 0x7fef6c0d0de0 2024-03-22T04:21:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c0b6c90 con 0x7fef6c0d0de0 2024-03-22T04:21:46.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c0b67a0 con 0x7fef6c0d0de0 2024-03-22T04:21:46.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef66000640 1 -- 172.21.15.86:0/3627944163 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fef5c0be060 con 0x7fef6c0d0de0 2024-03-22T04:21:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef66000640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef50067b20 0x7fef50069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef66000640 1 -- 172.21.15.86:0/3627944163 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7fef5c11f780 con 0x7fef6c0d0de0 2024-03-22T04:21:46.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.650+0000 7fef71e00640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef50067b20 0x7fef50069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.654+0000 7fef71e00640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef50067b20 0x7fef50069fe0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fef60000f70 tx=0x7fef60007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.654+0000 7fef66000640 1 -- 172.21.15.86:0/3627944163 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fef5c016070 con 0x7fef6c0d0de0 2024-03-22T04:21:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.810+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fef6c068650 con 0x7fef50067b20 2024-03-22T04:21:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef66000640 1 -- 172.21.15.86:0/3627944163 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fef6c068650 con 0x7fef50067b20 2024-03-22T04:21:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 0 to deep-scrub 2024-03-22T04:21:46.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef50067b20 msgr2=0x7fef50069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef50067b20 0x7fef50069fe0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fef60000f70 tx=0x7fef60007b60 comp rx=0 tx=0).stop 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0d0de0 msgr2=0x7fef6c0cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0d0de0 0x7fef6c0cbc40 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fef5c01a0b0 tx=0x7fef5c01acc0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 shutdown_connections 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fef50067b20 0x7fef50069fe0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef6c15fed0 0x7fef6c14c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c0d0de0 0x7fef6c0cbc40 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 --2- 172.21.15.86:0/3627944163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef6c0ce6b0 0x7fef6c0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 >> 172.21.15.86:0/3627944163 conn(0x7fef6c06f910 msgr2=0x7fef6c1658a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 shutdown_connections 2024-03-22T04:21:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.814+0000 7fef74800640 1 -- 172.21.15.86:0/3627944163 wait complete. 2024-03-22T04:21:46.830 INFO:tasks.ceph:Scrubbing osd.1 2024-03-22T04:21:46.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-22T04:21:46.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63b6400640 1 Processor -- start 2024-03-22T04:21:46.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63b6400640 1 -- start start 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63b6400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f63b0144a90 0x7f63b00727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63b6400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 0x7f63b00b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63b6400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b0144e70 con 0x7f63b00b35f0 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63b6400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f63b0073150 con 0x7f63b0144a90 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63b6400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f63b00b3e00 con 0x7f63b0072cf0 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63afe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f63b00b35f0 0x7f63b0071620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45934/0 (socket says 172.21.15.86:45934) 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63afe00640 1 -- 172.21.15.86:0/784447026 learned_addr learned my addr 172.21.15.86:0/784447026 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63af400640 1 --2- 172.21.15.86:0/784447026 >> v2:172.21.15.196:3300/0 conn(0x7f63b0144a90 0x7f63b00727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63a7e00640 1 --2- 172.21.15.86:0/784447026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 0x7f63b00b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63aea00640 1 -- 172.21.15.86:0/784447026 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314859369 0 0) 0x7f63b0144e70 con 0x7f63b00b35f0 2024-03-22T04:21:46.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63aea00640 1 -- 172.21.15.86:0/784447026 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6394003530 con 0x7f63b00b35f0 2024-03-22T04:21:46.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63aea00640 1 -- 172.21.15.86:0/784447026 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1635626669 0 0) 0x7f6394003530 con 0x7f63b00b35f0 2024-03-22T04:21:46.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63af400640 1 -- 172.21.15.86:0/784447026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 msgr2=0x7f63b00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:46.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63af400640 1 --2- 172.21.15.86:0/784447026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 0x7f63b00b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63af400640 1 -- 172.21.15.86:0/784447026 >> v1:172.21.15.86:6789/0 conn(0x7f63b00b35f0 legacy=0x7f63b0071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.974+0000 7f63af400640 1 -- 172.21.15.86:0/784447026 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b00b40e0 con 0x7f63b0144a90 2024-03-22T04:21:46.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63af400640 1 --2- 172.21.15.86:0/784447026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b0144a90 0x7f63b00727b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f63a00097c0 tx=0x7f63a001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265373e1c86c8e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63aea00640 1 -- 172.21.15.86:0/784447026 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a00b0070 con 0x7f63b0144a90 2024-03-22T04:21:46.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63aea00640 1 -- 172.21.15.86:0/784447026 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a00ab400 con 0x7f63b0144a90 2024-03-22T04:21:46.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63aea00640 1 -- 172.21.15.86:0/784447026 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8001a00 con 0x7f63b0144a90 2024-03-22T04:21:46.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63b6400640 1 -- 172.21.15.86:0/784447026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b0144a90 msgr2=0x7f63b00727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63b6400640 1 --2- 172.21.15.86:0/784447026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b0144a90 0x7f63b00727b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f63a00097c0 tx=0x7f63a001c390 comp rx=0 tx=0).stop 2024-03-22T04:21:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63b6400640 1 -- 172.21.15.86:0/784447026 shutdown_connections 2024-03-22T04:21:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63b6400640 1 --2- 172.21.15.86:0/784447026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 0x7f63b00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63b6400640 1 --2- 172.21.15.86:0/784447026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b0144a90 0x7f63b00727b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63b6400640 1 -- 172.21.15.86:0/784447026 >> 172.21.15.86:0/784447026 conn(0x7f63b006b660 msgr2=0x7f63b006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63b6400640 1 -- 172.21.15.86:0/784447026 shutdown_connections 2024-03-22T04:21:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.978+0000 7f63b6400640 1 -- 172.21.15.86:0/784447026 wait complete. 2024-03-22T04:21:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 Processor -- start 2024-03-22T04:21:46.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 -- start start 2024-03-22T04:21:46.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 0x7f63b0157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b00b35f0 0x7f63b0137c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63b0144a90 0x7f63b0138150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63b00b3bc0 con 0x7f63b0144a90 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63b00b3a40 con 0x7f63b00b35f0 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f63b0167d00 con 0x7f63b0072cf0 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63b0144a90 0x7f63b0138150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63b0144a90 0x7f63b0138150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51256/0 (socket says 172.21.15.86:51256) 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 -- 172.21.15.86:0/3177139274 learned_addr learned my addr 172.21.15.86:0/3177139274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63a7e00640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b00b35f0 0x7f63b0137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63af400640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 0x7f63b0157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 -- 172.21.15.86:0/3177139274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 msgr2=0x7f63b0157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0072cf0 0x7f63b0157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 -- 172.21.15.86:0/3177139274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b00b35f0 msgr2=0x7f63b0137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b00b35f0 0x7f63b0137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 -- 172.21.15.86:0/3177139274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b013b0d0 con 0x7f63b0144a90 2024-03-22T04:21:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63afe00640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63b0144a90 0x7f63b0138150 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f63a8001f60 tx=0x7f63a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8012400 con 0x7f63b0144a90 2024-03-22T04:21:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a80125a0 con 0x7f63b0144a90 2024-03-22T04:21:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8012950 con 0x7f63b0144a90 2024-03-22T04:21:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b013b3b0 con 0x7f63b0144a90 2024-03-22T04:21:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.982+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b013b970 con 0x7f63b0144a90 2024-03-22T04:21:46.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63b013ae80 con 0x7f63b0144a90 2024-03-22T04:21:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f63a8012af0 con 0x7f63b0144a90 2024-03-22T04:21:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63a7400640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f638c067b20 0x7f638c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f63a80871d0 con 0x7f63b0144a90 2024-03-22T04:21:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63af400640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f638c067b20 0x7f638c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63a7400640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7f638c06e100 0x7f638c070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63a801fec0 con 0x7f638c06e100 2024-03-22T04:21:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7f63a80875c0 con 0x7f63b0144a90 2024-03-22T04:21:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63a7e00640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7f638c06e100 0x7f638c070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:46.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63af400640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f638c067b20 0x7f638c069fe0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f63a00ad000 tx=0x7f63a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.986+0000 7f63a7e00640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7f638c06e100 0x7f638c070540 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:46.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:46.990+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 <== osd.1 v2:172.21.15.86:6800/3761623609 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63a801fec0 con 0x7f638c06e100 2024-03-22T04:21:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 --> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f63b0074580 con 0x7f638c06e100 2024-03-22T04:21:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63a7400640 1 -- 172.21.15.86:0/3177139274 <== osd.1 v2:172.21.15.86:6800/3761623609 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f63b0074580 con 0x7f638c06e100 2024-03-22T04:21:46.999 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:46.999 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-22T04:21:46.999 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7f638c06e100 msgr2=0x7f638c070540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6800/3761623609,v1:172.21.15.86:6801/3761623609] conn(0x7f638c06e100 0x7f638c070540 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f638c067b20 msgr2=0x7f638c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f638c067b20 0x7f638c069fe0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f63a00ad000 tx=0x7f63a0008040 comp rx=0 tx=0).stop 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63b0144a90 msgr2=0x7f63b0138150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 --2- 172.21.15.86:0/3177139274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63b0144a90 0x7f63b0138150 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f63a8001f60 tx=0x7f63a800a430 comp rx=0 tx=0).stop 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63afe00640 1 -- 172.21.15.86:0/3177139274 reap_dead start 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 shutdown_connections 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 >> 172.21.15.86:0/3177139274 conn(0x7f63b006b660 msgr2=0x7f63b01421a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 shutdown_connections 2024-03-22T04:21:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.002+0000 7f63b6400640 1 -- 172.21.15.86:0/3177139274 wait complete. 2024-03-22T04:21:47.017 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-22T04:21:47.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56de000640 1 Processor -- start 2024-03-22T04:21:47.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56de000640 1 -- start start 2024-03-22T04:21:47.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56de000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d815b3f0 0x7f56d80729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56de000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f56d80ae540 0x7f56d8072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56de000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d8071920 con 0x7f56d80cc3e0 2024-03-22T04:21:47.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56de000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f56d80716d0 con 0x7f56d80ae540 2024-03-22T04:21:47.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56de000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f56d806d990 con 0x7f56d815b3f0 2024-03-22T04:21:47.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f56d80cc3e0 0x7f56d80ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45938/0 (socket says 172.21.15.86:45938) 2024-03-22T04:21:47.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6a00640 1 -- 172.21.15.86:0/3133887432 learned_addr learned my addr 172.21.15.86:0/3133887432 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307314168 0 0) 0x7f56d8071920 con 0x7f56d80cc3e0 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56bc003530 con 0x7f56d80cc3e0 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7e00640 1 --2- 172.21.15.86:0/3133887432 >> v2:172.21.15.196:3300/0 conn(0x7f56d80ae540 0x7f56d8072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7400640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d815b3f0 0x7f56d80729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 642024483 0 0) 0x7f56bc003530 con 0x7f56d80cc3e0 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d815b3f0 msgr2=0x7f56d80729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d815b3f0 0x7f56d80729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 >> v2:172.21.15.196:3300/0 conn(0x7f56d80ae540 msgr2=0x7f56d8072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 --2- 172.21.15.86:0/3133887432 >> v2:172.21.15.196:3300/0 conn(0x7f56d80ae540 0x7f56d8072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d8071c00 con 0x7f56d80cc3e0 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f56c4001680 con 0x7f56d80cc3e0 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 >> v1:172.21.15.86:6789/0 conn(0x7f56d80cc3e0 legacy=0x7f56d80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0039c0 0x7f56bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d815b3f0 0x7f56bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56d80716d0 con 0x7f56bc004d50 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56d806d990 con 0x7f56bc0039c0 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc00b9d0 con 0x7f56d815b3f0 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7e00640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7400640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0039c0 0x7f56bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d6a00640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d815b3f0 0x7f56bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7e00640 1 -- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d815b3f0 msgr2=0x7f56bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7e00640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d815b3f0 0x7f56bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7e00640 1 -- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0039c0 msgr2=0x7f56bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7e00640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0039c0 0x7f56bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.158+0000 7f56d7e00640 1 -- 172.21.15.86:0/3133887432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc003530 con 0x7f56bc004d50 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56d7e00640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56bc0051d0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f56d8070ef0 tx=0x7f56d001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4134478e99f8279 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d00b2070 con 0x7f56bc004d50 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d00a5b60 con 0x7f56bc004d50 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56d6000640 1 -- 172.21.15.86:0/3133887432 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0009c10 con 0x7f56bc004d50 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56de000640 1 -- 172.21.15.86:0/3133887432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 msgr2=0x7f56bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56de000640 1 --2- 172.21.15.86:0/3133887432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56bc0051d0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f56d8070ef0 tx=0x7f56d001cc60 comp rx=0 tx=0).stop 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56d7e00640 1 -- 172.21.15.86:0/3133887432 reap_dead start 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56de000640 1 -- 172.21.15.86:0/3133887432 shutdown_connections 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56de000640 1 -- 172.21.15.86:0/3133887432 >> 172.21.15.86:0/3133887432 conn(0x7f56d80b1df0 msgr2=0x7f56d80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56de000640 1 -- 172.21.15.86:0/3133887432 shutdown_connections 2024-03-22T04:21:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.162+0000 7f56de000640 1 -- 172.21.15.86:0/3133887432 wait complete. 2024-03-22T04:21:47.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 Processor -- start 2024-03-22T04:21:47.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 -- start start 2024-03-22T04:21:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56d81f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d8153150 0x7f56d81590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56d81535d0 0x7f56d8159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56d80b30f0 con 0x7f56bc004d50 2024-03-22T04:21:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56d80721c0 con 0x7f56d81535d0 2024-03-22T04:21:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f56d80b3270 con 0x7f56d8153150 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56d81f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56d81f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51270/0 (socket says 172.21.15.86:51270) 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 -- 172.21.15.86:0/3676623912 learned_addr learned my addr 172.21.15.86:0/3676623912 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7e00640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56d81535d0 0x7f56d8159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d6a00640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d8153150 0x7f56d81590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 -- 172.21.15.86:0/3676623912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d8153150 msgr2=0x7f56d81590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d8153150 0x7f56d81590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 -- 172.21.15.86:0/3676623912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56d81535d0 msgr2=0x7f56d8159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56d81535d0 0x7f56d8159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 -- 172.21.15.86:0/3676623912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d81520b0 con 0x7f56bc004d50 2024-03-22T04:21:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56d7400640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56d81f6cf0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f56c8011a00 tx=0x7f56c8011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56cf400640 1 -- 172.21.15.86:0/3676623912 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c800fb50 con 0x7f56bc004d50 2024-03-22T04:21:47.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56cf400640 1 -- 172.21.15.86:0/3676623912 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c800fcf0 con 0x7f56bc004d50 2024-03-22T04:21:47.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56cf400640 1 -- 172.21.15.86:0/3676623912 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c800fea0 con 0x7f56bc004d50 2024-03-22T04:21:47.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d8152390 con 0x7f56bc004d50 2024-03-22T04:21:47.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.166+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d8152950 con 0x7f56bc004d50 2024-03-22T04:21:47.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.170+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d8068650 con 0x7f56bc004d50 2024-03-22T04:21:47.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.170+0000 7f56cf400640 1 -- 172.21.15.86:0/3676623912 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f56c8012040 con 0x7f56bc004d50 2024-03-22T04:21:47.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.170+0000 7f56cf400640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f56b8067b20 0x7f56b8069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.170+0000 7f56cf400640 1 -- 172.21.15.86:0/3676623912 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f56c8086490 con 0x7f56bc004d50 2024-03-22T04:21:47.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.170+0000 7f56d6a00640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f56b8067b20 0x7f56b8069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.170+0000 7f56d6a00640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f56b8067b20 0x7f56b8069fe0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f56d8153c30 tx=0x7f56c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.174+0000 7f56cf400640 1 -- 172.21.15.86:0/3676623912 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f56c8019030 con 0x7f56bc004d50 2024-03-22T04:21:47.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f56d806c670 con 0x7f56b8067b20 2024-03-22T04:21:47.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56cf400640 1 -- 172.21.15.86:0/3676623912 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f56d806c670 con 0x7f56b8067b20 2024-03-22T04:21:47.327 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 1 to deep-scrub 2024-03-22T04:21:47.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f56b8067b20 msgr2=0x7f56b8069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f56b8067b20 0x7f56b8069fe0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f56d8153c30 tx=0x7f56c4009040 comp rx=0 tx=0).stop 2024-03-22T04:21:47.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 msgr2=0x7f56d81f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56d81f6cf0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f56c8011a00 tx=0x7f56c8011ec0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 shutdown_connections 2024-03-22T04:21:47.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f56b8067b20 0x7f56b8069fe0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56d81535d0 0x7f56d8159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56d8153150 0x7f56d81590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 --2- 172.21.15.86:0/3676623912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc004d50 0x7f56d81f6cf0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 >> 172.21.15.86:0/3676623912 conn(0x7f56d80b1df0 msgr2=0x7f56d80b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:47.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 shutdown_connections 2024-03-22T04:21:47.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.330+0000 7f56de000640 1 -- 172.21.15.86:0/3676623912 wait complete. 2024-03-22T04:21:47.348 INFO:tasks.ceph:Scrubbing osd.2 2024-03-22T04:21:47.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-22T04:21:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242fa00640 1 Processor -- start 2024-03-22T04:21:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242fa00640 1 -- start start 2024-03-22T04:21:47.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f242815e670 0x7f242806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242fa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24280d05c0 con 0x7f242809aa70 2024-03-22T04:21:47.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242fa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24280d08a0 con 0x7f242806c980 2024-03-22T04:21:47.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f242815ea50 con 0x7f242815e670 2024-03-22T04:21:47.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f242815e670 0x7f242806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f242815e670 0x7f242806d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51276/0 (socket says 172.21.15.86:51276) 2024-03-22T04:21:47.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242d000640 1 -- 172.21.15.86:0/2208962847 learned_addr learned my addr 172.21.15.86:0/2208962847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:47.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242d000640 1 -- 172.21.15.86:0/2208962847 >> v1:172.21.15.196:6789/0 conn(0x7f242806c980 legacy=0x7f24280d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:47.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242d000640 1 -- 172.21.15.86:0/2208962847 >> v1:172.21.15.86:6789/0 conn(0x7f242809aa70 legacy=0x7f24280cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.490+0000 7f242d000640 1 -- 172.21.15.86:0/2208962847 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24280d0b80 con 0x7f242815e670 2024-03-22T04:21:47.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f242d000640 1 --2- 172.21.15.86:0/2208962847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242815e670 0x7f242806d870 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2418009e50 tx=0x7f241801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d425d95d312f17a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f2427400640 1 -- 172.21.15.86:0/2208962847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180a57d0 con 0x7f242815e670 2024-03-22T04:21:47.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f2427400640 1 -- 172.21.15.86:0/2208962847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24180a5970 con 0x7f242815e670 2024-03-22T04:21:47.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f2427400640 1 -- 172.21.15.86:0/2208962847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180a5d40 con 0x7f242815e670 2024-03-22T04:21:47.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f242fa00640 1 -- 172.21.15.86:0/2208962847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242815e670 msgr2=0x7f242806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f242fa00640 1 --2- 172.21.15.86:0/2208962847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242815e670 0x7f242806d870 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2418009e50 tx=0x7f241801c9f0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f242fa00640 1 -- 172.21.15.86:0/2208962847 shutdown_connections 2024-03-22T04:21:47.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f242fa00640 1 --2- 172.21.15.86:0/2208962847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242815e670 0x7f242806d870 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f242fa00640 1 -- 172.21.15.86:0/2208962847 >> 172.21.15.86:0/2208962847 conn(0x7f24280b3ba0 msgr2=0x7f24280b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:47.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f242fa00640 1 -- 172.21.15.86:0/2208962847 shutdown_connections 2024-03-22T04:21:47.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.494+0000 7f242fa00640 1 -- 172.21.15.86:0/2208962847 wait complete. 2024-03-22T04:21:47.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 Processor -- start 2024-03-22T04:21:47.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 -- start start 2024-03-22T04:21:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242806c980 0x7f242815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242809aa70 0x7f24280c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f242815e670 0x7f2428154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f242816cc60 con 0x7f242806c980 2024-03-22T04:21:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f242809b2b0 con 0x7f242809aa70 2024-03-22T04:21:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f242816cde0 con 0x7f242815e670 2024-03-22T04:21:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242806c980 0x7f242815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242806c980 0x7f242815a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51288/0 (socket says 172.21.15.86:51288) 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 -- 172.21.15.86:0/361291934 learned_addr learned my addr 172.21.15.86:0/361291934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f2427e00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242809aa70 0x7f24280c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242da00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f242815e670 0x7f2428154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 -- 172.21.15.86:0/361291934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f242815e670 msgr2=0x7f2428154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f242815e670 0x7f2428154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 -- 172.21.15.86:0/361291934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242809aa70 msgr2=0x7f24280c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f242809aa70 0x7f24280c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 -- 172.21.15.86:0/361291934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24280c9880 con 0x7f242806c980 2024-03-22T04:21:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242d000640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242806c980 0x7f242815a5d0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2418019600 tx=0x7f2418019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418009250 con 0x7f242806c980 2024-03-22T04:21:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24180b8680 con 0x7f242806c980 2024-03-22T04:21:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180b8a70 con 0x7f242806c980 2024-03-22T04:21:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24280c9b60 con 0x7f242806c980 2024-03-22T04:21:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24280ca120 con 0x7f242806c980 2024-03-22T04:21:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.498+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2428074580 con 0x7f242806c980 2024-03-22T04:21:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f24180c0020 con 0x7f242806c980 2024-03-22T04:21:47.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2424c00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2408067b20 0x7f2408069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f24181217f0 con 0x7f242806c980 2024-03-22T04:21:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2427e00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2408067b20 0x7f2408069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2424c00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7f240806e100 0x7f2408070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24180a5970 con 0x7f240806e100 2024-03-22T04:21:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7f2418121be0 con 0x7f242806c980 2024-03-22T04:21:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f242da00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7f240806e100 0x7f2408070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2427e00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2408067b20 0x7f2408069fe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f24280b77c0 tx=0x7f2414008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f242da00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7f240806e100 0x7f2408070540 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.502+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 <== osd.2 v2:172.21.15.86:6816/4132808680 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24180a5970 con 0x7f240806e100 2024-03-22T04:21:47.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 --> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f24280b46c0 con 0x7f240806e100 2024-03-22T04:21:47.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f2424c00640 1 -- 172.21.15.86:0/361291934 <== osd.2 v2:172.21.15.86:6816/4132808680 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f24280b46c0 con 0x7f240806e100 2024-03-22T04:21:47.514 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-22T04:21:47.515 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7f240806e100 msgr2=0x7f2408070540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6816/4132808680,v1:172.21.15.86:6817/4132808680] conn(0x7f240806e100 0x7f2408070540 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2408067b20 msgr2=0x7f2408069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f2408067b20 0x7f2408069fe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f24280b77c0 tx=0x7f2414008040 comp rx=0 tx=0).stop 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242806c980 msgr2=0x7f242815a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 --2- 172.21.15.86:0/361291934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242806c980 0x7f242815a5d0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2418019600 tx=0x7f2418019790 comp rx=0 tx=0).stop 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242da00640 1 -- 172.21.15.86:0/361291934 reap_dead start 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 shutdown_connections 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 >> 172.21.15.86:0/361291934 conn(0x7f24280b3ba0 msgr2=0x7f24280cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 shutdown_connections 2024-03-22T04:21:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.518+0000 7f242fa00640 1 -- 172.21.15.86:0/361291934 wait complete. 2024-03-22T04:21:47.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-22T04:21:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19d0c00640 1 Processor -- start 2024-03-22T04:21:47.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19d0c00640 1 -- start start 2024-03-22T04:21:47.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19d0c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f19cc0ad330 0x7f19cc15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19d0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19cc06ef40 0x7f19cc06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19d0c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19cc15c7c0 con 0x7f19cc0ad710 2024-03-22T04:21:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19d0c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f19cc06dc40 con 0x7f19cc06ef40 2024-03-22T04:21:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19d0c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f19cc06dea0 con 0x7f19cc0ad330 2024-03-22T04:21:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19c9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19cc06ef40 0x7f19cc06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19c9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19cc06ef40 0x7f19cc06d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51304/0 (socket says 172.21.15.86:51304) 2024-03-22T04:21:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.670+0000 7f19c9400640 1 -- 172.21.15.86:0/4008265832 learned_addr learned my addr 172.21.15.86:0/4008265832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:47.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19c9400640 1 -- 172.21.15.86:0/4008265832 >> v2:172.21.15.196:3300/0 conn(0x7f19cc0ad330 msgr2=0x7f19cc15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19c9400640 1 --2- 172.21.15.86:0/4008265832 >> v2:172.21.15.196:3300/0 conn(0x7f19cc0ad330 0x7f19cc15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19c9400640 1 -- 172.21.15.86:0/4008265832 >> v1:172.21.15.196:6789/0 conn(0x7f19cc0ad710 legacy=0x7f19cc13fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:47.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19c9400640 1 -- 172.21.15.86:0/4008265832 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc06e180 con 0x7f19cc06ef40 2024-03-22T04:21:47.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19c9400640 1 --2- 172.21.15.86:0/4008265832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc06ef40 0x7f19cc06d700 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f19cc06fbe0 tx=0x7f19b801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce1ebedf5095c13 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19c8a00640 1 -- 172.21.15.86:0/4008265832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80b1070 con 0x7f19cc06ef40 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19c8a00640 1 -- 172.21.15.86:0/4008265832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b80ac620 con 0x7f19cc06ef40 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19c8a00640 1 -- 172.21.15.86:0/4008265832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80ac9f0 con 0x7f19cc06ef40 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19d0c00640 1 -- 172.21.15.86:0/4008265832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc06ef40 msgr2=0x7f19cc06d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.674+0000 7f19d0c00640 1 --2- 172.21.15.86:0/4008265832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc06ef40 0x7f19cc06d700 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f19cc06fbe0 tx=0x7f19b801c4b0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 -- 172.21.15.86:0/4008265832 shutdown_connections 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 --2- 172.21.15.86:0/4008265832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc06ef40 0x7f19cc06d700 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 --2- 172.21.15.86:0/4008265832 >> v2:172.21.15.196:3300/0 conn(0x7f19cc0ad330 0x7f19cc15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 -- 172.21.15.86:0/4008265832 >> 172.21.15.86:0/4008265832 conn(0x7f19cc0b1df0 msgr2=0x7f19cc0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 -- 172.21.15.86:0/4008265832 shutdown_connections 2024-03-22T04:21:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 -- 172.21.15.86:0/4008265832 wait complete. 2024-03-22T04:21:47.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 Processor -- start 2024-03-22T04:21:47.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 -- start start 2024-03-22T04:21:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc06ef40 0x7f19cc165610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc0ad330 0x7f19cc1fa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19cc0ad710 0x7f19cc0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc1690e0 con 0x7f19cc0ad330 2024-03-22T04:21:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc168f60 con 0x7f19cc06ef40 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19d0c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc169260 con 0x7f19cc0ad710 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc0ad330 0x7f19cc1fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc0ad330 0x7f19cc1fa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51320/0 (socket says 172.21.15.86:51320) 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 -- 172.21.15.86:0/830776280 learned_addr learned my addr 172.21.15.86:0/830776280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 -- 172.21.15.86:0/830776280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19cc0ad710 msgr2=0x7f19cc0afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19cc0ad710 0x7f19cc0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9e00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc06ef40 0x7f19cc165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 -- 172.21.15.86:0/830776280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc06ef40 msgr2=0x7f19cc165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc06ef40 0x7f19cc165610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 -- 172.21.15.86:0/830776280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc164250 con 0x7f19cc0ad330 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9e00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc06ef40 0x7f19cc165610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c9400640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc0ad330 0x7f19cc1fa570 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f19b80af040 tx=0x7f19b801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c2000640 1 -- 172.21.15.86:0/830776280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80b1040 con 0x7f19cc0ad330 2024-03-22T04:21:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.678+0000 7f19c2000640 1 -- 172.21.15.86:0/830776280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b80bd070 con 0x7f19cc0ad330 2024-03-22T04:21:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc164530 con 0x7f19cc0ad330 2024-03-22T04:21:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19c2000640 1 -- 172.21.15.86:0/830776280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80b7880 con 0x7f19cc0ad330 2024-03-22T04:21:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc164af0 con 0x7f19cc0ad330 2024-03-22T04:21:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19cc068650 con 0x7f19cc0ad330 2024-03-22T04:21:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19c2000640 1 -- 172.21.15.86:0/830776280 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f19b80b7a20 con 0x7f19cc0ad330 2024-03-22T04:21:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19c2000640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19ac067b20 0x7f19ac069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19c2000640 1 -- 172.21.15.86:0/830776280 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f19b81204d0 con 0x7f19cc0ad330 2024-03-22T04:21:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19c9e00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19ac067b20 0x7f19ac069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.682+0000 7f19c9e00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19ac067b20 0x7f19ac069fe0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f19bc007eb0 tx=0x7f19bc0073c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.686+0000 7f19c2000640 1 -- 172.21.15.86:0/830776280 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f19b80160d0 con 0x7f19cc0ad330 2024-03-22T04:21:47.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f19cc0717e0 con 0x7f19ac067b20 2024-03-22T04:21:47.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19c2000640 1 -- 172.21.15.86:0/830776280 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f19cc0717e0 con 0x7f19ac067b20 2024-03-22T04:21:47.840 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 2 to deep-scrub 2024-03-22T04:21:47.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19ac067b20 msgr2=0x7f19ac069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19ac067b20 0x7f19ac069fe0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f19bc007eb0 tx=0x7f19bc0073c0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc0ad330 msgr2=0x7f19cc1fa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc0ad330 0x7f19cc1fa570 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f19b80af040 tx=0x7f19b801acf0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 shutdown_connections 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f19ac067b20 0x7f19ac069fe0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19cc0ad710 0x7f19cc0afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc0ad330 0x7f19cc1fa570 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 --2- 172.21.15.86:0/830776280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19cc06ef40 0x7f19cc165610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 >> 172.21.15.86:0/830776280 conn(0x7f19cc0b1df0 msgr2=0x7f19cc141120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 shutdown_connections 2024-03-22T04:21:47.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:47.842+0000 7f19d0c00640 1 -- 172.21.15.86:0/830776280 wait complete. 2024-03-22T04:21:47.860 INFO:tasks.ceph:Scrubbing osd.3 2024-03-22T04:21:47.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-22T04:21:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.002+0000 7f3d2c400640 1 Processor -- start 2024-03-22T04:21:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.002+0000 7f3d2c400640 1 -- start start 2024-03-22T04:21:48.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d2c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 0x7f3d240727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d2c400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d24072cf0 0x7f3d240b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d2c400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d24144e70 con 0x7f3d240b35f0 2024-03-22T04:21:48.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d2c400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3d24073150 con 0x7f3d24072cf0 2024-03-22T04:21:48.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d2c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3d240b3e00 con 0x7f3d24144a90 2024-03-22T04:21:48.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d2a400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3d240b35f0 0x7f3d24071620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45962/0 (socket says 172.21.15.86:45962) 2024-03-22T04:21:48.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d2a400640 1 -- 172.21.15.86:0/3995137596 learned_addr learned my addr 172.21.15.86:0/3995137596 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:48.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d29a00640 1 --2- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 0x7f3d240727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d29000640 1 --2- 172.21.15.86:0/3995137596 >> v2:172.21.15.196:3300/0 conn(0x7f3d24072cf0 0x7f3d240b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d23e00640 1 -- 172.21.15.86:0/3995137596 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956591739 0 0) 0x7f3d24144e70 con 0x7f3d240b35f0 2024-03-22T04:21:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d23e00640 1 -- 172.21.15.86:0/3995137596 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d0c003530 con 0x7f3d240b35f0 2024-03-22T04:21:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d29000640 1 -- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 msgr2=0x7f3d240727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d29000640 1 --2- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 0x7f3d240727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d29000640 1 -- 172.21.15.86:0/3995137596 >> v1:172.21.15.86:6789/0 conn(0x7f3d240b35f0 legacy=0x7f3d24071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d29000640 1 -- 172.21.15.86:0/3995137596 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d240b40e0 con 0x7f3d24072cf0 2024-03-22T04:21:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.006+0000 7f3d29a00640 1 --2- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 0x7f3d240727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T04:21:48.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d29000640 1 --2- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d24072cf0 0x7f3d240b30b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3d100097c0 tx=0x7f3d1001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5577bfef7bd667ff server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d23e00640 1 -- 172.21.15.86:0/3995137596 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100a7070 con 0x7f3d24072cf0 2024-03-22T04:21:48.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d23e00640 1 -- 172.21.15.86:0/3995137596 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d100b0070 con 0x7f3d24072cf0 2024-03-22T04:21:48.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d23e00640 1 -- 172.21.15.86:0/3995137596 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100ab890 con 0x7f3d24072cf0 2024-03-22T04:21:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d24072cf0 msgr2=0x7f3d240b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 --2- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d24072cf0 0x7f3d240b30b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3d100097c0 tx=0x7f3d1001c470 comp rx=0 tx=0).stop 2024-03-22T04:21:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- 172.21.15.86:0/3995137596 shutdown_connections 2024-03-22T04:21:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 --2- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d24072cf0 0x7f3d240b30b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 --2- 172.21.15.86:0/3995137596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 0x7f3d240727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- 172.21.15.86:0/3995137596 >> 172.21.15.86:0/3995137596 conn(0x7f3d2406b660 msgr2=0x7f3d2406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- 172.21.15.86:0/3995137596 shutdown_connections 2024-03-22T04:21:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- 172.21.15.86:0/3995137596 wait complete. 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 Processor -- start 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- start start 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d24072cf0 0x7f3d240aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d240b35f0 0x7f3d2413bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 0x7f3d240b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d240b3b80 con 0x7f3d240b35f0 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d240b3a00 con 0x7f3d24072cf0 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d2c400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3d24165cf0 con 0x7f3d24144a90 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d29000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d240b35f0 0x7f3d2413bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.010+0000 7f3d29000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d240b35f0 0x7f3d2413bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51326/0 (socket says 172.21.15.86:51326) 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d29000640 1 -- 172.21.15.86:0/376176568 learned_addr learned my addr 172.21.15.86:0/376176568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d29000640 1 -- 172.21.15.86:0/376176568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 msgr2=0x7f3d240b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d29000640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d24144a90 0x7f3d240b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d29000640 1 -- 172.21.15.86:0/376176568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d24072cf0 msgr2=0x7f3d240aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d29000640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d24072cf0 0x7f3d240aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d29000640 1 -- 172.21.15.86:0/376176568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2413ddf0 con 0x7f3d240b35f0 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d29000640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d240b35f0 0x7f3d2413bde0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ad040 tx=0x7f3d1001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100bf610 con 0x7f3d240b35f0 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d100abe50 con 0x7f3d240b35f0 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100bfa10 con 0x7f3d240b35f0 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d240af3b0 con 0x7f3d240b35f0 2024-03-22T04:21:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2413c9f0 con 0x7f3d240b35f0 2024-03-22T04:21:48.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d24074580 con 0x7f3d240b35f0 2024-03-22T04:21:48.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f3d100bd020 con 0x7f3d240b35f0 2024-03-22T04:21:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d21600640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d04067b20 0x7f3d04069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f3d10120b40 con 0x7f3d240b35f0 2024-03-22T04:21:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.014+0000 7f3d29a00640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d04067b20 0x7f3d04069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.018+0000 7f3d21600640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7f3d0406e100 0x7f3d04070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.018+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d100bde80 con 0x7f3d0406e100 2024-03-22T04:21:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.018+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7f3d10120ef0 con 0x7f3d240b35f0 2024-03-22T04:21:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.018+0000 7f3d2a400640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7f3d0406e100 0x7f3d04070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.018+0000 7f3d29a00640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d04067b20 0x7f3d04069fe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009fc0 tx=0x7f3d1800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.018+0000 7f3d2a400640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7f3d0406e100 0x7f3d04070540 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.018+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 <== osd.3 v2:172.21.15.86:6822/2837220459 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d100bde80 con 0x7f3d0406e100 2024-03-22T04:21:48.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 --> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3d2413ec40 con 0x7f3d0406e100 2024-03-22T04:21:48.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d21600640 1 -- 172.21.15.86:0/376176568 <== osd.3 v2:172.21.15.86:6822/2837220459 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3d2413ec40 con 0x7f3d0406e100 2024-03-22T04:21:48.028 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:48.029 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-22T04:21:48.029 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7f3d0406e100 msgr2=0x7f3d04070540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6822/2837220459,v1:172.21.15.86:6823/2837220459] conn(0x7f3d0406e100 0x7f3d04070540 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d04067b20 msgr2=0x7f3d04069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f3d04067b20 0x7f3d04069fe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009fc0 tx=0x7f3d1800b040 comp rx=0 tx=0).stop 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d240b35f0 msgr2=0x7f3d2413bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 --2- 172.21.15.86:0/376176568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d240b35f0 0x7f3d2413bde0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ad040 tx=0x7f3d1001acb0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2a400640 1 -- 172.21.15.86:0/376176568 reap_dead start 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 shutdown_connections 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 >> 172.21.15.86:0/376176568 conn(0x7f3d2406b660 msgr2=0x7f3d2413f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 shutdown_connections 2024-03-22T04:21:48.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.030+0000 7f3d2c400640 1 -- 172.21.15.86:0/376176568 wait complete. 2024-03-22T04:21:48.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-22T04:21:48.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a68200640 1 Processor -- start 2024-03-22T04:21:48.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a68200640 1 -- start start 2024-03-22T04:21:48.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a68200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a600ad330 0x7f9a6015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a68200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a6006ef40 0x7f9a6006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a68200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6015c7c0 con 0x7f9a600ad710 2024-03-22T04:21:48.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a68200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9a6006dc40 con 0x7f9a600ad330 2024-03-22T04:21:48.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a68200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6006dea0 con 0x7f9a6006ef40 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a65800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a600ad330 0x7f9a6015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a65800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a600ad330 0x7f9a6015d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51336/0 (socket says 172.21.15.86:51336) 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a65800640 1 -- 172.21.15.86:0/190493138 learned_addr learned my addr 172.21.15.86:0/190493138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a65800640 1 -- 172.21.15.86:0/190493138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a6006ef40 msgr2=0x7f9a6006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a64e00640 1 --2- 172.21.15.86:0/190493138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a6006ef40 0x7f9a6006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a65800640 1 --2- 172.21.15.86:0/190493138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a6006ef40 0x7f9a6006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a65800640 1 -- 172.21.15.86:0/190493138 >> v1:172.21.15.86:6789/0 conn(0x7f9a600ad710 legacy=0x7f9a6013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a65800640 1 -- 172.21.15.86:0/190493138 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6006e180 con 0x7f9a600ad330 2024-03-22T04:21:48.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.190+0000 7f9a64e00640 1 --2- 172.21.15.86:0/190493138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a6006ef40 0x7f9a6006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:48.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a65800640 1 --2- 172.21.15.86:0/190493138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a600ad330 0x7f9a6015d4a0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009e50 tx=0x7f9a5401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65ce86c21cd2b0e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a5fe00640 1 -- 172.21.15.86:0/190493138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a540a57e0 con 0x7f9a600ad330 2024-03-22T04:21:48.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a5fe00640 1 -- 172.21.15.86:0/190493138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a540a5980 con 0x7f9a600ad330 2024-03-22T04:21:48.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a5fe00640 1 -- 172.21.15.86:0/190493138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a540a5d50 con 0x7f9a600ad330 2024-03-22T04:21:48.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a68200640 1 -- 172.21.15.86:0/190493138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a600ad330 msgr2=0x7f9a6015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a68200640 1 --2- 172.21.15.86:0/190493138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a600ad330 0x7f9a6015d4a0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009e50 tx=0x7f9a5401ca00 comp rx=0 tx=0).stop 2024-03-22T04:21:48.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a68200640 1 -- 172.21.15.86:0/190493138 shutdown_connections 2024-03-22T04:21:48.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a68200640 1 --2- 172.21.15.86:0/190493138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a6006ef40 0x7f9a6006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a68200640 1 --2- 172.21.15.86:0/190493138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a600ad330 0x7f9a6015d4a0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a68200640 1 -- 172.21.15.86:0/190493138 >> 172.21.15.86:0/190493138 conn(0x7f9a600b1df0 msgr2=0x7f9a600b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:48.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a68200640 1 -- 172.21.15.86:0/190493138 shutdown_connections 2024-03-22T04:21:48.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.194+0000 7f9a68200640 1 -- 172.21.15.86:0/190493138 wait complete. 2024-03-22T04:21:48.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 Processor -- start 2024-03-22T04:21:48.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 -- start start 2024-03-22T04:21:48.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6006ef40 0x7f9a600b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a600ad330 0x7f9a60152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a600ad710 0x7f9a60154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601690e0 con 0x7f9a6006ef40 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60168f60 con 0x7f9a600ad330 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9a60169260 con 0x7f9a600ad710 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6006ef40 0x7f9a600b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6006ef40 0x7f9a600b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51344/0 (socket says 172.21.15.86:51344) 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 -- 172.21.15.86:0/196631646 learned_addr learned my addr 172.21.15.86:0/196631646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a64e00640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a600ad330 0x7f9a60152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a66200640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a600ad710 0x7f9a60154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 -- 172.21.15.86:0/196631646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a600ad710 msgr2=0x7f9a60154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a600ad710 0x7f9a60154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 -- 172.21.15.86:0/196631646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a600ad330 msgr2=0x7f9a60152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a600ad330 0x7f9a60152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 -- 172.21.15.86:0/196631646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a601543b0 con 0x7f9a6006ef40 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a65800640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6006ef40 0x7f9a600b4f00 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f9a540af040 tx=0x7f9a540197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a5d600640 1 -- 172.21.15.86:0/196631646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a540b8670 con 0x7f9a6006ef40 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a5d600640 1 -- 172.21.15.86:0/196631646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a540be070 con 0x7f9a6006ef40 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a5d600640 1 -- 172.21.15.86:0/196631646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a540b8a90 con 0x7f9a6006ef40 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a601597f0 con 0x7f9a6006ef40 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a60154900 con 0x7f9a6006ef40 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.198+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a60068650 con 0x7f9a6006ef40 2024-03-22T04:21:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.202+0000 7f9a5d600640 1 -- 172.21.15.86:0/196631646 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f9a540b8c30 con 0x7f9a6006ef40 2024-03-22T04:21:48.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.202+0000 7f9a5d600640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a40067b20 0x7f9a40069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.202+0000 7f9a5d600640 1 -- 172.21.15.86:0/196631646 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f9a541210c0 con 0x7f9a6006ef40 2024-03-22T04:21:48.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.202+0000 7f9a64e00640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a40067b20 0x7f9a40069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.202+0000 7f9a64e00640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a40067b20 0x7f9a40069fe0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c006fd0 tx=0x7f9a4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.206+0000 7f9a5d600640 1 -- 172.21.15.86:0/196631646 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9a54016620 con 0x7f9a6006ef40 2024-03-22T04:21:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f9a60154ad0 con 0x7f9a40067b20 2024-03-22T04:21:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a5d600640 1 -- 172.21.15.86:0/196631646 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9a60154ad0 con 0x7f9a40067b20 2024-03-22T04:21:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 3 to deep-scrub 2024-03-22T04:21:48.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a40067b20 msgr2=0x7f9a40069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a40067b20 0x7f9a40069fe0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c006fd0 tx=0x7f9a4c008040 comp rx=0 tx=0).stop 2024-03-22T04:21:48.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6006ef40 msgr2=0x7f9a600b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6006ef40 0x7f9a600b4f00 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f9a540af040 tx=0x7f9a540197a0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 shutdown_connections 2024-03-22T04:21:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f9a40067b20 0x7f9a40069fe0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a600ad330 0x7f9a60152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a600ad710 0x7f9a60154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 --2- 172.21.15.86:0/196631646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6006ef40 0x7f9a600b4f00 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 >> 172.21.15.86:0/196631646 conn(0x7f9a600b1df0 msgr2=0x7f9a6015dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 shutdown_connections 2024-03-22T04:21:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.362+0000 7f9a68200640 1 -- 172.21.15.86:0/196631646 wait complete. 2024-03-22T04:21:48.381 INFO:tasks.ceph:Scrubbing osd.4 2024-03-22T04:21:48.381 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-22T04:21:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.522+0000 7f00c8400640 1 Processor -- start 2024-03-22T04:21:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.522+0000 7f00c8400640 1 -- start start 2024-03-22T04:21:48.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c8400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c0144a90 0x7f00c00727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c8400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f00c0072cf0 0x7f00c00b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c8400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c0144e70 con 0x7f00c00b35f0 2024-03-22T04:21:48.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c8400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f00c0073150 con 0x7f00c0072cf0 2024-03-22T04:21:48.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c8400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f00c00b3e00 con 0x7f00c0144a90 2024-03-22T04:21:48.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c5000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f00c0072cf0 0x7f00c00b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c5a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c0144a90 0x7f00c00727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c6400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f00c00b35f0 0x7f00c0071620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:39224/0 (socket says 172.21.15.86:39224) 2024-03-22T04:21:48.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c6400640 1 -- 172.21.15.86:0/1787545728 learned_addr learned my addr 172.21.15.86:0/1787545728 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:48.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00bfe00640 1 -- 172.21.15.86:0/1787545728 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431676647 0 0) 0x7f00c0144e70 con 0x7f00c00b35f0 2024-03-22T04:21:48.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00bfe00640 1 -- 172.21.15.86:0/1787545728 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a8005740 con 0x7f00c00b35f0 2024-03-22T04:21:48.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c5a00640 1 -- 172.21.15.86:0/1787545728 >> v2:172.21.15.196:3300/0 conn(0x7f00c0072cf0 msgr2=0x7f00c00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:48.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c5a00640 1 --2- 172.21.15.86:0/1787545728 >> v2:172.21.15.196:3300/0 conn(0x7f00c0072cf0 0x7f00c00b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c5a00640 1 -- 172.21.15.86:0/1787545728 >> v1:172.21.15.196:6789/0 conn(0x7f00c00b35f0 legacy=0x7f00c0071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.526+0000 7f00c5a00640 1 -- 172.21.15.86:0/1787545728 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c00b40e0 con 0x7f00c0144a90 2024-03-22T04:21:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c5a00640 1 --2- 172.21.15.86:0/1787545728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c0144a90 0x7f00c00727b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f00b40097c0 tx=0x7f00b401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c118c1ac7c4697ec server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00bfe00640 1 -- 172.21.15.86:0/1787545728 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40a7070 con 0x7f00c0144a90 2024-03-22T04:21:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00bfe00640 1 -- 172.21.15.86:0/1787545728 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b40b0070 con 0x7f00c0144a90 2024-03-22T04:21:48.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00bfe00640 1 -- 172.21.15.86:0/1787545728 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40ab890 con 0x7f00c0144a90 2024-03-22T04:21:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 -- 172.21.15.86:0/1787545728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c0144a90 msgr2=0x7f00c00727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 --2- 172.21.15.86:0/1787545728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c0144a90 0x7f00c00727b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f00b40097c0 tx=0x7f00b401c470 comp rx=0 tx=0).stop 2024-03-22T04:21:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 -- 172.21.15.86:0/1787545728 shutdown_connections 2024-03-22T04:21:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 --2- 172.21.15.86:0/1787545728 >> v2:172.21.15.196:3300/0 conn(0x7f00c0072cf0 0x7f00c00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 --2- 172.21.15.86:0/1787545728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c0144a90 0x7f00c00727b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 -- 172.21.15.86:0/1787545728 >> 172.21.15.86:0/1787545728 conn(0x7f00c006b660 msgr2=0x7f00c006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 -- 172.21.15.86:0/1787545728 shutdown_connections 2024-03-22T04:21:48.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 -- 172.21.15.86:0/1787545728 wait complete. 2024-03-22T04:21:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 Processor -- start 2024-03-22T04:21:48.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.530+0000 7f00c8400640 1 -- start start 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c0072cf0 0x7f00c0155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c00b35f0 0x7f00c00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0144a90 0x7f00c00ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00c00b3b80 con 0x7f00c0072cf0 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00c00b3a00 con 0x7f00c0144a90 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f00c006d530 con 0x7f00c00b35f0 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c0072cf0 0x7f00c0155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c0072cf0 0x7f00c0155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51350/0 (socket says 172.21.15.86:51350) 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 -- 172.21.15.86:0/976155823 learned_addr learned my addr 172.21.15.86:0/976155823 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5000640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c00b35f0 0x7f00c00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c6400640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0144a90 0x7f00c00ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 -- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c00b35f0 msgr2=0x7f00c00ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f00c00b35f0 0x7f00c00ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 -- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0144a90 msgr2=0x7f00c00ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00c0144a90 0x7f00c00ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 -- 172.21.15.86:0/976155823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c0159710 con 0x7f00c0072cf0 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c5a00640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c0072cf0 0x7f00c0155810 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f00b40097c0 tx=0x7f00b4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40abc30 con 0x7f00c0072cf0 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b40abdd0 con 0x7f00c0072cf0 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40bf690 con 0x7f00c0072cf0 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c01599f0 con 0x7f00c0072cf0 2024-03-22T04:21:48.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c0159fb0 con 0x7f00c0072cf0 2024-03-22T04:21:48.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f00c0074580 con 0x7f00c0072cf0 2024-03-22T04:21:48.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.534+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f00b40bf830 con 0x7f00c0072cf0 2024-03-22T04:21:48.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00bd600640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f00a0067b20 0x7f00a0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f00b4120620 con 0x7f00c0072cf0 2024-03-22T04:21:48.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00c5000640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f00a0067b20 0x7f00a0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00bd600640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f00a006e100 0x7f00a0070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00b40b3e90 con 0x7f00a006e100 2024-03-22T04:21:48.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7f00b4120ab0 con 0x7f00c0072cf0 2024-03-22T04:21:48.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00c6400640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f00a006e100 0x7f00a0070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00c5000640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f00a0067b20 0x7f00a0069fe0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f00c00ccec0 tx=0x7f00ac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00c6400640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f00a006e100 0x7f00a0070540 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.538+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 <== osd.4 v2:172.21.15.196:6800/655314492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f00b40b3e90 con 0x7f00a006e100 2024-03-22T04:21:48.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.550+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 --> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f00c00b5220 con 0x7f00a006e100 2024-03-22T04:21:48.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.550+0000 7f00bd600640 1 -- 172.21.15.86:0/976155823 <== osd.4 v2:172.21.15.196:6800/655314492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f00c00b5220 con 0x7f00a006e100 2024-03-22T04:21:48.550 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:48.550 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-22T04:21:48.550 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f00a006e100 msgr2=0x7f00a0070540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.196:6800/655314492,v1:172.21.15.196:6801/655314492] conn(0x7f00a006e100 0x7f00a0070540 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f00a0067b20 msgr2=0x7f00a0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f00a0067b20 0x7f00a0069fe0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f00c00ccec0 tx=0x7f00ac009190 comp rx=0 tx=0).stop 2024-03-22T04:21:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c0072cf0 msgr2=0x7f00c0155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 --2- 172.21.15.86:0/976155823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00c0072cf0 0x7f00c0155810 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f00b40097c0 tx=0x7f00b4019210 comp rx=0 tx=0).stop 2024-03-22T04:21:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c6400640 1 -- 172.21.15.86:0/976155823 reap_dead start 2024-03-22T04:21:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 shutdown_connections 2024-03-22T04:21:48.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 >> 172.21.15.86:0/976155823 conn(0x7f00c006b660 msgr2=0x7f00c00b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:48.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 shutdown_connections 2024-03-22T04:21:48.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.554+0000 7f00c8400640 1 -- 172.21.15.86:0/976155823 wait complete. 2024-03-22T04:21:48.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-22T04:21:48.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.706+0000 7f1643000640 1 Processor -- start 2024-03-22T04:21:48.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.706+0000 7f1643000640 1 -- start start 2024-03-22T04:21:48.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.706+0000 7f1643000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f163c147880 0x7f163c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.706+0000 7f1643000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f163c0b4a60 0x7f163c15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.706+0000 7f1643000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f163c15ea90 con 0x7f163c06cec0 2024-03-22T04:21:48.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.706+0000 7f1643000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f163c0b6320 con 0x7f163c0b4a60 2024-03-22T04:21:48.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.706+0000 7f1643000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f163c06d4f0 con 0x7f163c147880 2024-03-22T04:21:48.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f1641000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f163c0b4a60 0x7f163c15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f1641000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f163c0b4a60 0x7f163c15ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51364/0 (socket says 172.21.15.86:51364) 2024-03-22T04:21:48.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f1641000640 1 -- 172.21.15.86:0/715803985 learned_addr learned my addr 172.21.15.86:0/715803985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:48.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f163be00640 1 --2- 172.21.15.86:0/715803985 >> v2:172.21.15.196:3300/0 conn(0x7f163c147880 0x7f163c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f1641000640 1 -- 172.21.15.86:0/715803985 >> v2:172.21.15.196:3300/0 conn(0x7f163c147880 msgr2=0x7f163c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:48.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f1641000640 1 --2- 172.21.15.86:0/715803985 >> v2:172.21.15.196:3300/0 conn(0x7f163c147880 0x7f163c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f1641000640 1 -- 172.21.15.86:0/715803985 >> v1:172.21.15.196:6789/0 conn(0x7f163c06cec0 legacy=0x7f163c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:48.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f1641000640 1 -- 172.21.15.86:0/715803985 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c06d7d0 con 0x7f163c0b4a60 2024-03-22T04:21:48.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f1641000640 1 --2- 172.21.15.86:0/715803985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15ef40 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f1634009870 tx=0x7f163401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b802f088b2a151 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f163b400640 1 -- 172.21.15.86:0/715803985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340b0070 con 0x7f163c0b4a60 2024-03-22T04:21:48.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f163b400640 1 -- 172.21.15.86:0/715803985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16340ab600 con 0x7f163c0b4a60 2024-03-22T04:21:48.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.710+0000 7f163b400640 1 -- 172.21.15.86:0/715803985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340ab9d0 con 0x7f163c0b4a60 2024-03-22T04:21:48.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- 172.21.15.86:0/715803985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 msgr2=0x7f163c15ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 --2- 172.21.15.86:0/715803985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15ef40 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f1634009870 tx=0x7f163401c520 comp rx=0 tx=0).stop 2024-03-22T04:21:48.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- 172.21.15.86:0/715803985 shutdown_connections 2024-03-22T04:21:48.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 --2- 172.21.15.86:0/715803985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15ef40 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 --2- 172.21.15.86:0/715803985 >> v2:172.21.15.196:3300/0 conn(0x7f163c147880 0x7f163c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- 172.21.15.86:0/715803985 >> 172.21.15.86:0/715803985 conn(0x7f163c06ba60 msgr2=0x7f163c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:48.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- 172.21.15.86:0/715803985 shutdown_connections 2024-03-22T04:21:48.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- 172.21.15.86:0/715803985 wait complete. 2024-03-22T04:21:48.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 Processor -- start 2024-03-22T04:21:48.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- start start 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f163c06cec0 0x7f163c1679f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f163c147880 0x7f163c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f163c15fe10 con 0x7f163c0b4a60 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f163c15fc90 con 0x7f163c06cec0 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1643000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f163c15ff90 con 0x7f163c147880 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1633400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1633400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51370/0 (socket says 172.21.15.86:51370) 2024-03-22T04:21:48.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1633400640 1 -- 172.21.15.86:0/436375316 learned_addr learned my addr 172.21.15.86:0/436375316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f163be00640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f163c06cec0 0x7f163c1679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1641000640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f163c147880 0x7f163c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1633400640 1 -- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f163c147880 msgr2=0x7f163c0cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1633400640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f163c147880 0x7f163c0cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1633400640 1 -- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f163c06cec0 msgr2=0x7f163c1679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1633400640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f163c06cec0 0x7f163c1679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f1633400640 1 -- 172.21.15.86:0/436375316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c141750 con 0x7f163c0b4a60 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.714+0000 7f163be00640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f163c06cec0 0x7f163c1679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1633400640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15df30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f162400c420 tx=0x7f162400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1638c00640 1 -- 172.21.15.86:0/436375316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240052d0 con 0x7f163c0b4a60 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1638c00640 1 -- 172.21.15.86:0/436375316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1624005470 con 0x7f163c0b4a60 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c141a30 con 0x7f163c0b4a60 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1638c00640 1 -- 172.21.15.86:0/436375316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624007260 con 0x7f163c0b4a60 2024-03-22T04:21:48.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c141ff0 con 0x7f163c0b4a60 2024-03-22T04:21:48.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163c068650 con 0x7f163c0b4a60 2024-03-22T04:21:48.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1638c00640 1 -- 172.21.15.86:0/436375316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1624007400 con 0x7f163c0b4a60 2024-03-22T04:21:48.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1638c00640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1618067b20 0x7f1618069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:48.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f1638c00640 1 -- 172.21.15.86:0/436375316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f16240853b0 con 0x7f163c0b4a60 2024-03-22T04:21:48.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f163be00640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1618067b20 0x7f1618069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:48.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.718+0000 7f163be00640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1618067b20 0x7f1618069fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f162c003fd0 tx=0x7f162c0039e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:48.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.722+0000 7f1638c00640 1 -- 172.21.15.86:0/436375316 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1624055de0 con 0x7f163c0b4a60 2024-03-22T04:21:48.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f163c072220 con 0x7f1618067b20 2024-03-22T04:21:48.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1638c00640 1 -- 172.21.15.86:0/436375316 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f163c072220 con 0x7f1618067b20 2024-03-22T04:21:48.880 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 4 to deep-scrub 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1618067b20 msgr2=0x7f1618069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1618067b20 0x7f1618069fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f162c003fd0 tx=0x7f162c0039e0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 msgr2=0x7f163c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15df30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f162400c420 tx=0x7f162400c8e0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 shutdown_connections 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1618067b20 0x7f1618069fe0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f163c147880 0x7f163c0cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c0b4a60 0x7f163c15df30 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 --2- 172.21.15.86:0/436375316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f163c06cec0 0x7f163c1679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:48.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 >> 172.21.15.86:0/436375316 conn(0x7f163c06ba60 msgr2=0x7f163c0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 shutdown_connections 2024-03-22T04:21:48.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:48.882+0000 7f1643000640 1 -- 172.21.15.86:0/436375316 wait complete. 2024-03-22T04:21:48.900 INFO:tasks.ceph:Scrubbing osd.5 2024-03-22T04:21:48.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-22T04:21:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc9e00640 1 Processor -- start 2024-03-22T04:21:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc9e00640 1 -- start start 2024-03-22T04:21:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc9e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 0x7fcdc40727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcdc40b37c0 0x7fcdc40ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc9e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc4144e70 con 0x7fcdc4072cf0 2024-03-22T04:21:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc9e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcdc4073150 con 0x7fcdc40b37c0 2024-03-22T04:21:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc9e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc40b3e00 con 0x7fcdc4144a90 2024-03-22T04:21:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcdc40b37c0 0x7fcdc40ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcdc40b37c0 0x7fcdc40ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51376/0 (socket says 172.21.15.86:51376) 2024-03-22T04:21:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc3400640 1 -- 172.21.15.86:0/1797191613 learned_addr learned my addr 172.21.15.86:0/1797191613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.042+0000 7fcdc2a00640 1 --2- 172.21.15.86:0/1797191613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 0x7fcdc40727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc3400640 1 -- 172.21.15.86:0/1797191613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 msgr2=0x7fcdc40727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc3400640 1 --2- 172.21.15.86:0/1797191613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 0x7fcdc40727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc3400640 1 -- 172.21.15.86:0/1797191613 >> v1:172.21.15.196:6789/0 conn(0x7fcdc4072cf0 legacy=0x7fcdc40b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc3400640 1 -- 172.21.15.86:0/1797191613 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc40b40e0 con 0x7fcdc40b37c0 2024-03-22T04:21:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc2a00640 1 --2- 172.21.15.86:0/1797191613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 0x7fcdc40727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc3400640 1 --2- 172.21.15.86:0/1797191613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 0x7fcdc40ce3d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00d150 tx=0x7fcdbc01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f9f9c0e7125718 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc1600640 1 -- 172.21.15.86:0/1797191613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00b2d0 con 0x7fcdc40b37c0 2024-03-22T04:21:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc1600640 1 -- 172.21.15.86:0/1797191613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdbc00b470 con 0x7fcdc40b37c0 2024-03-22T04:21:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc1600640 1 -- 172.21.15.86:0/1797191613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0a7c10 con 0x7fcdc40b37c0 2024-03-22T04:21:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1797191613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 msgr2=0x7fcdc40ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.046+0000 7fcdc9e00640 1 --2- 172.21.15.86:0/1797191613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 0x7fcdc40ce3d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00d150 tx=0x7fcdbc01ec30 comp rx=0 tx=0).stop 2024-03-22T04:21:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1797191613 shutdown_connections 2024-03-22T04:21:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 --2- 172.21.15.86:0/1797191613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 0x7fcdc40ce3d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 --2- 172.21.15.86:0/1797191613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 0x7fcdc40727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1797191613 >> 172.21.15.86:0/1797191613 conn(0x7fcdc406b660 msgr2=0x7fcdc406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1797191613 shutdown_connections 2024-03-22T04:21:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1797191613 wait complete. 2024-03-22T04:21:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 Processor -- start 2024-03-22T04:21:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 -- start start 2024-03-22T04:21:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc4072cf0 0x7fcdc415a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 0x7fcdc4139230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 0x7fcdc40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc40cf2a0 con 0x7fcdc40b37c0 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc40cf120 con 0x7fcdc4072cf0 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc9e00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc40cf420 con 0x7fcdc4144a90 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 0x7fcdc4139230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 0x7fcdc4139230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51382/0 (socket says 172.21.15.86:51382) 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2000640 1 -- 172.21.15.86:0/1928205682 learned_addr learned my addr 172.21.15.86:0/1928205682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2a00640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc4072cf0 0x7fcdc415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc3400640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 0x7fcdc40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2000640 1 -- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 msgr2=0x7fcdc40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2000640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc4144a90 0x7fcdc40ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2000640 1 -- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc4072cf0 msgr2=0x7fcdc415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2000640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc4072cf0 0x7fcdc415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.050+0000 7fcdc2000640 1 -- 172.21.15.86:0/1928205682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc415adb0 con 0x7fcdc40b37c0 2024-03-22T04:21:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdc2000640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 0x7fcdc4139230 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00c420 tx=0x7fcdac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0052d0 con 0x7fcdc40b37c0 2024-03-22T04:21:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac005470 con 0x7fcdc40b37c0 2024-03-22T04:21:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc413b750 con 0x7fcdc40b37c0 2024-03-22T04:21:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac007260 con 0x7fcdc40b37c0 2024-03-22T04:21:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc4137b80 con 0x7fcdc40b37c0 2024-03-22T04:21:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcdc4074580 con 0x7fcdc40b37c0 2024-03-22T04:21:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcdac007400 con 0x7fcdc40b37c0 2024-03-22T04:21:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcda0067b20 0x7fcda0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7fcdac085340 con 0x7fcdc40b37c0 2024-03-22T04:21:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdc2a00640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcda0067b20 0x7fcda0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7fcda006e100 0x7fcda0070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcdac00ae50 con 0x7fcda006e100 2024-03-22T04:21:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7fcdac08c050 con 0x7fcdc40b37c0 2024-03-22T04:21:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdc3400640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7fcda006e100 0x7fcda0070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.054+0000 7fcdc2a00640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcda0067b20 0x7fcda0069fe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40045f0 tx=0x7fcdb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.058+0000 7fcdc3400640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7fcda006e100 0x7fcda0070540 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.058+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 <== osd.5 v2:172.21.15.196:6808/2743139076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcdac00ae50 con 0x7fcda006e100 2024-03-22T04:21:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 --> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcdc4072ad0 con 0x7fcda006e100 2024-03-22T04:21:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdbb400640 1 -- 172.21.15.86:0/1928205682 <== osd.5 v2:172.21.15.196:6808/2743139076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcdc4072ad0 con 0x7fcda006e100 2024-03-22T04:21:49.069 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-22T04:21:49.069 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7fcda006e100 msgr2=0x7fcda0070540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.196:6808/2743139076,v1:172.21.15.196:6809/2743139076] conn(0x7fcda006e100 0x7fcda0070540 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcda0067b20 msgr2=0x7fcda0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcda0067b20 0x7fcda0069fe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40045f0 tx=0x7fcdb4008040 comp rx=0 tx=0).stop 2024-03-22T04:21:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 msgr2=0x7fcdc4139230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 --2- 172.21.15.86:0/1928205682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc40b37c0 0x7fcdc4139230 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00c420 tx=0x7fcdac00c8e0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc3400640 1 -- 172.21.15.86:0/1928205682 reap_dead start 2024-03-22T04:21:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 shutdown_connections 2024-03-22T04:21:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 >> 172.21.15.86:0/1928205682 conn(0x7fcdc406b660 msgr2=0x7fcdc4143cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 shutdown_connections 2024-03-22T04:21:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.070+0000 7fcdc9e00640 1 -- 172.21.15.86:0/1928205682 wait complete. 2024-03-22T04:21:49.087 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-22T04:21:49.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb225600640 1 Processor -- start 2024-03-22T04:21:49.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb225600640 1 -- start start 2024-03-22T04:21:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb225600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb220147880 0x7fb22006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb225600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2200b4a60 0x7fb22015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb225600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb22015ea90 con 0x7fb22006cec0 2024-03-22T04:21:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb225600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb2200b6320 con 0x7fb2200b4a60 2024-03-22T04:21:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb225600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb22006d4f0 con 0x7fb220147880 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb21f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2200b4a60 0x7fb22015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb21ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb220147880 0x7fb22006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb21e000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb22006cec0 0x7fb2200b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.86:39240/0 (socket says 172.21.15.86:39240) 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb21f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2200b4a60 0x7fb22015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.86:43126/0 (socket says 172.21.15.86:43126) 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.226+0000 7fb21e000640 1 -- 172.21.15.86:0/719367637 learned_addr learned my addr 172.21.15.86:0/719367637 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb21f400640 1 -- 172.21.15.86:0/719367637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb220147880 msgr2=0x7fb22006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb21f400640 1 --2- 172.21.15.86:0/719367637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb220147880 0x7fb22006c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb21f400640 1 -- 172.21.15.86:0/719367637 >> v1:172.21.15.196:6789/0 conn(0x7fb22006cec0 legacy=0x7fb2200b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb21f400640 1 -- 172.21.15.86:0/719367637 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22006d7d0 con 0x7fb2200b4a60 2024-03-22T04:21:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb21f400640 1 --2- 172.21.15.86:0/719367637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 0x7fb22015ef40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb2180097c0 tx=0x7fb21801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a555007b49d3cb55 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb21d600640 1 -- 172.21.15.86:0/719367637 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180b0070 con 0x7fb2200b4a60 2024-03-22T04:21:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb21d600640 1 -- 172.21.15.86:0/719367637 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2180ab600 con 0x7fb2200b4a60 2024-03-22T04:21:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb21d600640 1 -- 172.21.15.86:0/719367637 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180ab9d0 con 0x7fb2200b4a60 2024-03-22T04:21:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb225600640 1 -- 172.21.15.86:0/719367637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 msgr2=0x7fb22015ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb225600640 1 --2- 172.21.15.86:0/719367637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 0x7fb22015ef40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb2180097c0 tx=0x7fb21801c4a0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb225600640 1 -- 172.21.15.86:0/719367637 shutdown_connections 2024-03-22T04:21:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb225600640 1 --2- 172.21.15.86:0/719367637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 0x7fb22015ef40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb225600640 1 --2- 172.21.15.86:0/719367637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb220147880 0x7fb22006c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.230+0000 7fb225600640 1 -- 172.21.15.86:0/719367637 >> 172.21.15.86:0/719367637 conn(0x7fb22006ba60 msgr2=0x7fb22006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:49.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 -- 172.21.15.86:0/719367637 shutdown_connections 2024-03-22T04:21:49.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 -- 172.21.15.86:0/719367637 wait complete. 2024-03-22T04:21:49.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 Processor -- start 2024-03-22T04:21:49.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 -- start start 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006cec0 0x7fb220146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 0x7fb220138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220147880 0x7fb220143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb22015fe10 con 0x7fb220147880 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb22015fc90 con 0x7fb2200b4a60 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb225600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb22015ff90 con 0x7fb22006cec0 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220147880 0x7fb220143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21ea00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006cec0 0x7fb220146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220147880 0x7fb220143b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51398/0 (socket says 172.21.15.86:51398) 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 -- 172.21.15.86:0/2859625326 learned_addr learned my addr 172.21.15.86:0/2859625326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:49.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21e000640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 0x7fb220138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 -- 172.21.15.86:0/2859625326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006cec0 msgr2=0x7fb220146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006cec0 0x7fb220146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 -- 172.21.15.86:0/2859625326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 msgr2=0x7fb220138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 0x7fb220138810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 -- 172.21.15.86:0/2859625326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb220138d50 con 0x7fb220147880 2024-03-22T04:21:49.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.234+0000 7fb21f400640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220147880 0x7fb220143b40 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fb21801a0d0 tx=0x7fb218019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb216a00640 1 -- 172.21.15.86:0/2859625326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180abe60 con 0x7fb220147880 2024-03-22T04:21:49.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb216a00640 1 -- 172.21.15.86:0/2859625326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2180b9070 con 0x7fb220147880 2024-03-22T04:21:49.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb216a00640 1 -- 172.21.15.86:0/2859625326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180b0040 con 0x7fb220147880 2024-03-22T04:21:49.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2201456f0 con 0x7fb220147880 2024-03-22T04:21:49.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb220145cb0 con 0x7fb220147880 2024-03-22T04:21:49.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb220068650 con 0x7fb220147880 2024-03-22T04:21:49.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb216a00640 1 -- 172.21.15.86:0/2859625326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fb2180b4650 con 0x7fb220147880 2024-03-22T04:21:49.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb216a00640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb216a00640 1 -- 172.21.15.86:0/2859625326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7fb21811f930 con 0x7fb220147880 2024-03-22T04:21:49.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb21ea00640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.238+0000 7fb21ea00640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb210011340 tx=0x7fb21000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.242+0000 7fb216a00640 1 -- 172.21.15.86:0/2859625326 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb218016090 con 0x7fb220147880 2024-03-22T04:21:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.398+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb2201612a0 con 0x7fb200067b20 2024-03-22T04:21:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.398+0000 7fb216a00640 1 -- 172.21.15.86:0/2859625326 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb2201612a0 con 0x7fb200067b20 2024-03-22T04:21:49.397 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 5 to deep-scrub 2024-03-22T04:21:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 msgr2=0x7fb200069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb210011340 tx=0x7fb21000b040 comp rx=0 tx=0).stop 2024-03-22T04:21:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220147880 msgr2=0x7fb220143b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220147880 0x7fb220143b40 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fb21801a0d0 tx=0x7fb218019240 comp rx=0 tx=0).stop 2024-03-22T04:21:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 shutdown_connections 2024-03-22T04:21:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fb200067b20 0x7fb200069fe0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2200b4a60 0x7fb220138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220147880 0x7fb220143b40 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 --2- 172.21.15.86:0/2859625326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22006cec0 0x7fb220146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 >> 172.21.15.86:0/2859625326 conn(0x7fb22006ba60 msgr2=0x7fb2201626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 shutdown_connections 2024-03-22T04:21:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.402+0000 7fb225600640 1 -- 172.21.15.86:0/2859625326 wait complete. 2024-03-22T04:21:49.418 INFO:tasks.ceph:Scrubbing osd.6 2024-03-22T04:21:49.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-22T04:21:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.558+0000 7f68dac00640 1 Processor -- start 2024-03-22T04:21:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.558+0000 7f68dac00640 1 -- start start 2024-03-22T04:21:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.558+0000 7f68dac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d4072ec0 0x7f68d40b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.558+0000 7f68dac00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f68d40b35f0 0x7f68d40ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.558+0000 7f68dac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d4144e70 con 0x7f68d4144a90 2024-03-22T04:21:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.558+0000 7f68dac00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f68d40b3e00 con 0x7f68d40b35f0 2024-03-22T04:21:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.558+0000 7f68dac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f68d40b40e0 con 0x7f68d4072ec0 2024-03-22T04:21:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f68d4144a90 0x7f68d40727b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45976/0 (socket says 172.21.15.86:45976) 2024-03-22T04:21:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3e00640 1 -- 172.21.15.86:0/2110181409 learned_addr learned my addr 172.21.15.86:0/2110181409 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d8c00640 1 --2- 172.21.15.86:0/2110181409 >> v2:172.21.15.196:3300/0 conn(0x7f68d40b35f0 0x7f68d40ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3400640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d4072ec0 0x7f68d40b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673038375 0 0) 0x7f68d4144e70 con 0x7f68d4144a90 2024-03-22T04:21:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68b8005740 con 0x7f68d4144a90 2024-03-22T04:21:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1174799767 0 0) 0x7f68b8005740 con 0x7f68d4144a90 2024-03-22T04:21:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d4072ec0 msgr2=0x7f68d40b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d4072ec0 0x7f68d40b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 >> v2:172.21.15.196:3300/0 conn(0x7f68d40b35f0 msgr2=0x7f68d40ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 --2- 172.21.15.86:0/2110181409 >> v2:172.21.15.196:3300/0 conn(0x7f68d40b35f0 0x7f68d40ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d40cdbd0 con 0x7f68d4144a90 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7f68c4001680 con 0x7f68d4144a90 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 >> v1:172.21.15.86:6789/0 conn(0x7f68d4144a90 legacy=0x7f68d40727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b8005bd0 0x7f68b8005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68b8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d4072ec0 0x7f68b80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68d40b3e00 con 0x7f68b8006a20 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68d40b40e0 con 0x7f68b8005bd0 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f68b800dbe0 con 0x7f68d4072ec0 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3400640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68b8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3e00640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b8005bd0 0x7f68b8005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d8c00640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d4072ec0 0x7f68b80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3400640 1 -- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d4072ec0 msgr2=0x7f68b80073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3400640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d4072ec0 0x7f68b80073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3400640 1 -- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b8005bd0 msgr2=0x7f68b8005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3400640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b8005bd0 0x7f68b8005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.562+0000 7f68d3400640 1 -- 172.21.15.86:0/2110181409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b8005740 con 0x7f68b8006a20 2024-03-22T04:21:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68d3400640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68b8006ea0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009ae0 tx=0x7f68bc01c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c3201f0bd8eb1a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b1070 con 0x7f68b8006a20 2024-03-22T04:21:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc0a4ab0 con 0x7f68b8006a20 2024-03-22T04:21:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68d2a00640 1 -- 172.21.15.86:0/2110181409 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0096b0 con 0x7f68b8006a20 2024-03-22T04:21:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68dac00640 1 -- 172.21.15.86:0/2110181409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 msgr2=0x7f68b8006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68dac00640 1 --2- 172.21.15.86:0/2110181409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68b8006ea0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009ae0 tx=0x7f68bc01c750 comp rx=0 tx=0).stop 2024-03-22T04:21:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68d8c00640 1 -- 172.21.15.86:0/2110181409 reap_dead start 2024-03-22T04:21:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68dac00640 1 -- 172.21.15.86:0/2110181409 shutdown_connections 2024-03-22T04:21:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68dac00640 1 -- 172.21.15.86:0/2110181409 >> 172.21.15.86:0/2110181409 conn(0x7f68d406b660 msgr2=0x7f68d406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68dac00640 1 -- 172.21.15.86:0/2110181409 shutdown_connections 2024-03-22T04:21:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68dac00640 1 -- 172.21.15.86:0/2110181409 wait complete. 2024-03-22T04:21:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68dac00640 1 Processor -- start 2024-03-22T04:21:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.566+0000 7f68dac00640 1 -- start start 2024-03-22T04:21:49.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68d415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d40c74c0 0x7f68d40c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d413fe70 0x7f68d41402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68d40b3a20 con 0x7f68b8006a20 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68d40cd880 con 0x7f68d413fe70 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f68d40b3ba0 con 0x7f68d40c74c0 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68d415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68d415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51430/0 (socket says 172.21.15.86:51430) 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 -- 172.21.15.86:0/242981440 learned_addr learned my addr 172.21.15.86:0/242981440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d8c00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d413fe70 0x7f68d41402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3400640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d40c74c0 0x7f68d40c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 -- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d40c74c0 msgr2=0x7f68d40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68d40c74c0 0x7f68d40c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 -- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d413fe70 msgr2=0x7f68d41402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d413fe70 0x7f68d41402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 -- 172.21.15.86:0/242981440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d40c7f80 con 0x7f68b8006a20 2024-03-22T04:21:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d8c00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68d413fe70 0x7f68d41402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:49.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68d3e00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68d415a7f0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f68c4012340 tx=0x7f68c4012800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c401c070 con 0x7f68b8006a20 2024-03-22T04:21:49.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c4014750 con 0x7f68b8006a20 2024-03-22T04:21:49.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4014b40 con 0x7f68b8006a20 2024-03-22T04:21:49.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d40c8260 con 0x7f68b8006a20 2024-03-22T04:21:49.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d40c6320 con 0x7f68b8006a20 2024-03-22T04:21:49.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68d40c8780 con 0x7f68b8006a20 2024-03-22T04:21:49.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.570+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f68c4014ce0 con 0x7f68b8006a20 2024-03-22T04:21:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68cbe00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f68b0067b50 0x7f68b006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f68c4088710 con 0x7f68b8006a20 2024-03-22T04:21:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68d3400640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f68b0067b50 0x7f68b006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68cbe00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f68b006e130 0x7f68b0070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68c401bec0 con 0x7f68b006e130 2024-03-22T04:21:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7f68c408f050 con 0x7f68b8006a20 2024-03-22T04:21:49.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68d8c00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f68b006e130 0x7f68b0070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68d3400640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f68b0067b50 0x7f68b006a010 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0ae000 tx=0x7f68bc0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68d8c00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f68b006e130 0x7f68b0070570 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.574+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 <== osd.6 v2:172.21.15.196:6816/1648471396 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68c401bec0 con 0x7f68b006e130 2024-03-22T04:21:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.586+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 --> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f68d4074580 con 0x7f68b006e130 2024-03-22T04:21:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.586+0000 7f68cbe00640 1 -- 172.21.15.86:0/242981440 <== osd.6 v2:172.21.15.196:6816/1648471396 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f68d4074580 con 0x7f68b006e130 2024-03-22T04:21:49.586 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:49.586 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-22T04:21:49.586 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f68b006e130 msgr2=0x7f68b0070570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.196:6816/1648471396,v1:172.21.15.196:6818/1648471396] conn(0x7f68b006e130 0x7f68b0070570 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f68b0067b50 msgr2=0x7f68b006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f68b0067b50 0x7f68b006a010 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0ae000 tx=0x7f68bc0046d0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 msgr2=0x7f68d415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 --2- 172.21.15.86:0/242981440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68b8006a20 0x7f68d415a7f0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f68c4012340 tx=0x7f68c4012800 comp rx=0 tx=0).stop 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68d8c00640 1 -- 172.21.15.86:0/242981440 reap_dead start 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 shutdown_connections 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 >> 172.21.15.86:0/242981440 conn(0x7f68d406b660 msgr2=0x7f68d40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 shutdown_connections 2024-03-22T04:21:49.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.590+0000 7f68dac00640 1 -- 172.21.15.86:0/242981440 wait complete. 2024-03-22T04:21:49.605 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-22T04:21:49.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c7c00640 1 Processor -- start 2024-03-22T04:21:49.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c7c00640 1 -- start start 2024-03-22T04:21:49.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c7c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff3c00730d0 0x7ff3c00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c7c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3c00cc3e0 0x7ff3c00ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c7c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c0071920 con 0x7ff3c015b3f0 2024-03-22T04:21:49.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c7c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c00716d0 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c7c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c006d990 con 0x7ff3c00730d0 2024-03-22T04:21:49.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3c00cc3e0 0x7ff3c00ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3c00cc3e0 0x7ff3c00ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51440/0 (socket says 172.21.15.86:51440) 2024-03-22T04:21:49.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c5c00640 1 -- 172.21.15.86:0/986642547 learned_addr learned my addr 172.21.15.86:0/986642547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:49.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/986642547 >> v2:172.21.15.196:3300/0 conn(0x7ff3c00730d0 0x7ff3c00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c5c00640 1 -- 172.21.15.86:0/986642547 >> v2:172.21.15.196:3300/0 conn(0x7ff3c00730d0 msgr2=0x7ff3c00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:49.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c5c00640 1 --2- 172.21.15.86:0/986642547 >> v2:172.21.15.196:3300/0 conn(0x7ff3c00730d0 0x7ff3c00ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c5c00640 1 -- 172.21.15.86:0/986642547 >> v1:172.21.15.196:6789/0 conn(0x7ff3c015b3f0 legacy=0x7ff3c00729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.746+0000 7ff3c5c00640 1 -- 172.21.15.86:0/986642547 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c0071c00 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c5c00640 1 --2- 172.21.15.86:0/986642547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c00ae370 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ff3c006f270 tx=0x7ff3b801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161f2e35d9a955f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3bf400640 1 -- 172.21.15.86:0/986642547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b800b2d0 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3bf400640 1 -- 172.21.15.86:0/986642547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b800b470 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3bf400640 1 -- 172.21.15.86:0/986642547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80a7bd0 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 -- 172.21.15.86:0/986642547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 msgr2=0x7ff3c00ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/986642547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c00ae370 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ff3c006f270 tx=0x7ff3b801ebf0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 -- 172.21.15.86:0/986642547 shutdown_connections 2024-03-22T04:21:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/986642547 >> v2:172.21.15.196:3300/0 conn(0x7ff3c00730d0 0x7ff3c00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/986642547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c00ae370 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 -- 172.21.15.86:0/986642547 >> 172.21.15.86:0/986642547 conn(0x7ff3c00b1df0 msgr2=0x7ff3c00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 -- 172.21.15.86:0/986642547 shutdown_connections 2024-03-22T04:21:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 -- 172.21.15.86:0/986642547 wait complete. 2024-03-22T04:21:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 Processor -- start 2024-03-22T04:21:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.750+0000 7ff3c7c00640 1 -- start start 2024-03-22T04:21:49.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3c00730d0 0x7ff3c0099300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c014f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c015b3f0 0x7ff3c014fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c006e320 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c006e1a0 con 0x7ff3c015b3f0 2024-03-22T04:21:49.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c006e4a0 con 0x7ff3c00730d0 2024-03-22T04:21:49.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c014f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c014f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51454/0 (socket says 172.21.15.86:51454) 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 -- 172.21.15.86:0/83298116 learned_addr learned my addr 172.21.15.86:0/83298116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c5c00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c015b3f0 0x7ff3c014fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c5200640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3c00730d0 0x7ff3c0099300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 -- 172.21.15.86:0/83298116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3c00730d0 msgr2=0x7ff3c0099300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3c00730d0 0x7ff3c0099300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 -- 172.21.15.86:0/83298116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c015b3f0 msgr2=0x7ff3c014fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c015b3f0 0x7ff3c014fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 -- 172.21.15.86:0/83298116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c0152b10 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c014f530 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00e930 tx=0x7ff3ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bcc00640 1 -- 172.21.15.86:0/83298116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac017070 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bcc00640 1 -- 172.21.15.86:0/83298116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ac00ca00 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c0152df0 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bcc00640 1 -- 172.21.15.86:0/83298116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00cdb0 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c01533b0 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c0068650 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.754+0000 7ff3bcc00640 1 -- 172.21.15.86:0/83298116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff3ac01a070 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.758+0000 7ff3bcc00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff3a0067b20 0x7ff3a0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:49.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.758+0000 7ff3bcc00640 1 -- 172.21.15.86:0/83298116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7ff3ac086230 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.758+0000 7ff3c5200640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff3a0067b20 0x7ff3a0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:49.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.758+0000 7ff3c5200640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff3a0067b20 0x7ff3a0069fe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00b1290 tx=0x7ff3b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:49.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.762+0000 7ff3bcc00640 1 -- 172.21.15.86:0/83298116 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff3ac056db0 con 0x7ff3c00cc3e0 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.918+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7ff3c006c670 con 0x7ff3a0067b20 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.918+0000 7ff3bcc00640 1 -- 172.21.15.86:0/83298116 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff3c006c670 con 0x7ff3a0067b20 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 6 to deep-scrub 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff3a0067b20 msgr2=0x7ff3a0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff3a0067b20 0x7ff3a0069fe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00b1290 tx=0x7ff3b0009040 comp rx=0 tx=0).stop 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 msgr2=0x7ff3c014f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c014f530 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00e930 tx=0x7ff3ac00edf0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 shutdown_connections 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff3a0067b20 0x7ff3a0069fe0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c015b3f0 0x7ff3c014fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c00cc3e0 0x7ff3c014f530 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 --2- 172.21.15.86:0/83298116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3c00730d0 0x7ff3c0099300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 >> 172.21.15.86:0/83298116 conn(0x7ff3c00b1df0 msgr2=0x7ff3c0157330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 shutdown_connections 2024-03-22T04:21:49.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:49.922+0000 7ff3c7c00640 1 -- 172.21.15.86:0/83298116 wait complete. 2024-03-22T04:21:49.938 INFO:tasks.ceph:Scrubbing osd.7 2024-03-22T04:21:49.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-22T04:21:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd97a00640 1 Processor -- start 2024-03-22T04:21:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd97a00640 1 -- start start 2024-03-22T04:21:50.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd97a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd9009b180 0x7fcd9006d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd97a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd900d05c0 con 0x7fcd9006c980 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd97a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd900d08a0 con 0x7fcd9015e670 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd97a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9015ea50 con 0x7fcd9009b180 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd95a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd9009b180 0x7fcd9006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd8fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcd9006c980 0x7fcd9009aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45978/0 (socket says 172.21.15.86:45978) 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd8fe00640 1 -- 172.21.15.86:0/167787837 learned_addr learned my addr 172.21.15.86:0/167787837 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd95a00640 1 -- 172.21.15.86:0/167787837 >> v1:172.21.15.196:6789/0 conn(0x7fcd9015e670 legacy=0x7fcd900d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd95a00640 1 -- 172.21.15.86:0/167787837 >> v1:172.21.15.86:6789/0 conn(0x7fcd9006c980 legacy=0x7fcd9009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.082+0000 7fcd95a00640 1 -- 172.21.15.86:0/167787837 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd900d0b80 con 0x7fcd9009b180 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd95a00640 1 --2- 172.21.15.86:0/167787837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 0x7fcd9006d870 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009d90 tx=0x7fcd8801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cebe1a450e4f62 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd8f400640 1 -- 172.21.15.86:0/167787837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880b2070 con 0x7fcd9009b180 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd8f400640 1 -- 172.21.15.86:0/167787837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880a58a0 con 0x7fcd9009b180 2024-03-22T04:21:50.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd8f400640 1 -- 172.21.15.86:0/167787837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880a5c70 con 0x7fcd9009b180 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd97a00640 1 -- 172.21.15.86:0/167787837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 msgr2=0x7fcd9006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd97a00640 1 --2- 172.21.15.86:0/167787837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 0x7fcd9006d870 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009d90 tx=0x7fcd8801c930 comp rx=0 tx=0).stop 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd97a00640 1 -- 172.21.15.86:0/167787837 shutdown_connections 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd97a00640 1 --2- 172.21.15.86:0/167787837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 0x7fcd9006d870 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd97a00640 1 -- 172.21.15.86:0/167787837 >> 172.21.15.86:0/167787837 conn(0x7fcd900b3ba0 msgr2=0x7fcd900b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd97a00640 1 -- 172.21.15.86:0/167787837 shutdown_connections 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.086+0000 7fcd97a00640 1 -- 172.21.15.86:0/167787837 wait complete. 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 Processor -- start 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 -- start start 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd9006c980 0x7fcd9009ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 0x7fcd900cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd9015e670 0x7fcd9009c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9016c450 con 0x7fcd9009b180 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd900cd160 con 0x7fcd9006c980 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9016c5d0 con 0x7fcd9015e670 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 0x7fcd900cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 0x7fcd900cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51468/0 (socket says 172.21.15.86:51468) 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 -- 172.21.15.86:0/1844186172 learned_addr learned my addr 172.21.15.86:0/1844186172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd95000640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd9006c980 0x7fcd9009ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd95a00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd9015e670 0x7fcd9009c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 -- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd9015e670 msgr2=0x7fcd9009c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd9015e670 0x7fcd9009c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 -- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd9006c980 msgr2=0x7fcd9009ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd9006c980 0x7fcd9009ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 -- 172.21.15.86:0/1844186172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9009f0f0 con 0x7fcd9009b180 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8fe00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 0x7fcd900cb670 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c430 tx=0x7fcd7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c017070 con 0x7fcd9009b180 2024-03-22T04:21:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c005cf0 con 0x7fcd9009b180 2024-03-22T04:21:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9009f3d0 con 0x7fcd9009b180 2024-03-22T04:21:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0072f0 con 0x7fcd9009b180 2024-03-22T04:21:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9009f990 con 0x7fcd9009b180 2024-03-22T04:21:50.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.090+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd90074580 con 0x7fcd9009b180 2024-03-22T04:21:50.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fcd7c01a070 con 0x7fcd9009b180 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd8cc00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcd70067b50 0x7fcd7006a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7fcd7c085180 con 0x7fcd9009b180 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd95000640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcd70067b50 0x7fcd7006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd8cc00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7fcd7006e130 0x7fcd70070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd7c016ec0 con 0x7fcd7006e130 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=129) ==== 24+0+0 (secure 0 0 0) 0x7fcd7c08c050 con 0x7fcd9009b180 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd95a00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7fcd7006e130 0x7fcd70070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd95000640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcd70067b50 0x7fcd7006a010 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcd840098e0 tx=0x7fcd84009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd95a00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7fcd7006e130 0x7fcd70070570 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.094+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 <== osd.7 v2:172.21.15.196:6817/2847924285 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd7c016ec0 con 0x7fcd7006e130 2024-03-22T04:21:50.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 --> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcd9006c350 con 0x7fcd7006e130 2024-03-22T04:21:50.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd8cc00640 1 -- 172.21.15.86:0/1844186172 <== osd.7 v2:172.21.15.196:6817/2847924285 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcd9006c350 con 0x7fcd7006e130 2024-03-22T04:21:50.107 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-22T04:21:50.107 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-22T04:21:50.107 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7fcd7006e130 msgr2=0x7fcd70070570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.196:6817/2847924285,v1:172.21.15.196:6819/2847924285] conn(0x7fcd7006e130 0x7fcd70070570 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcd70067b50 msgr2=0x7fcd7006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fcd70067b50 0x7fcd7006a010 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcd840098e0 tx=0x7fcd84009060 comp rx=0 tx=0).stop 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 msgr2=0x7fcd900cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 --2- 172.21.15.86:0/1844186172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd9009b180 0x7fcd900cb670 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c430 tx=0x7fcd7c00c8f0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd95a00640 1 -- 172.21.15.86:0/1844186172 reap_dead start 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 shutdown_connections 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 >> 172.21.15.86:0/1844186172 conn(0x7fcd900b3ba0 msgr2=0x7fcd9015ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 shutdown_connections 2024-03-22T04:21:50.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.110+0000 7fcd97a00640 1 -- 172.21.15.86:0/1844186172 wait complete. 2024-03-22T04:21:50.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-22T04:21:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88f600640 1 Processor -- start 2024-03-22T04:21:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88f600640 1 -- start start 2024-03-22T04:21:50.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88f600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8880aff50 0x7ff88806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8881444f0 0x7ff88815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88f600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8880ae5a0 con 0x7ff88806cbb0 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff888144960 con 0x7ff8881444f0 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88f600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff88806d010 con 0x7ff8880aff50 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8881444f0 0x7ff88815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8881444f0 0x7ff88815eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51476/0 (socket says 172.21.15.86:51476) 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88d600640 1 -- 172.21.15.86:0/722900954 learned_addr learned my addr 172.21.15.86:0/722900954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff887e00640 1 --1- 172.21.15.86:0/722900954 >> v1:172.21.15.86:6789/0 conn(0x7ff88806cbb0 0x7ff888143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45994/0 (socket says 172.21.15.86:45994) 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88cc00640 1 --2- 172.21.15.86:0/722900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8880aff50 0x7ff88806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88d600640 1 -- 172.21.15.86:0/722900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8880aff50 msgr2=0x7ff88806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88d600640 1 --2- 172.21.15.86:0/722900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8880aff50 0x7ff88806c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88d600640 1 -- 172.21.15.86:0/722900954 >> v1:172.21.15.86:6789/0 conn(0x7ff88806cbb0 legacy=0x7ff888143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.266+0000 7ff88d600640 1 -- 172.21.15.86:0/722900954 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff888144b80 con 0x7ff8881444f0 2024-03-22T04:21:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88d600640 1 --2- 172.21.15.86:0/722900954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8881444f0 0x7ff88815eb80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7ff880009e50 tx=0x7ff88001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bf0649957015b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff887400640 1 -- 172.21.15.86:0/722900954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8800a5750 con 0x7ff8881444f0 2024-03-22T04:21:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff887400640 1 -- 172.21.15.86:0/722900954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8800a58f0 con 0x7ff8881444f0 2024-03-22T04:21:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff887400640 1 -- 172.21.15.86:0/722900954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8800a5cc0 con 0x7ff8881444f0 2024-03-22T04:21:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 -- 172.21.15.86:0/722900954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8881444f0 msgr2=0x7ff88815eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 --2- 172.21.15.86:0/722900954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8881444f0 0x7ff88815eb80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7ff880009e50 tx=0x7ff88001c9a0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 -- 172.21.15.86:0/722900954 shutdown_connections 2024-03-22T04:21:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 --2- 172.21.15.86:0/722900954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8881444f0 0x7ff88815eb80 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 --2- 172.21.15.86:0/722900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8880aff50 0x7ff88806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 -- 172.21.15.86:0/722900954 >> 172.21.15.86:0/722900954 conn(0x7ff888071fe0 msgr2=0x7ff8880723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 -- 172.21.15.86:0/722900954 shutdown_connections 2024-03-22T04:21:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 -- 172.21.15.86:0/722900954 wait complete. 2024-03-22T04:21:50.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 Processor -- start 2024-03-22T04:21:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.270+0000 7ff88f600640 1 -- start start 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff88806cbb0 0x7ff888155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8880aff50 0x7ff88813b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8881444f0 0x7ff888156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff888163090 con 0x7ff8880aff50 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff888162f10 con 0x7ff8881444f0 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff8880b0330 con 0x7ff88806cbb0 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8880aff50 0x7ff88813b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8880aff50 0x7ff88813b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51488/0 (socket says 172.21.15.86:51488) 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 -- 172.21.15.86:0/3712215322 learned_addr learned my addr 172.21.15.86:0/3712215322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88d600640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8881444f0 0x7ff888156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88cc00640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff88806cbb0 0x7ff888155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 -- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff88806cbb0 msgr2=0x7ff888155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff88806cbb0 0x7ff888155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 -- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8881444f0 msgr2=0x7ff888156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8881444f0 0x7ff888156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 -- 172.21.15.86:0/3712215322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8881386e0 con 0x7ff8880aff50 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88cc00640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff88806cbb0 0x7ff888155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff887e00640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8880aff50 0x7ff88813b040 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ff87400c420 tx=0x7ff87400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff884c00640 1 -- 172.21.15.86:0/3712215322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8740074e0 con 0x7ff8880aff50 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff884c00640 1 -- 172.21.15.86:0/3712215322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff874007680 con 0x7ff8880aff50 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff884c00640 1 -- 172.21.15.86:0/3712215322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874004a30 con 0x7ff8880aff50 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff888139740 con 0x7ff8880aff50 2024-03-22T04:21:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff888139d00 con 0x7ff8880aff50 2024-03-22T04:21:50.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.274+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88813a010 con 0x7ff8880aff50 2024-03-22T04:21:50.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.278+0000 7ff884c00640 1 -- 172.21.15.86:0/3712215322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7ff87401a020 con 0x7ff8880aff50 2024-03-22T04:21:50.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.278+0000 7ff884c00640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff868067b20 0x7ff868069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.278+0000 7ff884c00640 1 -- 172.21.15.86:0/3712215322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7ff874085690 con 0x7ff8880aff50 2024-03-22T04:21:50.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.278+0000 7ff88cc00640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff868067b20 0x7ff868069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.278+0000 7ff88cc00640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff868067b20 0x7ff868069fe0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff88813e460 tx=0x7ff87c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.282+0000 7ff884c00640 1 -- 172.21.15.86:0/3712215322 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff8880ae5a0 con 0x7ff8880aff50 2024-03-22T04:21:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.434+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7ff888068650 con 0x7ff868067b20 2024-03-22T04:21:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.434+0000 7ff884c00640 1 -- 172.21.15.86:0/3712215322 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff888068650 con 0x7ff868067b20 2024-03-22T04:21:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 7 to deep-scrub 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff868067b20 msgr2=0x7ff868069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff868067b20 0x7ff868069fe0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff88813e460 tx=0x7ff87c008040 comp rx=0 tx=0).stop 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8880aff50 msgr2=0x7ff88813b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8880aff50 0x7ff88813b040 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ff87400c420 tx=0x7ff87400c8e0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 shutdown_connections 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7ff868067b20 0x7ff868069fe0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8881444f0 0x7ff888156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8880aff50 0x7ff88813b040 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 --2- 172.21.15.86:0/3712215322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff88806cbb0 0x7ff888155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 >> 172.21.15.86:0/3712215322 conn(0x7ff888071fe0 msgr2=0x7ff88813e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 shutdown_connections 2024-03-22T04:21:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.438+0000 7ff88f600640 1 -- 172.21.15.86:0/3712215322 wait complete. 2024-03-22T04:21:50.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:21:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.590+0000 7f6986c00640 1 Processor -- start 2024-03-22T04:21:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.590+0000 7f6986c00640 1 -- start start 2024-03-22T04:21:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f6986c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f698015b3f0 0x7f69800729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f6986c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6980071920 con 0x7f69800ae540 2024-03-22T04:21:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f6986c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6980071c00 con 0x7f69800cc3e0 2024-03-22T04:21:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f6986c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f69800716d0 con 0x7f698015b3f0 2024-03-22T04:21:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f6984c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f69800ae540 0x7f69800b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46010/0 (socket says 172.21.15.86:46010) 2024-03-22T04:21:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f6984c00640 1 -- 172.21.15.86:0/6479325 learned_addr learned my addr 172.21.15.86:0/6479325 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:21:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f697fe00640 1 --2- 172.21.15.86:0/6479325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f698015b3f0 0x7f69800729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f697ea00640 1 -- 172.21.15.86:0/6479325 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1662728575 0 0) 0x7f6980071920 con 0x7f69800ae540 2024-03-22T04:21:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f697ea00640 1 -- 172.21.15.86:0/6479325 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6964003530 con 0x7f69800ae540 2024-03-22T04:21:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f697ea00640 1 -- 172.21.15.86:0/6479325 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2174736119 0 0) 0x7f6980071c00 con 0x7f69800cc3e0 2024-03-22T04:21:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f697ea00640 1 -- 172.21.15.86:0/6479325 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6980071920 con 0x7f69800cc3e0 2024-03-22T04:21:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f697fe00640 1 -- 172.21.15.86:0/6479325 >> v1:172.21.15.196:6789/0 conn(0x7f69800cc3e0 legacy=0x7f69800ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f697fe00640 1 -- 172.21.15.86:0/6479325 >> v1:172.21.15.86:6789/0 conn(0x7f69800ae540 legacy=0x7f69800b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.594+0000 7f697fe00640 1 -- 172.21.15.86:0/6479325 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6980071ee0 con 0x7f698015b3f0 2024-03-22T04:21:50.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f697fe00640 1 --2- 172.21.15.86:0/6479325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f698015b3f0 0x7f69800729c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f69700097f0 tx=0x7f697001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90459a67d2243998 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f697ea00640 1 -- 172.21.15.86:0/6479325 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700b0070 con 0x7f698015b3f0 2024-03-22T04:21:50.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f697ea00640 1 -- 172.21.15.86:0/6479325 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69700ab600 con 0x7f698015b3f0 2024-03-22T04:21:50.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f697ea00640 1 -- 172.21.15.86:0/6479325 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700ab9d0 con 0x7f698015b3f0 2024-03-22T04:21:50.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 -- 172.21.15.86:0/6479325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f698015b3f0 msgr2=0x7f69800729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 --2- 172.21.15.86:0/6479325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f698015b3f0 0x7f69800729c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f69700097f0 tx=0x7f697001c4e0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 -- 172.21.15.86:0/6479325 shutdown_connections 2024-03-22T04:21:50.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 --2- 172.21.15.86:0/6479325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f698015b3f0 0x7f69800729c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 -- 172.21.15.86:0/6479325 >> 172.21.15.86:0/6479325 conn(0x7f69800b1df0 msgr2=0x7f69800b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:50.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 -- 172.21.15.86:0/6479325 shutdown_connections 2024-03-22T04:21:50.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 -- 172.21.15.86:0/6479325 wait complete. 2024-03-22T04:21:50.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 Processor -- start 2024-03-22T04:21:50.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.598+0000 7f6986c00640 1 -- start start 2024-03-22T04:21:50.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69800ae540 0x7f6980151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69800cc3e0 0x7f698009dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698015b3f0 0x7f6980152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69800b30f0 con 0x7f698015b3f0 2024-03-22T04:21:50.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69800b4c60 con 0x7f69800ae540 2024-03-22T04:21:50.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f69800b3270 con 0x7f69800cc3e0 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698015b3f0 0x7f6980152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698015b3f0 0x7f6980152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51490/0 (socket says 172.21.15.86:51490) 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 -- 172.21.15.86:0/3599622247 learned_addr learned my addr 172.21.15.86:0/3599622247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f697fe00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69800ae540 0x7f6980151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f697f400640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69800cc3e0 0x7f698009dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 -- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69800cc3e0 msgr2=0x7f698009dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69800cc3e0 0x7f698009dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 -- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69800ae540 msgr2=0x7f6980151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69800ae540 0x7f6980151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 -- 172.21.15.86:0/3599622247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801503b0 con 0x7f698015b3f0 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f697f400640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69800cc3e0 0x7f698009dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:21:50.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6984c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698015b3f0 0x7f6980152450 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f697800ca80 tx=0x7f697800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6977e00640 1 -- 172.21.15.86:0/3599622247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978017070 con 0x7f698015b3f0 2024-03-22T04:21:50.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6977e00640 1 -- 172.21.15.86:0/3599622247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697800ac30 con 0x7f698015b3f0 2024-03-22T04:21:50.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6980150690 con 0x7f698015b3f0 2024-03-22T04:21:50.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6977e00640 1 -- 172.21.15.86:0/3599622247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978012400 con 0x7f698015b3f0 2024-03-22T04:21:50.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6980150c50 con 0x7f698015b3f0 2024-03-22T04:21:50.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6980068650 con 0x7f698015b3f0 2024-03-22T04:21:50.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.602+0000 7f6977e00640 1 -- 172.21.15.86:0/3599622247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f697801a070 con 0x7f698015b3f0 2024-03-22T04:21:50.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.606+0000 7f6977e00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6960067b20 0x7f6960069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:21:50.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.606+0000 7f6977e00640 1 -- 172.21.15.86:0/3599622247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f6978085af0 con 0x7f698015b3f0 2024-03-22T04:21:50.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.606+0000 7f697fe00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6960067b20 0x7f6960069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:21:50.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.606+0000 7f697fe00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6960067b20 0x7f6960069fe0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f69700ad000 tx=0x7f69700a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:21:50.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.610+0000 7f6977e00640 1 -- 172.21.15.86:0/3599622247 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6978055f60 con 0x7f698015b3f0 2024-03-22T04:21:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.758+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69801563e0 con 0x7f6960067b20 2024-03-22T04:21:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6977e00640 1 -- 172.21.15.86:0/3599622247 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13492 (secure 0 0 0) 0x7f69801563e0 con 0x7f6960067b20 2024-03-22T04:21:50.759 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:21:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6960067b20 msgr2=0x7f6960069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6960067b20 0x7f6960069fe0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f69700ad000 tx=0x7f69700a6000 comp rx=0 tx=0).stop 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698015b3f0 msgr2=0x7f6980152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698015b3f0 0x7f6980152450 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f697800ca80 tx=0x7f697800cf40 comp rx=0 tx=0).stop 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 shutdown_connections 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f6960067b20 0x7f6960069fe0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698015b3f0 0x7f6980152450 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69800cc3e0 0x7f698009dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 --2- 172.21.15.86:0/3599622247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69800ae540 0x7f6980151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 >> 172.21.15.86:0/3599622247 conn(0x7f69800b1df0 msgr2=0x7f6980159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 shutdown_connections 2024-03-22T04:21:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:21:50.762+0000 7f6986c00640 1 -- 172.21.15.86:0/3599622247 wait complete. 2024-03-22T04:21:50.779 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-03-22T04:21:49.946907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263148,"kb_used_data":13704,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711572,"statfs":{"total":773094113280,"available":772824649728,"internally_reserved":0,"allocated":14032896,"data_stored":7186088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.928088"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":371,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-03-22T04:21:42.028727+0000","last_change":"2024-03-22T04:10:23.659491+0000","last_active":"2024-03-22T04:21:42.028727+0000","last_peered":"2024-03-22T04:21:42.028727+0000","last_clean":"2024-03-22T04:21:42.028727+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:21:42.028727+0000","last_undegraded":"2024-03-22T04:21:42.028727+0000","last_fullsized":"2024-03-22T04:21:42.028727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T04:10:22.641480+0000","last_clean_scrub_stamp":"2024-03-22T04:10:22.641480+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T16:07:47.671812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509580,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33328,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338512,"statfs":{"total":96636764160,"available":96602636288,"internally_reserved":0,"allocated":2199552,"data_stored":1341037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509580,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33328,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338512,"statfs":{"total":96636764160,"available":96602636288,"internally_reserved":0,"allocated":2199552,"data_stored":1341037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32752,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:21:50.782 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-22T04:10:22.641480+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=22, tm_hour=4, tm_min=10, tm_sec=22, tm_wday=4, tm_yday=82, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=22, tm_hour=4, tm_min=21, tm_sec=45, tm_wday=4, tm_yday=82, tm_isdst=0) 2024-03-22T04:21:50.782 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-22T04:21:59.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:21:59.198 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:22:10.784 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-22T04:22:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5caa00640 1 Processor -- start 2024-03-22T04:22:10.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5caa00640 1 -- start start 2024-03-22T04:22:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5caa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c40ad710 0x7fd5c4142690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5caa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c415c7c0 con 0x7fd5c40ad330 2024-03-22T04:22:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5caa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c406dea0 con 0x7fd5c406ef40 2024-03-22T04:22:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5caa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c406e180 con 0x7fd5c40ad710 2024-03-22T04:22:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd5c40ad330 0x7fd5c413fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53470/0 (socket says 172.21.15.86:53470) 2024-03-22T04:22:10.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c3e00640 1 -- 172.21.15.86:0/3844134309 learned_addr learned my addr 172.21.15.86:0/3844134309 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:22:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104890867 0 0) 0x7fd5c415c7c0 con 0x7fd5c40ad330 2024-03-22T04:22:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c8a00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c40ad710 0x7fd5c4142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5a8003530 con 0x7fd5c40ad330 2024-03-22T04:22:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1638710807 0 0) 0x7fd5a8003530 con 0x7fd5c40ad330 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c40ad710 msgr2=0x7fd5c4142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c40ad710 0x7fd5c4142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 >> v1:172.21.15.196:6789/0 conn(0x7fd5c406ef40 legacy=0x7fd5c41402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c406e460 con 0x7fd5c40ad330 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 447898506 0 0) 0x7fd5b4001680 con 0x7fd5c40ad330 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 >> v1:172.21.15.86:6789/0 conn(0x7fd5c40ad330 legacy=0x7fd5c413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80039f0 0x7fd5a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c40ad710 0x7fd5a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a8004d50 0x7fd5a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c406e180 con 0x7fd5a80039f0 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a800bab0 con 0x7fd5a8004d50 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a800bcb0 con 0x7fd5c40ad710 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.934+0000 7fd5c3e00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80039f0 0x7fd5a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:10.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c3400640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c40ad710 0x7fd5a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c8a00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a8004d50 0x7fd5a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c3e00640 1 -- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c40ad710 msgr2=0x7fd5a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:22:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c3e00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c40ad710 0x7fd5a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c3e00640 1 -- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a8004d50 msgr2=0x7fd5a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:22:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c3e00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a8004d50 0x7fd5a80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c3e00640 1 -- 172.21.15.86:0/3844134309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a8003530 con 0x7fd5a80039f0 2024-03-22T04:22:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c3e00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80039f0 0x7fd5a8003e10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400c160 tx=0x7fd5b401cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5f05bbfd68324e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4007c70 con 0x7fd5a80039f0 2024-03-22T04:22:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b40a9070 con 0x7fd5a80039f0 2024-03-22T04:22:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c2a00640 1 -- 172.21.15.86:0/3844134309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4005d80 con 0x7fd5a80039f0 2024-03-22T04:22:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5caa00640 1 -- 172.21.15.86:0/3844134309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80039f0 msgr2=0x7fd5a8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5caa00640 1 --2- 172.21.15.86:0/3844134309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80039f0 0x7fd5a8003e10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400c160 tx=0x7fd5b401cb70 comp rx=0 tx=0).stop 2024-03-22T04:22:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5c8a00640 1 -- 172.21.15.86:0/3844134309 reap_dead start 2024-03-22T04:22:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5caa00640 1 -- 172.21.15.86:0/3844134309 shutdown_connections 2024-03-22T04:22:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5caa00640 1 -- 172.21.15.86:0/3844134309 >> 172.21.15.86:0/3844134309 conn(0x7fd5c40b1df0 msgr2=0x7fd5c40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:22:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5caa00640 1 -- 172.21.15.86:0/3844134309 shutdown_connections 2024-03-22T04:22:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.938+0000 7fd5caa00640 1 -- 172.21.15.86:0/3844134309 wait complete. 2024-03-22T04:22:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 Processor -- start 2024-03-22T04:22:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 -- start start 2024-03-22T04:22:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a80039f0 0x7fd5c409d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c415fe80 0x7fd5c41591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c4159720 0x7fd5c41f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c406d700 con 0x7fd5c415fe80 2024-03-22T04:22:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41409d0 con 0x7fd5a80039f0 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c406d880 con 0x7fd5c4159720 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c415fe80 0x7fd5c41591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c415fe80 0x7fd5c41591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59320/0 (socket says 172.21.15.86:59320) 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 -- 172.21.15.86:0/1147726741 learned_addr learned my addr 172.21.15.86:0/1147726741 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c8a00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c4159720 0x7fd5c41f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3e00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a80039f0 0x7fd5c409d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 -- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c4159720 msgr2=0x7fd5c41f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c4159720 0x7fd5c41f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 -- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a80039f0 msgr2=0x7fd5c409d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a80039f0 0x7fd5c409d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 -- 172.21.15.86:0/1147726741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c415a740 con 0x7fd5c415fe80 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3e00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a80039f0 0x7fd5c409d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5c3400640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c415fe80 0x7fd5c41591e0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000e500 tx=0x7fd5b000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5bbe00640 1 -- 172.21.15.86:0/1147726741 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00053a0 con 0x7fd5c415fe80 2024-03-22T04:22:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5bbe00640 1 -- 172.21.15.86:0/1147726741 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b0018070 con 0x7fd5c415fe80 2024-03-22T04:22:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5bbe00640 1 -- 172.21.15.86:0/1147726741 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0003820 con 0x7fd5c415fe80 2024-03-22T04:22:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c4158700 con 0x7fd5c415fe80 2024-03-22T04:22:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.942+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c4160300 con 0x7fd5c415fe80 2024-03-22T04:22:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.946+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c4068650 con 0x7fd5c415fe80 2024-03-22T04:22:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.946+0000 7fd5bbe00640 1 -- 172.21.15.86:0/1147726741 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd5b00039c0 con 0x7fd5c415fe80 2024-03-22T04:22:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.946+0000 7fd5bbe00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd5a4067b20 0x7fd5a4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.946+0000 7fd5bbe00640 1 -- 172.21.15.86:0/1147726741 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7fd5b0085b60 con 0x7fd5c415fe80 2024-03-22T04:22:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.946+0000 7fd5c3e00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd5a4067b20 0x7fd5a4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.946+0000 7fd5c3e00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd5a4067b20 0x7fd5a4069fe0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80057f0 tx=0x7fd5b40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:10.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:10.950+0000 7fd5bbe00640 1 -- 172.21.15.86:0/1147726741 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd5b00565a0 con 0x7fd5c415fe80 2024-03-22T04:22:11.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 --> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5c41570b0 con 0x7fd5a4067b20 2024-03-22T04:22:11.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5bbe00640 1 -- 172.21.15.86:0/1147726741 <== mgr.4104 v2:172.21.15.86:6832/1135494009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fd5c41570b0 con 0x7fd5a4067b20 2024-03-22T04:22:11.103 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:22:11.103 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-22T04:22:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd5a4067b20 msgr2=0x7fd5a4069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd5a4067b20 0x7fd5a4069fe0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80057f0 tx=0x7fd5b40a7070 comp rx=0 tx=0).stop 2024-03-22T04:22:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c415fe80 msgr2=0x7fd5c41591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c415fe80 0x7fd5c41591e0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000e500 tx=0x7fd5b000e9c0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 shutdown_connections 2024-03-22T04:22:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd5a4067b20 0x7fd5a4069fe0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5c4159720 0x7fd5c41f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5c415fe80 0x7fd5c41591e0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 --2- 172.21.15.86:0/1147726741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a80039f0 0x7fd5c409d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 >> 172.21.15.86:0/1147726741 conn(0x7fd5c40b1df0 msgr2=0x7fd5c40b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:22:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 shutdown_connections 2024-03-22T04:22:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.106+0000 7fd5caa00640 1 -- 172.21.15.86:0/1147726741 wait complete. 2024-03-22T04:22:11.124 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-03-22T04:22:09.951363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263140,"kb_used_data":13704,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711580,"statfs":{"total":773094113280,"available":772824657920,"internally_reserved":0,"allocated":14032896,"data_stored":7186088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002689"},"pg_stats":[{"pgid":"1.0","version":"129'119","reported_seq":382,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-03-22T04:21:48.169604+0000","last_change":"2024-03-22T04:21:48.169629+0000","last_active":"2024-03-22T04:21:48.169604+0000","last_peered":"2024-03-22T04:21:48.169604+0000","last_clean":"2024-03-22T04:21:48.169604+0000","last_became_active":"2024-03-22T04:10:23.658315+0000","last_became_peered":"2024-03-22T04:10:23.658315+0000","last_unstale":"2024-03-22T04:21:48.169604+0000","last_undegraded":"2024-03-22T04:21:48.169604+0000","last_fullsized":"2024-03-22T04:21:48.169604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"129'119","last_scrub_stamp":"2024-03-22T04:21:48.169547+0000","last_deep_scrub":"129'119","last_deep_scrub_stamp":"2024-03-22T04:21:48.169547+0000","last_clean_scrub_stamp":"2024-03-22T04:21:48.169547+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T15:12:02.485469+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2199552,"data_stored":1341037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2199552,"data_stored":1341037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32752,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1605632,"data_stored":750669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T04:22:11.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-22T04:22:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e8ba00640 1 Processor -- start 2024-03-22T04:22:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e8ba00640 1 -- start start 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e8ba00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e8406e7e0 0x7f1e8406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e8ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e84072650 0x7f1e84072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e8ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8415c770 con 0x7f1e840b1b80 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e8ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1e84071350 con 0x7f1e84072650 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e8ba00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8406f100 con 0x7f1e8406e7e0 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e84072650 0x7f1e84072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e89a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1e840b1b80 0x7f1e840b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53476/0 (socket says 172.21.15.86:53476) 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e89a00640 1 -- 172.21.15.86:0/1546562495 learned_addr learned my addr 172.21.15.86:0/1546562495 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e84072650 0x7f1e84072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59334/0 (socket says 172.21.15.86:59334) 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e89000640 1 --2- 172.21.15.86:0/1546562495 >> v2:172.21.15.196:3300/0 conn(0x7f1e8406e7e0 0x7f1e8406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e83e00640 1 -- 172.21.15.86:0/1546562495 >> v2:172.21.15.196:3300/0 conn(0x7f1e8406e7e0 msgr2=0x7f1e8406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:22:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e83e00640 1 --2- 172.21.15.86:0/1546562495 >> v2:172.21.15.196:3300/0 conn(0x7f1e8406e7e0 0x7f1e8406ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e83e00640 1 -- 172.21.15.86:0/1546562495 >> v1:172.21.15.86:6789/0 conn(0x7f1e840b1b80 legacy=0x7f1e840b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e83e00640 1 -- 172.21.15.86:0/1546562495 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e84073150 con 0x7f1e84072650 2024-03-22T04:22:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e89000640 1 --2- 172.21.15.86:0/1546562495 >> v2:172.21.15.196:3300/0 conn(0x7f1e8406e7e0 0x7f1e8406ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T04:22:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.262+0000 7f1e83400640 1 -- 172.21.15.86:0/1546562495 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806223257 0 0) 0x7f1e7c001120 con 0x7f1e840b1b80 2024-03-22T04:22:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e83e00640 1 --2- 172.21.15.86:0/1546562495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e84072b00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f1e840ce650 tx=0x7f1e7001ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964c2965eb358f32 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e83400640 1 -- 172.21.15.86:0/1546562495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700b4070 con 0x7f1e84072650 2024-03-22T04:22:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e83400640 1 -- 172.21.15.86:0/1546562495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e700a7a60 con 0x7f1e84072650 2024-03-22T04:22:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e83400640 1 -- 172.21.15.86:0/1546562495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700a7e30 con 0x7f1e84072650 2024-03-22T04:22:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e8ba00640 1 -- 172.21.15.86:0/1546562495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 msgr2=0x7f1e84072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/1546562495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e84072b00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f1e840ce650 tx=0x7f1e7001ea90 comp rx=0 tx=0).stop 2024-03-22T04:22:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e8ba00640 1 -- 172.21.15.86:0/1546562495 shutdown_connections 2024-03-22T04:22:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/1546562495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e84072b00 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/1546562495 >> v2:172.21.15.196:3300/0 conn(0x7f1e8406e7e0 0x7f1e8406ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e8ba00640 1 -- 172.21.15.86:0/1546562495 >> 172.21.15.86:0/1546562495 conn(0x7f1e840cc940 msgr2=0x7f1e840ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:22:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e8ba00640 1 -- 172.21.15.86:0/1546562495 shutdown_connections 2024-03-22T04:22:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.266+0000 7f1e8ba00640 1 -- 172.21.15.86:0/1546562495 wait complete. 2024-03-22T04:22:11.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 Processor -- start 2024-03-22T04:22:11.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 -- start start 2024-03-22T04:22:11.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8406e7e0 0x7f1e84149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e840cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e840b1b80 0x7f1e84148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84072080 con 0x7f1e84072650 2024-03-22T04:22:11.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e840ce930 con 0x7f1e8406e7e0 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1e84150e20 con 0x7f1e840b1b80 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e840cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e840cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59342/0 (socket says 172.21.15.86:59342) 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 -- 172.21.15.86:0/3904165693 learned_addr learned my addr 172.21.15.86:0/3904165693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 -- 172.21.15.86:0/3904165693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e840b1b80 msgr2=0x7f1e84148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e840b1b80 0x7f1e84148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 -- 172.21.15.86:0/3904165693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8406e7e0 msgr2=0x7f1e84149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8406e7e0 0x7f1e84149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 -- 172.21.15.86:0/3904165693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e84151100 con 0x7f1e84072650 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e83e00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e840cbd30 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f1e7001c6c0 tx=0x7f1e7001b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e80c00640 1 -- 172.21.15.86:0/3904165693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700b4040 con 0x7f1e84072650 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e80c00640 1 -- 172.21.15.86:0/3904165693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e700b8420 con 0x7f1e84072650 2024-03-22T04:22:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e80c00640 1 -- 172.21.15.86:0/3904165693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700b87f0 con 0x7f1e84072650 2024-03-22T04:22:11.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e841513e0 con 0x7f1e84072650 2024-03-22T04:22:11.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e841516c0 con 0x7f1e84072650 2024-03-22T04:22:11.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.270+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e84074470 con 0x7f1e84072650 2024-03-22T04:22:11.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.274+0000 7f1e80c00640 1 -- 172.21.15.86:0/3904165693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7f1e700b8990 con 0x7f1e84072650 2024-03-22T04:22:11.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.274+0000 7f1e80c00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e64067b20 0x7f1e64069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.274+0000 7f1e80c00640 1 -- 172.21.15.86:0/3904165693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7f1e70120b40 con 0x7f1e84072650 2024-03-22T04:22:11.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.274+0000 7f1e89000640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e64067b20 0x7f1e64069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.274+0000 7f1e89000640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e64067b20 0x7f1e64069fe0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009eb0 tx=0x7f1e740094a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:11.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.278+0000 7f1e80c00640 1 -- 172.21.15.86:0/3904165693 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1e70018740 con 0x7f1e84072650 2024-03-22T04:22:11.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.434+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f1e840b24e0 con 0x7f1e84072650 2024-03-22T04:22:11.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.438+0000 7f1e80c00640 1 -- 172.21.15.86:0/3904165693 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f1e700f1570 con 0x7f1e84072650 2024-03-22T04:22:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.438+0000 7f1e80c00640 1 -- 172.21.15.86:0/3904165693 <== mon.0 v2:172.21.15.86:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1e70126080 con 0x7f1e84072650 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e64067b20 msgr2=0x7f1e64069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e64067b20 0x7f1e64069fe0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009eb0 tx=0x7f1e740094a0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 msgr2=0x7f1e840cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e840cbd30 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f1e7001c6c0 tx=0x7f1e7001b830 comp rx=0 tx=0).stop 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 shutdown_connections 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7f1e64067b20 0x7f1e64069fe0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e840b1b80 0x7f1e84148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84072650 0x7f1e840cbd30 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 --2- 172.21.15.86:0/3904165693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8406e7e0 0x7f1e84149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 >> 172.21.15.86:0/3904165693 conn(0x7f1e840cc940 msgr2=0x7f1e840b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:22:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 shutdown_connections 2024-03-22T04:22:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.442+0000 7f1e8ba00640 1 -- 172.21.15.86:0/3904165693 wait complete. 2024-03-22T04:22:11.459 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-22T04:22:11.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T04:22:11.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8f5a00640 1 Processor -- start 2024-03-22T04:22:11.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8f5a00640 1 -- start start 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8f5a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8f006fc60 0x7fd8f00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8f5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8f0070040 0x7fd8f00d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8f5a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f00d1810 con 0x7fd8f00d1190 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8f5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f00d15e0 con 0x7fd8f0070040 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8f5a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f00ce270 con 0x7fd8f006fc60 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ee000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8f0070040 0x7fd8f00d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ee000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8f0070040 0x7fd8f00d0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59350/0 (socket says 172.21.15.86:59350) 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ee000640 1 -- 172.21.15.86:0/3364812261 learned_addr learned my addr 172.21.15.86:0/3364812261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ef400640 1 --1- 172.21.15.86:0/3364812261 >> v1:172.21.15.86:6789/0 conn(0x7fd8f00d1190 0x7fd8f01457f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53484/0 (socket says 172.21.15.86:53484) 2024-03-22T04:22:11.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8eea00640 1 --2- 172.21.15.86:0/3364812261 >> v2:172.21.15.196:3300/0 conn(0x7fd8f006fc60 0x7fd8f00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ee000640 1 -- 172.21.15.86:0/3364812261 >> v2:172.21.15.196:3300/0 conn(0x7fd8f006fc60 msgr2=0x7fd8f00b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-22T04:22:11.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ee000640 1 --2- 172.21.15.86:0/3364812261 >> v2:172.21.15.196:3300/0 conn(0x7fd8f006fc60 0x7fd8f00b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ee000640 1 -- 172.21.15.86:0/3364812261 >> v1:172.21.15.86:6789/0 conn(0x7fd8f00d1190 legacy=0x7fd8f01457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ee000640 1 -- 172.21.15.86:0/3364812261 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f00d1af0 con 0x7fd8f0070040 2024-03-22T04:22:11.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.598+0000 7fd8ed600640 1 -- 172.21.15.86:0/3364812261 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226400816 0 0) 0x7fd8f00d1810 con 0x7fd8f00d1190 2024-03-22T04:22:11.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8ee000640 1 --2- 172.21.15.86:0/3364812261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f00d0c50 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00705a0 tx=0x7fd8dc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5e27b3f78b73e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:11.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8ed600640 1 -- 172.21.15.86:0/3364812261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0aa650 con 0x7fd8f0070040 2024-03-22T04:22:11.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8ed600640 1 -- 172.21.15.86:0/3364812261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd8dc0af070 con 0x7fd8f0070040 2024-03-22T04:22:11.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8ed600640 1 -- 172.21.15.86:0/3364812261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0aaa30 con 0x7fd8f0070040 2024-03-22T04:22:11.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8f5a00640 1 -- 172.21.15.86:0/3364812261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 msgr2=0x7fd8f00d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/3364812261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f00d0c50 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00705a0 tx=0x7fd8dc01c4f0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8f5a00640 1 -- 172.21.15.86:0/3364812261 shutdown_connections 2024-03-22T04:22:11.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/3364812261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f00d0c50 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/3364812261 >> v2:172.21.15.196:3300/0 conn(0x7fd8f006fc60 0x7fd8f00b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8f5a00640 1 -- 172.21.15.86:0/3364812261 >> 172.21.15.86:0/3364812261 conn(0x7fd8f00b75b0 msgr2=0x7fd8f00b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:22:11.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8f5a00640 1 -- 172.21.15.86:0/3364812261 shutdown_connections 2024-03-22T04:22:11.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.602+0000 7fd8f5a00640 1 -- 172.21.15.86:0/3364812261 wait complete. 2024-03-22T04:22:11.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 Processor -- start 2024-03-22T04:22:11.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 -- start start 2024-03-22T04:22:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f006fc60 0x7fd8f013e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f00d1190 0x7fd8f015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00cf370 con 0x7fd8f0070040 2024-03-22T04:22:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00cf1f0 con 0x7fd8f00d1190 2024-03-22T04:22:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f00cf4f0 con 0x7fd8f006fc60 2024-03-22T04:22:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f015ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59354/0 (socket says 172.21.15.86:59354) 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 -- 172.21.15.86:0/712722490 learned_addr learned my addr 172.21.15.86:0/712722490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8eea00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f006fc60 0x7fd8f013e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ef400640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f00d1190 0x7fd8f015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 -- 172.21.15.86:0/712722490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f006fc60 msgr2=0x7fd8f013e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f006fc60 0x7fd8f013e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 -- 172.21.15.86:0/712722490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f00d1190 msgr2=0x7fd8f015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f00d1190 0x7fd8f015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 -- 172.21.15.86:0/712722490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f013edc0 con 0x7fd8f0070040 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8ee000640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f015ab10 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc01a120 tx=0x7fd8dc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8e6a00640 1 -- 172.21.15.86:0/712722490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0b3800 con 0x7fd8f0070040 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8e6a00640 1 -- 172.21.15.86:0/712722490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd8dc0a6040 con 0x7fd8f0070040 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8e6a00640 1 -- 172.21.15.86:0/712722490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0b3c00 con 0x7fd8f0070040 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f013f0a0 con 0x7fd8f0070040 2024-03-22T04:22:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f013f660 con 0x7fd8f0070040 2024-03-22T04:22:11.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.606+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f0068650 con 0x7fd8f0070040 2024-03-22T04:22:11.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.610+0000 7fd8e6a00640 1 -- 172.21.15.86:0/712722490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 87459+0+0 (secure 0 0 0) 0x7fd8dc0b2380 con 0x7fd8f0070040 2024-03-22T04:22:11.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.610+0000 7fd8e6a00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8d0067b20 0x7fd8d0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T04:22:11.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.610+0000 7fd8e6a00640 1 -- 172.21.15.86:0/712722490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5507+0+0 (secure 0 0 0) 0x7fd8dc120070 con 0x7fd8f0070040 2024-03-22T04:22:11.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.610+0000 7fd8eea00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8d0067b20 0x7fd8d0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T04:22:11.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.610+0000 7fd8eea00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8d0067b20 0x7fd8d0069fe0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0003fd0 tx=0x7fd8e00039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T04:22:11.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.614+0000 7fd8e6a00640 1 -- 172.21.15.86:0/712722490 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd8dc016110 con 0x7fd8f0070040 2024-03-22T04:22:11.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8f00b67e0 con 0x7fd8f0070040 2024-03-22T04:22:11.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8e6a00640 1 -- 172.21.15.86:0/712722490 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+3877 (secure 0 0 0) 0x7fd8dc0aae20 con 0x7fd8f0070040 2024-03-22T04:22:11.798 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-22T04:22:11.799 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1726763841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1726763841},{"type":"v1","addr":"172.21.15.196:6834","nonce":1726763841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1720734268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1720734268},{"type":"v1","addr":"172.21.15.196:6835","nonce":1720734268}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1765441058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1765441058},{"type":"v1","addr":"172.21.15.86:6836","nonce":1765441058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2486756361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2486756361},{"type":"v1","addr":"172.21.15.86:6837","nonce":2486756361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[]} 2024-03-22T04:22:11.799 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 114 2024-03-22T04:22:11.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8d0067b20 msgr2=0x7fd8d0069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8d0067b20 0x7fd8d0069fe0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0003fd0 tx=0x7fd8e00039a0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 msgr2=0x7fd8f015ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f015ab10 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc01a120 tx=0x7fd8dc01ad30 comp rx=0 tx=0).stop 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 shutdown_connections 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:6832/1135494009,v1:172.21.15.86:6833/1135494009] conn(0x7fd8d0067b20 0x7fd8d0069fe0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f00d1190 0x7fd8f015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8f0070040 0x7fd8f015ab10 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 --2- 172.21.15.86:0/712722490 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f006fc60 0x7fd8f013e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 >> 172.21.15.86:0/712722490 conn(0x7fd8f00b75b0 msgr2=0x7fd8f00b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 shutdown_connections 2024-03-22T04:22:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-22T04:22:11.802+0000 7fd8f5a00640 1 -- 172.21.15.86:0/712722490 wait complete. 2024-03-22T04:22:11.818 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-22T04:22:11.819 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-22T04:22:11.819 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-22T04:22:11.819 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:11.831 INFO:tasks.ceph.mds.a:Stopped 2024-03-22T04:22:11.831 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-22T04:22:11.831 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:11.884 INFO:tasks.ceph.mds.c:Stopped 2024-03-22T04:22:11.885 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-22T04:22:11.885 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:11.898 INFO:tasks.ceph.mds.b:Stopped 2024-03-22T04:22:11.898 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-22T04:22:11.899 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:11.955 INFO:tasks.ceph.mds.d:Stopped 2024-03-22T04:22:11.955 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-22T04:22:11.955 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-22T04:22:11.955 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:12.029 INFO:tasks.ceph.osd.0:Stopped 2024-03-22T04:22:12.030 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-22T04:22:12.030 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:12.127 INFO:tasks.ceph.osd.1:Stopped 2024-03-22T04:22:12.128 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-22T04:22:12.128 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:12.223 INFO:tasks.ceph.osd.2:Stopped 2024-03-22T04:22:12.224 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-22T04:22:12.224 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:18.326 INFO:tasks.ceph.osd.3:Stopped 2024-03-22T04:22:18.326 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-22T04:22:18.326 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:18.386 INFO:tasks.ceph.osd.4:Stopped 2024-03-22T04:22:18.387 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-22T04:22:18.387 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:18.470 INFO:tasks.ceph.osd.5:Stopped 2024-03-22T04:22:18.470 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-22T04:22:18.470 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:19.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-22T04:22:24.573 INFO:tasks.ceph.osd.6:Stopped 2024-03-22T04:22:24.573 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-22T04:22:24.573 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:24.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-03-22T04:22:24.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-03-22T04:22:24.627 INFO:tasks.ceph.osd.7:Stopped 2024-03-22T04:22:24.628 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-22T04:22:24.628 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-22T04:22:24.628 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:24.687 INFO:tasks.ceph.mgr.y:Stopped 2024-03-22T04:22:24.688 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-22T04:22:24.688 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:24.744 INFO:tasks.ceph.mgr.x:Stopped 2024-03-22T04:22:24.744 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-22T04:22:24.744 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-22T04:22:24.744 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:24.764 INFO:tasks.ceph.mon.a:Stopped 2024-03-22T04:22:24.764 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-22T04:22:24.764 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:24.781 INFO:tasks.ceph.mon.b:Stopped 2024-03-22T04:22:24.781 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-22T04:22:24.781 INFO:teuthology.orchestra.run:waiting for 300 2024-03-22T04:22:24.842 INFO:tasks.ceph.mon.c:Stopped 2024-03-22T04:22:24.842 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-22T04:22:24.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-22T04:22:24.879 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:15:27.121050+0000 mds.a (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-03-22T04:22:24.880 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-22T04:22:24.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-22T04:22:24.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-22T04:22:25.045 INFO:teuthology.orchestra.run.smithi086.stdout:2024-03-22T04:17:15.598395+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-22T04:22:25.046 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:22:25.046 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-22T04:22:25.354 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:22:25.354 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-22T04:22:25.514 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:22:25.514 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-22T04:22:25.677 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:22:25.677 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-22T04:22:25.834 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:22:25.835 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-22T04:22:25.999 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:22:25.999 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-22T04:22:26.169 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:22:26.169 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-22T04:22:26.351 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:22:26.351 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-22T04:22:26.534 INFO:tasks.ceph:Archiving mon data... 2024-03-22T04:22:26.535 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/data/mon.a.tgz 2024-03-22T04:22:26.536 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-03-22T04:22:26.545 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.X9j73kX476 2024-03-22T04:22:26.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.X9j73kX476 2024-03-22T04:22:26.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.X9j73kX476 2024-03-22T04:22:26.766 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.X9j73kX476 is 448KB 2024-03-22T04:22:26.832 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.X9j73kX476 2024-03-22T04:22:26.840 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/data/mon.b.tgz 2024-03-22T04:22:26.840 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-03-22T04:22:26.849 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.LlIC89miYl 2024-03-22T04:22:26.850 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.LlIC89miYl 2024-03-22T04:22:27.003 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.LlIC89miYl 2024-03-22T04:22:27.081 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.LlIC89miYl is 464KB 2024-03-22T04:22:27.143 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.LlIC89miYl 2024-03-22T04:22:27.151 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/data/mon.c.tgz 2024-03-22T04:22:27.151 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-03-22T04:22:27.200 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.N1lAWIMiYj 2024-03-22T04:22:27.200 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.N1lAWIMiYj 2024-03-22T04:22:27.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.N1lAWIMiYj 2024-03-22T04:22:27.425 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.N1lAWIMiYj is 465KB 2024-03-22T04:22:27.487 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.N1lAWIMiYj 2024-03-22T04:22:27.496 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-22T04:22:27.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-22T04:22:27.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-22T04:22:27.569 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-22T04:22:27.604 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-22T04:22:27.604 INFO:tasks.ceph:Archiving crash dumps... 2024-03-22T04:22:27.606 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/remote/smithi086/crash 2024-03-22T04:22:27.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-22T04:22:27.847 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/remote/smithi196/crash 2024-03-22T04:22:27.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-22T04:22:27.904 INFO:tasks.ceph:Shutting down logrotate 2024-03-22T04:22:27.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:22:27.928 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-22T04:22:28.094 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-22T04:22:28.115 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-22T04:22:28.135 INFO:tasks.ceph:Compressing logs... 2024-03-22T04:22:28.135 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T04:22:28.164 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T04:22:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36331.log 2024-03-22T04:22:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49339.log 2024-03-22T04:22:28.205 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54588.log 2024-03-22T04:22:28.205 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36331.log.gz 2024-03-22T04:22:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52977.log 2024-03-22T04:22:28.206 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49339.log.gz 2024-03-22T04:22:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45222.log 2024-03-22T04:22:28.207 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54588.log.gz 2024-03-22T04:22:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53685.log 2024-03-22T04:22:28.208 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27600.log 2024-03-22T04:22:28.209 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45222.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.52977.log.gz 2024-03-22T04:22:28.209 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36705.log 2024-03-22T04:22:28.209 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45222.log.gz 2024-03-22T04:22:28.209 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40530.log 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27600.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27600.log.gz 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27696.log 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.32966.log 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27552.log 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27696.log.gz 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.32966.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.32966.log.gz 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27552.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-22T04:22:28.217 INFO:teuthology.orchestra.run.smithi196.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.27552.log.gz 2024-03-22T04:22:28.218 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33073.log 2024-03-22T04:22:28.218 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-22T04:22:28.218 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.3.33073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27672.log 2024-03-22T04:22:28.218 INFO:teuthology.orchestra.run.smithi196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33073.log.gz 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53685.log.gz 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43126.log 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36705.log.gz 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40530.log.gz 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47298.log 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43126.log.gz 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38198.log 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44026.log 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38198.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47298.log.gz 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34740.log 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38198.log.gz 2024-03-22T04:22:28.219 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54172.log 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44026.log.gz 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46822.log 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34740.log.gz 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38434.log 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54172.log.gz 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43582.log 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46822.log.gz 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39664.log 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38434.log.gz 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49860.log 2024-03-22T04:22:28.220 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43582.log.gz 2024-03-22T04:22:28.223 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27648.log 2024-03-22T04:22:28.223 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27672.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-22T04:22:28.224 INFO:teuthology.orchestra.run.smithi196.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.27672.log.gz 2024-03-22T04:22:28.224 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27528.log 2024-03-22T04:22:28.224 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27648.log.gz 2024-03-22T04:22:28.224 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.27528.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-22T04:22:28.224 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-22T04:22:28.224 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27528.log.gz 2024-03-22T04:22:28.224 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27624.log 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34241.log 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39664.log.gz 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38354.log 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49860.log.gz 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36149.log 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34241.log.gz 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35641.log 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38354.log.gz 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36149.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-22T04:22:28.225 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36149.log.gz 2024-03-22T04:22:28.226 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40986.log 2024-03-22T04:22:28.226 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35641.log.gz 2024-03-22T04:22:28.226 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52619.log 2024-03-22T04:22:28.226 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51031.log 2024-03-22T04:22:28.226 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40986.log.gz 2024-03-22T04:22:28.228 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-22T04:22:28.228 INFO:teuthology.orchestra.run.smithi196.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-22T04:22:28.228 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27624.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-22T04:22:28.228 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27624.log.gz 2024-03-22T04:22:28.228 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-22T04:22:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.52619.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.0.52619.log.gz /var/log/ceph/ceph-client.admin.53990.log 2024-03-22T04:22:28.229 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-03-22T04:22:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41829.log 2024-03-22T04:22:28.229 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51031.log.gz 2024-03-22T04:22:28.230 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53349.log 2024-03-22T04:22:28.230 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53990.log.gz 2024-03-22T04:22:28.230 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54510.log 2024-03-22T04:22:28.230 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41829.log.gz 2024-03-22T04:22:28.230 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.53349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46976.log 2024-03-22T04:22:28.230 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53349.log.gz 2024-03-22T04:22:28.231 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-22T04:22:28.231 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36999.log 2024-03-22T04:22:28.232 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54510.log.gz 2024-03-22T04:22:28.232 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.46976.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.0.46976.log.gz /var/log/ceph/ceph-client.admin.35237.log 2024-03-22T04:22:28.232 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-03-22T04:22:28.232 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46848.log 2024-03-22T04:22:28.232 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36999.log.gz 2024-03-22T04:22:28.232 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51514.log 2024-03-22T04:22:28.233 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35237.log.gz 2024-03-22T04:22:28.233 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-22T04:22:28.234 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54224.log 2024-03-22T04:22:28.234 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46848.log.gz 2024-03-22T04:22:28.234 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38759.log 2024-03-22T04:22:28.235 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51514.log.gz 2024-03-22T04:22:28.235 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39349.log 2024-03-22T04:22:28.235 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54224.log.gz 2024-03-22T04:22:28.236 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51570.log 2024-03-22T04:22:28.236 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38759.log.gz 2024-03-22T04:22:28.237 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27576.log 2024-03-22T04:22:28.237 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.32984.log 2024-03-22T04:22:28.237 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27576.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27576.log.gz 2024-03-22T04:22:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48734.log 2024-03-22T04:22:28.239 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39349.log.gz 2024-03-22T04:22:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51570.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51570.log.gz 2024-03-22T04:22:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36679.log 2024-03-22T04:22:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34346.log 2024-03-22T04:22:28.239 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48734.log.gz 2024-03-22T04:22:28.240 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33068.log 2024-03-22T04:22:28.241 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.32984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.32984.log.gz 2024-03-22T04:22:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44000.log 2024-03-22T04:22:28.242 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36679.log.gz 2024-03-22T04:22:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49183.log 2024-03-22T04:22:28.243 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34346.log.gz 2024-03-22T04:22:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40934.log 2024-03-22T04:22:28.243 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44000.log.gz 2024-03-22T04:22:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2024-03-22T04:22:28.243 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49183.log.gz 2024-03-22T04:22:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40615.log 2024-03-22T04:22:28.245 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40934.log.gz 2024-03-22T04:22:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log 2024-03-22T04:22:28.245 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2024-03-22T04:22:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53711.log 2024-03-22T04:22:28.245 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40615.log.gz 2024-03-22T04:22:28.247 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44300.log 2024-03-22T04:22:28.247 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2024-03-22T04:22:28.247 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38609.log 2024-03-22T04:22:28.247 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53711.log.gz 2024-03-22T04:22:28.248 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49022.log 2024-03-22T04:22:28.249 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44300.log.gz 2024-03-22T04:22:28.249 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.38609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38609.log.gz 2024-03-22T04:22:28.249 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49443.log 2024-03-22T04:22:28.250 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.3.33068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33068.log.gz 2024-03-22T04:22:28.251 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35775.log 2024-03-22T04:22:28.251 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49022.log.gz 2024-03-22T04:22:28.251 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45818.log 2024-03-22T04:22:28.251 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49443.log.gz 2024-03-22T04:22:28.251 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.35775.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.45975.log -- replaced with /var/log/ceph/ceph-client.0.35775.log.gz 2024-03-22T04:22:28.251 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-03-22T04:22:28.252 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45818.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-22T04:22:28.252 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45818.log.gz 2024-03-22T04:22:28.252 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44866.log 2024-03-22T04:22:28.252 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45975.log.gz 2024-03-22T04:22:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50406.log 2024-03-22T04:22:28.254 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42232.log 2024-03-22T04:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44866.log.gz 2024-03-22T04:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39850.log 2024-03-22T04:22:28.256 INFO:teuthology.orchestra.run.smithi086.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50406.log.gz 2024-03-22T04:22:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54146.log 88.2% 2024-03-22T04:22:28.256 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42232.log.gz 2024-03-22T04:22:28.257 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2024-03-22T04:22:28.257 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39850.log.gz 2024-03-22T04:22:28.258 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54146.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54146.log.gz 2024-03-22T04:22:28.258 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46874.log 2024-03-22T04:22:28.259 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46348.log 2024-03-22T04:22:28.259 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz 2024-03-22T04:22:28.260 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48708.log 2024-03-22T04:22:28.260 INFO:teuthology.orchestra.run.smithi086.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46874.log.gz 2024-03-22T04:22:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46348.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46348.log.gz 2024-03-22T04:22:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51161.log 2024-03-22T04:22:28.262 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50797.log 2024-03-22T04:22:28.262 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48708.log.gz 2024-03-22T04:22:28.263 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33771.log 2024-03-22T04:22:28.263 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51161.log.gz 2024-03-22T04:22:28.264 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50797.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50797.log.gz 2024-03-22T04:22:28.264 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 2024-03-22T04:22:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42595.log 2024-03-22T04:22:28.265 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33771.log.gz 2024-03-22T04:22:28.266 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50901.log 2024-03-22T04:22:28.266 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2024-03-22T04:22:28.267 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40725.log 2024-03-22T04:22:28.267 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42595.log.gz 2024-03-22T04:22:28.267 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37706.log 2024-03-22T04:22:28.267 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50901.log.gz 2024-03-22T04:22:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34372.log 2024-03-22T04:22:28.268 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40725.log.gz 2024-03-22T04:22:28.269 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46633.log 2024-03-22T04:22:28.269 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37706.log.gz 2024-03-22T04:22:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37576.log 2024-03-22T04:22:28.270 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34372.log.gz 2024-03-22T04:22:28.271 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43918.log 2024-03-22T04:22:28.271 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46633.log.gz 2024-03-22T04:22:28.272 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41774.log 2024-03-22T04:22:28.272 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37576.log.gz 2024-03-22T04:22:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.43918.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43918.log.gz 2024-03-22T04:22:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38038.log 2024-03-22T04:22:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43768.log 2024-03-22T04:22:28.274 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41774.log.gz 2024-03-22T04:22:28.275 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48448.log 2024-03-22T04:22:28.275 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38038.log.gz 2024-03-22T04:22:28.276 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53581.log 2024-03-22T04:22:28.276 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43768.log.gz 2024-03-22T04:22:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34941.log 2024-03-22T04:22:28.278 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48448.log.gz 2024-03-22T04:22:28.278 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38655.log 2024-03-22T04:22:28.278 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53581.log.gz 2024-03-22T04:22:28.279 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53912.log 2024-03-22T04:22:28.279 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34941.log.gz 2024-03-22T04:22:28.280 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52099.log 2024-03-22T04:22:28.280 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38655.log.gz 2024-03-22T04:22:28.280 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53633.log 2024-03-22T04:22:28.281 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53912.log.gz 2024-03-22T04:22:28.281 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40641.log 2024-03-22T04:22:28.282 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52099.log.gz 2024-03-22T04:22:28.282 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35341.log 2024-03-22T04:22:28.283 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53633.log.gz 2024-03-22T04:22:28.284 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41405.log 2024-03-22T04:22:28.284 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40641.log.gz 2024-03-22T04:22:28.285 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35185.log 2024-03-22T04:22:28.285 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35341.log.gz 2024-03-22T04:22:28.286 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47352.log 2024-03-22T04:22:28.286 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41405.log.gz 2024-03-22T04:22:28.286 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33875.log 2024-03-22T04:22:28.287 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35185.log.gz 2024-03-22T04:22:28.287 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38408.log 2024-03-22T04:22:28.288 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47352.log.gz 2024-03-22T04:22:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35575.log 2024-03-22T04:22:28.289 INFO:teuthology.orchestra.run.smithi086.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.33875.log.gz 2024-03-22T04:22:28.289 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38408.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38408.log.gz 2024-03-22T04:22:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50849.log 2024-03-22T04:22:28.290 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38985.log 2024-03-22T04:22:28.291 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.35575.log.gz 2024-03-22T04:22:28.293 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49391.log 2024-03-22T04:22:28.293 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50849.log.gz 2024-03-22T04:22:28.294 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52434.log 2024-03-22T04:22:28.294 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38985.log.gz 2024-03-22T04:22:28.295 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35523.log 2024-03-22T04:22:28.295 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49391.log.gz 2024-03-22T04:22:28.295 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50200.log 2024-03-22T04:22:28.295 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52434.log.gz 2024-03-22T04:22:28.295 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46270.log 2024-03-22T04:22:28.295 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35523.log.gz 2024-03-22T04:22:28.296 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37446.log 2024-03-22T04:22:28.296 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50200.log.gz 2024-03-22T04:22:28.297 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41092.log 2024-03-22T04:22:28.297 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46270.log.gz 2024-03-22T04:22:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42991.log 2024-03-22T04:22:28.298 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37446.log.gz 2024-03-22T04:22:28.299 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33453.log 2024-03-22T04:22:28.299 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41092.log.gz 2024-03-22T04:22:28.300 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34587.log 2024-03-22T04:22:28.300 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42991.log.gz 2024-03-22T04:22:28.301 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-03-22T04:22:28.301 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33453.log.gz 2024-03-22T04:22:28.302 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41565.log 2024-03-22T04:22:28.302 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34587.log.gz 2024-03-22T04:22:28.302 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41883.log 2024-03-22T04:22:28.303 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-03-22T04:22:28.304 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52019.log 2024-03-22T04:22:28.304 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41565.log.gz 2024-03-22T04:22:28.304 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37238.log 2024-03-22T04:22:28.305 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41883.log.gz 2024-03-22T04:22:28.306 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52019.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52019.log.gz 2024-03-22T04:22:28.306 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37472.log 2024-03-22T04:22:28.306 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45248.log 2024-03-22T04:22:28.307 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37238.log.gz 2024-03-22T04:22:28.307 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44052.log 2024-03-22T04:22:28.308 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37472.log.gz 2024-03-22T04:22:28.308 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-03-22T04:22:28.308 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45248.log.gz 2024-03-22T04:22:28.309 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48286.log 2024-03-22T04:22:28.309 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44052.log.gz 2024-03-22T04:22:28.310 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37934.log 2024-03-22T04:22:28.310 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-03-22T04:22:28.311 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48234.log 2024-03-22T04:22:28.311 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48286.log.gz 2024-03-22T04:22:28.312 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49670.log 2024-03-22T04:22:28.312 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37934.log.gz 2024-03-22T04:22:28.313 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37264.log 2024-03-22T04:22:28.313 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48234.log.gz 2024-03-22T04:22:28.314 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52279.log 2024-03-22T04:22:28.314 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49670.log.gz 2024-03-22T04:22:28.315 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42699.log 2024-03-22T04:22:28.315 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37264.log.gz 2024-03-22T04:22:28.316 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42512.log 2024-03-22T04:22:28.316 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52279.log.gz 2024-03-22T04:22:28.316 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34561.log 2024-03-22T04:22:28.316 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42699.log.gz 2024-03-22T04:22:28.317 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51830.log 2024-03-22T04:22:28.317 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42512.log.gz 2024-03-22T04:22:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34561.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34561.log.gz 2024-03-22T04:22:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50823.log 2024-03-22T04:22:28.319 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41144.log 2024-03-22T04:22:28.320 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51830.log.gz 2024-03-22T04:22:28.320 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53194.log 2024-03-22T04:22:28.320 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50823.log.gz 2024-03-22T04:22:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51877.log 2024-03-22T04:22:28.321 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41144.log.gz 2024-03-22T04:22:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41064.log 2024-03-22T04:22:28.322 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53194.log.gz 2024-03-22T04:22:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.51877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51877.log.gz 2024-03-22T04:22:28.323 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41513.log 2024-03-22T04:22:28.323 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37524.log 2024-03-22T04:22:28.324 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41064.log.gz 2024-03-22T04:22:28.325 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36918.log 2024-03-22T04:22:28.325 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41513.log.gz 2024-03-22T04:22:28.325 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41196.log 2024-03-22T04:22:28.326 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37524.log.gz 2024-03-22T04:22:28.327 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36918.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-22T04:22:28.327 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36918.log.gz 2024-03-22T04:22:28.327 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34639.log 2024-03-22T04:22:28.327 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41196.log.gz 2024-03-22T04:22:28.328 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49048.log 2024-03-22T04:22:28.329 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53302.log 2024-03-22T04:22:28.329 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34639.log.gz 2024-03-22T04:22:28.330 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42258.log 2024-03-22T04:22:28.330 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49048.log.gz 2024-03-22T04:22:28.330 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43204.log 2024-03-22T04:22:28.331 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53302.log.gz 2024-03-22T04:22:28.332 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52253.log 2024-03-22T04:22:28.332 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42258.log.gz 2024-03-22T04:22:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37498.log 2024-03-22T04:22:28.333 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43204.log.gz 2024-03-22T04:22:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34215.log 2024-03-22T04:22:28.334 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52253.log.gz 2024-03-22T04:22:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35367.log 2024-03-22T04:22:28.335 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37498.log.gz 2024-03-22T04:22:28.336 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2024-03-22T04:22:28.336 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34215.log.gz 2024-03-22T04:22:28.337 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33585.log 2024-03-22T04:22:28.338 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35367.log: /var/log/ceph/ceph-client.admin.44740.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35367.log.gz 2024-03-22T04:22:28.338 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45435.log 2024-03-22T04:22:28.339 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2024-03-22T04:22:28.339 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36565.log 2024-03-22T04:22:28.340 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33585.log.gz 2024-03-22T04:22:28.340 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33823.log 2024-03-22T04:22:28.341 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45435.log.gz 2024-03-22T04:22:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49726.log 2024-03-22T04:22:28.341 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36565.log.gz 2024-03-22T04:22:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40751.log 2024-03-22T04:22:28.343 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33823.log.gz 2024-03-22T04:22:28.343 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44995.log 2024-03-22T04:22:28.344 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49726.log.gz 2024-03-22T04:22:28.344 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2024-03-22T04:22:28.344 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40751.log.gz 2024-03-22T04:22:28.345 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34111.log 2024-03-22T04:22:28.346 INFO:teuthology.orchestra.run.smithi196.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-22T04:22:28.347 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46053.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44995.log.gz 2024-03-22T04:22:28.347 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34949.log 2024-03-22T04:22:28.347 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-03-22T04:22:28.348 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34111.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34111.log.gz 2024-03-22T04:22:28.348 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34429.log 2024-03-22T04:22:28.348 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36279.log 2024-03-22T04:22:28.349 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34949.log.gz 2024-03-22T04:22:28.350 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34429.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-22T04:22:28.351 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40563.log 2024-03-22T04:22:28.351 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34429.log.gz 2024-03-22T04:22:28.351 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41118.log 2024-03-22T04:22:28.351 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36279.log.gz 2024-03-22T04:22:28.351 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35757.log 2024-03-22T04:22:28.352 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40563.log.gz 2024-03-22T04:22:28.353 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52460.log 2024-03-22T04:22:28.353 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41118.log.gz 2024-03-22T04:22:28.354 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.35757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43465.log 2024-03-22T04:22:28.354 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35757.log.gz 2024-03-22T04:22:28.355 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49287.log 2024-03-22T04:22:28.355 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52460.log.gz 2024-03-22T04:22:28.355 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45071.log 2024-03-22T04:22:28.356 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43465.log.gz 2024-03-22T04:22:28.356 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49313.log 2024-03-22T04:22:28.356 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49287.log.gz 2024-03-22T04:22:28.358 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34509.log 2024-03-22T04:22:28.358 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45071.log.gz 2024-03-22T04:22:28.358 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49313.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49313.log.gz 2024-03-22T04:22:28.359 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50927.log 2024-03-22T04:22:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51702.log 2024-03-22T04:22:28.361 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34509.log.gz 2024-03-22T04:22:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34926.log 2024-03-22T04:22:28.361 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50927.log.gz 2024-03-22T04:22:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54042.log 2024-03-22T04:22:28.364 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51702.log.gz 2024-03-22T04:22:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34926.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-22T04:22:28.364 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34926.log.gz 2024-03-22T04:22:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43071.log 2024-03-22T04:22:28.364 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54042.log.gz 2024-03-22T04:22:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45123.log 2024-03-22T04:22:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43071.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43071.log.gz 2024-03-22T04:22:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48552.log 2024-03-22T04:22:28.367 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42006.log 2024-03-22T04:22:28.367 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.45123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45123.log.gz 2024-03-22T04:22:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38012.log 2024-03-22T04:22:28.368 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48552.log.gz 2024-03-22T04:22:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44531.log 2024-03-22T04:22:28.368 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42006.log.gz 2024-03-22T04:22:28.369 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39167.log 2024-03-22T04:22:28.369 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38012.log.gz 2024-03-22T04:22:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49235.log 2024-03-22T04:22:28.370 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44531.log.gz 2024-03-22T04:22:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39323.log 2024-03-22T04:22:28.371 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39167.log.gz 2024-03-22T04:22:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42913.log 2024-03-22T04:22:28.372 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49235.log.gz 2024-03-22T04:22:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39323.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39323.log.gz 2024-03-22T04:22:28.374 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54198.log 2024-03-22T04:22:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37077.log 2024-03-22T04:22:28.375 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42913.log.gz 2024-03-22T04:22:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38172.log 2024-03-22T04:22:28.376 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54198.log.gz 2024-03-22T04:22:28.377 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47800.log 2024-03-22T04:22:28.377 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37077.log.gz 2024-03-22T04:22:28.377 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34938.log 2024-03-22T04:22:28.378 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38172.log: /var/log/ceph/ceph-client.admin.47800.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38172.log.gz 2024-03-22T04:22:28.379 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40804.log 2024-03-22T04:22:28.379 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47800.log.gz 2024-03-22T04:22:28.380 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45662.log 2024-03-22T04:22:28.380 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34938.log.gz 2024-03-22T04:22:28.381 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39585.log 2024-03-22T04:22:28.381 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40804.log.gz 2024-03-22T04:22:28.382 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45118.log 2024-03-22T04:22:28.382 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45662.log.gz 2024-03-22T04:22:28.383 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42562.log 2024-03-22T04:22:28.383 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39585.log.gz 2024-03-22T04:22:28.384 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.45118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40882.log 2024-03-22T04:22:28.384 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45118.log.gz 2024-03-22T04:22:28.385 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49365.log 2024-03-22T04:22:28.385 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42562.log.gz 2024-03-22T04:22:28.386 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39297.log 2024-03-22T04:22:28.386 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40882.log.gz 2024-03-22T04:22:28.387 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46027.log 2024-03-22T04:22:28.387 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49365.log.gz 2024-03-22T04:22:28.388 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34738.log 2024-03-22T04:22:28.388 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39297.log.gz 2024-03-22T04:22:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34455.log 2024-03-22T04:22:28.389 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46027.log.gz 2024-03-22T04:22:28.390 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35799.log 2024-03-22T04:22:28.390 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34738.log.gz 2024-03-22T04:22:28.391 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39193.log 2024-03-22T04:22:28.391 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34455.log.gz 2024-03-22T04:22:28.392 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44612.log 2024-03-22T04:22:28.392 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35799.log.gz 2024-03-22T04:22:28.393 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51345.log 2024-03-22T04:22:28.394 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39193.log.gz 2024-03-22T04:22:28.394 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53860.log 2024-03-22T04:22:28.395 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44612.log.gz 2024-03-22T04:22:28.396 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43974.log 2024-03-22T04:22:28.396 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51345.log.gz 2024-03-22T04:22:28.396 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42334.log 2024-03-22T04:22:28.397 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53860.log.gz 2024-03-22T04:22:28.398 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36383.log 2024-03-22T04:22:28.398 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43974.log.gz 2024-03-22T04:22:28.402 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47402.log 2024-03-22T04:22:28.402 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42334.log.gz 2024-03-22T04:22:28.402 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53055.log 2024-03-22T04:22:28.402 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36383.log.gz 2024-03-22T04:22:28.403 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51371.log 2024-03-22T04:22:28.403 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47402.log.gz 2024-03-22T04:22:28.403 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40259.log 2024-03-22T04:22:28.404 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52954.log 2024-03-22T04:22:28.404 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.53055.log.gz 2024-03-22T04:22:28.404 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51371.log.gz 2024-03-22T04:22:28.404 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38707.log 2024-03-22T04:22:28.405 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40259.log.gz 2024-03-22T04:22:28.405 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37420.log 2024-03-22T04:22:28.405 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52954.log.gz 2024-03-22T04:22:28.406 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45274.log 2024-03-22T04:22:28.406 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38707.log.gz 2024-03-22T04:22:28.407 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48838.log 2024-03-22T04:22:28.407 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37420.log.gz 2024-03-22T04:22:28.408 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36809.log 2024-03-22T04:22:28.408 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45274.log.gz 2024-03-22T04:22:28.409 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43716.log 2024-03-22T04:22:28.410 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48838.log.gz 2024-03-22T04:22:28.410 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.36809.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50025.log 2024-03-22T04:22:28.411 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43716.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36809.log.gz 2024-03-22T04:22:28.411 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-03-22T04:22:28.411 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43716.log.gz 2024-03-22T04:22:28.412 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47480.log 2024-03-22T04:22:28.412 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50025.log.gz 2024-03-22T04:22:28.413 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43019.log 2024-03-22T04:22:28.414 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-03-22T04:22:28.414 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44714.log 2024-03-22T04:22:28.415 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47480.log.gz 2024-03-22T04:22:28.415 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39772.log 2024-03-22T04:22:28.416 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43019.log.gz 2024-03-22T04:22:28.416 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35993.log 2024-03-22T04:22:28.417 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44714.log.gz 2024-03-22T04:22:28.418 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34743.log 2024-03-22T04:22:28.418 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39772.log.gz 2024-03-22T04:22:28.419 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33426.log 2024-03-22T04:22:28.419 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35993.log.gz 2024-03-22T04:22:28.420 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46796.log 2024-03-22T04:22:28.420 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33426.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34743.log.gz 2024-03-22T04:22:28.421 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37803.log 2024-03-22T04:22:28.421 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33426.log.gz 2024-03-22T04:22:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46796.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46796.log.gz 2024-03-22T04:22:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52201.log 2024-03-22T04:22:28.422 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37803.log: /var/log/ceph/ceph-client.admin.43871.log 2024-03-22T04:22:28.423 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37803.log.gz 2024-03-22T04:22:28.423 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41539.log 2024-03-22T04:22:28.424 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52201.log.gz 2024-03-22T04:22:28.425 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51423.log 2024-03-22T04:22:28.425 INFO:teuthology.orchestra.run.smithi086.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.43871.log.gz 2024-03-22T04:22:28.426 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40507.log 2024-03-22T04:22:28.426 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41539.log.gz 2024-03-22T04:22:28.427 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47096.log 2024-03-22T04:22:28.427 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51423.log.gz 2024-03-22T04:22:28.428 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53354.log 2024-03-22T04:22:28.428 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2024-03-22T04:22:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38959.log 2024-03-22T04:22:28.430 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.53354.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.53354.log.gz -5 2024-03-22T04:22:28.430 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-03-22T04:22:28.430 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47096.log.gz 2024-03-22T04:22:28.431 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38959.log: /var/log/ceph/ceph-client.admin.39611.log 2024-03-22T04:22:28.432 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38959.log.gz 2024-03-22T04:22:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-03-22T04:22:28.432 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-03-22T04:22:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43152.log 2024-03-22T04:22:28.433 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39611.log.gz 2024-03-22T04:22:28.434 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33300.log 2024-03-22T04:22:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37908.log: /var/log/ceph/ceph-client.admin.43152.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-03-22T04:22:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38120.log 2024-03-22T04:22:28.436 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43152.log.gz 2024-03-22T04:22:28.437 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49157.log 2024-03-22T04:22:28.437 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33300.log.gz 2024-03-22T04:22:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36970.log 2024-03-22T04:22:28.439 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38120.log.gz 2024-03-22T04:22:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34665.log 2024-03-22T04:22:28.439 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49157.log.gz 2024-03-22T04:22:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53028.log 2024-03-22T04:22:28.440 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36970.log.gz 2024-03-22T04:22:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50875.log 2024-03-22T04:22:28.441 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34665.log.gz 2024-03-22T04:22:28.441 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51676.log 2024-03-22T04:22:28.443 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50875.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.53028.log.gz 2024-03-22T04:22:28.443 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53130.log 2024-03-22T04:22:28.443 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50875.log.gz 2024-03-22T04:22:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39746.log 2024-03-22T04:22:28.444 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51676.log.gz 2024-03-22T04:22:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51397.log 2024-03-22T04:22:28.445 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53130.log.gz 2024-03-22T04:22:28.446 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39746.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39746.log.gz 2024-03-22T04:22:28.446 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54432.log 2024-03-22T04:22:28.447 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.51397.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51540.log 2024-03-22T04:22:28.448 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54432.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51397.log.gz 2024-03-22T04:22:28.448 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42757.log 2024-03-22T04:22:28.448 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54432.log.gz 2024-03-22T04:22:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46502.log 2024-03-22T04:22:28.449 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51540.log.gz 2024-03-22T04:22:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52786.log 2024-03-22T04:22:28.450 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42757.log.gz 2024-03-22T04:22:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.46502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46502.log.gz 2024-03-22T04:22:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39954.log 2024-03-22T04:22:28.452 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41012.log 2024-03-22T04:22:28.452 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52786.log.gz 2024-03-22T04:22:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36461.log 2024-03-22T04:22:28.453 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39954.log.gz 2024-03-22T04:22:28.454 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36227.log 2024-03-22T04:22:28.455 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36461.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41012.log.gz 2024-03-22T04:22:28.455 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41038.log 2024-03-22T04:22:28.456 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36461.log.gz 2024-03-22T04:22:28.456 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36227.log.gz 2024-03-22T04:22:28.456 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38927.log 2024-03-22T04:22:28.457 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47146.log 2024-03-22T04:22:28.458 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41038.log: /var/log/ceph/ceph-client.admin.38927.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41038.log.gz 2024-03-22T04:22:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48500.log 2024-03-22T04:22:28.459 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38927.log.gz 2024-03-22T04:22:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33797.log 2024-03-22T04:22:28.460 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47146.log.gz 2024-03-22T04:22:28.461 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41644.log 2024-03-22T04:22:28.461 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48500.log.gz 2024-03-22T04:22:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46109.log 2024-03-22T04:22:28.462 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33797.log.gz 2024-03-22T04:22:28.463 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46971.log 2024-03-22T04:22:28.463 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41644.log.gz 2024-03-22T04:22:28.463 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41801.log 2024-03-22T04:22:28.464 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1%/var/log/ceph/ceph-client.0.46971.log: -- replaced with /var/log/ceph/ceph-client.admin.46109.log.gz 2024-03-22T04:22:28.464 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46971.log.gz 2024-03-22T04:22:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37602.log 2024-03-22T04:22:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41801.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41801.log.gz 2024-03-22T04:22:28.466 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44969.log 2024-03-22T04:22:28.467 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-03-22T04:22:28.467 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37602.log.gz 2024-03-22T04:22:28.468 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39824.log 2024-03-22T04:22:28.469 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44969.log.gz 2024-03-22T04:22:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52396.log 2024-03-22T04:22:28.470 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1%/var/log/ceph/ceph-client.admin.39824.log: -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-03-22T04:22:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39718.log 2024-03-22T04:22:28.471 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39824.log.gz 2024-03-22T04:22:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39037.log 2024-03-22T04:22:28.472 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52396.log.gz 2024-03-22T04:22:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37342.log 2024-03-22T04:22:28.473 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39718.log.gz 2024-03-22T04:22:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43742.log 2024-03-22T04:22:28.474 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39037.log.gz 2024-03-22T04:22:28.475 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38146.log 2024-03-22T04:22:28.475 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37342.log.gz 2024-03-22T04:22:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.43742.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48930.log 2024-03-22T04:22:28.476 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6%/var/log/ceph/ceph-client.admin.38146.log: -- replaced with /var/log/ceph/ceph-client.admin.43742.log.gz 2024-03-22T04:22:28.477 INFO:teuthology.orchestra.run.smithi196.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-22T04:22:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-22T04:22:28.477 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38146.log.gz 2024-03-22T04:22:28.478 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49938.log 2024-03-22T04:22:28.478 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48930.log.gz 2024-03-22T04:22:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36357.log 2024-03-22T04:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.49938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40404.log 2024-03-22T04:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49938.log.gz 2024-03-22T04:22:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44137.log 2024-03-22T04:22:28.481 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36357.log.gz 2024-03-22T04:22:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.40404.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50693.log 2024-03-22T04:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44137.log: 91.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.40404.log.gz 2024-03-22T04:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45923.log 2024-03-22T04:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.44137.log.gz 2024-03-22T04:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50459.log 2024-03-22T04:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50693.log.gz 2024-03-22T04:22:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39637.log 2024-03-22T04:22:28.485 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45923.log.gz 2024-03-22T04:22:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39876.log 2024-03-22T04:22:28.486 INFO:teuthology.orchestra.run.smithi086.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50459.log.gz 2024-03-22T04:22:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43439.log 2024-03-22T04:22:28.487 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39637.log.gz 2024-03-22T04:22:28.488 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33613.log 2024-03-22T04:22:28.488 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39876.log.gz 2024-03-22T04:22:28.489 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37129.log 2024-03-22T04:22:28.490 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43439.log.gz 2024-03-22T04:22:28.491 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48474.log 2024-03-22T04:22:28.491 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33613.log.gz 2024-03-22T04:22:28.491 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40136.log 2024-03-22T04:22:28.492 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37129.log.gz 2024-03-22T04:22:28.493 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35471.log 2024-03-22T04:22:28.493 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48474.log.gz 2024-03-22T04:22:28.494 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45513.log 2024-03-22T04:22:28.494 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.40136.log.gz 2024-03-22T04:22:28.495 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33665.log 2024-03-22T04:22:28.495 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35471.log.gz 2024-03-22T04:22:28.498 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40333.log 2024-03-22T04:22:28.498 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45513.log: /var/log/ceph/ceph-client.admin.33665.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45513.log.gz 2024-03-22T04:22:28.498 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34729.log 2024-03-22T04:22:28.498 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33665.log.gz 2024-03-22T04:22:28.498 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51451.log 2024-03-22T04:22:28.499 INFO:teuthology.orchestra.run.smithi086.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40333.log.gz 2024-03-22T04:22:28.499 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47558.log 2024-03-22T04:22:28.499 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34729.log.gz 2024-03-22T04:22:28.500 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34163.log 2024-03-22T04:22:28.501 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51451.log.gz 2024-03-22T04:22:28.501 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44402.log 2024-03-22T04:22:28.501 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47558.log.gz 2024-03-22T04:22:28.502 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36201.log 2024-03-22T04:22:28.502 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34163.log.gz 2024-03-22T04:22:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.0.44402.log: --verbose -- /var/log/ceph/ceph-client.admin.43820.log 2024-03-22T04:22:28.503 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44402.log.gz 2024-03-22T04:22:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40006.log 2024-03-22T04:22:28.504 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36201.log.gz 2024-03-22T04:22:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50097.log 2024-03-22T04:22:28.506 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43820.log.gz 2024-03-22T04:22:28.507 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53394.log 2024-03-22T04:22:28.507 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40006.log.gz 2024-03-22T04:22:28.508 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-03-22T04:22:28.508 INFO:teuthology.orchestra.run.smithi086.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.50097.log.gz 2024-03-22T04:22:28.509 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53607.log 2024-03-22T04:22:28.509 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53394.log.gz 2024-03-22T04:22:28.510 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38512.log 2024-03-22T04:22:28.510 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-03-22T04:22:28.511 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41670.log 2024-03-22T04:22:28.511 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53607.log.gz 2024-03-22T04:22:28.512 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47826.log 2024-03-22T04:22:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41670.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38512.log.gz 2024-03-22T04:22:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46497.log 2024-03-22T04:22:28.513 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41670.log.gz 2024-03-22T04:22:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51083.log 2024-03-22T04:22:28.515 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47826.log.gz 2024-03-22T04:22:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.46497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40110.log 2024-03-22T04:22:28.515 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46497.log.gz 2024-03-22T04:22:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53226.log 2024-03-22T04:22:28.517 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51083.log.gz 2024-03-22T04:22:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45357.log 2024-03-22T04:22:28.518 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40110.log.gz 2024-03-22T04:22:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43923.log 2024-03-22T04:22:28.519 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53226.log.gz 2024-03-22T04:22:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-03-22T04:22:28.520 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45357.log.gz 2024-03-22T04:22:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.43923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43923.log.gz 2024-03-22T04:22:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-03-22T04:22:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-03-22T04:22:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49749.log: /var/log/ceph/ceph-client.admin.45792.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-03-22T04:22:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48151.log 2024-03-22T04:22:28.523 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-03-22T04:22:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36253.log 2024-03-22T04:22:28.524 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-03-22T04:22:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48312.log 2024-03-22T04:22:28.526 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36253.log: 91.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.48151.log.gz 2024-03-22T04:22:28.527 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49700.log 2024-03-22T04:22:28.527 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36253.log.gz 2024-03-22T04:22:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41980.log 2024-03-22T04:22:28.527 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48312.log.gz 2024-03-22T04:22:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38861.log 2024-03-22T04:22:28.528 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49700.log.gz 2024-03-22T04:22:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.41980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41980.log.gz 2024-03-22T04:22:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-22T04:22:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41352.log 2024-03-22T04:22:28.530 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38861.log.gz 2024-03-22T04:22:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48656.log 2024-03-22T04:22:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43360.log 2024-03-22T04:22:28.532 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41352.log.gz 2024-03-22T04:22:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40457.log 2024-03-22T04:22:28.534 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48656.log.gz 2024-03-22T04:22:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43360.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43360.log.gz 2024-03-22T04:22:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37212.log 2024-03-22T04:22:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40457.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40457.log.gz 2024-03-22T04:22:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42436.log 2024-03-22T04:22:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51005.log 2024-03-22T04:22:28.538 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37212.log.gz 2024-03-22T04:22:28.538 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39219.log 2024-03-22T04:22:28.539 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51005.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42436.log.gz 2024-03-22T04:22:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36305.log 2024-03-22T04:22:28.540 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51005.log.gz 2024-03-22T04:22:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48260.log 2024-03-22T04:22:28.541 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39219.log.gz 2024-03-22T04:22:28.542 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36097.log 2024-03-22T04:22:28.543 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36305.log.gz 2024-03-22T04:22:28.543 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36591.log 2024-03-22T04:22:28.543 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48260.log.gz 2024-03-22T04:22:28.544 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50633.log 2024-03-22T04:22:28.545 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36097.log.gz 2024-03-22T04:22:28.546 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33849.log 2024-03-22T04:22:28.546 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36591.log.gz 2024-03-22T04:22:28.547 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46189.log 2024-03-22T04:22:28.547 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50633.log.gz 2024-03-22T04:22:28.548 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54406.log 2024-03-22T04:22:28.548 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33849.log.gz 2024-03-22T04:22:28.549 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38883.log 2024-03-22T04:22:28.549 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46189.log.gz 2024-03-22T04:22:28.551 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48073.log 2024-03-22T04:22:28.551 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54406.log.gz 2024-03-22T04:22:28.552 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54484.log 2024-03-22T04:22:28.552 INFO:teuthology.orchestra.run.smithi086.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.38883.log.gz 2024-03-22T04:22:28.553 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37829.log 2024-03-22T04:22:28.553 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48073.log.gz 2024-03-22T04:22:28.554 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54484.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54484.log.gz 2024-03-22T04:22:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40084.log 2024-03-22T04:22:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37829.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37829.log.gz 2024-03-22T04:22:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36435.log 2024-03-22T04:22:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43178.log 2024-03-22T04:22:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40084.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40084.log.gz 2024-03-22T04:22:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36435.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36435.log.gz 2024-03-22T04:22:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53446.log 2024-03-22T04:22:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43178.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43178.log.gz 2024-03-22T04:22:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38789.log 2024-03-22T04:22:28.560 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35445.log 2024-03-22T04:22:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53446.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53446.log.gz 2024-03-22T04:22:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38789.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38789.log.gz 2024-03-22T04:22:28.563 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36840.log 2024-03-22T04:22:28.563 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35445.log.gz 2024-03-22T04:22:28.563 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41378.log 2024-03-22T04:22:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42835.log 2024-03-22T04:22:28.565 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36840.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36840.log.gz 2024-03-22T04:22:28.566 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35885.log 2024-03-22T04:22:28.566 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41378.log.gz 2024-03-22T04:22:28.567 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47043.log 2024-03-22T04:22:28.567 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42835.log.gz 2024-03-22T04:22:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.2.35885.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.2.35885.log.gz -5 2024-03-22T04:22:28.568 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.0.44397.log 2024-03-22T04:22:28.569 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41487.log 2024-03-22T04:22:28.570 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47043.log.gz 2024-03-22T04:22:28.570 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.44397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44397.log.gz 2024-03-22T04:22:28.571 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33233.log 2024-03-22T04:22:28.571 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41487.log.gz 2024-03-22T04:22:28.572 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44082.log 2024-03-22T04:22:28.573 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48812.log 2024-03-22T04:22:28.574 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33233.log: /var/log/ceph/ceph-client.admin.44082.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33233.log.gz 2024-03-22T04:22:28.574 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44082.log.gz 2024-03-22T04:22:28.575 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2024-03-22T04:22:28.575 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-22T04:22:28.575 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48812.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48812.log.gz 2024-03-22T04:22:28.576 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36783.log 2024-03-22T04:22:28.577 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52371.log 2024-03-22T04:22:28.578 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51780.log: /var/log/ceph/ceph-client.admin.36783.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2024-03-22T04:22:28.578 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36783.log.gz 2024-03-22T04:22:28.578 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40032.log 2024-03-22T04:22:28.579 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52371.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.52371.log.gz 2024-03-22T04:22:28.580 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34268.log 2024-03-22T04:22:28.581 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38839.log 2024-03-22T04:22:28.581 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40032.log.gz 2024-03-22T04:22:28.582 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34268.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34268.log.gz 2024-03-22T04:22:28.582 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41229.log 2024-03-22T04:22:28.583 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38839.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38839.log.gz 2024-03-22T04:22:28.584 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47506.log 2024-03-22T04:22:28.585 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51650.log 2024-03-22T04:22:28.585 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41229.log.gz 2024-03-22T04:22:28.586 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43690.log 2024-03-22T04:22:28.586 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47506.log.gz 2024-03-22T04:22:28.587 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51650.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51650.log.gz 2024-03-22T04:22:28.587 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42052.log 2024-03-22T04:22:28.588 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53817.log 2024-03-22T04:22:28.588 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43690.log.gz 2024-03-22T04:22:28.590 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39427.log 2024-03-22T04:22:28.590 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42052.log.gz 2024-03-22T04:22:28.591 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53817.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53817.log.gz 2024-03-22T04:22:28.591 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41722.log 2024-03-22T04:22:28.591 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45383.log 2024-03-22T04:22:28.592 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39427.log.gz 2024-03-22T04:22:28.593 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2024-03-22T04:22:28.593 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41722.log.gz 2024-03-22T04:22:28.594 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45383.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45383.log.gz 2024-03-22T04:22:28.594 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48370.log 2024-03-22T04:22:28.595 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41933.log 2024-03-22T04:22:28.596 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz 2024-03-22T04:22:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39141.log 2024-03-22T04:22:28.596 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48370.log.gz 2024-03-22T04:22:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41933.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41933.log.gz 2024-03-22T04:22:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39271.log 2024-03-22T04:22:28.599 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38733.log 2024-03-22T04:22:28.599 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39141.log.gz 2024-03-22T04:22:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45305.log 2024-03-22T04:22:28.600 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39271.log.gz 2024-03-22T04:22:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38733.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38733.log.gz 2024-03-22T04:22:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45949.log 2024-03-22T04:22:28.602 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45714.log 2024-03-22T04:22:28.602 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45305.log.gz 2024-03-22T04:22:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49592.log 2024-03-22T04:22:28.604 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45949.log.gz 2024-03-22T04:22:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45714.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45714.log.gz 2024-03-22T04:22:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36643.log 2024-03-22T04:22:28.606 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37738.log 2024-03-22T04:22:28.606 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49592.log.gz 2024-03-22T04:22:28.607 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49964.log 2024-03-22T04:22:28.607 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36643.log.gz 2024-03-22T04:22:28.608 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37738.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37738.log.gz 2024-03-22T04:22:28.608 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38224.log 2024-03-22T04:22:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35890.log 2024-03-22T04:22:28.610 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49964.log.gz 2024-03-22T04:22:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38224.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38224.log.gz 2024-03-22T04:22:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50656.log 2024-03-22T04:22:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.2.35890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35890.log.gz 2024-03-22T04:22:28.612 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51598.log 2024-03-22T04:22:28.613 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37986.log 2024-03-22T04:22:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50656.log: /var/log/ceph/ceph-client.admin.51598.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.50656.log.gz 2024-03-22T04:22:28.615 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51598.log.gz 2024-03-22T04:22:28.615 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37986.log: -- /var/log/ceph/ceph-client.admin.47902.log 2024-03-22T04:22:28.616 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37986.log.gz 2024-03-22T04:22:28.616 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35263.log 2024-03-22T04:22:28.617 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54250.log 2024-03-22T04:22:28.618 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47902.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47902.log.gz 2024-03-22T04:22:28.618 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35263.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35263.log.gz 2024-03-22T04:22:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48019.log 2024-03-22T04:22:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54250.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54250.log.gz 2024-03-22T04:22:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38092.log 2024-03-22T04:22:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47954.log 2024-03-22T04:22:28.622 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48019.log: /var/log/ceph/ceph-client.admin.38092.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48019.log.gz 2024-03-22T04:22:28.622 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38092.log.gz 2024-03-22T04:22:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37882.log 2024-03-22T04:22:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.47954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47954.log.gz 2024-03-22T04:22:28.624 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33505.log 2024-03-22T04:22:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38614.log 2024-03-22T04:22:28.625 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37882.log.gz 2024-03-22T04:22:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44425.log 2024-03-22T04:22:28.627 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33505.log.gz 2024-03-22T04:22:28.627 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.38614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38614.log.gz 2024-03-22T04:22:28.627 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39980.log 2024-03-22T04:22:28.629 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39690.log 2024-03-22T04:22:28.629 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44425.log.gz 2024-03-22T04:22:28.630 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34294.log 2024-03-22T04:22:28.630 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39980.log.gz 2024-03-22T04:22:28.631 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39690.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-22T04:22:28.631 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39690.log.gz 2024-03-22T04:22:28.632 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34294.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34294.log.gz 2024-03-22T04:22:28.633 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42939.log 2024-03-22T04:22:28.634 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47748.log 2024-03-22T04:22:28.635 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42939.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42939.log.gz 2024-03-22T04:22:28.636 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47748.log: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-22T04:22:28.637 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47748.log.gz 2024-03-22T04:22:28.637 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40908.log 2024-03-22T04:22:28.637 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48898.log 2024-03-22T04:22:28.638 INFO:teuthology.orchestra.run.smithi196.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-22T04:22:28.639 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-03-22T04:22:28.639 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40908.log.gz 2024-03-22T04:22:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47641.log 2024-03-22T04:22:28.640 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48898.log.gz 2024-03-22T04:22:28.641 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52678.log 2024-03-22T04:22:28.641 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz 2024-03-22T04:22:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47641.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47641.log.gz 2024-03-22T04:22:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52730.log 2024-03-22T04:22:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40699.log 2024-03-22T04:22:28.643 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52678.log.gz 2024-03-22T04:22:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47248.log 2024-03-22T04:22:28.644 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52730.log.gz 2024-03-22T04:22:28.645 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40699.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40699.log.gz 2024-03-22T04:22:28.646 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35419.log 2024-03-22T04:22:28.647 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42384.log 2024-03-22T04:22:28.648 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35419.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47248.log.gz 2024-03-22T04:22:28.648 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35419.log.gz 2024-03-22T04:22:28.649 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54538.log 2024-03-22T04:22:28.650 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47584.log 2024-03-22T04:22:28.650 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42384.log.gz 2024-03-22T04:22:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52517.log 2024-03-22T04:22:28.651 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54538.log.gz 2024-03-22T04:22:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41748.log 2024-03-22T04:22:28.653 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47584.log.gz 2024-03-22T04:22:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2024-03-22T04:22:28.654 INFO:teuthology.orchestra.run.smithi086.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.52517.log.gz 2024-03-22T04:22:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39011.log 2024-03-22T04:22:28.655 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41748.log.gz 2024-03-22T04:22:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34953.log 2024-03-22T04:22:28.656 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz 2024-03-22T04:22:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33719.log 2024-03-22T04:22:28.657 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39011.log.gz 2024-03-22T04:22:28.658 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52146.log 2024-03-22T04:22:28.658 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34953.log.gz 2024-03-22T04:22:28.659 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33719.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33719.log.gz 2024-03-22T04:22:28.659 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46712.log 2024-03-22T04:22:28.660 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.52146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52146.log.gz 2024-03-22T04:22:28.660 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51221.log 2024-03-22T04:22:28.662 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48864.log 2024-03-22T04:22:28.662 INFO:teuthology.orchestra.run.smithi086.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.46712.log.gz 2024-03-22T04:22:28.662 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45591.log 2024-03-22T04:22:28.663 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51221.log.gz 2024-03-22T04:22:28.664 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42127.log 2024-03-22T04:22:28.664 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48864.log.gz 2024-03-22T04:22:28.665 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45591.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45591.log.gz 2024-03-22T04:22:28.665 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39245.log 2024-03-22T04:22:28.666 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53763.log 2024-03-22T04:22:28.667 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42127.log.gz/var/log/ceph/ceph-client.admin.39245.log: 2024-03-22T04:22:28.667 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39245.log.gz 2024-03-22T04:22:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48422.log 2024-03-22T04:22:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53763.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53763.log.gz 2024-03-22T04:22:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45409.log 2024-03-22T04:22:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50979.log 2024-03-22T04:22:28.670 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48422.log.gz 2024-03-22T04:22:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36487.log 2024-03-22T04:22:28.672 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45409.log.gz 2024-03-22T04:22:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34189.log 2024-03-22T04:22:28.673 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50979.log.gz 2024-03-22T04:22:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36487.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36487.log.gz 2024-03-22T04:22:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39115.log 2024-03-22T04:22:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34189.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34189.log.gz 2024-03-22T04:22:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37186.log 2024-03-22T04:22:28.676 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53737.log 2024-03-22T04:22:28.677 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39115.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39115.log.gz 2024-03-22T04:22:28.677 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37186.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37186.log.gz 2024-03-22T04:22:28.678 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44451.log 2024-03-22T04:22:28.679 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53737.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53737.log.gz 2024-03-22T04:22:28.679 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-03-22T04:22:28.680 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42621.log 2024-03-22T04:22:28.681 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44451.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44451.log.gz/var/log/ceph/ceph-client.admin.46374.log: 2024-03-22T04:22:28.681 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-03-22T04:22:28.682 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44816.log 2024-03-22T04:22:28.683 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42621.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42621.log.gz 2024-03-22T04:22:28.683 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44479.log 2024-03-22T04:22:28.684 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-03-22T04:22:28.685 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44816.log: /var/log/ceph/ceph-client.admin.44479.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44816.log.gz 2024-03-22T04:22:28.685 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44479.log.gz 2024-03-22T04:22:28.686 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.45636.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45045.log 2024-03-22T04:22:28.686 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-03-22T04:22:28.687 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50583.log 2024-03-22T04:22:28.688 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35914.log 2024-03-22T04:22:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45045.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45045.log.gz 2024-03-22T04:22:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50583.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50583.log.gz 2024-03-22T04:22:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-03-22T04:22:28.690 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35914.log.gz 2024-03-22T04:22:28.691 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44505.log 2024-03-22T04:22:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43794.log 2024-03-22T04:22:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50458.log: /var/log/ceph/ceph-client.admin.44505.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-03-22T04:22:28.694 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44505.log.gz 2024-03-22T04:22:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40777.log 2024-03-22T04:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43794.log.gz 2024-03-22T04:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36731.log 2024-03-22T04:22:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50354.log 2024-03-22T04:22:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40777.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40777.log.gz 2024-03-22T04:22:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36731.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36731.log.gz 2024-03-22T04:22:28.698 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50745.log 2024-03-22T04:22:28.699 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50354.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50354.log.gz 2024-03-22T04:22:28.699 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53168.log 2024-03-22T04:22:28.701 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48682.log 2024-03-22T04:22:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50745.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50745.log.gz 2024-03-22T04:22:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53168.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53168.log.gz 2024-03-22T04:22:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52049.log 2024-03-22T04:22:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48682.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48682.log.gz 2024-03-22T04:22:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49545.log 2024-03-22T04:22:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36617.log 2024-03-22T04:22:28.705 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52049.log: /var/log/ceph/ceph-client.0.49545.log: 0.0% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52049.log.gz 2024-03-22T04:22:28.706 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.0.49545.log.gz 2024-03-22T04:22:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52857.log 2024-03-22T04:22:28.707 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36617.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36617.log.gz 2024-03-22T04:22:28.707 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51754.log 2024-03-22T04:22:28.707 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40667.log 2024-03-22T04:22:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52857.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52857.log.gz/var/log/ceph/ceph-client.admin.51754.log: 2024-03-22T04:22:28.709 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51754.log.gz 2024-03-22T04:22:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38064.log 2024-03-22T04:22:28.710 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40667.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40667.log.gz 2024-03-22T04:22:28.710 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45137.log 2024-03-22T04:22:28.711 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45331.log 2024-03-22T04:22:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38064.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38064.log.gz 2024-03-22T04:22:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45137.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.45137.log.gz 2024-03-22T04:22:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39928.log 2024-03-22T04:22:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45331.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45331.log.gz 2024-03-22T04:22:28.715 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48578.log 2024-03-22T04:22:28.716 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39928.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39928.log.gz 2024-03-22T04:22:28.716 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43664.log 2024-03-22T04:22:28.717 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42783.log 2024-03-22T04:22:28.718 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48578.log.gz 2024-03-22T04:22:28.719 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43664.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43664.log.gz 2024-03-22T04:22:28.719 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-03-22T04:22:28.720 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42783.log.gz 2024-03-22T04:22:28.720 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38250.log 2024-03-22T04:22:28.722 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43948.log 2024-03-22T04:22:28.722 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-03-22T04:22:28.723 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54016.log 2024-03-22T04:22:28.723 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38250.log.gz 2024-03-22T04:22:28.725 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43948.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43948.log.gz 2024-03-22T04:22:28.725 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36409.log 2024-03-22T04:22:28.725 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54016.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54016.log.gz 2024-03-22T04:22:28.725 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33479.log 2024-03-22T04:22:28.726 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34691.log 2024-03-22T04:22:28.727 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36409.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36409.log.gz 2024-03-22T04:22:28.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33479.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33479.log.gz 2024-03-22T04:22:28.728 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43295.log 2024-03-22T04:22:28.729 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34691.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34691.log.gz 2024-03-22T04:22:28.729 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37290.log 2024-03-22T04:22:28.730 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33925.log 2024-03-22T04:22:28.731 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43295.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43295.log.gz/var/log/ceph/ceph-client.admin.37290.log: 2024-03-22T04:22:28.731 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37290.log.gz 2024-03-22T04:22:28.732 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50771.log 2024-03-22T04:22:28.733 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33925.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33925.log.gz 2024-03-22T04:22:28.733 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44561.log 2024-03-22T04:22:28.734 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37654.log 2024-03-22T04:22:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50771.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50771.log.gz/var/log/ceph/ceph-client.admin.44561.log: 2024-03-22T04:22:28.736 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44561.log.gz 2024-03-22T04:22:28.737 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38460.log 2024-03-22T04:22:28.737 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37654.log.gz 2024-03-22T04:22:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47172.log 2024-03-22T04:22:28.739 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40282.log 2024-03-22T04:22:28.741 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38460.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38460.log.gz/var/log/ceph/ceph-client.admin.47172.log: 2024-03-22T04:22:28.741 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34320.log 2024-03-22T04:22:28.741 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.47172.log.gz/var/log/ceph/ceph-client.admin.40282.log: 2024-03-22T04:22:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37960.log 2024-03-22T04:22:28.742 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40282.log.gz 2024-03-22T04:22:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40186.log 2024-03-22T04:22:28.743 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34320.log.gz 2024-03-22T04:22:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39455.log 2024-03-22T04:22:28.744 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37960.log.gz 2024-03-22T04:22:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40186.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40186.log.gz 2024-03-22T04:22:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44664.log 2024-03-22T04:22:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39488.log 2024-03-22T04:22:28.746 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39455.log.gz 2024-03-22T04:22:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54328.log 2024-03-22T04:22:28.748 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44664.log.gz/var/log/ceph/ceph-client.admin.39488.log: 2024-03-22T04:22:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53498.log 2024-03-22T04:22:28.749 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39488.log.gz 2024-03-22T04:22:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54328.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54328.log.gz 2024-03-22T04:22:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34535.log 2024-03-22T04:22:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53498.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53498.log.gz 2024-03-22T04:22:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2024-03-22T04:22:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33335.log 2024-03-22T04:22:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34535.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34535.log.gz/var/log/ceph/ceph-client.admin.47664.log: 2024-03-22T04:22:28.754 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33335.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz 2024-03-22T04:22:28.755 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33335.log.gz 2024-03-22T04:22:28.755 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49912.log 2024-03-22T04:22:28.756 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27528.log 2024-03-22T04:22:28.757 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27631.log 2024-03-22T04:22:28.757 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49912.log.gz 2024-03-22T04:22:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39798.log 2024-03-22T04:22:28.758 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27528.log.gz 2024-03-22T04:22:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27631.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.27631.log.gz 2024-03-22T04:22:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52904.log 2024-03-22T04:22:28.760 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39798.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39798.log.gz 2024-03-22T04:22:28.761 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51135.log 2024-03-22T04:22:28.762 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51993.log 2024-03-22T04:22:28.763 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52904.log: /var/log/ceph/ceph-client.admin.51135.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52904.log.gz 2024-03-22T04:22:28.763 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51135.log.gz 2024-03-22T04:22:28.764 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-03-22T04:22:28.764 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51993.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51993.log.gz 2024-03-22T04:22:28.765 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48396.log 2024-03-22T04:22:28.766 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37855.log 2024-03-22T04:22:28.766 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41459.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2024-03-22T04:22:28.767 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48396.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48396.log.gz 2024-03-22T04:22:28.767 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38681.log 2024-03-22T04:22:28.768 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37855.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37855.log.gz 2024-03-22T04:22:28.769 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47691.log 2024-03-22T04:22:28.770 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35967.log 2024-03-22T04:22:28.770 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38681.log.gz 2024-03-22T04:22:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47691.log: /var/log/ceph/ceph-client.admin.35967.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35967.log.gz 2024-03-22T04:22:28.773 INFO:teuthology.orchestra.run.smithi086.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.47691.log.gz 2024-03-22T04:22:28.773 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36071.log 2024-03-22T04:22:28.774 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47969.log 2024-03-22T04:22:28.775 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52175.log 2024-03-22T04:22:28.775 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36071.log.gz 2024-03-22T04:22:28.776 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41985.log 2024-03-22T04:22:28.777 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52175.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.47969.log.gz 2024-03-22T04:22:28.777 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52175.log.gz 2024-03-22T04:22:28.778 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.41985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46400.log 2024-03-22T04:22:28.778 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41985.log.gz 2024-03-22T04:22:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50302.log 2024-03-22T04:22:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35133.log 2024-03-22T04:22:28.781 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46400.log.gz/var/log/ceph/ceph-client.admin.50302.log: 2024-03-22T04:22:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40233.log 2024-03-22T04:22:28.782 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50302.log.gz 2024-03-22T04:22:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35133.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35133.log.gz 2024-03-22T04:22:28.783 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46296.log 2024-03-22T04:22:28.784 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.40233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40233.log.gz 2024-03-22T04:22:28.784 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52567.log 2024-03-22T04:22:28.785 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36892.log 2024-03-22T04:22:28.786 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46296.log: 88.7%/var/log/ceph/ceph-client.admin.52567.log: -- replaced with /var/log/ceph/ceph-client.admin.46296.log.gz 2024-03-22T04:22:28.787 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52567.log.gz 2024-03-22T04:22:28.787 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43608.log 2024-03-22T04:22:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36892.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36892.log.gz 2024-03-22T04:22:28.788 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38328.log 2024-03-22T04:22:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51882.log 2024-03-22T04:22:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43608.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43608.log.gz 2024-03-22T04:22:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38328.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38328.log.gz 2024-03-22T04:22:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45170.log 2024-03-22T04:22:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.51882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51882.log.gz 2024-03-22T04:22:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-22T04:22:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54120.log 2024-03-22T04:22:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45170.log: 89.4%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.45170.log.gz 2024-03-22T04:22:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54120.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54120.log.gz 2024-03-22T04:22:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-22T04:22:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46450.log 2024-03-22T04:22:28.797 INFO:teuthology.orchestra.run.smithi086.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-22T04:22:28.798 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39063.log 2024-03-22T04:22:28.800 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46450.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46450.log.gz 2024-03-22T04:22:28.800 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.39063.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38905.log 2024-03-22T04:22:28.801 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39063.log.gz 2024-03-22T04:22:28.802 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46689.log 2024-03-22T04:22:28.803 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38905.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-22T04:22:28.803 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38905.log.gz 2024-03-22T04:22:28.804 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53659.log 2024-03-22T04:22:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46689.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46689.log.gz 2024-03-22T04:22:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51057.log 2024-03-22T04:22:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54562.log 2024-03-22T04:22:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53659.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53659.log.gz 2024-03-22T04:22:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51057.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51057.log.gz 2024-03-22T04:22:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-03-22T04:22:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54562.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54562.log.gz 2024-03-22T04:22:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54380.log 2024-03-22T04:22:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-22T04:22:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43638.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-03-22T04:22:28.811 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54380.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54380.log.gz 2024-03-22T04:22:28.811 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43230.log 2024-03-22T04:22:28.813 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49129.log 2024-03-22T04:22:28.814 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43230.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43230.log.gz 2024-03-22T04:22:28.815 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49129.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49129.log.gz 2024-03-22T04:22:28.815 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-03-22T04:22:28.817 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35941.log 2024-03-22T04:22:28.818 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48786.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-03-22T04:22:28.819 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35941.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35941.log.gz 2024-03-22T04:22:28.820 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34137.log 2024-03-22T04:22:28.820 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33745.log 2024-03-22T04:22:28.822 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34137.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34137.log.gz 2024-03-22T04:22:28.823 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33745.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33745.log.gz 2024-03-22T04:22:28.823 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49261.log 2024-03-22T04:22:28.825 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38382.log 2024-03-22T04:22:28.826 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49261.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49261.log.gz 2024-03-22T04:22:28.827 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38382.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38382.log.gz 2024-03-22T04:22:28.828 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33693.log 2024-03-22T04:22:28.829 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34483.log 2024-03-22T04:22:28.830 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33693.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33693.log.gz 2024-03-22T04:22:28.831 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34483.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34483.log.gz 2024-03-22T04:22:28.832 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49209.log 2024-03-22T04:22:28.833 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38815.log 2024-03-22T04:22:28.835 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49209.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49209.log.gz 2024-03-22T04:22:28.835 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38815.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38815.log.gz 2024-03-22T04:22:28.836 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36757.log 2024-03-22T04:22:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49994.log 2024-03-22T04:22:28.839 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36757.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36757.log.gz 2024-03-22T04:22:28.839 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49994.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49994.log.gz 2024-03-22T04:22:28.840 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-03-22T04:22:28.841 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54302.log 2024-03-22T04:22:28.843 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46581.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-03-22T04:22:28.843 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54302.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54302.log.gz 2024-03-22T04:22:28.844 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52652.log 2024-03-22T04:22:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38276.log 2024-03-22T04:22:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52652.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52652.log.gz 2024-03-22T04:22:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38276.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38276.log.gz 2024-03-22T04:22:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51967.log 2024-03-22T04:22:28.849 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52756.log 2024-03-22T04:22:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51967.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51967.log.gz 2024-03-22T04:22:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52756.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52756.log.gz 2024-03-22T04:22:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54458.log 2024-03-22T04:22:28.854 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34054.log 2024-03-22T04:22:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54458.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54458.log.gz 2024-03-22T04:22:28.857 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35289.log 2024-03-22T04:22:28.860 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41696.log 2024-03-22T04:22:28.860 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53555.log 2024-03-22T04:22:28.860 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35289.log.gz 2024-03-22T04:22:28.861 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41696.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41696.log.gz 2024-03-22T04:22:28.862 INFO:teuthology.orchestra.run.smithi086.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.34054.log.gz 2024-03-22T04:22:28.862 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2024-03-22T04:22:28.863 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37160.log 2024-03-22T04:22:28.865 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53555.log: /var/log/ceph/ceph-client.admin.34613.log: 88.9% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2024-03-22T04:22:28.865 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53555.log.gz 2024-03-22T04:22:28.866 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47774.log 2024-03-22T04:22:28.867 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37160.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37160.log.gz 2024-03-22T04:22:28.868 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47774.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47774.log.gz 2024-03-22T04:22:28.869 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44919.log 2024-03-22T04:22:28.870 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42731.log 2024-03-22T04:22:28.872 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44919.log: 91.1%/var/log/ceph/ceph-client.admin.42731.log: -- replaced with /var/log/ceph/ceph-client.admin.44919.log.gz 2024-03-22T04:22:28.873 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42731.log.gz 2024-03-22T04:22:28.873 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53082.log 2024-03-22T04:22:28.874 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47428.log 2024-03-22T04:22:28.878 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53082.log: /var/log/ceph/ceph-client.admin.47428.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.53082.log.gz 2024-03-22T04:22:28.878 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-03-22T04:22:28.878 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47428.log.gz 2024-03-22T04:22:28.879 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42647.log 2024-03-22T04:22:28.880 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51109.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-03-22T04:22:28.881 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42647.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42647.log.gz 2024-03-22T04:22:28.882 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54276.log 2024-03-22T04:22:28.883 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38302.log 2024-03-22T04:22:28.885 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54276.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54276.log.gz 2024-03-22T04:22:28.885 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38302.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38302.log.gz 2024-03-22T04:22:28.886 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36539.log 2024-03-22T04:22:28.887 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47949.log 2024-03-22T04:22:28.889 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36539.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36539.log.gz 2024-03-22T04:22:28.889 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.47949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47949.log.gz 2024-03-22T04:22:28.890 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44790.log 2024-03-22T04:22:28.891 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35497.log 2024-03-22T04:22:28.893 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44790.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44790.log.gz 2024-03-22T04:22:28.893 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35497.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35497.log.gz 2024-03-22T04:22:28.894 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49100.log 2024-03-22T04:22:28.895 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49886.log 2024-03-22T04:22:28.897 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49100.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49100.log.gz 2024-03-22T04:22:28.897 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49886.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49886.log.gz 2024-03-22T04:22:28.898 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39902.log 2024-03-22T04:22:28.899 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40238.log 2024-03-22T04:22:28.901 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39902.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39902.log.gz 2024-03-22T04:22:28.901 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.40238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40238.log.gz 2024-03-22T04:22:28.902 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49828.log 2024-03-22T04:22:28.906 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47718.log 2024-03-22T04:22:28.906 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.49828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49828.log.gz 2024-03-22T04:22:28.907 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47718.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.47718.log.gz 2024-03-22T04:22:28.907 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50719.log 2024-03-22T04:22:28.908 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43045.log 2024-03-22T04:22:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50719.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50719.log.gz 2024-03-22T04:22:28.911 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43045.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43045.log.gz 2024-03-22T04:22:28.912 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47532.log 2024-03-22T04:22:28.913 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53252.log 2024-03-22T04:22:28.915 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47532.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47532.log.gz 2024-03-22T04:22:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53252.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.53252.log.gz 2024-03-22T04:22:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48604.log 2024-03-22T04:22:28.917 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46163.log 2024-03-22T04:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48604.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48604.log.gz 2024-03-22T04:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46163.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46163.log.gz 2024-03-22T04:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38562.log 2024-03-22T04:22:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53529.log 2024-03-22T04:22:28.923 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38562.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38562.log.gz 2024-03-22T04:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53529.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53529.log.gz 2024-03-22T04:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46924.log 2024-03-22T04:22:28.926 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43097.log 2024-03-22T04:22:28.927 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46924.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46924.log.gz 2024-03-22T04:22:28.928 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43097.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43097.log.gz 2024-03-22T04:22:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46081.log 2024-03-22T04:22:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2024-03-22T04:22:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46081.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46081.log.gz 2024-03-22T04:22:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49417.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2024-03-22T04:22:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47615.log 2024-03-22T04:22:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43412.log 2024-03-22T04:22:28.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47615.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47615.log.gz 2024-03-22T04:22:28.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43412.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43412.log.gz 2024-03-22T04:22:28.937 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35211.log 2024-03-22T04:22:28.938 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34948.log 2024-03-22T04:22:28.940 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35211.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35211.log.gz 2024-03-22T04:22:28.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34948.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34948.log.gz 2024-03-22T04:22:28.941 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44350.log 2024-03-22T04:22:28.942 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51624.log 2024-03-22T04:22:28.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44350.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44350.log.gz 2024-03-22T04:22:28.945 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51624.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51624.log.gz 2024-03-22T04:22:28.945 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46244.log 2024-03-22T04:22:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49540.log 2024-03-22T04:22:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46244.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46244.log.gz 2024-03-22T04:22:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.49540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49540.log.gz 2024-03-22T04:22:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43493.log 2024-03-22T04:22:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46770.log 2024-03-22T04:22:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43493.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43493.log.gz 2024-03-22T04:22:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46770.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46770.log.gz 2024-03-22T04:22:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51941.log 2024-03-22T04:22:28.955 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-03-22T04:22:28.956 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51941.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51941.log.gz 2024-03-22T04:22:28.957 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45688.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-03-22T04:22:28.958 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51253.log 2024-03-22T04:22:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-03-22T04:22:28.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51253.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51253.log.gz 2024-03-22T04:22:28.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44274.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-03-22T04:22:28.962 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33400.log 2024-03-22T04:22:28.963 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47222.log 2024-03-22T04:22:28.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33400.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33400.log.gz 2024-03-22T04:22:28.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47222.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47222.log.gz 2024-03-22T04:22:28.966 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50277.log 2024-03-22T04:22:28.967 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40589.log 2024-03-22T04:22:28.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50277.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50277.log.gz 2024-03-22T04:22:28.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2024-03-22T04:22:28.971 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40589.log.gz 2024-03-22T04:22:28.971 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50380.log 2024-03-22T04:22:28.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42965.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2024-03-22T04:22:28.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50380.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50380.log.gz 2024-03-22T04:22:28.974 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37025.log 2024-03-22T04:22:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53938.log 2024-03-22T04:22:28.977 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37025.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37025.log.gz 2024-03-22T04:22:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53938.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53938.log.gz 2024-03-22T04:22:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39375.log 2024-03-22T04:22:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34398.log 2024-03-22T04:22:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39375.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39375.log.gz 2024-03-22T04:22:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46215.log 2024-03-22T04:22:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34398.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34398.log.gz 2024-03-22T04:22:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49618.log 2024-03-22T04:22:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46215.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46215.log.gz 2024-03-22T04:22:28.985 INFO:teuthology.orchestra.run.smithi086.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-22T04:22:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35393.log 2024-03-22T04:22:28.986 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49618.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49618.log.gz 2024-03-22T04:22:28.986 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36866.log 2024-03-22T04:22:28.987 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37103.log 2024-03-22T04:22:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35393.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35393.log.gz 2024-03-22T04:22:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36866.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36866.log.gz 2024-03-22T04:22:28.989 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49774.log 2024-03-22T04:22:28.990 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37103.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37103.log.gz 2024-03-22T04:22:28.990 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45487.log 2024-03-22T04:22:28.991 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33557.log 2024-03-22T04:22:28.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49774.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49774.log.gz 2024-03-22T04:22:28.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45487.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45487.log.gz 2024-03-22T04:22:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42029.log 2024-03-22T04:22:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33557.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33557.log.gz 2024-03-22T04:22:28.994 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33639.log 2024-03-22T04:22:28.995 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42308.log 2024-03-22T04:22:28.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42029.log: 90.4%/var/log/ceph/ceph-client.admin.33639.log: -- replaced with /var/log/ceph/ceph-client.admin.42029.log.gz 2024-03-22T04:22:28.996 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33639.log.gz 2024-03-22T04:22:28.997 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34727.log 2024-03-22T04:22:28.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42308.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42308.log.gz 2024-03-22T04:22:28.998 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2024-03-22T04:22:28.999 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41591.log 2024-03-22T04:22:29.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34727.log: 89.1%/var/log/ceph/ceph-client.admin.50953.log: -- replaced with /var/log/ceph/ceph-client.admin.34727.log.gz 2024-03-22T04:22:29.000 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2024-03-22T04:22:29.001 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52835.log 2024-03-22T04:22:29.001 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41591.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41591.log.gz 2024-03-22T04:22:29.002 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45461.log 2024-03-22T04:22:29.003 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41261.log 2024-03-22T04:22:29.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52835.log: 89.9%/var/log/ceph/ceph-client.admin.45461.log: -- replaced with /var/log/ceph/ceph-client.admin.52835.log.gz 2024-03-22T04:22:29.004 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45461.log.gz 2024-03-22T04:22:29.004 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54094.log 2024-03-22T04:22:29.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41261.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41261.log.gz 2024-03-22T04:22:29.005 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46529.log 2024-03-22T04:22:29.006 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2024-03-22T04:22:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54094.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54094.log.gz/var/log/ceph/ceph-client.admin.46529.log: 2024-03-22T04:22:29.008 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46529.log.gz 2024-03-22T04:22:29.008 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52879.log 2024-03-22T04:22:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50328.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2024-03-22T04:22:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40856.log 2024-03-22T04:22:29.010 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37368.log 2024-03-22T04:22:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52879.log: 89.5%/var/log/ceph/ceph-client.admin.40856.log: -- replaced with /var/log/ceph/ceph-client.admin.52879.log.gz 2024-03-22T04:22:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51915.log 2024-03-22T04:22:29.012 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40856.log.gz 2024-03-22T04:22:29.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37368.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37368.log.gz 2024-03-22T04:22:29.013 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2024-03-22T04:22:29.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40332.log 2024-03-22T04:22:29.015 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51915.log.gz 2024-03-22T04:22:29.015 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43521.log 2024-03-22T04:22:29.016 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9%/var/log/ceph/ceph-client.admin.40332.log: -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2024-03-22T04:22:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52704.log 2024-03-22T04:22:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43521.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40332.log.gz 2024-03-22T04:22:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42809.log 2024-03-22T04:22:29.018 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43521.log.gz 2024-03-22T04:22:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46001.log 2024-03-22T04:22:29.019 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52704.log.gz 2024-03-22T04:22:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46993.log 2024-03-22T04:22:29.021 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42809.log.gz 2024-03-22T04:22:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46001.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46001.log.gz 2024-03-22T04:22:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-03-22T04:22:29.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46993.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46993.log.gz 2024-03-22T04:22:29.023 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54354.log 2024-03-22T04:22:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37628.log 2024-03-22T04:22:29.025 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-03-22T04:22:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46607.log 2024-03-22T04:22:29.025 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54354.log.gz 2024-03-22T04:22:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37628.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37628.log.gz 2024-03-22T04:22:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35159.log 2024-03-22T04:22:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-03-22T04:22:29.028 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46607.log.gz 2024-03-22T04:22:29.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36019.log 2024-03-22T04:22:29.029 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35159.log.gz 2024-03-22T04:22:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48760.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-03-22T04:22:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27412.log 2024-03-22T04:22:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40305.log 2024-03-22T04:22:29.031 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36019.log.gz 2024-03-22T04:22:29.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-03-22T04:22:29.033 INFO:teuthology.orchestra.run.smithi086.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.27412.log.gz/var/log/ceph/ceph-client.admin.40305.log: 2024-03-22T04:22:29.034 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35315.log 2024-03-22T04:22:29.034 INFO:teuthology.orchestra.run.smithi086.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.40305.log.gz 2024-03-22T04:22:29.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50026.log 2024-03-22T04:22:29.036 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-03-22T04:22:29.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35315.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35315.log.gz 2024-03-22T04:22:29.037 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46137.log 2024-03-22T04:22:29.037 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50026.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50026.log.gz 2024-03-22T04:22:29.038 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42887.log 2024-03-22T04:22:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42182.log 2024-03-22T04:22:29.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46137.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46137.log.gz/var/log/ceph/ceph-client.admin.42887.log: 2024-03-22T04:22:29.041 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42887.log.gz 2024-03-22T04:22:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48201.log 2024-03-22T04:22:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50250.log 2024-03-22T04:22:29.042 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42182.log.gz 2024-03-22T04:22:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45565.log 2024-03-22T04:22:29.044 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48201.log.gz 2024-03-22T04:22:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43556.log 2024-03-22T04:22:29.046 INFO:teuthology.orchestra.run.smithi086.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.50250.log.gz 2024-03-22T04:22:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45565.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45565.log.gz 2024-03-22T04:22:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52151.log 2024-03-22T04:22:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34934.log 2024-03-22T04:22:29.047 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43556.log.gz 2024-03-22T04:22:29.048 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.52151.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.52312.log -- replaced with /var/log/ceph/ceph-client.0.52151.log.gz 2024-03-22T04:22:29.048 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-03-22T04:22:29.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34934.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34934.log.gz 2024-03-22T04:22:29.050 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52813.log 2024-03-22T04:22:29.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2024-03-22T04:22:29.051 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52312.log.gz 2024-03-22T04:22:29.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51479.log 2024-03-22T04:22:29.053 INFO:teuthology.orchestra.run.smithi086.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52813.log.gz 2024-03-22T04:22:29.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44218.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2024-03-22T04:22:29.053 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49074.log 2024-03-22T04:22:29.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36045.log 2024-03-22T04:22:29.055 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51479.log.gz 2024-03-22T04:22:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36175.log 2024-03-22T04:22:29.056 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49074.log.gz 2024-03-22T04:22:29.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36045.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36045.log.gz 2024-03-22T04:22:29.057 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22603.log 2024-03-22T04:22:29.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33531.log 2024-03-22T04:22:29.058 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36175.log.gz 2024-03-22T04:22:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.tmp-client.admin.22603.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22603.log.gz 2024-03-22T04:22:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43386.log 2024-03-22T04:22:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48338.log 2024-03-22T04:22:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33531.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33531.log.gz 2024-03-22T04:22:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39089.log 2024-03-22T04:22:29.061 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43386.log.gz 2024-03-22T04:22:29.061 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48338.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48338.log.gz 2024-03-22T04:22:29.062 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51319.log 2024-03-22T04:22:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53472.log 2024-03-22T04:22:29.063 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39089.log.gz 2024-03-22T04:22:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40058.log 2024-03-22T04:22:29.064 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51319.log.gz 2024-03-22T04:22:29.065 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53472.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53472.log.gz 2024-03-22T04:22:29.065 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46663.log 2024-03-22T04:22:29.067 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41431.log 2024-03-22T04:22:29.067 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40058.log.gz 2024-03-22T04:22:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37680.log 2024-03-22T04:22:29.068 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46663.log.gz 2024-03-22T04:22:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41431.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41431.log.gz 2024-03-22T04:22:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48630.log 2024-03-22T04:22:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41326.log 2024-03-22T04:22:29.070 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37680.log.gz 2024-03-22T04:22:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42486.log 2024-03-22T04:22:29.071 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48630.log.gz 2024-03-22T04:22:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41326.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41326.log.gz 2024-03-22T04:22:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54068.log 2024-03-22T04:22:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51728.log 2024-03-22T04:22:29.074 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42486.log.gz 2024-03-22T04:22:29.075 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46734.log 2024-03-22T04:22:29.075 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54068.log.gz 2024-03-22T04:22:29.076 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51728.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51728.log.gz 2024-03-22T04:22:29.076 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52227.log 2024-03-22T04:22:29.077 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34935.log 2024-03-22T04:22:29.078 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4%/var/log/ceph/ceph-client.admin.52227.log: -- replaced with /var/log/ceph/ceph-client.admin.46734.log.gz 2024-03-22T04:22:29.078 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52227.log.gz 2024-03-22T04:22:29.079 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48526.log 2024-03-22T04:22:29.080 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34935.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34935.log.gz 2024-03-22T04:22:29.080 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40960.log 2024-03-22T04:22:29.081 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45196.log 2024-03-22T04:22:29.082 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48526.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48526.log.gz 2024-03-22T04:22:29.082 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40960.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40960.log.gz 2024-03-22T04:22:29.083 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52491.log 2024-03-22T04:22:29.083 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45196.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45196.log.gz 2024-03-22T04:22:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41855.log 2024-03-22T04:22:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50150.log 2024-03-22T04:22:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52491.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52491.log.gz 2024-03-22T04:22:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41855.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41855.log.gz 2024-03-22T04:22:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53789.log 2024-03-22T04:22:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52614.log 2024-03-22T04:22:29.089 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50150.log.gz 2024-03-22T04:22:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35549.log 2024-03-22T04:22:29.089 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53789.log.gz 2024-03-22T04:22:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.52614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52614.log.gz 2024-03-22T04:22:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50223.log 2024-03-22T04:22:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35549.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35549.log.gz 2024-03-22T04:22:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53886.log 2024-03-22T04:22:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47852.log 2024-03-22T04:22:29.094 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50223.log: /var/log/ceph/ceph-client.admin.53886.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.50223.log.gz 2024-03-22T04:22:29.094 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53886.log.gz 2024-03-22T04:22:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41170.log 2024-03-22T04:22:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47852.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47852.log.gz 2024-03-22T04:22:29.095 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43263.log 2024-03-22T04:22:29.096 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37550.log 2024-03-22T04:22:29.097 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41170.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41170.log.gz 2024-03-22T04:22:29.099 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43263.log.gz 2024-03-22T04:22:29.099 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41618.log 2024-03-22T04:22:29.099 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37550.log.gz 2024-03-22T04:22:29.100 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46322.log 2024-03-22T04:22:29.101 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37051.log 2024-03-22T04:22:29.101 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41618.log.gz 2024-03-22T04:22:29.102 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39401.log 2024-03-22T04:22:29.102 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46322.log.gz 2024-03-22T04:22:29.104 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51187.log 2024-03-22T04:22:29.104 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37051.log.gz 2024-03-22T04:22:29.104 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39401.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39401.log.gz 2024-03-22T04:22:29.104 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49823.log 2024-03-22T04:22:29.106 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49493.log 2024-03-22T04:22:29.106 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51187.log.gz 2024-03-22T04:22:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.49823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49823.log.gz 2024-03-22T04:22:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38486.log 2024-03-22T04:22:29.110 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53420.log 2024-03-22T04:22:29.110 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49493.log.gz 2024-03-22T04:22:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40830.log 2024-03-22T04:22:29.111 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38486.log.gz 2024-03-22T04:22:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36944.log 2024-03-22T04:22:29.111 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53420.log.gz 2024-03-22T04:22:29.111 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42673.log 2024-03-22T04:22:29.111 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40830.log.gz 2024-03-22T04:22:29.112 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36944.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36944.log.gz 2024-03-22T04:22:29.113 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49644.log 2024-03-22T04:22:29.114 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-03-22T04:22:29.114 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42673.log.gz 2024-03-22T04:22:29.115 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-03-22T04:22:29.115 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49644.log.gz 2024-03-22T04:22:29.116 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44244.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-03-22T04:22:29.116 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36513.log 2024-03-22T04:22:29.117 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34745.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-03-22T04:22:29.118 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36513.log.gz 2024-03-22T04:22:29.263 INFO:teuthology.orchestra.run.smithi086.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-22T04:22:29.473 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-22T04:22:29.613 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-03-22T04:22:29.626 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-22T04:22:29.635 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-22T04:22:29.663 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-22T04:22:29.666 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-03-22T04:22:29.666 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m1.483s 2024-03-22T04:22:29.666 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m6.207s 2024-03-22T04:22:29.666 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.544s 2024-03-22T04:22:29.775 INFO:teuthology.orchestra.run.smithi086.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-22T04:22:29.776 INFO:teuthology.orchestra.run.smithi086.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-22T04:22:30.040 INFO:teuthology.orchestra.run.smithi086.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-22T04:22:30.049 INFO:teuthology.orchestra.run.smithi086.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-22T04:22:30.051 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-03-22T04:22:30.051 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m1.883s 2024-03-22T04:22:30.051 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m7.626s 2024-03-22T04:22:30.051 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m1.695s 2024-03-22T04:22:30.052 INFO:tasks.ceph:Archiving logs... 2024-03-22T04:22:30.052 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/remote/smithi086/log 2024-03-22T04:22:30.053 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T04:22:31.359 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/remote/smithi196/log 2024-03-22T04:22:31.360 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T04:22:32.119 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-22T04:22:32.129 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-22T04:22:32.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T04:22:32.133 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T04:22:32.212 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-22T04:22:32.221 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-22T04:22:32.222 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T04:22:32.225 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T04:22:32.237 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2024-03-22T04:22:32.238 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2024-03-22T04:22:32.238 INFO:teuthology.orchestra.run.smithi086.stdout:*hv01.front.sepi 44.4.53.2 2 u 31 64 377 0.068 -6.216 1.973 2024-03-22T04:22:32.238 INFO:teuthology.orchestra.run.smithi086.stdout:+hv02.front.sepi 63.231.80.2 3 u 27 64 377 0.107 -2.026 2.262 2024-03-22T04:22:32.238 INFO:teuthology.orchestra.run.smithi086.stdout:+hv03.front.sepi 74.6.168.72 3 u 30 64 377 0.078 -3.972 2.046 2024-03-22T04:22:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-03-22T04:22:32.240 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2024-03-22T04:22:32.240 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2024-03-22T04:22:32.240 INFO:teuthology.orchestra.run.smithi196.stdout:*hv01.front.sepi 44.4.53.2 2 u 28 64 377 0.076 -4.862 1.655 2024-03-22T04:22:32.240 INFO:teuthology.orchestra.run.smithi196.stdout:+hv02.front.sepi 63.231.80.2 3 u 30 64 377 0.085 -0.932 1.734 2024-03-22T04:22:32.241 INFO:teuthology.orchestra.run.smithi196.stdout:+hv03.front.sepi 74.6.168.72 3 u 32 64 377 0.067 -0.649 2.864 2024-03-22T04:22:32.241 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-03-22T04:22:32.241 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-22T04:22:32.251 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-22T04:22:32.252 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-22T04:22:32.260 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-22T04:22:32.269 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-22T04:22:32.278 INFO:teuthology.task.internal:Duration was 1411.181858 seconds 2024-03-22T04:22:32.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-22T04:22:32.287 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-22T04:22:32.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T04:22:32.291 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T04:22:32.382 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-22T04:22:32.382 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-03-22T04:22:32.382 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T04:22:32.403 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-03-22T04:22:32.403 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T04:22:32.423 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-22T04:22:32.423 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T04:22:32.447 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T04:22:32.506 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-22T04:22:32.506 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T04:22:32.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T04:22:32.620 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-22T04:22:32.632 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-22T04:22:32.632 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T04:22:32.652 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T04:22:32.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-22T04:22:32.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T04:22:32.695 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T04:22:32.710 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-03-22T04:22:32.737 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-03-22T04:22:32.781 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T04:22:32.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:22:32.789 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T04:22:32.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T04:22:32.835 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-22T04:22:32.846 INFO:teuthology.task.internal:Transferring archived files... 2024-03-22T04:22:32.846 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/remote/smithi086 2024-03-22T04:22:32.847 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T04:22:32.873 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927/remote/smithi196 2024-03-22T04:22:32.873 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T04:22:32.901 INFO:teuthology.task.internal:Removing archive directory... 2024-03-22T04:22:32.901 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T04:22:32.916 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T04:22:32.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-22T04:22:32.959 INFO:teuthology.task.internal:Not uploading archives. 2024-03-22T04:22:32.959 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-22T04:22:33.060 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-22T04:22:33.060 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T04:22:33.063 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T04:22:33.068 INFO:teuthology.orchestra.run.smithi086.stdout: 658116 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 22 04:22 /home/ubuntu/cephtest 2024-03-22T04:22:33.071 INFO:teuthology.orchestra.run.smithi196.stdout: 655436 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 22 04:22 /home/ubuntu/cephtest 2024-03-22T04:22:33.073 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-22T04:22:33.083 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-22T04:22:33.105 INFO:teuthology.nuke:Checking targets against current locks 2024-03-22T04:22:33.130 DEBUG:teuthology.nuke:shortname: smithi086 2024-03-22T04:22:33.130 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T04:22:33.154 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 03:49:47.696768', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPscXJYCGC+Y6ORG361G3Yy186lnUntU1Q7Mbs8PRDPUv/JsXU0OaJC4vPQHxfSoJ5xSvu8tDGn2eB3sAvggPtc='} 2024-03-22T04:22:33.158 DEBUG:teuthology.nuke:shortname: smithi196 2024-03-22T04:22:33.158 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T04:22:33.179 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615927', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 03:49:47.695009', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDx+T9vmsFO1rwbEhqQ66c/mjZ446kth81nrjsA2cR/9py1eqhPY/yGlKG+PKmRKciCFFls8ssOVEgnSDY9R34w='} 2024-03-22T04:22:33.207 INFO:teuthology.orchestra.console.smithi086:Power off 2024-03-22T04:22:33.207 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T04:22:33.231 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-03-22T04:22:33.231 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T04:22:33.253 INFO:teuthology.orchestra.console.smithi196:Power off 2024-03-22T04:22:33.253 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T04:22:33.263 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-03-22T04:22:33.276 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-03-22T04:22:33.276 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T04:22:33.299 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-22T04:22:37.264 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T04:22:37.300 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T04:22:37.375 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-03-22T04:22:37.412 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-22T04:22:41.378 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T04:22:41.413 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T04:22:41.488 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-03-22T04:22:41.523 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-03-22T04:22:41.590 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-03-22T04:22:41.624 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-03-22T04:22:41.729 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 client.1-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1411.1818583011627 failure_reason: '"2024-03-22T04:17:15.598395+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-22T04:22:41.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T04:22:41.808 INFO:teuthology.run:FAIL